5d99d667ec
" This patch series attempts to decouple package build and release infrastructure, which is internal to Scylla (the company). The goal of this series is to make it easy for humans and machines to build the full Scylla distribution package artifacts, and make it easy to quickly verify them. The improvements to build system are done in the following steps. 1. Make scylla.git a super-module, which has git submodules for scylla-jmx and scylla-tools. A clone of scylla.git is now all that is needed to access all source code of all the different components that make up a Scylla distribution, which is a preparational step to adding "dist" ninja build target. A scripts/sync-submodules.sh helper script is included, which allows easy updating of the submodules to the latest head of the respective git repositories. 2. Make builds reproducible by moving the remaining relocatable package specific build options from reloc/build_reloc.sh to the build system. After this step, you can build the exact same binaries from the git repository by using the dbuild version from scylla.git. 3. Add a "dist" target to ninja build, which builds all .rpm and .deb packages with one command. To build a release, run: $ ./tools/toolchain/dbuild ./configure.py --mode release $ ./tools/toolchain/dbuild ninja-build dist and you will now have .rpm and .deb packages to all the components of a Scylla distribution. 4. Add a "dist-check" target to ninja build for verification of .rpm and .deb packages in one command. To verify all the built packages, run: $ ninja-build dist-check Please note that you must run this step on the host, because the target uses Docker under the hood to verify packages by installing them on different Linux distributions. Currently only CentOS 7 verification is supported. All these improvements are done so that backward compatibility is retained. That is, any existing release infrastructure or other build scripts are completely unaffacted. Future improvements to consider: - Package repository generation: add a "ninja repo" command to generate a .rpm and .deb repositories, which can be uploaded to a web site. This makes it possible to build a downloadable Scylla distribution from scylla.git. The target requires some configuration, which user has to provide. For example, download URL locations and package signing keys. - Amazon Machine Image (AMI) support: add a "ninja ami" command to simplify the steps needed to generate a Scylla distribution AMI. - Docker image support: add a "ninja docker" command to simplify the steps needed to generate a Scylla distribution Docker image. - Simplify and unify package build: simplify and unify the various shell scripts needed to build packages in different git repositories. This step will break backward compatiblity and can be done only after relevant build scripts and release infrastructure is updated. " * 'penberg/packaging/v5' of github.com:penberg/scylla: docs: Update packaging documentation build: Add "dist-check" target scripts/testing: Add "dist-check" for package verification build: Add "dist" target reloc: Add '--builddir' option to build_deb.sh build: Add "-ffile-prefix-map" to cxxflags docs: Document sync-submodules.sh script in maintainer.md sync-submodules.sh: Add script for syncing submodules Add scylla-tools submodule Add scylla-jmx submodule
Scylla
Quick-start
Scylla is fairly fussy about its build environment, requiring very recent versions of the C++20 compiler and of many libraries to build. The document HACKING.md includes detailed information on building and developing Scylla, but to get Scylla building quickly on (almost) any build machine, Scylla offers offers a frozen toolchain, This is a pre-configured Docker image which includes recent versions of all the required compilers, libraries and build tools. Using the frozen toolchain allows you to avoid changing anything in your build machine to meet Scylla's requirements - you just need to meet the frozen toolchain's prerequisites (mostly, Docker or Podman being available).
Building and running Scylla with the frozen toolchain is as easy as:
$ ./tools/toolchain/dbuild ./configure.py
$ ./tools/toolchain/dbuild ninja build/release/scylla
$ ./tools/toolchain/dbuild ./build/release/scylla --developer-mode 1
Running Scylla
- Run Scylla
./build/release/scylla
- run Scylla with one CPU and ./tmp as work directory
./build/release/scylla --workdir tmp --smp 1
- For more run options:
./build/release/scylla --help
Testing
See test.py manual.
Scylla APIs and compatibility
By default, Scylla is compatible with Apache Cassandra and its APIs - CQL and Thrift. There is also experimental support for the API of Amazon DynamoDB, but being experimental it needs to be explicitly enabled to be used. For more information on how to enable the experimental DynamoDB compatibility in Scylla, and the current limitations of this feature, see Alternator and Getting started with Alternator.
Documentation
Documentation can be found in ./docs and on the wiki. There is currently no clear definition of what goes where, so when looking for something be sure to check both. Seastar documentation can be found here. User documentation can be found here.
Training
Training material and online courses can be found at Scylla University. The courses are free, self-paced and include hands-on examples. They cover a variety of topics including Scylla data modeling, administration, architecture, basic NoSQL concepts, using drivers for application development, Scylla setup, failover, compactions, multi-datacenters and how Scylla integrates with third-party applications.
Building a CentOS-based Docker image
Build a Docker image with:
cd dist/docker/redhat
docker build -t <image-name> .
This build is based on executables downloaded from downloads.scylladb.com, not on the executables built in this source directory. See further instructions in dist/docker/redhat/README.md to build a docker image from your own executables.
Run the image with:
docker run -p $(hostname -i):9042:9042 -i -t <image name>