fc79da5912
There are many differences between resharding and compaction that are
artificial, arising more from the way we ended up implementing it than
necessity. This patch attempts to pass the creator and replacer functions
through the compaction_descriptor.
There is a difference between the creator function for resharding and
regular compaction: resharding has to pass the shard number on behalf
of which the SSTable is created. However regular compactions can just
ignore this. No need to have a special path just for this.
After this is done, the constructor for the compaction object can be
greatly simplified. In further patches I intend to simplify it a bit
further, but some more cleanup has to happen first.
To make that happen we have to construct a compaction_descriptor object
inside the resharding function. This is temporary: resharding currently
works with a descriptor, but at some point that descriptor is lost and
broken into pieces to be passed to this function. The overarching goal
of this work is exactly to be able to keep that descriptor for as long
as possible, which should simplify things a lot.
Callers are patched, but there are plenty for sstable_datafile_test.cc.
For their benefit, a helper function is provided to keep the previous
signature (test only).
Signed-off-by: Glauber Costa <glauber@scylladb.com>
(cherry picked from commit e8801cd77b)
Scylla
Quick-start
To get the build going quickly, Scylla offers a frozen toolchain which would build and run Scylla using a pre-configured Docker image. Using the frozen toolchain will also isolate all of the installed dependencies in a Docker container. Assuming you have met the toolchain prerequisites, which is running Docker in user mode, building and running is as easy as:
$ ./tools/toolchain/dbuild ./configure.py
$ ./tools/toolchain/dbuild ninja build/release/scylla
$ ./tools/toolchain/dbuild ./build/release/scylla --developer-mode 1
Please see HACKING.md for detailed information on building and developing Scylla.
Note: GCC >= 8.1.1 is required to compile Scylla.
Running Scylla
- Run Scylla
./build/release/scylla
- run Scylla with one CPU and ./tmp as work directory
./build/release/scylla --workdir tmp --smp 1
- For more run options:
./build/release/scylla --help
Testing
See test.py manual.
Scylla APIs and compatibility
By default, Scylla is compatible with Apache Cassandra and its APIs - CQL and Thrift. There is also experimental support for the API of Amazon DynamoDB, but being experimental it needs to be explicitly enabled to be used. For more information on how to enable the experimental DynamoDB compatibility in Scylla, and the current limitations of this feature, see Alternator and Getting started with Alternator.
Documentation
Documentation can be found in ./docs and on the wiki. There is currently no clear definition of what goes where, so when looking for something be sure to check both. Seastar documentation can be found here. User documentation can be found here.
Training
Training material and online courses can be found at Scylla University. The courses are free, self-paced and include hands-on examples. They cover a variety of topics including Scylla data modeling, administration, architecture, basic NoSQL concepts, using drivers for application development, Scylla setup, failover, compactions, multi-datacenters and how Scylla integrates with third-party applications.
Building Fedora-based Docker image
Build a Docker image with:
cd dist/docker
docker build -t <image-name> .
Run the image with:
docker run -p $(hostname -i):9042:9042 -i -t <image name>