Asias He e31ffbf2e6 compaction_manager: Avoid stall in perform_cleanup
The following stall was seen during a cleanup operation:

scylla: Reactor stalled for 16262 ms on shard 4.

| std::_MakeUniq<locator::tokens_iterator_impl>::__single_object std::make_unique<locator::tokens_iterator_impl, locator::tokens_iterator_impl&>(locator::tokens_iterator_impl&) at /usr/include/fmt/format.h:1158
|  (inlined by) locator::token_metadata::tokens_iterator::tokens_iterator(locator::token_metadata::tokens_iterator const&) at ./locator/token_metadata.cc:1602
| locator::simple_strategy::calculate_natural_endpoints(dht::token const&, locator::token_metadata&) const at simple_strategy.cc:?
|  (inlined by) locator::simple_strategy::calculate_natural_endpoints(dht::token const&, locator::token_metadata&) const at ./locator/simple_strategy.cc:56
| locator::abstract_replication_strategy::get_ranges(gms::inet_address, locator::token_metadata&) const at /usr/include/fmt/format.h:1158
| locator::abstract_replication_strategy::get_ranges(gms::inet_address) const at /usr/include/fmt/format.h:1158
| service::storage_service::get_ranges_for_endpoint(seastar::basic_sstring<char, unsigned int, 15u, true> const&, gms::inet_address const&) const at /usr/include/fmt/format.h:1158
| service::storage_service::get_local_ranges(seastar::basic_sstring<char, unsigned int, 15u, true> const&) const at /usr/include/fmt/format.h:1158
|  (inlined by) operator() at ./sstables/compaction_manager.cc:691
|  (inlined by) _M_invoke at /usr/include/c++/9/bits/std_function.h:286
| std::function<std::vector<seastar::lw_shared_ptr<sstables::sstable>, std::allocator<seastar::lw_shared_ptr<sstables::sstable> > > (table const&)>::operator()(table const&) const at /usr/include/fmt/format.h:1158
|  (inlined by) compaction_manager::rewrite_sstables(table*, sstables::compaction_options, std::function<std::vector<seastar::lw_shared_ptr<sstables::sstable>, std::allocator<seastar::lw_shared_ptr<sstables::sstable> > > (table const&)>) at ./sstables/compaction_manager.cc:604
| compaction_manager::perform_cleanup(table*) at /usr/include/fmt/format.h:1158

To fix, we furturize the function to get local ranges and sstables.

In addition, this patch removes the dependency to global storage_service object.

Fixes #6662

(cherry picked from commit 07e253542d)
2020-08-27 13:11:39 +03:00
2020-08-11 12:35:31 +03:00
2020-08-11 12:35:31 +03:00
2020-08-11 13:33:45 +03:00
2020-02-07 08:59:39 +01:00
2020-08-11 12:35:31 +03:00
2019-12-19 15:43:04 +02:00
2020-08-11 12:35:32 +03:00
2020-01-30 11:10:08 +01:00
2020-02-17 10:59:06 +01:00
2020-03-03 11:34:00 +01:00
2020-01-30 14:10:18 +02:00
2020-02-17 10:59:15 +01:00
2020-01-29 14:05:01 -08:00
2020-02-17 10:59:06 +01:00
2020-03-03 11:34:00 +01:00
2020-02-17 10:59:06 +01:00
2020-03-03 11:34:00 +01:00
2020-03-03 11:34:00 +01:00
2020-03-03 11:34:00 +01:00
2020-03-16 13:42:07 +02:00
2020-08-17 00:38:43 +03:00
2020-03-03 11:34:00 +01:00
2020-03-03 11:34:00 +01:00

Scylla

Quick-start

To get the build going quickly, Scylla offers a frozen toolchain which would build and run Scylla using a pre-configured Docker image. Using the frozen toolchain will also isolate all of the installed dependencies in a Docker container. Assuming you have met the toolchain prerequisites, which is running Docker in user mode, building and running is as easy as:

$ ./tools/toolchain/dbuild ./configure.py
$ ./tools/toolchain/dbuild ninja build/release/scylla
$ ./tools/toolchain/dbuild ./build/release/scylla --developer-mode 1

Please see HACKING.md for detailed information on building and developing Scylla.

Note: GCC >= 8.1.1 is required to compile Scylla.

Running Scylla

  • Run Scylla
./build/release/scylla

  • run Scylla with one CPU and ./tmp as work directory
./build/release/scylla --workdir tmp --smp 1
  • For more run options:
./build/release/scylla --help

Testing

See test.py manual.

Scylla APIs and compatibility

By default, Scylla is compatible with Apache Cassandra and its APIs - CQL and Thrift. There is also experimental support for the API of Amazon DynamoDB, but being experimental it needs to be explicitly enabled to be used. For more information on how to enable the experimental DynamoDB compatibility in Scylla, and the current limitations of this feature, see Alternator and Getting started with Alternator.

Documentation

Documentation can be found in ./docs and on the wiki. There is currently no clear definition of what goes where, so when looking for something be sure to check both. Seastar documentation can be found here. User documentation can be found here.

Training

Training material and online courses can be found at Scylla University. The courses are free, self-paced and include hands-on examples. They cover a variety of topics including Scylla data modeling, administration, architecture, basic NoSQL concepts, using drivers for application development, Scylla setup, failover, compactions, multi-datacenters and how Scylla integrates with third-party applications.

Building Fedora-based Docker image

Build a Docker image with:

cd dist/docker
docker build -t <image-name> .

Run the image with:

docker run -p $(hostname -i):9042:9042 -i -t <image name>

Contributing to Scylla

Hacking howto Guidelines for contributing

Description
No description provided
Readme 397 MiB
Languages
C++ 72.5%
Python 26.2%
CMake 0.4%
GAP 0.3%
Shell 0.3%