mirror of
https://github.com/scylladb/scylladb.git
synced 2026-05-31 03:56:42 +00:00
54ab13eb8e
"Currently, a shared sstable is rewritten at all shards it belongs to, and only after that, it's deleted. This new algorithm adds the ability to reshard a set of sstables together at a single shard and produce unshared sstable for all shards involved. That's important for the leveled compaction strategy issue, in which the number of sstables growing considerably after resharding. What happened is that every sstable was being split into N ones, so we could end up with tons of small sstables. Now, we will reshard together a set of adjacent sstables." * 'sstable_resharding_revamp_v9' of github.com:raphaelsc/scylla: tests: add test for new sstable resharding database: kill column_family::start_rewrite database: wire up new resharding algorithm database: implement new sstable resharding algorithm database: introduce function to replace new sstables by their ancestors prevent regular compaction from choosing shared sstables compaction_strategy: implement resharding strategy for compaction strategies sstables: store more info in foreign_sstable_open_info sstables: make it possible to get open info from loaded sstable database: export column family dir database: inform if column family has shared tables sstables: add method to export ancestors lcs: implement get_level_count compaction_manager: introduce method to check if manager stopped lcs: restore invariant instead of sending overlapping sst to L0 sstables: extend compaction for new resharding sstables: allow shard A to correctly create sstable for shard B compaction: rework compacting_sstable_writer to work with multiple writers compaction: prepare compacting_sstable_writer to work with writers sstables: rework compaction to make it easy to extend
Scylla
Building Scylla
In addition to required packages by Seastar, the following packages are required by Scylla.
Submodules
Scylla uses submodules, so make sure you pull the submodules first by doing:
git submodule init
git submodule update --init --recursive
Building and Running Scylla on Fedora
- Installing required packages:
sudo dnf install yaml-cpp-devel lz4-devel zlib-devel snappy-devel jsoncpp-devel thrift-devel antlr3-tool antlr3-C++-devel libasan libubsan gcc-c++ gnutls-devel ninja-build ragel libaio-devel cryptopp-devel xfsprogs-devel numactl-devel hwloc-devel libpciaccess-devel libxml2-devel python3-pyparsing lksctp-tools-devel protobuf-devel protobuf-compiler systemd-devel libunwind-devel
- Build Scylla
./configure.py --mode=release --with=scylla --disable-xen
ninja-build build/release/scylla -j2 # you can use more cpus if you have tons of RAM
- Run Scylla
./build/release/scylla
- run Scylla with one CPU and ./tmp as data directory
./build/release/scylla --datadir tmp --commitlog-directory tmp --smp 1
- For more run options:
./build/release/scylla --help
Building Fedora RPM
As a pre-requisite, you need to install Mock on your machine:
# Install mock:
sudo yum install mock
# Add user to the "mock" group:
usermod -a -G mock $USER && newgrp mock
Then, to build an RPM, run:
./dist/redhat/build_rpm.sh
The built RPM is stored in /var/lib/mock/<configuration>/result directory.
For example, on Fedora 21 mock reports the following:
INFO: Done(scylla-server-0.00-1.fc21.src.rpm) Config(default) 20 minutes 7 seconds
INFO: Results and/or logs in: /var/lib/mock/fedora-21-x86_64/result
Building Fedora-based Docker image
Build a Docker image with:
cd dist/docker
docker build -t <image-name> .
Run the image with:
docker run -p $(hostname -i):9042:9042 -i -t <image name>