Avi Kivity dd76943125 Merge "Segregate data when streaming by timestamp for time window compaction strategy" from Botond
"
When writing streamed data into sstables, while using time window
compaction strategy, we have to emit a new sstable for each time window.
Otherwise we can end up with sstables, mixing data from wildly different
windows, ruining the compaction strategy's ability to drop entire
sstables when all data within is expired. This gets worse as these mixed
sstables get compacted together with sstables that used to contain a
single time window.

This series provides a solution to this by segregating the data by its
atom's the time-windows. This is done on the new RPC streaming and the
new row-level, repair, memtable-flush and compaction, ensuring that the
segregation requirement is respected at all times.

Fixes: #2687
"

* 'segregate-data-into-sstables-by-time-window-streaming/v2.1' of ssh://github.com/denesb/scylla:
  streaming,repair: restore indentation
  repair: pass the data stream through the compaction strategy's interposer consumer
  streaming: pass the data stream through the compaction strategy's interposer consumer
  TWCS: implement add_interposer_consumer()
  compaction_strategy: add add_interposer_consumer()
  Add mutation_source_metadata
  tests: add unit test for timestamp_based_splitting_writer
  Add timestamp_based_splitting_writer
  Introduce mutation_writer namespace
2019-06-26 19:18:52 +03:00
2019-06-25 19:06:30 +03:00
2019-06-25 19:05:29 +03:00
2018-12-03 11:18:02 +02:00
2019-05-07 09:27:27 +03:00
2019-06-25 19:05:29 +03:00
2019-02-20 08:03:46 -08:00
2019-04-28 15:50:12 +02:00
2019-03-28 14:21:10 +02:00
2019-06-25 19:05:29 +03:00
2019-05-07 09:55:53 +03:00
2019-05-14 13:27:30 +03:00
2019-06-26 15:45:59 +03:00
2019-06-25 12:01:33 +03:00
2019-06-17 13:09:54 +01:00

Scylla

Quick-start

To get the build going quickly, Scylla offers a frozen toolchain which would build and run Scylla using a pre-configured Docker image. Using the frozen toolchain will also isolate all of the installed dependencies in a Docker container. Assuming you have met the toolchain prerequisites, which is running Docker in user mode, building and running is as easy as:

$ ./tools/toolchain/dbuild ./configure.py
$ ./tools/toolchain/dbuild ninja build/release/scylla
$ ./tools/toolchain/dbuild ./build/release/scylla --developer-mode 1

Please see HACKING.md for detailed information on building and developing Scylla.

Note: GCC >= 8.1.1 is required to compile Scylla.

Running Scylla

  • Run Scylla
./build/release/scylla

  • run Scylla with one CPU and ./tmp as data directory
./build/release/scylla --datadir tmp --commitlog-directory tmp --smp 1
  • For more run options:
./build/release/scylla --help

Building Fedora RPM

As a pre-requisite, you need to install Mock on your machine:

# Install mock:
sudo yum install mock

# Add user to the "mock" group:
usermod -a -G mock $USER && newgrp mock

Then, to build an RPM, run:

./dist/redhat/build_rpm.sh

The built RPM is stored in /var/lib/mock/<configuration>/result directory. For example, on Fedora 21 mock reports the following:

INFO: Done(scylla-server-0.00-1.fc21.src.rpm) Config(default) 20 minutes 7 seconds
INFO: Results and/or logs in: /var/lib/mock/fedora-21-x86_64/result

Building Fedora-based Docker image

Build a Docker image with:

cd dist/docker
docker build -t <image-name> .

Run the image with:

docker run -p $(hostname -i):9042:9042 -i -t <image name>

Contributing to Scylla

Hacking howto Guidelines for contributing

Description
No description provided
Readme 473 MiB
Languages
C++ 72.2%
Python 26.6%
CMake 0.3%
GAP 0.3%
Shell 0.3%