Calle Wilund 1e37e1d40c commitlog: Add optional use of O_DSYNC mode
Refs #3929

Optionally enables O_DSYNC mode for segment files, and when
enabled ignores actual flushing and just barriers any ongoing
writes.

Iff using O_DSYNC mode, we will not only truncate the file
to max size, but also do an actual initial write of zero:s
to it, since XFS (intended target) has observably less good
behaviour on non-physical file blocks. Once written (and maybe
recycled) we should have rather satisfying throughput on writes.

Note that the O_DSYNC behaviour is hidden behind a default
disabled option. While user should probably seldom worry about
this, we should add some sort of logic i main/init that unless
specified by user, evaluates the commitlog disk and sets this
to true if it is using XFS and looks ok. This is because using
O_DSYNC on things like EXT4 etc has quite horrible performance.

All above statements about performance and O_DSYNC behaviour
are based on a sampling of benchmark results (modified fsqual)
on a statistically non-ssignificant selection of disks. However,
at least there the observed behaviour is a rather large
difference between ::fallocate:ed disk area vs. actually written
using O_DSYNC on XFS, and O_DSYNC on EXT4.

Note also that measurements on O_DSYNC vs. no O_DSYNC does not
take into account the wall-clock time of doing manual disk flush.
This is intentionally ignored, since in the commitlog case, at
least using periodic mode, flushes are relatively rare.

Message-Id: <20190520120331.10229-1-calle@scylladb.com>
2019-05-20 15:10:48 +03:00
2019-05-13 19:30:50 +02:00
2018-11-21 00:01:44 +02:00
2018-12-03 11:18:02 +02:00
2018-11-26 18:59:41 +01:00
2019-05-07 09:27:27 +03:00
2019-05-12 13:32:41 +03:00
2018-11-27 13:01:02 +02:00
2019-05-07 10:01:15 +03:00
2018-11-21 00:01:44 +02:00
2018-11-21 00:01:44 +02:00
2018-11-21 00:01:44 +02:00
2018-11-21 00:01:44 +02:00
2019-05-07 09:26:58 +03:00
2019-02-20 08:03:46 -08:00
2018-11-21 00:01:44 +02:00
2019-04-28 15:50:12 +02:00
2018-11-01 13:16:17 +00:00
2019-03-28 14:21:10 +02:00
2019-04-23 16:13:39 +02:00
2019-05-07 09:55:53 +03:00
2019-02-11 09:25:25 +01:00
2019-05-14 13:27:30 +03:00
2018-11-28 23:54:03 +01:00
2018-11-21 00:01:44 +02:00
2018-11-21 00:01:44 +02:00
2018-11-21 00:01:44 +02:00
2018-11-21 00:01:44 +02:00
2018-12-12 16:49:01 +08:00

Scylla

Quick-start

$ git submodule update --init --recursive
$ sudo ./install-dependencies.sh
$ ./configure.py --mode=release
$ ninja-build -j4 # Assuming 4 system threads.
$ ./build/release/scylla
$ # Rejoice!

Please see HACKING.md for detailed information on building and developing Scylla.

Note: GCC >= 8.1.1 is require to compile Scylla.

Note: See frozen toolchain for a way to build and run on an older distribution.

Running Scylla

  • Run Scylla
./build/release/scylla

  • run Scylla with one CPU and ./tmp as data directory
./build/release/scylla --datadir tmp --commitlog-directory tmp --smp 1
  • For more run options:
./build/release/scylla --help

Building Fedora RPM

As a pre-requisite, you need to install Mock on your machine:

# Install mock:
sudo yum install mock

# Add user to the "mock" group:
usermod -a -G mock $USER && newgrp mock

Then, to build an RPM, run:

./dist/redhat/build_rpm.sh

The built RPM is stored in /var/lib/mock/<configuration>/result directory. For example, on Fedora 21 mock reports the following:

INFO: Done(scylla-server-0.00-1.fc21.src.rpm) Config(default) 20 minutes 7 seconds
INFO: Results and/or logs in: /var/lib/mock/fedora-21-x86_64/result

Building Fedora-based Docker image

Build a Docker image with:

cd dist/docker
docker build -t <image-name> .

Run the image with:

docker run -p $(hostname -i):9042:9042 -i -t <image name>

Contributing to Scylla

Guidelines for contributing

Description
No description provided
Readme 487 MiB
Languages
C++ 72.1%
Python 26.7%
CMake 0.3%
GAP 0.3%
Shell 0.3%