Commit Graph

11381 Commits

Author SHA1 Message Date
Tomasz Grabiec
2cc27f72ca memtable: Accept all mutation_source parameters 2017-02-23 18:23:52 +01:00
Tomasz Grabiec
53b1a257cc streamed_mutation: Implement fast_forward_to() in stream merger 2017-02-23 18:23:52 +01:00
Tomasz Grabiec
e0a7ed48b0 streamed_mutation: Add generic implementation of forwardable streamed_mutation
Generic but not very efficient wrapper which simply drops
fragments from the original stream.
2017-02-23 18:23:51 +01:00
Tomasz Grabiec
301cd4912b streamed_mutation: Add fast_forward_to() API 2017-02-23 18:23:28 +01:00
Tomasz Grabiec
8fd19a71ff position_in_partition: Introduce position_range 2017-02-21 16:49:36 +01:00
Tomasz Grabiec
78c563ea6a position_in_partition: Introduce position constructor for right after the static row 2017-02-21 16:43:09 +01:00
Tomasz Grabiec
ce58706b50 streamed_mutation: Make cast to view non-explicit 2017-02-21 16:43:09 +01:00
Tomasz Grabiec
b4fd3a08e6 streamed_mutation: Make schema() getter non-copying 2017-02-21 14:18:57 +01:00
Duarte Nunes
65b21e3a99 schema_registry: Don't leak schemas
When loading a schema asynchronously, we're leaving a strong
reference to the loaded schema in the entry's shared future. This
patch fixed this by storing a shared_promised, which is reset when the
schema is loaded.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20170220193654.17439-1-duarte@scylladb.com>
2017-02-21 09:56:21 +01:00
Tomasz Grabiec
33457cc9a9 sstables: Fix detection of repeated tombstones
The check was not catching range tombstone repeated immediately after
itself.
Message-Id: <1487596098-17409-1-git-send-email-tgrabiec@scylladb.com>
2017-02-20 15:35:15 +00:00
Tomasz Grabiec
cc439df542 Revert "sstables: Simplify sstable_streamed_mutation::read_next()"
This reverts commit 1e2c01ff49.

We do not detect repeated tombstone if it follows an in-range
tombstone following a skipped clustering row, because _in_progress
will be disengaged after such tombstone is emitted.
Message-Id: <1487596080-21480-1-git-send-email-tgrabiec@scylladb.com>
2017-02-20 15:34:58 +00:00
Tomasz Grabiec
9f63e172fb tests: compaction_manager_test: Fix abort on exception
Message-Id: <1487343901-12745-1-git-send-email-tgrabiec@scylladb.com>
2017-02-17 15:53:55 +00:00
Avi Kivity
113ed9e963 Merge seastar upstream
* seastar 28a143a...5088065 (8):
  > configure.py: switch cmake to build c-ares to do out-of-source-tree build
  > iotune: make sure help is working
  > collectd: send double correctly for gauge
  > tls: make shutdown/close do "clean" handshake shutdown in background
  > tls: Make sink/source (i.e. streams) first class channel owners
  > native-stack: Make sink/source (i.e. streams) first class channel owners
  > posix-stack: Make sink/source (i.e. streams) first class channel owners
  > Merge "Detector for tasks blocked" from Glauber

Fixes #2085.

Packaging updated to require cmake, drop libtool and automake.
2017-02-16 19:34:28 +02:00
Raphael S. Carvalho
53d9008052 sstables/deletion_manager: kill dead code
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <2b24d9e622238030a737fbbe12b8439853d5d075.1487095059.git.raphaelsc@scylladb.com>
2017-02-16 18:38:54 +02:00
Vlad Zolotarov
f2e4629254 main.cc: expose scylla version as a gauge metrics
Add a new metric that exposes the current ScyllaDB version as a
gauge metrics.

The version is exposed as a label with the "version" key.

Fixes #1979

Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
Message-Id: <1487083703-27929-1-git-send-email-vladz@scylladb.com>
2017-02-16 16:57:55 +02:00
Piotr Jastrzebski
2b8e340761 Replace deprecated BOOST_MESSAGE with BOOST_TEST_MESSAGE
BOOST Unit test deprecated BOOST_MESSAGE as early as 1.34 and had it
been perminently removed.  This patch replaces all uses of BOOST_MESSAGE
with BOOST_TEST_MESSAGE.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Message-Id: <f1732018912a864cea229b0f7cd48170fd927dc2.1487238426.git.piotr@scylladb.com>
2017-02-16 10:49:03 +01:00
Avi Kivity
b8c4b35b57 Merge "Fixes for counter cell locking" from Paweł
"This series contains some fixes and a unit test for the logic responsible
for locking counter cells."

* 'pdziepak/cell-locking-fixes/v1' of github.com:cloudius-systems/seastar-dev:
  tests: add test for counter cell locker
  cell_locking: fix schema upgrades
  cell_locker: make locker non-movable
  cell_locking: allow to be included by anyone
2017-02-15 17:36:48 +02:00
Paweł Dziepak
f7f89df782 tests: add test for counter cell locker 2017-02-15 15:09:40 +00:00
Paweł Dziepak
2eb3e35815 cell_locking: fix schema upgrades
* cell_entry destructor may be called when the former is unlinked
 * update pointer to schema in partition_entry on schema upgrade
 * use correct bucket count when creating a new hash table
2017-02-15 15:09:40 +00:00
Paweł Dziepak
fa9c712263 cell_locker: make locker non-movable
locker keep iteratos to potentially internally stored data and moving
the object would invalidate them.
2017-02-15 13:48:47 +00:00
Paweł Dziepak
fc9145671d cell_locking: allow to be included by anyone 2017-02-15 13:48:47 +00:00
Tomasz Grabiec
9da078a18a tests: logalloc_test: Print debugging info and abort on failure
The test started to fail sporadically on jenkins after
7a00dd6985 due to quiesce() timing out. It's not
clear though if this is a regression because before the series such timeouts
would not cause test failure if the future resulves eventually, timeout was
only logged.

I was not able to reproduce it on my setup nor on jenkins, so let's add more
debugging output and trigger a coredump next time the test fails.

Message-Id: <1487089576-27147-1-git-send-email-tgrabiec@scylladb.com>
2017-02-15 14:41:49 +02:00
Takuya ASADA
9c8515eeed dist/redhat: stop backporting ninja-build from Fedora, install it from EPEL instead
ninja-build-1.6.0-2.fc23.src.rpm on fedora web site deleted for some
reason, but there is ninja-build-1.7.2-2 on EPEL, so we don't need to
backport from Fedora anymore.

Fixes #2087

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1487155729-13257-1-git-send-email-syuu@scylladb.com>
2017-02-15 12:58:00 +02:00
Paweł Dziepak
a5476b4e7d Merge "Emit only range tombstones relevant for query restrictions" from Tomasz
"Immediate reason to do this is to ensure that forwarding of streamed_mutation
will give the same mutations as slicing would, and have unit tests which
verify that those two access methods are consistent with each other.

Secondary reason is performance, to avoid processing unnecessary data.

Note that this should not cause digest mismatch of data queries during rolling
upgrade, because data queries are checksumming only tombstones affecting rows
in the results, so only relevant tombstones.

Fixes #1254."

* tag 'tgrabiec/only-relevant-range-tombstones-v2' of github.com:scylladb/seastar-dev:
  tests: mutation_source_test: Test that slicing returns only relevant range tombstones
  tests: Pass all mutation source parameters
  tests: mutation_source_tests: Ensure timestamps are strictly monotonic
  tests: streamed_mutation_assertions: Add more expectation methods
  tests: streamed_mutation_assertions: Make produces_end_of_stream() give better error messages
  sstables: Simplify sstable_streamed_mutation::read_next()
  sstables: Emit only relevant range tombstones
  range_tombstone: Introduce end_position()
  position_in_partition: Print position when printing fragment
  position_in_partition: Make printable
  position_in_partition: Add cast to view
  position_in_partition: Generalize from-bound_view constructor
  bound_view: Extract converters for range start and end bounds
  mutation_partition: Drop unneeded range tombstones
  mutation_partition: Simplify row removal
  range_tombstone_list: Introduce erase()
  partition_snapshot_reader: Emit only relevant tombstones
  range_tombstone_stream: Add slicing apply() overload
  range_tombstone_list: Introduce slice()
2017-02-14 11:18:51 +00:00
Tomasz Grabiec
7ec8c4cf54 tests: mutation_source_test: Test that slicing returns only relevant range tombstones 2017-02-13 20:52:50 +01:00
Tomasz Grabiec
2b8bd10dca tests: Pass all mutation source parameters 2017-02-13 20:52:49 +01:00
Tomasz Grabiec
25dffef6ae tests: mutation_source_tests: Ensure timestamps are strictly monotonic 2017-02-13 16:19:32 +01:00
Tomasz Grabiec
e6a95fd8cc tests: streamed_mutation_assertions: Add more expectation methods 2017-02-13 16:19:32 +01:00
Tomasz Grabiec
62843175ea tests: streamed_mutation_assertions: Make produces_end_of_stream() give better error messages 2017-02-13 16:19:32 +01:00
Tomasz Grabiec
1e2c01ff49 sstables: Simplify sstable_streamed_mutation::read_next()
mp_row_consumer doesn't split row fragments on repeated range
tombstones any more.
2017-02-13 16:12:16 +01:00
Tomasz Grabiec
6324876f24 sstables: Emit only relevant range tombstones 2017-02-13 16:12:16 +01:00
Tomasz Grabiec
72d74b7b40 range_tombstone: Introduce end_position() 2017-02-13 16:12:16 +01:00
Tomasz Grabiec
79d982cd86 position_in_partition: Print position when printing fragment 2017-02-13 16:12:16 +01:00
Tomasz Grabiec
f2e1f2938b position_in_partition: Make printable 2017-02-13 16:12:16 +01:00
Tomasz Grabiec
fb67dab548 position_in_partition: Add cast to view 2017-02-13 16:12:16 +01:00
Tomasz Grabiec
9ddb5a5173 position_in_partition: Generalize from-bound_view constructor
We will need to create positions corresponding to general range
bounds, not only corresponding to range tombstones.
2017-02-13 16:12:16 +01:00
Tomasz Grabiec
69911a87d3 bound_view: Extract converters for range start and end bounds 2017-02-13 16:12:16 +01:00
Tomasz Grabiec
2489a0f82e mutation_partition: Drop unneeded range tombstones
Fixes #1254.
2017-02-13 16:12:16 +01:00
Tomasz Grabiec
884858078a mutation_partition: Simplify row removal 2017-02-13 16:12:15 +01:00
Tomasz Grabiec
fb42366552 range_tombstone_list: Introduce erase() 2017-02-13 16:12:15 +01:00
Tomasz Grabiec
fcf3391785 partition_snapshot_reader: Emit only relevant tombstones
Refs #1254.
2017-02-13 16:12:15 +01:00
Tomasz Grabiec
440e50b76a range_tombstone_stream: Add slicing apply() overload 2017-02-13 16:12:15 +01:00
Tomasz Grabiec
8b7f93175c range_tombstone_list: Introduce slice() 2017-02-13 16:12:15 +01:00
Paweł Dziepak
8b1d34f39d mutation_partition_serializer: avoid creating atomic_cell object
write_{live, counter, expiring, dead}_cell() take a const reference to
an atomic cell as argument. However, their caller (which is
write_row_cells) passes to them an atomic_cell_view.
There is an appropriate implicit constructor so instead of compiler
complaints we get atomic_cell objects being constructed from views which
involves an allocation and a copy.
Message-Id: <20170213100106.9071-1-pdziepak@scylladb.com>
2017-02-13 11:23:23 +01:00
Avi Kivity
bcf34b9a58 Merge seastar upstream
* seastar 83a41c8...28a143a (5):
  > prometheus: send one MetricFamily per unique metric name
  > tests: Add test for circular_buffer::erase()
  > circular_buffer: Introduce erase()
  > protect against infinite do_until loop
  > metrics: alternative metrics creation with labels
2017-02-12 21:56:11 +02:00
Gleb Natapov
bb72425b61 storage_proxy: fix send_to_endpoint() to use correct create_write_response_handler() overload
There are several problems with storage_proxy::send_to_endpoint right
now. It uses create_write_response_handler() overload that is specific
to read repair which is suboptimal and creates incorrect logs, it does
not process errors and it does not hold storage_proxy object until write
is complete. The patch fixes all of the problems.

Message-Id: <20170208101949.GA19474@scylladb.com>
Reviewed-by: Nadav Har'El <nyh@scylladb.com>
2017-02-12 10:46:13 +02:00
Tomasz Grabiec
c70ebc7ca5 lsa: Make reclaim_timer enclose segment_pool::reclaim_segments()
LSA timing did not include segment migration. It does after this
change.
Message-Id: <1486657046-9378-1-git-send-email-tgrabiec@scylladb.com>
2017-02-09 17:07:59 +00:00
Avi Kivity
5f15388e7a Merge "Size-based buffering of mutation_fragments" from Paweł
"This series changes buffering of mutation fragments in streamed mutations
so that the size of the fragments is taken into account.
The original implementation buffered up to 16 fragments which was pretty
much meaningless since it could be far too much if the fargments were
large or not nearly enough in case they were small

Fixes #2036.."

* 'pdziepak/buffer-mfs-by-size/v1' of github.com:cloudius-systems/seastar-dev:
  streamed_mutation: size-based mutation_fragment buffer limit
  mutation_fragment: cache size in memory
  mutation_fragment: make write access more explicit
2017-02-09 16:42:42 +02:00
Paweł Dziepak
3079b1661e streamed_mutation: size-based mutation_fragment buffer limit
Currently, streamed mutations buffer up to 16 mutation fragments. This
may be too much, not enough or a perfect choice depending on the
mutation fragment size.

This patch makes streamed mutation choose how much mutation fragments to
keep in the buffer depending on their size, so that we avoid using too
much memory in case of large mutation fragments and are able to buffer a
lot of fragments if they are small.
2017-02-09 10:51:11 +00:00
Paweł Dziepak
cd0dc7734a mutation_fragment: cache size in memory 2017-02-09 10:50:51 +00:00