This is the mutation fragment level equivalent of mutation_merger.
It merges fragments produced by different sources. Mutation
fragments are not as self-contained as streamed mutations, they have
external context, e.g. the partition they belong to. To support this
mutation_fragment_merger operates on a producer instead of a vector of
fragments. Producer can have internal state and can do side-actions as
fragments are consumed.
For now only the interface is converted, behind the scenes the previous
implementation remains, it's output is simply converted by
flat_mutation_reader_from_mutation_reader. The implementation will be
converted in the following patches.
This simple code-movement and patch lays the groundwork for splitting
the logic in combined_mutation_reader into two blocks:
* one that takes care of moving the readers in lockstep and emits their
output as a non-decreasing stream of streamed_mutations and
* one that takes care of merging the above stream into
strictly-increasing stream of streamed_mutations.
This in turn is preparation-work to the transformation of
combined_mutation_reader into a flat_mutation_reader::impl.
"A delayed task can fail to execute, for example if the consistency
level the task required can't be achieves, so we should ensure it is
retried.
Fixes#3038"
* 'auth-retry/v2' of https://github.com/duarten/scylla:
auth/standard_role_manager: Extend exception handling
auth/common: Add exception handling and retry to task scheduling
auth/standard_role_manager: Lift async block to caller
Also handle exceptions thrown by has_existing_roles(), and print a
similar message to Apache Cassandra in case of error.
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
This follows the implementation in Apache Cassandra. The auth tasks
executed by delay_until_system_ready() usually perform a query with
QUORUM consistency level, which can fail if some nodes are
unavailable. So, we provide both exception handling and a retry
mechanism.
Fixes#3038
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
has_existing_roles() creates a seastar thread, but that can be
lifted to the caller for prettier code.
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
We are getting package build error on dh_auto_install which is invoked by
pybuild.
But since we handle all installation on debian/scylla-server.install, we can
simply skip running dh_auto_install.
Fixes#3036
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1512065117-15708-1-git-send-email-syuu@scylladb.com>
"This series makes it easier to comprehend assertion failures which
involve printing mutation contents."
* 'tgrabiec/mutation-printout' of github.com:scylladb/seastar-dev:
tests: Introduce mutation_diff script
mutation: Make printout more concise
mutation_partition: Don't print absent elements
mutation_partition: Make row_marker printout similar to other partition elements
database: Move operator<<() overloads to appropriate source files
mutation_partition: Use multi-line printout
position_in_partition: Improve printout
Convert to a multi line output, which is easier to read for a human.
After:
{ks.cf key {key: pk{000c706b30303030303030303030}, token:-2018791535786252460} data {mutation_partition: {tombstone: none},
range_tombstones: {},
static: cont=1 {row: },
clustered: {
{rows_entry: cont=true dummy=false {position: clustered,ckp{000c636b30303030303030303030},0} {deletable_row: {row: }}},
{rows_entry: cont=true dummy=true {position: clustered,ckp{000c636b30303030303030303031},0} {deletable_row: {row: }}}}}}
Before:
{position: type clustered, bound_weight -1, key ckp{000c636b30303030303030303033}}
After:
{position: clustered,ckp{000c636b30303030303030303033},-1}
Benefits:
- most significant parts appear first.
bound_weight, which is least significant, was in the middle before.
- shorter, so a bit easier to parse assertion failures.
The universal reference was introduced so we could bind an rvalue to
the argument, but it would have sufficed to make the argument a const
reference. This is also more consistent with the function's other
overload.
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20171129132758.19654-1-duarte@scylladb.com>
Fix two issues with serializing non-compound range tombstones as
compound: convert a non-compound clustering element to compound and
actually advertise the issue to other nodes.
* git@github.com:duarten/scylla.git rt-compact-fixes/v1:
compound_compact: Allow rvalues in size()
sstables/sstables: Convert non-compound clustering element to compound
tests/sstable_mutation_test: Verify we can write/read non-correct RTs
service/storage_service: Export non-compound RT feature
Add test to verify we can write and read non-compound tombstones and
compound ones for backward compatibility.
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
576ea421dc introduced a regression
as it didn't change the assumption that all clustering elements where
compound when writing a range tombstone, compound or non-compound, as
compound. Thus, we serialized a non-compound element while we should
have serialized a compound one.
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
after 7f8b62bc0b, its move operator and ctor broke. That potentially
leads to error because data_consume_context dtor moves sstable ref
to continuation when waiting for in-flight reads from input stream.
Otherwise, sstable can be destroyed meanwhile and file descriptor
would be invalid, leading to EBADF.
Fixes#3020.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <20171129014917.11841-1-raphaelsc@scylladb.com>
"This simplifies implementation of mutation_partition merging by relaxing
exception guarantees it needs to provide. This allows reverters to be dropped.
Direct motivation for this is to make it easier to implement new semantics
for merging of clustering range continuity.
Implementation details:
We only need strong exception guarantees when applying to the memtable, which is
using MVCC. Instead of calling apply() with strong exception guarantees on the latest
version, we will move the incoming mutation to a new partition_version and then
use monotonic apply() to merge them. If that merging fails, we attach the version with
the remainder, which cannot fail. This way apply() always succeeds if the allocation
of partition_version object succeeds.
Results of `perf_simple_query_g -c1 -m1G --write` (high overwrite rate):
Before:
101011.13 tps
102498.07 tps
103174.68 tps
102879.55 tps
103524.48 tps
102794.56 tps
103565.11 tps
103018.51 tps
103494.37 tps
102375.81 tps
103361.65 tps
After:
101785.37 tps
101366.19 tps
103532.26 tps
100834.83 tps
100552.11 tps
100891.31 tps
101752.06 tps
101532.00 tps
100612.06 tps
102750.62 tps
100889.16 tps
Fixes #2012."
* tag 'tgrabiec/drop-reversible-apply-v1' of github.com:scylladb/seastar-dev:
mutation_partition: Drop apply_reversibly()
mutation_partition: Relax exception guarantees of apply()
mutation_partition: Introduce apply_weak()
tests: mvcc: Add test for atomicity of partition_entry::apply()
tests: Move failure_injecting_allocation_strategy to a header
tests: mutation_partition: Test exception guarantees of apply_monotonically()
mvcc: Use apply_monotonically() where sufficient
mvcc: partition_version: Use apply_monotonically() to provide atomicity
mvcc: Extract partition_entry::add_version()
mutation_partition: Introduce apply_monotonically()
mutation_partition: Introduce row::consume_with()
The uses which needed strong or weak exception guarantees were
switched to a solution involving apply_monotonically(). All remaining
uses don't need any exception guarantees.
