mirrors/scylladb
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

repair: Postpone data segregation to off-strategy compaction

Browse Source 
With data segregation on repair, thousands of sstables are potentially
added to maintenance set which causes high latency due to stalls.

That's because N*M sstables are created by a repair,
	where N = # of ranges
	and M = # of segregations

For TWCS, M = # of windows.

Assuming N = 768 and M = 20, ~15k sstables end up in sstable set

To fix this problem, let's avoid performing data segregation in repair,
as offstrategy will already perform the segregation anyway.

So from now on, only N non-overlapping sstables will be added to set.
Read amplification isn't affected because a query will only touch one
sstable in maintenance set.
When offstrategy starts, it will pick all sstables from set and
compact them in a single step while performing data segregation,
so data is properly laid out before integrated into the main set.

tests:
	- sstable_compaction_test.twcs_reshape_with_disjoint_set_test
	- mode(dev)
	- manual test using repair-based bootstrap

Fixes #9199.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <20210824185043.76475-1-raphaelsc@scylladb.com>
This commit is contained in:
Raphael S. Carvalho
2021-08-24 15:50:43 -03:00
committed by Avi Kivity
parent b012040a76
commit a4053dbb72
1 changed files with 9 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
streaming/consumer.cc
View File

@@ -47,7 +47,15 @@ std::function<future<> (flat_mutation_reader)> make_streaming_consumer(sstring o
auto metadata = mutation_source_metadata{};
auto& cs = cf->get_compaction_strategy();
const auto adjusted_estimated_partitions = cs.adjust_partition_estimate(metadata, estimated_partitions);
auto consumer = cs.make_interposer_consumer(metadata,
auto make_interposer_consumer = [&cs, offstrategy] (const mutation_source_metadata& ms_meta, reader_consumer end_consumer) mutable {
// postpone data segregation to off-strategy compaction if enabled
if (offstrategy) {
return end_consumer;
}
return cs.make_interposer_consumer(ms_meta, std::move(end_consumer));
};
auto consumer = make_interposer_consumer(metadata,
[cf = std::move(cf), adjusted_estimated_partitions, use_view_update_path, &vug, origin = std::move(origin), offstrategy, reason] (flat_mutation_reader reader) {
sstables::shared_sstable sst;
try {