mirror of
https://github.com/scylladb/scylladb.git
synced 2026-04-27 20:05:10 +00:00
d606e9bfa2
Off-strategy suffers with a 100% space overhead, as it adopted a sort of all or nothing approach. Meaning all input sstables, living in maintenance set, are kept alive until they're all reshaped according to the strategy criteria. Input sstables in off-strategy are very likely to be mostly disjoint, so it can greatly benefit from incremental compaction. The incremental compaction approach is not only good for decreasing disk usage, but also memory usage (as metadata of input and output live in memory), and file desc count, which takes memory away from OS. Turns out that this approach also greatly simplifies the off-strategy impl in compaction manager, as it no longer have to maintain new unused sstables and mark them for deletion on failure, and also unlink intermediary sstables used between reshape rounds. Fixes https://github.com/scylladb/scylladb/issues/14992. Backport notes: relatively easy to backport, had to include **replica: Make compaction_group responsible for deleting off-strategy compaction input** and **compaction/leveled_compaction_strategy: ideal_level_for_input: special case max_sstable_size==0** Closes #15793 * github.com:scylladb/scylladb: test: Verify that off-strategy can do incremental compaction compaction/leveled_compaction_strategy: ideal_level_for_input: special case max_sstable_size==0 compaction: Clear pending_replacement list when tombstone GC is disabled compaction: Enable incremental compaction on off-strategy compaction: Extend reshape type to allow for incremental compaction compaction: Move reshape_compaction in the source compaction: Enable incremental compaction only if replacer callback is engaged replica: Make compaction_group responsible for deleting off-strategy compaction input