"This feature is intended to make compaction more efficient at getting rid of
droppable tombstone and expired data wasting disk space. So far, people have
been dealing with it manually through major compaction.
With strategies other than date tiered, large sstables will be left untouched
for a long time even though it's all expired. Date tiered suffers from it when
mixing data with different TTL because it only includes for compaction sstable
that is fully expired.
sstables keeps as metadata a histogram which allows us to easily estimate
droppable data ratio from gc_before. sstables which droppable data ratio is
above 20% (default value for tombstone_threshold option) will be considered
candidates for the operation.
Like in C*, we will only do tombstone removal compaction when there's nothing
to compact in standard way. It would be interesting to trigger it too when
disk usage is above a given threshold, but I decided to leave this for later.
Fixes #2306."
* 'tombstone_removal_compaction_v4' of github.com:raphaelsc/scylla:
tests: more testing for tombstone compaction options
tests: basic tombstone compaction test for date tiered
compaction/dtcs: add support for tombstone compaction
tests: basic test of tombstone compaction with lcs
compaction/lcs: add support for tombstone compaction
tests: basic tombstone compaction test for size tiered
compaction/stcs: add support for tombstone compaction
tests: add test for estimation of droppable tombstone ratio
sstables: introduce function to estimate droppable tombstone ratio
compaction_manager: periodically submit cfs for compaction
streaming_histogram: fix coding style
tests: add streaming_histogram_test
streaming_histogram: implement sum
tests: add test for sstable with bad tombstone histogram
sstables: discard bad streaming histogram for future use
tests: add sstable tombstone histogram test
streaming_histogram: fix update
streaming_histogram: move it to utils
streaming_histogram: do not limit it to be used by sstables
sstables: update tombstone_histogram for cells with expiration time
fc966c0c4c