mirror of
https://github.com/scylladb/scylladb.git
synced 2026-04-23 18:10:39 +00:00
9fd312d157
https://github.com/scylladb/scylladb/issues/24962 introduced memtable overlap checks to cache tombstone GC. This was observed to be very strict and greatly reduce the effectiveness of tombstone GC in the cache, especially for MV workloads, which regularly recycle old timestamp into new writes, so the memtable often has smaller min live timestamp than the timestamp of the tombstones in the cache. When creating a new memtable, save a snapshot of the tombstone gc state. This snapshot is used later to exclude this memtable from overlap checks for tombstones, whose token have an expiry time larger than that of the tombstone, meaning: all writes in this memtable were produced at a point in time when the current tombstone has already expired. This has the following implications: * The partition the tombstone is part of was already repaired at the time the memtable was created. * All writes in the memtable were produced *after* this tombstone's expiry time, these writes cannot be possibly relevant for this tombstone. Based on this, such memtables are excluded from the overlap checks. With adequately frequent memtable flushes -- so that the tombstone gc state snapshot is refreshed -- most memtables should be excluded from overlap checks, greatly helping the cache's tombstone GC efficiency. Fixes: https://github.com/scylladb/scylladb/issues/24962 Fixes a regression introduced by https://github.com/scylladb/scylladb/pull/23255 which was backported to all releases, needs backport to all releases as well Closes scylladb/scylladb#25033 * github.com:scylladb/scylladb: docs/dev/tombstone.md: document the memtable overlap check elision optimization test/boost/row_cache_test: add test for memtable overlap check elision db/cache_mutation_reader: obtain gc-before and min-live-ts lazily mutation/mutation_compactor: use max_purgeable::can_purge and max_purgeable::purge_result db/cache_mutation_reader: use max_purgeable::can_purge() replica/table: get_max_purgeable_fn_for_cache_underlying_reader(): use max_purgable::combine() replica/database: memtable_list::get_max_purgeable(): set expiry-treshold compaction/compaction_garbage_collector: max_purgeable: add expiry_treshold replica/table: propagate gc_state to memtable_list replica/memtable_list: add tombstone_gc_state* member replica/memtable: add tombstone_gc_state_snapshot tombstone_gc: introduce tombstone_gc_state_snapshot tombstone_gc: extract shared state into shared_tombstone_gc_state tombstone_gc: per_table_history_maps::_group0_gc_time: make it a value tombstone_gc: fold get_group0_gc_time() into its caller tombstone_gc: fold get_or_create_group0_gc_time() into update_group0_refresh_time() tombstone_gc: fold get_or_create_repair_history_for_table() into update_repair_time() tombstone_gc: refactor get_or_greate_repair_history_for_table() replica/memtable_list: s/min_live_timestamp()/get_max_purgeable()/ db/read_context: return max_purgeable from get_max_purgeable() compaction/compaction_garbage_collector: add formatter for max_purgeable mutation: move definition of gc symbols to compaction.cc compaction/compaction_garbage_collector: refactor max_purgeable into a class test/boost/row_cache_test: refactor test_populating_reader_tombstone_gc_with_data_in_memtable test: rewrite test_compacting_reader_tombstone_gc_with_data_in_memtable in C++ test/boost/row_cache_test: refactor cache tombstone GC with memtable overlap tests