mirror of
https://github.com/scylladb/scylladb.git
synced 2026-05-22 15:52:13 +00:00
3fce43124a
Allows static configuration of number of compaction groups per table per shard. To bootstrap the project, config option x_log2_compaction_groups was added which controls both number of groups and partitioning within a shard. With a value of 0 (default), it means 1 compaction group, therefore all tokens go there. With a value of 3, it means 8 compaction groups, and 3 most-significant-bits of tokens being used to decide which group owns the token. And so on. It's still missing: - integration with repair / streaming - integration with reshard / reshape. perf/perf_simple_query --smp 1 --memory 1G BEFORE ----- median 61358.55 tps ( 71.1 allocs/op, 12.2 tasks/op, 56375 insns/op, 0 errors) median 61322.80 tps ( 71.1 allocs/op, 12.2 tasks/op, 56391 insns/op, 0 errors) median 61058.58 tps ( 71.1 allocs/op, 12.2 tasks/op, 56386 insns/op, 0 errors) median 61040.94 tps ( 71.1 allocs/op, 12.2 tasks/op, 56381 insns/op, 0 errors) median 61118.40 tps ( 71.1 allocs/op, 12.2 tasks/op, 56379 insns/op, 0 errors) AFTER ----- median 61656.12 tps ( 71.1 allocs/op, 12.2 tasks/op, 56486 insns/op, 0 errors) median 61483.29 tps ( 71.1 allocs/op, 12.2 tasks/op, 56495 insns/op, 0 errors) median 61638.05 tps ( 71.1 allocs/op, 12.2 tasks/op, 56494 insns/op, 0 errors) median 61726.09 tps ( 71.1 allocs/op, 12.2 tasks/op, 56509 insns/op, 0 errors) median 61537.55 tps ( 71.1 allocs/op, 12.2 tasks/op, 56491 insns/op, 0 errors) Closes #12139 * github.com:scylladb/scylladb: test: mutation_test: Test multiple compaction groups test: database_test: Test multiple compaction groups test: database_test: Adapt it to compaction groups db: Add config for setting static number of compaction groups replica: Introduce static compaction groups test: sstable_test: Stop referencing single compaction group api: compaction_manager: Stop a compaction type for all groups api: Estimate pending tasks on all compaction groups api: storage_service: Run maintenance compactions on all compaction groups replica: table: Adapt assertion to compaction groups replica: database: stop and disable compaction on behalf of all groups replica: Introduce table::parallel_foreach_table_state() replica: disable auto compaction on behalf of all groups replica: table: Rework compaction triggers for compaction groups replica: Adapt table::get_sstables_including_compacted_undeleted() to compaction groups replica: Adapt table::rebuild_statistics() to compaction groups replica: table: Perform major compaction on behalf of all groups replica: table: Perform off-strategy compaction on behalf of all groups replica: table: Perform cleanup compaction on behalf of all groups replica: Extend table::discard_sstables() to operate on all compaction groups replica: table: Create compound sstable set for all groups replica: table: Set compaction strategy on behalf of all groups replica: table: Return min memtable timestamp across all groups replica: Adapt table::stop() to compaction groups replica: Adapt table::clear() to compaction groups replica: Adapt table::can_flush() to compaction groups replica: Adapt table::flush() to compaction groups replica: Introduce parallel_foreach_compaction_group() replica: Adapt table::set_schema() to compaction groups replica: Add memtables from all compaction groups for reads replica: Add memtable_count() method to compaction_group replica: table: Reserve reader list capacity through a callback replica: Extract addition of memtables to reader list into a new function replica: Adapt table::occupancy() to compaction groups replica: Adapt table::active_memtable() to compaction groups replica: Introduce table::compaction_groups() replica: Preparation for multiple compaction groups scylla-gdb: Fix backward compatibility of scylla_memtables command