mirror of
https://github.com/scylladb/scylladb.git
synced 2026-04-19 16:15:07 +00:00
b58dbe57aa
Add a buffer hint to the multishard reader. This is an internal hint, used by the multishard reader to provide a hint to the shard reader, on how much data exactly is needed by the multishard reader from the respective shard. This hint allows eliminating extraneous cross-shard round-trips and possible shard reader evict-recreate cycles. Building on this, repair sets its own row buffer size as the max buffer size on the multishard reader, ensuring that the row buffer is filled with the minimum amount of cross-shard round trips and minimal reader recreation. To further eliminate unnecessary evictions, this PR also disables the multishard reader's read-ahead which is a mechanism that was designed to reduce latency for user-reads but it can be too aggressive for repair, causing unnecessary extra congestion on the already struggling streaming semaphores. Refs: https://github.com/scylladb/scylladb/issues/18269 Fixes: https://github.com/scylladb/scylladb/issues/21113 The performance impact was measured with an SCT test, which creates a cluster of 3 nodes with 16 shards, then adds a 4th one with 12 shards. Currently, it is the bootstrap time which is the worse in the case of mixed shard clusters, see below for the improvement measured during bootstrap: | | master | buffer-hint | metric | | ------------ | ------------- | ------------- | --------------------------------------------------- | | evictions | 0.9M | 93.0K | scylla_database_paused_reads_permit_based_evictions | | read (bytes) | 9.0T | 3.9T | scylla_reactor_aio_bytes_read | | read (ops) | 88.0M | 33.5M | scylla_reactor_aio_reads | | time | 56min | 20min | N/A | This is a performance improvement, no backport required. Closes scylladb/scylladb#20815 * github.com:scylladb/scylladb: test/boost/mutation_reader_test: add test for multishard reader buffer hint repair/row_level: disable read-ahead db/config: introduce repair_multishard_reader_enable_read_ahead readers/multishard: implement the read_ahead flag replica/database: make_multishard_streaming_reader(): expose the read_ahead parameter readers/multishard: add read_ahead parameter repair/row_level: set max buffer size on multishard reader replica/database: make_multishard_streaming_reader(): expose buffer_hint parameter db/config: introduce enable_repair_multishard_reader_buffer_hint readers/multishard: multishard_reader: pass hint to shard_reader readers/multishard: shard_reader_v2::fill_reader_buffer(): respect the hint readers/multishard: propagate fill_buffer_hint to shard_reader:fill_reader_buffer() readers/multishard: shard_reader: extract buffer-fill into its own method