sstables: Dynamically adjust I/O size for index reads
Currently, index reader uses 128 KiB I/O size with read-ahead. That is a waste of bandwidth if index entries contain large promoted index and binary search will be used within the promoted index, which may not need to access as much. The read-ahead is wasted both when using binary search and when using the scanning cursor. On the other hand, large I/O is optimal if there is no promoted index and we're going to parse the whole page. There is no way to predict which case it is up front before reading the index. Attaching dynamic adjustments (per-sstable) lets the system auto adjust to the workload from past history. The large promoted index workload will settle on reading 32 KiB (with read-ahead). This is still not optimal, we should lower the buffer size even more. But that requires a seastar change, so is deferred.
This commit is contained in:
@@ -360,6 +360,7 @@ class index_reader {
|
||||
options.buffer_size = sst->sstable_buffer_size;
|
||||
options.read_ahead = 2;
|
||||
options.io_priority_class = pc;
|
||||
options.dynamic_adjustments = sst->_index_history;
|
||||
return options;
|
||||
}
|
||||
|
||||
|
||||
@@ -488,6 +488,7 @@ private:
|
||||
|
||||
lw_shared_ptr<file_input_stream_history> _single_partition_history = make_lw_shared<file_input_stream_history>();
|
||||
lw_shared_ptr<file_input_stream_history> _partition_range_history = make_lw_shared<file_input_stream_history>();
|
||||
lw_shared_ptr<file_input_stream_history> _index_history = make_lw_shared<file_input_stream_history>();
|
||||
|
||||
//FIXME: Set by sstable_writer to influence sstable writing behavior.
|
||||
// Remove when doing #3012
|
||||
|
||||
Reference in New Issue
Block a user