mirrors/scylladb
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
a1ab46d54d3fa8554070f4bbd2183e4c19d87ee1
scylladb/utils
History
Tomasz Grabiec 86ebca4621 utils: cached_file: Mark permit as awaiting on page miss 
Otherwise, the read will be considered as on-cpu during promoted index
search, which will severely underutlize the disk because by default
on-cpu concurrency is 1.

I verified this patch on the worst case scenario, where the workload
reads missing rows from a large partition. So partition index is
cached (no IO) and there is no data file IO. But there is IO during
promoted index search (via cached_file). Before the patch this
workload was doing 4k req/s, after the patch it does 30k req/s.

The problem is much less pronounced if there is data file or index
file IO involved because that IO will signal read concurrency
semaphore to invite more concurrency.

(cherry picked from commit 0f2101b055)
2024-12-09 17:45:04 +01:00
..
abi
util: replace <tab> with spaces
2023-09-11 14:38:46 +03:00
arch/powerpc/crc32-vpmsum
…
gz
utils/gs/barrett.hh: aarch64: s/brarett/barrett/
2022-11-17 11:09:59 +02:00
lsa
utils/stall_free: introduce reserve_gently
2024-06-18 23:36:30 +05:30
s3
code-cleanup: add missing header guards
2024-07-09 18:31:35 +03:00
allocation_strategy.hh
logalloc: add hold_reserve
2024-07-08 16:08:27 +02:00
amortized_reserve.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
anchorless_list.hh
treewide: use defaulted operator!=() and operator==()
2023-04-27 10:24:46 +03:00
array-search.cc
…
array-search.hh
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
ascii.cc
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
ascii.hh
…
atomic_vector.hh
utils: atomic_vector: mark for_each functions as const
2023-09-04 16:14:38 +03:00
aws_sigv4.cc
utils/aws: Export timepoint formatter
2023-04-17 11:14:45 +03:00
aws_sigv4.hh
utils/aws: Brush up the aws_sigv4.hh header
2023-04-17 11:16:45 +03:00
base64.cc
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
base64.hh
…
big_decimal.cc
treewide: do not include unused header
2023-05-07 19:01:29 +03:00
big_decimal.hh
treewide: replace formatter<std::string_view> with formatter<string_view>
2024-04-19 07:44:07 +03:00
bit_cast.hh
utils: bit_cast: drop unused #includes
2023-12-12 21:09:51 +08:00
bloom_calculations.cc
bloom_filter: mark internal help function static
2023-03-24 15:41:45 +08:00
bloom_calculations.hh
bloom_filter: add more constness to false positive rate tables
2023-03-24 15:41:45 +08:00
bloom_filter.cc
utils/i_filter: introduce get_filter_size()
2024-06-24 12:06:01 +05:30
bloom_filter.hh
utils/i_filter: introduce get_filter_size()
2024-06-24 12:06:01 +05:30
bounded_stats_deque.hh
…
bptree.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
buffer_input_stream.cc
…
buffer_input_stream.hh
…
buffer_view-to-managed_bytes_view.hh
utils/managed_bytes, serializer: add conversion between buffer_view<bytes_ostream> and managed_bytes_view
2023-05-07 17:17:34 +03:00
build_id.cc
…
build_id.hh
…
cached_file_stats.hh
utils: cached_file: deglobalize cached_file metrics
2023-09-01 22:34:41 +02:00
cached_file.hh
utils: cached_file: Mark permit as awaiting on page miss
2024-12-09 17:45:04 +01:00
chunked_vector.hh
utils: chunked_vector: add ctor from std::initializer_list
2024-06-25 12:08:06 +03:00
class_registrator.hh
…
clmul.hh
utils: clmul: add 64-bit constexpr clmul
2022-11-12 17:04:05 +02:00
CMakeLists.txt
utils: add on_internal_error with common logger
2024-01-31 16:45:09 +02:00
coarse_steady_clock.hh
logalloc: prevent false positives in reclaim_timer
2022-10-02 13:41:40 +03:00
collection-concepts.hh
…
compact-radix-tree.hh
Typos: fix typos in code
2023-12-05 15:18:11 +02:00
composite_abort_source.hh
everywhere: include seastar headers using angle brackets
2024-05-06 10:00:31 +03:00
config_file_impl.hh
Merge 'config: round-trip boolean configuration variables' from Avi Kivity
2024-09-13 17:54:37 +03:00
config_file.cc
config: avoid binding an lvalue reference to an rvalue reference
2024-06-27 19:36:13 +03:00
config_file.hh
config: avoid binding an lvalue reference to an rvalue reference
2024-06-27 19:36:13 +03:00
contiguous_shared_buffer.hh
Merge 'sstables: Fix use-after-free on page cache buffer when parsing promoted index entries across pages' from Tomasz Grabiec
2024-12-05 09:50:07 +02:00
coroutine.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
crc.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
cross-shard-barrier.hh
cross-shard-barrier: Capture shared barrier in complete
2022-09-16 08:21:02 +03:00
data_input.hh
…
date.h
treewide: apply codespell to the comments in source code
2023-12-20 10:25:03 +02:00
digest_algorithm.hh
feature: grandfather DIGEST_FOR_NULL_VALUES
2024-05-18 00:24:00 +03:00
digester.hh
feature: grandfather DIGEST_FOR_NULL_VALUES
2024-05-18 00:24:00 +03:00
directories.cc
treewide: include fmt/ranges.h and/or fmt/std.h
2024-04-19 22:56:16 +08:00
directories.hh
directories: prevent inode cache fragmentation by orderly verifying data directory contents
2024-02-01 12:20:23 +05:30
disk-error-handler.cc
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
disk-error-handler.hh
treewide: replace std::result_of_t with std::invoke_result_t
2024-05-26 16:45:42 +03:00
div_ceil.hh
…
double-decker.hh
treewide: use defaulted operator!=() and operator==()
2023-04-27 10:24:46 +03:00
dynamic_bitset.cc
…
dynamic_bitset.hh
…
entangled.hh
…
enum_option.hh
config, enum_option: allow round-trip string conversion
2024-07-10 20:39:01 +03:00
error_injection.cc
…
error_injection.hh
utils/error_injection: add set_parameter()
2024-06-11 04:17:48 -04:00
estimated_histogram.hh
utils/estimated_histogram: estimated_histogram: add constructor taking buckets
2024-03-13 02:06:30 -04:00
exception_container.hh
treewide: replace formatter<std::string_view> with formatter<string_view>
2024-04-19 07:44:07 +03:00
exceptions.cc
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
exceptions.hh
exceptions: s/#warn/#warning/
2024-02-01 14:50:17 +02:00
exponential_backoff_retry.hh
treewide: replace std::result_of_t with std::invoke_result_t
2024-05-26 16:45:42 +03:00
extremum_tracking.hh
…
file_lock.cc
utils: Remove unused operator<< for file_lock object
2024-02-02 15:20:40 +01:00
file_lock.hh
utils: Remove unused operator<< for file_lock object
2024-02-02 15:20:40 +01:00
flush_queue.hh
treewide: replace std::result_of_t with std::invoke_result_t
2024-05-26 16:45:42 +03:00
fmt-compat.hh
…
fragment_range.hh
treewide: replace formatter<std::string_view> with formatter<string_view>
2024-04-19 07:44:07 +03:00
fragmented_temporary_buffer.hh
Merge 'sstables: Fix use-after-free on page cache buffer when parsing promoted index entries across pages' from Tomasz Grabiec
2024-12-05 09:50:07 +02:00
hash.hh
…
hashers.cc
utils: hashing: add simple_xx_hasher
2023-04-24 14:06:43 +03:00
hashers.hh
utils: hashers: add HasherReturning concept
2023-04-24 14:06:40 +03:00
hashing.hh
treewide: include used headers
2024-05-27 17:34:38 +03:00
histogram_metrics_helper.cc
…
histogram_metrics_helper.hh
histogram_metrics_helper: support native histogram
2024-01-23 13:12:34 +02:00
histogram.hh
utils/histogram.hh: Make summary support inifinite bucket.
2024-08-27 12:12:39 +00:00
http.hh
code-cleanup: add missing header guards
2024-07-09 18:31:35 +03:00
human_readable.cc
utils/human_readable: add fmt::formatter for human_readable_value
2024-03-12 14:53:55 +08:00
human_readable.hh
treewide: replace formatter<std::string_view> with formatter<string_view>
2024-04-19 07:44:07 +03:00
i_filter.cc
utils/i_filter: introduce get_filter_size()
2024-06-24 12:06:01 +05:30
i_filter.hh
utils/i_filter: introduce get_filter_size()
2024-06-24 12:06:01 +05:30
immutable-collection.hh
…
input_stream.hh
…
int_range.hh
interval: rename nonwrapping_interval to interval
2024-02-21 19:43:17 +02:00
intrusive_btree.hh
Typos: fix typos in code
2023-12-05 15:18:11 +02:00
intrusive-array.hh
Typos: fix typos in code
2023-12-05 15:18:11 +02:00
large_bitset.cc
utils/large_bitset: replace reserve_partial with utils::reserve_gently
2024-06-18 23:36:30 +05:30
large_bitset.hh
…
latency.hh
…
lexicographical_compare.hh
treewide: use std::lexicographical_compare_threeway
2023-04-21 14:28:18 +03:00
like_matcher.cc
…
like_matcher.hh
…
limiting_data_source.cc
…
limiting_data_source.hh
…
linearizing_input_stream.hh
…
lister.cc
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
lister.hh
utils/lister: Limit the API of scan_dir() to fs::path
2023-09-05 20:50:42 +03:00
loading_cache.hh
treewide: replace seastar::future::get0() with seastar::future::get()
2024-02-02 22:12:57 +08:00
loading_shared_values.hh
loading_shared_values: Replace static-assert with concept
2024-05-09 16:29:49 +03:00
log_heap.hh
treewide: use defaulted operator!=() and operator==()
2023-04-27 10:24:46 +03:00
logalloc.cc
logalloc: add hold_reserve
2024-07-08 16:08:27 +02:00
logalloc.hh
test/perf: report also log_allocations/op
2024-05-02 18:42:41 +03:00
lru.hh
config: add index_cache_fraction
2023-09-01 22:34:23 +02:00
managed_bytes.cc
utils/managed_bytes: add fmt::formatters for managed_bytes and friends
2024-02-23 11:32:41 +08:00
managed_bytes.hh
utils/managed_bytes: add support for fmt::to_string() to bytes and friends
2024-04-19 22:56:13 +08:00
managed_ref.hh
…
managed_vector.hh
util: do not use variable length array
2023-11-20 23:02:41 +02:00
maybe_yield.hh
…
memory_data_sink.hh
s3/client: Unmark put-object lambdas from mutable
2024-07-04 10:07:48 +03:00
memory_limit_reached.hh
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
multiprecision_int.cc
…
multiprecision_int.hh
treewide: use defaulted operator!=() and operator==()
2023-04-27 10:24:46 +03:00
murmur_hash.cc
utils/murmur_hash: replace rotl64() with std::rotl()
2024-06-24 08:24:43 +03:00
murmur_hash.hh
utils/murmur_hash: replace rotl64() with std::rotl()
2024-06-24 08:24:43 +03:00
mutable_view.hh
utils: managed_bytes: rewrite managed_bytes methods in terms of managed_bytes_view
2024-02-09 17:00:33 +01:00
neat-object-id.hh
…
observable.hh
utils: observer: qualify seastar::noncopyable_function
2022-11-28 21:58:30 +02:00
on_internal_error.cc
utils: add on_internal_error with common logger
2024-01-31 16:45:09 +02:00
on_internal_error.hh
utils: add on_internal_error with common logger
2024-01-31 16:45:09 +02:00
overloaded_functor.hh
…
phased_barrier.hh
…
preempt.hh
utils: preempt: add preemption_source
2024-02-07 18:31:28 +01:00
pretty_printers.cc
utils/pretty_printers: add "I" specifier support
2024-01-11 14:33:47 +08:00
pretty_printers.hh
treewide: fix misspellings in code comments
2024-01-31 09:16:10 +02:00
ranges.hh
…
rate_limiter.cc
…
rate_limiter.hh
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
recent_entries_map.hh
utils: introduce recent_entries_map datatype to track least recent visited entries.
2023-02-03 19:04:32 +01:00
result_combinators.hh
…
result_loop.hh
treewide: replace seastar::future::get0() with seastar::future::get()
2024-02-02 22:12:57 +08:00
result_try.hh
util/result_try: pass template arg list explicitly
2024-06-05 13:19:45 +08:00
result.hh
…
reusable_buffer.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
rjson.cc
alternator: yield while converting large responses to json text
2024-06-02 13:07:13 +03:00
rjson.hh
treewide: fix typos in comment and error messages
2024-05-26 11:54:36 +03:00
rpc_utils.hh
storage_proxy: handler responses, use pointers to default constructed values instead of nulls
2023-06-26 11:10:38 +03:00
runtime.cc
…
runtime.hh
…
sequenced_set.hh
sequenced_set: add extract_vector method
2023-05-21 11:33:38 +04:00
serialization.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
serialized_action.hh
serialized_action: make serialized_action abortable
2023-05-11 16:31:23 +03:00
simple_hashers.hh
utils: hashing: add simple_xx_hasher
2023-04-24 14:06:43 +03:00
small_vector.hh
treewide: do not define FMT_DEPRECATED_OSTREAM
2024-04-19 22:57:36 +08:00
sorting.hh
code-cleanup: add missing header guards
2024-07-09 18:31:35 +03:00
stall_free.hh
utils/stall_free: introduce reserve_gently
2024-06-18 23:36:30 +05:30
streaming_histogram.hh
…
tagged_integer.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
throttle.hh
test: Extract throttle object to separate header
2023-06-21 00:58:24 +02:00
to_string.cc
nodetool: rebuild: add force option
2024-08-21 09:37:14 +03:00
to_string.hh
utils/to_string: include fmt/std.h if fmt >= v10
2024-04-23 12:09:05 +03:00
top_k.hh
treewide: include used headers
2024-05-27 17:34:38 +03:00
tuple_utils.hh
storage_proxy.cc: refactor encode_replica_exception_for_rpc
2023-06-15 15:52:50 +04:00
unconst.hh
db: Introduce mutation_partition_v2
2023-01-27 19:15:39 +01:00
updateable_value.cc
…
updateable_value.hh
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
user_provided_param.hh
nodetool: rebuild: add force option
2024-08-21 09:37:14 +03:00
utf8.cc
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
utf8.hh
…
UUID_gen.cc
utils: UUID_gen: include <atomic>
2024-04-30 09:07:22 +03:00
UUID_gen.hh
utils: UUID: Make get_time_UUID() respect the clock offset
2024-12-04 14:19:47 +01:00
uuid.cc
utils: Add uuid_xor_to_uint32 helper
2024-02-01 10:27:55 +02:00
UUID.hh
treewide: replace formatter<std::string_view> with formatter<string_view>
2024-04-19 07:44:07 +03:00
variant_element.hh
utils/variant_element: simplify is_variant_element with right fold
2022-11-27 16:34:34 +02:00
vle.hh
…
xx_hasher.hh
feature: grandfather DIGEST_FOR_NULL_VALUES
2024-05-18 00:24:00 +03:00