mirrors/scylladb
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
2ea97d8c19dbad31991a93c93b940a773ab5f560
scylladb/utils
History
Tomasz Grabiec 0851e3fba7 Merge '[Backport 6.2] utils: cached_file: Mark permit as awaiting on page miss' from ScyllaDB 
Otherwise, the read will be considered as on-cpu during promoted index
search, which will severely underutlize the disk because by default
on-cpu concurrency is 1.

I verified this patch on the worst case scenario, where the workload
reads missing rows from a large partition. So partition index is
cached (no IO) and there is no data file IO (relies on https://github.com/scylladb/scylladb/pull/20522).
But there is IO during promoted index search (via cached_file).

Before the patch this workload was doing 4k req/s, after the patch it does 30k req/s.

The problem is much less pronounced if there is data file or partition index IO involved
because that IO will signal read concurrency semaphore to invite more concurrency.

Fixes #21325

(cherry picked from commit 868f5b59c4)

(cherry picked from commit 0f2101b055)

Refs #21323

Closes scylladb/scylladb#21358

* github.com:scylladb/scylladb:
  utils: cached_file: Mark permit as awaiting on page miss
  utils: cached_file: Push resource_unit management down to cached_file
2024-12-16 19:55:00 +01:00
..
abi
util: replace <tab> with spaces
2023-09-11 14:38:46 +03:00
arch/powerpc/crc32-vpmsum
…
gz
…
lsa
utils/stall_free: introduce reserve_gently
2024-06-18 23:36:30 +05:30
s3
s3/client: Reuse class multipart_upload in do_upload_file
2024-09-12 18:38:16 +03:00
allocation_strategy.hh
logalloc: add hold_reserve
2024-07-08 16:08:27 +02:00
amortized_reserve.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
anchorless_list.hh
…
array-search.cc
…
array-search.hh
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
ascii.cc
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
ascii.hh
…
assert.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
atomic_vector.hh
utils: atomic_vector: mark for_each functions as const
2023-09-04 16:14:38 +03:00
aws_sigv4.cc
treewide: use seastar::format() or fmt::format() explicitly
2024-09-11 23:21:40 +03:00
aws_sigv4.hh
…
base64.cc
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
base64.hh
…
big_decimal.cc
treewide: use seastar::format() or fmt::format() explicitly
2024-09-11 23:21:40 +03:00
big_decimal.hh
treewide: replace formatter<std::string_view> with formatter<string_view>
2024-04-19 07:44:07 +03:00
bit_cast.hh
utils: bit_cast: drop unused #includes
2023-12-12 21:09:51 +08:00
bloom_calculations.cc
…
bloom_calculations.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
bloom_filter.cc
utils/i_filter: introduce get_filter_size()
2024-06-24 12:06:01 +05:30
bloom_filter.hh
utils/i_filter: introduce get_filter_size()
2024-06-24 12:06:01 +05:30
bounded_stats_deque.hh
…
bptree.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
buffer_input_stream.cc
…
buffer_input_stream.hh
…
buffer_view-to-managed_bytes_view.hh
…
build_id.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
build_id.hh
…
cached_file_stats.hh
utils: cached_file: deglobalize cached_file metrics
2023-09-01 22:34:41 +02:00
cached_file.hh
utils: cached_file: Mark permit as awaiting on page miss
2024-12-09 23:18:00 +01:00
chunked_vector.hh
utils: chunked_vector: add ctor from std::initializer_list
2024-06-25 12:08:06 +03:00
class_registrator.hh
…
clmul.hh
…
CMakeLists.txt
utils: add on_internal_error with common logger
2024-01-31 16:45:09 +02:00
coarse_steady_clock.hh
…
collection-concepts.hh
…
compact-radix-tree.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
composite_abort_source.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
config_file_impl.hh
config: specialize from-string conversion for bool
2024-07-18 18:38:22 +03:00
config_file.cc
config: avoid binding an lvalue reference to an rvalue reference
2024-06-27 19:36:13 +03:00
config_file.hh
treewide: use seastar::format() or fmt::format() explicitly
2024-09-11 23:21:40 +03:00
contiguous_shared_buffer.hh
sstables, utils: Allow parsers to work with different buffer types
2024-10-01 01:38:47 +00:00
coroutine.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
crc.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
cross-shard-barrier.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
data_input.hh
…
date.h
treewide: apply codespell to the comments in source code
2023-12-20 10:25:03 +02:00
digest_algorithm.hh
feature: grandfather DIGEST_FOR_NULL_VALUES
2024-05-18 00:24:00 +03:00
digester.hh
feature: grandfather DIGEST_FOR_NULL_VALUES
2024-05-18 00:24:00 +03:00
directories.cc
treewide: include fmt/ranges.h and/or fmt/std.h
2024-04-19 22:56:16 +08:00
directories.hh
directories: prevent inode cache fragmentation by orderly verifying data directory contents
2024-02-01 12:20:23 +05:30
disk-error-handler.cc
extensions: Add exception types for IO extensions and handle in memtable write path
2024-08-11 13:52:35 +03:00
disk-error-handler.hh
treewide: replace std::result_of_t with std::invoke_result_t
2024-05-26 16:45:42 +03:00
div_ceil.hh
utils/div_ceil: add constraints to template arguments
2024-08-04 15:32:01 +03:00
double-decker.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
dynamic_bitset.cc
…
dynamic_bitset.hh
…
entangled.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
enum_option.hh
config, enum_option: allow round-trip string conversion
2024-07-10 20:39:01 +03:00
error_injection.cc
…
error_injection.hh
utils: Ensure const correctness of injection_handler::get().
2024-08-20 14:15:50 +02:00
estimated_histogram.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
exception_container.hh
treewide: replace formatter<std::string_view> with formatter<string_view>
2024-04-19 07:44:07 +03:00
exceptions.cc
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
exceptions.hh
exceptions: s/#warn/#warning/
2024-02-01 14:50:17 +02:00
exponential_backoff_retry.hh
treewide: replace std::result_of_t with std::invoke_result_t
2024-05-26 16:45:42 +03:00
extremum_tracking.hh
…
file_lock.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
file_lock.hh
utils: Remove unused operator<< for file_lock object
2024-02-02 15:20:40 +01:00
flush_queue.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
fmt-compat.hh
…
fragment_range.hh
treewide: replace formatter<std::string_view> with formatter<string_view>
2024-04-19 07:44:07 +03:00
fragmented_temporary_buffer.hh
sstables, utils: Allow parsers to work with different buffer types
2024-10-01 01:38:47 +00:00
hash.hh
…
hashers.cc
…
hashers.hh
…
hashing.hh
treewide: include used headers
2024-05-27 17:34:38 +03:00
histogram_metrics_helper.cc
…
histogram_metrics_helper.hh
histogram_metrics_helper: support native histogram
2024-01-23 13:12:34 +02:00
histogram.hh
utils/histogram.hh: Make summary support inifinite bucket.
2024-08-22 23:34:24 +03:00
http.hh
code-cleanup: add missing header guards
2024-07-09 18:31:35 +03:00
human_readable.cc
utils/human_readable: add fmt::formatter for human_readable_value
2024-03-12 14:53:55 +08:00
human_readable.hh
treewide: replace formatter<std::string_view> with formatter<string_view>
2024-04-19 07:44:07 +03:00
i_filter.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
i_filter.hh
utils/i_filter: introduce get_filter_size()
2024-06-24 12:06:01 +05:30
immutable-collection.hh
…
input_stream.hh
…
int_range.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
intrusive_btree.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
intrusive-array.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
large_bitset.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
large_bitset.hh
…
latency.hh
…
lexicographical_compare.hh
…
like_matcher.cc
…
like_matcher.hh
…
limiting_data_source.cc
…
limiting_data_source.hh
…
linearizing_input_stream.hh
…
lister.cc
treewide: use seastar::format() or fmt::format() explicitly
2024-09-11 23:21:40 +03:00
lister.hh
directory_lister: Add noexcept default move-constructor
2024-09-05 11:10:21 +03:00
loading_cache.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
loading_shared_values.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
log_heap.hh
…
logalloc.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
logalloc.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
lru.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
managed_bytes.cc
utils/managed_bytes: add fmt::formatters for managed_bytes and friends
2024-02-23 11:32:41 +08:00
managed_bytes.hh
utils/managed_bytes: add support for fmt::to_string() to bytes and friends
2024-04-19 22:56:13 +08:00
managed_ref.hh
…
managed_vector.hh
util: do not use variable length array
2023-11-20 23:02:41 +02:00
maybe_yield.hh
…
memory_data_sink.hh
s3/client: Unmark put-object lambdas from mutable
2024-07-04 10:07:48 +03:00
memory_limit_reached.hh
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
multiprecision_int.cc
…
multiprecision_int.hh
…
murmur_hash.cc
utils/murmur_hash: replace rotl64() with std::rotl()
2024-06-24 08:24:43 +03:00
murmur_hash.hh
utils/murmur_hash: replace rotl64() with std::rotl()
2024-06-24 08:24:43 +03:00
mutable_view.hh
utils: managed_bytes: rewrite managed_bytes methods in terms of managed_bytes_view
2024-02-09 17:00:33 +01:00
neat-object-id.hh
…
observable.hh
…
on_internal_error.cc
utils: add on_internal_error with common logger
2024-01-31 16:45:09 +02:00
on_internal_error.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
overloaded_functor.hh
…
phased_barrier.hh
…
preempt.hh
utils: preempt: add preemption_source
2024-02-07 18:31:28 +01:00
pretty_printers.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
pretty_printers.hh
treewide: fix misspellings in code comments
2024-01-31 09:16:10 +02:00
ranges.hh
…
rate_limiter.cc
…
rate_limiter.hh
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
recent_entries_map.hh
…
result_combinators.hh
…
result_loop.hh
treewide: replace seastar::future::get0() with seastar::future::get()
2024-02-02 22:12:57 +08:00
result_try.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
result.hh
…
reusable_buffer.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
rjson.cc
utils/rjson.cc: include the function name in exception message
2024-09-12 15:22:49 +03:00
rjson.hh
utils/rjson.cc: correct a comment about assert()
2024-10-09 21:50:26 +03:00
rpc_utils.hh
storage_proxy: handler responses, use pointers to default constructed values instead of nulls
2023-06-26 11:10:38 +03:00
runtime.cc
…
runtime.hh
…
sequenced_set.hh
…
serialization.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
serialized_action.hh
…
simple_hashers.hh
…
small_vector.hh
treewide: do not define FMT_DEPRECATED_OSTREAM
2024-04-19 22:57:36 +08:00
sorting.hh
utils/sorting: allow to pass any container as verticies
2024-08-08 10:42:09 +02:00
stall_free.hh
utils/stall_free: introduce reserve_gently
2024-06-18 23:36:30 +05:30
streaming_histogram.hh
…
tagged_integer.hh
utils/tagged_integer: remove conversion to underlying integer
2024-08-15 02:12:58 +02:00
throttle.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
to_string.cc
nodetool: rebuild: add force option
2024-08-19 17:20:12 +03:00
to_string.hh
utils/to_string: include fmt/std.h if fmt >= v10
2024-04-23 12:09:05 +03:00
top_k.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
tuple_utils.hh
storage_proxy.cc: refactor encode_replica_exception_for_rpc
2023-06-15 15:52:50 +04:00
unconst.hh
…
updateable_value.cc
…
updateable_value.hh
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
user_provided_param.hh
nodetool: rebuild: add force option
2024-08-19 17:20:12 +03:00
utf8.cc
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
utf8.hh
…
UUID_gen.cc
utils: UUID_gen: include <atomic>
2024-04-30 09:07:22 +03:00
UUID_gen.hh
utils: UUID: Make get_time_UUID() respect the clock offset
2024-12-04 14:18:16 +01:00
uuid.cc
treewide: use seastar::format() or fmt::format() explicitly
2024-09-11 23:21:40 +03:00
UUID.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
value_or_reference.hh
kl::reader::make_reader: Unify interface with mx::reader::make_reader
2024-08-13 10:02:43 +02:00
variant_element.hh
…
vle.hh
…
xx_hasher.hh
feature: grandfather DIGEST_FOR_NULL_VALUES
2024-05-18 00:24:00 +03:00