mirrors/scylladb
1
0
Fork 0
mirror of https://github.com/scylladb/scylladb.git synced 2026-04-21 17:10:35 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
020ccbd76a134ed9642480a36fa15b4e902f30b6
scylladb/utils
History
Avi Kivity 020ccbd76a Merge 'utils: cached_file: Mark permit as awaiting on page miss' from Tomasz Grabiec 
Otherwise, the read will be considered as on-cpu during promoted index
search, which will severely underutlize the disk because by default
on-cpu concurrency is 1.

I verified this patch on the worst case scenario, where the workload
reads missing rows from a large partition. So partition index is
cached (no IO) and there is no data file IO (relies on https://github.com/scylladb/scylladb/pull/20522).
But there is IO during promoted index search (via cached_file).

Before the patch this workload was doing 4k req/s, after the patch it does 30k req/s.

The problem is much less pronounced if there is data file or partition index IO involved
because that IO will signal read concurrency semaphore to invite more concurrency.

Fixes #21325

Closes scylladb/scylladb#21323

* github.com:scylladb/scylladb:
  utils: cached_file: Mark permit as awaiting on page miss
  utils: cached_file: Push resource_unit management down to cached_file
2024-10-29 16:15:21 +02:00
..
abi
util: replace <tab> with spaces
2023-09-11 14:38:46 +03:00
arch/powerpc/crc32-vpmsum
…
gz
…
lsa
utils: lsa: chunked_managed_vector: replace boost with std
2024-10-08 12:03:30 +03:00
s3
Merge 'Do not leak S3 file-uploading parts on exceptions' from Pavel Emelyanov
2024-10-23 10:12:29 +03:00
allocation_strategy.hh
logalloc: add hold_reserve
2024-07-08 16:08:27 +02:00
amortized_reserve.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
anchorless_list.hh
…
array-search.cc
…
array-search.hh
treewide: de-static namespace scope functions in headers
2024-10-01 14:02:50 +03:00
ascii.cc
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
ascii.hh
…
assert.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
atomic_vector.hh
utils: atomic_vector: mark for_each functions as const
2023-09-04 16:14:38 +03:00
aws_sigv4.cc
treewide: use seastar::format() or fmt::format() explicitly
2024-09-11 23:21:40 +03:00
aws_sigv4.hh
treewide: de-static namespace scope functions in headers
2024-10-01 14:02:50 +03:00
base64.cc
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
base64.hh
…
big_decimal.cc
treewide: use seastar::format() or fmt::format() explicitly
2024-09-11 23:21:40 +03:00
big_decimal.hh
treewide: replace formatter<std::string_view> with formatter<string_view>
2024-04-19 07:44:07 +03:00
bit_cast.hh
utils: bit_cast: drop unused #includes
2023-12-12 21:09:51 +08:00
bloom_calculations.cc
…
bloom_calculations.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
bloom_filter.cc
utils/i_filter: introduce get_filter_size()
2024-06-24 12:06:01 +05:30
bloom_filter.hh
utils/i_filter: introduce get_filter_size()
2024-06-24 12:06:01 +05:30
bounded_stats_deque.hh
…
bptree.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
buffer_input_stream.cc
…
buffer_input_stream.hh
…
buffer_view-to-managed_bytes_view.hh
…
build_id.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
build_id.hh
…
cached_file_stats.hh
utils: cached_file: deglobalize cached_file metrics
2023-09-01 22:34:41 +02:00
cached_file.hh
utils: cached_file: Mark permit as awaiting on page miss
2024-10-28 19:54:58 +01:00
chunked_vector.hh
utils: chunked_vector: replace boost with std
2024-10-07 21:11:23 +03:00
class_registrator.hh
utils: class_registrator: replace boost with std
2024-10-07 21:16:03 +03:00
clmul.hh
…
CMakeLists.txt
build: cmake: detect and link against GnuTLS library
2024-10-18 08:36:52 +08:00
coarse_steady_clock.hh
…
collection-concepts.hh
…
compact-radix-tree.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
composite_abort_source.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
config_file_impl.hh
config: specialize from-string conversion for bool
2024-07-18 18:38:22 +03:00
config_file.cc
utils: config_file: replace boost with std
2024-10-08 12:03:15 +03:00
config_file.hh
config: do not provide default value for set_value() and friends
2024-09-25 15:45:42 +03:00
contiguous_shared_buffer.hh
sstables, utils: Allow parsers to work with different buffer types
2024-09-27 01:24:54 +02:00
coroutine.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
crc.hh
utils: crc: replace boost for_each with built-in range for
2024-10-07 21:19:14 +03:00
cross-shard-barrier.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
data_input.hh
…
date.h
date: do not put space before literal operator
2024-10-21 11:21:52 +03:00
digest_algorithm.hh
feature: grandfather DIGEST_FOR_NULL_VALUES
2024-05-18 00:24:00 +03:00
digester.hh
feature: grandfather DIGEST_FOR_NULL_VALUES
2024-05-18 00:24:00 +03:00
directories.cc
directories: mark verification_error() with [[noreturn]]
2024-09-30 12:07:15 +08:00
directories.hh
directories: prevent inode cache fragmentation by orderly verifying data directory contents
2024-02-01 12:20:23 +05:30
disk-error-handler.cc
extensions: Add exception types for IO extensions and handle in memtable write path
2024-08-11 13:52:35 +03:00
disk-error-handler.hh
treewide: replace std::result_of_t with std::invoke_result_t
2024-05-26 16:45:42 +03:00
div_ceil.hh
utils/div_ceil: add constraints to template arguments
2024-08-04 15:32:01 +03:00
double-decker.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
dynamic_bitset.cc
…
dynamic_bitset.hh
…
entangled.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
enum_option.hh
config, enum_option: allow round-trip string conversion
2024-07-10 20:39:01 +03:00
error_injection.cc
…
error_injection.hh
error_injection: Re-use enter() code in inject() overloads
2024-10-28 21:37:20 +02:00
estimated_histogram.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
exception_container.hh
treewide: replace formatter<std::string_view> with formatter<string_view>
2024-04-19 07:44:07 +03:00
exceptions.cc
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
exceptions.hh
exceptions: s/#warn/#warning/
2024-02-01 14:50:17 +02:00
exponential_backoff_retry.hh
treewide: replace std::result_of_t with std::invoke_result_t
2024-05-26 16:45:42 +03:00
extremum_tracking.hh
…
file_lock.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
file_lock.hh
utils: Remove unused operator<< for file_lock object
2024-02-02 15:20:40 +01:00
flush_queue.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
fmt-compat.hh
…
fragment_range.hh
utils: fragment_range: replace boost with std
2024-10-07 21:32:16 +03:00
fragmented_temporary_buffer.hh
sstables, utils: Allow parsers to work with different buffer types
2024-09-27 01:24:54 +02:00
hash.hh
…
hashers.cc
…
hashers.hh
Merge 'db: remove unused includes ' from Kefu Chai
2024-10-07 10:55:48 +03:00
hashing.hh
treewide: include used headers
2024-05-27 17:34:38 +03:00
histogram_metrics_helper.cc
…
histogram_metrics_helper.hh
histogram_metrics_helper: support native histogram
2024-01-23 13:12:34 +02:00
histogram.hh
utils/histogram.hh: Make summary support inifinite bucket.
2024-08-22 23:34:24 +03:00
http.hh
treewide: move log.hh into utils/log.hh
2024-10-22 06:54:46 +03:00
human_readable.cc
utils/human_readable: add fmt::formatter for human_readable_value
2024-03-12 14:53:55 +08:00
human_readable.hh
treewide: replace formatter<std::string_view> with formatter<string_view>
2024-04-19 07:44:07 +03:00
i_filter.cc
treewide: move log.hh into utils/log.hh
2024-10-22 06:54:46 +03:00
i_filter.hh
Merge 'db: remove unused includes ' from Kefu Chai
2024-10-07 10:55:48 +03:00
immutable-collection.hh
…
input_stream.hh
…
int_range.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
intrusive_btree.hh
utils: intrusive_btree: improve conformity with iterator requirements
2024-10-07 17:26:01 +03:00
intrusive-array.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
large_bitset.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
large_bitset.hh
…
latency.hh
…
lexicographical_compare.hh
…
like_matcher.cc
…
like_matcher.hh
…
limiting_data_source.cc
…
limiting_data_source.hh
…
linearizing_input_stream.hh
…
lister.cc
treewide: use seastar::format() or fmt::format() explicitly
2024-09-11 23:21:40 +03:00
lister.hh
treewide: de-static namespace scope functions in headers
2024-10-01 14:02:50 +03:00
loading_cache.hh
treewide: move log.hh into utils/log.hh
2024-10-22 06:54:46 +03:00
loading_shared_values.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
log_heap.hh
…
log.hh
treewide: move log.hh into utils/log.hh
2024-10-22 06:54:46 +03:00
logalloc.cc
treewide: move log.hh into utils/log.hh
2024-10-22 06:54:46 +03:00
logalloc.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
lru.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
managed_bytes.cc
utils/managed_bytes: add fmt::formatters for managed_bytes and friends
2024-02-23 11:32:41 +08:00
managed_bytes.hh
utils/managed_bytes: add support for fmt::to_string() to bytes and friends
2024-04-19 22:56:13 +08:00
managed_ref.hh
…
managed_vector.hh
util: do not use variable length array
2023-11-20 23:02:41 +02:00
maybe_yield.hh
…
memory_data_sink.hh
s3/client: Unmark put-object lambdas from mutable
2024-07-04 10:07:48 +03:00
memory_limit_reached.hh
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
multiprecision_int.cc
…
multiprecision_int.hh
…
murmur_hash.cc
utils/murmur_hash: replace rotl64() with std::rotl()
2024-06-24 08:24:43 +03:00
murmur_hash.hh
treewide: de-static namespace scope functions in headers
2024-10-01 14:02:50 +03:00
mutable_view.hh
utils: managed_bytes: rewrite managed_bytes methods in terms of managed_bytes_view
2024-02-09 17:00:33 +01:00
neat-object-id.hh
…
observable.hh
utils: observable: replace boost with std
2024-10-07 21:11:07 +03:00
on_internal_error.cc
utils: add on_internal_error with common logger
2024-01-31 16:45:09 +02:00
on_internal_error.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
overloaded_functor.hh
…
phased_barrier.hh
…
preempt.hh
utils: preempt: add preemption_source
2024-02-07 18:31:28 +01:00
pretty_printers.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
pretty_printers.hh
treewide: fix misspellings in code comments
2024-01-31 09:16:10 +02:00
rate_limiter.cc
…
rate_limiter.hh
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
recent_entries_map.hh
…
result_combinators.hh
…
result_loop.hh
treewide: replace seastar::future::get0() with seastar::future::get()
2024-02-02 22:12:57 +08:00
result_try.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
result.hh
…
reusable_buffer.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
rjson.cc
utils/rjson.cc: include the function name in exception message
2024-09-12 15:22:49 +03:00
rjson.hh
utils/rjson.cc: correct a comment about assert()
2024-10-06 12:47:51 +03:00
rpc_utils.hh
storage_proxy: handler responses, use pointers to default constructed values instead of nulls
2023-06-26 11:10:38 +03:00
runtime.cc
…
runtime.hh
…
sequenced_set.hh
…
serialization.hh
treewide: de-static namespace scope functions in headers
2024-10-01 14:02:50 +03:00
serialized_action.hh
…
simple_hashers.hh
…
small_vector.hh
utils/small_vector: refactor expansion condition in reserve*()
2024-10-22 17:12:11 +03:00
sorting.hh
utils/sorting: allow to pass any container as verticies
2024-08-08 10:42:09 +02:00
stall_free.hh
utils/stall_free: introduce reserve_gently
2024-06-18 23:36:30 +05:30
streaming_histogram.hh
…
tagged_integer.hh
utils/tagged_integer: remove conversion to underlying integer
2024-08-15 02:12:58 +02:00
throttle.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
to_string.cc
nodetool: rebuild: add force option
2024-08-19 17:20:12 +03:00
to_string.hh
utils/to_string: include fmt/std.h if fmt >= v10
2024-04-23 12:09:05 +03:00
top_k.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
tuple_utils.hh
storage_proxy.cc: refactor encode_replica_exception_for_rpc
2023-06-15 15:52:50 +04:00
unconst.hh
utils/unconst, mutation_partition: switch to ranges
2024-10-07 17:30:12 +03:00
updateable_value.cc
…
updateable_value.hh
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
user_provided_param.hh
nodetool: rebuild: add force option
2024-08-19 17:20:12 +03:00
utf8.cc
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
utf8.hh
…
UUID_gen.cc
utils: UUID_gen: include <atomic>
2024-04-30 09:07:22 +03:00
UUID_gen.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
uuid.cc
treewide: use seastar::format() or fmt::format() explicitly
2024-09-11 23:21:40 +03:00
UUID.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
value_or_reference.hh
kl::reader::make_reader: Unify interface with mx::reader::make_reader
2024-08-13 10:02:43 +02:00
variant_element.hh
…
vle.hh
treewide: de-static namespace scope functions in headers
2024-10-01 14:02:50 +03:00
xx_hasher.hh
feature: grandfather DIGEST_FOR_NULL_VALUES
2024-05-18 00:24:00 +03:00