mirrors/scylladb
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
ec4040496b410041ba5ccfd5fdd2d3ecda2a8d52
scylladb/utils
History
Raphael S. Carvalho 050ce9ef1d cached_file: Evict unused pages that aren't linked to LRU yet 
It was found that cached_file dtor can hit the following assert
after OOM

cached_file_test: utils/cached_file.hh:379: cached_file::~cached_file(): Assertion _cache.empty()' failed.`

cached_file's dtor iterates through all entries and evict those
that are linked to LRU, under the assumption that all unused
entries were linked to LRU.

That's partially correct. get_page_ptr() may fetch more than 1
page due to read ahead, but it will only call cached_page::share()
on the first page, the one that will be consumed now.

share() is responsible for automatically placing the page into
LRU once refcount drops to zero.

If the read is aborted midway, before cached_file has a chance
to hit the 2nd page (read ahead) in cache, it will remain there
with refcount 0 and unlinked to LRU, in hope that a subsequent
read will bring it out of that state.

Our main user of cached_file is per-sstable index caching.
If the scenario above happens, and the sstable and its associated
cached_file is destroyed, before the 2nd page is hit, cached_file
will not be able to clear all the cache because some of the
pages are unused and not linked.

A page read ahead will be linked into LRU so it doesn't sit in
memory indefinitely. Also allowing for cached_file dtor to
clear all cache if some of those pages brought in advance
aren't fetched later.

A reproducer was added.

Fixes #14814.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>

Closes #14818
2023-07-27 00:01:46 +02:00
..
abi
utils/abi: don't use the deprecated std::unexpected_handler
2022-11-27 12:25:20 +02:00
arch/powerpc/crc32-vpmsum
…
gz
utils/gs/barrett.hh: aarch64: s/brarett/barrett/
2022-11-17 11:09:59 +02:00
lsa
utils/chunked_managed_vector: use operator<=> when appropriate
2023-04-28 15:59:08 +03:00
s3
s3/client: add tagging ops
2023-07-11 09:30:46 +03:00
allocation_strategy.hh
code: Convert nothrow construction assertions into concepts
2022-02-24 19:44:50 +03:00
amortized_reserve.hh
db: range_tombstone_list: Avoid quadratic behavior when applying
2022-08-05 20:34:07 +03:00
anchorless_list.hh
treewide: use defaulted operator!=() and operator==()
2023-04-27 10:24:46 +03:00
array-search.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
array-search.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
ascii.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
ascii.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
atomic_vector.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
aws_sigv4.cc
utils/aws: Export timepoint formatter
2023-04-17 11:14:45 +03:00
aws_sigv4.hh
utils/aws: Brush up the aws_sigv4.hh header
2023-04-17 11:16:45 +03:00
base64.cc
utils: throw error on malformed input in base64 decode
2023-01-16 14:36:23 +01:00
base64.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
big_decimal.cc
treewide: do not include unused header
2023-05-07 19:01:29 +03:00
big_decimal.hh
utils: big_decimal: replace compare() with <=> operator
2023-04-15 12:52:30 +08:00
bit_cast.hh
utils: bit_cast: require TriviallyCopyable To
2022-08-08 08:02:27 +03:00
bloom_calculations.cc
bloom_filter: mark internal help function static
2023-03-24 15:41:45 +08:00
bloom_calculations.hh
bloom_filter: add more constness to false positive rate tables
2023-03-24 15:41:45 +08:00
bloom_filter.cc
treewide: remove empty comments in top-of-files
2022-05-13 07:11:58 +02:00
bloom_filter.hh
treewide: remove empty comments in top-of-files
2022-05-13 07:11:58 +02:00
bounded_stats_deque.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
bptree.hh
treewide: use defaulted operator!=() and operator==()
2023-04-27 10:24:46 +03:00
buffer_input_stream.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
buffer_input_stream.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
buffer_view-to-managed_bytes_view.hh
utils/managed_bytes, serializer: add conversion between buffer_view<bytes_ostream> and managed_bytes_view
2023-05-07 17:17:34 +03:00
build_id.cc
build_id: cache the value
2022-06-02 11:21:05 +03:00
build_id.hh
treewide: extent copyright statements to present day
2021-06-06 19:18:49 +03:00
cached_file.hh
cached_file: Evict unused pages that aren't linked to LRU yet
2023-07-27 00:01:46 +02:00
chunked_vector.hh
utils: to_string: generalize range helpers
2023-05-02 10:48:46 +03:00
class_registrator.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
clmul.hh
utils: clmul: add 64-bit constexpr clmul
2022-11-12 17:04:05 +02:00
CMakeLists.txt
build: cmake: include pretty_printers.cc in util
2023-07-11 09:16:33 +03:00
coarse_steady_clock.hh
logalloc: prevent false positives in reclaim_timer
2022-10-02 13:41:40 +03:00
collection-concepts.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
compact-radix-tree.hh
treewide: use defaulted operator!=() and operator==()
2023-04-27 10:24:46 +03:00
config_file_impl.hh
utils: s/std::regex/boost::regex/
2023-04-06 09:51:28 -04:00
config_file.cc
treewide: do not include unused header
2023-05-07 19:01:29 +03:00
config_file.hh
utils: config_file: add a space after =
2023-03-29 19:22:21 +08:00
coroutine.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
crc.hh
code: Convert is_integral assertions to concepts
2022-02-24 19:44:29 +03:00
cross-shard-barrier.hh
cross-shard-barrier: Capture shared barrier in complete
2022-09-16 08:21:02 +03:00
data_input.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
date.h
date: use std::in_range() to check for invalid year
2023-02-15 22:56:49 +08:00
digest_algorithm.hh
utils: move hashing related files to utils/ module
2023-02-17 07:19:52 +02:00
digester.hh
utils: move hashing related files to utils/ module
2023-02-17 07:19:52 +02:00
directories.cc
Update seastar submodule
2023-06-07 20:25:49 +03:00
directories.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
disk-error-handler.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
disk-error-handler.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
div_ceil.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
double-decker.hh
treewide: use defaulted operator!=() and operator==()
2023-04-27 10:24:46 +03:00
dynamic_bitset.cc
utils: dynamic_bitset: mark functions noexcept
2022-07-27 13:32:36 +03:00
dynamic_bitset.hh
utils: dynamic_bitset: mark functions noexcept
2022-07-27 13:32:36 +03:00
entangled.hh
compile: Fix headers so that *-headers targets compile cleanly.
2022-03-25 16:19:26 +02:00
enum_option.hh
config: fix printing of experimental feature list
2022-07-11 09:17:30 +02:00
error_injection.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
error_injection.hh
utils: error injection: check if it is an ongoing one-shot injection in is_enabled
2023-07-13 15:56:33 +02:00
estimated_histogram.hh
estimated_histogram: do not use dynamic format_string
2023-07-08 15:10:41 +03:00
exception_container.hh
exception_container: switch impl to std::shared_ptr and make copyable
2022-02-22 16:08:52 +01:00
exceptions.cc
utils/exceptions.cc: don't rethrow in is_timeout_exception
2022-07-05 16:41:09 +02:00
exceptions.hh
exceptions: Mark storage_io_error::code() with noexcept
2022-10-03 18:50:06 +03:00
exponential_backoff_retry.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
extremum_tracking.hh
min_max_tracker: mark functions noexcept
2022-07-27 13:17:27 +03:00
fb_utilities.hh
treewide: remove empty comments in top-of-files
2022-05-13 07:11:58 +02:00
file_lock.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
file_lock.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
flush_queue.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
fmt-compat.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
fragment_range.hh
utils: fragment_range: specialize fmt::formatter<FragmentedView>
2023-04-11 16:09:38 +03:00
fragmented_temporary_buffer.hh
treewide: use defaulted operator!=() and operator==()
2023-04-27 10:24:46 +03:00
hash.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
hashers.cc
utils: hashing: add simple_xx_hasher
2023-04-24 14:06:43 +03:00
hashers.hh
utils: hashers: add HasherReturning concept
2023-04-24 14:06:40 +03:00
hashing.hh
utils: hashers: add HasherReturning concept
2023-04-24 14:06:40 +03:00
histogram_metrics_helper.cc
histogram_metrics_helper.hh: Add to_metrics_summary function
2022-07-27 16:58:52 +03:00
histogram_metrics_helper.hh
histogram_metrics_helper.hh: Add to_metrics_summary function
2022-07-27 16:58:52 +03:00
histogram.hh
utils/histogram: drop defaulted copy ctor and assignment operator
2023-02-14 19:22:19 +08:00
human_readable.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
human_readable.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
i_filter.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
i_filter.hh
treewide: remove empty comments in top-of-files
2022-05-13 07:11:58 +02:00
immutable-collection.hh
compile: Fix headers so that *-headers targets compile cleanly.
2022-03-25 16:19:26 +02:00
in.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
input_stream.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
int_range.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
intrusive_btree.hh
treewide: use defaulted operator!=() and operator==()
2023-04-27 10:24:46 +03:00
intrusive-array.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
large_bitset.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
large_bitset.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
latency.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
lexicographical_compare.hh
treewide: use std::lexicographical_compare_threeway
2023-04-21 14:28:18 +03:00
like_matcher.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
like_matcher.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
limiting_data_source.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
limiting_data_source.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
linearizing_input_stream.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
lister.cc
lister: Make lister::dir_entry_types an enum_set
2022-11-17 19:01:45 +03:00
lister.hh
lister: Make lister::dir_entry_types an enum_set
2022-11-17 19:01:45 +03:00
loading_cache.hh
treewide: use defaulted operator!=() and operator==()
2023-04-27 10:24:46 +03:00
loading_shared_values.hh
treewide: use defaulted operator!=() and operator==()
2023-04-27 10:24:46 +03:00
log_heap.hh
treewide: use defaulted operator!=() and operator==()
2023-04-27 10:24:46 +03:00
logalloc.cc
utils: logalloc: correct asan_interface.h location
2023-05-29 23:03:25 +03:00
logalloc.hh
utils/logalloc: add use_standard_allocator_segment_pool_backend()
2022-09-16 13:02:40 +03:00
lru.hh
row_cache, lru: Introduce evict_shallow()
2023-01-27 21:56:31 +01:00
managed_bytes.cc
utils: managed_bytes: add bidirectional conversion between bytes_opt and managed_bytes_opt
2023-05-07 17:16:38 +03:00
managed_bytes.hh
utils/managed_bytes, serializer: add conversion between buffer_view<bytes_ostream> and managed_bytes_view
2023-05-07 17:17:34 +03:00
managed_ref.hh
code: Convert nothrow construction assertions into concepts
2022-02-24 19:44:50 +03:00
managed_vector.hh
code: Convert nothrow construction assertions into concepts
2022-02-24 19:44:50 +03:00
maybe_yield.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
memory_data_sink.hh
memory_data_sink: Add move ctor
2023-03-20 07:55:20 +02:00
multiprecision_int.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
multiprecision_int.hh
treewide: use defaulted operator!=() and operator==()
2023-04-27 10:24:46 +03:00
murmur_hash.cc
build: disable implicit fallthrough
2023-07-10 19:36:06 +02:00
murmur_hash.hh
build: disable implicit fallthrough
2023-07-10 19:36:06 +02:00
mutable_view.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
neat-object-id.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
observable.hh
utils: observer: qualify seastar::noncopyable_function
2022-11-28 21:58:30 +02:00
overloaded_functor.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
phased_barrier.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
preempt.hh
mutation_partition_v2: Accept arbitrary preemption source in apply_monotonically()
2023-01-27 21:56:31 +01:00
pretty_printers.cc
utils: drop operator<< for pretty printers
2023-07-17 14:02:13 +08:00
pretty_printers.hh
utils: drop operator<< for pretty printers
2023-07-17 14:02:13 +08:00
ranges.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
rate_limiter.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
rate_limiter.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
recent_entries_map.hh
utils: introduce recent_entries_map datatype to track least recent visited entries.
2023-02-03 19:04:32 +01:00
result_combinators.hh
result_combinators: add result_wrap_unpack
2022-02-22 16:25:21 +01:00
result_loop.hh
utils: result_loop: remove invalid and incorrect constraint
2022-04-18 12:27:18 +03:00
result_try.hh
util/result_try: reference this explicitly
2023-02-28 21:56:55 +08:00
result.hh
utils/result.hh: add missing header includes for boost.outcome
2022-03-16 12:28:47 +03:00
reusable_buffer.hh
utils: redesign reusable_buffer
2023-04-26 22:09:17 +02:00
rjson.cc
utils: yield when streaming json in print()
2023-01-23 13:46:06 +01:00
rjson.hh
alternator: evaluate expressions as false for stored malformed binary
2023-01-16 15:15:27 +01:00
rpc_utils.hh
storage_proxy: handler responses, use pointers to default constructed values instead of nulls
2023-06-26 11:10:38 +03:00
runtime.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
runtime.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
sequenced_set.hh
sequenced_set: add extract_vector method
2023-05-21 11:33:38 +04:00
serialization.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
serialized_action.hh
serialized_action: make serialized_action abortable
2023-05-11 16:31:23 +03:00
simple_hashers.hh
utils: hashing: add simple_xx_hasher
2023-04-24 14:06:43 +03:00
small_vector.hh
utils: to_string: generalize range helpers
2023-05-02 10:48:46 +03:00
stall_free.hh
clear_gently: remove noexcept for rvalue references overload
2023-05-24 12:08:19 +04:00
streaming_histogram.hh
treewide: remove empty comments in top-of-files
2022-05-13 07:11:58 +02:00
tagged_integer.hh
utils: tagged_integer: implement std::numeric_limits::{min,max}
2023-05-15 10:19:39 +03:00
throttle.hh
test: Extract throttle object to separate header
2023-06-21 00:58:24 +02:00
to_string.cc
to_string: add print operators for std::{weak,partial}_ordering
2023-02-15 11:09:04 +02:00
to_string.hh
utils: to_string: get rid of utils::join
2023-05-02 10:59:58 +03:00
top_k.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
tuple_utils.hh
storage_proxy.cc: refactor encode_replica_exception_for_rpc
2023-06-15 15:52:50 +04:00
unconst.hh
db: Introduce mutation_partition_v2
2023-01-27 19:15:39 +01:00
updateable_value.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
updateable_value.hh
utils/updateable_value: add transforming_value_updater
2022-10-05 10:52:49 +03:00
utf8.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
utf8.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
UUID_gen.cc
utils: move hashing related files to utils/ module
2023-02-17 07:19:52 +02:00
UUID_gen.hh
test: use correct timestamp resolution in test_group0_history_clearing_old_entries
2023-07-13 10:33:52 +03:00
uuid.cc
utils: UUID: specialize fmt::formatter for UUID and tagged_uuid<>
2023-03-20 14:25:45 +08:00
UUID.hh
utils/UUID: reference timeuuid_tri_compare in UUID::operator<=> comment
2023-07-11 13:19:50 +02:00
variant_element.hh
utils/variant_element: simplify is_variant_element with right fold
2022-11-27 16:34:34 +02:00
vle.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
xx_hasher.hh
utils: move hashing related files to utils/ module
2023-02-17 07:19:52 +02:00