mirrors/scylladb
1
0
Fork 0
mirror of https://github.com/scylladb/scylladb.git synced 2026-04-24 18:40:38 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
98a6002a1a9c505851ca0b09dd3afff23297d269
scylladb/utils
History
Michał Chojnowski 4a18a08284 utils/lsa/chunked_managed_vector: fix the calculation of max_chunk_capacity() 
`chunked_managed_vector` is a vector-like container which splits
its contents into multiple contiguous allocations if necessary,
in order to fit within LSA's max preferred contiguous allocation
limits.

Each limited-size chunk is stored in a `managed_vector`.
`managed_vector` is unaware of LSA's size limits.
It's up to the user of `managed_vector` to pick a size which
is small enough.

This happens in `chunked_managed_vector::max_chunk_capacity()`.
But the calculation is wrong, because it doesn't account for
the fact that `managed_vector` has to place some metadata
(the backreference pointer) inside the allocation.
In effect, the chunks allocated by `chunked_managed_vector`
are just a tiny bit larger than the limit, and the limit is violated.

Fix this by accounting for the metadata.

Also, before the patch `chunked_managed_vector::max_contiguous_allocation`,
repeats the definition of logalloc::max_managed_object_size.
This is begging for a bug if `logalloc::max_managed_object_size`
changes one day. Adjust it so that `chunked_managed_vector` looks
directly at `logalloc::max_managed_object_size`, as it means to.

Fixes scylladb/scylladb#23854

(cherry picked from commit 7f9152babc)

Closes scylladb/scylladb#24369
2025-06-03 18:10:36 +03:00
..
abi
util: replace <tab> with spaces
2023-09-11 14:38:46 +03:00
arch/powerpc/crc32-vpmsum
…
gz
utils/gs/barrett.hh: aarch64: s/brarett/barrett/
2022-11-17 11:09:59 +02:00
lsa
utils/lsa/chunked_managed_vector: fix the calculation of max_chunk_capacity()
2025-06-03 18:10:36 +03:00
s3
s3/client: Reuse class multipart_upload in do_upload_file
2024-09-12 18:38:16 +03:00
allocation_strategy.hh
logalloc: add hold_reserve
2024-07-08 16:08:27 +02:00
amortized_reserve.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
anchorless_list.hh
…
array-search.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
array-search.hh
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
ascii.cc
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
ascii.hh
…
assert.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
atomic_vector.hh
utils: atomic_vector: mark for_each functions as const
2023-09-04 16:14:38 +03:00
aws_sigv4.cc
treewide: use seastar::format() or fmt::format() explicitly
2024-09-11 23:21:40 +03:00
aws_sigv4.hh
…
base64.cc
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
base64.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
big_decimal.cc
treewide: use seastar::format() or fmt::format() explicitly
2024-09-11 23:21:40 +03:00
big_decimal.hh
treewide: replace formatter<std::string_view> with formatter<string_view>
2024-04-19 07:44:07 +03:00
bit_cast.hh
utils: bit_cast: drop unused #includes
2023-12-12 21:09:51 +08:00
bloom_calculations.cc
…
bloom_calculations.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
bloom_filter.cc
utils/i_filter: introduce get_filter_size()
2024-06-24 12:06:01 +05:30
bloom_filter.hh
utils/i_filter: introduce get_filter_size()
2024-06-24 12:06:01 +05:30
bounded_stats_deque.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
bptree.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
buffer_input_stream.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
buffer_input_stream.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
buffer_view-to-managed_bytes_view.hh
utils/managed_bytes, serializer: add conversion between buffer_view<bytes_ostream> and managed_bytes_view
2023-05-07 17:17:34 +03:00
build_id.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
build_id.hh
…
cached_file_stats.hh
utils: cached_file: deglobalize cached_file metrics
2023-09-01 22:34:41 +02:00
cached_file.hh
utils: cached_file: Mark permit as awaiting on page miss
2024-12-09 23:18:00 +01:00
chunked_vector.hh
utils: chunked_vector: add ctor from std::initializer_list
2024-06-25 12:08:06 +03:00
class_registrator.hh
…
clmul.hh
…
CMakeLists.txt
utils: add on_internal_error with common logger
2024-01-31 16:45:09 +02:00
coarse_steady_clock.hh
…
collection-concepts.hh
…
compact-radix-tree.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
composite_abort_source.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
config_file_impl.hh
config: specialize from-string conversion for bool
2024-07-18 18:38:22 +03:00
config_file.cc
config: avoid binding an lvalue reference to an rvalue reference
2024-06-27 19:36:13 +03:00
config_file.hh
treewide: use seastar::format() or fmt::format() explicitly
2024-09-11 23:21:40 +03:00
contiguous_shared_buffer.hh
sstables, utils: Allow parsers to work with different buffer types
2024-10-01 01:38:47 +00:00
coroutine.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
crc.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
cross-shard-barrier.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
data_input.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
date.h
treewide: apply codespell to the comments in source code
2023-12-20 10:25:03 +02:00
digest_algorithm.hh
feature: grandfather DIGEST_FOR_NULL_VALUES
2024-05-18 00:24:00 +03:00
digester.hh
feature: grandfather DIGEST_FOR_NULL_VALUES
2024-05-18 00:24:00 +03:00
directories.cc
treewide: include fmt/ranges.h and/or fmt/std.h
2024-04-19 22:56:16 +08:00
directories.hh
directories: prevent inode cache fragmentation by orderly verifying data directory contents
2024-02-01 12:20:23 +05:30
disk-error-handler.cc
extensions: Add exception types for IO extensions and handle in memtable write path
2024-08-11 13:52:35 +03:00
disk-error-handler.hh
treewide: replace std::result_of_t with std::invoke_result_t
2024-05-26 16:45:42 +03:00
div_ceil.hh
utils/div_ceil: add constraints to template arguments
2024-08-04 15:32:01 +03:00
double-decker.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
dynamic_bitset.cc
utils: dynamic_bitset: mark functions noexcept
2022-07-27 13:32:36 +03:00
dynamic_bitset.hh
utils: dynamic_bitset: mark functions noexcept
2022-07-27 13:32:36 +03:00
entangled.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
enum_option.hh
config, enum_option: allow round-trip string conversion
2024-07-10 20:39:01 +03:00
error_injection.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
error_injection.hh
utils/error_injection: add a way to set parameters from error injection points
2025-04-10 07:33:09 -04:00
estimated_histogram.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
exception_container.hh
treewide: replace formatter<std::string_view> with formatter<string_view>
2024-04-19 07:44:07 +03:00
exceptions.cc
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
exceptions.hh
exceptions: s/#warn/#warning/
2024-02-01 14:50:17 +02:00
exponential_backoff_retry.hh
treewide: replace std::result_of_t with std::invoke_result_t
2024-05-26 16:45:42 +03:00
extremum_tracking.hh
…
file_lock.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
file_lock.hh
utils: Remove unused operator<< for file_lock object
2024-02-02 15:20:40 +01:00
flush_queue.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
fmt-compat.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
fragment_range.hh
treewide: replace formatter<std::string_view> with formatter<string_view>
2024-04-19 07:44:07 +03:00
fragmented_temporary_buffer.hh
sstables, utils: Allow parsers to work with different buffer types
2024-10-01 01:38:47 +00:00
hash.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
hashers.cc
utils: hashing: add simple_xx_hasher
2023-04-24 14:06:43 +03:00
hashers.hh
utils: hashers: add HasherReturning concept
2023-04-24 14:06:40 +03:00
hashing.hh
treewide: include used headers
2024-05-27 17:34:38 +03:00
histogram_metrics_helper.cc
…
histogram_metrics_helper.hh
histogram_metrics_helper: support native histogram
2024-01-23 13:12:34 +02:00
histogram.hh
utils/histogram.hh: Make summary support inifinite bucket.
2024-08-22 23:34:24 +03:00
http.hh
code-cleanup: add missing header guards
2024-07-09 18:31:35 +03:00
human_readable.cc
utils/human_readable: add fmt::formatter for human_readable_value
2024-03-12 14:53:55 +08:00
human_readable.hh
treewide: replace formatter<std::string_view> with formatter<string_view>
2024-04-19 07:44:07 +03:00
i_filter.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
i_filter.hh
utils/i_filter: introduce get_filter_size()
2024-06-24 12:06:01 +05:30
immutable-collection.hh
compile: Fix headers so that *-headers targets compile cleanly.
2022-03-25 16:19:26 +02:00
input_stream.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
int_range.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
intrusive_btree.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
intrusive-array.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
large_bitset.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
large_bitset.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
latency.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
lexicographical_compare.hh
…
like_matcher.cc
…
like_matcher.hh
…
limiting_data_source.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
limiting_data_source.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
linearizing_input_stream.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
lister.cc
treewide: use seastar::format() or fmt::format() explicitly
2024-09-11 23:21:40 +03:00
lister.hh
directory_lister: Add noexcept default move-constructor
2024-09-05 11:10:21 +03:00
loading_cache.hh
utils::loading_cache: gracefully skip timer if gate closed
2025-05-06 10:19:32 +02:00
loading_shared_values.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
log_heap.hh
…
logalloc.cc
utils/lsa/chunked_managed_vector: fix the calculation of max_chunk_capacity()
2025-06-03 18:10:36 +03:00
logalloc.hh
utils/lsa/chunked_managed_vector: fix the calculation of max_chunk_capacity()
2025-06-03 18:10:36 +03:00
lru.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
managed_bytes.cc
utils/managed_bytes: add fmt::formatters for managed_bytes and friends
2024-02-23 11:32:41 +08:00
managed_bytes.hh
managed_bytes: in the copy constructor, respect the target preferred allocation size
2025-04-18 07:55:23 +00:00
managed_ref.hh
code: Convert nothrow construction assertions into concepts
2022-02-24 19:44:50 +03:00
managed_vector.hh
utils/lsa/chunked_managed_vector: fix the calculation of max_chunk_capacity()
2025-06-03 18:10:36 +03:00
maybe_yield.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
memory_data_sink.hh
s3/client: Unmark put-object lambdas from mutable
2024-07-04 10:07:48 +03:00
memory_limit_reached.hh
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
multiprecision_int.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
multiprecision_int.hh
…
murmur_hash.cc
utils/murmur_hash: replace rotl64() with std::rotl()
2024-06-24 08:24:43 +03:00
murmur_hash.hh
utils/murmur_hash: replace rotl64() with std::rotl()
2024-06-24 08:24:43 +03:00
mutable_view.hh
utils: managed_bytes: rewrite managed_bytes methods in terms of managed_bytes_view
2024-02-09 17:00:33 +01:00
neat-object-id.hh
…
observable.hh
utils: observer: qualify seastar::noncopyable_function
2022-11-28 21:58:30 +02:00
on_internal_error.cc
utils: add on_internal_error with common logger
2024-01-31 16:45:09 +02:00
on_internal_error.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
overloaded_functor.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
phased_barrier.hh
…
preempt.hh
utils: preempt: add preemption_source
2024-02-07 18:31:28 +01:00
pretty_printers.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
pretty_printers.hh
treewide: fix misspellings in code comments
2024-01-31 09:16:10 +02:00
ranges.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
rate_limiter.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
rate_limiter.hh
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
recent_entries_map.hh
utils: introduce recent_entries_map datatype to track least recent visited entries.
2023-02-03 19:04:32 +01:00
result_combinators.hh
result_combinators: add result_wrap_unpack
2022-02-22 16:25:21 +01:00
result_loop.hh
treewide: replace seastar::future::get0() with seastar::future::get()
2024-02-02 22:12:57 +08:00
result_try.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
result.hh
utils/result.hh: add missing header includes for boost.outcome
2022-03-16 12:28:47 +03:00
reusable_buffer.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
rjson.cc
utils/rjson.cc: include the function name in exception message
2024-09-12 15:22:49 +03:00
rjson.hh
utils/rjson.cc: correct a comment about assert()
2024-10-09 21:50:26 +03:00
rpc_utils.hh
storage_proxy: handler responses, use pointers to default constructed values instead of nulls
2023-06-26 11:10:38 +03:00
runtime.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
runtime.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
sequenced_set.hh
sequenced_set: add extract_vector method
2023-05-21 11:33:38 +04:00
serialization.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
serialized_action.hh
serialized_action: make serialized_action abortable
2023-05-11 16:31:23 +03:00
simple_hashers.hh
utils: hashing: add simple_xx_hasher
2023-04-24 14:06:43 +03:00
small_vector.hh
treewide: do not define FMT_DEPRECATED_OSTREAM
2024-04-19 22:57:36 +08:00
sorting.hh
utils/sorting: allow to pass any container as verticies
2024-08-08 10:42:09 +02:00
stall_free.hh
utils/stall_free: introduce reserve_gently
2024-06-18 23:36:30 +05:30
streaming_histogram.hh
treewide: remove empty comments in top-of-files
2022-05-13 07:11:58 +02:00
tagged_integer.hh
utils/tagged_integer: remove conversion to underlying integer
2024-08-15 02:12:58 +02:00
throttle.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
to_string.cc
nodetool: rebuild: add force option
2024-08-19 17:20:12 +03:00
to_string.hh
utils/to_string: include fmt/std.h if fmt >= v10
2024-04-23 12:09:05 +03:00
top_k.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
tuple_utils.hh
storage_proxy.cc: refactor encode_replica_exception_for_rpc
2023-06-15 15:52:50 +04:00
unconst.hh
db: Introduce mutation_partition_v2
2023-01-27 19:15:39 +01:00
updateable_value.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
updateable_value.hh
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
user_provided_param.hh
nodetool: rebuild: add force option
2024-08-19 17:20:12 +03:00
utf8.cc
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
utf8.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
UUID_gen.cc
utils: UUID_gen: include <atomic>
2024-04-30 09:07:22 +03:00
UUID_gen.hh
utils: UUID: Make get_time_UUID() respect the clock offset
2024-12-04 14:18:16 +01:00
uuid.cc
treewide: use seastar::format() or fmt::format() explicitly
2024-09-11 23:21:40 +03:00
UUID.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
value_or_reference.hh
kl::reader::make_reader: Unify interface with mx::reader::make_reader
2024-08-13 10:02:43 +02:00
variant_element.hh
utils/variant_element: simplify is_variant_element with right fold
2022-11-27 16:34:34 +02:00
vle.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
xx_hasher.hh
feature: grandfather DIGEST_FOR_NULL_VALUES
2024-05-18 00:24:00 +03:00