mirrors/scylladb
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
ed6bec2cacd3eaedaaa2ff1cf5de78ca282bbb61
scylladb/utils
History
Ernest Zaslavsky dd51e50f60 s3_client: add memory fallback in chunked_download_source 
Introduce fallback logic in `chunked_download_source` to handle
memory exhaustion. When memory is low, feed the `deque` with only
one uncounted buffer at a time. This allows slow but steady progress
without getting stuck on the memory semaphore.

Fixes: https://github.com/scylladb/scylladb/issues/25453
Fixes: https://github.com/scylladb/scylladb/issues/25262

Closes scylladb/scylladb#25452
2025-08-14 09:52:10 +03:00
..
abi
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
azure/identity
utils: azure: Get access token with default credentials
2025-07-16 17:14:08 +03:00
gz
treewide: Replace __builtin_expect with (un)likely
2025-07-03 13:34:04 +03:00
lsa
utils/lsa/chunked_managed_vector: fix the calculation of max_chunk_capacity()
2025-04-28 12:30:13 +02:00
rest
utils: rest: Add interface for request/response redaction logic
2025-07-16 17:14:08 +03:00
s3
s3_client: add memory fallback in chunked_download_source
2025-08-14 09:52:10 +03:00
advanced_rpc_compressor_protocol.hh
utils: add advanced_rpc_compressor
2024-12-23 23:37:02 +01:00
advanced_rpc_compressor.cc
treewide: use angle brackets for including seastar headers
2025-03-17 10:03:06 +02:00
advanced_rpc_compressor.hh
utils: use std::to_underlying() when appropriate
2025-02-19 12:12:28 +03:00
alien_worker.cc
main: utils: add thread names to alien workers
2025-07-15 23:29:21 +02:00
alien_worker.hh
main: utils: add thread names to alien workers
2025-07-15 23:29:21 +02:00
allocation_strategy.hh
utils/logalloc: enforce LSA allocation size limits
2025-05-29 13:05:08 +02:00
amortized_reserve.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
anchorless_list.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
array-search.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
array-search.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
ascii.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
ascii.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
assert.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
atomic_vector.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
aws_sigv4.cc
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
aws_sigv4.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
base64.cc
utils: Introduce base64url_{encode,decode}
2025-07-16 17:14:08 +03:00
base64.hh
utils: Introduce base64url_{encode,decode}
2025-07-16 17:14:08 +03:00
big_decimal.cc
utils/big_decimal: fix scale overflow when parsing values with large exponents
2025-06-26 15:29:28 +03:00
big_decimal.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
bit_cast.hh
utils/bit_cast: add object_representation()
2025-07-23 17:03:05 +02:00
bloom_calculations.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
bloom_calculations.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
bloom_filter.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
bloom_filter.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
bounded_stats_deque.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
bptree.hh
B+tree: Clean const_iterator->iterator conversion
2025-02-26 23:17:27 +02:00
buffer_input_stream.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
buffer_input_stream.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
buffer_view-to-managed_bytes_view.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
build_id.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
build_id.hh
…
cached_file_stats.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
cached_file.hh
utils/cached_file: add get_shared_page()
2025-08-05 00:56:50 +02:00
checked-file-impl.hh
code: Move checked-file-impl.hh to util/
2025-03-06 10:22:05 +02:00
chunked_vector.hh
utils: Allow chunked_vector::erase to work with non-default-constructible type
2025-07-29 13:43:17 +08:00
class_registrator.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
clmul.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
CMakeLists.txt
utils: azure: Declare all Azure credential types
2025-07-16 17:14:08 +03:00
coarse_steady_clock.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
collection-concepts.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
compact-radix-tree.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
composite_abort_source.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
config_file_impl.hh
tree: Remove unused boost headers
2025-02-25 10:32:32 +03:00
config_file.cc
tree: Remove unused boost headers
2025-02-25 10:32:32 +03:00
config_file.hh
Merge 'config: prevent SIGHUP from changing non-liveupdatable parameters' from Andrzej Jackowski
2025-02-06 11:33:59 +03:00
contiguous_shared_buffer.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
coroutine.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
crc.hh
PowerPC: remove ppc stuff
2025-07-08 10:38:23 +03:00
cross-shard-barrier.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
data_input.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
date.h
date: do not put space before literal operator
2024-10-21 11:21:52 +03:00
dict_trainer.cc
utils/hashers: add get_sha256()
2025-04-01 00:07:28 +02:00
dict_trainer.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
digest_algorithm.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
digester.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
directories.cc
tree: Fix polymorphic exception handling by using references
2025-02-26 23:15:16 +02:00
directories.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
disk_space_monitor.cc
disk_space_monitor: add space_source_registration
2025-06-04 16:25:24 +03:00
disk_space_monitor.hh
disk_space_monitor: add space_source_registration
2025-06-04 16:25:24 +03:00
disk-error-handler.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
disk-error-handler.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
div_ceil.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
double-decker.hh
B+tree: Clean const_iterator->iterator conversion
2025-02-26 23:17:27 +02:00
dynamic_bitset.cc
treewide: migrate from boost::adaptors::reversed to std::views::reverse
2025-01-07 13:22:00 +02:00
dynamic_bitset.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
entangled.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
enum_option.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
error_injection.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
error_injection.hh
test: Verify partitioned set store split and unsplit correctly
2025-04-29 15:47:33 -03:00
estimated_histogram.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
exception_container.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
exceptions.cc
utils/exceptions.cc: Added check for exceptions::request_timeout_exception in is_timeout_exception function.
2025-06-26 12:25:38 +02:00
exceptions.hh
exceptions: Add try_catch_nested to universally handle nested exceptions of the same type.
2025-03-26 11:15:13 +01:00
exponential_backoff_retry.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
extremum_tracking.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
file_lock.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
file_lock.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
flush_queue.hh
utils: flush_queue: use named_gate
2025-04-12 11:47:02 +03:00
fragment_range.hh
treewide: Replace __builtin_expect with (un)likely
2025-07-03 13:34:04 +03:00
fragmented_temporary_buffer.hh
treewide: Replace __builtin_expect with (un)likely
2025-07-03 13:34:04 +03:00
from_chars_exactly.hh
utils: introduce from_chars_exactly()
2025-01-09 11:14:49 +02:00
hash.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
hashers.cc
utils/hashers: add get_sha256()
2025-04-01 00:07:28 +02:00
hashers.hh
utils/hashers: add get_sha256()
2025-04-01 00:07:28 +02:00
hashing.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
histogram_metrics_helper.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
histogram_metrics_helper.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
histogram.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
http.cc
http: Add missing thread_local specifier for static
2025-06-17 10:23:52 +03:00
http.hh
utils::http::dns_connection_factory: Use a shared certificate_credentials
2025-06-10 11:20:21 +03:00
human_readable.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
human_readable.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
i_filter.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
i_filter.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
immutable-collection.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
input_stream.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
int_range.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
interval.hh
treewide: Move misc files to utils directory
2025-07-21 11:56:40 +03:00
intrusive_btree.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
intrusive-array.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
io-wrappers.cc
utils: Add "io-wrappers", useful IO helper types
2025-03-20 14:54:09 +00:00
io-wrappers.hh
utils: Add "io-wrappers", useful IO helper types
2025-03-20 14:54:09 +00:00
labels.cc
Adding the __level and features labels
2025-03-03 16:58:38 +02:00
labels.hh
Adding the __level and features labels
2025-03-03 16:58:38 +02:00
large_bitset.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
large_bitset.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
latency.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
lexicographical_compare.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
like_matcher.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
like_matcher.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
limiting_data_source.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
limiting_data_source.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
linearizing_input_stream.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
lister.cc
code: Move checked-file-impl.hh to util/
2025-03-06 10:22:05 +02:00
lister.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
loading_cache.hh
utils::loading_cache: gracefully skip timer if gate closed
2025-04-30 16:43:22 +03:00
loading_shared_values.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
log_heap.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
log.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
logalloc.cc
Merge 'utils/logalloc: enforce the max contiguous allocation size limit' from Michał Chojnowski
2025-05-29 22:11:41 +03:00
logalloc.hh
Merge 'utils/logalloc: enforce the max contiguous allocation size limit' from Michał Chojnowski
2025-05-29 22:11:41 +03:00
lru.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
managed_bytes.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
managed_bytes.hh
managed_bytes: make empty managed_bytes constexpr friendly
2025-07-29 23:51:43 +03:00
managed_ref.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
managed_string.hh
utils/managed_string.hh: Introduce managed_string and fragmented_ostringstream
2025-06-30 19:12:08 +02:00
managed_vector.hh
utils/lsa/chunked_managed_vector: fix the calculation of max_chunk_capacity()
2025-04-28 12:30:13 +02:00
map_difference.hh
treewide: Move misc files to utils directory
2025-07-21 11:56:40 +03:00
maybe_yield.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
memory_data_sink.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
memory_limit_reached.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
multiprecision_int.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
multiprecision_int.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
murmur_hash.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
murmur_hash.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
mutable_view.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
neat-object-id.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
observable.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
on_internal_error.cc
utils: add "fatal" version of utils::on_internal_error()
2025-08-01 02:15:04 +03:00
on_internal_error.hh
utils: add "fatal" version of utils::on_internal_error()
2025-08-01 02:15:04 +03:00
only_on_shard0.hh
storage_service, tablets: Collect per-node capacity in load_stats
2025-03-06 12:17:32 +01:00
overloaded_functor.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
phased_barrier.hh
utils: phased_barrier, pluggable: use named gate
2025-04-12 11:47:00 +03:00
pluggable.hh
utils: phased_barrier, pluggable: use named gate
2025-04-12 11:47:00 +03:00
preempt.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
pretty_printers.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
pretty_printers.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
rate_limiter.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
rate_limiter.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
recent_entries_map.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
refcounted.hh
utils: add advanced_rpc_compressor
2024-12-23 23:37:02 +01:00
reservoir_sampling.hh
utils: introduce reservoir_sampling
2024-12-23 23:37:02 +01:00
result_combinators.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
result_loop.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
result_try.hh
treewide: Replace __builtin_expect with (un)likely
2025-07-03 13:34:04 +03:00
result.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
reusable_buffer.hh
utils/reusable_buffer: accept non-throwing writer callbacks via result_with_exception
2025-07-17 16:40:02 +02:00
rjson.cc
alternator: avoid oversized allocation in Query/Scan
2025-07-14 18:41:34 +03:00
rjson.hh
alternator: avoid oversized allocation in Query/Scan
2025-07-14 18:41:34 +03:00
rpc_utils.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
runtime.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
runtime.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
scoped_item_list.hh
utils: add a scoped_item_list utility class
2025-08-01 02:15:04 +03:00
sequenced_set.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
sequential_producer.hh
utils: refactor sequential_producer as abortable
2025-07-08 16:29:55 +02:00
serialization.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
serialized_action.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
shared_dict.hh
utils/hashers: add get_sha256()
2025-04-01 00:07:28 +02:00
simple_hashers.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
small_vector.hh
treewide: Replace __builtin_expect with (un)likely
2025-07-03 13:34:04 +03:00
sorting.hh
utils/sorting: Fix VerticesContainer concept constraints
2025-02-26 23:23:53 +02:00
stall_free.hh
utils: clear_gently: add support for sets
2025-07-13 12:30:45 +03:00
stream_compressor.cc
utils/stream_compressor: allocate memory for zstd compressors externally
2025-05-27 12:43:11 +03:00
stream_compressor.hh
utils/stream_compressor: allocate memory for zstd compressors externally
2025-05-27 12:43:11 +03:00
streaming_histogram.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
tagged_integer.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
throttle.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
to_string.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
to_string.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
top_k.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
tuple_utils.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
unconst.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
unique_view.hh
utils: implement drop-in replacement for replacing boost::adaptors::uniqued
2025-01-21 16:24:45 +08:00
updateable_value.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
updateable_value.hh
utils: don't allow do discard updateable_value observer
2025-06-23 17:54:01 +02:00
user_provided_param.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
utf8.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
utf8.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
UUID_gen.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
UUID_gen.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
uuid.cc
db/config, utils: allow using UUID as a config option
2025-07-23 15:36:45 +02:00
UUID.hh
db/config, utils: allow using UUID as a config option
2025-07-23 15:36:45 +02:00
value_or_reference.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
variant_element.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
vle.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
xx_hasher.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00