mirrors/scylladb
1
0
Fork 0
mirror of https://github.com/scylladb/scylladb.git synced 2026-04-22 09:30:45 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
fdae3e4f3a9f8c264fc3999ef2de76bb3f148d33
scylladb/utils
History
Avi Kivity fdae3e4f3a Merge '[Backport 2026.1] s3_client: Fix s3 part size and number of parts calculation' from Scylladb[bot] 
- Correct `calc_part_size` function since it could return more than 10k parts
- Add tests
- Add more checks in `calc_part_size` to comply with S3 limits

Fixes: https://scylladb.atlassian.net/browse/SCYLLADB-640
Must be ported back to 2025.3/4 and 2026.1 since we may encounter this bug in production clusters

- (cherry picked from commit 289e910cec)

- (cherry picked from commit 6280cb91ca)

- (cherry picked from commit 960adbb439)

Parent PR: #28592

Closes scylladb/scylladb#28697

* github.com:scylladb/scylladb:
  s3_client: add more constrains to the calc_part_size
  s3_client: add tests for calc_part_size
  s3_client: correct multipart part-size logic to respect 10k limit
2026-02-24 14:23:33 +02:00
..
abi
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
azure/identity
code: Use tls_options::bye_timeout instead of deprecated switch
2025-10-06 16:25:35 +03:00
gcp
utils/gcp/object_storage: URL-encode object names in URL:s
2026-02-17 19:32:19 +00:00
gz
utils: mark crc barrett tables const
2025-08-31 11:37:39 +03:00
lsa
utils/lsa/chunked_managed_vector: fix the calculation of max_chunk_capacity()
2025-04-28 12:30:13 +02:00
rest
utils::rest::client: Add abort_source support
2025-10-13 08:53:25 +00:00
s3
Merge '[Backport 2026.1] s3_client: Fix s3 part size and number of parts calculation' from Scylladb[bot]
2026-02-24 14:23:33 +02:00
abstract_formatter.hh
Attach names to allocating sections for better debuggability
2025-12-07 14:14:25 +02:00
alien_worker.cc
main: utils: add thread names to alien workers
2025-07-15 23:29:21 +02:00
alien_worker.hh
main: utils: add thread names to alien workers
2025-07-15 23:29:21 +02:00
allocation_strategy.hh
utils/logalloc: enforce LSA allocation size limits
2025-05-29 13:05:08 +02:00
amortized_reserve.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
anchorless_list.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
array-search.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
array-search.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
ascii.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
ascii.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
assert.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
atomic_vector.hh
migration_listener: fix deadlock in nested notifications
2025-12-17 14:00:28 +01:00
aws_sigv4.cc
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
aws_sigv4.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
base64.cc
utils/base64: fix misleading code and comment (no functional change)
2025-09-01 08:57:50 +03:00
base64.hh
utils: Introduce base64url_{encode,decode}
2025-07-16 17:14:08 +03:00
big_decimal.cc
utils/big_decimal: fix scale overflow when parsing values with large exponents
2025-06-26 15:29:28 +03:00
big_decimal.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
bit_cast.hh
utils/bit_cast: add a new overload of write_unaligned()
2025-09-07 00:30:15 +02:00
bloom_calculations.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
bloom_calculations.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
bloom_filter.cc
utils/bloom_filter: add add(const hashed_key&)
2025-09-29 13:01:21 +02:00
bloom_filter.hh
utils/bloom_filter: add add(const hashed_key&)
2025-09-29 13:01:21 +02:00
bounded_stats_deque.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
bptree.hh
B+tree: Clean const_iterator->iterator conversion
2025-02-26 23:17:27 +02:00
buffer_input_stream.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
buffer_input_stream.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
buffer_view-to-managed_bytes_view.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
build_id.cc
…
build_id.hh
…
cached_file_stats.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
cached_file.hh
Attach names to allocating sections for better debuggability
2025-12-07 14:14:25 +02:00
checked-file-impl.hh
Update seastar submodule
2025-12-30 19:37:13 +03:00
chunked_vector.hh
utils: Allow chunked_vector::erase to work with non-default-constructible type
2025-07-29 13:43:17 +08:00
class_registrator.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
clmul.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
CMakeLists.txt
utils: make crypt_sha512.cc to compile
2025-12-10 15:36:18 +01:00
coarse_steady_clock.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
collection-concepts.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
compact-radix-tree.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
composite_abort_source.hh
treewide: Add missing license boilerplate
2025-09-01 14:58:32 +03:00
config_file_impl.hh
tree: Remove unused boost headers
2025-02-25 10:32:32 +03:00
config_file.cc
tree: Remove unused boost headers
2025-02-25 10:32:32 +03:00
config_file.hh
utils/config_file: fix a missing allowed_values propagation in one of named_value constructors
2025-10-06 15:33:11 +03:00
contiguous_shared_buffer.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
coroutine.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
crc.hh
PowerPC: remove ppc stuff
2025-07-08 10:38:23 +03:00
cross-shard-barrier.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
crypt_sha512.cc
auth: utils: add yielding to crypt_sha512
2025-12-10 15:36:18 +01:00
crypt_sha512.hh
utils: coroutinize crypt_sha512
2025-12-10 15:36:18 +01:00
data_input.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
date.h
date: do not put space before literal operator
2024-10-21 11:21:52 +03:00
digest_algorithm.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
digester.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
directories.cc
directories: fix exception rethrowing
2025-12-12 18:10:20 +01:00
directories.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
disk_space_monitor.cc
disk_space_monitor: add subscription API for threshold-based disk space monitoring
2025-08-28 18:06:37 +02:00
disk_space_monitor.hh
disk_space_monitor: add subscription API for threshold-based disk space monitoring
2025-08-28 18:06:37 +02:00
disk-error-handler.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
disk-error-handler.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
div_ceil.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
double-decker.hh
B+tree: Clean const_iterator->iterator conversion
2025-02-26 23:17:27 +02:00
dynamic_bitset.cc
treewide: migrate from boost::adaptors::reversed to std::views::reverse
2025-01-07 13:22:00 +02:00
dynamic_bitset.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
entangled.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
enum_option.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
error_injection.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
error_injection.hh
utils: error_injection: allow aborting wait_for_message
2025-12-29 19:13:55 +01:00
estimated_histogram.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
exception_container.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
exceptions.cc
utils/exceptions.cc: Added check for exceptions::request_timeout_exception in is_timeout_exception function.
2025-06-26 12:25:38 +02:00
exceptions.hh
exceptions: Add try_catch_nested to universally handle nested exceptions of the same type.
2025-03-26 11:15:13 +01:00
exponential_backoff_retry.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
extremum_tracking.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
file_lock.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
file_lock.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
flush_queue.hh
utils: flush_queue: use named_gate
2025-04-12 11:47:02 +03:00
fragment_range.hh
treewide: Replace __builtin_expect with (un)likely
2025-07-03 13:34:04 +03:00
fragmented_temporary_buffer.hh
transport: replace throwing protocol_exception with returns
2025-08-28 23:31:36 +02:00
from_chars_exactly.hh
utils: introduce from_chars_exactly()
2025-01-09 11:14:49 +02:00
hash.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
hashers.cc
utils/hashers: add get_sha256()
2025-04-01 00:07:28 +02:00
hashers.hh
utils/hashers: add get_sha256()
2025-04-01 00:07:28 +02:00
hashing.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
histogram_metrics_helper.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
histogram_metrics_helper.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
histogram.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
http.cc
dns_connection_factory: refine the move constructor
2026-01-27 22:43:08 +00:00
http.hh
connection_factory: includes cleanup
2026-01-27 22:43:08 +00:00
human_readable.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
human_readable.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
i_filter.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
i_filter.hh
utils/bloom_filter: add add(const hashed_key&)
2025-09-29 13:01:21 +02:00
immutable-collection.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
input_stream.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
int_range.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
interval.hh
interval: split interval_bound implementation for const references
2025-09-24 13:57:21 +02:00
intrusive_btree.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
intrusive-array.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
io-wrappers.cc
Merge 'Switch to seastar API level 9 (no more packet-s in output_stream/data_sink API)' from Pavel Emelyanov
2025-10-22 01:51:43 +03:00
io-wrappers.hh
utils::io-wrappers: Add ranged data_source
2025-10-13 08:53:25 +00:00
labels.cc
Adding the __level and features labels
2025-03-03 16:58:38 +02:00
labels.hh
treewide: Add missing #pragma once
2025-09-01 14:58:21 +03:00
large_bitset.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
large_bitset.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
latency.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
lexicographical_compare.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
like_matcher.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
like_matcher.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
limiting_data_source.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
limiting_data_source.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
linearizing_input_stream.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
lister.cc
Update seastar submodule
2025-12-30 19:37:13 +03:00
lister.hh
directory_lister: add ctor with opened directory
2026-01-04 11:05:18 +02:00
loading_cache.hh
utils::loading_cache: gracefully skip timer if gate closed
2025-04-30 16:43:22 +03:00
loading_shared_values.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
log_heap.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
log.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
logalloc.cc
lsa: Export metrics for reclaim/evict/compact time
2026-01-19 12:08:16 +03:00
logalloc.hh
Attach names to allocating sections for better debuggability
2025-12-07 14:14:25 +02:00
lru_string_map.hh
treewide: #include Seastar headers with angle brackets
2026-01-13 14:56:15 +02:00
lru.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
managed_bytes.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
managed_bytes.hh
managed_bytes: make empty managed_bytes constexpr friendly
2025-07-29 23:51:43 +03:00
managed_ref.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
managed_string.hh
utils/managed_string.hh: Introduce managed_string and fragmented_ostringstream
2025-06-30 19:12:08 +02:00
managed_vector.hh
utils/lsa/chunked_managed_vector: fix the calculation of max_chunk_capacity()
2025-04-28 12:30:13 +02:00
map_difference.hh
treewide: Move misc files to utils directory
2025-07-21 11:56:40 +03:00
maybe_yield.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
memory_data_sink.hh
util: Re-use seastar::util::memory_data_sink
2026-01-14 08:54:00 +02:00
memory_limit_reached.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
multiprecision_int.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
multiprecision_int.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
murmur_hash.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
murmur_hash.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
mutable_view.hh
sstables/trie: perform the BTI-encoding of position_in_partition eagerly
2025-09-29 04:10:40 +02:00
neat-object-id.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
observable.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
on_internal_error.cc
utils: add "fatal" version of utils::on_internal_error()
2025-08-01 02:15:04 +03:00
on_internal_error.hh
utils: add "fatal" version of utils::on_internal_error()
2025-08-01 02:15:04 +03:00
only_on_shard0.hh
storage_service, tablets: Collect per-node capacity in load_stats
2025-03-06 12:17:32 +01:00
overloaded_functor.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
phased_barrier.hh
utils: phased_barrier, pluggable: use named gate
2025-04-12 11:47:00 +03:00
pluggable.hh
utils: phased_barrier, pluggable: use named gate
2025-04-12 11:47:00 +03:00
preempt.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
pretty_printers.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
pretty_printers.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
rate_limiter.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
rate_limiter.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
recent_entries_map.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
refcounted.hh
utils: add advanced_rpc_compressor
2024-12-23 23:37:02 +01:00
replicator.hh
utils: Introduce helper for replicated data structures
2025-11-19 15:21:02 +01:00
reservoir_sampling.hh
utils: introduce reservoir_sampling
2024-12-23 23:37:02 +01:00
result_combinators.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
result_loop.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
result_try.hh
treewide: Replace __builtin_expect with (un)likely
2025-07-03 13:34:04 +03:00
result.hh
utils: add result_with_exception_ptr
2025-08-28 23:31:04 +02:00
reusable_buffer.hh
utils/reusable_buffer: accept non-throwing writer callbacks via result_with_exception
2025-07-17 16:40:02 +02:00
rjson.cc
utils/rjson: fix error messages from rjson::parse()
2025-12-11 11:17:01 +02:00
rjson.hh
use rjson::to_string_view when rjson::value gets converted using GetStringLength
2025-12-09 19:27:21 +01:00
rpc_utils.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
runtime.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
runtime.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
scoped_item_list.hh
utils: add a scoped_item_list utility class
2025-08-01 02:15:04 +03:00
seekable_source.hh
utils::seekable_source: Add a seekable IO source type
2025-10-13 08:53:24 +00:00
sequenced_set.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
sequential_producer.hh
utils: refactor sequential_producer as abortable
2025-07-08 16:29:55 +02:00
serialization.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
serialized_action.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
simple_hashers.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
simple_value_with_expiry.hh
Add simple_value_with_expiry util class
2025-12-29 08:32:52 +01:00
small_vector.hh
treewide: Replace __builtin_expect with (un)likely
2025-07-03 13:34:04 +03:00
sorting.hh
utils/sorting: Fix VerticesContainer concept constraints
2025-02-26 23:23:53 +02:00
stall_free.hh
stall_free: make variadic dispose_gently sequential
2025-11-20 07:16:16 +03:00
streaming_histogram.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
tagged_integer.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
throttle.hh
…
to_string.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
to_string.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
top_k.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
tuple_utils.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
unconst.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
unique_view.hh
utils: implement drop-in replacement for replacing boost::adaptors::uniqued
2025-01-21 16:24:45 +08:00
updateable_value.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
updateable_value.hh
utils: don't allow do discard updateable_value observer
2025-06-23 17:54:01 +02:00
upload_progress.hh
s3::upload_progress: Promote to general util type
2025-10-13 08:53:25 +00:00
user_provided_param.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
utf8.cc
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
utf8.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00
UUID_gen.cc
Add precompiled headers to CMakeLists.txt
2025-11-21 12:27:41 +02:00
UUID_gen.hh
utils: fix get_random_time_UUID_from_micros to generate correct time uuid
2025-11-20 10:27:29 +02:00
uuid.cc
db/config, utils: allow using UUID as a config option
2025-07-23 15:36:45 +02:00
UUID.hh
db/config, utils: allow using UUID as a config option
2025-07-23 15:36:45 +02:00
value_or_reference.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
variant_element.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
vle.hh
treewide: relicense to ScyllaDB-Source-Available-1.0
2024-12-18 17:45:13 +02:00
xx_hasher.hh
utils: do not include unused headers
2025-01-14 07:56:39 -05:00