mirrors/scylladb
1
0
Fork 0
mirror of https://github.com/scylladb/scylladb.git synced 2026-05-13 03:12:13 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
05adee4c824ae881cf4cdbbf558dafe89de0af75
scylladb/utils
History
Pavel Emelyanov a02e65c649 s3_client: Add bucket lister 
The lister resembles the directory_lister from util -- it returns
entries upon its .get() invocation, and should be .close()d at the end.

Internally the lister issues ListObjectsV2 request with provided prefix
and limits the server with the amount of entries returned not to consume
too much local memory (we don't have streaming XML parser for response).
If the result is indeed truncated, the subsequent calls include the
continuation token as per [1]

[1] https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
2024-08-13 21:15:43 +03:00
..
abi
util: replace <tab> with spaces
2023-09-11 14:38:46 +03:00
arch/powerpc/crc32-vpmsum
…
gz
…
lsa
utils/stall_free: introduce reserve_gently
2024-06-18 23:36:30 +05:30
s3
s3_client: Add bucket lister
2024-08-13 21:15:43 +03:00
allocation_strategy.hh
logalloc: add hold_reserve
2024-07-08 16:08:27 +02:00
amortized_reserve.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
anchorless_list.hh
treewide: use defaulted operator!=() and operator==()
2023-04-27 10:24:46 +03:00
array-search.cc
…
array-search.hh
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
ascii.cc
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
ascii.hh
…
assert.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
atomic_vector.hh
utils: atomic_vector: mark for_each functions as const
2023-09-04 16:14:38 +03:00
aws_sigv4.cc
utils/aws: Export timepoint formatter
2023-04-17 11:14:45 +03:00
aws_sigv4.hh
utils/aws: Brush up the aws_sigv4.hh header
2023-04-17 11:16:45 +03:00
base64.cc
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
base64.hh
…
big_decimal.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
big_decimal.hh
treewide: replace formatter<std::string_view> with formatter<string_view>
2024-04-19 07:44:07 +03:00
bit_cast.hh
utils: bit_cast: drop unused #includes
2023-12-12 21:09:51 +08:00
bloom_calculations.cc
bloom_filter: mark internal help function static
2023-03-24 15:41:45 +08:00
bloom_calculations.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
bloom_filter.cc
utils/i_filter: introduce get_filter_size()
2024-06-24 12:06:01 +05:30
bloom_filter.hh
utils/i_filter: introduce get_filter_size()
2024-06-24 12:06:01 +05:30
bounded_stats_deque.hh
…
bptree.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
buffer_input_stream.cc
…
buffer_input_stream.hh
…
buffer_view-to-managed_bytes_view.hh
utils/managed_bytes, serializer: add conversion between buffer_view<bytes_ostream> and managed_bytes_view
2023-05-07 17:17:34 +03:00
build_id.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
build_id.hh
…
cached_file_stats.hh
utils: cached_file: deglobalize cached_file metrics
2023-09-01 22:34:41 +02:00
cached_file.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
chunked_vector.hh
utils: chunked_vector: add ctor from std::initializer_list
2024-06-25 12:08:06 +03:00
class_registrator.hh
…
clmul.hh
…
CMakeLists.txt
utils: add on_internal_error with common logger
2024-01-31 16:45:09 +02:00
coarse_steady_clock.hh
…
collection-concepts.hh
…
compact-radix-tree.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
composite_abort_source.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
config_file_impl.hh
config: specialize from-string conversion for bool
2024-07-18 18:38:22 +03:00
config_file.cc
config: avoid binding an lvalue reference to an rvalue reference
2024-06-27 19:36:13 +03:00
config_file.hh
config: avoid binding an lvalue reference to an rvalue reference
2024-06-27 19:36:13 +03:00
coroutine.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
crc.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
cross-shard-barrier.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
data_input.hh
…
date.h
treewide: apply codespell to the comments in source code
2023-12-20 10:25:03 +02:00
digest_algorithm.hh
feature: grandfather DIGEST_FOR_NULL_VALUES
2024-05-18 00:24:00 +03:00
digester.hh
feature: grandfather DIGEST_FOR_NULL_VALUES
2024-05-18 00:24:00 +03:00
directories.cc
treewide: include fmt/ranges.h and/or fmt/std.h
2024-04-19 22:56:16 +08:00
directories.hh
directories: prevent inode cache fragmentation by orderly verifying data directory contents
2024-02-01 12:20:23 +05:30
disk-error-handler.cc
extensions: Add exception types for IO extensions and handle in memtable write path
2024-08-11 13:52:35 +03:00
disk-error-handler.hh
treewide: replace std::result_of_t with std::invoke_result_t
2024-05-26 16:45:42 +03:00
div_ceil.hh
utils/div_ceil: add constraints to template arguments
2024-08-04 15:32:01 +03:00
double-decker.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
dynamic_bitset.cc
…
dynamic_bitset.hh
…
entangled.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
enum_option.hh
config, enum_option: allow round-trip string conversion
2024-07-10 20:39:01 +03:00
error_injection.cc
…
error_injection.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
estimated_histogram.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
exception_container.hh
treewide: replace formatter<std::string_view> with formatter<string_view>
2024-04-19 07:44:07 +03:00
exceptions.cc
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
exceptions.hh
exceptions: s/#warn/#warning/
2024-02-01 14:50:17 +02:00
exponential_backoff_retry.hh
treewide: replace std::result_of_t with std::invoke_result_t
2024-05-26 16:45:42 +03:00
extremum_tracking.hh
…
file_lock.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
file_lock.hh
utils: Remove unused operator<< for file_lock object
2024-02-02 15:20:40 +01:00
flush_queue.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
fmt-compat.hh
…
fragment_range.hh
treewide: replace formatter<std::string_view> with formatter<string_view>
2024-04-19 07:44:07 +03:00
fragmented_temporary_buffer.hh
fragmented_temporary_buffer: Add const iterator access to underlying buffers
2023-11-21 08:42:33 +00:00
hash.hh
…
hashers.cc
utils: hashing: add simple_xx_hasher
2023-04-24 14:06:43 +03:00
hashers.hh
utils: hashers: add HasherReturning concept
2023-04-24 14:06:40 +03:00
hashing.hh
treewide: include used headers
2024-05-27 17:34:38 +03:00
histogram_metrics_helper.cc
…
histogram_metrics_helper.hh
histogram_metrics_helper: support native histogram
2024-01-23 13:12:34 +02:00
histogram.hh
utils/histogram: drop defaulted copy ctor and assignment operator
2023-02-14 19:22:19 +08:00
http.hh
code-cleanup: add missing header guards
2024-07-09 18:31:35 +03:00
human_readable.cc
utils/human_readable: add fmt::formatter for human_readable_value
2024-03-12 14:53:55 +08:00
human_readable.hh
treewide: replace formatter<std::string_view> with formatter<string_view>
2024-04-19 07:44:07 +03:00
i_filter.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
i_filter.hh
utils/i_filter: introduce get_filter_size()
2024-06-24 12:06:01 +05:30
immutable-collection.hh
…
input_stream.hh
…
int_range.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
intrusive_btree.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
intrusive-array.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
large_bitset.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
large_bitset.hh
…
latency.hh
…
lexicographical_compare.hh
treewide: use std::lexicographical_compare_threeway
2023-04-21 14:28:18 +03:00
like_matcher.cc
…
like_matcher.hh
…
limiting_data_source.cc
…
limiting_data_source.hh
…
linearizing_input_stream.hh
…
lister.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
lister.hh
utils/lister: Limit the API of scan_dir() to fs::path
2023-09-05 20:50:42 +03:00
loading_cache.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
loading_shared_values.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
log_heap.hh
treewide: use defaulted operator!=() and operator==()
2023-04-27 10:24:46 +03:00
logalloc.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
logalloc.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
lru.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
managed_bytes.cc
utils/managed_bytes: add fmt::formatters for managed_bytes and friends
2024-02-23 11:32:41 +08:00
managed_bytes.hh
utils/managed_bytes: add support for fmt::to_string() to bytes and friends
2024-04-19 22:56:13 +08:00
managed_ref.hh
…
managed_vector.hh
util: do not use variable length array
2023-11-20 23:02:41 +02:00
maybe_yield.hh
…
memory_data_sink.hh
s3/client: Unmark put-object lambdas from mutable
2024-07-04 10:07:48 +03:00
memory_limit_reached.hh
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
multiprecision_int.cc
…
multiprecision_int.hh
treewide: use defaulted operator!=() and operator==()
2023-04-27 10:24:46 +03:00
murmur_hash.cc
utils/murmur_hash: replace rotl64() with std::rotl()
2024-06-24 08:24:43 +03:00
murmur_hash.hh
utils/murmur_hash: replace rotl64() with std::rotl()
2024-06-24 08:24:43 +03:00
mutable_view.hh
utils: managed_bytes: rewrite managed_bytes methods in terms of managed_bytes_view
2024-02-09 17:00:33 +01:00
neat-object-id.hh
…
observable.hh
…
on_internal_error.cc
utils: add on_internal_error with common logger
2024-01-31 16:45:09 +02:00
on_internal_error.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
overloaded_functor.hh
…
phased_barrier.hh
…
preempt.hh
utils: preempt: add preemption_source
2024-02-07 18:31:28 +01:00
pretty_printers.cc
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
pretty_printers.hh
treewide: fix misspellings in code comments
2024-01-31 09:16:10 +02:00
ranges.hh
…
rate_limiter.cc
…
rate_limiter.hh
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
recent_entries_map.hh
utils: introduce recent_entries_map datatype to track least recent visited entries.
2023-02-03 19:04:32 +01:00
result_combinators.hh
…
result_loop.hh
treewide: replace seastar::future::get0() with seastar::future::get()
2024-02-02 22:12:57 +08:00
result_try.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
result.hh
…
reusable_buffer.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
rjson.cc
alternator: destroy streamed json values gently
2024-08-05 00:35:52 +03:00
rjson.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
rpc_utils.hh
storage_proxy: handler responses, use pointers to default constructed values instead of nulls
2023-06-26 11:10:38 +03:00
runtime.cc
…
runtime.hh
…
sequenced_set.hh
sequenced_set: add extract_vector method
2023-05-21 11:33:38 +04:00
serialization.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
serialized_action.hh
serialized_action: make serialized_action abortable
2023-05-11 16:31:23 +03:00
simple_hashers.hh
utils: hashing: add simple_xx_hasher
2023-04-24 14:06:43 +03:00
small_vector.hh
treewide: do not define FMT_DEPRECATED_OSTREAM
2024-04-19 22:57:36 +08:00
sorting.hh
utils/sorting: allow to pass any container as verticies
2024-08-08 10:42:09 +02:00
stall_free.hh
utils/stall_free: introduce reserve_gently
2024-06-18 23:36:30 +05:30
streaming_histogram.hh
…
tagged_integer.hh
utils: do not include unused headers
2024-01-18 12:50:06 +02:00
throttle.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
to_string.cc
utils: add fmt::formatter for std::strong_ordering and friends
2024-03-12 14:53:55 +08:00
to_string.hh
utils/to_string: include fmt/std.h if fmt >= v10
2024-04-23 12:09:05 +03:00
top_k.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
tuple_utils.hh
storage_proxy.cc: refactor encode_replica_exception_for_rpc
2023-06-15 15:52:50 +04:00
unconst.hh
db: Introduce mutation_partition_v2
2023-01-27 19:15:39 +01:00
updateable_value.cc
…
updateable_value.hh
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
utf8.cc
Typos: fix typos in comments
2023-12-02 22:37:22 +02:00
utf8.hh
…
UUID_gen.cc
utils: UUID_gen: include <atomic>
2024-04-30 09:07:22 +03:00
UUID_gen.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
uuid.cc
utils: Add uuid_xor_to_uint32 helper
2024-02-01 10:27:55 +02:00
UUID.hh
treewide: change assert() to SCYLLA_ASSERT()
2024-08-05 08:23:35 +03:00
variant_element.hh
…
vle.hh
…
xx_hasher.hh
feature: grandfather DIGEST_FOR_NULL_VALUES
2024-05-18 00:24:00 +03:00