mirrors/scylladb
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
3e92bcaa20f259715c63c33af0545d36f5e17a4e
scylladb/utils
History
Botond Dénes 3e92bcaa20 Merge 'utils: redesign reusable_buffer' from Michał Chojnowski 
Common compression libraries work on contiguous buffers.
Contiguous buffers are a problem for the allocator. However, as long as they are short-lived,
we can avoid the expensive allocations by reusing buffers across tasks.

This idea is already applied to the compression of CQL frames, but with some deficiencies.
`utils: redesign reusable_buffer` attempts to improve upon it in a few ways. See its commit message for an extended discussion.

Compression buffer reuse also happens in the zstd SSTable compressor, but the implementation is misguided. Every `zstd_processor` instance reuses a buffer, but each instance has its own buffer. This is very bad, because a healthy database might have thousands of concurrent instances (because there is one for each sstable reader). Together, the buffers might require gigabytes of memory, and the reuse actually *increases* memory pressure significantly, instead of reducing it.
`zstd: share buffers between compressor instances` aims to improve that by letting a single buffer be shared across all instances on a shard.

Closes #13324

* github.com:scylladb/scylladb:
  zstd: share buffers between compressor instances
  utils: redesign reusable_buffer
2023-04-27 09:09:09 +03:00
..
abi
utils/abi: don't use the deprecated std::unexpected_handler
2022-11-27 12:25:20 +02:00
arch/powerpc/crc32-vpmsum
…
gz
utils/gs/barrett.hh: aarch64: s/brarett/barrett/
2022-11-17 11:09:59 +02:00
lsa
utils/chunked_managed_vector: Fix sigsegv during reserve()
2022-04-12 16:37:11 +03:00
s3
s3/client: Zeroify stat by default
2023-04-25 09:53:47 +02:00
allocation_strategy.hh
code: Convert nothrow construction assertions into concepts
2022-02-24 19:44:50 +03:00
amortized_reserve.hh
db: range_tombstone_list: Avoid quadratic behavior when applying
2022-08-05 20:34:07 +03:00
anchorless_list.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
array-search.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
array-search.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
ascii.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
ascii.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
atomic_vector.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
aws_sigv4.cc
utils/aws: Export timepoint formatter
2023-04-17 11:14:45 +03:00
aws_sigv4.hh
utils/aws: Brush up the aws_sigv4.hh header
2023-04-17 11:16:45 +03:00
base64.cc
utils: throw error on malformed input in base64 decode
2023-01-16 14:36:23 +01:00
base64.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
big_decimal.cc
utils: big_decimal: replace compare() with <=> operator
2023-04-15 12:52:30 +08:00
big_decimal.hh
utils: big_decimal: replace compare() with <=> operator
2023-04-15 12:52:30 +08:00
bit_cast.hh
utils: bit_cast: require TriviallyCopyable To
2022-08-08 08:02:27 +03:00
bloom_calculations.cc
bloom_filter: mark internal help function static
2023-03-24 15:41:45 +08:00
bloom_calculations.hh
bloom_filter: add more constness to false positive rate tables
2023-03-24 15:41:45 +08:00
bloom_filter.cc
treewide: remove empty comments in top-of-files
2022-05-13 07:11:58 +02:00
bloom_filter.hh
treewide: remove empty comments in top-of-files
2022-05-13 07:11:58 +02:00
bounded_stats_deque.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
bptree.hh
treewide: do not define/capture unused variables
2023-02-15 22:57:18 +02:00
buffer_input_stream.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
buffer_input_stream.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
build_id.cc
build_id: cache the value
2022-06-02 11:21:05 +03:00
build_id.hh
treewide: extent copyright statements to present day
2021-06-06 19:18:49 +03:00
cached_file.hh
utils: lru: remove unlink_from_lru()
2022-10-17 12:07:27 +02:00
chunked_vector.hh
util/chunked_vector: more complete comment
2022-06-23 10:33:35 +03:00
class_registrator.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
clmul.hh
utils: clmul: add 64-bit constexpr clmul
2022-11-12 17:04:05 +02:00
CMakeLists.txt
utils: move generation-number to gms
2023-04-23 08:37:32 +03:00
coarse_steady_clock.hh
logalloc: prevent false positives in reclaim_timer
2022-10-02 13:41:40 +03:00
collection-concepts.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
compact-radix-tree.hh
treewide: do not define/capture unused variables
2023-02-15 22:57:18 +02:00
config_file_impl.hh
utils: s/std::regex/boost::regex/
2023-04-06 09:51:28 -04:00
config_file.cc
utils: config_file: add config_source::API
2022-11-09 11:53:20 +02:00
config_file.hh
utils: config_file: add a space after =
2023-03-29 19:22:21 +08:00
coroutine.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
crc.hh
code: Convert is_integral assertions to concepts
2022-02-24 19:44:29 +03:00
cross-shard-barrier.hh
cross-shard-barrier: Capture shared barrier in complete
2022-09-16 08:21:02 +03:00
data_input.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
date.h
date: use std::in_range() to check for invalid year
2023-02-15 22:56:49 +08:00
digest_algorithm.hh
utils: move hashing related files to utils/ module
2023-02-17 07:19:52 +02:00
digester.hh
utils: move hashing related files to utils/ module
2023-02-17 07:19:52 +02:00
directories.cc
lister: move to utils
2022-02-28 12:36:03 +02:00
directories.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
disk-error-handler.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
disk-error-handler.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
div_ceil.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
double-decker.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
dynamic_bitset.cc
utils: dynamic_bitset: mark functions noexcept
2022-07-27 13:32:36 +03:00
dynamic_bitset.hh
utils: dynamic_bitset: mark functions noexcept
2022-07-27 13:32:36 +03:00
entangled.hh
compile: Fix headers so that *-headers targets compile cleanly.
2022-03-25 16:19:26 +02:00
enum_option.hh
config: fix printing of experimental feature list
2022-07-11 09:17:30 +02:00
error_injection.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
error_injection.hh
treewide: use system-#include (angle brackets) for seastar
2022-04-26 14:46:42 +03:00
estimated_histogram.hh
estimated_histogram: add missing getter method
2022-07-26 15:59:33 +03:00
exception_container.hh
exception_container: switch impl to std::shared_ptr and make copyable
2022-02-22 16:08:52 +01:00
exceptions.cc
utils/exceptions.cc: don't rethrow in is_timeout_exception
2022-07-05 16:41:09 +02:00
exceptions.hh
exceptions: Mark storage_io_error::code() with noexcept
2022-10-03 18:50:06 +03:00
exponential_backoff_retry.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
extremum_tracking.hh
min_max_tracker: mark functions noexcept
2022-07-27 13:17:27 +03:00
fb_utilities.hh
treewide: remove empty comments in top-of-files
2022-05-13 07:11:58 +02:00
file_lock.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
file_lock.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
flush_queue.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
fmt-compat.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
fragment_range.hh
utils: fragment_range: specialize fmt::formatter<FragmentedView>
2023-04-11 16:09:38 +03:00
fragmented_temporary_buffer.hh
utils: redesign reusable_buffer
2023-04-26 22:09:17 +02:00
hash.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
hashers.cc
utils: hashing: add simple_xx_hasher
2023-04-24 14:06:43 +03:00
hashers.hh
utils: hashers: add HasherReturning concept
2023-04-24 14:06:40 +03:00
hashing.hh
utils: hashers: add HasherReturning concept
2023-04-24 14:06:40 +03:00
histogram_metrics_helper.cc
histogram_metrics_helper.hh: Add to_metrics_summary function
2022-07-27 16:58:52 +03:00
histogram_metrics_helper.hh
histogram_metrics_helper.hh: Add to_metrics_summary function
2022-07-27 16:58:52 +03:00
histogram.hh
utils/histogram: drop defaulted copy ctor and assignment operator
2023-02-14 19:22:19 +08:00
human_readable.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
human_readable.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
i_filter.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
i_filter.hh
treewide: remove empty comments in top-of-files
2022-05-13 07:11:58 +02:00
immutable-collection.hh
compile: Fix headers so that *-headers targets compile cleanly.
2022-03-25 16:19:26 +02:00
in.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
input_stream.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
int_range.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
intrusive_btree.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
intrusive-array.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
large_bitset.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
large_bitset.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
latency.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
lexicographical_compare.hh
treewide: use std::lexicographical_compare_threeway
2023-04-21 14:28:18 +03:00
like_matcher.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
like_matcher.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
limiting_data_source.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
limiting_data_source.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
linearizing_input_stream.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
lister.cc
lister: Make lister::dir_entry_types an enum_set
2022-11-17 19:01:45 +03:00
lister.hh
lister: Make lister::dir_entry_types an enum_set
2022-11-17 19:01:45 +03:00
loading_cache.hh
cql3: adding missing privileged on cache size eviction metric
2023-02-26 14:33:46 +02:00
loading_shared_values.hh
loading_cache: Make invalidation take immediate effect
2022-03-09 16:13:07 +02:00
log_heap.hh
utils: log_heap: mark functions noexcept
2022-07-27 13:43:17 +03:00
logalloc.cc
treewide: do not define/capture unused variables
2023-02-28 21:56:53 +08:00
logalloc.hh
utils/logalloc: add use_standard_allocator_segment_pool_backend()
2022-09-16 13:02:40 +03:00
lru.hh
row_cache, lru: Introduce evict_shallow()
2023-01-27 21:56:31 +01:00
managed_bytes.cc
utils: fragment_range: specialize fmt::formatter<FragmentedView>
2023-04-11 16:09:38 +03:00
managed_bytes.hh
bytes_ostream: base on managed_bytes
2022-07-12 00:23:29 +03:00
managed_ref.hh
code: Convert nothrow construction assertions into concepts
2022-02-24 19:44:50 +03:00
managed_vector.hh
code: Convert nothrow construction assertions into concepts
2022-02-24 19:44:50 +03:00
maybe_yield.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
memory_data_sink.hh
memory_data_sink: Add move ctor
2023-03-20 07:55:20 +02:00
multiprecision_int.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
multiprecision_int.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
murmur_hash.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
murmur_hash.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
mutable_view.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
neat-object-id.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
observable.hh
utils: observer: qualify seastar::noncopyable_function
2022-11-28 21:58:30 +02:00
overloaded_functor.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
phased_barrier.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
preempt.hh
mutation_partition_v2: Accept arbitrary preemption source in apply_monotonically()
2023-01-27 21:56:31 +01:00
ranges.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
rate_limiter.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
rate_limiter.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
recent_entries_map.hh
utils: introduce recent_entries_map datatype to track least recent visited entries.
2023-02-03 19:04:32 +01:00
result_combinators.hh
result_combinators: add result_wrap_unpack
2022-02-22 16:25:21 +01:00
result_loop.hh
utils: result_loop: remove invalid and incorrect constraint
2022-04-18 12:27:18 +03:00
result_try.hh
util/result_try: reference this explicitly
2023-02-28 21:56:55 +08:00
result.hh
utils/result.hh: add missing header includes for boost.outcome
2022-03-16 12:28:47 +03:00
reusable_buffer.hh
utils: redesign reusable_buffer
2023-04-26 22:09:17 +02:00
rjson.cc
utils/rjson.cc: ignore buggy GCC warning
2022-03-02 19:10:58 +02:00
rjson.hh
alternator: evaluate expressions as false for stored malformed binary
2023-01-16 15:15:27 +01:00
runtime.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
runtime.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
sequenced_set.hh
utils: sequenced_set: expose set and contains method
2022-08-08 17:31:00 +03:00
serialization.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
serialized_action.hh
serialized_action: Allow being observer for updateable_value
2022-07-06 08:17:08 +03:00
simple_hashers.hh
utils: hashing: add simple_xx_hasher
2023-04-24 14:06:43 +03:00
small_vector.hh
utils: small_vector: define operator<=>
2023-02-28 20:04:22 +02:00
source_location-compat.hh
cross-tree: allow std::source_location in clang 14
2022-12-11 20:28:49 +02:00
stall_free.hh
utils: clear_gently: add variants for optional values
2023-04-23 21:34:02 +03:00
streaming_histogram.hh
treewide: remove empty comments in top-of-files
2022-05-13 07:11:58 +02:00
tagged_integer.hh
utils: add tagged_integer
2023-04-23 08:37:32 +03:00
to_string.cc
to_string: add print operators for std::{weak,partial}_ordering
2023-02-15 11:09:04 +02:00
to_string.hh
utils: specialize fmt::formatter<optional<>>
2023-04-12 10:57:03 +08:00
top_k.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
unconst.hh
db: Introduce mutation_partition_v2
2023-01-27 19:15:39 +01:00
updateable_value.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
updateable_value.hh
utils/updateable_value: add transforming_value_updater
2022-10-05 10:52:49 +03:00
utf8.cc
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
utf8.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
UUID_gen.cc
utils: move hashing related files to utils/ module
2023-02-17 07:19:52 +02:00
UUID_gen.hh
utils: UUID_gen: accept decimicroseconds in min_time_UUID
2023-04-21 10:33:02 +02:00
uuid.cc
utils: UUID: specialize fmt::formatter for UUID and tagged_uuid<>
2023-03-20 14:25:45 +08:00
UUID.hh
utils: UUID: specialize fmt::formatter for UUID and tagged_uuid<>
2023-03-20 14:25:45 +08:00
variant_element.hh
utils/variant_element: simplify is_variant_element with right fold
2022-11-27 16:34:34 +02:00
vle.hh
treewide: use Software Package Data Exchange (SPDX) license identifiers
2022-01-18 12:15:18 +01:00
xx_hasher.hh
utils: move hashing related files to utils/ module
2023-02-17 07:19:52 +02:00