mirrors/scylladb
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
7ac59dcc9837658bb9fc1ff06dc4c95fdabef6f8
scylladb/utils
History
Avi Kivity 7ac59dcc98 lsa: decay reserves 
The log-structured allocator (LSA) reserves memory when performing
operations, since its operations are performed with reclaiming disabled
and if it runs out, it cannot evict cache to gain more. The amount of
memory to reserve is remembered across calls so that it does not have
to repeat the fail/increase-reserve/retry cycle for every operation.

However, we currently lack decaying the amount to reserve. This means
that if a single operation increased the reserve in the distant past,
all current operations also require this large reserve. Large reserves
are expensive since they can cause large amounts of cache to be evicted.

This patch adds reserve decay. The time-to-decay is inversely proportional
to reserve size: 10GB/reserve. This means that a 20MB reserve is halved
after 500 operations (10GB/20MB) while a 20kB reserve is halved after
500,000 operations (10GB/20kB). So large, expensive reserves are decayed
quickly while small, inexpensive reserves are decayed slowly to reduce
the risk of allocation failures and exceptions.

A unit test is added.

Fixes #325.
2020-09-08 15:59:25 +03:00
..
arch/powerpc/crc32-vpmsum
utils: crc32: mark power crc32 assembly as not requiring an executable stack
2018-10-02 18:48:23 +01:00
gz
utils: add missing include for ssize_t
2020-01-30 14:10:18 +02:00
allocation_strategy.hh
allocation_strategy: Mark returning lambda as noexcept
2020-07-09 14:41:23 +03:00
anchorless_list.hh
mvcc: Introduce partition_version_list
2018-05-30 12:18:56 +02:00
array-search.hh
utils: AVX searcher
2020-08-06 15:41:31 +03:00
ascii.cc
utils: add fast ascii string validation
2018-12-24 09:58:08 +02:00
ascii.hh
utils: add fast ascii string validation
2018-12-24 09:58:08 +02:00
atomic_vector.hh
atomic_vetor: Don't pass references to callbacks
2020-04-23 16:06:37 +03:00
big_decimal.cc
big_decimal: Correctly handle negative scales
2020-06-25 15:34:10 -07:00
big_decimal.hh
big_decimal: Add a as_rational member function
2020-06-25 15:33:31 -07:00
bloom_calculations.cc
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
bloom_calculations.hh
utils: convert sprint() to format()
2018-11-01 13:16:17 +00:00
bloom_filter.cc
Update seastar submodule
2020-03-23 11:59:30 +02:00
bloom_filter.hh
utils: Use dedicated enum for Bloom filter format instead of a boolean.
2018-05-10 09:47:41 +03:00
bounded_stats_deque.hh
treewide: add missing headers and/or forward declarations
2020-03-23 09:29:45 +02:00
bptree.hh
bptree: Special intra-node key search when possible
2020-08-06 15:41:31 +03:00
buffer_input_stream.cc
buffer_input_stream: make it possible to specify chunk size
2018-04-16 21:11:13 +02:00
buffer_input_stream.hh
buffer_input_stream: make it possible to specify chunk size
2018-04-16 21:11:13 +02:00
build_id.cc
build_id: simplify callback pointer type casting
2020-08-03 15:55:18 +03:00
build_id.hh
build_id: mv sources to utils/
2020-08-03 15:55:16 +03:00
cached_file.hh
utils: cached_file: Fix compilation error
2020-06-18 14:08:29 +03:00
chunked_vector.hh
utils::chunked_vector: add rbegin() and related iterators
2019-08-01 18:39:47 +03:00
class_registrator.hh
class_registry: Use std::string_view in (un)?qualified_name
2020-07-03 12:28:14 -07:00
clmul.hh
utils: Extract clmul() from crc.hh
2018-12-03 14:36:08 +01:00
collection-concepts.hh
intrusive-array: Array with trusted bounds
2020-07-14 16:29:49 +03:00
config_file_impl.hh
utils: config_src::add_command_line_options(): drop name and desc args
2020-07-28 18:00:29 +03:00
config_file.cc
utils: config_src::add_command_line_options(): drop name and desc args
2020-07-28 18:00:29 +03:00
config_file.hh
db/config_file.hh: named_value: remove unused members _name and _desc
2020-08-03 12:51:16 +03:00
coroutine.hh
treewide: add missing headers and/or forward declarations
2020-03-23 09:29:45 +02:00
crc.hh
treewide: update concepts language from the Concepts TS to C++20
2020-06-02 09:12:21 +03:00
data_input.hh
Replace std::experimental types with C++17 std version.
2019-01-08 13:16:36 +02:00
date.h
date: use correct casts for years
2017-04-17 23:03:15 +03:00
directories.cc
messaging_service: Move initialization to messaging/
2020-08-19 13:08:12 +03:00
directories.hh
treewide: add missing headers and/or forward declarations
2020-03-23 09:29:45 +02:00
disk-error-handler.cc
utils: Move disk-error-handler into it
2020-02-09 17:26:52 +02:00
disk-error-handler.hh
utils: do_io_check: adjust indentation
2020-08-06 19:01:18 +03:00
div_ceil.hh
utils: introduce div_ceil()
2017-05-17 12:30:03 +03:00
double-decker.hh
memtable: Switch onto B+ rails
2020-07-14 16:30:02 +03:00
dynamic_bitset.cc
dynamic_bitset: optimize for large sets
2018-04-07 14:52:58 +03:00
dynamic_bitset.hh
dynamic_bitset: optimize for large sets
2018-04-07 14:52:58 +03:00
enum_option.hh
treewide: update concepts language from the Concepts TS to C++20
2020-06-02 09:12:21 +03:00
error_injection.cc
utils: add error injection
2020-03-20 19:37:48 +01:00
error_injection.hh
codebase wide: replace find(...) != end() with contains
2020-08-11 13:28:50 +03:00
estimated_histogram.hh
approx_exponential_histogram: Makes the implementation clearer
2020-06-18 14:18:21 +03:00
exceptions.cc
utils: Use on_internal_error from seastar
2020-02-29 19:28:57 +02:00
exceptions.hh
utils: Use on_internal_error from seastar
2020-02-29 19:28:57 +02:00
exponential_backoff_retry.hh
treewide: add missing headers and/or forward declarations
2020-03-23 09:29:45 +02:00
extremum_tracking.hh
utils/extremum_tracking: drop default constructor
2019-02-07 18:31:25 +02:00
fb_utilities.hh
treewide: add const qualifiers throughout the code base
2019-11-26 02:24:49 +03:00
file_lock.cc
treewide: replace calls to engine().some_api() with some_api()
2020-04-05 12:46:04 +03:00
file_lock.hh
file_lock: Work with fs::path, not sstring
2019-12-12 17:32:10 +03:00
flush_queue.hh
codebase wide: replace count with contains
2020-08-15 20:26:02 +03:00
fragment_range.hh
treewide: update concepts language from the Concepts TS to C++20
2020-06-02 09:12:21 +03:00
fragmented_temporary_buffer.hh
treewide: update concepts language from the Concepts TS to C++20
2020-06-02 09:12:21 +03:00
generation-number.cc
storage_service: Move get_generation_number to util/
2020-06-01 09:08:40 +03:00
generation-number.hh
storage_service: Move get_generation_number to util/
2020-06-01 09:08:40 +03:00
hash.hh
database: Use a flat_hash_map for _ks_cf_to_uuid
2020-06-14 08:18:39 -07:00
histogram_metrics_helper.hh
approx_exponential_histogram: Makes the implementation clearer
2020-06-18 14:18:21 +03:00
histogram.hh
Update seastar submodule
2018-11-21 00:01:44 +02:00
i_filter.cc
utils: convert sprint() to format()
2018-11-01 13:16:17 +00:00
i_filter.hh
utils: Use dedicated enum for Bloom filter format instead of a boolean.
2018-05-10 09:47:41 +03:00
in.hh
treewide: add const qualifiers throughout the code base
2019-11-26 02:24:49 +03:00
input_stream.hh
Merge seastar upstream
2016-09-28 17:34:16 +03:00
int_range.hh
treewide: add missing headers and/or forward declarations
2020-03-23 09:29:45 +02:00
intrusive-array.hh
intrusive-array: Array with trusted bounds
2020-07-14 16:29:49 +03:00
joinpoint.hh
treewide: replace calls to engine().some_api() with some_api()
2020-04-05 12:46:04 +03:00
large_bitset.cc
large_bitset: use a chunked_vector internally and simplify API
2018-04-10 10:25:06 +03:00
large_bitset.hh
large_bitset: be more accurate with memory usage
2018-05-15 11:22:21 -04:00
latency.hh
histograms: do not use latency_in_nano
2016-11-14 18:01:43 +02:00
like_matcher.cc
cql3: Allow repeated LIKE on same column
2020-02-27 09:34:51 -05:00
like_matcher.hh
cql3: Allow repeated LIKE on same column
2020-02-27 09:34:51 -05:00
limiting_data_source.cc
Introduce make_limiting_data_source
2018-04-16 20:56:30 +02:00
limiting_data_source.hh
Introduce make_limiting_data_source
2018-04-16 20:56:30 +02:00
linearizing_input_stream.hh
treewide: update concepts language from the Concepts TS to C++20
2020-06-02 09:12:21 +03:00
loading_cache.hh
Update seastar submodule
2020-08-04 17:54:45 +03:00
loading_shared_values.hh
treewide: update concepts language from the Concepts TS to C++20
2020-06-02 09:12:21 +03:00
log_heap.hh
treewide: update concepts language from the Concepts TS to C++20
2020-06-02 09:12:21 +03:00
logalloc.cc
lsa: decay reserves
2020-09-08 15:59:25 +03:00
logalloc.hh
lsa: decay reserves
2020-09-08 15:59:25 +03:00
managed_bytes.cc
linearization_context: remove non-trivial operations from fast path
2018-01-30 18:33:25 +01:00
managed_bytes.hh
managed_bytes: Delete dead 'if'
2020-07-17 11:58:53 +03:00
managed_ref.hh
managed_ref: add external_memory_usage()
2019-10-15 15:41:42 +03:00
managed_vector.hh
managed_vector: Make external_memory_usage() ignore reserved space
2018-09-03 17:09:54 +03:00
memory_data_sink.hh
treewide: add missing headers and/or forward declarations
2020-03-23 09:29:45 +02:00
meta.hh
treewide: add missing headers and/or forward declarations
2020-03-23 09:29:45 +02:00
multiprecision_int.cc
utils: introduce multiprecision_int
2020-03-04 12:42:57 +02:00
multiprecision_int.hh
utils: introduce multiprecision_int
2020-03-04 12:42:57 +02:00
murmur_hash.cc
everywhere: Insert space after switch
2020-08-18 14:31:04 +03:00
murmur_hash.hh
everywhere: Insert space after switch
2020-08-18 14:31:04 +03:00
mutable_view.hh
treewide: add missing headers and/or forward declarations
2020-03-23 09:29:45 +02:00
neat-object-id.hh
utils: B+ tree implementation
2020-07-14 16:29:43 +03:00
observable.hh
observable: allow an observable to disconnect() twice without penalty
2018-07-11 10:15:01 +01:00
overloaded_functor.hh
utils: Add overloaded_functor helper.
2018-07-20 13:50:17 -07:00
phased_barrier.hh
utils: phased_barrier: Make advance_and_await() have strong exception guarantees
2018-11-20 16:15:12 +00:00
preempt.hh
treewide: add missing headers and/or forward declarations
2020-03-23 09:29:45 +02:00
ranges.hh
utils: introduce ranges::to()
2020-05-31 12:58:59 +03:00
rate_limiter.cc
Replace std::experimental types with C++17 std version.
2019-01-08 13:16:36 +02:00
rate_limiter.hh
treewide: replace calls to engine().some_api() with some_api()
2020-04-05 12:46:04 +03:00
reusable_buffer.hh
treewide: update concepts language from the Concepts TS to C++20
2020-06-02 09:12:21 +03:00
rjson.cc
rjson: Add templated get/set overloads and optional get<T>
2020-07-15 08:10:23 +00:00
rjson.hh
rjson: Add templated get/set overloads and optional get<T>
2020-07-15 08:10:23 +00:00
runtime.cc
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
runtime.hh
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
sequenced_set.hh
Update seastar submodule
2020-03-23 11:59:30 +02:00
serialization.hh
treewide: update concepts language from the Concepts TS to C++20
2020-06-02 09:12:21 +03:00
serialized_action.hh
Update seastar submodule
2020-08-19 17:18:57 +03:00
small_vector.hh
utils/small_vector: Add missing include
2020-03-03 21:23:40 +02:00
stall_free.hh
utils: Add clear_gently
2020-08-11 19:37:47 +08:00
streaming_histogram.hh
streaming_histogram: add missing include for uint64_t
2020-05-23 11:09:10 +03:00
top_k.hh
Merge "nodetool toppartitions" from Rafi & Avi
2018-12-28 16:31:24 +01:00
updateable_value.cc
treewide: replace calls to engine().some_api() with some_api()
2020-04-05 12:46:04 +03:00
updateable_value.hh
utils::updateable_value: add operator=(T)
2020-02-27 18:11:54 +02:00
utf8.cc
utf8: Print invalid UTF-8 character position
2020-09-07 18:11:21 +03:00
utf8.hh
utf8: Print invalid UTF-8 character position
2020-09-07 18:11:21 +03:00
UUID_gen.cc
Avoid including cryptopp headers
2019-02-20 08:03:46 -08:00
UUID_gen.hh
cql3: time_uuid_fcts: validate time UUID
2020-01-27 11:09:01 +02:00
uuid.cc
Update seastar submodule
2018-11-21 00:01:44 +02:00
UUID.hh
utils: UUID: assert UUID.is_timestamp
2020-01-26 18:54:36 +02:00