Currently overflow values are stored in incorrect bucket (last one
instead of special "overflow" one) and percentile() function throws
if there is overflow value. The patch fixes the code to store overflow
value in corespondent bucket and makes percentile() to take it into
account instead of throwing.
Message-Id: <20170911131752.27369-2-gleb@scylladb.com>
"This series fixes the problem of active reads causing OOM due to the fact that
partition snapshots they hold are not evictable. In particular, a single scan
of a partition larger than memory will bad_alloc due to itself.
After this, when partition entry is evicted from cache, data in all the snapshots
is also evicted. We still don't have row-level eviction, but this series lays some
grounds for it by making cache readers prepared for the possibility of rows
being evicted.
Fixes#2775.
Fixes #2730."
* tag 'tgrabiec/snapshot-evicition-in-cache-v1' of github.com:scylladb/seastar-dev:
tests: Add test for partition_entry::evict()
mutation_partition: Introduce range continuity checking methods
mutation_partition: Enable rows_entry::compare() on position_in_partition_views
tests: Extract mvcc tests to separate file
tests: row_cache: Add evicition tests
tests: simple_schema: Add new_tombstone() helper
tests: streamed_mutation_assertions: Introduce produces(mutation&)
streamed_mutation: Allow setting buffer capacity
row_cache: Evict partition snapshots
mvcc: Introduce partition_entry::evict()
row_cache: Handle eviction in partition reader
tests: row_cache_test: Don't assume mvcc snapshots are not evictable
row_cache: Reuse allocation_strategy::invalidate_references()
row_cache: Don't invalidate references on insertion
lsa: Move reclaim counter concept to allocation_strategy level
mvcc: Ensure partition_snapshot always destroys versions using proper allocator
mvcc: Encapsulate reference stability check in partition_snapshot
mvcc: Store LSA region reference in partition_snapshot
If snapshots are not evicted, they may pin unbouned amount of memory
for a long time in cache, which may lead to OOM. Evict snapshots
together with the entry.
Fixes#2775.
Fixes#2730.
The test was not updating the underlying mutation source but still
expecting to get the right data after calling invalidate(). If
snapshots are evictable, that's not guaranteed. Apply to underlying as
well, so data is read from underlying if necessary.
modification_count is currently only used to detect invalidation of
references, intended to be incremented on erasure.
Insertion into intrusive set doesn't invalidate references, so no
need to increment the counter.
partition_snapshot is managed by lw_shared_ptr. Currently it is
assumed that before it dies, maybe_merge_versions() is called on it,
which destroyes it in the right allocator context. It's not very
safe. This patch improves safety by using the right allocator in
snapshot's destructor.
This fixes a failure in view_schema_test, which starts many instances
of single_node_cql_env. cancel_atomic_deletions() causes later
deletions to fail, which causes some of the test cases to fail.
Message-Id: <1505311250-3118-2-git-send-email-tgrabiec@scylladb.com>
Fixes a hang on shutdown with --smp 2 in perf_fast_forward. The hang
is in sstables::await_background_jobs_on_all_shards(), which is
waiting on sstable deletions. Not all shards agree to delete certain
sstables, because e.g. not all shards decide to compact them
yet. Cancel those deletes after database is stopped on all shards,
like we do in main.cc
Fixes#2796.
Message-Id: <1505292239-26032-1-git-send-email-tgrabiec@scylladb.com>
Finds live objects on seastar heap of current shard which contain
given value. Prints results in 'scylla ptr' format.
Example:
(gdb) scylla find 0x600005321900
thread 1, small (size <= 512), live (0x6000000f3800 +48)
thread 1, small (size <= 56), live (0x6000008a1230 +32)
Message-Id: <1505284614-19577-1-git-send-email-tgrabiec@scylladb.com>
compaction_strategy.hh throws an exception, but it doesn't add the
exception header. It is working in-tree because of inclusion order,
but it broke one of my yet-out-of-tree changes.
In any case, it is best to add the headers we will need to the files,
and that is what this patch does.
Signed-off-by: Glauber Costa <glauber@scylladb.com>
Message-Id: <20170912233326.26114-1-glauber@scylladb.com>
filter.cc has just two smallish functions, which are part of the sstable
class. Move them to sstables.cc where the rest of the class members are defined.
Message-Id: <20170912080541.7836-1-avi@scylladb.com>
The test used apply() variant which assumed that it was invoked in a
seastar thread, which is no longer the case after commit d22fdf4. Fix
by copying outisde cache update, and use non-deferring apply() variant
for cache update.
Message-Id: <1505200142-3650-1-git-send-email-tgrabiec@scylladb.com>
This patchset reduces includes of sstables.hh, reducing compile time
by both reducing the amount of code compiled, and the amount of
needless recompiles caused by false dependencies. It does so by
replacing lw_shared_ptr<sstable>, which requires a complete class,
with a new custom type shared_sstable, which allows an incomplete
sstable class definition.
* https://github.com/avikivity/scylla deps2/v2.1
database: change truncate() to flush while compaction is disabled
database: make run_with_compaction_disabled() a non-template
database: add indirection to compaction_manager instance
database: remove dependency on compaction.hh and compaction_manager.hh
size_estimates_virtual_reader.hh: add missing include
system_keyspace: add missing include
main: add missing include
storage_service: add missing include
repair: add missing include
compaction.hh: add missig include and forward declaration
compaction_manager: add missing include
shared_index_lists.hh: add missing include
perf_fast_forward: add missing include
sstable_mutation_test: add missing include
sstables: extract version and format enum into a separate header file
database.hh: add missing forward declaration for
foreign_sstable_open_info
cql_test_env: add forward declaration
database: make column_family::disable_sstable_write() out-of-line
sstables: introduce make_sstable() as a shortcut for
make_lw_shared<sstable>
treewide: use shared_sstable, make_sstable in place of
lw_shared_ptr<sstable>
sstables: use support for lw_shared_ptr with incomplete type for
shared_sstable
sstables: reduce dependencies
streaming: remove unneeded includes
When table is created, it doesn't contain any data, so we can mark the whole
data range as continuous in cache. This way reads will immediately hit, and
flushes will populate. If sstables are later attached, the attaching process
is supposed to invalidate affected ranges (and it does).
Fixes#2536.
Message-Id: <1505200269-4031-1-git-send-email-tgrabiec@scylladb.com>
Use the lw_shared_ptr deleter support to define shared_sstable without
pulling the definition of class sstable, reducing compile time and
dependencies if only shared_sstable is needed.
