Since invalidate() may allocate, we need to take the region lock to
keep m.partitions references valid around whole clear_and_dispose(),
which relies on that.
SSTables already have a priority argument wired to their read path. However,
most of our reads do not call that interface directly, but employ the services
of a mutation reader instead.
Some of those readers will be used to read through a mutation_source, and those
have to patched as well.
Right now, whenever we need to pass a class, we pass Seastar's default priority
class.
Signed-off-by: Glauber Costa <glauber@scylladb.com>
Currently for wrap around the "begin" iterator would not meet with the
"end" iterator, invoking undefined behavior in erase_and_dispose()
which results in a crash.
Fixes#785
There is one current schema for given column_family. Entries in
memtables and cache can be at any of the previous schemas, but they're
always upgraded to current schema on access.
The intent is to make data returned by queries always conform to a
single schema version, which is requested by the client. For CQL
queries, for example, we want to use the same schema which was used to
compile the query. The other node expects to receive data conforming
to the requested schema.
Interface on shard level accepts schema_ptr, across nodes we use
table_schema_version UUID. To transfer schema_ptr across shards, we
use global_schema_ptr.
Because schema is identified with UUID across nodes, requestors must
be prepared for being queried for the definition of the schema. They
must hold a live schema_ptr around the request. This guarantees that
schema_registry will always know about the requested version. This is
not an issue because for queries the requestor needs to hold on to the
schema anyway to be able to interpret the results. But care must be
taken to always use the same schema version for making the request and
parsing the results.
Schema requesting across nodes is currently stubbed (throws runtime
exception).
Schema is tracked in memtable and cache per-entry. Entries are
upgraded lazily on access. Incoming mutations are upgraded to table's
current schema on given shard.
Mutating nodes need to keep schema_ptr alive in case schema version is
requested by target node.
Before this change, populations could race with update from flushed
memtable, which might result in cache being populated with older
data. Populations started before the flush are not considering the
memtable nor its sstable.
The fix employed here is to make update wait for populations which
were started before the flushed memtable's sstable was added to the
undrelying data source. All populatinos started after that are
guaranteed to see the new data.
This reader enables range queries on row cache. An underlying key_reader
is used to obtain information about partitions that belong to the
specified range and if any of them isn't in the cache an underlying
mutation reader is used to read the missing data.
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
This mutation reader returns mutations from cache that are in a given
range. There may be other mutations in the system (e.g. in sstables)
that won't be returned, so this reader on its own cannot really satisfy
any query.
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
Cache has a tendency to eat up all available memory. It is evicted
on-demand, but this happens at certain points in time (during large
allocation requests). Small allocations which are served from small
object pools won't usually trigger this. Large allocations happen for
example when LSA region needs a new segment, eg. when row cache is
populated. If large allocations happen for certain period only inside
row_cache::update(), then eviction will not be able to make forward
progress because cache's LSA region is locked inside
row_cache::update(). While it's locked, data can't be evicted from
it.
The solution is to use allocating_section.
Fixes#376.
Restrict the impact of flushing a memtable to row_cache to 20% of the
cpu. This is accomplished by converting the code to a thread (with
bad indentation to improve patch readability) and using a thread
scheduling group.
Currently cache update which from a flushed memtable affects hits and
misses, which may be confusing. Let's reserve hits and misses for
reads. Cache update will affect counters called "insertions" and
"merges".
See #259.
When transferring mutations between memtable and cache, lsa sometimes
runs out of memory. This solves the first two points, keeping reserve
filled up and adjusting the amount of reserve based on execution
history.
This expose the cache tracker and the num entries in the row cache so it
can be used by the API.
And it adds a const getter for the region.
Both are const and are used for inspecting only.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
When LSA reclaimer cannot reclaim more space by compaction, it
will reclaim data by evicting from evictable regions.
Currently the only evictable region is the one owned by the row cache.
_lru_len may get stale when row_cache instance goes out of scope
purging all its partitions from cache. I'm assuming we're not really
interested in the number of partitions here, but rather a measure of
occupancy, so I applied a simple fix of using LSA region occupancy
instead.
row_cache class is meant to cache data for given table by wrapping
some underlying data source. It gives away a mutation_reader which
uses in-memory data if possible, or delegates to the underlying reader
and populates the cache on-the-fly.
Accesses to data in cache is tracked for eviction purposes by a
separate entity, the cache_tracker. There is one such tracker for the
whole shard.
