Merged pull request https://github.com/scylladb/scylla/pull/6427
by Piotr Jastrzębski:
CDC Log is a time series so it makes sense to use time window compaction
strategy for it.
Our support for time series is limited so we make sure that we don't create
more than 24 sstables.
If TTL is configured to 0, meaning data does not expire, we don't use time
window compaction strategy.
This PR also sets gc_grace_seconds to 0 when TTL is not set to 0.
CDC Log is a time series with data TTLed by default to 24 hours so
it makes sense to use for it a time window compaction.
A window size is adjusted to the TTL configured for CDC Log so that
no more than 24 sstables will be created.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
"
We inherited from Origin a `caching` table parameter. It's a map of named caching parameters. Before this PR two caching parameters were expected: `keys` and `rows_per_partition`. So far we have been ignoring them. This PR adds a new caching parameter called `enabled` which can be set to `true` or `false` and controls the usage of the cache for the table. By default, it's set to `true` which reflects Scylla behavior before this PR.
This new capability is used to disable caching for CDC Log table. It is desirable because CDC Log entries are not expected to be read often. They also put much more pressure on memory than entries in Base Table. This is caused by the fact that some writes to Base Table can override previous writes. Every write to CDC Log is unique and does not invalidate any previous entry.
Fixes#6098Fixes#6146
Tests: unit(dev, release), manual
"
* haaawk-dont_cache_cdc:
cdc: Don't cache CDC Log table
table: invalidate disabled cache on memtable flush
table: Add cache_enabled member function
cf_prop_defs: persist caching_options in schema
property_definitions: add get that returns variant
feature: add PER_TABLE_CACHING feature
caching_options: add enabled parameter
"
CDC has to create CDC streams that are co-located with corresponding BaseTable data. This is not always easy. Especially for small vnodes. This PR introduces new partitioner which allows us to easily find such stream ids that the stream belongs to a given vnode and shard.
The idea is that a partitioner accepts only keys that are a blob composed of two int64 numbers. The first number is the token of the key.
Tests: unit(dev), dtests(CDC)
"
* haaawk-cdc_partitioner:
cdc:use CDCPartitioner for CDC Log
dht: Add find_first_token_for_shard
dht: use long_token in token::to_int64
cdc: add CDCPartitioner
stream_id: add token_from_bytes static function
i_partitioner: Stop distinguishing whether keys order is preserved
CDC writes are not expected to be read multiple times so it makes little sense
to cache them. Moreover, CDC Log puts much bigger pressure on memory usage than
Base Table because some updates to the Base Table override existing data while
related CDC Log updates are always a new entry in a memtable.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
1. Remove the `versioned_value::factory` class, it didn't add any value. It just
forced us to create an object for making `versioned_value`s, for no sensible
reason.
2. Move some `versioned_value` deserialization code (string -> internal data
structures) into the versioned_value module. Previously, it was scattered all
around the place.
3. Make `gossiper::get_seeds` const and return a const reference.
I needed these refactors for a PR I was preparing to fix an issue with CDC. The
attempt of fixing the issue failed (I'm trying something different now), but the
refactors might be useful anyway.
* kbr--vv-refactor:
gossiper: make `get_seeds` method const and return a const ref
versioned_value: remove versioned_value::factory class
gms: move TOKENS string deserialization code into versioned_value
This commit resolves the client-observable effect of CDC read
consistencies. I wrapped the preimage's SELECT query in try-catch to
intercept the `unavailable_exception`, which led to misleading
`NoHostAvailable` in Python and Java drivers. Now client gets a new
error code and a message specific to the issue of CL not being met
by the preimage query.
Fixes#5746
This will allow deterministic stream_id generation
and would remove the risk of not being able to generate
a stream id for some vnode.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
This is a special partitioner that will be used by
CDC Log. It works only with partition key that is blob
composed of two ints. The first int is a token this
partitioner will map the key to. The second int is there
to make it possible to create multiple keys that are different
from each other but map to the same token.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
CL of LOCAL_QUORUM used to be hardcoded into CDC preimage query
and led to an error when number of replicas was lower than CL
would require. The solution here is to link the CLs of writes
to base table with the CLs of CDC reads, so the client will get
the (limited) control over the consistency of preimage SELECTs
(instead of getting error every time).
The algorithm is as follows:
1. If write that caused CDC activity was done with CL = ANY,
then do preimage read with CL = ONE.
2. If write that caused CDC activity was done with CL = ALL,
then do preimage read with CL = QUORUM.
3. SERIAL and LOCAL_SERIAL writes cause preimage read with QUORUM
and LOCAL_QUORUM, respectively.
4. In other cases do preimage read with the same CL as base write.
And do the same with CDC_STREAMS_TIMESTAMP.
The code that took a list of tokens represented as a string inside
versioned_value (for gossiping) and deserialized it into
an `unordered_set<dht::token>` lived in the storage_service module,
while the code that did the serializing (set -> string) lived in
versioned_value. There was a similar situation with the CDC generation
timestamp.
To increase maintanability and reusability, the deserialization code is
now placed next to the serialization code in versioned_value.
Furthermore, the `make_full_token_string`, `make_token_string`, and
`make_cdc_streams_timestamp_string` (serialization functions) are moved
out of versioned_value::factory and made static methods of
versioned_value instead.
For a column of type `frozen<list<T>>` in base table, a corresponding
column of type `frozen<map<timeuuid, T>>` is created in cdc log.
Although a similar change of type takes place in case of non-frozen
lists, this is unneeded in case of frozen lists - frozen collections are
atomic, therefore there is no need for complicated type that will be
able to represent a column update that depends on its previous value
(e.g. appending elements to the end of the list).
Moreover, only cdc log table creation logic performs this type change
for frozen lists. The logic of `transformer::transform`, which is
responsible for creation of mutations to cdc log, assumes that atomic
columns will have their types unchanged in cdc log table. It simply
copies new value of the column from original mutation to the cdc log
mutation. A serialized frozen list might be copied to a field that is of
frozen map type, which may cause the field to become impossible to
deserialize.
This patch causes frozen list base table columns to have a corresponding
column in cdc log with the same type.
A test is added which asserts that the type of cdc log columns is not
changed in the case of frozen base columns.
Tests: unit(dev)
Fixes#6172
Fixes#6143
When doing post-image generation, we also write values for columns not
in delta (actual update), based on data selected in pre-image row.
However, if we are doing initial update/insert with only a subset of
columns, when the pre-image result set is nil, this cannot be done.
Adds check to non-touched column post-image code. Also uses the
pre-image value extractor to handle non-atomic sets properly.
Tests updated.
Fixes#6073
The logic with pre/post image was tangled into looking at "rs"
and would cause pre-image info to be stored even if only post-image
data was enabled.
Now only generate keys (and rows for them) iff explicitly enabled.
And only generate pre-image key iff we have pre-image data.
Fixes#6070
When mutation splitting was added, non-atomic column assignments were broken
into two invocation of transform. This means the second (actual data assignment)
does not know about the tombstone in first one -> postimage is created as if
we were _adding_ to the collection, not replacing it.
While not pretty, we can handle this knowing that we always get
invoked in timestamp order -> tombstone first, then assign.
So we simply keep track of non-atomic columns deleted across calls
and filter out preimage data post this.
Added test cases for all non-atomics
Merged pull request https://github.com/scylladb/scylla/pull/6030 from
Piotr Dulikowski:
Adds CDC-related metrics.
Following counters are added, both for total and failed operations:
Total number of CDC operations that did/did not perform splitting,
Total number of CDC operations that touched a particular mutation part.
Total number of preimage selects.
Fixes#6002.
Tests: unit(dev, debug)
* 'cdc-metrics' of github.com:piodul/scylla:
storage_proxy: track CDC operations in LWT flow
storage_proxy: track CDC operations in logged batches
storage_proxy: track CDC operations in standard flow
storage_proxy: add cdc tracker hooks to write response handlers
storage_proxy: move "else if" remainder into "else" block
cdc: create an operation_result_tracker object
cdc: add an object for tracking progress of cdc mutations
cdc: count touched mutation parts in transformer::transform
cdc: track preimage selects in metrics
cdc: register metric counters
cdc: fix non-atomic updates in splitting
CDC metrics, apart from tracking "total" metrics for all performed CDC
operations, also track metrics for "failed" operations. Because the
result of the CDC operation depends on whether all CDC mutations were
written successfully by storage_proxy, checking for failure and
incrementing appropriate counters is deferred after all write response
handlers finish.
The `cdc::operation_result_tracker` object was created for that purpose.
It contains all the details needed to accurately update the metrics
based on what actually happened in the `augment_mutation_call` function,
and holds a flag which tells if any of write response handlers failed.
This object is supposed to be referenced by write response handlers for
CDC mutations created after the same `augment_mutation_call`. After all
write response handlers are destroyed, the destructor of
`operation_result_tracker` will update appropriate metrics.
Actual creating and attaching this object to write response handlers
will be done in subsequent commits.
Modifies the transformer::transform so that it also returns a set of
flags indicating what parts of the mutation (e.g. rows, tombstones,
collections, etc.) were processed during transforming.
This patch defines a CDC metrics object and registers all of its
counters.
storage_proxy is chosen as the owner of the metrics object. Because in
subsequent commits it will become possible for CDC metrics to be updated
after a write operation ends, and because the cdc_service has shorter
lifetime than storage_proxy, we could risk a use-after-free if we placed
this object inside cdc_service.
This patch fixes a bug in mutation splitting logic of CDC. In the part
that handles updates of non-atomic clustering columns, the column
definition was fetched from a static column of the same id instead of
the actual definition of the clustering column. It could cause the value
to be written to a wrong column.
Tests: unit(dev)
This patch fixes a bug in mutation splitting logic of CDC. In the part
that handles updates of non-atomic clustering columns, the schema for
serializing that column was looked up incorrectly in the table schema -
instead of a `regular_column`, a `static_column` was looked up.
Due to how the `column_at` function works, a correct schema was always
returned if the table had no static columns. Therefore, in order for
this bug to manifest, a table with a static column and a regular column
with non-atomic collection was needed.
Primary key and clustering key column should not have a corresponding
"cdc$deleted_<name>" column in cdc log table, because it does not make
sense to delete such a column from a row.
Fixes: #6049
Tests: unit(dev)
The `result_callback` was a callback returned by `augment_mutation_call`
that was supposed to be used in the CDC postimage implementation.
Because CDC postimage was implemented without using this callback, and
currently a no-op function is always returned, this callback can safely
be removed.
Merged pull request https://github.com/scylladb/scylla/pull/5996 from
Calle Wilund:
Fixes#4992
Implements post-image support by synthesizing it from
pre-image + delta.
Post-image data differs from the delta data in two ways:
1.) It merges non-atomics into an actual result value
2.) It contains all columns of the row, not just
those affected by the update.
For a non-atomic field, the post-image value of a column
is either the pre-image or the delta (maybe null)
Tested by adding post-image checks to pre-image test
and collection/udt tests
Fixes#4992
Implements post-image support by synthesizing it from
pre-image + delta.
Post-image data differs from the delta data in two ways:
1.) It merges non-atomics into an actual result value
2.) It contains _all_ columns of the row, not just
those affected by the update.
For a non-atomic field, the post-image value of a column
is either the pre-image or the delta (maybe null)
Tested by adding post-image checks to pre-image test
and collection/udt tests
The is_log_for_some_table function incorrectly assumed that
database::find_schema would return a null pointer in case the queried
schema does not exist. This patch fixes that, and now this function
checks for existence of the schema using database::has_schema.
Tests: unit(dev)
If base mutation has at least one row tombstone, its preimage log
entry is constructed from all the base columns.
Fixes#5709
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
In case a static and a clustering row is written at the same time, but
a clustering row with given key was not present, the preimage query was
incorrectly configured and no rows were returned. This resulted in an
empty preimage, while a preimage for static row should be present.
This patch fixes this and now the static row is correctly written to cdc
log in the case above.
Tests: unit(dev)
This change disallows creating CDC log tables for already existing CDC
log tables. CDC logs nested in that way are not really useful and do not
work at the moment, therefore disallowing their creation prevents
confusion.
Instead of storing a raw map of options inside `cdc_extension`, the
extension now converts them into `cdc_options` directly on construction.
This removes the need to construct `cdc_options` object multiple times.
With #5950 we changed the representation of stream_id
in CDC Log from two int columns to a single blob column.
This PR cleans up stream_id representation internally.
Now stream_id is stored as blob both in-memory and in
internal CDC tables.
Tests: unit(dev)
* hawk/stream_id_representation:
cdc: store stream_ids as blobs in internal tables
cdc: improve do_update_streams_description
cdc: Fix generate_topology_description
cdc: add stream_id::operator<
cdc: change stream_id representation
If a batch update is performed with a sequence of changes with a single
timestamp, they will now show up in CDC with a single timeuuid in the
`time` column, distinguished by different `batch_seq_no` values.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
`for_each_change` is like `split` but it doesn't return a vector of
mutations representing each change; instead, it takes as a parameter
a function which gets called on each mutation.
This reduced the memory usage and allows to preserve common context
when handling each change (will be useful in next commits).
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
In new CDC Log format stream_id is represented by a single
blob column so it makes sense to store it in the same form
everywhere - including internal CDC tables.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Use std::set::insert that takes range instead of
looping through elements and adding them one by one.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
In new CDC Log format we store only a single stream_id column.
This means generate_topology_description has to use appropriate
schema for generating tokens for stream_ids.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
New CDC Log format stores stream ids as blobs.
It makes sense to keep them internally in the same form.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Currently, writes to a static row in a base table are not reflected
at all in the corresponding cdc log. This patch causes such writes
to be properly logged.
Fixes: #5744
Tests: unit(dev)
* piodul/5744-handle-static-row-correctly-in-cdc:
cdc_test: add tests for handling static row
cdc: fix indentation in transformer::transform
cdc: handle static rows separately in transformer::transform
cdc: move process_cells higher (and fix captured variables)
cdc: reduce dependencies on captured variables in process_cells
cdc: fix preimage query for static rows
It is possible to produce an empty mutation using CQL. For example, the
following query:
DELETE FROM ks.tbl WHERE pk = 0 AND ck < 1 AND ck > 2;
will attempt to delete from an empty range of rows. This is translated
to the following mutation:
{ks.tbl {key: pk{000400000000}, token:-3485513579396041028}
{mutation_partition:
static: cont=1 {row: },
clustered: {}}}
Such mutation does not contain any timestamp, therefore it is difficult
to determine what timestamp was used while making the query. This is
problematic for CDC, because an entry in CDC log should be written with
the same timestamp as a part of the mutation.
Because an empty mutation does not modify the table in any way, we can
safely skip logging such mutations in CDC and still preserve the
ability to reconstruct the current state of the base table from full
CDC log.
Tests: unit(dev)
