Commit Graph

7 Commits

Author SHA1 Message Date
Piotr Dulikowski
338e473946 cdc: fix non-atomic updates in splitting
This patch fixes a bug in mutation splitting logic of CDC. In the part
that handles updates of non-atomic clustering columns, the column
definition was fetched from a static column of the same id instead of
the actual definition of the clustering column. It could cause the value
to be written to a wrong column.

Tests: unit(dev)
2020-03-23 13:47:23 +01:00
Kamil Braun
3200d415da cdc: use a single timeuuid value for a batch of changes
If a batch update is performed with a sequence of changes with a single
timestamp, they will now show up in CDC with a single timeuuid in the
`time` column, distinguished by different `batch_seq_no` values.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
2020-03-05 12:32:57 +01:00
Kamil Braun
292eba9da0 cdc: replace split with for_each_change
`for_each_change` is like `split` but it doesn't return a vector of
mutations representing each change; instead, it takes as a parameter
a function which gets called on each mutation.

This reduced the memory usage and allows to preserve common context
when handling each change (will be useful in next commits).

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
2020-03-05 12:05:08 +01:00
Piotr Dulikowski
204e204586 cdc: do not attempt to log empty mutations
It is possible to produce an empty mutation using CQL. For example, the
following query:

DELETE FROM ks.tbl WHERE pk = 0 AND ck < 1 AND ck > 2;

will attempt to delete from an empty range of rows. This is translated
to the following mutation:

{ks.tbl {key: pk{000400000000}, token:-3485513579396041028}
 {mutation_partition:
  static: cont=1 {row: },
  clustered: {}}}

Such mutation does not contain any timestamp, therefore it is difficult
to determine what timestamp was used while making the query. This is
problematic for CDC, because an entry in CDC log should be written with
the same timestamp as a part of the mutation.

Because an empty mutation does not modify the table in any way, we can
safely skip logging such mutations in CDC and still preserve the
ability to reconstruct the current state of the base table from full
CDC log.

Tests: unit(dev)
2020-03-05 08:32:54 +01:00
Kamil Braun
529d30ef66 cdc: add split function
This function takes a mutation and returns a set of mutations, each
representing a separate change with a single timestamp and ttl.
2020-03-03 13:17:51 +01:00
Kamil Braun
132ea89c32 cdc: add extract_changes function
This commit introduces a bunch of new structs describing a change made
to a table, and an `extract_changes` function which takes a mutation and
returns the set of changes contained in this mutation, separated by
timestamp and ttl.
2020-03-03 13:17:51 +01:00
Kamil Braun
b5c944370e cdc: add should_split function
The function checks if there are multiple timestamps and/or ttls inside
a mutation, which means separate changes should be created for this
mutation in CDC.
2020-03-03 13:17:50 +01:00