Cassandra disallows adding a column with the same name as a collection
that existed in the past in that table if the types aren't compatible.
To enforce that Scylla needs to keep track of all collections that ever
existed in the column family.
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
When a column is dropped its name and deletion timestamp are added
to schema::_raw._dropped_columns to prevent data resurrection in case a
column with the same name is added. To reduce the number of lookups in
_dropped_columns this patch makes each instance of column_definition
to caches this information (i.e. timestamp of the latest removal of a
column with the same name).
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
Knowing which columns were dropped (and when) is important to prevent
the data from the dropped ones reappearing if a new column is added with
the same name.
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
We want the node's schema version to change whenever
table_schema_version of any table changes. The latter is calculated by
hashing mutations so we should also use mutation hash when calculating
schema digest.
We need to track which schema version were synced with on current node
to avoid triggering the sync on every mutation. We need to sync before
mutating to be able to apply the incoming mutation using current
node's schema, possibly applying irreverdible transformations to it to
make it conform.
Right now in some places we use column_id, and in some places
size_t. Solve it by using column_count_type whose meaning is "an
integer sufficiently large for indexing columns". Note that we cannot
use column_id because it has more meaning to it than that.
The version needs to change value not only on structural changes but
also temporal. This is needed for nodes to detect if the version they
see was already synchronized with or not even if it has the same
structure as the past versions. We also need to end up with the same
version on all nodes when schema changes are commuted.
For regular mutable schemas version will be calculated from underlying
mutations when schema is announced. For static schemas of system
keyspace it is calculated by hashing scylla version and column id,
because we don't have mutations at the time of building the schema.
For static and regular (row) columns it is very convenient in some
cases to utilize the fact that columns ordered by ids are also ordered
by name. It currently holds, so make schema export this guarantee and
enable consumers to rely on.
The static schema::row_column_ids_are_ordered_by_name field is about
allowing code external to schema to make it very explicit (via
static_assert) that it relies on this guarantee, and be easily
discoverable in case we would have to relax this.
Instead of accepting a column resolver callable, accept a schema and
column_kind or column_selector. Makes the interface easier to use and
enables us to move implementation to .cc file.
We will use those properties during initialization - for instance, to calculate
thrift_bits.is_on_all_components. In order to do that, it has to be available at
schema creation, and not through the schema builder.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
There is a small but potentially serious mistake in the way we are computing
the number of regular columns. We are using an incorrect offset, and the
number of regular columns in a table like this:
schema(...,
{{"pk", utf8_type}},
{{"ck", utf8_type}, {"ck2", utf8_type}},
{
{"reg1", long_type},
{"reg2", long_type},
},
{{"static", utf8_type}},
utf8_type,
"comment"
);
is being reported as 3.
Fix this
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Because we don't support alter table, we won't have anyone actually manipulating this.
But we will set up so it appears properly as an empty map in queries.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
When added to the schema and handled by legacy_schema_tables.cc, will then
appear correctly in the respective system tables.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
It will be stored in the schema, so move there, where it belongs. We'll need
to do more than just store a type, so provide a class that encapsulates it.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Because we will do from and to sstring operations on it, it needs
a default value. The lack of it is causing our tests to fail.
This should have been included in the original patch but I have somehow
missed it. Sorry.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Origin has another column_kind, that we lack: compact_value. This kind is
used to identify regular columns of dense tables.
Take for instance, the following table:
CREATE TABLE ks2.compact (
ks text,
cl1 text,
cl2 text,
PRIMARY KEY (ks, cl1)
) WITH COMPACT STORAGE
cqlsh> select keyspace_name, columnfamily_name, column_name, type from system.schema_columns \
where keyspace_name='ks2' and columnfamily_name='compact';
keyspace_name | columnfamily_name | column_name | type
---------------+-------------------+-------------+----------------
ks2 | compact | cl1 | clustering_key
ks2 | compact | cl2 | compact_value
ks2 | compact | ks | partition_key
We will treat those columns as regular columns for most purposes. Because of
that, we don't need to separate them from the regular columns when we sort
initially, for instance. All we have to do is change its type.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
We are deviating a bit from Origin here: In Origin, we would store a full
comparator class. However, due to the fact that our types are very different,
and as a consequence we will not call a serializer directly on the cell name,
that is not necessary.
The only information that we will need to store is whether or not the table is
compound. Some functions to manipulate it will be presented in the next patch.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Get values from cf->schema instead of using hardcoded threshold
values. In addition, move DEFAULT_MIN_COMPACTION_THRESHOLD and
DEFAULT_MAX_COMPACTION_THRESHOLD to schema.hh so as not to have
knowledge duplicated.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Make all callers go through the builder. Current callers - there are many
in the system tables code, are patched.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
