Commit Graph

121 Commits

Author SHA1 Message Date
Duarte Nunes
fbb4c9edda schema: Provide all-selecting partition slice
This patch introduces schema::full_slice(), which returns a
partition_slice selecting the full clustering range, as well as all
static and regular columns. No options aside from the default are
set in that partition_slice.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1507732800-9448-1-git-send-email-duarte@scylladb.com>
2017-10-17 11:25:35 +02:00
Duarte Nunes
e988121dbb schema_builder: Replace type when re-dropping column
Fixes #2634

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20170725183933.5311-1-duarte@scylladb.com>
2017-07-26 13:26:29 +02:00
Duarte Nunes
9c831b4e97 schema: Remove unnecessary print
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20170725174000.71061-1-duarte@scylladb.com>
2017-07-26 12:01:51 +02:00
Duarte Nunes
7eecda3a61 schema: Support compaction enabled attribute
Fixes #2547

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20170721132206.3037-1-duarte@scylladb.com>
2017-07-21 15:38:45 +02:00
Tomasz Grabiec
a9237c1666 schema: Revert back to the 1.7 layout of static compact tables in memory
We are using C* 3.x compatible layout in schema tables but want to
keep using the 1.7 layout in memory for compatibility during rolling
upgrade. This patch switches the schema and schema_builder classes
back to the old layout. Translation of layout happens when converting
to/from schema mutations.

Notable changes:

 1) Includes a revert of commit 6260f31e08
    "thrift: Update CQL mapping of static CFs".

 2) Brings back the "default_validation_class" schema attribute. In v3
    it can be dervied from column definitions, but in v2 it can't, so
    we have to store it.

 3) legacy_schema_migrator and schema_builder don't have to do
    conversions to v3, this is now handled by the v3_columns
    class. schema_builder works with the same layout as schema, that
    is v2.

 4) Includes a revert of commit 66991a7ccb
    "v3 schema test fixes"

Fixes #2555.
2017-07-19 09:52:15 +02:00
Tomasz Grabiec
dc2dc056a4 schema: Use v3 column layout when converting to/from schema mutations 2017-07-19 09:52:15 +02:00
Tomasz Grabiec
dc463ef644 schema: Encapsulate column layout translations in the v3_columns class 2017-07-19 09:52:15 +02:00
Duarte Nunes
6b464da67d schema: Get rid of regular_columns_by_name
They are unused.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20170717103635.6473-2-duarte@scylladb.com>
2017-07-17 12:52:41 +02:00
Tomasz Grabiec
34dae0588c schema: Use proper name comparator
This replaces column_definition::name_comparator, which incorrectly
assumes that names are always utf8, with name_compare moved from
schema::rebuild() and unifies usages.
2017-07-17 09:40:06 +02:00
Tomasz Grabiec
b919c50d21 schema: Use proper column name type for static columns
After f5dae826ce, static columns not
always have utf8 column names. For static compact tables it's
determined by the cell name comparator type, which is equal to the
type of the synthetic clustering column.

Caused various errors with static thrift tables with non-utf8
comparator.
2017-07-17 09:40:06 +02:00
Tomasz Grabiec
f685f7f8a1 schema: Fix column_name_type() for static compact tables
Introduced in f5dae826ce.
2017-07-17 09:40:06 +02:00
Tomasz Grabiec
84536a4a75 schema: Introduce clustering_column_at() 2017-07-17 09:40:06 +02:00
Duarte Nunes
4886b7ed5e schema: Lift maybe_quote() into cql3/util
It's a more natural place given its current and future usages.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-06-15 19:55:52 +00:00
Duarte Nunes
6260f31e08 thrift: Update CQL mapping of static CFs
This patch updates the mapping of static CFs so that their CQL
representation is a non-compound, non-dense schema with static
columns, instead of regular ones. This matches the representation os
static CFs in Cassandra 3.x.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-05-10 16:44:48 +00:00
Calle Wilund
6c8b5fc09d schema_tables: Use v3 schema tables and formats
Switches system/schema_* for system_schema/*, updates schema/schema
builder and uses to hold/expect v3 style info (i.e. types & dropped).
2017-05-10 16:44:48 +00:00
Calle Wilund
0e6ae8dec2 schema: rename column accessors to be in line with origin
More pointedly: Expose columns as is (currently
all_columns_in_select_order), expose name->column mapping more
appropriately named. 

Renaming like this is not strictly neccesary, but there is a point to
trying to keep nomenclature similar-ish with origin, esp. when select
order column need to become filtered (spoiler alert).
2017-05-10 16:44:48 +00:00
Calle Wilund
1c328a4166 schema_builder: Add helper to generate unique column names akin origin 2017-05-10 16:44:48 +00:00
Duarte Nunes
5387ac98f8 schema: Add utility functions for static columns
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-05-10 16:44:48 +00:00
Duarte Nunes
0439d83d1e schema: Use heterogeneous comparator for columns bounds
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-05-10 16:44:47 +00:00
Calle Wilund
f5fcadf0b1 schema: Add "as_cql_string" for column_def + quote-wrapper 2017-05-09 13:48:55 +00:00
Pekka Enberg
35bb6dedd8 schema: Kill index_type enum 2017-05-08 10:19:34 +03:00
Pekka Enberg
06564afedb schema: Kill index_info class
It's no longer used. Indices are managed by the index_metadata class.
2017-05-08 10:19:34 +03:00
Pekka Enberg
25e2777344 schema: Add all_indices() to schema class 2017-05-04 14:59:12 +03:00
Pekka Enberg
830591b092 schema: Remove add_default_index_names() from schema_builder class
The add_default_index_names() is part of the old and incomplete
secondary index implementation in Scylla. Drop it as it's no longer
used.
2017-05-04 14:59:12 +03:00
Pekka Enberg
ef29520c8e schema: Add get_default_index_name() to index_metadata class 2017-05-04 14:59:11 +03:00
Pekka Enberg
c6e7d4484a database: Make existing_index_names() per-keyspace operation 2017-05-04 14:59:11 +03:00
Pekka Enberg
5e8f2f49c3 schema: Add indices() to schema class 2017-05-04 14:59:11 +03:00
Pekka Enberg
5abd4b8041 schema: Add has_index() to schema class 2017-05-04 14:59:11 +03:00
Pekka Enberg
1fb1828aa2 schema: Add index_names() to schema class 2017-05-04 14:59:11 +03:00
Pekka Enberg
1c1a767408 schema: Add find_index_noname() to schema class
This adds a find_index_nomame() helper to the schema class, which
searches for index that is otherwise equal but ignores the name of the
index in comparison. This is needed to for CREATE INDEX to reject
duplicate index creation.
2017-05-04 14:59:11 +03:00
Pekka Enberg
62fba73a05 schema_builder: Add index_metadata support 2017-05-04 14:59:11 +03:00
Pekka Enberg
05e12a1d2b schema: Add index_metadata maps to raw_schema class 2017-05-04 13:22:12 +03:00
Pekka Enberg
2f83232a02 schema: Add index_metadata class 2017-05-02 10:29:18 +03:00
Duarte Nunes
bfb8a3c172 materialized views: Replace db::view::view class
The write path uses a base schema at a particular version, and we
want it to use the materialized views at the corresponding version.

To achieve this, we need to map the state currently in db::view::view
to a particular schema version, which this patch does by introducing
the view_info class to hold the state previously in db::view::view,
and by having a view schema directly point to it.

The changes in the patch are thus:

1) Introduce view_info to hold the extra view state;
2) Point to the view_info from the schema;
3) Make the functions in the now stateless db::view::view non-member;
4) Remove the db::view::view class.

All changes are structural and don't affect current behavior.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-03-15 15:50:05 +01:00
Duarte Nunes
a64c47f315 schema: Move raw_view_info outside of raw_schema
In preparation of an upcoming patch, where the schema
won't directly store the raw_view_info.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-03-15 15:38:31 +01:00
Duarte Nunes
4b209be8b8 view_info: Rename to raw_view_info
In preparation for upcoming patches, which will deal with
moving the state in db::view::view to view_info.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-03-15 15:38:31 +01:00
Paweł Dziepak
4cbbbc67f0 schema: verify that there are no both counter and non-counter columns 2017-02-07 15:17:14 +00:00
Paweł Dziepak
4b3c0db5cc schema: make is_counter() return correct value 2017-02-02 10:35:14 +00:00
Paweł Dziepak
83f6018ea2 schema: keep counter information in column definition 2017-02-02 10:35:13 +00:00
Calle Wilund
5da92db432 cell_comparator: Better fix (i.e. potentially correct) for compound/clustered desc.
As Tomek pointed out, previous code, regardless of version mismatch, of generating
comparator description string was not correct (as in: in sync with origin).
This modifies it to look at
1.) Actual clustring size
2.) Compound-ness
3.) Dense-ness

to determine whether we should generate a compound desc, and whether it
should contain a trailing utf8-desc type.

v2: Simplify non-dense base column addition and ensure it handles
    thrift non-utf8 (as per comments from tomek)
Message-Id: <1484670171-18362-1-git-send-email-calle@scylladb.com>
2017-01-17 18:03:11 +01:00
Calle Wilund
7d2a4defcf schema: Fix version check for comparator desc string formatting
Fixes #2019

According to the Java driver and cassandra, all versions < 3
include the PK in the comparator descriptor string.

This broke for us when bumping the cassandra version 2.1 -> 2.2

Message-Id: <1484657580-14411-1-git-send-email-calle@scylladb.com>
2017-01-17 14:59:47 +02:00
Duarte Nunes
a9e5b7f124 view_info: Fix comparison
Two view_info object are equal if their fields are equal, not
different.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1482253839-2736-1-git-send-email-duarte@scylladb.com>
2016-12-20 18:36:39 +01:00
Duarte Nunes
d0ed8fa29b schema: Add view_ptr class
The view_ptr class contains a schema_ptr known to represent a
materialized view. It is intended to be used by functions that require
such a schema, and thus obviate the need for the function to check for
schema::is_view().

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00
Duarte Nunes
82ce8eedbd schema: Add view_info field
This patch adds a view_info optional field to the schema. It's
presence indicates the schema represents a materialized view.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00
Duarte Nunes
4b3ac42914 materialized views: Add view_info class
The view_info class is meant to augment a schema with
fields relevant for materialized views.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00
Tomasz Grabiec
f451014785 schema: Implement operator<< for column_mapping
Message-Id: <1481310679-14074-1-git-send-email-tgrabiec@scylladb.com>
2016-12-13 12:20:46 +02:00
Tomasz Grabiec
059a1a4f22 db: Fix commitlog replay to not drop cell mutations with older schema
column_mapping is not safe to access across shards, because data_type
is not safe to access. One of the manifestation of this is that
abstract_type::is_value_compatible_with() always fails if the two
types belong to different shards.

During replay, column_mapping lives on the replaying shard, and is
used by converting_mutation_partition_applier against the schema on
the target shard. Since types in the mapping will be considered
incompatible with types in the schema, all cells will be dropped.

Fix by using column_mapping in a safe way, by copying it to the target
shard if necessary. Each shard maintains its own cache of column
mappings.

Fixes #1924.
Message-Id: <1481310463-13868-1-git-send-email-tgrabiec@scylladb.com>
2016-12-13 12:19:32 +02:00
Duarte Nunes
7c28ed3dfc schema: Extract default compressor
This patch extracts the definition of the default compressor into the
compression_parameters class, so that the table and view creation
statements don't have to explicitly deal with it.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-10-18 01:18:52 +00:00
Duarte Nunes
cb0516a76c schema: Remove compact_column concept
This is a confusing one, and can be replaced the fact that dense
schemas have a single regular column.

Ref #1542

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-08-03 17:21:41 +00:00
Duarte Nunes
529c3a3ae6 column_kind: Drop compact_column
A compact column is a dense schema's single regular column. The fact
that it is a different column_kind has lead to various bugs (#1535,
derived by the schema being dense and the column being regular.

Fixes #1542

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-08-03 17:21:37 +00:00