Commit Graph

177 Commits

Author SHA1 Message Date
Piotr Sarna
b0ab4c28cf schema: add column_definition::is_hidden_from_cql
Right now the only columns hidden from CQL are view virtual columns,
but in case of expanding this set, a helper function is provided.
2019-02-27 15:07:54 +01:00
Duarte Nunes
fa2b0384d2 Replace std::experimental types with C++17 std version.
Replace stdx::optional and stdx::string_view with the C++ std
counterparts.

Some instances of boost::variant were also replaced with std::variant,
namely those that called seastar::visit.

Scylla now requires GCC 8 to compile.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20190108111141.5369-1-duarte@scylladb.com>
2019-01-08 13:16:36 +02:00
Tomasz Grabiec
789fac9884 schema: Optimize column count getters 2018-11-21 14:04:27 +01:00
Avi Kivity
775b7e41f4 Update seastar submodule
* seastar d59fcef...b924495 (2):
  > build: Fix protobuf generation rules
  > Merge "Restructure files" from Jesse

Includes fixup patch from Jesse:

"
Update Seastar `#include`s to reflect restructure

All Seastar header files are now prefixed with "seastar" and the
configure script reflects the new locations of files.

Signed-off-by: Jesse Haber-Kucharsky <jhaberku@scylladb.com>
Message-Id: <5d22d964a7735696fb6bb7606ed88f35dde31413.1542731639.git.jhaberku@scylladb.com>
"
2018-11-21 00:01:44 +02:00
Avi Kivity
a71ab365e3 toplevel: convert sprint() to format()
sprint() recently became more strict, throwing on sprint("%s", 5). Replace
with the more modern format().

Mechanically converted with https://github.com/avikivity/unsprint.
2018-11-01 13:16:17 +00:00
Vladimir Krivopalov
399f815a89 schema: Add helper method returning the count of columns of specified kind.
Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
2018-10-25 17:07:20 -07:00
Nadav Har'El
0a1d93138d schema: add "view virtual" flag to schema's column_definition
In this patch we add a flag, "view virtual", that we can mark on on a
column defined in a schema. In following patches, we will add such virtual
columns to materialized views to allow view rows to remain alive despite
having no data (refs #3362).

After this patch, the "view virtual" flag exists in our in-memory
representation of the schema, but not persisted to disk - we will
fix this in the next patch.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
2018-08-16 15:23:09 +03:00
Paweł Dziepak
6c54a97320 schema: column_mapping_entry: cache abstract_type::is_atomic()
IDL deserialisation code calls is_atomic() for each cell. An additional
indirection and a virtual call can be avoided by caching that value in
column_mapping_entry. There is already very similar optimisation done
for column_definitions.
2018-06-28 22:16:42 +01:00
Piotr Sarna
bc019205b3 schema: fix typos in a comment
Message-Id: <2b2a169e8a511fa9e0e1556ac7559ce9bef896e1.1525431353.git.sarna@scylladb.com>
2018-05-04 15:26:51 +01:00
Calle Wilund
ff41f47a08 db::extensions: Allow extensions to modify (system) schemas
Allows extensions/config listeners to potentially augument
(system) schemas at boot time. This is only useful for schemas
who do not pass through system_schema tables.
2018-03-26 11:58:28 +00:00
Calle Wilund
3ab760b375 schema: Add opaque type to represent extensions
A virtual opaque object meant to represent the "extensions" mapping
in schema_tables::tables/views
2018-02-07 10:11:45 +00:00
Duarte Nunes
fbb4c9edda schema: Provide all-selecting partition slice
This patch introduces schema::full_slice(), which returns a
partition_slice selecting the full clustering range, as well as all
static and regular columns. No options aside from the default are
set in that partition_slice.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1507732800-9448-1-git-send-email-duarte@scylladb.com>
2017-10-17 11:25:35 +02:00
Paweł Dziepak
7b0f75c0d1 sstables: avoid indirect calls to abstract_type::is_multi_cell() 2017-07-26 14:38:27 +01:00
Duarte Nunes
7eecda3a61 schema: Support compaction enabled attribute
Fixes #2547

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20170721132206.3037-1-duarte@scylladb.com>
2017-07-21 15:38:45 +02:00
Tomasz Grabiec
a9237c1666 schema: Revert back to the 1.7 layout of static compact tables in memory
We are using C* 3.x compatible layout in schema tables but want to
keep using the 1.7 layout in memory for compatibility during rolling
upgrade. This patch switches the schema and schema_builder classes
back to the old layout. Translation of layout happens when converting
to/from schema mutations.

Notable changes:

 1) Includes a revert of commit 6260f31e08
    "thrift: Update CQL mapping of static CFs".

 2) Brings back the "default_validation_class" schema attribute. In v3
    it can be dervied from column definitions, but in v2 it can't, so
    we have to store it.

 3) legacy_schema_migrator and schema_builder don't have to do
    conversions to v3, this is now handled by the v3_columns
    class. schema_builder works with the same layout as schema, that
    is v2.

 4) Includes a revert of commit 66991a7ccb
    "v3 schema test fixes"

Fixes #2555.
2017-07-19 09:52:15 +02:00
Tomasz Grabiec
dc2dc056a4 schema: Use v3 column layout when converting to/from schema mutations 2017-07-19 09:52:15 +02:00
Tomasz Grabiec
dc463ef644 schema: Encapsulate column layout translations in the v3_columns class 2017-07-19 09:52:15 +02:00
Duarte Nunes
6b464da67d schema: Get rid of regular_columns_by_name
They are unused.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20170717103635.6473-2-duarte@scylladb.com>
2017-07-17 12:52:41 +02:00
Tomasz Grabiec
34dae0588c schema: Use proper name comparator
This replaces column_definition::name_comparator, which incorrectly
assumes that names are always utf8, with name_compare moved from
schema::rebuild() and unifies usages.
2017-07-17 09:40:06 +02:00
Tomasz Grabiec
b919c50d21 schema: Use proper column name type for static columns
After f5dae826ce, static columns not
always have utf8 column names. For static compact tables it's
determined by the cell name comparator type, which is equal to the
type of the synthetic clustering column.

Caused various errors with static thrift tables with non-utf8
comparator.
2017-07-17 09:40:06 +02:00
Tomasz Grabiec
84536a4a75 schema: Introduce clustering_column_at() 2017-07-17 09:40:06 +02:00
Duarte Nunes
4886b7ed5e schema: Lift maybe_quote() into cql3/util
It's a more natural place given its current and future usages.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-06-15 19:55:52 +00:00
Calle Wilund
6c8b5fc09d schema_tables: Use v3 schema tables and formats
Switches system/schema_* for system_schema/*, updates schema/schema
builder and uses to hold/expect v3 style info (i.e. types & dropped).
2017-05-10 16:44:48 +00:00
Calle Wilund
0e6ae8dec2 schema: rename column accessors to be in line with origin
More pointedly: Expose columns as is (currently
all_columns_in_select_order), expose name->column mapping more
appropriately named. 

Renaming like this is not strictly neccesary, but there is a point to
trying to keep nomenclature similar-ish with origin, esp. when select
order column need to become filtered (spoiler alert).
2017-05-10 16:44:48 +00:00
Calle Wilund
d2dc7898aa schema: Add "is_static_compact_table" 2017-05-10 16:44:48 +00:00
Duarte Nunes
5387ac98f8 schema: Add utility functions for static columns
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-05-10 16:44:48 +00:00
Calle Wilund
f5fcadf0b1 schema: Add "as_cql_string" for column_def + quote-wrapper 2017-05-09 13:48:55 +00:00
Pekka Enberg
35bb6dedd8 schema: Kill index_type enum 2017-05-08 10:19:34 +03:00
Pekka Enberg
06564afedb schema: Kill index_info class
It's no longer used. Indices are managed by the index_metadata class.
2017-05-08 10:19:34 +03:00
Pekka Enberg
25e2777344 schema: Add all_indices() to schema class 2017-05-04 14:59:12 +03:00
Pekka Enberg
ef29520c8e schema: Add get_default_index_name() to index_metadata class 2017-05-04 14:59:11 +03:00
Pekka Enberg
5e8f2f49c3 schema: Add indices() to schema class 2017-05-04 14:59:11 +03:00
Pekka Enberg
5abd4b8041 schema: Add has_index() to schema class 2017-05-04 14:59:11 +03:00
Pekka Enberg
1fb1828aa2 schema: Add index_names() to schema class 2017-05-04 14:59:11 +03:00
Pekka Enberg
1c1a767408 schema: Add find_index_noname() to schema class
This adds a find_index_nomame() helper to the schema class, which
searches for index that is otherwise equal but ignores the name of the
index in comparison. This is needed to for CREATE INDEX to reject
duplicate index creation.
2017-05-04 14:59:11 +03:00
Pekka Enberg
05e12a1d2b schema: Add index_metadata maps to raw_schema class 2017-05-04 13:22:12 +03:00
Pekka Enberg
2f83232a02 schema: Add index_metadata class 2017-05-02 10:29:18 +03:00
Duarte Nunes
bfb8a3c172 materialized views: Replace db::view::view class
The write path uses a base schema at a particular version, and we
want it to use the materialized views at the corresponding version.

To achieve this, we need to map the state currently in db::view::view
to a particular schema version, which this patch does by introducing
the view_info class to hold the state previously in db::view::view,
and by having a view schema directly point to it.

The changes in the patch are thus:

1) Introduce view_info to hold the extra view state;
2) Point to the view_info from the schema;
3) Make the functions in the now stateless db::view::view non-member;
4) Remove the db::view::view class.

All changes are structural and don't affect current behavior.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-03-15 15:50:05 +01:00
Duarte Nunes
a64c47f315 schema: Move raw_view_info outside of raw_schema
In preparation of an upcoming patch, where the schema
won't directly store the raw_view_info.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-03-15 15:38:31 +01:00
Duarte Nunes
4b209be8b8 view_info: Rename to raw_view_info
In preparation for upcoming patches, which will deal with
moving the state in db::view::view to view_info.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-03-15 15:38:31 +01:00
Paweł Dziepak
4b3c0db5cc schema: make is_counter() return correct value 2017-02-02 10:35:14 +00:00
Paweł Dziepak
83f6018ea2 schema: keep counter information in column definition 2017-02-02 10:35:13 +00:00
Duarte Nunes
76276f1a53 alter_table_statement: Update materialized view
This patch ensures that changes to a base table's schema
are reflected in that table's materialized views.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00
Duarte Nunes
d0ed8fa29b schema: Add view_ptr class
The view_ptr class contains a schema_ptr known to represent a
materialized view. It is intended to be used by functions that require
such a schema, and thus obviate the need for the function to check for
schema::is_view().

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00
Duarte Nunes
82ce8eedbd schema: Add view_info field
This patch adds a view_info optional field to the schema. It's
presence indicates the schema represents a materialized view.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00
Duarte Nunes
4b3ac42914 materialized views: Add view_info class
The view_info class is meant to augment a schema with
fields relevant for materialized views.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00
Tomasz Grabiec
f451014785 schema: Implement operator<< for column_mapping
Message-Id: <1481310679-14074-1-git-send-email-tgrabiec@scylladb.com>
2016-12-13 12:20:46 +02:00
Tomasz Grabiec
059a1a4f22 db: Fix commitlog replay to not drop cell mutations with older schema
column_mapping is not safe to access across shards, because data_type
is not safe to access. One of the manifestation of this is that
abstract_type::is_value_compatible_with() always fails if the two
types belong to different shards.

During replay, column_mapping lives on the replaying shard, and is
used by converting_mutation_partition_applier against the schema on
the target shard. Since types in the mapping will be considered
incompatible with types in the schema, all cells will be dropped.

Fix by using column_mapping in a safe way, by copying it to the target
shard if necessary. Each shard maintains its own cache of column
mappings.

Fixes #1924.
Message-Id: <1481310463-13868-1-git-send-email-tgrabiec@scylladb.com>
2016-12-13 12:19:32 +02:00
Duarte Nunes
7c28ed3dfc schema: Extract default compressor
This patch extracts the definition of the default compressor into the
compression_parameters class, so that the table and view creation
statements don't have to explicitly deal with it.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-10-18 01:18:52 +00:00
Duarte Nunes
cb0516a76c schema: Remove compact_column concept
This is a confusing one, and can be replaced the fact that dense
schemas have a single regular column.

Ref #1542

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-08-03 17:21:41 +00:00