Commit Graph

413 Commits

Author SHA1 Message Date
Pekka Enberg
82394debe6 cql3/statements: Multiple index targets for CREATE INDEX 2017-05-04 14:59:12 +03:00
Pekka Enberg
fe315bd31a cql3/statements: Use index_metadata in create_index_statement class 2017-05-04 14:59:12 +03:00
Pekka Enberg
651af0f45a cql3/statements: Use feature flag in create_index_statement class 2017-05-04 14:59:12 +03:00
Pekka Enberg
4391faaf45 cql3/statements: Add constants to index_target 2017-05-04 14:59:11 +03:00
Pekka Enberg
546d1e47dd cql3/statements: Add as_cql_string() to index_target class 2017-05-04 14:59:11 +03:00
Pekka Enberg
58b90655d2 cql3/statements: Add to_string(target_type) 2017-05-04 14:59:11 +03:00
Pekka Enberg
1f5a52d03f cql3/statements: Use namespaces in index_target.cc file 2017-05-04 14:59:11 +03:00
Avi Kivity
7d16cfa5f0 Merge branch 'penberg/create-index-stmt-cleanup/v1' of github.com:cloudius-systems/seastar-dev
"The version of create_index_statement class that was translated to C++
is pretty old by now. This series of cleanups brings it closer to Apache
Cassandra trunk to make it easier to bring over more secondary index
code to Scylla."

* 'penberg/create-index-stmt-cleanup/v1' of github.com:cloudius-systems/seastar-dev:
  cql3/statements/create_index_statement: Move target validation
  cql3/statements/create_index_statement: Remove static column validation
  cql3/statements/create_index_statement: Extract validations
  cql3/statements/create_index_statement: Kill bogus custom validation
  cql3/statements/create_index_statement: Add materialized view to validate()
  cql3/statements/create_index_statement: Remove validation
2017-04-13 13:27:53 +03:00
Pekka Enberg
8c45038729 cql3/statements/create_index_statement: Move target validation
Move index target validation to preserve the same code structure as
Apache Cassandra and simplify support for multiple index targets.
2017-04-12 20:50:09 +03:00
Pekka Enberg
528c33a05b cql3/statements/create_index_statement: Remove static column validation
Apache Cassandra supports secondary indices on static columns since
commit 9e74891 ("Add support for secondary indexes on static columns").
2017-04-12 20:46:29 +03:00
Pekka Enberg
975e1c8fc6 cql3/statements/create_index_statement: Extract validations
Extract specific validations to separate functions to preserve the same
structure as Apache Cassandra code and make it easier to add support for
multiple index targets.
2017-04-12 20:44:15 +03:00
Pekka Enberg
940d6de1b8 cql3/statements/create_index_statement: Kill bogus custom validation
Rejecting custom indices is bogus because it's just a configuration
mechanism like replication strategy, for example. Furthermore, it's
needed for SASI indices, which we likely need to be compatible with.
2017-04-12 20:17:09 +03:00
Pekka Enberg
cfadb70565 cql3/statements/create_index_statement: Add materialized view to validate()
Apache Cassandra does not support secondary indices on materialized
views so neither should we.
2017-04-12 20:15:20 +03:00
Pekka Enberg
1706f346f2 cql3/statements/create_index_statement: Remove validation
The validation was removed in Apacha Cassandra commit 0626be8 ("New 2i
API and implementations for built in indexes"). Let's also remove it
from our code so that we remove one dependency to the obsolete db/index/
code.
2017-04-12 20:14:52 +03:00
Vlad Zolotarov
75fbc7c558 cql3::statements::batch_statement: add a constructor that doesn't receive the "bound_terms" value
This constructor should be used when we know that there are no bound terms in the current
batch statement.

Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
2017-04-12 12:24:08 -04:00
Vlad Zolotarov
ff55b76562 cql3::query_processor: use weak_ptr for passing the prepared statements around
Use seastar::checked_ptr<weak_ptr<pepared_statement>> instead of shared_ptr for passing prepared statements around.
This allows an easy tracking and handling of statements invalidation.

This implementation will throw an exception every time an invalidated
statement reference is dereferenced.

Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
2017-04-12 12:24:03 -04:00
Pekka Enberg
ecb8ee4efd cql3/statements: Cleanup create_index_statement.cc
Use namespaces and fix formatting issues to make the source file easier
on the eyes.

Message-Id: <1491977964-26629-1-git-send-email-penberg@scylladb.com>
2017-04-12 16:48:45 +03:00
Avi Kivity
27c42359bc Merge seastar upstream
* seastar 6b21197...2ebe842 (6):
  > Merge "Various improvements to execution stages" from Paweł
  > app-template: allow apps to specify a name for help message
  > bool_class: avoid initializing object of incomplete type
  > app-template: make sure we can still get help with required options
  > prometheus: Http handler that returns prometheus 0.4 protobuf or text format
  > Update DPDK to 17.02

Includes patch from Pawel to adjust to updated execution_stage interface.
2017-03-26 10:50:21 +03:00
Duarte Nunes
e215f25b11 migration_manager: Atomically migrate table and views
This patch changes the migration path for table updates such that the
base table mutations are sent and applied atomically with the view
schema mutations.

This ensures that after schema merging, we have a consistent mapping
of base table versions to view table versions, which will be used in
later patches.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-03-15 16:03:56 +01:00
Duarte Nunes
bfb8a3c172 materialized views: Replace db::view::view class
The write path uses a base schema at a particular version, and we
want it to use the materialized views at the corresponding version.

To achieve this, we need to map the state currently in db::view::view
to a particular schema version, which this patch does by introducing
the view_info class to hold the state previously in db::view::view,
and by having a view schema directly point to it.

The changes in the patch are thus:

1) Introduce view_info to hold the extra view state;
2) Point to the view_info from the schema;
3) Make the functions in the now stateless db::view::view non-member;
4) Remove the db::view::view class.

All changes are structural and don't affect current behavior.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-03-15 15:50:05 +01:00
Gleb Natapov
d34f3a0440 batchlog: introduce batch_size_fail_threshold_in_kb option
Add batch_size_fail_threshold_in_kb to prevent huge batch from been
applied and causing troubles. Also do not warn or fail if only one
partition is affected.

Fixes: #2128

Message-Id: <20170309111247.GE8197@scylladb.com>
2017-03-09 12:20:17 +01:00
Paweł Dziepak
827357cb08 cql3: make batch statement an execution stage 2017-03-09 09:27:43 +00:00
Paweł Dziepak
dce785089a cql3: make modification statement an execution stage 2017-03-09 09:27:43 +00:00
Paweł Dziepak
d005b20071 cql3: make select statement an execution stage 2017-03-09 09:27:43 +00:00
Calle Wilund
0a4edca756 counters/cql: allow wormholing actual counter values (with shards) via cql
Adds yet another magic function "SCYLLA_COUNTER_SHARD_LIST", indicating that
argument value, which must be a list of tuples <int, UUID, long, long>,
should be inserted as an actual counter value, not update.

This of course to allow counters to be read from sstable loader.

Note that we also need to allow timestamps for counter mutations,
as well as convince the counter code itself to treat the data as
already baked. So ugly wormhole galore.

v2:
* Changed flag names
* More explicit wormholing, bypassing normal counter path, to
  avoid read-before-write etc
* throw exceptions on unhandled shard types in marshalling
v3:
* Added counter id ordering check
* Added batch statement check for mixing normal and raw counter updates
Message-Id: <1487683665-23426-2-git-send-email-calle@scylladb.com>
2017-02-22 09:19:46 +00:00
Paweł Dziepak
fa29ef3cc0 cql3: allow counters in CREATE TABLE statements 2017-02-02 10:35:14 +00:00
Paweł Dziepak
1e8814f5ce storage_proxy: support counter updates 2017-02-02 10:35:14 +00:00
Pekka Enberg
2bd560118e cql3/statements/select_statement: Unset value support 2017-01-27 09:24:36 +02:00
Pekka Enberg
be0351b49c cql3: Introduce raw_value and raw_value_view types
Currently, the code is using bytes_opt and bytes_view_opt to represent
CQL values, which can hold a value or null. In preparation for
supporting a third state, unset value introduced in CQL v4, introduce
new raw_value and raw_value_view types and use them instead.

The new types are based on boost::variant<> and are capable of holding
null, unset values, and blobs that represent a value.
2017-01-26 13:50:04 +02:00
Avi Kivity
a9403877e4 cql3: add more metrics for batch statements
- how many statements are in a batch
 - different types of batches
 - whether we were able to convert a logged batch to an unlogged batch
2017-01-20 13:19:00 +02:00
Avi Kivity
e3c003544d cql3: optimize batch_statement when the same partition is mutated multiple times
Batch statements are often used to insert multiple rows into the same
partition.  Recognize this case and merge mutations to the same partition.

If the result is a single mutation, there is an additional win (already
present in the code), where a logged batch can be converted into an unlogged
batch.

Ref #1689.
2017-01-20 13:18:56 +02:00
Piotr Jastrzebski
4bbe05dd47 mutation_partition: take schema in find_row and clustered_row
This will allow intrusive set implementation that does not
store schema.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
2017-01-05 11:26:03 +01:00
Pekka Enberg
302035577e cql3/statements: Make batch_statement::_type private
The _type member variable is never accessed outside of the
batch_statement class so make it private.
Message-Id: <1482921073-28485-1-git-send-email-penberg@scylladb.com>
2016-12-28 12:08:05 +01:00
Pekka Enberg
20daf43403 cql3/statements: Move batch_statement implementation to source file
Clean up batch_statement class by moving implementation to the
batch_statement.cc source file to make it easier to modify the class.

Message-Id: <1482920872-28303-1-git-send-email-penberg@scylladb.com>
2016-12-28 12:30:03 +02:00
Avi Kivity
1d9ee358f1 Revert "Merge "Reduce the size of mutation_partition" from Piotr"
This reverts commit aa392810ff, reversing
changes made to a24ff47c637e6a5fd158099b8a65f1191fc2d023; it uses
boost::intrusive::detail directly, which it must not, and doesn't compile on
all boost versions as a consequence.
2016-12-25 16:07:48 +02:00
Piotr Jastrzebski
2af6ff68d9 mutation_partition: take schema in find_row and clustered_row
This will allow intrusive set implementation that does not
store schema.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
2016-12-23 11:29:07 +01:00
Tomasz Grabiec
bc6486b304 Use gc_clock instead of db_clock where possible
Some code paths were obtaining db_clock timestamp to only convert it
to gc_clock later. Avoid this. In the future we could make gc_clock
cheaper cause it has low precision.

Message-Id: <1482401190-2035-1-git-send-email-tgrabiec@scylladb.com>
2016-12-22 13:27:55 +02:00
Duarte Nunes
124802e196 cql3: Add function to build view's select statement
This patch adds an utility function that creates a raw select
statement from a set of columns and a where clause. It is intended to
be used to create the prepared select statement used by the view
class.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00
Duarte Nunes
088dfdb108 select_statement: Consider materialized views
This patch considers materialized views in
select_statement::check_access().

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00
Duarte Nunes
5511dab914 cql3: Add drop view statement
This patch adds the drop_view_statement, which enables users to drop a
given materialized view.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00
Duarte Nunes
3025ea63fc cql3: Add alter view statement
This patch adds the alter_view_statement, which enables users to
change the properties of a materialized view.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00
Duarte Nunes
8792fed651 create_view_statement: Complete implementation
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00
Duarte Nunes
02bc0d2ab3 create_view_statement: Require MV feature
This patch adds the MATERIALIZED_VIEWS_FEATURE to the set of cluster
features and requires its presence to allow creating a view. This
ensures view schemas can be safely propagated across nodes.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00
Duarte Nunes
59682c95a1 create_view_statement: Require experimental switch
Creating a materialized view requires running Scylla with the
experimental switch.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00
Duarte Nunes
c626c983f4 create_view_statement: Reuse validation code
This replace some validation logic with a call to
validation::validate_column_family.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00
Duarte Nunes
5bd74abee8 create_view_statement: Implement check_access
This patch implements check_access according to Cassandra's
implementation.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00
Duarte Nunes
a9c17b0a52 select_statement: Propagate for_view argument
This patch propagates the for_view argument, used by
statement_restrictions to ensure IS NOT NULL can be used when creating
a materialized view.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00
Duarte Nunes
65535b3444 modification_statement: Check access for tables with views
This patch checks for additional permissions when modifying a table
with views, since that update will require reading from the table and
writing into its views.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00
Duarte Nunes
5187fdbb3a modification_statement: Views aren't updated directly
This patch ensures that views cannot be modified directly through an
insert or update statement.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00
Duarte Nunes
21e34c5054 alter_type_statement: Consider materialized views
This patch ensures we also update materialized views where the type
being updated occurs.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00