Commit Graph

283 Commits

Author SHA1 Message Date
Tomasz Grabiec
5f44afa311 cql3: batch_statement: Execute statements sequentially
Currently we execute all statements in parallel, but some statements
depend on order, in particular list append/prepend. Fix by executing
sequentially.

Fixes cql_additional_tests.py:TestCQL.batch_and_list_test dtest.

Fixes #1075.

Message-Id: <1458672874-4749-1-git-send-email-tgrabiec@scylladb.com>
2016-03-22 20:59:40 +02:00
Tomasz Grabiec
c518e852ee modificiation_statement: Use result_view::do_with()
Reduces code duplication.

Message-Id: <1458336592-22065-1-git-send-email-tgrabiec@scylladb.com>
2016-03-20 15:14:28 +02:00
Tomasz Grabiec
b42d3a90b3 cql3: create_table_statement: Sort _defined_names by text
Currently they are sorted by address in memory, which breaks the
check for column name duplicates, which assumes sorting by text.

Fixes #975.

Message-Id: <1456937400-20475-1-git-send-email-tgrabiec@scylladb.com>
2016-03-02 18:53:43 +02:00
Pekka Enberg
3a6d43c784 cql3: Fix duplicate column definition check
We cannot use shared_ptr *instances* for checking duplicate column
definitions because they are never equal. Store column definition name
in the unordered_map instead.

Fixes cql_additional_tests.py:TestCQL.identifier_test.

Spotted by Shlomi.

Message-Id: <1456840506-13941-1-git-send-email-penberg@scylladb.com>
2016-03-01 16:46:33 +02:00
Paweł Dziepak
92f9c9428e cql3: don't insert row marker if schema is_cql3_table()
Checking schema::is_dense() is not enough to know whether row marker
should be inserted or not as there may be compact storage tables that
are not considered dense (namely, a table with now clustering key).

Row marker should only be insterted if schema::is_cql3_table() is true.

Fixes #931.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
Message-Id: <1456834937-1630-1-git-send-email-pdziepak@scylladb.com>
2016-03-01 13:29:53 +01:00
Tomasz Grabiec
1ecf9a7427 query: result_view: Introduce do_with()
Encapsulates linearization. Abstracts away the fact that result_view
can't work with discontiguous storage yet.
2016-02-26 12:26:13 +01:00
Pekka Enberg
8e2c924de3 cql3: Fix quadratic behavior in update_statement::parsed_insert::prepare_internal()
This fixes a quadratic search for duplicate columns in prepare_internal().

Refs #822.

Message-Id: <1456405104-16482-1-git-send-email-penberg@scylladb.com>
2016-02-25 15:06:56 +02:00
Pekka Enberg
dfcc48d82a transport: Add result metadata to PREPARED message
The gocql driver assumes that there's a result metadata section in the
PREPARED message. Technically, Scylla is not at fault here as the CQL
specification explicitly states in Section 4.2.5.4. ("Prepared") that the
section may be empty:

   - <result_metadata> is defined exactly as <metadata> but correspond to the
      metadata for the resultSet that execute this query will yield. Note that
      <result_metadata> may be empty (have the No_metadata flag and 0 columns, See
      section 4.2.5.2) and will be for any query that is not a Select. There is
      in fact never a guarantee that this will non-empty so client should protect
      themselves accordingly. The presence of this information is an

However, Cassandra always populates the section so lets do that as well.

Fixes #912.

Message-Id: <1456317082-31688-1-git-send-email-penberg@scylladb.com>
2016-02-24 14:43:24 +02:00
Pekka Enberg
4ff1692248 cql3: Make 'CREATE TYPE' error message human readable
We don't support the 'CREATE TYPE' statement for now. The user-visible
error message, however, is unreadable because our CQL parser doesn't
even recognize the statement.

  cqlsh:ks1> CREATE TYPE config (url text);
  SyntaxException: <ErrorMessage code=2000 [Syntax error in CQL query] message=" : cannot match to any predicted input...

Implement just enough of 'CREATE TYPE' parsing to be able to report a
human readable error message if someone tries to execute such
statements:

  cqlsh:ks1> CREATE TYPE config (url text);
  ServerError: <ErrorMessage code=0000 [Server error] message="User-defined types are not supported yet">
Message-Id: <1456148719-9473-2-git-send-email-penberg@scylladb.com>
2016-02-22 14:50:25 +01:00
Tomasz Grabiec
d376167bd4 cql: create_table_statement: Optimize duplicate column names detection
Current algorithm is O(N^2) where N is the column count. This causes
limits.py:TestLimits.max_columns_and_query_parameters_test to timeout
because CREATE TABLE statement takes too long.

This change replaces it with an algorithm of O(N)
complexity. _defined_names are already sorted so if any duplicates
exist, they must be next to each other.

Message-Id: <1456058447-5080-1-git-send-email-tgrabiec@scylladb.com>
2016-02-21 14:55:03 +02:00
Tomasz Grabiec
09dc79f245 cql3: select_statement: Set desired serialization format 2016-02-15 17:05:55 +01:00
Tomasz Grabiec
63006e5dd2 query: Serialize collection cells using CQL format
We want the format of query results to be eventually defined in the
IDL and be independent of the format we use in memory to represent
collections. This change is a step in this direction.

The change decouples format of collection cells in query results from
our in-memory representation. We currently use collection_mutation_view,
after the change we will use CQL binary protocol format. We use that because
it requires less transformations on the coordinator side.

One complication is that some list operations need to retrieve keys
used in list cells, not only values. To satisfy this need, new query
option was added called "collections_as_maps" which will cause lists
and sets to be reinterpreted as maps matching their underlying
representation. This allows the coordinator to generate mutations
referencing existing items in lists.
2016-02-15 17:05:55 +01:00
Tomasz Grabiec
383296c05b cql3: Fix handling of lists with static columns
List operations and prefetching were not handling static columns
correctly. One issue was that prefetching was attaching static column
data to row data using ids which might overlap with clustered columns.

Another problem was that list operations were always constructing
clustering key even if they worked on a static column. For static
columns the key would be always empty and lookup would fail.

The effect was that list operations which depend on curent state had
no effect. Similar problem could be observed on C* 2.1.9, but not on 2.2.3.

Fixes #903.
2016-02-15 17:05:55 +01:00
Tomasz Grabiec
9d11968ad8 Rename serialization_format to cql_serialization_format 2016-02-15 16:53:56 +01:00
Erich Keane
4197ceeedb raw_statement::is_reversed rewrite to avoid VLA
The is_reversed function uses a variable length array, which isn't
spec-abiding C++.  Additionally, the Clang compiler doesn't allow them
with non-POD types, so this function wouldn't compile.

After reading through the function it seems that the array wasn't
necessary as the check could be calculated inline rather than
separately.  This version should be more performant (since it no longer
requires the VLA lookup performance hit) while taking up less memory in
all but the smallest of edge-cases (when the clustering_key_size *
sizeof(optional<bool>) < sizeof(size_type) - sizeof(uint32_t) +
sizeof(bool).

This patch uses  relation_order_unsupported it assure that the exception
order is consistent with the preivous version.  The throw would
otherwise be moved into the initial for-loop.

There are two derrivations in behavior:
The first is the initial assert.  It however should not change the apparent
behavior besides causing orderings() to be looked up 2x in debug
situations.

The second is the conversion of is_reversed_ from an optional to a bool.
The result is that the final return value is now well-defined to be
false in the release-condition where orderings().size() == 0, rather
than be the ill-defined *is_reversed_ that was there previously.

Signed-off-by: Erich Keane <erich.keane@verizon.net>
Message-Id: <1454546285-16076-4-git-send-email-erich.keane@verizon.net>
2016-02-07 10:38:17 +02:00
Tomasz Grabiec
b8c3fa4d46 cql3: Print only column name in error message
Printing column_definition prints all fields of the struct, we want
only name here.
Message-Id: <1453207531-16589-1-git-send-email-tgrabiec@scylladb.com>
2016-01-19 20:22:37 +02:00
Gleb Natapov
f97eed0c94 fix batch size checking
warn_threshold is in kbytes v.size is in bytes size is in kbytes.

Message-Id: <20160118090620.GZ6705@scylladb.com>
2016-01-18 11:08:13 +02:00
Paweł Dziepak
00f7a873a5 cql3: forbid re-adding collection with incompatible type
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-01-18 08:35:38 +01:00
Calle Wilund
e935c9cd34 select_statement: Make sure all aggregate queries use paging
Mainly to make sure we respect row limits. Since normal result
generation does not for aggregates.

Fixes #752 

Message-Id: <1452681048-30171-2-git-send-email-calle@scylladb.com>
2016-01-14 19:03:37 +02:00
Tomasz Grabiec
054f1df0a5 cql3: Disable ALTER TABLE unless experimental features are on 2016-01-14 13:21:13 +01:00
Calle Wilund
6a5f075107 batch_statement: Modify verify_batch_size to match current origin
Fixes #614

* Use warning threshold from config
* Don't throw exceptions. We're only supposed to warn.
* Try to actually estimate mutation data payload size, not
  number of mutations.
Message-Id: <1452615759-23213-1-git-send-email-calle@scylladb.com>
2016-01-13 12:26:49 +01:00
Paweł Dziepak
0276919819 cql3: complete translation of alter table statement
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-01-11 10:34:54 +01:00
Tomasz Grabiec
d8ff9ee441 schema_tables: Make merge_tables() compare by mutations
Schema version is calculated from mutations, so merge_schema should
also look at mutation changes to detect schema changes whenever
version changes.
2016-01-11 10:34:53 +01:00
Tomasz Grabiec
04eb58159a query: Add schema_version field to read_command 2016-01-11 10:34:51 +01:00
Tomasz Grabiec
dae531554a create_index_statement: Use textual column name in all messages
As pointed out by Pawel, we can rely on operator<<()
Message-Id: <1452243656-3376-1-git-send-email-tgrabiec@scylladb.com>
2016-01-08 11:06:09 +02:00
Tomasz Grabiec
5d6d039297 create_index_statement: Use textual representation of column name
Before:

  InvalidRequest: code=2200 [Invalid query] message="No column definition found for column 736368656d615f76657273696f6e"

After:

  InvalidRequest: code=2200 [Invalid query] message="No column definition found for column schema_version"
Message-Id: <1452243156-2923-1-git-send-email-tgrabiec@scylladb.com>
2016-01-08 10:53:37 +02:00
Paweł Dziepak
3ca4e27dba cql3: convert alter_table_statement to c++
Everything except alter_table_statement::announce_migration() is
translated. announce_migration() has to wait for multi schema support to
be merged.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-01-05 09:49:04 +01:00
Paweł Dziepak
35edda76c4 cql3: import AlterTableStatement.java
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-01-05 09:49:04 +01:00
Paweł Dziepak
b615bfa47e cql3: add cf_prop_defs::get_default_time_to_live()
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-01-05 09:49:04 +01:00
Pekka Enberg
b754de8f4a cql3: Move delete_statement implementation to source file 2015-12-18 13:29:58 +02:00
Pekka Enberg
227e517852 cql3: Move modification_statement implementation to source file 2015-12-18 13:29:58 +02:00
Pekka Enberg
ca963d470e cql3: Move parsed_statement implementation to source file 2015-12-18 13:07:55 +02:00
Pekka Enberg
ff994cfd39 cql3: Move property_definitions implementation to source file 2015-12-18 13:04:32 +02:00
Pekka Enberg
d7db5e91b6 cql3: Move select_statement implementation to source file 2015-12-18 12:59:22 +02:00
Pekka Enberg
8b780e3958 cql3: Move update_statement implementation to source file 2015-12-18 12:54:19 +02:00
Pekka Enberg
e56bf8933f Improve not implemented errors
Print out the function name where we're throwing the exception from to
make it easier to debug such exceptions.
2015-12-18 10:51:37 +01:00
Avi Kivity
79f7431a03 db: change collection_mutation::{one,view} not to use nested classes
Nested classes cannot be forward-declared, so change the naming
not to use them.  Follows atomic_cell{,_view}.
2015-11-13 17:13:07 +02:00
Calle Wilund
fdc549cd47 select_statement: Handle aggregate queries
Fixes #549.

Being clinically absent-minded, aggregate query support (i.e. count(...))
was left out of the "paging" change set.

This adds repeated paged querying to do aggregate queries (similar to
origin). Uses "batched" paging.
2015-11-11 18:41:47 +01:00
Calle Wilund
ecd7674867 select_statement: Paging
Check if paging might be needed, and if so, use a paging object.
Similar to origin, but without all the filters.
2015-11-10 13:16:06 +01:00
Calle Wilund
4a1a17defc cql3::selection: Move result set building visitor to result_set_builder
Allows its use (and partial override - hint hint) in more place than
one.
2015-11-10 13:12:33 +01:00
Avi Kivity
2c3591cbd9 data_value de-any-fication
We use boost::any to convert to and from database values (stored in
serlialized form) and native C++ values.  boost::any captures information
about the data type (how to copy/move/delete etc.) and stores it inside
the boost::any instance.  We later retrieve the real value using
boost::any_cast.

However, data_value (which has a boost::any member) already has type
information as a data_type instance.  By teaching data_type intances about
the corresponding native type, we can elimiante the use of boost::any.

While boost::any is evil and eliminating it improves efficiency somewhat,
the real goal is growing native type support in data_type.  We will use that
later to store native types in the cache, enabling O(log n) access to
collections, O(1) access to tuples, and more efficient large blob support.
2015-10-30 17:38:51 +01:00
Pekka Enberg
e48f8d25e2 cql3: Move cf_statement class implementation to source file
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-23 16:17:17 +03:00
Pekka Enberg
d79d8b00f3 cql3: Move create_keyspace_statement class implementation to source file
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-23 16:17:17 +03:00
Pekka Enberg
baea79d17d cql3: Move schema_altering_statement class implementation to source file
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-23 16:17:17 +03:00
Pekka Enberg
6dbee22d2e cql3: Move use_statement class implementation to source file
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-23 16:17:17 +03:00
Pekka Enberg
ff93a840ae cql3: Remove ifdef'd code from use_statement.hh
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-23 15:52:02 +03:00
Pekka Enberg
83e6c428be transport: Add messages_fwd.hh and use it
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-23 15:52:02 +03:00
Pekka Enberg
1890d276b9 cql3: Add depends_on_{keyspace|column_family} helper to cql_statement
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-15 09:18:52 +03:00
Pekka Enberg
921dc9ab9f cql3: Add keyspace() and column_family() to select_statement
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-14 09:12:40 +03:00
Pekka Enberg
b66154e43a cql3: Fix capture-by-reference in drop_keyspace_statement
We need to capture the "is_local_only" boolean by value because it's an
argument to the function. Fixes an annoying bug where we failed to update
schema version because we pass "true" accidentally.

Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-06 14:53:35 +03:00