Commit Graph

79 Commits

Author SHA1 Message Date
Avi Kivity
ebaeefa02b Merge seatar upstream (seastar namespace)
- introcduced "seastarx.hh" header, which does a "using namespace seastar";
 - 'net' namespace conflicts with seastar::net, renamed to 'netw'.
 - 'transport' namespace conflicts with seastar::transport, renamed to
   cql_transport.
 - "logger" global variables now conflict with logger global type, renamed
   to xlogger.
 - other minor changes
2017-05-21 12:26:15 +03:00
Duarte Nunes
03f765c468 modification_statement: Fix detection of clustering keys
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-05-10 19:54:42 +02:00
Duarte Nunes
d7701087af cql3/restrictions/statement_restrictions: Consider statement type
Now that update_statement uses statement_restrictions, we need our
validation logic to take the statement type into account, in
particular to deal with insertion statements which only set static
columns but specify clustering values.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-05-10 19:54:42 +02:00
Duarte Nunes
c2041753c9 cql3/statements/modification_statement: Extract statement_type
This patch extracts the statement_type into its own file. The type
will be later passed to statement_restrictions for validation
purposes.

Further along, we could add methods to it that currently live in other
statements so we can move more validation into statement_restrictions.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-05-10 19:54:42 +02:00
Pekka Enberg
dfee4d2bb0 cql3: Fix partition key bind indices for prepared statements
Fix the CQL front-end to populate the partition key bind index array in
result message prepared metadata, which is needed for CQL binary
protocol v4 to function correctly.

Fixes #2355.

Message-Id: <1494247871-3148-1-git-send-email-penberg@scylladb.com>
2017-05-08 16:33:17 +03:00
Duarte Nunes
9e88b60ef5 mutation: Set cell using clustering_key_prefix
Change the clustering key argument in mutation::set_cell from
exploded_clustering_prefix to clustering_key_prefix, which allows for
some overall code simplification and fewer copies. This mostly affects
the cql3 layer.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-05-04 15:59:50 +02:00
Duarte Nunes
f7bc88734a modification_statement: Use statement_restrictions for where clause
This patch replaces the custom where clause processing by adding and
using a statement_restrictions field to modification_statement.

This improves code reuse and also moves some checks to prepare-time.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-05-04 15:59:49 +02:00
Vlad Zolotarov
ff55b76562 cql3::query_processor: use weak_ptr for passing the prepared statements around
Use seastar::checked_ptr<weak_ptr<pepared_statement>> instead of shared_ptr for passing prepared statements around.
This allows an easy tracking and handling of statements invalidation.

This implementation will throw an exception every time an invalidated
statement reference is dereferenced.

Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
2017-04-12 12:24:03 -04:00
Avi Kivity
27c42359bc Merge seastar upstream
* seastar 6b21197...2ebe842 (6):
  > Merge "Various improvements to execution stages" from Paweł
  > app-template: allow apps to specify a name for help message
  > bool_class: avoid initializing object of incomplete type
  > app-template: make sure we can still get help with required options
  > prometheus: Http handler that returns prometheus 0.4 protobuf or text format
  > Update DPDK to 17.02

Includes patch from Pawel to adjust to updated execution_stage interface.
2017-03-26 10:50:21 +03:00
Paweł Dziepak
dce785089a cql3: make modification statement an execution stage 2017-03-09 09:27:43 +00:00
Calle Wilund
0a4edca756 counters/cql: allow wormholing actual counter values (with shards) via cql
Adds yet another magic function "SCYLLA_COUNTER_SHARD_LIST", indicating that
argument value, which must be a list of tuples <int, UUID, long, long>,
should be inserted as an actual counter value, not update.

This of course to allow counters to be read from sstable loader.

Note that we also need to allow timestamps for counter mutations,
as well as convince the counter code itself to treat the data as
already baked. So ugly wormhole galore.

v2:
* Changed flag names
* More explicit wormholing, bypassing normal counter path, to
  avoid read-before-write etc
* throw exceptions on unhandled shard types in marshalling
v3:
* Added counter id ordering check
* Added batch statement check for mixing normal and raw counter updates
Message-Id: <1487683665-23426-2-git-send-email-calle@scylladb.com>
2017-02-22 09:19:46 +00:00
Paweł Dziepak
1e8814f5ce storage_proxy: support counter updates 2017-02-02 10:35:14 +00:00
Duarte Nunes
65535b3444 modification_statement: Check access for tables with views
This patch checks for additional permissions when modifying a table
with views, since that update will require reading from the table and
writing into its views.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00
Duarte Nunes
5187fdbb3a modification_statement: Views aren't updated directly
This patch ensures that views cannot be modified directly through an
insert or update statement.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-20 13:06:11 +00:00
Asias He
937f28d2f1 Convert to use dht::partition_range_vector and dht::token_range_vector 2016-12-19 14:08:50 +08:00
Asias He
e5485f3ea6 Get rid of query::partition_range
Use dht::partition_range instead
2016-12-19 08:09:25 +08:00
Vlad Zolotarov
6e1d27bed1 cql3::query_processor: add a counter for a number of CQL modification requests ("writes")
- Add a inserts, updates, deletes members to cql_stats.
   - Store cql_stats& in a modification_statement and increment the corresponding counter according to the value of a "type" field.
   - Store cql_stats& in a batch_statement and increment the statistics for each BATCH member.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-11-03 11:49:15 -04:00
Vlad Zolotarov
7606588267 cql3::query_processor: add cql_stats
- Add cql_stats member.
   - Pass it to cql3::raw::parsed_statement::prepare() virtual method.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-11-03 11:48:57 -04:00
Vlad Zolotarov
abae3b05e7 tracing: set table_name in a modification statement
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-08-23 17:58:42 +03:00
Vlad Zolotarov
baa6496816 service::storage_proxy: READ instrumentation: store trace state object in abstract_read_executor
Having a trace_state_ptr in the storage_proxy level is needed to trace code bits in this level.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:59 +03:00
Vlad Zolotarov
4c16df9e4c service: instrument MUTATE flow with tracing
Store the trace state in the abstract_write_response_handler.
Instrument send_mutation RPC to receive an additional
rpc::optional parameter that will contain optional<trace_info>
value.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Tomasz Grabiec
7328a8eff8 cql: modification_statement: Avoid copying keyspace and table names
Message-Id: <1468574135-4701-1-git-send-email-tgrabiec@scylladb.com>
2016-07-15 10:36:53 +01:00
Avi Kivity
10213c4211 cql3: extract raw modification_statement into raw sub-namespace 2016-05-31 20:53:37 +03:00
Avi Kivity
c8f98c5981 cql3: move cf_statement into raw hierarchy
cql3::statements::cf_statement
    -> cql3::statements::raw::cf_statement
Message-Id: <1464609556-3756-3-git-send-email-avi@scylladb.com>
2016-05-31 09:09:21 +03:00
Avi Kivity
caf8d4f0e6 cql3: separate parsed_statement and parsed_statment::prepared
cql3::statements::parsed_statement
    -> cql3::statements::raw::parsed_statement
  cql3::statements::parsed_statement::prepared
    -> cql3::statements::prepared_statement
Message-Id: <1464609556-3756-2-git-send-email-avi@scylladb.com>
2016-05-31 09:09:10 +03:00
Calle Wilund
3906dc9f0d cql3::statements: Change check_access to future<> + implement 2016-04-19 11:49:05 +00:00
Pekka Enberg
38a54df863 Fix pre-ScyllaDB copyright statements
People keep tripping over the old copyrights and copy-pasting them to
new files. Search and replace "Cloudius Systems" with "ScyllaDB".

Message-Id: <1460013664-25966-1-git-send-email-penberg@scylladb.com>
2016-04-08 08:12:47 +03:00
Tomasz Grabiec
c518e852ee modificiation_statement: Use result_view::do_with()
Reduces code duplication.

Message-Id: <1458336592-22065-1-git-send-email-tgrabiec@scylladb.com>
2016-03-20 15:14:28 +02:00
Tomasz Grabiec
63006e5dd2 query: Serialize collection cells using CQL format
We want the format of query results to be eventually defined in the
IDL and be independent of the format we use in memory to represent
collections. This change is a step in this direction.

The change decouples format of collection cells in query results from
our in-memory representation. We currently use collection_mutation_view,
after the change we will use CQL binary protocol format. We use that because
it requires less transformations on the coordinator side.

One complication is that some list operations need to retrieve keys
used in list cells, not only values. To satisfy this need, new query
option was added called "collections_as_maps" which will cause lists
and sets to be reinterpreted as maps matching their underlying
representation. This allows the coordinator to generate mutations
referencing existing items in lists.
2016-02-15 17:05:55 +01:00
Tomasz Grabiec
383296c05b cql3: Fix handling of lists with static columns
List operations and prefetching were not handling static columns
correctly. One issue was that prefetching was attaching static column
data to row data using ids which might overlap with clustered columns.

Another problem was that list operations were always constructing
clustering key even if they worked on a static column. For static
columns the key would be always empty and lookup would fail.

The effect was that list operations which depend on curent state had
no effect. Similar problem could be observed on C* 2.1.9, but not on 2.2.3.

Fixes #903.
2016-02-15 17:05:55 +01:00
Tomasz Grabiec
04eb58159a query: Add schema_version field to read_command 2016-01-11 10:34:51 +01:00
Pekka Enberg
227e517852 cql3: Move modification_statement implementation to source file 2015-12-18 13:29:58 +02:00
Avi Kivity
79f7431a03 db: change collection_mutation::{one,view} not to use nested classes
Nested classes cannot be forward-declared, so change the naming
not to use them.  Follows atomic_cell{,_view}.
2015-11-13 17:13:07 +02:00
Pekka Enberg
1890d276b9 cql3: Add depends_on_{keyspace|column_family} helper to cql_statement
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-15 09:18:52 +03:00
Avi Kivity
d5cf0fb2b1 Add license notices 2015-09-20 10:43:39 +03:00
Paweł Dziepak
39e5f92433 cql3: modification_statement: avoid copying mutations
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-09-03 10:30:18 +02:00
Glauber Costa
4e83530c3f do not "throw new"
This is how Java does. But in C++, "throw new", although valid, would require
the catcher to catch a pointer to the exception - which isn't really what we
do.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-23 07:07:17 +03:00
Tomasz Grabiec
ad99e84505 storage_proxy: Take schema_ptr in query()
It will be needed for reconciliation.
2015-07-12 12:54:38 +02:00
Pekka Enberg
86d913954a db/legacy_schema_tables: Store CF "is_dense" to system tables
Persist column family's "is_dense" value to system tables. Please note
that we throw an exception if "is_dense" is null upon read. That needs
to be fixed later by inferring the value from other information like
Origin does.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-07 12:36:50 +02:00
Calle Wilund
41cbd0d267 Implement modification_statement::execute_internal 2015-07-06 08:21:15 +02:00
Calle Wilund
72ebb6360f modification_statement bugfix: don't move out of shared pointer in loop 2015-07-05 16:04:41 +03:00
Paweł Dziepak
290a7ca1bf query: add timestamp to read_command
Read command needs a timestamp in order to determine which cells have
already expired.

Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-07-02 17:01:19 +02:00
Tomasz Grabiec
3779506990 db: query: Make partition_range hold ring_position
Current model was not really correct because Origin doesn't support
querying of partition ranges by their value. We can query slices
according to dht::decorated_key ordering, which orders partitions
first by token then by key value.

ring_position encapsulates range constraint. Key value is optional, in
which case only token is constrained.
2015-06-18 15:47:40 +02:00
Gleb Natapov
a338407e29 make storage_proxy object distributed
storage_proxy holds per cpu state now to track clustering, so it has to
be distributed otherwise smp setup does not work.
2015-06-17 15:14:06 +02:00
Gleb Natapov
b7155ad862 pass partitions_ranges separately from from read_command
partitions_ranges will be manipulated upon to be split for different
destination, so provide it separately from read_command to not copy the
later for each destination.
2015-06-11 15:18:07 +03:00
Calle Wilund
15b8267dab Add thrift_schema and placeholder "has_compound_comparator()"
thrift_schema == place to collect thrift compatibility aspects of
schema definition.
2015-06-03 10:13:53 +02:00
Calle Wilund
1631ce132e Add "storage_proxy&" argument to cql_statement::validate
To make db, schemas etc reachable
2015-06-03 10:13:52 +02:00
Avi Kivity
750543fc04 cql3: fix shared_ptr misuse in modification_statement
A shared_ptr is mutable, so it must be thread_local, not static.
2015-06-01 17:31:57 +02:00
Tomasz Grabiec
731a63e371 schema: Embed raw_schema inside schema
Public fields got encapsulated.
2015-04-24 18:01:01 +02:00
Avi Kivity
3d38708434 cql3: pass a database& instance to most foo::raw::prepare() variants
To prepare a user-defined type, we need to look up its name in the keyspace.
While we get the keyspace name as an argument to prepare(), it is useless
without the database instance.

Fix the problem by passing a database reference along with the keyspace.
This precolates through the class structure, so most cql3 raw types end up
receiving this treatment.

Origin gets along without it by using a singleton.  We can't do this due
to sharding (we could use a thread-local instance, but that's ugly too).

Hopefully the transition to a visitor will clean this up.
2015-04-20 16:15:34 +03:00