Commit Graph

4628 Commits

Author SHA1 Message Date
Tomasz Grabiec
ec65d7fa73 Introduce partition_slice_builder 2015-07-09 19:55:01 +02:00
Tomasz Grabiec
31081f91c4 tests: Fix row_assertion::matches() to deal with empty boost::any properly 2015-07-09 19:55:01 +02:00
Tomasz Grabiec
197196c6b0 tests: Introduce result_set_assertions::has_size() 2015-07-09 19:55:00 +02:00
Tomasz Grabiec
9aec8b9591 query: result_set: Use type's to_string() to get prettier printout 2015-07-09 19:55:00 +02:00
Tomasz Grabiec
cb9b48d9be query: result_set: Add static columns even if there are no clustered rows 2015-07-09 19:55:00 +02:00
Tomasz Grabiec
5af1769326 tests: Add more CQL query tests 2015-07-09 19:55:00 +02:00
Tomasz Grabiec
9724b84bb3 db: Fix query of partitions with no live clustered rows
When partition has no live regular rows, but has some data live in the
static row, then it should appear in the results, even though we
didn't select any static column.

To reproduce:

  create table cf (k blob, c blob, v blob, s1 blob static, primary key (k, c));
  update cf set s1 = 0x01 where k = 0x01;
  update cf set s1 = 0x02 where k = 0x02;
  select k from cf;

The "select" statement should return 2 rows, but was returning 0.

The following query worked fine, because static columns were included:

  select * from cf;

The data query should contain only live data, so we shouldn't write a
partition entry if it's supposed to be absent from the results. We
can'r tell that though until we've processed all the data. To solve
this problem, query result writer is using an optimistic approach,
where the partition header will be retracted from the buffer
(cheaply), if it turns out there's no live data in it.
2015-07-09 19:55:00 +02:00
Tomasz Grabiec
c742529e52 bytes_ostream: Introduce retract()
Useful for optimistic writers, where it's easier/cheaper to retract
later than to calculate the decision up front.
2015-07-09 19:55:00 +02:00
Tomasz Grabiec
09ed972068 mutation_partition: Remove redundant slice parameter from query()
The slice used by partition_writer must match the one used by query()
anyway.
2015-07-09 19:47:32 +02:00
Tomasz Grabiec
c975e7fb56 db: atomic_cell: Fix misnamed parameter 2015-07-09 19:46:30 +02:00
Tomasz Grabiec
33778231d9 db: atomic_cell: Introduce is_covered_by() helper 2015-07-09 19:46:29 +02:00
Tomasz Grabiec
da937897cf memtable: Introduce as_data_source() 2015-07-09 19:46:29 +02:00
Tomasz Grabiec
8a18d2b699 Extract memtable implementation to memtable.cc 2015-07-09 19:46:29 +02:00
Tomasz Grabiec
176fa9d049 storage_proxy: Simplify query_local() 2015-07-09 18:53:03 +02:00
Tomasz Grabiec
c3f7856853 bytes_ostream: Introduce fragment iterator 2015-07-09 18:53:03 +02:00
Tomasz Grabiec
a03fc3549b query-result: Add missing include 2015-07-09 18:53:03 +02:00
Avi Kivity
50236b69b3 Merge "Fix floating point comparison" from Paweł
"This change makes comparing floating point values behave exactly like
it does in case of Java compareTo() function, which is different than
the usual IEEE 754 behaviour."
2015-07-09 19:09:18 +03:00
Paweł Dziepak
92740919eb tests/type: add test for comparing floating point values
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-07-09 17:52:45 +02:00
Paweł Dziepak
cf07b4c09a types: make comparing floating point values follow Java behavior
In Java method compareTo() of Float and Double types doesn't strictly
follow IEEE 754. Firstly, NaN is equal NaN and is greater than any other
value. Secondly, positive zero is larger than negative zero.

Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-07-09 17:52:45 +02:00
Avi Kivity
28652c669f Merge "UTF8 validation in CQL binary protocol" from Paweł
"These patches add verification that strings passed using CQL binary protocol
are valid utf8. Exception hierarchy is also adjusted to make sure that
protocol exceptions are properly translated to an appropriate error code.

This makes DTEST cql_tests.py:TestCQL.invalid_string_literals_test pass."

Reviewed-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-09 17:44:09 +03:00
Tomasz Grabiec
06917666f9 Merge tag 'bootstrap-v3' from git@github.com:glommer/urchin.git
Among other things, saving of system keyspace schema, from Glauber.
2015-07-09 15:31:27 +03:00
Glauber Costa
bac0a8a680 legacy_schema_tables: convert save_system_keyspace_schema()
This function is called at bootstrap, to make sure the system tables exist in
the keyspace list. I honestly don't know why do we have to force a delete +
reconstruct. But let's keep consistency with Origin here.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-09 08:12:47 -04:00
Paweł Dziepak
4273c6d390 transport: verify that strings are valid utf8
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-07-09 13:41:35 +02:00
Paweł Dziepak
ac776d2618 exceptions: remove transport_exception
transport_exception is an interface implemented by both
cassandra_exception and protocol_exception. The logic implemented in
both these subclasses is identical.

This patch removes transport_exception and makes protocl_exception a
subclass of cassandra_exception.

Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-07-09 13:39:31 +02:00
Tomasz Grabiec
31b0c60640 Merge branch 'penberg/cql-drop-table' from seastar-dev.git
From Pekka:

"This series enables DROP TABLE statement in the front-end. Our
back-end is not really prepared to deal with it so throw "not
implemented" exception from migration manager."
2015-07-09 10:53:48 +02:00
Avi Kivity
8857f42c5f Merge "Wire up streaming and ssstable" from Asias 2015-07-09 11:37:22 +03:00
Tomasz Grabiec
724099abe9 query: result_set_builder: Use the set of columns which was queried for
This builder is the one used to build the convenient result_set (not
on fast path). The builder was assuming that the whole set of columns
was always queried, which resulted in buffer underflow exceptions
during parsing of the results if this was not the case. Let's also
handle queries which have narrower column sets.
2015-07-09 11:35:31 +03:00
Asias He
b7b0aa3318 streaming: Negotiate core to core connection.
In streaming code, we need core to core connection(the second connection
from B to A). That is when node A initiates a stream to node B, it is
possible that node A will transfer data to node B and vice verse, so we
need two connections. When node A creates a tcp connection (within the
messaging_service) to node B, we have a connection ip_a:core_a to
ip_b:core_b. When node B creates a connection to node B, we can not
guarantee it is ip_b:core_b to ip_a:core_a.

Current messaging_service does not support core to core connection yet,
although we use shard_id{ip, cpu_id} as the destination of the message.

We can solve the issue in upper layer. We can pass extra cpu_id as a
user msg.

Node A sends stream_init_message with my_cpu_id = current_cpu_id

Node B receives stream_init_message, it runs on whatever cpu this
connection goes to, then it sends response back with Node B's
current_cpu_id.

After this, each node knows which cpu_id to send to each other.

TODO: we need to handle the case when peer node reboots with different
number of cpus.
2015-07-09 15:52:28 +08:00
Asias He
853175fc61 streaming: Implement prepare_message handler
This is a bit different from Origin. We always send back a
prepare_message even if the initializer requested no data from the
follower, to unify the handling.
2015-07-09 15:52:28 +08:00
Asias He
abfcf7d825 streaming: Send prepare_message message 2015-07-09 15:52:28 +08:00
Asias He
7ae860a43a streaming: Enable init_receiving_side 2015-07-09 15:52:28 +08:00
Asias He
f1dc4f21d5 streaming: Send stream_init_message to remote 2015-07-09 15:52:28 +08:00
Asias He
3256a21556 streaming: Use frozen_mutation to send mutations
Each outgoing_file_message might contain multiple mutations. Send them
one mutation per RPC call (using frozen_mutation), instead of one big
outgoing_file_message per one RPC call.
2015-07-09 15:52:28 +08:00
Asias He
cdf50aa078 messaging_service: Add STREAM_MUTATION verb
It is used by streaming service to send frozen_mutation.
2015-07-09 15:52:28 +08:00
Asias He
1dd80bac65 streaming: Make get_local_db() and ms() public 2015-07-09 15:52:28 +08:00
Asias He
1aab3c7bb9 streaming: Drop serialization interface for {outgoing,incoming}_file_message
We will send mutations using the frozen_mutation verb
2015-07-09 15:52:28 +08:00
Asias He
4718211d4a streaming: Wire up stream_transfer_task::add_transfer_file
Wire up with outgoing_file_message
2015-07-09 15:52:27 +08:00
Asias He
ad3692f666 streaming: Implement stream_session::add_transfer_ranges
Given keyspace names, ranges and column_families names, figure out
mutation_readers to transfer.
2015-07-09 15:52:27 +08:00
Asias He
3d42a9f9ad streaming: Introduce stream_detail
It is used to hold mutations which we will send over network.
2015-07-09 15:52:27 +08:00
Asias He
4b676bfac5 streaming: Init streaming service in main
We need a reference to <distributed> db.
2015-07-09 15:52:27 +08:00
Asias He
c41872a64d messaging_service: Move serialization code frozen_mutation to source file 2015-07-09 15:52:27 +08:00
Asias He
a773c151a3 messaging_service: Move serialization code for sstring to source file 2015-07-09 15:52:27 +08:00
Asias He
ff5e80bd35 messaging_service: Move serialization code for messaging_verb to source file
It is *HUGE* pain to modify one line of the code in messaging_verb.hh and
wait like forever until urchin compiles.
2015-07-09 15:52:27 +08:00
Pekka Enberg
31248ee0cf cql3: Enable DROP TABLE statement grammar
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-09 09:36:39 +03:00
Pekka Enberg
b68f93191e cql3: Convert DropTableStatement to C++
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-09 09:36:39 +03:00
Pekka Enberg
a64eab772c migration_manager: Convert announceColumnFamilyDrop to C++ as stub
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-09 09:22:02 +03:00
Glauber Costa
4641e4166f db/query_context: expose next_timestamp from query_processor
Helper function only

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-09 00:12:08 -04:00
Glauber Costa
d1c8b4fb05 query_context: accessor for storage proxy
Will be useful later to invoke things like mutate_locally

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-09 00:12:08 -04:00
Glauber Costa
cb103449a5 db: move query_context to a header
Do this so we can use it from legacy_schema_tables as well

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-09 00:12:08 -04:00
Glauber Costa
cdeab4a3d3 cql3: provide a mechanism to get a timestamp higher than any
It is common for some operations, like system table updates, to try and guarantee
some particular ordering of operations.

The way Origin does it, is by simply incrementing one to the current timestamp.
Our calls, however, are being dispatched through our internal query processor, which
has a builtin client_state.

Our client_state has a mechanism to guarantee monotonicity, by adding 1 if needed
to operations that happen subsequentially. By using a clock that is not wired up
to this mechanism, we can't really guarantee that if other operations happened to
get in between.

If we expose this mechanism through the query_processor, we will be able to guarantee
that.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-09 00:12:08 -04:00