Commit Graph

53948 Commits

Author SHA1 Message Date
Asias He
f1dc4f21d5 streaming: Send stream_init_message to remote 2015-07-09 15:52:28 +08:00
Asias He
3256a21556 streaming: Use frozen_mutation to send mutations
Each outgoing_file_message might contain multiple mutations. Send them
one mutation per RPC call (using frozen_mutation), instead of one big
outgoing_file_message per one RPC call.
2015-07-09 15:52:28 +08:00
Asias He
cdf50aa078 messaging_service: Add STREAM_MUTATION verb
It is used by streaming service to send frozen_mutation.
2015-07-09 15:52:28 +08:00
Asias He
1dd80bac65 streaming: Make get_local_db() and ms() public 2015-07-09 15:52:28 +08:00
Asias He
1aab3c7bb9 streaming: Drop serialization interface for {outgoing,incoming}_file_message
We will send mutations using the frozen_mutation verb
2015-07-09 15:52:28 +08:00
Asias He
4718211d4a streaming: Wire up stream_transfer_task::add_transfer_file
Wire up with outgoing_file_message
2015-07-09 15:52:27 +08:00
Asias He
ad3692f666 streaming: Implement stream_session::add_transfer_ranges
Given keyspace names, ranges and column_families names, figure out
mutation_readers to transfer.
2015-07-09 15:52:27 +08:00
Asias He
3d42a9f9ad streaming: Introduce stream_detail
It is used to hold mutations which we will send over network.
2015-07-09 15:52:27 +08:00
Asias He
4b676bfac5 streaming: Init streaming service in main
We need a reference to <distributed> db.
2015-07-09 15:52:27 +08:00
Asias He
c41872a64d messaging_service: Move serialization code frozen_mutation to source file 2015-07-09 15:52:27 +08:00
Asias He
a773c151a3 messaging_service: Move serialization code for sstring to source file 2015-07-09 15:52:27 +08:00
Asias He
ff5e80bd35 messaging_service: Move serialization code for messaging_verb to source file
It is *HUGE* pain to modify one line of the code in messaging_verb.hh and
wait like forever until urchin compiles.
2015-07-09 15:52:27 +08:00
Pekka Enberg
31248ee0cf cql3: Enable DROP TABLE statement grammar
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-09 09:36:39 +03:00
Pekka Enberg
b68f93191e cql3: Convert DropTableStatement to C++
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-09 09:36:39 +03:00
Pekka Enberg
a64eab772c migration_manager: Convert announceColumnFamilyDrop to C++ as stub
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-09 09:22:02 +03:00
Glauber Costa
4641e4166f db/query_context: expose next_timestamp from query_processor
Helper function only

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-09 00:12:08 -04:00
Glauber Costa
d1c8b4fb05 query_context: accessor for storage proxy
Will be useful later to invoke things like mutate_locally

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-09 00:12:08 -04:00
Glauber Costa
cb103449a5 db: move query_context to a header
Do this so we can use it from legacy_schema_tables as well

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-09 00:12:08 -04:00
Glauber Costa
cdeab4a3d3 cql3: provide a mechanism to get a timestamp higher than any
It is common for some operations, like system table updates, to try and guarantee
some particular ordering of operations.

The way Origin does it, is by simply incrementing one to the current timestamp.
Our calls, however, are being dispatched through our internal query processor, which
has a builtin client_state.

Our client_state has a mechanism to guarantee monotonicity, by adding 1 if needed
to operations that happen subsequentially. By using a clock that is not wired up
to this mechanism, we can't really guarantee that if other operations happened to
get in between.

If we expose this mechanism through the query_processor, we will be able to guarantee
that.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-09 00:12:08 -04:00
Glauber Costa
9a691503ff add default_validator to schema_builder
We eventually need to wire them all up. This one is needed for describe
keyspaces to work.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-09 00:12:08 -04:00
Glauber Costa
0cb43b42d0 query_options: use native_protocol instead of 3 for internal options as well.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-09 00:12:08 -04:00
Pekka Enberg
69c56997e2 cql3: Enable DROP KEYSPACE statement grammar
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-08 18:40:30 +02:00
Pekka Enberg
816254ed45 cql3: Convert DropKeyspaceStatement to C++
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-08 18:40:30 +02:00
Pekka Enberg
386792e63f migration_manager: Convert announceKeyspaceDrop to C++ as stub
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-08 18:40:30 +02:00
Pekka Enberg
d3eb9f9b57 version.hh: Fix release version to be "2.2.0"
Fix release version to be compatible with origin to avoid confusing
clients.

Before:

  [penberg@nero apache-cassandra-2.1.7]$ ./bin/cqlsh --no-color 127.0.0.1
  Connected to Test Cluster at 127.0.0.1:9042.
  [cqlsh 5.0.1 | Cassandra SeastarDB v0.1 | CQL spec 3.2.0 | Native protocol v3]
  Use HELP for help.
  cqlsh>

After:

  [penberg@nero apache-cassandra-2.1.7]$ ./bin/cqlsh --no-color 127.0.0.1
  Connected to Test Cluster at 127.0.0.1:9042.
  [cqlsh 5.0.1 | Cassandra 2.2.0 | CQL spec 3.2.0 | Native protocol v3]
  Use HELP for help.
  cqlsh>

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-08 19:21:14 +03:00
Gleb Natapov
b0c087b2a5 storage_proxy: fix check for singular range on read
A singular range which a ring_position that does not contains a key may
operate on more than one key and thus is not really singular.
2015-07-08 18:38:39 +03:00
Asias He
ee699eb0ac mutation_reader: Take vector instead of initializer_list 2015-07-08 17:07:48 +03:00
Avi Kivity
a2e270f3d4 Merge "wire update tokens" from Glauber
"As previously said, there were some unidentified bugs that prevented update_tokens from
working properly. The first one was sent alongside the series, the second one took me more
time, but it is fixed here."
2015-07-08 16:01:37 +03:00
Avi Kivity
b9ae77a5a9 Merge branch 'master' of github.com:cloudius-systems/urchin into db 2015-07-08 15:18:16 +03:00
Avi Kivity
79dcdfd90f Merge "BatchLogMana" from Calle
"(Partial?) implementation of BatchLogManager.
Requires the token function/restriction series.

Functional as in that it can create batchlog mutations, and do replay
of data in this system table.
Since range queries does not yet work, it only handles a very small
table contents.

It is not used yet either, but will eventually be needed for batch statements
etc."
2015-07-08 14:48:33 +03:00
Pekka Enberg
3ca9a8c2fb migration_manager: Fix announce_new_column_family() validation
Throw an exception if a column family already exists.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-08 13:03:28 +02:00
Avi Kivity
a209c91662 Merge "Fix empty sstable_compression property" from Paweł
"This patches prevent compression validation code from rejecting statements
with an empty sstable_compression property. Correct behaviour in such cases
is not to use compression at all.

This would have been a 10 minuts fix if antlr3 didn't prepare surprise of its
own. Empty STRING_LITERALS weren't handled properly and a workaround for that
problem was introduced."
2015-07-08 13:14:12 +03:00
Calle Wilund
6cf2230cb6 Add batchlog_manager_test
Very simple, and limited, but at least checks that very basic
replay works.
2015-07-08 10:59:57 +02:00
Calle Wilund
ef2cc9b05d BatchLogManager.java -> C++
Somewhat simplifies version of the Origin code, since from what I 
can see, there is less need for us to do explicit query sends in 
the BLM itself, instead we can just go through storage_proxy. 
I could be wrong though.
2015-07-08 10:59:57 +02:00
Calle Wilund
c1a5627c12 Import BatchlogManager.java 2015-07-08 10:59:57 +02:00
Calle Wilund
4ba0bf7ac6 system_keyspace : add support for storing and reading truncation
* Straight re-impl of origin code
* Uses system tables not yet created, so does not actuall work...
2015-07-08 10:59:57 +02:00
Pekka Enberg
918b30b2d8 transport/server: Fix "already exists" error serialization
The serialization for "already exists" error is special as explained in
the CQL binary protocol specification:

  0x2400    Already_exists: The query attempted to create a keyspace or a
            table that was already existing. The rest of the ERROR message
            body will be <ks><table> where:
              <ks> is a [string] representing either the keyspace that
                   already exists, or the keyspace in which the table that
                   already exists is.
              <table> is a [string] representing the name of the table that
                      already exists. If the query was attempting to create a
                      keyspace, <table> will be present but will be the empty
                      string.

Fix that to unconfuse "cqlsh" when attemting to create a duplicate
keyspace or table.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-08 11:57:48 +03:00
Calle Wilund
0e500b6759 db::serializer : add replay_position serializer 2015-07-08 10:50:46 +02:00
Calle Wilund
8a8694cbbf Add naive rate limiter object
This is mostly a placeholder, since the "limiting" is rather coarse
and stuttering
2015-07-08 10:50:46 +02:00
Calle Wilund
806fc04b37 token_metadata.hh : implement get_all_endpoints
+ add number_of_endpoints()
2015-07-08 10:50:46 +02:00
Paweł Dziepak
b8c3d8de4b tests/cql: add test for empty sstable_compression property
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-07-08 10:47:50 +02:00
Paweł Dziepak
148d6b9db2 compress: allow empty sstable_compression
Fixes #13.

Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-07-08 10:47:11 +02:00
Paweł Dziepak
8405361e81 Cql.g: support empty STRING_LITERALS
setText() is meant to override the token value. However, when antlr3 is
asked to return the token value and needs to determine whether setText()
was called it checks if the string set by setText() is empty or not.
This basically means that it is impossible to override token value with
an empty string.

This problem is solved by using a string with a single byte set to -1 to
represent empty string.

Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-07-08 10:38:33 +02:00
Avi Kivity
957b4076a2 Merge "CQL: Handle token function & restrictions" from Calle
"Implementation of:

* Token func
* Token restriction
* Token relation
* Token cql parsing

This series contains some hefty refactoring of the cql3::restrictions
interfaces, to handle the slightly different conditions of the token
relation. (See individual commit comment)"
2015-07-08 11:35:22 +03:00
Calle Wilund
84b5ba5270 query_processor bug fix: execute_internal did not keep options alive
... across the whole call chain. Should fix test failure.
2015-07-08 11:03:57 +03:00
Calle Wilund
94fcce501e Cql3.g: handle token in grammar 2015-07-08 10:02:31 +02:00
Calle Wilund
bb6677b47a TokenRelation: Java -> c++ 2015-07-08 10:02:31 +02:00
Calle Wilund
d7c10ff3b6 cql3::relation - fix constness 2015-07-08 10:02:31 +02:00
Calle Wilund
73421cb413 TokenRestriction: Java->c++ 2015-07-08 10:02:31 +02:00
Calle Wilund
4a5a5359b0 cql3::restrictions - refactor and clean up
Needed to reasonably cleanly implement token restrictions. 

* Fixed constness for various virtuals. 
* primary_key_restrictions now inherit abstract_restriction,
  similuar to Origin (for better or for worse), to avoid 
  duplicating attributes etc. 
* primary_key_restrictions bounds & values renamed (so not to
  collide with restriction), and some logic pushed downwards 
  (building bounds), to avoid abstraction breakage in 
  statement_restrictions
* primary_key_restrictions merging is now potentially replacing
  to make dispatching token/multicolumn restrictions simpler
2015-07-08 10:02:31 +02:00