Commit Graph

225 Commits

Author SHA1 Message Date
Glauber Costa
afee9ab72a system_keyspaces: implement remove_endpoint
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-09 11:58:31 -04:00
Glauber Costa
e559959632 system_keyspace: implement update_peer_info
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-09 11:58:31 -04:00
Glauber Costa
38465dea70 system_keyspaces: implement load_dc_rack_info
To implement that, we will resort to a cache mechanism, instead of doing the
query all the time.  This is mainly because we want to avoid overfuturization
of the callers, that are usually just interested in passing simple strings
around.

We will be able to intercept all updates to it, and maintain consistency with our
internal cache. The updates are not done in this patchset.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-09 11:58:30 -04:00
Glauber Costa
bac0a8a680 legacy_schema_tables: convert save_system_keyspace_schema()
This function is called at bootstrap, to make sure the system tables exist in
the keyspace list. I honestly don't know why do we have to force a delete +
reconstruct. But let's keep consistency with Origin here.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-09 08:12:47 -04:00
Glauber Costa
4641e4166f db/query_context: expose next_timestamp from query_processor
Helper function only

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-09 00:12:08 -04:00
Glauber Costa
d1c8b4fb05 query_context: accessor for storage proxy
Will be useful later to invoke things like mutate_locally

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-09 00:12:08 -04:00
Glauber Costa
cb103449a5 db: move query_context to a header
Do this so we can use it from legacy_schema_tables as well

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-09 00:12:08 -04:00
Glauber Costa
9a691503ff add default_validator to schema_builder
We eventually need to wire them all up. This one is needed for describe
keyspaces to work.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-09 00:12:08 -04:00
Avi Kivity
a2e270f3d4 Merge "wire update tokens" from Glauber
"As previously said, there were some unidentified bugs that prevented update_tokens from
working properly. The first one was sent alongside the series, the second one took me more
time, but it is fixed here."
2015-07-08 16:01:37 +03:00
Calle Wilund
ef2cc9b05d BatchLogManager.java -> C++
Somewhat simplifies version of the Origin code, since from what I 
can see, there is less need for us to do explicit query sends in 
the BLM itself, instead we can just go through storage_proxy. 
I could be wrong though.
2015-07-08 10:59:57 +02:00
Calle Wilund
c1a5627c12 Import BatchlogManager.java 2015-07-08 10:59:57 +02:00
Calle Wilund
4ba0bf7ac6 system_keyspace : add support for storing and reading truncation
* Straight re-impl of origin code
* Uses system tables not yet created, so does not actuall work...
2015-07-08 10:59:57 +02:00
Calle Wilund
0e500b6759 db::serializer : add replay_position serializer 2015-07-08 10:50:46 +02:00
Glauber Costa
12dc7ebd26 system.local: convert update tokens
At this point, users of the interface are futurized already, so we
just need to make sure they call the right function.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-07 20:24:43 -04:00
Avi Kivity
dd29ac9593 Merge "cqlsh" from Glauber
System table Work to make cqlsh connect.
2015-07-07 19:33:23 +03:00
Glauber Costa
137cf22abf system.local: convert host_id functions
They are already futurized, we just have to implement them.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-07 11:38:23 -04:00
Glauber Costa
a9e69a34cb system.local: convert bootstrap state
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-07 11:38:23 -04:00
Glauber Costa
6d58675092 system.local: force blocking flush
Allows a CF to be flushed, and wait for it to finish.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-07 11:38:23 -04:00
Glauber Costa
072de63427 system.local: convert check_health
This function is called at startup and makes sure that the cluster_name field
in system.local exists, and if it exists, that it matches the expected value.

To simplifly things, I am leaving the sstable check out. For us, that would be
a map-reduce operation, and if the sstables are indeed corrupted, we would have
caught that already for sure.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-07 11:38:23 -04:00
Glauber Costa
30008bc0b1 system.local: schema version
This patch provide a function to store the current schema version.
Currently, it is called every time the node boots, with a random schema.
That is incorrect and will be fixed shortly. But for now, cqlsh needs
to see a valid value here, so this will do.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-07 11:38:23 -04:00
Glauber Costa
e5d27d6485 system.local: setup version fields
Done every time the node boots, unconditionally.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-07 11:38:23 -04:00
Glauber Costa
5545e08bf7 database: introduce flush method
We will have to flush it from other places as well, so wrap the flushing code
into a method - specially because the current code has issues and it will be
easier to deal with it if it is in a single place.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-07 11:38:23 -04:00
Glauber Costa
0f68010152 futurize get_local_host_id and set_local_host_id
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-07 11:38:23 -04:00
Glauber Costa
157f5991ac system keyspace: setup queries
We won't create our own versions of database and query_proxy, so we need
some setup to be done. The current code will capture those variables and
setup the structure used to conduct the queries. Later on, it will also
carry on the database's basic setup.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-07 11:38:22 -04:00
Pekka Enberg
a358990855 db/legacy_schema_tables: Pass storage_proxy by reference
We always operate on the local storage proxy so pass it by reference.
This simplifies DEFINITIONS_UPDATE message handler where all we have is
a "this" pointer to the local storage proxy.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-07 16:27:58 +03:00
Pekka Enberg
86d913954a db/legacy_schema_tables: Store CF "is_dense" to system tables
Persist column family's "is_dense" value to system tables. Please note
that we throw an exception if "is_dense" is null upon read. That needs
to be fixed later by inferring the value from other information like
Origin does.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-07 12:36:50 +02:00
Pekka Enberg
3437a49d26 cql3: Enable CF "bloom_filter_fp_chance"
Enable column family "bloom_filter_fp_chance" from the CQL front-end and
make sure its persisted to system tables.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-07 13:09:58 +03:00
Pekka Enberg
c24b7d42ce db/legacy_schema_tables: Store CF key validator in system tables
Store the column family key validator in system tables. Please note that
we derive the validator from CQL partition keys and never actually read
it from the database. This is different from Origin which uses
CompositeType that is both stored and read from the system tables.

Fixes #7.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Tested-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-06 20:33:05 +03:00
Vlad Zolotarov
45ce351f60 db: consistency_level.hh: added is_sufficient_live_nodes()
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-07-05 17:34:56 +03:00
Vlad Zolotarov
501737cb84 db: consistency_level.hh: Complete the implementation of filter_for_query()
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>

New in v2:
   - Use std::partition_copy() and boost::range::algorithm::partition().
   - Don't use std::move() when returning a local vector variable.
2015-07-05 17:34:50 +03:00
Vlad Zolotarov
a9a3bd1927 db: consistency_level.hh: Styling in filter_for_query()
- Make live_endpoints.erase() call more readable.
   - Adjust the comments to our naming.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-07-05 17:15:23 +03:00
Tomasz Grabiec
48d2c96b3c db/serializer: Introduce write() which works with bytes_ostream 2015-07-04 13:46:31 +02:00
Avi Kivity
b0fd850463 Merge "Code missing for a NetworkTopologyStrategy integration" from Vlad
"This series add a code missing for an integration of a NetworkTopologyStrategy
with a current clustering WRITE path."
2015-07-02 17:31:49 +03:00
Vlad Zolotarov
77c50dc013 db: consistency_level.hh: complete assure_sufficient_live_nodes()
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>

New in v2:
   - Use static_cast instead of a dynamic_cast.
2015-07-02 16:00:17 +03:00
Vlad Zolotarov
a4a6c0d69e db: consistency_level.hh: implement is_local()
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-07-02 15:59:40 +03:00
Vlad Zolotarov
ff770a61a5 db: consistency_level.hh: complete block_for() function
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>

New in v2:
   - Use static_cast instead of a dynamic_cast.
2015-07-02 15:58:50 +03:00
Vlad Zolotarov
6b609d5b35 db: consistency_level.hh: implement local_quorum_for()
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>

New in v2:
   - Use static_cast instead of a dynamic_cast.
2015-07-02 15:56:56 +03:00
Tomasz Grabiec
a1f6dec067 result_set: Introduce from_raw_result() factory method 2015-07-02 13:25:46 +02:00
Tomasz Grabiec
c9e5508e3c result_set_builder: Make build() return unwrapped object
It's better to let the user decide which kind (if any) of smart
pointer to wrap it into.
2015-07-02 13:25:46 +02:00
Gleb Natapov
4b9661c608 initial read clustering code
Works only if all replicas (participating in CL) has the same live
data. Does not detects mismatch in tombstones (no infrastructure yet).
Does not report timeout yet.
2015-07-01 13:36:30 +03:00
Vlad Zolotarov
e47e0dcde9 locator: moved endpoint_dc_rack from db::system_keyspace to locator namespace
This is meant to resolve to dependecy loop between token_metadata.hh
and system_keyspace.hh.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-06-29 14:35:47 +03:00
Paweł Dziepak
bad8f7cc72 db/legacy_schema_tables: support compression_parameters
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-06-29 09:36:28 +02:00
Glauber Costa
a0b5c5d13e do not move altered inside capture
If we move altered inside this lambda, the next invocations of the outter
lambda will see a corrupted value. So in the case of smp, invoke_on_all will
invoke correctly in the first shard, but then see a bogus altered vector on
the subsequent ones.

Capturing a reference is fine, because the outer lambda will not be destroyed
until the inner one completes - so the object will live in the capture list.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-06-25 22:00:18 +02:00
Glauber Costa
7fa8071dde fix creation of keyspaces
When we call merge_keyspaces, we end up calling db.add_keyspace, which means we
don't do parts of creation of a keyspace - like creating the directory. This works
in some situations where all the work was already done, but not in others.

We should be calling create_keyspace instead, which will take care of all that
for us. That will include creating a directory when one is needed.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-06-24 12:44:05 -04:00
Tomasz Grabiec
dc6bfb63da system_keyspace: Introduce query()
Simple method to get all data from a system table.

Note: Perhaps we should move storage_proxy::query_local() here as well.
2015-06-24 18:13:02 +03:00
Avi Kivity
3d22623a6b Merge "Flush schema changes to disk" from Glauber
"This is the current patchset to flush and persist schema changes to disk.
It is not perfect, in the sense that older changes still in flight won't be
waited for. But as we discussed - at this moment we'll just note that, and
leave the fix for later"
2015-06-24 17:08:33 +03:00
Gleb Natapov
7d846e842c use write_request_timeout_in_ms for write request timeout
Fixes another fixme. Also change default value to 2000 which seams to
be what origin uses.
2015-06-24 12:51:33 +03:00
Glauber Costa
d6b14c967f db: flush system tables to disk
We flush the mutation we receive, and this is not complete yet.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-06-23 15:23:37 -04:00
Glauber Costa
83d4a47a21 stop using get() / get0() arbitrarily
Being able to just get() a future inside a thread is a very nice functionality,
but we should be more careful a to when we do it. In particular, we are doing this
in a function inside db::legacy_schema_tables, because that function is supposed to
be used - so far - only inside a threaded function.

That approach simplify things, but at a great reusability cost. That means that this
function can now only be reused inside thread code.

This patch turns the offender into a normal future function, and moves the get0() to
code that is directly under seastar::async.

As a rule of thumb, we should stay away from ever doing get / get0 in functions that
are not used directly by async code.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-06-23 21:16:56 +03:00
Tomasz Grabiec
b9288d9fa7 db: Make column_family managed by lw_shared_ptr<>
It will be share-owned by readers.
2015-06-23 13:49:24 +02:00