Commit Graph

401 Commits

Author SHA1 Message Date
Pekka Enberg
da7b741f64 service/migration_manager: Implement announce_keyspace_drop()
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-10-06 14:53:35 +03:00
Tomasz Grabiec
bc1d159c1b Merge branch 'penberg/cql-drop-table/v3' from seastar-dev.git
From Pekka:

This patch series implements support for CQL DROP TABLE. It uses the newly
added truncate infrastructure under the hood. After this series, the
test_table CQL test in dtest passes:

  [penberg@nero urchin-dtest]$ nosetests -v cql_tests.py:TestCQL.table_test
  table_test (cql_tests.TestCQL) ... ok

  ----------------------------------------------------------------------
  Ran 1 test in 23.841s

  OK
2015-10-06 13:39:25 +02:00
Pekka Enberg
042e9252d5 service/migration_manager: Implement announce_column_family_drop()
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-10-06 11:28:55 +03:00
Glauber Costa
651937becf Revert "pass db::config to storage service as well"
This reverts commit c2b981cd82.
2015-10-05 13:21:33 +02:00
Glauber Costa
639ba2b99d incremental backups: move control to the CF level
Currently, we control incremental backups behavior from the storage service.
This creates some very concrete problems, since the storage service is not
always available and initialized.

The solution is to move it to the column family (and to the keyspace so we can
properly propagate the conf file value). When we change this from the api, we will
have to iterate over all of them, changing the value accordingly.

Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-10-05 13:16:11 +02:00
Glauber Costa
b619d244e8 storage_service: public access to the database object
Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-10-05 13:15:27 +02:00
Glauber Costa
a5fb145084 storage_service: incremental backups
Query and set the state of incremental backups. The initial value comes from
the configuration file through the local db reference. Later on, it can be
changed through the interface.

Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-10-02 18:23:27 +02:00
Glauber Costa
c2b981cd82 pass db::config to storage service as well
We would like to access configuration, but don't want to poke other services
in order to do so.

Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-10-02 18:23:26 +02:00
Gleb Natapov
2998b891f3 storage_proxy: fix crash during background read repair
Lazy digest calculation code introduced a bug in background read repair.
The problem is that digest_read_resolver::resolve() destroys one data
result (it is moved to a caller to be sent as a reply), so during
background digest match there is no value to calculate a digest from.
Copying data to the caller would be most elegant solution, but also
slowest one, so lets just treat the case where there is only one
target queried and skip digest calculation in this case since we know
digest_match() will do nothing.
2015-09-30 16:35:12 +03:00
Calle Wilund
d0864be20f storage_proxy: Implement "truncate_blocking" 2015-09-30 09:09:43 +02:00
Calle Wilund
80ade2e2d3 storage_proxy: Add TRUNCATE verb handler 2015-09-30 09:09:43 +02:00
Avi Kivity
d5cf0fb2b1 Add license notices 2015-09-20 10:43:39 +03:00
Avi Kivity
987294a412 Add missing copyrights 2015-09-20 10:16:11 +03:00
Asias He
2d99476bb1 storage_service: Fix schedule_schema_pull
It might block for a very long time. Don't wait for it otherwise it will
block the whole gossip round.
2015-09-17 09:13:20 +03:00
Gleb Natapov
ab5f52fde3 storage_proxy: lazily calculate digest from data results during query
Do not calculate digest from data on arrival, do it during digest
matching check, also skip it entirely if there is only one digest
to match.
2015-09-16 17:40:22 +03:00
Gleb Natapov
5f76cacc90 storage_proxy: handle some more exceptions during write 2015-09-16 14:08:11 +02:00
Gleb Natapov
396570b002 storage_proxy: fix digest_request_resolve completion reporting
Digest resolver is broken in a way that prevents read completion to
be reported if data arrives after enough digests for cl were already
received. This happens because the code tried to save on a state and
used _cl_responses as an indicator that completion was reported already,
but this is incorrect since there can be enough responses for cl, but no
data yet. Fix by introducing special state to track completion reporting.

Fixes #331
2015-09-10 15:46:15 +03:00
Gleb Natapov
bfaa771b87 storage_proxy: measure latency for failed writes too
This was the case before 0149a22f69
2015-09-10 15:45:21 +03:00
Paweł Dziepak
6a0d4e3ade client_state: verify that keyspace exist
Fixes #323.

Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-09-10 13:58:48 +03:00
Gleb Natapov
b96ad54cbf do not leak write response handler in logged batch.
If exception is thrown during creation of a write response handler those
that are already created need to be removed explicitly.
2015-09-09 14:46:22 +03:00
Gleb Natapov
e08160b1ec storage_proxy: do not report disconnects as errors
Connection drop during read operation is not an error and should not be
reported as such. Furthermore disconnects are already reported by
gossip, so no need to report it for each ongoing read again.

Fixes #320
2015-09-09 12:37:52 +03:00
Avi Kivity
e2d474ea56 Merge "Enabling the nodetool info" from Amnon
"This series enable the nodetool info, by completing the missing APIs.

The main change is returning fixed value for storage_service
is_rpc_server_running, is_native_transport_running and get_exception_count.

After this series it will be possible to run:
nodetool info (while the jmx is runnning) and to get the results without errors
or crashes."
2015-09-09 11:40:06 +03:00
Gleb Natapov
df468504b6 schema_table: convert code to use distributed<storage_proxy> instead of storage_proxy&
All database code was converted to is when storage_proxy was made
distributed, but then new code was written to use storage_proxy& again.
Passing distributed<> object is safer since it can be passed between
shards safely. There was a patch to fix one such case yesterday, I found
one more while converting.
2015-09-09 10:19:30 +03:00
Amnon Heiman
ea5fa7e469 storage_service: return values for exception, native_transport and
rpc_server

This patch changes the behaviour of is_native_transport_running and
is_rpc_server_running to return true and not to fail, we assume that
they are running. It should be changed when an API to start and stop
them will be added.

The get_exception_count will return 0, the definition for it in origin
is exception that were not cought in a thread.

We should re-think about what it means in our implementation, meanwhile
return 0, for no exception, is a reasonable approach.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-09-09 09:37:18 +03:00
Gleb Natapov
031f6e1aeb storage_proxy: do not capture storage_proxy reference in rpc callback
Callback may be called on different cpus so shared pointer cannot be
captured.
2015-09-08 09:55:23 +02:00
Gleb Natapov
a223c33278 storage_proxy: remove unused variable 2015-09-08 09:53:32 +02:00
Asias He
89f2959536 gossip: Rework stop() and shutdown()
Consolidate stop() and shutdown() into one function.

Fix crash:

scylla: urchin/seastar/core/future.hh:315: void
future_state<>::set(): Assertion `_u.st == state::future' failed.

=== stop gossip
$ curl -X DELETE --header "Accept: application/json"
"http://127.0.0.1:10000/storage_service/gossiping"

=== start gossip
$ curl -X POST --header "Content-Type: application/json" --header
"Accept: application/json"
"http://127.0.0.1:10000/storage_service/gossiping"
2015-09-08 12:20:53 +08:00
Asias He
312daed342 storage_service: Fix is_starting API
Query _operation_mode on CPU 0.

$ curl -X GET --header "Accept: application/json"
"http://127.0.0.1:10000/storage_service/is_starting"
2015-09-08 11:07:13 +08:00
Asias He
5e3d8a56b2 storage_service: Fix get_operation_mode API
Route request to CPU 0. _operation_mode is not replicated to other CPUS.

Without this:

$ curl -X GET --header "Accept: application/json"
"http://127.0.0.1:10000/storage_service/operation_mode"

returns "NORMAL" and "STARTING" randomly.
2015-09-08 10:55:50 +08:00
Asias He
0d88570286 storage_service: Fix is_gossip_running API and friends
Only cpu 0 instance of gossip has the correct information, route request
to cpu 0.

Fix a bug where

$ curl -X GET --header "Accept: application/json"
 "http://172.31.5.77:10000/storage_service/gossiping"

returns true and false randomly.
2015-09-08 10:45:25 +08:00
Gleb Natapov
0149a22f69 storage_proxy: use parallel_for_each in mutate() instead of semaphore
If several mutation in a batch throw exceptions have_cl.broken() will be
called more then once. Fix this by dropping ad hoc have_cl and use
parallel_for_each() that does the same thing that current code is doing.

Fixes #297
2015-09-07 19:29:34 +03:00
Gleb Natapov
327e27b67b storage_proxy: stop query timeout timer when all replies are received
Fixes #285
2015-09-07 15:21:51 +02:00
Gleb Natapov
41f16159b3 storage_proxy: track reference to storage_proxy during mutate/query operations
This patch makes sure that storage_proxy cannot be deleted while
mutate/query operation is in progress.
2015-09-07 14:46:13 +02:00
Gleb Natapov
b884aba147 storage_proxy: drop superfluous captures 2015-09-07 14:46:13 +02:00
Gleb Natapov
5af2d18b6f stroage_proxy: change storage_proxy::mutate_locally to use do_with 2015-09-07 14:46:13 +02:00
Asias He
f89a25562c storage_service: Fix is_auto_bootstrap
Get the value from cfg option.
2015-09-07 12:53:58 +03:00
Avi Kivity
42dc29619d Merge "Optimize mutation copies and moves" from Paweł
"This series deals with copies and moves of mutation. The former are dealt
with by adding std::move() and missing 'mutable' (in case of lambdas). The
latter are improved by storing mutation_partition externally thus removing
the need for moving mutation_partition each time mutation is moved.

Storing mutation_partition externally is obviously trading the cost of
move constructor for the cost of allocation which shows in perf_mutation
results since mutations aren't moved in that test.

perf_mutation (-c 1):
before: 3289520.06 tps
after:  3183023.37 tps
diff: -3.24%

perf_simple_query (read):
before: 526954.05 tps
after:  577225.16 tps
diff +9.54%

perf_simple_query (write):
before: 731832.70 tps
after:  734923.60 tps
diff: +0.42%

Fixes #150 (well, not completely)."
2015-09-03 12:05:28 +03:00
Paweł Dziepak
08e3192b52 service: avoid copying mutations
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-09-03 10:30:36 +02:00
Paweł Dziepak
2d0112b11f migration_manager: add missing mutable
If lambda is not marked as mutable objects from its capture list cannot
be moved.

Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-09-03 10:30:22 +02:00
Glauber Costa
1ed6333566 storage_proxy: log failed attempts at communicating
If an exception happens in the query path, we'll never know about it. They are
currently being ignored.

Investigating this, I found out that this is because the readers in
storage_proxy.cc handles them - but don't log they anywhere.

This patch introduces such logging. the error() function takes an sstring not
an exception_ptr: this is so we can reuse it in the future to also log problems
from other hosts (currently not done).

We have a separate helper to extract the message from the current exception
before we pass it to error()

Fixes #110

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-09-03 09:10:51 +03:00
Asias He
7d04fc641e storage_service: fail unimplemented function
They are used by APIs. Let user know it is not implemented instead of
returning incorrect results or not performing the actual operation
silently.
2015-09-01 14:31:44 +08:00
Asias He
f244c85ea4 storage_service: Stub get_exception_count
Needed by API.
2015-09-01 14:31:44 +08:00
Asias He
3331e9fafe storage_service: Stub rebuild
Needed by API.
2015-09-01 14:31:44 +08:00
Asias He
c88b0fd987 storage_service: Stub get_load_map
Needed by API.
2015-09-01 14:31:44 +08:00
Asias He
21874eb8eb storage_service: Stub get_load
Needed by API.
2015-09-01 14:31:44 +08:00
Asias He
d7736062ef storage_service: Stub drain
Needed by API.
2015-09-01 14:31:44 +08:00
Asias He
ea691397f6 storage_service: Stub remove_node
Needed by API.
2015-09-01 14:31:44 +08:00
Asias He
a1ab9e60b7 storage_service: Stub decommission
Needed by API.
2015-09-01 14:31:44 +08:00
Asias He
babefbc47b storage_service: Stub is_native_transport_running
Needed by API.
2015-09-01 14:31:43 +08:00
Asias He
5df469517b storage_service: Stub stop_native_transport
Needed by API.
2015-09-01 14:31:43 +08:00