Commit Graph

5663 Commits

Author SHA1 Message Date
Vlad Zolotarov
a3cda17bfb locator::gossiping_property_file_snitch: simplify the start()
- Call for read_property_file() directly from the start().
   - Immediately return ready future from the start() for non-IO
     CPUs.
   - Remove the not needed invoke_on_all() invocations.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-08-12 16:21:18 +03:00
Vlad Zolotarov
4eeed09572 tests: gossiping_property_file_snitch_test: stop() the distributed in an error case
If snitch has been created while it had to fail we have to stop the
global (distributed) snitch in order to avoid the assert.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-08-12 16:19:06 +03:00
Vlad Zolotarov
0b17f2ad75 locator: simple and rack_inferring snitches: add "override" qualifier to get_name() method
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-08-12 16:19:06 +03:00
Vlad Zolotarov
c9f9d8164e locator::gossiping_property_file_snitch: make get_name() public as it should be
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-08-12 16:19:06 +03:00
Vlad Zolotarov
31dbca22d7 locator: snitch_base.hh: cleanups
- Fixed a typo.
   - snitch_ptr: make operator= return a reference to the parent object.
   - i_endpoint_snitch: set the _state in a default start() implementation.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-08-12 16:18:58 +03:00
Gleb Natapov
0b3d2de2f1 Fix mutation write timeout exception reporting
Make it compatible with CQL specification
2015-08-12 14:58:48 +03:00
Avi Kivity
149b08889a Merge "gossip cleanup" from Asias 2015-08-12 12:40:23 +03:00
Asias He
0f2ea6d7c0 gossip: Remove one TODO for SHUTDOWN handling 2015-08-12 17:35:46 +08:00
Asias He
c72c96f8aa gossip: Remove a outdated TODO
It is fixed already.
2015-08-12 17:35:46 +08:00
Asias He
5831dcba28 gossip: Add error handling for GOSSIP_SHUTDOWN and GOSSIP_DIGEST_ACK verb 2015-08-12 17:35:46 +08:00
Avi Kivity
b38d6b8132 Merge "storage_service update" from Asias
"I'm leaving the following functions

   get_saved_tokens()
   get_bootstrap_state()
   set_bootstrap_state()

for people more familiar with execute_cql."
2015-08-12 11:55:22 +03:00
Avi Kivity
c453a2075a Merge 2015-08-12 11:49:20 +03:00
Avi Kivity
c90e3c4bb2 Merge "CQL server cleanups" from Pekka
"Cleanups to the CQL server implementation. The biggest change is moving
event notifier to a separate source file in an attempt to make server.cc
smaller and more modularized."
2015-08-12 11:42:04 +03:00
Avi Kivity
ecc3ccc716 lsa: emergency segment reserve for compaction
To free memory, we need to allocate memory.  In lsa compaction, we convert
N segments with average occupancy of (N-1)/N into N-1 new segments.  However,
to do that, we need to allocate segments, which we may not be able to do
due to the low memory condition which caused us to compact anyway.

Fix by introducing a segment reserve, which we normally try to ensure is
full.  During low memory conditions, we temporarily allow allocating from
the emergency reserve.
2015-08-12 11:29:09 +03:00
Asias He
74e2f0156a messaging_service: Ignore cpu id for shard_id hash
Since we do not support shard to shard connections at the moment, ip
address should fully decide if a connection to a remote node exists or
not. messaging_service maintains connections to remote node using

   std::unordered_map<shard_id, shard_info, shard_id::hash> _clients;

With this patch, we can possibly reduce number of tcp connections
between two nodes.
2015-08-12 10:25:06 +03:00
Pekka Enberg
5c99a58b27 transport: Move cql_server and event_notifier to transport namespace
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-12 09:59:36 +03:00
Pekka Enberg
3ab87c216e transport/server: Use pragma once as include guard
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-12 09:59:36 +03:00
Pekka Enberg
a3c194b050 transport/server: Move event_notifier class to separate file
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-12 09:59:35 +03:00
Pekka Enberg
3ad4e4e829 transport/server: Move connection class definition to server.hh
Move the connection class to server.hh so that we can move event
notifier implementation to a separate source file.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-12 09:59:09 +03:00
Pekka Enberg
42f865a3de transport/server: Clean up connection class definition
Move member function definitions outside of the class definition in
preparation for moving the latter to a header file.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-12 09:25:09 +03:00
Asias He
dd34f4b0a4 storage_service: Enable update_local_tokens 2015-08-12 08:02:07 +08:00
Asias He
ce927105d8 db/system_keyspace: Implement update_local_tokens 2015-08-12 07:50:26 +08:00
Asias He
b3f7507e0a storage_service: Enable gossiper.replacement_quarantine in handle_state_normal 2015-08-12 07:50:26 +08:00
Asias He
4d3f333ec0 storage_service: Enable call to remove_endpoint in on_remove 2015-08-12 07:50:26 +08:00
Asias He
95dd307597 db/system_keyspace: Remove duplicated commented out code
I'm not sure what happened. We have the same commented code in both .hh
and .cc. It is very confusing when enabling some of the code. Let's
remove the duplicated code in .cc and leave the in .hh only.
2015-08-12 07:50:26 +08:00
Asias He
96fe749141 db/system_keyspace: Stub get_bootstrap_state and friends 2015-08-12 07:50:26 +08:00
Asias He
951c0d192b storage_service: Enable _is_survey_mode logic in join_token_ring 2015-08-12 07:50:26 +08:00
Asias He
6874663c9d storage_service: Enable current in prepare_to_join 2015-08-12 07:50:26 +08:00
Asias He
645700d261 storage_service: Implement join_ring
Join the ring by operator request.
2015-08-12 07:50:26 +08:00
Asias He
3ea91504ba storage_service: Enable get_saved_tokens and get_initial_tokens 2015-08-12 07:50:26 +08:00
Asias He
5cb5050ca1 system_keyspace: Stub get_saved_tokens 2015-08-12 07:50:26 +08:00
Raphael S. Carvalho
ce2fea2976 tests: add basic test to compaction manager functionality
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-08-11 20:14:48 +03:00
Gleb Natapov
36c7c2ac5f Provide correct data_present for read_timeout_exception
Fix fixmes.
2015-08-11 19:45:59 +03:00
Gleb Natapov
6046316352 Untemplatize continuation in storage_proxy::query
Nothing wrong with it besides that it crashs my eclips indexer for some
reason.
2015-08-11 19:45:59 +03:00
Avi Kivity
09ae029f87 Merge seastar upstream
* seastar 8f97b50...7e7cef2 (1):
  > future: make futurize::apply() more flexible
2015-08-11 18:44:28 +03:00
Avi Kivity
ca6ab0a6d1 Merge "Implement "atomic" batch statement processing" from Calle
"I.e. implement storage_proxy::mutate_atomically, which in turn means
roughly the same as mutate, with write/remove from the batchlog table
intermixed.

This patch restructures some stuff in storage_proxy to avoid to much code
duplication, with the assumption (amongst other) that dead nodes will be few
etc."
2015-08-11 18:33:34 +03:00
Avi Kivity
8d449190d1 thrift: handle exceptions thrown in non-continuation part of thrift handler
Our thrift code performs an elaborate dance to convert a result/exception
reported in a future<> to the cob/exn_cob flow required by the thrift
library.  However, if the exception if thrown before the first continuation,
no one will catch it will be leaked, eventually resulting in a crash.

Fix by replacing the complete() infrastructure, which took a future as a
parameter, with a with_cob() helper that instead takes a function to
execute.  This allows it to catch both exceptions thrown directly and
exceptions reported via the future.

Fixes #133.
2015-08-11 18:29:58 +03:00
Calle Wilund
81f2f80963 Add batchlog_manager to cqlenv + add batch statement test to cql test 2015-08-11 17:10:18 +02:00
Calle Wilund
b7c7c97295 StorageProxy: implement mutate_atomically
Atomically == add to batch log before doing actual mutate
2015-08-11 17:10:17 +02:00
Calle Wilund
7cacf6382f main: initialize global batchlog_manager in startup code 2015-08-11 17:10:17 +02:00
Calle Wilund
9a52ad84b1 BatchlogManager: make blm globally reachable distributed like other objects 2015-08-11 17:10:17 +02:00
Calle Wilund
0ded44eeee BatchlogManager: make endpoint_filter method + implement 2015-08-11 17:10:16 +02:00
Glauber Costa
799a6b5962 sstables: change summary_la to summary_ka
What we implement is ka, not la. Since the summary is the one element that
actually changed in the 2.2 implementation, it is particularly important that
we get this one right. I have previously missed this.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-08-11 17:47:48 +03:00
Raphael S. Carvalho
9823164c89 db: introduce compaction manager
Currently, each column family creates a fiber to handle compaction requests
in parallel to the system. If there are N column families, N compactions
could be running in parallel, which is definitely horrible.

To solve that problem, a per-database compaction manager is introduced here.

Compaction manager is a feature used to service compaction requests from N
column families. Parallelism is made available by creating more than one
fiber to service the requests. That being said, N compaction requests will
be served by M fibers.

A compaction request being submitted will go to a job queue shared between
all fibers, and the fiber with the lowest amount of pending jobs will be
signalled.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-08-11 17:25:46 +03:00
Avi Kivity
3c5f12bb97 Merge seastar upstream
* seastar a1933df...8f97b50 (1):
  > sharded: fix move assignment operator
2015-08-11 17:03:20 +03:00
Avi Kivity
e00deca209 Merge "CQL protocol fixes" from Pekka
"Various fixes to CQL protocol error handling."
2015-08-11 14:49:21 +03:00
Pekka Enberg
d461443bf2 transport/server: Fix error handling in parse_frame()
Throw a protocol exception instead of killing the process with abort().

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-11 14:33:30 +03:00
Pekka Enberg
48abeefdda transport/server: Fix connection version mismatch validation
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-11 14:33:30 +03:00
Pekka Enberg
bb067782d7 transport/server: Fix CQL binary protocol version validation
We only support up to CQL protocol v3. Fixes #117.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-11 14:33:30 +03:00
Pekka Enberg
ad041207a4 transport/server: Catch all request processing errors
We need to also catch exceptions in top-level connection::process() so
that they are converted to proper CQL protocol errors.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-11 14:33:21 +03:00