Commit Graph

53948 Commits

Author SHA1 Message Date
Avi Kivity
c175025bb6 db: place all memtables into a single region_group
We can use this to track the amount of unevictable memory in the
system.
2015-08-19 19:36:41 +03:00
Avi Kivity
9ed2bbb25c lsa: introduce region_group
A region_group is a nestable group of regions, for cumulative statistics
purposes.
2015-08-19 19:36:40 +03:00
Raphael S. Carvalho
32ce27f00d tests: fix possible failure on compaction manager test
If sleep time isn't enough for compaction manager to select the
submitted cf for compaction, then the test will fail because the
compaction will not take place and subsequent checks will fail.
A solution is to sleep until the required condition becomes true.

Problem and solution found by Shlomi.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-08-19 18:26:54 +03:00
Avi Kivity
dd4d76980d Merge seastar upstream
* seastar 4ff14c4...d96bfcd (1):
  > rpc: keep reference to server::connection while reply is sent
2015-08-19 15:11:28 +03:00
Avi Kivity
71aad57ca8 lsa: make region::impl a top-level class
Makes using forward declarations possible.
2015-08-19 14:43:17 +03:00
Avi Kivity
7b67b04822 db: wire up max memtable size configuration 2015-08-19 13:17:27 +03:00
Avi Kivity
c317391f62 db: trigger memtable flush based on actual memory usage
Rather than using _mutation_count as a poor proxy.
2015-08-19 12:59:52 +03:00
Avi Kivity
e7272d27cc tests: perf_mutation: convert to app_template
Won't work with lsa without it, due to too small default memory size.
2015-08-19 11:18:07 +03:00
Avi Kivity
5252d5ec9b managed_bytes: fix self-assignment 2015-08-19 11:18:07 +03:00
Avi Kivity
00f39c4e1a managed_bytes: add small string optimization 2015-08-19 11:18:07 +03:00
Avi Kivity
0db6962a89 Merge "gossip fix" from Asias
Fixes #162, #144, #134.
2015-08-19 11:11:56 +03:00
Asias He
8415218dba gossip: Save one seastar thread inside remove_endpoint
Now all the caller are inside a seastar thread. Kill one thread inside
remove_endpoint.
2015-08-19 14:46:44 +08:00
Avi Kivity
176ab06f77 db: demote commitlog reorderign detected log message to debug
It's less rare than we thought and also less interesting.
2015-08-19 09:26:23 +03:00
Asias He
b560a50ea7 gossip: Fix a name typo
evict_from_membershipg -> evict_from_membership
2015-08-19 14:22:54 +08:00
Asias He
54a42b4549 gossip: Fix a iterate and delete issue in do_status_check
evict_from_membership might delete an element inside endpoint_state_map
while iterating it.

Fixes #162
2015-08-19 14:21:11 +08:00
Asias He
016dfdc8e1 gossip: Gossip error messages
We are printing out error messages when a remote connection is closed

   ERROR   [shard 0] gossip - Fail to send GossipDigestACK2 to 127.0.0.2:0: rpc::closed_error (connection is closed)
   ERROR   [shard 0] gossip - Fail to handle GOSSIP_DIGEST_ACK: rpc::closed_error (connection is closed)
   WARN    [shard 0] unimplemented

this is causing issues with DTEST as it validates after finishing a run
that there are no ERRORs in the log

The rule is:
   We can handle it correctly if error occurs -> log warn
   We can not handle it correctly when error occurs -> log error

Fixes #144
2015-08-19 14:21:05 +08:00
Asias He
009f9e7f21 gossip: Add timeout for send_gossip_digest_syn in do_shadow_round
Fixes #134
2015-08-19 14:20:52 +08:00
Avi Kivity
ddee5e817a Workaround boost::any_cast bug
any_cast<X> is supposed to return X, but boost 1.55's any_cast<X> returns
X&&.  This means the lifetime-extending construct

   auto&& x = boost::any_cast<X>(...);

will not work, because the result of the expression is an rvalue reference,
not a true temporary.

Fix by using a temporary, not a lifetime-extending reference.

Fixes #163.
2015-08-19 09:15:31 +03:00
Avi Kivity
2b4eaf83ab transport: set TCP_NODELAY
Reduces latency.
2015-08-18 15:48:36 +03:00
Avi Kivity
2e92759943 thrift: set TCP_NODELAY
Reduces latency.
2015-08-18 15:48:36 +03:00
Avi Kivity
c98a738656 Merge seastar upstream
* seastar 2afc6c8...39eeca5 (5):
  > rpc: set TCP_NODELAY on rpc sockets
  > net: implement accessors for Nagle's algorithm (TCP_NODELAY)
  > posix: improve getsockopt() interface
  > future: slightly optimize then() signature
  > net: add missing tcp_nodelay thunks
2015-08-18 15:48:16 +03:00
Avi Kivity
2354611920 Merge "storage_service udpate" from Asias 2015-08-18 12:34:14 +03:00
Avi Kivity
8047e5ed8a Merge seastar upstream
* seastar 69edf16...2afc6c8 (3):
  > Rebase dpdk to v2.1.0
  > future: don't use get() in future_state::forward_to()
  > future: add get_value(), and use it in then()
2015-08-18 12:32:43 +03:00
Avi Kivity
0b01b74444 build: disable seastar Xen support
Not needed, and conflicts with dpdk.
2015-08-18 12:31:26 +03:00
Avi Kivity
e9a46215ef build: change project name
The configure script originated from seastar, need a name change.
2015-08-18 12:29:05 +03:00
Asias He
1b97cd988d storage_service: Fix indentation in init_server 2015-08-18 17:06:03 +08:00
Asias He
eda11a35e6 storage_service: Partially implement handle_state_removing 2015-08-18 17:06:03 +08:00
Asias He
0f2e4003ce storage_service: Implement handle_state_moving 2015-08-18 17:06:03 +08:00
Asias He
8a7f7d5d8b storage_service: Implement handle_state_left 2015-08-18 17:06:03 +08:00
Asias He
455ea0e87c storage_service: Implement handle_state_leaving 2015-08-18 17:06:03 +08:00
Asias He
7c72ed227b storage_service: Enable !join_ring case in init_server 2015-08-18 17:06:03 +08:00
Asias He
c111f8395c storage_service: Enable load_tokens logic in init_server 2015-08-18 17:06:03 +08:00
Asias He
67953a65b6 db/system_keyspace: Stub load_host_ids 2015-08-18 17:06:03 +08:00
Asias He
ab40ab6c19 db/system_keyspace: Stub load_tokens 2015-08-18 17:06:02 +08:00
Asias He
a8efcac89b storage_service: Run init_server inside seastar::async
This helps enabling more code inside it.
2015-08-18 17:06:02 +08:00
Asias He
63d32fb681 storage_service: Enable remove_endpoint in bootstrap 2015-08-18 17:06:02 +08:00
Asias He
c7d2d07fc1 storage_service: Kill one FIXME for is_replacing 2015-08-18 17:06:02 +08:00
Asias He
7a0bc19c98 storage_service: Enable set_bootstrap_state and friends.
It is enabled in system_keyspace now.
2015-08-18 17:06:02 +08:00
Asias He
63a577c34c tests/cql_test_env: Init system_keyspace's local_cache
Soon storage_service will access it, we need to init it.
2015-08-18 17:06:02 +08:00
Asias He
7f98a89968 db/system_keyspace: Introduce init_local_cache 2015-08-18 17:06:02 +08:00
Raphael S. Carvalho
820ba6f4d2 adapt compaction manager for column family removal
We need a way to remove a column family from the compaction manager
because when dropping a column family we need to make sure that the
compaction manager doesn't hold a reference to it anymore.

So compaction manager queue is now of column_family, allowing us
to cancel requests pertaining to a column family being dropped.
There may be an ongoing compaction for the column family being
dropped, so we also need to wait for its termination.

Testcase for compaction manager was also adapted and improved.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-08-18 11:38:06 +03:00
Amnon Heiman
47ce728572 API: Adding a stub implementation for the storage_service metrics
This adds a stub implementation for the storge service metrics. The
implementation returns the currect type with a stub value.
After this patch the following url will be available:
/storage_service/metrics/load
/storage_service/metrics/exceptions
/storage_service/metrics/hints_in_progress
/storage_service/metrics/total_hints

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-18 11:19:52 +03:00
Amnon Heiman
34e60faaca API: Adding the storage service metrics definition
This adds the storage service metrics that is based on the
StorageServiceMetrics class.
The following command where added:
get_metrics_load
get_exceptions
get_total_hints_in_progress
get_total_hints

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-18 11:19:51 +03:00
Amnon Heiman
cae3de162c API: Remove empty line from empty parameters list in
storage_service.json

Just for styling, empty parameters list will not include an empty line.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-18 11:19:51 +03:00
Avi Kivity
583d3b8438 Merge "Implement token and bootstrap functions from system keyspace" from Glauber
Fixes #138
2015-08-18 10:56:45 +03:00
Raphael S. Carvalho
2608427469 sstables: add support to range tombstone of a clustered row
Range tombstone for a clustered row wasn't supported, so an assert
to remember that was being triggered.
Testcase was added.

Fixes #158.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-08-18 10:41:25 +03:00
Glauber Costa
4cfe7de292 schema: correctly compound collections
We are currently using the ColumnToCollectionType wrongly: we are wrapping
by that string to every collection. But that is not how Origin operates: a single
ColumnToCollectionType hosts all collections a schema has.

Funny enough, sstable2json seems to work all right without any comparator - and
that is how it worked before, but when a comparator is present, it expects it to
abide by what Origin expects. That causes us to crash.

Fixes #148

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-08-18 10:40:04 +03:00
Glauber Costa
0177c7fed1 system keyspace: implement get_bootstrap_state
To avoid spreading the futures all over, we will resort to a cache with this,
the same way we did for the dc/rack information.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-08-17 11:03:37 -07:00
Glauber Costa
20590db87f system keyspace: implement set_bootstrap_state
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-08-17 11:03:37 -07:00
Glauber Costa
8a50534119 system keyspace: implement get_saved_tokens
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-08-17 11:03:37 -07:00