Commit Graph

7516 Commits

Author SHA1 Message Date
Tomasz Grabiec
d64db98943 query: Convert serialization of query::result to use db::serializer<>
That's what we're trying to standardize on.

This patch also fixes an issue with current query::result::serialize()
not being const-qualified, because it modifies the
buffer. messaging_service did a const cast to work this around, which
is not safe.
2015-12-03 09:19:11 +01:00
Tomasz Grabiec
d4d3a5b620 bytes_ostream: Make size_type and value_type public 2015-12-03 09:19:11 +01:00
Tomasz Grabiec
f0cfa61968 Relax header dependencies 2015-12-03 09:10:02 +01:00
Amnon Heiman
1812fe9e70 API: Add the get_version to messaging_service swagger definition file
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-12-02 14:45:44 +02:00
Amnon Heiman
ae53604ed7 API: Add the get_version implementation to messaging service
This patch adds the implementation to the get_version.
After this patch the following url will be available:
messaging_service/version?addr=127.0.0.1

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-12-02 13:29:40 +02:00
Avi Kivity
53e3e79349 Merge "API: Stubing the compaction manager" from Amnon
"This series allows the compaction manager to be used by the nodetool as a stub implementation.

It has two changes:
* Add to the compaction manager API a method that returns a compaction info
object

* Stub all the compaction method so that it will create an unimplemented
warning but will not fail, the API implementation will be reverted when the
work on compaction will be completed."
2015-12-02 13:28:34 +02:00
Takuya ASADA
871bfb1c94 dist: generate correct distribution codename on debian/changelog
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-12-02 12:38:52 +02:00
Takuya ASADA
b61ea247d2 dist: check supported Ubuntu release
Warn if unsupported release.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-12-02 12:38:52 +02:00
Takuya ASADA
0c66c25250 dist: fix typo on scylla_prepare
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-12-02 11:30:15 +02:00
Avi Kivity
f667e05e08 Merge "backport gosisp and storage_service fix" from Asias
"This contains most bug fixes from imported version commit
38847a6bd967e4f41bc7b1fc83629161a2c214dc to c* 2.1.11 for gossip and
storage_service."
2015-12-01 14:42:41 +02:00
Asias He
dc6f2157e7 Update ORIGIN for gossip and storage_service 2015-12-01 19:45:04 +08:00
Amnon Heiman
3674ee2fc1 API: get snapshot size
This patch adds the column family API that return the snapshot size.
The changes in the swagger definition file follo origin so the same API will be used for the metric and the
column_family.

The implementation is based on the get_snapshot_details in the
column_family.

This fix:
425

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-12-01 11:41:52 +02:00
Asias He
56df32ba56 gossip: Mark node as dead even if already left
Backport: CASSANDRA-10205

484e645 Mark node as dead even if already left
2015-12-01 17:29:25 +08:00
Asias He
59694a8e43 failure_detector: Print versions for gossip states in gossipinfo
Backport: CASSANDRA-10330

ae4cd69 Print versions for gossip states in gossipinfo

For instance, the version for each state, which can be useful for
diagnosing the reason for any missing states. Also instead of just
omitting the TOKENS state, let's indicate whether the state was actually
present or not.
2015-12-01 17:29:25 +08:00
Asias He
af91a8f31b storage_service: Fix transition from write survey to normal mode
Backport: CASSANDRA-9740

52dbc3f Can't transition from write survey to normal mode
2015-12-01 17:29:25 +08:00
Asias He
2f071d9648 storage_service: Refuse to decommission if not in state NORMAL
Backport: CASSANDRA-8741

5bc56c3 refuse to decomission if not in state NORMAL
2015-12-01 17:29:25 +08:00
Asias He
224db2ba37 failure_detector: Don't mark nodes down before the max local pause interval once paused
Backport: CASSANDRA-9446

7fba3d2 Don't mark nodes down before the max local pause interval once paused
2015-12-01 17:29:25 +08:00
Asias He
51fcc48700 failure_detector: Failure detector detects and ignores local pauses
Backport: CASSANDRA-9183

4012134 Failure detector detects and ignores local pauses
2015-12-01 17:29:25 +08:00
Asias He
1b9e350614 gossip: Do not print node is now part of the cluster during gossip shadow round
With

   Node 1 (Seed node, Port 7000 is opened, 10.184.9.144)
   Node 2 (Port 7000 is opened, 10.184.9.145)
   Node 3 (Port 7000 is blocked by firewall)

On Node 3, we saw the following error which was very confusing: Node 3
saw Node 1 and Node 3 but it complained it can not contact any seeds.

The message "Node 10.184.9.144 is now part of the cluster" and friends
are actually messages printed during the gossip shadow round where Node
3 connects to Node 1's port 7000 and Node 1 returns all info it knows to
Node 3, so that Node 3 knows Node 1 and Node 2 and we see the "Node
10.184.9.144/145 is now part of the cluster" message.

However, during the normal gossip round, Node 3 will not mark Node 1 and
Node 2 UP until the Seed node initiates a gossip round to Node 3, (note
port 7000 on node 3 is blocked in this case). So Node 3 will not mark
Node 1 and Node 2 UP and we see the "Unable to contact any seeds" error.

[shard 0] storage_service - Loading persisted ring state
[shard 0] gossip - Node 10.184.9.144 is now part of the cluster
[shard 0] gossip - inet_address 10.184.9.144 is now UP
[shard 0] gossip - Node 10.184.9.145 is now part of the cluster
[shard 0] gossip - inet_address 10.184.9.145 is now UP
[shard 0] storage_service - Starting up server gossip
scylla_run[12479]: Start gossiper service ...
[shard 0] storage_service - JOINING: waiting for ring information
[shard 0] storage_service - JOINING: schema complete, ready to bootstrap
[shard 0] storage_service - JOINING: waiting for pending range calculation
[shard 0] storage_service - JOINING: calculation complete, ready to bootstrap
[shard 0] storage_service - JOINING: getting bootstrap token
[shard 0] storage_service - JOINING: sleeping 5000 ms for pending range setup
scylla_run[12479]: Exiting on unhandled exception of type 'std::runtime_error': Unable to contact any seeds!
2015-12-01 17:29:25 +08:00
Asias He
f62a6f234b gossip: Add shutdown gossip state
Backported: CASSANDRA-8336 and CASSANDRA-9871

84b2846 remove redundant state
b2c62bb Add shutdown gossip state to prevent timeouts during rolling restarts
8f9ca07 Cannot replace token does not exist - DN node removed as Fat Client

Fixes:

When X is shutdown, X sends SHUTDOWN message to both Y and Z, but for
some reason, only Y receives the message and Z does not receive the
message. If Z has a higher gossip version for X than Y has for
X, Z will initiate a gossip with Y and Y will mark X alive again.

X ------> Y
 \      /
  \    /
    Z
2015-12-01 17:29:25 +08:00
Gleb Natapov
8c02ad0e9e messaging: log connection dropping event 2015-11-30 19:42:04 +02:00
Avi Kivity
b85f3ad130 Merge "Commit log replay - handle corrupted data silently, as non-fatal"
Fixes: #593

"Changes the parser/replayer to treat data corruption as non-fatal,
skipping as little as possible to get the most data out of a segment,
but keeping track of, and reporting, the amount corrupted.

Replayer handles this and reports any non-fatal errors on replay finish.
Also added tests for corruption cases.

This patch series contains a cleanup-patch for commitlog_tests that was
previously submitted, but got lost."
2015-11-30 19:13:31 +02:00
Gleb Natapov
5b9f3bff7d storage_proxy: simplify error handling by using this/handle_exception
It is cleaner to use this/handle_exception instead of then_wrapped if
normal and error flow do not share any state.
2015-11-30 17:41:32 +02:00
Gleb Natapov
5484f25091 storage_proxy: remove unneeded continuation
make_ready_future() around when_all() is not longer needed. It was
added to catch mutate_locally() exceptions, but now it is handled in
lower level.
2015-11-30 17:41:28 +02:00
Gleb Natapov
cf95c3f681 storage_proxy: introduce unique_response_handler object to prevent write request leaks
If something bad happens between write request handler creation and
request execution the request handler have to be destroyed. Currently
code tries to do that explicitly in all places where request may be
abandoned, but it misses some (at least one). This patch replaces this
by introducing unique_response_handler object that will remove the handler
automatically if request is not executed for some reason.
2015-11-30 17:41:27 +02:00
Gleb Natapov
d8afc6014e storage_proxy: catch exception thrown by mutate_locally in mutate verb handler
Also simplify error logging.
2015-11-30 17:41:25 +02:00
Avi Kivity
3c9ded27cc Update scylla-ami submodule
* ami/files/scylla-ami 3f37184...07b7118 (1):
  > Use /etc/scylla as SCYLLA_CONF directory
2015-11-30 16:39:49 +02:00
Takuya ASADA
616903de12 dist: use distribution version of antlr3, on Ubuntu 15.10
Rename antlr3-tool to antlr3 (same as distribution package), and use distribution version if it's available

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-11-30 16:37:36 +02:00
Pekka Enberg
0e8f80b5ee Merge "Relax bootstrapping/leaving/moving nodes check" from Asias 2015-11-30 11:53:07 +02:00
Asias He
2022117234 failure_detector: Enable phi_convict_threshold option
Adjusts the sensitivity of the failure detector on an exponential scale.

Use as:

$ scylla --phi-convict-threshold 9

Default to 8.
2015-11-30 11:09:36 +02:00
Asias He
db70643fe3 failure_detector: Print application_state properly 2015-11-30 11:08:40 +02:00
Asias He
aaca88a1e7 token_metadata: Add print_pending_ranges for debug print
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-11-30 11:07:42 +02:00
Avi Kivity
2c59e2f81f Merge "Fix race between population and update in row cache" from Tomasz
"Before this change, populations could race with update from flushed
memtable, which might result in cache being populated with older
data. Populations started before the flush are not considering the
memtable nor its sstable.

The fix employed here is to make update wait for populations which
were started before the flushed memtable's sstable was added to the
undrelying data source. All populatinos started after that are
guaranteed to see the new data. The update() call will wait only for
current populating reads to complete, it will not wait for readers to
get advanced by the consumer for instance."
2015-11-30 11:06:23 +02:00
Tomasz Grabiec
8d88ece896 schema_tables: Fix "comment" property not being loaded from storage 2015-11-30 10:57:36 +02:00
Pekka Enberg
a64fa3db03 Merge "range_streamer fix and cleanup" from Asias
Do not use hard-coded value for is_replacing and rangemovement.
2015-11-30 10:47:06 +02:00
Asias He
879a4ad4d3 storage_service: Update pending ranges immediately after update of normal tokens
To avoid a race where natural endpoint was updated to contain node A,
but A was not yet removed from pending endpoints.

This fixes the root cause of commit d9d8f87c1 (storage_proxy: filter out
natural endpoints from pending endpoint). This patch alone fixes #539,
but we still want commit d9d8f87c1 to be safe.
2015-11-30 10:20:59 +02:00
Asias He
0af7fb5509 range_streamer: Kill FIXME in use_strict_consistency for consistent_rangemovement 2015-11-30 09:15:42 +08:00
Asias He
f80e3d7859 range_streamer: Simplify multiple_map to map conversion in add_ranges 2015-11-30 09:15:42 +08:00
Asias He
21882f5122 range_streamer: Kill one leftover comment 2015-11-30 09:15:42 +08:00
Asias He
6b258f1247 range_streamer: Kill FIXME for is_replacing 2015-11-30 09:15:42 +08:00
Asias He
aa2b11f21b database: Move is_replacing and get_replace_address to database class
So they can be used outside storage_service.
2015-11-30 09:15:42 +08:00
Asias He
80d1d4d161 storage_service: Relax bootstrapping/leaving/moving nodes check in check_for_endpoint_collision
When other bootstrapping/leaving/moving nodes are found during
bootstrap, instead of throwing immediately, sleep and try again for one
minute, hoping other nodes will finish the operation soon.

Since we are retrying using shadow gossip round more than once, we need
to put the gossip state back to shadow round after each shadow round, to
make shadow round works correctly.

This is useful when starting an empty cluster for testing. E.g,

   $ scylla --listen-address 127.0.0.1
   $ sleep 3
   $ scylla --listen-address 127.0.0.2
   $ sleep 3
   $ scylla --listen-address 127.0.0.3

Without this patch, node 3 will hit the check.

   TIME  STATUS
   -----------------------
   Node  1:
   32:00 Starts
   32:00 In NORMAL status

   Node  2:
   32:03 Starts
   32:04 In BOOT status
   32:10 In NORMAL status

   Node  3:
   32:06 Starts
   32:06 Found node 2 in BOOT status, hit the check, sleep and try again
   32:11 Found node 2 in NORMAL status, can keep going now
   32:12 In BOOT status
   32:18 In NORMAL status
2015-11-30 09:07:57 +08:00
Asias He
8b19373536 storage_service: Relax bootstrapping/leaving/moving nodes check in join_token_ring
When other bootstrapping/leaving/moving nodes are found during
bootstrap, instead of throwing immediately, sleep and try again for one
minute, hoping other nodes will finish the operation soon.

This is useful when starting an empty cluster for testing. E.g,

   $ scylla --listen-address 127.0.0.1
   $ scylla --listen-address 127.0.0.2
   $ scylla --listen-address 127.0.0.3

Without this patch, node 3 will hit the check.

   TIME  STATUS
   -----------------------
   Node  1:
   25:19 Starts
   25:20 In NORMAL status

   Node  2:
   25:19 Starts
   25:23 In BOOT status
   25:28 In NORMAL status

   Node  3:
   25:19 Starts
   25:24 Found node 2 in BOOT status, hit the check, sleep and try again
   25:29 Found node 2 in NORMAL status, can keep going now
   25:29 In BOOT status
   25:34 In NORMAL status
2015-11-30 09:07:57 +08:00
Tomasz Grabiec
df46542832 tests: Add test for populate and update race 2015-11-29 16:25:22 +01:00
Tomasz Grabiec
6f69d4b700 tests: Avoid potential use after free on partition range 2015-11-29 16:25:21 +01:00
Tomasz Grabiec
de75f3fa69 row_cache: Add default value for partition range in make_reader() 2015-11-29 16:25:21 +01:00
Tomasz Grabiec
ab328ead3d mutation: Introduce ring_position() 2015-11-29 16:25:21 +01:00
Tomasz Grabiec
32ac2ccc4a memtable: Introduce apply(memtable&) 2015-11-29 16:25:21 +01:00
Tomasz Grabiec
7c3e6c306b row_cache: Wait for in-flight populations on update
Before this change, populations could race with update from flushed
memtable, which might result in cache being populated with older
data. Populations started before the flush are not considering the
memtable nor its sstable.

The fix employed here is to make update wait for populations which
were started before the flushed memtable's sstable was added to the
undrelying data source. All populatinos started after that are
guaranteed to see the new data.
2015-11-29 16:25:21 +01:00
Tomasz Grabiec
a3e3add28a utils: Introduce phased_barrier
Utility for waiting on a group of async actions started before certain
point in time.
2015-11-29 16:25:21 +01:00