Commit Graph

7476 Commits

Author SHA1 Message Date
Asias He
aaca88a1e7 token_metadata: Add print_pending_ranges for debug print
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-11-30 11:07:42 +02:00
Avi Kivity
2c59e2f81f Merge "Fix race between population and update in row cache" from Tomasz
"Before this change, populations could race with update from flushed
memtable, which might result in cache being populated with older
data. Populations started before the flush are not considering the
memtable nor its sstable.

The fix employed here is to make update wait for populations which
were started before the flushed memtable's sstable was added to the
undrelying data source. All populatinos started after that are
guaranteed to see the new data. The update() call will wait only for
current populating reads to complete, it will not wait for readers to
get advanced by the consumer for instance."
2015-11-30 11:06:23 +02:00
Tomasz Grabiec
8d88ece896 schema_tables: Fix "comment" property not being loaded from storage 2015-11-30 10:57:36 +02:00
Pekka Enberg
a64fa3db03 Merge "range_streamer fix and cleanup" from Asias
Do not use hard-coded value for is_replacing and rangemovement.
2015-11-30 10:47:06 +02:00
Asias He
879a4ad4d3 storage_service: Update pending ranges immediately after update of normal tokens
To avoid a race where natural endpoint was updated to contain node A,
but A was not yet removed from pending endpoints.

This fixes the root cause of commit d9d8f87c1 (storage_proxy: filter out
natural endpoints from pending endpoint). This patch alone fixes #539,
but we still want commit d9d8f87c1 to be safe.
2015-11-30 10:20:59 +02:00
Asias He
0af7fb5509 range_streamer: Kill FIXME in use_strict_consistency for consistent_rangemovement 2015-11-30 09:15:42 +08:00
Asias He
f80e3d7859 range_streamer: Simplify multiple_map to map conversion in add_ranges 2015-11-30 09:15:42 +08:00
Asias He
21882f5122 range_streamer: Kill one leftover comment 2015-11-30 09:15:42 +08:00
Asias He
6b258f1247 range_streamer: Kill FIXME for is_replacing 2015-11-30 09:15:42 +08:00
Asias He
aa2b11f21b database: Move is_replacing and get_replace_address to database class
So they can be used outside storage_service.
2015-11-30 09:15:42 +08:00
Tomasz Grabiec
df46542832 tests: Add test for populate and update race 2015-11-29 16:25:22 +01:00
Tomasz Grabiec
6f69d4b700 tests: Avoid potential use after free on partition range 2015-11-29 16:25:21 +01:00
Tomasz Grabiec
de75f3fa69 row_cache: Add default value for partition range in make_reader() 2015-11-29 16:25:21 +01:00
Tomasz Grabiec
ab328ead3d mutation: Introduce ring_position() 2015-11-29 16:25:21 +01:00
Tomasz Grabiec
32ac2ccc4a memtable: Introduce apply(memtable&) 2015-11-29 16:25:21 +01:00
Tomasz Grabiec
7c3e6c306b row_cache: Wait for in-flight populations on update
Before this change, populations could race with update from flushed
memtable, which might result in cache being populated with older
data. Populations started before the flush are not considering the
memtable nor its sstable.

The fix employed here is to make update wait for populations which
were started before the flushed memtable's sstable was added to the
undrelying data source. All populatinos started after that are
guaranteed to see the new data.
2015-11-29 16:25:21 +01:00
Tomasz Grabiec
a3e3add28a utils: Introduce phased_barrier
Utility for waiting on a group of async actions started before certain
point in time.
2015-11-29 16:25:21 +01:00
Pekka Enberg
a26ffefd53 transport/server: Remove CQL text type from encoding
The text data type is no longer present in CQL binary protocol v3 and
later. We don't need it for encoding earlier versions either because
it's an alias for varchar which is present in all CQL binary protocol
versions.

Fixes #526.

Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-11-27 09:13:56 +01:00
Pekka Enberg
2599b78583 Merge "CQL notification + storage_service fix" from Asias
"pushed_notifications_test.py dtest is now passing"
2015-11-27 10:09:56 +02:00
Asias He
da0e80a286 storage_service: Fix failed bootstrap/replace attempts being persisted in system.peers
Backported from:

ac46747 Fix failed bootstrap/replace attempts being persisted in system.peers (CASSANDRA-9180)
2015-11-27 15:31:56 +08:00
Asias He
36b2de10ed failure_detector: Improve FD logging when the arrival time is ignored
Backport from:

eb9c5bb Improve FD logging when the arrival time is ignored.
2015-11-27 15:31:56 +08:00
Asias He
ed9cd23a2d transport: Fix duplicate up/down messages sent to native clients
This patch plus pekka's previous commit 3c72ea9f96

   "gms: Fix gossiper::handle_major_state_change() restart logic"

fix CASSANDRA-7816.

Backported from:

   def4835 Add missing follow on fix for 7816 only applied to
           cassandra-2.1 branch in 763130bdbde2f4cec2e8973bcd5203caf51cc89f
   763130b Followup commit for 7816
   2199a87 Fix duplicate up/down messages sent to native clients

Tested by:
   pushed_notifications_test.py:TestPushedNotifications.restart_node_test
2015-11-27 15:31:56 +08:00
Asias He
25bb889c2a transport: Fix wrong message for UP and DOWN event 2015-11-27 15:31:56 +08:00
Asias He
ca8c4f3e77 storage_service: Fix MOVED_NODE client event (CASSANDRA-8516)
Backport from:

b296c55f956c6ef07c8330dc28ef8c351e5bcfe2 (Fix MOVED_NODE client event)

Fixes:

DISABLE_VNODES=true nosetests
pushed_notifications_test.py:TestPushedNotifications.move_single_node_test
2015-11-27 15:31:56 +08:00
Gleb Natapov
ad358300a9 cql server: remove connection from notifiers earlier
Remove connection from notifiers lists just before closing it to prevent
attempts to send notification on already closed connection.
2015-11-26 18:50:08 +02:00
Pekka Enberg
569d288891 cql3: Add TRUNCATE TABLE alias for TRUNCATE
CQL 3.2.1 introduces a "TRUNCATE TABLE X" alias for "TRUNCATE X":

  4e3555c1d9

Fix our CQL grammar to also support that.

Please note that we don't bump up advertised CQL version yet because our
cqlsh clients won't be able to connect by default until we upgrade them
to C* 2.1.10 or later.

Fixes #576

Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-11-26 18:45:50 +02:00
Gleb Natapov
96f40d535e cql server: add missing gate during connection access
cql connection access is protected by a gate, but event notifiers have
omitted taking it. Fix it.
2015-11-26 13:05:59 +02:00
Tomasz Grabiec
a7c11d1e30 db: Fix handling of missing column family
The FIXMEs are no longer valid, we load schema on bootstrap and don't
support hot-plugging of column families via file system (nor does
Cassandra).

Handling of missing tables matches Cassandra 2.1, applies log
it and continue, queries propagate the error.
2015-11-25 16:59:15 +02:00
Tomasz Grabiec
3a402db1be storage_proxy: Remove dead signature 2015-11-25 16:57:03 +02:00
Asias He
d03b452322 storage_service: Remove RPC client in on_dead
When gossip mark a node down, we should close all the RPC connections to
that node.
2015-11-25 16:30:14 +02:00
Gleb Natapov
d9d8f87c1b storage_proxy: filter out natural endpoints from pending endpoint
If request comes after natural endpoint was updated to contain node A,
but A was not yet removed from pending endpoints it will be in both and
write request logic cannot handle this properly. Filter nodes which are
already in natural endpoint from pending endpoint to fix this.

Fixes #539.
2015-11-25 16:28:55 +02:00
Pekka Enberg
cf7541020f Merge "Enable more config options" from Asias 2015-11-25 16:09:22 +02:00
Tomasz Grabiec
c3f03d5c96 Merge branch 'pdziepak/random-lsa-patches/v3' from seastar-dev.git
LSA fixes from Paweł.
2015-11-25 10:26:23 +01:00
Paweł Dziepak
89f7f746cb lsa: fix printing object_descriptor::_alignment
object_descriptor::_alignment is of type uint8_t which is actually an
unsigned char.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-11-24 20:13:29 +01:00
Paweł Dziepak
65875124b7 lsa: guarantee that segment_heap doesn't throw
boost::heap::binomial_heap allocates helper object in push() and,
therefore, may throw an exception. This shouldn't happen during
compaction.

The solution is to reserve space for this helper object in
segment_descriptor and use a custom allocator with
boost::heap::binomial_heap.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-11-24 19:51:22 +01:00
Paweł Dziepak
273b8daeeb lsa: add no-op default constructor for segment
Zero initialization of segment::data when segment is value initialized
is undesirable.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-11-24 16:37:37 +01:00
Paweł Dziepak
e6cf3e915f lsa: add counters for memory used by large objects
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-11-24 16:36:27 +01:00
Paweł Dziepak
9396956955 scylla-gdb.py: show lsa statistics and regions
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-11-24 16:36:20 +01:00
Paweł Dziepak
aaecf5424c scylla-gdb.py: show free, used and total memory
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-11-24 16:36:16 +01:00
Paweł Dziepak
6b113a9a7a lsa: fix eviction of large blobs
LSA memory reclaimer logic assumes that the amount of memory used by LSA
equals: segments_in_use * segment_size. However, LSA is also responsible
for eviction of large objects which do not affect the used segmentcount,
e.g. region with no used segments may still use a lot of memory for
large objects. The solution is to switch from measuring memory in used
segments to used bytes count that includes also large objects.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-11-24 16:29:09 +01:00
Takuya ASADA
4a8c79ca0e dist: re-initialize RAID on ephemeral disk when stop/restart AMI instance
Since this won't check disk types, may re-initialize RAID on EBS when first block was lost.
But in such condition, probably re-initialize RAID is the only choice we can take, so this is fine.
Fixes #364.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-11-24 10:46:10 +02:00
Asias He
3a9200db03 config: Add more document for options
For consistent_rangemovement, join_ring, load_ring_state, etc.
2015-11-24 10:07:31 +08:00
Asias He
7ddf8963f5 config: Enable broadcast_rpc_address option
With this patch, start two nodes

node 1:
scylla --rpc-address 127.0.0.1 --broadcast-rpc-address 127.0.0.11

node 2:
scylla --rpc-address 127.0.0.2 --broadcast-rpc-address 127.0.0.12

On node 1:
cqlsh> SELECT rpc_address from system.peers;

 rpc_address
-------------
  127.0.0.12

which means client should use this address to connect node 2 for cql and
thrift protocol.
2015-11-24 10:07:31 +08:00
Asias He
33ef58c5c9 utils: Add get_broadcast_rpc_address and set_broadcast_rpc_address helper 2015-11-24 10:07:31 +08:00
Asias He
1e55aa38c1 storage_service: Implement is_replacing 2015-11-24 10:07:29 +08:00
Asias He
644c226d58 config: Enable replace_address and replace_address_first_boot option
It is same as

   -Dcassandra.replace_address
   -Dcassandra.replace_address_first_boot

in cassandra.
2015-11-24 10:07:24 +08:00
Asias He
bfe26ea208 config: Enable replace_token option
It is same as

   -Dcassandra.replace_token

in cassandra.

Use it as:

   $ scylla --replace-token $token1,$token2,$token3
2015-11-24 10:07:20 +08:00
Asias He
730abbc421 config: Enable replace_node option
It is same as

   -Dcassandra.replace_node

in cassandra.

Use it as:

   $ scylla --replace-node $node_uuid
2015-11-24 10:07:16 +08:00
Asias He
2513d6ddbe config: Enable load_ring_state option
It is same as

   -Dcassandra.load_ring_state

in cassandra.

Use it as:

   $ scylla --load-ring-state 0

or

   $ scylla --load-ring-state 1
2015-11-24 10:07:12 +08:00
Asias He
6e72e78e0d config: Enable join_ring option
It is same as

   -Dcassandra.join_ring

in cassandra.

Use it as:

   $ scylla --join-ring 0

or

   $ scylla --join-ring 1
2015-11-24 10:07:07 +08:00