Commit Graph

5490 Commits

Author SHA1 Message Date
Avi Kivity
d4bb1e2e35 Merge "CQL topology and status change events" from Pekka
"This series implements CQL topology and status change events. We send
out all other events except for the "leave cluster" topology event which
doesn't have the relevant code converted in storage service.

This fixes #88 and fixes #89."
2015-08-06 15:54:29 +03:00
Avi Kivity
48a1ce28fc Merge "Switch to log-structured allocator" from Tomasz 2015-08-06 15:45:39 +03:00
Pekka Enberg
c8128de562 transport/server: Hook event notifier to lifecycle events
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-06 15:26:48 +03:00
Pekka Enberg
dc4171e8d7 transport/server: Fix response::write_inet()
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-06 15:26:40 +03:00
Pekka Enberg
096e02469e transport/server: Fix response::write_byte()
The last two arguments to vector::insert() are flipped which causes us
to fill the vector with 'b' bytes of value 0x01. Switch to the single
element insertion variant to fix the issue.

Spotted by dtest push notification tests.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-06 15:26:40 +03:00
Pekka Enberg
92c813d562 transport/server: Topology and status change event encoding
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-06 15:26:40 +03:00
Pekka Enberg
f5ac4a3738 transport/event: Convert TopologyChange and StatusChange to C++
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-06 15:26:40 +03:00
Pekka Enberg
921d9386cc service/storage_service: Endpoint lifecycle subscriber hooks
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-06 15:26:40 +03:00
Pekka Enberg
8d0d60168e service: Convert IEndpointLifecycleSubscriber to C++
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-06 15:26:40 +03:00
Pekka Enberg
bbe2c52d9b service: Import IEndpointLifecycleSubscriber.java
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-06 15:26:40 +03:00
Tomasz Grabiec
cd12e365c5 mutation_partition: Avoid copy in merge_column() 2015-08-06 14:05:16 +02:00
Tomasz Grabiec
e7f1ae51b4 mutation_partition: Reduce duplication in apply() 2015-08-06 14:05:16 +02:00
Tomasz Grabiec
44910b9742 row_cache: Don't try to populate partitions already in cache 2015-08-06 14:05:16 +02:00
Tomasz Grabiec
926509525f row_cache: Switch to using LSA 2015-08-06 14:05:16 +02:00
Tomasz Grabiec
8978d0ba1a tests: Test memtable data survives full compaction 2015-08-06 14:05:16 +02:00
Tomasz Grabiec
18ec9c3643 db: Move column_family::flush() to source file 2015-08-06 14:05:16 +02:00
Tomasz Grabiec
3b92ba2857 db: Add memtable flush logging 2015-08-06 14:05:16 +02:00
Tomasz Grabiec
cda31eccf7 db: Use LSA to allocate data inside memtable 2015-08-06 14:05:16 +02:00
Tomasz Grabiec
c4acdb2068 db: Switch from bytes to managed_bytes for storing data
We need a container which can be used with compacting
allocators. "bytes" can't be used with compacting allocator because it
can't handle its external storage being moved.
2015-08-06 14:05:16 +02:00
Tomasz Grabiec
593fa725e9 dht: Relax dependencies on bytes const&
In preparation to switching to a bytes container which is not "bytes"
switch to bytes_view.
2015-08-06 14:05:16 +02:00
Tomasz Grabiec
fe4c75dee6 memtable: Remove unused find_partition() 2015-08-06 14:05:16 +02:00
Tomasz Grabiec
1046ee6e80 memtable: Remove all_partitions()
Preferred way to access the memtable is via reader.
2015-08-06 14:05:16 +02:00
Tomasz Grabiec
658c21a060 tests: Add LSA tests 2015-08-06 14:05:16 +02:00
Tomasz Grabiec
e9322b9803 utils: Introduce managed_bytes<> class
It's a bytes container which can be used with LSA.
2015-08-06 14:05:16 +02:00
Tomasz Grabiec
5508d9ff17 utils: Introduce managed_ref<> 2015-08-06 14:05:16 +02:00
Tomasz Grabiec
df6f0c35df utils: lsa: Add reclaimer hook which compacts regions 2015-08-06 14:05:15 +02:00
Tomasz Grabiec
5a9e296803 utils: lsa: Introduce log-structured allocator 2015-08-06 14:05:15 +02:00
Tomasz Grabiec
e7e79af435 tests: Add allocation_strategy_test 2015-08-06 12:52:43 +02:00
Tomasz Grabiec
581eceea41 utils: Introduce allocation_strategy 2015-08-06 12:52:43 +02:00
Avi Kivity
640db3383a Revert "tests: give cql_test_env a directory"
This reverts commit 6b2be41df00bc42331eccd423b7031b345cf979d; tests should
work without a data directory, so let's find why they don't and fix it
instead.
2015-08-06 13:09:06 +03:00
Vlad Zolotarov
a77ca1c29b cql_server::do_accepts(): Use logger.debug() for connection errors logging
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-08-06 13:03:20 +03:00
Pekka Enberg
dae1119796 database: Fix create keyspace ASan error
ASan does not like commit 05c23c7f73
("database: Add create_keyspace_on_all() helper"):

  ==8112==WARNING: AddressSanitizer failed to allocate 0x7f88b84fc690 bytes
  ==8112==AddressSanitizer's allocator is terminating the process instead of returning 0
  ==8112==If you don't like this behavior set allocator_may_return_null=1
  ==8112==Sanitizer CHECK failed: ../../../../libsanitizer/sanitizer_common/sanitizer_allocator.cc:147 ((0)) != (0) (0, 0)

I was not able to determine the source of the bug. Make ASan happy by
reverting the code movement and using the "cpu zero" trick we use for
table creation.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-06 13:02:58 +03:00
Nadav Har'El
91d804de9b storage_service: fix compilation on gcc 4.9
gcc 4.9 fails to compiles the following legal code (which compiles fine on
gcc 5.1):

	#include <unordered_set>
	std::unordered_set<int> hi() {
        	return {};
	}

Work around this problem to make Scylla compile on gcc 4.9 again.

Note that this bug is specific to std::unordered_set - we have other places
in the code code which "return {}" for std::set, and those work fine.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-08-06 13:01:31 +03:00
Avi Kivity
dec3430abb Merge "Enable more code in storage_service" from Asias
"The most visiable change to user is, now it takes less time to boot up the
non-seed node."
2015-08-06 11:07:00 +03:00
Asias He
c21fd58189 storage_service: Sleep less if we get schema version
This drops the second node boot up time from ~12 to ~7 seconds.
2015-08-06 15:23:51 +08:00
Asias He
b27201bd56 storage_service: Pass db into storage_service
It is needed for db.get_version(). I really hated to pass &db everywhere
If we had a global helper function like get_local_db(), life will be much
easier.
2015-08-06 15:23:51 +08:00
Asias He
470e9972a3 storage_service: Use set_mode
Now that set_mode is enabled, use it.
2015-08-06 15:23:51 +08:00
Asias He
5764e98b49 storage_service: Enable set_mode 2015-08-06 15:23:51 +08:00
Asias He
006554821c storage_service: Improve prepare_to_join
Use seastar::async to simplify the code. This code is only executed
during boot up, so it is fine to use seastar::async.

More commented out code are enabled.
2015-08-06 15:23:51 +08:00
Asias He
efe067fd74 storage_service: Enable more code in prepare_to_join 2015-08-06 15:23:51 +08:00
Asias He
f7b3b9646f storage_service: Stub prepare_replacement_info() 2015-08-06 15:23:51 +08:00
Asias He
93de64a061 storage_service: Add helper to get property and friends
They are used but we don't support them yet. Add stub helpers for now.
2015-08-06 15:23:51 +08:00
Glauber Costa
c2eca19737 sstable_test: fix check_toc_func
We are currently failing the sstable test. The reason is that we use the store()
function for test purposes, and that function does not store the TOC component.
It was removed by Aviccident in 3a5e3c88.

Because that function is only used for testing purposes, it doesn't need to write
the Index and Data components: we can then remove them from the list.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-08-06 10:11:55 +03:00
Glauber Costa
fb34ac7f65 database: fix scan_dir
When probing for the type, I have made the classical mistake of using
as a parameter part of a structure that is moved into the capture. That
is what broke our tests.

But also, when stat'ing, de.name will give us only the component relative to
the current path. We need to add the directory so the stat will succeed.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-08-06 10:10:56 +03:00
Glauber Costa
ece8f01d06 database: make sure a type is present.
Our directory scanner currently requires a type to be passed, and we have a
FIXME saying that we should stat when there is none. In some filesystems,
in particular, XFS, getdents won't return a type, meaning we should manually
probe it.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-08-05 22:52:07 +03:00
Tomasz Grabiec
6c9a41bfcf cql3/functions/token_fct: Add unimplemented guard for validation 2015-08-05 17:09:08 +02:00
Tomasz Grabiec
ea4f374486 mutation: Fix token() getter 2015-08-05 17:09:08 +02:00
Pekka Enberg
464c96d901 db/schema_tables: Wire up CF creation CQL event
The code is merge_tables() is a twisted maze of tricks that is hard to
restructure so that event notification can be done cleanly like with
keyspaces.

The problem there is that we need to run bunch of database operations
for the merging that really need to happen on all the shards.  To fix
the issue, lets cheat a little and simply only run CQL event
notification on cpu zero.

This seems to fix cluster schema propagation issues in urchin-dtest. I
can now run TestSimpleCluster.simple_create_insert_select_test without
any additional delays inserted into the test code.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-05 17:33:59 +03:00
Avi Kivity
891b0fe6fc Merge seastar upstream
* seastar c09488e...6f1dd3c (3):
  > net: make udp send more robust wrt. errors
  > net: remove packet constructors with template Deleter parameter
  > memory: Support for discovering allocator's address range
2015-08-05 17:23:32 +03:00
Pekka Enberg
e5ca713e72 transport/server: Fix schema change event encoding
We also need to encode the event type in the response message. Fixes the
following dtest breakage:

  cassandra.connection: ERROR: Error decoding response from Cassandra. opcode: 000c; message contents: '\x83\x00\xff\xff\x0c\x00\x00\x00\x17\x00\x07CREATED\x00\x08KEYSPACE\x00\x02ks'
  Traceback (most recent call last):
    File "/usr/lib64/python2.7/site-packages/cassandra/connection.py", line 431, in process_msg
      flags, opcode, body, self.decompressor)
    File "/usr/lib64/python2.7/site-packages/cassandra/protocol.py", line 123, in decode_response
      msg = msg_class.recv_body(body, protocol_version, user_type_map)
    File "/usr/lib64/python2.7/site-packages/cassandra/protocol.py", line 803, in recv_body
      raise NotSupportedError('Unknown event type %r' % event_type)
  NotSupportedError: Unknown event type u'CREATED'

Reported-by: Shlomi Livne <shlomi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-05 15:16:50 +03:00