Commit Graph

53948 Commits

Author SHA1 Message Date
Raphael S. Carvalho
7dc9ba1714 sstables: write summary position field as little endian
The field entries must be written in memory order.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-05-17 11:55:35 +03:00
Avi Kivity
a4880ac0c5 tests: convert mutation_test to use SEASTAR_TEST_CASE()
Following changes introduce a dependency on engine().cpu_id(), which requires
SEASTAR_TEST_CASE().
2015-05-17 10:31:22 +03:00
Avi Kivity
ad54edb047 tests: use memtable instead of column_family where appropriate
Requires less configuration.
2015-05-17 10:31:22 +03:00
Avi Kivity
40c2d91cd8 db: add memtable::find_or_create_row_slow()
Useful for tests that do not need a column_family.
2015-05-17 10:31:22 +03:00
Glauber Costa
81e4c28898 sstable: fix signed conversion build problems
Unfortunately, for some unknown reason, my compiler is not locally warning me
of signe comparison mismatches, but our builder's is.

We have a currently build failure that is hopefully fixed by this patch.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-14 17:37:37 +03:00
Avi Kivity
830765b959 Merge branch 'shlomi/support_net_service_stop_v4_rebase' of github.com:cloudius-systems/seastar-dev into db
Support stop of gossiper and failure_detector, from Shlomi.

Reviewed-by: Asias He <asias@cloudius-systems.com>
2015-05-14 16:40:27 +03:00
Asias He
fbb3d9d434 main: Init messaging_service before thrift and cql server
Init messaging_service before thrift and cql server, since
thrift/cql may not rely on gossip having all the data, but they may
query.
2015-05-14 15:38:05 +03:00
Shlomi Livne
7c879ea842 Add a test to start/stop messaging_service,gossiper,failure-detector
Signed-off-by: Shlomi Livne <shlomi@cloudius-systems.com>
2015-05-14 17:01:26 +08:00
Shlomi Livne
0ad0a02d93 Change failure_detector registration of listeners to accept a ptr
Signed-off-by: Shlomi Livne <shlomi@cloudius-systems.com>
2015-05-14 17:01:18 +08:00
Shlomi Livne
89b9443127 Adding gossiper stop and internal handler::stop
Signed-off-by: Shlomi Livne <shlomi@cloudius-systems.com>
2015-05-14 17:01:10 +08:00
Shlomi Livne
a73adc39f3 Rename gossiper stop to shutdown to allow creation of stop() needed for distributed<>::stop
Signed-off-by: Shlomi Livne <shlomi@cloudius-systems.com>
2015-05-14 17:01:01 +08:00
Shlomi Livne
fbeafa67cb Add failure_dector stop() that will be called by distributed<>::stop
Signed-off-by: Shlomi Livne <shlomi@cloudius-systems.com>
2015-05-14 17:00:46 +08:00
Nadav Har'El
902d5b21ca sleep: Fix use-after-free in sleep()
The implementation of sleep() looks like a game of Seastar golf - doing
something in the minimum number of lines possible :-) Unfortunately, it
looks very clever, but not quite right. sleep() usually works correctly,
but the sanitizer (in the debug build) catches a use after free.

The problem was that we delete an object which contains a timer which
contains the callback (and std::function) - from inside this callback.

The workaround in this patch is to use our future chaining to only delete
the sleeper object after its future became ready - and at that point, none
of the sleeper object or code is needed any more.

This patch also includes a regression test for this issue. The test looks
silly (just sleeps and checks nothing), but in the debugging build it
failed (with a sanitizer reporting use-after-free) before this patch.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-14 11:26:58 +03:00
Asias He
64d9fca4ef main: Move messaging_service init code into a helper function 2015-05-14 11:10:29 +03:00
Avi Kivity
b1d604d584 Merge branch 'sst2mutation-v4' of github.com:glommer/urchin into db
sstable -> mutation path, from Glauber.
2015-05-14 10:50:57 +03:00
Avi Kivity
159fe294f2 Merge branch 'gossip' into db
Gossip integration, from Asias.
2015-05-14 10:37:58 +03:00
Asias He
1adacecb41 main: Fix indentation
Spotted when reading the code.
2015-05-14 10:37:27 +03:00
Asias He
bb87afcd1f main: Wire up gossip and friends
Integrate messaging_service and failure_detector and gossiper into urchin.

To start a 3 nodes cluster on a single host:

./seastar --listen-address 127.0.0.1 --seed-provider-parameters 127.0.0.1
./seastar --listen-address 127.0.0.2 --seed-provider-parameters 127.0.0.1
./seastar --listen-address 127.0.0.3 --seed-provider-parameters 127.0.0.1
2015-05-14 10:37:26 +03:00
Asias He
979bb60d78 gms: Resolve localhost in inet_address
In db:config, "localhost" is used as the default IP address for
listen_address, rpc_address. We do not have a name resolver at the
moment.

Add a minimal resolver for localhost for now.
2015-05-14 10:37:26 +03:00
Asias He
827300ebe1 gossip: Fix sending ECHO message
The msg parameter is missing.

Fix a bug where Node B does not not recognize Node A.

Node A
$ ./gossip --seed 127.0.0.1  --listen-address 127.0.0.1

Node B
$ ./gossip --seed 127.0.0.1  --listen-address 127.0.0.2

The issue is that in gossiper::mark_alive(), the parameter for ECHO
message is wrong and after commit 1a8c4b75f5 (message: do not erase client's
rpc call type), we deduce the rpc handler using the parameters supplied
to messaging_service::send_message(), so we will use a wrong handler for
the ECHO message and the message will never reply thus we never mark the
peer node alive.

empty_msg was used as a placeholder when messaging_service does not handle void
return type correctly. Since we support it now, drop it.
2015-05-14 10:37:25 +03:00
Calle Wilund
46480ca836 config: Add support for maps in command lines + fix seed_provider
* Allow boost::config to translate string_maps via iostream
* Special case seed_provider (struct) into two parameters (for lack of a
good way to express it on command line)

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2015-05-14 10:36:13 +03:00
Glauber Costa
8de096ba81 sstable: add tests for mutation transformation
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-13 18:28:37 -04:00
Glauber Costa
e3f87033b7 sstables: convert collections to mutation
We do our best to avoid serializing every cell we see. We will accumulate them
all in an internal data structure, and then finally serialize it to our internal
format all at the same time.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-13 17:40:20 -04:00
Glauber Costa
53b6162976 sstables: rework collection detection code
We currently have code (which is wrong, btw), to throw an exception
when we find a collection. Replace that code with something that aside
from working, has the added benefit of actually capturing the collection
information correctly. This is the first step into implementing conversion
for collections.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-13 17:39:43 -04:00
Glauber Costa
e7ab34d644 sstables: handle row tombstone
These cases were previously unhandled.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-13 17:39:36 -04:00
Glauber Costa
d31b12a3fd sstables: implement conversion of range tombstone
Only the simple case is supported so far. We will throw where unsupported.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-13 17:38:56 -04:00
Glauber Costa
d00c228409 sstables: turn some column parser fields static
They don't depend on internal state (or can be made not to). We want to reuse
those fields later for detecting static columns in tombstones.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-13 17:14:03 -04:00
Glauber Costa
c4dfefe3d1 sstable key: accept markers other than zero
Composites have an extra byte at the end of each element. The middle bytes
are expected to be zero, but the last one may not always be. In cases like
those of a range tombstone, we will have a marker with special significance.

Since we are exploding the composite into its components, we don't actually
need to retrieve it. We merely need to make sure that the marker is the one
we expect.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-13 17:14:03 -04:00
Glauber Costa
34c6cca845 sstable types: convert a deletion time to a tombstone
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-13 17:14:03 -04:00
Glauber Costa
0a4a5914a8 sstables: add helper method for deletion_time
That should make it easier for code to test if a cell or range is live whenever
needed.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-13 17:14:03 -04:00
Glauber Costa
7ca7b69761 sstables: do not short-cut cql row marker
As Nadav and Tomek both pointed out, the \0\0\0 is just a special case of a row
marker for a schema with no clustering keys. There is no need for us to handle
that separately.

And in fact, short-cutting it means that we will have to take care to update
the row timestamp in two places - which we weren't doing.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-13 17:14:03 -04:00
Glauber Costa
2fba948ad8 sstables: move timestamps to signed integer
This is to follow Origin

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-13 17:14:02 -04:00
Glauber Costa
590abb800e sstables: pass a key_view instead of bytes_view to consume_row_start
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-13 17:14:02 -04:00
Glauber Costa
4dde0386de sstable key: provide equality and inequality operator
So we don't have to convert to a bytes view unnecessarily.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-13 17:14:02 -04:00
Glauber Costa
936bb2c97d sstables: small fixes for mutation converted
1) pass schema by const reference to avoid bumping shared_ptr ref count
2) move construct the atomic cells.
3) typo fixups

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-13 17:14:02 -04:00
Glauber Costa
f190beaa4e sstables: return a mutation_opt instead of a pointer
As suggested by Avi, we can return an actual mutation by moving it out of our
consumer. We will encapsulate it within an optional, to handle the cases where
the mutation cannot be found.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-13 17:13:55 -04:00
Avi Kivity
fa344eaeb8 Merge branch 'master' of github.com:cloudius-systems/urchin into sstable-excl 2015-05-13 18:06:19 +03:00
Avi Kivity
520cd91e90 sstables: don't overwrite datafiles
Prevent data loss resulting from erronously overwrting an sstable.

Reviewed-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-05-13 18:05:41 +03:00
Gleb Natapov
8d9fb8a96c message: consolidate send_message() and send_message_oneway()
send_message() and send_message_oneway() are almost identical, implement
the later in terms of the former. The patch also fixes send_message() to
work properly with MsgIn = void.

Reviewed-by: Asias He <asias@cloudius-systems.com>
2015-05-13 13:41:24 +03:00
Avi Kivity
5c3f538b29 Merge branch 'master' of github.com:cloudius-systems/seastar into db 2015-05-13 13:24:07 +03:00
Avi Kivity
2f10793898 core: add open_flags::exclusive (O_EXCL) 2015-05-13 13:23:18 +03:00
Avi Kivity
8c09dbdbe8 Merge branch 'tgrabiec/cleanups' of github.com:cloudius-systems/seastar-dev into db
Database core cleanups, from Tomasz.
2015-05-13 11:08:21 +03:00
Avi Kivity
097aeb261e Merge branch 'master' of github.com:cloudius-systems/seastar into db 2015-05-13 10:57:53 +03:00
Avi Kivity
bf527c2842 future: add range-based parallel_for_each variant 2015-05-13 10:53:13 +03:00
Tomasz Grabiec
8fedebd166 db: Add clarifying description to query_command and partition_slice 2015-05-13 08:56:54 +02:00
Tomasz Grabiec
f656ae8ed4 db: Encapsulate deletable_row fields 2015-05-13 08:56:54 +02:00
Tomasz Grabiec
dbc40dfb09 db: Encapsulate the "row" class
Reduces coupling. User's should not rely on the fact that it's an
std::map<>.  It also allows us to extend row's interface with
domain-specific methods, which are a lot easier to discover than free
functions.
2015-05-13 08:56:54 +02:00
Tomasz Grabiec
56bea440a7 mutation_partition: Pass schema by const& where applicable
If method doesn't want to share schema ownership it doesn't have to
take it by shared pointer. The benefit is that it's slightly cheaper
and those methods may now be called from places which don't own
schema.
2015-05-13 08:56:54 +02:00
Tomasz Grabiec
a5ff3818f7 mutation_partition: Visually segregate mutators from getters 2015-05-13 08:56:53 +02:00
Tomasz Grabiec
431958c621 keys: Fix make_empty()
make_empty() is used from thrift to create a clustering_key for a
table's row without clustering key columns. The implementation was
misleading because it seemed to be handling any number of components in
the key while only no-component case is supposed to work.
2015-05-13 08:56:53 +02:00