Commit Graph

3272 Commits

Author SHA1 Message Date
Glauber Costa
d00c228409 sstables: turn some column parser fields static
They don't depend on internal state (or can be made not to). We want to reuse
those fields later for detecting static columns in tombstones.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-13 17:14:03 -04:00
Glauber Costa
c4dfefe3d1 sstable key: accept markers other than zero
Composites have an extra byte at the end of each element. The middle bytes
are expected to be zero, but the last one may not always be. In cases like
those of a range tombstone, we will have a marker with special significance.

Since we are exploding the composite into its components, we don't actually
need to retrieve it. We merely need to make sure that the marker is the one
we expect.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-13 17:14:03 -04:00
Glauber Costa
34c6cca845 sstable types: convert a deletion time to a tombstone
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-13 17:14:03 -04:00
Glauber Costa
0a4a5914a8 sstables: add helper method for deletion_time
That should make it easier for code to test if a cell or range is live whenever
needed.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-13 17:14:03 -04:00
Glauber Costa
7ca7b69761 sstables: do not short-cut cql row marker
As Nadav and Tomek both pointed out, the \0\0\0 is just a special case of a row
marker for a schema with no clustering keys. There is no need for us to handle
that separately.

And in fact, short-cutting it means that we will have to take care to update
the row timestamp in two places - which we weren't doing.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-13 17:14:03 -04:00
Glauber Costa
2fba948ad8 sstables: move timestamps to signed integer
This is to follow Origin

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-13 17:14:02 -04:00
Glauber Costa
590abb800e sstables: pass a key_view instead of bytes_view to consume_row_start
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-13 17:14:02 -04:00
Glauber Costa
4dde0386de sstable key: provide equality and inequality operator
So we don't have to convert to a bytes view unnecessarily.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-13 17:14:02 -04:00
Glauber Costa
936bb2c97d sstables: small fixes for mutation converted
1) pass schema by const reference to avoid bumping shared_ptr ref count
2) move construct the atomic cells.
3) typo fixups

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-13 17:14:02 -04:00
Glauber Costa
f190beaa4e sstables: return a mutation_opt instead of a pointer
As suggested by Avi, we can return an actual mutation by moving it out of our
consumer. We will encapsulate it within an optional, to handle the cases where
the mutation cannot be found.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-13 17:13:55 -04:00
Avi Kivity
fa344eaeb8 Merge branch 'master' of github.com:cloudius-systems/urchin into sstable-excl 2015-05-13 18:06:19 +03:00
Avi Kivity
520cd91e90 sstables: don't overwrite datafiles
Prevent data loss resulting from erronously overwrting an sstable.

Reviewed-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-05-13 18:05:41 +03:00
Gleb Natapov
8d9fb8a96c message: consolidate send_message() and send_message_oneway()
send_message() and send_message_oneway() are almost identical, implement
the later in terms of the former. The patch also fixes send_message() to
work properly with MsgIn = void.

Reviewed-by: Asias He <asias@cloudius-systems.com>
2015-05-13 13:41:24 +03:00
Avi Kivity
5c3f538b29 Merge branch 'master' of github.com:cloudius-systems/seastar into db 2015-05-13 13:24:07 +03:00
Avi Kivity
2f10793898 core: add open_flags::exclusive (O_EXCL) 2015-05-13 13:23:18 +03:00
Avi Kivity
8c09dbdbe8 Merge branch 'tgrabiec/cleanups' of github.com:cloudius-systems/seastar-dev into db
Database core cleanups, from Tomasz.
2015-05-13 11:08:21 +03:00
Avi Kivity
097aeb261e Merge branch 'master' of github.com:cloudius-systems/seastar into db 2015-05-13 10:57:53 +03:00
Avi Kivity
bf527c2842 future: add range-based parallel_for_each variant 2015-05-13 10:53:13 +03:00
Tomasz Grabiec
8fedebd166 db: Add clarifying description to query_command and partition_slice 2015-05-13 08:56:54 +02:00
Tomasz Grabiec
f656ae8ed4 db: Encapsulate deletable_row fields 2015-05-13 08:56:54 +02:00
Tomasz Grabiec
dbc40dfb09 db: Encapsulate the "row" class
Reduces coupling. User's should not rely on the fact that it's an
std::map<>.  It also allows us to extend row's interface with
domain-specific methods, which are a lot easier to discover than free
functions.
2015-05-13 08:56:54 +02:00
Tomasz Grabiec
56bea440a7 mutation_partition: Pass schema by const& where applicable
If method doesn't want to share schema ownership it doesn't have to
take it by shared pointer. The benefit is that it's slightly cheaper
and those methods may now be called from places which don't own
schema.
2015-05-13 08:56:54 +02:00
Tomasz Grabiec
a5ff3818f7 mutation_partition: Visually segregate mutators from getters 2015-05-13 08:56:53 +02:00
Tomasz Grabiec
431958c621 keys: Fix make_empty()
make_empty() is used from thrift to create a clustering_key for a
table's row without clustering key columns. The implementation was
misleading because it seemed to be handling any number of components in
the key while only no-component case is supposed to work.
2015-05-13 08:56:53 +02:00
Avi Kivity
ee5b6d0f2e Merge branch 'penberg/sstable-cleanups' of github.com:cloudius-systems/seastar-dev into db
Sstable cleanups, from Pekka.
2015-05-12 19:03:08 +03:00
Avi Kivity
3e04c30c00 Merge branch 'penberg/cql-cleanups' of github.com:cloudius-systems/seastar-dev into db
cql cleanups, from Pekka.
2015-05-12 17:04:10 +03:00
Avi Kivity
f3c8994535 distributed: add map_reduce() variant accepting an initial value
Separating the initial value (and accumulator) from the reducer function
can result in simpler invocations.

Unfortunately, the name conflicts with another variant, so we have to name
the method map_reduce0.
2015-05-12 16:57:50 +03:00
Pekka Enberg
f2438d0124 sstables: Formatting cleanups
Clean up some formatting issues spotted while reading the code.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-12 16:57:24 +03:00
Pekka Enberg
5c515dcad8 sstables: Define public members first
Shuffle code in sstables.hh so that public members are defined first.
Makes the code easier to digest.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-12 16:52:15 +03:00
Pekka Enberg
11b633208d cql3: Remove Java imports from C++ files
Remove left-over Java imports from files that are already translated to
C++.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-12 16:41:12 +03:00
Pekka Enberg
c6da7e6844 cql3: Use std::nullopt constant
There's a std::nullopt constant for unset optionals. Use it.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-12 16:37:29 +03:00
Pekka Enberg
d50139351f cql3: Use pragma once everywhere
There's no benefit to using C include guards so switch to pragma once
everywhere for consistency.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-12 16:32:56 +03:00
Avi Kivity
b300a05468 Merge branch 'master' of github.com:cloudius-systems/urchin into db 2015-05-12 15:07:16 +03:00
Avi Kivity
548646d4ba Merge branch 'master' of github.com:cloudius-systems/seastar into db
Should fix use-after-free when a frozen_mutation is applied to the
local shard.

Includes two adjustments to urchin collectd usage from Calle:

  - Updated thrift collectd registration to use proper move semantics
  - Commitlog: Fix collectd registration to use move semantics + test

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2015-05-12 14:47:07 +03:00
Avi Kivity
50d65f1d86 smp: protect lifetime of function parameter to submit_to()
When submit_to() calls to a different core, and when the function to
be executed is a temporary (the usual case), we copy it to the heap for
the duration of execution.  However when the function happens to execute
locally, we don't copy it, which can lead to a use-after-free if the function
defers.

Fix by detecting the case of local execution of a temporary function, and
copying it to the heap in that case.
2015-05-12 14:33:18 +03:00
Tomasz Grabiec
c779fa880f Merge branch 'penberg/keyspace-merging-improvements/v2' from seastar-dev.git
From Pekka:

 "This series implements a Maps.difference() function in C++, changes
 storage_proxy::query_local() to not return foreign_ptr>, and finally
 changes the keyspace merging code to follow Origin."
2015-05-12 13:08:02 +02:00
Pekka Enberg
96a1955cea db/legacy_schema_tables: Use map_difference() instead of open-coding it
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-12 13:18:56 +03:00
Pekka Enberg
d3f9af7239 Make storage_proxy::query_local() return bare lw_shared_ptr<>
Don't return foreign_ptr<> which is not copyable so that we can use
map_difference for maps with result_set in them.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-12 13:18:56 +03:00
Avi Kivity
6173d1af62 Merge branch 'calle/scollectd-fixes' of github.com:cloudius-systems/seastar-dev
collectd fixes, from Calle.
2015-05-12 12:42:19 +03:00
Calle Wilund
ad5feda8ae Collectd: Fix registration/deregistration and reg. anchor managment
Obviously, I was sleeping or something when I wrote the reg/unreg code, since
using copy semantics for anchors is equivalent with double unregistrations.
Luckily, unregister was broken as well, so counters did stay active. Which
however broke things once actual non-persistent counters were added. Doh.

* Anchors must be non-copyable
* Above makes creating std::vector<registration> from initializer list
tricky, so added helper type "registrations" which inherits vector<reg>
but constructs from initializer_list<type_instance_id>, avoiding illegal
copying.
* Both register and unregister were broken (map semantics does not overwrite
on insert, only [] or iterator operation).
* Modified the various registration callsites to use registrations and move
semantics.
2015-05-12 11:30:44 +02:00
Calle Wilund
2646fa188f Collectd: Add virtual destructor to value_list
Since we bind actual std::functions etc, a virtual destructor is needed.
2015-05-12 11:30:44 +02:00
Calle Wilund
e941b5a378 Collectd: Make sure get_instance_ids only returns active id:s 2015-05-12 11:30:44 +02:00
Calle Wilund
db9869639c Collectd: change string types to sstring 2015-05-12 11:30:44 +02:00
Pekka Enberg
a264f42bd6 Add map_difference::difference() helper
This implements Maps.difference() helper function in C++. We need it to
translate various Origin call-sites that use it. The implementation is
written from scratch with Guava code used as reference to preserve
semantics.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-12 10:19:52 +03:00
Raphael S. Carvalho
7bb43cb961 gitorderfile: make changes into *.py files appear first
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-05-12 10:13:25 +03:00
Raphael S. Carvalho
e1de49e233 tests: move sstable data file generation testcases to another file
I didn't want to create another executable for it because doing so
would take a lot more of disk space. The motivation behind this
change is to debloat the sstables test file, which is growing very
quickly. Suggested by Glauber Costa.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-05-11 19:37:29 +03:00
Avi Kivity
7630fe332d db: pass correct mutation size to commitlog
Use serialized_size() instead of reprentation().size(), to account for the
size header.
2015-05-11 19:19:23 +03:00
Avi Kivity
dff2b74a60 Merge branch 'master' of github.com:cloudius-systems/seastar into db 2015-05-11 18:10:20 +03:00
Avi Kivity
22e261e24b Merge branch 'sstables-write' of github.com:raphaelsc/urchin into db
Sstable support for expiring and tombstone cells, from Raphael.
2015-05-11 18:09:41 +03:00
Avi Kivity
4cea444fb2 collectd: avoid using an exception as end-of-packet indicator
Collectd uses an exception to signal that the buffer space in the packet
is exhausted and a new packet needs to be started.  This violates the
"exceptions are for exceptional conditions" guideline, but more practically,
makes it hard to use the gdb "catch throw" command to trap exceptions.

Fix by using a data member to store the overflow condition instead.
2015-05-11 18:02:41 +03:00