Store a lw_shared_ptr<keyspace_metadata> in struct keyspace so callers
in migration manager, for example, can look it up.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
This is my turn to make the same mistake as Nadav recently made: I have
written down the expected position from the test output itself, instead
of checking the file.
A position of 0x400c0000000000 is obviously ridiculous. The file itself is not
that big. My recent patch to fix the summary broke the test, and so we need to
fix it.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
This patch is built on the previous one, where access to members of sstable
class was made possible.
Index file is being properly generated, but it's important mentioning that
promoted indexes aren't supported yet.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
The function write_datafile was renamed to write_components and made a member
of sstable class because write of components requires access to private
members. This change is an important step towards the generation of components
other than data file.
The respective testcases were adapted to the changes.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
From Avi:
"This patchset prepares for adding sstables to the read path. Because sstables
involve I/O, their APIs return futures, which means that APIs that may call
those sstable APIs also need to return futures.
This patchset uses the two-space indent + do_with + reference aliases trick
to make patches more readable. Cleanup patches will follow once it is merged."
Pekka says:
"Clean up keyspace creation functions by using keyspace_metadata. While
at it, simplify creation by moving replication strategy initialization
to database.cc. This paves the way for adding keyspace metadata lookup
functionality to database that's needed by migration manager."
Our input_stream::consume() mechanism allows feeding data from an input
stream into a consumer, piece by piece, until the consumer doesn't want
any more. It currently assumed the input can block (when reading from disk),
but the consumption is assumed to be immediate. This patch adds support for
blocking in the consumption function: The consumer now returns a future
which it promises to fulfill after consuming the given buffer.
This patch goes further by somewhat simplifying (?) the interface of the
consumer. Instead of receiving a mysterious "done" lambda the consumer
is supposed to call when done (doesn't want any more input), the consumer
now returns a future<optional<temporary_buffer<char>>, which means:
1. The future is fulfilled when the consumer is done with this buffer
and either wants more - or wants to stop.
2. If the consumer wants to stop, it returns the *remaining* part of the
buffer it didn't want to process (this will be pushed back into the
input stream).
3. If the consumer is not done, and wants to consume more, it returns an
unset optional.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Initialize replication strategy when keyspace is being created now that
we have access to keyspace_metadata.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Use the keyspace_metadata type for keyspace creation functions. This is
needed to be able to have a mapping from keyspace name to keyspace
metadata for various call-sites.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Follow the naming convention set by user_types_metadata and rename
ks_meta_data to keyspace_metadata.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Pekka says:
"There's a user_types_metadata type defined in database.hh. Use that and
remove the left-over ut_meta_data from initial CQL translations."
Conflicts:
thrift/handler.cc
There's a user_types_metadata type in database.hh. Use it and drop the
left-over ut_meta_data from Origin.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
This patchset adds thread support to seastar. Threads can use the normal
seastar API, and may also "block" by calling future<>::get() on an
unavailable future. Synchronous and asynchronous code may be intermixed.
Threads may not issue blocking operating system calls.
Unfortunately, for some unknown reason, my compiler is not locally warning me
of signe comparison mismatches, but our builder's is.
We have a currently build failure that is hopefully fixed by this patch.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
The implementation of sleep() looks like a game of Seastar golf - doing
something in the minimum number of lines possible :-) Unfortunately, it
looks very clever, but not quite right. sleep() usually works correctly,
but the sanitizer (in the debug build) catches a use after free.
The problem was that we delete an object which contains a timer which
contains the callback (and std::function) - from inside this callback.
The workaround in this patch is to use our future chaining to only delete
the sleeper object after its future became ready - and at that point, none
of the sleeper object or code is needed any more.
This patch also includes a regression test for this issue. The test looks
silly (just sleeps and checks nothing), but in the debugging build it
failed (with a sanitizer reporting use-after-free) before this patch.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
send_message() and send_message_oneway() are almost identical, implement
the later in terms of the former. The patch also fixes send_message() to
work properly with MsgIn = void.
Reviewed-by: Asias He <asias@cloudius-systems.com>
Reduces coupling. User's should not rely on the fact that it's an
std::map<>. It also allows us to extend row's interface with
domain-specific methods, which are a lot easier to discover than free
functions.
If method doesn't want to share schema ownership it doesn't have to
take it by shared pointer. The benefit is that it's slightly cheaper
and those methods may now be called from places which don't own
schema.
Separating the initial value (and accumulator) from the reducer function
can result in simpler invocations.
Unfortunately, the name conflicts with another variant, so we have to name
the method map_reduce0.
Should fix use-after-free when a frozen_mutation is applied to the
local shard.
Includes two adjustments to urchin collectd usage from Calle:
- Updated thrift collectd registration to use proper move semantics
- Commitlog: Fix collectd registration to use move semantics + test
Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
From Pekka:
"This series implements a Maps.difference() function in C++, changes
storage_proxy::query_local() to not return foreign_ptr>, and finally
changes the keyspace merging code to follow Origin."
This implements Maps.difference() helper function in C++. We need it to
translate various Origin call-sites that use it. The implementation is
written from scratch with Guava code used as reference to preserve
semantics.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
I didn't want to create another executable for it because doing so
would take a lot more of disk space. The motivation behind this
change is to debloat the sstables test file, which is growing very
quickly. Suggested by Glauber Costa.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
