send_message() and send_message_oneway() are almost identical, implement
the later in terms of the former. The patch also fixes send_message() to
work properly with MsgIn = void.
Reviewed-by: Asias He <asias@cloudius-systems.com>
Reduces coupling. User's should not rely on the fact that it's an
std::map<>. It also allows us to extend row's interface with
domain-specific methods, which are a lot easier to discover than free
functions.
If method doesn't want to share schema ownership it doesn't have to
take it by shared pointer. The benefit is that it's slightly cheaper
and those methods may now be called from places which don't own
schema.
Separating the initial value (and accumulator) from the reducer function
can result in simpler invocations.
Unfortunately, the name conflicts with another variant, so we have to name
the method map_reduce0.
Should fix use-after-free when a frozen_mutation is applied to the
local shard.
Includes two adjustments to urchin collectd usage from Calle:
- Updated thrift collectd registration to use proper move semantics
- Commitlog: Fix collectd registration to use move semantics + test
Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
From Pekka:
"This series implements a Maps.difference() function in C++, changes
storage_proxy::query_local() to not return foreign_ptr>, and finally
changes the keyspace merging code to follow Origin."
This implements Maps.difference() helper function in C++. We need it to
translate various Origin call-sites that use it. The implementation is
written from scratch with Guava code used as reference to preserve
semantics.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
I didn't want to create another executable for it because doing so
would take a lot more of disk space. The motivation behind this
change is to debloat the sstables test file, which is growing very
quickly. Suggested by Glauber Costa.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Introduce frozen_mutation, from Tomasz:
"The immediate motivation for introducing frozen_mutation is inability to
deserialize current "mutation" object, which needs schema reference at the
time it's constructed. It needs schema to initialize its internal maps with
proper key comparators, which depend on schema. We can't lookup schema before
we deserialize column family ID. Another problem is that even if we had the ID
somehow, low level RPC layer doesn't know how to lookup the schema.
This form is primarily destined to be sent over the network channel. Data can
be wrapped in frozen_mutation without schema information, the schema is only
needed to access some of the fields.
frozen_mutation supports reading via visiting, without having to explode it
into an object graph. Because of that, application should be more efficient
because of fewer cache misses.
We also don't have to serialize it back for the commit log, it's serialized
only once on the coordinator node.
Different serialization formats are not supported yet."
Reviewed-by: Pekka Enberg <penberg@cloudius-systems.com>
When the temporary buffer has enough data for a uint64 to be
consumed, we readily consume it.
The problem is that we were wrongly storing the uint64 into
a uint32 variable.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
For fluent asserting mutations:
assert_that(what_we_got).is_equal_to(what_we_expect);
We could extend that in future with more specific checks, like
has_cell() etc.
The immediate motivation for introducing frozen_mutation is inability
to deserialize current "mutation" object, which needs schema reference
at the time it's constructed. It needs schema to initialize its
internal maps with proper key comparators, which depend on schema.
frozen_mutation is an immutable, compact form of a mutation. It
doesn't use complex in-memory strucutres, data is stored in a linear
buffer. In case of frozen_mutation schema needs to be supplied only at
the time mutation partition is visited. Therefore it can be trivially
deserialized without schema.
Fix inconsistency between handler's parameter types and what is actually
passed to rpc call. Also get rid of unneeded local variables.
Reviewed-by: Asias He <asias@cloudius-systems.com>
Origin does that, so should we. Both ttl and expiry time are stored in
sstables. The value of ttl seems to be used to calculate the read
digest (expiry is not used for that).
The API for creating atomic_cells changed a bit.
To create a non-expiring cell:
atomic_cell::make_live(timestamp, value);
To create an expiring cell:
atomic_cell::make_live(timestamp, value, expiry, ttl);
or:
// Expiry is calculated based on current clock reading
atomic_cell::make_live(timestamp, value, ttl_optional);
Generic read-all-stream from a commit log segmen file.
Provides a byte view for each data entry, doing CRC checks and padding skips.
Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
In preparation for multiple memtables, move column_family::partitions into
its own class, and forward relevant calls from column_family.
A testonly_all_memtables() function was added to support sstable_test.
Fix replication strategy class lookup to support unqualified names such
as "SimpleStrategy". The AbstractReplicationStrategy.getClass() method
in origin does the same thing.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Read mutations from sstables, from Glauber.
Conflicts:
sstables/key.cc
sstables/key.hh
sstables/sstables.cc
sstables/sstables.hh
[avi: adjust sstables/partition.cc:149 for ambiguity due to new
basic_sstring range constructor]
From Avi:
'''
start() returns immediately, then the listen object is destroyed (it is
on the stack), then the lambda runs. [&] is also very dangerous.
'''
This fixes a bug in initialization of messaging_service.
[asias@hjpc urchin]$ ./message --listen-address 127.0.0.100
listen =127.0.0.100
protocol::server::server: addr=127.0.0.100:7000
opts.reuse_address=1, _reuseport=1
messaging_service on cpu_id=0, ip=127.0.0.100,
listen_address=127.0.0.100
protocol::server::server: addr=0.36.145.24:7000
opts.reuse_address=1, _reuseport=1
messaging_service on cpu_id=1, ip=0.36.145.24,
listen_address=0.36.145.24
Messaging server listening on port 7000 ...
Each testcase generates a data file from a given schema + a mutation,
and then check the correctness of the generated data file.
The data used to check the correctness of the generated data file
is from C* data file itself.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
There is one gossiper instance per node and it runs on cpu0 only. We can
not guarantee there will always be a core to core tcp connection within
messaging service, so messaging service needs to listen on all cpus.
When a remote node connects to local node with a connection bound to cpu
other than cpu0, we need to forward this message to cpu0.
message_service: create object using placement new()
Object reference may point to a unit member that was not properly
created, so assign will not work on it. Use placement new() instead.
