The comment says the directory is created on the local shard only, but when
this code was converted, it ended up being moved inside the invoke_on_all lambda,
which means now it is called from all shards. Fix it by reorganizing the code.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
There is only one user in tree, and as Tomek pointed out, it is buggy. The
reason is that ksm is a shard-local structure, and it is currently used
indiscriminately by all shards which can easily lead to problems.
We could fix it with some tricks, but it is way better and safer to make sure
the callers are doing it right instead. There is only one caller, so let's fix
that.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Fix keyspace strategy options to preserve key-value ordering by
switching to std::map. We need this to be able to store the map in
database as JSON because unordered maps can cause the schema merging
code to attempt a keyspace update, which we don't support, even though
the values did not change.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
The original code just assigns value to _ks_name which is a sstring
so the try { } catch { } clause only gave the wrong impression that
something is checked.
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
Store a lw_shared_ptr<keyspace_metadata> in struct keyspace so callers
in migration manager, for example, can look it up.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
When thrift sees an exception that was not declared as part of the interface,
it wraps it using std::exception::what() for the exception text. This is
often cryptic, so add an "Internal server error" prefix.
From Avi:
"This patchset prepares for adding sstables to the read path. Because sstables
involve I/O, their APIs return futures, which means that APIs that may call
those sstable APIs also need to return futures.
This patchset uses the two-space indent + do_with + reference aliases trick
to make patches more readable. Cleanup patches will follow once it is merged."
Initialize replication strategy when keyspace is being created now that
we have access to keyspace_metadata.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Use the keyspace_metadata type for keyspace creation functions. This is
needed to be able to have a mapping from keyspace name to keyspace
metadata for various call-sites.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Follow the naming convention set by user_types_metadata and rename
ks_meta_data to keyspace_metadata.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Pekka says:
"There's a user_types_metadata type defined in database.hh. Use that and
remove the left-over ut_meta_data from initial CQL translations."
Conflicts:
thrift/handler.cc
There's a user_types_metadata type in database.hh. Use it and drop the
left-over ut_meta_data from Origin.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
This is slightly awkwards, since the directory structure is not sharded.
This requires some processing to occur outside the shard, while the rest
is sharded.
Returning a reference to the keyspace is dangerous in that the keyspace can
be moved away, when we start futurizing the add_keyspace() process. Make
it return void and look up the keyspace at the point of use.
Reduces coupling. User's should not rely on the fact that it's an
std::map<>. It also allows us to extend row's interface with
domain-specific methods, which are a lot easier to discover than free
functions.
The immediate motivation for introducing frozen_mutation is inability
to deserialize current "mutation" object, which needs schema reference
at the time it's constructed. It needs schema to initialize its
internal maps with proper key comparators, which depend on schema.
frozen_mutation is an immutable, compact form of a mutation. It
doesn't use complex in-memory strucutres, data is stored in a linear
buffer. In case of frozen_mutation schema needs to be supplied only at
the time mutation partition is visited. Therefore it can be trivially
deserialized without schema.
Origin does that, so should we. Both ttl and expiry time are stored in
sstables. The value of ttl seems to be used to calculate the read
digest (expiry is not used for that).
The API for creating atomic_cells changed a bit.
To create a non-expiring cell:
atomic_cell::make_live(timestamp, value);
To create an expiring cell:
atomic_cell::make_live(timestamp, value, expiry, ttl);
or:
// Expiry is calculated based on current clock reading
atomic_cell::make_live(timestamp, value, ttl_optional);
A lookup can cause several data sources to be merged, in which case we will
have to return a temporary (containing data from all the data sources).
For simplicity, we start by always returning a temporary.
Ensure that read-side accessors are const. This is important in preparation
for multiple memtables (and later, sstables) since a read-side
mutation_partition may be a temporary object coming from multiple memtables
(and sstables) while a write-side mutation_partition is guaranteed to belong
to a single memtable (and thus, not be temporary).
Since writers will want non-const mutation_partitions to write to, they won't
be able to use the read-side accessors by accident.
* A commitlog is created in "work" dirs when initing the db
from a datadir. However, since we have neither disk data storage,
nor replay capability yet (and no real db config), the settings
are basically to just write in-memory serialization, write them to
disk and then discard them. So in fact, pointless. But at least using
the log...
* Moved the actual "apply" of mutation into database. If a commitlog
is active, add an entry to it before applying mutation.
Partitions should be ordered using Origin's ordering, which is first
by token, then by Origin's representation of the key. That is the
natural ordering of decorated_key.
This also changes mutation class to hold decorated_key, to avoid
decoration overhead at different layers.
bytes and sstring are distinct types, since their internal buffers are of
different length, but bytes_view is an alias of sstring_view, which makes
it possible of objects of different types to leak across the abstraction
boundary.
Fix this by making bytes a basic_sstring<int8_t, ...> instead of using char.
int8_t is a 'signed char', which is a distinct type from char, so now
bytes_view is a distinct type from sstring_view.
uint8_t would have been an even better choice, but that diverges from Origin
and would have required an audit.
This patch converts (for very small value of 'converts') some
replication related classes. Only static topology is supported (it is
created in keyspace::create_replication_strategy()). During mutation
no replication is done, since messaging service is not ready yet,
only endpoints are calculated.
* database now holds all keyspace + column family object
* column families are mapped by uuid, either generated or explicit
* lookup by name tuples or uuid
* finder functions now return refs + throws on missing obj
