Origin's --seed-provider-parameters format is seeds=<ip1>,<ip2>,<ip3> to
align with yaml configuration file format and command line options a
different seperator must be used instead of "," - switched to using ";"
Signed-off-by: Shlomi Livne <shlomi@cloudius-systems.com>
Fix keyspace strategy options to preserve key-value ordering by
switching to std::map. We need this to be able to store the map in
database as JSON because unordered maps can cause the schema merging
code to attempt a keyspace update, which we don't support, even though
the values did not change.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
- Forbid explicit snitch creation with constructor.
- Allow the creation of snitches only with locator::make_snitch() template
function.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
New in v4:
- Make sure the snitch is stopped before it's destroyed when _snitch_is_ready
is returned in an exceptional state.
New in v2:
- Change snitch_ptr to be std::unique_ptr<i_endpoint_snitch>
- abstract_replication_strategy::create_replication_strategy(): explicitly
specify (template) types of create_object() parameters.
- Re-arrange the loop in marge_keyspaces() so that lambdas that depend on
"this" complete before there is a chance that "this" gets destroyed.
- create_keyspace(): Don't add a new keyspace if a keyspace with this name
already exists.
- i_endpoint_snitch: added a stop() virtual method
- Added a stop() pure virtual method.
- Added an enum class snitch_state and a _state member initialized to snitch_state::initializing,
added an assert() in a destructor requiring _state to become snitch_state::stopped,
which should be set when stop() is complete.
- rack_inferring_snitch: added a stop() method.
- simple_snitch: added a stop() method.
- Added stop() methods to abstract_replication_strategy and keyspace.
- Updated database::stop() to wait for all keyspaces in _keyspaces to stop.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
New in v2:
- storage_service: add a non-const version of get_token_metadata().
- get_broadcast_address(): check if net::get_messaging_service().local_is_initialized()
before calling net::get_local_messaging_service().listen_address().
- get_broadcast_address(): return an inet_address by value.
- system_keyspace: introduce db::system_keyspace::endpoint_dc_rack
- fb_utilities: use listen_address as broadcast_address for now
To handle the fact that --data-file-directories is supposed to be 1+
folders.
Note that boost::program_ops already "reserves" the use of std::vector
as reciever of values for multitoken options (i.e. those with more than
one value). Thus, values recieving a list of tokens via command line
should adhere to the multi-token rules, i.e. space separated values.
End result is that --data-file-directories now accept multiple paths,
white space separated,
i.e. --data-file-directories <path1> <path2>
And as it turns out, this is really a nicer way of writing stuff than
using "," or ":" seperation of paths etc, so...
Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
Fix the check in read_schema_for_keyspaces() to not insert empty
result sets in the return value. There's no functional change as the
merge algorithms already deal with the case. However, the code is now
closer to what origin does.
Spotted while reading the code.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Switch to schema_result::value_type instead of the open-coded std::pair
so that the actual types are defined in one place.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Keys that are in "entries_only_on_right" need to be looked up from
"after". Fixes a regression introduced in commit 5418e32
("map_difference: Simplify difference value").
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Simplify the return value of map_difference() to return set of keys.
This makes it possible to use the function with value types such as
foreign_ptr<> that are non-copyable.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
There's no need to use decorated_key as schema results map key because
we just convert it back to a keyspace name string. It does, however,
cause problems when trying to pass the results around to different CPUs
so just switch to the sstring type.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
If map value type is lw_shared_ptr<T>, for example, we need to be able
to pass our own value comparison function to the difference() function.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Store a lw_shared_ptr<keyspace_metadata> in struct keyspace so callers
in migration manager, for example, can look it up.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Pekka says:
"Clean up keyspace creation functions by using keyspace_metadata. While
at it, simplify creation by moving replication strategy initialization
to database.cc. This paves the way for adding keyspace metadata lookup
functionality to database that's needed by migration manager."
This adds a method to return a vector with full-path to the active
segment names. It will be used by the API.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Use the keyspace_metadata type for keyspace creation functions. This is
needed to be able to have a mapping from keyspace name to keyspace
metadata for various call-sites.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Follow the naming convention set by user_types_metadata and rename
ks_meta_data to keyspace_metadata.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
* Allow boost::config to translate string_maps via iostream
* Special case seed_provider (struct) into two parameters (for lack of a
good way to express it on command line)
Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
Should fix use-after-free when a frozen_mutation is applied to the
local shard.
Includes two adjustments to urchin collectd usage from Calle:
- Updated thrift collectd registration to use proper move semantics
- Commitlog: Fix collectd registration to use move semantics + test
Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
From Pekka:
"This series implements a Maps.difference() function in C++, changes
storage_proxy::query_local() to not return foreign_ptr>, and finally
changes the keyspace merging code to follow Origin."
Don't return foreign_ptr<> which is not copyable so that we can use
map_difference for maps with result_set in them.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
- # segments
- # allocting segments
- # unused segments
- # allocations
- # cycles (disk writes)
- # flush
- # total bytes allocated
- # total bytes disk slack (due to dma blocks)
Counters are per-commitlog (shard). Can be extended to be per-segment also,
but would be transient and probably not much more useful.
Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
The immediate motivation for introducing frozen_mutation is inability
to deserialize current "mutation" object, which needs schema reference
at the time it's constructed. It needs schema to initialize its
internal maps with proper key comparators, which depend on schema.
frozen_mutation is an immutable, compact form of a mutation. It
doesn't use complex in-memory strucutres, data is stored in a linear
buffer. In case of frozen_mutation schema needs to be supplied only at
the time mutation partition is visited. Therefore it can be trivially
deserialized without schema.
Generic read-all-stream from a commit log segmen file.
Provides a byte view for each data entry, doing CRC checks and padding skips.
Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
