"This series adds the storage proxy metrics API. The definition are based on the
StorageProxyMetrics definition. This series also adds stats object to the
storage_proxy a getter function for it and an implementation based on it, but
it is currently does not adds the code to manipulate the counters."
The API needs to call the storage_proxy, for that a reference to the
distribute storage_proxy is added to the context and is set in main.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Right now when we initiate the database, we exist with just an exception if the
data directory does not exist. That does not tell much to the user about what
is going on.
It would be nice to at the very least catch the exception and turn it into a
user friendly message. But we can obviously do much better and create the
directory.
If we fail, then we can capture and tell the user why.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
We won't create our own versions of database and query_proxy, so we need
some setup to be done. The current code will capture those variables and
setup the structure used to conduct the queries. Later on, it will also
carry on the database's basic setup.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
The messaging service is initialized very early, before we have the
proxy or query processor initialized. It is mostly fine, except for
the fact that the messaging service also finishes the initialization
of the storage service. That part will issue queries agains the system
tables, (as soon as we support them), and need to happen later.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
In order for us to call some function from db::legacy_schema_tables, we need
a working storage proxy. We will use those functions in order to leverage the
work done in keyspace / table creation from mutations
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Soon I will introduce some changes that make init_from_data_directory dependent
on a working storage proxy.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
This patch adds support to the Swagger-UI. It does so by adding
reference to the Swagger-UI target directory, after applying this patch
/ui/ will display the Swagger-UI page with the available APIs.
From the Swagger-UI page it is possible to run the different APIs.
The target directory of the ui can be override from the command line.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Snitch class semantics defined to be per-Node. To make it so we
introduce here a static member in an i_endpoint_snitch class that
has to contain the pointer to the relevant snitch class instance.
Since the snitch contents are not always pure const it has to be per
shard, therefore we'll make it a "distributed". All the I/O is going
to take place on a single shard and if there are changes - they are going
to be propagated to the rest of the shards.
The application is responsible to initialize this distributed<shnitch>
before it's used for the first time.
This patch effectively reverts most of the "locator: futurize
snitch creation" a2594015f9 patch - the part that modifies the
code that was creating the snitch instance. Since snitch is
created explicitly by the application and all the rest of the code
simply assumes that the above global is initialized we won't need
all those changes any more and the code will get back to be nice and simple
as it was before the patch above.
So, to summarize, this patch does the following:
- Reverts the changes introduced by a2594015f9 related to the fact that
every time a replication strategy was created there should have been created
a snitch that would have been stored in this strategy object. More specifically,
methods like keyspace::create_replication_strategy() do not return a future<>
any more and this allows to simplify the code that calls it significantly.
- Introduce the global distributed<snitch_ptr> object:
- It belongs to the i_endpoint_snitch class.
- There has been added a corresponding interface to access both global and
shard-local instances.
- locator::abstract_replication_strategy::create_replication_strategy() does
not accept snitch_ptr&& - it'll get and pass the corresponding shard-local
instance of the snitch to the replication strategy's constructor by itself.
- Adjusted the existing snitch infrastructure to the new semantics:
- Modified the create_snitch() to create and start all per-shard snitch
instances and update the global variable.
- Introduced a static i_endpoint_snitch::stop_snitch() function that properly
stops the global distributed snitch.
- Added the code to the gossiping_property_file_snitch that distributes the
changed data to all per-shard snitch objects.
- Made all existing snitches classes properly maintain their state in order
to be able to shut down cleanly.
- Patched both urchin and cql_query_test to initialize a snitch instance before
all other services.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
New in v6:
- Rebased to the current master.
- Extended a commit message a little - the summary.
New in v5:
- database::create_keyspace(): added a missing _keyspaces.emplace()
New in v4:
- Kept the database::create_keyspace() to return future<> by Glauber's request
and added a description to this method that needs to be changed when Glauber
adds his bits that require this interface.
Moved setting of configuration variables after the configuration file
has been read.
Updated the code parsing seeds to comply with configuration file format
- seeds: <ip1>,<ip2>,<ip3>
Signed-off-by: Shlomi Livne <shlomi@cloudius-systems.com>
Code calls failure_detector::is_alive on all cpus, so we start
failure_detector on all cpus. However, the internal data of failure_detector
is modified on cpu zero and it is not replicated to non-zero cpus.
This is fine since the user of failure_detector (the gossiper) accesses
it on cpu0 only.
_token_metadata is needed by replication strategy code on all cpus.
Changes to _token_metadata are done on cpu 0. Replicate it to all cpus.
We may copy only if _token_metadata actually changes. As a starter, we
always copy in gossip modification callbacks.
Currently if you run more than one instance on the same machine, they
all bind to the same local address and port (we enable an option that
makes this non error) and cql/thrift requests are distributed randomly
among them.
The API needs the database distribute object to get information from it.
This adds a database reference the API context so it would be
available in the API.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Integrate messaging_service and failure_detector and gossiper into urchin.
To start a 3 nodes cluster on a single host:
./seastar --listen-address 127.0.0.1 --seed-provider-parameters 127.0.0.1
./seastar --listen-address 127.0.0.2 --seed-provider-parameters 127.0.0.1
./seastar --listen-address 127.0.0.3 --seed-provider-parameters 127.0.0.1
This adds the API to urchin, all API related files will be placed under
the api directory.
The API server uses a context object to access global parameters,
typically the different servers themselves.
After this patch the api-doc will be available, though empty under:
http://localhost:10000/api-doc
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This patch adds initial support for PREPARE and EXECUTE requests which
are used by the CQL binary protocol for prepared statements. The use of
prepared statement gives a nice 2.5x single core performance boost for
Urchin:
$ ./build/release/seastar --data data --smp 1
$ ./tools/bin/cassandra-stress write -mode cql3 simplenative -rate threads=32
Results:
op rate : 31728
partition rate : 31728
row rate : 31728
latency mean : 1.0
latency median : 0.9
latency 95th percentile : 1.8
latency 99th percentile : 1.8
latency 99.9th percentile : 5.6
latency max : 181.7
Total operation time : 00:00:30
END
$ ./tools/bin/cassandra-stress write -mode cql3 simplenative prepared -rate threads=32
Results:
op rate : 75033
partition rate : 75033
row rate : 75033
latency mean : 0.4
latency median : 0.4
latency 95th percentile : 0.7
latency 99th percentile : 0.8
latency 99.9th percentile : 3.4
latency max : 205.0
Total operation time : 00:00:30
END
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
When engine.exit() is used, e.g., on trapping SIGINT, the engine
exits before doing db.stop(), causing an assertion failure.
This patch adds an at_exit([&db] { return db.stop(); }). This means
that before the engine exits, it runs db.stop() and waits for the
future that it returns to be completed. This is exactly what we need.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
s/database/distributed<database>/ everywhere.
Use simple distribution rules: writes are broadcast, reads are local.
This causes tremendous data duplication, but will change soon.
cql/, where the binary protocol code resides, is too similar to cql3/, where
the grammar lives. Rename it to transport/ to reduce confusion. While the
name isn't perfect, it matches origin.
If data directory does not exist, Urchin starts up without printing
anything and hangs. Make sure the error is shown to user by terminating
the application.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
This patch implement initial support for CQL binary protocol versions 1,
2, 3, and 4. The CQL server is able to handshake with cqlsh and
cassandra-stress from Cassandra 2.1 which uses the STARTUP and OPTIONS
messages. Queries or other functionality is not supported.
To try it out, start Urchin:
$ build/release/seastar --smp 1 --datadir data
CQL server listening on port 9042 ...
Thrift server listening on port 9160 ...
Then try to login with cqlsh:
$ ./bin/cqlsh 127.0.0.1
Urchin side will fail with:
CQL_VERSION => 3.2.0
warning: ignoring event registration
warning: ignoring query SELECT peer, data_center, rack, tokens, rpc_address, schema_version FROM system.peers
seastar: cql/server.cc:222: future<> cql_server::connection::process_query(uint16_t, temporary_buffer<char>&): Assertion `0' failed.
Aborted (core dumped)
TODO:
- Compression is not supported.
- Authentication is not supported.
- Supported options are defined to make cqlsh and cassandra-stress
happy. We really need to decide which CQL versions and compression
algorithms we want to support.
- std::string is used everywhere because sstring does not work with
std::map, std::multimap, and others.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
This patch includes a helper function that executes a function for each entry
in a directory. It is future based and can include in the future, future-based
code to asychronously read, for instance, an sstable.
At the moment, it only scan all keyspaces and make sure they appear in the
keyspaces hash.
Both the database and keyspace classes gain a populate<T> factory that returns a
populated database. At this point, the names found are just listed, but not really
stored anywhere.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
This is the directory from which we will read the sstables.
Code to actually parse them will come later.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
