"
- Introduce a global distributed snitch object.
- Add the corresponding methods in i_endpoint_snitch class needed to work with
this object.
- Added additional check to gossiping_property_file_snitch_test.
"
This tests the basic compaction functionality: I created three small
tables using Cassandra (see commands below), compact them into one,
load the resulting table and check its content.
This test demonstrates, but is commented out to make the test succeed,
a bug: If a partition had old values and then a newer deletion (tombstone)
in another sstable, both values and tombstones are left behind in the
compacted table. This will be fixed (and the test uncommented) in a later
patch.
The three sstables were created with:
USE try1;
CREATE TABLE compaction (
name text,
age int,
height int,
PRIMARY KEY (name)
);
INSERT INTO compaction (name, age) VALUES ('nadav', 40);
INSERT INTO compaction (name, age) VALUES ('john', 30);
<flush>
INSERT INTO compaction (name, height) VALUES ('nadav', 186);
INSERT INTO compaction (name, age, height) VALUES ('jerry', 40, 170);
<flush>
DELETE FROM compaction WHERE name = 'nadav';
INSERT INTO compaction (name, age) VALUES ('john', 20);
INSERT INTO compaction (name, age, height) VALUES ('tom', 20, 180);
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Snitch class semantics defined to be per-Node. To make it so we
introduce here a static member in an i_endpoint_snitch class that
has to contain the pointer to the relevant snitch class instance.
Since the snitch contents are not always pure const it has to be per
shard, therefore we'll make it a "distributed". All the I/O is going
to take place on a single shard and if there are changes - they are going
to be propagated to the rest of the shards.
The application is responsible to initialize this distributed<shnitch>
before it's used for the first time.
This patch effectively reverts most of the "locator: futurize
snitch creation" a2594015f9 patch - the part that modifies the
code that was creating the snitch instance. Since snitch is
created explicitly by the application and all the rest of the code
simply assumes that the above global is initialized we won't need
all those changes any more and the code will get back to be nice and simple
as it was before the patch above.
So, to summarize, this patch does the following:
- Reverts the changes introduced by a2594015f9 related to the fact that
every time a replication strategy was created there should have been created
a snitch that would have been stored in this strategy object. More specifically,
methods like keyspace::create_replication_strategy() do not return a future<>
any more and this allows to simplify the code that calls it significantly.
- Introduce the global distributed<snitch_ptr> object:
- It belongs to the i_endpoint_snitch class.
- There has been added a corresponding interface to access both global and
shard-local instances.
- locator::abstract_replication_strategy::create_replication_strategy() does
not accept snitch_ptr&& - it'll get and pass the corresponding shard-local
instance of the snitch to the replication strategy's constructor by itself.
- Adjusted the existing snitch infrastructure to the new semantics:
- Modified the create_snitch() to create and start all per-shard snitch
instances and update the global variable.
- Introduced a static i_endpoint_snitch::stop_snitch() function that properly
stops the global distributed snitch.
- Added the code to the gossiping_property_file_snitch that distributes the
changed data to all per-shard snitch objects.
- Made all existing snitches classes properly maintain their state in order
to be able to shut down cleanly.
- Patched both urchin and cql_query_test to initialize a snitch instance before
all other services.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
New in v6:
- Rebased to the current master.
- Extended a commit message a little - the summary.
New in v5:
- database::create_keyspace(): added a missing _keyspaces.emplace()
New in v4:
- Kept the database::create_keyspace() to return future<> by Glauber's request
and added a description to this method that needs to be changed when Glauber
adds his bits that require this interface.
Similar to std::shared_mutex, allows shared and exclusive locking of a
resource (also known as a reader/writer lock). This implementation is
strictly FIFO.
In a couple of places in sstable_datafile_test.cc, we had
make_shared<memtable> instead of the usual make_lw_shared<memtable>.
It appears that since commit bc468f9a0e,
which made memtables inherit from enable_lw_shared_from_this (not the "lw"),
this no longer works correctly. I'm not sure I can really explain the
details of what's wrong, but with some refactoring I did on the sstable
writing code, it stopped working giving various strange crashes which appear
like the object protected by this shared_ptr got prematurely destructed.
Changing the test to use make_lw_shared() fixes the problems I was having.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
From Glauber:
"Shlomi found a bug when writing summary files under certain conditions.
The current patchset fixes it, while also performing some generally useful
cleanups."
From Pekka:
"This series improves the CQL test for 'create table' statement. It
hardens the test_create_statement() test case and converts
create_table() calls to use CQL where possible. Please note that we need
to go through the remaining create_table() call-sites and fix potential
issues in the CQL front-end or schema management code."
Add a require_table_exists() assertion after 'create table' statement is
executed to ensure in-memory schema is updated accordingly.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
We have recently had a bug where we were messing up with the summary generation
in certain corner cases. In particular, when the number of keys was an exact multiple
of the sampling rate.
Add a test case to make sure we never regress on this.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
"only lz4 is supported so far, but other algorithms should be fairly easy to
add, given that the infrastructure for compression is introduced here.
NOTE:
our sstable read code already reads compressed data file, so a test was added
to guarantee that data from a compressed sstable generated by our write code
can be successfully loaded."
To test the messaging service API it needs to be included in the
messaging service test.
To test it, start the server with --stay-alive true
then you can use the API to get the messges information:
http://localhost:10001/messaging_service/messages/exception
will return the number of exception per connection
http://localhost:10001/messaging_service/command/completed
will return the number of completed command per connection.
Note that because the two servers are running on the same machine, to
prevent port conflict, the server port will be increment by one (i.e
10001)
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
From Vlad:
"Currently database always created a SimpleSnitch and ignores the corresponding parameter
provided by the user. This series fixes this situation:
- Changes the snitch creation interface to comply the Java-like interface that
has already been used in a topology_strategy classes family.
- Fix all the places where a SimpleSnitch has been created ignoring the user configuration."
