This allows for us to delete an existing snapshot. It works at the column
family level, and removing it from the list of keyspace snapshots needs to
happen only when all CFs are processed. Therefore, that is provided as a
separate operation.
The filesystem code is a bit ugly: it can be made better by making our file
lister more generic. First step would be to call it walker, not lister...
For now, we'll use the fact that there are mostly two levels in the snapshot
hierarchy to our advantage, and avoid a full recursion - using the same lambda
for all calls would require us to provide a separate class to handle the state,
that's part of making this generic.
Signed-off-by: Glauber Costa <glommer@scylladb.com>
There are situations in which we would like to match more than one directory
type. One example of that, would be a recursive delete operation: we need to
delete the files inside directories and the directories themselves, but we
still don't want a "delete all" since finding anything other than a directory
or a file is an error, and we should treat it as such.
Since there aren't that many times, it should be ok performance wise to just
use a list. I am using an unordered_set here just because it is easy enough,
but we could actually relax it later if needed. In any case, users of the
interface should not worry about that, and that decision is abstracted away
into lister::dir_entry_types.
Signed-off-by: Glauber Costa <glommer@scylladb.com>
This is certainly the right thing to do and seems to fix#403. However
I didn't manage to convince myself that this would cause problems for
binomial_heap, given that binomial_heap::erase() calls siftup()
anyway:
void erase(handle_type handle)
{
node_pointer n = handle.node_;
siftup(n, force_inf());
top_element = n;
pop();
}
void increase (handle_type handle)
{
node_pointer n = handle.node_;
siftup(n, *this);
update_top_element();
sanity_check();
}
There was a confusion between the snapshot key and the keyspace in the
snapshot details, this fixes it.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
"Those are fixes needed for the snapshotting process itself. I have bundled this
in the create_snapshot series before to avoid a rebase, but since I will have to
rewrite that to get rid of the snapshot manager (and go to the filesystem),
I am sending those out on their own."
test_setup::do_with_test_directory is missing. For some reason,
the test wasn't failing without it until now. Adding it is the
correct thing to do anyway.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
"This patchset introduces leveled compaction to Scylla.
We don't handle all corner cases yet, but we already have the strategy
and compaction working as expected. Test cases were written and I also
tested the stability with a load of cassandra-stress.
Leveled compaction may output more than one sstable because there is
a limit on the size of sstables. 160M by default.
Related to handling of partial compaction, it's still something to be
worked on.
Anyway, it will not be a big problem. Why? Suppose that a leveled
compaction will generate 2 sstables, and scylla is interrupted after
the first sstable is completely written but before the second one is
completely written. The next boot will delete the second sstable,
because it was partially written, but will not do anything with the
first one as it was completely written.
As a result, we will have two sstables with redundant data."
With the distribute-and-sync method we are using, if an exception happens in
the snapshot creation for any reason (think file permissions, etc), that will
just hang the server since our shard won't do the necessary work to
synchronize and note that we done our part (or tried to) in snapshot creation.
Make the then clause a finally, so that the sync part is always executed.
Signed-off-by: Glauber Costa <glommer@scylladb.com>
create_links will fail in one of the shards if one of the SSTables happen to be
shared. It should be fine if the link already exists, so let's just ignore that case.
Signed-off-by: Glauber Costa <glommer@scylladb.com>
Explicitly use up all the memory in the system as best as we can instead
Still not super reliable, but should have less side effects. And work better
with pre-allocated segment files
Make production_snitch_base constructor signature consistent with
the rest of production snitches.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
A non-empty default value of a configuration file name was preventing
the db::config::get_conf_dir() to kick in when a default snitch constructor
was used (this is the way it's always used from scylla).
Fixes issue #459
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
When building the in-memory schema for a column family, we were
ignoring compaction strategy class because of a bug in the
existing code. Example: suppose that you create a column family
with leveled compaction strategy. This option would be ignored
and the default strategy (size-tiered) would be used instead.
Found this problem while working on leveled compaction.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
"This series cleans up a few places in the snitches
code that has been noticed during the work on issues #464
and #459.
The last patch actually fixes the issue #464"
Make production_snitch_base constructor signature consistent with
the rest of production snitches.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
A non-empty default value of a configuration file name was preventing
the db::config::get_conf_dir() to kick in when a default snitch constructor
was used (this is the way it's always used from scylla).
Fixes issue #459
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
