When the container_to_vec helper function has a string that contains
space, a boost exection is thrown.
This fixes it by using std::string for the conversion that the boost
recognize as a string type.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The current 4K write buffer is ridiculously small and forces Urchin to
issue small I/O batches. Increase the buffer size to 64K.
Before:
Results:
op rate : 27265
partition rate : 27265
row rate : 27265
latency mean : 1.2
latency median : 0.9
latency 95th percentile : 2.4
latency 99th percentile : 10.6
latency 99.9th percentile : 14.3
latency max : 44.7
Total operation time : 00:00:30
END
After:
Results:
op rate : 35365
partition rate : 35365
row rate : 35365
latency mean : 0.9
latency median : 0.8
latency 95th percentile : 1.8
latency 99th percentile : 8.8
latency 99.9th percentile : 21.8
latency max : 272.2
Total operation time : 00:00:34
END
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Keys that are in "entries_only_on_right" need to be looked up from
"after". Fixes a regression introduced in commit 5418e32
("map_difference: Simplify difference value").
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Adapt for_all_partitions() to use futures instead of iterators,
as that will be the interface to sstables. We drop use of nway_merger as
that is not able to use futures and instead open-code the heap
functionality.
* Forward commitlog replay_position to column_family.memtable, updating
highest RP if needed
* When flushing memtable, signal back to commitlog that RP has been dealt with
to potentially remove finished segment(s)
Note: since memtable flushing right now is _not_ explicitly ordered,
this does not actually work, since we need to guarantee ordering with
regards to RP. I.e. if we flush N blocks, we must guarantee that:
a.) We report "flushed RP" in RP order
b.) For a given RP1, all RP* lower than RP1 must also have been flushed.
(The latter means that it is fine to say, flush X tables at the same time, as long as we report a single RP that is the highest, and no lower RP:s exist in non-flushed tables)
I am however letting someone else deal with ensuring MT->sstable flush order.
Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
This patch replaces the sstable read APIs from having "push" style,
to having "pull style".
The sstable read code has two APIs:
1. An API for sequentially consuming low-level sstable items - sstable
row's beginning and end, cells, tombstones, etc.
2. An API for sequentially consuming entire sstable rows in our "mutation"
format.
Before this patch, both APIs were in "push style": The user supplies
callback functions, and the sstable read code "pushes" to these functions
the desired items (low-level sstable parts, or whole mutations).
However, a push API is very inconvenient for users, like the query
processing code, or the compaction code, which both iterate over mutations.
Such code wants to control its own progression through the iteration -
the user prefers to "pull" the next mutation when it wants it; Moreover,
the user wants to *stop* pulling more mutations if it wants, without
worrying about various continuations that are still scheduled in the
background (the latter concern was especially problematic in the "push"
design).
The modified APIs are:
1. The functions for iterating over mutations, sstable::read_rows() et al.,
now return a "mutation_reader" object which can be used for iterating
over the mutation: mutation_reader::read() asks for the next mutation,
and returns a future to it (or an unassigned value on EOF).
You can see an example on how it is used in sstable_mutation_test.cc.
2. The functions for consuming low-level sstable items (row begin, cell,
etc.) are still partially push-style - the items are still fed into
the consume object - but consumpton now *stops* (instead of defering
and continuing later, as in the old code) when the consumer asks to.
The caller can resume the consumption later when it wishes to (in
this sense, this is a "pull" API, because the user asks for more
input when it wants to).
This patch does *not* remove input_stream's feature of a consumer
function returning a non-ready future. However, this feature is no longer
used anywhere in our code - the new sstable reader code stops the
consumption when old sstable reader code paused it temporarily with
a non-ready future.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
This change how the json formatter handle float and adds double support.
float and double conversion will throw exceptions when try to convert
inf or nan.
It also contains tests for both float and double including inf, -inf and
nan for both float and double.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Simplify the return value of map_difference() to return set of keys.
This makes it possible to use the function with value types such as
foreign_ptr<> that are non-copyable.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
c_stats is a thread local variable, which was being re-allocated
for every thrift row. It's intended to keep track of its stats.
Shlomi reported the sigsegv while running cassandra-stress, and
bisect pointed to the commit that introduce write of statistics.
After some investigation, I realized that the problem was about
c_stats, which was being explicitly deallocated by our code, and
that the implicit deallocator would find it after freed.
In addition to the problem above, Avi realized that our code to
write statistics was broken when concurrently writing sstables,
given that the same c_stats would be used for both.
To fix both problems, c_stats and functions to write sstables
were made members of the class sstable.
Reported-by: Shlomi Livne <shlomi@cloudius-systems.com>
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
That was needed because new methods of sstable class will have a
file writer as a parameter, and thus the definition of the file
writer must be available from sstables header.
From Paweł:
"These series contains some improvements in the way thrift keyspace-related
operations are handled (checks whether set_keyspace was given an existing
keyspace, more informative error message when adding keyspace with
invalid/unsupported placement strategy) and initial implementation of
describe_keyspace and describe_keyspaces."
Without the check added in this patch if the class doesn't exist
a std::bad_function_call is thrown which is not very informative.
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
The original code just assigns value to _ks_name which is a sstring
so the try { } catch { } clause only gave the wrong impression that
something is checked.
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
