handle_exception() should really discard the future's value automatically,
and in an upcoming version of Seastar, won't. So instead of
sp.execute().handle_exception(...)
(where execute() returns a future which is *not* future<>)
We need to write
sp.execute().discard_result().handle_exception(...)
This already works in today's Seastar (the extra discard_result()
doesn't cause any harm), and will be necessary when handle_exception()
in Seastar is improved (I'll send a patch soon).
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Loading data from memory tends to be the most expensive part of the comparison
operations. Because we don't have a tri_compare function for tokens, we end up
having to do an equality test, which will load the token's data in memory, and
then, because all we know is that they are not equal, we need to do another
one.
Having two dereferences is harmful, and shows up in my simple benchmark. This
is because before writing to sstables, we must order the keys in decorated key
order, which is heavy on the comparisons.
The proposed change speeds up index write benchmark by 8.6%:
Before:
41458.14 +- 1.49 partitions / sec (30 runs)
After:
45020.81 +- 3.60 partitions / sec (30 runs)
Parameters:
--smp 6 --partitions 500000
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
This is a test that allow us to query the performance of our sstable index
reads and writes (currently only writes implemented). A lot of potentially
common code is put into a header, which will make writing new tests easier if
needed.
We don't want to take shortcuts for this, so all reading and writing is done
through public sstable interfaces.
For writing, there is no way to write the index without writing the datafile.
But because we are only writing the primary key, the datafile will not contain
anything else. This is the closest we can get to an index testing with the
public interfaces.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
if a directory is found, recursively delete it. This will be useful for
allowing the creation of test structures like test/cpuX/sstable
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Our normal test directory may not be good enough for performance testing. The
reason is, that while our git tree with its relative path will usually be
sitting in a standard ext4 filesystem, we want the performance tests to be run
against XFS, which is our deployment target.
It is a lot easier to point the perf test to an already mounted xfs directory,
than to meddle with mounts into the codebase's relative path for this alone.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
In some situations, it is useful to have the test directory persistent. To do that,
expose the inner function that creates it.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
resume_io() is different from start() in that it won't try to read to configuration
and will only restart the periodic I/O task (if any).
This also means that resume_io() may not fail while start() will return an
exceptional future if it fails to read the configuration.
pause_io() is a counterpart of resume_io() - it stops the periodic I/O task (if any).
After it returns a ready future - snitch will not try to read any configuration until
either start() or resume_io() are called.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
- production_snitch_base: Store a distributed object pointer in the snitch.
- i_endpoint_snitch::init_snitch_obj(): Set the distributed<> mentioned above.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Initializes the given distributed<snitch_ptr> object but
not start()s the local snitch instances.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
- Call for read_property_file() directly from the start().
- Immediately return ready future from the start() for non-IO
CPUs.
- Remove the not needed invoke_on_all() invocations.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
If snitch has been created while it had to fail we have to stop the
global (distributed) snitch in order to avoid the assert.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
- Fixed a typo.
- snitch_ptr: make operator= return a reference to the parent object.
- i_endpoint_snitch: set the _state in a default start() implementation.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
This adds the implementation to in the API to the row size histogram.
It adds a map_cf method that perform a map operation over all column
family on the different shards.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds the latency histogram support to the storage_proxy.
It uses a the latency object to mark the opetation latency, if there
will be an impact on performance, it can be changed from all operations
to sample of the operation.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds the estimated_histogram to the utils definition file.
The estimated_histogram holds a list of buckets and a list of buckets
offsets.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds the implementation for min and max row size in column family.
It uses the column family map redudce helper function with the addtional
function to get the min and max row size.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This helper function wraps the std min and max template for int64_t, it
makes it easier to pass them as a value in need.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This changes the constructor initilization of the metadata_collecr, it
would call the constructor directly without the java-like assignment.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This do the following chagnes in the estimated_histogram, it uses
int64_t over unsigned to be compatible to origin and the API.
It adds a getter to the buckets and change the getteer to the
bucket_offset to be const.
It adds a get min and max similiar to origin. And it adds a merge
function to merge estimated histogram.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
"Cleanups to the CQL server implementation. The biggest change is moving
event notifier to a separate source file in an attempt to make server.cc
smaller and more modularized."
To free memory, we need to allocate memory. In lsa compaction, we convert
N segments with average occupancy of (N-1)/N into N-1 new segments. However,
to do that, we need to allocate segments, which we may not be able to do
due to the low memory condition which caused us to compact anyway.
Fix by introducing a segment reserve, which we normally try to ensure is
full. During low memory conditions, we temporarily allow allocating from
the emergency reserve.
Since we do not support shard to shard connections at the moment, ip
address should fully decide if a connection to a remote node exists or
not. messaging_service maintains connections to remote node using
std::unordered_map<shard_id, shard_info, shard_id::hash> _clients;
With this patch, we can possibly reduce number of tcp connections
between two nodes.
Move the connection class to server.hh so that we can move event
notifier implementation to a separate source file.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
The default constructor need to set the the max_bin size, so it was
combine with the non default one, with a default value.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Move member function definitions outside of the class definition in
preparation for moving the latter to a header file.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
