DPDK network statistics, from Vlad:
- Re-arranges and adds a few new queue networking statistics counters.
- Adds port networking statistics counters for DPDK backend.
- Rework the collectd registration and representation of networking statistics:
- Use Rx/Tx statistics tuples where possible.
In Java it is possible to create an object by knowing its class name in
runtime. Replication strategies are created this way (I presume class
name comes from configuration somehow), so when I translated the code to
urchin I wrote replication_strategy_registry class to map a class name to
a factory function. Now I see that this is used in other places too (I
see that snitch class created in the same way), so instead of repeating
the same code for each class hierarchy that is created from its name in
origin this patch tries to introduce an infrastructure to do that easily.
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
This gives about 30% increase in tps in:
build/release/tests/perf/perf_simple_query -c1 --query-single-key
This patch switches query result format from a structured one to a
serialized one. The problems with structured format are:
- high level of indirection (vector of vectors of vectors of blobs), which
is not CPU cache friendly
- high allocation rate due to fine-grained object structure
On replica side, the query results are probably going to be serialized
in the transport layer anyway, so this change only subtracts
work. There is no processing of the query results on replica other
than concatenation in case of range queries. If query results are
collected in serialized form from different cores, we can concatenate
them without copying by simply appending the fragments into the
packet. This optimization is not implemented yet.
On coordinator side, the query results would have to be parsed from
the transport layer buffers anyway, so this also doesn't add work, but
again saves allocations and copying. The CQL server doesn't need
complex data structures to process the results, it just goes over it
linearly consuming it. This patch provides views, iterators and
visitors for consuming query results in serialized form. Currently the
iterators assume that the buffer is contiguous but we could easily
relax this in future so that we can avoid linearization of data
received from seastar sockets.
The coordinator side could be optimized even further for CQL queries
which do not need processing (eg. select * from cf where ...) we
could make the replica send the query results in the format which is
expected by the CQL binary protocol client. So in the typical case the
coordinator would just pass the data using zero-copy to the client,
prepending a header.
We do need structure for prefetched rows (needed by list
manipulations), and this change adds query result post-processing
which converts serialized query result into a structured one, tailored
particularly for prefetched rows needs.
This change also introduces partition_slice options. In some queries
(maybe even in typical ones), we don't need to send partition or
clustering keys back to the client, because they are already specified
in the query request, and not queried for. The query results hold now
keys as optional elements. Also, meta-data like cell timestamp and
ttl is now also optional. It is only needed if the query has
writetime() or ttl() functions in it, which it typically won't have.
This:
set_if<item>(cond);
is a more efficient version of:
if (cond) {
set<item>();
}
The implementation can leverage internal representation to avoid
branching and thus make the operation cheaper.
UUID_gen::create_time_safe() does not synchronize across cores. The
comment says that it assumes it runs on a single core. This is no
longer true, we can run urchin on many cores. This easily leads to
UUID conflicts with more than one core. Fix by adding a per-core
unique number to the node part of the UUID.
