An earlier patch of mine was using should_yield to do the same. That
is a better direction, but should_yield() was demonstrably more
expensive so for now we'll go with need_preempt() - since this is
hurting pretty much every latency-dependent workload.
I am also including the scripts that I have used to measure and
compare the various versions of this patch.
The data() method already returns a bytes_opt so there's no need to call to_bytes_opt() again.
Fixes compliation failure on CentOS:
In file included from ./cql3/query_options.hh:51:0,
from ./cql3/cql_statement.hh:47,
from ./cql3/statements/raw/select_statement.hh:45,
from build/release/gen/cql3/CqlParser.hpp:65,
from build/release/gen/cql3/CqlParser.cpp:44:
./cql3/values.hh: In function 'bytes_opt to_bytes_opt(const cql3::raw_value&)':
./cql3/values.hh:184:37: error: no matching function for call to 'to_bytes_opt(bytes_opt)'
return to_bytes_opt(value.data());
Message-Id: <1485761863-28236-1-git-send-email-penberg@scylladb.com>
The reason is the same as why foreground writes are reported instead of
total writes (049ae37d08): It is much easier to see what is going on
this way.
Also fixes a typo in a counter's description.
Fixes#1217
Message-Id: <20170129093412.GS11469@scylladb.com>
This patch series adds support for "unset values" that were introduced
in CQL binary protocol v4. They allow bound statements to skip updates
to some or all of the bound variables.
Unset values are specified using the BoundStatement.unset() method in
the Java driver:
http://docs.datastax.com/en/drivers/java/3.1/com/datastax/driver/core/BoundStatement.html#unset-int-
and using the UNSET_VALUE constant in the Python driver:
https://datastax.github.io/python-driver/api/cassandra/query.html#cassandra.query.UNSET_VALUEFixes#2039.
* 'penberg/cql-unset-values/v2' of github.com:cloudius-systems/seastar-dev:
transport/server: CQL unset value support
cql3/statements/select_statement: Unset value support
cql3/user_types: Unset value support
cql3/tuples: Unset value support
cql3/maps: Unset value support
cql3/sets: Unset value support
cql3/lists: Unset value support
cql3/constants: UNSET_VALUE constant
cql3/constants: Unset value support
cql3/attributes: Unset value support
types.hh: Add field_name_as_string() to user_type_impl type
cql3: Introduce raw_value and raw_value_view types
The quota check is quite old at the moment, and dates back to a time in
which the infrastructure in seastar threads was lacking a lot. It is a
bad check since it will not take into consideration the size of the
partition or the time it takes to merge them.
A better check would at least take need_preempt() into account, so that
we would respect the task quota. That check is now embedded into
should_yield(), so there would no need to check anything else.
Although should_yield() does the job, it is still currently quite
expensive. And because we are in a seastar thread with a computationally
intensive loop, it can hurt latency a lot.
So as a temporary measure, let's at least check for need_preempt() - as
it is hurting real users at the moment - and soon work on making
should_yield() cheaper.
Signed-off-by: Glauber Costa <glauber@scylladb.com>
update is one of our biggest sources of performance issues as far as the
cache is concerned. systemtap can be useful in helping tracking some of
them down.
Signed-off-by: Glauber Costa <glauber@scylladb.com>
Currently, the code is using bytes_opt and bytes_view_opt to represent
CQL values, which can hold a value or null. In preparation for
supporting a third state, unset value introduced in CQL v4, introduce
new raw_value and raw_value_view types and use them instead.
The new types are based on boost::variant<> and are capable of holding
null, unset values, and blobs that represent a value.
This reverts commit d61002cc33.
Introduced a regression in row_cache_alloc_stress.
The problem is that reclaim_from_evictable() evicts way too much after
the refactor due to the stop condition not taking into account how
much data was evicted so far and only looking at occupancy of the
minimal segment. This may lead to eviction of the whole region.
parse_time() adds hourse, minutes, etc to a final value 'result'.
However, it is of type std::chrono::nanoseconds which means it is not
zeroed at initialization unless it is explicitly asked to do so.
Fixed debug mode failures in types_tyes and cql_query_test.
Message-Id: <20170125155239.1253-1-pdziepak@scylladb.com>
"This patchset properly implements range_tombstone_list::difference(),
which was very broken. We add unit tests for the function and ensure
we always randomly generate range_tombstones in other unit tests so
other problems aren't hidden."
This patch ensures the mutation_merger emits any deferred tombstones
that it still may be holding before closing the stream.
Together with the range_tombstone_list: Properly implement
difference() patch set, this fixes breakage of streamed_mutation_test
and row_cache_test.
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20170123195643.9876-1-duarte@scylladb.com>
* seastar 6d80c6a...397685c (4):
> Merge "add label to the io_queue" from Amnon
> rpc: Modify the shutdown code to wait and handle exceptions
> tls.cc: Fix shutdown_input/output to conform with expected socket behaviour
> core: Add counter for polls
The difference method wasn't properly implemented. The version in this
patch correctly computes the difference and returns a range tombstone
list contains those range tombstones in "this" but absent from the
other, specified range tombstone list.
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
* seastar 38aaa4a...6d80c6a (2):
> DPDK: Change the metrics registration with label support
> metric: Fix the error: could not convert {...} from <brace-enclosed initializer list> to struct metric_definition_impl
Add a counter field to RELEASE, just before the date, and fix it at zero.
This allows custom package builds to override it in a way that sorts before
the official packages.
Example:
Official release: 1.6.0-0.20160120.<githash>
Custom release 1: 1.6.0-1.avi.20160121.<githash>
Custom release 2: 1.6.0-2.avi.20160122.<githash>
The counter (0/1/2) ensures that the build number dominates over the date
when sorting.
Message-Id: <20170122102814.19649-1-avi@scylladb.com>
From Avi:
In many cases, batch statements are used to mutate a single partition, or
a number of partitions that is smaller than the number of statements within
the batch. We can detect this case and reduce the numbers of mutations
applied, and in some cases, convert a logged batch into an unlogged batch.
Ref #1689.
continuous_data_consumer::fast_forward_to() returns a future which was
later ignored by data_consume_context::fast_forward_to().
With the current implementation, the future in question is always ready
and that's why the problem didn't manifest itself in the form of crashes
or invalid results.
Message-Id: <20170120105746.7300-1-pdziepak@scylladb.com>
Batch statements are often used to insert multiple rows into the same
partition. Recognize this case and merge mutations to the same partition.
If the result is a single mutation, there is an additional win (already
present in the code), where a logged batch can be converted into an unlogged
batch.
Ref #1689.
Add a boolean to short circuit the read path on empty range
hoping for some speedup.
tested in read write with cs using:
cl=QUORUM duration=1m -mode native cql3 -rate threads=700 -node localhost
Will do some additional benchmark.
Fixes#1056
Signed-off-by: Benoît Canet <benoit@scylladb.com>
Message-Id: <20170118194451.16836-1-benoit@scylladb.com>
