For drop_column_family(), we want to first remove the column_family from
lookup tables and truncate after that to avoid races. Introduce a
truncate() variant that takes keyspace and column_family references.
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
We need to capture the "is_local_only" boolean by value because it's an
argument to the function. Fixes an annoying bug where we failed to update
schema version because we pass "true" accidentally. Spotted by ASan.
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
"This series adds the functionality that is required so the nodetool cfstats
would work.
It complete the histogram support for read and write latency and add stub for
functionality that is needed but is not supported yet."
This adds the implementation that return the estimated total latency of
the read and of the write.
First the method that sum the count was renamed to get_cf_stats_count
and a method was added named get_cf_stats_sum to sum the estimated
latencies.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The histogram that are used typically only sample the data, so to get an
estimation of the actual sum, we use the estimated mean multiply by the
actuall count.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This patch contains two changes to the histogram implementation. It uses
a simpler method to calculate the estimated mean (simply divide the
estimated sum with the number of samples) and to make sure that there
will always be values in the histogram, it start with taking a sample
(when there are no samples) and then use the mask to decide if to sample
or not.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This patch contains the following changes, in the definition of the read
and write latency histogram it removes the mask value, so the the
default value will be used.
To support the gothering of the read latency histogram the query method
cannot be const as it modifies the histogram statistics.
The read statistic is sample based and it should have no real impact on
performance, if there will be an impact, we can always change it in the
future to a lower sampling rate.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds the API definition with stub implementation that would make
the nodetool cfstats to run.
After this patch the nodetool cfstats command would work, but with stub
imlementation.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This patch add some missing definition for cas read an write, the API
definition is for completness only as we do not support cas yet.
It also change a part of the definition from storage_service to
storage_proxy as it should be.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Only tables that arise from flushes are backed up. Compacted tables are not.
Therefore, the place for that to happen is right after our flush.
Note that due to our sharded architecture, it is possible that in the face of a
value change some shards will backup sstables while others won't.
This is, in theory, possible to mitigate through a rwlock. However, this
doesn't differ from the situation where all tables are coming from a single
shard and the toggle happens in the middle of them.
The code as is guarantees that we'll never partially backup a single sstable,
so that is enough of a guarantee.
Signed-off-by: Glauber Costa <glommer@scylladb.com>
Query and set the state of incremental backups. The initial value comes from
the configuration file through the local db reference. Later on, it can be
changed through the interface.
Signed-off-by: Glauber Costa <glommer@scylladb.com>
When we convert exceptions into CQL server errors, type information is
not preserved. Therefore, improve exception error messages to make
debugging dtest failures, for example, slightly easier.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
For a lot of users, running Scylla in some kinds of filesystems that do not support
O_DIRECT is quite frustrating: it will fail at some point, with random error messages
that aren't really meaningful.
We should try to check for that, and fail with a good error message. Also, since our
performance claims won't really hold in anything other than XFS, we should warn the user
if that is not the setup we encounter.
Fixes#409
Signed-off-by: Glauber Costa <glommer@scylladb.com>
Lazy digest calculation code introduced a bug in background read repair.
The problem is that digest_read_resolver::resolve() destroys one data
result (it is moved to a caller to be sent as a reply), so during
background digest match there is no value to calculate a digest from.
Copying data to the caller would be most elegant solution, but also
slowest one, so lets just treat the case where there is only one
target queried and skip digest calculation in this case since we know
digest_match() will do nothing.
"First iteration implementation of CQL truncate, transposed from
Origin.
Includes a workable impl. of snapshots, since that is sort of an integral
part of the origin code.
Note: This is still incomplete/incorrect in two ways:
1.) Since we have no way to ensure sstables are finished writing,
the flush-snapshots are unreliable. Needs basically the same
fix as correct commitlog management, namely flush queues and
the ability to wait-force "active" flushes to finish before
continuing.
2.) System table truncation record saving does not handle sharding.
This means we basically save the "last" RP from any of the shards
truncating, and consequently if we have a crash and do commitlog
replay, we could resurrect truncated data.
Fix is to have truncation records be per cf+shard just as RP:s
are per shard.
However, since some people are waiting for at least a semi-functional
truncate, I'm submitting this without fixing the two above issues,
since they can be dealt with in subsequent patches."
Align with rest of file (for better or worse). This allows calls from
entity without query_processor handy (i.e. storage_proxy).
Added "minimal" setup method for the "global" state, to facilitate
tests. Doing a full setup either in cql_test_env or after it is created
breaks badly. (Not sure why). So quick workaround.
Updated the current two users (batchlog_manager and commitlog_replayer)
callsites to conform.
* seastar 5fe596a...283901a (10):
> Add filesystem "link_file"
> scripts: posix_net_conf.sh: take care of the case with more than 32 CPUs
> posix: Add explanatory string to throw_system_error_on()
> tests: fix memory leak in thread_test test_thread_1
> tests: fix memory leak in timertest cancellation test
> README: require xfsprogs-devel
> file: query dma alignment from OS
> file: separate disk and memory dma alignment
> scripts: posix_net_conf.sh: Exclude CPU0 in RDS config for EN.
> README: Add missing Ubuntu 14.04 dependencies to README.md
Although it is technically a seastar problem, most complains about that is
coming from the Scylla side. I prefer to keep the message here so we can reference
a Scylla issue.
Signed-off-by: Glauber Costa <glommer@scylladb.com>
