Allows detecting changes of lower_bound().
Result of advance_to() is not enough. When we get false from
advance_to() twice in a row, lower bound may or may not have changed.
Simplifies the code a bit, but also will make it easier to calculate
the next position we should skip to after forwarding, taking into
consideration both the position forwarded to as well as clustering
ranges of the query. That will be just calling
_ck_ranges_walker->lower_bound() after it is trimmed.
In general mp_row_consumer has better information about the next
position to read. It could be after the position we forward to if
there are clustering restrictions. This will be exploited in the
following patches.
Problem is that column family field of task wasn't being set for resharding,
so column family wasn't being properly removed from compaction manager.
In addition to fixing this issue, we'll also interrupt ongoing compactions
when dropping a column family, exactly like we do with shutdown.
Fixes#2291.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <20170418125807.7712-1-raphaelsc@scylladb.com>
streaming generates lots of small sstables with large token range,
which triggers O(N^2) in space in interval map.
level 0 sstables will now be stored in a structure that has O(N)
in space complexity and which will be included for every read.
Fixes#2287.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <20170417185509.6633-1-raphaelsc@scylladb.com>
"This series fixes some errors found by clang, with the aim of enabling
clang/zapcc as a supported compiler. A few more fixes are needed to produce
a binary."
* tag 'clang/1/v1' of https://github.com/avikivity/scylla:
logalloc: avoid auto in function argument declaration
thrift: avoid auto in function argument declaration
streamed_mutation: fix non-POD argument to C-style variadic function
mutation_partition_serializer: avoid auto in function argument declaration
date: use correct casts for years
streaming: avoid auto in function argument declaration
repair: avoid auto in function argument declaration
gms: expose gms::inet_address streaming operator
murmur3_partitioner: fix build on clang
i_partitioner: remove unused function
byte_ordered_partitioner: fix bad operator precedence
result_set: pass comparator by reference to std::sort()
to_string: move standard container overloads of to_string to std:: namespace
cql_type: fix bad enum syntax on clang
build: disable more warnings for clang
build: fix detection of unsupported warnings on clang
'auto' in a non-lambda function argument is not legal C++, and is hard
to read besides. Replace with the right type.
Since the right type is private, add some friendship.
Clang warns that passing a non-POD to a C-style variadic function will
result in an abort(). That happens to be exactly what we want, but to
silence the warning, use a template instead. Since templates aren't
allowed in local classes, move the containing class to namespace scope.
The standard says, and clang enforces, that declaring a function via
a friend declaration is not sufficient for ADL to kick in. Add a namespace
level declaration so ADL works.
Clang complains about some error without it, I could not understand it, but
I'm not going to argue with it.
Since std::sort() will copy the comparator, it's better to pass using an
std::ref(), and everyone is happy.
Argument-dependent lookup will not find to_string() overloads in the global
namespace if the argument and the caller are in other namespaces.
Move these to_string() overloads to std:: so ADL will find them.
Found by clang.
The diagnostic that clang spits out when it sees an unrecognized warning
is itself a warning, so the test compilation succeeds and we don't notice
the warning is not supported.
Adding -Werror turns the warning about the unrecognized warning into an
error, allowing the detection machinery to work.
The blocked task detector introduced in
113ed9e963 was seeing
the initialization phase of perf_ssttable as a blocked
task.
Tranform this part of the code in a futurized loop
to make to blocked task detector happy.
Signed-off-by: Benoît Canet <benoit@scylladb.com>
Message-Id: <20170413132506.17806-1-benoit@scylladb.com>
"The version of create_index_statement class that was translated to C++
is pretty old by now. This series of cleanups brings it closer to Apache
Cassandra trunk to make it easier to bring over more secondary index
code to Scylla."
* 'penberg/create-index-stmt-cleanup/v1' of github.com:cloudius-systems/seastar-dev:
cql3/statements/create_index_statement: Move target validation
cql3/statements/create_index_statement: Remove static column validation
cql3/statements/create_index_statement: Extract validations
cql3/statements/create_index_statement: Kill bogus custom validation
cql3/statements/create_index_statement: Add materialized view to validate()
cql3/statements/create_index_statement: Remove validation
We take a reference of endpoint_state entry in endpoint_state_map. We
access it again after code which defers, the reference can be invalid
after the defer if someone deletes the entry during the defer.
Fix this by checking take the reference again after the defering code.
I also audited the code to remove unsafe reference to endpoint_state_map entry
as much as possible.
Fixes the following SIGSEGV:
Core was generated by `/usr/bin/scylla --log-to-syslog 1 --log-to-stdout
0 --default-log-level info --'.
Program terminated with signal SIGSEGV, Segmentation fault.
(this=<optimized out>) at /usr/include/c++/5/bits/stl_pair.h:127
127 in /usr/include/c++/5/bits/stl_pair.h
[Current thread is 1 (Thread 0x7f1448f39bc0 (LWP 107308))]
Fixes#2271
Message-Id: <529ec8ede6da884e844bc81d408b93044610afd2.1491960061.git.asias@scylladb.com>
This introduce offline installer generator.
It will generate self-extractable archive witch contains Scylla packages
and dependency packages.
Package installation automatically starts when the archive executed.
Limitation: Only supported CentOS at this point.
Fixes#2268
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1491997091-15323-1-git-send-email-syuu@scylladb.com>
"This series was initially meant to only transition the keyspace based backend to work
on top of prepared statements but there were a few potential issues found on the way.
In addition the original Tracing series has been expanded with a few patches in the cql3 layer
that are improving the generic clq3 layer but are not obvious without the context of the
following Tracing patches.
The "main" patch contains a heavy rework of trace_keyspace_helper:
- Use prepared statements for updating tables instead of manually constructing mutations:
- We intentionally decrease the level of code robustness from "paranoid" to "normal".
- The code gets a lot more simple, e.g. we don't need to cache columns definitions any more.
- We are loosing some performance here but:
- Tracing write is not in the fast path.
- Tracing write events should be rare.
- Currently the performance loss (for the actual write time of all trace records) for a "SELECT" query with a specific key is
about 45%: 144us vs 99us."
* 'tracing_rework_using_prepared-v6' of github.com:cloudius-systems/seastar-dev:
tracing: use prepared statment for updating tables
tracing::trace_keyspace_helper: add a bad_column_family constructor that accepts an std::exception parameter
tracing::trace_keyspace_helper: introduce a table_helper class
tracing::trace_keyspace_helper: add static qualifier to make_monotonic_UUID_tp() and elapsed_to_micros() methods
tracing::tracing: allow slow query TTL only in the signed 32-bit integer range
cql3::query_processor::prepare(): futurize the error case
cql3::query_options: add a factory method for creation of options for a BATCH statement
cql3::statements::batch_statement: add a constructor that doesn't receive the "bound_terms" value
cql3::query_processor: use weak_ptr for passing the prepared statements around
When compacting a fully expired sstable, we're not allowing that sstable
to be purged because expired cell is *unconditionally* converted into a
dead cell. Why not check if the expired cell can be purged instead using
gc before and max purgeable timestamp?
Currently, we need two compactions to get rid of a fully expired sstable
which cells could have always been purged.
look at this sstable with expired cell:
{
"partition" : {
"key" : [ "2" ],
"position" : 0
},
"rows" : [
{
"type" : "row",
"position" : 120,
"liveness_info" : { "tstamp" : "2017-04-09T17:07:12.702597Z",
"ttl" : 20, "expires_at" : "2017-04-09T17:07:32Z", "expired" : true },
"cells" : [
{ "name" : "country", "value" : "1" },
]
now this sstable data after first compaction:
[shard 0] compaction - Compacted 1 sstables to [...]. 120 bytes to 79
(~65% of original) in 229ms = 0.000328997MB/s.
{
...
"rows" : [
{
"type" : "row",
"position" : 79,
"cells" : [
{ "name" : "country", "deletion_info" :
{ "local_delete_time" : "2017-04-09T17:07:12Z" },
"tstamp" : "2017-04-09T17:07:12.702597Z"
},
]
now another compaction will actually get rid of data:
compaction - Compacted 1 sstables to []. 79 bytes to 0 (~0% of original)
in 1ms = 0MB/s. ~2 total partitions merged to 0
NOTE:
It's a waste of time to wait for second compaction because the expired
cell could have been purged at first compaction because it satisfied
gc_before and max purgeable timestamp.
Fixes#2249, #2253
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <20170413001049.9663-1-raphaelsc@scylladb.com>
