Use name of the existing preceeding column with restriction
(last_column) instead of assuming that the column right after the
current column already has restrictions.
This will yield an error message that is different from that of
Cassandra, albeit still a correct one.
Fixes#2421
Signed-off-by: Botond Dénes <bdenes@scylladb.com>
Message-Id: <40335768a2c8bd6c911b881c27e9ea55745c442e.1499781685.git.bdenes@scylladb.com>
DowngradingConsistencyRetryPolicy uses live replicas count from
Unavailable exception to adjust CL for retry, but when there are pending
nodes CL is increased internally by a coordinator and that may prevent
retried query from succeeding. Adjust live replica count in case of
pending node presence so that retried query will be able to proceed.
Fixes#2535
Message-Id: <20170710085238.GY2324@scylladb.com>
"most of changes are to improve maintainability of the strategy but
the ones that are introduced by the following patches:
lcs: do not check if level 0 can be promoted twice
lcs: remove quadratic behavior from L0 compaction
lcs: partially sort candidates that will be trimmed
lcs: only demote sstable from level higher than target one"
* 'lcs_improvements_2' of github.com:raphaelsc/scylla:
lcs: only demote sstable from level higher than target one
lcs: improve indentation for get_overlapping_starved_sstables
lcs: improve indentation for get_compaction_candidates
lcs: partially sort candidates that will be trimmed
lcs: remove quadratic behavior from L0 compaction
lcs: introduce private interface
lcs: make some member functions static
lcs: make some functions const qualified
lcs: remove add method
lcs: extract code for higher levels compaction from get_candidates_for
lcs: simplify code to get candidates for higher levels
lcs: extract round-robin heuristic for even distribution of keys into function
lcs: update outdated comments for level 0 compaction
lcs: improve worth_promoting_L0_candidates interface
lcs: do not check if level 0 can be promoted twice
lcs: extract code for level 0 compaction from get_candidates_for
CQL reply may contain metadata that describes columns present in the
response including the information about their type.
However, Scylla incorrectly reports counter types as bigint. The
serialised format of counters and bigint is exactly the same, which
could explain why the problem hasn't been noticed earlier but it is a
bug nevertheless.
Fixes#2569.
Message-Id: <20170711130520.27603-1-pdziepak@scylladb.com>
if we are compacting level 1 into level 2, we only want to demote
a sstable from level 3 or higher.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
L0 compaction triggers quadratic behavior when many newly created
sstables are needed for promotion due to their size being relatively
low to max sstable size parameter. So until L0 is worth promoting,
the strategy will compact every new sstable with all the existing
ones in L0. To fix it, let's do STCS on level 0 until it becomes
worth promoting.
Fixes#2432.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
get rid of unneeded loop for dealing with suspect sstables and
std::advance because vector allows random access.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
some comments are no longer relevant, especially the ones that
talk about dealing with busy sstables due to parallel compaction,
which isn't done by us for lcs.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
can_promote flag will be used to carry info about whether or not
level 0 can promoted. That will avoid a single iteration for higher
levels too which can contain tens of thousands of sstables.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
I will split code for higher levels compaction into functions first
before putting it into its own function too.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
The default of 2ms is somewhat arbitrary. Now that we have a lot more
mileage deploying Scylla applications in production it does sound not
only arbitrary, but high.
In particular, it is really hard to achieve 1ms latencies in the face of
CPU-heavy workloads with it.
Signed-off-by: Glauber Costa <glauber@scylladb.com>
Message-Id: <1499354495-27173-1-git-send-email-glauber@scylladb.com>
Otherwise we may deadlock, as explained in commit 5e8f0efc8:
Table drop starts with creating a snapshot on all shards. All shards
must use the same snapshot timestamp which, among other things, is
part of the snapshot name. The timestamp is generated using supplied
timestamp generating function (joinpoint object). The joinpoint object
will wait for all shards to arrive and then generate and return the
timestamp.
However, we drop tables in parallel, using the same joinpoint
instance. So joinpoint may be contacted by snapshotting shards of
tables A and B concurrently, generating timestamp t1 for some shards
of table A and some shards of table B. Later the remaining shards of
table A will get a different timestamp. As a result, different shards
may use different snapshot names for the same table. The snapshot
creation will never complete because the sealing fiber waits for all
shards to signal it, on the same name.
Message-Id: <1499762663-21967-1-git-send-email-tgrabiec@scylladb.com>
"most of changes are to improve maintainability of the strategy but
the ones that are introduced by the following patches:
lcs: do not check if level 0 can be promoted twice
lcs: remove quadratic behavior from L0 compaction
lcs: partially sort candidates that will be trimmed
lcs: only demote sstable from level higher than target one"
* 'lcs_improvements' of github.com:raphaelsc/scylla: (21 commits)
lcs: only demote sstable from level higher than target one
lcs: improve indentation for get_overlapping_starved_sstables
lcs: improve indentation for get_compaction_candidates
lcs: partially sort candidates that will be trimmed
lcs: remove quadratic behavior from L0 compaction
lcs: introduce private interface
lcs: make some member functions static
lcs: make some functions const qualified
lcs: remove add method
lcs: extract code for higher levels compaction from get_candidates_for
lcs: simplify code to get candidates for higher levels
lcs: extract round-robin heuristic for even distribution of keys into function
lcs: update outdated comments for level 0 compaction
lcs: improve worth_promoting_L0_candidates interface
lcs: do not check if level 0 can be promoted twice
lcs: extract code for level 0 compaction from get_candidates_for
dist/offline_installer: add --skip-setup option to offline installer
dist/offline_installer/debian: install python-minimal package before installing scylla deps
migration_manager: Give empty response to schema pulls from incompatible nodes
migration_manager: Don't pull schema from incompatible nodes
...
if we are compacting level 1 into level 2, we only want to demote
a sstable from level 3 or higher.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
L0 compaction triggers quadratic behavior when many newly created
sstables are needed for promotion due to their size being relatively
low to max sstable size parameter. So until L0 is worth promoting,
the strategy will compact every new sstable with all the existing
ones in L0. To fix it, let's do STCS on level 0 until it becomes
worth promoting.
Fixes#2432.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
get rid of unneeded loop for dealing with suspect sstables and
std::advance because vector allows random access.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
some comments are no longer relevant, especially the ones that
talk about dealing with busy sstables due to parallel compaction,
which isn't done by us for lcs.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
can_promote flag will be used to carry info about whether or not
level 0 can promoted. That will avoid a single iteration for higher
levels too which can contain tens of thousands of sstables.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
I will split code for higher levels compaction into functions first
before putting it into its own function too.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
