range_tombstone_list::apply() has no exception safety guarantees about
the logical state. The target mutation_partition in cache should be
assumed to be left in unspecified state. In particular, some of the
preexisting overlapping tombstones may be removed and not reinserted,
so the cache would be missing some of the range tombstone information
in case the whole allocating section fails.
Use apply_monotonically() which provides the needed guarantees.
Fixes#2938.
erase_undo_op() constructor takes ownership of *it, and destroys it
when it goes out of scope. If emplace_back() fails, *it would be
destroyed before being removed from its container (_dst._tombstones).
Fix by making sure _ops.emplace_back() won't fail.
If exception is thrown from _row_tombstones.apply(), _rows will be
left uncleared. This will trigger assertion in bi::set_member_hook
destructor, which assrts that the hook is not linked.
Always clear _rows.
Because of the small size optimization, not all copies will call the
allocator, so allocation failure injection may miss this site if the
value is not large enough. Make the testing more effective by marking
this place explicitly as an allocation point.
* seastar d71922c...8040cab (4):
> util: Introduce support for allocation failure injection
> Adding dpdk-port-index as a command line option with default value of 0
> core/sharded: Introduce invoke_on_others()
> noncopyable_function: improve support for capturing mutable lambdas
"Fixes #2933
Fixes regressions introduced by config restructuring.
Allows "base" config to handle errors by warning, while
other uses can opt otherwise."
[pdziepak: resolved merge conflict]
* 'calle/cfgfix' of github.com:scylladb/seastar-dev:
config_test: Use error handler (ignore errors) + add error test
config: Resurrect command line aliases that where lost
main: Use error handler for config parse
config_file: Add optional "error_handler" to yaml parse functions
"This patch series adds support for secondary index queries using the
backing index view that's created when CREATE INDEX statement is
executed.
Example:
-- Create keyspace and table:
CREATE KEYSPACE ks WITH REPLICATION = {'class' : 'SimpleStrategy', 'replication_factor' : 1};
CREATE TABLE ks.users (
userid uuid,
name text,
email text,
country text,
PRIMARY KEY (userid)
);
-- Create secondary indexes:
CREATE INDEX ON ks.users (email);
CREATE INDEX ON ks.users (country);
-- Insert some data:
INSERT INTO ks.users (userid, name, email, country) VALUES (uuid(), 'Bondie Easseby', 'beassebyv@house.gov', 'France');
INSERT INTO ks.users (userid, name, email, country) VALUES (uuid(), 'Demetri Curror', 'dcurrorw@techcrunch.com', 'France');
INSERT INTO ks.users (userid, name, email, country) VALUES (uuid(), 'Langston Paulisch', 'lpaulischm@reverbnation.com', 'United States');
INSERT INTO ks.users (userid, name, email, country) VALUES (uuid(), 'Channa Devote', 'cdevote14@marriott.com', 'Denmark');
-- Query on the secondary-index backed non-primary keys:
SELECT * FROM ks.users WHERE email = 'beassebyv@house.gov';
userid | country | email | name
--------+---------+-------+------
022238c8-5213-44b5-959e-4e3e1b032f85 | France | beassebyv@house.gov | Bondie Easseby
(1 rows)
SELECT * FROM ks.users WHERE country = 'France';
userid | country | email | name
--------------------------------------+---------+-------------------------+----------------
2152d85a-61f6-4eab-af4d-e7e7d0872319 | France | beassebyv@house.gov | Bondie Easseby
59fddb6d-bfc9-4636-a9a0-85383fd815ee | France | dcurrorw@techcrunch.com | Demetri Curror
Known imitations:
- Only regular column indexes return results. Indexing primary key
components like clustering keys return empty result set because of
index view query partition key serialization issues that will be fixed
in subsequent patches.
- Secondary index queries are not paginated, which can cause problems
for queries that return a large number of rows.
- Multiple restrictions don't work correctly if one of them is backed by
a secondary-index.
- Only one secondary-indexed restriction per query is supported -- other
restrictions are ignored.
- Compound partition keys are not supported.
- ALLOW FILTERING on non-primary key columns does not work correctly
without secondary index (see issue #2200)."
* 'penberg/cql-2i-queries/v2' of github.com:penberg/scylla:
tests/cql_query_test: Add test case for secondary index queries
cql3: Secondary-index backed select statements
index: Fix index view schema when primary key component is indexed
tests/cql_query_test: Fix view creation in test_duration_restrictions()
cql3/restrictions: Add statement_restrictions::index_restrictions() helper
index: Implement index::supports_expression() for EQ operator
cql3: Make operator_type class non-copyable
index: Fix index::supports_expression() operator parameter type
cql3: Implement statement_restriction index validation
Before the patch we appended and queried at the front. Insert at the
front instead, so that writes and reads overlap. Stresses eviction and
population more.
Message-Id: <1506369562-14892-1-git-send-email-tgrabiec@scylladb.com>
"We currently can't insert row entries at any position_in_partition,
but only at full keys and after all keys. If a query range has bounds
such that we have to insert a dummy entry at non-representable position
then information about range continuity will not be fully populated.
In particular, single-row queries of a row which is not present in sstables
will miss when repeated again.
The series fixes the problem by marking the whole query range as continuous
by inserting dummy entries at boundaries when necessary.
Refs #2579."
* tag 'tgrabiec/cache-range-continuity-v2' of github.com:scylladb/seastar-dev:
tests: row_cache: Add test for population of single rows
tests: Add test for population of continuity
tests: mutation_reader_assertions: Introduce produces_compacted()
mutation: Introduce apply(mutation_fragment)
cache: Document invariants of cache_streamed_mutation::_lower_bound
cache_streamed_mutation: Special-case population for singular ranges
query: Introduce is_single_row()
cache_streamed_mutation: Increment mispopulation counter when can't populate due to eviction
cache_streamed_mutation: Override continuity of older versions when populating
cache_streamed_mutation: Mark whole query range as continuous
tests: cache_streamed_mutation: Allow creating expected_row at any position_in_partition
cache_streamed_mutation: Populate continuity when range adjacent to non-latest version rows
cache_streamed_mutation: Avoid lookup in maybe_add_to_cache() in more cases
row_cache: Make read_context::key() valid before reading from underlying starts
mutation_partition: Allow creating rows_entry at any clustered position_in_partition
position_in_partition: Do not use -2 and +2 weights
clustering_ranges_walker: Make contains() drop range tombstones adjacent to query range
mutation_partition: Remove delegating_compare()
mvcc: Print iterators in operator<< for partition_snapshot_row_cursor
mvcc: Introduce partition_snapshot_row_weakref
mvcc: Make the null state of partition_snapshot::change_mark explicit
mvcc: Add partition_snapshot::region() getter
mvcc: Add partition_snapshot::schema() getter
position_in_partition: Introduce before_key()
position_in_partition: Introduce min()
position_in_partition: Introduce for_static_row()
While not being a real unit tests memory_footprint can be a quite useful
tool and running it among other tests will ensure that we will notice
when it gets broken.
Message-Id: <20171102160233.6756-2-pdziepak@scylladb.com>
When created cache registers several metrics, since attempts to create
an already existing metrics result in an exception being thrown it is no
longer possible to have two cache instances at the same time. This is
exactly what happens in memory_footprint: one (useless) cache object is
created through a call to do_with_cql_env() and, then, memory_footprint
explicitly creates another one (not a useless one).
The tests itself doesn't really need a full cql environment and the only
reason it was added is so that storage_service is initialised and various
code paths can query for the available cluster features. This can be
done in a much lightweight way using storage_service_for_tests.
Fixes memory_footprint failure (until next time we decide there is
nothing wrong with globals).
Message-Id: <20171102160233.6756-1-pdziepak@scylladb.com>
$basearch isn't parsed as expected, the finaly baseurl is wrong.
We only have x86_64 arch in external 3rdparty repository, and
the conf file is only for x86_64, so it's fine to use hardcode
x86_64.
The problem was introduced by commit
b5e83ebd94 ("dist/redhat: switch 3rdparty
packages to external build service").
Fixes#2930
Signed-off-by: Amos Kong <amos@scylladb.com>
Message-Id: <708f46a7c36623e86fee278462c80db1eff3b820.1509700430.git.amos@scylladb.com>
This patch adds support for secondary-index backed select statements.
Current select_statement class is split into two separate classes:
primary_key_select_statement that retains regular query behavior and
indexed_table_select_statement that introduces the new secondary-index
backed query logic. One of the two behaviors is selected at query
preparation time to minimize overhead for non-indexed queries.
This fixes index view schema to exclude indexed column when a primary
key component like clustering key is indexed. This fixes a server crash
when CREATE INDEX statement is executed on a clustering key column.
The materialized view created in test_duration_restriction() restricts
on a non-PK column. Since Scylla's ALLOW FILTERING and secondary index
validation path is broken, once we start to do secondary index queries,
query processor thinks there's a secondary index backing that non-PK
column and fails because it's unable to find such column.
Fix up the view to only trigger the duration type validation error we're
interested in here.
Fixes the case of continuity not being populated when the row which is
the upper bound of the population range belongs to a non-latest
version. In such case we wouldn't mark the range as continuous,
because we can't modify rows of non-latest versions. To fix this,
create an empty entry in latest version which will just override the
continuity flag of the old entry.
Before this patch only ranges between returned row fragments were
marked as continuous. In the extreme case, there could be no such
fragments, in which case next read would miss as well. To avoid this,
mark whole query range as continuous by inserting dummy entries when
necessary.
Refs #2579.
Current code will not mark the range as continuous if the previous
entry does not come in the latest version. Fix that by switching
to partition_snapshot_row_pointer, which is capable of checking
in older versions as necessary.
Also, we avoid the key comparison if we know that the iterator
is still valid.
So that we can call cache_streamed_mutation::can_populate() before
we start reading from underlying. Will be needed in upcoming changes
which insert dummy entries when falling back to underlying.
::weight() is using those values for excl_end and excl_start in order
to be able to represent non-overlapping ranges. In their model the end
bound is inclusive. We don't need this, since position_range has end
bound exclusive.
This change makes that:
position_in_partition::after_key(y)
== position_in_partition::for_range_end(clutering_range::make({x}, {y})
