Instead of returning a coordinator result and making a caller parse it
later, read_posting_list now extracts rows by itself.
This change is later needed when querying is replaced with a pager.
A standard way for passing a timeout parameter is specifying
a time_point, while pagers used to take a duration in order
to compute time points on the fly. This patch adds a timeout
parameter, which is a time_point, to fetch_page().
This patchset addresses multiple errors in normalizing_reader
implementation found during review.
I have decided to not make a clustering key full inside
before_key()/after_key() helpers. The reason is that for this they
would need schema to be passed as another parameter so existing
methods don't suit. OTOH, introducing new members for a class using
for testing purposes only seems an overkill.
* github.com/argenet/scylla.git projects/sstables-30/normalizing_reader_fixes/v1:
range_tombstone: Add constructor accepting position_in_partition_views
for range bounds.
tests: Make sure range tombstone is properly split over rows with
non-full keys.
tests: Multiple fixes for draining and clearing range tombstones in
normalizing_reader.
We only wait from the last test case, so if an individual test is executed,
a memory leak may be reported.
Fix by waiting from all test cases.
Message-Id: <20180926203723.18026-1-avi@scylladb.com>
When executing a prepared select statement with a multicolumn IN, the
system returned incorrect results due to a memory violation (a bytes view
referring to an out of scope bytes object).
Added test for the prepared statement results correctness.
Tests:
1. unit (release) with the new test.
2. Python script.
Signed-off-by: Eliran Sinvani <eliransin@scylladb.com>
Message-Id: <36c9cf9ed3fe72e3b4801e3cd120678429ce218a.1537947897.git.eliransin@scylladb.com>
"
This patchset fixes several issues in SSTables 3.x ('mc') writing and
parsing and extends existing SSTables unit tests to cover the new
format.
The only test enabled temporarily is check_multi_schema because it
turned out that reading SSTables 3.x with a different schema has not
been implemented in full. This will be addressed in a separate patchset.
This patchset depends on the "Support SSTables 3.x in Scylla runtime"
patchset.
Tests: unit {release}
"
* 'projects/sstables-30/unit-tests/v3' of https://github.com/argenet/scylla: (25 commits)
tests: Enable existing SSTables tests for 'mc' format.
tests: Fix test_wrong_range_tombstone_order for 'mc' format.
tests: Extend reader assertions to check clustering keys made full.
tests: Disable test_old_format_non_compound_range_tombstone_is_read for 'mc' format.
tests: Disable check_multi_schema for 'mc' format.
tests: Fix test_promoted_index_read for 'mc' format by using normalizing_reader.
tests: Fix promoted_index_read to not rely on a specific index length
tests: Add 'mc' files for test_wrong_range_tombstone_order
tests: Add 'mc' files for test_wrong_counter_shard_order
tests: Add 'mc' files for summary_test
tests: Add 'mc' files for test_promoted_index_read
tests: Add 'mc' files for test_partition_skipping
tests: Add 'mc' files for large_partition tests (promoted_index_read, sub_partition_read, sub_partitions_read
tests: Add 'mc' files for test_counter_read
tests: Add 'mc' files for test_broken_promoted_index_is_skipped
tests: SSTables 'mc' files for sliced_mutation_reads_test.
tests: Introduce normalizing_reader helper for SSTables tests.
mutation_fragment: Add range_tombstone_stream::empty() method.
sstables: Make key full when setting a range tombstone start from end open marker.
sstables: For 'mc' format, use excl_start when split an RT over a row with a full key.
...
"
This patchset makes it possible to use SSTables 'mc' format, commonly
referred to as 'SSTables 3.x', when running Scylla instance.
Several bugs found on this way are fixed. Also, a configuration option
is introduced to allow running Scylla either with 'mc' or 'la' format
as default.
Tests: unit {release}
+ tested Scylla with both 'la' and 'mc' formats to work fine:
cqlsh> CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1}; [3/1890]
cqlsh> USE test;
cqlsh:test> CREATE TABLE cfsst3 (pk int, ck int, rc int, PRIMARY KEY (pk, ck)) WITH compression = {'sstable_compression': ''};
cqlsh:test> INSERT INTO cfsst3 (pk, ck, rc) VALUES ( 4, 7, 8);
<<flush>>
cqlsh:test> DELETE from cfsst3 WHERE pk = 4 and ck> 3 and ck < 8;
<<flush>>
cqlsh:test> INSERT INTO cfsst3 (pk, ck) VALUES ( 2, 3);
cqlsh:test> INSERT INTO cfsst3 (pk, ck) VALUES ( 4, 6);
cqlsh:test> SELECT * FROM cfsst3 ;
pk | ck | rc
----+----+------
2 | 3 | null
4 | 6 | null
(2 rows)
<<Scylla restart>>
cqlsh:test> INSERT INTO cfsst3 (pk, ck) VALUES ( 5, 7);
cqlsh:test> INSERT INTO cfsst3 (pk, ck) VALUES ( 6, 8);
cqlsh:test> INSERT INTO cfsst3 (pk, ck) VALUES ( 7, 9);
cqlsh:test> INSERT INTO cfsst3 (pk, ck) VALUES ( 8, 10);
cqlsh:test> SELECT * from cfsst3 ;
pk | ck | rc
----+----+------
5 | 7 | null
8 | 10 | null
2 | 3 | null
4 | 6 | null
7 | 9 | null
6 | 8 | null
(6 rows)
"
* 'projects/sstables-30/try-runtime/v8' of https://github.com/argenet/scylla:
database: Honour enable_sstables_mc_format configuration option.
sstables: Support SSTables 'mc' format as a feature.
db: Add configuration option for enabling SSTables 'mc' format.
tests: Add test for reading a complex column with zero subcolumns (SST3).
sstables: Fix parsing of complex columns with zero subcolumns.
sstables: Explicitly cast api::timestamp_type to uint64_t when delta-encoding.
sstables: Use parser_type instead of abstract_type::parse_type in column_translation.
bytes: Add helper for turning bytes_view into sstring_view.
sstables: Only forward the call to fast_forwarding_to in mp_row_consumer_m if filter exists.
sstables: Fix string formatting for exception messages in m_format_read_helpers.
sstables: Don't validate timestamps against the max value on parsing.
sstables: Always store only min bases in serialization_header.
sstables: Support 'mc' version parsing from filename.
SST3: Make sure we call consume_partition_end
This test is not applicable to the 'mc' format as it covers a backward
compatibility case which may only occur with SSTables generated by older
Scylla versions in 'ka' format.
Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
This is a helper flat_mutation_reader that wraps another reader and
splits range tombstones over rows before emitting them.
It is used to produce the same mutation streams for both old (ka/la) and
new (mc) SSTables formats in unit tests.
Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
This fixes the monotonicity issue as otherwise the range tombstone
emitted after such clustering row has a start position that should be
ordered before that of the row.
Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
If we go past the slice to be read with a range tombstone being opened
we need to emit an RT corresponding to this slice.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Only enable SSTables 'mc' format if the entire cluster supports it and
it is enabled in the configuration file.
Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
This flag will only be used for testing purposes until Scylla 3.o
release and will be removed once SSTables 'mc' testing is completed.
Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
Before this fix, a complex column with zero subcolumns would be
incorrecty parsed as it would re-read the deletion time twice.
Now, this case is handled properly.
Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
