"
Cleanup various cases related to updating of metatdata stats and encoding stats
updating in preparation for 64-bit gc_clock (#3353).
Fixes#4026Fixes#4033Fixes#4035Fixes#4041
Refs #3353
"
* 'projects/encoding-stats-fixes/v6' of https://github.com/bhalevy/scylla:
sstables: remove duplicated code in data_consume_rows_context CELL_VALUE_BYTES
sstables: mc: use api::timestamp_type in write_liveness_info
sstables: mc: sstable_write encoding_stats are const
mp_row_consumer_k_l::consume_deleted_cell rename ttl param to local_deletion_time
memtable: don't use encoding_stats epochs as default
memtable: mc: udpate min_ttl encoding stats for dead row marker
memtable: mc: add comment regarding updating encoding stats of collection tombstones
sstables: metadata_collector: add update tombstone stats
sstables: assert that delete_time is not live when updating stats
sstables: move update_deletion_time_stats to metadata collector
sstables: metadata_collector: introduce update_local_deletion_time_and_tombstone_histogram
sstables: mc: write_liveness_info and write_collection should update tombstone_histogram
sstables: update_local_deletion_time for row marker deletion_time and expiration
Presence checker is constructed and destroyed in the standard
allocator context, but the presence check was invoked in the LSA
context. If the presence checker allocates and caches some managed
objects, there will be alloc-dealloc mismatch.
That is the case with LeveledCompactionStrategy, which uses
incremental_selector.
Fix by invoking the presence check in the standard allocator context.
Fixes#4063.
Message-Id: <1547547700-16599-1-git-send-email-tgrabiec@scylladb.com>
"
This mini-series adds counters for the inactive reads registered in the
reader concurrency semaphore.
"
* 'reader-concurrency-semaphore-counters/v6' of https://github.com/denesb/scylla:
tests/querier_cache: use stats to get the no. of inactive reads
reader_concurrency_semaphore: add counters for inactive reads
The existence of LZ4_compress_default is a property of the lz4
library, not seastar.
With this patch scylla does its own configure check instead of
depending on the one done by seastar.
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
Message-Id: <20190114013737.5395-1-espindola@scylladb.com>
"
Currently the logic is scattered between types.*, cql3_types.* and
sstables/mc/writer.cc.
This patchset places all the logic in types.* and makes sure we
correctly add "frozen<...>" and "FrozenType(...)" to the names of
tuples and UDTs.
Fixes#4087
Tests: unit(release)
"
* 'haaawk/4087_v1' of github.com:scylladb/seastar-dev:
Add comment explaining tuple type name creation
Add "FrozenType(...)" to UDT name only when it's frozen
Move "FrozenType(...)" addition to UDT name to user_type_impl
Add "frozen<...>" to tuple CQL name only when it's frozen
Move "frozen<...>" addition to tuple CQL name to tuple_type_impl
Merge make_cql3_tuple_type into tuple_type_impl::as_cql3_type
Add "frozen<...>" to UDT CQL name only when it's frozen
Move "frozen<...>" addition to UDT CQL name to user_type_impl
Why default to an artificial minimum when you can do better
with zero effort? Track the actual minima in the memtable instead.
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
Update min ttl with expired_liveness_ttl (although it's value of max int32
is not expected to affect the minimum).
Fixes#4041
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
When the row flag has_complex_deletion is set, some collection columns may have
deletion tombstones and some may not. we don't strictly need to update stats
will not affect the encoding_stats anyway.
Fixes#4035
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
Now that continuous_data_consumer::position() is meaningful (since
36dd660), we can use our position in the stream to calculate offsets
instead of duplicating state machine in offset calculations.
The value of position() - data.size() always holds the current offset
in the stream.
Message-Id: <1547219234-21182-1-git-send-email-tgrabiec@scylladb.com>
Put package names one per line. This makes it easier to review changes,
and to backport changes to this file. No content changes.
Message-Id: <20190112091024.21878-1-avi@scylladb.com>
Do not allow write access to the sstable list via this accessor. Luckily
there are no violations, and now we enforce it.
Message-Id: <20190111151049.16953-1-avi@scylladb.com>
To keep format compatibiliti we never wrap tuple type name
into "org.apache.cassandra.db.marshal.FrozenType(...)".
Even when the tuple is frozen.
This patch adds a comment in tuple_type_impl::make_name that
explains the situation.
For more details see #4087
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
At the moment Scylla supports only frozen UDTs but
the code should be able to handle non-frozen UDTs as well.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
At the moment Scylla supports only frozen tuples but
the code should be able to handle non-frozen tuples as well.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
At the moment Scylla supports only frozen UDTs but
the code should be able to handle non-frozen UDTs as well.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Race condition takes place when one of the sstables selected by snapshot
is deleted by compaction. Snapshot fails because it tries to link a
sstable that was previously unlinked by compaction's sstable deletion.
Fixes#4051.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <20190110194048.26051-1-raphaelsc@scylladb.com>
sstable_file_io_extensions() return an array of pointers to extensions,
but do_for_each() may defer and the array will be destroyed. The match
keeps it alive until do_for_each completes.
Message-Id: <20190110125656.GC3172@scylladb.com>
make_sstable_reader needs to deal with single-key and scanning reads, and
with restricting and non-restricting (in terms of read concurrency) readers.
Right now it does this combinatorically - there are separate cases for
restricting single-key reads, non-restricting single-key reads, restricing
scans, and non-restricting scans.
This makes further changes more complicated, so separate the two concepts.
The patch splits the code into two stages; the first selects between a single-key
and a scan, and the second selects between a restricting and non-restricting read.
This slightly pessimizes non-restricting reads (a mutation_source is created and
immediately destroyed), but that's not the common case.
Tests: unit(release)
Message-Id: <20190109175804.9352-1-avi@scylladb.com>
compare_row_marker_for_merge compares deletion_time also for row markers
that have missing timestamps. This happened to succeed due to implicit
initialization to 0. However, we prefer the initialization to be explicit
and allow calling row_marker::deletion_time() in all states.
Fixes#4068
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
Message-Id: <20190110102949.17896-1-bhalevy@scylladb.com>
Serialization header stores column types for all
columns in sstable. If any of them is a UDT then it
has to be wrapped into
"org.apache.cassandra.db.marshal.FrozenType(...)".
This patch adds a test case to verify that.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Serialization header stores type names of all
columns in a table. Including partition key columns,
clustering key columns, static columns and regular columns.
If one of those types is a user defined type then we need to
wrap its name into
"org.apache.cassandra.db.marshal.FrozenType(...)".
Fixes#4073
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
This renames some variables and functions to make it clear that they
refer to partitions and not rows.
Old versions of sstablemetadata used to refer to a row histogram, but
current versions now mention a partition histogram instead.
This patch doesn't change the exposed API names.
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
Message-Id: <20181229223311.4184-2-espindola@scylladb.com>
Both build_rpm.sh/build_deb.sh are failing at beginning of the script
when relocatable package does not exist, need to prevent it and show
user friendly message.
Fixes#4071
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20190109094353.16690-1-syuu@scylladb.com>
Compaction manager holds reference to all cleaning sstables till the very
end, and that becomes a problem because disk space of cleaned sstables
cannot be reclaimed due to respective file descriptors opened.
Fixes#3735.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <20181221000941.15024-1-raphaelsc@scylladb.com>
Replace stdx::optional and stdx::string_view with the C++ std
counterparts.
Some instances of boost::variant were also replaced with std::variant,
namely those that called seastar::visit.
Scylla now requires GCC 8 to compile.
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20190108111141.5369-1-duarte@scylladb.com>
