Commit Graph

22 Commits

Author SHA1 Message Date
Avi Kivity
fcb8d040e8 treewide: use Software Package Data Exchange (SPDX) license identifiers
Instead of lengthy blurbs, switch to single-line, machine-readable
standardized (https://spdx.dev) license identifiers. The Linux kernel
switched long ago, so there is strong precedent.

Three cases are handled: AGPL-only, Apache-only, and dual licensed.
For the latter case, I chose (AGPL-3.0-or-later and Apache-2.0),
reasoning that our changes are extensive enough to apply our license.

The changes we applied mechanically with a script, except to
licenses/README.md.

Closes #9937
2022-01-18 12:15:18 +01:00
Avi Kivity
a55b434a2b treewide: extent copyright statements to present day 2021-06-06 19:18:49 +03:00
Avi Kivity
a96b8e8aed sstables: read_vint(): convert enable_if to concepts
Convert read_vint() to a concept. The explicitly deleted version
is no longer needed since wrongly-typed inputs will be rejected
by the constraint. Similarly the static assert can be dropped
for the same reason.
2021-03-18 18:45:05 +02:00
Avi Kivity
a00fca1a69 sstables: move bound_kind_m formatter to its natural place
Move bound_kind_m's formatter to the same header file where
is is defined. This prevents cases where the compiler decays
the type (an enum) to the underlying integral type because it
does not see the formatter declaration, resulting in the wrong
output.
2020-10-12 20:36:10 +03:00
Avi Kivity
69c3533d97 sstables: deinline bound_kind_m formatter
The formatter is by no means hot code and should not be inlined.
2020-10-12 20:35:08 +03:00
Benny Halevy
12393c5ec2 sstables: rename mc folder to mx
Prepare for supporting the md format.

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
2020-08-10 18:53:04 +03:00
Benny Halevy
89b3974e56 sstables: print invalid boundary type as unsigned int
Otherwise it prints a binary value to the log and corrupting it.
Seen when testing scrub with randomly-corrupted sstable
using scrub_with_one_node_expect_data_loss_test
as of https://github.com/scylladb/scylla-dtest/pull/1414

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
Message-Id: <20200407055617.1045977-1-bhalevy@scylladb.com>
2020-04-07 10:18:19 +02:00
Benny Halevy
93270dd8e0 gc_clock: make 64 bit
Fixes: #3353

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
2019-01-22 15:34:32 +02:00
Benny Halevy
1ccd72f115 sstables: mc: use int64_t for local_deletion_time and ttl
In preparation for changing gc_clock::duration::rep to int64_t.

Refs #3353

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
2019-01-22 15:34:32 +02:00
Benny Halevy
33314cec3f sstables: safely convert ttl and local_deletion_time to int32_t
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
2019-01-22 15:34:32 +02:00
Benny Halevy
844a2de263 sstables: mc: prevent signed integer overflow
Fix runtime error: signed integer overflow
introduced by 2dc3776407

Delta-encoded values may wrap around if the encoded value is
less than the base value.  This could happen in two places:
In the mc-format serialization header itself, where the base values are implicit
Cassandra epoch time, and in the sstables data files, where the base values
are taken from the encoding_stats (later written to the serialization_header).

In these cases, when the calculation is done using signed integer/long we may see
"runtime error: signed integer overflow" messages in debug mode
(with -fsanitize=undefined / -fsanitize=signed-integer-overflow).

Overflow here is expected and harmless since we do not gurantee that
neither the base values in the serialization header are greater than
or equal to Cassandra's epoch now that the delta-encoded values are
always greater than or equal to the respective base values in
the serialization header.

To prevent these warnings, the subtraction/addition should be done with unsigned
(two's complement) arithmetic and the result converted to the signed type.

Note that to keep the code simple where possible, when also rely on implicit
conversion of signed integers to unsigned when either one of added value is unsigned
and the other is signed.

Fixes: #4098

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
Message-Id: <20190120142950.15776-1-bhalevy@scylladb.com>
2019-01-20 16:59:46 +02:00
Benny Halevy
60323b79d1 sstables: mc: sign-extend delta local_deletion_time and delta ttl
Follow Cassandra's encoding so that values that are less than the
baseline encoding_stats will wrap-around in 64-bits rather tham 32.

Fixes #4074
Refs #3353

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
Message-Id: <20190109192703.18371-1-bhalevy@scylladb.com>
2019-01-09 21:43:30 +02:00
Tomasz Grabiec
bd7e9ad3ab sstables: mc: Extract bound_kind_m related stuff into mc/types.hh 2018-12-12 12:06:46 +01:00
Vladimir Krivopalov
e71cc5ab20 sstables: Introduce TTL limitation and special 'expired TTL' value.
This allows to store expired liveness info in SSTables 3.x format
without introducing a possible conflict with real TTL values.

As per Cassandra, TTL cannot exceed 20 years so taking the maximum value
as a special value for indicating expired liveness info is safe.

Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
2018-10-10 11:44:14 -07:00
Vladimir Krivopalov
fc901eb700 sstables: Fix string formatting for exception messages in m_format_read_helpers.
Before this fix, the code was a potential undefined behaviour and crash
because it would add a large value to a const char* and try to create a
std::string out of it.

Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
2018-09-25 17:23:40 -07:00
Vladimir Krivopalov
84341821b1 sstables: Don't validate timestamps against the max value on parsing.
Internally, timestamps are represented as signed integers (int64_t) but
stored as unsigned ones. So it is quite possible to store data with
timestamp that is represented as a number larger than the max value of
int64_t type.
One such example is api::min_timestamp() that is used when generating
system schema tables ("keyspaces"). When cast to uint64_t, it turns into
a large value.

Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
2018-09-25 17:23:40 -07:00
Vladimir Krivopalov
bdca27ae41 sstables: Always store only min bases in serialization_header.
There previously was an inconsistency in treating min values stored in a
serialization_header. They are written to or read from a Statistics.db
as deltas against fixed bases, but when we parse timeouts from the data
file, we need the full bases, not just deltas.

This inconsistency causes wrong timestamp values if we write an sstable
and then read from it using one and the same sstable object because we
turn min values into bases on write and then don't adjust them back
because we already have them in memory.

Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
2018-09-25 17:23:40 -07:00
Vladimir Krivopalov
c8422c9a91 sstables: Add operator<< overload for bound_kind_m.
Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
2018-08-20 16:22:53 -07:00
Vladimir Krivopalov
917528c427 sstables: Read promoted index stored in SSTables 3.x ('mc') format.
Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
2018-07-20 13:51:13 -07:00
Piotr Jastrzebski
d0f8c71e28 sstables: move bound_kind_m to header
and add helper methods.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
2018-07-20 13:50:17 -07:00
Piotr Jastrzebski
b2f9841dd4 Add helper methods for parsing simple types.
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
2018-05-23 16:39:52 +02:00
Piotr Jastrzebski
6b4df2d424 Add helpers to read/parse vints
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
2018-05-09 15:46:46 +02:00