Commit Graph

32 Commits

Author SHA1 Message Date
Benny Halevy
ec77777bda bytes: compare_unsigned: do not pass nullptr to memcmp
If any of the compared bytes_view's is empty
consider the empty prefix is same and proceed to compare
the size of the suffix.

A similar issue exists in legacy_compound_view::tri_comparator::operator().
It too must not pass nullptr to memcmp if any of the compared byte_view's
is empty.

Fixes #6797
Refs #6814

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>

Test: unit(dev)
Branches: all
Message-Id: <20200709123453.955569-1-bhalevy@scylladb.com>
2020-07-09 17:54:46 +03:00
Tomasz Grabiec
714da3c644 compound_compat: Allow constructing composite from a view 2020-06-16 16:15:23 +02:00
Botond Dénes
2e09a0317c types, compound: pass std::current_exception() to on_internal_error()
So that  nested exceptions are not lost. Also, marshal exceptions, the
ones we have in these places, already have a backtrace, so might as well
use that, instead of creating a new one, loosing unwound frames.

Signed-off-by: Botond Dénes <bdenes@scylladb.com>
Message-Id: <20200507091405.244544-1-bdenes@scylladb.com>
2020-05-07 11:25:25 +02:00
Botond Dénes
e17d8af3c6 compound_compat: composite::iterator cover error-paths with on_internal_error()
But only non-validation error paths. When validating we do expect it to
maybe fail, so we don't want to generate cores for validation.
Validation is in fact a de-serialization pass with some additional
checks. To be able to keep reusing the same code for de-serialization
and validation just with different error handling, introduce a
`strict_mode` flag that can be passed to `composite::iterator`
constructor. When in strict mode (the default) the iterator will convert
any `marshal_exception` thrown during the de-serialization to
`on_internal_error()`.

We don't want anybody to use the iterator in non-strict mode, besides
validation, so the iterator constructors are made private. This is
standard practice for iterators anyway.
2020-04-07 13:18:03 +03:00
Botond Dénes
e0e9b6d9b0 compound_compat: composite_view: add is_valid()
Until now this was open-coded in `sstables::validate_min_max_metadata()`.
We want to cover non-validation compound de-serialization error-paths
with `on_internal_error()` and so we need more control over how
compounds are validated. As a first step we want to centralize
validation in the class itself as in the next patches they will use
private APIs to bypass `on_internal_error()` in the error paths during
validation.
2020-04-07 11:45:45 +03:00
Botond Dénes
4054ba0c45 serialization: accept any CharOutputIterator
Not just bytes::output_iterator. Allow writing into streams other than
just `bytes`. In fact we should be very careful with writing into
`bytes` as they require potentially large contiguous allocations.

The `write()` method is now templatized also on the type of its first
argument, which now accepts any CharOutputIterator. Due to our poor
usage of namespace this now collides with `write` defined inside
`db/commitlog/commitlog.cc`. Luckily, the latter doesn't really have to
be templatized on the data type it reads from, and de-templatizing it
resolves the clash.
2019-12-02 10:10:31 +02:00
Avi Kivity
a71ab365e3 toplevel: convert sprint() to format()
sprint() recently became more strict, throwing on sprint("%s", 5). Replace
with the more modern format().

Mechanically converted with https://github.com/avikivity/unsprint.
2018-11-01 13:16:17 +00:00
Piotr Jastrzebski
e1e23ec555 Pass sstable version to describe_type
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
2018-04-24 11:30:26 +02:00
Avi Kivity
28be4ff5da Revert "Merge "Implement loading sstables in 3.x format" from Piotr"
This reverts commit 513479f624, reversing
changes made to 01c36556bf. It breaks
booting.

Fixes #3376.
2018-04-23 06:47:00 +03:00
Piotr Jastrzebski
26ab3056ae Pass sstable version to describe_type
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
2018-04-22 14:41:11 +02:00
Vladimir Krivopalov
b35c2fe177 Attach backtrace to marshal_exception-s thrown from generic functions.
Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
Message-Id: <06ad18c3563855771dd3ea8d0ec99533642e1919.1511931828.git.vladimir@scylladb.com>
2017-12-05 16:14:55 +01:00
Duarte Nunes
cda3ddd146 compound_compact: Change universal reference to const reference
The universal reference was introduced so we could bind an rvalue to
the argument, but it would have sufficed to make the argument a const
reference. This is also more consistent with the function's other
overload.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20171129132758.19654-1-duarte@scylladb.com>
2017-11-29 14:41:35 +01:00
Duarte Nunes
ec8ce3388e compound_compact: Allow rvalues in size()
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-11-29 14:17:49 +01:00
Duarte Nunes
576ea421dc compound_compat: Don't assume compoundness
This patch changes some factory functions so that they don't assume
the schema is compound.

This enables some code simplification in
sstables::write_column_name().

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2017-11-23 19:14:15 +00:00
Vladimir Krivopalov
61b1988aa1 Use meaningful error messages when throwing a marshal_exception
Fixes #2977

Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
Message-Id: <20171121005108.23074-1-vladimir@scylladb.com>
2017-11-21 16:05:43 +02:00
Paweł Dziepak
4a0385e908 compound_compat: explode: reserve some elements in a vector
When we are exploding a compound key we know already that there is more
than one component, but we have no easy way of determining how many of
them are going to be there. Let's reserve space for a few elements so
that we avoid an excessive number of reallocations in case of
medium-sized keys.
2017-07-26 14:38:27 +01:00
Paweł Dziepak
28c105e4a7 sstables: avoid copying key components 2017-07-26 14:38:27 +01:00
Tomasz Grabiec
6c6be0f7e4 compound_compat: Accept marker value in serialize_value() 2017-03-28 18:10:39 +02:00
Tomasz Grabiec
1331d10811 compound_compat: Add trichotomic comparator 2017-03-28 18:10:39 +02:00
Tomasz Grabiec
2121c2ee8a compound_compat: Make composites printable 2017-03-28 18:10:39 +02:00
Tomasz Grabiec
38e14ab3c8 compound_compat: Introduce composite::serialize_static()
Generelized from static_prefix().
2017-03-28 18:10:39 +02:00
Tomasz Grabiec
bef677b57d compound_compat: Introduce composite_view::last_eoc() 2017-03-28 18:10:39 +02:00
Tomasz Grabiec
18a057aa81 compound_compat: Return composite from serialize_value()
To make the code more type-safe. Also, mark constructor from bytes
explicit.
2017-03-28 18:10:39 +02:00
Duarte Nunes
05c3d4f22b composite: Use operator[] instead of at()
Since we already do bounds checking on is_static(), we can use
bytes_view::operator[] instead of bytes_view::at() to avoid repeating
the bounds checking.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1469616205-4550-3-git-send-email-duarte@scylladb.com>
2016-07-28 11:11:54 +02:00
Duarte Nunes
6c9076fdd7 composite_view: Fix is_static
composite_view's is_static function is wrong because:

1) It doesn't guard against the composite being a compound;
2) Doesn't deal with widening due to integral promotions and
   consequent sign extension.

This patch fixes this by ensuring there's only one correct
implementation of is_static, to avoid code duplication and
enforce test coverage.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1469616205-4550-2-git-send-email-duarte@scylladb.com>
2016-07-28 11:11:38 +02:00
Duarte Nunes
7a81553d17 compound_compat: Only compound values can be static
If a composite is not a compound, then it doesn't carry a length
prefix where static information is encoded. In its absence, a
non-compound composite can never be static.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1469397561-7748-1-git-send-email-duarte@scylladb.com>
2016-07-25 10:05:19 +02:00
Duarte Nunes
4eca7632ec sstables: Replace composite fields with raw bytes
This patch fixes a regression introduced in
f81329be60, which made keys compound by
default when using a particular ctor, in turn leading to mismatches
when comparing the same key built with functions that properly
consider compoundness.

As a temporary fix, the sstable::key and sstable::key_view classes
store raw bytes instead of a composite.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1468339295-3924-1-git-send-email-duarte@scylladb.com>
2016-07-12 18:08:04 +02:00
Duarte Nunes
b179d8d378 compound_compat: Parse legacy compound values
This patch adds support for parsing legacy compound values by
introducing the composite class, a wrapper around a sequence of bytes
serialized in the legacy format for compounds. Compound values can be
sent though the thrift API.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-07-11 16:55:07 +02:00
Pekka Enberg
38a54df863 Fix pre-ScyllaDB copyright statements
People keep tripping over the old copyrights and copy-pasting them to
new files. Search and replace "Cloudius Systems" with "ScyllaDB".

Message-Id: <1460013664-25966-1-git-send-email-penberg@scylladb.com>
2016-04-08 08:12:47 +03:00
Avi Kivity
d5cf0fb2b1 Add license notices 2015-09-20 10:43:39 +03:00
Tomasz Grabiec
d54388e1f6 compound_compat: Remove leftover code
It's a remnant after failed experiment. I forgot to drop it.
2015-04-30 15:40:02 +02:00
Tomasz Grabiec
89dca2eee4 compound: Introduce legacy format adaptors 2015-04-30 11:16:52 +02:00