Commit Graph

36 Commits

Author SHA1 Message Date
Nikos Dragazis
2991b09c8e test/lib: Add option to create uncompressed random schemas
Extend the `random_schema_specification` to support creating both
compressed and uncompressed schemas.

Signed-off-by: Nikos Dragazis <nikolaos.dragazis@scylladb.com>
2024-09-11 13:12:32 +03:00
Avi Kivity
aa1270a00c treewide: change assert() to SCYLLA_ASSERT()
assert() is traditionally disabled in release builds, but not in
scylladb. This hasn't caused problems so far, but the latest abseil
release includes a commit [1] that causes a 1000 insn/op regression when
NDEBUG is not defined.

Clearly, we must move towards a build system where NDEBUG is defined in
release builds. But we can't just define it blindly without vetting
all the assert() calls, as some were written with the expectation that
they are enabled in release mode.

To solve the conundrum, change all assert() calls to a new SCYLLA_ASSERT()
macro in utils/assert.hh. This macro is always defined and is not conditional
on NDEBUG, so we can later (after vetting Seastar) enable NDEBUG in release
mode.

[1] 66ef711d68

Closes scylladb/scylladb#20006
2024-08-05 08:23:35 +03:00
Botond Dénes
0d7335dd27 test/lib/random_schema: remove assert on min number of regular columns
It is legal for a schema to have 0 regular columns, so remove the assert
on the schema specification's regular column count.
2024-06-13 01:32:17 -04:00
Kefu Chai
5ca9a46a91 test/lib: do not include unused headers
these unused includes were identified by clangd. see
https://clangd.llvm.org/guides/include-cleaner#unused-include-warning
for more details on the "Unused include" warning.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#18515
2024-05-05 23:31:48 +03:00
Botond Dénes
d43670046b test/lib: random_schema: disallow boolean_type in keys
They result in poor distribution and poor cardinality, interfering with
tests which want to generate N partitions or rows.

Fixes: #17821

Closes scylladb/scylladb#17856
2024-04-03 09:52:36 +03:00
Botond Dénes
e412673c44 test/lib/random_schema: add uncompatible_timestamp_generator()
Guarantees that produced mutations will not be compactible.
2024-03-12 11:05:18 -04:00
Avi Kivity
51df8b9173 interval: rename nonwrapping_interval to interval
Our interval template started life as `range`, and was supported
wrapping to follow Cassandra's convention of wrapping around the
maximum token.

We later recognized that an interval type should usually be non-wrapping
and split it into wrapping_range and nonwrapping_range, with `range`
aliasing wrapping_range to preserve compatibility.

Even later, we realized the name was already taken by C++ ranges and
so renamed it to `interval`. Given that intervals are usually non-wrapping,
the default `interval` type is non-wrapping.

We can now simplify it further, recognizing that everyone assumes
that an interval is non-wrapping and so doesn't need the
nonwrapping_interval_designation. We just rename nonwrapping_interval
to `interval` and remove the type alias.
2024-02-21 19:43:17 +02:00
Avi Kivity
605bf6e221 range.hh: retire
range.hh was deprecated in bd794629f9 (2020) since its names
conflict with the C++ library concept of an iterator range. The name
::range also mapped to the dangerous wrapping_interval rather than
nonwrapping_interval.

Complete the deprecation by removing range.hh and replacing all the
aliases by the names they point to from the interval library. Note
this now exposes uses of wrapping intervals as they are now explicit.

The unit tests are renamed and range.hh is deleted.

Closes scylladb/scylladb#17428
2024-02-21 00:24:25 +02:00
Pavel Emelyanov
6ead9a5255 test: Don't use require_table_exists() in test/lib/random_schema
This check is pointless. The subsequent call to find_column_family()
would call on_internal_error() in case schema is not found, and since
cql_test_env sets abort-on-internal-error to true, this would fail just
like that

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
2023-08-12 11:46:36 +03:00
Botond Dénes
99c9a71d93 test/lib/random_schema: add create_with_cql()
Allowing the generated schema to be created as a CQL table, so that
queries can be run against it.
2023-03-27 01:44:00 -04:00
Botond Dénes
10a44fee06 test/lib/random_schema: fix udt handling
* generate lowercase names (upper-case seems to cause problems);
* preserve dependency order between UDTs when dumping them from schema;
* use built-in describe() to dump to CQL string;
* drop single arg dump_udts() overlad, which was not recursive, unlike
  the vector variant;
2023-03-27 01:44:00 -04:00
Botond Dénes
b2ddc60c10 test/lib/random_schema: type_generator(): also generate frozen types
For regular and static columns, to introduce some further randomness.
So far frozen types were generated only for primary key members and
embedded types.
2023-03-27 01:44:00 -04:00
Botond Dénes
1cb4b1fc83 test/lib/random_schema: type_generator(): make static column generation conditional
On the schema having clustering columns. Otherwise static column is
illegal.
2023-03-27 01:44:00 -04:00
Botond Dénes
2a7cccd1a8 test/lib/random_schema: type_generator(): don't generate duration_type for keys
And for any embedded type (collection, tuple members, etc.).
Its not allowed as I recently learned it.
2023-03-27 01:44:00 -04:00
Botond Dénes
c9f54e539d test/lib/random_schema: generate_random_mutations(): add overload with seed 2023-03-27 01:44:00 -04:00
Botond Dénes
394909869d test/lib/random_schema: generate_random_mutations(): respect range tombstone count param
Even though there is a parameter determining the number of range
tombstones to be generated, the method disregards it and generates
just 4. Fix that.
2023-03-27 01:43:59 -04:00
Botond Dénes
477b26f7af test/lib/random_schema: generate_random_mutations(): add yields 2023-03-27 01:43:59 -04:00
Botond Dénes
fd8a50035a test/lib/random_schema: generate_random_mutations(): fix indentation 2023-03-27 01:43:59 -04:00
Botond Dénes
71fdec7b42 test/lib/random_schema: generate_random_mutations(): coroutinize method 2023-03-27 01:43:59 -04:00
Kefu Chai
3ae11de204 treewide: do not define/capture unused variables
these warnings are found by Clang-17 after removing
`-Wno-unused-lambda-capture` and '-Wno-unused-variable' from
the list of disabled warnings in `configure.py`.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2023-02-28 21:56:53 +08:00
Avi Kivity
69a385fd9d Introduce schema/ module
Schema related files are moved there. This excludes schema files that
also interact with mutations, because the mutation module depends on
the schema. Those files will have to go into a separate module.

Closes #12858
2023-02-15 11:01:50 +02:00
Avi Kivity
c5e4bf51bd Introduce mutation/ module
Move mutation-related files to a new mutation/ directory. The names
are kept in the global namespace to reduce churn; the names are
unambiguous in any case.

mutation_reader remains in the readers/ module.

mutation_partition_v2.cc was missing from CMakeLists.txt; it's added in this
patch.

This is a step forward towards librarization or modularization of the
source base.

Closes #12788
2023-02-14 11:19:03 +02:00
Botond Dénes
04ca710a95 test/lib/random_schema.hh: value_generator: add min_size_in_bytes
Allow caller to specify the minimum size in bytes of the generated
value. Only really works with string-like types (and collections of
these).
Also fixed max size enforcement for strings: before this patch, the
provided max size was dividied by wide string size, instead of the
char width of the actual string type the value is generated for.
2023-01-30 01:11:31 -05:00
Botond Dénes
98f3d516a2 test/lib/random_schema: add a simpler overload for fixed partition count
Some tests want to generate a fixed amount of random partitions, make
their life easier.
2022-05-05 14:33:37 +03:00
Avi Kivity
fcb8d040e8 treewide: use Software Package Data Exchange (SPDX) license identifiers
Instead of lengthy blurbs, switch to single-line, machine-readable
standardized (https://spdx.dev) license identifiers. The Linux kernel
switched long ago, so there is strong precedent.

Three cases are handled: AGPL-only, Apache-only, and dual licensed.
For the latter case, I chose (AGPL-3.0-or-later and Apache-2.0),
reasoning that our changes are extensive enough to apply our license.

The changes we applied mechanically with a script, except to
licenses/README.md.

Closes #9937
2022-01-18 12:15:18 +01:00
Kamil Braun
9f0e13dd0b test: random_schema: fix clustering column printing in random_schema::cql
Also leave a FIXME to include the key ordering in the string as well.
2021-11-15 17:30:59 +01:00
Avi Kivity
a55b434a2b treewide: extent copyright statements to present day 2021-06-06 19:18:49 +03:00
Avi Kivity
e2e723cc4c build: enable -Wrange-loop-construct warning
This warning triggers when a range for ("for (auto x : range)") causes
non-trivial copies, prompting the developer to replace with a capture
by reference. A few minor violations in the test suite are corrected.

Closes #8699
2021-05-26 10:32:56 +03:00
Pavel Solodovnikov
fff7ef1fc2 treewide: reduce boost headers usage in scylla header files
`dev-headers` target is also ensured to build successfully.

Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
2021-05-20 01:33:18 +03:00
Kamil Braun
b22aa6dbde test: generalize random_subset and move to header 2020-11-30 11:55:44 +01:00
Piotr Jastrzebski
c001374636 codebase wide: replace count with contains
C++20 introduced `contains` member functions for maps and sets for
checking whether an element is present in the collection. Previously
`count` function was often used in various ways.

`contains` does not only express the intend of the code better but also
does it in more unified way.

This commit replaces all the occurences of the `count` with the
`contains`.

Tests: unit(dev)

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Message-Id: <b4ef3b4bc24f49abe04a2aba0ddd946009c9fcb2.1597314640.git.piotr@scylladb.com>
2020-08-15 20:26:02 +03:00
Botond Dénes
ec36c7cb2f test: random_schema: remove redundant gc grace period from tombstone expiry
Compaction automatically adds gc grace period to expiry times already,
no need to add it when creating the tombstones. Remove the redundant
additions form the code. The direct impact is really minor as this is
only used in tests, but it might confuse readers who are looking at how
tombstones are created across the codebase.

Signed-off-by: Botond Dénes <bdenes@scylladb.com>
Message-Id: <20200323120948.92104-1-bdenes@scylladb.com>
2020-03-23 15:12:25 +02:00
Avi Kivity
3c772757c0 treewide: use utils::multiprecision_int for varint implementation
The goal is to forward-declare utils::multiprecision_int, something
beyond my capabilities for boost::multiprecision::cpp_int, to reduce
compile time bloat.

The patch is mostly search-and-replace, with a few casts added to
disambiguate conversions the compiler had trouble with.
2020-03-04 13:28:16 +02:00
Piotr Jastrzebski
a18c791f6f random_schema: stop taking partitioner in constructor
random_schema already has a _schema field which in turn
has a get_partitioner() function. Store partitioner
in random_schema is redundant.

At the moment all uses of random_schema are based on
default partitioner so it is not necessary to set it
explicitly. If in the future we need random_schema to
work with other partitioners we will add the constructor
back and fix the creation of _schema to contain it. It's
not needed now though.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
2020-02-17 10:59:15 +01:00
Rafael Ávila de Espíndola
bd93a0af52 types: Return bytes_opt from data_value::serialize
Since a data_value can contain a null value, returning bytes from
serialize() was losing information as it was mapping null to empty.

This also introduces a serialize_nonnull that still returns bytes, but
results in an internal error if called with a null value.

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2020-01-29 14:04:59 -08:00
Konstantin Osipov
1c8736f998 tests: move all test source files to their new locations
1. Move tests to test (using singular seems to be a convention
   in the rest of the code base)
2. Move boost tests to test/boost, other
   (non-boost) unit tests to test/unit, tests which are
   expected to be run manually to test/manual.

Update configure.py and test.py with new paths to tests.
2019-12-16 17:47:42 +03:00