Commit Graph

35 Commits

Author SHA1 Message Date
Nadav Har'El
9c50d29a00 test/boost: fix flaky test_inject_future_disabled
The test boost/error_injection_test.cc::test_inject_future_disabled
checks what happens when a sleep injection is *disabled*: The test
has a 10-millisecond-sleep injection and measures how much it takes.
The test expects it to take less than 10 milliseconds - in fact it
should take almost zero. But this is not guaranteed - on a slow debug
build and an overcommitted server this do-nothing injection can take
some time, and in one run (#27798) it took 14 milliseconds - and the
test failed.

The solution is easy - make the sleep-that-doesn't-happen much longer -
e.g., 10 whole seconds. Since this sleep still doesn't happen, we
expect the injection to return in less - much less - than 10 seconds.
This 10 seconds is so ridiculously high we don't expect the do-nothing
injection to take 10 seconds, not even a ridiculously busy test machine.

Fixes #27798

Signed-off-by: Nadav Har'El <nyh@scylladb.com>

Closes scylladb/scylladb#27874
2025-12-25 20:46:31 +02:00
Takuya ASADA
03461d6a54 test: compile unit tests into a single executable
To reduce test executable size and speed up compilation time, compile unit
tests into a single executable.

Here is a file size comparison of the unit test executable:

- Before applying the patch
$ du -h --exclude='*.o' --exclude='*.o.d' build/release/test/boost/ build/debug/test/boost/
11G	build/release/test/boost/
29G	build/debug/test/boost/

- After applying the patch
du -h --exclude='*.o' --exclude='*.o.d' build/release/test/boost/ build/debug/test/boost/
5.5G	build/release/test/boost/
19G	build/debug/test/boost/

It reduces executable sizes 5.5GB on release, and 10GB on debug.

Closes #9155

Closes scylladb/scylladb#21443
2024-12-22 19:14:09 +02:00
Avi Kivity
f3eade2f62 treewide: relicense to ScyllaDB-Source-Available-1.0
Drop the AGPL license in favor of a source-available license.
See the blog post [1] for details.

[1] https://www.scylladb.com/2024/12/18/why-were-moving-to-a-source-available-license/
2024-12-18 17:45:13 +02:00
Pavel Emelyanov
440c1e3e3f error_injection: Remove unused inject(sleep, then invoke) overload
The overload was introduced by a8b14b0227 (utils: add timeout error
injection with lambda), but is only used by the test nowadays.

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>

Closes scylladb/scylladb#21377
2024-11-05 09:56:08 +02:00
Kefu Chai
6ead5a4696 treewide: move log.hh into utils/log.hh
the log.hh under the root of the tree was created keep the backward
compatibility when seastar was extracted into a separate library.
so log.hh should belong to `utils` directory, as it is based solely
on seastar, and can be used all subsystems.

in this change, we move log.hh into utils/log.hh to that it is more
modularized. and this also improves the readability, when one see
`#include "utils/log.hh"`, it is obvious that this source file
needs the logging system, instead of its own log facility -- please
note, we do have two other `log.hh` in the tree.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2024-10-22 06:54:46 +03:00
Kefu Chai
40ce52c3cc test: use generic boost_test_print_type()
in this change, we trade the `boost_test_print_type()` overloads
for the generic template of `boost_test_print_type()`, except for
those in the very small tests, which presumably want to keep
themselves relative self-contained.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#18727
2024-05-20 12:56:20 +03:00
Patryk Jędrzejczak
c5c4cc7d00 error_injection: allow injection handlers to not share messages
For a single injection, all created injection handlers share all
received messages. In particular, it means that one received message
unblocks all handlers waiting for the first message. This behavior
is often desired, for example, if multiple fibers execute the
injected code and we want to unblock them all with a single message.
However, there is a problem if we want to block every execution
of the injected code. Apart from the first created handler, all
handlers will be instantly unblocked by messages from the past that
have already unblocked the first handler.

In one of the following commits, we add a test that needs to block
the CDC generation publisher's loop twice. Since it looks like there
are no good workarounds for this arguably general problem, we extend
injections with handlers in a way that solves it. We introduce the
new `share_messages` parameter. Depending on its value, handlers
will share messages or not. The details are described in the new
comments in `error_injection.hh`.

We also add some basic unit tests for the new funcionality.
2024-03-21 14:35:38 +01:00
Pavel Emelyanov
1f44a374b8 error_injection: Overload inject() instead of inject_with_handler()
The inject_with_handler() method accepts a coroutine that can be called
wiht injection_handler. With such function as an argument, there's no
need in distinctive inject_with_handler() name for a method, it can be
overload of all the existing inject()-s

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
2024-03-11 19:30:19 +03:00
Kefu Chai
97587a2ea4 test/boost: do not include unused headers
these unused includes were identified by clangd. see
https://clangd.llvm.org/guides/include-cleaner#unused-include-warning
for more details on the "Unused include" warning.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#17139
2024-02-06 13:22:16 +02:00
Avi Kivity
7cb1c10fed treewide: replace seastar::future::get0() with seastar::future::get()
get0() dates back from the days where Seastar futures carried tuples, and
get0() was a way to get the first (and usually only) element. Now
it's a distraction, and Seastar is likely to deprecate and remove it.

Replace with seastar::future::get(), which does the same thing.
2024-02-02 22:12:57 +08:00
Kefu Chai
1d33a68dd7 tests: utils: error injection: print time duration instead of count
instead of casting / comparing the count of duration unit, let's just
compare the durations, so that boost.test is able to print the duration
in a more informative and user friendly way (line wrapped)

test/boost/error_injection_test.cc(167): fatal error:
    in "test_inject_future_disabled":
      critical check wait_time > sleep_msec has failed [23839ns <= 10ms]

Refs #15902
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2024-01-25 19:10:24 +08:00
Mikołaj Grzebieluch
ef712e5d21 tests: utils: error injection: add test for injection's parameters 2023-07-13 10:10:52 +02:00
Mikołaj Grzebieluch
7e5c42af0a tests: utils: error injections: add test for inject_with_handler
Add a test checking the correctness of the `inject_with_handler` method
in presence of concurrency.
2023-07-06 12:34:53 +02:00
Kefu Chai
3ae11de204 treewide: do not define/capture unused variables
these warnings are found by Clang-17 after removing
`-Wno-unused-lambda-capture` and '-Wno-unused-variable' from
the list of disabled warnings in `configure.py`.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2023-02-28 21:56:53 +08:00
Raphael S. Carvalho
3c5afb2d5c test: Enable Scylla test command line options for boost tests
We have enabled the command line options without changing a
single line of code, we only had to replace old include
with scylla_test_case.hh.

Next step is to add x-log-compaction-groups options, which will
determine the number of compaction groups to be used by all
instantiations of replica::table.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2023-02-01 20:14:51 -03:00
Benny Halevy
59acc58920 test: error_injection: test_inject_noop: do no rely on wall clock timing
This unit test may fail in debug mode
since .then may yield, even if inject returns
a ready future, so the wall clock timing has no basis.

As seen in https://jenkins.scylladb.com/job/releng/job/Scylla-CI/932/testReport/junit/boost.error_injection_test.debug/test_boost_error_injection_test/test_inject_noop/
```
[Exception] - critical check wait_time.count() < sleep_msec.count() has failed [47 >= 10]
 == [File] - test/boost/error_injection_test.cc
 == [Line] -45
 ```

Instead, just verify that inject returns
a successful, ready future, when using
a non-existing errr injection name.

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>

Closes #10842
2022-06-21 11:58:13 +03:00
Nadav Har'El
1ce73c2ab3 Merge 'utils::is_timeout_exception: Ensure we handle nested exception types' from Calle Wilund
Fixes #9922

storage proxy uses is_timeout_exception to traverse different code paths.
a6202ae079 broke this (because bit rot and
intermixing), by wrapping exception for information purposes.

This adds check of nested types in exception handling, as well as a test
for the routine itself.

Closes #9932

* github.com:scylladb/scylla:
  database/storage_proxy: Use "is_timeout_exception" instead of catch match
  utils::is_timeout_exception: Ensure we handle nested exception types
2022-01-18 23:49:41 +02:00
Avi Kivity
fcb8d040e8 treewide: use Software Package Data Exchange (SPDX) license identifiers
Instead of lengthy blurbs, switch to single-line, machine-readable
standardized (https://spdx.dev) license identifiers. The Linux kernel
switched long ago, so there is strong precedent.

Three cases are handled: AGPL-only, Apache-only, and dual licensed.
For the latter case, I chose (AGPL-3.0-or-later and Apache-2.0),
reasoning that our changes are extensive enough to apply our license.

The changes we applied mechanically with a script, except to
licenses/README.md.

Closes #9937
2022-01-18 12:15:18 +01:00
Calle Wilund
97bb1be6f7 utils::is_timeout_exception: Ensure we handle nested exception types
Fixes #9922

storage proxy uses is_timeout_exception to traverse different code paths.
a6202ae079 broke this (because bit rot and
intermixing), by wrapping exception for information purposes.

This adds check of nested types in exception handling, as well as a test
for the routine itself.
2022-01-17 08:43:41 +00:00
Avi Kivity
6221b90b89 secondary_index_manager: stop including expression.hh
Use a forward declaration of cql3::expr::oper_t to reduce the
number of translation units depending on expression.hh.

Before:

    $ find build/dev -name '*.d' | xargs cat | grep -c expression.hh
    272

After:

    $ find build/dev -name '*.d' | xargs cat | grep -c expression.hh
    154

Some translation units adjust their includes to restore access
to required headers.

Closes #9229
2021-08-22 21:21:46 +03:00
Avi Kivity
a55b434a2b treewide: extent copyright statements to present day 2021-06-06 19:18:49 +03:00
Alejo Sanchez
a8b14b0227 utils: add timeout error injection with lambda
Even though calling then() on a ready future does not allocate a
continuation, calling then on the result of it will allocate.

This error injection only adds a continuation in the dependency
chain if error injections are enabled at compile timeand this particular
error injection is enabled.

Signed-off-by: Alejo Sanchez <alejo.sanchez@scylladb.com>
2020-06-03 14:44:00 +02:00
Alejo Sanchez
0321172677 utils: error injection add enter() for control flow
For control flow (i.e. return) and simplicity add enter() method.

For disabled injections, this method is const returning false,
therefore it has no overhead.

Add boost test.

Signed-off-by: Alejo Sanchez <alejo.sanchez@scylladb.com>
2020-06-03 14:42:48 +02:00
Alejo Sanchez
bb08b5ad5a utils: error injections provide error exceptions
Provide non-timeout error exception
to facilitate control flow in injected errors.

Signed-off-by: Alejo Sanchez <alejo.sanchez@scylladb.com>
2020-05-28 11:13:55 +02:00
Pavel Solodovnikov
014883d560 failure_injector: implement CQL API for failure injector class
The following UDFs are defined to control failure injector API usage:
 * enable_injection(name, args)
 * disable_injection(name)

All arguments have string type.

As currently function(terminal) is not supported by the parser,
the arguments must come from selected rows.

Added boost test for CQL API.

Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
Signed-off-by: Alejo Sanchez <alejo.sanchez@scylladb.com>
2020-05-28 11:13:55 +02:00
Alejo Sanchez
2c7e01a3b6 lwt: fix disabled error injection templates
Fix disabled injection templates to match enabled ones.
Fix corresponding test to not be a continuation.

Signed-off-by: Alejo Sanchez <alejo.sanchez@scylladb.com>
2020-05-28 11:13:55 +02:00
Rafael Ávila de Espíndola
543a9ebd9b tests: Wait for a few futures
GCC 10 now warns on these. This fixes the dev build with gcc 10.

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
Message-Id: <20200424161006.17857-1-espindola@scylladb.com>
2020-04-26 15:20:40 +03:00
Alejo Sanchez
bd849764e0 utils: error injection sleep add support for manual_clock
Requested by @tgrabiec in previous patch (already merged).

Adds support for sleep using manual clock.
Add test.

NOTE: Removes system_clock support (and test) as sleep is not explicitly
      instantiated in seastar/src/core/reactor.cc

Branch URL: https://github.com/alecco/scylla/tree/error_injection_5_manual_clock

Tests: unit ({dev})

Signed-off-by: Alejo Sanchez <alejo.sanchez@scylladb.com>
Message-Id: <20200417081518.868900-1-alejo.sanchez@scylladb.com>
2020-04-17 11:45:05 +02:00
Alejo Sanchez
3a4dd0a856 utils: error injection inject() returning a future
Make inject() return a future.

Suggested by Gleb.
Botond helped on dealing with complex function/lambda overload.

Refs #3295 (closed)

Tests: unit ({dev})

Signed-off-by: Alejo Sanchez <alejo.sanchez@scylladb.com>
Message-Id: <20200331143839.1781424-7-alejo.sanchez@scylladb.com>
2020-04-01 16:22:52 +02:00
Alejo Sanchez
8bae38cef9 utils: error injection support multiple clocks
Use template to support multiple clock classes for time point
for deadline injection.

Refs: #3295   (closed)

Signed-off-by: Alejo Sanchez <alejo.sanchez@scylladb.com>
Message-Id: <20200331143839.1781424-6-alejo.sanchez@scylladb.com>
2020-04-01 16:22:45 +02:00
Alejo Sanchez
71f2f423bc utils: error injection reorder args for exceptions
Move exception factory to end of argument list.

Refs: #3295   (closed)

Signed-off-by: Alejo Sanchez <alejo.sanchez@scylladb.com>
Message-Id: <20200331143839.1781424-5-alejo.sanchez@scylladb.com>
2020-04-01 16:22:38 +02:00
Alejo Sanchez
fd1eb6a466 utils: error injection simplify API
Split error injection C++ API to have

1. sleep duration
2. sleep to deadline (timeout)

TODO: support multiple types of clocks

Refs: #3295   (closed)

Signed-off-by: Alejo Sanchez <alejo.sanchez@scylladb.com>
Message-Id: <20200331143839.1781424-4-alejo.sanchez@scylladb.com>
2020-04-01 16:22:30 +02:00
Alejo Sanchez
4a3b98facc utils: error injection fix deadline test timeout
Rafael reported test_inject_future_sleep_timeout_short failed
sometimes as limit is too close. Bump limit.

Refs #3295 (closed)

Repro:
./test.py --mode=dev -v boost/error_injection_test --repeat 300

Tests: unit ({dev})

Signed-off-by: Alejo Sanchez <alejo.sanchez@scylladb.com>
Message-Id: <20200328204454.1326514-3-alejo.sanchez@scylladb.com>
2020-03-31 11:58:38 +02:00
Alejo Sanchez
febcced4f1 utils: error injection with timeout/deadline
Most of Scylla code runs with a user-supplied query timeout, expressed as
absolute clock (deadline). When injecting test sleeps into such code, we most
often want to not sleep beyond the user supplied deadline. Extend error
injection API to optionally accept a deadline, and, if it is provided,
sleep no more than up to the deadline. If current time is beyond deadline,
sleep injection is skipped altogether.

Signed-off-by: Alejo Sanchez <alejo.sanchez@scylladb.com>
Message-Id: <20200326091600.1037717-2-alejo.sanchez@scylladb.com>
2020-03-26 12:41:10 +01:00
Pavel Solodovnikov
057adc8b4d utils: add error injection
Error injection class is implemented in order to allow injecting
various errors (exceptions, stalls, etc.) in code for testing
purposes.

Error injection is enabled via compile flag
 SCYLLA_ENABLE_ERROR_INJECTION

TODO: manage shard instances

Enable error injection in debug/dev/sanitize modes.

Unit tests for error injection class.

Closes #3295

Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
Signed-off-by: Alejo Sanchez <alejo.sanchez@scylladb.com>
2020-03-20 19:37:48 +01:00