There is no need to call c_str() on the name of the directory entry.
In fact, the used overload std::stoi() takes an std::string as its
argument. Providing seastar::sstring instead of const char* is more
efficient because we can allocate just the right amount of memory
and std::memcpy it, i.e. call std::string(const char*, std::size_t).
Using the overload std::string(const char*) would need to first
traverse the string to find the null byte.
This is a small change, all the more because paths don't tend to
be long, but it's some gain nonetheless.
The commit also inserts a few empty lines to make the code less
compact and improve readability as a result.
An anonymous namespace is a safer mechanism than the static
keyword. When adding a new piece of code, it's easy to
forget about adding the static. In that case, that code
might undergo external linkage. However, when code is put
in an anonymous namespace (when it should not), the linker
will immediately detect it (in most cases), and
the programmer will be able to spot and fix their mistake
right away.
these options for disabling warnings are not necessary anymore, for
one of the following reasons:
* the code which caused the warning were either fixed or removed
* the toolchain were updated, so the false alarms do not exist
with the latest frozen toolchain.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#15450
do not use muti-line comment. this silences the warning from GCC:
```
In file included from ./cql3/prepare_context.hh:19,
from ./cql3/statements/raw/parsed_statement.hh:14,
from build/debug/gen/cql3/CqlParser.hpp:62,
from build/debug/gen/cql3/CqlParser.cpp:44:
./cql3/expr/expression.hh:490:1: error: multi-line comment [-Werror=comment]
490 | /// Custom formatter for an expression. Supports multiple modes:\
| ^
```
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#15471
this target mirrors the target named `{mode}e-test` in the
`build.ninja` build script created by `configure.py`.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#15448
because we should have a frozeon toolchain built with fedora38, and f38
provides cmake v3.27.4, we can assume the availability of cmake v3.27.4
when building scylla with the toolchain.
in this change, the minimum required CMake version is changed to
3.27.
this also allows us to simplify the implementation of
`add_whole_archive()`, and remove the buggy branch for supporting
CMake < 3.24, as we should have used `${name}` in place of `auth` there.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#15446
This PR modularizes `manager.{hh, cc}` by dividing the files into separate smaller units. The changes improve overall readability of code and help reason about it. Each file has a specific purpose now.
This is the first step in refactoring the Hinted Handoff module.
Refs scylladb/scylla#15358Closesscylladb/scylladb#15378
* github.com:scylladb/scylladb:
db/hints: Remove unused aliases from manager.hh
db/hints: Rename end_point_hints_manager
db/hints: Rename sender to hint_sender
db/hints: Move the rebalancing logic to hint_storage
db/hints: Move the implementation of sender
db/hints: Move the declaration of sender to hint_sender.hh
db/hints: Move sender::replay_allowed() to the source file
db/hints: Put end_point_hints_manager in internal namespace
db/hints: Move the implementation of end_point_hints_manager
db/hints: Move the declaration of end_point_hints_manager
db/hints: Move definitions of functions using shard hint manager
db/hints: Introduce hint_storage.hh
db/hints: Extract the logger from manager.cc
db/hints: Extract common types from manager.hh
Currently, when creating the table, permissions may be mistakenly
granted to the user even if the table is already existing. This
can happen in two cases:
The query has a IF NOT EXISTS clause - as a result no exception
is thrown after encountering the existing table, and the permission
granting is not prevented.
The query is handled by a non-zero shard - as a result we accept
the query with a bounce_to_shard result_message, again without
preventing the granting of permissions.
These two cases are now avoided by checking the result_message
generated when handling the query - now we only grant permissions
when the query resulted in a schema_change message.
Additionally, a test is added that reproduces both of the mentioned
cases.
CVE-2023-33972
Fixes#15467.
* 'no-grant-on-no-create' of github.com:scylladb/scylladb-ghsa-ww5v-p45p-3vhq:
auth: do not grant permissions to creator without actually creating
transport: add is_schema_change() method to result_message
this target mirrors the "dist-server-debuginfo-{mode}" target in the `build.ninja` created by `configure.py`.
Closesscylladb/scylladb#15441
* github.com:scylladb/scylladb:
build: cmake: add "dist-server-debuginfo" target
build: cmake: remove debian dep from relocatable pkg
Populating of non-system keyspaces is now done by listing datadirs and assuming that each subdir found is a keyspace. For S3-backed keyspaces this is also true, but it's a bug (#13020). The loop needs to walk the list of known keyspaces instead, and try to find the keyspace storage later, based on the storage option.
Closesscylladb/scylladb#15436
* github.com:scylladb/scylladb:
distributed_loader: Indentation fix after previous patch
distributed_loader: Generalize datadir parallelizm loop
distributed_loader: Provide keyspace ref to populate_keyspace
distributed_loader: Walk list of keyspaces instead of directories
always initialize returned values. the branches which
return these unitiailized returned values handles the
unmatched cases, so this change should not have any
impact on the behavior.
ANTLR3's C++ code generator does not assign any value
to the value, if it runs into failure or encounter
exceptions. for instance, following rule assigns the
value of `isStatic` to `isStaticColumn` only if
nothing goes wrong.
```
cfisStatic returns [bool isStaticColumn]
@init{
bool isStatic = false;
}
: (K_STATIC { isStatic=true; })?
{
$isStaticColumn = isStatic;
}
;
```
as shown in the generated C++ code:
```c++
switch (alt118)
{
case 1:
// build/debug/gen/cql3/Cql.g:989:8: K_STATIC
{
this->matchToken(K_STATIC, &FOLLOW_K_STATIC_in_cfisStatic5870);
if (this->hasException())
{
goto rulecfisStaticEx;
}
if (this->hasFailed())
{
return isStaticColumn;
}
if ( this->get_backtracking()==0 )
{
isStaticColumn=isStatic;
}
}
break;
}
```
when `this->hasException()` or `this->hasFailed()`,
`isStaticColumn` is returned right away without being
initialized, because we don't assign any initial value
to it, neither do we customize the exception handling
for this rule.
and, the parser bails out when its smells something bad
after it tries to match the specified rule. also, the
parser is a stateful tokenizer, its failure state is
carried by the parser itself. also, the matchToken()
*could* fail when trying to find the matched token,
this is the runtime behavior of parser, that's why the
compiler cannot be certain that the error path won't
be taken.
anyway, let's always initialize the values without
default constructor. the return values whose type
is of scoped enum are initialized with zero
initialization, because their types don't provide an
"invalid" value.
this change should silence warnings like:
```
clang++ -MD -MT build/debug/gen/cql3/CqlParser.o -MF build/debug/gen/cql3/CqlParser.o.d -I/home/kefu/dev/scylladb/seastar/include -I/home/kefu/dev/scylladb/build/debug/seastar/gen/include -U_FORTIFY_SOURCE -DSEASTAR_SSTRING -Werror=unused-result -fstack-clash-protection -fsanitize=address -fsanitize=undefined -fno-sanitize=vptr -DSEASTAR_API_LEVEL=7 -DSEASTAR_BUILD_SHARED_LIBS -DSEASTAR_SCHEDULING_GROUPS_COUNT=16 -DSEASTAR_DEBUG -DSEASTAR_DEFAULT_ALLOCATOR -DSEASTAR_SHUFFLE_TASK_QUEUE -DSEASTAR_DEBUG_SHARED_PTR -DSEASTAR_LOGGER_TYPE_STDOUT -DSEASTAR_TYPE_ERASE_MORE -DBOOST_NO_CXX98_FUNCTION_BASE -DFMT_SHARED -I/usr/include/p11-kit-1 -ffile-prefix-map=/home/kefu/dev/scylladb=. -march=westmere -DDEBUG -DSANITIZE -DDEBUG_LSA_SANITIZER -DSCYLLA_ENABLE_ERROR_INJECTION -Og -DSCYLLA_BUILD_MODE=debug -g -gz -iquote. -iquote build/debug/gen --std=gnu++20 -ffile-prefix-map=/home/kefu/dev/scylladb=. -march=westmere -DBOOST_TEST_DYN_LINK -DNOMINMAX -DNOMINMAX -fvisibility=hidden -Wall -Werror -Wextra -Wno-deprecated-copy -Wno-mismatched-tags -Wno-missing-field-initializers -Wno-c++11-narrowing -Wno-ignored-qualifiers -Wno-overloaded-virtual -Wno-unsupported-friend -Wno-unused-parameter -Wno-implicit-int-float-conversion -Wno-error=deprecated-declarations -DXXH_PRIVATE_API -DSEASTAR_TESTING_MAIN -DFMT_DEPRECATED_OSTREAM -Wno-parentheses-equality -O1 -fno-sanitize-address-use-after-scope -c -o build/debug/gen/cql3/CqlParser.o build/debug/gen/cql3/CqlParser.cpp
build/debug/gen/cql3/CqlParser.cpp:26645:28: error: variable 'perm' is uninitialized when used here [-Werror,-Wuninitialized]
return perm;
^~~~
build/debug/gen/cql3/CqlParser.cpp:26616:5: note: variable 'perm' is declared here
auth::permission perm;
^
build/debug/gen/cql3/CqlParser.cpp:52577:28: error: variable 'op' is uninitialized when used here [-Werror,-Wuninitialized]
return op;
^~
build/debug/gen/cql3/CqlParser.cpp:52518:5: note: variable 'op' is declared here
oper_t op;
^
```
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#15451
instead of checking the availability of a required program, let's
use the `REQUIRED` argument introduced by CMake 3.18, simpler this
way.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#15447
Handle abort_requested_exception exactly like
sleep_aborted, as an expected error when startup
is aborted mid-way.
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
Closesscylladb/scylladb#15443
Some time ago populating of tables from sstables was reworked to use sstable states instead of full paths (#12707). Since then few places in the populator was left that still operate on the state-based subdirectory name. This PR collects most of those dangling ends
refs: #13020Closesscylladb/scylladb#15421
* github.com:scylladb/scylladb:
distributed_loader: Print sstable state explicitly
distributed_loader: Move check for the missing dir upper
distributed_loader: Use state as _sstable_directories key
when compiling the tests with -Wsign-compare, the compiler complains like:
```
/home/kefu/.local/bin/clang++ -DBOOST_ALL_DYN_LINK -DBOOST_NO_CXX98_FUNCTION_BASE -DDEBUG -DDEBUG_LSA_SANITIZER -DFMT_DEPRECATED_OSTREAM -DFMT_SHARED -DSANITIZE -DSCYLLA_BUILD_MODE=debug -DSCYLLA_ENABLE_ERROR_INJECTION -DSEASTAR_API_LEVEL=7 -DSEASTAR_BROKEN_SOURCE_LOCATION -DSEASTAR_DEBUG -DSEASTAR_DEBUG_SHARED_PTR -DSEASTAR_DEFAULT_ALLOCATOR -DSEASTAR_LOGGER_TYPE_STDOUT -DSEASTAR_SCHEDULING_GROUPS_COUNT=16 -DSEASTAR_SHUFFLE_TASK_QUEUE -DSEASTAR_TESTING_MAIN -DSEASTAR_TYPE_ERASE_MORE -DXXH_PRIVATE_API -I/home/kefu/dev/scylladb -I/home/kefu/dev/scylladb/build/cmake/gen -I/home/kefu/dev/scylladb/seastar/include -I/home/kefu/dev/scylladb/build/cmake/seastar/gen/include -isystem /home/kefu/dev/scylladb/build/cmake/rust -Wall -Werror -Wextra -Wno-error=deprecated-declarations -Wimplicit-fallthrough -Wno-c++11-narrowing -Wno-mismatched-tags -Wno-overloaded-virtual -Wno-unsupported-friend -Wno-unused-parameter -Wno-missing-field-initializers -Wno-deprecated-copy -Wno-ignored-qualifiers -march=westmere -Og -g -gz -std=gnu++20 -fvisibility=hidden -U_FORTIFY_SOURCE -DSEASTAR_SSTRING -Wno-error=unused-result "-Wno-error=#warnings" -fstack-clash-protection -fsanitize=address -fsanitize=undefined -fno-sanitize=vptr -MD -MT test/boost/CMakeFiles/tablets_test.dir/tablets_test.cc.o -MF test/boost/CMakeFiles/tablets_test.dir/tablets_test.cc.o.d -o test/boost/CMakeFiles/tablets_test.dir/tablets_test.cc.o -c /home/kefu/dev/scylladb/test/boost/tablets_test.cc
/home/kefu/dev/scylladb/test/boost/tablets_test.cc:1335:53: error: comparison of integers of different signs: 'int' and 'size_t' (aka 'unsigned long') [-Werror,-Wsign-compare]
for (int log2_tablets = 0; log2_tablets < tablet_count_bits; ++log2_tablets) {
~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~
```
in this case, it should be safe to use an signed int as the loop
variable to be compared with `tablet_count_bits`, but let's just
appease the compiler so we can enable the warning option project-wide
to prevent any potential issues caused by signed-unsigned comparision.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#15449
We add garbage collection for the `CDC_GENERATIONS_V3` table to prevent
it from endlessly growing. This mechanism is especially needed because
we send the entire contents of `CDC_GENERATIONS_V3` as a part of the
group 0 snapshot.
The solution is to keep a clean-up candidate, which is one of the
already published CDC generations. The CDC generation publisher
introduced in #15281 continually uses this candidate to remove all
generations with timestamps not exceeding the candidate's and sets a new
candidate when needed.
We also add `test_cdc_generation_clearing.py` that verifies this new
mechanism.
Fixes#15323Closesscylladb/scylladb#15413
* github.com:scylladb/scylladb:
test: add test_cdc_generation_clearing
raft topology: remove obsolete CDC generations
raft topology: set CDC generation clean-up candidate
topology_coordinator: refactor publish_oldest_cdc_generation
system_keyspace: introduce decode_cdc_generation_id
system_keyspace: add cleanup_candidate to CDC_GENERATIONS_V3
in other words, do not create bpo::value unless transfer it to an
option_description.
`boost::program_options::value()` create a new typed_value<T> object,
without holding it with a shared_ptr. boost::program_options expects
developer to construct a `bpo::option_description` right away from it.
and `boost::program_options::option_description` takes the ownership
of the `type_value<T>*` raw pointer, and manages its life cycle with
a shared_ptr. but before passing it to a `bpo::option_description`,
the pointer created by `boost::program_options::value()` is a still
a raw pointer.
before this change, we initialize `operations_with_func` as global
variables using `boost::program_options::value()`. but unfortunately,
we don't always initialize a `bpo::option_description` from it --
we only do this on demand when the corresponding subcommand is
called.
so, if the corresponding subcommand is not called, the created
`typed_value<T>` objects are leaked. hence LeakSanitizer warns us.
after this change, we create the option map as a static
local variable in a function so it is created on demand as well.
as an alternative, we could initialize the options map as local
variable where it used. but to be more consistent with how
`global_option` is specified. and to colocate them in a single
place, let's keep the existing code layout.
this change is quite similar to 374bed8c3d
Fixes https://github.com/scylladb/scylladb/issues/15429Closesscylladb/scylladb#15430
* github.com:scylladb/scylladb:
tools/scylla-nodetools: reindent
tools/scylla-nodetools: do not create unowned bpo::value
column_stats::update_local_deletion_time() is not used anywhere,
what is being used is
`column_stats::update_local_deletion_time_and_tombstone_histogram(time_point)`.
while `update_local_deletion_time_and_tombstone_histogram(int32_t)`
is only used internally by a single caller.
neither is `column_stats::update(const deletion_time&)` used.
so let's drop them. and merge
`update_local_deletion_time_and_tombstone_histogram(int32_t)`
into its caller.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#15189
in this series, we do not assume the existence of "build" build directory. and prefer using the version files located under the directory specified with the `--build-dir` option.
Refs #15241Closesscylladb/scylladb#15402
* github.com:scylladb/scylladb:
create-relocatable-package.py: prefer $build_dir/SCYLLA-RELEASE-FILE
create-relocatable-package.py: create SCYLLA-RELOCATABLE-FILE with tempfile
in other words, do not create bpo::value unless transfer it to an
option_description.
`boost::program_options::value()` create a new typed_value<T> object,
without holding it with a shared_ptr. boost::program_options expects
developer to construct a `bpo::option_description` right away from it.
and `boost::program_options::option_description` takes the ownership
of the `type_value<T>*` raw pointer, and manages its life cycle with
a shared_ptr. but before passing it to a `bpo::option_description`,
the pointer created by `boost::program_options::value()` is a still
a raw pointer.
before this change, we initialize `operations_with_func` as global
variables using `boost::program_options::value()`. but unfortunately,
we don't always initialize a `bpo::option_description` from it --
we only do this on demand when the corresponding subcommand is
called.
so, if the corresponding subcommand is not called, the created
`typed_value<T>` objects are leaked. hence LeakSanitizer warns us.
after this change, we create the option map as a static
local variable in a function so it is created on demand as well.
as an alternative, we could initialize the options map as local
variable where it used. but to be more consistent with how
`global_option` is specified. and to colocate them in a single
place, let's keep the existing code layout.
this change is quite similar to 374bed8c3dFixes#15429
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
```console
$ journalctl --user start scylla-server -xe
Failed to add match 'start': Invalid argument
```
`journalctl` expects a match filter as its positional arguments.
but apparently, start is not a filter. we could use `--unit`
to specify a unit though, like:
```console
$ journalctl --user --unit scylla-server.service -xe
```
but it would flood the stdout with the logging messages printed
by scylla. this is not what a typical user expects. probably a better
use experience can be achieved using
```console
$ systemctl --user status scylla-server
```
which also print the current status reported by the service, and
the command line arguments. they would be more informative in typical
use cases.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#15390
The docker/podman tooling is destructive: it will happily
overwrite images locally and on the server. If a maintainer
forgets to update tools/toolchain/image, this can result
in losing an older toolchain container image.
To prevent that, check that the image name is new.
Closesscylladb/scylladb#15397
update commented out experimental_features to reflect the latest
experimental features:
- in 4f23eec4, "raft" was renamed to "consistent-topology-changes".
- in 2dedb5ea, "alternator-ttl" was moved out of experimental features.
- in 5b1421cc, "broadcast-tables" was added to experimental features.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#15407
this target mirrors the "dist-server-debuginfo-{mode}" target in
the `build.ninja` created by `configure.py`.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
`create-relocatable-package.py` does not use or include
`${CMAKE_CURRENT_BINARY_DIR}/debian`. so there is no
need to include this directory as a dependency.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
pull_gitgub_pr.sh adds a "Closes #xyz" tag so github can close
the pull request after next promotion. Convert it to an absolute
refefence (scylladb/scylladb#xyz) so the commit can be cherry-picked
into another repository without the reference dangling.
Closes#15424
this change change `const` to `constexpr`. because the string literal
defined here is not only immutable, but also initialized at
compile-time, and can be used by constexpr expressions and functions.
this change is introduced to reduce the size of the change when moving
to compile-time format string in future. so far, seastar::format() does
not use the compile-time format string, but we have patches pending on
review implementing this. and the author of this change has local
branches implementing the changes on scylla side to support compile-time
format string, which practically replaces most of the `format()` calls
with `seastar::format()`.
without this change, if we use compile-time format check, compiler fails
like:
```
/home/kefu/dev/scylladb/tools/scylla-nodetool.cc:276:44: error: call to consteval function 'fmt::basic_format_string<char, const char *const &, seastar::basic_sstring<char, unsigned int, 15>>::basic_format_string<const char *, 0>' is not a constant expression
.description = seastar::format(description_template, app_name, boost::algorithm::join(operations | boost::adaptors::transformed([] (const auto& op) {
^
/usr/include/fmt/core.h:3148:67: note: read of non-constexpr variable 'description_template' is not allowed in a constant expression
FMT_CONSTEVAL FMT_INLINE basic_format_string(const S& s) : str_(s) {
^
/home/kefu/dev/scylladb/tools/scylla-nodetool.cc:276:44: note: in call to 'basic_format_string(description_template)'
.description = seastar::format(description_template, app_name, boost::algorithm::join(operations | boost::adaptors::transformed([] (const auto& op) {
^
/home/kefu/dev/scylladb/tools/scylla-nodetool.cc:258:16: note: declared here
const auto description_template =
^
```
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closes#15432
change the loop variable to `int` to silence warning like
```
/home/kefu/.local/bin/clang++ -DBOOST_NO_CXX98_FUNCTION_BASE -DDEBUG -DDEBUG_LSA_SANITIZER -DFMT_DEPRECATED_OSTREAM -DFMT_SHARED -DSANITIZE -DSCYLLA_BUILD_MODE=debug -DSCYLLA_ENABLE_ERROR_INJECTION -DSEASTAR_API_LEVEL=7 -DSEASTAR_BROKEN_SOURCE_LOCATION -DSEASTAR_DEBUG -DSEASTAR_DEBUG_SHARED_PTR -DSEASTAR_DEFAULT_ALLOCATOR -DSEASTAR_LOGGER_TYPE_STDOUT -DSEASTAR_SCHEDULING_GROUPS_COUNT=16 -DSEASTAR_SHUFFLE_TASK_QUEUE -DSEASTAR_TYPE_ERASE_MORE -DXXH_PRIVATE_API -I/home/kefu/dev/scylladb -I/home/kefu/dev/scylladb/seastar/include -I/home/kefu/dev/scylladb/build/cmake/seastar/gen/include -I/home/kefu/dev/scylladb/build/cmake/gen -Wall -Werror -Wextra -Wno-error=deprecated-declarations -Wimplicit-fallthrough -Wno-c++11-narrowing -Wno-mismatched-tags -Wno-overloaded-virtual -Wno-unsupported-friend -Wno-unused-parameter -Wno-missing-field-initializers -Wno-deprecated-copy -Wno-ignored-qualifiers -march=westmere -Og -g -gz -std=gnu++20 -fvisibility=hidden -U_FORTIFY_SOURCE -DSEASTAR_SSTRING -Wno-error=unused-result "-Wno-error=#warnings" -fstack-clash-protection -fsanitize=address -fsanitize=undefined -fno-sanitize=vptr -MD -MT tools/CMakeFiles/tools.dir/scylla-nodetool.cc.o -MF tools/CMakeFiles/tools.dir/scylla-nodetool.cc.o.d -o tools/CMakeFiles/tools.dir/scylla-nodetool.cc.o -c /home/kefu/dev/scylladb/tools/scylla-nodetool.cc
/home/kefu/dev/scylladb/tools/scylla-nodetool.cc:215:28: error: comparison of integers of different signs: 'unsigned int' and 'int' [-Werror,-Wsign-compare]
for (unsigned i = 0; i < argc; ++i) {
~ ^ ~~~~
```
`i` is used as the index in a plain C-style array, it's perfectly fine
to use a signed integer as index in this case. as per C++ standard,
> The expression E1[E2] is identical (by definition) to *((E1)+(E2))
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closes#15431
Population of keyspaces happens first fo system keyspaces, then for
non-system ones. Both methods iterate over config datadirs to populate
from all configured directories. This patch generalizes this loop into
the populate_keyspace() method.
(indentation is deliberately left broken)
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
The method in question tries to find keyspace reference on the database
by the given keyspace name. However, one of the callers aready has the
keyspace reference at hands and can just pass it. The other calls can
find the keyspace on its own.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
When populating non-system keyspaces the dist. loader lists the
directories with keyspaces in datadirs, then tries to call
populate_keyspace() with the found name. If the keyspace in question is
not found on the database, a warning is printed and population
continues.
S3-backed keyspaces are nowadays populated with this process just
because there's a bug #13020 -- even such keyspaces still create empty
directories in datadirs. When the bug gets fixed, population would omit
such keyspaces. This patch prepares this by making population walk the
known keyspaces from the database. BTW, population of system keyspaces
already works by iterating over the list of known keyspaces, not the
datadir subdirectories.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
in hope to lower the bar to testing object store.
* add language specifier for better readability of the document.
to highlight the config with YAML syntax
* add more specific comment on the AWS related settings
* explain that endpoint should match in the CREATE KEYSPACE
statement and the one defined by the YAML configuration.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closes#15433
Move node_ops related classes to node_ops/ so that they
are consistently grouped and could be access from
many modules.
Closes#15351
* github.com:scylladb/scylladb:
node_ops: extract classes related to node operations
node_ops: repair: move node_ops_id to node_ops directory
This commit adds the information that ScyllaDB Enterprise
supports FIPS-compliant systems in versions
2023.1.1 and later.
The information is excluded from OSS docs with
the "only" directive, because the support was not
added in OSS.
This commit must be backported to branch-5.2 so that
it appears on version 2023.1 in the Enterprise docs.
Closes#15415
We make the CDC generation publisher continually remove the
obsolete CDC generation data to prevent CDC_GENERATIONS_V3 from
endlessly growing. To achieve this, we use the clean-up candidate.
If it exists and can be safely removed, we remove it together with
all older CDC generations. We also mark the lack of a new
candidate. The next published CDC generation will become one.
Note this solution does not have any guarantee about "when"
it removes obsolete generations. Formally, it guarantees that
if there is a candidate that can be removed and the CDC generation
publisher attempts to remove it, all generations up to the
candidate are removed. In practice, when a new generation appears,
the publisher makes a new candidate or tries to remove an old
candidate, so obsolete generations can stay for a long time only
if no generation appears for a long time. But it is fine because
we only want to prevent CDC_GENERATIONS_V3 from growing too much.
Moreover, providing any guarantees would require a new wake-up
mechanism for the publisher, which would be hard to implement.
We want to use the clean-up candidates to remove the obsolete CDC
generation data, but first, we need to set suitable generations as
a candidate when there is no candidate. Since CDC generations must
be published before we remove them, a generation that is being
published is a good candidate.
In the following commits, we add a new task for the CDC generation
publisher -- clearing obsolete CDC generation data. This task
can be done together with the publishing under one group 0 guard.
We refactor publish_oldest_cdc_generation to make it possible.
Now, this function is more like a command builder. It takes guard
by const reference and updates the vector of mutations and the
reason string. The CDC generation publisher uses them directly to
update the topology at the end after finishing building the
command. This logic will be more visible after adding the clearing
task.
