since we do not rely on FMT_DEPRECATED_OSTREAM to define the
fmt::formatter for us anymore, let's stop defining `FMT_DEPRECATED_OSTREAM`.
in this change,
* utils: drop the range formatters in to_string.hh and to_string.c, as
we don't use them anymore. and the tests for them in
test/boost/string_format_test.cc are removed accordingly.
* utils: use fmt to print chunk_vector and small_vector. as
we are not able to print the elements using operator<< anymore
after switching to {fmt} formatters.
* test/boost: specialize fmt::details::is_std_string_like<bytes>
due to a bug in {fmt} v9, {fmt} fails to format a range whose
element type is `basic_sstring<uint8_t>`, as it considers it
as a string-like type, but `basic_sstring<uint8_t>`'s char type
is signed char, not char. this issue does not exist in {fmt} v10,
so, in this change, we add a workaround to explicitly specialize
the type trait to assure that {fmt} format this type using its
`fmt::formatter` specialization instead of trying to format it
as a string. also, {fmt}'s generic ranges formatter calls the
pair formatter's `set_brackets()` and `set_separator()` methods
when printing the range, but operator<< based formatter does not
provide these method, we have to include this change in the change
switching to {fmt}, otherwise the change specializing
`fmt::details::is_std_string_like<bytes>` won't compile.
* test/boost: in tests, we use `BOOST_REQUIRE_EQUAL()` and its friends
for comparing values. but without the operator<< based formatters,
Boost.Test would not be able to print them. after removing
the homebrew formatters, we need to use the generic
`boost_test_print_type()` helper to do this job. so we are
including `test_utils.hh` in tests so that we can print
the formattable types.
* treewide: add "#include "utils/to_string.hh" where
`fmt::formatter<optional<>>` is used.
* configure.py: do not define FMT_DEPRECATED_OSTREAM
* cmake: do not define FMT_DEPRECATED_OSTREAM
Refs #13245
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
This is a different way attempting to combine building an optimized clang (using LTO, PGO and BOLT, based on compiling ScyllaDB) to dbuild. Per Avi's request, there are 3 options: skip this phase (which is the current default), build it and build + install it to the default path.
Fixes: #10985Fixes: scylladb/scylla-enterprise#2539
Add --build-dir option to specify build directory.
This is needed for optimized clang support, since it requires to build
Scylla in tools/toolchain/prepare, w/o deleting current build/
directory.
before this change, SCYLLA-{PRODUCT,VERSION,RELEASE}-FILE is generated
at the first run of `configure.py`, once these files are around, they
are not updated despite that `SCYLLA_VERSION_GEN` does not generate
them as long as the release string retrieved from git sha1 is identical
the one stored in `SCYLLA-RELEASE-FILE`, because we don't rerun
`SCYLLA_VERSION_GEN` at all.
but the pain is, when performing incremental build, like other built
artifacts, these generated files stay with the build directory, so
even if the sha1 of the workspace changes, the SCYLLA-RELEASE-FILE
keeps the same -- it still contains the original git sha1 when it
was created. this could leads to confusion if developer or even our
CI perform incremental build using the same workspace and build
directory, as the built scylla executables always report the same
version number.
in this change, we always rebuilt the said
SCYLLA-{PRODUCT,VERSION,RELEASE}-FILE files, and instruct ninja
to re-stat the output files, see
https://ninja-build.org/manual.html#ref_rule, in order to avoid
unnecessary rebuild. so the downside is that `SCYLLA_VERSION_GEN`
is executed every time we run `ninja` even if all targets are updated.
but the upside is that the release number reported by scylla is
accurate even if we perform incremental build.
also, since we encode the product, version and release stored
in the above files in the generated `build.ninja` file, in this change,
these three files are added as dependencies of `build.ninja`,
so that this file is regenerated if any of them is newer than
`build.ninja`.
Fixes#8255
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#17974
`raft_service_level_distributed_data_accessor` works this way:
- on read path it reads service levels from `SYSTEM.SERVICE_LEVELS_V2`
table with CL = LOCAL_ONE
- on write path it starts group0 operation and it makes the change
using raft command
New keyspace is added similarly as system_schema keyspace,
it's being registred via system_keyspace::make which calls
all_tables to build its schema.
Dummy table 'roles' is added as keyspaces are being currently
registered by walking through their tables. Full table schemas
will be added in subsequent commits.
Change can be observed via cqlsh:
cassandra@cqlsh> describe keyspaces;
system_auth_v2 system_schema system system_distributed_everywhere
system_auth system_distributed system_traces
cassandra@cqlsh> describe keyspace system_auth_v2;
CREATE KEYSPACE system_auth_v2 WITH replication = {'class': 'LocalStrategy'} AND durable_writes = true;
CREATE TABLE system_auth_v2.roles (
role text PRIMARY KEY
) WITH bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}
AND comment = 'comment'
AND compaction = {'class': 'SizeTieredCompactionStrategy'}
AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.0
AND default_time_to_live = 0
AND gc_grace_seconds = 604800
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
RPC calls lose information about the type of returned exception.
Thus, if a table is dropped on receiver node, but it still exists
on a sender node and sender node streams the table's data, then
the whole operation fails.
To prevent that, add a method which synchronizes schema and then
checks, if the exception was caused by table drop. If so,
the exception is swallowed.
Use the method in streaming and repair to continue them when
the table is dropped in the meantime.
Fixes: #17028.
Fixes: #15370.
Fixes: #15598.
Closesscylladb/scylladb#17231
* github.com:scylladb/scylladb:
repair: handle no_such_column_family from remote node gracefully
test: test drop table on receiver side during streaming
streaming: fix indentation
streaming: handle no_such_column_family from remote node gracefully
repair: add methods to skip dropped table
range.hh was deprecated in bd794629f9 (2020) since its names
conflict with the C++ library concept of an iterator range. The name
::range also mapped to the dangerous wrapping_interval rather than
nonwrapping_interval.
Complete the deprecation by removing range.hh and replacing all the
aliases by the names they point to from the interval library. Note
this now exposes uses of wrapping intervals as they are now explicit.
The unit tests are renamed and range.hh is deleted.
Closesscylladb/scylladb#17428
`-Wno-unused-command-line-argument` is used to disable the warning of
`-Wunused-command-line-argument`, which is in turn used to split
warnings if any of the command line arguments passed to the compiler
driver is not used. see
https://clang.llvm.org/docs/DiagnosticsReference.html#wunused-command-line-argument
but it seems we are not passing unused command line arguments to
the compiler anymore. so let's drop this option.
this change helps to
* reduce the discrepencies between the compiling options used by
CMake-generated rules and those generated directly using
`configure.py`
* reenable the warning so we are aware if any of the options
is not used by compiler. this could a sign that the option fails
to serve its purpose.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#17195
ldflags are passed to ld (the linker), while cxxflags are passed to the
C++ compiler. the compiler does not understand the ldflags. if we
pass ldflags to it, it complains if `-Wunused-command-line-argument` is
enabled.
in this change, we do not include the ldflags in cxxflags, this helps
us to enable the warning option of `-Wunused-command-line-argument`,
so we don't need to disabled it.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#17328
Schema propagation is async so one node can see the table while on
the other node it is already dropped. So, if the nodes stream
the table data, the latter node throws no_such_column_family.
The exception is propagated to the other node, but its type is lost,
so the operation fails on the other node.
Add method which waits until all raft changes are applied and then
checks whether given table exists.
Add the function which uses the above to determine, whether the function
failed because of dropped table (eg. on the remote node so the exact
exception type is unknown). If so, the exception isn't rethrown.
`-fno-sanitize-address-use-after-scope` is used to disable the check for
stack-use-after-scope bugs, but this check is only performed when ASan
is enabled. if we pass this option when ASan is not enabled, we'd have
following warning, so let's apply it only when ASan is enabled.
```
clang-16: error: argument unused during compilation:
'-fno-sanitize-address-use-after-scope' [-Werror,-Wunused-command-line-argument]
```
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#17329
The true motivation for this patch is a certain problem with configure.py
in scylla-enterprise, which can only be solved by moving the `extra_cxxflags`
lines before configure_seastar(). This patch does that by hoisting
get_extra_cxxflags() up to create_build_system().
But this patch makes sense even if we disregard the real motivation.
It's weird that a function called `write_build_file()` adds additional
build flags on its own.
Closesscylladb/scylladb#17189
Adds a test reproducing https://github.com/scylladb/scylladb/issues/16759, and the instrumentation needed for it.
Closesscylladb/scylladb#17208
* github.com:scylladb/scylladb:
row_cache_test: test cache consistency during memtable-to-cache merge
row_cache: use preemption_source in update()
utils: preempt: add preemption_source
PEP 632 deprecates distutils module, and it is remove from Python 3.12.
we are actually using the one vendored by setuptools, if we are using
3.12. so let's use shutil for finding ninja executable.
see https://peps.python.org/pep-0632/
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#17271
we recently added -Wextra to configure.py, and this option enables
a bunch of warning options, including `-Wignored-qualifiers`. so
there is no need to enable this specific warning anymore. this change
remove ths option from both `configure.py` and the CMake building system.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#17272
Because Seastar now defaults to C++23, we downgrade it explicitly to
C++20.
* seastar 289ad5e593...5d3ee98073 (10):
> Update supported C++ standards to C++23 and C++20 (dropping C++17)
> docker: install clang-tools-18
> http: add handler_base::verify_mandatory_params()
> coroutine/exception: document return_exception_ptr()
> http: use structured-binding when appropriate
> test/http: Read full server response before sending next
> doc/lambda-coroutine-fiasco: fix a syntax error
> util/source_location-compat: use __cpp_consteval
> Fix incorrect class name in documentation.
> Add support for missing HTTP PATCH method.
Closesscylladb/scylladb#17268
While `preemption_check` can be passed to functions to control
their preemption points, there is no way to inspect the
state of the system after the preemption results in a yield.
`preemption_source` is a superset of `preemption_check`,
which also allows for customizing the yield, not just the preemption
check. An implementation passed by a test can hook the yield to
put the tested function to sleep, run some code, and then wake the
function up.
We use the preprocessor to minimize the impact on release builds.
Only dev-mode preemption_source is hookable. When it's used in other
modes, it should compile to direct reactor calls, as if it wasn't used.
also disable some more warnings which are failing the build after
`-Wextra` is enabled. we can fix them on a case-by-case basis, if
they are geniune issues. but before that, we just disable them.
this goal of this change is to reduce the discrepancies between
the compile options used by CMake and those used by configure.py.
the side effect is that we enable some more warning enabeld by
`-Wextra`, for instance, `-Wsign-compare` is enable now. for
the full list of the enabled warnings when building with Clang,
please see https://clang.llvm.org/docs/DiagnosticsReference.html#wextra.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
This PR fixes the bug of certain calls to the `mintimeuuid()` CQL function which large negative timestamps could crash Scylla. It turns out we already had protections in place against very positive timestamps, but very negative timestamps could still cause bugs.
The actual fix in this series is just a few lines, but the bigger effort was improving the test coverage in this area. I added tests for the "date" type (the original reproducer for this bug used totimestamp() which takes a date parameter), and also reproducers for this bug directly, without totimestamp() function, and one with that function.
Finally this PR also replaces the assert() which made this molehill-of-a-bug into a mountain, by a throw.
Fixes#17035Closesscylladb/scylladb#17073
* github.com:scylladb/scylladb:
utils: replace assert() by on_internal_error()
utils: add on_internal_error with common logger
utils: add a timeuuid minimum, like we had maximum
test/cql-pytest: tests for "date" type
Seastar's on_internal_error() is a useful replacement for assert()
but it's inconvenient that it requires each caller to supply a logger -
which is often inconvenient, especially when the caller is a header file.
So in this patch we introduce a utils::on_internal_error() function
which is the same as seastar::on_internal_error() (the former calls
the latter), except it uses a single logger instead of asking the caller
to pass a logger.
Refs #7871
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
This reverts commit 370fbd346c, reversing
changes made to 0912d2a2c6.
This makes scylla-manager mis-interpret the data_file_directories
somehow, issue #17078
we add `-DBOOST_TEST_DYN_LINK` to the cflags when `--static-boost` is
not passed to `configure.py`. but we don't never pass this option to
`configure.py` in our CI/CD. also, we don't install `boost-static` in
`install-dependencies.sh`, so the linker always use the boost shared
libraries when building scylla and other executables in this project.
this fact has been verified with the latest master HEAD, after building
scylla from `build.ninja` which was in turn created using `configure.py`.
Seastar::seastar_testing exposes `Boost::dynamic_linking` in its public
interface, and `Boost::dynamic_linking` exposes `-DBOOST_ALL_DYN_LINK`
as one of its cflags.
so, when building testings using CMake, the tests are compiled with
`-DBOOST_ALL_DYN_LINK`, while when building tests using `configure.py`,
they are compiled with `-DBOOST_TEST_DYN_LINK`. the former is exposed
by `Boost::dynamic_linking`, the latter is hardwired using
`configure.py`. but the net results are identical. it would be better
to use identical cflags on these two building systems. so, let's use
`-DBOOST_ALL_DYN_LINK` in `configure.py` also. furthermore, this is what
non-static-boost implies.
please note, we don't consume the cflags exposed by
`seastar-testing.pc`, so they don't override the ones we set using
`configure.py`.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#17070
`db::config` is a class, that is used in many places across the code base. When it is changed, its clients' code need to be recompiled. It represents the configuration of the database. Some fields of the configuration that describe the location of directories may be empty. In such cases `db::config::setup_directories()` function is called - it modifies the provided configuration. Such modification is not good - it is better to keep `db::config` intact.
This PR:
- extends the public interface of utils::directories class to provide required directory paths to the users
- removes 'db::config::setup_directories()' to avoid altering the fields of configuration object
- replaces usages of db::config object with utils::directories object in places that require obtaining paths to dirs
Fixes: scylladb#5626
Closesscylladb/scylladb#16787
* github.com:scylladb/scylladb:
utils/directories: make utils::directories::set an internal type
db::config: keep dir paths unchanged
cql_transport/controler: use utils::directories to get paths of dirs
service/storage_proxy: use utils::directories to get paths of dirs
api/storage_service.cc: use utils::directories to get paths of dirs
tools/scylla-sstable.cc: use utils::directories to get paths
db/commitlog: do not use db::config to get dirs
Use utils::directories to get dirs paths in replica::database
Allow utils::directories to provide paths to dirs
Clean-up of utils::directories
This allows the user of `raft::server` to cause it to create a snapshot
and truncate the Raft log (leaving no trailing entries; in the future we
may extend the API to specify number of trailing entries left if
needed). In a later commit we'll add a REST endpoint to Scylla to
trigger group 0 snapshots.
One use case for this API is to create group 0 snapshots in Scylla
deployments which upgraded to Raft in version 5.2 and started with an
empty Raft log with no snapshot at the beginning. This causes problems,
e.g. when a new node bootstraps to the cluster, it will not receive a
snapshot that would contain both schema and group 0 history, which would
then lead to inconsistent schema state and trigger assertion failures as
observed in scylladb/scylladb#16683.
In 5.4 the logic of initial group 0 setup was changed to start the Raft
log with a snapshot at index 1 (ff386e7a44)
but a problem remains with these existing deployments coming from 5.2,
we need a way to trigger a snapshot in them (other than performing 1000
arbitrary schema changes).
Another potential use case in the future would be to trigger snapshots
based on external memory pressure in tablet Raft groups (for strongly
consistent tables).
The PR adds the API to `raft::server` and a HTTP endpoint that uses it.
In a follow-up PR, we plan to modify group 0 server startup logic to automatically
call this API if it sees that no snapshot is present yet (to automatically
fix the aforementioned 5.2 deployments once they upgrade.)
Closesscylladb/scylladb#16816
* github.com:scylladb/scylladb:
raft: remove `empty()` from `fsm_output`
test: add test for manual triggering of Raft snapshots
api: add HTTP endpoint to trigger Raft snapshots
raft: server: add `trigger_snapshot` API
raft: server: track last persisted snapshot descriptor index
raft: server: framework for handling server requests
raft: server: inline `poll_fsm_output`
raft: server: fix indentation
raft: server: move `io_fiber`'s processing of `batch` to a separate function
raft: move `poll_output()` from `fsm` to `server`
raft: move `_sm_events` from `fsm` to `server`
raft: fsm: remove constructor used only in tests
raft: fsm: move trace message from `poll_output` to `has_output`
raft: fsm: extract `has_output()`
raft: pass `max_trailing_entries` through `fsm_output` to `store_snapshot_descriptor`
raft: server: pass `*_aborted` to `set_exception` call
This change extends utils::directories class in
the following way:
- adds new member variables that correspond to
fields from db::config that describe paths
of directories
- introduces a public interface to retrieve the
values of the new members
- allows construction of utils::directories
object based on db::config to setup internal
member variables related to paths to dirs
The new members of utils::directories are overriden
when the provided values are empty. The way of setting
paths is taken from db::config.
To ensure that the new logic works correctly
`utils_directories_test` has been created.
Refs: scylladb#5626
Signed-off-by: Patryk Wrobel <patryk.wrobel@scylladb.com>
The `topology_coordinator` is a large class (>1000 loc) which resides in
an even larger source file (storage_service.cc, ~7800 loc). This PR
moves the topology_coordinator class out of the storage_service.cc file
in order to improve modularity and recompilation times during
development.
As a first step, the `topology_mutation_builder` and
`topology_node_mutation_builder` classes are also moved from
storage_service.cc to their own, new header/source files as they are an
important abstraction used both by the topology coordinator code and
some other code in storage_service.cc that won't be moved.
Then, the `topology_coordinator` is moved out. The
`topology_coordinator` class is completely hidden in the new
topology_coordinator.cc file and can only be started and waited on to
finish via the new `run_topology_coordinator` function.
Fixes: scylladb/scylladb#16605Closesscylladb/scylladb#16609
* github.com:scylladb/scylladb:
service: move topology coordinator to a separate file
storage_service: introduce run_topology_coordinator function
service: move topology mutation builder out of storage_service
storage_service: detemplate topology_node_mutation_builder::set
The topology coordinator is a large class that sits in an even larger
storage_service.cc file. For the sake of code modularization and
reducing recompilation time, move the topology coordinator outside
storage_service.cc.
The topology_coordinator class is moved to the new
topology_coordinator.cc unchanged. Along with it, the following items
are moved:
- wait_for_ip function - it's used both by storage_service and
topology_coordinator, so in order for the new topology_coordinator.cc
not to depend on storage service, it is moved to the new file,
- raft_topology logger - for the same reason as wait_for_ip,
- run_topology_coordinator - serves as the main interface for the
topology coordinator. The topology coordinator class is not exposed at
all, it's only possible to start the coordinator and wait until it
shuts down itself via that function.
This uses the `trigger_snapshot()` API added in previous commit on a
server running for the given Raft group.
It can be used for example in tests or in the context of disaster
recovery (ref scylladb/scylladb#16683).
The topology_mutation_builder, topology_node_mutation_builder and
topology_request_tracking_mutation_builder are currently used by
storage service - mainly, but not exclusively, by the topology
coordinator logic. As we are going to extract the topology coordinator
to a separate file, we need to move the builders to their own file as
well so that they will be accessible both by the topology coordinator
and the storage service.
it seems that the tree builds just fine with this warning enabled.
and narrowing is a potentially unsafe numeric conversion. so let's
enable this warning option.
this change also helps to reduce the difference between the rules
generated by configure.py and those generated by CMake.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#16929
We already have a dedicated coverage build, however, this build is
dedicated mostly for coverage in boost and standalone unit tests.
This added configuration option will compile every configured
build mode with coverage profiling support (excluding 'coverage' mode).
It also does targeted profiling that is narrowed down only to ScyllaDB
code and doesn't instrument seastar and testing code, this should give
a more accurate coverage reporting and also impact performance less, as
one example, the reactor loop in seastar will not be profiled (along
with everything else).
The targeted profiling is done with the help of the newly added
`coverage_sources.list` file which excludes all seastar sub directories
from the profiling.
Also an extra measure is taken to make sure that the seastar
library will not be linked with the coverage framework
(so it will not dump confusing empty profiles).
Some of the seastar headers are still going to be included in the
profile since they are indirectly included by profiled source files in
order to remove them from the final report a processing step on the
resulting profile will need to take place.
A note about expected performance impact:
It is expected to have minimal impact on performance since the
instrumentation adds counter increments without locking.
Ref: https://clang.llvm.org/docs/UsersManual.html#cmdoption-fprofile-update
This means that the numbers themselves are less reliable but all covered
lines are guarantied to have at least non-zero value.
Signed-off-by: Eliran Sinvani <eliransin@scylladb.com>
a209ae15 addresses that last -Wimplicit-int-float-conversion warning
in the tree, so we now have the luxury of enabling this warning.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#16640
`--static-boost` is an option provided by `configure.py`. this option is
not used by our CI or building scripts. but in order to be compatible
with the existing behavior of `configure.py`, let's support this option
when building with CMake.
`Boost_USE_STATIC_LIBS` is a cmake variable supported by CMake's
FindBoost and Boost's own `BoostConfig.cmake`. see
https://cmake.org/cmake/help/latest/module/FindBoost.html#other-variables
by default boost is linked via its shared libraries. by setting
this variable, we link boost's static libraries.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#16545
this preserves the existing behavior of `configure.py` in the CMake
generated `build.ninja`.
* configure.py: map 'release' to 'RelWithDebInfo'
* cmake: rename cmake/mode.Release.cmake to cmake/mode.RelWithDebInfo.cmake
* CMakeLists.txt: s/Release/RelWithDebInfo/
Closesscylladb/scylladb#16479
* github.com:scylladb/scylladb:
build: cmake: map 'release' to 'RelWithDebInfo'
build: define BuildType for enclosing build_by_default
It enables interaction with the node through CQL protocol without authentication. It gives full-permission access.
The maintenance socket is available by Unix domain socket with file permissions `755`, thus it is not accessible from outside of the node and from other POSIX groups on the node.
It is created before the node joins the cluster.
To set up the maintenance socket, use the `maintenance-socket` option when starting the node.
* If set to `ignore` maintenance socket will not be created.
* If set to `workdir` maintenance socket will be created in `<node's workdir>/cql.m`.
* Otherwise maintenance socket will be created in the specified path.
The default value is `ignore`.
* With python driver
```python
from cassandra.cluster import Cluster
from cassandra.connection import UnixSocketEndPoint
from cassandra.policies import HostFilterPolicy, RoundRobinPolicy
socket = "<node's workdir>/cql.m"
cluster = Cluster([UnixSocketEndPoint(socket)],
# Driver tries to connect to other nodes in the cluster, so we need to filter them out.
load_balancing_policy=HostFilterPolicy(RoundRobinPolicy(), lambda h: h.address == socket))
session = cluster.connect()
```
Merge note: apparently cqlsh does not support unix domain sockets; it
will have to be fixed in a follow-up.
Closesscylladb/scylladb#16172
* github.com:scylladb/scylladb:
test.py: add maintenance socket test
test.py: enable maintenance socket in tests by default
docs: add maintenance socket documentation
main: add maintenance socket
main: refactor initialization of cql controller and auth service
auth/service: don't create system_auth keyspace when used by maintenance socket
cql_controller: maintenance socket: fix indentation
cql_controller: add option to start maintenance socket
db/config: add maintenance_socket_enabled bool class
auth: add maintenance_socket_role_manager
db/config: add maintenance_socket variable
this preserves the existing behavior of `configure.py` in the CMake
generated `build.ninja`.
* configure.py: map 'release' to 'RelWithDebInfo'
* cmake: rename cmake/mode.Release.cmake to cmake/mode.RelWithDebInfo.cmake
* CMakeLists.txt: s/Release/RelWithDebInfo/
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
in existing `modes` defined in `configure.py`, "release" is mapped to
"RelWithDebInfo". this behavior matches that of seastar's
`configure.py`, where we also map "release" build mode to
"RelWithDebInfo" CMAKE_BUILD_TYPE.
but in scylladb's existing cmake settings, it maps "release" to
"Release", despite "Release" is listed as one of the typical
CMAKE_BUILD_TYPE values.
so, in this change, to prepare for the mapping, `BuildType` is
introduced to map a build mode to its related settings. the
building settings are still kept in `cmake.${CMAKE_BUILD_TYPE}.cmake`,
but the other settings, like if a build type should be enabled or
its mappings, are stored in `BuildType` in `configure.py`.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Add `maintenance_socket_role_manager` which will disable all operations
associated with roles to not depend on system_auth keyspace, which may
be not yet created when the maintenance socket starts listening
The task for splitting compaction will run until all sstables
in the main set are split. The only exceptions are shutdown
or user has explicitly asked for abort.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Token group is an abstraction that allows us to easily segregate a
mutation stream into buckets. Groups share the same properties as
compaction groups. Groups follow the ring order and they don't
overlap each other. Groups are defined according to a classifier,
which return an id given a token. It's expected that classifier
return ids in monotonic increasing order.
The reasons for this abstraction are:
1) we don't want to make segregator aware of compaction groups
2) splitting happens before tablet metadata is changed, so the
the segregator will have to classify based on whether the token
belongs to left (group id 0) or right (group id 1) side of
the range to be split.
The reason for not extending sstable writer instead, is that
today, writer consumer can only tell producer to switch to a
new writer, when consuming the end of a partition, but that
would be too late for us, as we have to decide to move to
a new writer at partition start instead.
It will be wired into compaction when it happens in split mode.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
For all compaction types which can be started with api, add an asynchronous
version of api, which returns task_id of the corresponding task manager
task. With the task_id a user can check task status, abort, or wait for it,
using task manager api.
as part of the efforts to migrate to the CMake-based building system,
this change enables us to `configure.py` to optionally create
`build.ninja` with CMake.
in this change, we add a new option named `--use-cmake` to
`configure.py` so we can create `build.ninja`. please note,
instead of using the "Ninja" generator used by Seastar's
`configure.py` script, we use "Ninja Multi-Config" generator
along with `CMAKE_CROSS_CONFIGS` setting in this project.
so that we can generate a `build.ninja` which is capable of
building the same artifacts with multiple configuration.
Refs #15379
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#15916
* github.com:scylladb/scylladb:
build: cmake: add compatibility target of dev-headers
build: add an option to use CMake as the build build system
our CI builds "dev-headers" as a gating check. but the target names
generated by CMake's Ninja Multi-Config generator does not follow
this naming convention. we could have headers:Dev, but still, it's
different from what we are using, before completely switching to
CMake, let's keep this backward compatibility by adding a target
with the same name.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
