Commit Graph

296 Commits

Author SHA1 Message Date
Kefu Chai
50964c423e hints: host_filter: add formatter for hints::host_filter
before this change, we rely on the default-generated fmt::formatter
created from operator<<, but fmt v10 dropped the default-generated
formatter.

in this change, we define formatters for `hints::host_filter`. its
operator<< is preserved as it's still used by the homebrew generic
formatter for vector<>, which is in turn used by db/config.cc.

Refs #13245

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#17347
2024-02-16 19:03:11 +03:00
Avi Kivity
7cb1c10fed treewide: replace seastar::future::get0() with seastar::future::get()
get0() dates back from the days where Seastar futures carried tuples, and
get0() was a way to get the first (and usually only) element. Now
it's a distraction, and Seastar is likely to deprecate and remove it.

Replace with seastar::future::get(), which does the same thing.
2024-02-02 22:12:57 +08:00
Pavel Emelyanov
7c5c89ba8d Revert "Merge 'Use utils::directories instead of db::config to get dirs' from Patryk Wróbel"
This reverts commit 370fbd346c, reversing
changes made to 0912d2a2c6.

This makes scylla-manager mis-interpret the data_file_directories
somehow, issue #17078
2024-01-31 15:08:14 +03:00
Patryk Wrobel
781a6a5071 utils/directories: make utils::directories::set an internal type
Previously, utils::directories::set could have been used by
clients of utils::directories class to provide dirs for creation.
Due to moving the responsibility for providing paths of dirs from
db::config to utils::directories, such usage is no longer the case.

This change:
 - defines utils::directories::set in utils/directories.cc to disallow
   its usage by the clients of utils::directories
 - makes utils::directories::create_and_verify() member function
   private; now it is used only by the internals of the class
 - introduces a new member function to utils::directories called
   create_and_verify_sharded_directory() to limit the functionality
   provided to clients

Signed-off-by: Patryk Wrobel <patryk.wrobel@scylladb.com>
2024-01-29 13:20:41 +01:00
Dawid Medrek
b92fb3537a main: Postpone start-up of hint manager
In this commit, we postpone the start-up
of the hint manager until we obtain information
about other nodes in the cluster.

When we start the hint managers, one of the
things that happen is creating endpoint
managers -- structures managed by
db::hints::manager. Whether we create
an instance of endpoint manager depends on
the value returned by host_filter::can_hint_for,
which, in turn, may depend on the current state
of locator::topology.

If locator::topology is incomplete, some endpoint
managers may not be started even though they
should (because the target node IS part of the
cluster and we SHOULD send hints to it if there
are some).

The situation like that can happen because we
start the hint managers too early. This commit
aims to solve that problem. We only start
the hint managers when we've gathered information
about the other nodes in the cluster and created
the locator::topology using it.

Hinted Handoff is not negatively affected by these
changes since in between the previous point of
starting the hint managers and the current one,
all of the mutations performed by
service::storage_proxy target the local node, so
no hints would need to be generated anyway.

Fixes scylladb/scylladb#11870
Closes scylladb/scylladb#16511
2024-01-26 12:49:40 +01:00
Kefu Chai
be364d30fd db: do not include unused headers
these unused includes were identified by clangd. see
https://clangd.llvm.org/guides/include-cleaner#unused-include-warning
for more details on the "Unused include" warning.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#16664
2024-01-09 11:44:19 +02:00
Petr Gusev
e50dbef3e2 database: get_token_metadata -> new token_metadata
database::get_token_metadata() is switched to token_metadata2.

get_all_ips method is added to the host_id-based token_metadata, since
its convenient and will be used in several places. It returns all current
nodes converted to inet_address by means of the topology
contained within token_metadata.

hint_sender::can_send: if the node has already left the
cluster we may not find its host_id. This case is handled
in the same way as if it's not a normal token owner - we
simply send a hint to all replicas.
2023-12-12 23:19:53 +04:00
Botond Dénes
d2a88cd8de Merge 'Typos: fix typos in code' from Yaniv Kaul
Fixes some more typos as found by codespell run on the code. In this commit, there are more user-visible errors.

Refs: https://github.com/scylladb/scylladb/issues/16255

Closes scylladb/scylladb#16289

* github.com:scylladb/scylladb:
  Update unified/build_unified.sh
  Update main.cc
  Update dist/common/scripts/scylla-housekeeping
  Typos: fix typos in code
2023-12-06 07:36:41 +02:00
Yaniv Kaul
ae2ab6000a Typos: fix typos in code
Fixes some more typos as found by codespell run on the code.
In this commit, there are more user-visible errors.

Refs: https://github.com/scylladb/scylladb/issues/16255
2023-12-05 15:18:11 +02:00
Benny Halevy
6e79d647e6 db/hints/manager: use locator::topology rather than fb_utilities
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
2023-12-05 08:42:49 +02:00
Yaniv Kaul
c658bdb150 Typos: fix typos in comments
Fixes some typos as found by codespell run on the code.
In this commit, I was hoping to fix only comments, not user-visible alerts, output, etc.
Follow-up commits will take care of them.

Refs: https://github.com/scylladb/scylladb/issues/16255
Signed-off-by: Yaniv Kaul <yaniv.kaul@scylladb.com>
2023-12-02 22:37:22 +02:00
Kefu Chai
55103f4a6b hints: move formatter of db::hints::sync_point to test
the operator<<() based formatter is only used in its test, so
let's move it to where it is used.
we can always bring it back later if it is required in other places.
but better off implementing it as a fmt::formatter<> then.

Refs #13245
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#16142
2023-11-23 11:22:31 +02:00
Dawid Medrek
6fdca0d3a8 db/hints/manager: Reword comments about state
The current comments should be clearer to someone
not familiar with the module. This commit also makes
them abide by the limit of 120 characters per line.
2023-10-06 13:25:30 +02:00
Dawid Medrek
aa38ea3642 db/hints/manager: Unfriend space_watchdog
space_watchdog is a friend of shard hint manager just to
be able to execute one of its functions. This commit changes
that by unfriending the class and exposing the function.
2023-10-06 13:25:30 +02:00
Dawid Medrek
6cd0153954 db/hints: Remove a redundant alias 2023-10-06 13:25:30 +02:00
Dawid Medrek
ddc385bce0 db/hints: Remove an unused namespace 2023-10-06 13:25:30 +02:00
Dawid Medrek
76d414012b db/hints: Coroutinize change_host_filter() 2023-10-06 13:25:30 +02:00
Dawid Medrek
09eb30e6f1 db/hints: Coroutinize drain_for()
This commit turns the function into a coroutine
and makes the code less compact and more readable.
2023-10-06 13:25:30 +02:00
Dawid Medrek
907a572e24 db/hints: Clean up can_hint_for()
This commit gets rid of unnecessary additional calls to functions
and makes all lines abide by the limit of 120 characters.
2023-10-06 13:25:30 +02:00
Dawid Medrek
596e1f9859 db/hints: Clean up store_hint()
This commit makes the function abide by the limit
of 120 characters per line.
2023-10-06 13:25:30 +02:00
Dawid Medrek
8a43f94ca6 db/hints: Clean up too_many_in_flight_hints_for()
This commit makes the return statement more readable.
It also makes the comment abide by the limit of 120 characters per line.
2023-10-06 13:25:30 +02:00
Dawid Medrek
96a5906621 db/hints: Refactor get_ep_manager() 2023-10-06 13:25:30 +02:00
Dawid Medrek
8b591be3c3 db/hints: Coroutinize wait_for_sync_point()
This commit coroutinizes the function and adds
a comment explaining a non-trivial case.
2023-10-06 13:25:27 +02:00
Dawid Medrek
fee3aafd80 db/hints: Use std::span in calculate_current_sync_point
std::span is a lot more flexible than std::vector as it allows
for arbitrary contiguous ranges.
2023-10-06 12:36:05 +02:00
Dawid Medrek
64fd4d6323 db/hints: Clean up manager::forbid_hints_for_eps_with_pending_hints() 2023-10-06 12:26:55 +02:00
Dawid Medrek
58cd5c4167 db/hints: Clean up manager::forbid_hints() 2023-10-06 12:26:55 +02:00
Dawid Medrek
f8ed93f5bc db/hints: Clean up manager::allow_hints() 2023-10-06 12:26:52 +02:00
Dawid Medrek
bfe32bcf89 db/hints: Coroutinize compute_hints_dir_device_id() 2023-10-06 12:18:30 +02:00
Dawid Medrek
8f28eb6522 db/hints: Clean up manager::stop()
This commit gets rid of boilerplate in the function,
leverages a range pipe and explicit types to make
the code more readable, and changes the logs to
make it clearer what happens.
2023-10-06 12:18:30 +02:00
Dawid Medrek
a384caece0 db/hints: Clean up manager::start()
This commit coroutinizes the function and makes it less compact.
2023-10-06 12:18:30 +02:00
Dawid Medrek
2db97aaf81 db/hints/manager: Clean up the constructor
fmt::to_string should be preferred to seastar::format.
It's clearer and simpler. Besides that, this commit makes
the code abide by the limit of 120 characters per line.
2023-10-06 12:18:30 +02:00
Dawid Medrek
6c10a86791 db/hints: Remove boilerplate drain_lock() 2023-10-06 12:18:30 +02:00
Dawid Medrek
f1f35ba819 db/hints: Let drain_for() return a future
Currently, the function doesn't return anything.
However, if the futurue doesn't need to be awaited,
the caller can decide that. There is no reason
to make that decision in the function itself.
2023-10-06 12:18:25 +02:00
Dawid Medrek
79e1412f14 db/hints: Remove ep_managers_end
The methods are redundant and are effectively
code boilerplate.
2023-10-06 12:15:04 +02:00
Dawid Medrek
cfbacb29bb db/hints: Remove find_ep_manager
The methods are redundant and are effectively
code boilerplate.
2023-10-06 12:15:04 +02:00
Dawid Medrek
1c70a18fc7 db/hints: Use manager as API for hint_endpoint_manager
This commit makes with_file_update_mutex() a method of hint_endpoint_manager
and introduces db::hints::manager::with_file_update_mutex_for() for accessing
it from the outside. This way, hint_endpoint_manager is hidden and no one
needs to know about its existence.
2023-10-06 12:15:01 +02:00
Dawid Medrek
d068143b83 db/hints: Don't mark have_ep_manager()'s definition as inline
Doing that doesn't allow for external linkage, so
it's not accessible from other files.
2023-10-06 11:54:15 +02:00
Dawid Medrek
58249363bc db/hints: Remove make_directory_initializer()
The function is never used. It's not even implemented.
2023-10-06 11:54:15 +02:00
Dawid Medrek
f47a669f75 db/hints/manager: Order constructors
This commit orders constructors of db::hints::manager for readability.
2023-10-06 11:54:15 +02:00
Dawid Medrek
4663f72990 db/hints: Move ~manager() and mark it as noexcept
The destructor is trivial and there is no reason
to keep in the source file. We mark it as noexcept too.
2023-10-06 11:54:15 +02:00
Dawid Medrek
18a2831186 db/hints: Use reference for storage proxy
This commit makes db::hints::manager store service::storage_proxy
as a reference instead of a seastar::shared_ptr. The manager is
owned by storage proxy, so it only lives as long as storage proxy
does. Hence, it makes little sense to store the latter as a shared
pointer; in fact, it's very confusing and may be error-prone.
The field never changes, so it's safe to keep it as a reference
(especially because copy and move constructors of db::hints::manager
are both deleted). What's more, we ensure that the hint manager
has access to storage proxy as soon as it's created.

The same changes were applied to db::hints::resource_manager.
The rationale is the same.
2023-10-06 11:54:15 +02:00
Dawid Medrek
3c347cc196 db/hints/manager: Explicitly delete copy constructor
This commit explicitly deletes the copy constructor of
db::hints::manager and its copy assignment. They're not
used in the code, and they should not.
2023-10-06 11:54:15 +02:00
Dawid Medrek
ee5a5c1661 db/hints: Capitalize constants
This is a common convention. Follow it for readability.
2023-10-06 11:54:15 +02:00
Dawid Medrek
fd30bac7b1 db/hints/manager: Hide declarations 2023-10-06 11:54:15 +02:00
Dawid Medrek
4b03cba1bf db/hints/manager: Move the defintions of static members to the header
If the variables are accessible from the outside, it makes
sense to also expose their initial values to the user.
This commit moves them to the header and marks as inline.
2023-10-06 11:54:15 +02:00
Dawid Medrek
c3ab28f5e9 db/hints: Move make_dummy() to the header
The function is trivial. It can also be marked as noexcept.
2023-10-06 11:54:15 +02:00
Dawid Medrek
5e333f0a52 db/hints: Don't explicitly define ~directory_initializer()
The destructor is the default destructor, and it is safe
to drop it altogether.
2023-10-06 11:53:02 +02:00
Dawid Medrek
9f215d3cf1 db/hints: Change the order of logging in ensure_created_and_verified()
The new logging order seems to make more sense, i.e.
we first log that we're creating and validating directories,
and only then do we start doing that.
The previous order when those actions were reversed didn't
match the log's message because the action was already
done when we informed the user of it.
2023-10-06 11:14:41 +02:00
Dawid Medrek
4ad3e8d37b db/hints: Coroutinize ensure_rebalanced() 2023-10-06 11:14:41 +02:00
Dawid Medrek
672cdb5c05 db/hints: Coroutinize ensure_created_and_verified() 2023-10-06 11:14:41 +02:00