Commit Graph

210 Commits

Author SHA1 Message Date
Pavel Emelyanov
7c5c89ba8d Revert "Merge 'Use utils::directories instead of db::config to get dirs' from Patryk Wróbel"
This reverts commit 370fbd346c, reversing
changes made to 0912d2a2c6.

This makes scylla-manager mis-interpret the data_file_directories
somehow, issue #17078
2024-01-31 15:08:14 +03:00
Patryk Wrobel
781a6a5071 utils/directories: make utils::directories::set an internal type
Previously, utils::directories::set could have been used by
clients of utils::directories class to provide dirs for creation.
Due to moving the responsibility for providing paths of dirs from
db::config to utils::directories, such usage is no longer the case.

This change:
 - defines utils::directories::set in utils/directories.cc to disallow
   its usage by the clients of utils::directories
 - makes utils::directories::create_and_verify() member function
   private; now it is used only by the internals of the class
 - introduces a new member function to utils::directories called
   create_and_verify_sharded_directory() to limit the functionality
   provided to clients

Signed-off-by: Patryk Wrobel <patryk.wrobel@scylladb.com>
2024-01-29 13:20:41 +01:00
Kefu Chai
be364d30fd db: do not include unused headers
these unused includes were identified by clangd. see
https://clangd.llvm.org/guides/include-cleaner#unused-include-warning
for more details on the "Unused include" warning.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes scylladb/scylladb#16664
2024-01-09 11:44:19 +02:00
Benny Halevy
6e79d647e6 db/hints/manager: use locator::topology rather than fb_utilities
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
2023-12-05 08:42:49 +02:00
Dawid Medrek
ddc385bce0 db/hints: Remove an unused namespace 2023-10-06 13:25:30 +02:00
Dawid Medrek
76d414012b db/hints: Coroutinize change_host_filter() 2023-10-06 13:25:30 +02:00
Dawid Medrek
09eb30e6f1 db/hints: Coroutinize drain_for()
This commit turns the function into a coroutine
and makes the code less compact and more readable.
2023-10-06 13:25:30 +02:00
Dawid Medrek
907a572e24 db/hints: Clean up can_hint_for()
This commit gets rid of unnecessary additional calls to functions
and makes all lines abide by the limit of 120 characters.
2023-10-06 13:25:30 +02:00
Dawid Medrek
596e1f9859 db/hints: Clean up store_hint()
This commit makes the function abide by the limit
of 120 characters per line.
2023-10-06 13:25:30 +02:00
Dawid Medrek
8a43f94ca6 db/hints: Clean up too_many_in_flight_hints_for()
This commit makes the return statement more readable.
It also makes the comment abide by the limit of 120 characters per line.
2023-10-06 13:25:30 +02:00
Dawid Medrek
96a5906621 db/hints: Refactor get_ep_manager() 2023-10-06 13:25:30 +02:00
Dawid Medrek
8b591be3c3 db/hints: Coroutinize wait_for_sync_point()
This commit coroutinizes the function and adds
a comment explaining a non-trivial case.
2023-10-06 13:25:27 +02:00
Dawid Medrek
fee3aafd80 db/hints: Use std::span in calculate_current_sync_point
std::span is a lot more flexible than std::vector as it allows
for arbitrary contiguous ranges.
2023-10-06 12:36:05 +02:00
Dawid Medrek
64fd4d6323 db/hints: Clean up manager::forbid_hints_for_eps_with_pending_hints() 2023-10-06 12:26:55 +02:00
Dawid Medrek
58cd5c4167 db/hints: Clean up manager::forbid_hints() 2023-10-06 12:26:55 +02:00
Dawid Medrek
f8ed93f5bc db/hints: Clean up manager::allow_hints() 2023-10-06 12:26:52 +02:00
Dawid Medrek
bfe32bcf89 db/hints: Coroutinize compute_hints_dir_device_id() 2023-10-06 12:18:30 +02:00
Dawid Medrek
8f28eb6522 db/hints: Clean up manager::stop()
This commit gets rid of boilerplate in the function,
leverages a range pipe and explicit types to make
the code more readable, and changes the logs to
make it clearer what happens.
2023-10-06 12:18:30 +02:00
Dawid Medrek
a384caece0 db/hints: Clean up manager::start()
This commit coroutinizes the function and makes it less compact.
2023-10-06 12:18:30 +02:00
Dawid Medrek
2db97aaf81 db/hints/manager: Clean up the constructor
fmt::to_string should be preferred to seastar::format.
It's clearer and simpler. Besides that, this commit makes
the code abide by the limit of 120 characters per line.
2023-10-06 12:18:30 +02:00
Dawid Medrek
6c10a86791 db/hints: Remove boilerplate drain_lock() 2023-10-06 12:18:30 +02:00
Dawid Medrek
f1f35ba819 db/hints: Let drain_for() return a future
Currently, the function doesn't return anything.
However, if the futurue doesn't need to be awaited,
the caller can decide that. There is no reason
to make that decision in the function itself.
2023-10-06 12:18:25 +02:00
Dawid Medrek
79e1412f14 db/hints: Remove ep_managers_end
The methods are redundant and are effectively
code boilerplate.
2023-10-06 12:15:04 +02:00
Dawid Medrek
cfbacb29bb db/hints: Remove find_ep_manager
The methods are redundant and are effectively
code boilerplate.
2023-10-06 12:15:04 +02:00
Dawid Medrek
1c70a18fc7 db/hints: Use manager as API for hint_endpoint_manager
This commit makes with_file_update_mutex() a method of hint_endpoint_manager
and introduces db::hints::manager::with_file_update_mutex_for() for accessing
it from the outside. This way, hint_endpoint_manager is hidden and no one
needs to know about its existence.
2023-10-06 12:15:01 +02:00
Dawid Medrek
d068143b83 db/hints: Don't mark have_ep_manager()'s definition as inline
Doing that doesn't allow for external linkage, so
it's not accessible from other files.
2023-10-06 11:54:15 +02:00
Dawid Medrek
4663f72990 db/hints: Move ~manager() and mark it as noexcept
The destructor is trivial and there is no reason
to keep in the source file. We mark it as noexcept too.
2023-10-06 11:54:15 +02:00
Dawid Medrek
18a2831186 db/hints: Use reference for storage proxy
This commit makes db::hints::manager store service::storage_proxy
as a reference instead of a seastar::shared_ptr. The manager is
owned by storage proxy, so it only lives as long as storage proxy
does. Hence, it makes little sense to store the latter as a shared
pointer; in fact, it's very confusing and may be error-prone.
The field never changes, so it's safe to keep it as a reference
(especially because copy and move constructors of db::hints::manager
are both deleted). What's more, we ensure that the hint manager
has access to storage proxy as soon as it's created.

The same changes were applied to db::hints::resource_manager.
The rationale is the same.
2023-10-06 11:54:15 +02:00
Dawid Medrek
ee5a5c1661 db/hints: Capitalize constants
This is a common convention. Follow it for readability.
2023-10-06 11:54:15 +02:00
Dawid Medrek
4b03cba1bf db/hints/manager: Move the defintions of static members to the header
If the variables are accessible from the outside, it makes
sense to also expose their initial values to the user.
This commit moves them to the header and marks as inline.
2023-10-06 11:54:15 +02:00
Dawid Medrek
c3ab28f5e9 db/hints: Move make_dummy() to the header
The function is trivial. It can also be marked as noexcept.
2023-10-06 11:54:15 +02:00
Dawid Medrek
5e333f0a52 db/hints: Don't explicitly define ~directory_initializer()
The destructor is the default destructor, and it is safe
to drop it altogether.
2023-10-06 11:53:02 +02:00
Dawid Medrek
9f215d3cf1 db/hints: Change the order of logging in ensure_created_and_verified()
The new logging order seems to make more sense, i.e.
we first log that we're creating and validating directories,
and only then do we start doing that.
The previous order when those actions were reversed didn't
match the log's message because the action was already
done when we informed the user of it.
2023-10-06 11:14:41 +02:00
Dawid Medrek
4ad3e8d37b db/hints: Coroutinize ensure_rebalanced() 2023-10-06 11:14:41 +02:00
Dawid Medrek
672cdb5c05 db/hints: Coroutinize ensure_created_and_verified() 2023-10-06 11:14:41 +02:00
Dawid Medrek
a5f14cb130 db/hints: Improve formatting of directory_initializer::impl
The implementation class has been divided into clear sections.
The indentation has also been adjusted to what is commonly
used in the codebase.
2023-10-06 11:14:41 +02:00
Dawid Medrek
500175d738 db/hints: Do not rely on the values of enums
These changes move away from relying on specific
values of enum variants. The code based on the arithmetic
of them is trivial, and there is no reason to not operator==
and operator!= instead. This should make the code less error
prone and easier to understand.
2023-10-06 11:14:41 +02:00
Dawid Medrek
d0b4d9f14f db/hints: Move the implementation of directory_initializer
This commit moves said code to the top of manager.cc
to match its position in the header file. That should
make navigation easier.
2023-10-06 11:14:41 +02:00
Dawid Medrek
b516fe1fc0 db/hints: Prefer nested namespaces
This reduces the amount of boilerplate.
2023-10-06 11:14:41 +02:00
Dawid Medrek
fc80c57bec db/hints: Reorder includes in manager.hh and .cc
These changes improve the readability of the included headers.
2023-10-06 11:14:41 +02:00
Dawid Medrek
aba85c9c98 db/hints: Rename rebalance to rebalance_hints
The new name conveys the idea clearly.
2023-09-27 18:49:08 +02:00
Dawid Medrek
d46437a87b db/hints: Rename end_point_hints_manager
This commit renames `end_point_hints_manager` to `hint_endpoint_manager`
to be consistent with other names used in the module (they all start
with `hint_`).
2023-09-15 03:46:15 +02:00
Dawid Medrek
4ad0f8907c db/hints: Move the rebalancing logic to hint_storage
This commit continues modularizing manager.hh.
2023-09-15 03:46:15 +02:00
Dawid Medrek
507054012d db/hints: Move the implementation of end_point_hints_manager
This commit continues moving end_point_hints_manager to its
dedicated files. After moving the declaration of the class,
these changes move the implementation.
2023-09-15 03:46:15 +02:00
Dawid Medrek
f72c423984 db/hints: Move the declaration of end_point_hints_manager
This commit is yet another step in modularizing manager.hh.
We move the declaration of the class to a dedicated header file.
The implementation will follow in a future commit.
2023-09-15 03:46:15 +02:00
Dawid Medrek
854cc0c939 db/hints: Move definitions of functions using shard hint manager
We move definitions of inline methods of end_point_hints_manager
and sender accessing shard hint manager to the source file,
effectively un-inlining them. We need to do that to prepare for
moving said structures out of manager.hh. This commit is yet
another step in modularizing manager.hh.
2023-09-15 03:45:57 +02:00
Dawid Medrek
4814b3b19a db/hints: Extract the logger from manager.cc
This commit extracts the logger used in manager.cc
to prepare the ground for modularization of manager.hh
into separate smaller files. We want to preserve
the logging behavior (at least for the time being),
which means new files should use the same logger.
These changes serve that purpose.
2023-09-15 02:24:20 +02:00
Dawid Medrek
efd6d1f57a db/hints: Extract common types from manager.hh
Currently, data structures used in manager.hh
use their own aliases for gms::inet_address.
It is clear they all should use the same type
and having different names for it only reduces
readability of the code. This commit introduces
a common alias -- endpoint_id -- and gets rid
of the other ones.

This commit is also the first step in modularizing
manager.hh by extracting common types to another
file.
2023-09-15 02:23:30 +02:00
Dawid Medrek
c7fe5d7f94 utils/lister: Limit the API of scan_dir() to fs::path
Right now, the function allows for passing the path to a file as a seastar::sstring,
which is then converted to std::filesystem::path -- implicitly to the caller.
However, the function performs I/O, and there is no reason to accept any other type
than std::filesystem::path, especially because the conversion is straightforward.
Callers can perform it on their own.

This commit introduces the more constrained API.

Closes #15266
2023-09-05 20:50:42 +03:00
Benny Halevy
2c54d7a35a view, storage_proxy: carry effective_replication_map along with endpoints
When sending mutation to remote endpoint,
the selected endpoints must be in sync with
the current effective_replication_map.

Currently, the endpoints are sent down the storage_proxy
stack, and later on an effective_replication_map is retrieved
again, and it might not match the target or pending endpoints,
similar to the case seen in https://github.com/scylladb/scylladb/issues/15138

The correct way is to carry the same effective replication map
used to select said endpoints and pass it down the stack.
See also https://github.com/scylladb/scylladb/pull/15141

Fixes scylladb/scylladb#15144
Fixes scylladb/scylladb#14730

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>

Closes #15142
2023-08-29 09:08:42 +03:00