Pavel Emelyanov
632a65bffa
Merge 'repair: row_level: coroutinize more functions' from Avi Kivity
...
Coroutinize more functions in row-level repair to improve maintainability.
The functions all deal with repair buffers, so coroutinization does not affect performance.
Cleanup, no reason to backport
Closes scylladb/scylladb#20464
* github.com:scylladb/scylladb:
repair: row_level: restore indentation
repair: row_level: coroutinize repair_service::insert_repair_meta()
repair: row_level: coroutinize repair_meta::get_full_row_hashes()
repair: row_level: coroutinize repair_meta::apply_rows_on_follower()
repair: row_level: coroutinize repair_meta::clear_working_row_buf()
repair: row_level: coroutinize get_common_diff_detect_algorithm()
repair: row_level: coroutinize repair_service::remove_repair_meta() (non-selective overload)
repair: row_level: coroutinize repair_service::remove_repair_meta() (by-address overload)
repair: row_level: coroutinize repair_service::remove_repair_meta() (by-id overload)
repair: row_level: row_level_repair::run()
repair: row_level: row_level_repair::send_missing_rows_to_follower_nodes()
repair: row_level: row_level_repair::get_missing_rows_from_follower_nodes()
repair: row_level: row_level_repair::negotiate_sync_boundary()
repair: row_level: coroutinize repair_put_row_diff_with_rpc_stream_process_op()
repair: row_level: coroutinize repair_meta::get_sync_boundary_handler()
repair: row_level: coroutinize repair_meta::get_sync_boundary()
repair: row_level: coroutinize repair_meta::repair_set_estimated_partitions_handler()
repair: row_level: coroutinize repair_meta::repair_set_estimated_partitions()
repair: row_level: coroutinize repair_meta::repair_get_estimated_partitions_handler()
repair: row_level: coroutinize repair_meta::repair_get_estimated_partitions()
repair: row_level: coroutinize repair_meta::repair_row_level_stop_handler()
repair: row_level: coroutinize repair_meta::repair_row_level_stop()
repair: row_level: coroutinize repair_meta::repair_row_level_start_handler()
repair: row_level: coroutinize repair_meta::repair_row_level_start()
repair: row_level: coroutinize repair_meta::get_combined_row_hash_handler()
repair: row_level: coroutinize repair_meta::get_combined_row_hash()
repair: row_level: coroutinize repair_meta::get_full_row_hashes_handler()
repair: row_level: coroutinize repair_meta::get_full_row_hashes_with_rpc_stream()
repair: row_level: coroutinize repair_meta::request_row_hashes()
2024-09-12 15:35:57 +03:00
Kefu Chai
3e84d43f93
treewide: use seastar::format() or fmt::format() explicitly
...
before this change, we rely on `using namespace seastar` to use
`seastar::format()` without qualifying the `format()` with its
namespace. this works fine until we changed the parameter type
of format string `seastar::format()` from `const char*` to
`fmt::format_string<...>`. this change practically invited
`seastar::format()` to the club of `std::format()` and `fmt::format()`,
where all members accept a templated parameter as its `fmt`
parameter. and `seastar::format()` is not the best candidate anymore.
despite that argument-dependent lookup (ADT for short) favors the
function which is in the same namespace as its parameter, but
`using namespace` makes `seastar::format()` more competitive,
so both `std::format()` and `seastar::format()` are considered
as the condidates.
that is what is happening scylladb in quite a few caller sites of
`format()`, hence ADT is not able to tell which function the winner
in the name lookup:
```
/__w/scylladb/scylladb/mutation/mutation_fragment_stream_validator.cc:265:12: error: call to 'format' is ambiguous
265 | return format("{} ({}.{} {})", _name_view, s.ks_name(), s.cf_name(), s.id());
| ^~~~~~
/usr/bin/../lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/format:4290:5: note: candidate function [with _Args = <const std::basic_string_view<char> &, const seastar::basic_sstring<char, unsigned int, 15> &, const seastar::basic_sstring<char, unsigned int, 15> &, const utils::tagged_uuid<table_id_tag> &>]
4290 | format(format_string<_Args...> __fmt, _Args&&... __args)
| ^
/__w/scylladb/scylladb/seastar/include/seastar/core/print.hh:143:1: note: candidate function [with A = <const std::basic_string_view<char> &, const seastar::basic_sstring<char, unsigned int, 15> &, const seastar::basic_sstring<char, unsigned int, 15> &, const utils::tagged_uuid<table_id_tag> &>]
143 | format(fmt::format_string<A...> fmt, A&&... a) {
| ^
```
in this change, we
change all `format()` to either `fmt::format()` or `seastar::format()`
with following rules:
- if the caller expects an `sstring` or `std::string_view`, change to
`seastar::format()`
- if the caller expects an `std::string`, change to `fmt::format()`.
because, `sstring::operator std::basic_string` would incur a deep
copy.
we will need another change to enable scylladb to compile with the
latest seastar. namely, to pass the format string as a templated
parameter down to helper functions which format their parameters.
to miminize the scope of this change, let's include that change when
bumping up the seastar submodule. as that change will depend on
the seastar change.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com >
2024-09-11 23:21:40 +03:00
Avi Kivity
384a09585b
repair: row_level: repair_get_row_diff_with_rpc_stream_process_op: simplify return value
...
During review of 0857b63259Closes scylladb/scylladb#20441
2024-09-06 11:39:21 +03:00
Avi Kivity
c57b8dd0bf
repair: row_level: restore indentation
2024-09-05 18:38:43 +03:00
Avi Kivity
710977ef88
repair: row_level: coroutinize repair_service::insert_repair_meta()
...
Some of the indentation was broken, and is partially repaired by this
change.
2024-09-05 17:59:42 +03:00
Avi Kivity
f23a32ed84
repair: row_level: coroutinize repair_meta::get_full_row_hashes()
2024-09-05 17:56:27 +03:00
Avi Kivity
607747beb1
repair: row_level: coroutinize repair_meta::apply_rows_on_follower()
2024-09-05 17:55:07 +03:00
Avi Kivity
89d4394d12
repair: row_level: coroutinize repair_meta::clear_working_row_buf()
2024-09-05 17:52:32 +03:00
Avi Kivity
4cfc25f8d7
repair: row_level: coroutinize get_common_diff_detect_algorithm()
...
The function is threaded, but the inner lambda can be coroutinized.
2024-09-05 17:47:27 +03:00
Avi Kivity
20b79816f1
repair: row_level: coroutinize repair_service::remove_repair_meta() (non-selective overload)
2024-09-04 18:43:19 +03:00
Avi Kivity
3b9ac51b6b
repair: row_level: coroutinize repair_service::remove_repair_meta() (by-address overload)
2024-09-04 18:39:21 +03:00
Avi Kivity
704e3f5432
repair: row_level: coroutinize repair_service::remove_repair_meta() (by-id overload)
2024-09-04 18:37:48 +03:00
Avi Kivity
9612c4d790
repair: row_level: row_level_repair::run()
...
The function itself is threaded, but the inner lambdas are coroutinized
(except one which is expected to run in a thread, and so is threaded).
2024-09-04 18:34:45 +03:00
Avi Kivity
2b94ee981b
repair: row_level: row_level_repair::send_missing_rows_to_follower_nodes()
...
The function itself is threaded, but the inner lambda is coroutinized.
2024-09-04 18:28:27 +03:00
Avi Kivity
c768448339
repair: row_level: row_level_repair::get_missing_rows_from_follower_nodes()
...
The function itself is threaded, but the inner lambda is coroutinized.
2024-09-04 18:28:12 +03:00
Avi Kivity
d2f1b44487
repair: row_level: row_level_repair::negotiate_sync_boundary()
...
The function itself is threaded, but the inner lambda is coroutinized.
2024-09-04 18:21:39 +03:00
Avi Kivity
645e39e746
repair: row_level: coroutinize repair_put_row_diff_with_rpc_stream_process_op()
...
Both the outer function and the inner lambda are coroutinized.
2024-09-04 18:10:43 +03:00
Avi Kivity
4c05d0b965
repair: row_level: coroutinize repair_meta::get_sync_boundary_handler()
2024-09-04 15:33:40 +03:00
Avi Kivity
eea011fad5
repair: row_level: coroutinize repair_meta::get_sync_boundary()
...
Not really helping anything, but a coroutine is a safer platform for
future changes in administrative APIs.
2024-09-04 15:31:57 +03:00
Avi Kivity
91b88df956
repair: row_level: coroutinize repair_meta::repair_set_estimated_partitions_handler()
2024-09-04 15:20:53 +03:00
Avi Kivity
b73194c9bf
repair: row_level: coroutinize repair_meta::repair_set_estimated_partitions()
...
Not really helping anything, but a coroutine is a safer platform for
future changes in administrative APIs.
2024-09-04 15:18:33 +03:00
Avi Kivity
a69fb626bd
repair: row_level: coroutinize repair_meta::repair_get_estimated_partitions_handler()
2024-09-04 15:17:42 +03:00
Avi Kivity
5cd8207ac7
repair: row_level: coroutinize repair_meta::repair_get_estimated_partitions()
...
Not really helping anything, but a coroutine is a safer platform for
future changes in administrative APIs.
2024-09-04 15:16:32 +03:00
Avi Kivity
e108f867a9
repair: row_level: coroutinize repair_meta::repair_row_level_stop_handler()
2024-09-04 15:15:42 +03:00
Avi Kivity
ffbb973063
repair: row_level: coroutinize repair_meta::repair_row_level_stop()
...
Not really helping anything, but a coroutine is a safer platform for
future changes in administrative APIs.
2024-09-04 15:14:08 +03:00
Avi Kivity
587b6fe400
repair: row_level: coroutinize repair_meta::repair_row_level_start_handler()
2024-09-04 15:12:49 +03:00
Avi Kivity
db7b1014ff
repair: row_level: coroutinize repair_meta::repair_row_level_start()
2024-09-04 15:10:45 +03:00
Avi Kivity
17b82265ae
repair: row_level: coroutinize repair_meta::get_combined_row_hash_handler()
2024-09-04 15:08:58 +03:00
Avi Kivity
bacbdde791
repair: row_level: coroutinize repair_meta::get_combined_row_hash()
2024-09-04 15:07:27 +03:00
Avi Kivity
8b8dc5092f
repair: row_level: coroutinize repair_meta::get_full_row_hashes_handler()
2024-09-04 15:05:28 +03:00
Avi Kivity
21e01990ff
repair: row_level: coroutinize repair_meta::get_full_row_hashes_with_rpc_stream()
...
The when_all_succeed() call is changed to the safer coroutine::when_all(),
which avoids the temporary futures.
2024-09-04 15:03:00 +03:00
Avi Kivity
572fbfde09
repair: row_level: coroutinize repair_meta::request_row_hashes()
2024-09-04 14:07:59 +03:00
Pavel Emelyanov
0857b63259
Merge 'repair: row_level: coroutinize some slow-path functions' from Avi Kivity
...
This series coroutinizes up some functions in repair/row_level.cc. This enhances
readability and reduces bloat:
```
size build/release/repair/row_level.o.{before,after}
text data bss dec hex filename
1650619 48 524 1651191 1931f7 build/release/repair/row_level.o.before
1604610 48 524 1605182 187e3e build/release/repair/row_level.o.after
```
46kB of text were saved.
Functions that only touch a single mutation fragment were not coroutinized to avoid
adding a allocation in a fast path. In one case a function was split into a fast path and a
slow path.
Clean-up series, backport not needed.
Closes scylladb/scylladb#20283
* github.com:scylladb/scylladb:
repair: row_level: restore indentation
repair: row_level: coroutinize repair_meta::get_full_row_hashes_sink_op()
repair: row_level: coroutinize repair_meta::get_full_row_hashes_source_op()
repair: row_level: coroutinize repair_get_full_row_hashes_with_rpc_stream_handler()
repair: row_level: coroutinize repair_put_row_diff_with_rpc_stream_handler()
repair: row_level: coroutinize repair_get_row_diff_with_rpc_stream_handler()
repair: row_level: coroutinize repair_get_full_row_hashes_with_rpc_stream_process()
repair: row_level: coroutinize repair_get_row_diff_with_rpc_stream_process_op_slow_path()
repair: row_level: split repair_get_row_diff_with_rpc_stream_process_op() into fast and slow paths
repair: row_level: coroutinize repair_meta::put_row_diff_handler()
repair: row_level: coroutinize repair_meta::put_row_diff_sink_op()
repair: row_level: coroutinize repair_meta::put_row_diff_source_op()
repair: row_level: coroutinize repair_meta::put_row_diff()
repair: row_level: coroutinize repair_meta::get_row_diff_handler()
repair: row_level: coroutinize repair_meta::get_row_diff_sink_op()
repair: row_level: coroutinize repair_meta::to_repair_rows_on_wire()
repair: row_level: coroutinize repair_meta::do_apply_rows()
repair: row_level: coroutinize repair_meta::copy_rows_from_working_row_buf_within_set_diff()
repair: row_level: coroutinize repair_meta::copy_rows_from_working_row_buf()
repair: row_level: coroutinize repair_meta::row_buf_csum()
repair: row_level: coroutinize repair_meta::get_repairs_row_size()
repair: row_level: coroutinize repair_meta::set_estimated_partitions()
repair: row_level: coroutinize repair_meta::get_estimated_partitions()
repair: row_level: coroutinize repair_meta::do_estimate_partitions_on_local_shard()
repair: row_level: coroutinize repair_reader::close()
repair: row_level: coroutinize repair_reader::end_of_stream()
repair: row_level: coroutinize sink_source_for_repair::close()
repair: row_level: coroutinize sink_source_for_repair::get_sink_source()
2024-09-03 14:41:22 +03:00
Avi Kivity
61e6a77a99
repair: row_level: restore indentation
2024-08-30 23:00:59 +03:00
Avi Kivity
a35942e09a
repair: row_level: coroutinize repair_meta::get_full_row_hashes_sink_op()
...
Extra care is needed for exception handling.
2024-08-30 22:55:16 +03:00
Avi Kivity
8e9ebd82fc
repair: row_level: coroutinize repair_meta::get_full_row_hashes_source_op()
2024-08-30 22:55:16 +03:00
Avi Kivity
f7d19e237d
repair: row_level: coroutinize repair_get_full_row_hashes_with_rpc_stream_handler()
...
Both the handle_exception() and finally() blocks need some extra care.
2024-08-30 22:55:16 +03:00
Avi Kivity
bb8751f4b5
repair: row_level: coroutinize repair_put_row_diff_with_rpc_stream_handler()
...
Both the handle_exception() and finally() blocks need some extra care.
2024-08-30 22:55:16 +03:00
Avi Kivity
7ba0642da2
repair: row_level: coroutinize repair_get_row_diff_with_rpc_stream_handler()
...
Both the handle_exception() and finally() blocks need some extra care.
2024-08-30 22:55:16 +03:00
Avi Kivity
61bbf452c6
repair: row_level: coroutinize repair_get_full_row_hashes_with_rpc_stream_process()
2024-08-30 22:55:16 +03:00
Avi Kivity
01a578f608
repair: row_level: coroutinize repair_get_row_diff_with_rpc_stream_process_op_slow_path()
2024-08-30 22:55:16 +03:00
Avi Kivity
3733105f78
repair: row_level: split repair_get_row_diff_with_rpc_stream_process_op() into fast and slow paths
...
This allows coroutinization of the slow path without affecting the fast path.
2024-08-30 22:55:16 +03:00
Avi Kivity
e17c3b71a8
repair: row_level: coroutinize repair_meta::put_row_diff_handler()
2024-08-30 22:55:16 +03:00
Avi Kivity
74ea2b9663
repair: row_level: coroutinize repair_meta::put_row_diff_sink_op()
...
Exception handling is a bit awkward since can't co_await in a catch block.
2024-08-30 22:55:16 +03:00
Avi Kivity
e4362a5b7b
repair: row_level: coroutinize repair_meta::put_row_diff_source_op()
2024-08-30 22:55:16 +03:00
Avi Kivity
b998d69f09
repair: row_level: coroutinize repair_meta::put_row_diff()
2024-08-30 22:55:16 +03:00
Avi Kivity
3f2b5fe5dc
repair: row_level: coroutinize repair_meta::get_row_diff_handler()
2024-08-30 22:55:16 +03:00
Avi Kivity
cd63971501
repair: row_level: coroutinize repair_meta::get_row_diff_sink_op()
...
Since sink.close() is called from an exception handler, some code
movement is needed so it isn't co_awaited from a catch block.
2024-08-30 22:55:16 +03:00
Avi Kivity
3f28dec88c
repair: row_level: coroutinize repair_meta::to_repair_rows_on_wire()
...
coroutine::maybe_yield() introduced to compensate for loss of
stall-protected do_for_each()
2024-08-30 22:55:16 +03:00
Avi Kivity
1a84f1a73d
repair: row_level: coroutinize repair_meta::do_apply_rows()
...
coroutine::maybe_yield() introduced to compensate for loss of
stall-protected do_for_each()
2024-08-30 22:55:16 +03:00
