mirror of
https://github.com/scylladb/scylladb.git
synced 2026-05-29 19:21:01 +00:00
0e0282cdf1
Currently all management of CDC generations happens in storage_service, which is a big ball of mud that does many unrelated things. This PR introduces a new service crafted to handle CDC generation management: listening and reacting to generation changes in the cluster. We plug the service in, initializing it in main and test code, passing a reference to storage_service and having storage_service call the service (using the `after_join` method): the service only starts doing its job after the node joins the token ring (either on bootstrap or restart). Some parts of generation management still remain in storage_service: the bootstrap procedure, which happens inside storage_service, must also do some initialization regarding CDC generations, for example: on restart it must retrieve the latest known generation timestamp from disk; on bootstrap it must create a new generation and announce it to other nodes. The order of these operations w.r.t the rest of the startup procedure is important, hence the startup procedure is the only right place for them. We may try decoupling these services even more in follow-up PRs, but that requires a bit of careful reasoning. What this PR does is a low-hanging fruit. Still, what remains in storage_service is a small part of the entire CDC generation management logic; most of it has been moved to the new service. This includes listening for generation changes and updating the data structures for performing CDC log writes (cdc::metadata). Furthermore these handling functions now return futures (and are internally coroutines), where previously they required a seastar::async context. This PR is a prerequisite to fixing #7985. The fact that all the CDC generation management code was in storage_service is technical debt. It will be easier to modify the management algorithms when they sit in their own module. Tests: unit (dev) and cdc_tests.py dtest (dev), and local replication test using scylla-cdc-java Closes #8172 * github.com:scylladb/scylla: cdc: move (most of) CDC generation management code to the new service cdc: coroutinize make_new_cdc_generation cdc: coroutinize update_streams_description cdc: introduce cdc::generation_service main: move cdc_service initialization just prior to storage_service initialization