mirror of
https://github.com/scylladb/scylladb.git
synced 2026-04-23 10:00:35 +00:00
9c4e32d2e2
When `io_fiber` fetched a batch with a configuration that does not contain this node, it would send the entries committed in this batch to `applier_fiber` and proceed by any remaining entry dropping waiters (if the node was no longer a leader). If there were waiters for entries committed in this batch, it could either happen that `applier_fiber` received and processed those entries first, notifying the waiters that the entries were committed and/or applied, or it could happen that `io_fiber` reaches the dropping waiters code first, causing the waiters to be resolved with `commit_status_unknown`. The second scenario is undesirable. For example, when a follower tries to remove the current leader from the configuration using `modify_config`, if the second scenario happens, the follower will get `commit_status_unknown` - this can happen even though there are no node or network failures. In particular, this caused `randomized_nemesis_test.remove_leader_with_forwarding_finishes` to fail from time to time. Fix it by serializing the notifying and dropping of waiters in a single fiber - `applier_fiber`. We decided to move all management of waiters into `applier_fiber`, because most of that management was already there (there was already one `drop_waiters` call, and two `notify_waiters` calls). Now, when `io_fiber` observes that we've been removed from the config and no longer a leader, instead of dropping waiters, it sends a message to `applier_fiber`. `applier_fiber` will drop waiters when receiving that message. Improve an existing test to reproduce this scenario more frequently. Fixes #11235. Closes #11308 * github.com:scylladb/scylladb: test: raft: randomized_nemesis_test: more chaos in `remove_leader_with_forwarding_finishes` raft: server: drop waiters in `applier_fiber` instead of `io_fiber` raft: server: use `visit` instead of `holds_alternative`+`get`