Patryk Jędrzejczak a2c23793ab raft_group0: join_group0: fix join hang when node joins group 0 before post_server_start ...

A joining node hung forever if the topology coordinator added it to the
group 0 configuration before the node reached `post_server_start`. In
that case, `server->get_configuration().contains(my_id)` returned true
and the node broke out of the join loop early, skipping
`post_server_start`. `_join_node_group0_started` was therefore never set,
so the node's `join_node_response` RPC handler blocked indefinitely.
Meanwhile the topology coordinator's `respond_to_joining_node` call
(which has no timeout) hung forever waiting for the reply that never came.

Fix by only taking the early-break path when not starting as a follower
(i.e. when the node is the discovery leader or is restarting). A joining
node must always reach `post_server_start`.

We also provide a regression test. It takes 6s in dev mode.

Fixes SCYLLADB-959

Closes scylladb/scylladb#29266

(cherry picked from commit b9f82f6f23)

Closes scylladb/scylladb#29291

Closes scylladb/scylladb#29308

2026-04-09 15:53:43 +02:00

..

auth_cluster

Merge 'service_levels: mark v2 migration complete on empty legacy table' from Alex Dathskovsky

2026-04-06 17:51:34 +03:00

dtest

Merge '[Backport 2025.4] Make commitlog replay handle files with corrupt file header (non-zero) as data loss, not startup failure' from Scylladb[bot]

2026-01-19 06:39:29 +02:00

lwt

tests(lwt): new test for LWT testing during tablet resize

2025-11-04 12:47:24 +01:00

mv

db/view/view_update_generator: move discover_staging_sstables to start

2026-01-12 10:33:11 +01:00

object_store

streaming: fix loop break condition in tablet_sstable_streamer::stream

2025-11-25 11:42:34 +02:00

random_failures

db/view: Require rf_rack_valid_keyspaces when creating view

2025-10-06 13:19:54 +00:00

tasks

test: Fix drain api in task_manager_client.py

2025-08-11 10:10:07 +08:00

__init__.py

…

conftest.py

pylib: extract upgrade helpers from test_sstable_compression_dictionaries_upgrade.py

2025-09-15 12:34:45 +02:00

suite.yaml

test: dtest: limits_test.py: make the tests work

2025-10-01 22:40:29 +02:00

test_aggregation.py

…

test_alternator.py

Merge '[Backport 2025.4] alternator: fix batch writes during intranode tablet migrations' from Scylladb[bot]

2026-01-08 16:40:20 +02:00

test_automatic_cleanup.py

…

test_bad_initial_token.py

test: cluster: add test_bad_initial_token

2025-04-25 12:25:15 +02:00

test_batchlog_manager.py

test: extend test_batchlog_replay_failure_during_repair

2025-12-22 14:45:08 +01:00

test_blocked_bootstrap.py

…

test_boot_after_ip_change.py

…

test_boot_nodes.py

test: Add test_boot_nodes.py

2025-07-10 10:56:53 +08:00

test_bootstrap_with_quick_group0_join.py

raft_group0: join_group0: fix join hang when node joins group 0 before post_server_start

2026-04-09 15:53:43 +02:00

test_bti_index.py

test/cluster/test_bti_index.py: avoid a race with CQL tracing

2025-10-20 10:32:58 +03:00

test_cdc_generation_clearing.py

test_cdc_generation_clearing: wait for generations to propagate

2025-06-09 12:59:04 +02:00

test_cdc_generation_data.py

raft_group0: split shutdown into abort_and_drain and destroy

2025-07-25 17:16:14 +02:00

test_cdc_generation_publishing.py

test_cdc_generation_publishing: fix to read monotonically

2025-05-30 08:35:56 +02:00

test_cdc_with_alter.py

test: test concurrent writes with column drop with cdc preimage

2025-11-16 09:29:27 +01:00

test_cdc_with_tablets.py

test: cdc: extend cdc with tablets tests

2025-10-30 02:44:47 +00:00

test_change_ip.py

test/cluster: Adjust simple tests to RF-rack-validity

2025-05-10 16:30:18 +02:00

test_change_replication_factor_1_to_0.py

test: cluster: deflake consistency checks after decommission

2025-09-09 19:01:12 +02:00

test_change_rpc_address.py

test/cluster: Adjust simple tests to RF-rack-validity

2025-05-10 16:30:18 +02:00

test_cluster_features.py

…

test_commitlog_segment_data_resurrection.py

…

test_commitlog.py

…

test_compaction_backpressure.py

compaction_manager: fix maybe_wait_for_sstable_count_reduction() hanging forever

2026-02-27 01:39:05 +02:00

test_concurrent_schema.py

…

test_config_live_updates.py

test: add test for live updates of generic server config

2025-06-23 17:56:26 +02:00

test_config.py

…

test_conflicting_keys_read_repair.py

test/cluster: Adjust simple tests to RF-rack-validity

2025-05-10 16:30:18 +02:00

test_coordinator_queue_management.py

test.py: rework log_browsing for dtest migration

2025-05-19 11:50:55 +00:00

test_crash_coordinator_before_streaming.py

…

test_create_table_during_node_shutdown.py

migration_listener: fix deadlock in nested notifications

2026-02-18 12:47:30 +02:00

test_data_resurrection_after_cleanup.py

test: cluster: deflake consistency checks after decommission

2025-09-09 19:01:12 +02:00

test_data_resurrection_in_memtable.py

test/cluster: Adjust simple tests to RF-rack-validity

2025-05-10 16:30:18 +02:00

test_decommission_kill_then_replace.py

topology coordinator: complete pending operation for a replaced node

2026-01-19 09:42:20 +02:00

test_decommission.py

test: cluster: deflake consistency checks after decommission

2025-09-09 19:01:12 +02:00

test_deprecating_cluster_features.py

…

test_describe.py

cql3: Represent create_statement using managed_string

2025-07-01 12:58:02 +02:00

test_different_group0_ids.py

test.py: rewrite the wait_for_first_completed

2025-10-22 18:12:52 +02:00

test_encryption.py

…

test_error_becoming_voter.py

…

test_fencing.py

test_fencing: add test_lwt_fencing_upgrade

2025-09-15 12:34:45 +02:00

test_global_ignore_nodes.py

test/cluster: Adjust simple tests to RF-rack-validity

2025-05-10 16:30:18 +02:00

test_gossip_boot.py

…

test_gossiper_empty_self_id_on_shadow_round.py

gossiper: fix empty initial local node state

2025-09-08 11:38:31 +02:00

test_gossiper_orphan_remover.py

test: test_gossiper_orphan_remover: get host ID of the bootstrapping node before it crashes

2026-01-29 11:27:37 +01:00

test_gossiper_race.py

gossiper: check for a race condition in do_apply_state_locally

2025-09-08 11:38:30 +02:00

test_gossiper.py

…

test_group0_schema_versioning.py

test: test_group0_schema_versioning: wait for schema sync in system.local

2026-01-19 16:35:49 +01:00

test_hints.py

test: use exclusive driver connection in test_limited_concurrency_of_writes

2026-04-07 14:22:48 +03:00

test_incremental_repair.py

repair: Fix sstable_list_to_mark_as_repaired with multishard writer

2026-01-19 09:39:49 +02:00

test_initial_token.py

test/cluster/conftest: cluster_con: provide default values for port and use_ssl

2025-08-22 09:51:24 +03:00

test_ip_mappings.py

test: test_full_shutdown_during_replace: retry replace after the replacing node is removed from gossip

2026-03-10 16:48:05 +01:00

test_keyspace_rf.py

test/cqlpy: add keyspace creation default replication factor tests

2025-08-28 01:42:34 +02:00

test_left_node_notification.py

raft topology: Notify that a node was removed only once

2025-12-30 11:17:41 +01:00

test_long_join.py

test: improve async execution in test_long_join

2025-09-08 17:14:37 +02:00

test_long_query_timeout_erm.py

test.py: rewrite the wait_for_first_completed

2025-10-22 18:12:52 +02:00

test_lwt_semaphore.py

…

test_maintenance_mode.py

test: test_maintenance_mode: enable maintenance mode properly

2026-02-03 11:33:53 +01:00

test_major_compaction.py

compaction: fix use after free when strategy is altered during compaction

2025-10-21 00:59:33 +00:00

test_metadata_id.py

test/cluster/conftest: cluster_con: provide default values for port and use_ssl

2025-08-22 09:51:24 +03:00

test_multidc.py

Merge 'cql3: Warn when creating RF-rack-invalid keyspace' from Dawid Mędrek

2025-08-22 11:33:32 +02:00

test_mutation_schema_change.py

…

test_mv.py

tombstone_gc: don't use 'repair' mode for colocated tables

2025-11-26 08:36:52 +01:00

test_no_dc_rack_change.py

test: cluster: introduce test_no_dc_rack_change

2025-04-17 16:22:58 +02:00

test_no_removed_node_event_on_ip_change.py

test/cluster/conftest: cluster_con: provide default values for port and use_ssl

2025-08-22 09:51:24 +03:00

test_node_isolation.py

tiering (test.py): introduce tiering labels

2025-08-04 15:38:16 +03:00

test_node_ops_metrics.py

test/pylib/rest_client: fix ScyllaMetrics filtering

2025-08-10 10:16:00 +02:00

test_node_shutdown_waits_for_pending_requests.py

…

test_nodetool.py

…

test_not_enough_token_owners.py

test: cluster: Fix NoHostAvailable error in test_not_enough_token_owners

2026-01-09 19:10:11 +01:00

test_query_rebounce.py

…

test_raft_cluster_features.py

…

test_raft_fix_broken_snapshot.py

…

test_raft_ignore_nodes.py

…

test_raft_no_quorum.py

test: test_raft_no_quorum: decrease group0_raft_op_timeout_in_ms after quorum loss

2026-03-18 10:15:34 +01:00

test_raft_recovery_basic.py

…

test_raft_recovery_during_join.py

test: test_raft_recovery_during_join: get host ID of the bootstrapping node before it crashes

2026-01-22 18:19:36 +01:00

test_raft_recovery_entry_loss.py

test: test_raft_recovery_entry_loss: fix the typo in the test case name

2025-10-17 10:27:33 +00:00

test_raft_recovery_majority_loss.py

…

test_raft_recovery_stuck.py

test: test_raft_recovery_stuck: ensure mutual visibility before using driver

2025-11-20 10:36:54 +02:00

test_raft_recovery_user_data.py

test: deflake driver reconnections in the recovery procedure tests

2025-09-22 17:21:06 +02:00

test_raft_snapshot_request.py

…

test_raft_snapshot_truncation.py

…

test_raft_voters.py

test/cluster/conftest: cluster_con: provide default values for port and use_ssl

2025-08-22 09:51:24 +03:00

test_random_tables.py

…

test_read_repair.py

test/cluster/test_read_repair: write 100 rows in trace test

2025-06-27 16:23:08 +03:00

test_refresh.py

Add nodetool refresh --scope option

2025-05-29 16:12:09 +03:00

test_remove_alive_node.py

…

test_remove_rpc_client_with_pending_requests.py

test/cluster: Adjust simple tests to RF-rack-validity

2025-05-10 16:30:18 +02:00

test_repair.py

repair: Allow min max range to be updated for repair history

2026-01-08 16:39:10 +02:00

test_replace_alive_node.py

…

test_replace_ignore_nodes.py

…

test_replace_with_encryption.py

…

test_replace_with_same_ip_twice.py

…

test_replace.py

test.py: rework log_browsing for dtest migration

2025-05-19 11:50:55 +00:00

test_restart_cluster.py

…

test_resurrection.py

…

test_reversed_queries_during_simulated_upgrade_process.py

test/cluster: Adjust simple tests to RF-rack-validity

2025-05-10 16:30:18 +02:00

test_rpc_compression.py

test/cluster: Adjust simple tests to RF-rack-validity

2025-05-10 16:30:18 +02:00

test_select_from_mutation_fragments.py

test/cluster: Adjust simple tests to RF-rack-validity

2025-05-10 16:30:18 +02:00

test_shutdown_hang.py

…

test_snapshot.py

test: add type creation to test_snapshot

2025-07-10 10:46:55 +02:00

test_sstable_cleanup_stop.py

test/cluster: fix flaky test_cleanup_stop by using asyncio.sleep

2026-04-07 14:22:27 +03:00

test_sstable_compression_config.py

schema: Add initializer for compression defaults

2026-01-28 12:42:10 +02:00

test_sstable_compression_dictionaries_autotrain.py

test: fix configuration of test_autoretrain_dict

2026-02-26 09:55:57 +02:00

test_sstable_compression_dictionaries_basic.py

db/config: Deprecate sstable_compression_dictionaries_allow_in_ddl

2025-11-04 15:40:46 +02:00

test_sstable_compression_dictionaries_upgrade.py

pylib: extract upgrade helpers from test_sstable_compression_dictionaries_upgrade.py

2025-09-15 12:34:45 +02:00

test_sstable_set.py

test: Verify partitioned set store split and unsplit correctly

2025-04-29 15:47:33 -03:00

test_start_bootstrapped_with_invalid_seed.py

test: disable test_start_bootstrapped_with_invalid_seed

2026-01-15 17:01:30 +02:00

test_streaming_deadlock.py

test: limit test_streaming_deadlock_removenode concurrency

2025-09-19 12:50:20 +03:00

test_table_desc_read_barrier.py

…

test_table_drop.py

test: test table drop during flush

2025-04-23 14:29:28 +02:00

test_tablet_repair_scheduler.py

repair: Add tablet repair progress report support

2026-01-19 09:39:13 +02:00

test_tablet_stats.py

topology_coordinator: Make tablet_load_stats_refresh_interval configurable

2025-07-31 14:31:55 +03:00

test_tablets2.py

test: add test and reproducer for load_stats refresh exception

2026-02-02 16:35:08 +01:00

test_tablets_colocation.py

test: fix test flakiness in test_colocated_tables_gc_mode

2025-12-19 17:32:12 +01:00

test_tablets_cql.py

test/cluster: Adjust simple tests to RF-rack-validity

2025-05-10 16:30:18 +02:00

test_tablets_intranode.py

…

test_tablets_lwt.py

test_lwt_shutdown: fix flakiness by removing storage_proxy::stop injection

2026-01-23 19:24:06 +00:00

test_tablets_merge.py

test_tablets_merge: test_tablet_split_merge_with_many_tables: reduce number of tables in debug mode

2025-09-29 15:30:13 +03:00

test_tablets_migration.py

test: test_restart_leaving_replica_during_cleanup: reconnect driver after restart

2026-02-18 12:43:31 +02:00

test_tablets_removenode.py

test/cluster: Disable rf_rack_valid_keyspaces in problematic tests

2025-05-10 16:30:49 +02:00

test_tablets.py

test: Verify that repair doesn't block disabling of tablet load balancing

2026-02-02 21:26:18 +01:00

test_tls.py

test/cluster: Adjust simple tests to RF-rack-validity

2025-05-10 16:30:18 +02:00

test_tombstone_gc.py

test: test group0 tombstone GC in the Raft-based recovery procedure

2025-10-22 17:13:34 +00:00

test_topology_failure_recovery.py

…

test_topology_ops_encrypted.py

test: cluster: deflake consistency checks after decommission

2025-09-09 19:01:12 +02:00

test_topology_ops.py

test: cluster: deflake consistency checks after decommission

2025-09-09 19:01:12 +02:00

test_topology_recovery_basic.py

test.py: apply the nightly label on test_topology_recovery_basic

2025-09-01 14:16:29 +02:00

test_topology_recovery_majority_loss.py

test/cluster/conftest: cluster_con: provide default values for port and use_ssl

2025-08-22 09:51:24 +03:00

test_topology_rejoin.py

…

test_topology_remove_decom.py

raft_topology: Modify the conditional logic in remove node operation to enhance concurrency for raft enabled clusters.

2025-09-17 15:23:32 +05:30

test_topology_remove_garbage_group0.py

test: test_remove_garbage_group0_members: wait for token ring and group0 consistency before removenode

2026-03-24 16:09:02 +01:00

test_topology_schema.py

…

test_topology_smp.py

test/cluster: Adjust simple tests to RF-rack-validity

2025-05-10 16:30:18 +02:00

test_topology_upgrade_not_stuck_after_recent_removal.py

…

test_topology_upgrade_stuck.py

test.py: rewrite the wait_for_first_completed

2025-10-22 18:12:52 +02:00

test_topology_upgrade.py

…

test_truncate_concurrent_writes.py

truncate: add test for truncate with concurrent writes

2025-08-05 13:54:14 +02:00

test_truncate_with_drop.py

system_keyspace: Prune dropped tables from truncation on start/drop

2025-09-03 07:25:34 +03:00

test_truncate_with_tablets.py

topology coordinator: allow running multiple global commands in parallel

2025-06-11 11:29:33 +03:00

test_unfinished_writes_during_shutdown.py

storage_service: Cancel all write requests on storage_proxy shutdown

2025-07-22 15:03:30 +02:00

test_vector_store.py

index: allow vector indexes without rf_rack_valid_keyspces

2025-12-05 20:13:02 +01:00

test_view_build_status.py

test/cluster: add view build status tests

2025-08-27 10:23:04 +02:00

test_view_building_coordinator.py

test/cluster/test_view_building_coordinator: fix flakiness in test_file_streaming

2026-01-08 16:42:08 +02:00

test_write_query_during_cql_server_shutdown.py

generic_server: Two-step connection shutdown.

2025-07-28 10:08:06 +02:00

test_writes_to_previous_cdc_generations.py

…

test_zero_token_nodes_multidc.py

test: test_zero_token_nodes_multidc: properly handle reads with CL=LOCAL_ONE

2026-01-22 18:22:05 +01:00

test_zero_token_nodes_no_replication.py

test/cluster/conftest: cluster_con: provide default values for port and use_ssl

2025-08-22 09:51:24 +03:00

test_zero_token_nodes_topology_ops.py

test/cluster/test_zero_token_nodes_topology_ops: Adjust to RF-rack-validity

2025-05-10 16:30:34 +02:00

util.py

Revert "Merge 'transport: service_level_controller: create and use driver service level' from Andrzej Jackowski"

2025-09-22 09:32:46 +03:00