Yaniv Michael Kaul c0bf728edb raft/group0: fix destroy assertion on startup failure ...

If start_server_for_group0() successfully registers a server in
_raft_gr._servers but a subsequent step (e.g. enable_in_memory_state_machine())
throws, the server is never destroyed because abort_and_drain()/destroy()
check std::get_if<raft::group_id>(&_group0) which was only set after the
entire with_scheduling_group block completed.

Move _group0.emplace<raft::group_id>() inside the lambda, immediately after
start_server_for_group() succeeds, so that cleanup paths can always find
and destroy the registered server.

This fixes the assertion:
  "raft_group_registry - stop(): server for group ... is not destroyed"

which manifests during shutdown after an upgrade where topology_state_load()
fails due to netw::unknown_address.

Backport: Yes, to 2026.1, 2026.2, as it causes a crash on upgrades

Refs: SCYLLADB-1217
Refs: CUSTOMER-340
Refs: CUSTOMER-335
Fixes: SCYLLADB-1809
Signed-off-by: Yaniv Kaul <yaniv.kaul@scylladb.com>
AI-assisted: Yes, Opencode/Opus 4.6

Closes scylladb/scylladb#29702

(cherry picked from commit 6179406467)

Closes scylladb/scylladb#29742

Closes scylladb/scylladb#29754

2026-05-08 11:24:02 +02:00

..

auth_cluster

test: add reproducer for auth cache crash on missing permissions column

2026-05-06 13:17:06 +02:00

dtest

dtest/alternator: stop concurrent-requests test when workers hit limit

2026-04-21 19:31:22 +02:00

lwt

test: Use ManagerClient.{disable,enable}_tablet_balancing()

2026-01-13 00:38:00 +01:00

mv

test/mv/test_mv_staging: wait for cql after restart

2026-04-24 17:56:35 +03:00

object_store

reduce test_backup.py and test_refresh.py datasets

2026-01-21 10:47:36 +02:00

random_failures

test: Use ManagerClient.{disable,enable}_tablet_balancing()

2026-01-13 00:38:00 +01:00

tasks

Merge 'service: tasks: return successful status if a table was dropped' from Aleksandra Martyniuk

2026-04-06 14:25:13 +03:00

__init__.py

…

conftest.py

test.py: fix the link for the failed_test directory

2026-01-13 10:22:38 +02:00

suite.yaml

test: dtest: reproducer for parallel rebuild failure

2026-01-08 21:55:18 +02:00

test_aggregation.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_alternator_proxy_protocol.py

test: test_alternator_proxy_protocol: wait for the node to report itself as serving

2026-01-29 22:46:11 +02:00

test_alternator.py

raft_topology: log sub-step progress in local_topology_barrier

2026-05-06 15:00:44 +03:00

test_automatic_cleanup.py

replica/database: consolidate the two database_apply error injections

2026-04-24 18:31:11 +03:00

test_bad_initial_token.py

…

test_batchlog_manager.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_blocked_bootstrap.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_boot_after_ip_change.py

…

test_boot_nodes.py

…

test_bootstrap_with_quick_group0_join.py

raft_group0: join_group0: fix join hang when node joins group 0 before post_server_start

2026-04-01 09:58:20 +02:00

test_bti_index.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_cdc_generation_clearing.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_cdc_generation_data.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_cdc_generation_publishing.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_cdc_with_alter.py

test: test concurrent writes with column drop with cdc preimage

2025-11-13 17:00:08 +01:00

test_cdc_with_tablets.py

test: Use ManagerClient.{disable,enable}_tablet_balancing()

2026-01-13 00:38:00 +01:00

test_change_ip.py

test.py: add pid to ServerInfo

2025-12-12 15:11:03 +02:00

test_change_replication_factor_1_to_0.py

test: cluster: deflake consistency checks after decommission

2025-09-09 19:01:12 +02:00

test_change_rpc_address.py

…

test_client_routes.py

Merge 'test: add missing awaits in test_client_routes_upgrade' from Andrzej Jackowski

2026-03-05 17:46:39 +02:00

test_cluster_features.py

test: test_downgrade_after_partial_upgrade: check that feature is disabled on all nodes after partial upgrade

2025-12-30 17:34:56 +01:00

test_commitlog_segment_data_resurrection.py

…

test_commitlog.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_compaction_backpressure.py

compaction_manager: fix maybe_wait_for_sstable_count_reduction() hanging forever

2026-02-27 01:38:13 +02:00

test_concurrent_schema.py

…

test_config_live_updates.py

…

test_config.py

…

test_conflicting_keys_read_repair.py

…

test_coordinator_queue_management.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_counter_write_timeout_metric.py

storage_proxy: mark write_timeouts metric for counter write timeouts

2026-01-14 17:50:46 +02:00

test_counters_with_tablets.py

test: Use ManagerClient.{disable,enable}_tablet_balancing()

2026-01-13 00:38:00 +01:00

test_crash_coordinator_before_streaming.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_create_table_during_node_shutdown.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_data_resurrection_after_cleanup.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_data_resurrection_in_memtable.py

replica/database: consolidate the two database_apply error injections

2026-04-24 18:31:11 +03:00

test_decommission_kill_then_replace.py

topology coordinator: complete pending operation for a replaced node

2026-01-14 13:11:27 +01:00

test_decommission.py

test: cluster: deflake consistency checks after decommission

2025-09-09 19:01:12 +02:00

test_deprecating_cluster_features.py

…

test_describe.py

…

test_different_group0_ids.py

db/config: don't use RBNO for scaling

2026-02-27 06:32:15 +02:00

test_encryption.py

test/cluster/test_encryption.py: use single-partition reads in read_verify_workload()

2026-04-16 21:09:09 +03:00

test_error_becoming_voter.py

…

test_failure_after_group0_server_registration.py

raft/group0: fix destroy assertion on startup failure

2026-05-08 11:24:02 +02:00

test_fencing.py

test: Use ManagerClient.{disable,enable}_tablet_balancing()

2026-01-13 00:38:00 +01:00

test_global_ignore_nodes.py

…

test_gossip_boot.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_gossiper_empty_self_id_on_shadow_round.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_gossiper_orphan_remover.py

Merge 'test: fix race condition in test_crashed_node_substitution' from Sergey Zolotukhin

2026-04-06 14:21:46 +03:00

test_gossiper_race.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_gossiper.py

…

test_group0_recovers_after_partial_command_application.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_group0_schema_versioning.py

test: test_group0_schema_versioning: wait for schema sync in system.local

2026-01-14 09:55:45 +01:00

test_hints.py

test/cluster: fix flaky test_hints_consistency_during_replace

2026-04-24 17:55:33 +03:00

test_incremental_repair.py

repair/replica: Fix race window where post-repair data is wrongly promoted to repaired

2026-05-08 11:25:26 +03:00

test_initial_token.py

test/cluster/conftest: cluster_con: provide default values for port and use_ssl

2025-08-22 09:51:24 +03:00

test_internode_compression.py

test_internode_compression: Add await for "run" coro:s

2026-03-20 10:32:31 +02:00

test_ip_mappings.py

test: test_full_shutdown_during_replace: retry replace after the replacing node is removed from gossip

2026-03-03 10:21:11 +01:00

test_keyspace_rf.py

test: Generalize tests to work with both numeric RF and rack lists

2025-10-29 23:32:58 +01:00

test_left_node_notification.py

raft topology: Notify that a node was removed only once

2025-12-29 14:22:34 +01:00

test_long_join.py

test: improve async execution in test_long_join

2025-09-08 17:14:37 +02:00

test_long_query_timeout_erm.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_lwt_semaphore.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_maintenance_mode.py

test: test_maintenance_mode: enable maintenance mode properly

2026-02-02 17:02:16 +00:00

test_major_compaction.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_metadata_id.py

test/cluster/conftest: cluster_con: provide default values for port and use_ssl

2025-08-22 09:51:24 +03:00

test_multidc.py

Merge 'test: cluster: Deflake test_startup_with_keyspaces_violating_rf_rack_valid_keyspaces' from Dawid Mędrek

2026-04-06 14:22:08 +03:00

test_mutation_schema_change.py

…

test_mv.py

tombstone_gc: don't use 'repair' mode for colocated tables

2025-11-25 09:15:46 +01:00

test_no_dc_rack_change.py

…

test_no_removed_node_event_on_ip_change.py

test.py: add pid to ServerInfo

2025-12-12 15:11:03 +02:00

test_node_isolation.py

topology: let banned node know that it is banned

2025-11-24 17:12:13 +01:00

test_node_ops_metrics.py

test/pylib/rest_client: fix ScyllaMetrics filtering

2025-08-10 10:16:00 +02:00

test_node_shutdown_waits_for_pending_requests.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_nodetool.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_not_enough_token_owners.py

test: cluster: Fix NoHostAvailable error in test_not_enough_token_owners

2026-01-08 21:53:47 +02:00

test_prepare_race.py

Merge 'cql3: pin prepared cache entry in prepare() to avoid invalid weak handle race' from Alex Dathskovsky

2026-03-20 10:27:04 +02:00

test_proxy_protocol.py

Merge 'test_proxy_protocol: fix flaky system.clients visibility checks' from Piotr Smaron

2026-03-10 22:48:14 +02:00

test_query_rebounce.py

…

test_raft_cluster_features.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_raft_fix_broken_snapshot.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_raft_ignore_nodes.py

…

test_raft_no_quorum.py

test: test_raft_no_quorum: decrease group0_raft_op_timeout_in_ms after quorum loss

2026-03-17 18:04:27 +01:00

test_raft_recovery_basic.py

…

test_raft_recovery_during_join.py

test: test_raft_recovery_during_join: get host ID of the bootstrapping node before it crashes

2026-01-22 06:55:16 +02:00

test_raft_recovery_entry_loss.py

test: unskip test_raft_recovery_entry_loss

2025-10-24 21:23:41 +03:00

test_raft_recovery_majority_loss.py

…

test_raft_recovery_stuck.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_raft_recovery_user_data.py

test: test_raft_recovery_user_data: prevent repeated ALTER KEYSPACE request

2026-01-22 14:13:42 +03:00

test_raft_snapshot_request.py

…

test_raft_snapshot_truncation.py

…

test_raft_voters.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_random_tables.py

…

test_read_repair.py

replica/database: consolidate the two database_apply error injections

2026-04-24 18:31:11 +03:00

test_refresh.py

reduce test_backup.py and test_refresh.py datasets

2026-01-21 10:47:36 +02:00

test_remove_alive_node.py

…

test_remove_rpc_client_with_pending_requests.py

…

test_repair.py

replica/database: consolidate the two database_apply error injections

2026-04-24 18:31:11 +03:00

test_replace_alive_node.py

…

test_replace_ignore_nodes.py

…

test_replace_with_encryption.py

…

test_replace_with_same_ip_twice.py

…

test_replace.py

…

test_replica_exceptions.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_rest_api_on_startup.py

test: add test_rest_api_on_startup

2025-12-03 15:35:59 +01:00

test_restart_cluster.py

…

test_resurrection.py

test: Use ManagerClient.{disable,enable}_tablet_balancing()

2026-01-13 00:38:00 +01:00

test_reversed_queries_during_simulated_upgrade_process.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_rpc_compression.py

…

test_select_from_mutation_fragments.py

…

test_shutdown_hang.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_size_based_load_balancing.py

test/cluster: Remove short_tablet_stats_refresh_interval injection

2026-02-17 08:35:16 +01:00

test_snapshot.py

…

test_sstable_cleanup_stop.py

test/cluster: fix flaky test_cleanup_stop by using asyncio.sleep

2026-04-06 14:20:46 +03:00

test_sstable_compression_config.py

schema: Add initializer for compression defaults

2026-01-13 20:45:59 +02:00

test_sstable_compression_dictionaries_autotrain.py

test: fix configuration of test_autoretrain_dict

2026-02-26 09:26:23 +02:00

test_sstable_compression_dictionaries_basic.py

db/config: Deprecate sstable_compression_dictionaries_allow_in_ddl

2025-10-29 20:13:08 +02:00

test_sstable_compression_dictionaries_upgrade.py

test: add a missing reconnect_driver in test_sstable_compression_dictionaries_upgrade.py

2026-04-06 22:07:57 +03:00

test_sstable_set.py

test: Use ManagerClient.{disable,enable}_tablet_balancing()

2026-01-13 00:38:00 +01:00

test_start_bootstrapped_with_invalid_seed.py

test: disable test_start_bootstrapped_with_invalid_seed

2026-01-15 10:25:45 +01:00

test_streaming_deadlock.py

test: limit test_streaming_deadlock_removenode concurrency

2025-09-19 12:50:20 +03:00

test_strong_consistency.py

test/cluster: add test_strong_consistency.py

2026-01-21 14:56:01 +01:00

test_table_desc_read_barrier.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_table_drop.py

sstables_loader: prevent use-after-free on table drop during streaming

2026-04-21 15:11:49 +02:00

test_tablet_repair_scheduler.py

repair: Implement auto repair for tablet repair

2026-01-09 16:11:39 +08:00

test_tablet_stats.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_tablets2.py

replica/database: consolidate the two database_apply error injections

2026-04-24 18:31:11 +03:00

test_tablets_colocation.py

test: Use ManagerClient.{disable,enable}_tablet_balancing()

2026-01-13 00:38:00 +01:00

test_tablets_cql.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_tablets_intranode.py

test: Use ManagerClient.{disable,enable}_tablet_balancing()

2026-01-13 00:38:00 +01:00

test_tablets_lwt.py

raft_topology: log sub-step progress in local_topology_barrier

2026-05-06 15:00:44 +03:00

test_tablets_merge.py

Merge 'tablets: Fix deadlock in background storage group merge fiber' from Tomasz Grabiec

2026-03-21 01:37:30 +01:00

test_tablets_migration.py

test: test_restart_leaving_replica_during_cleanup: reconnect driver after restart

2026-02-17 10:03:50 +01:00

test_tablets_parallel_decommission.py

test_tablets_parallel_decommission: Fix flakiness due to delayed task appearance

2026-02-20 16:35:39 +00:00

test_tablets_removenode.py

test: Use ManagerClient.{disable,enable}_tablet_balancing()

2026-01-13 00:38:00 +01:00

test_tablets.py

Merge 'service: add missing replicas if tablet rebuild was rolled back' from Aleksandra Martyniuk

2026-04-20 14:18:56 +03:00

test_tls.py

…

test_tombstone_gc.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_topology_failure_recovery.py

topology_coordinator, tablets: Fail draining operations when tablet migration fails due to critical disk utilization

2026-01-18 15:36:07 +01:00

test_topology_ops_encrypted.py

test: Use ManagerClient.{disable,enable}_tablet_balancing()

2026-01-13 00:38:00 +01:00

test_topology_ops_with_rf_rack_valid.py

test: fix test_remove_node_violating_rf_rack_with_rack_list

2026-02-18 12:36:52 +02:00

test_topology_ops.py

test: Use ManagerClient.{disable,enable}_tablet_balancing()

2026-01-13 00:38:00 +01:00

test_topology_recovery_basic.py

test.py: apply the nightly label on test_topology_recovery_basic

2025-09-01 14:16:29 +02:00

test_topology_recovery_majority_loss.py

test/cluster/conftest: cluster_con: provide default values for port and use_ssl

2025-08-22 09:51:24 +03:00

test_topology_rejoin.py

…

test_topology_remove_decom.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_topology_remove_garbage_group0.py

test: test_remove_garbage_group0_members: wait for token ring and group0 consistency before removenode

2026-03-20 10:22:40 +02:00

test_topology_schema.py

…

test_topology_smp.py

…

test_topology_upgrade_not_stuck_after_recent_removal.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_topology_upgrade_stuck.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_topology_upgrade.py

…

test_truncate_concurrent_writes.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_truncate_with_drop.py

system_keyspace: Prune dropped tables from truncation on start/drop

2025-09-03 07:25:34 +03:00

test_truncate_with_tablets.py

Merge 'table: don't create new split compaction groups if main compaction group is disabled' from Ferenc Szili

2026-04-16 10:39:29 +03:00

test_unfinished_writes_during_shutdown.py

Merge 'topology coordinator: restrict node join/remove to preserve RF-rack validity' from Michael Litvak

2026-01-09 10:01:23 +02:00

test_vector_store.py

index: allow vector indexes without rf_rack_valid_keyspces

2025-12-05 09:26:26 +02:00

test_view_build_status.py

test: Use ManagerClient.{disable,enable}_tablet_balancing()

2026-01-13 00:38:00 +01:00

test_view_building_coordinator.py

test: Use ManagerClient.{disable,enable}_tablet_balancing()

2026-01-13 00:38:00 +01:00

test_write_query_during_cql_server_shutdown.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_writes_to_previous_cdc_generations.py

test.py: convert skip_mode function to pytest.mark

2026-01-08 21:55:16 +02:00

test_zero_token_nodes_multidc.py

test: test_zero_token_nodes_multidc: properly handle reads with CL=LOCAL_ONE

2026-01-21 15:17:42 +01:00

test_zero_token_nodes_no_replication.py

test/cluster/conftest: cluster_con: provide default values for port and use_ssl

2025-08-22 09:51:24 +03:00

test_zero_token_nodes_topology_ops.py

…

util.py

test: use DROP KEYSPACE IF EXISTS in new_test_keyspace cleanup

2026-04-24 17:58:41 +03:00