mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-08-25 00:27:11 +00:00

Files

T

Chris LuandGitHub 301d83cc7a test: wait for the master to register the volume servers before failover tests run (#10871 )

The failover harness treated an open volume server port as readiness, but the
master only learns of a volume server from its heartbeat. A lone master refuses
heartbeats until its bootstrap check elects it, and the servers back off and
retry, so registration lands seconds after the ports answer. Tests that started
writing in that window assigned against an empty topology, which fails with
"no free volumes left" and reaches the mount as ENOSPC.

2026-08-21 23:05:14 -07:00

framework_test.go

test: wait for the master to register the volume servers before failover tests run (#10871 )

2026-08-21 23:05:14 -07:00

README.md

Recover from a dead volume server on the mount read path (#10798 )

2026-08-17 17:30:33 -07:00

volume_failover_test.go

test: wait for the master to register the volume servers before failover tests run (#10871 )

2026-08-21 23:05:14 -07:00

README.md

FUSE volume server failover tests

Integration tests for what happens to FUSE mounts when a volume server goes away, comes back, or restarts underneath in-flight IO. They automate the manual matrix reported in discussion #10206: one mount appends to a file while a second mount tails it, and a volume server is stopped, started or restarted mid-stream.

The cluster is 1 master (-defaultReplication=001), 3 volume servers, 1 filer and 2 mounts, all as local processes. With 001 every chunk has a copy on two of the three servers, so losing any single server must be invisible to both mounts.

Reported scenario	Test
control: append + tail with nothing failing	`TestAppendWithoutChaos`
read a file while one volume server is down	`TestReadWithVolumeServerDown`
"STOP volumes": append + tail, kill a server mid-stream	`TestAppendWhileVolumeServerStops`
"Start volumes": append + tail with a server down, start it mid-stream	`TestAppendWhileVolumeServerStarts`
"Re-start volumes": append + tail, restart a server mid-stream	`TestAppendWhileVolumeServerRestarts`
part 2: large file copy instead of small appends	`TestLargeWriteWhileVolumeServerStops`

Volume servers are dropped with SIGKILL, the closest local equivalent of a Swarm task disappearing from the overlay network: no deregistration, and the address stops answering.

Running

go build -o weed/weed ./weed
WEED_BINARY=$PWD/weed/weed go test -v -count=1 -timeout=30m ./test/fuse_failover/...

Needs FUSE and, on Linux, a working /dev/fuse. Logs from a failed run are copied to /tmp/seaweedfs-fuse-failover-logs/.