mirror of
https://github.com/versity/scoutfs.git
synced 2026-07-19 06:22:24 +00:00
Auke Kok
4a8d2211fe
A newly elected leader fences the previous quorum leader as it's removed from the quorum, and the same rid is also a mounted client that the new server will scan and prepare for recovery. If that fence is still pending when the 30s recovery timeout fires, the rid is fenced a second time and the two fence requests collide on the rid-named sysfs dir. Close the window at its source: reclaim the pending fences and wait for them to fully drain before scanning the mounted client btree for recovery. Reclaim removes a fenced rid from that btree, so once the fence list is empty the previous leader is no longer a recovery candidate and the recovery timeout can't fence it again. Add scoutfs_fence_drained() as a non-blocking predicate in the fence layer: it reports an empty pending list and surfaces any errored fence. The server owns the wait in wait_for_fence_drain(), blocking on server->waitq so it wakes promptly when the server stops. queue_reclaim_work() moves ahead of start_recovery() so reclaim runs during the drain. Draining is only an optimization, since the existing fence dedup already makes a double fence non-fatal. The backstop timeout is therefore best effort: on a large system a single reclaim can legitimately run longer than the timeout, and aborting a healthy reclaim would recreate the recovery-failure loop we're avoiding. On timeout the server warns and starts recovery anyway, letting reclaim finish in the background and dedup absorb any double-submit. Startup is only aborted if a fence errored (the node couldn't be fenced, so recovery isn't safe) or the server is already stopping. Signed-off-by: Auke Kok <auke.kok@versity.com>
Introduction
scoutfs is a clustered in-kernel Linux filesystem designed to support large archival systems. It features additional interfaces and metadata so that archive agents can perform their maintenance workflows without walking all the files in the namespace. Its cluster support lets deployments add nodes to satisfy archival tier bandwidth targets.
The design goal is to reach file populations in the trillions, with the archival bandwidth to match, while remaining operational and responsive.
Highlights of the design and implementation include:
- Fully consistent POSIX semantics between nodes
- Atomic transactions to maintain consistent persistent structures
- Integrated archival metadata replaces syncing to external databases
- Dynamic seperation of resources lets nodes write in parallel
- 64bit throughout; no limits on file or directory sizes or counts
- Open GPLv2 implementation
Community Mailing List
Please join us on the open scoutfs-devel@scoutfs.org mailing list hosted on Google Groups