mirrors/scoutfs
1
0
Fork 0
mirror of https://github.com/versity/scoutfs.git synced 2026-07-19 06:22:24 +00:00
Code Issues Packages Projects Releases Wiki Activity

Don't shut down the server when fencing a rid twice

Browse Source 
A node only needs to be fenced once, but scoutfs_fence_start() can be
called for the same rid more than once. When a new leader starts it
fences the previous leader as it removes it from the quorum
(quorum_block_leader), and that same rid can also be a mounted client
that then fails to recover within the timeout (client_recovery).

The second fence call collides on that name, sysfs returns -EEXIST,
and the error is propagated to fence_pending_recov_worker() which
treats any error as fatal and shuts the server down. On the next
mount a new leader hits the same stale set and the same collision,
so the filesystem can never finish recovery.

  Jun 15 09:22:35 kernel: scoutfs f.000000.r.222222: fencing previous leader f.000000.r.111111 at term 183942 in slot 3 with address x.x.x.x:6000
  Jun 15 09:22:36 scoutfs-fenced[9194]: [2026-06-15 09:22:36.588037842] server f.000000.r.222222 fencing rid 1111111111111111 at IP x.x.x.x for quorum_block_leader
  Jun 15 09:23:09 kernel: scoutfs f.000000.r.222222 error: 30000 ms recovery timeout expired for client rid 1111111111111111, fencing
  Jun 15 09:23:09 kernel: sysfs: cannot create duplicate filename '/fs/scoutfs/f.000000.r.222222/fence/1111111111111111'
  Jun 15 09:23:09 kernel: scoutfs f.000000.r.222222 error: fence returned err -17, shutting down server

The reclaim path keys off the rid, not the reason, so one pending fence
covers both needs. Skip submitting a duplicate fence for an rid that
already has one pending, and treat a -EEXIST from the sysfs create as
success to also cover a race or a stale rid-named dir.

Signed-off-by: Auke Kok <auke.kok@versity.com>
This commit is contained in:
Auke Kok
2026-06-15 11:37:48 -07:00
parent 31e7101319
commit 313674a439
1 changed files with 22 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
kmod/src/fence.c
+22
View File
@@ -238,6 +238,25 @@ int scoutfs_fence_start(struct super_block *sb, u64 rid, __be32 ipv4_addr, int r
struct pending_fence *fence;
int ret;
/*
* A node only needs to be fenced once. A given rid can be submitted
* for fencing more than once: the previous quorum leader is fenced as
* it's removed from the quorum, and that same rid can also be a
* mounted client that fails to recover in time. The reclaim path
* keys off the rid, not the reason, so a single pending fence is
* sufficient. Skip creating a duplicate, which would otherwise
* collide on the rid-named sysfs dir and return -EEXIST to the caller,
* shutting the server down.
*/
spin_lock(&fi->lock);
list_for_each_entry(fence, &fi->list, entry) {
if (fence->rid == rid) {
spin_unlock(&fi->lock);
return 0;
}
}
spin_unlock(&fi->lock);
fence = kzalloc(sizeof(struct pending_fence), GFP_NOFS);
if (!fence) {
ret = -ENOMEM;
@@ -258,6 +277,9 @@ int scoutfs_fence_start(struct super_block *sb, u64 rid, __be32 ipv4_addr, int r
&fence->ssa, fence_attrs,
"%016llx", rid);
if (ret < 0) {
/* highly unlikely race collision */
if (ret == -EEXIST)
ret = 0;
kfree(fence);
goto out;
}