From 7e935898abd4081ffde36ee081eda6ccbb044f69 Mon Sep 17 00:00:00 2001 From: Zach Brown Date: Fri, 30 Jul 2021 13:00:15 -0700 Subject: [PATCH] Avoid premature metadata enospc server_get_log_trees() sets the low flag in a mount's meta_avail allocator, triggering enospc for any space consuming allocatins in the mount, if the server's global meta_vail pool falls below the reserved block count. Before each server transaction opens we swap the global meta_avail and meta_freed allocators to ensure that the transaction has at least the reserved count of blocks available. This creates a risk of premature enospc as the global meta_avail pool drains and swaps to the larger meta_freed. The pool can be close to the reserved count, perhaps at it exactly. _get_log_trees can fill the client's mount, even a little, and drop the global meta_avail total under the reserved count, triggering enospc, even though meta_Freed could have had quite a lot of blocks. The fix is to ensure that the global meta_avail has 2x the reserved count and swapping if it falls under that. This ensures that a server transaction can consume an entire reserved count and still have enough to avoid triggering enospc. This fixes a scattering of rare premature enospc returns that were hitting during tests. It was rare for meta_avail to fall just at the reserved count and for get_log_trees to have to refill the client allocator, but it happened. Signed-off-by: Zach Brown --- kmod/src/server.c | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/kmod/src/server.c b/kmod/src/server.c index d605d0ca..510114aa 100644 --- a/kmod/src/server.c +++ b/kmod/src/server.c @@ -323,7 +323,6 @@ static void scoutfs_server_commit_func(struct work_struct *work) struct commit_waiter *cw; struct commit_waiter *pos; struct llist_node *node; - u64 reserved; int ret; trace_scoutfs_server_commit_work_enter(sb, 0, 0); @@ -389,16 +388,19 @@ static void scoutfs_server_commit_func(struct work_struct *work) server->other_freed = &super->server_meta_freed[server->other_ind]; /* - * The reserved metadata blocks includes the max size of - * outstanding allocators and a server transaction could be - * asked to refill all those allocators from meta_avail. If our - * meta_avail falls below the reserved count, and freed is still - * above it, then swap so that we don't start returning enospc - * until we're truly low. + * get_log_trees sets ALLOC_LOW when its allocator drops below + * the reserved blocks after having filled the log trees's avail + * allocator during its transaction. To avoid prematurely + * setting the low flag and causing enospc we make sure that the + * next transaction's meta_avail has 2x the reserved blocks so + * that it can consume a full reserved amount and still have + * enough to avoid enospc. We swap to freed if avail is under + * the buffer and freed is larger. */ - reserved = scoutfs_server_reserved_meta_blocks(sb); - if (le64_to_cpu(server->meta_avail->total_len) <= reserved && - le64_to_cpu(server->meta_freed->total_len) > reserved) + if ((le64_to_cpu(server->meta_avail->total_len) < + (scoutfs_server_reserved_meta_blocks(sb) * 2)) && + (le64_to_cpu(server->meta_freed->total_len) > + le64_to_cpu(server->meta_avail->total_len))) swap(server->meta_avail, server->meta_freed); ret = 0;