mirrors/scoutfs
1
0
Fork 0
mirror of https://github.com/versity/scoutfs.git synced 2026-02-11 04:51:08 +00:00
Code Issues Packages Projects Releases Wiki Activity

scoutfs: minimize commit writeback latencies

Browse Source 
Our simple transaction machinery causes high commit latencies if we let
too much dirty file data accumulate.

Small files have a natural limit on the amount of dirty data because
they have more dirty items per dirty page.  They fill up the single
segment sooner and kick off a commit which finds a relatively small
amount of dirty file data.

But large files can reference quite a lot of dirty data with a small
amount of extent items which don't fill up the transaction's segment.
During large streaming writes we can fill up memory with dirty file data
before filling a segment with mapping extent metadata.  This can lead to
high commit latencies when memory is full of dirty file pages.

Regularly kicking off background writeback behind streaming write
positions reduces the amount of dirty data that commits will find and
have to write out.

Signed-off-by: Zach Brown <zab@versity.com>
This commit is contained in:
Zach Brown
2018-06-21 17:07:29 -07:00
committed by Zach Brown
parent 59170f41b1
commit fddc3a7a75
1 changed files with 36 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
kmod/src/data.c
View File

@@ -20,6 +20,7 @@
#include <linux/hash.h>
#include <linux/log2.h>
#include <linux/falloc.h>
#include <linux/writeback.h>
#include "format.h"
#include "super.h"
@@ -874,6 +875,20 @@ out:
return ret;
}
/* kinda like __filemap_fdatawrite_range! :P */
static int writepages_sync_none(struct address_space *mapping, loff_t start,
loff_t end)
{
struct writeback_control wbc = {
.sync_mode = WB_SYNC_NONE,
.nr_to_write = LONG_MAX,
.range_start = start,
.range_end = end,
};
return mapping->a_ops->writepages(mapping, &wbc);
}
static int scoutfs_write_end(struct file *file, struct address_space *mapping,
loff_t pos, unsigned len, unsigned copied,
struct page *page, void *fsdata)
@@ -900,6 +915,27 @@ static int scoutfs_write_end(struct file *file, struct address_space *mapping,
scoutfs_release_trans(sb);
scoutfs_inode_index_unlock(sb, &wbd->ind_locks);
kfree(wbd);
/*
* Currently transactions are kept very simple. Only one is
* open at a time and commit excludes concurrent dirtying. It
* writes out all dirty file data during commit. This can lead
* to very long commit latencies with lots of dirty file data.
*
* This hack tries to minimize these writeback latencies while
* keeping concurrent large file strreaming writes from
* suffering too terribly. Every N bytes we kick off background
* writbeack on the previous N bytes. By the time transaction
* commit comes along it will find that dirty file blocks have
* already been written.
*/
#define BACKGROUND_WRITEBACK_BYTES (16 * 1024 * 1024)
#define BACKGROUND_WRITEBACK_MASK (BACKGROUND_WRITEBACK_BYTES - 1)
if (ret > 0 && ((pos + ret) & BACKGROUND_WRITEBACK_MASK) == 0)
writepages_sync_none(mapping,
pos + ret - BACKGROUND_WRITEBACK_BYTES,
pos + ret - 1);
return ret;
}