Now that we have the inode refreshing flags let's add them to the
callers that want to have a current inode after they have their lock.
Callers locking newly created items use the new inode flag to reset the
refresh gen.
A few inode tests are moved down to after locking so that it can test
the current refreshed inode.
Signed-off-by: Zach Brown <zab@versity.com>
Lock callers can specify that they want inode fields reread from items
after the lock is acquired. dlmglue sets a refresh_gen in the locks
that we store in inodes to track when they were last refreshed and if
they need a refresh.
Signed-off-by: Zach Brown <zab@versity.com>
In addition to setting NEEDS_REFRESH when locks are acquired out of NL,
we now also give them a refresh_gen counter that is increased by
incrementing a long lived counter in the super.
This gives callers a strictly increasing read-only indication that the
lock has changed. They don't have to serialize users to clear
NEEDS_REFRESH and transfer it to some other serialized state.
scoutfs will use with the multiple inodes that are refreshed with
respect to the lock's refresh_gen.
Signed-off-by: Zach Brown <zab@versity.com>
This is based on Mark Fasheh <mfasheh@versity.com>'s series that
introduced inode refreshing after locking and a trylock for readpage.
Rework the inode locking function so that it's more clearly named and
takes flags and the inode struct.
We have callers that want to lock the logical inode but aren't doing
anything with the vfs inode so we provide that specific entry point.
Signed-off-by: Zach Brown <zab@versity.com>
This portion of the port needs a bit of work before we can use
it in scoutfs. In the meantime, disable it so that we can build
on debug kernels.
Signed-off-by: Mark Fasheh <mfasheh@versity.com>
These make it hard to read the header and are very ocfs2-specific
functions that would get moved when we merge this upstream anyway.
Signed-off-by: Mark Fasheh <mfasheh@versity.com>
When we're not the server node, 'mani' is NULL, so derefing it in our
loop causes a crash. That said, we don't need it anyway - the loop will
eventually end when our btree walk (via btree_prev_overlap_or_next())
ends.
Signed-off-by: Mark Fasheh <mfasheh@versity.com>
We move struct ocfs2_lock_res_ops and flags to dlmglue.c so that
locks.c can get access to it. Similarly, we export
ocfs2_lock_res_init_common() for locks.c can initialize each lockres
before use. Also, free_lock_tree() now has to happen before we shut
down the dlm - this gives dlmglue the opportunity to unlock their
underlying dlm locks before we go off freeing the structures.
Signed-off-by: Mark Fasheh <mfasheh@versity.com>
Ultimataly the direct dlm lock calls will go away. For now though we
grab the lockspace off our cluster connection object. In order to get
this going, I stubbed out our recovery callbacks which now gets us a
print when a node goes down.
Signed-off-by: Mark Fasheh <mfasheh@versity.com>
These work with little modification. We comment out a couple
ocfs2-specific lines. And decouple a few more variables from the osb
structure. As it stands, ocfs2 could also use these init/shutdown
functions with little modification.
Signed-off-by: Mark Fasheh <mfasheh@versity.com>
Dlmglue is built on top of this. Bring in the portions we need which
includes the stackglue API as well as most of the fs/dlm implementation.
I left off the Ocfs2 specific version and connection handling. Also
left out is the old Ocfs2 dlm support which we'll never want.
Like dlmglue, we keep as much of the generic stackglue code in tact
here. This will make translating to/from upstream patches much easier.
Signed-off-by: Mark Fasheh <mfasheh@versity.com>
This only leaked into the bast function. I retained the debug print -
it'll be turned off in our build anyway, and that's what we'd
want to do upstream.
Signed-off-by: Mark Fasheh <mfasheh@versity.com>
We need this for the lockres name. It also turns out to be the only
thing we need from fs/ocfs2/ocfs2_lockid.h.
Signed-off-by: Mark Fasheh <mfasheh@versity.com>
Some of this leaks through even after the big #ifdef'ing - ocfs2 had
to special case printing the name of dentry locks. We don't have such
a need so it's easy to drop those calls.
Signed-off-by: Mark Fasheh <mfasheh@versity.com>
We only want the generic stuff. Long term the Ocfs2 specific code would be
what's left in fs/ocfs2/dlmglue.[ch].
Signed-off-by: Mark Fasheh <mfasheh@versity.com>
Like the mtime index, this index is unused. Removing it is a near
identical task. Running the same createmany test from our last
patch gives us the following:
$ createmany -o '/scoutfs/file_%lu' 10000000
total: 10000000 creates in 598.28 seconds: 16714.59 creates/second
real 9m58.292s
user 0m7.420s
sys 5m44.632s
So after both indices are gone, we go from a 12m56 run time to 9m58s,
saving almost 3 minutes which translates into a total performance
increase of about 23%.
Signed-off-by: Mark Fasheh <mfasheh@versity.com>
This index is unused - we can gain some create performance by removing it.
To verify this, I ran createmany for 10 million files:
$ createmany -o '/scoutfs/file_%lu' 10000000
Before this patch:
total: 10000000 creates in 776.54 seconds: 12877.56 creates/second
real 12m56.557s
user 0m7.861s
sys 6m56.986s
After this patch:
total: 10000000 creates in 691.92 seconds: 14452.46 creates/second
real 11m31.936s
user 0m7.785s
sys 6m19.328s
So removing the index gained us about a minute and a half on the test or a
12% performance increase.
Signed-off-by: Mark Fasheh <mfasheh@versity.com>
Truncation updates extents that intersect with the input range. It
starts with the first block in the range and iterates until it has
searched for all the extents that could cover the range.
Extents are stored in items at their final block location so that we can
use _next to find intersections. Truncation was searching for the next
extent after the full extent that it was still searching for. That
means it was starting the search at the last block in the extent, not
the first. It would miss all the extents that didn't overlap with the
last block it was searching for.
This fixed by searching from a temporary single block extent at the
start of the search range.
Signed-off-by: Zach Brown <zab@versity.com>
Offline extents weren't being merged because they all had their physical
blkno set to 0 and all the extent calculations didn't treat them
specially. They would only merge if the physical blocks of two extent
were contiguous. Instead of special casing offline extents everywhere
we store them with a physical blkno set to the logical blk_off. This
lets all the current extent calculations work as expected.
Signed-off-by: Zach Brown <zab@versity.com>
Release tries to re-instate extents if it sees an error during release.
Those item manipulations need to be covered by the transaction.
Signed-off-by: Zach Brown <zab@versity.com>
The existing release interface specified byte regions to release but
that didn't match what the underlying file data mapping structure is
capable of. What happens if you specify a single byte to release? Does
it release the whole block? Does it release nothing? Does it return an
error?
By making the interface match the capability of the operation we make
the functioning of the system that much more predictable. Callers are
forced to think about implementing their desires in terms of block
granular releasing.
Signed-off-by: Zach Brown <zab@versity.com>
The ->statfs method was still using the super_block in the super_info
that was read during mount. This will get progressively more out
of date.
We add a network message to ask the server for the current fields that
impact statfs. This is always racy and the fields are mostly nonsense,
but we try our best.
Signed-off-by: Zach Brown <zab@versity.com>
Delete inode index items when deleting all the items associated with an
inode after its been unlinked and had all its references dropped.
The index items should always match the fields in the inode item so we
read it to determine the index items that should be deleted, regardless
of if we have the vfs inode cached or not. We take the opportunity to
collapse the two callers of item deletion which looked up the inode into
item deletion so that it can use the inode fields.
The deletion of index items is partially verified by an inode index test
in xfstests which makes sure that unlinked files are no longer present
in the index.
Signed-off-by: Zach Brown <zab@versity.com>
Directories were getting added to the data_seq index. It might have
looked like they weren't because their data_seqs were always 0 but when
inodes are created they don't have 'have_item' set so all the fields are
added regardless of their current value.
We'd rather not have to wade their directories when looking for regular
file data in the data_seq index so let's explicitly test for regular
files when updating the data_seq index items.
Signed-off-by: Zach Brown <zab@versity.com>
The updating of the inode index items was racey. It loaded the inode
values, updated the items, loaded the fields again, and then stored the
fields in the inode info. All without locking. Concurrent attempts
could get the fields scrambled and racing with other paths that update
the fields could get the items and inode info out of sync.
This fixes up the two races by only reading the inode fields once and
performing the multi-stage update under a mutex. We add a new lock to
avoid ordering problems with trying to add an existing lock at these
points in the locking heirarchy. We specifically use a mutex because
the item functions can block.
Now the inode index field update just has to safely race with concurrent
access to the fields.
This was found by generic/037 once getattr started refreshing the inode.
It now passes again.
Signed-off-by: Zach Brown <zab@versity.com>
Use MODULE_ALIAS_FS() to register the "scoutfs" fs alias so that
modprobe can find the module if it's installed and visible to depmod.
We don't yet have clever enough xfstests to mess around with modules. I
manually verified this by installing the module in /lib/modules and
trying mount -t scoutfs before and after the change.
Signed-off-by: Zach Brown <zab@versity.com>
The networking code was really suffering by trying to combine the client
and server processing paths into one file. The code can be a lot
simpler by giving the client and server their own processing paths that
take their different socket lifecysles into account.
The client maintains a single connection. Blocked senders work on the
socket under a sending mutex. The recv path runs in work that can be
canceled after first shutting down the socket.
A long running server work function acquires the listener lock, manages
the listening socket, and accepts new sockets. Each accepted socket has
a single recv work blocked waiting for requests. That then spawns
concurrent processing work which sends replies under a sending mutex.
All of this is torn down by shutting down sockets and canceling work
which frees its context.
All this restructuring makes it a lot easier to track what is happening
in mount and unmount between the client and server. This fixes bugs
where unmount was failing because the monolithic socket shutdown
function was queueing other work while running while draining.
Signed-off-by: Zach Brown <zab@versity.com>
The rhashtable API has changed over time. Continuing to use it means
having to worry about maintaining different APIs in different kernel
generations.
We have a static pool of cursors so we don't need the flexibility of the
resizable rhashtable. We can roll a simple array of hlist heads to use
as a hash table.
And finally, these cursors will probably disappear eventually anyway.
Let's not invest too much in them.
Signed-off-by: Zach Brown <zab@versity.com>
Raw [su]{8,16,32,64} types keep leaking into our exported headers where
they break userspace builds. Make sure that we only use the exported __
types and add a check to break our build if we get it wrong.
Signed-off-by: Zach Brown <zab@versity.com>
It's handy to quickly find the git commit that built a given module. We
add a MOD_INFO() tag for it so we can see it in modinfo on the built
module. We add a ELF note that the kernel tracks in
/sys/modules/$m/notes/ when the module is loaded.
Signed-off-by: Zach Brown <zab@versity.com>
Zach Brown
Mark Fasheh
