This is pretty straight forward - we define a new item type,
SCOUTFS_ORPHAN_KEY. We don't need to store any value with this, the inode
and type fields are enough for us to find what inode has been orphaned.
Otherwise this works as one would expect. Unlink sets the item, and
->evict_inode removes it. On mount, we scan for orphan items and remove any
corresponding inodes.
Signed-off-by: Mark Fasheh <mfasheh@versity.com>
Signed-off-by: Zach Brown <zab@versity.com>
Adding in an 'all' target allows us to use canned build scripts for any
of the scoutfs related repositories.
Signed-off-by: Nic Henke <nic.henke@versity.com>
Signed-off-by: Zach Brown <zab@zabbo.net>
When running make in a limited shell or in docker, there is no PWD from
shell. By using CURDIR we avoid worrying about the environment and let
make take care of this for us.
Signed-off-by: Nic Henke <nic.henke@versity.com>
Signed-off-by: Zach Brown <zab@zabbo.net>
We don't overwrite existing data. Every file data write has to allocate
new blocks and update block mapping items.
We can search for inodes whose data has changed by filtering block
mapping item walks by the sequence number. We do this by using the
exact same code for finding changed inodes but using the block mapping
key type.
Signed-off-by: Zach Brown <zab@versity.com>
We make an event class for the two most common btree op patterns, and reuse
that to make our tracepoints for each function. This covers all the entry
points listed in btree.h. We don't get every single parameter of every
function but this is enough that we can see which keys are being queried /
inserted.
Signed-off-by: Mark Fasheh <mfasheh@versity.com>
Signed-off-by: Zach Brown <zab@versity.com>
The upcoming allocator changes have a need to forget dirty blocks so
they're not written. It proabably won't be the only one.
Signed-off-by: Zach Brown <zab@versity.com>
I accidentally left some lock tracing in the btree locking commit that
is very noisy and not particularly useful. Let's remove it.
Signed-off-by: Zach Brown <zab@versity.com>
We already have a function that zeros the end of a block starting at a
given offset. Some callers have a pointer to the byte to zero from so
let's add a convenience function that calculates the offset from the
pointer.
Signed-off-by: Zach Brown <zab@versity.com>
The pseudo random byte wrapper function used the intel instructions
so that it could deal with high call rates, like initializing random
node priorities for a large treap.
But this is obviously not remotely portable and has the annoying habit
of tripping up versions of valgrind that haven't yet learned about these
instructions.
We don't actually have high bandwidth callers so let's back off and just
let openssl take care of this for us.
Signed-off-by: Zach Brown <zab@versity.com>
The btree locking so far was a quick interim measure to get the rest of
the system going. We want to clean it up both for correctness and
performance but also to make way for using the btree for block
allocation.
We were unconditionally using the buffer head lock for tree block
locking. This is bad for at least four reasons: it's invisible to
lockdep, it doesn't allow concurrent reads, it doesn't allow reading
while a block is being written during the transaction, and it's not
necessary at all when the for stable read-only blocks.
Instead we add a rwsem to the buffer head private which we use to lock
the block when it's writable. We clean up the locking functions to make
it clearer that btree_walk holds one lock at a time and either returns
it to the caller with the buffer head or unlocks the parent if its
returning an error.
We also add the missing sibling block locking during splits and merges.
Locking the parent prevented walks from descending down our path but it
didn't protect against previous walks that were already down at our
sibling's level.
Getting all this working with lockdep adds a bit more class/subclass
plumbing calls but nothing too ornerous.
Signed-off-by: Zach Brown <zab@versity.com>
The btree cursor was built to address two problems. First it
accelerates iteration by avoiding full descents down the tree by holding
on to leaf blocks. Second it lets callers reference item value contents
directly to avoid copies.
But it also has serious complexity costs. It pushes refcounting and
locking out to the caller. There have already been a few bugs where
callers did things while holding the cursor without realizing that
they're holding a btree lock and can't perform certain btree operations
or even copies to user space.
Future changes to the allocator to use the btree motivates cleaning up
the tree locking which is complicated by the cursor being a stand alone
lock reference. Instead of continuing to layer complexity onto this
construct let's remove it.
The iteration acceleration will be addressed the same way we're going to
accelerate the other btree operations: with per-cpu cached leaf block
references. Unlike the cursor this doesn't push interface changes out
to callers who want repeated btree calls to perform well.
We'll leave the value copying for now. If it becomes an issue we can
add variants that call a function to operate on the value. Let's hope
we don't have to go there.
This change replaces the cursor with a vector to memory that the value
should be copied to and from. The vector has a fixed number of elements
and is wrapped in a struct for easy declaration and initialization.
This change to the interface looks noisy but each caller's change is
pretty mechanical. They tend to involve:
- replace the cursor with the value struct and initialization
- allocate some memory to copy the value in to
- reading functions return the number of value bytes copied
- verify copied bytes makes sense for item being read
- getting rid of confusing ((ret = _next())) looping
- _next now returns -ENOENT instead of 0 for no next item
- _next iterators now need to increase the key themselves
- make sure to free allocated mem
Sometimes the order of operations changes significantly. Now that we
can't modify in place we need to read, modify, write. This looks like
changing a modification of the item through the cursor to a
lookup/update pattern.
The symlink item iterators didn't need to use next because they walk a
contiguous set of keys. They're changed to use simple insert or lookup.
Signed-off-by: Zach Brown <zab@versity.com>
The final symlink item insertion was taking the min of the entire path
and the max symlink item size, not the min of the remaining length of
the path after having created all the previous items. For paths larger
than the max item size this could use too much space.
Signed-off-by: Zach Brown <zab@versity.com>
This is analagous to scoutfs_inc_key(). It decreases the next highest
order key value each time its decrement wraps.
Signed-off-by: Zach Brown <zab@versity.com>
The btree functions currently don't take a specific root argument. They
assume, deep down in btree_walk, that there's only one btree in the
system. We're going to be adding a few more to support richer
allocation.
To prepare for this we have the btree functions take an explicit btree
argument. This should make no functional difference.
Signed-off-by: Zach Brown <zab@versity.com>
The buddy allocator had the test for non-existant stable bitmap blocks
backwards. An uninitialized block implies that all the bits are marked
free and we don't need to test that the specific bits are free.
Signed-off-by: Zach Brown <zab@versity.com>
Initialize the free_order field in all the slots of the buddy index
block so that the kernel will try to allocate from them and will
initialize and populate the first block.
Signed-off-by: Zach Brown <zab@versity.com>
The check for aligned buffer head data pointers was trying to
dereference a bad IS_ERR pointer when allocation of a new block failed
with ENOSPC.
Signed-off-by: Zach Brown <zab@versity.com>
The btree block merging code knew to try and compact the destination
block if it was going to move more bytes worth of items than there was
contiguous free space in the destination block. But it missed the case
where item movement moves more than the hint because the last item it
moves was big. In the worst case this creates an item which overlaps
the item offsets and ends up looking like corrupt items.
Signed-off-by: Zach Brown <zab@versity.com>
Add a BUG_ON() assertion for the case where we create an item that
starts in the item offset array. This happens if the callers free space
calculations are incorrect. It shouldn't be triggerable by corrupt
blocks if we're verifying the blocks as we read them in.
Signed-off-by: Zach Brown <zab@versity.com>
Our follow_link method forgot to fill the nameidata with the target path
of the symlink. The uninitialized nameidata tripped up the generic
readlink code in a debugging kernel.
Signed-off-by: Zach Brown <zab@versity.com>
We don't want very large transactions to build up and create huge commit
latencies. All blocks are written to free space so we use a count of
allocations to count dirty blocks. We arbitrarily limit the transaction
to 128MB and try to kick off commits when we release transactions that
have gotten that big.
Signed-off-by: Zach Brown <zab@versity.com>
Wire up the inode callbacks that let us remove all the persistent items
associated with an unlinked inode as its final reference is dropped.
This is the first part of full truncate and orphan inode support.
Signed-off-by: Zach Brown <zab@versity.com>
If block validation failed then we'd end up trying to unlock an IS_ERR
buffer_head pointer. Fix it so that we drop the ref and set the
pointer after unlocking.
Signed-off-by: Zach Brown <zab@versity.com>
To do a credible job of this we need to track the number of free blocks.
We add counters of order allocations free to the indirect blocks so that
we can quickly scan them. We also need a bit of help to count inodes.
Finally I noticed that we were miscalculating the number of slots in the
indirect blocks because we were using the size of the buddy block
header, not the size of the indirect block header.
Signed-off-by: Zach Brown <zab@versity.com>
Add commands that use the find-xattr ioctls to show the inode numbers of
inodes which probably contain xattrs matching the specified name or
value.
Signed-off-by: Zach Brown <zab@versity.com>
Add ioctls that return the inode numbers that probably contain the given
xattr name or value. To support these we add items that index inodes by
the presence of xattr items whose names or values hash to a give hash
value.
Signed-off-by: Zach Brown <zab@versity.com>
The previous %llu for the key type came from the weird tracing functions
that cast all the arguments to long long. Those have since been
removed.
Signed-off-by: Zach Brown <zab@versity.com>
Now that we have the link backrefs let's add support for hard links so
we can verify that an inode can have multiple backrefs. (It can.)
It's a straight forward refactoring of mknod to let callers either
allocate or use existing inodes. We push all the btree item specific
work into a function called by mknod and link.
The only surprising bit is the small max link count. It's limiting
the worst case buffer size for the inode_paths ioctl.
Signed-off-by: Zach Brown <zab@versity.com>
Add the inode-paths command which uses the ioctl to display all the
paths that lead to the given inode. We add support for printing
the new link backref items and inode and dirent fields.
Signed-off-by: Zach Brown <zab@versity.com>
This adds the ioctl that returns all the paths from the root to a given
inode. The implementation only traverses btree items to keep it
isolated from the vfs object locking and life cycles, but that could be
a performance problem. This is another motivation to accelerate the
btree code.
Signed-off-by: Zach Brown <zab@versity.com>
Up to this point we'd been storing file data in large fixed size items.
This obviously needed to change to get decent large file IO patterns.
This wires the file IO into the usual page cache and buffer head paths
so that we write data blocks into allocations referenced by btree items.
We're aggressively trying to find the highest ratio of performance to
implementation complexity.
Writing dirty metadata blocks during transaction commit changes a bit.
We need to discover if we have dirty blocks before trying to sync the
inodes. We add our _block_has_dirty() function back and use it to avoid
write attempts during transaction commit.
Signed-off-by: Zach Brown <zab@versity.com>
We can certainly have btree update callers that haven't yet dirtied the
blocks but who can deal with errors. So make it return errors and have
its only current caller freak out if it fails. This will let the file
data block mapping code attempt to get a dirty item without first
dirtying.
Signed-off-by: Zach Brown <zab@versity.com>
Add helpers to discover if a given allocation was free and to free all
the buddy order allocations that make up an abritrary block extent.
These are going to be used by the file data block mapping code.
Signed-off-by: Zach Brown <zab@versity.com>
Now that we are using fixed smaller blocks we can make the btree format
significantly simpler. The fixed small block size limits the number of
items that will be stored in each block. We can use a simple sorted
array of item offsets to maintain the item sort order instead of
the treap.
Getting rid of the treap not only removes a bunch of code, it makes
tasks like verifying or repairing a btree block a lot simpler.
The main impact on the code is that now an item doesn't record its
position in the sort order. Users of sorted item position now need to
track an items sorted position instead of just the item.
Signed-off-by: Zach Brown <zab@versity.com>
The buffer head rewrite got rid of the only caller who needed to ensure
that a free couldn't fail. Let's get rid of this. We can always bring
it back if it's needed again.
Signed-off-by: Zach Brown <zab@versity.com>
