Prefer named to anonymous enums. This helps readability a little.
Use enum as param type if possible (a couple spots).
Remove unused enum in lock_server.c.
Define enum spbm_flags using shift notation for consistency.
Rename get_file_block()'s "gfb" parameter to "flags" for consistency.
Signed-off-by: Andy Grover <agrover@versity.com>
Not initializing wid[] can cause incorrect output.
Also, we only need 6 columns if we reference the array from 0.
Signed-off-by: Andy Grover <agrover@versity.com>
The xfstests generic/067 test is a bit of a stinker in that it's trying
to make sure a mount failes when the device is invalid. It does this
with raw mount calls without any filesystem-specific conventions. Our
mount fails, so the test passes, but not for the reason the test
assumes. It's not a great test. But we expect it to not be great and
produce this message.
Signed-off-by: Zach Brown <zab@versity.com>
Add another expected message that comes from attempting to mount an ext4
filesystem from a device that returns read errors.
Signed-off-by: Zach Brown <zab@versity.com>
The tests were checking that the literal string was zero, which it never
was. Once we check the value of the variable then we notice that the
sense of some tests went from -n || to -n &&, so switch those to -z.
Signed-off-by: Zach Brown <zab@versity.com>
For xfstests, we need to be able to specify both for scratch device as
well.
using -e and -f for now, but we should really be switching to long options.
Signed-off-by: Andy Grover <agrover@versity.com>
[zab@versity.com: minor arg message fixes]
Add -z option to run-tests.sh to specify metadata device.
Do a bunch of things twice.
Fix up setup-error-teardown test.
Signed-off-by: Andy Grover <agrover@versity.com>
[zab@versity.com: minor arg message fixes, golden output]
mkfs: Take two block devices as arguments. Write everything to metadata
dev, and the superblock to the data dev. UUIDs match. Differentiate by
checking a bit in a new "flags" field in the superblock.
Refactor device_size() a little. Convert spaces to tabs.
Move code to pretty-print sizes to dev.c so we can use it in error
messages there, as well as in mkfs.c.
print: Include flags in output.
Add -D and -M options for setting max dev sizes
Allow sizes to be specified using units like "K", "G" etc.
Note: -D option replaces -S option, and uses above units rather than
the number of 4k data blocks.
Update man pages for cmdline changes.
Signed-off-by: Andy Grover <agrover@versity.com>
Update the README.md introduction to scoutfs to mention the need for and
use of metadata and data block devices.
Signed-off-by: Zach Brown <zab@versity.com>
Require a second path to metadata bdev be given via mount option.
Verify meta sb matches sb also written to data sb. Change code as needed
in super.c to allow both to be read. Remove check for overlapping
meta and data blknos, since they are now on entirely separate bdevs.
Use meta_bdev for superblock, quorum, and block.c reads and writes.
Signed-off-by: Andy Grover <agrover@versity.com>
It was too tricky to pick out the difference between metadata and data
usage in the previous format. This makes it much more clear which
values are for either metadata or data.
Signed-off-by: Zach Brown <zab@versity.com>
Write locks are given an increasing version number as they're granted
which makes its way into items in the log btrees and is used to find the
most recent version of an item.
The initialization of the lock server's next write_version for granted
locks dates back to the initial prototype of the forest of log btrees.
It is only initialized to zero as the module is loaded. This means that
reloading the module, perhaps by rebooting, resets all the item versions
to 0 and can lead to newly written items being ignored in favour of
older existing items with greater versions from a previous mount.
To fix this we initialize the lock server's write_version to the
greatest of all the versions in items in log btrees. We add a field to
the log_trees struct which records the greatest version which is
maintained as we write out items in transactions. These are read by the
server as it starts.
Then lock recovery needs to include the write_version so that the
lock_server can be sure to set the next write_version past the greatest
version in the currently granted locks.
Signed-off-by: Zach Brown <zab@versity.com>
The log_trees structs store the data that is used by client commits.
The primary struct is communicated over the wire so it includes the rid
and nr that identify the log. The _val struct was stored in btree item
values and was missing the rid and nr because those were stored in the
item's key.
It's madness to duplicate the entire struct just to shave off those two
fields. We can remove the _val struct and store the main struct in item
values, including the rid and nr.
Signed-off-by: Zach Brown <zab@versity.com>
Add a test which makes sure that we don't initialize the lock server's
write version to a version less than existing log tree items.
Signed-off-by: Zach Brown <zab@versity.com>
Audit code for structs allocated on stack without initialization, or
using kmalloc() instead of kzalloc().
- avl.c: zero padding in avl_node on insert.
- btree.c: Verify item padding is zero, or WARN_ONCE.
- inode.c: scoutfs_inode contains scoutfs_timespecs, which have padding.
- net.c: zero pad in net header.
- net.h: scoutfs_net_addr has padding, zero it in scoutfs_addr_from_sin().
- xattr.c: scoutfs_xattr has padding, zero it.
- forest.c: item_root in forest_next_hint() appears to either be
assigned-to or unused, so no need to zero it.
- key.h: Ensure padding is zeroed in scoutfs_key_set_{zeros,ones}
Signed-off-by: Andy Grover <agrover@versity.com>
Instead, explicitly add padding field, and adjust member ordering to
eliminate compiler-added padding between members, and at the end of the
struct (if possible: some structs end in a u8[0] array.)
This should prevent unaligned accesses. Not a big deal on x86_64, but
other archs like aarch64 really want this.
Signed-off-by: Andy Grover <agrover@versity.com>
This will ensure structs, which are internally 8 byte aligned, will remain
so when in the item cache.
16 bytes alignment doesn't seem like it's needed so just do 8.
Signed-off-by: Andy Grover <agrover@versity.com>
We were using a trailing owner offset to iterate over btree item values
from the back of the block towards the front. We did this to reclaim
fragmented free space in a block to satisfy an allocation instead of
having to split the block, which is expensive mostly because it has to
allocate and free metadata blocks.
In the before times, we used to compact items by sorting items by their
offset, moving them, and then sorting them by their keys again. The
sorting by keys was expensive so we added these owner offsets to be able
to compact without sorting.
But the complexity of maintaining the owner metadata is not worth it.
We can avoid the expensive sorting by keys by allocating a temporary
array of item offsets and sorting only it by the value offset. That's
nice and quick, it was the key comparisons that were expensive. Then we
can remove the owner offset entirely, as well as the block header final
free region that compaction needed.
And we also don't compact as often in the modern era because we do the
bulk of our work in the item cache instead of in the btree, and we've
changed the split/merge/compaction heuristics to avoid constantly
splitting/merging/comapcting and an item population happens to hover
right around a shared threshold.
Signed-off-by: Zach Brown <zab@versity.com>
dev.c includes linux/fs.h which includes linux/types.h, which defines
these types, __be16 etc. These are also defined in sparse.h, but I don't
think these are needed.
Definitions in linux/types.h includes stuff to set attr(bitwise) if
__CHECKER__ is defined, so we can remove __sp_biwise.
Signed-off-by: Andy Grover <agrover@versity.com>
The check for a small device didn't return an error code because it was
copied from error tests of ret for an error code. It has to generate
one, do so.
Signed-off-by: Zach Brown <zab@versity.com>
Add the df command which uses the new alloc_detail ioctl to show df for
the metadata and data devices separately.
Signed-off-by: Zach Brown <zab@versity.com>
Use little helpers to insert items into new single block btrees for
mkfs. We're about to insert a whole bunch more items.
Signed-off-by: Zach Brown <zab@versity.com>
Remove the old superblock fields which were used to track free blocks
found in the radix allocators. We now walk all the allocators when we
need to know the free totals, rather than trying to keep fields in sync.
Signed-off-by: Zach Brown <zab@versity.com>
Before the introduction of the AVL tree to sort btree items, the items
were sorted by sorting a small packed array of offsets. The final
offset in that array pointed to the item in the block with the greatest
key.
With the move to sorting items in an AVL tree by nodes embedded in item
structs, we now don't have the array of offsets and instead have a dense
array of items. Creation and deletion of items always works with the
final item in the array.
last_item() used to return the item with the greatest key by returning
the item pointed to by the final entry in the sorted offset array, then
it returned the final entry in the item array for creation and deletion
but that was no longer the item with the greatest key.
But spliting and joining still used last_item() to find the item in the
block with the greatest key for updating references to blocks in
parents. Since the introduction of the AVL tree splitting and joining
has been corrrupting the tree by setting parent block reference keys to
whatever item happened to be at the end of the array, not the item with
the greatest key.
The extent code recently pushed hard enough to hit this by working with
relatively random extent items in the core allocation btrees.
Eventually the parent block reference keys got out of sync and we'd fail
to find items by descending into the wrong children when looking for
them. Extent deletion hit this during allocation, returned -ENOENT, and
the allocator turned that into -ENOSPC.
With this fixed we can repetedly create and delte millions of files with
heavily fragmented extents in a tiny metadata device. Eventually it
actually runs out of space instead of spuriously returning ENOSPC in a
matter of minutes.
Signed-off-by: Zach Brown <zab@versity.com>
Andy GroverandZach Brown
Zach BrownandAndy Grover