Commit Graph

118 Commits

Author SHA1 Message Date
Zach Brown
c6fd807638 Use recov to manage lock recovery
Now that we have the recov layer we can have the lock server use it to
track lock recovery.  The lock server no longer needs its own recovery
tracking structures and can instead call recov.  We add a call for the
server to call to kick lock processing once lock recovery finishes.  We
can get rid of the persistent lock_client items now that the server is
driving recovery from the mounted_client items.

Signed-off-by: Zach Brown <zab@versity.com>
2021-04-13 12:10:35 -07:00
Andy Grover
efe5d92458 Reserve space in superblock for IPv6 addresses
Define a family field, and add a union for IPv4 and v6 variants, although
v6 is not supported yet.

Family field is now used to determine presence of address in a quorum slot,
instead of checking if addr is zero.

Signed-off-by: Andy Grover <agrover@versity.com>
2021-03-12 14:10:42 -08:00
Zach Brown
f18fa0e97a Update scoutfs print for centralized block_ref
Update scoutfs print to use the new block_ref struct instead of the
handful of per-block type ref structs that we had accumulated.

Signed-off-by: Zach Brown <zab@versity.com>
2021-03-01 09:49:17 -08:00
Zach Brown
57f34e90e9 Use mounted_client item as sign of farewell
As clients unmount they send a farewell request that cleans up
persistent state associated with the mount.  The client needs to be sure
that it gets processed, and we must maintain a majority of quorum
members mounted to be able to elect a server to process farewell
requests.

We had a mechanism using the unmount_barrier fields in the greeting and
super_block to let the final unmounting quorum majority know that their
farewells have been processed and that they didn't need to keep trying
to reconnect.

But we missed that we also need this out of band farewell handling
signal for non-quorum member clients as well.  The server can send
farewells to a non-member client as well as the final majority and then
tear down all the connections before the non-quorum client can see its
farewell response.  It also needs to be able to know that its farewell
has been processed before the server let the final majority unmount.

We can remove the custom unmount_barrier method and instead have all
unmounting clients check for their mounted_client item in the server's
btree.  This item is removed as the last step of farewell processing so
if the client sees that it has been removed it knows that it doesn't
need to resend the farewell and can finish unmounting.

This fixes a bug where a non-quorum unmount could hang if it raced with
the final majority unmounting.  I was able to trigger this hang in our
tests with 5 mounts and 3 quorum members.

Signed-off-by: Zach Brown <zab@versity.com>
2021-02-22 13:28:38 -08:00
Zach Brown
87fcad5428 Update scoutfs mkfs and print for quorum slots
Signed-off-by: Zach Brown <zab@versity.com>
2021-02-22 13:28:38 -08:00
Andy Grover
d731c1577e Filesystem version instead of format hash check
Instead of hashing headers, define an interop version. Do not mount
superblocks that have a different version, either higher or lower.

Since this is pretty much the same as the format hash except it's a
constant, minimal code changes are needed.

Initial dev version is 0, with the intent that version will be bumped to
1 immediately prior to tagging initial release version.

Update README. Fix comments.

Add interop version to notes and modinfo.

Signed-off-by: Andy Grover <agrover@versity.com>
2021-01-15 10:53:00 -08:00
Andy Grover
d48b447e75 Do not set -Wpadded except for checking kmod-shared headers
Remove now-unneeded manual padding in arg structs.

Signed-off-by: Andy Grover <agrover@versity.com>
2021-01-12 16:29:42 -08:00
Andy Grover
e0a2175c2e Use argp info instead of duplicating for cmd_register()
Make it static and then use it both for argp_parse as well as
cmd_register_argp.

Split commands into five groups, to help understanding of their
usefulness.

Mention that each command has its own help text, and that we are being
fancy to keep the user from having to give fs path.

Signed-off-by: Andy Grover <agrover@versity.com>
2021-01-12 16:29:42 -08:00
Andy Grover
7befc61482 Implement argp support for mkfs and add --force
Support max-meta-size and max-data-size using KMGTP units with rounding.

Detect other fs signatures using blkid library.

Detect ScoutFS super using magic value.

Move read_block() from print.c into util.c since blkid also needs it.

Signed-off-by: Andy Grover <agrover@versity.com>
2021-01-12 16:29:42 -08:00
Andy Grover
6b5ddf2b3a Implement argp support for print
Print warning if printing a data dev, you probably wanted the meta dev.

Change read_block to return err value. Otherwise there are confusing
ENOMEM messages when pread() fails. e.g. try to print /dev/null.

Signed-off-by: Andy Grover <agrover@versity.com>
2021-01-12 10:47:47 -08:00
Andy Grover
8f72d16609 scoutfs-utils: Use separate block devices for metadata and data
mkfs: Take two block devices as arguments. Write everything to metadata
dev, and the superblock to the data dev. UUIDs match. Differentiate by
checking a bit in a new "flags" field in the superblock.

Refactor device_size() a little. Convert spaces to tabs.

Move code to pretty-print sizes to dev.c so we can use it in error
messages there, as well as in mkfs.c.

print: Include flags in output.

Add -D and -M options for setting max dev sizes

Allow sizes to be specified using units like "K", "G" etc.

Note: -D option replaces -S option, and uses above units rather than
the number of 4k data blocks.

Update man pages for cmdline changes.

Signed-off-by: Andy Grover <agrover@versity.com>
2020-11-19 11:41:54 -08:00
Zach Brown
66c6331131 scoutfs-utils: add max item vers to log trees
Add a field to the log_trees struct which records the greatest item
version seen in items in the tree.

Signed-off-by: Zach Brown <zab@versity.com>
2020-10-30 11:12:52 -07:00
Zach Brown
42bf0980b6 scoutfs-utils: remove scoutfs_log_trees_val
We're just using the one log_trees struct for both network messages and
persistent btree item values.

Signed-off-by: Zach Brown <zab@versity.com>
2020-10-30 11:12:52 -07:00
Andy Grover
5701182665 scoutfs-utils: Enable -Wpadded
The compiler will complain if it sees any padding.

Fix a spot in print.c for this.

Signed-off-by: Andy Grover <agrover@versity.com>
2020-10-29 14:15:22 -07:00
Zach Brown
6b1dd980f0 scoutfs-utils: remove btree item owner
We no longer have an owner offset trailing btree item values.

Signed-off-by: Zach Brown <zab@versity.com>
2020-10-29 14:15:22 -07:00
Zach Brown
ea7c41d876 scoutfs-utils: remove free_*_blocks super fields
The kernel is no longer storing the total free space in all allocators
in super block fields.

Signed-off-by: Zach Brown <zab@versity.com>
2020-10-26 15:19:41 -07:00
Zach Brown
838e293413 scoutfs-utils: update compaction item printing
We now only use one srch file compaction struct and we store it in
PENDING and BUSY key types.

Signed-off-by: Zach Brown <zab@versity.com>
2020-10-26 15:19:41 -07:00
Zach Brown
23711f05f6 scoutfs-utils: alloc and data uses full extents
Signed-off-by: Zach Brown <zab@versity.com>
2020-10-26 15:19:41 -07:00
Zach Brown
e2a919492d scoutfs-utils: remove unused xattr index items
We're now using the .srch. xattr tags.

Signed-off-by: Zach Brown <zab@versity.com>
2020-08-26 14:39:28 -07:00
Zach Brown
f04a636229 scoutfs-utils: add support for srch
Signed-off-by: Zach Brown <zab@versity.com>
2020-08-26 14:39:28 -07:00
Zach Brown
5f0dbc5f85 scoutfs-utils: remove radix _first fields
The recent cleanup of the radix allocator included removing tracking of
the first set bits or references in blocks.

Signed-off-by: Zach Brown <zab@versity.com>
2020-08-26 14:39:28 -07:00
Zach Brown
39993d8b5f scoutfs-utils: use larger metadata blocks
Signed-off-by: Zach Brown <zab@versity.com>
2020-08-26 14:39:28 -07:00
Zach Brown
b86a1bebbb scoutfs-utils: support btree avl and hash
Update the internal structure of btree blocks to use the avl item index
and hash table direct item lookup.

Signed-off-by: Zach Brown <zab@versity.com>
2020-08-26 14:39:28 -07:00
Zach Brown
aa84f7c601 scoutfs-utils: use scoutfs_key as btree key
Track the kernel changes to use the scoutfs_key struct as the btree key
instead of a big-endian binary blob.

Signed-off-by: Zach Brown <zab@versity.com>
2020-08-26 14:39:28 -07:00
Zach Brown
91c64dfa2d scoutfs-utils: print packed extents
Add support for printing the invidual extents stored in packed extent
items.

Signed-off-by: Zach Brown <zab@versity.com>
2020-03-02 12:11:39 -08:00
Zach Brown
ec782fff8d scoutfs-utils: meta and data free blocks
The super block now tracks free metadata and data blocks in separate
counters.

Signed-off-by: Zach Brown <zab@versity.com>
2020-02-25 12:04:17 -08:00
Zach Brown
6b66e583f2 scoutfs-utils: fix printing block hdr fields
The block header printing helper had the identifiers for the blkno and
seq in the format string swapped.

Signed-off-by: Zach Brown <zab@versity.com>
2020-02-25 12:04:17 -08:00
Zach Brown
ff436db49b scoutfs-utils: add support for radix alloc
Add support for initializing radix allocator blocks that describe free
space in mkfs and support for printing them out.

Signed-off-by: Zach Brown <zab@versity.com>
2020-02-25 12:04:17 -08:00
Zach Brown
e0a49c46a7 scoutfs-utils: add packed extents and bitmaps
Signed-off-by: Zach Brown <zab@versity.com>
2020-01-17 11:22:04 -08:00
Zach Brown
3776c18c66 scoutfs-utils: switch to btree forest
Remove all the lsm code from mkfs and print, replacing
it with the forest of btrees.

Signed-off-by: Zach Brown <zab@versity.com>
2020-01-17 11:22:04 -08:00
Zach Brown
7cd8738add scoutfs-utils: net uses rid instead of node_id
Now that networking is identifing clients by their rid some persistent
structures are using that to store records of clients.

Signed-off-by: Zach Brown <zab@versity.com>
2019-08-20 15:52:17 -07:00
Zach Brown
3670a5b80d scoutfs-utils: remove quorum slot config
The format no longer has statically configured named slots.  The only
persistent config is the number of monts that must be voting to reach
quorum.  The quorum blocks now have a log of successfull elections.

Signed-off-by: Zach Brown <zab@versity.com>
2019-08-20 15:52:17 -07:00
Zach Brown
fc15b816b0 scoutfs-utils: update format for rid
Signed-off-by: Zach Brown <zab@versity.com>
2019-08-20 15:52:17 -07:00
Zach Brown
674224d454 scoutfs-utils: hidden and indexed xattrs
Add support for the xattr tags which can hide or index xattrs by their
name.  We get an item that indexes inodes by the presence of an xattr, a
listxattr_raw ioctl which can show hidden xattrs, and an ioctl that
finds inodes which have an xattr.

Signed-off-by: Zach Brown <zab@versity.com>
2019-06-24 10:08:35 -07:00
Zach Brown
77bd0c20ab scoutfs-utils: add flags to quorum block
Signed-off-by: Zach Brown <zab@versity.com>
2019-04-12 10:54:20 -07:00
Zach Brown
514418421c scoutfs-utils: add support for unmount_barrier
Signed-off-by: Zach Brown <zab@versity.com>
2019-04-12 10:54:20 -07:00
Zach Brown
3c9eeeb2ef scoutfs-utils: add transaction seq btree
Signed-off-by: Zach Brown <zab@versity.com>
2019-04-12 10:54:20 -07:00
Zach Brown
3d64c46fcd scoutfs-utils: add lock clients btree
Show the lock client btree entries in print.

Signed-off-by: Zach Brown <zab@versity.com>
2019-04-12 10:54:20 -07:00
Zach Brown
64bdda717c scoutfs-utils: move super id to block hdr magic
Move the magic value that identifies the super block into the block
header and use it for btree blocks as well.

Signed-off-by: Zach Brown <zab@versity.com>
2019-04-12 10:54:20 -07:00
Zach Brown
ea969a5dde scoutfs-utils: update format.h for quorum
Signed-off-by: Zach Brown <zab@versity.com>
2019-04-12 10:54:20 -07:00
Zach Brown
bbfa71361f scoutfs-utils: compaction request format update
Signed-off-by: Zach Brown <zab@versity.com>
2018-08-28 15:34:33 -07:00
Zach Brown
078d2f6073 scoutfs-utils: update format for greeting node_id
Signed-off-by: Zach Brown <zab@versity.com>
2018-08-28 15:34:33 -07:00
Zach Brown
ea2ec838ec scoutfs-utils: use one super and verify its crc
Signed-off-by: Zach Brown <zab@versity.com>
2018-06-29 15:56:36 -07:00
Zach Brown
cfc8cb8800 scoutfs-utils: support server extent allocation
Signed-off-by: Zach Brown <zab@versity.com>
2018-06-29 14:42:08 -07:00
Zach Brown
35e4ab92f0 scoutfs-utils: support file and node free extents
Add support for printing the items used to track file mapping extents
and free extents.

Signed-off-by: Zach Brown <zab@versity.com>
2018-06-29 14:42:08 -07:00
Zach Brown
8e6c18a0fa scoutfs-utils: support small keys
Make the changes to support the new small key struct.  mkfs and print
work with simpler keys, segment items, and manifest entries.  The item
cache keys ioctl now just needs to work with arrays of keys.

Signed-off-by: Zach Brown <zab@versity.com>
2018-04-04 09:15:54 -05:00
Zach Brown
0770cc8c57 scoutfs-utils: support single dirent format
Signed-off-by: Zach Brown <zab@versity.com>
2018-04-04 09:15:54 -05:00
Zach Brown
8119a56c92 scoutfs-utils: update format for xattr cleanups
xattr items are now stored at the hash of the name and have a header in
the first part.

Signed-off-by: Zach Brown <zab@versity.com>
2018-03-15 09:23:53 -07:00
Zach Brown
2527b4906e scoutfs-utils: remove inode blocks field
It's the sum of oneline and offline and is redundant.

Signed-off-by: Zach Brown <zab@versity.com>
2018-02-21 09:36:49 -08:00
Zach Brown
e68a999ed5 scoutfs-utils: remove locks command
scoutfs now directly uses the kernel dlm subsystem and offsers a debugfs
file with the current lock state.  We don't need userspace to read and
format the contents of a debugging file.

Signed-off-by: Zach Brown <zab@versity.com>
2018-02-14 15:01:16 -08:00