Commit Graph

102 Commits

Author SHA1 Message Date
Zach Brown
7bbe49fde2 scoutfs-utils: sort keys by zones, then types
Our item cache protocol is tied to holding DLM locks which cover a
region of the item namespace.  We want locks to cover all the data
associated with an inode and other locks to cover the indexes.  So we
resort the items first by major (index, fs) then by inode type (inode,
dirent, etc).

Signed-off-by: Zach Brown <zab@versity.com>
2017-07-19 10:33:12 -07:00
Zach Brown
6c37e3dee0 scoutfs-utils: add btree ring storage
Manifest entries and segment allocation bitmap regions are now stored in
btree items instead of the ring log.  This lets us work with them
incrementally and share them between nodes.

Signed-off-by: Zach Brown <zab@versity.com>
2017-07-17 13:43:37 -07:00
Zach Brown
d78649e065 scoutfs-utils: support symlink item with nr field
Signed-off-by: Zach Brown <zab@versity.com>
2017-06-27 14:08:50 -07:00
Zach Brown
6ae8e9743f scoutfs-utils: add support for skip list segments
Signed-off-by: Zach Brown <zab@versity.com>
2017-06-27 14:07:25 -07:00
Zach Brown
c6eaccbf90 scoutfs-utils: add item cache keys commands
Add ioctls to get the keys for cached ranges and items and print them.

Signed-off-by: Zach Brown <zab@versity.com>
2017-06-15 22:22:10 -07:00
Zach Brown
51ae302d81 scoutfs-utils: add key printing
Just lift the key printer from the kernel and use it to print
item keys in segments and in manifest entries.

Signed-off-by: Zach Brown <zab@versity.com>
2017-06-15 21:54:04 -07:00
Zach Brown
228c5d8b4b scoutfs-utils: support meta and data seqs
Signed-off-by: Zach Brown <zab@versity.com>
2017-05-23 12:17:52 -07:00
Zach Brown
08aaa5b430 scoutfs-utils: add stat command
The scoutfs stat command is modeled after stat(1) and uses the STAT_MORE
ioctl.

Signed-off-by: Zach Brown <zab@versity.com>
2017-05-23 12:17:50 -07:00
Zach Brown
9fc99a8c31 scoutfs-utils: add support for inode index items
Add support for the inode index items which are replacing the seq walks
from the old btree structures.  We create the index items for the root
inode, can print out the items, and add a commmand to walk the indices.

Signed-off-by: Zach Brown <zab@versity.com>
2017-05-16 11:14:10 -07:00
Zach Brown
9c00602051 scoutfs-utils: print extent flags
Signed-off-by: Zach Brown <zab@versity.com>
2017-05-16 11:14:10 -07:00
Zach Brown
a9cb464d49 scoutfs-utils: rename __bitwise
Recent kernel headers have leaked __bitwise into userspace.  Rename our
use of __bitwise in userspace sparse builds to avoid the collision.

Signed-off-by: Zach Brown <zab@versity.com>
2017-05-16 11:14:10 -07:00
Zach Brown
4585d57153 scoutfs-utils: only print recent super
It's a bit confusing to always see both the old and current super block.
Let's only print the first one.  We could add an argument to print all
of them.

Signed-off-by: Zach Brown <zab@versity.com>
2017-05-16 11:13:59 -07:00
Zach Brown
1c9a407059 scoutfs-utils: print extent items
Signed-off-by: Zach Brown <zab@versity.com>
2017-05-16 11:13:47 -07:00
Zach Brown
e09a216762 Support simpler ring entries
Add mkfs and print support for the simpler rings that the segment bitmap
allocator and manifest are now using.  Some other recent format header
updates come along for the ride.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
bd54995599 Add a simple native bitmap
Nothing fancy at all.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
f86ce74ffd Add BITS_PER_LONG define
Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
a147239022 Remove dead block, btree, and buddy code
Remove the last bits of the dead code from the old btree design.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
2e2ee3b2f1 Print symlink items
Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
77d0268cb2 Add printing xattrs
For now we only print the xattr names, not the values.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
13b2d9bb88 Remove find_xattr commands
We're no longer maintaining xattr backrefs.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
02993a2dd7 Update ino_path for the large cursor
Previously we could iterate over backref items with a small u64.  Now we
need a larger opaque buffer.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
16da3c182a Add printing link backref items
Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
acda5a3bf1 Add support for free_segs in super
The allocator records the total number of free segments in the super
block.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
44f8551fb6 Print data items
Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
52291b2c75 Update format for readdir_pos
We now track each parent dir's next readdir pos and the readdir pos of
each dirent.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
38c8a4901f Print orphan items
Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
c4f2563cc1 Update tools to new segment item layout
The segment item struct used to have fiddly packed offsets and lengths.
Now it's just normal fields so we can work with them directly and get
rid of the native item indirection.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
e81c256a22 Remove the bitops helpers
We don't have any use for the bitops today, we'll resurrect this in
simpler form if it's needed again.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
34c62824e5 Use a treap walker to print segments
We were using a bitmap to record segments during manifest printing and
then walking that bitmap to print segments.  It's a little silly to have
a second data structure record the referenced segments when we could
just walk the manifest again to print the segments.

So refactor node printing into a treap walker that calls a function for
each node.  Then we can have functions that print the node data
structurs for each treap and then one that prints the segments that are
referenced by manifest nodes.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
26a4266964 Set manifest keys to precise segment keys
We had changed the manifest keys to fully cover the space around the
segments in the hopes that it'd let item reading easily find negative
cached regions around items.

But that makes compaction think that segments intersect with items when
they really don't.  We'd much rather avoid unnecessary compaction by
having the manifest entries precisely reflect the keys in the segment.

Item reading can do more work at run time to find the bounds of the key
space that are around the edges of the segments it works with.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
c2b47d84c1 Add next_seg_seq field to super
Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
484b34057a Update mkfs and print for treap ring
Update mkfs and print now that the manifest and allocator are stored in
treaps in the ring.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
7c4bc528c6 Make sure manifests cover all keys
Make sure that the manifest entries for a given level fully
cover the possible key space.  This helps item reading describe
cached key ranges that extend around items.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:42 -07:00
Zach Brown
c3b6dd0763 Describe ring log with index,nr
Update mkfs and print to describe the ring blocks with a starting index
and number of blocks instead of a head and tail index.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:42 -07:00
Zach Brown
19b674cb38 Print dirent and readdir items
Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:42 -07:00
Zach Brown
7cd70ab2bb Don't double increment segno when printing
Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:42 -07:00
Zach Brown
818e149643 Update mkfs and print for lsm writing
Adapt mkfs and print for the format changes made to support writing
segments.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:42 -07:00
Zach Brown
eb4baa88f5 Print LSM structures
Print segments and their items instead of btree blocks.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:29 -07:00
Zach Brown
c96b833a36 mkfs LSM segment and ring stuctures
Make a new file system by writing a root inode in a segment and storing
a manifest entry in the ring that references the segment.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:02 -07:00
Zach Brown
9d3fe27929 Add data_since command
Add a data_since command that operates just like inodes-since.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-18 10:52:25 -08:00
Zach Brown
5fcf70b53e Catch up to kernel's scoutfs_extent
Signed-off-by: Zach Brown <zab@versity.com>
2016-11-17 19:48:05 -08:00
Zach Brown
41e3ca0f41 Consistently use __u8 in format.h
Signed-off-by: Zach Brown <zab@versity.com>
2016-11-17 19:47:39 -08:00
Zach Brown
ec702b9bb3 Update the data_version ioctl to return the u64
We updated the code to use the new iteration of the data_version ioctl
but we forgot to update the ioctl definition so it didn't actually work.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-17 15:45:56 -08:00
Zach Brown
c3f122a5f1 Fix mkfs buddy initialization
mkfs was starting setting free blk bits from 0 instead of from
the blkno offset of the first free block.  This resulted in
the highest order above a used blkno being marked free.  Freeing
that blkno would set its lowest order blkno.  Now that blkno can be
allocated from two orders.  That, eventually, can lead to blocks
being doubly allocated and users trampling on each other.

While auditing the code to chase this bug down I also noticed that
write_buddy_blocks() was using a min() that makes no sense at all.  Here
'blk' is inclusive, the modulo math works on its own.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-17 15:43:23 -08:00
Zach Brown
932b0776d1 Add commands for working with offline data
Add the data_version, stage, and release commands for working with
offline extents of file data.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-16 14:40:09 -08:00
Zach Brown
22140c93d1 Print extents instead of bmap items
Print the extent items now that we're not using bmap items any more.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-16 14:40:04 -08:00
Zach Brown
c2cfb0227f Print the new inode data_version field
We've added a data_version field to the inode for tracking changes to
the file's data.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-16 14:39:39 -08:00
Zach Brown
f1d8955303 Support the ino_path ioctl
We updated the inode_paths ioctl to return one path and use a counter
cursor and renamed it to ino_path.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-16 14:39:00 -08:00
Zach Brown
fb16af7b7d btree nr_items is now a le16
The btree block now has a le16 nr_items field to make room for the
number of items that larger blocks can hold.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-16 14:38:04 -08:00
Zach Brown
cd0d045c93 Add support for full radix buddy blocks
Update mkfs and print for the full radix buddy allocators.  mkfs has to
calculate the number of blocks and the height of the tree and has to
initialize the paths down the left and right side of the tree.
Print needs to dump the new radix blockx and super block fields.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-04 14:16:25 -07:00