Commit Graph

77 Commits

Author SHA1 Message Date
Zach Brown
38c8a4901f Print orphan items
Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
c4f2563cc1 Update tools to new segment item layout
The segment item struct used to have fiddly packed offsets and lengths.
Now it's just normal fields so we can work with them directly and get
rid of the native item indirection.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
e81c256a22 Remove the bitops helpers
We don't have any use for the bitops today, we'll resurrect this in
simpler form if it's needed again.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
34c62824e5 Use a treap walker to print segments
We were using a bitmap to record segments during manifest printing and
then walking that bitmap to print segments.  It's a little silly to have
a second data structure record the referenced segments when we could
just walk the manifest again to print the segments.

So refactor node printing into a treap walker that calls a function for
each node.  Then we can have functions that print the node data
structurs for each treap and then one that prints the segments that are
referenced by manifest nodes.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
26a4266964 Set manifest keys to precise segment keys
We had changed the manifest keys to fully cover the space around the
segments in the hopes that it'd let item reading easily find negative
cached regions around items.

But that makes compaction think that segments intersect with items when
they really don't.  We'd much rather avoid unnecessary compaction by
having the manifest entries precisely reflect the keys in the segment.

Item reading can do more work at run time to find the bounds of the key
space that are around the edges of the segments it works with.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
c2b47d84c1 Add next_seg_seq field to super
Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
484b34057a Update mkfs and print for treap ring
Update mkfs and print now that the manifest and allocator are stored in
treaps in the ring.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:43 -07:00
Zach Brown
7c4bc528c6 Make sure manifests cover all keys
Make sure that the manifest entries for a given level fully
cover the possible key space.  This helps item reading describe
cached key ranges that extend around items.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:42 -07:00
Zach Brown
c3b6dd0763 Describe ring log with index,nr
Update mkfs and print to describe the ring blocks with a starting index
and number of blocks instead of a head and tail index.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:42 -07:00
Zach Brown
19b674cb38 Print dirent and readdir items
Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:42 -07:00
Zach Brown
7cd70ab2bb Don't double increment segno when printing
Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:42 -07:00
Zach Brown
818e149643 Update mkfs and print for lsm writing
Adapt mkfs and print for the format changes made to support writing
segments.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:42 -07:00
Zach Brown
eb4baa88f5 Print LSM structures
Print segments and their items instead of btree blocks.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:29 -07:00
Zach Brown
c96b833a36 mkfs LSM segment and ring stuctures
Make a new file system by writing a root inode in a segment and storing
a manifest entry in the ring that references the segment.

Signed-off-by: Zach Brown <zab@versity.com>
2017-04-18 14:20:02 -07:00
Zach Brown
9d3fe27929 Add data_since command
Add a data_since command that operates just like inodes-since.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-18 10:52:25 -08:00
Zach Brown
5fcf70b53e Catch up to kernel's scoutfs_extent
Signed-off-by: Zach Brown <zab@versity.com>
2016-11-17 19:48:05 -08:00
Zach Brown
41e3ca0f41 Consistently use __u8 in format.h
Signed-off-by: Zach Brown <zab@versity.com>
2016-11-17 19:47:39 -08:00
Zach Brown
ec702b9bb3 Update the data_version ioctl to return the u64
We updated the code to use the new iteration of the data_version ioctl
but we forgot to update the ioctl definition so it didn't actually work.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-17 15:45:56 -08:00
Zach Brown
c3f122a5f1 Fix mkfs buddy initialization
mkfs was starting setting free blk bits from 0 instead of from
the blkno offset of the first free block.  This resulted in
the highest order above a used blkno being marked free.  Freeing
that blkno would set its lowest order blkno.  Now that blkno can be
allocated from two orders.  That, eventually, can lead to blocks
being doubly allocated and users trampling on each other.

While auditing the code to chase this bug down I also noticed that
write_buddy_blocks() was using a min() that makes no sense at all.  Here
'blk' is inclusive, the modulo math works on its own.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-17 15:43:23 -08:00
Zach Brown
932b0776d1 Add commands for working with offline data
Add the data_version, stage, and release commands for working with
offline extents of file data.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-16 14:40:09 -08:00
Zach Brown
22140c93d1 Print extents instead of bmap items
Print the extent items now that we're not using bmap items any more.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-16 14:40:04 -08:00
Zach Brown
c2cfb0227f Print the new inode data_version field
We've added a data_version field to the inode for tracking changes to
the file's data.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-16 14:39:39 -08:00
Zach Brown
f1d8955303 Support the ino_path ioctl
We updated the inode_paths ioctl to return one path and use a counter
cursor and renamed it to ino_path.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-16 14:39:00 -08:00
Zach Brown
fb16af7b7d btree nr_items is now a le16
The btree block now has a le16 nr_items field to make room for the
number of items that larger blocks can hold.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-16 14:38:04 -08:00
Zach Brown
cd0d045c93 Add support for full radix buddy blocks
Update mkfs and print for the full radix buddy allocators.  mkfs has to
calculate the number of blocks and the height of the tree and has to
initialize the paths down the left and right side of the tree.
Print needs to dump the new radix blockx and super block fields.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-04 14:16:25 -07:00
Zach Brown
40b9f19ec4 Add bitops.c for find_next_bit_le()
The upcoming buddy changes are going to need a find_next_bit_le().

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-04 14:16:25 -07:00
Zach Brown
871db60fb2 Add U16_MAX
Add a simple U16_MAX define for upcoming buddy changes.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-04 14:16:25 -07:00
Zach Brown
a901db2ff7 Print seqs in bmap items
The bmap items now have the sequence number that wrote each mapped
block.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-04 14:16:25 -07:00
Zach Brown
b436772376 Add orphan key
Printing the raw item is enough, it doesn't have a value to decode.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-04 14:16:25 -07:00
Zach Brown
e6222223c2 Update format.h for kernel code helpers
Update format.h for some format defines that have so far only been used
by the kernel code.

Signed-off-by: Zach Brown <zab@versity.com>
2016-11-04 14:16:22 -07:00
Zach Brown
0dff7f55a6 Use openssl for pseudo random bytes
The pseudo random byte wrapper function used the intel instructions
so that it could deal with high call rates, like initializing random
node priorities for a large treap.

But this is obviously not remotely portable and has the annoying habit
of tripping up versions of valgrind that haven't yet learned about these
instructions.

We don't actually have high bandwidth callers so let's back off and just
let openssl take care of this for us.

Signed-off-by: Zach Brown <zab@versity.com>
2016-09-27 09:47:50 -07:00
Zach Brown
4ccb80a8ec Initialize all the buddy slot free order fields
Initialize the free_order field in all the slots of the buddy index
block so that the kernel will try to allocate from them and will
initialize and populate the first block.

Signed-off-by: Zach Brown <zab@versity.com>
2016-09-08 16:40:39 -07:00
Zach Brown
86ffdf24a2 Add symlink support
Print out the raw symlink items.

Signed-off-by: Zach Brown <zab@versity.com>
2016-08-29 10:25:46 -07:00
Zach Brown
a89f6c10b1 Add buddy indirect order totals
The total counts of all the set order bits in all the child buddy blocks
is needed for statfs.

Signed-off-by: Zach Brown <zab@versity.com>
2016-08-23 16:41:57 -07:00
Zach Brown
2f91a9a735 Make command listing less noisy
It's still not great, but at least it's a little clearer.

Signed-off-by: Zach Brown <zab@versity.com>
2016-08-23 12:31:03 -07:00
Zach Brown
c17a7036ed Add find xattr commands
Add commands that use the find-xattr ioctls to show the inode numbers of
inodes which probably contain xattrs matching the specified name or
value.

Signed-off-by: Zach Brown <zab@versity.com>
2016-08-23 12:21:47 -07:00
Zach Brown
43619a245d Add inode-paths via link backrefs
Add the inode-paths command which uses the ioctl to display all the
paths that lead to the given inode.  We add support for printing
the new link backref items and inode and dirent fields.

Signed-off-by: Zach Brown <zab@versity.com>
2016-08-11 16:50:57 -07:00
Zach Brown
be4a137479 Add support for printing block map items
Signed-off-by: Zach Brown <zab@versity.com>
2016-08-10 15:19:09 -07:00
Zach Brown
25e3b03d94 Add support for simpler btree block
Update mkfs and print to the new simpler btree block format.

Signed-off-by: Zach Brown <zab@versity.com>
2016-08-02 13:31:06 -07:00
Zach Brown
0af40547b5 Update to smaller block size
We're going to try using a smaller fixed block size to reduce complexity
in the file data extent code.

Signed-off-by: Zach Brown <zab@versity.com>
2016-08-02 13:30:40 -07:00
Zach Brown
6a97aa3c9a Add support for the radix buddy bitmaps
Update mkfs and print to support the buddy allocator that's indexed by
radix blocks.

Signed-off-by: Zach Brown <zab@versity.com>
2016-08-02 13:29:51 -07:00
Zach Brown
4b86256904 Ignore sparse warning for builtin fpclassify
Signed-off-by: Zach Brown <zab@versity.com>
2016-07-27 13:56:01 -07:00
Zach Brown
99167f6d66 Expand little endian bitops functions
We had the start of functions that operated on little endian bitmaps.
This adds more operations and uses __packed to support unaligned bitmaps
on platforms where unaligned accesses are a problem.

Signed-off-by: Zach Brown <zab@versity.com>
2016-07-27 13:50:51 -07:00
Zach Brown
c48e08a378 Add -fno-strict-aliasing
We modify the same memory through pointers of different types all the
live long day.

Signed-off-by: Zach Brown <zab@versity.com>
2016-07-27 13:49:27 -07:00
Zach Brown
1cacc50de0 Remove old unused lebitmap code
Signed-off-by: Zach Brown <zab@versity.com>
2016-07-22 15:04:07 -07:00
Zach Brown
fc37ece26b Remove homebrew tracing
Happily, it turns out that there are crash extensions for extracting
trace messages from crash dumps.  That's good enough for us.

Signed-off-by: Zach Brown <zab@versity.com>
2016-07-22 13:54:10 -07:00
Zach Brown
54044508fa Add inodes-since command
The kernel now has an ioctl to give us inode numbers with their sequence
number for every inode that's been modified since a given tree update
sequence number.

Update mkfs and print to the on-disk format changes and add a trivial
inodes-since command which calls the ioctl and prints the results.

Signed-off-by: Zach Brown <zab@versity.com>
2016-07-05 17:49:13 -04:00
Zach Brown
a069bdd945 Add format header updates for xattrs
The kernel now has items and structs for xattrs.

Signed-off-by: Zach Brown <zab@versity.com>
2016-07-04 11:02:07 -07:00
Zach Brown
d774e5308b Add support for printing traces from files
We can extract the formats and records from a crash dump and
print them from the files.

Signed-off-by: Zach Brown <zab@versity.com>
2016-05-28 12:41:30 -07:00
Zach Brown
54867b0f9c Add support for printing kernel traces
Add a 'trace' command which uses the debugfs file created by the scoutfs
kernel module to read and print trace messages.

Signed-off-by: Zach Brown <zab@versity.com>
2016-05-28 11:10:08 -07:00