Commit Graph

24 Commits

Author SHA1 Message Date
Hunter Shaffer
2be15d416d Add Quota support
Adds a function to to insert quota rules as filesystem items.
This will then have an outward facing function that takes a
writer and a mirror of the _squota_rule struct in quota.c
and is called _parallel_restore_quota_rule. Adds testing
to make sure we are restoring a test quota.

Signed-off-by: Hunter Shaffer <hunter.shaffer@versity.com>
2024-10-17 12:01:04 -07:00
Hunter Shaffer
0d910eb7ab Check if source device has been mounted
The filesystem we are restoring into needs to be empty
and never mounted. Here we check the all of the quorum blocks
timestamps to see whether the device we are restoring into has
been mounted before. Adds a test in the test script that attempts
to restore a previously mounted device.

Signed-off-by: Hunter Shaffer <hunter.shaffer@versity.com>
Signed-off-by: Auke Kok <auke.kok@versity.com>
2024-10-17 12:01:04 -07:00
Auke Kok
130e10626d Copy a tree using parallel restore library.
This tool compies a source tree (whether it's scoutfs or not)
into an offline scoutfs meta device. It has only those 2 parameters
and does a single-process walk of the tree to restore all items
while preservice as much of the metadata as possible.

Signed-off-by: Auke Kok <auke.kok@versity.com>
2024-10-17 11:42:16 -07:00
Auke Kok
281cd4f87a Create a 4k offline extent for each regular file.
After this change, all files have a single offline extent:

```
$ sudo src/filefrag-gc57857a5 -b4096 -v /mnt/scratch/top-0/file-1094
Filesystem type is: 554f4353
File size of /mnt/scratch/top-0/file-1094 is 4096 (1 block of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..       0:          0..         0:      1:             last,unknown_loc,eof
/mnt/scratch/top-0/file-1094: 1 extent found
```

Signed-off-by: Auke Kok <auke.kok@versity.com>
2024-10-17 11:42:16 -07:00
Auke Kok
c78b5cdecc Detect child process exiting with errors.
Adds a signal handler for SIGCHLD and sets up a signal handler with
SA_SIGINFO. This way we can inspect the exit code emitted by the
child and abort processing when a child process exits with an error.

Without this handler, any child process that exits with e.g. ENOSPC
will keep the parent hanging indefinitely.

Signed-off-by: Auke Kok <auke.kok@versity.com>
2024-10-17 11:42:16 -07:00
Auke Kok
6da7034d48 Pass meta_seq and data_seq to _restore_inode.
This allows callers to pass in seq values for generated inodes. The
tester code initializes them now before calling, instead of being
hard set in the library.

Signed-off-by: Auke Kok <auke.kok@versity.com>
2024-10-17 11:42:16 -07:00
Zach Brown
1ac0e5bfd3 Add test for parallel restore
Signed-off-by: Zach Brown <zab@versity.com>
2024-10-17 11:42:16 -07:00
Zach Brown
82c2d0b1d0 Add o_tmpfile_linkat test binary
Add a test binary that uses o_tmpfile and linkat to create a file in a
given dir.  We have something similar, but it's weirdly specific to a
given test.  This is a simpler building block that could be used by more
tests.

Signed-off-by: Zach Brown <zab@versity.com>
2024-06-28 15:09:05 -07:00
Zach Brown
bf21699ad7 bulk_create_paths test tool takes xattr name
Previously the bulk_create_paths test tool used the same xattr name for
each category of xattrs it was creating.

This created a problem where two tests got their xattrs confused with
each other.  The first test created a bunch of srch xattrs, failed, and
didn't clean up after itself.  The second test saw these search xattrs
as its own and got very confused when there were far more srch xattrs
than it thought it had created.

This lets each test specify the srch xattr names that are created by
bulk_create_paths so that tests can work with their xattrs independent
of each other.

Signed-off-by: Zach Brown <zab@versity.com>
2023-11-09 14:15:44 -08:00
Auke Kok
a7704e0b56 Allow the kernel to return -ESTALE from orphan-inode test
In newer kernels, we always get -ESTALE because the inode has been
marked immediately as deleting. Since this is expected behavior we
should not fail the test here on this error value.

Signed-off-by: Auke Kok <auke.kok@versity.com>
2023-10-09 15:35:40 -04:00
Zach Brown
9ba2ee5c88 Add testing of O_TMPFILE umask
There were kernels that didn't apply the current umask to inode modes
created with O_TMPFILE without acls.  Let's have a test running to make
sure that we're not surprised if we come across one.

Signed-off-by: Zach Brown <zab@versity.com>
2023-01-09 14:49:23 -08:00
Zach Brown
acb94dd9b7 Add test of large fragmented free lists
Add a test which gives the server a transaction with a free list block
that contains blknos that each dirty an individiaul btree blocks in the
global data free extent btree.

Signed-off-by: Zach Brown <zab@versity.com>
2022-07-29 11:25:01 -07:00
Zach Brown
90518a0fbd Add handle_fsetxattr test utility
Add a quick little binary that spins opening an inode by a handle and
calling fsetxattr.

Signed-off-by: Zach Brown <zab@versity.com>
2022-03-10 11:43:11 -08:00
Bryant G. Duffy-Ly
16ea0ef671 Add syscall wrapper for renameat2
Signed-off-by: Bryant G. Duffy-Ly <bduffyly@versity.com>
2021-11-19 17:54:08 -06:00
Zach Brown
932a842ae3 Remove valid_bytes from stat _more ioctls
The idea here was that we'd expand the size of the struct and
valid_bytes would tell the kernel which fields were present in
userspace's struct.  That doesn't combine well with the ioctl convention
of having the size of the type baked into the ioctl number.   We'll
remove this to make the world less surprising.  If we expand the
interface we'd add additional ioctls and types.

Signed-off-by: Zach Brown <zab@versity.com>
2021-10-28 12:30:47 -07:00
Zach Brown
a9baeab22e stage_tmpfile test gets current data_version
The stage_tmpfile test util was written when fallocate didn't update
data_version for size extensions.  It is more correct to get the
data_version after fallocate changes data_versions for however many
transactions, extent allocations, and i_size extensions it took to
allocate space.

Signed-off-by: Zach Brown <zab@versity.com>
2021-07-30 13:22:42 -07:00
Zach Brown
120c2d342a Add create_xattr_loop test tool
Add a quick tool that creates xattrs in a tight loop.

Signed-off-by: Zach Brown <zab@versity.com>
2021-07-07 14:13:14 -07:00
Andy Grover
0deb232d3f Support O_TMPFILE and allow MOVE_BLOCKS into released extents
Support O_TMPFILE: Create an unlinked file and put it on the orphan list.
If it ever gains a link, take it off the orphan list.

Change MOVE_BLOCKS ioctl to allow moving blocks into offline extent ranges.
Ioctl callers must set a new flag to enable this operation mode.

RH-compat: tmpfile support it actually backported by RH into 3.10 kernel.
We need to use some of their kabi-maintaining wrappers to use it:
use a struct inode_operations_wrapper instead of base struct
inode_operations, set S_IOPS_WRAPPER flag in i_flags. This lets
RH's modified vfs_tmpfile() find our tmpfile fn pointer.

Add a test that tests both creating tmpfiles as well as moving their
contents into a destination file via MOVE_BLOCKS.

xfstests common/004 now runs because tmpfile is supported.

Signed-off-by: Andy Grover <agrover@versity.com>
2021-04-05 14:23:44 -07:00
Zach Brown
88aefc381a scoutfs-tests: add find_xattrs
Add a utility that mimics our search_xattrs ioctl with directory entry
walking and fgetxattr as efficiently as it can so we can use it to test
large file populations.

Signed-off-by: Zach Brown <zab@versity.com>
2020-12-03 13:40:33 -08:00
Zach Brown
8982750266 scoutfs-tests: bulk create more clearly sets xattr
Just set the value using a single char, this messed up and set the size
of the pointer.

Signed-off-by: Zach Brown <zab@versity.com>
2020-12-03 13:40:33 -08:00
Zach Brown
85a27b2198 scoutfs-utils: per sec bulk_create_path banners
bulk_create_paths was inspired by createmany when it was outputting
status lines every 10000 files.  That's far too often if we're creating
files very quickly.  And it only tried to output a line after entire
directories, so output could stall for very large directories.

Behave more inline with vmstat, iostat, etc, and output a line at a
regular time interval.

Signed-off-by: Zach Brown <zab@versity.com>
2020-08-26 14:39:39 -07:00
Zach Brown
c1229644da scoutfs-tests: create xattrs in bulk_create_paths
Add options to bulk_create_paths for creating xattrs as we create files.
We can create normal xattrs, or .srch. tagged xattrs where all, some, or
none of the files share the same xattr name.

Signed-off-by: Zach Brown <zab@versity.com>
2020-08-26 14:39:39 -07:00
Zach Brown
d2d32c8776 scoutfs-tests: add bulk_create_paths
Add a test utility which efficiently recreates paths read from stdin.

Signed-off-by: Zach Brown <zab@versity.com>
2020-06-02 10:12:50 -07:00
Zach Brown
b9bd7d1293 scoutfs-tests: initial commit
The first commit of the scoutfs-tests suite which uses multiple mounts
on one host to test multi-node scoutfs.

Signed-off-by: Zach Brown <zab@versity.com>
2019-08-02 16:51:34 -07:00