We don't directly mount the underlying devices for each mount because
the kernel notices multiple mounts and doesn't setup a new super block
for each.
Previously the script used loopback devices to create the local shared
block construct 'cause it was easy. This introduced corruption of
blocks that saw concurrent read and write IOs. The buffered kernel file
IO paths that loopback eventually degrades into by default (via splice)
could have buffered readers copying out of pages without the page lock
while writers modified the page. This manifest as occasional crc
failure of blocks that we knowingly issue concurrent reads and writes to
from multiple mounts (the quorum and super blocks).
This changes the script to use device-mapper linear passthrough devices.
Their IOs don't hit a caching layer and don't provide an opportunity to
corrupt blocks.
Signed-off-by: Zach Brown <zab@versity.com>
In older versions of coreutils, quoted strings are occasionally
output using utf-8 open/close single quotes.
New versions of coreutils will exclusively use the ASCII single quote
character "'" when the output is not a TTY - as is the case with
all test scripts.
We can avoid most of these problems by always setting LC_ALL=C in
testing, however.
Signed-off-by: Auke Kok <auke.kok@versity.com>
The fence script we use for our single node multi-mount tests only knows
how to fence by using forced unmount to destroy a mount. As of now, the
tests only generate failing nodes that need to be fenced by using forced
unmount as well. This results in the awkward situation where the
testing fence script doesn't have anything to do because the mount is
already gone.
When the test fence script has nothing to do we might not notice if it
isn't run. This adds explicit verification to the fencing tests that
the script was really run. It adds per-invocation logging to the fence
script and the test makes sure that it was run.
While we're at it, we take the opportunity to tidy up some of the
scripting around this. We use a sysfs file with the data device
major:minor numbers so that the fencing script can find and unmount
mounts without having to ask them for their rid. They may not be
operational.
Signed-off-by: Zach Brown <zab@versity.com>
The local-force-unmount fenced fencing script only works when all the
mounts are on the local host and it uses force unmount. It is only
used in our specific local testing scripts. Packaging it as an example
lead people to believe that it could be used to cobble together a
multi-host testing network, however temporary.
Move it from being in utils and packged to being private to our tests so
that it doesn't present an attractive nuisance.
Signed-off-by: Zach Brown <zab@versity.com>
The test harness might as well use all cpus when building. It's
reasonably safe to assume both that the test systems are otherwise idle
and that the build is likely to succeed.
Signed-off-by: Zach Brown <zab@versity.com>
We've grown some test names that are prefixes of others
(createmany-parallel, createmany-parallel-mounts). When we're searching
for lines with the test name we have to search for the exact test name,
by terminating the name with a space, instead of searching for a line
that starts with the test name.
This fixes strange output and saved passed stats for the names that
share a prefix.
Signed-off-by: Zach Brown <zab@versity.com>
We can lose interesting state if the mounts are unmounted as tests fail,
only unmount if all the tests pass.
Signed-off-by: Zach Brown <zab@versity.com>
Weirdly, run-tests was treating trace_printk not as an option to enable
trace_printk() traces but as an option to print trace events to the
console with printk? That's not a thing.
Make -P really enable trace_printk tracing and collect it as it would
enabled trace events. It needs to be treated seperately from the -t
options that enable trace events.
While we're at it treat the -P trace dumping option as a stand-alone
option that works without -t arguments.
Signed-off-by: Zach Brown <zab@versity.com>
run-tests.sh has a -t argument which takes a whitespace seperated string
of globs of events to enable. This was hard to use and made it very
easy to accidentally expand the globs at the wrong place in the script.
This makes each -t argument specify a single word glob which is stored
in an array so the glob isn't expanded until it's applied to the trace
event path. We also add an error for -t globs that didn't match any
events and add a message with the count of -t arguments and enabled
events.
Signed-off-by: Zach Brown <zab@versity.com>
We were checking for the wrong magic value.
We now need to use -f when running mkfs in run-tests for things to work.
Signed-off-by: Andy Grover <agrover@versity.com>
When we had three repos the run-tests harness helped by checking
branches in kmod and utils repos to build and test. Now that we have
one repo we can just use the sibling kmod/ and utils/ dirs in the repo.
Signed-off-by: Zach Brown <zab@versity.com>
The tests were checking that the literal string was zero, which it never
was. Once we check the value of the variable then we notice that the
sense of some tests went from -n || to -n &&, so switch those to -z.
Signed-off-by: Zach Brown <zab@versity.com>
For xfstests, we need to be able to specify both for scratch device as
well.
using -e and -f for now, but we should really be switching to long options.
Signed-off-by: Andy Grover <agrover@versity.com>
[zab@versity.com: minor arg message fixes]
Add -z option to run-tests.sh to specify metadata device.
Do a bunch of things twice.
Fix up setup-error-teardown test.
Signed-off-by: Andy Grover <agrover@versity.com>
[zab@versity.com: minor arg message fixes, golden output]
It can be handy to skip checking out specific branches from the
required repos, so -s option will skip doing so for kmod/utils/xfstests.
Also fix utils die messages to reference -U/u instead of -K/k.
Signed-off-by: Andy Grover <agrover@versity.com>
The dmesg check was creating false positives when unexpected messages
from before the test run were forced out of the ring. The evicted
messages were showing up as removals in the diff.
We only want to see new messages that were created during the test run.
So we format the diff to only output added lines.
Signed-off-by: Zach Brown <zab@versity.com>
We add directories of our built binaries for tests to find. Let's
prepend them to PATH so that we find them before any installed
binaries in the system.
Signed-off-by: Zach Brown <zab@versity.com>
Add a -y argument so we can specify additional args to ./xfstests, and
clean up our xfstest a bit while we're in there.
Signed-off-by: Zach Brown <zab@versity.com>
When running a test we only create the test dir through one mount, but
we were off-by-one when deciding that we were iterating through the
first mount.
Signed-off-by: Zach Brown <zab@versity.com>
We can't use cmd() to create the results dir because it tries to
redirect output to the results dir, which fails, so mkdir isn't run and
we don't create the results dir.
Signed-off-by: Zach Brown <zab@versity.com>
We check out the specified git branch with "origin/" prepended, but we
weren't verifying that same full branch so the verification failed
because it couldn't distinguish differentiate amongst possible named
branches.
Signed-off-by: Zach Brown <zab@versity.com>
The first commit of the scoutfs-tests suite which uses multiple mounts
on one host to test multi-node scoutfs.
Signed-off-by: Zach Brown <zab@versity.com>
