mirrors/scoutfs
1
0
Fork 0
mirror of https://github.com/versity/scoutfs.git synced 2026-01-07 12:35:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add option to shuffle test order.

Browse Source 
The `-R` option will shuffle the order in which tests are executed.

The testing order shouldn't affect the outcome of any of the tests, but
in practice many of these tests will execute code slightly different
based on the history of the filesystem, resources allocated, memory
usage etc. of tests that were executed before. Shuffling the order of
tests therefore introduces small semi-random variations in the
enviroment.

The xfstests test is the only one that can't be shuffled yet into the
mix, so it is kept at the end. This is because it leaves the filesystems
unmounted. At a later point we may want to address this.

Signed-off-by: Auke Kok <auke.kok@versity.com>
This commit is contained in:
Auke Kok
2025-10-03 14:20:56 -07:00
parent 9741d40e10
commit bf87ea0a1c
1 changed files with 18 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
tests/run-tests.sh
View File

@@ -69,6 +69,7 @@ $(basename $0) options:
-r <dir> | Specify the directory in which to store results of -r <dir> | Specify the directory in which to store results of
| test runs. The directory will be created if it doesn't | test runs. The directory will be created if it doesn't
| exist. Previous results will be deleted as each test runs. | exist. Previous results will be deleted as each test runs.
-R | shuffle the test order randomly using shuf
-s | Skip git repo checkouts. -s | Skip git repo checkouts.
-t | Enabled trace events that match the given glob argument. -t | Enabled trace events that match the given glob argument.
| Multiple options enable multiple globbed events. | Multiple options enable multiple globbed events.
@@ -164,6 +165,9 @@ while true; do
T_RESULTS="$2" T_RESULTS="$2"
shift shift
;; ;;
-R)
T_SHUF="1"
;;
-s) -s)
T_SKIP_CHECKOUT="1" T_SKIP_CHECKOUT="1"
;; ;;
@@ -261,13 +265,26 @@ for e in T_META_DEVICE T_DATA_DEVICE T_EX_META_DEV T_EX_DATA_DEV T_KMOD T_RESULT
eval $e=\"$(readlink -f "${!e}")\" eval $e=\"$(readlink -f "${!e}")\"
done done
# permute sequence?
T_SEQUENCE=sequence
if [ -n "$T_SHUF" ]; then
msg "shuffling test order"
shuf sequence -o sequence.shuf
# keep xfstests at the end
if grep -q 'xfstests.sh' sequence.shuf ; then
sed -i '/xfstests.sh/d' sequence.shuf
echo "xfstests.sh" >> sequence.shuf
fi
T_SEQUENCE=sequence.shuf
fi
# include everything by default # include everything by default
test -z "$T_INCLUDE" && T_INCLUDE="-e '.*'" test -z "$T_INCLUDE" && T_INCLUDE="-e '.*'"
# (quickly) exclude nothing by default # (quickly) exclude nothing by default
test -z "$T_EXCLUDE" && T_EXCLUDE="-e '\Zx'" test -z "$T_EXCLUDE" && T_EXCLUDE="-e '\Zx'"
# eval to strip re ticks but not expand # eval to strip re ticks but not expand
tests=$(grep -v "^#" sequence | tests=$(grep -v "^#" $T_SEQUENCE |
eval grep "$T_INCLUDE" | eval grep -v "$T_EXCLUDE") eval grep "$T_INCLUDE" | eval grep -v "$T_EXCLUDE")
test -z "$tests" && \ test -z "$tests" && \
die "no tests found by including $T_INCLUDE and excluding $T_EXCLUDE" die "no tests found by including $T_INCLUDE and excluding $T_EXCLUDE"