Allow compacting logs down to a single page

The k-way merge function at the core of the srch file entry merging had
some bookkeeping math (calculating number of parents) that couldn't
handle merging a single incoming entry stream, so it threw a warning and
returned an error.  When refusing to handle that case, it was assuming
that caller was trying to merge down a single log file which doesn't
make any sense.

But in the case of multiple small unsorted logs we can absolutely end up
with their entries stored in one sorted page.   We have one sorted input
page that's merging multiple log files.  The merge function is also the
path that writes to the output file so we absolutely need to handle this
case.

We more carefully calculate the number of parents, clamping it to one
parent when we'd otherwise get "(roundup(1) -> 1) - 1 == 0" when
calculating the number of parents from the number of inputs.  We can
relax the warning and error to refuse to merge nothing.

The test triggers this case by putting single search entries in the log
files for mounts and unmounting them to force rotation of the mount log
files into mergable rotated log files.

Signed-off-by: Zach Brown <zab@versity.com>
This commit is contained in:
Zach Brown
2021-10-06 11:33:31 -07:00
parent cf512c5fcf
commit 75f9aabe75
3 changed files with 29 additions and 2 deletions

View File

@@ -1481,10 +1481,11 @@ static int kway_merge(struct super_block *sb,
int ind;
int i;
if (WARN_ON_ONCE(nr <= 1))
if (WARN_ON_ONCE(nr <= 0))
return -EINVAL;
nr_parents = roundup_pow_of_two(nr) - 1;
/* always at least one parent for single leaf */
nr_parents = max_t(unsigned long, 1, roundup_pow_of_two(nr) - 1);
/* root at [1] for easy sib/parent index calc, final pad for odd sib */
nr_nodes = 1 + nr_parents + nr + 1;
tnodes = __vmalloc(nr_nodes * sizeof(struct tourn_node),

View File

@@ -2,6 +2,7 @@
== update existing xattr
== remove an xattr
== remove xattr with files
== trigger small log merges by rotating single block with unmount
== create entries in current log
== delete small fraction
== remove files

View File

@@ -42,6 +42,31 @@ echo "== remove xattr with files"
rm -f "$T_D0/"{create,update}
diff_srch_find scoutfs.srch.test
echo "== trigger small log merges by rotating single block with unmount"
sv=$(t_server_nr)
i=1
while [ "$i" -lt "8" ]; do
for nr in $(t_fs_nrs); do
# not checking, can go over limit by fs_nrs
((i++))
if [ $nr == $sv ]; then
continue;
fi
eval path="\$T_D${nr}/single-block-$i"
touch "$path"
setfattr -n scoutfs.srch.single-block-logs -v $i "$path"
t_umount $nr
t_mount $nr
((i++))
done
done
# wait for srch compaction worker delay
sleep 10
rm -rf "$T_D0/single-block-*"
echo "== create entries in current log"
DIR="$T_D0/dir"
NR=$((LOG / 4))