mirror of
https://github.com/SCST-project/scst.git
synced 2026-10-22 06:02:33 +00:00
ci: Run the scst_user Ginkgo specs on Linux 5.8 and 5.9
scst_user pins its data buffers from Linux 5.9, the first release in which fork() gives the child a copy of a pinned page (commit 70e806e4e645, "mm: Do early cow for pinned pages during fork() for ptes"). The Ginkgo "handler forks" spec checks this, but CI boots only the current Fedora kernel, so the boundary itself is never tested. Add a Ginkgo job matrix that runs the local "user" specs on Fedora 33's Linux 5.8.18 and 5.9.16, on both sides of the boundary. On 5.9.16 the fork spec relies on the pin alone; on 5.8.18 the handler protects its buffers with MADV_DONTFORK, as required there. Both must keep correct geometry and data. The jobs are named after their kernels, like the regression jobs. - kernel-5.8.env, kernel-5.9.env and their SHA256SUMS pin those kernels' rpms from Fedora's koji, like kernel.env and kernel.SHA256SUMS. Fedora's only 5.9.0 build is a rawhide kernel that defines RHEL_MAJOR 8 and RHEL_MINOR 99, so SCST's backports treat it as RHEL and the build fails; 5.9.16 has the same fork() behavior. - build-image takes an optional pin name, such as "kernel-5.9", for Ginkgo images. The image is otherwise built the same way. - The Dockerfile downloads exactly the rpms listed in the selected checksum file, since 5.9 has no kernel-modules-core package. - vm.py --local-only boots only the target guest, without the iSCSI initiator or the private network, and expects only the local report. The guest then writes a config without an iscsi section. The label filter is restricted to local specs: "local && (<filter>)", or "local" without one.
This commit is contained in:
@@ -53,7 +53,14 @@ paths = [
|
||||
checks = ["ginkgo", "kunit"]
|
||||
|
||||
[[rules]]
|
||||
paths = [".github/workflows/ginkgo.yml"]
|
||||
paths = [
|
||||
".github/workflows/ginkgo.yml",
|
||||
# Only the scst_user Ginkgo jobs boot these kernels.
|
||||
"scripts/ci/kernel-5.8.SHA256SUMS",
|
||||
"scripts/ci/kernel-5.8.env",
|
||||
"scripts/ci/kernel-5.9.SHA256SUMS",
|
||||
"scripts/ci/kernel-5.9.env",
|
||||
]
|
||||
checks = ["ginkgo"]
|
||||
|
||||
[[rules]]
|
||||
|
||||
@@ -66,3 +66,63 @@ jobs:
|
||||
${{ runner.temp }}/ginkgo-image/*.json
|
||||
${{ runner.temp }}/ginkgo-image/*.txt
|
||||
${{ runner.temp }}/ginkgo-results/
|
||||
|
||||
user-kernels:
|
||||
# Run the local scst_user specs on both sides of the Linux 5.9 boundary of
|
||||
# dev_user_map_buf(). Named like the regression jobs; the build step checks
|
||||
# each name against its scripts/ci/<pin>.env.
|
||||
name: ${{ matrix.name }}
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 120
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- pin: kernel-5.8
|
||||
name: 5.8.18-300.fc33^Fedora^33
|
||||
- pin: kernel-5.9
|
||||
name: 5.9.16-200.fc33^Fedora^33
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
fetch-depth: 0
|
||||
- name: Select Ginkgo checks
|
||||
id: changes
|
||||
env:
|
||||
BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
run: |
|
||||
python3 scripts/ci/select_checks.py --check ginkgo --event "$GITHUB_EVENT_NAME" \
|
||||
--base "$BASE_SHA" --head "$HEAD_SHA"
|
||||
- name: Install build and VM tools
|
||||
if: steps.changes.outputs.run == 'true'
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y qemu-system-x86 qemu-utils openssh-client e2fsprogs fakeroot
|
||||
- name: Build SCST image with the kernel from scripts/ci/${{ matrix.pin }}.env
|
||||
if: steps.changes.outputs.run == 'true'
|
||||
timeout-minutes: 60
|
||||
env:
|
||||
PIN: ${{ matrix.pin }}
|
||||
NAME: ${{ matrix.name }}
|
||||
run: |
|
||||
source "scripts/ci/$PIN.env"
|
||||
[[ $NAME == "$KERNEL_BUILD^"* ]] || { echo "::error::$NAME does not match $PIN.env"; exit 1; }
|
||||
bash scripts/ci/build-image ginkgo "$RUNNER_TEMP/ginkgo-image" "$PIN"
|
||||
- name: Run local scst_user tests against the compiled SCST
|
||||
if: steps.changes.outputs.run == 'true'
|
||||
run: |
|
||||
python3 scripts/ci/vm.py ginkgo "$RUNNER_TEMP/ginkgo-image" \
|
||||
"$RUNNER_TEMP/ginkgo-results" --timeout 3000 --local-only --ginkgo-label user
|
||||
- name: Save build logs, reports and serial console
|
||||
if: always() && steps.changes.outputs.run == 'true'
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: ginkgo-${{ matrix.pin }}-results
|
||||
retention-days: 7
|
||||
path: |
|
||||
${{ runner.temp }}/ginkgo-image/*.log
|
||||
${{ runner.temp }}/ginkgo-image/*.json
|
||||
${{ runner.temp }}/ginkgo-image/*.txt
|
||||
${{ runner.temp }}/ginkgo-results/
|
||||
|
||||
@@ -32,6 +32,15 @@ Kernel matrices are maintained through `.github/kernel-matrix.conf` and
|
||||
`nightly/conf/nightly.conf`; VM kernel pins are in `scripts/ci/kernel.env`
|
||||
and `kernel.SHA256SUMS` and must be updated together.
|
||||
|
||||
Further Ginkgo jobs run the local `user` specs on Fedora 33's Linux 5.8.18 and
|
||||
5.9.16, pinned in `scripts/ci/kernel-5.8.env` and `kernel-5.9.env` with their
|
||||
`SHA256SUMS`. Linux 5.9 is the first release where `fork()` copies pinned
|
||||
ordinary anonymous pages, which scst_user relies on. The 5.8 fork fixture
|
||||
uses the documented `MADV_DONTFORK` protection; the 5.9 fixture exercises
|
||||
pinning without it. Both require correct geometry and data after the fork.
|
||||
The invalid-capacity specs check the exact SCSI rejection on both kernels.
|
||||
The images are otherwise built the same way; these jobs have no iSCSI initiator.
|
||||
|
||||
## Results
|
||||
|
||||
Start with the failed job's log. KUnit/Ginkgo upload build logs, test reports
|
||||
@@ -66,6 +75,10 @@ python3 scripts/ci/vm.py ginkgo /tmp/scst-ginkgo-image /tmp/scst-ginkgo-results
|
||||
```
|
||||
|
||||
Use `--ginkgo-label ci-push` for the master selection; omit it for all specs.
|
||||
For the scst_user kernel jobs, add the pin name, such as `kernel-5.9`, to
|
||||
`build-image` and run `vm.py` with `--local-only --ginkgo-label user`.
|
||||
`--local-only` intersects the resolved label expression with `local`; without
|
||||
`--ginkgo-label`, it selects all local specs.
|
||||
QEMU tries KVM then TCG; `--accel tcg` forces emulation. To check the VM
|
||||
supervisor itself, use `vm.py self-test` with a KUnit image; this intentionally
|
||||
crashes a guest.
|
||||
|
||||
@@ -2,9 +2,12 @@
|
||||
FROM fedora:44@sha256:43b29f65a41eb9c35e1cd5323e3bdf3b655c2357a9f4f1ff2f9c2798e5045d80 AS runtime
|
||||
ARG SUITE
|
||||
ARG KERNEL_BUILD
|
||||
# The pinned kernel rpms, matching KERNEL_BUILD. The package split varies by
|
||||
# release, so the checksum file also lists which rpms to install.
|
||||
ARG KERNEL_SUMS=kernel.SHA256SUMS
|
||||
ENV KVER=${KERNEL_BUILD}.x86_64
|
||||
ENV CGO_ENABLED=0 GOTOOLCHAIN=local GOCACHE=/opt/go-cache GOMODCACHE=/opt/go-mod
|
||||
COPY scripts/ci/kernel.SHA256SUMS /tmp/kernel.SHA256SUMS
|
||||
COPY scripts/ci/${KERNEL_SUMS} /tmp/kernel.SHA256SUMS
|
||||
RUN dnf install -y --setopt=install_weak_deps=False \
|
||||
make kmod python3 perl perl-Config-General perl-Data-Dumper \
|
||||
systemd systemd-udev dracut util-linux procps-ng iproute \
|
||||
@@ -12,17 +15,14 @@ RUN dnf install -y --setopt=install_weak_deps=False \
|
||||
findutils diffutils tar gzip curl-minimal && \
|
||||
if [ "$SUITE" = ginkgo ]; then dnf install -y golang; fi && \
|
||||
mkdir /tmp/kernel && cd /tmp/kernel && \
|
||||
for package in kernel-core kernel-modules-core kernel-modules \
|
||||
kernel-modules-extra kernel-modules-internal kernel-devel; do \
|
||||
for package in $(awk '{print $2}' /tmp/kernel.SHA256SUMS); do \
|
||||
curl --fail --location --retry 3 \
|
||||
"https://kojipkgs.fedoraproject.org/packages/kernel/${KERNEL_BUILD%-*}/${KERNEL_BUILD#*-}/x86_64/$package-$KVER.rpm" \
|
||||
--output "$package-$KVER.rpm"; \
|
||||
"https://kojipkgs.fedoraproject.org/packages/kernel/${KERNEL_BUILD%-*}/${KERNEL_BUILD#*-}/x86_64/$package" \
|
||||
--output "$package"; \
|
||||
done && \
|
||||
sha256sum --check /tmp/kernel.SHA256SUMS && \
|
||||
dnf install -y --setopt=install_weak_deps=False \
|
||||
"./kernel-core-$KVER.rpm" "./kernel-modules-core-$KVER.rpm" \
|
||||
"./kernel-modules-$KVER.rpm" "./kernel-modules-extra-$KVER.rpm" \
|
||||
"./kernel-modules-internal-$KVER.rpm" && \
|
||||
$(find . -name 'kernel-*.rpm' ! -name 'kernel-devel-*' | sort) && \
|
||||
depmod "$KVER" && \
|
||||
dracut --force --no-hostonly --no-hostonly-cmdline --no-early-microcode \
|
||||
--add-drivers 'virtio_pci virtio_blk virtio_net 9p 9pnet_virtio' \
|
||||
|
||||
@@ -2,17 +2,20 @@
|
||||
# SPDX-License-Identifier: GPL-2.0-only
|
||||
# Build a bootable, version-matched test image without booting or loading SCST.
|
||||
set -euo pipefail
|
||||
if [[ $# != 2 || ! $1 =~ ^(kunit|ginkgo)$ ]]; then
|
||||
# KERNEL selects another pin pair, such as kernel-5.9.env and its SHA256SUMS.
|
||||
if [[ ! ($# == 2 && $1 =~ ^(kunit|ginkgo)$ || $# == 3 && $1 == ginkgo && $3 =~ ^kernel-[0-9]+\.[0-9]+$) ]]; then
|
||||
echo "Usage: $0 kunit|ginkgo OUTPUT_DIRECTORY" >&2
|
||||
echo " $0 ginkgo OUTPUT_DIRECTORY kernel-MAJOR.MINOR" >&2
|
||||
exit 2
|
||||
fi
|
||||
suite=$1
|
||||
out=$(realpath -m "$2")
|
||||
kernel=${3:-kernel}
|
||||
mkdir "$out"
|
||||
root=$(git rev-parse --show-toplevel)
|
||||
commit=$(git rev-parse HEAD)
|
||||
# shellcheck source=kernel.env
|
||||
source "$root/scripts/ci/kernel.env"
|
||||
source "$root/scripts/ci/$kernel.env"
|
||||
work=$(mktemp -d)
|
||||
cleanup() {
|
||||
rm -rf -- "$work"
|
||||
@@ -22,7 +25,8 @@ mkdir "$work/source" "$work/root"
|
||||
# A committed snapshot excludes local profiles, generated files and Git credentials.
|
||||
git -C "$root" archive "$commit" | tar -x -C "$work/source"
|
||||
docker build --progress=plain \
|
||||
--build-arg "KERNEL_BUILD=$KERNEL_BUILD" --build-arg "SUITE=$suite" \
|
||||
--build-arg "KERNEL_BUILD=$KERNEL_BUILD" --build-arg "KERNEL_SUMS=$kernel.SHA256SUMS" \
|
||||
--build-arg "SUITE=$suite" \
|
||||
--build-arg "SOURCE_COMMIT=$commit" --output "type=tar,dest=$work/rootfs.tar" \
|
||||
-f "$work/source/scripts/ci/Dockerfile" "$work/source" 2>&1 | tee "$out/image-build.log"
|
||||
# Preserve guest owners without host root, mounts or loop devices. Extraction
|
||||
|
||||
+20
-12
@@ -43,10 +43,15 @@ kunit)
|
||||
2>&1 | tee /ci/reports/tests.log
|
||||
;;
|
||||
ginkgo|initiator)
|
||||
while [[ ! -e /ci/input/ssh-ready ]]; do sleep 1; done
|
||||
ip link set eth0 up
|
||||
if [[ $mode == initiator ]]; then address=192.0.2.2; else address=192.0.2.1; fi
|
||||
ip address add "$address/24" dev eth0
|
||||
# A local-only run has no initiator guest and no network.
|
||||
local_only=
|
||||
if [[ -e /ci/input/local-only ]]; then local_only=1; fi
|
||||
if [[ ! $local_only ]]; then
|
||||
while [[ ! -e /ci/input/ssh-ready ]]; do sleep 1; done
|
||||
ip link set eth0 up
|
||||
if [[ $mode == initiator ]]; then address=192.0.2.2; else address=192.0.2.1; fi
|
||||
ip address add "$address/24" dev eth0
|
||||
fi
|
||||
if [[ $mode == initiator ]]; then
|
||||
install -m 600 /ci/input/authorized_keys /root/.ssh/authorized_keys
|
||||
install -m 600 /ci/input/ssh_host_key /run/ci-host-key
|
||||
@@ -73,23 +78,26 @@ PY
|
||||
touch /ci/reports/ready
|
||||
sleep infinity
|
||||
else
|
||||
install -m 600 /ci/input/id_ed25519 /root/.ssh/id_ed25519
|
||||
install -m 600 /ci/input/known_hosts /root/.ssh/known_hosts
|
||||
while [[ ! -e /ci/input/peer-ready ]]; do sleep 1; done
|
||||
if [[ ! $local_only ]]; then
|
||||
install -m 600 /ci/input/id_ed25519 /root/.ssh/id_ed25519
|
||||
install -m 600 /ci/input/known_hosts /root/.ssh/known_hosts
|
||||
while [[ ! -e /ci/input/peer-ready ]]; do sleep 1; done
|
||||
fi
|
||||
# This disk is created by the host supervisor exclusively for this run.
|
||||
test "$(lsblk -dn -o SERIAL /dev/disk/by-id/virtio-scst-ci-data)" = scst-ci-data
|
||||
python3 - <<'PY'
|
||||
import json, pathlib
|
||||
python3 - "$local_only" <<'PY'
|
||||
import json, pathlib, sys
|
||||
config = dict(disposable_host=True,
|
||||
admin_command=['/usr/bin/perl', '-I/scst/scstadmin/scstadmin.sysfs/scst-1.0.0/lib',
|
||||
'/scst/scstadmin/scstadmin.sysfs/scstadmin'],
|
||||
report_dir='/ci/reports/ginkgo', backing_dir='/ci/backing',
|
||||
command_timeout='90s', cleanup_timeout='5m', settle_timeout='30s', suite_timeout='75m',
|
||||
io=dict(block_device='/dev/disk/by-id/virtio-scst-ci-data', region_mb=16, timeout='5m'),
|
||||
iscsi=dict(target_address='192.0.2.1', port=3260, login_timeout='30s',
|
||||
io=dict(block_device='/dev/disk/by-id/virtio-scst-ci-data', region_mb=16, timeout='5m'))
|
||||
if not sys.argv[1]:
|
||||
config['iscsi'] = dict(target_address='192.0.2.1', port=3260, login_timeout='30s',
|
||||
initiator=dict(host='192.0.2.2', user='root', port=22,
|
||||
identity_file='/root/.ssh/id_ed25519', lock_file='/tmp/scst-ginkgo.lock',
|
||||
disposable_host=True)))
|
||||
disposable_host=True))
|
||||
pathlib.Path('/ci/config.json').write_text(json.dumps(config))
|
||||
PY
|
||||
label=$(cat /ci/input/ginkgo-label)
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
73d1291b951df9c6b5c3e567331bc6529f7c56d2a274a7055039c8b9e2fb76f3 kernel-core-5.8.18-300.fc33.x86_64.rpm
|
||||
457bbef5665da9003f2a931fb7722066a5a9269fa5ae578a0da10ad81acf7dde kernel-modules-5.8.18-300.fc33.x86_64.rpm
|
||||
cf2624e20fa4e6f7cf66ee2b948115828f9ffc231ac4b8f9c21f505f9fc8e610 kernel-devel-5.8.18-300.fc33.x86_64.rpm
|
||||
73b9b699f0b259b41e28ad191ff6ac0ef95f900324783c48903da56e062ae986 kernel-modules-extra-5.8.18-300.fc33.x86_64.rpm
|
||||
e546c88bce1f6ccfeaf4d8b23b8ab9d145529c87d01b6e3761db53ac3cf6e236 kernel-modules-internal-5.8.18-300.fc33.x86_64.rpm
|
||||
@@ -0,0 +1,5 @@
|
||||
# SPDX-License-Identifier: GPL-2.0-only
|
||||
# The last release before Linux 5.9: scst_user does not pin buffers here, so the
|
||||
# fork spec uses the required MADV_DONTFORK protection. Keep the runtime kernel
|
||||
# and build headers together; update kernel-5.8.SHA256SUMS as well.
|
||||
KERNEL_BUILD=5.8.18-300.fc33
|
||||
@@ -0,0 +1,5 @@
|
||||
37bb114221751c1b1e1ccae23139c138c151efbe5c120f257fdf88f0d44612c2 kernel-core-5.9.16-200.fc33.x86_64.rpm
|
||||
2cc1d80c853d8f58277e5023b65293fc3bcfb38a719705e4f5803f8a3de3d9a6 kernel-modules-5.9.16-200.fc33.x86_64.rpm
|
||||
d13297be0d5948fef81f4fd7e76e19d8a644e4845df735f40702fe87209a3204 kernel-devel-5.9.16-200.fc33.x86_64.rpm
|
||||
6343da31deb38b11d72e2fa8b997bfbbdb16bff3064e9a57cfa5723960fd9de2 kernel-modules-extra-5.9.16-200.fc33.x86_64.rpm
|
||||
528b9b46163de94c98027dec0cd09b3bdb8a67cd5108ac80366280b13920c85d kernel-modules-internal-5.9.16-200.fc33.x86_64.rpm
|
||||
@@ -0,0 +1,7 @@
|
||||
# SPDX-License-Identifier: GPL-2.0-only
|
||||
# Linux 5.9 is the first release in which fork() gives the child a copy of a
|
||||
# pinned page (commit 70e806e4e645), which scst_user relies on. Fedora's only
|
||||
# 5.9.0 build defines RHEL_MAJOR, which SCST's backports treat as RHEL, so use
|
||||
# Fedora 33's last 5.9 kernel. Keep the runtime kernel and build headers
|
||||
# together; update kernel-5.9.SHA256SUMS as well.
|
||||
KERNEL_BUILD=5.9.16-200.fc33
|
||||
+70
-6
@@ -5,6 +5,7 @@ from contextlib import ExitStack, redirect_stdout
|
||||
import io
|
||||
import json
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
@@ -137,6 +138,45 @@ class SupervisorTests(unittest.TestCase):
|
||||
vm.validate_completion(self.output, 'ginkgo', 'this-run', self.manifest,
|
||||
expected)
|
||||
|
||||
def test_local_only_ginkgo_requires_exactly_the_local_report(self):
|
||||
self.manifest['suite'] = 'ginkgo'
|
||||
self.completion(mode='ginkgo')
|
||||
for groups in ((), ('iscsi',), ('local', 'iscsi'), ('local',)):
|
||||
with self.subTest(groups=groups):
|
||||
shutil.rmtree(self.output / 'ginkgo', ignore_errors=True)
|
||||
# The runner names each report directory run-<random>.
|
||||
for index, group in enumerate(groups):
|
||||
(self.output / 'ginkgo' / f'run-{index}').mkdir(parents=True)
|
||||
(self.output / 'ginkgo' / f'run-{index}' / 'report.json').write_text(json.dumps([
|
||||
dict(SuiteSucceeded=True, SuiteLabels=[group],
|
||||
SuiteConfig=dict(LabelFilter='user'),
|
||||
PreRunStats=dict(SpecsThatWillRun=1),
|
||||
SpecReports=[dict(LeafNodeType='It', State='passed')])]))
|
||||
if groups == ('local',):
|
||||
vm.validate_completion(self.output, 'ginkgo', 'this-run', self.manifest,
|
||||
'user', local_only=True)
|
||||
else:
|
||||
with self.assertRaisesRegex(vm.VMError, 'local suite only'):
|
||||
vm.validate_completion(self.output, 'ginkgo', 'this-run', self.manifest,
|
||||
'user', local_only=True)
|
||||
|
||||
def test_local_only_marker_is_written_to_guest_input_before_boot(self):
|
||||
for local_only in (False, True):
|
||||
with self.subTest(local_only=local_only):
|
||||
work = self.output / f'work-{local_only}'
|
||||
output = self.output / f'output-{local_only}'
|
||||
work.mkdir()
|
||||
output.mkdir()
|
||||
|
||||
def boot(*args, **kwargs):
|
||||
self.assertEqual((work / 'target/local-only').exists(), local_only)
|
||||
return Mock()
|
||||
|
||||
with patch('vm.subprocess.run'), patch('vm.subprocess.Popen', side_effect=boot), \
|
||||
ExitStack() as stack:
|
||||
vm.launch(stack, self.output, work, output, 'target', 'ginkgo',
|
||||
'this-run', 'tcg', local_only=local_only)
|
||||
|
||||
def test_ginkgo_filter_is_written_to_guest_input_before_boot(self):
|
||||
for index, label in enumerate(('', 'basic || smoke')):
|
||||
with self.subTest(label=label):
|
||||
@@ -167,12 +207,14 @@ class ConsoleTests(unittest.TestCase):
|
||||
self.output = self.root / 'results'
|
||||
|
||||
def run_guest(self, mode, console=None, exit_code=0, completion='valid',
|
||||
error=None, serial=None, ginkgo_label=''):
|
||||
error=None, serial=None, ginkgo_label='', local_only=False):
|
||||
manifest = dict(commit='a' * 40, suite=mode, kernel='test-kernel')
|
||||
(self.image / 'image.json').write_text(json.dumps(manifest))
|
||||
self.launched = []
|
||||
|
||||
def launch(stack, image, work, output, role, guest_mode, token, accel, network,
|
||||
ginkgo_label=''):
|
||||
ginkgo_label='', local_only=False):
|
||||
self.launched.append((role, network is not None, local_only))
|
||||
directory = output / role
|
||||
directory.mkdir()
|
||||
if network:
|
||||
@@ -193,11 +235,12 @@ class ConsoleTests(unittest.TestCase):
|
||||
report.write_text(json.dumps(dict(status='PASS', counts=dict(passed=2),
|
||||
not_run_suites=[], module_state='unloaded')))
|
||||
else:
|
||||
for transport in ('local', 'iscsi'):
|
||||
report = directory / 'ginkgo' / transport / 'report.json'
|
||||
for transport in ('local',) if local_only else ('local', 'iscsi'):
|
||||
report = directory / 'ginkgo' / f'run-{transport}' / 'report.json'
|
||||
report.parent.mkdir(parents=True)
|
||||
report.write_text(json.dumps([dict(
|
||||
SuiteSucceeded=True, PreRunStats=dict(SpecsThatWillRun=2),
|
||||
SuiteSucceeded=True, SuiteLabels=[transport],
|
||||
PreRunStats=dict(SpecsThatWillRun=2),
|
||||
SuiteConfig=dict(LabelFilter=ginkgo_label),
|
||||
SpecReports=[dict(LeafNodeType='It', State='passed')])]))
|
||||
return Mock()
|
||||
@@ -205,7 +248,7 @@ class ConsoleTests(unittest.TestCase):
|
||||
stdout = io.StringIO()
|
||||
with patch('vm.launch', side_effect=launch), patch('vm.configure_ssh'), \
|
||||
patch('vm.wait_guest', side_effect=error), redirect_stdout(stdout):
|
||||
result = vm.run(self.image, self.output, mode, 10, 'tcg', ginkgo_label)
|
||||
result = vm.run(self.image, self.output, mode, 10, 'tcg', ginkgo_label, local_only)
|
||||
self.assertEqual(json.loads((self.output / 'result.json').read_text()), result)
|
||||
if console is not None:
|
||||
self.assertEqual((self.output / 'target/tests.log').read_bytes(), console)
|
||||
@@ -242,6 +285,27 @@ class ConsoleTests(unittest.TestCase):
|
||||
self.assertEqual(result['ginkgo_label'], expression)
|
||||
self.assertIn(f'Label filter: {expression}', output)
|
||||
|
||||
def test_local_only_ginkgo_boots_one_guest_without_a_network(self):
|
||||
for local_only, launched in ((False, [('target', True, False), ('initiator', True, False)]),
|
||||
(True, [('target', False, True)])):
|
||||
with self.subTest(local_only=local_only):
|
||||
self.output = self.root / f'local-only-{local_only}'
|
||||
result, _ = self.run_guest('ginkgo', ginkgo_label='user', local_only=local_only)
|
||||
self.assertEqual(result['status'], 'PASS')
|
||||
self.assertEqual(result['local_only'], local_only)
|
||||
self.assertEqual(self.launched, launched)
|
||||
|
||||
def test_local_only_restricts_the_label_filter_to_local_specs(self):
|
||||
for label, expression in (('', 'local'), ('user', 'local && (user)'),
|
||||
('basic || smoke', 'local && (basic || smoke)'),
|
||||
('ci-pr', 'local && (basic || smoke)')):
|
||||
with self.subTest(label=label):
|
||||
self.output = self.root / f'filter-{label or "all"}'
|
||||
result, output = self.run_guest('ginkgo', ginkgo_label=label, local_only=True)
|
||||
self.assertEqual(result['status'], 'PASS')
|
||||
self.assertEqual(result['ginkgo_label'], expression)
|
||||
self.assertIn(f'Label filter: {expression}', output)
|
||||
|
||||
def test_failed_tests_print_assertions_and_preserve_failed_verdict(self):
|
||||
for mode in ('kunit', 'ginkgo'):
|
||||
with self.subTest(mode=mode):
|
||||
|
||||
+27
-8
@@ -65,7 +65,7 @@ def wait_guest(target, peer, deadline, ready, release, started=None, boot_deadli
|
||||
raise VMError(f'QEMU exited with {target.returncode}')
|
||||
|
||||
|
||||
def validate_completion(directory, mode, token, manifest, ginkgo_label=''):
|
||||
def validate_completion(directory, mode, token, manifest, ginkgo_label='', local_only=False):
|
||||
try:
|
||||
result = json.loads((directory / 'completion.json').read_text())
|
||||
except (OSError, ValueError) as exc:
|
||||
@@ -84,7 +84,10 @@ def validate_completion(directory, mode, token, manifest, ginkgo_label=''):
|
||||
raise VMError('KUnit did not complete successfully')
|
||||
elif mode == 'ginkgo':
|
||||
paths = list((directory / 'ginkgo').glob('*/report.json'))
|
||||
if len(paths) != 2:
|
||||
if local_only:
|
||||
if len(paths) != 1:
|
||||
raise VMError('expected a report from the local suite only')
|
||||
elif len(paths) != 2:
|
||||
raise VMError('expected reports from both local and iSCSI suites')
|
||||
for path in paths:
|
||||
reports = json.loads(path.read_text())
|
||||
@@ -95,6 +98,9 @@ def validate_completion(directory, mode, token, manifest, ginkgo_label=''):
|
||||
raise VMError(f'incomplete or failed Ginkgo suite: {path}')
|
||||
if reports[0]['SuiteConfig']['LabelFilter'] != ginkgo_label:
|
||||
raise VMError(f'Ginkgo label filter differs from the requested selection: {path}')
|
||||
# Each run directory is random; the suite label names the transport.
|
||||
if local_only and reports[0].get('SuiteLabels') != ['local']:
|
||||
raise VMError(f'expected a report from the local suite only: {path}')
|
||||
|
||||
|
||||
def diagnostics(output):
|
||||
@@ -160,7 +166,8 @@ def print_results(output, result):
|
||||
print(f"\n{mode}: {result['status']} ({result['elapsed_seconds']}s); {output}", flush=True)
|
||||
|
||||
|
||||
def launch(stack, image, work, output, role, mode, token, accel, network=None, ginkgo_label=''):
|
||||
def launch(stack, image, work, output, role, mode, token, accel, network=None, ginkgo_label='',
|
||||
local_only=False):
|
||||
config, reports = work / role, output / role
|
||||
config.mkdir()
|
||||
reports.mkdir()
|
||||
@@ -168,6 +175,8 @@ def launch(stack, image, work, output, role, mode, token, accel, network=None, g
|
||||
(config / 'token').write_text(token)
|
||||
if mode == 'ginkgo':
|
||||
(config / 'ginkgo-label').write_text(ginkgo_label)
|
||||
if local_only:
|
||||
(config / 'local-only').touch()
|
||||
overlay = work / f'{role}.qcow2'
|
||||
subprocess.run(['qemu-img', 'create', '-q', '-f', 'qcow2', '-F', 'raw',
|
||||
'-b', str(image / 'rootfs.img'), str(overlay)], check=True)
|
||||
@@ -225,24 +234,29 @@ def configure_ssh(work):
|
||||
(peer / 'ssh-ready').touch()
|
||||
|
||||
|
||||
def run(image, output, mode, timeout, accel, ginkgo_label=''):
|
||||
def run(image, output, mode, timeout, accel, ginkgo_label='', local_only=False):
|
||||
manifest = json.loads((image / 'image.json').read_text())
|
||||
if mode in ('kunit', 'ginkgo') and manifest['suite'] != mode:
|
||||
raise VMError('image was built for a different test suite')
|
||||
if mode == 'ginkgo':
|
||||
ginkgo_label = resolve_ginkgo_label(ginkgo_label)
|
||||
if local_only:
|
||||
# Group the resolved expression so OR terms cannot select iSCSI.
|
||||
ginkgo_label = f'local && ({ginkgo_label})' if ginkgo_label.strip() else 'local'
|
||||
output.mkdir(parents=True, exist_ok=False)
|
||||
token = secrets.token_hex(16)
|
||||
result = dict(mode=mode, image=manifest, status='ERROR', error=None)
|
||||
if mode == 'ginkgo':
|
||||
result['ginkgo_label'] = ginkgo_label
|
||||
result['local_only'] = local_only
|
||||
started = time.monotonic()
|
||||
try:
|
||||
with tempfile.TemporaryDirectory(prefix='scst-ci-') as temporary, ExitStack() as stack:
|
||||
work = Path(temporary)
|
||||
network = work / 'network.sock' if mode == 'ginkgo' else None
|
||||
# Local-only runs need no iSCSI initiator, so no second guest or network.
|
||||
network = work / 'network.sock' if mode == 'ginkgo' and not local_only else None
|
||||
target = launch(stack, image, work, output, 'target', mode, token, accel, network,
|
||||
ginkgo_label)
|
||||
ginkgo_label, local_only)
|
||||
peer = None
|
||||
if network:
|
||||
# The listening QEMU must create its socket before its peer connects.
|
||||
@@ -254,7 +268,8 @@ def run(image, output, mode, timeout, accel, ginkgo_label=''):
|
||||
configure_ssh(work)
|
||||
wait_guest(target, peer, started + timeout, output / 'initiator/ready',
|
||||
work / 'target/peer-ready', output / 'target/started', started + 240)
|
||||
validate_completion(output / 'target', mode, token, manifest, ginkgo_label)
|
||||
validate_completion(output / 'target', mode, token, manifest, ginkgo_label,
|
||||
local_only)
|
||||
result['status'] = 'PASS'
|
||||
except (VMError, OSError, ValueError, KeyError, TypeError, subprocess.SubprocessError) as exc:
|
||||
result['error'] = str(exc)
|
||||
@@ -278,11 +293,15 @@ def main():
|
||||
parser.add_argument('--accel', choices=('auto', 'tcg', 'kvm'), default='auto')
|
||||
parser.add_argument('--ginkgo-label', default='',
|
||||
help='Ginkgo label set or expression (default: all tests)')
|
||||
parser.add_argument('--local-only', action='store_true',
|
||||
help='run only local Ginkgo specs, without an iSCSI initiator guest')
|
||||
args = parser.parse_args()
|
||||
if args.timeout <= 0:
|
||||
parser.error('--timeout must be positive')
|
||||
if args.ginkgo_label and args.mode != 'ginkgo':
|
||||
parser.error('--ginkgo-label requires ginkgo mode')
|
||||
if args.local_only and args.mode != 'ginkgo':
|
||||
parser.error('--local-only requires ginkgo mode')
|
||||
image, output = args.image.resolve(), args.output.resolve()
|
||||
# QEMU's key-value arguments cannot represent arbitrary comma-containing paths.
|
||||
if any(',' in str(path) for path in (image, output, Path(tempfile.gettempdir()))):
|
||||
@@ -301,7 +320,7 @@ def main():
|
||||
and (output / 'hang/target/started').is_file()
|
||||
and (output / 'hang/target/started').read_text().strip() == 'hang'))
|
||||
return int(run(image, output, args.mode, args.timeout, args.accel,
|
||||
args.ginkgo_label)['status'] != 'PASS')
|
||||
args.ginkgo_label, args.local_only)['status'] != 'PASS')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
|
||||
Reference in New Issue
Block a user