e2e: fail the suite when a label filter selects no specs (#10674)
Run the E2E test on kind / setup-test-matrix (push) Failing after 3s
e2e-test-kind.yaml / extract (push) Failing after 6s
Run the E2E test on kind / get-go-version (push) Failing after 7s
Run the E2E test on kind / build (push) Skipped
Run the E2E test on kind / run-e2e-test (push) Skipped
Run the E2E test on kind / run-csi-e2e-test (push) Skipped
push.yml / extract (push) Failing after 6s
Scorecard supply-chain security / Scorecard analysis (push) Skipped
Main CI / get-go-version (push) Failing after 7s
Main CI / Build (push) Skipped

* e2e: fail the suite when a label filter selects no specs

A label expression that matches nothing is reported as a pass. Ginkgo
runs zero specs, prints SUCCESS!, and exits 0, so a run that tested
nothing is indistinguishable from one that tested everything asked of it:

  $ GINKGO_LABELS='Basic && BackupsSync' make -C test/ run-e2e
  Ran 0 of 70 Specs in 0.001 seconds
  SUCCESS! -- 0 Passed | 0 Failed | 0 Pending | 70 Skipped

That expression looks reasonable and selects nothing, because the backup
sync case is labelled Backups rather than Basic. The same shape of
mistake kept the VolumeGroupSnapshot case from running for months, where
CSISnapshot did not select CSIVolumeGroupSnapshot because a bare token is
matched by equality rather than as a substring.

Pass --fail-on-empty, which ginkgo has supported for far longer than the
version in go.mod, so the above now ends in:

  FAIL! - Detected no specs ran and --fail-on-empty is set

It cannot affect a run that selects specs and then skips them at run
time, which this suite does constantly on kind. The flag tests
specs.CountWithoutSkip(), and that Skip field is set only while the label
filter is applied, before anything runs; a Skip() call during a spec sets
the spec's state instead. Checked all five label rows the kind workflow
uses against every labelled Describe in the suite: the smallest selects
two specs, so none of them starts failing.

run-perf keeps the old behaviour. It takes its own labels for a different
workload, and an empty selection there is not what this is about.

Signed-off-by: krishhna24 <krishhnatupedev@gmail.com>

* Add changelog for #10674

Signed-off-by: krishhna24 <krishhnatupedev@gmail.com>

---------

Signed-off-by: krishhna24 <krishhnatupedev@gmail.com>
This commit is contained in:
KrishhnaT
2026-10-21 06:10:55 -07:00
committed by GitHub
parent 642a5bdf87
commit 6093ea050c
2 changed files with 2 additions and 0 deletions
+1
View File
@@ -0,0 +1 @@
Fail the e2e suite when a Ginkgo label filter selects no specs
+1
View File
@@ -212,6 +212,7 @@ run-e2e: ginkgo
-v \
--junit-report e2e/report.xml \
--label-filter="$(GINKGO_LABELS)" \
--fail-on-empty \
--timeout=5h \
--fail-fast=$(FAIL_FAST) \
./e2e \