Files
seaweedfs/.github
Chris LuandGitHub 1c926e8fac test: systematic EC interruption verification — exhaustive model check + deterministic kill matrix (#10764)
* ec: bounded-exhaustive model check of the volume lifecycle

The randomized chaos harness samples the state space; this enumerates
it. The lifecycle is a state machine whose steps mirror the pipelines in
this package, and the checker explores every schedule within the bound:
a crash at every step boundary, an error return running the rollback
(itself crashable at every step), a volume-server restart applying the
startup reconciliation rules in every quiescent state, and the
prescribed restart-based recovery from every crashed state.

Checked in every reachable state: durability (a readable copy always
exists), at most one generation mounted, and — a property the sweep
discipline turns out to guarantee — at most one generation's files on
disk. From every quiescent state the recovery must converge to a clean
volume. Runs in well under a second.

* test: deterministic EC interruption matrix

Enumerate every phase of every interruptible EC operation and kill a
real weed shell exactly when the phase announces itself on the command
output, instead of at a random moment: four encode phases, four decode
phases, and the balance's move phase (set up with -rebalance=false so a
move is guaranteed). Each scenario prepares its precondition, kills at
the marker, runs the prescribed recovery, and verifies every stored byte
still reads back identical.

The interruption recoveries move out of the randomized ops into shared
chaosRun helpers both drivers use.

* test: make the randomized EC chaos walk opt-in

The systematic layers — the interruption matrix and the lifecycle model
check — carry the CI coverage deterministically; the randomized walk
stays for exploratory runs, behind EC_CHAOS_SEED.

* ci: bound the EC integration suite by the job budget, not go test's default

The suite with the interruption matrix runs close to the default 10m
binary timeout on slower runners.

* test: require every interruption-matrix marker to appear

A marker that never prints means a pipeline refactor renamed or dropped
the progress line; silently degenerating into a no-interruption run
would let CI pass without exercising the boundary the scenario names.
Also recheck the marker channel after the wait: a shell that prints and
exits at once makes both channels ready, and select picking the exit
case must not report a printed marker as missed.
2026-08-14 17:45:11 -07:00
..
2022-04-24 21:44:53 -07:00
2021-12-16 11:07:53 -08:00