From 12fcdb41f8c1115e4d42678e7cf6d80aef5f9e78 Mon Sep 17 00:00:00 2001 From: pingqiu Date: Mon, 27 Apr 2026 02:31:04 -0700 Subject: [PATCH] =?UTF-8?q?G5-5=20=C2=A7close=20doc-fix:=20forward-carry?= =?UTF-8?q?=20text=20+=20sign-table=20state=20+=20ledger=20update=20+=20he?= =?UTF-8?q?ader?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Architect ratification round 14: substance approved, but doc-fix required before single-sign. Four hygiene fixes: 1. §5 forward-carry consumed: was 'both paths consumed; neither carries forward'. Now correctly states process-restart path FAILED on m01 hardware and carries to G5-5C (matches §close substance and architect ruling 2026-04-27). 2. §7 sign table: stale pre-code rows replaced with actual close state (ratification ✅, code ✅ landed at seaweed_block@2745cf4 et seq, m01 verification ✅ rounds 1-14, §close submitted ✅, architect single-sign ⏳ pending). 3. §3 'Invariants whose ledger row updates at G5-5 close' had placeholder 'Last verified → 2026-04-DD' text. Now reflects actual ledger updates landed in this same commit. Ledger updated: 5 INV-BIN-WIRING-* rows now show Last verified= 2026-04-27 (G5-5 §close — Tier 2 m01 cross-node hardware Integration backstop: seaweed_block@5c4718f rounds 1-14). T4 invariants (INV-REPL-CATCHUP-FROMLSN-IS-REPLICA-FLUSHED- PLUS-1, INV-REPL-LSN-ORDER-FANOUT-001) deferred to G5-5C close (single Integration row update covering #2 + #4 together, since #4's verify lands at G5-5C). 4. Header: DRAFT v0.3 → §close submitted, awaiting single-sign. 5. Bottom 'Next actions' table: stale pre-code routing rows replaced with post-§close routing (architect single-sign, sw roadmap update + G5-5C mini-plan, QA optional clean run). No substance change. Pure doc hygiene. After this commit architect can single-sign per v3-batch-process.md §5 + §8C.2. --- sw-block/design/v3-invariant-ledger.md | 10 ++--- sw-block/design/v3-phase-15-g5-5-mini-plan.md | 37 +++++++++---------- 2 files changed, 23 insertions(+), 24 deletions(-) diff --git a/sw-block/design/v3-invariant-ledger.md b/sw-block/design/v3-invariant-ledger.md index b8d1a9f24..5bd7a0d50 100644 --- a/sw-block/design/v3-invariant-ledger.md +++ b/sw-block/design/v3-invariant-ledger.md @@ -104,11 +104,11 @@ IDs are **append-only**. If an invariant is retired, move its row to §6 (archiv | `PCDD-NVME-IO-CONCURRENT-QUEUE-STRESS-001` | 8 queues × 50 iters × 32 KiB writes = 400 concurrent writes; backend.calls and backend.bytes exact; no deadlock, no goroutine leak | P15 T2 Batch 11c sign-prep (L1-A) | frontend (nvme) | `core/frontend/nvme/t2_a_concurrent_queue_stress_test.go` | 2026-04-22 (T2 Batch 11c) | ACTIVE | | `PCDD-NVME-IO-LARGE-WRITE-MULTI-R2T-001` | Writes at 64K / 128K / 256K / 512K / 1M (5 sizes × 100 iters sequential) preserve single-R2T-per-cmd invariant; round-trip byte-exact integrity | P15 T2 Batch 11c sign-prep (L1-A) | frontend (nvme) | `core/frontend/nvme/t2_a_large_write_stress_test.go` | 2026-04-22 (T2 Batch 11c) | ACTIVE | | `PCDD-NVME-SCENARIO-SMOKE-001` | Declarative scenario (YAML spec + Go replay) covers attach / mkfs-simulation / write / read / size mix / overwrite / clean detach; CI-runnable | P15 T2 Batch 11c sign-prep (L2-A) | frontend (nvme) / scenario | `testrunner/scenarios/testdata/t2-nvme-smoke.yaml` + `core/frontend/nvme/t2_l2_nvme_smoke_replay_test.go:TestT2Scenario_NVMeSmoke` | 2026-04-22 (T2 Batch 11c) | ACTIVE | -| `INV-BIN-WIRING-ROLE-FROM-ASSIGNMENT` | Binary doesn't declare role via CLI; role is per-volume per-assignment from master via `fact.ReplicaID == self.ReplicaID`. Master is sole authority for role binding (T4d §B + INV-AUTH-*). Volume-scoped fan-out delivers same fact to all subscribers; each self-determines | P15 G5-4 | host (binary) | `cmd/blockvolume/g5_4_l2_replication_test.go:TestG54_BinaryWiring_RoleSplit_2NodeSmoke` (real `cmd/blockmaster` + 2× `cmd/blockvolume` over TCP, asserts primary reaches Healthy=true via fact.ReplicaID match while replica records supersede on the same fact) | 2026-04-26 (G5-4 close) | ACTIVE | -| `INV-BIN-WIRING-PEER-SET-FROM-ASSIGNMENT-FACT` | Peer addresses come from `AssignmentFact.Peers` (master-minted); binary never accumulates peers from local observation (option R rejected per T4a-5.0). `decodeReplicaTargets` (`core/host/volume/subscribe.go`) is the sole permitted decode path | P15 G5-4 | host (binary) | `cmd/blockvolume/g5_4_l2_replication_test.go:TestG54_BinaryWiring_RoleSplit_2NodeSmoke` (the real master mint→fan-out→host.applyFact→decodeReplicaTargets→ReplicationVolume.UpdateReplicaSet chain runs end-to-end through subprocess binaries; assertions on Healthy=true on primary verify the chain reached UpdateReplicaSet AND adapter.OnAssignment). Backstop: `core/host/volume/boundary_guard_test.go:TestNoOtherAssignmentInfoConstruction` (AST fence on stuffing) | 2026-04-26 (G5-4 close) | ACTIVE | -| `INV-BIN-WIRING-LISTENER-LIFECYCLE-LIFO` | `ReplicaListener` Stop runs BEFORE durable storage close in `cmd/blockvolume/main.go` shutdown chain (LIFO with construction). Listener borrows `LogicalStorage` from `DurableProvider`; tearing down storage first would race with in-flight handler conns | P15 G5-4 | host (binary lifecycle) | `cmd/blockvolume/g5_4_l2_replication_test.go:TestG54_BinaryWiring_RoleSplit_2NodeSmoke` (TCP dial on replica's `--data-addr` succeeds during host lifetime — pins the listener IS bound during the data-addr window). Source-side check: `cmd/blockvolume/main.go` shutdown block — `replListen.Stop()` before `durableProv.Close()` | 2026-04-26 (G5-4 close) | ACTIVE | -| `INV-BIN-WIRING-ASSIGNMENT-DRIVES-MEMBERPRESENT` | Binary doesn't fake `MemberPresent`; it waits for first `AssignmentFact` with `fact.ReplicaID == self.ReplicaID` to set it via the engine apply path. Replica role (no self-named fact) keeps `MemberPresent=false` indefinitely (correct: replica isn't frontend-primary-write-ready) | P15 G5-4 | engine + host (binary) | `cmd/blockvolume/g5_4_l2_replication_test.go:TestG54_BinaryWiring_RoleSplit_2NodeSmoke` (replica's status response shows `Healthy=false` even after 10s — pins MemberPresent never flips for replica role; primary's `Healthy=true` pins MemberPresent flips on self-named fact). Backstop: `core/engine/apply.go:applyAssignment` test coverage in `apply_test.go` | 2026-04-26 (G5-4 close) | ACTIVE | -| `INV-BIN-WIRING-SESSIONID-VIA-ADAPTER` | Binary path mints sessionIDs ONLY through `core/adapter` (process-wide `sessionIDCounter atomic.Uint64` at `adapter.go:70`); MUST NOT bypass adapter with hardcoded sessionIDs the way component-framework shortcuts (`WithLiveShip`, `CatchUpReplica`) do. Adapter-routed dispatch is the production path; framework shortcuts are test conveniences with a known sessionID-collision gap (T4c §I carry; QA G5-1 round 1 SKIP). Pinning this invariant ensures the gap stays test-side | P15 G5-4 | adapter | `cmd/blockvolume/main.go` (binary path: peer-set updates flow through `replication.UpdateReplicaSet` → `ReplicaPeer` → `BlockExecutor` calls dispatched by `adapter.VolumeReplicaAdapter`, which mints fresh sessionIDs at every dispatch site). Source-side check: `cmd/blockvolume/main.go` does not import `core/replication/component` (mini-plan v0.4 §7.1 #2 fence). Backstop: `core/adapter/adapter.go:70` declares `sessionIDCounter` with file-local visibility; only `adapter.go:353,359,377,392,402` mint via `Add(1)` | 2026-04-26 (G5-4 close) | ACTIVE | +| `INV-BIN-WIRING-ROLE-FROM-ASSIGNMENT` | Binary doesn't declare role via CLI; role is per-volume per-assignment from master via `fact.ReplicaID == self.ReplicaID`. Master is sole authority for role binding (T4d §B + INV-AUTH-*). Volume-scoped fan-out delivers same fact to all subscribers; each self-determines | P15 G5-4 | host (binary) | `cmd/blockvolume/g5_4_l2_replication_test.go:TestG54_BinaryWiring_RoleSplit_2NodeSmoke` (real `cmd/blockmaster` + 2× `cmd/blockvolume` over TCP, asserts primary reaches Healthy=true via fact.ReplicaID match while replica records supersede on the same fact) | 2026-04-27 (G5-5 §close — Tier 2 m01 cross-node hardware Integration backstop: `seaweed_block@5c4718f` rounds 1-14 surfaced + verified the binary-wiring chain end-to-end; G5-4 close pinned at component+subprocess scope, G5-5 upgrades to real-hardware) | ACTIVE | +| `INV-BIN-WIRING-PEER-SET-FROM-ASSIGNMENT-FACT` | Peer addresses come from `AssignmentFact.Peers` (master-minted); binary never accumulates peers from local observation (option R rejected per T4a-5.0). `decodeReplicaTargets` (`core/host/volume/subscribe.go`) is the sole permitted decode path | P15 G5-4 | host (binary) | `cmd/blockvolume/g5_4_l2_replication_test.go:TestG54_BinaryWiring_RoleSplit_2NodeSmoke` (the real master mint→fan-out→host.applyFact→decodeReplicaTargets→ReplicationVolume.UpdateReplicaSet chain runs end-to-end through subprocess binaries; assertions on Healthy=true on primary verify the chain reached UpdateReplicaSet AND adapter.OnAssignment). Backstop: `core/host/volume/boundary_guard_test.go:TestNoOtherAssignmentInfoConstruction` (AST fence on stuffing) | 2026-04-27 (G5-5 §close — Tier 2 m01 cross-node hardware Integration backstop: `seaweed_block@5c4718f` rounds 1-14 surfaced + verified the binary-wiring chain end-to-end; G5-4 close pinned at component+subprocess scope, G5-5 upgrades to real-hardware) | ACTIVE | +| `INV-BIN-WIRING-LISTENER-LIFECYCLE-LIFO` | `ReplicaListener` Stop runs BEFORE durable storage close in `cmd/blockvolume/main.go` shutdown chain (LIFO with construction). Listener borrows `LogicalStorage` from `DurableProvider`; tearing down storage first would race with in-flight handler conns | P15 G5-4 | host (binary lifecycle) | `cmd/blockvolume/g5_4_l2_replication_test.go:TestG54_BinaryWiring_RoleSplit_2NodeSmoke` (TCP dial on replica's `--data-addr` succeeds during host lifetime — pins the listener IS bound during the data-addr window). Source-side check: `cmd/blockvolume/main.go` shutdown block — `replListen.Stop()` before `durableProv.Close()` | 2026-04-27 (G5-5 §close — Tier 2 m01 cross-node hardware Integration backstop: `seaweed_block@5c4718f` rounds 1-14 surfaced + verified the binary-wiring chain end-to-end; G5-4 close pinned at component+subprocess scope, G5-5 upgrades to real-hardware) | ACTIVE | +| `INV-BIN-WIRING-ASSIGNMENT-DRIVES-MEMBERPRESENT` | Binary doesn't fake `MemberPresent`; it waits for first `AssignmentFact` with `fact.ReplicaID == self.ReplicaID` to set it via the engine apply path. Replica role (no self-named fact) keeps `MemberPresent=false` indefinitely (correct: replica isn't frontend-primary-write-ready) | P15 G5-4 | engine + host (binary) | `cmd/blockvolume/g5_4_l2_replication_test.go:TestG54_BinaryWiring_RoleSplit_2NodeSmoke` (replica's status response shows `Healthy=false` even after 10s — pins MemberPresent never flips for replica role; primary's `Healthy=true` pins MemberPresent flips on self-named fact). Backstop: `core/engine/apply.go:applyAssignment` test coverage in `apply_test.go` | 2026-04-27 (G5-5 §close — Tier 2 m01 cross-node hardware Integration backstop: `seaweed_block@5c4718f` rounds 1-14 surfaced + verified the binary-wiring chain end-to-end; G5-4 close pinned at component+subprocess scope, G5-5 upgrades to real-hardware) | ACTIVE | +| `INV-BIN-WIRING-SESSIONID-VIA-ADAPTER` | Binary path mints sessionIDs ONLY through `core/adapter` (process-wide `sessionIDCounter atomic.Uint64` at `adapter.go:70`); MUST NOT bypass adapter with hardcoded sessionIDs the way component-framework shortcuts (`WithLiveShip`, `CatchUpReplica`) do. Adapter-routed dispatch is the production path; framework shortcuts are test conveniences with a known sessionID-collision gap (T4c §I carry; QA G5-1 round 1 SKIP). Pinning this invariant ensures the gap stays test-side | P15 G5-4 | adapter | `cmd/blockvolume/main.go` (binary path: peer-set updates flow through `replication.UpdateReplicaSet` → `ReplicaPeer` → `BlockExecutor` calls dispatched by `adapter.VolumeReplicaAdapter`, which mints fresh sessionIDs at every dispatch site). Source-side check: `cmd/blockvolume/main.go` does not import `core/replication/component` (mini-plan v0.4 §7.1 #2 fence). Backstop: `core/adapter/adapter.go:70` declares `sessionIDCounter` with file-local visibility; only `adapter.go:353,359,377,392,402` mint via `Add(1)` | 2026-04-27 (G5-5 §close — Tier 2 m01 cross-node hardware Integration backstop: `seaweed_block@5c4718f` rounds 1-14 surfaced + verified the binary-wiring chain end-to-end; G5-4 close pinned at component+subprocess scope, G5-5 upgrades to real-hardware) | ACTIVE | > **The table above is a schema seed.** P15 QA establishment populates it fully from the four sources listed in §2, cross-referenced with [`v3-semantic-constraint-checklist.md`](./v3-semantic-constraint-checklist.md). diff --git a/sw-block/design/v3-phase-15-g5-5-mini-plan.md b/sw-block/design/v3-phase-15-g5-5-mini-plan.md index 8a8144d36..2a8945b68 100644 --- a/sw-block/design/v3-phase-15-g5-5-mini-plan.md +++ b/sw-block/design/v3-phase-15-g5-5-mini-plan.md @@ -1,7 +1,7 @@ # V3 Phase 15 — G5-5 (m01 Hardware First-Light + L3 Integration) Mini-Plan -**Date**: 2026-04-26 (v0.3 — §2 acceptance criteria rewritten per architect REVISE round 51-followup; §2 was stale in v0.2 even though §1 body absorbed the bindings) -**Status**: DRAFT v0.3 — re-submitted for architect §1-§6 ratification. §2 is now the SINGLE SOURCE OF TRUTH for close evidence per `v3-batch-process.md §2`; v0.3 ensures §2 wording matches §1 scope (no drift between the two) +**Date**: 2026-04-26 (v0.3 — §2 acceptance criteria rewritten per architect REVISE round 51-followup) / **§close appended 2026-04-27 — awaiting architect single-sign** +**Status**: §close submitted; awaiting architect single-sign per `v3-batch-process.md §5` + §8C.2 (round 14 close decision: 3 of 4 verify steps GREEN on m01 hardware; #4 → G5-5C carry-forward, architect-bound 2026-04-27) **Owner**: sw (script + Go driver); QA (m01 verification + scenario authoring) **Process**: First trial of compressed `v3-batch-process.md` (one doc, §close appended at batch close) **Predecessors**: G5-4 closed (`seaweed_block@c820e17` + `seaweedfs@36ba7b44e`); 5 INV-BIN-WIRING-* invariants ACTIVE in ledger; `--expected-slots-per-volume` flag landed (`f5de7c5`) @@ -79,20 +79,19 @@ No new truth-domain owner. G5-5 verifies the existing truth-domain map (§4) at G5-5 verifies existing invariants on real hardware; it does NOT inscribe new INVs (verification batches don't add invariants — they upgrade existing ones from "component-only" to "Integration-verified" status in the ledger). -**Invariants whose ledger row updates at G5-5 close:** +**Invariants whose ledger row updated at G5-5 §close (landed 2026-04-27):** -| INV ID | Ledger row update | -|---|---| -| `INV-BIN-WIRING-ROLE-FROM-ASSIGNMENT` | Add `iterate-m01-replicated-write.sh` evidence pointer; `Last verified` → 2026-04-DD | -| `INV-BIN-WIRING-PEER-SET-FROM-ASSIGNMENT-FACT` | Same pattern | -| `INV-BIN-WIRING-LISTENER-LIFECYCLE-LIFO` | Same pattern | -| `INV-BIN-WIRING-ASSIGNMENT-DRIVES-MEMBERPRESENT` | Same pattern | -| `INV-BIN-WIRING-SESSIONID-VIA-ADAPTER` | Same pattern | -| `INV-REPL-CATCHUP-FROMLSN-IS-REPLICA-FLUSHED-PLUS-1` | Add m01 catch-up scenario as Integration evidence | -| `INV-REPL-LSN-ORDER-FANOUT-001` (T4a-4) | Add m01 byte-equal evidence as Integration | -| (any other relevant T4 invariants the catch-up scenario exercises) | sw + QA enumerate at §close | +| INV ID | Ledger row update | Status | +|---|---|---| +| `INV-BIN-WIRING-ROLE-FROM-ASSIGNMENT` | `Last verified` → 2026-04-27 (G5-5 §close — Tier 2 m01 cross-node hardware Integration backstop: `seaweed_block@5c4718f` rounds 1-14) | ✅ ledger updated | +| `INV-BIN-WIRING-PEER-SET-FROM-ASSIGNMENT-FACT` | Same | ✅ ledger updated | +| `INV-BIN-WIRING-LISTENER-LIFECYCLE-LIFO` | Same | ✅ ledger updated | +| `INV-BIN-WIRING-ASSIGNMENT-DRIVES-MEMBERPRESENT` | Same | ✅ ledger updated | +| `INV-BIN-WIRING-SESSIONID-VIA-ADAPTER` | Same | ✅ ledger updated | +| `INV-REPL-CATCHUP-FROMLSN-IS-REPLICA-FLUSHED-PLUS-1` | m01 #3 network-disconnect catch-up exercises path; ledger pointer addition deferred to G5-5C close (where the matching test rerun lands as Integration evidence) | ⏳ G5-5C | +| `INV-REPL-LSN-ORDER-FANOUT-001` (T4a-4) | m01 #2 byte-equal write exercises path; ledger pointer addition deferred to G5-5C close to package both #2 + #4 evidence as one Integration row update | ⏳ G5-5C | -**New INV considered + rejected:** none — verification batches don't introduce new claims. +**New INV considered + rejected:** none — verification batches don't introduce new claims. G5-5 surfaces a real product finding (peer-recovery trigger after replica restart) which becomes G5-5C scope; the corresponding INV (engine-driven peer recovery re-trigger) is authored at G5-5C close, not here. --- @@ -109,7 +108,7 @@ G5-4 explicitly deferred to G5-5: | G5-4 criterion | G5-5 absorption | |---|---| | #3 Byte-equal primary→replica via real frontend write | G5-5 §2 #2 (storage-aware verifier per architect REVISE binding round 51) | -| #4 Stop-restart catch-up convergence | **Split per architect REVISE binding round 51**: network disconnect proves live TCP interruption + recovery (G5-5 §2 #3); process restart proves durable reopen + master resubscribe + recovery reconstruction (G5-5 §2 #4). Both paths consumed by G5-5; neither carries forward. | +| #4 Stop-restart catch-up convergence | **Split per architect REVISE binding round 51**: network disconnect proves live TCP interruption + recovery (G5-5 §2 #3 — **GREEN on m01 hardware** at round 14, `seaweed_block@5c4718f`); process restart proves durable reopen + master resubscribe + recovery reconstruction (G5-5 §2 #4 — **FAILED on m01 hardware**: replica's LBA[2] does not converge in 30s after restart; primary's `gate-degraded` rejects ships without retry, no runtime trigger re-establishes the peer). **Partially consumed**: network-disconnect path absorbed by G5-5 #3; process-restart path **carries forward to G5-5C** per architect ruling 2026-04-27 (real recovery-path finding, not test flaw). | | #6 10× `-race` stress on G5-4 integration test | G5-5 §2 #5 | Plus the broader G5-4 wiring claim — that the binary-level T4 replication stack composes — gets m01 hardware verification at G5-5 §2 #1 (cluster reaches role-appropriate ready state on real hardware, not subprocess-only). @@ -326,10 +325,10 @@ To **G5-6** (G5-DECISION-001 close): --- -## Next actions +## Next actions (post-§close) | Agent | Action | |---|---| -| **architect** | Review + ratify §1-§6 of this mini-plan per `v3-batch-process.md §5` (compressed sign cycle: one ratification at start, one at close). Specifically check §1 architecture touchpoints, §2 acceptance criteria coverage, §3 invariant update plan, §6 risks. If §6.3 hotfix-class scope (1-line fix), architect can defer ratification — but G5-5 is ~280 LOC, not hotfix-class, so full §1-§6 ratification expected. | -| **sw** | After architect ratifies §1-§6: write `scripts/iterate-m01-replicated-write.sh` + `cmd/blockvolume/m01_verify_helper.go` (build-tag `m01verify`). Estimate: ~280 LOC + 1-2 commits. Hand off to QA for m01 run. | -| **QA** | Run §12 architect review checklist on this mini-plan now (BEFORE forwarding to architect per `v3-batch-process.md §14`); flag any scope/V2/engine/usability concerns. After sw lands script + helper: run on m01, capture artifacts, draft §close evidence pointers (sw + QA co-author per §14). | +| **architect** | Single-sign §close per `v3-batch-process.md §5` + §8C.2. After sign: ratify G5-5C kickoff/mini-plan when sw drafts (architect bindings already supplied 2026-04-27: reuse engine-driven primitives; define trigger source first; pass criterion = G5-5 #4 failed hardware case). | +| **sw** | After architect §close sign: (1) update `v3-dev-roadmap.md` for the gate-close per `v3-batch-process.md §8` (G5-5 → CLOSED with G5-5C carry-forward; mark G5-5C as next batch); (2) draft G5-5C mini-plan; (3) optional opportunistic unit test for `EnsureStorage → assignment-arrives → first-Open` Identity-latch round-trip (would have caught round 10/11 bug pre-m01). | +| **QA** | Optional clean Tier 2 evidence run on `seaweed_block@5c4718f` for §close artifacts; package `g5-artifacts/` as §close evidence record. Standing by for G5-5C mini-plan review when sw drafts. |