mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-08-27 19:37:00 +00:00
759d50ba5ddb4fc28f64470e0a3e2a19c8babcf6
13307
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 pingqiu
|
759d50ba5d |
sw-block/design: v3-recovery-inv-test-map — INV-WAL-CURSOR-MONOTONIC-FROM-PINLSN
Adds the new invariant pin row for §3.2 #3 unified WAL stream / cursor- rewind. Companion to the seaweed_block g7-redo/unified-wal-impl branch (checkpoint 3/N at commit 0550e44) which adds the 7 new tests cited. Row content: Definition: sender pump rewinds cursor to fromLSN once at session start; cursor advances monotonically per ScanLBAs callback; never decreases. Receiver enforces matching wire-level monotonic discipline (4-case per kickoff v0.3 §5.1: ==applied+1 apply / >applied+1 gap → FailureContract / ==applied exact-duplicate → FailureProtocol / <applied backward → FailureProtocol). Tests pinned (7): TestSender_PumpHappyPath_OnMemoryWAL TestSender_LiveWritesDuringSession_OnMemoryWAL TestSender_KindByte_FlipsOnceAtCatchUp TestSender_StreamUntilHead_CtxCancel TestReceiver_RejectsBackwardLSN_InSession TestReceiver_RejectsGap_InSession TestReceiver_RejectsExactDuplicate_InSession Status: ✅ pinned. |
||
|
pingqiu
|
04b9d1ea53 |
sw-block/design: relocate V3 recovery dev docs from seaweed_block
Per repository policy: dev/design docs live in
seaweedfs/sw-block/design/, not in seaweed_block/docs/. Formal
product docs come later. This commit relocates the 7 recovery
design markdown docs (4 trunk-merged in seaweed_block phase-15;
3 in-flight on g7-redo branches) plus the 1 hardware canonical
YAML to sw-block/design/ with v3-recovery-* prefix to match the
existing naming pattern (v3-recovery-live-line-backlog-spec.md).
Companion cleanup: a follow-on PR on seaweed_block removes the
docs from docs/ (and the YAML from testrunner/scenarios/) — that
PR is the seaweed_block side of the relocation.
Files added:
v3-recovery-pin-floor-wire.md — was docs/recovery-pin-floor-wire.md
on seaweed_block phase-15 (PR #11+#16)
v3-recovery-wiring-plan.md — was docs/recovery-wiring-plan.md
(PR #13)
v3-recovery-execution-institution.md — was docs/recovery-execution-institution.md
v3-recovery-inv-test-map.md — was docs/recovery-inv-test-map.md
(PR #11/#14/#15)
v3-recovery-unified-wal-stream-kickoff.md — was docs/recovery-unified-wal-stream-kickoff.md
g7-redo/unified-wal-kickoff (v0.3)
v3-recovery-unified-wal-stream-mini-plan.md — was docs/recovery-unified-wal-stream-mini-plan.md
g7-redo/unified-wal-mini-plan (v0.2)
v3-recovery-dual-lane-canonical-runbook.md — was docs/recovery-dual-lane-canonical-runbook.md
g7-redo/hardware-canonical-paper
v3-recovery-dual-lane-canonical.yaml — was testrunner/scenarios/recovery-dual-lane-canonical.yaml
g7-redo/hardware-canonical-paper
Internal cross-references updated in-place via sed:
- docs/recovery-inv-test-map.md → v3-recovery-inv-test-map.md
- docs/recovery-pin-floor-wire.md → v3-recovery-pin-floor-wire.md
- docs/recovery-wiring-plan.md → v3-recovery-wiring-plan.md
- testrunner/scenarios/recovery-dual-lane-canonical.yaml →
v3-recovery-dual-lane-canonical.yaml
Hand-edits:
- runbook §1 companion-YAML link: was
"../v3-recovery-dual-lane-canonical.yaml" (parent dir from
seaweed_block/docs); now same-directory link in design/.
- runbook §8 §3.2 #3 reference: was relative to seaweed_block
memory file (../../.claude/...); rewritten to point to
v3-recovery-unified-wal-stream-kickoff.md §4 directly.
- mini-plan Q15: docs/archive/ wording updated to
sw-block/design/archive/.
Stages-of-evidence still readable from the docs themselves
(kickoff §11, mini-plan §10 resolution logs, inv-test-map row
versions). Original seaweed_block branches preserve git
history for the in-flight content; the cleanup PR closes them
once this lands.
NOTE: this commit does NOT include the user's unrelated
ongoing edits in feature/sw-block (M v3-batch-process.md,
M v3-dev-roadmap.md, M v3-phase-15-g6-mini-plan.md, etc.).
Those stay uncommitted for the user to handle separately.
|
||
|
pingqiu
|
3904730c5a |
G7 mini-plan v0.1 — §harness-notes correction per QA pre-work survey
QA's G7 pre-work surfaced a discrepancy: the v0.1 §harness-notes
pointed at `exec_rebuild_started` / `exec_rebuild_completed` as
the harness markers. Those are RecoveryLog event names (internal
Orchestrator.Log ring buffer, process-local) — NOT visible in
primary.log on hardware. Hardware harnesses can't scrape them
without a /recovery-log HTTP surface (G5-3 forward-carry).
Hardware-visible markers (corrected):
- START: `executor: rebuild start replica=<id> sessionID=<n>
epoch=<n> EV=<n> targetLSN=<n>`
from core/transport/rebuild_sender.go:41
(added at G6 #1, seaweed_block@85475cd)
- COMPLETE: `executor: rebuild complete, sent <n> blocks
(targetLSN=<n>)`
from core/transport/rebuild_sender.go:120
(pre-existing T4d-4 part B / earlier)
Both produced via log.Printf in rebuild_sender.go and routed to
the daemon's stdout/stderr stream (which iterate harness captures
to ${REMOTE_RUN_DIR}/logs/primary.log). Both are sessionID-
correlatable for chained-scenario filtering. The G6 hardware run
already proved the START marker pattern; COMPLETE follows the
same shape.
Files corrected:
- §2 #7 acceptance row (harness helper text)
- §2 entry-marker table row
- §3 risks "Ambiguous rebuild done vs peer healthy" row
- §harness-notes (full rewrite with v0.1 correction note +
marker table + RecoveryLog clarification + recommended helper
shape with sessionID filter)
Negative-references to RecoveryLog event names retained in
explanatory context (so future readers don't re-introduce the
mistake by reading the engine code in isolation).
QA pre-work artifact V:\share\g5-test\scenarios\g7-helpers.sh is
already written against the corrected literals; this commit
brings the §harness-notes source-of-truth into alignment.
Standing by for architect §1.A ratification (Q1 topology / Q2
fold-G6 / Q3 deadline / etc.) before §1.H code-start audit.
|
||
|
pingqiu
|
4a876a9cd2 |
G6 §close + 5 INVs inscribed in ledger + roadmap closure
m01 single-run GREEN at 71 s on seaweed_block@96c51b4 — both §2 #4 (retention-OK catch-up) AND §2 #5 (sustained-write recycle → rebuild dispatch + 5000-LBA byte-equal) in one closed-loop run per architect §2 #6 binding. Logs: V:\share\g5-test\logs\g6-20260428T100217Z.log Scenarios: V:\share\g5-test\scenarios-g6.sh + scenarios\g6-d.sh Mini-plan §close: - §close.summary: 8-row table of bindings + commits + hardware + regression status, all GREEN. - §close.evidence: software-pin (3 commits, 10 tests / 14 cases PASS); hardware-pin (5 acceptance rows, all GREEN; single 71 s run). - §close.deltas: 2 entries documenting (a) physical-recycle NOT required for §2 #5 (engine recovery decision branch is load-bearing) and (b) harness discipline finding from QA. - §close.findings: 2 findings — (1) data-vs-state convergence harness discipline → new INV; (2) §1.H audit verdict was correct + resolved in-batch. - §close.forward-carries: G5-2/G5-6 (durability mode), G5-3 (peer-state surface), future replica-aware retention (β/γ), G7 (rebuild path semantics). 5 INVs inscribed in v3-invariant-ledger.md: - INV-G6-WALRECYCLE-DISPATCHES-REBUILD - INV-G6-CATCHUP-CONVERGES-WITHIN-RETENTION - INV-G6-RETENTION-POLICY-OPERATOR-VISIBLE - INV-G6-ENGINE-NO-REBUILD-PINNED-ON-OTHER-FAILURES - INV-G6-HARNESS-DATA-AND-STATE-CONVERGENCE (NEW from §close.findings #1) INV-G6-RETENTION-POLICY-REPLICA-AWARE NOT inscribed — reserved for future β/γ replica-aware retention batch (architect §1.A α ratification 2026-04-29). Roadmap §3 G6 line: ⏳ next → ✅ closed 2026-04-28 (retention-aware recovery; α config knob + escalation pin). Roadmap §7: G6 row added to recently-closed table. Roadmap §8 backlog: G6-T-WALRECYCLE-ESCALATE → "Closed backlog tickets" section with verdict (a) + resolution narrative. Awaiting architect single-sign on this §close. |
||
|
pingqiu
|
050c3ff875 |
G6 mini-plan v0.1 (kickoff draft for architect ratification)
Per architect ruling 2026-04-28 on G6 scope (post-G5-5C close): Bindings absorbed: - G6-T-WALRECYCLE-ESCALATE folded into G6 main acceptance, not a separate sub-batch (architect ruling #1). - §1 AC = single closed-loop covering retention-OK catch-up + recycle-triggered escalation in ONE hardware scenario (architect ruling #1, "现象上是一件事, 不重复跑 sustained"). - §1.A = WAL retention policy options (α config knob / β pin-window / γ replica-watermark-driven). sw recommends α for smallest diff + fastest ratification; β/γ are richer, naturally G6-followup territory if escalation path proves clean first. - §1.H = audit-then-decide on code surface; do NOT pre-declare "zero code". Three possible verdicts: verify-only / minor-patch / engine-evolution-batch (halt condition). §2 acceptance criteria (8 items): - #1 §1.H audit published as commit note before any production code. - #2 §1.A bound + landed. - #3 engine-layer dispatch test pinning RecoveryFailureWALRecycled → RebuildPinned=true → next decide() emits StartRebuild. - #4 hardware retention-OK catch-up GREEN. - #5 hardware recycle-escalation GREEN (rebuild dispatch within deadline OR documented operator-failure-mode if §1 binds rebuild-as-NON-GOAL — architect's product-句 escape clause). - #6 #4 + #5 pass in SAME hardware run (one closed-loop AC). - #7 no regression on G5-5C 6-step suite. - #8 zero diff under master/authority/proto (carries INV-G5-5C-NO-MASTER-PROTOCOL-CHANGE discipline). §3 INVs to inscribe at close (4 always + 1 conditional): - INV-G6-WALRECYCLE-DISPATCHES-REBUILD - INV-G6-CATCHUP-CONVERGES-WITHIN-RETENTION - INV-G6-RETENTION-POLICY-OPERATOR-VISIBLE (only if α) - INV-G6-RETENTION-POLICY-REPLICA-AWARE (only if β/γ) - INV-G6-ENGINE-NO-REBUILD-PINNED-ON-OTHER-FAILURES Forward-carry from G5-5C consumed (§5): - G6-T-WALRECYCLE-ESCALATE — primary scope of this batch. - Evidence: V:\share\g5-test\logs\bcd-20260428T072539Z.log D-section. - QA wait_until_rebuild_dispatched helper held until §1-§6 ratified. §7 sign table awaits architect §1-§6 ratification (especially §1.A α/β/γ pick) before sw runs §1.H audit. Standing by for architect ratification. |
||
|
pingqiu
|
1207fc5444 |
Roadmap §8: queue G6-T-WALRECYCLE-ESCALATE backlog ticket from G5-5C QA scenario D
Per architect ruling 2026-04-28 + sw §close.appendix: D's WALRecycled boundary finding is G6 territory, not a G5-5C reopener. Adding the backlog ticket here so it doesn't get lost between G5-5C close and G6 kickoff. Ticket text + evidence pointer + cross-references all preserved from the §close.appendix; this is the dev-roadmap-side mirror so the ticket surfaces when planning G6 scope. Standing by for architect final §close single-sign on G5-5C. |
||
|
pingqiu
|
5069e74445 |
G5-5C §close.appendix: QA scenario expansion (B/C confidence + D → G6 carry)
Per architect ruling 2026-04-28 on QA's expanded scenario report: - A (capacity): 🐛 → ✅ already-fixed at seaweed_block@a250b52, INV inscribed. - B (500 random LBAs over 65536-LBA volume): ✅ GREEN. Confidence bump on dirty-map skew + ship order under random write pattern. - C (kill replica mid-write-storm + restart + 200 LBAs converge): ✅ GREEN. Highest-signal recovery scenario in the expansion; validates G5-5C peer-recovery trigger under load. - D (5000-LBA sustained write → WALRecycled past replica LSN): 🐛 boundary finding. Architect: G6 territory, not G5-5C reopener. Catch-up requires WAL retention; rebuild path is for gap-beyond- WAL. Engine has dispatch-branch tests (Batch 4); runtime escalation path under sustained pressure is G6 acceptance scope. Doc updates: - New §close.appendix table with all 4 scenario rows + dispositions. - Semantic clarification on D — catch-up vs WAL recycle vs rebuild. - §close.forward-carries gets a NEW G6 entry with backlog ticket text, evidence pointer, cross-reference to INV-G5-5C-PROBE-BEFORE- CATCHUP, and explicit non-reopener rationale. - Logs + scenario script paths recorded for QA continuity. §close substance unchanged: G5-5C gate (verify_restart_catchup GREEN within 30 s) was met on the canonical case at seaweed_block@712cbc47 + capacity addendum at a250b52. B/C are strengthening, not gating; D is forward-carry. Awaiting architect final §close single-sign on this tree. |
||
|
pingqiu
|
a5c39fde34 |
G5-5C addendum: inscribe INV-G5-FRONTEND-CAPACITY-FROM-DURABLE-CONFIG
Per architect ruling 2026-04-28 + sw addendum landing at seaweed_block@a250b52: inscribe new INV in the ledger. Statement: iSCSI/NVMe externally-visible volume capacity and block size MUST derive from --durable-blocks × --durable-blocksize when --durable-root is set, not silently fall back to frontend defaults (DefaultVolumeBlocks=2048 × DefaultBlockSize=512 = 1 MiB). Without this plumb-through, a daemon configured for N MiB durable storage advertises a 1 MiB iSCSI/NVMe LUN and any workload above LBA 256 fails. Test pointers: cmd/blockvolume/frontend_capacity_test.go (6 tests: ProductOfBlocksAndBlockSize, RejectsZero, OverflowGuard, IscsiHandlerCapacity, NvmeHandlerCapacity, FrontendDefaults_ StillReturn1MiB). Source-side: cmd/blockvolume/main.go:: computeFrontendVolumeSize flows into both iscsi.TargetConfig and nvme.TargetConfig handler. First introduced: P15 G5-5C addendum (P0 product fix). Owner layer: host (binary, frontend wiring). Last verified: 2026-04-28 (G5-5C addendum P0; m01 hardware re- verification pending QA). Status: ACTIVE. Awaiting m01 hardware re-run for full §close ledger update. |
||
|
pingqiu
|
9a2c939b9a |
G5-5C §close: ALL 6 m01 hardware verify steps GREEN — L4 reached
m01 hardware run 4 at seaweed_block@712cbc47 (with Batch #7 per-peer adapter wiring) — full results: #1 verify_cluster_ready ✅ GREEN #2 verify_byte_equal ✅ GREEN #3 verify_network_catchup ✅ GREEN (9s) #4 verify_restart_catchup ✅ GREEN (9s) ← Batch #7 unblocked #5 verify_race_stress (×10) ✅ GREEN #6 verify_full_suite ✅ GREEN §close updated: - Header: closes at L4 Replicated IO with peer-restart resilience. - §close.evidence hardware-pin row table: run 4 results. - Earlier-runs row table preserved for artifact retention (run 2 port-release race; run 3 per-peer adapter gap; both root-caused and fixed). - §close.findings 'per-peer adapter gap' marked RESOLVED by Batch #7. - §close.deltas: forward-carry to G5-5D dropped (absorbed in-batch). - §close.forward-carries: G5-5D removed; only G5-5 deferred ledger pointers + G5-2/G5-3/future master observability remain. - architect-review-checklist: scope truth, engine impact, product level all updated to reflect L4 reached on hardware. INV-G5-5C-PER-PEER-ADAPTER-PER-PEER-ENGINE inscribed at this close (no longer deferred). Awaiting QA evidence verification + architect single-sign per v3-batch-process.md §5 + §8C.2. |
||
|
pingqiu
|
88cff6145c |
G5-5C: §1.I plan extension for Batch #7 (per-peer adapter wiring)
Architect approved Option B 2026-04-27: absorb the hardware-revealed gap into G5-5C as Batch #7 instead of carrying to G5-5D. §1.I scope: - core/host/volume/peer_command_executor.go (NEW, ~120 LOC) - core/host/volume/peer_adapter_registry.go (NEW, ~100 LOC) - core/replication/volume.go ConfigurePeerLifecycleHook (~30 LOC) - core/host/volume/probe_loop_wiring.go router signature (~20 net) - cmd/blockvolume/main.go registry wire-up (~20 net) - ~10 new tests, ~250 LOC test code INV INV-G5-5C-PER-PEER-ADAPTER-PER-PEER-ENGINE absorbed back in-batch (was previously deferred to G5-5D in pre-architect-ruling draft). Pass criterion unchanged: m01 verify_restart_catchup GREEN within 30s deadline; #1-#3 regression GREEN in the same run. §close updated: ceremony waits for Batch #7 land + hardware re-run; G5-5C closes at full L4 in one shot. |
||
|
pingqiu
|
389896b5e4 |
G5-5C §close: m01 #1-#3 GREEN, #4 RED — hardware-revealed gap, carries to G5-5D
m01 hardware run 3 at seaweed_block@ac9392d: - #1 verify_cluster_ready ✅ GREEN - #2 verify_byte_equal ✅ GREEN - #3 verify_network_catchup ✅ GREEN (9s) - #4 verify_restart_catchup ❌ RED (30s timeout) Root cause (verified in code + log): Primary log shows probe loop fired correctly post-restart and the wire probe SUCCEEDED twice (R=2 S=1 H=3), but no StartCatchUp ever dispatched. Engine apply.go:117-128 checkReplicaID drops events whose ReplicaID doesn't match the adapter's tracked Identity — cmd/blockvolume's host adapter tracks the PRIMARY'S OWN slot (ReplicaID=r1), not peer r2. Probe results for r2 are correctly dropped as wrong_replica. Component test (Batch #6) passed because cluster.go's WithEngineDrivenRecovery constructs c.primary.adapters[] — one per peer. cmd/blockvolume only constructs ONE adapter for the host's own slot. The component test exercised a different (architecturally-correct) wiring than production has. §1.H audit verdict was correct on engine SEMANTICS; it did not extend to whether the production binary CONSTRUCTS per-peer engine state. That layer was assumed; hardware revealed the assumption. §close decision: - G5-5C software pieces all sound, stay landed (50 unit + integ tests PASS; full ./... regression PASS). - Hardware finding carries to G5-5D — Per-peer adapter wiring for primary-side recovery dispatch. - G5-5D pass criterion = exact verify_restart_catchup case from this run; seed evidence = sw-block/design/g5-artifacts/primary-fail.log. - New INV to inscribe at G5-5D close: INV-G5-5D-PER-PEER-ADAPTER-PER-PEER-ENGINE. Doc updates: - §close.evidence: hardware-pin row table filled with run 3 results. - §close.deltas: 3 implicit assumptions surfaced. - §close.findings: 2 findings (#1 per-peer adapter gap; #2 script port-release race already fixed). - §close.forward-carries: G5-5D added as named carry. - architect-review-checklist: scope/audit/engine-impact/product level all updated to reflect actual reached state (L3+, not L4). Awaiting architect ratification of G5-5D scope at single-sign or earlier; sw drafts G5-5D mini-plan once architect rules. |
||
|
pingqiu
|
a15d13a02c |
G5-5C §close skeleton: software pin + hardware-pin TBD rows
Per v3-batch-process.md §2: §close drafted as soon as software is ready. Hardware row table left as TBD; sw fills evidence pointers once iterate-m01-replicated-write.sh completes. Forward-carries + deferred ledger pointers + architect-review-checklist all populated based on G5-5C scope already in-batch. Awaiting: 1. m01 hardware run completion → fill #1-#4 evidence rows 2. QA evidence verification → §close.deltas / findings if needed 3. architect single-sign per v3-batch-process.md §5 + §8C.2 |
||
|
pingqiu
|
9245446b59 |
G5-5C §1.H code-start audit: PROCEED — all halt-conditions clear
Per v0.5 §1.H step 3, sw publishes audit findings as a commit note
before any G5-5C production code change.
AUDIT METHOD: greped seaweed_block/core/{engine,replication,adapter}
for the structural backing of each in-scope INV; cited apply.go +
state.go + replication/volume.go + adapter/adapter.go line numbers
as evidence.
PER-INV FINDINGS:
[1] INV-G5-5C-PRIMARY-RECOVERY-AUTHORITY-BOUNDED
Owner: core/replication/volume.go (ReplicationVolume.peers map)
Status: ✅ PASS. peers map is sole probe target collection;
UpdateReplicaSet is sole mutator and is master-fact-driven only.
Halt-cond cleared.
[2] INV-G5-5C-GENERATION-FENCE
Owner: core/engine/apply.go:132-166 (stale event rejection) +
state.go:24-32 (IdentityTruth.{Epoch, EndpointVersion} carrier)
Status: ✅ PASS. Engine rejects events with epoch < Identity.Epoch
or (epoch == AND ev < Identity.EndpointVersion). identityChanged
triggers wholesale Recovery reset (line 166-169). Fence is
carried on engine state, not re-derived per call site.
Halt-cond cleared.
[3] INV-G5-5C-SINGLE-INFLIGHT-PER-PEER
Owner: core/engine/state.go:144-151 (SessionTruth single-slot) +
apply.go phase-guards at 183/236/364/417/442/455/472/507/536
Status: ✅ PASS. ReplicaState.Session is one slot per peer.
Engine FSM handlers explicitly skip / reject when Phase is
PhaseStarting or PhaseRunning. apply.go:536 "Skip if a rebuild
session already exists" pinned. In-flight is engine-explicit,
not implicit. Halt-cond cleared.
[4] INV-G5-5C-PROBE-BEFORE-CATCHUP
Owner: core/engine/state.go:84-121 (RecoveryTruth) +
decide() probe-driven decision path
Status: ✅ PASS. RecoveryTruth.Decision is derived from R/S/H
(boundaries from probe), NOT from transport reachability.
Engine's RebuildPinned guard prevents stale auto-probe from
downgrading Rebuild back to CatchUp mid-flight (line 105-120).
Halt-cond cleared.
[5] INV-G5-5C-RECOVERY-BACKOFF
Owner: engine retry budget (state.go:91-103
RecoveryTruth.Attempts + RuntimePolicy.MaxRetries from T4c-3) +
NEW G5-5C runtime cooldown (5s base → 10s → 20s → 40s → 60s cap;
reset on success)
Status: ⚠ PARTIAL — engine has retry budget but no exponential
cooldown. G5-5C adds the cooldown as a primary-runtime policy on
top of engine retry budget. NOT an engine FSM change. Acceptable
under §1.H "minimum evolution" criterion. Halt-cond cleared.
[6] INV-G5-5C-STALE-ACK-NO-HEALTH-PROMOTION
Owner: core/engine/apply.go:766-789 (Healthy gate)
Status: ✅ PASS. Healthy = true requires three conjuncts:
(a) Recovery.Decision == DecisionNone, (b) Reachability.Status
== ProbeReachable, (c) Identity.Epoch <= Reachability.FencedEpoch.
A barrier ack with AchievedLSN < TargetLSN does not transition
SessionTruth, decide() does not flip Decision to None on
insufficient achieved LSN — Healthy stays false. Halt-cond
cleared.
OVERALL VERDICT: PROCEED.
All six in-scope INVs have their backing infrastructure in engine
(state.go + apply.go) or replication (volume.go). G5-5C is a runtime
wiring batch + small policy extension (backoff). No engine FSM
rewrite needed. No halt-condition fires; no engine-evolution
mini-plan required.
NEXT STEP: implement primary-side probe loop +
ReplicaPeer.ProbeIfDegraded() + lifecycle/cooldown/dispatch tests +
component test, all under core/replication/. Probe loop owned by
ReplicationVolume lifecycle per architect binding. Test method
names to be concretized as code-start commit-note addendum to §2.
This audit commit fulfills §1.H step 3 (audit findings published) +
§2 #15 (audit commit note before production code).
|
||
|
pingqiu
|
74e92b974d |
G5-5C mini-plan v0.4.5 → v0.5: single-sign recorded + §1 scope-rule one-liner
Architect single-signed §1-§6 at seaweedfs@ba7bd0ba4 2026-04-27 with: - Option B trigger source (primary-side degraded-peer probe loop) - Probe loop placement = core/replication/ owned by ReplicationVolume - Master protocol unchanged - §1.H code-start audit gate before code This commit: 1. Records the single-sign in the doc header. 2. Adds a §1 scope-rule one-liner near the top so future readers find the architect-bound boundary without re-reading the v0.1→v0.5 trail: "master owns identity/topology; primary+engine own data recovery; the protocol aligns the two via (PeerSetGeneration, epoch, EndpointVersion) fences." §1.A already bound Option B in v0.4; no flip needed there. No design change. §1.H audit is the next sw step before any production code. |
||
|
pingqiu
|
ba7bd0ba48 |
G5-5C mini-plan v0.4.4 → v0.4.5: doc-hygiene cleanup + probe loop placement bound
Architect approves v0.4.4 substance (Option B; master unchanged; no PeerSetGeneration change) but requires five doc-hygiene fixes before single-sign: 1. §1 #3 V2 path "weed/server/" → V3 "core/replication/" + reword from "shipper re-arms" V2 vocabulary to "probe loop detects degraded peer reconnection". 2. §1 Architecture truth-domain check: dropped v0.3 / A1 / "publication / re-emission" residue. Now points cleanly to §1.C. 3. §2 "#3a/#3b/#4" v0.2 naming residue: rewritten to reference acceptance criteria #2-#15 with package-level verifier files (peer_test.go, probe_loop_test.go, volume_test.go, component/). Test method names concretized at code-start as commit-note addendum (no re-ratification needed). 4. Architect review checklist "Engine / adapter impact" reworded: "No new engine recovery primitive by default; engine-owned fences/state audited at §1.H code-start; if found insufficient, sw halts G5-5C and starts engine-evolution mini-plan rather than layering ifs in core/replication/." 5. §1.A loop owner row bound: probe loop placement = core/replication/ owned by ReplicationVolume lifecycle (NOT host layer). Reasoning: admitted peers + peer state + close/teardown + in-flight guard all in core/replication/; host only forwards flags/config. §1.H halt rule preserved: audit may still escalate to engine-evolution. §7 sign table records substance approval 2026-04-27 + probe loop placement binding + awaits single-sign of v0.4.5. Standing by for architect single-sign. |
||
|
pingqiu
|
9b6e103dde |
G5-5C mini-plan v0.4.3 → v0.4.4: engine/runtime/master split + 6 boundary rules in scope, 3 forward-carry, audit gate
Architect framing 2026-04-27: enumerate ten protocol boundary rules and address engine-evolution question. Engine vs primary runtime vs master split: - Engine owns: recovery FSM, single in-flight per peer, generation/epoch fence, probe→decision, backoff/cooldown policy, stale-ack-cannot-promote-health rule, recovery reason / projection - Primary runtime/adapter owns: timer / degraded-peer loop, transport probe execution, feeding probe result into engine, executing engine-emitted commands, ReplicationVolume / ReplicaPeer connection lifecycle - Master owns: identity / topology / assignment / health observation ONLY. No runtime recovery. No epoch bumps for short up/down. Six in-scope boundary rules (#1, #2, #3, #4, #7, #8): - #1 Admitted Peer Rule — already INV-G5-5C-PRIMARY-RECOVERY-AUTHORITY-BOUNDED - #2 Generation Fence — NEW INV-G5-5C-GENERATION-FENCE - #3 Single In-Flight Per Peer — NEW INV-G5-5C-SINGLE-INFLIGHT-PER-PEER - #4 Probe Before Catch-Up — NEW INV-G5-5C-PROBE-BEFORE-CATCHUP - #7 Backoff/Cooldown — NEW INV-G5-5C-RECOVERY-BACKOFF (extends v0.4 fixed-5s into 5s→10s→20s→40s→60s cap, reset on success) - #8 Stale Ack Guard — NEW INV-G5-5C-STALE-ACK-NO-HEALTH-PROMOTION (cross-refs G5-5 round-14 gate-degraded artifact) Three forward-carries OUT of G5-5C (per §5): - #5 Durability Mode Explicit → G5-2 / G5-6 - #6 RF Health Reporting Separate From Recovery → future master observability batch - #10 Status Surface (recovery reason, effective RF, last probe) → G5-3 metrics/backpressure One citation (#9 Replica-side lineage check): already enforced by T4 acceptMutationLineage gate; G5-5C cites, no new code. §1.H code-start audit gate: sw audits per-INV current owner location BEFORE writing any code. Halt-condition: if recovery FSM is embedded in ReplicationVolume, fence is re-derived per call site, in-flight is implicit, or stale-ack guard is missing — sw stops and re-scopes as engine-evolution batch instead of layering ifs in core/replication/. Audit findings published as commit note pre-code; PR includes audit-summary. §2 acceptance criteria: add #13 (stale-ack guard), #14 (backoff progression), #15 (code-start audit). Acceptance count now 15 covering 7 INVs (6 new + reconnect orthogonality from v0.4.3). Standing by for architect single-sign of v0.4.4. |
||
|
pingqiu
|
13eb8181d3 |
G5-5C mini-plan v0.4.2 → v0.4.3: add §1.F reconnect orthogonal axes
Architect framing 2026-04-27 (sharpening v0.4.2): reconnect splits
along two orthogonal dimensions — connection recovery vs identity /
lineage change. Each axis has different protocol semantics; G5-5C
must handle both correctly.
Architect's protocol judgment points:
1. PeerSetGeneration only changes for identity / address / lineage
change. Brief disconnects / restarts / freshness flapping do NOT
bump generation.
2. Primary's degraded-peer loop only acts on currently-admitted peers
(§1.E reaffirmed).
3. After reconnect, primary still probes R/S/H — reconnect alone is
not assumed sufficient.
4. If a higher PeerSetGeneration arrives during reconnect / probe,
the in-flight recovery must stop or invalidate.
Changes:
- New §1.F with two cases:
Case 1 (identity unchanged): primary retries existing peer
descriptor; new sessionID minted (sessions are session-scoped, not
peer-scoped); probe R/S/H; catch-up / rebuild as needed; no master
re-emit needed. This is G5-5C's core path.
Case 2 (identity changed): existing UpdateReplicaSet T4a-5 path
(volume.go:229-246) tears down + recreates; in-flight aborts via
Close(); new peer with new lineage takes over.
- Misread guards documented: "primary keeps retrying old address
forever" rejected by Case 2 + §1.E (c); "master must bump on every
blip" rejected by Case 1 + §1.D.
- New INV-G5-5C-RECONNECT-ORTHOGONAL-AXES in §3.
- New §2 #11 (reconnect Case 1 — identity unchanged, no re-emit) and
§2 #12 (reconnect Case 2 — lineage bump mid-flight).
This is structural reaffirmation: the V3 code already does Case 2
correctly (T4a-5 teardown). Case 1 is what the probe loop adds. The
new tests pin both axes against future drift.
Standing by for architect single-sign of v0.4.3.
|
||
|
pingqiu
|
5cf429595f |
G5-5C mini-plan v0.4.1 → v0.4.2: add §1.E authority-bounded primary recovery invariant
Architect framing 2026-04-27 (sharpening v0.4.1): §1.D ordering-
independence must NOT be misread as "primary may self-discover and
connect to any replica it sees on the network." Tighten with a
second protocol invariant.
Rule (architect verbatim): "Primary recovery loop may retry only peers
that were previously admitted by a master-issued assignment fact for
the current authority lineage."
Layering: master establishes identity once; primary owns retry /
recovery for that admitted peer until master revokes or changes the
assignment.
This is structurally true in V3 today (probe loop reads
ReplicationVolume.peers, which UpdateReplicaSet populates from master
facts) but v0.4.2 promotes it from implementation detail to protocol
invariant so future contributors don't widen the probe surface.
Changes:
- New §1.E with three scenarios:
(a) first-time replica join — disallowed without master fact
(b) brief outage + recovery (G5-5C core case) — allowed without
master re-emit
(c) epoch / assignment change — probe must stop; in-flight aborts
- Implementation requirement made explicit: ReplicaPeer.Close() must
abort in-flight probe synchronously.
- Authority alignment surface table: replicaID/epoch/EV → identity;
AssignmentFact.Peers → only legal probe targets;
PeerSetGeneration → existing lastAppliedGeneration guard preserved.
- New INV-G5-5C-PRIMARY-RECOVERY-AUTHORITY-BOUNDED in §3.
- New §2 #9 (authority-bounded targets test) and §2 #10 (lineage-
change-during-probe test).
- §1.A bound-shape Master-interaction row references §1.E.
- §1 Files peer.go row notes Close() must abort in-flight probe.
Standing by for architect single-sign of v0.4.2.
|
||
|
pingqiu
|
aebf668094 |
G5-5C mini-plan v0.4 → v0.4.1: add §1.D two-feedback-loop ordering-independence invariant
Architect framing 2026-04-27: when a replica goes down or recovers, both the control-plane identity/health loop and the data-plane governance loop receive feedback. Protocol must treat them as two independent loops with no ordering dependency, alignment via durable identity facts (replicaID/epoch/EV/peer address), and idempotency on primary-side dispatch absorbing duplicate triggers. This is a sharpening of v0.4, not a re-bind. Design unchanged: Option B primary-side probe loop, no master protocol change. Changes: - New §1.D: explicit two-loop table, five ordering scenarios all ending safe, anti-requirements (master re-emit NOT prerequisite, primary recovery NOT blocked on master), idempotency guarantees, future RF-health observability noted as different-batch scope. - New INV-G5-5C-TWO-LOOPS-ORDERING-INDEPENDENT in §3 with test pointer (peer_test.go simultaneous-fire test). - New §2 #8 acceptance criterion: unit test exercising the "simultaneous-fire" case (concurrent fact replay + concurrent ProbeIfDegraded on same degraded peer; idempotent absorption). Standing by for architect single-sign of v0.4.1. |
||
|
pingqiu
|
4f6e5d3e6a |
G5-5C mini-plan v0.3 → v0.4: retire Option A, bind Option B per layering correction
Architect re-ruling 2026-04-27: control-plane / data-plane layering. Master must own identity / topology / address / RF-health; it must NOT own runtime recovery scheduling. v0.2/v0.3's Option A (master observation-driven re-emission) forces master into recovery-scheduling territory and forces PeerSetGeneration to carry two distinct semantics (authority version + peer-set-view version). That's the wrong shape: master gets heavier; control-plane heartbeat cadence couples to data-plane recovery cadence; protocol cleanliness erodes. Bind Option B (primary-side degraded-peer probe loop) with explicit constraints. No master protocol change. Changes: - §1.A rewritten: Option B bound shape (only-on-degraded, 5s interval, per-peer cooldown, in-flight guard, max-concurrent-probes=1, CP4B-2 lifecycle discipline). Why-A-retired + Why-C-rejected sections. - §1.B replaced: master protocol explicitly unchanged. v0.3's PeerSetRevision proto field, ObservationStore.obsRev counter, and UpdateReplicaSet lex-compare upgrade — all three retired. - §1.C replaced: truth-domain matrix shows zero master-side write; one truth domain (primary data-control) writes; all others untouched. - §1 Files retired master-side rows; replaced with primary-side probe loop infrastructure (peer.go probe entry + replication probe loop + flags + lifecycle/cooldown/dispatch tests + component test). Total ~225 prod + ~310 test, all primary-side. Zero LOC master / proto. - §2 acceptance criteria rewritten: lifecycle correctness, cooldown + in-flight TOCTOU, dispatch branches, hardware GREEN. New criterion #7: zero diff under core/host/master/, core/authority/, proto/. - §3 INVs replaced: drop INV-MASTER-PEER-SET-GEN-REV-MONOTONIC; add INV-REPL-PEER-RECOVERY-PROBE-LOOP-001, retain INV-REPL-PEER-RECOVERY-NO-RETRIGGER-LOOP, add INV-G5-5C-NO-MASTER-PROTOCOL-CHANGE (anti-creep guard). - §6 risks rewritten around probe loop concerns: lifecycle bugs (CP4B-2 lessons), cooldown tuning, in-flight TOCTOU, scope-creep prevention via §3 INV + §2 #7 diff inspection. - §5 forward-carry: trigger source disposition updated to Option B. - §7 sign table records full ruling history v0.1 → v0.2 → v0.3 → v0.4 with retire/keep markings; awaiting single-sign of v0.4. Standing by for architect single-sign of v0.4. |
||
|
pingqiu
|
900e4d0cb3 |
G5-5C mini-plan v0.2 → v0.3: V3 paths + peer-set generation design + truth-domain wording
Architect REVISE ruling on v0.2 — three items, all addressed:
1. V3 paths (was: V2 weed/server + weed/storage/blockvol).
v0.3 §1 Files table corrected to seaweed_block paths:
- core/rpc/proto/control.proto (proto field add)
- core/host/master/services.go (A1 re-emission)
- core/authority/observation_store.go (obsRev tracking)
- core/replication/volume.go (lex compare in UpdateReplicaSet)
- core/host/volume/host.go (applyFact dispatch wiring)
- core/replication/peer.go (OnReappeared entry point)
- core/host/volume/apply_fact_test.go + master/services_test.go +
replication/volume_test.go + replication/component/...
Header now states Repo: seaweed_block (V3) explicitly.
2. Peer-set generation design (was: missing).
New §1.B enumerates three options the architect named (master-
maintained counter / observation revision folded / separate field)
with concrete V3 mechanics + tradeoff matrix. sw recommends
Option γ (separate PeerSetRevision field alongside existing
PeerSetGeneration). Stale-drop hazard cited at
replication/volume.go:194-209. UpdateReplicaSet stale-replay rule
becomes lex compare on (generation, revision). Open architect
choice within γ: per-slot vs per-volume rev (sw proposes per-volume
max).
New INV-MASTER-PEER-SET-GEN-REV-MONOTONIC inscribed in §3 with
test pointers for revision bump + lineage reset + lex-compare
stale-drop.
3. Truth-domain wording (was: A1 = "read").
New §1.C corrects: A1 is publication / re-emission of master truth,
not pure read. Remains authority-safe (no new lineage invented).
Per-domain matrix replaces v0.2's bullet list.
§2 acceptance criteria #2/#3 updated to reference (PeerSetGeneration,
PeerSetRevision) lex-compare semantics and pin V3 test file paths.
§6 risks add two new entries: obsRev overflow (none) + master-restart
revision reset (mitigation: first-attach bootstrap clears
lastAppliedGeneration/lastAppliedRevision). §7 sign table records
absorbed REVISE items + open single-sign.
Standing by for architect single-sign of v0.3.
|
||
|
pingqiu
|
b6267d8af7 |
G5-5C mini-plan v0.1 → v0.2: bind trigger source to Option A (A1+A2)
Architect REVISE ruling 2026-04-27: bind trigger source to Option A with both halves in scope (no split into G5-5B). Reject B and C. QA review v0.1 flagged: master-side scope must be explicit; pin §5 evidence path. Changes: - §1.A: collapse three-option proposal to bound Option A. Make A1 (master-side observation-driven re-emission) and A2 (primary-side recovery dispatch) explicit as two halves of one causal chain. Record B/C rejection rationale for future reference. - §1 Files: revise table with Side column (master/primary). Add master-side rows (A1 re-emit logic + ObservationStore freshness helper). Total estimate ~360 prod + ~150 test, split master ~90 / primary ~120 / tests ~150. - §2: rewrite criteria #1-#5 around bound Option A (drop per-Option deadline language). Split #2/#3 into A1 master-side + A2 primary-side criteria. Hardware deadline at #5 stays 30s. - §2 verifier note: file paths + test names pinned at code-start (acceptable for v0.2 per QA review). - §5: pin G5-5 seed evidence to actual artifact path V:\share\g5-test\logs\artifacts-20260427T092858Z\primary-fail.log (no future task — fact-pointer). - §7: trigger-source binding row marked done (architect REVISE); single-sign of v0.2 still pending. - Header: v0.1 → v0.2 status note updated. Standing by for architect single-sign of v0.2. No code starts until single-sign. |
||
|
pingqiu
|
d6a2fb92d6 |
G5-5 close handoff: roadmap update + G5-5C mini-plan v0.1 kickoff
Per architect single-sign of G5-5 §close (`seaweedfs@c78116fd2`): (a) v3-dev-roadmap.md - §3: G5 line note now mentions G5-5 closed at L3 + G5-5C carry-forward - §4: G5-5 row → CLOSED (link to seaweedfs@c78116fd2); G5-5C row added as next active gate with bound pass criterion - §7: G5-5 close commit appended to recently-closed table (seaweed_block@5c4718f + seaweedfs@c78116fd2, L3 reached, #4 carry) (b) v3-phase-15-g5-5c-mini-plan.md (new) v0.1 kickoff - §1 scope: peer recovery trigger after replica restart; reuse T4d-4 primitives (architect binding); no engine logic change - §1.A: three trigger source options (A master observation, B periodic probe, C transport reconnect) with tradeoffs; sw recommends A; final pick deferred to architect ratification - §2: 6 acceptance criteria, hardware step is exactly G5-5 #4 (verify_restart_catchup → GREEN with no harness changes) - §3: 2 new INVs proposed (REPL-PEER-RECOVERY-TRIGGER-001 + -NO-RETRIGGER-LOOP) + 2 deferred ledger pointers from G5-5 close - §4: G-1 N/A (new build, no V2 PORT) - §5: forward-carries from G5-5 §close all addressed - §6: 5 risks tabled - §7: sign table awaiting architect §1-§6 ratification including trigger source pick Standing by for architect ratification of trigger source binding. No code starts until §1-§6 signed. |
||
|
pingqiu
|
c78116fd2f |
G5-5 §close doc-fix #2: drop stale 'blocked' sign-table rows
Architect's round-15 hygiene callout: §7 sign table still had three pre-code 'blocked' rows after the real close-state rows landed in the prior doc-fix commit. Pure leftover from before the close-state update overwrote earlier rows but didn't delete the trailing pre-code rows. Removed: - 'Code start (script + Go helper) ... blocked on ratification' - 'm01 hardware verification run ... blocked' - '§close append + close sign ... blocked' Sign table now ends cleanly at the §close architect single-sign pending row. Ready for sign. |
||
|
pingqiu
|
12fcdb41f8 |
G5-5 §close doc-fix: forward-carry text + sign-table state + ledger update + header
Architect ratification round 14: substance approved, but doc-fix required before single-sign. Four hygiene fixes: 1. §5 forward-carry consumed: was 'both paths consumed; neither carries forward'. Now correctly states process-restart path FAILED on m01 hardware and carries to G5-5C (matches §close substance and architect ruling 2026-04-27). 2. §7 sign table: stale pre-code rows replaced with actual close state (ratification ✅, code ✅ landed at seaweed_block@2745cf4 et seq, m01 verification ✅ rounds 1-14, §close submitted ✅, architect single-sign ⏳ pending). 3. §3 'Invariants whose ledger row updates at G5-5 close' had placeholder 'Last verified → 2026-04-DD' text. Now reflects actual ledger updates landed in this same commit. Ledger updated: 5 INV-BIN-WIRING-* rows now show Last verified= 2026-04-27 (G5-5 §close — Tier 2 m01 cross-node hardware Integration backstop: seaweed_block@5c4718f rounds 1-14). T4 invariants (INV-REPL-CATCHUP-FROMLSN-IS-REPLICA-FLUSHED- PLUS-1, INV-REPL-LSN-ORDER-FANOUT-001) deferred to G5-5C close (single Integration row update covering #2 + #4 together, since #4's verify lands at G5-5C). 4. Header: DRAFT v0.3 → §close submitted, awaiting single-sign. 5. Bottom 'Next actions' table: stale pre-code routing rows replaced with post-§close routing (architect single-sign, sw roadmap update + G5-5C mini-plan, QA optional clean run). No substance change. Pure doc hygiene. After this commit architect can single-sign per v3-batch-process.md §5 + §8C.2. |
||
|
pingqiu
|
54feecb31d |
G5-5 §close: 3 of 4 verify steps GREEN on m01 hardware; #4 carry → G5-5C
§close summary per v3-batch-process.md §12 template:
Done:
- #1 verify_cluster_ready
- #2 verify_byte_equal — live iSCSI replicated write, byte-equal
verified via storage-aware m01verify (LBA[0]=0xab on cross-host
hardware)
- #3 verify_network_catchup — iptables disconnect+heal, replica
converges to LBA[1]=0xcd byte-equal in 8s via engine-driven
catch-up
- 14 bugs surfaced+fixed across 14 m01 self-iteration rounds
- 5 INV-BIN-WIRING-* invariants in v3-invariant-ledger.md from
G5-4 still load-bearing; G5-5 hardware run is Integration backstop
Not done:
- #4 verify_restart_catchup — kill replica + write while down +
restart: replica's LBA[2]=0xef does NOT converge in 30s. Per
architect ruling 2 (round 13): real recovery-path finding,
surface as G5-5C carry-forward.
- #5 verify_race_stress + #6 verify_full_suite — gated on #4 fix
or test sequencing rework.
Product level reached: L3 (Replicated IO) per v3-architecture.md §13.
Falls short of full L4 (Failure/recovery under IO) — process-restart
recovery is the gap, scoped as G5-5C.
Next gate that makes it usable: G5-5C Peer Recovery Trigger After
Replica Restart — fix engine-driven catch-up re-trigger when a
degraded peer becomes reachable again. After G5-5C: re-run #4 #5 #6
in this same harness; full L4 reached.
Forward-carries to G5-5C (architect-bound 2026-04-27):
- Reuse existing engine-driven recovery primitives (T4d-4); no
ad-hoc re-ship from replication layer.
- Define trigger source first: observation reappearance, periodic
probe loop, or stream/transport reconnect signal.
- Pass criterion: exactly the failed hardware case from G5-5 #4.
- Seed evidence: seaweed_block@5c4718f primary-fail.log shows the
gate-degraded + stale-barrier-ack pattern.
Forward-carries to opportunistic future hardening:
- Unit test for EnsureStorage→assignment-arrives→first-Open
Identity-latch path (would have caught round-10/11 bug pre-m01).
- Generalize start_cluster() pre-flight stale-state cleanup pattern
for future hardware harnesses.
Forward-carries to G5-6:
- G5-DECISION-001 (Path A vs Path B) — engine-state serializability
pinned in T4d still holds; G5-5 doesn't change posture.
Pending: architect single-sign on §close per v3-batch-process.md §5
+ §8C.2.
Refs: 24 commits in seaweed_block@phase-15 spanning rounds 1-14
(documented in §close.evidence.commits table).
|
||
|
pingqiu
|
774cee5bf4 |
G5-5 mini-plan v0.3: §2 acceptance criteria rewritten (architect REVISE round 51-followup)
Architect's v0.2 review caught that §1 absorbed the 3 binding revisions but §2 (the close contract per v3-batch-process.md §2) stayed stale: - §2 #2 still said "byte-equal on replica's walstore extent" - §2 had old #4 (race stress) instead of new #4 (process restart) - §2 #3 didn't name /status/recovery as the R/H source v0.3 rewrites §2 to match §1, with explicit verifier names: #1 verify_cluster_ready #2 verify_byte_equal — m01verify Go helper using walstore.OpenReadOnly + storage.LogicalStorage.Read(lba) + SHA-256 (NO raw extent peek) #3 verify_network_catchup — iptables disconnect + polls /status/recovery?volume=v1 for R/H; asserts RecoveryDecision="catch_up" #4 verify_restart_catchup — SIGTERM replica + restart same binary + same --durable-root; polls /status/recovery same as #3 #5 verify_race_stress — 10x -race on G5-4.5 integration test #6 verify_full_suite — go test ./... clean from m01 #7 v3-dev-roadmap.md updated at gate-close per v3-batch-process.md §8 §1 file map and §5 forward-carry table already match v0.3 numbering (grep confirmed no stale references). Implementation scope unchanged from v0.2 (~310 prod LOC + ~30 unit tests). v3-batch-process.md §2 single-source-of-truth discipline preserved: §2 acceptance criteria IS the close contract; §1 scope description stays in sync but is not load-bearing for close evidence. |
||
|
pingqiu
|
e0261bfd84 |
G5-5 mini-plan v0.2: architect REVISE-BEFORE-CODE responses
Addresses 3 architect revision requirements (round 51): REVISION 1 — process restart distinct from network disconnect: Split G5-4 #4 forward-carry into TWO scenarios: §2 #3 network disconnect (iptables) — proves live TCP interrupt + recovery without process restart §2 #4 replica process stop/restart — proves durable reopen + master resubscribe + recovery reconstruction G5-4 #4 is now FULLY consumed (was: only network proxy in v0.1). REVISION 2 — storage-aware byte verifier: Replace raw walstore .extent peek with storage-abstraction Read(lba): helper opens replica's walstore via core/storage/walstore (or equivalent OpenReadOnly path), invokes Read(lba) per LBA in the range, SHA-256 vs primary's known payload. Raw extent peek REJECTED — walstore on-disk includes WAL frames + checkpoints + sparse regions + potentially-stale-but-valid blocks; only Read(lba) returns the authoritative current value. Risk added: if walstore.OpenReadOnly is missing, sw adds it as part of this batch (small scope expansion contained in core/storage/walstore; read-only opener for verification only, NOT a substrate semantic change). REVISION 3 — named R/H observation source: /status?volume=v1 returns frontend.Projection (no R/S/H). G5-5 adds /status/recovery?volume=v1 returning engine.ReplicaProjection (Mode, R, S, H, RecoveryDecision); gated by new --status-recovery daemon flag (default off; production binaries don't enable). Loopback-only via existing isLoopbackRemote guard. ~30 prod LOC + ~30 unit tests. Engine/adapter logic unchanged — surfaces already-computed projection through HTTP. Updated §1 file map, §1.4 truth-domain check, §5 forward-carry table, §6 risks (3 new rows), §close template unchanged. Re-submitted for architect §1-§6 ratification. After ratify, sw codes per §1 file map; estimate ~310 prod LOC + ~30 unit tests. |
||
|
pingqiu
|
4045f8c8aa |
G5-5 mini-plan v0.1 — first trial of compressed v3-batch-process.md
Single doc per v3-batch-process.md §2: scope + acceptance + invariants + forward-carry + risks + sign table; §close appended at batch close (no separate kickoff / closure / G-1 docs). Scope: m01 hardware first-light, promoting G5-4's L1 (binary composition) result to L3 (Replicated IO) per v3-architecture.md §13: 1. iterate-m01-replicated-write.sh orchestration script 2. Real iSCSI write byte-equal primary→replica on hardware 3. iptables disconnect + engine-driven catch-up (within retention) 4. 10x -race stress on G5-4.5 integration test (m01 has CGO/gcc) Architecture touchpoints (v3-architecture.md): §6.1 Write Path, §6.3 Replication Path, §7 Recovery Explicit non-claims: rebuild path, NVMe target, durability modes, failover, backend-layer failure injection — all defer to follow-up batches per §1. Forward-carry consumed (G5-4 §close criteria 3+4+6). G-1 N/A (V3-native verification batch, no V2 muscle PORT). Per v3-batch-process.md §15: per-agent action list at end. |
||
|
pingqiu
|
087343dd14 | v3-batch-process §14: clarify architect/sw/QA are AI agents; user routes between them | ||
|
pingqiu
|
34dfbb66ef | v3-batch-process §15: per-agent action list at end of every response (cut user's routing load) | ||
 pingqiuandClaude Opus 4.7
|
0965a36b16 |
v3-batch-process §12-§13 + v3-architecture.md (architect first-order)
Architect additions to v3-batch-process.md:
§12 Architect review checklist — system-level drift check (4 questions:
scope truth, V2/new-build decision, engine/adapter impact,
product usability level). Close-sign template enforces honest
"Done / Not done / Product level reached / Next gate" statement.
§13 Failure modes process must prevent — 5 observed waste sources:
V2 porting drift, engine/adapter change without reason, function
close = happy path only, product unusable despite green tests,
missing architecture component (catches like binary wiring +
G9A placement gap).
§14 (renumbered from §12) — ownership table unchanged.
v3-architecture.md (NEW first-order doc, architect-authored):
369 lines, 15 sections covering component map, truth domains,
control-plane + data-plane flows, recovery architecture, failure
model, operator interface, P15 gate alignment, product completion
ladder, open architecture decisions, change discipline.
Bridges the gap surfaced in conversation: WHAT (gates) + behavior
contracts + anti-patterns existed; HOW the system fits together
was missing. v3-architecture.md is now peer of mvp-scope-gates.md
+ block-behavior-contract-index.md as first-order references.
v3-batch-process §8 control-doc table updated to include
v3-architecture.md as architect-owned, "component/responsibility/
flow changes" trigger.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
pingqiuandClaude Opus 4.7
|
a8b0999c45 |
v3-batch-process §12: ownership table
Codifies who owns each step based on what actually worked in T4 + G5:
- Gate scope: architect
- Batch sketch (mini-plan §1-§6): sw
- G-1 V2 read (when V2 PORT): sw
- Mini-plan ratification: architect signs + QA reviews
- Code + unit tests: sw
- Component scenarios + m01 verification: QA
- Ledger inscription (PR-atomic): sw
- §close append: sw drafts + QA verifies
- Close sign: architect single-sign
Why sw plans (not architect):
- Knows code feasibility + framework state
- Self-commits to deliverable scope (fewer revision cycles)
- Architect ratifies SCOPE but doesn't need implementation detail
(caught 2 binding clarifications at G5-4 v0.2→v0.3 — that's the
right level of architect involvement)
Why QA reviews (doesn't plan):
- Independent third party (not scope or implementation advocate)
- Catches discipline gaps sw + architect miss
- Owns m01 hardware + component scenarios
Edge cases:
- Process changes (this doc): QA proposes; architect signs
- Hotfix-class (§6.3): sw self-authors + self-merges; QA spot-reviews;
architect ratifies if invariant-affecting
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
pingqiuandClaude Opus 4.7
|
d0cbe66702 |
v3-batch-process.md (NEW): compressed batch governance
Codifies the lessons from T4 + G5-4 retrospective:
KEEP — earned its keep on T4:
- G-1 V2 PORT read (saved 5 hidden invariants on T4b-4, probe non-
mutation pin on T4c-2, 3 placement decisions on T4d-3)
- Mini-plan acceptance criteria (single source of truth for close)
- Invariant ledger discipline ("claim without test = wish")
- m01 -race verification (caught 2 engine bugs at T4d-4 part C)
- Architect single-sign at close (caught 4 stale refs at G5-4 close)
DROP — overhead without payoff:
- Separate kickoff PROPOSAL doc (mini-plan §1-§6 = same thing)
- Separate G-1 doc (inline §4 of mini-plan)
- Separate closure report doc (§close section of mini-plan)
- Separate forward-carry checklist (§5 of next-batch mini-plan)
- Separate QA scenario catalogue (write tests directly when ready)
- Multi-version doc churn (v0.1→v0.5)
- Cross-doc invariant restatement (ledger is sole source)
- Mixed T-track + G-N naming for same gate
Compressed sign cycles: was 4-5 architect signs per batch; now 2
(scope ratify + close sign).
Per-batch artifact count: was 5+ (kickoff + mini-plan + G-1 +
closure + checklist + scenario catalogue); now 1 (mini-plan with
§close appended).
Decision rules codified:
§6.1 G-1 yes/no (V2 PORT yes; V3-native no)
§6.2 T-track vs G-N naming (architect picks at kickoff)
§6.3 When to skip mini-plan (1-line hotfix-class)
§8 names the 6 first-order control docs to keep current
(v3-dev-roadmap, v3-phase-15-mvp-scope-gates, v3-invariant-ledger,
v3-block-behavior-contract-index, v3-product-placement-authority-
rationale, v2-v3-contract-bridge-catalogue).
§9 first trial: G5-5 m01 hardware first-light.
§11 honesty principle: documentation that catches bugs is
discipline; documentation that doesn't is ceremony. Drop ceremony,
keep discipline.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
pingqiu
|
0667f8edbf | v3-dev-roadmap: link v3-block-behavior-contract-index as first-order architect ref | ||
|
pingqiuandClaude Opus 4.7
|
270615e005 |
P15 doc cleanup pass 1: G9A placement gate + roadmap entry doc
3 changes for clearer dev roadmap:
1. v3-phase-15-mvp-scope-gates.md — added G9A Placement Controller MVP
per architect direction 2026-04-26. Sits between G9 lifecycle and
G10 snapshot. P0 priority. Source rationale: production block
storage needs V2-like operational ergonomics (operator asks for
intent → system computes placement → master mints assignment) but
V3 authority discipline must be preserved (no heartbeat-as-
authority, no V2 promote/demote). G9A bridges the two:
- flat-topology RF placement (NO rack/AZ awareness in P15)
- durable desired topology generation
- explainable candidate filtering (why selected, why rejected)
- replacement-on-drain/disk-loss
- master mints ONLY from desired topology
Explicit non-scope (defer to G20 / P16): rack-aware, hot rebalance,
automatic load movement, multi-master HA, V2 promote/demote.
Updated P0 table, dependency graph §4.5, closure rule §5 #13.
2. v3-dev-roadmap.md (NEW) — 1-page entry point for "where are we,
what's next." Lists 22 P15 gates with status emoji, current
batch state, naming decoder, source-of-truth pointers, recently
closed batches, prediction for after-G5. QA owns; updates at
every gate-close.
3. v3-phase-development-model.md — added §0 header note clarifying
this is methodology-only, NOT current state. Points to
v3-dev-roadmap.md as current-state entry. Methodology sections
(§1-§6, §8-§14) remain canonical.
Doc layer architecture now:
Methodology: v3-phase-development-model.md (stable)
Roadmap: v3-dev-roadmap.md (entry point; updated per gate-close)
Canonical: v3-phase-15-mvp-scope-gates.md (22 gates + closure)
Rationale: v3-product-placement-authority-rationale.md (why G9A)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
pingqiuandClaude Opus 4.7
|
daafc8e25b |
G5-4 mini-plan v0.5: architect close-sign + doc-lock
Architect ratification round 51 verbatim:
"APPROVED — G5-4 close. Binary T4 replication wiring is complete at
commit seaweed_block@c820e17; criteria 1/2/5/6/7 satisfied; criteria
3/4 explicitly relocated to G5-5 hardware first-light; --data-addr
correction accepted; 5 INV-BIN-WIRING-* rows ACTIVE. Close claim is
wiring-ready, not byte-movement-ready."
4 doc-lock corrections applied:
1. Header status v0.2 → v0.5 CLOSED + close-sign metadata
2. §1.2 + §1.5: --ctrl-addr → --data-addr correction inscribed
- executor dials peer.DataAddr (core/transport/executor.go:303)
- listener MUST bind the address master mints into
AssignmentFact.peers[*].DataAddr
- --ctrl-addr reserved for future control-plane split; verified
no current binder + no NVMe/iSCSI/status conflict
3. §4 #3 + #4: marked RELOCATED to G5-5 with rationale (in-process
subprocess can't drive real iSCSI/NVMe write without kernel
client; G5-5 m01 has the kernel tooling)
4. §4 #6: marked DONE (m01 -race ×10 PASS in 13.2s; was pending
in v0.4); §4 #1/#2/#5/#7 marked DONE with evidence pointers
Final state:
- 5 of 7 acceptance criteria satisfied (1, 2, 5, 6, 7)
- 2 criteria (3, 4) RELOCATED to G5-5 hardware first-light
- 5 INV-BIN-WIRING-* rows ACTIVE in v3-invariant-ledger.md
- Code: seaweed_block@c820e17 (binary wiring + integration test)
- Ledger: seaweedfs@36ba7b44e (5 invariant rows)
Close claim: wiring-ready, NOT byte-movement-ready (per architect).
G5-5 m01 first-light certifies byte-movement.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
pingqiu
|
36ba7b44e1 |
G5-4: 5 INV-BIN-WIRING-* invariants inscribed in ledger
PR-atomic with seaweed_block@c820e17 per architect binding round 50 (mini-plan v0.4 §4 #7): ledger inscription required at G5-4 close. - INV-BIN-WIRING-ROLE-FROM-ASSIGNMENT - INV-BIN-WIRING-PEER-SET-FROM-ASSIGNMENT-FACT - INV-BIN-WIRING-LISTENER-LIFECYCLE-LIFO - INV-BIN-WIRING-ASSIGNMENT-DRIVES-MEMBERPRESENT - INV-BIN-WIRING-SESSIONID-VIA-ADAPTER All 5 are ACTIVE with test pointers to cmd/blockvolume/g5_4_l2_replication_test.go (subprocess integration) + source-side checks in cmd/blockvolume/main.go and core/host/volume. Last verified 2026-04-26 (G5-4 close). |
||
|
pingqiuandClaude Opus 4.7
|
3892ab29ce |
G5-4 mini-plan v0.4: G-1 ceremony DROPPED; sw cleared to code
User question surfaced the overhead-vs-value of G-1 for V3-native
batches. Honest assessment:
G-1 ceremony EARNED its keep on T4 V2-PORT batches:
- T4b-4 G-1 caught 5 hidden invariants pre-code
- T4c-2 G-1 caught probe non-mutation discipline pin
- T4d-3 G-1 caught 3 placement decisions
G-1 ceremony does NOT earn its keep for G5-4:
- V3-native binary integration (not V2 muscle PORT)
- Mini-plan v0.3 already has scope + 7 acceptance criteria + 5
inscribed invariants + file map
- Architect's 2 binding questions (round 50) are small design
questions answerable in PR description, not separate ratified doc
v0.4 changes:
§7.1 #1 — G-1 deliverable struck through; replaced with PR-
description requirements for the 2 architect bindings
§3 #5 predicate — G-1 dropped; sw cleared to start G5-4.1
§8 sign table — code-start row "▶️ unblocked" (was "⏳ pending")
G5-4 close requirements unchanged: 7 acceptance criteria + 5
invariants in ledger + PR cites resolution of 2 architect bindings
+ architect single-sign per §8C.2
Process lesson: don't auto-port T4 governance template to every batch;
ask "does this step earn its keep" each time. Future V2-PORT batches
still get G-1 ceremony. Future V3-native batches: mini-plan + PR
review + G-2/G-3 gates is sufficient.
Sw next: code G5-4.1 → G5-4.2 → G5-4.3 → G5-4.4 → G5-4.5 in order.
PR description must cite resolution of 2 architect round-50 bindings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
pingqiuandClaude Opus 4.7
|
a68d94679c |
G5-4 mini-plan v0.3: architect round-50 RATIFY with 2 bindings
Architect ratification 2026-04-26: "Role inference, in-process acceptance, G5-DECISION-001 seam, and sessionID discipline are architecturally correct. G-1 must clarify replica readiness semantics and confirm ctrl-addr reuse or introduce repl-addr before code." 2 binding clarifications baked into v0.3: #1 — §4 #2 acceptance criterion split by role: - Primary: Healthy=true per existing frontend/write-ready projection - Replica: replication-ready / listener-bound + ApplyEntry byte-equal verified — MUST NOT report Healthy=true if existing field implies frontend-primary-write-ready - If existing status field is too coarse, G5-4.5 uses precise assertion names (assertReplicaReplicationReady, assertPrimaryFrontendReady) instead of unified assertHealthy #2 — §4 #7 acceptance criterion strengthened: - Catalogue inscription ALONE insufficient at G5-4 close - 5 INV-BIN-WIRING-* invariants MUST land in v3-invariant-ledger.md - Per v3-quality-system.md §6 "an invariant without a test is a wish" - Ledger updated as PR atomic with code (not after-the-fact) §7.1 G-1 deliverable extended (G-1-blocking subitems): - Replica readiness semantics — what existing volume.Status / ProjectionView field expresses replication-ready (vs Healthy)? G-1 either proposes new field OR specifies precise assertion names - --ctrl-addr reuse confirmation — verify NO conflict with NVMe/iSCSI control-plane traffic on same port. If conflict, G-1 introduces --repl-addr flag (small scope expansion, contained in this batch) §3 #4 predicate flipped to ✅ DONE (architect round 50). §8 sign table updated with explicit ledger requirement at close. Architect-pre-baked: ratification stays valid; no further mini-plan revisions needed before G-1. Sw next: produce G-1 V3-native PORT read deliverable per §7.1 (includes the 2 binding subitems). Code stays blocked until architect ratifies G-1. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
pingqiu
|
c46c52e1aa |
G5-4 mini-plan v0.2: QA round 1 review responses
Addresses QA's 3 notes + 1 clarification ask: Note 1 (role inference): §1.3 rewritten — fact.ReplicaID is master-minted (proto verified at control.proto:128-148 + mint site at services.go:198-205). Binary reads `fact.ReplicaID == self.ReplicaID` directly. No lex-smallest fallback (master always names exactly one bound replica per volume per line). Removes the binary-side authority inference that violated the master-authority rule. Note 2 (acceptance circular): §4 #2 verifier reframed to G5-4.5 in-process test. m01 hardware verification belongs to G5-5; G5-4 closes on the in-process pin. Note 3 (G5-DECISION-001 contradiction): §5 rewritten — G5-4 ships Path B runtime AND keeps Path A serializability seam open. T4d-4 part B's RoundTripJSON test already pins serializability; G5-4 preserves it. G5-6 architect ratification can promote to Path A by adding persistence on top of the existing struct, with no engine-state-shape change. Clarification ask (sessionID minting): §6 added INV-BIN-WIRING-SESSIONID-VIA-ADAPTER. Adapter mints unique sessionIDs via process-wide atomic counter at adapter.go:70; binary inherits for free as long as it dispatches via the adapter (never via framework shortcuts that hardcode sessionID=1, which is the known T4c §I + QA G5-1 round 1 SKIP gap). Pinning this invariant keeps the gap test-side. Re-submitted for QA re-review per parent kickoff §7 governance loop. |
||
|
pingqiu
|
c6b2685890 |
G5-4 mini-plan v0.1: binary T4 replication wiring
Mirror cmd/blockvolume to T4d-4 part B's WithEngineDrivenRecovery() framework binding. Single batch (~250 prod + ~150 tests), 5 ordered subtasks. Design decisions (a-d per kickoff §3 G5-4 row): (a) Role inference: assignment-driven, no new CLI flag (b) Peer discovery: AssignmentFact.Peers per T4a-5 P-refined (c) Listener lifecycle: --ctrl-addr reuse + LIFO Stop in host.Close() (d) Engine instantiation: one engine per volume, single --volume-id Pre-merge gates require G-1 V3-native PORT read of cluster.go:357-369 + V2 lesson check on weed/storage/blockvol/blockvol.go before code. 4 new invariants to inscribe at close (INV-BIN-WIRING-*). Submitted for QA + architect ratification per parent kickoff §7 governance loop. No code until ratify. |
||
|
pingqiuandClaude Opus 4.7
|
bf77e2b57a |
G5 kickoff v0.3: architect round-49 RATIFY WITH DOC FIXES
Architect sign by pingqiu 2026-04-26: "6-batch shape 批准; G5-4 governance loop 批准 (kickoff → mini-plan → G-1 → code); ordering 批准 (G5-1/2/3 可并行; G5-4 blocking G5-5; G5-6 closure last); G5-DECISION-001 timing 放在 G5-6 close 最合适." 5 doc fixes applied: 1. §4 #1 "5 G5 batches" → "6 G5 batches" with explicit batch list 2. §6 forward-carry table — G5-DECISION-001 → G5-6 + m01 → G5-5 3. §8 "5-batch shape" struck through with v0.3 ratify note 4. handoff doc title + §0 context renamed G5-4 → G5-5 for m01; added v0.3 architect-round-49 note explaining renumber 5. §7 status relaxed from "No G5 code begins until ratified" to "No G5-4 code begins until G5-4 mini-plan/G-1 ratifies" + explicit cleared-to-start list Sw + QA clearances effective immediately: - QA cleared: G5-1 scenario authoring (component-scope, no binary wiring needed) - QA cleared: G5-2 primary-only smoke - sw cleared: G5-3 metrics/backpressure assessment - sw cleared: G5-4 mini-plan + G-1 V2-native PORT read (T4d-4 part B component framework as PORT source) Held until further governance: - G5-4 binary-wiring CODE (waits for G5-4 mini-plan + G-1 ratify) - G5-5 m01 hardware first-light (depends on G5-4) - G5-6 G5-DECISION-001 architect resolution (at close) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
pingqiuandClaude Opus 4.7
|
ead22edcd5 |
G5: surface binary T4-wiring as new G5-4 batch (was implicit; now explicit)
Hand-off doc v0.3 + G5 kickoff v0.2: m01+M02 bring-up smoke surfaced
that cmd/blockvolume binary lacks T4 replication wiring entirely.
Sw-confirmed root cause:
- --t1-readiness HealthyPathExecutor is primary-only by design
- volume.Config.ReplicationVolume slot exists (host.go:73) with godoc
"T4a-5 production wiring sets this" — but T4a-5 only added the
field; the wiring NEVER landed
- T4d-4 part B wired WithEngineDrivenRecovery() for component test
framework (cluster.go:357-369), NOT for the binary
- Result: V3 components compose end-to-end (proven by T4d HARD GATE
#3); the production binary still constructs a primary-only data
plane
Sw confirmed this is real implementation work (150-300 LOC + design),
not a 50-LOC quick patch. Four design decisions needed:
1. Role inference (assignment vs CLI flag vs topology)
2. Peer discovery (from AssignmentFact.Peers)
3. Listener lifecycle (--data-addr reuse + Stop)
4. Engine instantiation (one engine per volume)
G5 kickoff revised to v0.2:
- 5 batches → 6 batches (binary wiring promoted to G5-4)
- G5-1/2/3 are NOT blocked by G5-4 (component framework already
binds T4d-4 part B; QA scenarios + walstore cadence at
component/primary-only scope can run in parallel)
- G5-4 binary wiring: needs full governance loop (kickoff →
architect ratify → mini-plan → architect ratify → G-1 → code).
G-1 source: T4d-4 part B component framework as V3-native PORT
- G5-5 m01 hardware first-light DEPENDS on G5-4 (script can't
drive replica scenarios until binary supports replicas)
- G5-6 G5-DECISION-001 resolution at G5 close (was G5-5 in v0.1)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
pingqiuandClaude Opus 4.7
|
fbcfe89e24 |
G5-4 bring-up hand-off v0.2 — RESOLVED via local debug
Root cause for "volume not ready" gate: missing
--expected-slots-per-volume 2 flag on blockmaster.
Default is 3; QA's 2-node topology had 2 slots; controller
silently rejected observation snapshot (cmd/blockmaster/main.go:39).
Fix verified locally on Windows (single-node, no m01/M02 needed):
- Add --expected-slots-per-volume 2 to blockmaster command
- Primary reaches Healthy=true with epoch=1
- assignment-received fires; durable storage opens; status
endpoint serves {"Healthy":true}
Lesson learned (process improvement): for V3-internal bring-up
debug, try single-node local reproduction FIRST. The cluster
bring-up gate is V3 logic, not network topology. Reproduces in
seconds locally with full source-code access; m01/M02 only needed
for cross-node-specific scenarios (real network conditions,
iptables, multi-host wire).
Secondary finding: replica r2 sees primary r1's assignment but
records "supersede, not applying to adapter" because T1
HealthyPathExecutor only handles primary case. For G5-4 replica
bring-up, sw needs to wire T4a-T4d ReplicationVolume + ReplicaPeer
+ ReplicaListener stack (not just --t1-readiness flag). This is
the actual next gap for G5-4.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
pingqiu
|
e21c686939 |
G5-4 m01+M02 bring-up — sw answer: --expected-slots-per-volume flag
Root cause: cmd/blockmaster/main.go hardcoded ExpectedSlotsPerVolume=3. QA's 2-slot topology silently failed validateVolumeTopology in the controller, so no assignments were minted, no master-log lines, and volumes timed out at durable open. Fix landed in seaweed_block@f5de7c5: --expected-slots-per-volume CLI flag, default 3, set 2 for the 2-node smoke. QA next: rebuild blockmaster, pass --expected-slots-per-volume 2 in §3.4 of the handoff command sequence; rest unchanged. |
||
|
pingqiuandClaude Opus 4.7
|
2d9c2be9f3 |
G5-4 m01+M02 cluster bring-up — hand-off to sw
Records QA's cross-node smoke attempt 2026-04-26: infrastructure
fully verified READY (m01+M02 reachability, SMB share for binary
distribution, master cross-node listen, network OK), but cluster
bring-up blocked at V3-internal gate.
Symptom: blockvolume on both nodes connects to master but logs
"durable open: frontend: volume not ready" — never reaches steady
state, status endpoint never binds, master log shows no heartbeat
or assignment-mint events.
Hand-off contents:
- §1 specific questions for sw (5 gaps to fill)
- §2 infrastructure verified READY (no action needed)
- §3 copy-pasteable commands sw can run/debug
(build → topology → master → primary → replica → cleanup)
- §4 QA's hypothesis on the gap (assignment-from-master flow)
- §5 debug suggestions for sw (log levels, integration test
references)
- §6 G5-4 script skeleton current state
- §7 QA's next steps once sw answers
Working dirs reproducible:
- Binaries: /mnt/smb/work/share/g5-binaries/{blockmaster,blockvolume}
- Run state: /tmp/g5sm/ on both nodes
- Logs: /tmp/g5sm/logs/{master,primary,replica}.log
Blocks: G5-4 implementation work (script scenario bodies, hardware
first-light scenarios). Does NOT block QA scenario authoring at
component scope (Cluster framework already covers that).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
pingqiuandClaude Opus 4.7
|
ce78fea36f |
G5 kickoff §7a: m01 + M02 infrastructure verification (QA pre-ratify)
Per QA infra-check round 2026-04-26, surfaces real readiness gaps before architect ratifies G5-4 schedule: m01 (192.168.1.181 — primary node): ✅ 32-day uptime; sudo password-less; 16 cores; 19 GiB RAM ✅ 177 GiB free disk; Go 1.26.2 installed ✅ iptables / netns / multi-process tools all available ✅ T2 m01 NVMe script template available as pattern reference M02 (192.168.1.184 — replica node): ✅ Reachable from m01 (0.92ms); same kernel; 178 GiB free disk ❌ Go NOT installed — must scp binaries from m01 Implication for G5-4: Build binaries on m01, scp to M02. Same cross-node binary pattern T2 already uses for its iSCSI target deployment. G5-4 skeleton at seaweed_block/scripts/iterate-m01-replicated-write.sh implements this build-then-scp flow. No infrastructure blockers. Architecture ready as soon as G5 mini-plan ratifies scenario list. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
pingqiuandClaude Opus 4.7
|
a792ed67e5 |
G5 kickoff PROPOSAL v0.1 (post-T4 close)
QA-authored proposal opening G5 collective close planning.
Inherits 5 forward-carries from T4d closure §I as G5 scope:
- m01 hardware first-light for replicated write path
- Multi-replica concurrent live + recovery scenarios
- G5-DECISION-001 resolution (Path A persist vs Path B rebuild)
- walstore flusher cadence verification + tuning policy
- Minimal metrics/backpressure assessment
5-batch shape proposed:
- G5-1 multi-replica scenarios (component) — QA + sw framework
- G5-2 walstore cadence verification — sw + architect
- G5-3 metrics/backpressure assessment — sw + architect
- G5-4 m01 hardware L3 first-light — QA + sw
- G5-5 G5-DECISION-001 resolution + closure report — architect + sw + QA
QA recommendations:
- G5-DECISION-001: Path B (rebuild from probe after restart) for
MVP scope. T4d-4 part B already structurally enables (ReplicaState
JSON-clean per TestG5Decision001_*); production restarts rare;
Path A's persistence work substantial. Backwards-compatible
upgrade later if production usage proves Path B insufficient.
- G5-5 timing at close (after G5-1/2/3/4 evidence informs decision)
- §2.2 explicit non-claims to prevent G5 scope creep:
* CARRY-T4D-LANE-CONTEXT-001 → post-G5 hardening backlog
* --durable-walsize CLI flag → post-G5
* Snapshot-based catch-up → post-G5
* Wire protocol versioning → post-G5
* Auth/encryption/mTLS → post-G5
Status: ⏸ DRAFT — awaiting architect ratification on §2 scope +
§3 batch shape + §4 acceptance bar + §5 G5-DECISION-001 path.
No G5 code work begins until ratified.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
pingqiuandClaude Opus 4.7
|
75d18e676f |
T4d batch close: catalogue invariant upgrades + checklist v0.3
Catalogue §3.3 — 12 T4d invariants flipped from ⏭ to ✓ PORTED with
specific commit hashes; 4 round-47/48 invariants newly inscribed:
Pre-existing flipped to ✓ PORTED:
- INV-REPL-NO-PER-LBA-DATA-REGRESSION → bd2de99 + 01f4ab9
- INV-REPL-RECOVERY-STALE-ENTRY-SKIP-PER-LBA → bd2de99
- INV-REPL-RECOVERY-COVERAGE-ADVANCES-ON-SKIP → bd2de99
- INV-REPL-LIVE-LANE-STALE-FAILS-LOUD → bd2de99
- INV-REPL-RECOVERY-COVERAGE-RESTART-SAFE → bd2de99
- INV-REPL-LANE-DERIVED-FROM-HANDLER-CONTEXT → 01f4ab9 + 44c60dd
(with named carry CARRY-T4D-LANE-CONTEXT-001 to post-G5)
- INV-REPL-TRANSPORT-STORAGE-CONTRACT-ONLY → 44c60dd + 1edeb36
- INV-REPL-CATCHUP-FROMLSN-IS-REPLICA-FLUSHED-PLUS-1 → 44c60dd
- INV-REPL-CATCHUP-FROMLSN-FROM-ENGINE-STATE-NOT-PROBE → 44c60dd
Newly inscribed (round-47 + round-48 architect additions):
- INV-REPL-CATCHUP-EXHAUSTION-ESCALATES-TO-REBUILD → 812d3fa + e642ae8
- INV-REPL-REBUILD-FAILURE-TERMINAL → 812d3fa
- INV-REPL-FAILED-SESSION-KIND-DRIVES-ESCALATION (part C bug #1) → e642ae8
- INV-REPL-REBUILD-ESCALATION-STICKY-UNTIL-TERMINAL (part C bug #2) → e642ae8
Forward-carry checklist v0.3:
- All per-batch focus rows resolved
- m01 -race verified across all T4d batches including T2A NVMe race fix
- Status transitions from "active gating" to "G5-baseline"
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|