mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-08-27 19:37:00 +00:00
T4 L1 survey round 2: sw pre-scan output + H6 narrowing + H7 + §3.13
Fulfills §5 step 1 pre-scan gate with concrete V3 source evidence and
propagates findings to §3 observations.
## Pre-scan output (§5 step 1)
5-row checklist table against V3 source:
- SetReplicaAddrs / ReplicaAddrs / replica fields: NONE in
`core/frontend/` or `core/storage/` (grep-clean)
- Sync/Write remote-ack semantics: NONE; all returns pure-local
(`types.go:50-78`, `logical_storage.go:57-70`)
- LogicalStorage.Write LSN: pure-local; distributed durability
is explicit non-contract (`logical_storage.go:45`)
- Ship/Replicate/Quorum/Barrier/Durability identifiers: none in
code; comments only
- Replication stubs: NONE; but three fully-implemented replica-
side primitives on LogicalStorage: ApplyEntry / AdvanceFrontier
/ AllBlocks, with impls in walstore.go + smartwal/store.go
Net: frontend/durable layer clean; LogicalStorage layer already
committed to a specific replica-side shape. L2 must ALIGN with
that shape, not override it.
## §3 updates driven by pre-scan
§3.11 (H6) narrowed with V3 existing-shape evidence:
- Option A unlikely (no supporting V3 shape; StorageBackend is
replication-unaware)
- Option B effectively ruled out (ApplyEntry/AdvanceFrontier sit
BELOW Backend on LogicalStorage; a ReplicatedBackend wrapper
would either reach past its wrapped contents or duplicate the
storage-layer contract)
- Option C leading (matches V3 existing Provider-owns-lifecycle
shape; generalizes BUG-005 lesson)
§3.12 (H7) new — LSN surface-up gap:
- `Backend.Write → (int, error)` discards LSN
- `LogicalStorage.Write → (lsn, error)` returns it
- Primary-side shipper needs per-write LSN
- H7a (extend Backend sig) unlikely; H7b (Provider intercepts
at LogicalStorage layer) natural fit with H6 Option C; H7c
(side-channel NextLSN+Boundaries delta) rejected as racy
- H7 resolution coupled to H6 — joint L2 LOCK
§3.13 new — replica-side bypasses Backend entirely:
- Structural finding already locked by V3 shape, NOT an L2 choice
- Primary-side traffic: session → handler → Backend → LogicalStorage
- Replica-side traffic: network frame → ReplicaReceiver →
LogicalStorage.ApplyEntry (bypasses Backend)
- Explicit so L2 builds on it rather than fighting
## Feedback-round log + change log
§5 feedback log gains round 2 entry; §6 change log gains full
round-2 detail with line-level citations.
No sign event; this is iterative informal feedback per §8C.8
lightweight cadence. L1 stays DRAFT until bundled T4 T-start
three-sign with L2 + L3.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.7
parent
b4adf76aa0
commit
de2767cd3c
@@ -172,12 +172,21 @@ These are patterns visible at L1 that will drive L2 bridge verdicts. **Not** rec
|
||||
8. **ReplicaReceiver's `ioMu.RLock` nesting around apply** — the lock is held across the entire apply path for restore/import exclusion. Only released inside `replicaAppendWithRetry` during WAL-full wait. Any V3 bridge must either preserve this nesting or explicitly rebuild the exclusion (with the rationale documented).
|
||||
9. **ShipperGroup double watermark** — `MinShippedLSN` (Ceph retention watermark, advisory) vs `MinReplicaFlushedLSNAll` (authoritative sync_all durability). Two consumers, two semantics; losing the distinction in V3 would silently break one or the other.
|
||||
10. **Cross-node epoch consistency observation window** (H5, architect-added). V2 `dist_group_commit` sync_quorum needs primary to know each replica's ack `{epoch, lsn}`; epoch mismatch → ack ignored for quorum. V2 makes this implicit — wire frame carries epoch, decoder in `repl_proto` extracts it. V3's `frontend.Identity.Epoch` is this-primary's view; the replica's epoch comes from replica-side `ProjectionView` (or from assignment directly). L2 decision required: does the V3 ack frame still carry epoch (wire-compat / simplest), OR does primary maintain a `per-replica epoch cache` (cleaner layering but needs refresh semantics on failover)? These choices produce different failover semantics and different rebuild-trigger conditions — cannot be deferred to implementation.
|
||||
11. **Write-path vs replication-path concurrency residence** (H6, architect-added). V2 `BlockVol.Write` is one function that does durable-local-write AND triggers `ShipperGroup` broadcast. V3's `StorageBackend.Write → LogicalStorage.Write` is strictly local durable. **Where does replication get triggered?** Three L2 options:
|
||||
- **Option A** — `StorageBackend.Write` calls local storage, then synchronously calls shipper. Matches V2 semantics, but adapter-layer gains a replication dependency — violates T3a layering.
|
||||
- **Option B** — introduce a `ReplicatedBackend` wrapping `StorageBackend + shipper`; `frontend.Provider` returns Replicated vs Plain based on RF. Clean layering; adds one entity.
|
||||
- **Option C** — replication lives inside `DurableProvider`; `StorageBackend` is unaware; Provider intercepts Write. Requires Provider to gain replication responsibility — rhymes with BUG-005's "Provider owns Backend lifecycle" lesson but extends it.
|
||||
11. **Write-path vs replication-path concurrency residence** (H6, architect-added; narrowed by sw pre-scan 2026-04-22). V2 `BlockVol.Write` is one function that does durable-local-write AND triggers `ShipperGroup` broadcast. V3's `StorageBackend.Write → LogicalStorage.Write` is strictly local durable. **Where does replication get triggered?** Three L2 options, updated with V3 existing-shape evidence:
|
||||
- **Option A** — `StorageBackend.Write` calls local storage, then synchronously calls shipper. Matches V2 semantics, but adapter-layer gains a replication dependency — violates T3a layering. **Verdict: unlikely.** No supporting V3 shape; requires expanding `StorageBackend` (currently 4 fields: storage, view, id, operational atomic — no replication surface).
|
||||
- **Option B** — introduce a `ReplicatedBackend` wrapping `StorageBackend + shipper`; `frontend.Provider` returns Replicated vs Plain based on RF. Clean layering; adds one entity. **Verdict: effectively ruled out by V3 existing shape.** V3 `LogicalStorage` already exposes three replica-side primitives below the `Backend` interface: `ApplyEntry(lba, data, lsn)` (`logical_storage.go:114`), `AdvanceFrontier(lsn)` (`:103`), `AllBlocks()` (`:118`), with full implementations in both `walstore.go:513/491/565` and `smartwal/store.go:343/322/367`. A `ReplicatedBackend` wrapping `StorageBackend` cannot own replica-side ingest because `ApplyEntry` is *below* `Backend` in the stack — either the wrapper reaches past its own contents (violates encapsulation), or the wrapper re-implements replica-side ingest in a second place (duplicates the storage contract V3 already has).
|
||||
- **Option C** — replication lives inside `DurableProvider`; `StorageBackend` is unaware; Provider intercepts Write + drives ShipperGroup. **Verdict: leading candidate.** Matches V3 existing shape: `DurableProvider` already owns `LogicalStorage` lifecycle per-volume (see `provider.go:83-91` `volumes map[string]*volHandle`), and BUG-005's "Provider owns Backend lifecycle" lesson generalizes cleanly to "Provider owns replication lifecycle". Primary-side Provider wraps `LogicalStorage.Write` and drives shippers; replica-side Provider exposes plain `LogicalStorage` for `ApplyEntry` ingest path — replica side bypasses `Backend` entirely (see structural note §3.13). `volumes` map likely needs to grow from `*volHandle` to include per-replica shipper refs; RF comes from master assignment via a new Provider setter.
|
||||
|
||||
L1 makes no recommendation; L2 must pick and LOCK this before L3 touches files.
|
||||
L1 narrows but does not LOCK; L2 must pick and write the final bridge shape. Option C is proposed as the default unless H5/H7 decisions shift the balance.
|
||||
|
||||
12. **LSN surface-up gap** (H7, sw-added 2026-04-22 from pre-scan). `frontend.Backend.Write(ctx, offset, p) → (int, error)` **discards the LSN** returned by `LogicalStorage.Write(lba, data) → (lsn, error)` (`logical_storage.go:57`). For replication, primary-side shipper MUST observe per-write LSN to broadcast ordered entries with contiguous LSN sequences (the V2 `WALShipper`/`ReplicaReceiver` contract, §2.1 + §2.3 invariants). Three L2 options:
|
||||
- **Option H7a** — extend `Backend.Write` signature to return LSN alongside bytes. Breaks all existing `frontend.Backend` consumers (iSCSI/NVMe session handlers use only the byte count); requires interface bump. **Verdict: unlikely.** iSCSI/NVMe don't need LSN; paying interface churn for a feature only replication uses is bad layering.
|
||||
- **Option H7b** — Provider intercepts Write at the `LogicalStorage` layer (not the Backend layer). Primary-side Provider wraps `LogicalStorage.Write` — captures LSN in the same call — and passes `{lba, data, lsn}` to ShipperGroup. `Backend.Write` unchanged; LSN never crosses the Backend boundary. **Verdict: natural fit with H6 Option C.** If H6 C is chosen, H7 b is implied.
|
||||
- **Option H7c** — side-channel: Provider reads `LogicalStorage.NextLSN()` before Write + frontier (`Boundaries().H`) after, infers per-call LSN from the delta. **Verdict: racy.** Concurrent Writes from different sessions make delta inference unsound; rejected.
|
||||
|
||||
H7's resolution is effectively coupled to H6: if H6 chooses C, H7 locks on H7b automatically; if H6 chooses A or B (both unlikely), H7 has to be re-opened. L2 must confirm jointly.
|
||||
|
||||
13. **Replica-side bypasses `Backend` entirely** (sw-added 2026-04-22 from pre-scan; structural finding). The `frontend.Backend` interface is designed for per-session host-client backends (iSCSI / NVMe session consumes it). Replica-side ingest uses `LogicalStorage.ApplyEntry` which is *below* `Backend`. Primary-side traffic goes through `Backend` (session → handler → Backend → LogicalStorage); replica-side traffic bypasses `Backend` entirely (network frame → ReplicaReceiver → LogicalStorage). V3 already assumes this asymmetry by putting ApplyEntry on `LogicalStorage`, not on `Backend`. L2 implication: replica-side `DurableProvider.Open()` does not return a `Backend` in the traditional host-session sense — it returns a replica-ingest handle (or it's a separate Provider method). This asymmetry is structural, not an L2 choice — it's already locked by V3 existing shape. Explicit here so L2 builds on it rather than fighting it.
|
||||
|
||||
---
|
||||
|
||||
@@ -206,6 +215,25 @@ Per §8C.8, there is only **one** three-sign — at **T4 T-start** on the bundle
|
||||
|
||||
Output: one line "V3 has no pre-baked replication assumptions" OR a short list "V3 has X at Y, L2 must align/override". Reply inline; no doc needed.
|
||||
|
||||
**sw pre-scan result (2026-04-22)** — see §3.11 narrowing + §3.12 H7 + §3.13 for L2 impact. Summary against the checklist:
|
||||
|
||||
| Check | V3 state | Source |
|
||||
|---|---|---|
|
||||
| `SetReplicaAddrs` / `ReplicaAddrs` / replica field on `Provider` / `Backend` / `LogicalStorage` | **None.** Zero grep matches across `core/frontend/` + `core/storage/`. | `grep -rn "SetReplicaAddrs\|ReplicaAddrs" core/` empty |
|
||||
| `Sync` / `Write` return type or callback implying remote-ack wait | **None.** `Backend.Write → (int, error)`, `Backend.Sync(ctx) → error`, `LogicalStorage.Write → (lsn, error)`, `LogicalStorage.Sync → (stableLSN, error)` — all pure-local. | `types.go:50-78`, `logical_storage.go:57-70` |
|
||||
| `LogicalStorage.Write` LSN model | **Pure-local.** Godoc: "advances the LSN, and returns the assigned LSN. NOT durable until Sync returns success." `Boundaries() (R, S, H)` is a per-node frontier tuple. Distributed durability is an explicit **non-contract** (`logical_storage.go:45` "distributed durability across nodes" listed under "what the contract does NOT cover"). | `logical_storage.go:52-92` |
|
||||
| `Ship` / `Replicate` / `Quorum` / `Barrier` / `Durability` identifiers | **None in identifiers**; comments only. No `Ship*`, no `Quorum*`, no `Barrier*`, no `Replicate*` types / methods / fields in `core/frontend/` or `core/storage/`. `Durability` appears only in comments + the T3 bug doc language. | `grep -rni "quorum\|ship[pe]\|replicate\|barrier" core/` — zero code matches, only comments |
|
||||
| Stub / comment referencing a future replication surface | **Three implemented primitives** already on `LogicalStorage`, NOT stubs: `ApplyEntry(lba, data, lsn)` (`logical_storage.go:114`, `walstore.go:513-548`, `smartwal/store.go:343-362`), `AdvanceFrontier(lsn)` (`logical_storage.go:103`, `walstore.go:491`, `smartwal/store.go:322`), `AllBlocks() map[uint32][]byte` (`logical_storage.go:118`, `walstore.go:565`, `smartwal/store.go:367`). Plus the non-contract comment at `logical_storage.go:45`. No stubs elsewhere. |
|
||||
|
||||
**Net**: `core/frontend/durable/` + `core/frontend/` is clean of replication assumptions. **BUT** `core/storage/LogicalStorage` already commits to a specific replica-side shape (pure-local LSN + ApplyEntry ingest below Backend + frontier-alignment primitive + block-enumeration primitive for rebuild). These are not assumptions — they are full implementations that L2 must align with, not override. Concrete impact on L2:
|
||||
|
||||
- **H6 narrowing** — Option B ruled out; Option A unlikely; Option C leading. Detail in §3.11.
|
||||
- **H7 new hazard** — `Backend.Write` discards LSN; replication primary-side needs LSN surfaced somewhere. Resolution coupled to H6. Detail in §3.12.
|
||||
- **Replica side bypasses `Backend`** — structural finding. Detail in §3.13.
|
||||
- **Replica-side ingest primitives ready** — L2 does not need to design `ApplyEntry` semantics; it needs to wire V2 `ReplicaReceiver` → V3 `LogicalStorage.ApplyEntry`. Bridge shape is split (receiver state machine + network frame handling on one side, storage primitive on the other) with a defined call interface already in place.
|
||||
|
||||
**No additional latent-drift concerns found**: `DurableProvider.volumes` is keyed by volumeID only (not by role, not by replica-ID) — for T4 this map likely needs a `*volHandle` extension to carry primary-vs-replica role + shipper refs, OR a separate map. Either is a clean additive change, not a drift. `StorageBackend` is replication-unaware by construction; that matches the Backend-bypass finding.
|
||||
|
||||
2. **sw + QA iterate on L2** — QA drafts bridge verdicts into `v2-v3-contract-bridge-catalogue.md` §3 Replication (shape tag + V3 embedding note + per-contract PRESERVE/REBUILD/BREAK verdicts) with H5 + H6 answers from architect in hand. sw reviews inline, comments lead to catalogue edits. No formal sign between rounds; iterate until both sides comfortable.
|
||||
|
||||
3. **sw + QA draft L3** — `v3-phase-15-t4-port-plan-sketch.md` (scope / V2 file classification / V3 target files / non-claims / test strategy / provisional ledger rows) derived from L1 + L2.
|
||||
@@ -214,6 +242,7 @@ Per §8C.8, there is only **one** three-sign — at **T4 T-start** on the bundle
|
||||
|
||||
**Feedback-round log** (informal; appended in §6 change log as received):
|
||||
- 2026-04-22 round 1 — architect F1/F2/F3 + H5/H6 + sw pre-scan gate (landed in `seaweedfs@d2588f5`).
|
||||
- 2026-04-22 round 2 — sw pre-scan performed; output filled into §5 step 1; §3 gains H7 (LSN surface-up) + §3.13 (replica-side Backend bypass); §3.11 H6 narrowed per V3 existing-shape evidence.
|
||||
|
||||
---
|
||||
|
||||
@@ -223,4 +252,5 @@ Per §8C.8, there is only **one** three-sign — at **T4 T-start** on the bundle
|
||||
|---|---|---|
|
||||
| 2026-04-22 | Initial L1 survey drafted post-T3 close; 9 entities identified; 9 L1-level observations recorded; 5 open questions raised for sw/architect/PM review | QA Owner |
|
||||
| 2026-04-22 | Architect feedback round 1 incorporated: F1 split RebuildBitmap into standalone §2.10 (10 entities total); F2 ShipperGroup gains "External deps" row citing master-assignment-provided RF as cross-entity contract; F3 ReplicaBarrier scope rewritten to "per-request call-closure BUT queue-state shared per-volume via cond.Wait"; H5 (cross-node epoch ack window) and H6 (Options A/B/C for write-path vs replication-path residence) added to §3 observations; §5 updated to require sw V3 pre-scan as blocking step before L1 three-sign | QA Owner |
|
||||
| 2026-04-22 | sw pre-scan round 2 landed: §5 step 1 output filled (5-row checklist table + net summary against L2); §3.11 H6 narrowed — Option B ruled out by existing V3 `LogicalStorage.ApplyEntry`/`AdvanceFrontier`/`AllBlocks` primitives at `walstore.go`/`smartwal/store.go`, Option A unlikely, Option C leading; §3.12 H7 added (LSN-surface-up: `Backend.Write` discards LSN from `LogicalStorage.Write`; H7a/b/c options, resolution coupled to H6); §3.13 added (replica-side bypasses `Backend` entirely — structural finding already locked by V3 shape, L2 builds on it rather than chooses) | sw |
|
||||
| 2026-04-22 | Architect scraped back governance overhead: no separate L1 three-sign (I had invented one not in §8C.8). Only one three-sign exists — at T4 T-start on bundled L1+L2+L3. §5 rewritten as lightweight cadence: sw pre-scan → sw+QA iterate on L2 → sw+QA draft L3 → T-start three-sign. Review rounds are informal, logged in this change-log as they happen | QA Owner |
|
||||
|
||||
Reference in New Issue
Block a user