mirrors/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-08-01 04:46:33 +00:00
Code Issues Packages Projects Releases Wiki Activity
13,359 Commits 158 Branches 353 Tags
ab2e131c63dd7bae073bf390f708d2e5e296683a
Commit Graph
13359 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
pingqiu ab2e131c63 docs(p15): update G15a blockcsi binary progress 2026-05-03 08:09:56 -07:00
pingqiu 468c7fb390 docs(p15): update G15a CSI static MVP progress 2026-05-03 08:02:21 -07:00
pingqiu b5d3cd1ee9 docs(p15): add G15a CSI static MVP port plan 2026-05-03 07:43:31 -07:00
pingqiu ed1cf4dcfa docs(p15): close G9G and define PR review cadence 2026-05-03 07:21:52 -07:00
pingqiu 3567f10c8d docs(p15): mark G9G-3 cluster spec implementation 2026-05-03 07:07:01 -07:00
pingqiu 1f44d0382a docs(p15): add G9G-3 cluster spec mini-plan 2026-05-03 07:02:24 -07:00
pingqiu 32a9993264 docs(p15): add G9G close snapshot and QA instruction 2026-05-03 06:55:29 -07:00
pingqiu 36ddbe78d7 docs(p15): mark G9G seed-file entry slice 2026-05-03 06:46:40 -07:00
pingqiu 8cff3e86b3 docs(p15): mark G9G subprocess L2 slice 2026-05-03 06:20:34 -07:00
pingqiu ceca2c1c6f docs(p15): mark G9G first product loop slice 2026-05-03 06:09:31 -07:00
pingqiu fb9476c5ee docs(p15): add G9G product loop mini-plan 2026-05-03 06:03:55 -07:00
pingqiu c4d1468a32 docs(p15): clarify G9F-2 slice binding status 2026-05-03 06:02:31 -07:00
pingqiu 0e649fcc5d docs(p15): reshuffle G9 path and add G9F-2 plan 2026-05-02 23:22:26 -07:00
pingqiu 8b0b9cd628 docs(p15): ratify G9F verify slice and add QA instruction 2026-05-02 22:56:11 -07:00
pingqiu 1a789789da docs(p15): add G9D G9F checkpoint review 2026-05-02 21:39:04 -07:00
pingqiu 4481c029fc docs(p15): add G9F placement authority bridge mini-plan 2026-05-02 21:04:19 -07:00
pingqiu babad86765 docs(p15): add control-plane structural guard pattern 2026-05-02 21:00:12 -07:00
pingqiu a53b61265c docs(p15): add G9D registration controller consensus 2026-05-02 20:30:54 -07:00
pingqiu 22ad260997 docs(p15): mark G9C status transition close-ready 2026-05-02 19:59:07 -07:00
pingqiu e34952f648 docs(p15): update G9C component ready evidence 2026-05-02 19:56:49 -07:00
pingqiu df7f3b183d docs(p15): update G9C post-close durable ack evidence 2026-05-02 19:54:28 -07:00
pingqiu 7643356f2d docs(p15): start G9C replica ready feed continuity plan 2026-05-02 19:49:20 -07:00
pingqiu 3367071d97 docs(p15): close G9B replica join lifecycle 2026-05-02 19:33:03 -07:00
pingqiu 0b840d8933 docs(p15): record G9B L2 join lifecycle smoke 2026-05-02 19:24:07 -07:00
pingqiu 08945de1a9 docs(p15): record G9B adapter lifecycle proof 2026-05-02 19:13:48 -07:00
pingqiu db3160565c docs(p15): update G9B replica ready lifecycle progress 2026-05-02 19:06:48 -07:00
pingqiu 52addf2b35 docs(p15): start G9B replica join lifecycle plan 2026-05-02 18:58:01 -07:00
pingqiu fc71db68ef docs(p15): record G9A RF3 ACK semantics 2026-05-02 18:46:53 -07:00
pingqiu bf6775b856 docs(p15): close G9A ACK reintegration policy 2026-05-02 18:39:11 -07:00
pingqiu e72ce0d530 docs(p15): mark G9A close-ready 2026-05-02 18:38:39 -07:00
pingqiu f5670b5dc4 docs(p15): record G9A recovering status role 2026-05-02 18:35:40 -07:00
pingqiu 4b5a7d62d7 docs(p15): record G9A returned-replica readiness oracle 2026-05-02 18:27:57 -07:00
pingqiu 72f22ec46e docs(p15): record G9A best-effort ACK oracle 2026-05-02 18:21:47 -07:00
pingqiu fa1ed676c2 docs(p15): update G9A ACK reintegration progress 2026-05-02 18:18:01 -07:00
pingqiu 5d411294e4 docs(p15): start G9A ACK reintegration plan 2026-05-02 16:43:30 -07:00
pingqiu 1a613afba4 docs(p15): close G8 failover data continuity 2026-05-02 16:22:57 -07:00
pingqiu 0483a035b0 docs(recovery): separate pin breach from primary flow control 2026-05-02 12:12:44 -07:00
pingqiu baaa1e7e45 docs(recovery): record coordinator progress fact gaps 2026-05-02 11:09:25 -07:00
pingqiu 6905fb8278 docs(recovery): plan progress fact recover governance 2026-05-02 11:00:28 -07:00
pingqiu eda40923a8 docs(recovery): add single-egress grill checklist 2026-05-01 23:24:59 -07:00
pingqiuandClaude Opus 4.7 3910cbd1cb docs(architecture): V3 egress single-decision-core principle 
Pins the architecture principle that V3 egress components (per-peer
shipper / session pump / flusher / barrier driver) must be modeled
as a single decision core with single-queue / single serializable
worker shape. Monotonic pointers (cursor, applied LSN, emit profile,
bound conn) advance only via the core's internal transitions; external
callers deliver commands/events; direct external mutation is a
design-debt side door.

Grounds the principle in three hardware-validated incidents on m01/M02
during 2026-05-02, all of which turn out to be the same shape:

  §3.1 executor.Ship overwrites the WalShipper's emit context mid-
       session (g7 #5: 498 LBA mismatches at concurrent-write range)
  §3.2 PrimaryBridge onStart/onClose dropped ReplicaID; engine and
       runtime peer state diverged (g7 #5/#6 dispatch never fires)
  §3.3 A-class sender→coord RecordBarrierWalLegOk side-write
       (reverted 2026-05-02 working tree per this principle)

Companion to v3-rebuild-from-lsn-pin-clarification.md. Anchors in
consensus: §I P1, §I P7, §6.8, INV-SINGLE.

No new wire field, predicate, or invariant; clarifies a rule that
earlier docs imply but don't articulate. Provides judgment criterion
for upcoming Ship/PushLiveWrite collapse and peer-state ownership
work.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-01 20:57:31 -07:00
pingqiuandClaude Opus 4.7 02ed4db104 docs(recovery): rebuild fromLSN pin sentinel translation clarification 
Pins the rebuild path's `fromLSN=0` sentinel semantic for the current
`StartRebuild` signature. Closes the gap §I P7 calls out ("transport
silently overwriting `fromLSN := 0` violates parity") in the absence of
an engine-published `fromLSN` for rebuild.

Hardware-validated by seaweed_block@bc4286e g7-dual-lane on m01/M02:
G7-#2 PASS (dispatch=1s, complete=1s, total=2s, 1000 LBAs byte-equal).

Sentinel rule:
  - Caller passes 0 ⇒ "rebuild — primary picks the pin"
  - Transport translates: sessionFromLSN := targetLSN
  - Receiver-visible fromLSN = targetLSN
  - Future catch-up (engine surfaces fromLSN := replicaLSN): passthrough

Three transport mechanics that satisfy the rule without violating any
consensus invariant: sentinel translation in startRebuildDualLane,
cursor-caught-up shortcut in WalShipper.DrainBacklog (preserves the
recycle gate's <= strictness verbatim), SeedWalApplied at SessionStart
so base-only rebuilds satisfy the A-class TryComplete conjunct.

Anti-discipline: no new wire fields, predicates, or invariants. Memo
clarifies the meaning of an existing transport-layer constant.

Anchors: v3-recovery-algorithm-consensus.md §I P7 / §6.9 / §6.10;
recover-semantics-adjustment-plan.md §1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-01 18:08:18 -07:00
pingqiu d6636e3497 docs(sw-block): mini-plan §11.9 slice-2A drive+L SHAs + Legacy default receipt 
Document 28fb142 (drive collapse) and d647db4 (LegacyOutOfOrderEmit false),
deprecation stance, Slice-2B Path B remainder; §8 v0.15; §12.1 bridge pointer.

Made-with: Cursor
 2026-04-30 22:06:55 -07:00
pingqiu f098e0e33c docs(sw-block): §11.8 WriteExtentDirect receipt; §6.3 CASE C ∅ noop v3.17 
- Mini-plan §11.8 documents g7-redo/wal-shipper-impl 6fccc62 and 9f7d918
- Pillar 2 supersession note (faulty-store vs wire-abort); §12.4 #7-10 backlog
- Consensus §6.3 Drive pseudocode CASE C comment (architect ∅ nit)
- v3.17 revision; mini-plan revision v0.14

Made-with: Cursor
 2026-04-30 20:16:08 -07:00
pingqiu 8110555978 docs(sw-block): dual-lane recv appendWAL vs writeExtentDirect; backlog iterator cost 
Consensus §6.10: WAL path uses appendWAL + bitmap; base uses
writeExtentDirect (no WAL/LSN); substrate must expose both. §6.3:
ring/sequential backlog model, stateful vs stateless ReadAtLSN table,
O(N) vs O(N log N). §7 routing + INV rows aligned. Mini-plan §12.4
checklist expanded (v0.12). Wal-shipper spec §7.4 note. v3.15.

Made-with: Cursor
 2026-04-30 13:26:33 -07:00
pingqiu c48804479c docs(sw-block): normative Drive/Apply pseudocode and cross-links 
Add §6.3 Drive(input) and §6.10 ApplyWAL/ApplyBASE blocks with state,
atomic envelope, and INV wiring; align cursor/head with exclusive-slot
semantics plus §6.1/CHK-REWIND note; revise §13 StrictRealtimeOrdering
for cursor convention. Wal-shipper spec §7.3 references consensus §6.3;
§7.4 points at §6.10 pseudocode. Mini-plan §12 defers Drive to consensus
§6.3 only (v0.11). Consensus v3.14.
 2026-04-30 13:06:40 -07:00
pingqiuandClaude Opus 4.7 7dfb0172d1 sw-block/design: mini-plan §11.7 — pillar 2/3 landed-SHA + test-name table 
Annotates pillar 2 (dual-line execution) and pillar 3 (receiver
convergence) rows with the landed commits + concrete test names so
the design narrative tracks the code:

  Pillar 2 — assembled-stack fault-injection
    7d051e2 — C3 fault-injection (recovery-package layer, spy sink):
              TestC3FaultInjection_BaseError_WalExitsViaCtx
              TestC3FaultInjection_OuterCancel_BothLanesWindDown
              TestC3FaultInjection_WalError_RunTerminates_NarrowVariantA
    9f62ebe — Pillar 2 fault-injection on the assembled stack
              (BlockExecutor + RecoverySink + Sender + WalShipper):
              TestPillar2A_BaseError_AssembledStack_FailReason
              TestPillar2B_LiveWrites_HighPressure_BarrierIntegrity
              TestPillar2C_WireAbortMidSession_AssembledStack_RestoresEmitContext

  Pillar 3 — receiver convergence under same-LBA conflict
    291e652 — slice-1: same-LBA arbitration on transport stack
    3495a12 — slice-1 polish (AchievedLSN assertion, replica==primary
              comparison, comment fixes, companion-SHA list)
              TestPillar3Slice1_ReceiverConvergence_LiveOverwritesBacklog_SameLBAs
              (transport-stack only; slice-2 lifts to engine path
               via dual_lane_engine_test.go)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-30 11:51:20 -07:00
pingqiuandClaude Opus 4.7 822c588d08 sw-block/design: mini-plan §11.6 Phase 0 SHA receipt + §11.7 heavy integration matrix 
§11.6 startRebuildDualLane row updated with landed-2026-04-30 commit
receipts on g7-redo/wal-shipper-impl:
  - e354813 fix(transport): unify replicaID in startRebuildDualLane
  - e5a8763 fix(test): align component cluster harness to replica-%d
Module-wide go test green (25 packages). Phase 0 closed.

§11.7 added — heavy integration proof matrix on the manager-assembled
session (BlockExecutor + PrimaryBridge + RecoverySink + recovery.Sender +
resident WalShipper + PeerShipCoordinator under unified replicaID).
Three pillars: (1) WalShipper backlog/realtime modes, (2) dual-line
execution + writeMu interleave, (3) receiver convergence. Order:
Phase 0 + SHA receipt → C3 fault-injection → widen matrix → T2/T7
hardware soak. Stop rule: failure at assembled-session layer fixes
algorithm/wiring before Phase 1 attempt binder.

Revision rows v0.7 + v0.8 record the §11.6 + §11.7 deltas.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-30 11:21:14 -07:00
pingqiu 91a313e1e1 sw-block/design: §13 E-WALSHIPPER-DUAL-MODE + mini-plan §11.2a + spec sync 
Architect-approved exception (consensus v3.9, 2026-04-30): WalShipper
dual-mode contract carves Realtime out of §6.3(B) timer + §6.8(3)(9)
send(incoming, debt) normative scope. Backlog mode is fully normative;
Realtime is per-append (T4a no-replay), with StrictRealtimeOrdering
as the production safety switch.

Files:

  v3-recovery-algorithm-consensus.md
    - NEW §13: Architect-approved exceptions, with E-WALSHIPPER-
      DUAL-MODE entry. Old §13 Revision → §14; old §14 Document
      map → §15. Cross-refs in §I, §6.3(A)(B), §6.8 checklist,
      §V §12 (CHK-WALSHIPPER-TIMER-DRAIN explicitly limited to
      ModeBacklog), §15 (index gains §13).
    - §14 revision row v3.9.

  v3-recovery-wal-shipper-mini-plan.md
    - §11.1 G-TIMER row updated to "C2 + dual-mode/§13".
    - §11.2 replaced (was: pre-merge "Realtime drains on cursor<head"
      bullets) with C2 dual-mode contract + §11.2a normative table:
        * Backlog: §6.3 / §6.8(3)(4)(9) fully apply (timer, scan,
          oldest-first, send(·, debt)).
        * Realtime: NotifyAppend per-LSN-in-order; lsn==cursor+1
          invariant; no substrate replay (§13 carve-out).
        * Production safety switch: StrictRealtimeOrdering.
    - §11.3 compliance receipt rebuilt as 1–9 ordered table with
      commit SHAs (P2d cb8ff1c, C1 294d4bf, C2 53c292f, C3 40f2935,
      review-fix 377bcb0).
    - §11.4 test anchor table gains StrictRealtimeOrdering opt-in.
    - §11.5 invariants gain (4) §13 dual-mode + (5) replicaID drift.
    - §11.6 reshaped as production-migration table:
        * Engine drives rebuild-on-gap before flipping
          StrictRealtimeOrdering=true.
        * Fix replicaID drift in startRebuildDualLane (sink uses
          engine replicaID; bridge.coord uses dl.ReplicaID).
        * Hardware-run carries: §IV T2 (barrier vs targetLSN),
          §IV T7 (R2 saturation under sustained pressure).
        * PR template line: ban `lsn > cursor + 1` debt detection
          (banned regression anchor; use `cursor < head` if it
          ever needs to come back, but only inside Backlog).
    - §8 revision: v0.6.

  v3-recovery-wal-shipper-spec.md
    - Goal/checklist front-matter cross-refs §13 + mini-plan §11.2a.
    - §10 revision row aligning with consensus v3.9.

Companion implementation on seaweed_block g7-redo/wal-shipper-impl:
  - 377bcb0 review-fix: T4a Realtime sequence guard + drainOpportunity TOCTOU
  - cb17338 docs: drainOpportunity comment matches §13 dual-mode contract
 2026-04-30 09:48:10 -07:00
pingqiu aeeb71f0e7 sw-block/design: mini-plan §11 — commit SHAs + review-derived invariants 
§11 hardening sequence is fully landed on
seaweed_block/g7-redo/wal-shipper-impl:

  C1 294d4bf  shared writeMu + post-emit RecordShipped hook
  C2 53c292f  WalShipper self-driving Backlog drain + DisableTimerDrain
  C3 40f2935  Sender.Run errgroup BASE ∥ WAL parallel
  348cb35     §6.8 review-derived doc invariants (DisableTimerDrain +
              lock order shipMu → writeMu)

§11.3 — compliance receipt expanded into a 9-row table mapping each
  §6.8 MUST to its commit SHA and implementation locus.

§11.4 — test anchor table with §6.8 # / repo path. Includes
  TestC2_NoGapDenseLSNEdge as the regression anchor banning
  `lsn > cursor + 1` debt detection (the user's correction).

§11.5 — three review-derived invariants now documented in code:
  - DisableTimerDrain test-only contract
  - shipMu → writeMu lock hierarchy
  - RecordShipped four-path coverage audit

§11.6 — open items carried forward (§IV T2 barrier vs targetLSN;
  §IV T7 R2 saturation under sustained pressure; PR template line
  banning `lsn > cursor + 1`).
 2026-04-30 08:57:10 -07:00