Commit Graph
11 Commits
Author SHA1 Message Date
pingqiuandClaude Opus 4.6 59a36013d4 feat: rebuild hardening A1-A5 + session-controlled execution path
A1 Engine kind-routing fix:
  SessionProgressObserved/Completed/Failed now respect active session
  Kind. Rebuild progress no longer leaks into catch-up aggregate.
  sessionKindMismatch guard + observeRebuildProgress helper.
  2 regression tests lock kind isolation.

A2 Retention pin:
  Rebuild session ack drives progress-based WAL retention floor.
  Pin installed at base_lsn on accepted, advances with wal_applied_lsn,
  released on completed/failed/cancelled. rebuildProgressPinFloor
  returns min across all active replicas.
  Retention pin test: 100 blocks fill WAL, 5 flusher cycles with
  20 pinned rebuild entries — all verified correct.

A3 Progress ack emission:
  Automatic sessionAck(running/base_complete/completed/failed) emitted
  from rebuild session lifecycle transitions. sessionAckLocked builds
  ack under session lock. emitRebuildSessionAck callback wired through
  SetOnRebuildSessionAck on BlockVol.
  ObserveReplicaRebuildSessionAck maps acks to core engine events.
  WireLocalReplicaRebuildSessionAcks bridges local callback to server.
  5 server tests proving ack→core, pin advance, pin cleanup.

A4 Deadline/timeout:
  rebuildAckWatch watchdog: armed on accepted/running/base_complete,
  refreshed on each ack, cleared on completed/failed. Timeout
  cancels local session + clears pin + fail-closes.
  2 tests: timeout→fail-close, progress→refresh.

A5 Session-controlled execution path:
  v2bridge.Executor.TransferFullBase now uses session-controlled loop:
  beginControlledFullBase → real sessionControl over TCP →
  transferExtentToSession via RebuildTransportClient →
  PrepareFullBaseRebuild → TryCompleteRebuildSession.
  ReplicaReceiver control channel handles MsgSessionControl alongside
  MsgBarrierReq. Session acks written back on same TCP connection.
  RebuildSessionBase request type separates new per-block stream from
  legacy raw extent stream. Full-base cleanup deferred until success.
  Deadlock fix: ApplyBaseBlock releases session lock before ioMu.
  Hydration skip for full-base sessions.

23 rebuild component tests (all pass):
  11 kernel correctness, 8 transport/runtime, 3 scenario-scale,
  including 1GB primary-initiated with CRC validation.

29 files changed, ~2500 insertions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 14:39:11 -07:00
pingqiuandClaude Opus 4.6 d2d57851b0 feat: rebuild MVP — dual-lane session with bitmap protection
Rebuild session protocol implementation for v2-rebuild-mvp-session-protocol.md.

New files:
- rebuild_bitmap.go: RebuildBitmap — session-scoped dense bitset for
  WAL-applied LBA tracking. MarkApplied on local WAL write (not receive).
  ShouldApplyBase returns false for WAL-covered LBAs (WAL always wins).

- rebuild_session.go: RebuildSession — replica-side two-line rebuild.
  WAL lane (ApplyWALEntry) + base lane (ApplyBaseBlock) with bitmap
  conflict resolution. TryComplete requires BOTH base_complete AND
  wal_applied_lsn >= target_lsn. Volume-level control surface:
  StartRebuildSession, ApplyRebuildSessionWALEntry/BaseBlock,
  MarkRebuildSessionBaseComplete, TryCompleteRebuildSession,
  CancelRebuildSession, ActiveRebuildSession.

- rebuild_mvp_test.go: 4 correctness tests — base+WAL converge,
  WAL-applied never overwritten by base, bitmap set on applied not
  received, control surface start/supersede/complete.

- rebuild_transport_test.go: 2 transport-level tests — two-line with
  real WAL shipping, live writes during base copy with bitmap conflict.

Design docs:
- v2-rebuild-mvp-session-protocol.md: MVP spec with message set, apply
  rules, completion/failure/crash rules, test matrix
- v2-sync-recovery-protocol.md: full protocol context (keepup/catchup/
  rebuild unified design, primary decision logic, two-line model)
- v2-session-protocol-shape.md: protocol shape overview

Protocol engine (reference, not production):
- sw-block/protocol/: 7-event engine with ~300 lines, 13 tests

6 rebuild tests pass, all existing component tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 14:30:34 -07:00
pingqiuandClaude Opus 4.6 d1a16fac03 feat: protocol-aware execution wave — phase gate for live WAL shipping
Add host-side protocol state seam that derives per-replica execution
state from V2 sender/session snapshots and blocks live-tail WAL
shipping while an active recovery session is in progress.

New file: weed/server/block_protocol_state.go
  - replicaProtocolExecutionState derived from engine snapshots
  - LiveEligible=false during active catch-up/rebuild sessions
  - bindProtocolExecutionPolicy wires policy into BlockVol
  - syncProtocolExecutionState called after assignments + core events

Data plane changes:
  - WALShipper.Ship() checks liveShippingPolicy before dial/send
  - BlockVol.SetLiveShippingPolicy persists across shipper group rebuilds
  - ShipperGroup propagates policy to all shippers

Design contract: sw-block/design/v2-protocol-aware-execution.md

Scope: WAL-first rollout only. Prevents illegal live-tail delivery
during active recovery. Does not change snapshot/build behavior or
move backlog. Next wave: bounded WAL catch-up under same contract.

Tests: 4 unit/component tests for phase gate behavior, plus bootstrap
seam tests that confirmed the two pre-existing bugs locally.

13 files changed, 900 insertions, 69 deletions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 23:47:07 -07:00
pingqiu 43dbebfa04 refactor: close bounded recovery drain and invalidation seams
Move removed-replica drain and replica-scoped invalidation onto explicit core-command paths so the widened multi-replica runtime no longer depends on coarse host-side recovery handling.

Made-with: Cursor
2026-04-04 11:01:12 -07:00
pingqiu 5fd9ec0edf refactor: widen bounded multi-replica catch-up startup ownership
Emit one core-owned start_recovery_task per primary catch-up replica so the bounded multi-replica startup path no longer depends on a single-replica assumption.

Made-with: Cursor
2026-04-04 10:21:28 -07:00
pingqiu 92c006eb29 refactor: aggregate bounded multi-replica catch-up conservatively
Track catch-up observations per replica so the volume-level recovery view stays in catching_up until all bounded replicas complete. This preserves the current bounded semantics while removing an overclaim that would block later multi-replica startup ownership work.

Made-with: Cursor
2026-04-04 09:27:03 -07:00
pingqiu 16ba70f856 refactor: make bounded recovery observation events replica-scoped
Carry replica-scoped addressing through bounded recovery planning and completion events so the core no longer depends on a volume-only observation seam. This preserves the current single-replica catch-up and rebuilding behavior while aligning the observation side with the replica-scoped command path.

Made-with: Cursor
2026-04-04 09:18:07 -07:00
pingqiuandClaude Opus 4.6 b304b8e212 refactor: make bounded recovery command addressing replica-scoped
Replace the remaining volume-scoped recovery command and pending slot
with replica-scoped addressing on the bounded core-present path. This
preserves the current single-replica catch-up and rebuilding behavior
while removing the structural blocker for later multi-replica startup
ownership.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 09:05:36 -07:00
pingqiuandClaude Opus 4.6 11c6aaf316 feat: Batch 7 + Phase 16C-E — command dispatch extraction + engine refinements
Batch 7: Command dispatch binding extraction
- New weed/server/blockcmd package: CommandHandler interface + DispatchCommands
- volume_server_block.go applyCoreCommandsWithAssignment delegates to dispatcher
- weed/server still owns RecordCommand, EmitCoreEvent, PublishProjection
- v2bridge NOT given command-switch or event-emission semantics

Phase 16C: Rebuilding assignment enters core command path
Phase 16D: Rebuild recovery-task startup is command-driven
Phase 16E: Catch-up recovery-task startup is command-driven

Engine refinements:
- RecoveryTarget on AssignmentDelivered event
- shouldStartRecoveryTask / shouldStartReceiver guards
- bootstrapReason: awaiting_rebuild_start

Bridge/contract updates:
- control_adapter.go: refined translation helpers
- contract.go: executor port alignment

Migration design docs (Batch 1-3 delivered, design artifacts):
- v2-first/second/third-migration-batch.md + task-pack.md
- v2-assignment-translation-unification.md
- v2-execution-muscles-inventory.md
- v2-separation-port-layer-audit.md
- v2-legacy-runtime-exit-criteria.md

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 02:13:08 -07:00
pingqiuandClaude Opus 4.6 a6fc8545b9 feat: Phase 14A+14B — V2 core publication ownership + command semantics
14A: Publication as explicit core-owned state
- state.go: PublicationView on VolumeState, explicit gate reasons
- engine.go: mode→readiness→publication chain with named gates
  (awaiting_role_apply, awaiting_shipper_configured, awaiting_barrier_durability)
- projection.go: PublicationProjection carries publication truth
- RF=1/no-replicas → allocated_only (CP13-9 constraint in core)
- phase14_core_test.go: strengthened publication closure + RF=1 proof

14B: Command emission bounded by semantic gap
- engine.go: repeated same-assignment skips redundant commands,
  repeated same-reason BarrierRejected skips duplicate invalidation,
  command-state tracking on VolumeState
- command.go: new command types for bounded emission
- event.go: new boundary events
- phase14_command_test.go: exact command sequences frozen as proofs
  (primary/replica repeated assignment, assignment changed, repeated failure)
- phase14_boundary_test.go: boundary/recovery structural tests

All tests pass in sw-block/engine/replication.
Phase 14 docs updated (14A accepted, 14B active→14C planned).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 16:52:55 -07:00
pingqiu fb0da91196 feat: start Phase 14 V2 core shell
Make the first V2 core owner explicit in sw-block by freezing Phase 14 docs, mode/readiness/publication semantics, and bounded command emission rules. This turns accepted Phase 13 constraints into executable core behavior without overclaiming live runtime cutover.

Made-with: Cursor
2026-04-03 16:11:38 -07:00