From a4a5ea1758baa9b12b2a4ba3afdf63ed29682d3d Mon Sep 17 00:00:00 2001 From: pingqiu Date: Sun, 3 May 2026 08:58:42 -0700 Subject: [PATCH] docs(p15): add G15b Kubernetes lab instruction --- ...3-phase-15-g15b-k8s-qa-test-instruction.md | 144 ++++++++++++++++++ ...3-phase-15-g15b-k8s-static-pv-mini-plan.md | 36 ++++- 2 files changed, 178 insertions(+), 2 deletions(-) create mode 100644 sw-block/design/test/v3-phase-15-g15b-k8s-qa-test-instruction.md diff --git a/sw-block/design/test/v3-phase-15-g15b-k8s-qa-test-instruction.md b/sw-block/design/test/v3-phase-15-g15b-k8s-qa-test-instruction.md new file mode 100644 index 000000000..a3772c7c1 --- /dev/null +++ b/sw-block/design/test/v3-phase-15-g15b-k8s-qa-test-instruction.md @@ -0,0 +1,144 @@ +# V3 Phase 15 G15b Kubernetes Static PV QA Test Instruction + +**Date**: 2026-05-03 +**Status**: K8s lab instruction for `p15-g15b/k8s-static-pv@32b3a13`; execution pending +**Scope**: single-node Kubernetes static PV/PVC/pod smoke through real V3 daemons and CSI. + +--- + +## Headline + +At `seaweed_block@32b3a13`, the G15b lab harness is staged to prove: + +```text +blockmaster + product-loop + r1/r2 blockvolume + -> CSI external-attacher calls ControllerPublish + -> blockcsi reads frontend target fact from blockmaster + -> kubelet NodeStage performs real iSCSI attach/mount + -> pod writes/reads checksum through PVC + -> cleanup leaves no dangling test iSCSI session +``` + +This is the first Kubernetes form of the G15a privileged m01 proof. The first run is single-node and preserves loopback-only frontend exposure. + +--- + +## Preconditions + +Required: + +- Kubernetes cluster API reachable by `kubectl`. +- Single Linux node with iSCSI initiator support. +- `iscsiadm`, mount, and kubelet CSI mount paths available on the node. +- Local images preloaded into the cluster: + - `sw-block:local` containing `/usr/local/bin/blockmaster` and `/usr/local/bin/blockvolume` + - `sw-block-csi:local` containing `/usr/local/bin/blockcsi` +- The cluster must allow privileged CSI node pods. + +Known current local limitation: + +- On the current dev workstation, `kubectl` context `rancher-desktop` exists but API server is not reachable. This instruction needs QA or a running K8s lab. + +--- + +## Commands + +Pre-flight from the code repo: + +```powershell +cd C:\work\seaweed_block_g9c +git checkout p15-g15b/k8s-static-pv +go test ./cmd/blockcsi -run TestG15b_Manifest -count=1 -v +go test ./core/csi ./cmd/blockcsi ./core/host/volume ./core/host/master ./core/authority ./cmd/blockmaster ./cmd/blockvolume -count=1 +``` + +Kubernetes lab run from Linux or WSL with `kubectl` configured: + +```bash +cd /path/to/seaweed_block +G15B_ARTIFACT_DIR=/mnt/smb/work/share/g15b-k8s/runs/$(date -u +%Y%m%dT%H%M%SZ) \ + bash scripts/run-g15b-k8s-static.sh "$PWD" +``` + +Expected result: + +- script exits 0; +- pod `default/sw-block-static-smoke` reaches `Succeeded`; +- artifact dir contains pod logs and product daemon logs; +- cleanup deletes the test PV/PVC/pod and product/CSI manifests. + +--- + +## Scenario Map + +| Scenario | Evidence / backing test | +|---|---| +| Manifest attach path requires `ControllerPublish`. | `TestG15b_Manifest_CSIDriverRequiresAttach` | +| Controller deploy uses external-attacher, not provisioner. | `TestG15b_Manifest_ControllerUsesAttacherNotProvisioner` | +| Product stack uses G9G cluster-spec/product-loop and RF=2 r1/r2. | `TestG15b_Manifest_ProductStackSingleNodeLoopbackShape` | +| Static PV does not carry target endpoint truth. | `TestG15b_Manifest_StaticPVDoesNotEmbedTargetFacts` | +| Node plugin has privileged host mount shape. | `TestG15b_Manifest_NodePluginPrivilegedShape` | +| Pod write/read checksum path. | `scripts/run-g15b-k8s-static.sh` pod phase + `pod.log` | +| Product logs captured for debug. | `blockmaster.log`, `blockvolume-r1.log`, `blockvolume-r2.log`, `blockcsi-controller.log` | + +--- + +## Artifact Expectations + +The run should write: + +- `run.log` +- `kubectl-version.txt` +- `nodes.before.txt` +- `apply-*.log` +- `pod.log` +- `pod.describe.txt` +- `blockmaster.log` +- `blockvolume-r1.log` +- `blockvolume-r2.log` +- `blockcsi-controller.log` +- `kube-system-pods.txt` +- `app-pv-pvc-pod.txt` +- `cleanup.log` + +If the run fails, preserve the entire artifact directory. + +--- + +## Non-Claims + +G15b first lab does not claim: + +- multi-node Kubernetes; +- routable iSCSI target exposure; +- dynamic CSI provisioning; +- snapshot/clone/expand; +- NVMe CSI; +- pod remount after failover; +- primary kill while mounted; +- plugin restart cleanup; +- performance or soak. + +--- + +## Follow-Up If First Run Fails + +Triage by first failing layer: + +1. Image pull / binary missing + - Fix image build/load pipeline; product code not implicated. + +2. external-attacher does not call `ControllerPublish` + - Inspect `CSIDriver`, `VolumeAttachment`, and attacher logs. + +3. `ControllerPublish` returns no publish_context + - Inspect `blockmaster.log`, `blockvolume-r1.log`, and master status facts. + +4. `NodeStage` fails before iSCSI login + - Inspect node plugin logs and CSI request context. + +5. `iscsiadm` login/mount fails + - Compare with G15a privileged m01 evidence; verify hostNetwork/loopback and `/etc/iscsi` mounts. + +6. Pod checksum fails + - Inspect `blockvolume-r1.log` for SCSI writes and replication barriers. diff --git a/sw-block/design/v3-phase-15-g15b-k8s-static-pv-mini-plan.md b/sw-block/design/v3-phase-15-g15b-k8s-static-pv-mini-plan.md index 9383bc002..c231ba2da 100644 --- a/sw-block/design/v3-phase-15-g15b-k8s-static-pv-mini-plan.md +++ b/sw-block/design/v3-phase-15-g15b-k8s-static-pv-mini-plan.md @@ -1,7 +1,7 @@ # V3 Phase 15 — G15b Kubernetes Static PV Mini-Plan **Date**: 2026-05-03 -**Status**: G15b-1 manifest skeleton + static guards implemented at `seaweed_block@62325c9` +**Status**: G15b-1 manifests implemented at `62325c9`; G15b-2 lab harness staged at `32b3a13`; Kubernetes run pending **Branch**: `p15-g15b/k8s-static-pv` from `ac49adb` **Goal**: prove a Kubernetes pod can consume a pre-provisioned V3 block volume through `cmd/blockcsi`, using real Kubernetes CSI control flow and real Linux iSCSI staging. @@ -162,11 +162,28 @@ Result: PASS on `62325c9`. ### G15b-2 — K8s Lab Harness +Status: **harness staged** at `seaweed_block@32b3a13`; real Kubernetes execution pending. + Artifacts: -- `V:\share\g15b-k8s\run-g15b-k8s-static.sh` +- `scripts/run-g15b-k8s-static.sh` - `sw-block/design/test/v3-phase-15-g15b-k8s-qa-test-instruction.md` +Additional manifests: + +- `deploy/k8s/g15b/block-stack.yaml` + - `sw-block-cluster-spec` ConfigMap + - `sw-blockmaster` Deployment + Service + - `sw-blockvolume-r1` Deployment + - `sw-blockvolume-r2` Deployment + +First topology: + +- single-node Kubernetes; +- `blockvolume` pods use `hostNetwork: true`; +- iSCSI remains `127.0.0.1:3260`; +- this intentionally preserves the G15a loopback-only frontend guard. + Harness responsibilities: 1. Build V3 binaries/images for `blockmaster`, `blockvolume`, and `blockcsi`. @@ -185,6 +202,21 @@ Pass: - Pod writes and reads byte-equal data. - No dangling iSCSI session for the test IQN after cleanup. +Pre-flight verification already green at `32b3a13`: + +```powershell +go test ./cmd/blockcsi -run TestG15b_Manifest -count=1 -v +go test ./core/csi ./cmd/blockcsi ./core/host/volume ./core/host/master ./core/authority ./cmd/blockmaster ./cmd/blockvolume -count=1 +``` + +Not yet proven: + +- Kubernetes API server availability; +- image build/load path; +- external-attacher calling `ControllerPublish`; +- kubelet calling `NodeStage` / `NodePublish`; +- pod checksum write/read. + ### G15b-3 — First Kubernetes Close Run Evidence target: