mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-08-19 05:36:58 +00:00
docs(p15): add G15b Kubernetes lab instruction
This commit is contained in:
pingqiu
@@ -0,0 +1,144 @@
|
||||
# V3 Phase 15 G15b Kubernetes Static PV QA Test Instruction
|
||||
|
||||
**Date**: 2026-05-03
|
||||
**Status**: K8s lab instruction for `p15-g15b/k8s-static-pv@32b3a13`; execution pending
|
||||
**Scope**: single-node Kubernetes static PV/PVC/pod smoke through real V3 daemons and CSI.
|
||||
|
||||
---
|
||||
|
||||
## Headline
|
||||
|
||||
At `seaweed_block@32b3a13`, the G15b lab harness is staged to prove:
|
||||
|
||||
```text
|
||||
blockmaster + product-loop + r1/r2 blockvolume
|
||||
-> CSI external-attacher calls ControllerPublish
|
||||
-> blockcsi reads frontend target fact from blockmaster
|
||||
-> kubelet NodeStage performs real iSCSI attach/mount
|
||||
-> pod writes/reads checksum through PVC
|
||||
-> cleanup leaves no dangling test iSCSI session
|
||||
```
|
||||
|
||||
This is the first Kubernetes form of the G15a privileged m01 proof. The first run is single-node and preserves loopback-only frontend exposure.
|
||||
|
||||
---
|
||||
|
||||
## Preconditions
|
||||
|
||||
Required:
|
||||
|
||||
- Kubernetes cluster API reachable by `kubectl`.
|
||||
- Single Linux node with iSCSI initiator support.
|
||||
- `iscsiadm`, mount, and kubelet CSI mount paths available on the node.
|
||||
- Local images preloaded into the cluster:
|
||||
- `sw-block:local` containing `/usr/local/bin/blockmaster` and `/usr/local/bin/blockvolume`
|
||||
- `sw-block-csi:local` containing `/usr/local/bin/blockcsi`
|
||||
- The cluster must allow privileged CSI node pods.
|
||||
|
||||
Known current local limitation:
|
||||
|
||||
- On the current dev workstation, `kubectl` context `rancher-desktop` exists but API server is not reachable. This instruction needs QA or a running K8s lab.
|
||||
|
||||
---
|
||||
|
||||
## Commands
|
||||
|
||||
Pre-flight from the code repo:
|
||||
|
||||
```powershell
|
||||
cd C:\work\seaweed_block_g9c
|
||||
git checkout p15-g15b/k8s-static-pv
|
||||
go test ./cmd/blockcsi -run TestG15b_Manifest -count=1 -v
|
||||
go test ./core/csi ./cmd/blockcsi ./core/host/volume ./core/host/master ./core/authority ./cmd/blockmaster ./cmd/blockvolume -count=1
|
||||
```
|
||||
|
||||
Kubernetes lab run from Linux or WSL with `kubectl` configured:
|
||||
|
||||
```bash
|
||||
cd /path/to/seaweed_block
|
||||
G15B_ARTIFACT_DIR=/mnt/smb/work/share/g15b-k8s/runs/$(date -u +%Y%m%dT%H%M%SZ) \
|
||||
bash scripts/run-g15b-k8s-static.sh "$PWD"
|
||||
```
|
||||
|
||||
Expected result:
|
||||
|
||||
- script exits 0;
|
||||
- pod `default/sw-block-static-smoke` reaches `Succeeded`;
|
||||
- artifact dir contains pod logs and product daemon logs;
|
||||
- cleanup deletes the test PV/PVC/pod and product/CSI manifests.
|
||||
|
||||
---
|
||||
|
||||
## Scenario Map
|
||||
|
||||
| Scenario | Evidence / backing test |
|
||||
|---|---|
|
||||
| Manifest attach path requires `ControllerPublish`. | `TestG15b_Manifest_CSIDriverRequiresAttach` |
|
||||
| Controller deploy uses external-attacher, not provisioner. | `TestG15b_Manifest_ControllerUsesAttacherNotProvisioner` |
|
||||
| Product stack uses G9G cluster-spec/product-loop and RF=2 r1/r2. | `TestG15b_Manifest_ProductStackSingleNodeLoopbackShape` |
|
||||
| Static PV does not carry target endpoint truth. | `TestG15b_Manifest_StaticPVDoesNotEmbedTargetFacts` |
|
||||
| Node plugin has privileged host mount shape. | `TestG15b_Manifest_NodePluginPrivilegedShape` |
|
||||
| Pod write/read checksum path. | `scripts/run-g15b-k8s-static.sh` pod phase + `pod.log` |
|
||||
| Product logs captured for debug. | `blockmaster.log`, `blockvolume-r1.log`, `blockvolume-r2.log`, `blockcsi-controller.log` |
|
||||
|
||||
---
|
||||
|
||||
## Artifact Expectations
|
||||
|
||||
The run should write:
|
||||
|
||||
- `run.log`
|
||||
- `kubectl-version.txt`
|
||||
- `nodes.before.txt`
|
||||
- `apply-*.log`
|
||||
- `pod.log`
|
||||
- `pod.describe.txt`
|
||||
- `blockmaster.log`
|
||||
- `blockvolume-r1.log`
|
||||
- `blockvolume-r2.log`
|
||||
- `blockcsi-controller.log`
|
||||
- `kube-system-pods.txt`
|
||||
- `app-pv-pvc-pod.txt`
|
||||
- `cleanup.log`
|
||||
|
||||
If the run fails, preserve the entire artifact directory.
|
||||
|
||||
---
|
||||
|
||||
## Non-Claims
|
||||
|
||||
G15b first lab does not claim:
|
||||
|
||||
- multi-node Kubernetes;
|
||||
- routable iSCSI target exposure;
|
||||
- dynamic CSI provisioning;
|
||||
- snapshot/clone/expand;
|
||||
- NVMe CSI;
|
||||
- pod remount after failover;
|
||||
- primary kill while mounted;
|
||||
- plugin restart cleanup;
|
||||
- performance or soak.
|
||||
|
||||
---
|
||||
|
||||
## Follow-Up If First Run Fails
|
||||
|
||||
Triage by first failing layer:
|
||||
|
||||
1. Image pull / binary missing
|
||||
- Fix image build/load pipeline; product code not implicated.
|
||||
|
||||
2. external-attacher does not call `ControllerPublish`
|
||||
- Inspect `CSIDriver`, `VolumeAttachment`, and attacher logs.
|
||||
|
||||
3. `ControllerPublish` returns no publish_context
|
||||
- Inspect `blockmaster.log`, `blockvolume-r1.log`, and master status facts.
|
||||
|
||||
4. `NodeStage` fails before iSCSI login
|
||||
- Inspect node plugin logs and CSI request context.
|
||||
|
||||
5. `iscsiadm` login/mount fails
|
||||
- Compare with G15a privileged m01 evidence; verify hostNetwork/loopback and `/etc/iscsi` mounts.
|
||||
|
||||
6. Pod checksum fails
|
||||
- Inspect `blockvolume-r1.log` for SCSI writes and replication barriers.
|
||||
@@ -1,7 +1,7 @@
|
||||
# V3 Phase 15 — G15b Kubernetes Static PV Mini-Plan
|
||||
|
||||
**Date**: 2026-05-03
|
||||
**Status**: G15b-1 manifest skeleton + static guards implemented at `seaweed_block@62325c9`
|
||||
**Status**: G15b-1 manifests implemented at `62325c9`; G15b-2 lab harness staged at `32b3a13`; Kubernetes run pending
|
||||
**Branch**: `p15-g15b/k8s-static-pv` from `ac49adb`
|
||||
**Goal**: prove a Kubernetes pod can consume a pre-provisioned V3 block volume through `cmd/blockcsi`, using real Kubernetes CSI control flow and real Linux iSCSI staging.
|
||||
|
||||
@@ -162,11 +162,28 @@ Result: PASS on `62325c9`.
|
||||
|
||||
### G15b-2 — K8s Lab Harness
|
||||
|
||||
Status: **harness staged** at `seaweed_block@32b3a13`; real Kubernetes execution pending.
|
||||
|
||||
Artifacts:
|
||||
|
||||
- `V:\share\g15b-k8s\run-g15b-k8s-static.sh`
|
||||
- `scripts/run-g15b-k8s-static.sh`
|
||||
- `sw-block/design/test/v3-phase-15-g15b-k8s-qa-test-instruction.md`
|
||||
|
||||
Additional manifests:
|
||||
|
||||
- `deploy/k8s/g15b/block-stack.yaml`
|
||||
- `sw-block-cluster-spec` ConfigMap
|
||||
- `sw-blockmaster` Deployment + Service
|
||||
- `sw-blockvolume-r1` Deployment
|
||||
- `sw-blockvolume-r2` Deployment
|
||||
|
||||
First topology:
|
||||
|
||||
- single-node Kubernetes;
|
||||
- `blockvolume` pods use `hostNetwork: true`;
|
||||
- iSCSI remains `127.0.0.1:3260`;
|
||||
- this intentionally preserves the G15a loopback-only frontend guard.
|
||||
|
||||
Harness responsibilities:
|
||||
|
||||
1. Build V3 binaries/images for `blockmaster`, `blockvolume`, and `blockcsi`.
|
||||
@@ -185,6 +202,21 @@ Pass:
|
||||
- Pod writes and reads byte-equal data.
|
||||
- No dangling iSCSI session for the test IQN after cleanup.
|
||||
|
||||
Pre-flight verification already green at `32b3a13`:
|
||||
|
||||
```powershell
|
||||
go test ./cmd/blockcsi -run TestG15b_Manifest -count=1 -v
|
||||
go test ./core/csi ./cmd/blockcsi ./core/host/volume ./core/host/master ./core/authority ./cmd/blockmaster ./cmd/blockvolume -count=1
|
||||
```
|
||||
|
||||
Not yet proven:
|
||||
|
||||
- Kubernetes API server availability;
|
||||
- image build/load path;
|
||||
- external-attacher calling `ControllerPublish`;
|
||||
- kubelet calling `NodeStage` / `NodePublish`;
|
||||
- pod checksum write/read.
|
||||
|
||||
### G15b-3 — First Kubernetes Close Run
|
||||
|
||||
Evidence target:
|
||||
|
||||
Reference in New Issue
Block a user