4.33

* feat(ec): add encode_ts_ns to EC shard metadata and the shard read RPC

EcShardConfig and VolumeEcShardReadRequest gain an int64 encode_ts_ns
(encode time in unix nanos). It rides in .vif and the read request so a
read can be scoped to the encode run that produced the index.

* fix(ec): stamp each encode and reject cross-run shard reads

Generate stamps EncodeTsNs into the volume's .vif. Reads carry it to the
shard's owning volume (resolved together via FindEcVolumeWithShard, so a
multi-disk server validates the disk that actually serves the bytes) and
reject a shard from a different encode run, recovering from parity. A
zero on either side (pre-upgrade volume) skips the guard.

* fix(ec): stamp the encode identity on the worker-generated .vif

The worker-local encode path now writes EncodeTsNs (and the resolved EC
ratio) into the .vif, so the read guard is not silently off for volumes
encoded by the maintenance worker.

* fix(ec): wipe stale EC artifacts before re-encoding

VolumeEcShardsGenerate evicts any in-memory EcVolume for the volume and
removes its on-disk shard/index/sidecar files before writing fresh ones,
so a retried encode never builds on a partial prior run and the unlink
frees the inodes instead of leaving open fds serving old bytes.

* fix(ec): unmount EC shards across all disks

UnmountEcShards walked only the first disk holding the shard, leaving a
duplicate copy mounted on a sibling disk (split-disk reconciled volumes)
still serving and heartbeating. Traverse every disk and emit one
deletion delta per disk.

* fix(ec): delete orphan shards without a local .ecx

deleteEcShardIdsForEachLocation gated shard-file removal on a local .ecx,
so it could not clean an orphan .ecNN left by a failed copy on a disk
with no index. Delete the requested shard files unconditionally; the
index-file (.ecx/.ecj/.vif) routing stays gated as before.

* fix(ec): clear stale EC shards cluster-wide before re-encoding

ec.encode unmounts and deletes EC shards for the target volumes on every
node before regenerating: fatal for the shards the topology reports
(mounted leftovers), best-effort for the rest (a sweep that catches
unmounted failed-copy orphans). A down node is a no-op.

* fix(ec): don't nil EC fds on close so reads can't race eviction

A reader resolves an EcVolume/shard under the lock then reads after it is
released, so an eviction that nils ecxFile/ecdFile would race that read
and panic. Close the fds without nilling the fields: the field is now
write-once (no data race) and a concurrent read hits a closed fd, getting
a clean error that the caller recovers from parity.

* fix(ec): wipe stale EC artifacts on every disk and surface failures

The pre-encode wipe only deleted beside the source volume, so a stale
shard on a sibling disk survived and could be mounted against the new
index at reconcile. Sweep every disk. Removal also ignored os.Remove
errors, reporting a failed cleanup as success and letting a stale shard
join the next generation; surface the first real failure (treating
already-gone as success) from removeStaleEcArtifacts and the shard delete.

* fix(ec): log when a local shard is skipped for a different encode run

The cross-run guard returned errShardNotLocal, indistinguishable in logs
from a genuinely-absent shard. Add a V(1) line naming both EncodeTsNs so
operators can tell "wrong encode generation" from "shard not here".

* fix(ec): surface metadata removal failures in the shard delete path

deleteEcShardIdsForEachLocation still dropped os.Remove errors on the
.ecx/.ecj/.vif/sidecar cleanup. A surviving stale .ecx is the orphan-index
condition this path prevents, so route those through removeFileIfExists and
return the first real failure instead of reporting cleanup as success.

* fix(ec): fail orphan cleanup when a reachable node's delete fails

The pre-encode orphan sweep swallowed every error for unreported (node,
volume) pairs. That is only safe for an unreachable node, which cannot
receive this encode's new generation. A reachable node whose delete
genuinely failed (permission/IO) keeps an orphan shard that a later copy
re-stamps with the new run's volume-level .vif identity, so the read guard
would accept stale data. Surface those; stay best-effort only for
unreachable nodes (gRPC Unavailable / no status).

* fix(ec): guard ecjFile under its lock in the EC delete path

EcVolume.Close nils ecjFile under ecjFileAccessLock; a delete that resolved
its .ecx lookup before a concurrent eviction (the generate-time
UnloadEcVolume) could then reach the journal append with a nil fd. Bail
with a clear "volume closed" error under the lock instead.

* fix(ec): reject an unstamped shard when the caller has an encode identity

The read guard required both identities nonzero, so a current (stamped)
caller accepted a holder with identity 0 and could be served a stale
pre-upgrade shard. Reject when the caller is stamped and the holder
differs (including unstamped); stay lenient only when the caller itself
has no identity (pre-upgrade reader). A skipped shard recovers from parity.

* fix(ec): full-teardown delete so cluster cleanup wipes a whole generation

The pre-encode cluster sweep deleted only the listed canonical shards on
remote nodes, leaving index/sidecar (and, on builds with versioned
generations, those too) behind. Add a full_teardown flag to
VolumeEcShardsDelete that evicts the volume and wipes every EC artifact for
it on every disk via removeStaleEcArtifacts; the shell and worker pre-encode
cleanup paths set it. Other delete callers (balance/decode/repair) are
unchanged.

* fix(ec): take ecjFileAccessLock before the nil-check in Sync and Close

Sync and Close read ev.ecjFile before acquiring ecjFileAccessLock while
Close nils it under the lock, a data race on the field. Take the lock
first, then nil-check inside, in both.

* fix(ec): acknowledge full_teardown so a pre-upgrade server can't fake success

An old volume server silently ignores full_teardown and returns success
for an ordinary delete, so the caller wrongly believes the generation was
wiped and copies a fresh gen-0 onto an unwiped node. Echo full_teardown_done
in the response; the worker destination cleanup fails when it is absent, and
the shell cluster sweep fails for a reported (mounted) leftover while staying
best-effort for an unreported node. encode_ts_ns stays an accepted transient
(an old server just skips the new read guard, no regression).

* fix(ec): fail the pre-encode sweep for any reachable node that can't ack teardown

A reachable pre-upgrade server ignores full_teardown and returns success
without wiping an orphan, which a later copy then folds into the new
generation. Treat a missing full_teardown_done ack as fatal for every
reachable node (best-effort only for a gRPC-unreachable one), not just for
topology-reported pairs.

* fix(ec): return the served shard identity and validate it client-side

The encode identity was only enforced server-side, so a pre-upgrade server
ignored the request field and served bytes unchecked. Echo the served
shard's EncodeTsNs on every read response chunk and have the client reject a
mismatch (including 0 from an old server), so the guard holds regardless of
server version; a rejected read recovers from parity.

* fix(ec): reject a short/empty remote shard read instead of serving zeros

doReadRemoteEcShardInterval accepted an immediate EOF or a short stream and
returned success with a partly zero-filled, unvalidated buffer (the server
stamps the identity only on chunks that carry bytes). A non-deleted interval
must arrive whole: require n == len(buf), exempting the is_deleted
short-circuit (n=0), matching readLocalEcShardInterval's local check. A short
read now fails so the caller recovers from parity.

* test(ec): fake volume server echoes the full_teardown acknowledgement

The worker now fails a teardown delete that isn't acknowledged (so a
pre-upgrade server can't silently skip the wipe). The fake server's no-op
VolumeEcShardsDelete returned an empty response, which the worker read as a
skipped teardown and aborted the encode. Echo full_teardown_done.

* feat(ec): mirror the encode-run identity guard + full_teardown into the Rust volume server

The Go volume server stamps an encode-run identity (encode_ts_ns) into the .vif
and rejects a read served from a shard of a different run; full_teardown wipes a
whole generation and acknowledges it. The Rust volume server had none of it.
Mirror the shared logic: load encode_ts_ns from the .vif onto the EcVolume,
stamp it on every read response, and reject a request/response mismatch on both
the server and the distributed-read client (recovering from parity); handle
full_teardown by evicting the volume and wiping every EC artifact on each disk,
echoing full_teardown_done so the caller can detect a server that ignored it.

* fix(ec): remove a stale .vif on full teardown of a shard-only node

A shard copy installs shards + .ecx before .vif, so an interrupted copy after a
teardown could mount the new files under the previous run's identity / version /
shard ratio / dat_file_size carried by the surviving .vif. Remove .vif during
full teardown, gated on .idx absence so a source-volume holder keeps its live
.vif. In Rust this lives in a teardown-only helper so the reconcile / load-
fallback paths (which share the base removal) still preserve .vif.

* fix(ec): treat a missing teardown ack as fatal, not as an unreachable node

isNodeUnreachable returned true for any non-gRPC-status error, so a reachable
pre-upgrade server's missing full_teardown_done ack (a plain error) was
classified unreachable and the unreported pair was silently skipped. Classify
only a real codes.Unavailable as unreachable, and wrap the missing ack in a
sentinel the sweep treats as fatal regardless. A genuinely down node still
surfaces as Unavailable from the RPC and stays best-effort.

* fix(ec): reject a short shard read in the local EC needle reader

read_ec_shard_needle ignored the byte count from shard.read_at and appended the
whole pre-sized buffer, so a truncated shard's zero-filled tail passed the later
length check and parsed as garbage. Require n == buf.len() per interval, erroring
on a short read like the local interval reader already does.

* fix(ec): probe reachability before skipping a node that returns Unavailable

The pre-encode sweep skipped any node whose teardown delete returned
codes.Unavailable, but a reachable volume server in maintenance mode also
returns that code for the maintenance-gated delete, so its stale EC files were
left behind on a node that can still receive the new generation. Confirm with a
non-maintenance-gated empty-target Ping: skip only when the node fails the probe
too (genuinely unreachable).

* fix(ec): use try_exists for the teardown .vif .idx guard

The teardown-only .vif removal gated on Path::exists(), which returns false on a
permission/IO stat error, so a stat failure on a present .idx would read as a
shard-only node and delete the live source volume's .vif. Gate on
try_exists() == Ok(false) instead, preserving the sidecar on any stat error.

* fix(ec): only skip a sweep node when a Ping confirms it is transport-down

The pre-encode sweep skipped a node whenever its teardown delete and a liveness
Ping both failed, but it treated ANY Ping error as down — an application-level
Internal/ResourceExhausted, or Unimplemented from a pre-Ping server, left a
reachable node's stale generation in place. Classify the Ping tri-state and skip
only when it transport-fails with codes.Unavailable; a reachable or inconclusive
node stays fatal.

* fix(ec): exclude sweep-skipped nodes from the encode's rebalance

The pre-encode sweep skips a genuinely-down node best-effort, but the rebalance
then recollected the current topology — a node that recovered between the two
could become a copy target and receive the new generation while still holding
its stale, never-cleared shards. Have the sweep return the skipped set and
exclude those nodes from the rebalance for this encode, so a node we could not
clean cannot receive the new generation. Standalone ec.balance is unaffected.

* fix(ec): re-sweep recovered nodes before generation so they aren't stranded

A node skipped as down by the pre-encode sweep is excluded from the rebalance,
but it can recover and become the generation host — mounting all shards locally,
then being excluded from distribution. Union-only verification accepts all
shards on one node and deletes the originals: a single point of failure. Re-sweep
the skipped nodes just before generation; one whose teardown now succeeds leaves
the skipped set and rebalances normally, while a node still down stays skipped.

* fix(ec): abort the encode if a selected source is still skipped after re-sweep

The re-sweep un-skips a recovered node, but the source was selected before it and
a node can stay down through the re-sweep then recover just in time to be the
generation host — mounting all shards locally while still excluded from the
rebalance, which union-only verification accepts before deleting the originals.
Abort the encode when a selected source remains skipped after the re-sweep.

* fix(ec): batch delete returns retriable 503 when a volume became EC mid-batch

If a volume is not EC at the batch-delete classification but is encoded to EC and
its .dat deleted before the regular-volume mutation, the mutation returns an exact
"not found" that the filer chunk-GC treats as completed, dropping the delete.
Recheck EC presence under the mutation lock and return a retriable 503 with the
"try again" token so the filer requeues it onto the EC path.

* fix(ec): recheck EC state before the regular batch-delete mutation

ec.encode mounts EC shards (copied from the .dat) before deleting the originals,
so a volume can be EC while its .dat still exists. The batch delete only rechecked
EC after a NotFound, so a successful regular-volume delete in that window wrote a
tombstone to the soon-removed .dat — the delete was lost and the needle resurrected
from the pre-tombstone shards. Recheck has_ec_volume under the write lock before
delete_volume_needle and return a retriable 503 so the filer requeues onto the EC path.

* fix(volume): make the metrics push test independent of test order

test_push_metrics_once asserted the pushed body contains the request-counter
family without ever touching the counter — a CounterVec with no children emits
nothing, so the assertion only held when another test had already created a
labelset in the shared registry. Create one in the test itself.

.github/workflows/container_dev.yml

@@ -111,7 +111,7 @@ jobs:
             org.opencontainers.image.vendor=Chris Lu
 
       - name: Set up QEMU
-        uses: docker/setup-qemu-action@v4
+        uses: docker/setup-qemu-action@v4.1.0
 
       - name: Create BuildKit config
         run: |

.github/workflows/container_foundationdb_version.yml

@@ -126,7 +126,7 @@ jobs:
           echo "seaweedfs_ref=$seaweed" >> "$GITHUB_OUTPUT"
 
       - name: Set up QEMU
-        uses: docker/setup-qemu-action@v4
+        uses: docker/setup-qemu-action@v4.1.0
 
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v4

.github/workflows/container_latest.yml

@@ -207,7 +207,7 @@ jobs:
             org.opencontainers.image.vendor=Chris Lu
       - name: Set up QEMU
         if: matrix.platform != 'amd64'
-        uses: docker/setup-qemu-action@v4
+        uses: docker/setup-qemu-action@v4.1.0
       - name: Create BuildKit config
         run: |
           cat > /tmp/buildkitd.toml <<EOF

.github/workflows/container_release_foundationdb.yml

@@ -43,7 +43,7 @@ jobs:
             org.opencontainers.image.vendor=Chris Lu
       -
         name: Set up QEMU
-        uses: docker/setup-qemu-action@v4
+        uses: docker/setup-qemu-action@v4.1.0
       -
         name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v4

.github/workflows/container_release_unified.yml

@@ -118,8 +118,9 @@ jobs:
     needs: [build-rust-binaries]
     runs-on: ubuntu-latest
     strategy:
-      # Build sequentially to avoid rate limits
-      max-parallel: 2
+      # All variants at once: pulls hit mirror.gcr.io and pushes go to GHCR only,
+      # so docker.io limits don't apply. Watch the 10 GB gha cache budget.
+      max-parallel: 5
       matrix:
         include:
           # Normal volume - multi-arch
@@ -219,7 +220,7 @@ jobs:
             
       - name: Set up QEMU
         if: (github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant) && contains(matrix.platforms, 'arm')
-        uses: docker/setup-qemu-action@v4
+        uses: docker/setup-qemu-action@v4.1.0
         
       - name: Create BuildKit config
         if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
@@ -272,93 +273,52 @@ jobs:
             BRANCH=${{ github.sha }}
             ${{ matrix.variant == 'rocksdb' && format('ROCKSDB_VERSION={0}', github.event.inputs.rocksdb_version || 'v10.10.1') || '' }}
             
-      - name: Clean up build artifacts
-        if: always() && (github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant)
-        run: |
-          sudo docker system prune -f
-          sudo rm -rf /tmp/go-build*
-
-  copy-to-dockerhub:
-    runs-on: ubuntu-latest
-    needs: [build]
-    if: github.event_name != 'pull_request'
-    strategy:
-      matrix:
-        variant: [normal, large_disk, full, large_disk_full, rocksdb]
-        include:
-          - variant: normal
-            tag_suffix: ""
-          - variant: large_disk
-            tag_suffix: _large_disk
-          - variant: full
-            tag_suffix: _full
-          - variant: large_disk_full
-            tag_suffix: _large_disk_full
-          - variant: rocksdb
-            tag_suffix: _large_disk_rocksdb
-            
-    steps:
-      - name: Login to Docker Hub
-        if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
-        uses: docker/login-action@v4.2.0
-        with:
-          username: ${{ secrets.DOCKER_USERNAME }}
-          password: ${{ secrets.DOCKER_PASSWORD }}
-          
-      - name: Login to GHCR
-        if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
-        uses: docker/login-action@v4.2.0
-        with:
-          registry: ghcr.io
-          username: ${{ secrets.GHCR_USERNAME }}
-          password: ${{ secrets.GHCR_TOKEN }}
-          
+      # Copy GHCR -> Docker Hub here, per variant, so it overlaps with the other
+      # variants still building instead of waiting on the whole matrix.
       - name: Install crane
-        if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
+        if: (github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant) && github.event_name != 'pull_request'
         run: |
           cd $(mktemp -d)
           curl -sL "https://github.com/google/go-containerregistry/releases/latest/download/go-containerregistry_Linux_x86_64.tar.gz" | tar xz
           sudo mv crane /usr/local/bin/
           crane version
-          
-      - name: Copy ${{ matrix.variant }} from GHCR to Docker Hub
-        if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
+
+      - name: Copy ${{ matrix.variant }} to Docker Hub
+        if: (github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant) && github.event_name != 'pull_request'
         run: |
-          # Function to retry with exponential backoff
           retry_with_backoff() {
             local max_attempts=5
             local timeout=1
             local attempt=1
             local exit_code=0
-            
             while [ $attempt -le $max_attempts ]; do
               if "$@"; then
                 return 0
               else
                 exit_code=$?
               fi
-              
               if [ $attempt -lt $max_attempts ]; then
                 echo "Attempt $attempt failed. Retrying in ${timeout}s..." >&2
                 sleep $timeout
                 timeout=$((timeout * 2))
               fi
-              
               attempt=$((attempt + 1))
             done
-            
             echo "Command failed after $max_attempts attempts" >&2
             return $exit_code
           }
-          
-          # Copy multi-arch image from GHCR to Docker Hub with retry
-          # This is much more efficient than pulling/pushing individual arch images
+
           echo "Copying ${{ matrix.variant }} from GHCR to Docker Hub..."
           retry_with_backoff crane copy \
             ghcr.io/chrislusf/seaweedfs:${{ env.RELEASE_TAG }}${{ matrix.tag_suffix }} \
             chrislusf/seaweedfs:${{ env.RELEASE_TAG }}${{ matrix.tag_suffix }}
-          
-          echo "Successfully copied ${{ matrix.variant }} to Docker Hub"
+          echo "Copied ${{ matrix.variant }} to Docker Hub"
+
+      - name: Clean up build artifacts
+        if: always() && (github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant)
+        run: |
+          sudo docker system prune -f
+          sudo rm -rf /tmp/go-build*
 
   # Report-only trivy scan: uploads fixable HIGH/CRITICAL findings to GitHub
   # Security for visibility, but never blocks the release. Releases (including
@@ -418,7 +378,7 @@ jobs:
   # trivy-scan: vuln findings are reported but do not block `latest`.
   tag-latest:
     runs-on: ubuntu-latest
-    needs: [copy-to-dockerhub]
+    needs: [build]
     if: github.event_name == 'push'
     strategy:
       matrix:
@@ -483,7 +443,7 @@ jobs:
 
   helm-release:
     runs-on: ubuntu-latest
-    needs: [copy-to-dockerhub]
+    needs: [build]
     if: github.event_name == 'push' || github.event_name == 'workflow_dispatch'
     permissions:
       contents: write

.github/workflows/container_rocksdb_version.yml

@@ -82,7 +82,7 @@ jobs:
           echo "seaweedfs_ref=$seaweed" >> "$GITHUB_OUTPUT"
 
       - name: Set up QEMU
-        uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a # v1
+        uses: docker/setup-qemu-action@06116385d9baf250c9f4dcb4858b16962ea869c3 # v1
 
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v1

.github/workflows/e2e.yml

@@ -31,6 +31,11 @@ jobs:
       with:
         go-version-file: 'go.mod'
 
+    - name: Configure Docker Hub mirror
+      run: |
+        echo '{"registry-mirrors": ["https://mirror.gcr.io"]}' | sudo tee /etc/docker/daemon.json
+        sudo systemctl restart docker
+
     - name: Set up Docker Buildx
       uses: docker/setup-buildx-action@v4
       

.github/workflows/helm_ci.yml

@@ -51,22 +51,22 @@ jobs:
           
           echo "=== Testing default configuration ==="
           helm template test $CHART_DIR > /tmp/default.yaml
-          echo "✓ Default configuration renders successfully"
+          echo "Default configuration renders successfully"
           
           echo "=== Testing with S3 enabled ==="
           helm template test $CHART_DIR --set s3.enabled=true > /tmp/s3.yaml
           grep -q "kind: Deployment" /tmp/s3.yaml && grep -q "seaweedfs-s3" /tmp/s3.yaml
-          echo "✓ S3 deployment renders correctly"
+          echo "S3 deployment renders correctly"
           
           echo "=== Testing with all-in-one mode ==="
           helm template test $CHART_DIR --set allInOne.enabled=true > /tmp/allinone.yaml
           grep -q "seaweedfs-all-in-one" /tmp/allinone.yaml
-          echo "✓ All-in-one deployment renders correctly"
+          echo "All-in-one deployment renders correctly"
           
           echo "=== Testing with security enabled ==="
           helm template test $CHART_DIR --set global.seaweedfs.enableSecurity=true > /tmp/security.yaml
           grep -q "security-config" /tmp/security.yaml
-          echo "✓ Security configuration renders correctly"
+          echo "Security configuration renders correctly"
 
           echo ""
           echo "=== Testing IAM gRPC opt-in path ==="
@@ -121,7 +121,7 @@ jobs:
           if configmap(out, "test-seaweedfs-security-config") is not None:
               failed.append("defaults: security ConfigMap should not render")
           else:
-              print("✓ defaults: no security-config ConfigMap (unchanged)")
+              print("defaults: no security-config ConfigMap (unchanged)")
 
           # Case 2: filerWrite=true alone is the documented opt-in for
           # the Admin UI Users tab. Configmap must render with
@@ -141,7 +141,7 @@ jobs:
               if "[grpc" in toml:
                   failed.append("filerWrite=true: security.toml unexpectedly has [grpc.*] (would need cert mounts)")
               if "[jwt.filer_signing]" in toml and "[grpc" not in toml:
-                  print("✓ filerWrite=true: security.toml has [jwt.filer_signing], no [grpc.*]")
+                  print("filerWrite=true: security.toml has [jwt.filer_signing], no [grpc.*]")
 
           # Case 3: filer + admin pods must MOUNT the security ConfigMap
           # under filerWrite=true so the JWT key reaches both processes.
@@ -154,7 +154,7 @@ jobs:
               if "security-config" not in vols or "security-config" not in mounts:
                   failed.append(f"filerWrite=true: {wl} does not mount security-config (IAM gRPC would still fail)")
               else:
-                  print(f"✓ filerWrite=true: {wl} mounts security-config")
+                  print(f"filerWrite=true: {wl} mounts security-config")
               cert_vols = {v for v in vols if v.endswith("-cert")}
               if cert_vols:
                   failed.append(f"filerWrite=true: {wl} unexpectedly has cert volumes {sorted(cert_vols)}")
@@ -172,7 +172,7 @@ jobs:
               if missing:
                   failed.append(f"enableSecurity=true: security.toml missing {missing}")
               else:
-                  print("✓ enableSecurity=true: security.toml has [jwt.signing] + [grpc.*] preserved")
+                  print("enableSecurity=true: security.toml has [jwt.signing] + [grpc.*] preserved")
 
           # Case 5: helper must tolerate explicit nulls (gemini-code-assist
           # PR review). securityConfig=null was the parens-pattern crash
@@ -187,7 +187,7 @@ jobs:
               if configmap(out, "test-seaweedfs-security-config") is not None:
                   failed.append(f"{null_path}=null: should not render configmap")
               else:
-                  print(f"✓ {null_path}=null: render tolerates explicit null")
+                  print(f"{null_path}=null: render tolerates explicit null")
 
           if failed:
               print("\nFAIL:", file=sys.stderr)
@@ -195,14 +195,14 @@ jobs:
                   print(f"  - {f}", file=sys.stderr)
               sys.exit(1)
           PYEOF
-          echo "✓ IAM gRPC decoupling tests passed"
+          echo "IAM gRPC decoupling tests passed"
 
           echo "=== Testing with monitoring enabled ==="
           helm template test $CHART_DIR \
             --set global.seaweedfs.monitoring.enabled=true \
             --set global.seaweedfs.monitoring.gatewayHost=prometheus \
             --set global.seaweedfs.monitoring.gatewayPort=9091 > /tmp/monitoring.yaml
-          echo "✓ Monitoring configuration renders correctly"
+          echo "Monitoring configuration renders correctly"
           
           echo "=== Testing with PVC storage ==="
           helm template test $CHART_DIR \
@@ -210,25 +210,25 @@ jobs:
             --set master.data.size=10Gi \
             --set master.data.storageClass=standard > /tmp/pvc.yaml
           grep -q "PersistentVolumeClaim" /tmp/pvc.yaml
-          echo "✓ PVC configuration renders correctly"
+          echo "PVC configuration renders correctly"
           
           echo "=== Testing with custom replicas ==="
           helm template test $CHART_DIR \
             --set master.replicas=3 \
             --set filer.replicas=2 \
             --set volume.replicas=3 > /tmp/replicas.yaml
-          echo "✓ Custom replicas configuration renders correctly"
+          echo "Custom replicas configuration renders correctly"
           
           echo "=== Testing filer with S3 gateway ==="
           helm template test $CHART_DIR \
             --set filer.s3.enabled=true \
             --set filer.s3.enableAuth=true > /tmp/filer-s3.yaml
-          echo "✓ Filer S3 gateway renders correctly"
+          echo "Filer S3 gateway renders correctly"
           
           echo "=== Testing SFTP enabled ==="
           helm template test $CHART_DIR --set sftp.enabled=true > /tmp/sftp.yaml
           grep -q "seaweedfs-sftp" /tmp/sftp.yaml
-          echo "✓ SFTP deployment renders correctly"
+          echo "SFTP deployment renders correctly"
           
           echo "=== Testing ingress configurations ==="
           helm template test $CHART_DIR \
@@ -237,12 +237,12 @@ jobs:
             --set s3.enabled=true \
             --set s3.ingress.enabled=true > /tmp/ingress.yaml
           grep -q "kind: Ingress" /tmp/ingress.yaml
-          echo "✓ Ingress configurations render correctly"
+          echo "Ingress configurations render correctly"
           
           echo "=== Testing COSI driver ==="
           helm template test $CHART_DIR --set cosi.enabled=true > /tmp/cosi.yaml
           grep -q "seaweedfs-cosi" /tmp/cosi.yaml
-          echo "✓ COSI driver renders correctly"
+          echo "COSI driver renders correctly"
           
           echo ""
           echo "=== Testing long release name: service names match DNS references ==="
@@ -285,7 +285,7 @@ jobs:
           [ "$MASTER_SVC" = "$MASTER_ADDR_SVC" ] || { echo "FAIL: master service name mismatch"; exit 1; }
           [ "$FILER_CLIENT_SVC" = "$FILER_ADDR_SVC" ] || { echo "FAIL: filer-client service name mismatch"; exit 1; }
           [ "$FILER_CLIENT_SVC" = "$S3_FILER_SVC" ] || { echo "FAIL: S3 -filer= does not match filer-client service"; exit 1; }
-          echo "✓ Normal mode: service names match DNS references with long release name"
+          echo "Normal mode: service names match DNS references with long release name"
 
           # --- All-in-one mode: all-in-one service vs both helper addresses ---
           helm template "$LONG_RELEASE" $CHART_DIR \
@@ -305,7 +305,7 @@ jobs:
 
           [ "$AIO_SVC" = "$AIO_MASTER_ADDR_SVC" ] || { echo "FAIL: all-in-one master address mismatch"; exit 1; }
           [ "$AIO_SVC" = "$AIO_FILER_ADDR_SVC" ] || { echo "FAIL: all-in-one filer address mismatch"; exit 1; }
-          echo "✓ All-in-one mode: service names match DNS references with long release name"
+          echo "All-in-one mode: service names match DNS references with long release name"
 
           echo ""
           echo "=== Testing security+S3: no blank lines in shell command blocks ==="
@@ -343,7 +343,7 @@ jobs:
                   print(f"FAIL: {e}", file=sys.stderr)
               print("Rendered with: global.seaweedfs.enableSecurity=true, filer.s3.enabled=true, s3.enabled=true, allInOne.enabled=true", file=sys.stderr)
               sys.exit(1)
-          print("✓ No blank lines in security+S3 command blocks")
+          print("No blank lines in security+S3 command blocks")
           PYEOF
 
           echo ""
@@ -421,7 +421,7 @@ jobs:
                             f"(got cert={has_cert} key={has_key} https={has_https})", file=sys.stderr)
                       failed = True
                   else:
-                      print(f"✓ {label}: cert/key/https args emitted together")
+                      print(f"{label}: cert/key/https args emitted together")
               else:
                   if has_cert or has_key or has_https:
                       print(f"FAIL: {label}: expected none of {cert_flag}/{key_flag}/{https_flag}; "
@@ -429,7 +429,7 @@ jobs:
                             f"(got cert={has_cert} key={has_key} https={has_https})", file=sys.stderr)
                       failed = True
                   else:
-                      print(f"✓ {label}: no TLS args emitted, main -port stays HTTP")
+                      print(f"{label}: no TLS args emitted, main -port stays HTTP")
 
               # bash -n: pin down that the rendered script parses. Guards against
               # a future helper change that leaves a dangling `\` with nothing
@@ -446,7 +446,49 @@ jobs:
           sys.exit(1 if failed else 0)
           PYEOF
 
-          echo "✅ All template rendering tests passed!"
+          echo ""
+          echo "=== Testing all-in-one env: a key in both global and component renders once ==="
+          # Regression: all-in-one looped global and component extraEnvironmentVars
+          # in two separate ranges, emitting duplicate env entries for any key set
+          # in both maps. Render a shared key and assert it appears exactly once in
+          # the all-in-one container, with the component value winning (consistent
+          # with the merge helper the other components already use). pyyaml is
+          # installed by the earlier IAM gRPC block in this same step.
+          helm template test $CHART_DIR \
+            --set allInOne.enabled=true \
+            --set global.seaweedfs.extraEnvironmentVars.WEED_SHARED=fromGlobal \
+            --set allInOne.extraEnvironmentVars.WEED_SHARED=fromComponent > /tmp/aio-env.yaml
+          python3 - /tmp/aio-env.yaml <<'PYEOF'
+          import sys, yaml
+          from collections import Counter
+          docs = [d for d in yaml.safe_load_all(open(sys.argv[1])) if d]
+          dep = next(d for d in docs if d.get("kind") == "Deployment"
+                     and d["metadata"]["name"].endswith("all-in-one"))
+          envs = [e["name"] for c in dep["spec"]["template"]["spec"]["containers"]
+                  for e in c.get("env", [])]
+          dups = {k: v for k, v in Counter(envs).items() if v > 1}
+          if dups:
+              print(f"FAIL: duplicate env entries in all-in-one container: {dups}", file=sys.stderr)
+              sys.exit(1)
+          val = next(e.get("value") for c in dep["spec"]["template"]["spec"]["containers"]
+                     for e in c.get("env", []) if e["name"] == "WEED_SHARED")
+          if val != "fromComponent":
+              print(f"FAIL: WEED_SHARED should take the component value 'fromComponent', got '{val}'",
+                    file=sys.stderr)
+              sys.exit(1)
+          print("all-in-one env: shared key renders once, component value wins")
+          PYEOF
+
+          echo "=== Testing bucket versioning: YAML bool false suspends like string \"false\" ==="
+          # bool false used to be a silent no-op while string "false" suspended.
+          BOOL_FALSE=$(helm template test $CHART_DIR \
+            --set s3.enabled=true \
+            --set s3.createBuckets[0].name=verbucket \
+            --set s3.createBuckets[0].versioning=false | grep 's3.bucket.versioning -name verbucket' || true)
+          echo "$BOOL_FALSE" | grep -q -- '-status Suspended' || { echo "FAIL: bool false versioning did not Suspend the bucket"; exit 1; }
+          echo "Bucket versioning: YAML bool false suspends consistently with string \"false\""
+
+          echo "All template rendering tests passed!"
 
       - name: Create kind cluster
         uses: helm/kind-action@v1.14.0

.github/workflows/kafka-quicktest.yml

@@ -32,6 +32,11 @@ jobs:
           **/go.sum
       id: go
 
+    - name: Configure Docker Hub mirror
+      run: |
+        echo '{"registry-mirrors": ["https://mirror.gcr.io"]}' | sudo tee /etc/docker/daemon.json
+        sudo systemctl restart docker
+
     - name: Set up Docker Buildx
       uses: docker/setup-buildx-action@v4
 

.github/workflows/nfs-tests.yml

@@ -1,137 +0,0 @@
-name: "NFS Integration Tests"
-
-on:
-  push:
-    branches: [ master, main ]
-    paths:
-      - 'weed/server/nfs/**'
-      - 'weed/command/nfs.go'
-      - 'weed/filer/filer_inode.go'
-      - 'weed/filer/filer_inode_index.go'
-      - 'weed/filer/filerstore_wrapper.go'
-      - 'weed/server/filer_grpc_server_rename.go'
-      - 'test/nfs/**'
-      - '.github/workflows/nfs-tests.yml'
-  pull_request:
-    branches: [ master, main ]
-    paths:
-      - 'weed/server/nfs/**'
-      - 'weed/command/nfs.go'
-      - 'weed/filer/filer_inode.go'
-      - 'weed/filer/filer_inode_index.go'
-      - 'weed/filer/filerstore_wrapper.go'
-      - 'weed/server/filer_grpc_server_rename.go'
-      - 'test/nfs/**'
-      - '.github/workflows/nfs-tests.yml'
-
-concurrency:
-  group: ${{ github.head_ref }}/nfs-tests
-  cancel-in-progress: true
-
-permissions:
-  contents: read
-
-env:
-  TEST_TIMEOUT: '15m'
-
-jobs:
-  nfs-integration:
-    name: NFS Integration Testing
-    runs-on: ubuntu-22.04
-    timeout-minutes: 20
-
-    steps:
-    - name: Checkout code
-      uses: actions/checkout@v6
-
-    - name: Set up Go
-      uses: actions/setup-go@v6
-      with:
-        go-version-file: 'go.mod'
-
-    - name: Build SeaweedFS
-      run: |
-        cd weed
-        go build -o weed .
-        chmod +x weed
-        ./weed version
-
-    - name: Run NFS Integration Tests
-      run: |
-        cd test/nfs
-
-        echo "Running NFS integration tests..."
-        echo "============================================"
-
-        # Install test dependencies
-        go mod download
-
-        # Run the protocol-layer tests. The kernel-mount tests require root
-        # for mount(2) and are exercised in their own privileged step below;
-        # skip them here so a "skipped because not root" line doesn't show
-        # up as noise on every CI run.
-        go test -v -timeout=${{ env.TEST_TIMEOUT }} -skip '^TestKernelMount' ./...
-
-        echo "============================================"
-        echo "NFS integration tests completed"
-
-    - name: Install kernel NFS client
-      run: |
-        # nfs-common provides mount.nfs; netbase provides /etc/protocols
-        # which mount.nfs's protocol-name lookups (`tcp`, `udp`) need.
-        sudo apt-get update
-        sudo apt-get install -y nfs-common netbase
-
-    - name: Run kernel-mount E2E tests
-      run: |
-        cd test/nfs
-
-        echo "Running kernel-mount end-to-end tests..."
-        echo "These mount the running 'weed nfs' subprocess via the actual"
-        echo "Linux NFS client to catch protocol regressions invisible to"
-        echo "the go-nfs-client-based tests above."
-        echo "============================================"
-
-        # mount(2) is privileged. Preserve PATH so 'go' (and the weed
-        # binary that test/nfs/framework.go locates via $PATH) resolve
-        # correctly under sudo, and pass through the Go module/cache dirs
-        # so we don't redownload modules under root.
-        sudo env "PATH=$PATH" \
-          GOMODCACHE="$(go env GOMODCACHE)" \
-          GOCACHE="$(go env GOCACHE)" \
-          go test -v -timeout=${{ env.TEST_TIMEOUT }} -run '^TestKernelMount' ./...
-
-        echo "============================================"
-        echo "Kernel-mount E2E tests completed"
-
-    - name: Test Summary
-      if: always()
-      run: |
-        echo "## NFS Integration Test Summary" >> $GITHUB_STEP_SUMMARY
-        echo "" >> $GITHUB_STEP_SUMMARY
-        echo "### Test Coverage" >> $GITHUB_STEP_SUMMARY
-        echo "- **Read/Write Round Trip**: Basic file create + read" >> $GITHUB_STEP_SUMMARY
-        echo "- **Directory Operations**: Mkdir, ReadDirPlus, RmDir" >> $GITHUB_STEP_SUMMARY
-        echo "- **Nested Directories**: Deep tree creation and leaf I/O" >> $GITHUB_STEP_SUMMARY
-        echo "- **Rename**: Content preserved across rename" >> $GITHUB_STEP_SUMMARY
-        echo "- **Overwrite + Truncate**: Setattr(size=0) + shorter write" >> $GITHUB_STEP_SUMMARY
-        echo "- **Large Files**: 3 MiB binary round trip" >> $GITHUB_STEP_SUMMARY
-        echo "- **Edge Payloads**: All 256 byte values + empty files" >> $GITHUB_STEP_SUMMARY
-        echo "- **Symlinks**: Symlink + Lookup" >> $GITHUB_STEP_SUMMARY
-        echo "- **Missing Path**: Remove on missing entry errors cleanly" >> $GITHUB_STEP_SUMMARY
-        echo "- **FSINFO**: Non-zero rtpref/wtpref advertised" >> $GITHUB_STEP_SUMMARY
-        echo "- **Sequential Append**: Two-part concatenation" >> $GITHUB_STEP_SUMMARY
-        echo "- **ReadDir After Remove**: Meta cache does not serve stale entries" >> $GITHUB_STEP_SUMMARY
-        echo "" >> $GITHUB_STEP_SUMMARY
-        echo "### Kernel-Mount E2E Coverage" >> $GITHUB_STEP_SUMMARY
-        echo "- **V3 over TCP**: baseline NFSv3 mount + readdir" >> $GITHUB_STEP_SUMMARY
-        echo "- **V3 with mountproto=udp**: regression test for UDP MOUNT v3 responder" >> $GITHUB_STEP_SUMMARY
-        echo "- **V4 rejects cleanly**: regression test for the v4 PROG_MISMATCH path (#9262)" >> $GITHUB_STEP_SUMMARY
-        echo "" >> $GITHUB_STEP_SUMMARY
-        echo "### Harness" >> $GITHUB_STEP_SUMMARY
-        echo "Most tests boot their own master + volume + filer + nfs subprocess" >> $GITHUB_STEP_SUMMARY
-        echo "stack on loopback and drive it via the NFSv3 RPC protocol using" >> $GITHUB_STEP_SUMMARY
-        echo "go-nfs-client. The kernel-mount E2E tests reuse the same harness" >> $GITHUB_STEP_SUMMARY
-        echo "but mount the export through the in-tree Linux NFS client to" >> $GITHUB_STEP_SUMMARY
-        echo "catch protocol regressions a Go-only client can't see; they run" >> $GITHUB_STEP_SUMMARY
-        echo "in a separate privileged step (mount(2) requires root)." >> $GITHUB_STEP_SUMMARY

.github/workflows/postgres-tests.yml

@@ -31,6 +31,11 @@ jobs:
     - name: Check out code
       uses: actions/checkout@v6
 
+    - name: Configure Docker Hub mirror
+      run: |
+        echo '{"registry-mirrors": ["https://mirror.gcr.io"]}' | sudo tee /etc/docker/daemon.json
+        sudo systemctl restart docker
+
     - name: Set up Docker Buildx
       uses: docker/setup-buildx-action@v4
 

.github/workflows/rust_binaries_release.yml

@@ -66,34 +66,42 @@ jobs:
           SEAWEEDFS_COMMIT: ${{ github.sha }}
         run: |
           cd seaweed-volume
-          cargo build --release --target ${{ matrix.target }}
+          cargo build --release --target ${{ matrix.target }} --target-dir target/large-disk
 
       - name: Build Rust volume server (normal)
         env:
           SEAWEEDFS_COMMIT: ${{ github.sha }}
         run: |
           cd seaweed-volume
-          cargo build --release --target ${{ matrix.target }} --no-default-features
+          cargo build --release --target ${{ matrix.target }} --no-default-features --target-dir target/normal
 
       - name: Package binaries
         run: |
           # Large disk (default, 5bytes feature)
-          cp seaweed-volume/target/${{ matrix.target }}/release/weed-volume weed-volume-large-disk
+          cp seaweed-volume/target/large-disk/${{ matrix.target }}/release/weed-volume weed-volume-large-disk
           tar czf weed-volume_large_disk_${{ matrix.asset_suffix }}.tar.gz weed-volume-large-disk
           rm weed-volume-large-disk
 
           # Normal volume size
-          cp seaweed-volume/target/${{ matrix.target }}/release/weed-volume weed-volume-normal
+          cp seaweed-volume/target/normal/${{ matrix.target }}/release/weed-volume weed-volume-normal
           tar czf weed-volume_${{ matrix.asset_suffix }}.tar.gz weed-volume-normal
           rm weed-volume-normal
 
+      - name: Generate md5 checksums
+        run: |
+          for f in weed-volume_large_disk_${{ matrix.asset_suffix }}.tar.gz weed-volume_${{ matrix.asset_suffix }}.tar.gz; do
+            md5sum "$f" > "$f.md5"
+          done
+
       - name: Upload release assets
         if: startsWith(github.ref, 'refs/tags/')
         uses: softprops/action-gh-release@v3
         with:
           files: |
             weed-volume_large_disk_${{ matrix.asset_suffix }}.tar.gz
+            weed-volume_large_disk_${{ matrix.asset_suffix }}.tar.gz.md5
             weed-volume_${{ matrix.asset_suffix }}.tar.gz
+            weed-volume_${{ matrix.asset_suffix }}.tar.gz.md5
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
 
@@ -104,7 +112,9 @@ jobs:
           name: rust-volume-${{ matrix.asset_suffix }}
           path: |
             weed-volume_large_disk_${{ matrix.asset_suffix }}.tar.gz
+            weed-volume_large_disk_${{ matrix.asset_suffix }}.tar.gz.md5
             weed-volume_${{ matrix.asset_suffix }}.tar.gz
+            weed-volume_${{ matrix.asset_suffix }}.tar.gz.md5
 
   build-rust-volume-darwin:
     permissions:
@@ -145,32 +155,40 @@ jobs:
           SEAWEEDFS_COMMIT: ${{ github.sha }}
         run: |
           cd seaweed-volume
-          cargo build --release --target ${{ matrix.target }}
+          cargo build --release --target ${{ matrix.target }} --target-dir target/large-disk
 
       - name: Build Rust volume server (normal)
         env:
           SEAWEEDFS_COMMIT: ${{ github.sha }}
         run: |
           cd seaweed-volume
-          cargo build --release --target ${{ matrix.target }} --no-default-features
+          cargo build --release --target ${{ matrix.target }} --no-default-features --target-dir target/normal
 
       - name: Package binaries
         run: |
-          cp seaweed-volume/target/${{ matrix.target }}/release/weed-volume weed-volume-large-disk
+          cp seaweed-volume/target/large-disk/${{ matrix.target }}/release/weed-volume weed-volume-large-disk
           tar czf weed-volume_large_disk_${{ matrix.asset_suffix }}.tar.gz weed-volume-large-disk
           rm weed-volume-large-disk
 
-          cp seaweed-volume/target/${{ matrix.target }}/release/weed-volume weed-volume-normal
+          cp seaweed-volume/target/normal/${{ matrix.target }}/release/weed-volume weed-volume-normal
           tar czf weed-volume_${{ matrix.asset_suffix }}.tar.gz weed-volume-normal
           rm weed-volume-normal
 
+      - name: Generate md5 checksums
+        run: |
+          for f in weed-volume_large_disk_${{ matrix.asset_suffix }}.tar.gz weed-volume_${{ matrix.asset_suffix }}.tar.gz; do
+            md5 -r "$f" > "$f.md5"
+          done
+
       - name: Upload release assets
         if: startsWith(github.ref, 'refs/tags/')
         uses: softprops/action-gh-release@v3
         with:
           files: |
             weed-volume_large_disk_${{ matrix.asset_suffix }}.tar.gz
+            weed-volume_large_disk_${{ matrix.asset_suffix }}.tar.gz.md5
             weed-volume_${{ matrix.asset_suffix }}.tar.gz
+            weed-volume_${{ matrix.asset_suffix }}.tar.gz.md5
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
 
@@ -181,7 +199,9 @@ jobs:
           name: rust-volume-${{ matrix.asset_suffix }}
           path: |
             weed-volume_large_disk_${{ matrix.asset_suffix }}.tar.gz
+            weed-volume_large_disk_${{ matrix.asset_suffix }}.tar.gz.md5
             weed-volume_${{ matrix.asset_suffix }}.tar.gz
+            weed-volume_${{ matrix.asset_suffix }}.tar.gz.md5
 
   build-rust-volume-windows:
     permissions:
@@ -213,33 +233,42 @@ jobs:
           SEAWEEDFS_COMMIT: ${{ github.sha }}
         run: |
           cd seaweed-volume
-          cargo build --release
+          cargo build --release --target-dir target/large-disk
 
       - name: Build Rust volume server (normal)
         env:
           SEAWEEDFS_COMMIT: ${{ github.sha }}
         run: |
           cd seaweed-volume
-          cargo build --release --no-default-features
+          cargo build --release --no-default-features --target-dir target/normal
 
       - name: Package binaries
         shell: bash
         run: |
-          cp seaweed-volume/target/release/weed-volume.exe weed-volume-large-disk.exe
+          cp seaweed-volume/target/large-disk/release/weed-volume.exe weed-volume-large-disk.exe
           7z a weed-volume_large_disk_windows_amd64.zip weed-volume-large-disk.exe
           rm weed-volume-large-disk.exe
 
-          cp seaweed-volume/target/release/weed-volume.exe weed-volume-normal.exe
+          cp seaweed-volume/target/normal/release/weed-volume.exe weed-volume-normal.exe
           7z a weed-volume_windows_amd64.zip weed-volume-normal.exe
           rm weed-volume-normal.exe
 
+      - name: Generate md5 checksums
+        shell: bash
+        run: |
+          for f in weed-volume_large_disk_windows_amd64.zip weed-volume_windows_amd64.zip; do
+            md5sum "$f" > "$f.md5"
+          done
+
       - name: Upload release assets
         if: startsWith(github.ref, 'refs/tags/')
         uses: softprops/action-gh-release@v3
         with:
           files: |
             weed-volume_large_disk_windows_amd64.zip
+            weed-volume_large_disk_windows_amd64.zip.md5
             weed-volume_windows_amd64.zip
+            weed-volume_windows_amd64.zip.md5
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
 
@@ -250,4 +279,6 @@ jobs:
           name: rust-volume-windows_amd64
           path: |
             weed-volume_large_disk_windows_amd64.zip
+            weed-volume_large_disk_windows_amd64.zip.md5
             weed-volume_windows_amd64.zip
+            weed-volume_windows_amd64.zip.md5

.github/workflows/s3-proxy-signature-tests.yml

@@ -28,6 +28,11 @@ jobs:
           go-version-file: 'go.mod'
         id: go
 
+      - name: Configure Docker Hub mirror
+        run: |
+          echo '{"registry-mirrors": ["https://mirror.gcr.io"]}' | sudo tee /etc/docker/daemon.json
+          sudo systemctl restart docker
+
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v4
 

.github/workflows/s3-spark-tests.yml

@@ -33,15 +33,19 @@ jobs:
           go-version-file: 'go.mod'
         id: go
 
-      - name: Set up Docker
-        uses: docker/setup-buildx-action@v4
+      - name: Configure Docker Hub mirror
+        run: |
+          echo '{"registry-mirrors": ["https://mirror.gcr.io"]}' | sudo tee /etc/docker/daemon.json
+          sudo systemctl restart docker
 
       - name: Install SeaweedFS
         run: |
           go install -buildvcs=false ./weed
 
       - name: Pre-pull Spark image
-        run: docker pull apache/spark:3.5.8
+        run: |
+          pull() { for i in 1 2 3; do docker pull "$1" && return 0; sleep 15; done; return 1; }
+          pull apache/spark:3.5.1
 
       - name: Run S3 Spark integration tests
         working-directory: test/s3/spark

.github/workflows/s3-tables-tests.yml

@@ -83,6 +83,11 @@ jobs:
           go-version-file: 'go.mod'
         id: go
 
+      - name: Configure Docker Hub mirror
+        run: |
+          echo '{"registry-mirrors": ["https://mirror.gcr.io"]}' | sudo tee /etc/docker/daemon.json
+          sudo systemctl restart docker
+
       - name: Run go mod tidy
         run: go mod tidy
 
@@ -139,11 +144,15 @@ jobs:
           go-version-file: 'go.mod'
         id: go
 
-      - name: Set up Docker
-        uses: docker/setup-buildx-action@v4
+      - name: Configure Docker Hub mirror
+        run: |
+          echo '{"registry-mirrors": ["https://mirror.gcr.io"]}' | sudo tee /etc/docker/daemon.json
+          sudo systemctl restart docker
 
       - name: Pre-pull Trino image
-        run: docker pull trinodb/trino:479
+        run: |
+          pull() { for i in 1 2 3; do docker pull "$1" && return 0; sleep 15; done; return 1; }
+          pull trinodb/trino:479
 
       - name: Run go mod tidy
         run: go mod tidy
@@ -205,14 +214,16 @@ jobs:
           go-version-file: 'go.mod'
         id: go
 
-      - name: Set up Docker
-        uses: docker/setup-buildx-action@v4
+      - name: Configure Docker Hub mirror
+        run: |
+          echo '{"registry-mirrors": ["https://mirror.gcr.io"]}' | sudo tee /etc/docker/daemon.json
+          sudo systemctl restart docker
 
-      - name: Pre-pull Dremio image
-        run: docker pull dremio/dremio-oss:25.2.0
-
-      - name: Pre-pull Python image for PyIceberg writer
-        run: docker pull python:3.11-slim
+      - name: Pre-pull images
+        run: |
+          pull() { for i in 1 2 3; do docker pull "$1" && return 0; sleep 15; done; return 1; }
+          pull dremio/dremio-oss:25.2.0
+          pull python:3.11-slim
 
       - name: Run go mod tidy
         run: go mod tidy
@@ -276,14 +287,16 @@ jobs:
           go-version-file: 'go.mod'
         id: go
 
-      - name: Set up Docker
-        uses: docker/setup-buildx-action@v4
+      - name: Configure Docker Hub mirror
+        run: |
+          echo '{"registry-mirrors": ["https://mirror.gcr.io"]}' | sudo tee /etc/docker/daemon.json
+          sudo systemctl restart docker
 
-      - name: Pre-pull Doris image
-        run: docker pull apache/doris:doris-all-in-one-2.1.0
-
-      - name: Pre-pull Python image for PyIceberg writer
-        run: docker pull python:3.11-slim
+      - name: Pre-pull images
+        run: |
+          pull() { for i in 1 2 3; do docker pull "$1" && return 0; sleep 15; done; return 1; }
+          pull apache/doris:doris-all-in-one-2.1.0
+          pull python:3.11-slim
 
       - name: Run go mod tidy
         run: go mod tidy
@@ -354,8 +367,15 @@ jobs:
         run: |
           go install -buildvcs=false ./weed
 
+      - name: Configure Docker Hub mirror
+        run: |
+          echo '{"registry-mirrors": ["https://mirror.gcr.io"]}' | sudo tee /etc/docker/daemon.json
+          sudo systemctl restart docker
+
       - name: Pre-pull Polaris image
-        run: docker pull apache/polaris:latest
+        run: |
+          pull() { for i in 1 2 3; do docker pull "$1" && return 0; sleep 15; done; return 1; }
+          pull apache/polaris:latest
 
       - name: Run Polaris Integration Tests
         timeout-minutes: 25
@@ -408,11 +428,15 @@ jobs:
           go-version-file: 'go.mod'
         id: go
 
-      - name: Set up Docker
-        uses: docker/setup-buildx-action@v4
+      - name: Configure Docker Hub mirror
+        run: |
+          echo '{"registry-mirrors": ["https://mirror.gcr.io"]}' | sudo tee /etc/docker/daemon.json
+          sudo systemctl restart docker
 
       - name: Pre-pull Spark image
-        run: docker pull apache/spark:3.5.1
+        run: |
+          pull() { for i in 1 2 3; do docker pull "$1" && return 0; sleep 15; done; return 1; }
+          pull apache/spark:3.5.1
 
       - name: Run go mod tidy
         run: go mod tidy
@@ -474,13 +498,16 @@ jobs:
           go-version-file: 'go.mod'
         id: go
 
-      - name: Set up Docker
-        uses: docker/setup-buildx-action@v4
+      - name: Configure Docker Hub mirror
+        run: |
+          echo '{"registry-mirrors": ["https://mirror.gcr.io"]}' | sudo tee /etc/docker/daemon.json
+          sudo systemctl restart docker
 
       - name: Pre-pull RisingWave image
         run: |
-          docker pull risingwavelabs/risingwave:v2.5.0
-          docker pull postgres:16-alpine
+          pull() { for i in 1 2 3; do docker pull "$1" && return 0; sleep 15; done; return 1; }
+          pull risingwavelabs/risingwave:v2.5.0
+          pull postgres:16-alpine
 
       - name: Run go mod tidy
         run: go mod tidy
@@ -542,11 +569,39 @@ jobs:
           go-version-file: 'go.mod'
         id: go
 
-      - name: Set up Docker
-        uses: docker/setup-buildx-action@v4
+      - name: Configure Docker Hub mirror
+        run: |
+          echo '{"registry-mirrors": ["https://mirror.gcr.io"]}' | sudo tee /etc/docker/daemon.json
+          sudo systemctl restart docker
 
-      - name: Pre-pull Python image
-        run: docker pull python:3
+      - name: Week stamp for image cache key
+        id: week
+        run: echo "week=$(date -u +%G-%V)" >> "$GITHUB_OUTPUT"
+
+      - name: Restore python:3 image cache
+        id: python-image
+        uses: actions/cache@v5
+        with:
+          path: /tmp/python3-image.tar
+          key: python3-image-${{ steps.week.outputs.week }}
+          restore-keys: |
+            python3-image-
+
+      - name: Load or pull python:3
+        run: |
+          if [ "${{ steps.python-image.outputs.cache-hit }}" = "true" ]; then
+            docker load -i /tmp/python3-image.tar
+            exit 0
+          fi
+          pull() { for i in 1 2 3; do docker pull "$1" && return 0; sleep 15; done; return 1; }
+          if pull python:3; then
+            docker save -o /tmp/python3-image.tar python:3
+          elif [ -f /tmp/python3-image.tar ]; then
+            # Docker Hub unreachable; fall back to last week's cached image
+            docker load -i /tmp/python3-image.tar
+          else
+            exit 1
+          fi
 
       - name: Run go mod tidy
         run: go mod tidy
@@ -608,15 +663,6 @@ jobs:
           go-version-file: 'go.mod'
         id: go
 
-      - name: Set up Docker
-        uses: docker/setup-buildx-action@v4
-
-      - name: Pre-pull Python image
-        run: docker pull python:3
-        
-      - name: Pre-pull LocalStack image (if needed)
-        run: docker pull localstack/localstack:latest || true
-
       - name: Run go mod tidy
         run: go mod tidy
 
@@ -677,14 +723,16 @@ jobs:
           go-version-file: 'go.mod'
         id: go
 
-      - name: Set up Docker
-        uses: docker/setup-buildx-action@v4
+      - name: Configure Docker Hub mirror
+        run: |
+          echo '{"registry-mirrors": ["https://mirror.gcr.io"]}' | sudo tee /etc/docker/daemon.json
+          sudo systemctl restart docker
 
-      - name: Pre-pull Unity Catalog image
-        run: docker pull unitycatalog/unitycatalog:v0.4.0
-
-      - name: Pre-pull Python image for delta-rs writer
-        run: docker pull python:3.11-slim
+      - name: Pre-pull images
+        run: |
+          pull() { for i in 1 2 3; do docker pull "$1" && return 0; sleep 15; done; return 1; }
+          pull unitycatalog/unitycatalog:v0.4.0
+          pull python:3.11-slim
 
       - name: Run go mod tidy
         run: go mod tidy

.github/workflows/s3tests.yml

@@ -45,7 +45,7 @@ jobs:
 
       - name: Fix S3 tests bucket creation conflicts
         run: |
-          python3 test/s3/fix_s3_tests_bucket_conflicts.py
+          python3 test/s3/compatibility/fix_s3_tests_bucket_conflicts.py
         env:
           S3_TESTS_PATH: s3-tests
 
@@ -388,7 +388,7 @@ jobs:
 
       - name: Fix S3 tests bucket creation conflicts
         run: |
-          python3 test/s3/fix_s3_tests_bucket_conflicts.py
+          python3 test/s3/compatibility/fix_s3_tests_bucket_conflicts.py
         env:
           S3_TESTS_PATH: s3-tests
 

.github/workflows/terraform_ci.yml

@@ -0,0 +1,80 @@
+name: "terraform: validate and test modules"
+
+on:
+  push:
+    branches: [ master ]
+    paths: ['terraform/**', '.github/workflows/terraform_ci.yml']
+  pull_request:
+    branches: [ master ]
+    paths: ['terraform/**', '.github/workflows/terraform_ci.yml']
+
+permissions:
+  contents: read
+
+jobs:
+  validate:
+    name: fmt, validate, plan-level tests
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Set up OpenTofu
+        uses: opentofu/setup-opentofu@v2
+        with:
+          tofu_version: 1.12.1
+
+      - name: fmt check
+        working-directory: terraform
+        run: tofu fmt -recursive -check -diff
+
+      - name: validate core
+        working-directory: terraform/modules/core
+        run: |
+          tofu init -backend=false -input=false
+          tofu validate
+
+      - name: validate security
+        working-directory: terraform/modules/security
+        run: |
+          tofu init -backend=false -input=false
+          tofu validate
+
+      - name: plan-level tests (core)
+        working-directory: terraform/modules/core
+        run: tofu test
+
+      - name: validate examples
+        run: |
+          set -e
+          for ex in terraform/examples/*/; do
+            echo "== validate $ex =="
+            tofu -chdir="$ex" init -backend=false -input=false
+            tofu -chdir="$ex" validate
+          done
+
+  smoke:
+    name: local cluster smoke test (real weed)
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Set up Go
+        uses: actions/setup-go@v6
+        with:
+          go-version-file: go.mod
+
+      - name: Build weed
+        run: go build -o "$RUNNER_TEMP/weed" ./weed
+
+      - name: Set up OpenTofu
+        uses: opentofu/setup-opentofu@v2
+        with:
+          tofu_version: 1.12.1
+
+      - name: Run local cluster harness
+        working-directory: terraform/test/local
+        run: WEED="$RUNNER_TEMP/weed" ./run_local_cluster.sh
+
+      - name: Run local mTLS cluster harness
+        working-directory: terraform/test/local-secure
+        run: WEED="$RUNNER_TEMP/weed" ./run_local_secure.sh

README.md

@@ -35,6 +35,7 @@ Your support will be really appreciated by me and other supporters!
 [![nodion](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/sponsor_nodion.png)](https://www.nodion.com)
 [![piknik](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/piknik.png)](https://www.piknik.com)
 [![keepsec](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/keepsec.png)](https://www.keepsec.ca)
+[![zyner](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/sponsor_zyner.png)](https://zyner.org)
 
 ---
 
@@ -654,7 +655,8 @@ Cluster Total: 3302.88 MiB/s, 550.51 obj/s over 43s.
 ## Enterprise ##
 
 For enterprise users, please visit [seaweedfs.com](https://seaweedfs.com) for the SeaweedFS Enterprise Edition, 
-which has a self-healing storage format with better data protection.
+which has advanced features, including data recovery, self-healing storage, 
+customizable erasure coding, EC vacuum and repair, etc.
 
 [Back to TOC](#table-of-contents)
 

docker/Dockerfile.go_build

@@ -1,4 +1,6 @@
-FROM golang:1.25-alpine AS builder
+# Pin the builder to the host arch and cross-compile the (CGO-free) Go binary,
+# so arm64/arm/386 targets skip QEMU emulation of the whole compile.
+FROM --platform=$BUILDPLATFORM golang:1.25-alpine AS builder
 RUN apk add git g++ fuse
 RUN mkdir -p /go/src/github.com/seaweedfs/
 ARG BRANCH=${BRANCH:-master}
@@ -12,9 +14,12 @@ RUN cd /go/src/github.com/seaweedfs/seaweedfs && \
       git checkout $BRANCH) || \
      (echo "ERROR: Branch/commit $BRANCH not found in repository" && \
       echo "Available branches:" && git branch -a && exit 1))
+ARG TARGETOS TARGETARCH TARGETVARIANT
 RUN cd /go/src/github.com/seaweedfs/seaweedfs/weed \
   && export LDFLAGS="-X github.com/seaweedfs/seaweedfs/weed/util/version.COMMIT=$(git rev-parse --short HEAD)" \
-  && CGO_ENABLED=0 go install -tags "$TAGS" -ldflags "-extldflags -static ${LDFLAGS}"
+  && export GOOS=$TARGETOS GOARCH=$TARGETARCH \
+  && case "$TARGETARCH" in arm) export GOARM="${TARGETVARIANT#v}";; esac \
+  && CGO_ENABLED=0 go build -tags "$TAGS" -ldflags "-extldflags -static ${LDFLAGS}" -o /go/bin/weed .
 
 # Rust volume server: use pre-built binary from CI when available (placed in
 # weed-volume-prebuilt/ by the build-rust-binaries job), otherwise compile

docker/Makefile

@@ -127,6 +127,9 @@ test_tarantool: tags = tarantool
 test_tarantool: build_tarantool_dev_env build
 	docker compose -f compose/test-tarantool-filer.yml -p seaweedfs up
 
+test_keycloak_s3: build
+	docker compose -f compose/test-keycloak-s3.yml -p seaweedfs up
+
 clean:
 	rm ./weed
 

docker/compose/test-keycloak-s3.yml

@@ -0,0 +1,46 @@
+services:
+  keycloak:
+    image: quay.io/keycloak/keycloak:26.0.7
+    command: ["start-dev", "--import-realm"]
+    environment:
+      KC_BOOTSTRAP_ADMIN_USERNAME: admin
+      KC_BOOTSTRAP_ADMIN_PASSWORD: admin
+      KC_HTTP_ENABLED: "true"
+      KC_HOSTNAME: "http://keycloak:8080"
+      KC_HOSTNAME_STRICT: "false"
+    ports:
+      - "8080:8080"
+    volumes:
+      - ../../test/s3/iam/seaweedfs-test-realm.json:/opt/keycloak/data/import/seaweedfs-test-realm.json:ro
+    healthcheck:
+      test: ["CMD-SHELL", "exec 3<>/dev/tcp/localhost/8080"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+      start_period: 60s
+
+  s3:
+    image: chrislusf/seaweedfs:local
+    command:
+      - -v=9
+      - server
+      - -ip=s3
+      - -filer
+      - -master.volumeSizeLimitMB=16
+      - -volume
+      - -volume.max=0
+      - -volume.preStopSeconds=1
+      - -s3
+      - -s3.port=8333
+      - -s3.config=/etc/seaweedfs/s3_config.json
+      - -s3.iam.config=/etc/seaweedfs/iam_config.json
+    environment:
+      WEED_MASTER_VOLUME_GROWTH_COPY_OTHER: 1
+    volumes:
+      - ../../test/s3/iam/test_config.json:/etc/seaweedfs/s3_config.json:ro
+      - ../../test/s3/iam/iam_config_docker.json:/etc/seaweedfs/iam_config.json:ro
+    ports:
+      - "8333:8333"
+    depends_on:
+      keycloak:
+        condition: service_healthy

go.mod

@@ -88,9 +88,9 @@ require (
 	go.etcd.io/etcd/client/v3 v3.6.10
 	go.mongodb.org/mongo-driver v1.17.9
 	go.opencensus.io v0.24.0 // indirect
-	gocloud.dev v0.45.0
+	gocloud.dev v0.46.0
 	gocloud.dev/pubsub/natspubsub v0.45.0
-	gocloud.dev/pubsub/rabbitpubsub v0.45.0
+	gocloud.dev/pubsub/rabbitpubsub v0.46.0
 	golang.org/x/crypto v0.52.0
 	golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f
 	golang.org/x/image v0.39.0
@@ -102,7 +102,7 @@ require (
 	golang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da // indirect
 	google.golang.org/api v0.278.0
 	google.golang.org/genproto v0.0.0-20260319201613-d00831a3d3e7 // indirect
-	google.golang.org/grpc v1.80.0
+	google.golang.org/grpc v1.81.1
 	google.golang.org/protobuf v1.36.11
 	gopkg.in/inf.v0 v0.9.1 // indirect
 	modernc.org/b v1.0.0 // indirect
@@ -117,19 +117,18 @@ require (
 	github.com/Jille/raft-grpc-transport v1.6.1
 	github.com/ThreeDotsLabs/watermill v1.5.1
 	github.com/a-h/templ v0.3.1001
-	github.com/apache/cassandra-gocql-driver/v2 v2.1.0
+	github.com/apache/cassandra-gocql-driver/v2 v2.1.1
 	github.com/apache/iceberg-go v0.5.0
 	github.com/apple/foundationdb/bindings/go v0.0.0-20250911184653-27f7192f47c3
 	github.com/arangodb/go-driver v1.6.9
 	github.com/armon/go-metrics v0.4.1
-	github.com/aws/aws-sdk-go-v2 v1.41.7
-	github.com/aws/aws-sdk-go-v2/config v1.32.14
-	github.com/aws/aws-sdk-go-v2/credentials v1.19.14
-	github.com/aws/aws-sdk-go-v2/service/s3 v1.101.0
+	github.com/aws/aws-sdk-go-v2 v1.41.12
+	github.com/aws/aws-sdk-go-v2/config v1.32.21
+	github.com/aws/aws-sdk-go-v2/credentials v1.19.20
+	github.com/aws/aws-sdk-go-v2/service/s3 v1.102.2
 	github.com/cognusion/imaging v1.0.3
 	github.com/fluent/fluent-logger-golang v1.10.1
 	github.com/getsentry/sentry-go v0.44.1
-	github.com/go-git/go-billy/v5 v5.9.0
 	github.com/go-ldap/ldap/v3 v3.4.13
 	github.com/golang-jwt/jwt/v5 v5.3.1
 	github.com/google/flatbuffers/go v0.0.0-20230108230133-3b8644d32c50
@@ -145,21 +144,19 @@ require (
 	github.com/rabbitmq/amqp091-go v1.11.0
 	github.com/rclone/rclone v1.74.1
 	github.com/rdleal/intervalst v1.5.0
-	github.com/redis/go-redis/v9 v9.19.0
+	github.com/redis/go-redis/v9 v9.20.0
 	github.com/schollz/progressbar/v3 v3.19.0
 	github.com/seaweedfs/go-fuse/v2 v2.9.3
 	github.com/shirou/gopsutil/v4 v4.26.3
 	github.com/tarantool/go-tarantool/v2 v2.4.2
 	github.com/testcontainers/testcontainers-go v0.40.0
 	github.com/tikv/client-go/v2 v2.0.7
-	github.com/willscott/go-nfs v0.0.4
-	github.com/willscott/go-nfs-client v0.0.0-20251022144359-801f10d98886
 	github.com/xeipuuv/gojsonschema v1.2.0
 	github.com/ydb-platform/ydb-go-sdk-auth-environ v0.5.1
-	github.com/ydb-platform/ydb-go-sdk/v3 v3.134.2
-	go.etcd.io/etcd/client/pkg/v3 v3.6.11
+	github.com/ydb-platform/ydb-go-sdk/v3 v3.139.5
+	go.etcd.io/etcd/client/pkg/v3 v3.6.12
 	go.uber.org/atomic v1.11.0
-	golang.org/x/sync v0.20.0
+	golang.org/x/sync v0.21.0
 	golang.org/x/tools/godoc v0.1.0-deprecated
 	google.golang.org/grpc/security/advancedtls v1.0.0
 )
@@ -182,7 +179,7 @@ require (
 	github.com/antlr4-go/antlr/v4 v4.13.1 // indirect
 	github.com/apache/arrow-go/v18 v18.5.2-0.20260220015023-a886a5722b87 // indirect
 	github.com/apache/thrift v0.23.0 // indirect
-	github.com/aws/aws-sdk-go-v2/service/signin v1.0.9 // indirect
+	github.com/aws/aws-sdk-go-v2/service/signin v1.1.2 // indirect
 	github.com/bahlo/generic-list-go v0.2.0 // indirect
 	github.com/bazelbuild/rules_go v0.46.0 // indirect
 	github.com/biogo/store v0.0.0-20201120204734-aad293a2328f // indirect
@@ -211,6 +208,7 @@ require (
 	github.com/dromara/dongle v1.0.1 // indirect
 	github.com/gin-gonic/gin v1.11.0 // indirect
 	github.com/go-asn1-ber/asn1-ber v1.5.8-0.20250403174932-29230038a667 // indirect
+	github.com/go-git/go-billy/v5 v5.9.0 // indirect
 	github.com/goccy/go-yaml v1.18.0 // indirect
 	github.com/golang/geo v0.0.0-20210211234256-740aa86cb551 // indirect
 	github.com/google/go-cmp v0.7.0 // indirect
@@ -233,7 +231,7 @@ require (
 	github.com/klauspost/asmfmt v1.3.2 // indirect
 	github.com/kr/pretty v0.3.1 // indirect
 	github.com/kr/text v0.2.0 // indirect
-	github.com/lib/pq v1.11.1 // indirect
+	github.com/lib/pq v1.12.0 // indirect
 	github.com/lithammer/fuzzysearch v1.1.8 // indirect
 	github.com/lithammer/shortuuid/v3 v3.0.7 // indirect
 	github.com/magiconair/properties v1.8.10 // indirect
@@ -257,7 +255,6 @@ require (
 	github.com/pquerna/otp v1.5.0 // indirect
 	github.com/pterm/pterm v0.12.82 // indirect
 	github.com/quic-go/qpack v0.6.0 // indirect
-	github.com/rasky/go-xdr v0.0.0-20170124162913-1a41d1a06c93 // indirect
 	github.com/rclone/Proton-API-Bridge v1.0.3 // indirect
 	github.com/rclone/go-proton-api v1.0.2 // indirect
 	github.com/rogpeppe/go-internal v1.14.1 // indirect
@@ -277,10 +274,10 @@ require (
 	github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
 	github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
 	github.com/zeebo/xxh3 v1.1.0 // indirect
-	go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.38.0 // indirect
-	go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.38.0 // indirect
+	go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.42.0 // indirect
+	go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.42.0 // indirect
 	go.opentelemetry.io/otel/exporters/zipkin v1.36.0 // indirect
-	go.opentelemetry.io/proto/otlp v1.9.0 // indirect
+	go.opentelemetry.io/proto/otlp v1.10.0 // indirect
 	go.uber.org/mock v0.5.2 // indirect
 	go.yaml.in/yaml/v2 v2.4.3 // indirect
 	go.yaml.in/yaml/v3 v3.0.4 // indirect
@@ -323,23 +320,22 @@ require (
 	github.com/andybalholm/cascadia v1.3.3 // indirect
 	github.com/appscode/go-querystring v0.0.0-20170504095604-0126cfb3f1dc // indirect
 	github.com/arangodb/go-velocypack v0.0.0-20200318135517-5af53c29c67e // indirect
-	github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.10 // indirect
-	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.21 // indirect
+	github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.11 // indirect
+	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.26 // indirect
 	github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.22.13 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.23 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.23 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/ini v1.8.6 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.24 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.9 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.15 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.23 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.23 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sns v1.39.7 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sqs v1.42.17 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sso v1.30.15 // indirect
-	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.19 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sts v1.41.10
-	github.com/aws/smithy-go v1.25.1
+	github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.26 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.26 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.27 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.10 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.18 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.26 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.25 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sns v1.39.14 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sqs v1.42.24 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sso v1.31.0 // indirect
+	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.36.3 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sts v1.43.0
+	github.com/aws/smithy-go v1.27.1
 	github.com/boltdb/bolt v1.3.1 // indirect
 	github.com/bradenaw/juniper v0.15.3 // indirect
 	github.com/bradfitz/iter v0.0.0-20191230175014-e8f45d346db8 // indirect
@@ -350,7 +346,7 @@ require (
 	github.com/cloudinary/cloudinary-go/v2 v2.15.0 // indirect
 	github.com/cloudsoda/go-smb2 v0.0.0-20250228001242-d4c70e6251cc // indirect
 	github.com/cloudsoda/sddl v0.0.0-20250224235906-926454e91efc // indirect
-	github.com/cncf/xds/go v0.0.0-20251210132809-ee656c7534f5 // indirect
+	github.com/cncf/xds/go v0.0.0-20260202195803-dba9d589def2 // indirect
 	github.com/colinmarc/hdfs/v2 v2.4.0 // indirect
 	github.com/creasty/defaults v1.8.0 // indirect
 	github.com/cronokirby/saferith v0.33.1-0.20250226174546-1f11f94ce488 // indirect
@@ -362,8 +358,8 @@ require (
 	github.com/elastic/gosigar v0.14.3 // indirect
 	github.com/emersion/go-message v0.18.2 // indirect
 	github.com/emersion/go-vcard v0.0.0-20241024213814-c9703dde27ff // indirect
-	github.com/envoyproxy/go-control-plane/envoy v1.36.0 // indirect
-	github.com/envoyproxy/protoc-gen-validate v1.3.0 // indirect
+	github.com/envoyproxy/go-control-plane/envoy v1.37.0 // indirect
+	github.com/envoyproxy/protoc-gen-validate v1.3.3 // indirect
 	github.com/fatih/color v1.18.0 // indirect
 	github.com/felixge/httpsnoop v1.0.4 // indirect
 	github.com/flynn/noise v1.1.0 // indirect
@@ -392,7 +388,7 @@ require (
 	github.com/gorilla/securecookie v1.1.2 // indirect
 	github.com/gorilla/sessions v1.4.0
 	github.com/grpc-ecosystem/go-grpc-middleware v1.4.0 // indirect
-	github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.3 // indirect
+	github.com/grpc-ecosystem/grpc-gateway/v2 v2.28.0 // indirect
 	github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
 	github.com/hashicorp/go-hclog v1.6.3 // indirect
 	github.com/hashicorp/go-immutable-radix v1.3.1 // indirect
@@ -441,7 +437,7 @@ require (
 	github.com/pelletier/go-toml/v2 v2.2.4 // indirect
 	github.com/pengsrc/go-shared v0.2.1-0.20190131101655-1999055a4a14 // indirect
 	github.com/philhofer/fwd v1.2.0 // indirect
-	github.com/pierrec/lz4/v4 v4.1.26
+	github.com/pierrec/lz4/v4 v4.1.27
 	github.com/pingcap/errors v0.11.5-0.20211224045212-9687c2b0f87c // indirect
 	github.com/pingcap/failpoint v0.0.0-20220801062533-2eaa32854a6c // indirect
 	github.com/pingcap/kvproto v0.0.0-20230403051650-e166ae588106 // indirect
@@ -478,7 +474,7 @@ require (
 	github.com/vmihailenco/tagparser/v2 v2.0.0 // indirect
 	github.com/xanzy/ssh-agent v0.3.3 // indirect
 	github.com/yandex-cloud/go-genproto v0.0.0-20211115083454-9ca41db5ed9e // indirect
-	github.com/ydb-platform/ydb-go-genproto v0.0.0-20260311095541-ebbf792c1180 // indirect
+	github.com/ydb-platform/ydb-go-genproto v0.0.0-20260428144813-1c07baab7f7b // indirect
 	github.com/ydb-platform/ydb-go-yc v0.12.1 // indirect
 	github.com/ydb-platform/ydb-go-yc-metadata v0.6.1 // indirect
 	github.com/yunify/qingstor-sdk-go/v3 v3.2.0 // indirect
@@ -488,7 +484,7 @@ require (
 	go.etcd.io/bbolt v1.4.3 // indirect
 	go.etcd.io/etcd/api/v3 v3.6.10 // indirect
 	go.opentelemetry.io/auto/sdk v1.2.1 // indirect
-	go.opentelemetry.io/contrib/detectors/gcp v1.39.0 // indirect
+	go.opentelemetry.io/contrib/detectors/gcp v1.42.0 // indirect
 	go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.67.0 // indirect
 	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.67.0 // indirect
 	go.opentelemetry.io/otel v1.43.0 // indirect

go.sum

@@ -573,10 +573,6 @@ github.com/Azure/azure-sdk-for-go/sdk/storage/azfile v1.5.4 h1:tZh20RjgfMxKBxJiI
 github.com/Azure/azure-sdk-for-go/sdk/storage/azfile v1.5.4/go.mod h1:vGYAk36rhMVCfTP7v+RVruCR0zmPe6S+36KRpDCLySw=
 github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c h1:udKWzYgxTojEKWjV8V+WSxDXJ4NFATAsZjh8iIbsQIg=
 github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E=
-github.com/Azure/go-autorest v14.2.0+incompatible h1:V5VMDjClD3GiElqLWO7mz2MxNAK/vTfRHdAubSIPRgs=
-github.com/Azure/go-autorest v14.2.0+incompatible/go.mod h1:r+4oMnoxhatjLLJ6zxSWATqVooLgysK6ZNox3g/xq24=
-github.com/Azure/go-autorest/autorest/to v0.4.1 h1:CxNHBqdzTr7rLtdrtb5CMjJcDut+WNGCVv7OmS5+lTc=
-github.com/Azure/go-autorest/autorest/to v0.4.1/go.mod h1:EtaofgU4zmtvn1zT2ARsjRFdq9vXx0YWtmElwL+GZ9M=
 github.com/Azure/go-ntlmssp v0.1.1 h1:l+FM/EEMb0U9QZE7mKNEDw5Mu3mFiaa2GKOoTSsNDPw=
 github.com/Azure/go-ntlmssp v0.1.1/go.mod h1:NYqdhxd/8aAct/s4qSYZEerdPuH1liG2/X9DiVTbhpk=
 github.com/AzureAD/microsoft-authentication-extensions-for-go/cache v0.1.1 h1:WJTmL004Abzc5wDB5VtZG2PJk5ndYDgVacGqfirKxjM=
@@ -694,8 +690,8 @@ github.com/antlr4-go/antlr/v4 v4.13.1/go.mod h1:GKmUxMtwp6ZgGwZSva4eWPC5mS6vUAmO
 github.com/apache/arrow-go/v18 v18.5.2-0.20260220015023-a886a5722b87 h1:r/gg2gzUXiXoy72VU3jnODh8l/5rL0aslnTAbtmai/U=
 github.com/apache/arrow-go/v18 v18.5.2-0.20260220015023-a886a5722b87/go.mod h1:IJTMBTlHe7cDOhRh0ioGuEKBl5iTR6xPfl5BN4AgirU=
 github.com/apache/arrow/go/v10 v10.0.1/go.mod h1:YvhnlEePVnBS4+0z3fhPfUy7W1Ikj0Ih0vcRo/gZ1M0=
-github.com/apache/cassandra-gocql-driver/v2 v2.1.0 h1:VEbbeJ2ift4deKMZ6Fs55Vs3fq/RrkjCcxCnqUxhwf8=
-github.com/apache/cassandra-gocql-driver/v2 v2.1.0/go.mod h1:QH/asJjB3mHvY6Dot6ZKMMpTcOrWJ8i9GhsvG1g0PK4=
+github.com/apache/cassandra-gocql-driver/v2 v2.1.1 h1:DjmtmSRoUuIFswczA8LPBppRb8T9vxqNP5Txc/lQxhE=
+github.com/apache/cassandra-gocql-driver/v2 v2.1.1/go.mod h1:QH/asJjB3mHvY6Dot6ZKMMpTcOrWJ8i9GhsvG1g0PK4=
 github.com/apache/iceberg-go v0.5.0 h1:wQj4CK5YiXZcB+tj19gWG+Jf1I6MiORQ/StSL/E5gGQ=
 github.com/apache/iceberg-go v0.5.0/go.mod h1:F/rdP1yZmnO4mQ0Qew2HTGdc+ZV57cRfxbbq/uJm1eM=
 github.com/apache/thrift v0.22.0 h1:r7mTJdj51TMDe6RtcmNdQxgn9XcyfGDOzegMDRg47uc=
@@ -715,50 +711,50 @@ github.com/armon/go-metrics v0.4.1/go.mod h1:E6amYzXo6aW1tqzoZGT755KkbgrJsSdpwZ+
 github.com/atomicgo/cursor v0.0.1/go.mod h1:cBON2QmmrysudxNBFthvMtN32r3jxVRIvzkUiF/RuIk=
 github.com/aws/aws-sdk-go v1.55.8 h1:JRmEUbU52aJQZ2AjX4q4Wu7t4uZjOu71uyNmaWlUkJQ=
 github.com/aws/aws-sdk-go v1.55.8/go.mod h1:ZkViS9AqA6otK+JBBNH2++sx1sgxrPKcSzPPvQkUtXk=
-github.com/aws/aws-sdk-go-v2 v1.41.7 h1:DWpAJt66FmnnaRIOT/8ASTucrvuDPZASqhhLey6tLY8=
-github.com/aws/aws-sdk-go-v2 v1.41.7/go.mod h1:4LAfZOPHNVNQEckOACQx60Y8pSRjIkNZQz1w92xpMJc=
-github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.10 h1:gx1AwW1Iyk9Z9dD9F4akX5gnN3QZwUB20GGKH/I+Rho=
-github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.10/go.mod h1:qqY157uZoqm5OXq/amuaBJyC9hgBCBQnsaWnPe905GY=
-github.com/aws/aws-sdk-go-v2/config v1.32.14 h1:opVIRo/ZbbI8OIqSOKmpFaY7IwfFUOCCXBsUpJOwDdI=
-github.com/aws/aws-sdk-go-v2/config v1.32.14/go.mod h1:U4/V0uKxh0Tl5sxmCBZ3AecYny4UNlVmObYjKuuaiOo=
-github.com/aws/aws-sdk-go-v2/credentials v1.19.14 h1:n+UcGWAIZHkXzYt87uMFBv/l8THYELoX6gVcUvgl6fI=
-github.com/aws/aws-sdk-go-v2/credentials v1.19.14/go.mod h1:cJKuyWB59Mqi0jM3nFYQRmnHVQIcgoxjEMAbLkpr62w=
-github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.21 h1:NUS3K4BTDArQqNu2ih7yeDLaS3bmHD0YndtA6UP884g=
-github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.21/go.mod h1:YWNWJQNjKigKY1RHVJCuupeWDrrHjRqHm0N9rdrWzYI=
+github.com/aws/aws-sdk-go-v2 v1.41.12 h1:DIKX2c31ekm9RA2D9FBj1EWXx++9AdAqRw+e78Tq2Ck=
+github.com/aws/aws-sdk-go-v2 v1.41.12/go.mod h1:27+ACypSLljLAEKsCYOmrjKh83vuTRkuAe9Uv/3A4bg=
+github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.11 h1:h5+3VT69KUBK24grGuuA5saDJTj2IIjLb9au668Fo5I=
+github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.11/go.mod h1:dnakxebH6UwFvcvujL0LVggYQ8nEvBGjU4G/V79Nv94=
+github.com/aws/aws-sdk-go-v2/config v1.32.21 h1:1i1v5jWn6iNPl7zkH4VwIXQOb+OMqMRFNjdHQeZbFVo=
+github.com/aws/aws-sdk-go-v2/config v1.32.21/go.mod h1:7NXDJjk0e4QOMv3Y3ppwALx3ifdSQHzCACFtRX5J1hI=
+github.com/aws/aws-sdk-go-v2/credentials v1.19.20 h1:nIGz0+cQKwijxSsWNpggvPxjxFMtrAHNj16RYzGbu2Q=
+github.com/aws/aws-sdk-go-v2/credentials v1.19.20/go.mod h1:/UQHYia3DSeilGdchXJDEVzJws15XuZSdnTlwbTBg0Q=
+github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.26 h1:a1VPHz7eG7mVNHWHXUcCyC8rQDBEr4k+o4HJYbSoYZ0=
+github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.26/go.mod h1:9G8LPblMwaodeQdS+IfDBgPqABDctJLpyZ/M6jflE6s=
 github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.22.13 h1:uMC4oL6G3MNhodo358QEqSDjrgvzV3TUQ58nyQSGq2E=
 github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.22.13/go.mod h1:Cer86AE2686DvVUe57LPve3jUBmbujuaonSX8pNzGgw=
-github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.23 h1:GpT/TrnBYuE5gan2cZbTtvP+JlHsutdmlV2YfEyNde0=
-github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.23/go.mod h1:xYWD6BS9ywC5bS3sz9Xh04whO/hzK2plt2Zkyrp4JuA=
-github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.23 h1:bpd8vxhlQi2r1hiueOw02f/duEPTMK59Q4QMAoTTtTo=
-github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.23/go.mod h1:15DfR2nw+CRHIk0tqNyifu3G1YdAOy68RftkhMDDwYk=
-github.com/aws/aws-sdk-go-v2/internal/ini v1.8.6 h1:qYQ4pzQ2Oz6WpQ8T3HvGHnZydA72MnLuFK9tJwmrbHw=
-github.com/aws/aws-sdk-go-v2/internal/ini v1.8.6/go.mod h1:O3h0IK87yXci+kg6flUKzJnWeziQUKciKrLjcatSNcY=
-github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.24 h1:OQqn11BtaYv1WLUowvcA30MpzIu8Ti4pcLPIIyoKZrA=
-github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.24/go.mod h1:X5ZJyfwVrWA96GzPmUCWFQaEARPR7gCrpq2E92PJwAE=
-github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.9 h1:FLudkZLt5ci0ozzgkVo8BJGwvqNaZbTWb3UcucAateA=
-github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.9/go.mod h1:w7wZ/s9qK7c8g4al+UyoF1Sp/Z45UwMGcqIzLWVQHWk=
-github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.15 h1:ieLCO1JxUWuxTZ1cRd0GAaeX7O6cIxnwk7tc1LsQhC4=
-github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.15/go.mod h1:e3IzZvQ3kAWNykvE0Tr0RDZCMFInMvhku3qNpcIQXhM=
-github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.23 h1:pbrxO/kuIwgEsOPLkaHu0O+m4fNgLU8B3vxQ+72jTPw=
-github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.23/go.mod h1:/CMNUqoj46HpS3MNRDEDIwcgEnrtZlKRaHNaHxIFpNA=
-github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.23 h1:03xatSQO4+AM1lTAbnRg5OK528EUg744nW7F73U8DKw=
-github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.23/go.mod h1:M8l3mwgx5ToK7wot2sBBce/ojzgnPzZXUV445gTSyE8=
-github.com/aws/aws-sdk-go-v2/service/s3 v1.101.0 h1:etqBTKY581iwLL/H/S2sVgk3C9lAsTJFeXWFDsDcWOU=
-github.com/aws/aws-sdk-go-v2/service/s3 v1.101.0/go.mod h1:L2dcoOgS2VSgbPLvpak2NyUPsO1TBN7M45Z4H7DlRc4=
-github.com/aws/aws-sdk-go-v2/service/signin v1.0.9 h1:QKZH0S178gCmFEgst8hN0mCX1KxLgHBKKY/CLqwP8lg=
-github.com/aws/aws-sdk-go-v2/service/signin v1.0.9/go.mod h1:7yuQJoT+OoH8aqIxw9vwF+8KpvLZ8AWmvmUWHsGQZvI=
-github.com/aws/aws-sdk-go-v2/service/sns v1.39.7 h1:fovS7qGMT+BBSuifkySdVaMWxXTyaYT6qaBx/1y6Ij4=
-github.com/aws/aws-sdk-go-v2/service/sns v1.39.7/go.mod h1:gFahrattA8ulEtiS4XL/fQiQ77l+Urc52Y96/r1e6ks=
-github.com/aws/aws-sdk-go-v2/service/sqs v1.42.17 h1:ZNMxVFPayuHe14u/vn+BwLi3wxQvxcNTw8WdPv2gqBc=
-github.com/aws/aws-sdk-go-v2/service/sqs v1.42.17/go.mod h1:ZxqweFQ2w6NNznWMUvWV9AvkAfM6J8F/MC250Mb4n1I=
-github.com/aws/aws-sdk-go-v2/service/sso v1.30.15 h1:lFd1+ZSEYJZYvv9d6kXzhkZu07si3f+GQ1AaYwa2LUM=
-github.com/aws/aws-sdk-go-v2/service/sso v1.30.15/go.mod h1:WSvS1NLr7JaPunCXqpJnWk1Bjo7IxzZXrZi1QQCkuqM=
-github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.19 h1:dzztQ1YmfPrxdrOiuZRMF6fuOwWlWpD2StNLTceKpys=
-github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.19/go.mod h1:YO8TrYtFdl5w/4vmjL8zaBSsiNp3w0L1FfKVKenZT7w=
-github.com/aws/aws-sdk-go-v2/service/sts v1.41.10 h1:p8ogvvLugcR/zLBXTXrTkj0RYBUdErbMnAFFp12Lm/U=
-github.com/aws/aws-sdk-go-v2/service/sts v1.41.10/go.mod h1:60dv0eZJfeVXfbT1tFJinbHrDfSJ2GZl4Q//OSSNAVw=
-github.com/aws/smithy-go v1.25.1 h1:J8ERsGSU7d+aCmdQur5Txg6bVoYelvQJgtZehD12GkI=
-github.com/aws/smithy-go v1.25.1/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
+github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.2.3 h1:w5OoDiMN6x53ROmiIImGzmVcxXv2q1GXY+aKV4WAJYM=
+github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.2.3/go.mod h1:dAhgYp776bX3LuWvnSCFwQEjNs6fuFg7YXIy5PXcP3Q=
+github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.26 h1:dx1qc6V0exl5SK5/kwSc2IdOKCxLHoNbl/zXuYRwopw=
+github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.26/go.mod h1:iDMGb+drKqPTqRjv1kWc5meMYzT6HsvPXxJV/Axq1t0=
+github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.26 h1:ZjsIjuahUd/OxnxgfeK6OZqrReRNru1BcPfFD95IoN0=
+github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.26/go.mod h1:57VbRA0K0Xxx5XUUTUrVxZ/gm4u8Bk3LBflfufwHtKQ=
+github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.27 h1:i49S7g2smFWmlconi97Nn8dxb7jSQ0TeOFD20NX3M+o=
+github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.27/go.mod h1:qiLes3uVQAK+cpb837HUEhypYKGBz2ZCDcDl5BYQtpM=
+github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.10 h1:d5/908OJ4bXg8lyjeMPvXetEKqoDoLi5Owy1zNue3yg=
+github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.10/go.mod h1:a57l7Hwh+FWI+we50g5NPJHYUKeJKfXbc4w8SyXu8Ig=
+github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.18 h1:W/EyPFl9A5rXrtoilfwHYEvzHER+K4SpBPtMXi24Mos=
+github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.18/go.mod h1:UG50K+pvd/uy6xExbobg0rjqFBFZe6I3l75EPDZw4tg=
+github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.26 h1:oIRaBVJ2CxT7T61l3JkBCW/rnyh2TFZIRL71MvJTKRI=
+github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.26/go.mod h1:1gCTZPMlqAUSFoFbkl35hKtTpE96WFl7ORvDXmSLpRc=
+github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.25 h1:2pQEbwf+/6EDbiit/GcBE2K4IUpMZymaA0kOz3xK978=
+github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.25/go.mod h1:KvT6NCcQ0EZ+ZkVRrlBMt04Po3ok23YELEp7WimhLhM=
+github.com/aws/aws-sdk-go-v2/service/s3 v1.102.2 h1:ie4ElCmUKS26pzrZcIk/lmt4yWjAqLLcawstyQCh298=
+github.com/aws/aws-sdk-go-v2/service/s3 v1.102.2/go.mod h1:zjsomFeX5duj+4PlMB+o4JoWTIx+G0XMyzjYrUbQkN0=
+github.com/aws/aws-sdk-go-v2/service/signin v1.1.2 h1:qwU4CNor4G2gRW3Na24FWOCZSEvr967IAV3D+MXxdE0=
+github.com/aws/aws-sdk-go-v2/service/signin v1.1.2/go.mod h1:vwIj2P3URYf7ZtOQkO2iVbAaR8qFhqMv05w3aPPA5XA=
+github.com/aws/aws-sdk-go-v2/service/sns v1.39.14 h1:p8WdWDh5AwSZdp19Haa3XMyPCICi9Z375a/Nu3IIEZY=
+github.com/aws/aws-sdk-go-v2/service/sns v1.39.14/go.mod h1:NKVY7DER6VXHkt2I/ycmHakALNboi3Rqwt4eEf/1Cnk=
+github.com/aws/aws-sdk-go-v2/service/sqs v1.42.24 h1:JP2wjWGmUp8lTCZb13Dv0Eciyc1jbO8pd0HZVMHFlrc=
+github.com/aws/aws-sdk-go-v2/service/sqs v1.42.24/go.mod h1:Ql9ziDutk8ERAN9HMaYANCW3lop451ppebkxEJMLCTM=
+github.com/aws/aws-sdk-go-v2/service/sso v1.31.0 h1:cMDUWhi7vlmGFj6rldHofjnAQhLLBWwRvNRbtEoyZo4=
+github.com/aws/aws-sdk-go-v2/service/sso v1.31.0/go.mod h1:VfGCm6HLUGwcouY6zjQJd81PoFsb8sfSmKd5QCg9XXc=
+github.com/aws/aws-sdk-go-v2/service/ssooidc v1.36.3 h1:/pKy3mNES+ppFHf3g/XmI95+f3vdeXnkWYObByoYGAg=
+github.com/aws/aws-sdk-go-v2/service/ssooidc v1.36.3/go.mod h1:0z2rhlVAjsYeBzV8fdT4jR23Nq+fYhoJNSJN+4SBlqM=
+github.com/aws/aws-sdk-go-v2/service/sts v1.43.0 h1:HN43uYvF8D2NtaPTeE4L3HMvXgd+RtqEqrea4uUztkM=
+github.com/aws/aws-sdk-go-v2/service/sts v1.43.0/go.mod h1:qzhDlfw1BRYyoNofevemNzrlRpUXa0BJamWx7klrR8w=
+github.com/aws/smithy-go v1.27.1 h1:4T340VFndXtADGF52gYa1POyL7s9E4Z1OeZ1hCscIw8=
+github.com/aws/smithy-go v1.27.1/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
 github.com/bahlo/generic-list-go v0.2.0 h1:5sz/EEAK+ls5wF+NeqDpk5+iNdMDXrh3z3nPnH1Wvgk=
 github.com/bahlo/generic-list-go v0.2.0/go.mod h1:2KvAjgMlE5NNynlg/5iLrrCCZ2+5xWbdbCW3pNTGyYg=
 github.com/bazelbuild/rules_go v0.46.0 h1:CTefzjN/D3Cdn3rkrM6qMWuQj59OBcuOjyIp3m4hZ7s=
@@ -854,8 +850,8 @@ github.com/cncf/xds/go v0.0.0-20211011173535-cb28da3451f1/go.mod h1:eXthEFrGJvWH
 github.com/cncf/xds/go v0.0.0-20220314180256-7f1daf1720fc/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
 github.com/cncf/xds/go v0.0.0-20230105202645-06c439db220b/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
 github.com/cncf/xds/go v0.0.0-20230310173818-32f1caf87195/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
-github.com/cncf/xds/go v0.0.0-20251210132809-ee656c7534f5 h1:6xNmx7iTtyBRev0+D/Tv1FZd4SCg8axKApyNyRsAt/w=
-github.com/cncf/xds/go v0.0.0-20251210132809-ee656c7534f5/go.mod h1:KdCmV+x/BuvyMxRnYBlmVaq4OLiKW6iRQfvC62cvdkI=
+github.com/cncf/xds/go v0.0.0-20260202195803-dba9d589def2 h1:aBangftG7EVZoUb69Os8IaYg++6uMOdKK83QtkkvJik=
+github.com/cncf/xds/go v0.0.0-20260202195803-dba9d589def2/go.mod h1:qwXFYgsP6T7XnJtbKlf1HP8AjxZZyzxMmc+Lq5GjlU4=
 github.com/cockroachdb/apd/v3 v3.2.1 h1:U+8j7t0axsIgvQUqthuNm82HIrYXodOV2iWLWtEaIwg=
 github.com/cockroachdb/apd/v3 v3.2.1/go.mod h1:klXJcjp+FffLTHlhIG69tezTDvdP065naDsHzKhYSqc=
 github.com/cockroachdb/errors v1.11.3 h1:5bA+k2Y6r+oz/6Z/RFlNeVCesGARKuC6YymtcDrbC/I=
@@ -1000,16 +996,16 @@ github.com/envoyproxy/go-control-plane v0.10.3/go.mod h1:fJJn/j26vwOu972OllsvAgJ
 github.com/envoyproxy/go-control-plane v0.11.0/go.mod h1:VnHyVMpzcLvCFt9yUz1UnCwHLhwx1WguiVDV7pTG/tI=
 github.com/envoyproxy/go-control-plane v0.14.0 h1:hbG2kr4RuFj222B6+7T83thSPqLjwBIfQawTkC++2HA=
 github.com/envoyproxy/go-control-plane v0.14.0/go.mod h1:NcS5X47pLl/hfqxU70yPwL9ZMkUlwlKxtAohpi2wBEU=
-github.com/envoyproxy/go-control-plane/envoy v1.36.0 h1:yg/JjO5E7ubRyKX3m07GF3reDNEnfOboJ0QySbH736g=
-github.com/envoyproxy/go-control-plane/envoy v1.36.0/go.mod h1:ty89S1YCCVruQAm9OtKeEkQLTb+Lkz0k8v9W0Oxsv98=
+github.com/envoyproxy/go-control-plane/envoy v1.37.0 h1:u3riX6BoYRfF4Dr7dwSOroNfdSbEPe9Yyl09/B6wBrQ=
+github.com/envoyproxy/go-control-plane/envoy v1.37.0/go.mod h1:DReE9MMrmecPy+YvQOAOHNYMALuowAnbjjEMkkWOi6A=
 github.com/envoyproxy/go-control-plane/ratelimit v0.1.0 h1:/G9QYbddjL25KvtKTv3an9lx6VBE2cnb8wp1vEGNYGI=
 github.com/envoyproxy/go-control-plane/ratelimit v0.1.0/go.mod h1:Wk+tMFAFbCXaJPzVVHnPgRKdUdwW/KdbRt94AzgRee4=
 github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
 github.com/envoyproxy/protoc-gen-validate v0.6.7/go.mod h1:dyJXwwfPK2VSqiB9Klm1J6romD608Ba7Hij42vrOBCo=
 github.com/envoyproxy/protoc-gen-validate v0.9.1/go.mod h1:OKNgG7TCp5pF4d6XftA0++PMirau2/yoOwVac3AbF2w=
 github.com/envoyproxy/protoc-gen-validate v0.10.0/go.mod h1:DRjgyB0I43LtJapqN6NiRwroiAU2PaFuvk/vjgh61ss=
-github.com/envoyproxy/protoc-gen-validate v1.3.0 h1:TvGH1wof4H33rezVKWSpqKz5NXWg5VPuZ0uONDT6eb4=
-github.com/envoyproxy/protoc-gen-validate v1.3.0/go.mod h1:HvYl7zwPa5mffgyeTUHA9zHIH36nmrm7oCbo4YKoSWA=
+github.com/envoyproxy/protoc-gen-validate v1.3.3 h1:MVQghNeW+LZcmXe7SY1V36Z+WFMDjpqGAGacLe2T0ds=
+github.com/envoyproxy/protoc-gen-validate v1.3.3/go.mod h1:TsndJ/ngyIdQRhMcVVGDDHINPLWB7C82oDArY51KfB0=
 github.com/facebookgo/clock v0.0.0-20150410010913-600d898af40a h1:yDWHCSQ40h88yih2JAcL6Ls/kVkSE8GFACTGVnMPruw=
 github.com/facebookgo/clock v0.0.0-20150410010913-600d898af40a/go.mod h1:7Ga40egUymuWXxAe151lTNnCv97MddSOVsjpPPkityA=
 github.com/facebookgo/ensure v0.0.0-20200202191622-63f1cf65ac4c h1:8ISkoahWXwZR41ois5lSJBSVw4D0OV19Ht/JSTzvSv0=
@@ -1324,8 +1320,8 @@ github.com/grpc-ecosystem/grpc-gateway v1.16.0 h1:gmcG1KaJ57LophUzW0Hy8NmPhnMZb4
 github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw=
 github.com/grpc-ecosystem/grpc-gateway/v2 v2.7.0/go.mod h1:hgWBS7lorOAVIJEQMi4ZsPv9hVvWI6+ch50m39Pf2Ks=
 github.com/grpc-ecosystem/grpc-gateway/v2 v2.11.3/go.mod h1:o//XUCC/F+yRGJoPO/VU0GSB0f8Nhgmxx0VIRUvaC0w=
-github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.3 h1:NmZ1PKzSTQbuGHw9DGPFomqkkLWMC+vZCkfs+FHv1Vg=
-github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.3/go.mod h1:zQrxl1YP88HQlA6i9c63DSVPFklWpGX4OWAc9bFuaH4=
+github.com/grpc-ecosystem/grpc-gateway/v2 v2.28.0 h1:HWRh5R2+9EifMyIHV7ZV+MIZqgz+PMpZ14Jynv3O2Zs=
+github.com/grpc-ecosystem/grpc-gateway/v2 v2.28.0/go.mod h1:JfhWUomR1baixubs02l85lZYYOm7LV6om4ceouMv45c=
 github.com/hamba/avro/v2 v2.31.0 h1:wv3nmua7lCEIwWsb6vqsTS3pXktTxcKg5eoyNu0VhrU=
 github.com/hamba/avro/v2 v2.31.0/go.mod h1:t6lJYAGE5Mswfn17zjtyQsssRQgnqO6TXLBCHHWRqrw=
 github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
@@ -1511,8 +1507,8 @@ github.com/lanrat/extsort v1.4.2 h1:akbLIdo4PhNZtvjpaWnbXtGMmLtnGzXplkzfgl+XTTY=
 github.com/lanrat/extsort v1.4.2/go.mod h1:hceP6kxKPKebjN1RVrDBXMXXECbaI41Y94tt6MDazc4=
 github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
 github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
-github.com/lib/pq v1.11.1 h1:wuChtj2hfsGmmx3nf1m7xC2XpK6OtelS2shMY+bGMtI=
-github.com/lib/pq v1.11.1/go.mod h1:/p+8NSbOcwzAEI7wiMXFlgydTwcgTr3OSKMsD2BitpA=
+github.com/lib/pq v1.12.0 h1:mC1zeiNamwKBecjHarAr26c/+d8V5w/u4J0I/yASbJo=
+github.com/lib/pq v1.12.0/go.mod h1:/p+8NSbOcwzAEI7wiMXFlgydTwcgTr3OSKMsD2BitpA=
 github.com/linkedin/goavro/v2 v2.15.0 h1:pDj1UrjUOO62iXhgBiE7jQkpNIc5/tA5eZsgolMjgVI=
 github.com/linkedin/goavro/v2 v2.15.0/go.mod h1:KXx+erlq+RPlGSPmLF7xGo6SAbh8sCQ53x064+ioxhk=
 github.com/linxGnu/grocksdb v1.10.8 h1:Nau01Hhm/0kaVTR6d4viwD6npYbnDvZAfzwJCLzKRYo=
@@ -1691,8 +1687,8 @@ github.com/phpdave11/gofpdf v1.4.2/go.mod h1:zpO6xFn9yxo3YLyMvW8HcKWVdbNqgIfOOp2
 github.com/phpdave11/gofpdi v1.0.12/go.mod h1:vBmVV0Do6hSBHC8uKUQ71JGW+ZGQq74llk/7bXwjDoI=
 github.com/phpdave11/gofpdi v1.0.13/go.mod h1:vBmVV0Do6hSBHC8uKUQ71JGW+ZGQq74llk/7bXwjDoI=
 github.com/pierrec/lz4/v4 v4.1.15/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
-github.com/pierrec/lz4/v4 v4.1.26 h1:GrpZw1gZttORinvzBdXPUXATeqlJjqUG/D87TKMnhjY=
-github.com/pierrec/lz4/v4 v4.1.26/go.mod h1:EoQMVJgeeEOMsCqCzqFm2O0cJvljX2nGZjcRIPL34O4=
+github.com/pierrec/lz4/v4 v4.1.27 h1:+PhzhWDrjRj89TH2sw43nE3+4+W8lSxIuQadEHZyjUk=
+github.com/pierrec/lz4/v4 v4.1.27/go.mod h1:EoQMVJgeeEOMsCqCzqFm2O0cJvljX2nGZjcRIPL34O4=
 github.com/pierrre/compare v1.0.2 h1:k4IUsHgh+dbcAOIWCfxVa/7G6STjADH2qmhomv+1quc=
 github.com/pierrre/compare v1.0.2/go.mod h1:8UvyRHH+9HS8Pczdd2z5x/wvv67krDwVxoOndaIIDVU=
 github.com/pierrre/geohash v1.0.0 h1:f/zfjdV4rVofTCz1FhP07T+EMQAvcMM2ioGZVt+zqjI=
@@ -1783,8 +1779,6 @@ github.com/quic-go/quic-go v0.59.0 h1:OLJkp1Mlm/aS7dpKgTc6cnpynnD2Xg7C1pwL6vy/SA
 github.com/quic-go/quic-go v0.59.0/go.mod h1:upnsH4Ju1YkqpLXC305eW3yDZ4NfnNbmQRCMWS58IKU=
 github.com/rabbitmq/amqp091-go v1.11.0 h1:HxIctVm9Gid/Vtn706necmZ7Wj6pgGI2eqplRbEY8O8=
 github.com/rabbitmq/amqp091-go v1.11.0/go.mod h1:Hy4jKW5kQART1u+JkDTF9YYOQUHXqMuhrgxOEeS7G4o=
-github.com/rasky/go-xdr v0.0.0-20170124162913-1a41d1a06c93 h1:UVArwN/wkKjMVhh2EQGC0tEc1+FqiLlvYXY5mQ2f8Wg=
-github.com/rasky/go-xdr v0.0.0-20170124162913-1a41d1a06c93/go.mod h1:Nfe4efndBz4TibWycNE+lqyJZiMX4ycx+QKV8Ta0f/o=
 github.com/rclone/Proton-API-Bridge v1.0.3 h1:Bs7RC4xCFSN0BPIYVda/BNxp0qo3NV0gB2VZqx2KIew=
 github.com/rclone/Proton-API-Bridge v1.0.3/go.mod h1:26RAest751Ofk+F/d8xtl4UyWXrZvMQwn39U8rm/WKM=
 github.com/rclone/go-proton-api v1.0.2 h1:cJtJUab0MGJ3C6q5kiEJs3pbyhSLnOKMyYOQehA0PBc=
@@ -1795,8 +1789,8 @@ github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 h1:N/ElC8H3+5X
 github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4=
 github.com/rdleal/intervalst v1.5.0 h1:SEB9bCFz5IqD1yhfH1Wv8IBnY/JQxDplwkxHjT6hamU=
 github.com/rdleal/intervalst v1.5.0/go.mod h1:xO89Z6BC+LQDH+IPQQw/OESt5UADgFD41tYMUINGpxQ=
-github.com/redis/go-redis/v9 v9.19.0 h1:XPVaaPSnG6RhYf7p+rmSa9zZfeVAnWsH5h3lxthOm/k=
-github.com/redis/go-redis/v9 v9.19.0/go.mod h1:v/M13XI1PVCDcm01VtPFOADfZtHf8YW3baQf57KlIkA=
+github.com/redis/go-redis/v9 v9.20.0 h1:WnQYxLkgO2xiXTCJY0ldIiI8dNqCDlQAG+AtaH7a2a0=
+github.com/redis/go-redis/v9 v9.20.0/go.mod h1:v/M13XI1PVCDcm01VtPFOADfZtHf8YW3baQf57KlIkA=
 github.com/redis/rueidis v1.0.71 h1:pODtnAR5GAB7j4ekhldZ29HKOxe4Hph0GTDGk1ayEQY=
 github.com/redis/rueidis v1.0.71/go.mod h1:lfdcZzJ1oKGKL37vh9fO3ymwt+0TdjkkUCJxbgpmcgQ=
 github.com/redis/rueidis/rueidiscompat v1.0.71 h1:wNZ//kEjMZgBM0KCk7ncOX8KmAgROU2kDdDNpwheG4w=
@@ -2028,10 +2022,6 @@ github.com/vmihailenco/msgpack/v5 v5.4.1 h1:cQriyiUvjTwOHg8QZaPihLWeRAAVoCpE00IU
 github.com/vmihailenco/msgpack/v5 v5.4.1/go.mod h1:GaZTsDaehaPpQVyxrf5mtQlH+pc21PIudVV/E3rRQok=
 github.com/vmihailenco/tagparser/v2 v2.0.0 h1:y09buUbR+b5aycVFQs/g70pqKVZNBmxwAhO7/IwNM9g=
 github.com/vmihailenco/tagparser/v2 v2.0.0/go.mod h1:Wri+At7QHww0WTrCBeu4J6bNtoV6mEfg5OIWRZA9qds=
-github.com/willscott/go-nfs v0.0.4 h1:1vpOPAdECmoT2KmZ8u+ukO/jfvDjMEUNYhA2F1jGJtI=
-github.com/willscott/go-nfs v0.0.4/go.mod h1:VhNccO67Oug787VNXcyx9JDI3ZoSpqoKMT/lWMhUIDg=
-github.com/willscott/go-nfs-client v0.0.0-20251022144359-801f10d98886 h1:DtrBtkgTJk2XGt4T7eKdKVkd9A5NCevN2e4inLXtsqA=
-github.com/willscott/go-nfs-client v0.0.0-20251022144359-801f10d98886/go.mod h1:Tq++Lr/FgiS3X48q5FETemXiSLGuYMQT2sPjYNPJSwA=
 github.com/wk8/go-ordered-map/v2 v2.1.8 h1:5h/BUHu93oj4gIdvHHHGsScSTMijfx5PeYkE/fJgbpc=
 github.com/wk8/go-ordered-map/v2 v2.1.8/go.mod h1:5nJHM5DyteebpVlHnWMV0rPz6Zp7+xBAnxjb1X5vnTw=
 github.com/wsxiaoys/terminal v0.0.0-20160513160801-0940f3fc43a0 h1:3UeQBvD0TFrlVjOeLOBz+CPAI8dnbqNSVwUwRrkp7vQ=
@@ -2064,14 +2054,14 @@ github.com/yandex-cloud/go-genproto v0.0.0-20211115083454-9ca41db5ed9e h1:9LPdmD
 github.com/yandex-cloud/go-genproto v0.0.0-20211115083454-9ca41db5ed9e/go.mod h1:HEUYX/p8966tMUHHT+TsS0hF/Ca/NYwqprC5WXSDMfE=
 github.com/ydb-platform/ydb-go-genproto v0.0.0-20221215182650-986f9d10542f/go.mod h1:Er+FePu1dNUieD+XTMDduGpQuCPssK5Q4BjF+IIXJ3I=
 github.com/ydb-platform/ydb-go-genproto v0.0.0-20230528143953-42c825ace222/go.mod h1:Er+FePu1dNUieD+XTMDduGpQuCPssK5Q4BjF+IIXJ3I=
-github.com/ydb-platform/ydb-go-genproto v0.0.0-20260311095541-ebbf792c1180 h1:avIdi8eGXjKbn1WLokNR1Ofnz1k8t7tJ88YQLD/iCi8=
-github.com/ydb-platform/ydb-go-genproto v0.0.0-20260311095541-ebbf792c1180/go.mod h1:Er+FePu1dNUieD+XTMDduGpQuCPssK5Q4BjF+IIXJ3I=
+github.com/ydb-platform/ydb-go-genproto v0.0.0-20260428144813-1c07baab7f7b h1:xeiobG1riqe6dTMZuTcOvwOY0BbFidSk3gRLyVeLfJA=
+github.com/ydb-platform/ydb-go-genproto v0.0.0-20260428144813-1c07baab7f7b/go.mod h1:Er+FePu1dNUieD+XTMDduGpQuCPssK5Q4BjF+IIXJ3I=
 github.com/ydb-platform/ydb-go-sdk-auth-environ v0.5.1 h1:XaRxeVrOyl3y6v9CiYMWaFdZ6zevvYe+TRxOR8ifa2s=
 github.com/ydb-platform/ydb-go-sdk-auth-environ v0.5.1/go.mod h1:9YzkhlIymWaJGX6KMU3vh5sOf3UKbCXkG/ZdjaI3zNM=
 github.com/ydb-platform/ydb-go-sdk/v3 v3.44.0/go.mod h1:oSLwnuilwIpaF5bJJMAofnGgzPJusoI3zWMNb8I+GnM=
 github.com/ydb-platform/ydb-go-sdk/v3 v3.47.3/go.mod h1:bWnOIcUHd7+Sl7DN+yhyY1H/I61z53GczvwJgXMgvj0=
-github.com/ydb-platform/ydb-go-sdk/v3 v3.134.2 h1:Bzra5hUQIWrxknQ0oV6HE5q/eZU1VhHMKSZ/XWpqMgM=
-github.com/ydb-platform/ydb-go-sdk/v3 v3.134.2/go.mod h1:VYUUkRJkKuQPkIpgtZJj6+58Fa2g8ccAqdmaaK6HP5k=
+github.com/ydb-platform/ydb-go-sdk/v3 v3.139.5 h1:mTih9wy1RbI4PplGlFn/CMnkgxm1HzD/nKSdnNMy4pE=
+github.com/ydb-platform/ydb-go-sdk/v3 v3.139.5/go.mod h1:b9NEO6mgaiqsnOMkS003uS82XsKh6GL+ZTFfPqXWz+c=
 github.com/ydb-platform/ydb-go-yc v0.12.1 h1:qw3Fa+T81+Kpu5Io2vYHJOwcrYrVjgJlT6t/0dOXJrA=
 github.com/ydb-platform/ydb-go-yc v0.12.1/go.mod h1:t/ZA4ECdgPWjAb4jyDe8AzQZB5dhpGbi3iCahFaNwBY=
 github.com/ydb-platform/ydb-go-yc-metadata v0.6.1 h1:9E5q8Nsy2RiJMZDNVy0A3KUrIMBPakJ2VgloeWbcI84=
@@ -2111,8 +2101,8 @@ go.etcd.io/bbolt v1.4.3 h1:dEadXpI6G79deX5prL3QRNP6JB8UxVkqo4UPnHaNXJo=
 go.etcd.io/bbolt v1.4.3/go.mod h1:tKQlpPaYCVFctUIgFKFnAlvbmB3tpy1vkTnDWohtc0E=
 go.etcd.io/etcd/api/v3 v3.6.10 h1:jlwjtELjA8yi2VWpOFH+0w0lGr3K6mVDyn0RDB9aaAY=
 go.etcd.io/etcd/api/v3 v3.6.10/go.mod h1:pdV4VeFmvhdNjB4LWRkC8ReLyRBAxUOze3GarMhE2sk=
-go.etcd.io/etcd/client/pkg/v3 v3.6.11 h1:e41mp315Yn3QMGPmEzCyLsMINgJXTY/dX8kM++1csxU=
-go.etcd.io/etcd/client/pkg/v3 v3.6.11/go.mod h1:DysuMe/inqRyC/1tjRR6hReH/VV9Lufs27YKSKBWWJg=
+go.etcd.io/etcd/client/pkg/v3 v3.6.12 h1:36zzB+pQOdHbhN+kH2iJz/K8bJn0ZLtLfPPO7jozTDo=
+go.etcd.io/etcd/client/pkg/v3 v3.6.12/go.mod h1:hh2+ZXtfLzs3o6mn92ntgNPBrTJJOvXqICM5g3L3DMY=
 go.etcd.io/etcd/client/v3 v3.6.10 h1:J598zJ+C/ZPvImypmq5waj84+bovePrlZERHklf34y0=
 go.etcd.io/etcd/client/v3 v3.6.10/go.mod h1:iHhUDUcEwaKs1YFq3MgmI9U4zhTVasp/vgdVbFf1RS8=
 go.mongodb.org/mongo-driver v1.17.9 h1:IexDdCuuNJ3BHrELgBlyaH9p60JXAvdzWR128q+U5tU=
@@ -2128,8 +2118,8 @@ go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0=
 go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo=
 go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
 go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
-go.opentelemetry.io/contrib/detectors/gcp v1.39.0 h1:kWRNZMsfBHZ+uHjiH4y7Etn2FK26LAGkNFw7RHv1DhE=
-go.opentelemetry.io/contrib/detectors/gcp v1.39.0/go.mod h1:t/OGqzHBa5v6RHZwrDBJ2OirWc+4q/w2fTbLZwAKjTk=
+go.opentelemetry.io/contrib/detectors/gcp v1.42.0 h1:kpt2PEJuOuqYkPcktfJqWWDjTEd/FNgrxcniL7kQrXQ=
+go.opentelemetry.io/contrib/detectors/gcp v1.42.0/go.mod h1:W9zQ439utxymRrXsUOzZbFX4JhLxXU4+ZnCt8GG7yA8=
 go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.67.0 h1:yI1/OhfEPy7J9eoa6Sj051C7n5dvpj0QX8g4sRchg04=
 go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.67.0/go.mod h1:NoUCKYWK+3ecatC4HjkRktREheMeEtrXoQxrqYFeHSc=
 go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.60.0 h1:0tY123n7CdWMem7MOVdKOt0YfshufLCwfE5Bob+hQuM=
@@ -2138,14 +2128,14 @@ go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.67.0 h1:Oyrsyzu
 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.67.0/go.mod h1:C2NGBr+kAB4bk3xtMXfZ94gqFDtg/GkI7e9zqGh5Beg=
 go.opentelemetry.io/otel v1.43.0 h1:mYIM03dnh5zfN7HautFE4ieIig9amkNANT+xcVxAj9I=
 go.opentelemetry.io/otel v1.43.0/go.mod h1:JuG+u74mvjvcm8vj8pI5XiHy1zDeoCS2LB1spIq7Ay0=
-go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.38.0 h1:vl9obrcoWVKp/lwl8tRE33853I8Xru9HFbw/skNeLs8=
-go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.38.0/go.mod h1:GAXRxmLJcVM3u22IjTg74zWBrRCKq8BnOqUVLodpcpw=
+go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.42.0 h1:MdKucPl/HbzckWWEisiNqMPhRrAOQX8r4jTuGr636gk=
+go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.42.0/go.mod h1:RolT8tWtfHcjajEH5wFIZ4Dgh5jpPdFXYV9pTAk/qjc=
 go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.35.0 h1:0NIXxOCFx+SKbhCVxwl3ETG8ClLPAa0KuKV6p3yhxP8=
 go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.35.0/go.mod h1:ChZSJbbfbl/DcRZNc9Gqh6DYGlfjw4PvO1pEOZH1ZsE=
-go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.38.0 h1:GqRJVj7UmLjCVyVJ3ZFLdPRmhDUp2zFmQe3RHIOsw24=
-go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.38.0/go.mod h1:ri3aaHSmCTVYu2AWv44YMauwAQc0aqI9gHKIcSbI1pU=
-go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.38.0 h1:lwI4Dc5leUqENgGuQImwLo4WnuXFPetmPpkLi2IrX54=
-go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.38.0/go.mod h1:Kz/oCE7z5wuyhPxsXDuaPteSWqjSBD5YaSdbxZYGbGk=
+go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.42.0 h1:THuZiwpQZuHPul65w4WcwEnkX2QIuMT+UFoOrygtoJw=
+go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.42.0/go.mod h1:J2pvYM5NGHofZ2/Ru6zw/TNWnEQp5crgyDeSrYpXkAw=
+go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.42.0 h1:zWWrB1U6nqhS/k6zYB74CjRpuiitRtLLi68VcgmOEto=
+go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.42.0/go.mod h1:2qXPNBX1OVRC0IwOnfo1ljoid+RD0QK3443EaqVlsOU=
 go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.38.0 h1:aTL7F04bJHUlztTsNGJ2l+6he8c+y/b//eR0jjjemT4=
 go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.38.0/go.mod h1:kldtb7jDTeol0l3ewcmd8SDvx3EmIE7lyvqbasU3QC4=
 go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.43.0 h1:TC+BewnDpeiAmcscXbGMfxkO+mwYUwE/VySwvw88PfA=
@@ -2163,8 +2153,8 @@ go.opentelemetry.io/otel/trace v1.43.0/go.mod h1:/QJhyVBUUswCphDVxq+8mld+AvhXZLh
 go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI=
 go.opentelemetry.io/proto/otlp v0.15.0/go.mod h1:H7XAot3MsfNsj7EXtrA2q5xSNQ10UqI405h3+duxN4U=
 go.opentelemetry.io/proto/otlp v0.19.0/go.mod h1:H7XAot3MsfNsj7EXtrA2q5xSNQ10UqI405h3+duxN4U=
-go.opentelemetry.io/proto/otlp v1.9.0 h1:l706jCMITVouPOqEnii2fIAuO3IVGBRPV5ICjceRb/A=
-go.opentelemetry.io/proto/otlp v1.9.0/go.mod h1:xE+Cx5E/eEHw+ISFkwPLwCZefwVjY+pqKg1qcK03+/4=
+go.opentelemetry.io/proto/otlp v1.10.0 h1:IQRWgT5srOCYfiWnpqUYz9CVmbO8bFmKcwYxpuCSL2g=
+go.opentelemetry.io/proto/otlp v1.10.0/go.mod h1:/CV4QoCR/S9yaPj8utp3lvQPoqMtxXdzn7ozvvozVqk=
 go.uber.org/atomic v1.6.0/go.mod h1:sABNBOSYdrvTF6hTgEIbc7YasKWGhgEQZyfxyTvoXHQ=
 go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
 go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
@@ -2188,12 +2178,12 @@ go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0=
 go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8=
 go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
 go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
-gocloud.dev v0.45.0 h1:WknIK8IbRdmynDvara3Q7G6wQhmEiOGwpgJufbM39sY=
-gocloud.dev v0.45.0/go.mod h1:0kXKmkCLG6d31N7NyLZWzt7jDSQura9zD/mWgiB6THI=
+gocloud.dev v0.46.0 h1:niIuZwSjMtBx8K+ITB2s5kZullB13PGOS2ZoQPZxQ4Q=
+gocloud.dev v0.46.0/go.mod h1:ACQe+2qO+hEO+pdcvvsM+RB63r8TyGD1W3ESCLFyzvM=
 gocloud.dev/pubsub/natspubsub v0.45.0 h1:kfCupVejeynIQcS1GaIvkY3Nj/acLlM5yPWyxZN8wJ8=
 gocloud.dev/pubsub/natspubsub v0.45.0/go.mod h1:WU5SMDWuF7CCr2U8UpbrEONYYFOmvigCAvTpOOpUvYE=
-gocloud.dev/pubsub/rabbitpubsub v0.45.0 h1:hlejKjBFYQBUAj7ZsHKgW0JOER/3XUqrMFFxBOFUIOU=
-gocloud.dev/pubsub/rabbitpubsub v0.45.0/go.mod h1:Kqq6gppGY/RTrRQkKSHZq+/qbCufPEoQ0E1V3X389zM=
+gocloud.dev/pubsub/rabbitpubsub v0.46.0 h1:uWJ8z/F8xYxoTlBafTweoKvX2V85cwtN+/+nyRRtjZ0=
+gocloud.dev/pubsub/rabbitpubsub v0.46.0/go.mod h1:Ef5XHxv17zV9B3U9xoYAqJ8Rutslxt5Emb5e1cScuXc=
 golang.org/x/arch v0.20.0 h1:dx1zTU0MAE98U+TQ8BLl7XsJbgze2WnNKF/8tGp/Q6c=
 golang.org/x/arch v0.20.0/go.mod h1:bdwinDaKcfZUGpH09BB7ZmOfhalA8lQdzl62l8gGWsk=
 golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
@@ -2405,8 +2395,8 @@ golang.org/x/sync v0.4.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=
 golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
 golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
 golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
-golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
-golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
+golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM=
+golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
 golang.org/x/sys v0.0.0-20180810173357-98c5dad5d1a0/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@@ -2892,8 +2882,8 @@ google.golang.org/grpc v1.51.0/go.mod h1:wgNDFcnuBGmxLKI/qn4T+m5BtEBYXJPvibbUPsA
 google.golang.org/grpc v1.52.0/go.mod h1:pu6fVzoFb+NBYNAvQL08ic+lvB2IojljRYuun5vorUY=
 google.golang.org/grpc v1.53.0/go.mod h1:OnIrk0ipVdj4N5d9IUoFUx72/VlD7+jUsHwZgwSMQpw=
 google.golang.org/grpc v1.55.0/go.mod h1:iYEXKGkEBhg1PjZQvoYEVPTDkHo1/bjTnfwTeGONTY8=
-google.golang.org/grpc v1.80.0 h1:Xr6m2WmWZLETvUNvIUmeD5OAagMw3FiKmMlTdViWsHM=
-google.golang.org/grpc v1.80.0/go.mod h1:ho/dLnxwi3EDJA4Zghp7k2Ec1+c2jqup0bFkw07bwF4=
+google.golang.org/grpc v1.81.1 h1:VnnIIZ88UzOOKLukQi+ImGz8O1Wdp8nAGGnvOfEIWQQ=
+google.golang.org/grpc v1.81.1/go.mod h1:xGH9GfzOyMTGIOXBJmXt+BX/V0kcdQbdcuwQ/zNw42I=
 google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.1.0/go.mod h1:6Kw0yEErY5E/yWrBtf03jp27GLLJujG4z/JK95pnjjw=
 google.golang.org/grpc/examples v0.0.0-20250407062114-b368379ef8f6 h1:ExN12ndbJ608cboPYflpTny6mXSzPrDLh0iTaVrRrds=
 google.golang.org/grpc/examples v0.0.0-20250407062114-b368379ef8f6/go.mod h1:6ytKWczdvnpnO+m+JiG9NjEDzR1FJfsnmJdG7B8QVZ8=

k8s/charts/seaweedfs/Chart.yaml

@@ -1,6 +1,6 @@
 apiVersion: v1
 description: SeaweedFS
 name: seaweedfs
-appVersion: "4.29"
+appVersion: "4.33"
 # Dev note: Trigger a helm chart release by `git tag -a helm-<version>`
-version: 4.29.0
+version: 4.33.0

k8s/charts/seaweedfs/README.md

@@ -384,4 +384,4 @@ helm install seaweedfs seaweedfs/seaweedfs \
 ## Enterprise
 
 For enterprise users, please visit [seaweedfs.com](https://seaweedfs.com) for the SeaweedFS Enterprise Edition, 
-which has a self-healing storage format with better data protection.
+which has advanced features, including data recovery, self-healing storage, customizable erasure coding, EC vacuum and repair, etc.

k8s/charts/seaweedfs/dashboards/seaweedfs-grafana-dashboard.json

@@ -3666,6 +3666,291 @@
       ],
       "title": "S3 Bucket Object Count",
       "type": "timeseries"
+    },
+    {
+      "collapsed": false,
+      "datasource": {
+        "type": "prometheus",
+        "uid": "${DS_PROMETHEUS}"
+      },
+      "gridPos": {
+        "h": 1,
+        "w": 24,
+        "x": 0,
+        "y": 130
+      },
+      "id": 92,
+      "panels": [],
+      "title": "Filer Object Size Distribution",
+      "type": "row"
+    },
+    {
+      "datasource": {
+        "type": "prometheus",
+        "uid": "${DS_PROMETHEUS}"
+      },
+      "description": "Rate of new objects created, split by size range.",
+      "fieldConfig": {
+        "defaults": {
+          "color": {
+            "mode": "palette-classic"
+          },
+          "custom": {
+            "axisBorderShow": false,
+            "axisCenteredZero": false,
+            "axisColorMode": "text",
+            "axisLabel": "objects/s",
+            "axisPlacement": "auto",
+            "barAlignment": 0,
+            "drawStyle": "line",
+            "fillOpacity": 25,
+            "gradientMode": "none",
+            "hideFrom": {
+              "legend": false,
+              "tooltip": false,
+              "viz": false
+            },
+            "insertNulls": false,
+            "lineInterpolation": "linear",
+            "lineWidth": 1,
+            "pointSize": 4,
+            "scaleDistribution": {
+              "type": "linear"
+            },
+            "showPoints": "auto",
+            "spanNulls": false,
+            "stacking": {
+              "group": "A",
+              "mode": "normal"
+            },
+            "thresholdsStyle": {
+              "mode": "off"
+            }
+          },
+          "mappings": [],
+          "thresholds": {
+            "mode": "absolute",
+            "steps": [
+              {
+                "color": "green",
+                "value": null
+              }
+            ]
+          },
+          "unit": "short",
+          "unitScale": true
+        },
+        "overrides": []
+      },
+      "gridPos": {
+        "h": 8,
+        "w": 12,
+        "x": 0,
+        "y": 131
+      },
+      "id": 93,
+      "links": [],
+      "options": {
+        "legend": {
+          "calcs": [],
+          "displayMode": "list",
+          "placement": "bottom",
+          "showLegend": true
+        },
+        "tooltip": {
+          "mode": "single",
+          "sort": "none"
+        }
+      },
+      "pluginVersion": "8.1.2",
+      "targets": [
+        {
+          "exemplar": true,
+          "expr": "sum(rate(SeaweedFS_filer_object_size_bytes_bucket{le=\"1024\",namespace=\"$NAMESPACE\"}[$__rate_interval]))",
+          "format": "time_series",
+          "hide": false,
+          "instant": false,
+          "interval": "",
+          "intervalFactor": 2,
+          "legendFormat": "< 1KB",
+          "refId": "A",
+          "step": 60
+        },
+        {
+          "exemplar": true,
+          "expr": "clamp_min(sum(rate(SeaweedFS_filer_object_size_bytes_bucket{le=\"102400\",namespace=\"$NAMESPACE\"}[$__rate_interval])) - sum(rate(SeaweedFS_filer_object_size_bytes_bucket{le=\"1024\",namespace=\"$NAMESPACE\"}[$__rate_interval])), 0)",
+          "format": "time_series",
+          "hide": false,
+          "instant": false,
+          "interval": "",
+          "intervalFactor": 2,
+          "legendFormat": "1KB - 100KB",
+          "refId": "B",
+          "step": 60
+        },
+        {
+          "exemplar": true,
+          "expr": "clamp_min(sum(rate(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.048576e+06\",namespace=\"$NAMESPACE\"}[$__rate_interval])) - sum(rate(SeaweedFS_filer_object_size_bytes_bucket{le=\"102400\",namespace=\"$NAMESPACE\"}[$__rate_interval])), 0)",
+          "format": "time_series",
+          "hide": false,
+          "instant": false,
+          "interval": "",
+          "intervalFactor": 2,
+          "legendFormat": "100KB - 1MB",
+          "refId": "C",
+          "step": 60
+        },
+        {
+          "exemplar": true,
+          "expr": "clamp_min(sum(rate(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.048576e+08\",namespace=\"$NAMESPACE\"}[$__rate_interval])) - sum(rate(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.048576e+06\",namespace=\"$NAMESPACE\"}[$__rate_interval])), 0)",
+          "format": "time_series",
+          "hide": false,
+          "instant": false,
+          "interval": "",
+          "intervalFactor": 2,
+          "legendFormat": "1MB - 100MB",
+          "refId": "D",
+          "step": 60
+        },
+        {
+          "exemplar": true,
+          "expr": "clamp_min(sum(rate(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.073741824e+09\",namespace=\"$NAMESPACE\"}[$__rate_interval])) - sum(rate(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.048576e+08\",namespace=\"$NAMESPACE\"}[$__rate_interval])), 0)",
+          "format": "time_series",
+          "hide": false,
+          "instant": false,
+          "interval": "",
+          "intervalFactor": 2,
+          "legendFormat": "100MB - 1GB",
+          "refId": "E",
+          "step": 60
+        },
+        {
+          "exemplar": true,
+          "expr": "clamp_min(sum(rate(SeaweedFS_filer_object_size_bytes_count{namespace=\"$NAMESPACE\"}[$__rate_interval])) - sum(rate(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.073741824e+09\",namespace=\"$NAMESPACE\"}[$__rate_interval])), 0)",
+          "format": "time_series",
+          "hide": false,
+          "instant": false,
+          "interval": "",
+          "intervalFactor": 2,
+          "legendFormat": "> 1GB",
+          "refId": "F",
+          "step": 60
+        }
+      ],
+      "title": "Filer Object Write Rate by Size Range",
+      "type": "timeseries"
+    },
+    {
+      "datasource": {
+        "type": "prometheus",
+        "uid": "${DS_PROMETHEUS}"
+      },
+      "description": "Objects created per size range over the selected time window.",
+      "fieldConfig": {
+        "defaults": {
+          "color": {
+            "mode": "thresholds"
+          },
+          "mappings": [],
+          "min": 0,
+          "thresholds": {
+            "mode": "absolute",
+            "steps": [
+              {
+                "color": "green",
+                "value": null
+              }
+            ]
+          },
+          "unit": "short"
+        },
+        "overrides": []
+      },
+      "gridPos": {
+        "h": 8,
+        "w": 12,
+        "x": 12,
+        "y": 131
+      },
+      "id": 94,
+      "options": {
+        "displayMode": "gradient",
+        "orientation": "horizontal",
+        "reduceOptions": {
+          "calcs": [
+            "lastNotNull"
+          ],
+          "fields": "",
+          "values": false
+        },
+        "showUnfilled": true,
+        "valueMode": "color"
+      },
+      "pluginVersion": "8.1.2",
+      "targets": [
+        {
+          "exemplar": true,
+          "expr": "sum(increase(SeaweedFS_filer_object_size_bytes_bucket{le=\"1024\",namespace=\"$NAMESPACE\"}[$__range]))",
+          "format": "time_series",
+          "instant": true,
+          "interval": "",
+          "intervalFactor": 2,
+          "legendFormat": "< 1KB",
+          "refId": "A"
+        },
+        {
+          "exemplar": true,
+          "expr": "clamp_min(sum(increase(SeaweedFS_filer_object_size_bytes_bucket{le=\"102400\",namespace=\"$NAMESPACE\"}[$__range])) - sum(increase(SeaweedFS_filer_object_size_bytes_bucket{le=\"1024\",namespace=\"$NAMESPACE\"}[$__range])), 0)",
+          "format": "time_series",
+          "instant": true,
+          "interval": "",
+          "intervalFactor": 2,
+          "legendFormat": "1KB - 100KB",
+          "refId": "B"
+        },
+        {
+          "exemplar": true,
+          "expr": "clamp_min(sum(increase(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.048576e+06\",namespace=\"$NAMESPACE\"}[$__range])) - sum(increase(SeaweedFS_filer_object_size_bytes_bucket{le=\"102400\",namespace=\"$NAMESPACE\"}[$__range])), 0)",
+          "format": "time_series",
+          "instant": true,
+          "interval": "",
+          "intervalFactor": 2,
+          "legendFormat": "100KB - 1MB",
+          "refId": "C"
+        },
+        {
+          "exemplar": true,
+          "expr": "clamp_min(sum(increase(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.048576e+08\",namespace=\"$NAMESPACE\"}[$__range])) - sum(increase(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.048576e+06\",namespace=\"$NAMESPACE\"}[$__range])), 0)",
+          "format": "time_series",
+          "instant": true,
+          "interval": "",
+          "intervalFactor": 2,
+          "legendFormat": "1MB - 100MB",
+          "refId": "D"
+        },
+        {
+          "exemplar": true,
+          "expr": "clamp_min(sum(increase(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.073741824e+09\",namespace=\"$NAMESPACE\"}[$__range])) - sum(increase(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.048576e+08\",namespace=\"$NAMESPACE\"}[$__range])), 0)",
+          "format": "time_series",
+          "instant": true,
+          "interval": "",
+          "intervalFactor": 2,
+          "legendFormat": "100MB - 1GB",
+          "refId": "E"
+        },
+        {
+          "exemplar": true,
+          "expr": "clamp_min(sum(increase(SeaweedFS_filer_object_size_bytes_count{namespace=\"$NAMESPACE\"}[$__range])) - sum(increase(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.073741824e+09\",namespace=\"$NAMESPACE\"}[$__range])), 0)",
+          "format": "time_series",
+          "instant": true,
+          "interval": "",
+          "intervalFactor": 2,
+          "legendFormat": "> 1GB",
+          "refId": "F"
+        }
+      ],
+      "title": "Filer Object Size Distribution",
+      "type": "bargauge"
     }
   ],
   "refresh": "",

k8s/charts/seaweedfs/templates/all-in-one/all-in-one-deployment.yaml

@@ -81,8 +81,9 @@ spec:
           imagePullPolicy: {{ default "IfNotPresent" .Values.global.seaweedfs.imagePullPolicy }}
           env:
             {{- /* Determine default cluster alias and the corresponding env var keys to avoid conflicts */}}
-            {{- $envMerged := merge (.Values.global.seaweedfs.extraEnvironmentVars | default dict) (.Values.allInOne.extraEnvironmentVars | default dict) }}
-            {{- $clusterDefault := default "sw" (index $envMerged "WEED_CLUSTER_DEFAULT") }}
+            {{- $mergedExtraEnvironmentVars := dict }}
+            {{- include "seaweedfs.mergeExtraEnvironmentVars" (dict "global" .Values.global.seaweedfs "component" .Values.allInOne "target" $mergedExtraEnvironmentVars) }}
+            {{- $clusterDefault := default "sw" (index $mergedExtraEnvironmentVars "WEED_CLUSTER_DEFAULT") }}
             {{- $clusterUpper := upper $clusterDefault }}
             {{- $clusterMasterKey := printf "WEED_CLUSTER_%s_MASTER" $clusterUpper }}
             {{- $clusterFilerKey := printf "WEED_CLUSTER_%s_FILER" $clusterUpper }}
@@ -100,8 +101,8 @@ spec:
                   fieldPath: metadata.namespace
             - name: SEAWEEDFS_FULLNAME
               value: "{{ include "seaweedfs.fullname" . }}"
-            {{- if .Values.allInOne.extraEnvironmentVars }}
-            {{- range $key, $value := .Values.allInOne.extraEnvironmentVars }}
+            {{- range $key := keys $mergedExtraEnvironmentVars | sortAlpha }}
+            {{- $value := index $mergedExtraEnvironmentVars $key }}
             {{- if and (ne $key $clusterMasterKey) (ne $key $clusterFilerKey) }}
             - name: {{ $key }}
               {{- if kindIs "string" $value }}
@@ -112,20 +113,6 @@ spec:
               {{- end }}
             {{- end }}
             {{- end }}
-            {{- end }}
-            {{- if .Values.global.seaweedfs.extraEnvironmentVars }}
-            {{- range $key, $value := .Values.global.seaweedfs.extraEnvironmentVars }}
-            {{- if and (ne $key $clusterMasterKey) (ne $key $clusterFilerKey) }}
-            - name: {{ $key }}
-              {{- if kindIs "string" $value }}
-              value: {{ tpl $value $ | quote }}
-              {{- else }}
-              valueFrom:
-                {{ toYaml $value | nindent 16 }}
-              {{- end }}
-            {{- end }}
-            {{- end }}
-            {{- end }}
             # Inject computed cluster endpoints for the default cluster
             - name: {{ $clusterMasterKey }}
               value: {{ include "seaweedfs.cluster.masterAddress" . | quote }}
@@ -272,6 +259,9 @@ spec:
               {{- $authMethods := .Values.allInOne.sftp.authMethods | default .Values.sftp.authMethods }}
               {{- if $authMethods }}
               -sftp.authMethods={{ $authMethods }} \
+              {{- if contains "certificate" $authMethods }}
+              -sftp.trustedUserCAKeysFile=/etc/sw/sftp_ca/ca_user.pub \
+              {{- end }}
               {{- end }}
               {{- $maxAuthTries := .Values.allInOne.sftp.maxAuthTries | default .Values.sftp.maxAuthTries }}
               {{- if $maxAuthTries }}
@@ -319,6 +309,12 @@ spec:
               name: config-users
               readOnly: true
             {{- end }}
+            {{- $aioAuthMethods := .Values.allInOne.sftp.authMethods | default .Values.sftp.authMethods }}
+            {{- if and $aioAuthMethods (contains "certificate" $aioAuthMethods) }}
+            - mountPath: /etc/sw/sftp_ca
+              name: config-sftp-ca
+              readOnly: true
+            {{- end }}
             {{- end }}
             {{- if .Values.filer.notificationConfig }}
             - name: notification-config
@@ -454,6 +450,18 @@ spec:
             defaultMode: 420
             secretName: {{ default (printf "%s-sftp-secret" (include "seaweedfs.fullname" .)) (or .Values.allInOne.sftp.existingConfigSecret .Values.sftp.existingConfigSecret) }}
         {{- end }}
+        {{- $aioAuthMethodsVol := .Values.allInOne.sftp.authMethods | default .Values.sftp.authMethods }}
+        {{- if and $aioAuthMethodsVol (contains "certificate" $aioAuthMethodsVol) }}
+        {{- $aioCASecret := or .Values.allInOne.sftp.existingCAKeysSecret .Values.sftp.existingCAKeysSecret }}
+        {{- $aioCAKeys := or .Values.allInOne.sftp.trustedUserCAKeys .Values.sftp.trustedUserCAKeys }}
+        {{- if and (not $aioCASecret) (not $aioCAKeys) }}
+        {{- fail "allInOne.sftp.authMethods includes \"certificate\" but neither trustedUserCAKeys nor existingCAKeysSecret is set" }}
+        {{- end }}
+        - name: config-sftp-ca
+          secret:
+            defaultMode: 420
+            secretName: {{ default (printf "%s-sftp-ca-secret" (include "seaweedfs.fullname" .)) $aioCASecret }}
+        {{- end }}
         {{- end }}
         {{- if .Values.filer.notificationConfig }}
         - name: notification-config

k8s/charts/seaweedfs/templates/sftp/sftp-ca-secret.yaml

@@ -0,0 +1,40 @@
+{{- include "seaweedfs.compat" . -}}
+{{- /*
+Render a chart-managed Secret carrying the SSH user CA public key(s) for
+certificate-based SFTP authentication, but only when:
+  - SFTP (standalone or all-in-one) is enabled,
+  - "certificate" is in the relevant authMethods list,
+  - the user provided inline CA keys via sftp.trustedUserCAKeys, and
+  - no existingCAKeysSecret is set (which would supersede this Secret).
+*/}}
+{{- $sftpEnabled := or .Values.sftp.enabled (and .Values.allInOne.enabled .Values.allInOne.sftp.enabled) -}}
+{{- $authMethods := .Values.sftp.authMethods -}}
+{{- if and .Values.allInOne.enabled .Values.allInOne.sftp.authMethods -}}
+{{- $authMethods = .Values.allInOne.sftp.authMethods -}}
+{{- end -}}
+{{- $certInMethods := and $authMethods (contains "certificate" $authMethods) -}}
+{{- $inlineCAKeys := .Values.sftp.trustedUserCAKeys -}}
+{{- if and .Values.allInOne.enabled .Values.allInOne.sftp.trustedUserCAKeys -}}
+{{- $inlineCAKeys = .Values.allInOne.sftp.trustedUserCAKeys -}}
+{{- end -}}
+{{- $existingSecret := .Values.sftp.existingCAKeysSecret -}}
+{{- if and .Values.allInOne.enabled .Values.allInOne.sftp.existingCAKeysSecret -}}
+{{- $existingSecret = .Values.allInOne.sftp.existingCAKeysSecret -}}
+{{- end -}}
+{{- if and $sftpEnabled $certInMethods $inlineCAKeys (not $existingSecret) }}
+apiVersion: v1
+kind: Secret
+type: Opaque
+metadata:
+  name: {{ include "seaweedfs.fullname" . }}-sftp-ca-secret
+  namespace: {{ .Release.Namespace }}
+  labels:
+    app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
+    helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    app.kubernetes.io/managed-by: {{ .Release.Service }}
+    app.kubernetes.io/instance: {{ .Release.Name }}
+    app.kubernetes.io/component: sftp
+stringData:
+  ca_user.pub: |
+{{ $inlineCAKeys | indent 4 }}
+{{- end }}

k8s/charts/seaweedfs/templates/sftp/sftp-deployment.yaml

@@ -135,6 +135,9 @@ spec:
               {{- end }}
               {{- if .Values.sftp.authMethods }}
               -authMethods={{ .Values.sftp.authMethods }} \
+              {{- if contains "certificate" .Values.sftp.authMethods }}
+              -trustedUserCAKeysFile=/etc/sw/sftp_ca/ca_user.pub \
+              {{- end }}
               {{- end }}
               {{- if .Values.sftp.maxAuthTries }}
               -maxAuthTries={{ .Values.sftp.maxAuthTries }} \
@@ -176,6 +179,11 @@ spec:
             - mountPath: /etc/sw/ssh
               name: config-ssh
               readOnly: true
+            {{- if and .Values.sftp.authMethods (contains "certificate" .Values.sftp.authMethods) }}
+            - mountPath: /etc/sw/sftp_ca
+              name: config-sftp-ca
+              readOnly: true
+            {{- end }}
             {{- if include "seaweedfs.securityConfigEnabled" . }}
             - name: security-config
               readOnly: true
@@ -256,6 +264,19 @@ spec:
             {{- else }}
             secretName: {{ include "seaweedfs.fullname" . }}-sftp-ssh-secret
             {{- end }}
+        {{- if and .Values.sftp.authMethods (contains "certificate" .Values.sftp.authMethods) }}
+        {{- if and (not .Values.sftp.existingCAKeysSecret) (not .Values.sftp.trustedUserCAKeys) }}
+        {{- fail "sftp.authMethods includes \"certificate\" but neither sftp.trustedUserCAKeys nor sftp.existingCAKeysSecret is set" }}
+        {{- end }}
+        - name: config-sftp-ca
+          secret:
+            defaultMode: 420
+            {{- if .Values.sftp.existingCAKeysSecret }}
+            secretName: {{ .Values.sftp.existingCAKeysSecret }}
+            {{- else }}
+            secretName: {{ include "seaweedfs.fullname" . }}-sftp-ca-secret
+            {{- end }}
+        {{- end }}
         {{- if eq .Values.sftp.logs.type "hostPath" }}
         - name: logs
           hostPath:

k8s/charts/seaweedfs/templates/shared/post-install-bucket-hook.yaml

@@ -143,6 +143,8 @@ spec:
           {{- if kindIs "bool" .versioning }}
             {{- if .versioning }}
               {{- $bucketVersioning = "Enabled" }}
+            {{- else }}
+              {{- $bucketVersioning = "Suspended" }}
             {{- end }}
           {{- else if kindIs "string" .versioning }}
             {{- $versioningLower := lower .versioning }}

k8s/charts/seaweedfs/values.yaml

@@ -919,7 +919,7 @@ filer:
     # Buckets may be exposed publicly by setting `anonymousRead` to `true`
     # ttl format: [1-255][m|h|d|w|M|y] (e.g., 7d)
     # objectLock enables S3 Object Lock (irreversible, forces versioning)
-    # versioning: Enabled or Suspended (or true to enable)
+    # versioning: Enabled or Suspended (or bool true/false)
     # createBuckets:
     #   - name: bucket-a
     #     anonymousRead: true
@@ -972,7 +972,7 @@ s3:
   # Buckets may be exposed publicly by setting `anonymousRead` to `true`
   # ttl format: [1-255][m|h|d|w|M|y] (e.g., 7d)
   # objectLock enables S3 Object Lock (irreversible, forces versioning)
-  # versioning: Enabled or Suspended (or true to enable)
+  # versioning: Enabled or Suspended (or bool true/false)
   # createBuckets:
   #   - name: bucket-a
   #     anonymousRead: true
@@ -1138,7 +1138,7 @@ sftp:
   # SSH server configuration
   sshPrivateKey: "/etc/sw/seaweedfs_sftp_ssh_private_key"  # Path to the SSH private key file for host authentication
   hostKeysFolder: "/etc/sw/ssh"  # path to folder containing SSH private key files for host authentication
-  authMethods: "password,publickey"  # Comma-separated list of allowed auth methods: password, publickey, keyboard-interactive
+  authMethods: "password,publickey"  # Comma-separated list of allowed auth methods: password, publickey, certificate
   maxAuthTries: 6  # Maximum number of authentication attempts per connection
   bannerMessage: "SeaweedFS SFTP Server"  # Message displayed before authentication
   loginGraceTime: "2m"  # Timeout for authentication
@@ -1155,6 +1155,18 @@ sftp:
   # Set to the name of an existing kubernetes Secret with the list of ssh private keys for sftp
   existingSshConfigSecret: null
 
+  # SSH user-certificate authentication (CA-signed user certs). Mirrors
+  # OpenSSH `TrustedUserCAKeys` and MinIO `--sftp=trusted-user-ca-key`.
+  # Add "certificate" to `authMethods` to activate; when active, plain
+  # public keys are rejected on the public-key channel.
+  #
+  # Inline CA public keys in OpenSSH authorized_keys format (one per
+  # line). Ignored when `existingCAKeysSecret` is set.
+  trustedUserCAKeys: ""
+  # Set to the name of an existing kubernetes Secret carrying the CA
+  # public keys under data key `ca_user.pub`.
+  existingCAKeysSecret: null
+
   # Additional resources
   sidecars: []
   initContainers: ""
@@ -1514,7 +1526,7 @@ allInOne:
     # Buckets may be exposed publicly by setting `anonymousRead` to `true`
     # ttl format: [1-255][m|h|d|w|M|y] (e.g., 7d)
     # objectLock enables S3 Object Lock (irreversible, forces versioning)
-    # versioning: Enabled or Suspended (or true to enable)
+    # versioning: Enabled or Suspended (or bool true/false)
     # createBuckets:
     #   - name: bucket-a
     #     anonymousRead: true
@@ -1545,6 +1557,10 @@ allInOne:
     existingConfigSecret: null
     # Set to the name of an existing kubernetes Secret with the SSH keys
     existingSshConfigSecret: null
+    # SSH user-certificate authentication. See sftp.trustedUserCAKeys above.
+    # (null on either field inherits from the top-level sftp.* setting.)
+    trustedUserCAKeys: null
+    existingCAKeysSecret: null
 
   # Service settings
   service:

other/metrics/grafana_seaweedfs.json

@@ -623,6 +623,237 @@
               "show": true
             }
           ]
+        },
+        {
+          "aliasColors": {},
+          "bars": false,
+          "dashLength": 10,
+          "dashes": false,
+          "datasource": "${DS_PROMETHEUS}",
+          "editable": true,
+          "error": false,
+          "fill": 5,
+          "fillGradient": 0,
+          "grid": {},
+          "gridPos": {
+            "h": 7,
+            "w": 12,
+            "x": 0,
+            "y": 31
+          },
+          "hiddenSeries": false,
+          "id": 91,
+          "legend": {
+            "alignAsTable": true,
+            "avg": true,
+            "current": true,
+            "hideEmpty": false,
+            "hideZero": false,
+            "max": true,
+            "min": false,
+            "rightSide": true,
+            "show": true,
+            "total": false,
+            "values": true
+          },
+          "lines": true,
+          "linewidth": 2,
+          "links": [],
+          "nullPointMode": "null as zero",
+          "options": {
+            "alertThreshold": true
+          },
+          "percentage": false,
+          "pluginVersion": "8.1.2",
+          "pointradius": 5,
+          "points": false,
+          "renderer": "flot",
+          "seriesOverrides": [],
+          "spaceLength": 10,
+          "stack": true,
+          "steppedLine": false,
+          "targets": [
+            {
+              "expr": "sum(rate(SeaweedFS_filer_object_size_bytes_bucket{le=\"1024\"}[1m]))",
+              "format": "time_series",
+              "intervalFactor": 2,
+              "legendFormat": "< 1KB",
+              "refId": "A"
+            },
+            {
+              "expr": "clamp_min(sum(rate(SeaweedFS_filer_object_size_bytes_bucket{le=\"102400\"}[1m])) - sum(rate(SeaweedFS_filer_object_size_bytes_bucket{le=\"1024\"}[1m])), 0)",
+              "format": "time_series",
+              "intervalFactor": 2,
+              "legendFormat": "1KB - 100KB",
+              "refId": "B"
+            },
+            {
+              "expr": "clamp_min(sum(rate(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.048576e+06\"}[1m])) - sum(rate(SeaweedFS_filer_object_size_bytes_bucket{le=\"102400\"}[1m])), 0)",
+              "format": "time_series",
+              "intervalFactor": 2,
+              "legendFormat": "100KB - 1MB",
+              "refId": "C"
+            },
+            {
+              "expr": "clamp_min(sum(rate(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.048576e+08\"}[1m])) - sum(rate(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.048576e+06\"}[1m])), 0)",
+              "format": "time_series",
+              "intervalFactor": 2,
+              "legendFormat": "1MB - 100MB",
+              "refId": "D"
+            },
+            {
+              "expr": "clamp_min(sum(rate(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.073741824e+09\"}[1m])) - sum(rate(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.048576e+08\"}[1m])), 0)",
+              "format": "time_series",
+              "intervalFactor": 2,
+              "legendFormat": "100MB - 1GB",
+              "refId": "E"
+            },
+            {
+              "expr": "clamp_min(sum(rate(SeaweedFS_filer_object_size_bytes_count[1m])) - sum(rate(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.073741824e+09\"}[1m])), 0)",
+              "format": "time_series",
+              "intervalFactor": 2,
+              "legendFormat": "> 1GB",
+              "refId": "F"
+            }
+          ],
+          "thresholds": [],
+          "timeFrom": null,
+          "timeRegions": [],
+          "timeShift": null,
+          "title": "Filer Object Write Rate by Size Range",
+          "tooltip": {
+            "msResolution": false,
+            "shared": true,
+            "sort": 0,
+            "value_type": "individual"
+          },
+          "type": "graph",
+          "xaxis": {
+            "buckets": null,
+            "mode": "time",
+            "name": null,
+            "show": true,
+            "values": []
+          },
+          "yaxes": [
+            {
+              "format": "short",
+              "label": "objects/s",
+              "logBase": 1,
+              "max": null,
+              "min": 0,
+              "show": true
+            },
+            {
+              "format": "short",
+              "label": null,
+              "logBase": 1,
+              "max": null,
+              "min": null,
+              "show": true
+            }
+          ],
+          "yaxis": {
+            "align": false,
+            "alignLevel": null
+          }
+        },
+        {
+          "datasource": "${DS_PROMETHEUS}",
+          "description": "Objects created per size range over the selected time window.",
+          "fieldConfig": {
+            "defaults": {
+              "color": {
+                "mode": "thresholds"
+              },
+              "mappings": [],
+              "min": 0,
+              "thresholds": {
+                "mode": "absolute",
+                "steps": [
+                  {
+                    "color": "green",
+                    "value": null
+                  }
+                ]
+              },
+              "unit": "short"
+            },
+            "overrides": []
+          },
+          "gridPos": {
+            "h": 7,
+            "w": 12,
+            "x": 12,
+            "y": 31
+          },
+          "id": 92,
+          "options": {
+            "displayMode": "gradient",
+            "orientation": "horizontal",
+            "reduceOptions": {
+              "calcs": [
+                "lastNotNull"
+              ],
+              "fields": "",
+              "values": false
+            },
+            "showUnfilled": true,
+            "valueMode": "color"
+          },
+          "pluginVersion": "8.1.2",
+          "targets": [
+            {
+              "expr": "sum(increase(SeaweedFS_filer_object_size_bytes_bucket{le=\"1024\"}[$__range]))",
+              "format": "time_series",
+              "intervalFactor": 2,
+              "legendFormat": "< 1KB",
+              "refId": "A",
+              "instant": true
+            },
+            {
+              "expr": "clamp_min(sum(increase(SeaweedFS_filer_object_size_bytes_bucket{le=\"102400\"}[$__range])) - sum(increase(SeaweedFS_filer_object_size_bytes_bucket{le=\"1024\"}[$__range])), 0)",
+              "format": "time_series",
+              "intervalFactor": 2,
+              "legendFormat": "1KB - 100KB",
+              "refId": "B",
+              "instant": true
+            },
+            {
+              "expr": "clamp_min(sum(increase(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.048576e+06\"}[$__range])) - sum(increase(SeaweedFS_filer_object_size_bytes_bucket{le=\"102400\"}[$__range])), 0)",
+              "format": "time_series",
+              "intervalFactor": 2,
+              "legendFormat": "100KB - 1MB",
+              "refId": "C",
+              "instant": true
+            },
+            {
+              "expr": "clamp_min(sum(increase(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.048576e+08\"}[$__range])) - sum(increase(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.048576e+06\"}[$__range])), 0)",
+              "format": "time_series",
+              "intervalFactor": 2,
+              "legendFormat": "1MB - 100MB",
+              "refId": "D",
+              "instant": true
+            },
+            {
+              "expr": "clamp_min(sum(increase(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.073741824e+09\"}[$__range])) - sum(increase(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.048576e+08\"}[$__range])), 0)",
+              "format": "time_series",
+              "intervalFactor": 2,
+              "legendFormat": "100MB - 1GB",
+              "refId": "E",
+              "instant": true
+            },
+            {
+              "expr": "clamp_min(sum(increase(SeaweedFS_filer_object_size_bytes_count[$__range])) - sum(increase(SeaweedFS_filer_object_size_bytes_bucket{le=\"1.073741824e+09\"}[$__range])), 0)",
+              "format": "time_series",
+              "intervalFactor": 2,
+              "legendFormat": "> 1GB",
+              "refId": "F",
+              "instant": true
+            }
+          ],
+          "title": "Filer Object Size Distribution",
+          "type": "bargauge"
         }
       ],
       "repeat": null,

seaweed-volume/proto/volume_server.proto

@@ -450,8 +450,10 @@ message VolumeEcShardsDeleteRequest {
     uint32 volume_id = 1;
     string collection = 2;
     repeated uint32 shard_ids = 3;
+    bool full_teardown = 4; // pre-encode cleanup: wipe every EC artifact + generation for this volume, not just shard_ids
 }
 message VolumeEcShardsDeleteResponse {
+    bool full_teardown_done = 1; // set by a new server that performed full_teardown; absent from an old server lets the caller detect the silent no-op
 }
 
 message VolumeEcShardsMountRequest {
@@ -476,10 +478,13 @@ message VolumeEcShardReadRequest {
     int64 offset = 3;
     int64 size = 4;
     uint64 file_key = 5;
+    reserved 6;
+    int64 encode_ts_ns = 7; // caller's expected encode time; the server rejects a shard from a different encode run
 }
 message VolumeEcShardReadResponse {
     bytes data = 1;
     bool is_deleted = 2;
+    int64 encode_ts_ns = 3; // identity of the shard actually served; client rejects a mismatch (0 = pre-upgrade server)
 }
 
 message VolumeEcBlobDeleteRequest {
@@ -577,6 +582,7 @@ message VolumeInfo {
 message EcShardConfig {
     uint32 data_shards = 1;   // Number of data shards (e.g., 10)
     uint32 parity_shards = 2; // Number of parity shards (e.g., 4)
+    int64 encode_ts_ns = 3;   // encode time (unix nanos); a read served from a shard of a different encode run is rejected
 }
 message OldVersionVolumeInfo {
     repeated RemoteFile files = 1;

seaweed-volume/src/main.rs

@@ -545,6 +545,12 @@ async fn run(
                     whitelist.extend(sec.guard_white_list.iter().cloned());
                     let mut guard = state_reload.guard.write().unwrap();
                     guard.update_whitelist(&whitelist);
+                    guard.update_signing_keys(
+                        SigningKey(sec.jwt_signing_key),
+                        sec.jwt_signing_expires,
+                        SigningKey(sec.jwt_read_signing_key),
+                        sec.jwt_read_signing_expires,
+                    );
                 }
 
                 // Trigger heartbeat to report new volumes

seaweed-volume/src/metrics.rs

@@ -424,6 +424,10 @@ mod tests {
     #[tokio::test]
     async fn test_push_metrics_once() {
         register_metrics();
+        // A CounterVec with no children emits nothing, so create a labelset
+        // instead of depending on another test having touched the counter
+        // first (test order is nondeterministic).
+        REQUEST_COUNTER.with_label_values(&["GET", "200"]).inc();
 
         let captured = Arc::new(Mutex::new(None::<String>));
         let captured_clone = captured.clone();

seaweed-volume/src/security.rs

@@ -172,6 +172,26 @@ impl Guard {
         self.is_write_active = !is_empty_whitelist || !self.signing_key.is_empty();
     }
 
+    /// Refresh the JWT signing keys and their expirations in place so operators
+    /// can rotate keys via SIGHUP without restarting the process. Mirrors Go's
+    /// `Guard::UpdateSigningKeys`: it swaps the fields and recomputes
+    /// `is_write_active`.
+    pub fn update_signing_keys(
+        &mut self,
+        signing_key: SigningKey,
+        expires_after_sec: i64,
+        read_signing_key: SigningKey,
+        read_expires_after_sec: i64,
+    ) {
+        self.signing_key = signing_key;
+        self.expires_after_sec = expires_after_sec;
+        self.read_signing_key = read_signing_key;
+        self.read_expires_after_sec = read_expires_after_sec;
+
+        let is_empty_whitelist = self.whitelist_ips.is_empty() && self.whitelist_cidrs.is_empty();
+        self.is_write_active = !is_empty_whitelist || !self.signing_key.is_empty();
+    }
+
     /// Check if a remote IP is in the whitelist.
     /// Returns true if write security is inactive (no whitelist and no signing key),
     /// if the whitelist is empty, or if the IP matches.
@@ -463,6 +483,43 @@ mod tests {
         assert!(matches!(err, Err(JwtError::FileIdMismatch { .. })));
     }
 
+    #[test]
+    fn test_update_signing_keys_rotates() {
+        let mut guard = Guard::new(
+            &[],
+            SigningKey::from_string("old-write"),
+            10,
+            SigningKey::from_string("old-read"),
+            60,
+        );
+
+        guard.update_signing_keys(
+            SigningKey::from_string("new-write"),
+            11,
+            SigningKey::from_string("new-read"),
+            61,
+        );
+
+        let token = gen_jwt(&guard.signing_key, 60, "3,01637037d6").unwrap();
+        // Validates with the rotated key, not the old one.
+        assert!(decode_jwt(&guard.signing_key, &token).is_ok());
+        assert!(decode_jwt(&SigningKey::from_string("old-write"), &token).is_err());
+        assert_eq!(guard.expires_after_sec, 11);
+        assert_eq!(guard.read_expires_after_sec, 61);
+    }
+
+    #[test]
+    fn test_update_signing_keys_toggles_write_active() {
+        let mut guard = Guard::new(&[], SigningKey(vec![]), 0, SigningKey(vec![]), 0);
+        assert!(guard.check_jwt(None, true).is_ok());
+
+        guard.update_signing_keys(SigningKey::from_string("write"), 10, SigningKey(vec![]), 0);
+        assert!(guard.is_write_active);
+
+        guard.update_signing_keys(SigningKey(vec![]), 0, SigningKey(vec![]), 0);
+        assert!(!guard.is_write_active);
+    }
+
     #[test]
     fn test_extract_host() {
         assert_eq!(extract_host("192.168.1.1:8080"), "192.168.1.1");

seaweed-volume/src/server/grpc_server.rs

@@ -399,6 +399,27 @@ impl VolumeServer for VolumeGrpcService {
 
             if !is_ec_volume {
                 let mut store = self.state.store.write().unwrap();
+                // Recheck EC state under the write lock before mutating the .dat. The
+                // is_ec_volume snapshot was taken earlier under a separate read lock;
+                // ec.encode mounts EC shards (copied from the .dat) BEFORE deleting the
+                // originals, so the vid can be EC now while the .dat still exists. A
+                // delete_volume_needle here would tombstone that .dat, which the encode
+                // then removes — the delete is lost and the needle resurrected from the
+                // pre-tombstone shards. If the vid is now EC, return a retriable 503 so
+                // the filer requeues the delete onto the EC path.
+                if store.has_ec_volume(file_id.volume_id) {
+                    results.push(volume_server_pb::DeleteResult {
+                        file_id: fid_str.clone(),
+                        status: 503,
+                        error: format!(
+                            "volume {} became ec during delete, try again",
+                            file_id.volume_id
+                        ),
+                        size: 0,
+                        version: 0,
+                    });
+                    continue;
+                }
                 match store.delete_volume_needle(file_id.volume_id, &mut n) {
                     Ok(size) => {
                         if size.0 == 0 {
@@ -420,13 +441,35 @@ impl VolumeServer for VolumeGrpcService {
                         }
                     }
                     Err(e) => {
-                        results.push(volume_server_pb::DeleteResult {
-                            file_id: fid_str.clone(),
-                            status: 500,
-                            error: e.to_string(),
-                            size: 0,
-                            version: 0,
-                        });
+                        // The volume vanished between the is_ec_volume snapshot and
+                        // this mutation. If an EC volume now occupies the vid (ec.encode
+                        // mounted EC then deleted the .dat under us), the delete belongs
+                        // on the EC journal, not here — return a retriable 503 with the
+                        // "try again" token so the filer requeues it and the retry hits
+                        // the EC path. A bare exact "not found" would be dropped
+                        // permanently by the filer chunk-GC.
+                        if matches!(e, crate::storage::volume::VolumeError::NotFound)
+                            && store.has_ec_volume(file_id.volume_id)
+                        {
+                            results.push(volume_server_pb::DeleteResult {
+                                file_id: fid_str.clone(),
+                                status: 503,
+                                error: format!(
+                                    "volume {} became ec during delete, try again",
+                                    file_id.volume_id
+                                ),
+                                size: 0,
+                                version: 0,
+                            });
+                        } else {
+                            results.push(volume_server_pb::DeleteResult {
+                                file_id: fid_str.clone(),
+                                status: 500,
+                                error: e.to_string(),
+                                size: 0,
+                                version: 0,
+                            });
+                        }
                     }
                 }
             } else {
@@ -2133,6 +2176,12 @@ impl VolumeServer for VolumeGrpcService {
                 ec_shard_config: Some(crate::storage::volume::VifEcShardConfig {
                     data_shards: data_shards,
                     parity_shards: parity_shards,
+                    // This run's identity; the read path rejects a shard from a
+                    // different encode run.
+                    encode_ts_ns: std::time::SystemTime::now()
+                        .duration_since(std::time::UNIX_EPOCH)
+                        .unwrap_or_default()
+                        .as_nanos() as i64,
                 }),
                 ..Default::default()
             };
@@ -2593,12 +2642,35 @@ impl VolumeServer for VolumeGrpcService {
         self.state.check_maintenance()?;
         let req = request.into_inner();
         let vid = VolumeId(req.volume_id);
+
+        if req.full_teardown {
+            // Pre-encode cleanup: evict the volume and wipe every EC artifact for it
+            // on every disk, not just the listed shards, so a remote node retains no
+            // stale generation that a fresh gen-0 copy would collide with. Echo the
+            // acknowledgement so the caller can tell a pre-upgrade server apart.
+            {
+                let mut store = self.state.store.write().unwrap();
+                let _ = store.remove_ec_volume(vid);
+                for loc in &store.locations {
+                    loc.remove_ec_volume_files_full_teardown(&req.collection, vid);
+                }
+            }
+            self.state.volume_state_notify.notify_one();
+            return Ok(Response::new(
+                volume_server_pb::VolumeEcShardsDeleteResponse {
+                    full_teardown_done: true,
+                },
+            ));
+        }
+
         let mut store = self.state.store.write().unwrap();
         store.delete_ec_shards(vid, &req.collection, &req.shard_ids);
         drop(store);
         self.state.volume_state_notify.notify_one();
         Ok(Response::new(
-            volume_server_pb::VolumeEcShardsDeleteResponse {},
+            volume_server_pb::VolumeEcShardsDeleteResponse {
+                full_teardown_done: false,
+            },
         ))
     }
 
@@ -2672,6 +2744,21 @@ impl VolumeServer for VolumeGrpcService {
                 ))
             })?;
 
+        // Reject a shard whose identity doesn't match the caller's index; the caller
+        // then recovers from parity. Lenient only when the caller has no identity
+        // (pre-upgrade reader): a known caller must not accept an unstamped holder,
+        // which would serve a stale shard from a different encode run.
+        if req.encode_ts_ns != 0 && req.encode_ts_ns != ec_vol.encode_ts_ns {
+            return Err(Status::failed_precondition(format!(
+                "ec shard {}.{} belongs to a different encode run",
+                req.volume_id, req.shard_id
+            )));
+        }
+
+        // Identity of the shard actually served, echoed on every response chunk so
+        // the client can reject a different encode run even from a pre-upgrade server.
+        let served_encode_ts_ns = ec_vol.encode_ts_ns;
+
         // Check if the requested needle is deleted (via .ecx index, matching Go)
         if req.file_key > 0 {
             let needle_id = NeedleId(req.file_key);
@@ -2682,6 +2769,7 @@ impl VolumeServer for VolumeGrpcService {
                 if size.is_deleted() {
                     let results = vec![Ok(volume_server_pb::VolumeEcShardReadResponse {
                         is_deleted: true,
+                        encode_ts_ns: served_encode_ts_ns,
                         ..Default::default()
                     })];
                     return Ok(Response::new(Box::pin(tokio_stream::iter(results))));
@@ -2730,6 +2818,7 @@ impl VolumeServer for VolumeGrpcService {
             results.push(Ok(volume_server_pb::VolumeEcShardReadResponse {
                 data: buf,
                 is_deleted: false,
+                encode_ts_ns: served_encode_ts_ns,
             }));
             if n < chunk_size {
                 break; // short read means EOF

seaweed-volume/src/server/handlers.rs

@@ -23,6 +23,27 @@ use crate::pb::volume_server_pb;
 use crate::storage::needle::needle::Needle;
 use crate::storage::types::*;
 
+/// Slack added over the configured file-size limit when bounding the raw
+/// request body, to allow for multipart/form-data framing overhead. The exact
+/// per-file limit is still enforced on the parsed data after multipart parsing.
+const UPLOAD_BODY_OVERHEAD: usize = 16 * 1024 * 1024; // 16 MiB
+
+/// Upper bound on bytes we will materialize in memory for a single request when
+/// expanding stored content (gzip decompression or chunk-manifest assembly).
+/// Guards against gzip bombs and crafted/oversized manifest sizes OOM-killing
+/// the server; legitimate large objects are read via client-side chunk fetches.
+const MAX_EXPANSION_BYTES: u64 = 2 * 1024 * 1024 * 1024; // 2 GiB
+
+/// Why `maybe_decompress_gzip` failed, so callers can distinguish a recoverable
+/// "not valid gzip, use the raw bytes" from "the bomb cap was hit, reject it".
+#[derive(Debug)]
+enum GunzipError {
+    /// Input was not valid gzip / decode failed — callers may fall back to raw.
+    Decode,
+    /// Decompressed output would exceed `MAX_EXPANSION_BYTES` — must be rejected.
+    TooLarge,
+}
+
 // ============================================================================
 // Inflight Throttle Guard
 // ============================================================================
@@ -1452,12 +1473,16 @@ async fn get_or_head_handler_inner(
     if is_compressed {
         if needs_image_ops {
             // Always decompress for image operations (Go decompresses before resize/crop)
-            use flate2::read::GzDecoder;
-            use std::io::Read as _;
-            let mut decoder = GzDecoder::new(&data[..]);
-            let mut decompressed = Vec::new();
-            if decoder.read_to_end(&mut decompressed).is_ok() {
-                data = decompressed;
+            match maybe_decompress_gzip(&data) {
+                Ok(decompressed) => data = decompressed,
+                Err(GunzipError::TooLarge) => {
+                    return (
+                        StatusCode::PAYLOAD_TOO_LARGE,
+                        "compressed object exceeds decompression limit",
+                    )
+                        .into_response()
+                }
+                Err(GunzipError::Decode) => {} // not valid gzip; keep raw bytes
             }
         } else {
             let accept_encoding = headers
@@ -1474,12 +1499,16 @@ async fn get_or_head_handler_inner(
                 response_headers.insert(header::CONTENT_ENCODING, "gzip".parse().unwrap());
             } else {
                 // Decompress for client
-                use flate2::read::GzDecoder;
-                use std::io::Read as _;
-                let mut decoder = GzDecoder::new(&data[..]);
-                let mut decompressed = Vec::new();
-                if decoder.read_to_end(&mut decompressed).is_ok() {
-                    data = decompressed;
+                match maybe_decompress_gzip(&data) {
+                    Ok(decompressed) => data = decompressed,
+                    Err(GunzipError::TooLarge) => {
+                        return (
+                            StatusCode::PAYLOAD_TOO_LARGE,
+                            "compressed object exceeds decompression limit",
+                        )
+                            .into_response()
+                    }
+                    Err(GunzipError::Decode) => {} // not valid gzip; keep raw bytes
                 }
             }
         }
@@ -2134,15 +2163,30 @@ pub async fn post_handler(
         .and_then(|v| v.to_str().ok())
         .map(|s| s.to_string());
 
-    // Read body
-    let body = match axum::body::to_bytes(request.into_body(), usize::MAX).await {
+    // Read body, bounded by the configured file-size limit so a single upload
+    // cannot buffer unbounded memory and OOM-kill the server (mirrors Go's
+    // io.LimitReader(r.Body, sizeLimit+1)). A margin covers multipart framing;
+    // the exact per-file limit is still enforced on the parsed data below.
+    let body_limit = if state.file_size_limit_bytes > 0 {
+        // try_from (not `as usize`) so a >usize::MAX limit on 32-bit caps at
+        // usize::MAX instead of silently truncating/wrapping to a tiny value.
+        usize::try_from(state.file_size_limit_bytes)
+            .unwrap_or(usize::MAX)
+            .saturating_add(UPLOAD_BODY_OVERHEAD)
+    } else {
+        usize::MAX
+    };
+    let body = match axum::body::to_bytes(request.into_body(), body_limit).await {
         Ok(b) => b,
         Err(e) => {
-            return json_error_with_query(
-                StatusCode::BAD_REQUEST,
-                format!("read body: {}", e),
-                Some(&query),
-            )
+            // With a limit configured, an error here means the body exceeded it
+            // before we buffered the whole thing; report it like the size check.
+            let msg = if state.file_size_limit_bytes > 0 {
+                format!("file over the limited {} bytes", state.file_size_limit_bytes)
+            } else {
+                format!("read body: {}", e)
+            };
+            return json_error_with_query(StatusCode::BAD_REQUEST, msg, Some(&query));
         }
     };
 
@@ -2282,7 +2326,17 @@ pub async fn post_handler(
     };
 
     let uncompressed_data = if is_gzipped {
-        maybe_decompress_gzip(&body_data_raw).unwrap_or_else(|| body_data_raw.clone())
+        match maybe_decompress_gzip(&body_data_raw) {
+            Ok(d) => d,
+            Err(GunzipError::TooLarge) => {
+                return json_error_with_query(
+                    StatusCode::PAYLOAD_TOO_LARGE,
+                    "compressed object exceeds decompression limit",
+                    Some(&query),
+                );
+            }
+            Err(GunzipError::Decode) => body_data_raw.clone(),
+        }
     } else {
         body_data_raw.clone()
     };
@@ -2761,14 +2815,16 @@ pub async fn delete_handler(
     // If this is a chunk manifest, delete child chunks first
     if n.is_chunk_manifest() {
         let manifest_data = if n.is_compressed() {
-            use flate2::read::GzDecoder;
-            use std::io::Read as _;
-            let mut decoder = GzDecoder::new(&n.data[..]);
-            let mut decompressed = Vec::new();
-            if decoder.read_to_end(&mut decompressed).is_ok() {
-                decompressed
-            } else {
-                n.data.clone()
+            match maybe_decompress_gzip(&n.data) {
+                Ok(d) => d,
+                Err(GunzipError::TooLarge) => {
+                    return json_error_with_query(
+                        StatusCode::PAYLOAD_TOO_LARGE,
+                        "compressed manifest exceeds decompression limit",
+                        Some(&del_query),
+                    );
+                }
+                Err(GunzipError::Decode) => n.data.clone(),
             }
         } else {
             n.data.clone()
@@ -3124,14 +3180,19 @@ fn try_expand_chunk_manifest(
     last_modified_str: &Option<String>,
 ) -> Option<Response> {
     let data = if n.is_compressed() {
-        use flate2::read::GzDecoder;
-        use std::io::Read as _;
-        let mut decoder = GzDecoder::new(&n.data[..]);
-        let mut decompressed = Vec::new();
-        if decoder.read_to_end(&mut decompressed).is_err() {
-            return None;
+        match maybe_decompress_gzip(&n.data) {
+            Ok(d) => d,
+            Err(GunzipError::TooLarge) => {
+                return Some(
+                    (
+                        StatusCode::PAYLOAD_TOO_LARGE,
+                        "compressed manifest exceeds decompression limit",
+                    )
+                        .into_response(),
+                )
+            }
+            Err(GunzipError::Decode) => return None,
         }
-        decompressed
     } else {
         n.data.clone()
     };
@@ -3141,6 +3202,13 @@ fn try_expand_chunk_manifest(
         Err(_) => return None,
     };
 
+    // Guard the attacker-controlled manifest size before allocating: a negative
+    // value would wrap to a huge usize (capacity-overflow panic) and an oversized
+    // one would OOM-kill the server.
+    if manifest.size < 0 || manifest.size as u64 > MAX_EXPANSION_BYTES {
+        return None;
+    }
+
     // Read and concatenate all chunks
     let mut result = vec![0u8; manifest.size as usize];
     let store = state.store.read().unwrap();
@@ -3174,24 +3242,54 @@ fn try_expand_chunk_manifest(
                 )
             }
         }
-        let chunk_data = if chunk_needle.is_compressed() {
+        // Validate the attacker-controlled chunk offset before indexing: a
+        // negative value would wrap to a huge usize, and an out-of-range one has
+        // nowhere to land.
+        if chunk.offset < 0 {
+            return Some(
+                (
+                    StatusCode::INTERNAL_SERVER_ERROR,
+                    format!("invalid negative chunk offset in {}", chunk.fid),
+                )
+                    .into_response(),
+            );
+        }
+        let offset = chunk.offset as usize;
+        if offset >= result.len() {
+            continue;
+        }
+        // Write the chunk into its window in `result`. Compressed chunks inflate
+        // directly into the destination slice (bounded by the remaining window),
+        // so a chunk never allocates a second large buffer — peak memory stays at
+        // ~manifest.size instead of doubling. Bytes past the window are dropped,
+        // matching the prior truncation behavior.
+        if chunk_needle.is_compressed() {
             use flate2::read::GzDecoder;
             use std::io::Read as _;
             let mut decoder = GzDecoder::new(&chunk_needle.data[..]);
-            let mut decompressed = Vec::new();
-            if decoder.read_to_end(&mut decompressed).is_ok() {
-                decompressed
-            } else {
-                chunk_needle.data.clone()
+            let window = &mut result[offset..];
+            let mut written = 0usize;
+            let mut decode_failed = false;
+            while written < window.len() {
+                match decoder.read(&mut window[written..]) {
+                    Ok(0) => break,
+                    Ok(n) => written += n,
+                    Err(_) => {
+                        decode_failed = true;
+                        break;
+                    }
+                }
+            }
+            // On any decode failure, drop the partial output and fall back to the
+            // chunk's raw bytes (truncated to the window), preserving prior behavior.
+            if decode_failed {
+                window[..written].fill(0);
+                let copy_len = chunk_needle.data.len().min(window.len());
+                window[..copy_len].copy_from_slice(&chunk_needle.data[..copy_len]);
             }
         } else {
-            chunk_needle.data.clone()
-        };
-        let offset = chunk.offset as usize;
-        let end = std::cmp::min(offset + chunk_data.len(), result.len());
-        let copy_len = end - offset;
-        if copy_len > 0 {
-            result[offset..offset + copy_len].copy_from_slice(&chunk_data[..copy_len]);
+            let copy_len = chunk_needle.data.len().min(result.len() - offset);
+            result[offset..offset + copy_len].copy_from_slice(&chunk_needle.data[..copy_len]);
         }
     }
 
@@ -3564,13 +3662,22 @@ fn try_gzip_data(data: &[u8]) -> Option<Vec<u8>> {
     encoder.finish().ok()
 }
 
-fn maybe_decompress_gzip(data: &[u8]) -> Option<Vec<u8>> {
+fn maybe_decompress_gzip(data: &[u8]) -> Result<Vec<u8>, GunzipError> {
     use flate2::read::GzDecoder;
     use std::io::Read;
-    let mut decoder = GzDecoder::new(data);
+    // Cap the output so a crafted, highly-compressible (gzip-bomb) needle cannot
+    // OOM the server when decompressed on read or upload. take(limit+1) lets us
+    // tell "exactly at the limit" apart from "over it", which we reject as
+    // TooLarge so callers fail the request instead of silently using raw bytes.
+    let mut decoder = GzDecoder::new(data).take(MAX_EXPANSION_BYTES + 1);
     let mut decompressed = Vec::new();
-    decoder.read_to_end(&mut decompressed).ok()?;
-    Some(decompressed)
+    decoder
+        .read_to_end(&mut decompressed)
+        .map_err(|_| GunzipError::Decode)?;
+    if decompressed.len() as u64 > MAX_EXPANSION_BYTES {
+        return Err(GunzipError::TooLarge);
+    }
+    Ok(decompressed)
 }
 
 fn compute_md5_base64(data: &[u8]) -> String {
@@ -3812,7 +3919,11 @@ mod tests {
         let compressed = try_gzip_data(data).unwrap();
         let decompressed = maybe_decompress_gzip(&compressed).unwrap();
         assert_eq!(decompressed, data);
-        assert!(maybe_decompress_gzip(data).is_none());
+        // Non-gzip input is reported as a decode error (callers fall back to raw).
+        assert!(matches!(
+            maybe_decompress_gzip(data),
+            Err(GunzipError::Decode)
+        ));
     }
 
     #[test]

seaweed-volume/src/server/store_ec.rs

@@ -70,6 +70,10 @@ struct Snapshot {
     intervals: Vec<IntervalResult>,
     cached_locations: HashMap<ShardId, Vec<String>>,
     cache_refreshed_at: Option<Instant>,
+    /// This volume's encode identity, carried to peers on remote shard reads so a
+    /// shard from a different encode run is rejected rather than served at a
+    /// mismatched offset. 0 for a pre-feature volume (lenient).
+    encode_ts_ns: i64,
 }
 
 /// Top-level entry point. Returns `Ok(None)` for "not found" (matches
@@ -147,6 +151,7 @@ pub async fn read_ec_shard_needle_distributed(
                     &shard_locations,
                     snapshot.data_shards as usize,
                     snapshot.parity_shards as usize,
+                    snapshot.encode_ts_ns,
                 )
                 .await?;
                 assembled.push(buf);
@@ -259,6 +264,7 @@ fn snapshot_under_lock(
         intervals: interval_results,
         cached_locations,
         cache_refreshed_at,
+        encode_ts_ns: ecv.encode_ts_ns,
     }))
 }
 
@@ -383,6 +389,7 @@ async fn fetch_one_interval(
     shard_locations: &HashMap<ShardId, Vec<String>>,
     data_shards: usize,
     parity_shards: usize,
+    expected_encode_ts_ns: i64,
 ) -> io::Result<Vec<u8>> {
     // Direct peer read against the cached locations for this shard.
     if let Some(sources) = shard_locations.get(&shard_id) {
@@ -395,6 +402,7 @@ async fn fetch_one_interval(
                 shard_id,
                 shard_offset,
                 size,
+                expected_encode_ts_ns,
             )
             .await
             {
@@ -424,6 +432,7 @@ async fn fetch_one_interval(
         shard_locations,
         data_shards,
         parity_shards,
+        expected_encode_ts_ns,
     )
     .await
 }
@@ -436,6 +445,7 @@ async fn read_remote_ec_shard_interval(
     shard_id: ShardId,
     shard_offset: i64,
     size: usize,
+    expected_encode_ts_ns: i64,
 ) -> io::Result<Vec<u8>> {
     let mut last_err: Option<io::Error> = None;
     for src in sources {
@@ -447,6 +457,7 @@ async fn read_remote_ec_shard_interval(
             shard_id,
             shard_offset,
             size,
+            expected_encode_ts_ns,
         )
         .await
         {
@@ -470,6 +481,7 @@ async fn do_read_remote_ec_shard_interval(
     shard_id: ShardId,
     shard_offset: i64,
     size: usize,
+    expected_encode_ts_ns: i64,
 ) -> io::Result<Vec<u8>> {
     let grpc_addr =
         parse_grpc_address(source).map_err(|e| io::Error::new(io::ErrorKind::InvalidInput, e))?;
@@ -505,6 +517,7 @@ async fn do_read_remote_ec_shard_interval(
         offset: shard_offset,
         size: size as i64,
         file_key: needle_id.0,
+        encode_ts_ns: expected_encode_ts_ns,
     };
     let resp = client
         .volume_ec_shard_read(Request::new(req))
@@ -523,6 +536,18 @@ async fn do_read_remote_ec_shard_interval(
         .await
         .map_err(|e| io::Error::new(io::ErrorKind::Other, format!("recv: {}", e)))?
     {
+        // Validate the served shard's identity client-side, so the guard holds even
+        // against a pre-upgrade server that ignored the request field (returns 0).
+        // A mismatch fails the read; the caller recovers from parity.
+        if expected_encode_ts_ns != 0 && msg.encode_ts_ns != expected_encode_ts_ns {
+            return Err(io::Error::new(
+                io::ErrorKind::Other,
+                format!(
+                    "ec shard {}.{} from {} belongs to a different encode run (want {} got {})",
+                    vid.0, shard_id, source, expected_encode_ts_ns, msg.encode_ts_ns
+                ),
+            ));
+        }
         if !msg.data.is_empty() {
             out.extend_from_slice(&msg.data);
         }
@@ -554,6 +579,7 @@ async fn recover_one_remote_ec_shard_interval(
     shard_locations: &HashMap<ShardId, Vec<String>>,
     data_shards: usize,
     parity_shards: usize,
+    expected_encode_ts_ns: i64,
 ) -> io::Result<Vec<u8>> {
     let total_shards = data_shards + parity_shards;
     let rs = ReedSolomon::new(data_shards, parity_shards).map_err(|e| {
@@ -614,6 +640,7 @@ async fn recover_one_remote_ec_shard_interval(
                 sid,
                 shard_offset,
                 size,
+                expected_encode_ts_ns,
             )
             .await;
             (sid, res)

seaweed-volume/src/storage/disk_location.rs

@@ -21,7 +21,9 @@ use crate::storage::erasure_coding::ec_volume::EcVolume;
 use crate::storage::needle_map::NeedleMapKind;
 use crate::storage::super_block::ReplicaPlacement;
 use crate::storage::types::*;
-use crate::storage::volume::{remove_volume_files, volume_file_name, Volume, VolumeError};
+use crate::storage::volume::{
+    remove_volume_files, volume_file_name, VifVolumeInfo, Volume, VolumeError,
+};
 
 /// A single disk location managing volumes in one directory.
 pub struct DiskLocation {
@@ -131,8 +133,8 @@ impl DiskLocation {
                     volume_id = vid.0,
                     "volume was not completed: {}, removing files", note
                 );
-                remove_volume_files(&volume_name);
-                remove_volume_files(&idx_name);
+                remove_volume_files(&volume_name, false);
+                remove_volume_files(&idx_name, false);
                 continue;
             }
 
@@ -178,6 +180,20 @@ impl DiskLocation {
                 continue;
             }
 
+            // Load existing data only; never create a phantom `.dat`. A lone
+            // `.vif`/`.idx` (e.g. an EC sidecar whose `.ecx` is on a sibling
+            // disk) would otherwise have Volume::new write an 8-byte stub that
+            // the sibling-.dat prune deletes real shards against. Remote-tiered
+            // volumes also have no local `.dat`, but their `.vif` points at
+            // remote files and must still load via the remote path.
+            let dat_path = format!("{}.dat", volume_name);
+            if !check_dat_file_exists(&dat_path)
+                && !vif_references_remote_file(&format!("{}.vif", volume_name))
+                && !vif_references_remote_file(&format!("{}.vif", idx_name))
+            {
+                continue;
+            }
+
             match Volume::new(
                 &self.directory,
                 &self.idx_directory,
@@ -328,6 +344,37 @@ impl DiskLocation {
         }
     }
 
+    /// Full-teardown variant: everything remove_ec_volume_files clears, PLUS the
+    /// normal-volume <base>.vif on a SHARD-ONLY node. An interrupted shard copy
+    /// (which installs shards + .ecx before .vif) could otherwise mount a fresh
+    /// generation under the prior run's identity / ratio / dat_file_size carried by
+    /// a stale .vif. Gated on .idx absence so a source-volume holder keeps its live
+    /// .vif. Reconcile/load-fallback call remove_ec_volume_files directly and
+    /// intentionally preserve it, mirroring Go's removeEcVolumeFiles (reconcile) vs
+    /// removeStaleEcArtifacts (teardown) split.
+    pub(crate) fn remove_ec_volume_files_full_teardown(&self, collection: &str, vid: VolumeId) {
+        self.remove_ec_volume_files(collection, vid);
+        let base = volume_file_name(&self.directory, collection, vid);
+        let idx_base = volume_file_name(&self.idx_directory, collection, vid);
+        // try_exists, not exists(): a stat error on a PRESENT .idx must not read as
+        // "shard-only" and delete a live source volume's .vif. Only Ok(false) (genuine
+        // NotFound) clears it; on a stat error stay conservative and keep the sidecar.
+        if matches!(
+            std::path::Path::new(&format!("{}.idx", idx_base)).try_exists(),
+            Ok(false)
+        ) {
+            let _ = fs::remove_file(format!("{}.vif", idx_base));
+            if self.idx_directory != self.directory
+                && matches!(
+                    std::path::Path::new(&format!("{}.idx", base)).try_exists(),
+                    Ok(false)
+                )
+            {
+                let _ = fs::remove_file(format!("{}.vif", base));
+            }
+        }
+    }
+
     /// Find a volume by ID.
     pub fn find_volume(&self, vid: VolumeId) -> Option<&Volume> {
         self.volumes.get(&vid)
@@ -1018,6 +1065,17 @@ fn check_dat_file_exists(path: &str) -> bool {
     }
 }
 
+/// True when a `.vif` references remote-tier files: a remote-only volume
+/// that has no local `.dat` but must still load via the remote path,
+/// rather than be skipped as a lone EC sidecar.
+fn vif_references_remote_file(vif_path: &str) -> bool {
+    fs::read_to_string(vif_path)
+        .ok()
+        .and_then(|s| serde_json::from_str::<VifVolumeInfo>(&s).ok())
+        .map(|vif| !vif.files.is_empty())
+        .unwrap_or(false)
+}
+
 fn parse_volume_filename(filename: &str) -> Option<(String, VolumeId)> {
     let stem = filename
         .strip_suffix(".dat")

seaweed-volume/src/storage/erasure_coding/ec_volume.rs

@@ -57,6 +57,10 @@ pub struct EcVolume {
     pub shard_locations_refresh_time: std::sync::Mutex<Option<std::time::Instant>>,
     /// EC volume expiration time (unix epoch seconds), set during EC encode from TTL.
     pub expire_at_sec: u64,
+    /// Encode-run identity (unix nanos) loaded from the .vif EcShardConfig. A read
+    /// served from a shard of a different encode run is rejected (server- and
+    /// client-side); 0 for a pre-feature volume, which is treated leniently.
+    pub encode_ts_ns: i64,
 }
 
 /// Locate the `.vif` for a (collection, vid) by preferring the data dir
@@ -137,7 +141,7 @@ impl EcVolume {
         // for shard-size math and by the Store-level prune in
         // `store_ec_reconcile.rs` to verify a sibling-disk .dat is
         // plausibly the encoding source (#9478).
-        let (expire_at_sec, vif_version, vif_dat_file_size) = {
+        let (expire_at_sec, vif_version, vif_dat_file_size, encode_ts_ns) = {
             let vif_path = locate_vif_path(dir, dir_idx, collection, volume_id);
             if let Ok(vif_content) = std::fs::read_to_string(&vif_path) {
                 if let Ok(vif_info) =
@@ -148,12 +152,21 @@ impl EcVolume {
                     } else {
                         Version::current()
                     };
-                    (vif_info.expire_at_sec, ver, vif_info.dat_file_size)
+                    let cfg_encode_ts_ns = vif_info
+                        .ec_shard_config
+                        .as_ref()
+                        .map_or(0, |c| c.encode_ts_ns);
+                    (
+                        vif_info.expire_at_sec,
+                        ver,
+                        vif_info.dat_file_size,
+                        cfg_encode_ts_ns,
+                    )
                 } else {
-                    (0, Version::current(), 0)
+                    (0, Version::current(), 0, 0)
                 }
             } else {
-                (0, Version::current(), 0)
+                (0, Version::current(), 0, 0)
             }
         };
 
@@ -177,6 +190,7 @@ impl EcVolume {
             shard_locations: std::sync::RwLock::new(HashMap::new()),
             shard_locations_refresh_time: std::sync::Mutex::new(None),
             expire_at_sec,
+            encode_ts_ns,
         };
 
         // Open .ecx file (sorted index) in read/write mode for in-place deletion marking.
@@ -605,7 +619,19 @@ impl EcVolume {
                 })?;
 
             let mut buf = vec![0u8; interval.size as usize];
-            shard.read_at(&mut buf, shard_offset as u64)?;
+            let n = shard.read_at(&mut buf, shard_offset as u64)?;
+            if n != buf.len() {
+                return Err(io::Error::new(
+                    io::ErrorKind::UnexpectedEof,
+                    format!(
+                        "short read on ec shard {}: read {} of {} bytes for needle {}",
+                        shard_id,
+                        n,
+                        buf.len(),
+                        needle_id
+                    ),
+                ));
+            }
             bytes.extend_from_slice(&buf);
         }
 
@@ -1243,6 +1269,7 @@ mod tests {
             ec_shard_config: Some(crate::storage::volume::VifEcShardConfig {
                 data_shards: 6,
                 parity_shards: 3,
+                ..Default::default()
             }),
             ..Default::default()
         };
@@ -1268,6 +1295,7 @@ mod tests {
             ec_shard_config: Some(crate::storage::volume::VifEcShardConfig {
                 data_shards: 10,
                 parity_shards: 10,
+                ..Default::default()
             }),
             ..Default::default()
         };

seaweed-volume/src/storage/store.rs

@@ -598,8 +598,16 @@ impl Store {
                     as i32;
                 let mut max_count = vol_count + ec_equivalent;
 
-                if unclaimed > volume_size_limit as i64 {
-                    max_count += (unclaimed as u64 / volume_size_limit) as i32 - 1;
+                // One slot per full volume that fits in the unclaimed space.
+                // A "- 1" here used to zero the count when the disk had room for
+                // exactly one volume (free between 1x and 2x the limit), stranding
+                // auto-sized disks at max_volume_count 0 with no writable volume.
+                if unclaimed > 0 {
+                    max_count += (unclaimed as u64 / volume_size_limit) as i32;
+                }
+                // An auto-sized disk with free space always hosts at least one volume.
+                if max_count < 1 {
+                    max_count = 1;
                 }
 
                 loc.max_volume_count.store(max_count, Ordering::Relaxed);
@@ -1462,6 +1470,47 @@ mod tests {
         assert!(with_preallocate > without_preallocate);
     }
 
+    #[test]
+    fn test_maybe_adjust_volume_max_no_dead_zone() {
+        // With auto max (original_max_volume_count == 0) and a large volume size
+        // limit, a disk with room for at least one volume must not report 0
+        // slots. It once did so for disks sized between 1x and 2x the limit,
+        // leaving no writable volume and timing out every assign.
+        let tmp = TempDir::new().unwrap();
+        let dir = tmp.path().to_str().unwrap();
+
+        let (_, free) = crate::storage::disk_location::get_disk_stats(dir);
+        if free < 4 {
+            return; // cannot determine free disk space
+        }
+
+        let mut store = Store::new(NeedleMapKind::InMemory);
+        store
+            .add_location(
+                dir,
+                dir,
+                0, // auto
+                DiskType::HardDrive,
+                MinFreeSpace::Percent(1.0),
+                Vec::new(),
+            )
+            .unwrap();
+
+        // free disk is 1.5x the limit: room for one full volume, less than two.
+        let volume_size_limit = free * 2 / 3;
+        store
+            .volume_size_limit
+            .store(volume_size_limit, Ordering::Relaxed);
+
+        store.maybe_adjust_volume_max();
+
+        let max = store.locations[0].max_volume_count.load(Ordering::Relaxed);
+        assert!(
+            max >= 1,
+            "auto-sized disk with room for one volume reported max_volume_count={max}; want >= 1"
+        );
+    }
+
     #[test]
     fn test_find_free_location_predicate_prefers_more_capacity_and_skips_low_disk() {
         let tmp1 = TempDir::new().unwrap();

seaweed-volume/src/storage/store_ec_mirror.rs

@@ -327,6 +327,7 @@ mod tests {
             ec_shard_config: Some(VifEcShardConfig {
                 data_shards,
                 parity_shards,
+                ..Default::default()
             }),
             ..Default::default()
         };

seaweed-volume/src/storage/store_ec_reconcile.rs

@@ -203,14 +203,11 @@ impl Store {
     /// this server) also fall through unchanged because the `.dat`
     /// index never matches.
     ///
-    /// Before deleting any EC files we also check that the sibling
-    /// `.dat` is plausibly the encoding source: at least
-    /// `SUPER_BLOCK_SIZE` bytes long, and — when the EC's `.vif`
-    /// recorded a non-zero source size in `dat_file_size` — at least
-    /// that many bytes. A zero-byte shell or a truncated `.dat` does
-    /// not justify wiping the partial EC, because that EC shard may
-    /// still combine usefully with shards on other servers in a
-    /// recoverable distributed-EC layout.
+    /// The sibling `.dat` must be a credible encoding source before we
+    /// delete anything: at least the size `.vif` recorded at encode time,
+    /// or — when unknown (0) — more than a bare superblock so an empty
+    /// 8-byte stub can't pass. A truncated `.dat` leaves the partial EC
+    /// alone; those shards may still reconstruct from other servers.
     ///
     /// We don't have to push anything to a deleted-shards channel
     /// here: the Rust heartbeat path in `server/heartbeat.rs` diffs
@@ -257,14 +254,13 @@ impl Store {
                     // per-disk pass; don't second-guess it here.
                     continue;
                 }
-                // Decide whether the sibling .dat is credible. Prefer
-                // the size baked into .vif at encode time; fall back
-                // to "at least a superblock" for old EC volumes whose
-                // .vif predates the field.
+                // Credible source size: prefer .vif's encode-time size; when
+                // unknown (0) require more than a bare superblock so an empty
+                // 8-byte stub (e.g. a phantom .dat) can't pass.
                 let required = if ev.dat_file_size > 0 {
                     ev.dat_file_size as u64
                 } else {
-                    SUPER_BLOCK_SIZE as u64
+                    SUPER_BLOCK_SIZE as u64 + 1
                 };
                 if owner.size < required {
                     warn!(
@@ -459,6 +455,7 @@ mod tests {
             ec_shard_config: Some(VifEcShardConfig {
                 data_shards,
                 parity_shards,
+                ..Default::default()
             }),
             ..Default::default()
         };
@@ -469,6 +466,97 @@ mod tests {
         .unwrap();
     }
 
+    /// Regression: a lone `.vif` whose `.ecx` is on a sibling disk must not
+    /// make the loader create a phantom `.dat`, nor the sibling-.dat prune
+    /// delete the real shards on the sibling.
+    #[test]
+    fn test_lone_vif_does_not_create_phantom_dat_or_delete_shards() {
+        let tmp = TempDir::new().unwrap();
+        let d0 = tmp.path().join("data0");
+        let d1 = tmp.path().join("data1");
+        std::fs::create_dir_all(&d0).unwrap();
+        std::fs::create_dir_all(&d1).unwrap();
+        let coll = "warp-loadtest";
+        let vid = 57u32;
+
+        // d0: a self-contained but partial (2 < 10) EC volume.
+        write_shard(d0.to_str().unwrap(), coll, vid, 2);
+        write_shard(d0.to_str().unwrap(), coll, vid, 4);
+        write_index_files(d0.to_str().unwrap(), coll, vid, 10, 4);
+
+        // d1: ONLY the mirrored `.vif` — no `.ecx`, no shard, no `.dat`.
+        std::fs::copy(
+            d0.join(format!("{}_{}.vif", coll, vid)),
+            d1.join(format!("{}_{}.vif", coll, vid)),
+        )
+        .unwrap();
+
+        let mut store = Store::new(NeedleMapKind::InMemory);
+        for d in [&d0, &d1] {
+            store
+                .add_location(
+                    d.to_str().unwrap(),
+                    d.to_str().unwrap(),
+                    100,
+                    DiskType::HardDrive,
+                    MinFreeSpace::Percent(0.0),
+                    Vec::new(),
+                )
+                .unwrap();
+        }
+
+        let phantom = d1.join(format!("{}_{}.dat", coll, vid));
+        assert!(
+            !phantom.exists(),
+            "loader created a phantom .dat from a lone .vif"
+        );
+        let total: usize = store.locations.iter().map(|l| l.ec_shard_count()).sum();
+        assert_eq!(total, 2, "real EC shards were deleted (total={})", total);
+    }
+
+    /// Regression for the prune credibility gate: when the EC `.vif`
+    /// records no source size (`dat_file_size == 0`), an empty 8-byte
+    /// sibling `.dat` stub must NOT justify deleting partial EC shards.
+    #[test]
+    fn test_prune_refuses_unverifiable_8byte_dat() {
+        let tmp = TempDir::new().unwrap();
+        let d0 = tmp.path().join("data0");
+        let d1 = tmp.path().join("data1");
+        std::fs::create_dir_all(&d0).unwrap();
+        std::fs::create_dir_all(&d1).unwrap();
+        let coll = "warp-loadtest";
+        let vid = 99u32;
+
+        // d0: partial EC (2 shards); its `.vif` records no source size.
+        write_shard(d0.to_str().unwrap(), coll, vid, 0);
+        write_shard(d0.to_str().unwrap(), coll, vid, 1);
+        write_index_files(d0.to_str().unwrap(), coll, vid, 10, 4);
+
+        // d1: a real but empty 8-byte (superblock-sized) `.dat` stub.
+        std::fs::write(d1.join(format!("{}_{}.dat", coll, vid)), vec![0u8; 8]).unwrap();
+
+        let mut store = Store::new(NeedleMapKind::InMemory);
+        for d in [&d0, &d1] {
+            store
+                .add_location(
+                    d.to_str().unwrap(),
+                    d.to_str().unwrap(),
+                    100,
+                    DiskType::HardDrive,
+                    MinFreeSpace::Percent(0.0),
+                    Vec::new(),
+                )
+                .unwrap();
+        }
+
+        let total: usize = store.locations.iter().map(|l| l.ec_shard_count()).sum();
+        assert_eq!(
+            total, 2,
+            "prune deleted shards against an unverifiable 8-byte .dat (total={})",
+            total
+        );
+    }
+
     /// Reproduces the orphan-shard layout from issue #9212. Shards live
     /// on dir0; the .ecx / .ecj / .vif live on dir1. Without
     /// reconciliation, dir0's shards are silently dropped at startup.
@@ -966,15 +1054,15 @@ mod tests {
         std::fs::create_dir_all(&dat_dir).unwrap();
         std::fs::create_dir_all(&ec_dir).unwrap();
 
-        let collection = "";
+        // Real collection so the loader's volume_file_name-based `.ecx`
+        // lookup matches the helpers (empty collection emits `_122.*` vs the
+        // expected `122.*`).
+        let collection = "pics";
         let vid = 122u32;
 
-        // Disk A (sdd): a non-trivial .dat plus the regular-volume
-        // sidecars. The .dat content doesn't matter for the prune —
-        // load_existing_volumes refuses to mount it because the
-        // superblock is absent — but its file name has to be present
-        // so index_dat_owners records this disk as the .dat owner.
-        let dat_path = dat_dir.join(format!("{}_{}.dat", collection, vid).trim_start_matches('_'));
+        // Disk A (sdd): a .dat whose name must be present so index_dat_owners
+        // records this disk as the .dat owner (content doesn't matter).
+        let dat_path = dat_dir.join(format!("{}_{}.dat", collection, vid));
         std::fs::write(&dat_path, vec![0u8; 1024]).unwrap();
 
         // Disk B (sdf): partial EC — one shard, plus .ecx / .ecj / .vif.
@@ -1078,6 +1166,7 @@ mod tests {
             ec_shard_config: Some(VifEcShardConfig {
                 data_shards: 10,
                 parity_shards: 4,
+                ..Default::default()
             }),
             ..Default::default()
         };

seaweed-volume/src/storage/volume.rs

@@ -189,6 +189,10 @@ pub struct VifEcShardConfig {
     pub data_shards: u32,
     #[serde(default, rename = "parityShards")]
     pub parity_shards: u32,
+    /// Encode time (unix nanos). Shards and .ecx of one encode run share it,
+    /// so a read served from a different run's shard is rejected.
+    #[serde(default, rename = "encodeTsNs", with = "string_or_i64")]
+    pub encode_ts_ns: i64,
 }
 
 /// Serde-compatible representation of OldVersionVolumeInfo for legacy .vif JSON deserialization.
@@ -279,6 +283,7 @@ impl VifVolumeInfo {
             ec_shard_config: pb.ec_shard_config.as_ref().map(|c| VifEcShardConfig {
                 data_shards: c.data_shards,
                 parity_shards: c.parity_shards,
+                encode_ts_ns: c.encode_ts_ns,
             }),
         }
     }
@@ -309,6 +314,7 @@ impl VifVolumeInfo {
                 crate::pb::volume_server_pb::EcShardConfig {
                     data_shards: c.data_shards,
                     parity_shards: c.parity_shards,
+                    encode_ts_ns: c.encode_ts_ns,
                 }
             }),
         }
@@ -3071,11 +3077,15 @@ impl Volume {
                 let bytes = fake_del_needle.write_bytes(version);
                 dst_dat.write_all(&bytes)?;
 
+                // Record the tombstone's real .dat offset, like the normal delete path,
+                // so a deletion left at the .dat tail stays visible to the integrity
+                // check on reload. Offset 0 hid the trailing tombstone and falsely
+                // flipped the volume read-only.
                 let mut idx_entry_buf = [0u8; NEEDLE_MAP_ENTRY_SIZE];
                 crate::storage::types::idx_entry_to_bytes(
                     &mut idx_entry_buf,
                     key,
-                    Offset::from_actual_offset(0),
+                    Offset::from_actual_offset(dat_offset as i64),
                     Size(crate::storage::types::TOMBSTONE_FILE_SIZE.into()),
                 );
                 dst_idx.write_all(&idx_entry_buf)?;
@@ -3157,12 +3167,34 @@ impl Volume {
             }
         }
 
+        // A regular volume and an EC volume for the same id share <base>.vif.
+        // When EC artefacts coexist on this disk (e.g. shards distributed onto
+        // a source replica before it is deleted), keep the .vif so removing the
+        // regular volume does not strip the EC volume's info file.
+        let keep_vif = self.shares_vif_with_ec_volume();
         self.close();
-        remove_volume_files(&self.data_file_name());
-        remove_volume_files(&self.index_file_name());
+        remove_volume_files(&self.data_file_name(), keep_vif);
+        remove_volume_files(&self.index_file_name(), keep_vif);
         Ok(())
     }
 
+    /// Reports whether an EC volume for this id has a sealed .ecx on the same
+    /// disk, in which case its .vif is the same file as the regular volume's
+    /// and must outlive the regular volume's deletion. Mirrors the on-disk
+    /// portion of Go's Volume.sharesVifWithEcVolume / HasEcxFileOnDisk.
+    fn shares_vif_with_ec_volume(&self) -> bool {
+        let has_ecx = |base: &str| -> bool {
+            fs::metadata(format!("{}.ecx", base))
+                .map(|m| !m.is_dir() && m.len() > 0)
+                .unwrap_or(false)
+        };
+        if has_ecx(&volume_file_name(&self.dir_idx, &self.collection, self.id)) {
+            return true;
+        }
+        self.dir != self.dir_idx
+            && has_ecx(&volume_file_name(&self.dir, &self.collection, self.id))
+    }
+
     /// Check if an I/O error is EIO (errno 5) and record it for health monitoring.
     /// On success (None), clears any previously recorded EIO error.
     /// Matches Go's `checkReadWriteError` in volume_write.go.
@@ -3231,10 +3263,13 @@ fn get_append_at_ns(last: u64) -> u64 {
 
 /// Remove all files associated with a volume.
 /// .dat/.idx removals log at info level so destructive calls are traceable.
-pub(crate) fn remove_volume_files(base: &str) {
+pub(crate) fn remove_volume_files(base: &str, keep_vif: bool) {
     for ext in &[
         ".dat", ".idx", ".vif", ".sdx", ".cpd", ".cpx", ".note", ".rdb",
     ] {
+        if *ext == ".vif" && keep_vif {
+            continue;
+        }
         let path = format!("{}{}", base, ext);
         let size = fs::metadata(&path).map(|m| m.len()).unwrap_or(0);
         let existed = fs::remove_file(&path).is_ok();
@@ -4740,4 +4775,43 @@ mod tests {
             ".vif removed from data dir"
         );
     }
+
+    /// When an EC volume for the same id has a sealed .ecx on the same disk, the
+    /// .vif is shared with it and must survive the regular volume's deletion.
+    #[test]
+    fn test_destroy_keeps_vif_when_ec_coexists() {
+        let tmp = TempDir::new().unwrap();
+        let dir = tmp.path().to_str().unwrap();
+
+        let mut v = make_test_volume(dir);
+        let mut n = Needle {
+            id: NeedleId(1),
+            cookie: Cookie(1),
+            data: b"test".to_vec(),
+            data_size: 4,
+            ..Needle::default()
+        };
+        v.write_needle(&mut n, true).unwrap();
+
+        let vif_path = format!("{}/1.vif", dir);
+        std::fs::write(&vif_path, r#"{"version":3}"#).unwrap();
+        // A sealed .ecx marks a coexisting EC volume for the same id.
+        let ecx_path = format!("{}/1.ecx", dir);
+        std::fs::write(&ecx_path, b"ec-index").unwrap();
+
+        v.destroy(false, false).unwrap();
+
+        let dat_path = format!("{}/1.dat", dir);
+        let idx_path = format!("{}/1.idx", dir);
+        assert!(!std::path::Path::new(&dat_path).exists(), ".dat removed");
+        assert!(!std::path::Path::new(&idx_path).exists(), ".idx removed");
+        assert!(
+            std::path::Path::new(&vif_path).exists(),
+            ".vif kept: shared with the coexisting EC volume"
+        );
+        assert!(
+            std::path::Path::new(&ecx_path).exists(),
+            ".ecx is an EC sidecar, never touched here"
+        );
+    }
 }

terraform/.gitignore

@@ -0,0 +1,15 @@
+# Terraform / OpenTofu working files
+.terraform/
+.terraform.lock.hcl
+*.tfstate
+*.tfstate.*
+crash.log
+crash.*.log
+*.tfvars
+!*.tfvars.example
+override.tf
+override.tf.json
+*_override.tf
+*_override.tf.json
+.terraformrc
+terraform.rc

terraform/README.md

@@ -0,0 +1,128 @@
+# SeaweedFS on Terraform
+
+> **Experimental.** This Terraform support is an early scaffold under active
+> development. Interfaces (variables, outputs, module layout) may change without
+> notice, and not every tier is implemented yet. Not recommended for production
+> without your own review and testing.
+
+Self-contained Terraform/OpenTofu modules to deploy SeaweedFS on cloud VMs
+running the `weed` binary directly under systemd. No Helm and no Kubernetes
+required.
+
+What works today is verified end-to-end against a real `weed` cluster (see
+"Test it locally" below).
+
+## Layout
+
+```text
+terraform/
+  modules/
+    core/        cloud-agnostic renderer (ZERO cloud resources): turns an
+                 address map + config into per-node weed argv, systemd units,
+                 config files, disk-mount + secret-fetch scripts, and cloud-init.
+                 Both the cloud wrappers and the local harnesses consume `nodes`.
+    security/    cloud-agnostic CA + per-component mTLS certs (distinct CNs) +
+                 JWT signing keys (tls/random providers). Emits a `core_security`
+                 object ready for the core, plus PEMs for secret-store delivery.
+    aws/         thin AWS wrapper: reserves stable ENIs (fixed private IPs)
+                 first, feeds them to core, creates instances + protected EBS
+                 data disks + SG; with enable_security it generates certs/JWT,
+                 stores them in SSM SecureString, grants an instance role, and
+                 renders a boot fetch-secrets.sh. Keyed for_each throughout.
+  examples/
+    aws-ha-distributed/   3 masters + 3 volumes (1/AZ) + 2 filers + 1 S3,
+                          secure-by-default (mTLS via SSM-delivered certs)
+    aws-all-in-one/       single `weed server` instance via core directly
+  test/
+    local/         render a cluster with core and run it as real weed processes
+                   on 127.0.0.1 (no cloud, no docker), then assert it works
+    local-secure/  generate certs/JWT with the security module, run a real
+                   mTLS cluster, and assert it forms + enforces JWT auth
+```
+
+## Design in one paragraph
+
+The chart is the structural reference, not a dependency. A cloud-agnostic
+**core** renders everything portable; thin per-cloud wrappers provision infra.
+Addressing is an **input** to the core (wrapper reserves static IPs first), so
+the wrapper -> core dependency is one-way with no apply-time cycle. Stateful
+tiers (master/volume/filer) are keyed `for_each` maps, never `count`, so a
+middle node can be replaced without reindexing its peers or reattaching the
+wrong disk. Flag names are verified against the real `weed` binary
+(notably: volume uses `-mserver` for the master list; gRPC is `-port.grpc`,
+auto = http+10000; `minFreeSpacePercent` is a string).
+
+## Test it locally
+
+Requires `tofu` (or `terraform`), `jq`, `curl`, and a `weed` binary. Renders a
+3-master + volume + filer + S3 cluster from the core module and runs it as real
+processes, asserting quorum, volume registration, and filer/S3 round-trips:
+
+```bash
+cd terraform/test/local
+WEED=/path/to/weed ./run_local_cluster.sh
+# => 7 passed, 0 failed
+```
+
+The harness uses a high port range (29333/28080/28888/28333) so it does not
+collide with a SeaweedFS cluster already running on the machine, and aborts if a
+required port is taken. `KEEP=1 ./run_local_cluster.sh` leaves the cluster up.
+
+### mTLS end-to-end
+
+```bash
+cd terraform/test/local-secure
+WEED=/path/to/weed ./run_local_secure.sh
+# => generates a CA + component certs + JWT, renders security.toml, runs a real
+#    mTLS cluster, asserts master/volume/filer form over mutual TLS and that the
+#    filer enforces JWT signing (unsigned writes get 401). 5 passed.
+```
+
+## Plan-level tests (no cloud)
+
+```bash
+cd terraform/modules/core && tofu test
+# => 11 passed: peers list, -mserver vs -master, metrics gating,
+#    security.toml conditions, all-in-one inheritance, ...
+```
+
+## Validate the cloud wrappers
+
+```bash
+cd terraform/examples/aws-ha-distributed && tofu init && tofu validate
+```
+
+`apply` needs AWS credentials, a VPC, subnets, and an AMI with `weed` installed
+(bake with Packer, or install at boot).
+
+## Status
+
+Implemented and verified:
+- Tiers: master / volume / filer / s3 / all-in-one rendered by the core.
+- Disk mount: cloud-init runs a `mount-disks.sh` that auto-discovers (or takes
+  explicit candidate devices), `blkid`-guards `mkfs`, mounts, and persists to
+  `/etc/fstab` by UUID. The AWS wrapper wires the protected EBS disk to `/data`.
+- mTLS + JWT: the `security` module generates the CA + per-component certs
+  (distinct CNs) + JWT keys; the AWS wrapper stores them in SSM SecureString,
+  grants an instance role, and the core renders a boot fetch script so secrets
+  stay OUT of user_data. Proven end-to-end by `test/local-secure`.
+
+Not yet implemented:
+- sftp / admin / worker tiers (core does not render them yet).
+- GCP/Azure wrappers, the data-plane provider, and the K8s-native module.
+- CA in Vault PKI (currently TF-generated, so the CA key lives in state). KMS
+  decrypt in the IAM policy is scoped by
+  `ViaService` but uses `Resource="*"`; tighten to the SSM CMK in production.
+
+## Gotchas
+
+- This `weed` build starts an Iceberg REST catalog on `:8181` by default; set
+  `s3.iceberg_port = 0` to disable, or a port to relocate it.
+- Disk auto-discovery assumes a single attached data disk per node. For multiple
+  data disks, pass explicit `devices` candidates in `disk_mounts`.
+
+## Security note
+
+`tofu output`/state can contain secrets. Generate secrets outside Terraform
+(cloud secret manager / Vault) and fetch them at boot; never commit secret
+material.

terraform/examples/aws-all-in-one/main.tf

@@ -0,0 +1,133 @@
+# SeaweedFS all-in-one on a single AWS instance.
+#
+# Demonstrates the layered design directly: call the cloud-agnostic core to
+# render cloud-init for a `weed server` node, then attach it to one instance
+# with a protected data disk. No wrapper module needed for the simple case.
+#
+#   tofu init && tofu validate
+#   tofu apply   # requires AWS credentials + a weed AMI
+
+terraform {
+  required_version = ">= 1.3.0"
+  required_providers {
+    aws = {
+      source  = "hashicorp/aws"
+      version = ">= 5.40"
+    }
+  }
+}
+
+provider "aws" {
+  region = var.region
+}
+
+variable "region" {
+  type    = string
+  default = "us-east-1"
+}
+variable "vpc_id" {
+  type = string
+}
+variable "subnet_id" {
+  type = string
+}
+variable "availability_zone" {
+  type    = string
+  default = "us-east-1a"
+}
+variable "ami_id" {
+  type = string
+}
+variable "private_ip" {
+  type    = string
+  default = "10.0.1.50"
+}
+
+module "core" {
+  source      = "../../modules/core"
+  weed_binary = "/usr/bin/weed"
+
+  # disable the distributed tiers; run a single all-in-one process
+  master = { enabled = false }
+  volume = { enabled = false }
+  filer  = { enabled = false }
+
+  all_in_one = {
+    enabled = true
+    nodes   = { a0 = { address = var.private_ip, data_dir = "/data" } }
+    s3      = { enabled = true }
+  }
+
+  s3_identities = [{
+    name       = "anonymous"
+    access_key = ""
+    secret_key = ""
+    actions    = ["Read", "List"]
+  }]
+}
+
+resource "aws_network_interface" "aio" {
+  subnet_id       = var.subnet_id
+  private_ips     = [var.private_ip]
+  security_groups = [aws_security_group.aio.id]
+  tags            = { Name = "seaweedfs-all-in-one" }
+}
+
+resource "aws_security_group" "aio" {
+  name_prefix = "seaweedfs-aio-"
+  vpc_id      = var.vpc_id
+  lifecycle {
+    create_before_destroy = true
+  }
+}
+
+resource "aws_vpc_security_group_ingress_rule" "s3" {
+  security_group_id = aws_security_group.aio.id
+  cidr_ipv4         = "10.0.0.0/8"
+  ip_protocol       = "tcp"
+  from_port         = 8333
+  to_port           = 8333
+}
+
+resource "aws_vpc_security_group_egress_rule" "all" {
+  security_group_id = aws_security_group.aio.id
+  cidr_ipv4         = "0.0.0.0/0"
+  ip_protocol       = "-1"
+}
+
+resource "aws_instance" "aio" {
+  ami           = var.ami_id
+  instance_type = "m5.large"
+  user_data     = module.core.cloud_init_by_node["all-in-one-a0"]
+
+  network_interface {
+    network_interface_id = aws_network_interface.aio.id
+    device_index         = 0
+  }
+  metadata_options {
+    http_tokens = "required"
+  }
+  tags = { Name = "seaweedfs-all-in-one", Role = "all-in-one" }
+}
+
+resource "aws_ebs_volume" "data" {
+  availability_zone = var.availability_zone
+  size              = 200
+  type              = "gp3"
+  encrypted         = true
+  tags              = { Name = "seaweedfs-all-in-one-data" }
+  lifecycle {
+    prevent_destroy = true
+  }
+}
+
+resource "aws_volume_attachment" "data" {
+  device_name                    = "/dev/xvdf"
+  volume_id                      = aws_ebs_volume.data.id
+  instance_id                    = aws_instance.aio.id
+  stop_instance_before_detaching = true
+}
+
+output "instance_id" {
+  value = aws_instance.aio.id
+}

terraform/examples/aws-ha-distributed/main.tf

@@ -0,0 +1,117 @@
+# SeaweedFS HA on AWS: 3-master quorum + 3 volume servers (one per AZ) + 2
+# filers (leveldb2-replicated HA) + 1 standalone S3 gateway.
+#
+#   tofu init && tofu validate
+#   tofu apply   # requires AWS credentials, a VPC, subnets, and a weed AMI
+#
+# This is a scaffold: it provisions instances, protected EBS data disks, and the
+# security group. Mounting the EBS disk at /data and secret-store cert delivery
+# are documented follow-ups (see terraform/README.md).
+
+terraform {
+  required_version = ">= 1.3.0"
+  required_providers {
+    aws = {
+      source  = "hashicorp/aws"
+      version = ">= 5.40"
+    }
+  }
+}
+
+provider "aws" {
+  region = var.region
+}
+
+variable "region" {
+  type    = string
+  default = "us-east-1"
+}
+
+variable "vpc_id" {
+  type = string
+}
+
+variable "ami_id" {
+  description = "AMI with the weed binary at /usr/bin/weed."
+  type        = string
+}
+
+# subnet per AZ
+variable "subnet_a" { type = string }
+variable "subnet_b" { type = string }
+variable "subnet_c" { type = string }
+
+# AZ defaults assume region us-east-1. When you change `region`, override az_a/az_b/az_c
+# (and the matching subnets) with AZs that belong to that region, or the volume/filer
+# EBS volumes will fail to create in a mismatched AZ.
+variable "az_a" {
+  type    = string
+  default = "us-east-1a"
+}
+variable "az_b" {
+  type    = string
+  default = "us-east-1b"
+}
+variable "az_c" {
+  type    = string
+  default = "us-east-1c"
+}
+
+variable "client_ingress_cidrs" {
+  type    = list(string)
+  default = ["10.0.0.0/8"]
+}
+
+variable "ssh_ingress_cidrs" {
+  type    = list(string)
+  default = []
+}
+
+module "seaweedfs" {
+  source = "../../modules/aws"
+
+  name   = "seaweedfs"
+  vpc_id = var.vpc_id
+  ami_id = var.ami_id
+
+  # secure-by-default flagship: mTLS (certs + JWT generated by the security
+  # submodule, delivered via SSM and fetched at boot) plus monitoring.
+  enable_security    = true
+  monitoring_enabled = true
+
+  masters = {
+    m0 = { subnet_id = var.subnet_a, private_ip = "10.0.1.10" }
+    m1 = { subnet_id = var.subnet_b, private_ip = "10.0.2.10" }
+    m2 = { subnet_id = var.subnet_c, private_ip = "10.0.3.10" }
+  }
+
+  volumes = {
+    v0 = { subnet_id = var.subnet_a, availability_zone = var.az_a, private_ip = "10.0.1.20", rack = var.az_a, data_center = var.region, data_volume_size_gb = 500 }
+    v1 = { subnet_id = var.subnet_b, availability_zone = var.az_b, private_ip = "10.0.2.20", rack = var.az_b, data_center = var.region, data_volume_size_gb = 500 }
+    v2 = { subnet_id = var.subnet_c, availability_zone = var.az_c, private_ip = "10.0.3.20", rack = var.az_c, data_center = var.region, data_volume_size_gb = 500 }
+  }
+
+  filers = {
+    f0 = { subnet_id = var.subnet_a, availability_zone = var.az_a, private_ip = "10.0.1.30" }
+    f1 = { subnet_id = var.subnet_b, availability_zone = var.az_b, private_ip = "10.0.2.30" }
+  }
+
+  s3_nodes = {
+    s0 = { subnet_id = var.subnet_a, private_ip = "10.0.1.40" }
+  }
+
+  client_ingress_cidrs = var.client_ingress_cidrs
+  ssh_ingress_cidrs    = var.ssh_ingress_cidrs
+}
+
+output "master_peers" {
+  value = module.seaweedfs.master_peers
+}
+
+output "instance_ids" {
+  value = module.seaweedfs.instance_ids
+}
+
+output "security_group_id" {
+  value = module.seaweedfs.security_group_id
+}

terraform/modules/aws/main.tf

@@ -0,0 +1,205 @@
+# ---- step 1: reserve stable addressing (ENIs with fixed private IPs) -------
+# These exist BEFORE the core renders config, so the address map is known and
+# the core <- wrapper dependency is one-way (no apply-time cycle).
+
+resource "aws_network_interface" "master" {
+  for_each        = var.masters
+  subnet_id       = each.value.subnet_id
+  private_ips     = [each.value.private_ip]
+  security_groups = [aws_security_group.cluster.id]
+  tags            = merge(var.tags, { Name = "${var.name}-master-${each.key}", Role = "master" })
+}
+
+resource "aws_network_interface" "volume" {
+  for_each        = var.volumes
+  subnet_id       = each.value.subnet_id
+  private_ips     = [each.value.private_ip]
+  security_groups = [aws_security_group.cluster.id]
+  tags            = merge(var.tags, { Name = "${var.name}-volume-${each.key}", Role = "volume" })
+}
+
+resource "aws_network_interface" "filer" {
+  for_each        = var.filers
+  subnet_id       = each.value.subnet_id
+  private_ips     = [each.value.private_ip]
+  security_groups = [aws_security_group.cluster.id]
+  tags            = merge(var.tags, { Name = "${var.name}-filer-${each.key}", Role = "filer" })
+}
+
+resource "aws_network_interface" "s3" {
+  for_each        = var.s3_nodes
+  subnet_id       = each.value.subnet_id
+  private_ips     = [each.value.private_ip]
+  security_groups = [aws_security_group.cluster.id]
+  tags            = merge(var.tags, { Name = "${var.name}-s3-${each.key}", Role = "s3" })
+}
+
+# ---- step 2: render config with the cloud-agnostic core --------------------
+module "core" {
+  source = "../core"
+
+  weed_binary        = var.weed_binary
+  monitoring_enabled = var.monitoring_enabled
+  enable_security    = var.enable_security
+  security           = local.core_security
+
+  # Keep secrets out of cloud-init user_data when security is on; deliver them
+  # from SSM via the boot fetch script instead.
+  render_secret_files = !var.enable_security
+  boot_fetch_script   = local.fetch_script
+
+  master = {
+    nodes = { for k, v in var.masters : k => { address = v.private_ip } }
+  }
+
+  volume = {
+    enabled = length(var.volumes) > 0
+    nodes = { for k, v in var.volumes : k => {
+      address     = v.private_ip
+      rack        = v.rack
+      data_center = v.data_center
+      data_dirs   = [{ path = "/data", max_volumes = 0 }]
+      # auto-discover the single attached data disk and mount it at /data
+      disk_mounts = [{ mountpoint = "/data", fstype = "xfs" }]
+    } }
+  }
+
+  filer = {
+    enabled = length(var.filers) > 0
+    nodes = { for k, v in var.filers : k => {
+      address     = v.private_ip
+      data_dir    = "/data/filerldb2"
+      disk_mounts = [{ mountpoint = "/data", fstype = "xfs" }]
+    } }
+    s3 = var.embedded_s3
+  }
+
+  s3 = {
+    enabled = length(var.s3_nodes) > 0
+    nodes   = { for k, v in var.s3_nodes : k => { address = v.private_ip } }
+  }
+
+  s3_identities = var.s3_identities
+}
+
+# ---- step 3: instances (keyed for_each; user_data = rendered cloud-init) ----
+resource "aws_instance" "master" {
+  for_each                = var.masters
+  ami                     = var.ami_id
+  instance_type           = each.value.instance_type
+  key_name                = var.key_name
+  user_data               = module.core.cloud_init_by_node["master-${each.key}"]
+  iam_instance_profile    = var.enable_security ? aws_iam_instance_profile.node[0].name : null
+  disable_api_termination = var.termination_protection
+
+  network_interface {
+    network_interface_id = aws_network_interface.master[each.key].id
+    device_index         = 0
+  }
+  metadata_options {
+    http_tokens = "required" # IMDSv2
+  }
+  tags = merge(var.tags, { Name = "${var.name}-master-${each.key}", Role = "master" })
+}
+
+resource "aws_instance" "volume" {
+  for_each                = var.volumes
+  ami                     = var.ami_id
+  instance_type           = each.value.instance_type
+  key_name                = var.key_name
+  user_data               = module.core.cloud_init_by_node["volume-${each.key}"]
+  iam_instance_profile    = var.enable_security ? aws_iam_instance_profile.node[0].name : null
+  disable_api_termination = var.termination_protection
+
+  network_interface {
+    network_interface_id = aws_network_interface.volume[each.key].id
+    device_index         = 0
+  }
+  metadata_options {
+    http_tokens = "required"
+  }
+  tags = merge(var.tags, { Name = "${var.name}-volume-${each.key}", Role = "volume" })
+}
+
+resource "aws_instance" "filer" {
+  for_each                = var.filers
+  ami                     = var.ami_id
+  instance_type           = each.value.instance_type
+  key_name                = var.key_name
+  user_data               = module.core.cloud_init_by_node["filer-${each.key}"]
+  iam_instance_profile    = var.enable_security ? aws_iam_instance_profile.node[0].name : null
+  disable_api_termination = var.termination_protection
+
+  network_interface {
+    network_interface_id = aws_network_interface.filer[each.key].id
+    device_index         = 0
+  }
+  metadata_options {
+    http_tokens = "required"
+  }
+  tags = merge(var.tags, { Name = "${var.name}-filer-${each.key}", Role = "filer" })
+}
+
+resource "aws_instance" "s3" {
+  for_each             = var.s3_nodes
+  ami                  = var.ami_id
+  instance_type        = each.value.instance_type
+  key_name             = var.key_name
+  user_data            = module.core.cloud_init_by_node["s3-${each.key}"]
+  iam_instance_profile = var.enable_security ? aws_iam_instance_profile.node[0].name : null
+
+  network_interface {
+    network_interface_id = aws_network_interface.s3[each.key].id
+    device_index         = 0
+  }
+  metadata_options {
+    http_tokens = "required"
+  }
+  tags = merge(var.tags, { Name = "${var.name}-s3-${each.key}", Role = "s3" })
+}
+
+# ---- step 4: protected data disks (decoupled; survive instance replacement) -
+# The core's mount-disks.sh (wired via disk_mounts above) mkfs+mounts these at
+# /data before the weed unit starts.
+resource "aws_ebs_volume" "volume_data" {
+  for_each          = var.volumes
+  availability_zone = each.value.availability_zone
+  size              = each.value.data_volume_size_gb
+  type              = each.value.data_volume_type
+  iops              = each.value.data_volume_iops
+  encrypted         = true
+  tags              = merge(var.tags, { Name = "${var.name}-volume-${each.key}-data", Role = "volume" })
+
+  lifecycle {
+    prevent_destroy = true # break-glass: remove to allow destroy
+  }
+}
+
+resource "aws_volume_attachment" "volume_data" {
+  for_each                       = var.volumes
+  device_name                    = "/dev/xvdf"
+  volume_id                      = aws_ebs_volume.volume_data[each.key].id
+  instance_id                    = aws_instance.volume[each.key].id
+  stop_instance_before_detaching = true
+}
+
+resource "aws_ebs_volume" "filer_data" {
+  for_each          = var.filers
+  availability_zone = each.value.availability_zone
+  size              = each.value.data_volume_size_gb
+  type              = "gp3"
+  encrypted         = true
+  tags              = merge(var.tags, { Name = "${var.name}-filer-${each.key}-data", Role = "filer" })
+
+  lifecycle {
+    prevent_destroy = true
+  }
+}
+
+resource "aws_volume_attachment" "filer_data" {
+  for_each                       = var.filers
+  device_name                    = "/dev/xvdf"
+  volume_id                      = aws_ebs_volume.filer_data[each.key].id
+  instance_id                    = aws_instance.filer[each.key].id
+  stop_instance_before_detaching = true
+}

terraform/modules/aws/outputs.tf

@@ -0,0 +1,34 @@
+output "security_group_id" {
+  description = "Cluster security group id."
+  value       = aws_security_group.cluster.id
+}
+
+output "master_private_ips" {
+  description = "Master private IPs keyed by node id."
+  value       = { for k, v in var.masters : k => v.private_ip }
+}
+
+output "master_peers" {
+  description = "Master peer list passed to the weed processes."
+  value       = module.core.master_peers
+}
+
+output "filer_private_ips" {
+  description = "Filer private IPs keyed by node id."
+  value       = { for k, v in var.filers : k => v.private_ip }
+}
+
+output "s3_private_ips" {
+  description = "Standalone S3 gateway private IPs keyed by node id."
+  value       = { for k, v in var.s3_nodes : k => v.private_ip }
+}
+
+output "instance_ids" {
+  description = "All instance ids keyed by role-node."
+  value = merge(
+    { for k, v in aws_instance.master : "master-${k}" => v.id },
+    { for k, v in aws_instance.volume : "volume-${k}" => v.id },
+    { for k, v in aws_instance.filer : "filer-${k}" => v.id },
+    { for k, v in aws_instance.s3 : "s3-${k}" => v.id },
+  )
+}

terraform/modules/aws/security.tf

@@ -0,0 +1,136 @@
+# =============================================================================
+# mTLS + JWT generation and secret delivery (active when enable_security=true).
+#
+# Generates the CA + per-component certs + JWT keys with the security submodule,
+# stores them (plus the rendered security.toml / S3 config) in SSM Parameter
+# Store as SecureString, and renders a boot fetch-secrets.sh that pulls them via
+# the instance role. This keeps secrets OUT of cloud-init user_data / IMDS.
+# Secrets still live in Terraform state (TF-generated); prefer Vault PKI for the
+# CA in production.
+# =============================================================================
+
+locals {
+  all_private_ips = concat(
+    [for k, v in var.masters : v.private_ip],
+    [for k, v in var.volumes : v.private_ip],
+    [for k, v in var.filers : v.private_ip],
+    [for k, v in var.s3_nodes : v.private_ip],
+  )
+
+  ssm_prefix        = "/seaweedfs/${var.name}"
+  deliver_s3_config = length(var.s3_nodes) > 0 || var.embedded_s3.enabled
+
+  # security.toml is identical across nodes; take it from the first master.
+  first_master       = var.enable_security ? sort(keys(var.masters))[0] : ""
+  security_toml_path = "/etc/seaweedfs/security.toml"
+  s3_config_path     = "/etc/seaweedfs/s3_config.json"
+
+  # Boot fetch entries: cert files + security.toml + (optional) S3 config.
+  fetch_entries = var.enable_security ? concat(
+    [for m in module.security[0].secret_manifest : { param = "${local.ssm_prefix}/certs/${m.key}", path = m.path, mode = m.mode }],
+    [{ param = "${local.ssm_prefix}/security.toml", path = local.security_toml_path, mode = "0600" }],
+    local.deliver_s3_config ? [{ param = "${local.ssm_prefix}/s3_config.json", path = local.s3_config_path, mode = "0600" }] : [],
+  ) : []
+
+  fetch_script = var.enable_security ? templatefile("${path.module}/templates/fetch-secrets.sh.tftpl", {
+    run_as_user = "seaweedfs"
+    entries     = local.fetch_entries
+  }) : ""
+
+  core_security = var.enable_security ? module.security[0].core_security : var.security
+}
+
+module "security" {
+  count           = var.enable_security ? 1 : 0
+  source          = "../security"
+  internal_domain = var.internal_domain
+  cert_dir        = var.cert_dir
+  ip_sans         = local.all_private_ips
+}
+
+# ---- SSM SecureString parameters -------------------------------------------
+resource "aws_ssm_parameter" "cert" {
+  for_each = var.enable_security ? { for m in module.security[0].secret_manifest : m.key => m } : {}
+  name     = "${local.ssm_prefix}/certs/${each.key}"
+  type     = "SecureString"
+  value    = module.security[0].secret_contents[each.key]
+  tags     = var.tags
+}
+
+resource "aws_ssm_parameter" "security_toml" {
+  count = var.enable_security ? 1 : 0
+  name  = "${local.ssm_prefix}/security.toml"
+  type  = "SecureString"
+  value = module.core.secret_files_by_node["master-${local.first_master}"][local.security_toml_path]
+  tags  = var.tags
+}
+
+resource "aws_ssm_parameter" "s3_config" {
+  count = var.enable_security && local.deliver_s3_config ? 1 : 0
+  name  = "${local.ssm_prefix}/s3_config.json"
+  type  = "SecureString"
+  # identical content across s3/filer nodes; render once via the security module's
+  # JSON is not available here, so read it back from any node that carries it.
+  value = local.s3_config_source
+  tags  = var.tags
+}
+
+locals {
+  # Pull the rendered S3 identity JSON from whichever node carries it.
+  s3_config_source = !local.deliver_s3_config ? "" : (
+    length(var.s3_nodes) > 0 ?
+    module.core.secret_files_by_node["s3-${sort(keys(var.s3_nodes))[0]}"][local.s3_config_path] :
+    (length(var.filers) > 0 ? module.core.secret_files_by_node["filer-${sort(keys(var.filers))[0]}"][local.s3_config_path] : "")
+  )
+}
+
+# ---- IAM: instance role allowed to read the SSM params ----------------------
+data "aws_iam_policy_document" "assume" {
+  count = var.enable_security ? 1 : 0
+  statement {
+    actions = ["sts:AssumeRole"]
+    principals {
+      type        = "Service"
+      identifiers = ["ec2.amazonaws.com"]
+    }
+  }
+}
+
+data "aws_iam_policy_document" "ssm_read" {
+  count = var.enable_security ? 1 : 0
+  statement {
+    sid       = "ReadSeaweedfsParams"
+    actions   = ["ssm:GetParameter", "ssm:GetParameters"]
+    resources = ["arn:aws:ssm:*:*:parameter${local.ssm_prefix}/*"]
+  }
+  statement {
+    sid       = "DecryptSecureStrings"
+    actions   = ["kms:Decrypt"]
+    resources = [var.kms_key_arn] # defaults to "*"; set a CMK ARN in production
+    condition {
+      test     = "StringEquals"
+      variable = "kms:ViaService"
+      values   = ["ssm.*.amazonaws.com"]
+    }
+  }
+}
+
+resource "aws_iam_role" "node" {
+  count              = var.enable_security ? 1 : 0
+  name_prefix        = "${var.name}-node-"
+  assume_role_policy = data.aws_iam_policy_document.assume[0].json
+  tags               = var.tags
+}
+
+resource "aws_iam_role_policy" "ssm_read" {
+  count  = var.enable_security ? 1 : 0
+  name   = "ssm-read"
+  role   = aws_iam_role.node[0].id
+  policy = data.aws_iam_policy_document.ssm_read[0].json
+}
+
+resource "aws_iam_instance_profile" "node" {
+  count       = var.enable_security ? 1 : 0
+  name_prefix = "${var.name}-node-"
+  role        = aws_iam_role.node[0].name
+}

terraform/modules/aws/security_group.tf

@@ -0,0 +1,62 @@
+# Cluster security group + the SeaweedFS port matrix.
+# Intra-cluster: all TCP allowed within the SG (master 9333/19333, volume
+# 8080/18080, filer 8888/18888, admin 33646, ...). Client-facing: only S3 (8333)
+# and filer (8888). Metrics ports (9327) are never opened to clients.
+
+resource "aws_security_group" "cluster" {
+  name_prefix = "${var.name}-cluster-"
+  description = "SeaweedFS cluster"
+  vpc_id      = var.vpc_id
+  tags        = merge(var.tags, { Name = "${var.name}-cluster" })
+
+  lifecycle {
+    create_before_destroy = true
+  }
+}
+
+# All intra-cluster TCP via SG self-reference.
+resource "aws_vpc_security_group_ingress_rule" "intra_cluster" {
+  security_group_id            = aws_security_group.cluster.id
+  referenced_security_group_id = aws_security_group.cluster.id
+  ip_protocol                  = "tcp"
+  from_port                    = 0
+  to_port                      = 65535
+  description                  = "intra-cluster (http + gRPC for all roles)"
+}
+
+resource "aws_vpc_security_group_ingress_rule" "s3" {
+  for_each          = toset(var.client_ingress_cidrs)
+  security_group_id = aws_security_group.cluster.id
+  cidr_ipv4         = each.value
+  ip_protocol       = "tcp"
+  from_port         = 8333
+  to_port           = 8333
+  description       = "S3 gateway"
+}
+
+resource "aws_vpc_security_group_ingress_rule" "filer" {
+  for_each          = toset(var.client_ingress_cidrs)
+  security_group_id = aws_security_group.cluster.id
+  cidr_ipv4         = each.value
+  ip_protocol       = "tcp"
+  from_port         = 8888
+  to_port           = 8888
+  description       = "filer HTTP"
+}
+
+resource "aws_vpc_security_group_ingress_rule" "ssh" {
+  for_each          = toset(var.ssh_ingress_cidrs)
+  security_group_id = aws_security_group.cluster.id
+  cidr_ipv4         = each.value
+  ip_protocol       = "tcp"
+  from_port         = 22
+  to_port           = 22
+  description       = "SSH"
+}
+
+resource "aws_vpc_security_group_egress_rule" "all" {
+  security_group_id = aws_security_group.cluster.id
+  cidr_ipv4         = "0.0.0.0/0"
+  ip_protocol       = "-1"
+  description       = "all egress"
+}

terraform/modules/aws/templates/fetch-secrets.sh.tftpl

@@ -0,0 +1,21 @@
+#!/usr/bin/env bash
+# Rendered by terraform-aws-seaweedfs. Pulls certs + secret config from SSM
+# Parameter Store (SecureString) at boot using the instance role. Region is
+# auto-resolved by the AWS CLI from IMDS, so no region is baked in. Requires
+# the AWS CLI and a working instance profile (see security.tf).
+set -euo pipefail
+
+run_as_user="${run_as_user}"
+
+fetch() {
+  local param="$1" dest="$2" mode="$3"
+  mkdir -p "$(dirname "$dest")"
+  aws ssm get-parameter --with-decryption --name "$param" \
+    --query 'Parameter.Value' --output text > "$dest"
+  chmod "$mode" "$dest"
+  chown "$run_as_user:$run_as_user" "$dest" 2>/dev/null || true
+}
+
+%{ for e in entries ~}
+fetch "${e.param}" "${e.path}" "${e.mode}"
+%{ endfor ~}

terraform/modules/aws/variables.tf

@@ -0,0 +1,171 @@
+# =============================================================================
+# terraform-aws-seaweedfs - thin AWS wrapper around the cloud-agnostic core.
+#
+# Reserves stable per-node addressing (ENIs with fixed private IPs) FIRST, feeds
+# that address map to the core to render cloud-init, then creates instances and
+# protected EBS data disks. All tiers are keyed for_each (never count), so a
+# middle node can be replaced without reindexing its peers/disks.
+# =============================================================================
+
+variable "name" {
+  description = "Name prefix for all resources."
+  type        = string
+  default     = "seaweedfs"
+}
+
+variable "vpc_id" {
+  description = "VPC to deploy into."
+  type        = string
+}
+
+variable "ami_id" {
+  description = "AMI with the weed binary installed (e.g. baked with Packer)."
+  type        = string
+}
+
+variable "weed_binary" {
+  description = "Path to weed on the AMI."
+  type        = string
+  default     = "/usr/bin/weed"
+}
+
+variable "key_name" {
+  description = "EC2 key pair for SSH (optional)."
+  type        = string
+  default     = null
+}
+
+variable "monitoring_enabled" {
+  description = "Bind metrics ports and pass them through to the core."
+  type        = bool
+  default     = false
+}
+
+variable "enable_security" {
+  description = "Enable mTLS (security.toml). Cert material must be supplied via the security variable / secret store."
+  type        = bool
+  default     = false
+}
+
+variable "security" {
+  description = "Passed through to the core security variable when enable_security=false. When enable_security=true this is ignored and certs/JWT are generated by the security submodule."
+  type        = any
+  default     = {}
+}
+
+variable "internal_domain" {
+  description = "Internal domain for generated component cert CNs / peer-auth wildcard (enable_security=true)."
+  type        = string
+  default     = "seaweedfs.internal"
+}
+
+variable "cert_dir" {
+  description = "On-host directory where certs are fetched/placed."
+  type        = string
+  default     = "/usr/local/share/ca-certificates"
+}
+
+variable "kms_key_arn" {
+  description = "KMS key ARN the instance role may use to decrypt SSM SecureStrings. Defaults to \"*\" (the SSM default key); set to a specific CMK ARN in production."
+  type        = string
+  default     = "*"
+}
+
+variable "termination_protection" {
+  description = "Set disable_api_termination on stateful instances (masters, volumes)."
+  type        = bool
+  default     = true
+}
+
+# ---- stateful tiers: keyed node maps ---------------------------------------
+variable "masters" {
+  description = "Master nodes keyed by stable id (m0/m1/m2). Quorum size must be odd."
+  type = map(object({
+    subnet_id     = string
+    private_ip    = string
+    instance_type = optional(string, "t3.medium")
+  }))
+  validation {
+    condition     = length(var.masters) % 2 == 1
+    error_message = "Master quorum must be an odd number of nodes (1, 3, 5)."
+  }
+}
+
+variable "volumes" {
+  description = "Volume nodes keyed by stable id. Each owns one protected EBS data disk."
+  type = map(object({
+    subnet_id           = string
+    availability_zone   = string
+    private_ip          = string
+    instance_type       = optional(string, "m5.large")
+    rack                = optional(string)
+    data_center         = optional(string)
+    data_volume_size_gb = optional(number, 100)
+    data_volume_type    = optional(string, "gp3")
+    data_volume_iops    = optional(number, null)
+  }))
+  default = {}
+}
+
+variable "filers" {
+  description = "Filer nodes keyed by stable id. Each owns a protected EBS disk for its leveldb2 store."
+  type = map(object({
+    subnet_id           = string
+    availability_zone   = string
+    private_ip          = string
+    instance_type       = optional(string, "t3.medium")
+    data_volume_size_gb = optional(number, 50)
+  }))
+  default = {}
+}
+
+variable "s3_nodes" {
+  description = "Standalone S3 gateway nodes keyed by stable id (stateless)."
+  type = map(object({
+    subnet_id     = string
+    private_ip    = string
+    instance_type = optional(string, "t3.medium")
+  }))
+  default = {}
+}
+
+variable "embedded_s3" {
+  description = "Embed the S3 gateway in the filer instead of standalone nodes."
+  type = object({
+    enabled     = optional(bool, false)
+    port        = optional(number, 8333)
+    domain_name = optional(string, "")
+  })
+  default = {}
+}
+
+variable "s3_identities" {
+  description = "Non-admin S3 identities for the gateway config (admin key goes via EnvironmentFile)."
+  type = list(object({
+    name       = string
+    access_key = string
+    secret_key = string
+    actions    = list(string)
+  }))
+  default   = []
+  sensitive = true
+}
+
+# ---- networking ------------------------------------------------------------
+variable "client_ingress_cidrs" {
+  description = "CIDRs allowed to reach the client-facing S3 (8333) and filer (8888) ports."
+  type        = list(string)
+  default     = []
+}
+
+variable "ssh_ingress_cidrs" {
+  description = "CIDRs allowed SSH (22)."
+  type        = list(string)
+  default     = []
+}
+
+variable "tags" {
+  description = "Extra tags applied to all resources."
+  type        = map(string)
+  default     = {}
+}

terraform/modules/aws/versions.tf

@@ -0,0 +1,12 @@
+terraform {
+  required_version = ">= 1.3.0"
+  required_providers {
+    aws = {
+      source = "hashicorp/aws"
+      # >= 5.40 for the modern aws_vpc_security_group_*_rule resources.
+      # Provider >= 6.0 is recommended for the EBS-attachment no-replace behavior;
+      # bump the floor here once you standardize on it.
+      version = ">= 5.40"
+    }
+  }
+}

terraform/modules/core/main.tf

@@ -0,0 +1,373 @@
+# =============================================================================
+# SeaweedFS Terraform core - pure renderer (zero cloud resources).
+#
+# Computes, per node: the verified `weed` argv, the full systemd ExecStart,
+# config files, environment, and rendered systemd unit + cloud-init.
+# =============================================================================
+
+locals {
+  # ---- shared security.toml (rendered when mTLS on OR any JWT key set) ----
+  jwt_any = anytrue([
+    var.security.jwt_signing_key != "",
+    var.security.jwt_signing_read_key != "",
+    var.security.jwt_filer_signing_key != "",
+    var.security.jwt_filer_signing_read_key != "",
+  ])
+  render_security = var.enable_security || local.jwt_any
+  security_toml = local.render_security ? templatefile("${path.module}/templates/security.toml.tftpl", {
+    enable_security            = var.enable_security
+    cert_dir                   = var.security.cert_dir
+    allowed_wildcard_domain    = var.security.allowed_wildcard_domain
+    allowed_common_names       = var.security.allowed_common_names
+    jwt_signing_key            = var.security.jwt_signing_key
+    jwt_signing_read_key       = var.security.jwt_signing_read_key
+    jwt_filer_signing_key      = var.security.jwt_filer_signing_key
+    jwt_filer_signing_read_key = var.security.jwt_filer_signing_read_key
+  }) : ""
+  security_files = local.render_security ? { "/etc/seaweedfs/security.toml" = local.security_toml } : {}
+
+  # ---- S3 identity JSON (non-admin identities only) ----
+  # Empty access_key => anonymous identity (no credentials block), per SeaweedFS
+  # convention where the identity literally named "anonymous" grants anon access.
+  s3_config_json = jsonencode({
+    identities = [for id in var.s3_identities : {
+      name        = id.name
+      credentials = id.access_key != "" ? [{ accessKey = id.access_key, secretKey = id.secret_key }] : []
+      actions     = id.actions
+    }]
+  })
+
+  # ---- global flags (before the subcommand) ----
+  global_flags = compact([
+    format("-v=%d", var.log_level),
+    var.log_to_stderr ? "-logtostderr=true" : "",
+  ])
+
+  # ---- master peers (sorted by key for determinism) ----
+  master_keys = sort(keys(var.master.nodes))
+  master_peer_list = [for k in local.master_keys :
+    format("%s:%d", var.master.nodes[k].address, coalesce(var.master.nodes[k].port, var.master.port))
+  ]
+  master_peers = join(",", local.master_peer_list)
+
+  # ---- first filer (default s3 target + cluster env) ----
+  filer_keys  = sort(keys(var.filer.nodes))
+  first_filer = length(local.filer_keys) > 0 ? format("%s:%d", var.filer.nodes[local.filer_keys[0]].address, coalesce(var.filer.nodes[local.filer_keys[0]].port, var.filer.port)) : ""
+
+  # ---- cluster discovery env (WEED_CLUSTER_*) ----
+  cluster_env = merge(
+    { WEED_CLUSTER_DEFAULT = var.cluster_name },
+    local.master_peers != "" ? { "WEED_CLUSTER_${replace(upper(var.cluster_name), "/[^A-Z0-9_]/", "_")}_MASTER" = local.master_peers } : {},
+    local.first_filer != "" ? { "WEED_CLUSTER_${replace(upper(var.cluster_name), "/[^A-Z0-9_]/", "_")}_FILER" = local.first_filer } : {},
+  )
+
+  metrics_flags_master = var.monitoring_enabled ? compact([
+    format("-metricsPort=%d", var.master.metrics_port),
+    var.master.metrics_ip != "" ? format("-metricsIp=%s", var.master.metrics_ip) : "",
+  ]) : []
+  metrics_flags_volume = var.monitoring_enabled ? compact([
+    format("-metricsPort=%d", var.volume.metrics_port),
+    var.volume.metrics_ip != "" ? format("-metricsIp=%s", var.volume.metrics_ip) : "",
+  ]) : []
+  metrics_flags_filer = var.monitoring_enabled ? compact([
+    format("-metricsPort=%d", var.filer.metrics_port),
+    var.filer.metrics_ip != "" ? format("-metricsIp=%s", var.filer.metrics_ip) : "",
+  ]) : []
+  metrics_flags_s3 = var.monitoring_enabled ? [format("-metricsPort=%d", var.s3.metrics_port)] : []
+
+  # ===========================================================================
+  # MASTER nodes
+  # ===========================================================================
+  master_nodes = var.master.enabled ? {
+    for k in local.master_keys : "master-${k}" => {
+      role        = "master"
+      name        = "master-${k}"
+      address     = var.master.nodes[k].address
+      disk_mounts = var.master.nodes[k].disk_mounts
+      data_dir    = coalesce(var.master.nodes[k].data_dir, var.master.data_dir)
+      data_dirs   = [coalesce(var.master.nodes[k].data_dir, var.master.data_dir)]
+      ports = {
+        http    = coalesce(var.master.nodes[k].port, var.master.port)
+        metrics = var.master.metrics_port
+      }
+      env          = local.cluster_env
+      config_files = local.security_files
+      argv = concat(
+        local.global_flags,
+        ["master"],
+        [
+          format("-port=%d", coalesce(var.master.nodes[k].port, var.master.port)),
+          format("-mdir=%s", coalesce(var.master.nodes[k].data_dir, var.master.data_dir)),
+          format("-ip=%s", var.master.nodes[k].address),
+          format("-ip.bind=%s", var.master.ip_bind),
+          format("-peers=%s", local.master_peers),
+          format("-defaultReplication=%s", var.master.default_replication),
+          format("-volumeSizeLimitMB=%d", var.master.volume_size_limit_mb),
+          format("-electionTimeout=%s", var.master.election_timeout),
+          format("-heartbeatInterval=%s", var.master.heartbeat_interval),
+        ],
+        var.master.nodes[k].grpc_port != null ? [format("-port.grpc=%d", var.master.nodes[k].grpc_port)] : (
+        var.master.grpc_port != null ? [format("-port.grpc=%d", var.master.grpc_port)] : []),
+        var.master.volume_preallocate ? ["-volumePreallocate"] : [],
+        var.master.garbage_threshold != null ? [format("-garbageThreshold=%s", tostring(var.master.garbage_threshold))] : [],
+        var.master.raft_hashicorp ? ["-raftHashicorp"] : [],
+        var.master.resume_state ? ["-resumeState"] : [],
+        var.master.disable_http ? ["-disableHttp"] : [],
+        var.master.white_list != "" ? [format("-whiteList=%s", var.master.white_list)] : [],
+        local.metrics_flags_master,
+        var.master.extra_args,
+      )
+    }
+  } : {}
+
+  # ===========================================================================
+  # VOLUME nodes
+  # ===========================================================================
+  volume_keys = sort(keys(var.volume.nodes))
+  volume_nodes = var.volume.enabled ? {
+    for k in local.volume_keys : "volume-${k}" => {
+      role        = "volume"
+      name        = "volume-${k}"
+      address     = var.volume.nodes[k].address
+      disk_mounts = var.volume.nodes[k].disk_mounts
+      data_dirs   = [for d in(var.volume.nodes[k].data_dirs != null ? var.volume.nodes[k].data_dirs : var.volume.data_dirs) : d.path]
+      data_dir    = (var.volume.nodes[k].data_dirs != null ? var.volume.nodes[k].data_dirs : var.volume.data_dirs)[0].path
+      ports = {
+        http    = coalesce(var.volume.nodes[k].port, var.volume.port)
+        metrics = var.volume.metrics_port
+      }
+      env          = local.cluster_env
+      config_files = local.security_files
+      argv = concat(
+        local.global_flags,
+        ["volume"],
+        [
+          format("-port=%d", coalesce(var.volume.nodes[k].port, var.volume.port)),
+          format("-dir=%s", join(",", [for d in(var.volume.nodes[k].data_dirs != null ? var.volume.nodes[k].data_dirs : var.volume.data_dirs) : d.path])),
+          format("-max=%s", join(",", [for d in(var.volume.nodes[k].data_dirs != null ? var.volume.nodes[k].data_dirs : var.volume.data_dirs) : tostring(d.max_volumes)])),
+          format("-ip=%s", var.volume.nodes[k].address),
+          format("-ip.bind=%s", var.volume.ip_bind),
+          format("-mserver=%s", local.master_peers),
+          format("-readMode=%s", var.volume.read_mode),
+          format("-compactionMBps=%d", var.volume.compaction_mbps),
+          format("-minFreeSpacePercent=%s", var.volume.min_free_space_percent),
+          format("-index=%s", var.volume.index),
+        ],
+        var.volume.nodes[k].grpc_port != null ? [format("-port.grpc=%d", var.volume.nodes[k].grpc_port)] : [],
+        (var.volume.nodes[k].idx_dir != null && var.volume.nodes[k].idx_dir != "") ? [format("-dir.idx=%s", var.volume.nodes[k].idx_dir)] : (var.volume.idx_dir != "" ? [format("-dir.idx=%s", var.volume.idx_dir)] : []),
+        var.volume.nodes[k].data_center != null ? [format("-dataCenter=%s", var.volume.nodes[k].data_center)] : [],
+        var.volume.nodes[k].rack != null ? [format("-rack=%s", var.volume.nodes[k].rack)] : [],
+        var.volume.nodes[k].public_url != null ? [format("-publicUrl=%s", var.volume.nodes[k].public_url)] : [],
+        var.volume.file_size_limit_mb != null ? [format("-fileSizeLimitMB=%d", var.volume.file_size_limit_mb)] : [],
+        var.volume.images_fix_orientation ? ["-images.fix.orientation"] : [],
+        var.volume.white_list != "" ? [format("-whiteList=%s", var.volume.white_list)] : [],
+        local.metrics_flags_volume,
+        var.volume.extra_args,
+      )
+    }
+  } : {}
+
+  # ===========================================================================
+  # FILER nodes (+ optional embedded S3)
+  # ===========================================================================
+  filer_s3_files = var.filer.s3.enabled ? { (var.filer.s3.config_path) = local.s3_config_json } : {}
+  filer_nodes = var.filer.enabled ? {
+    for k in local.filer_keys : "filer-${k}" => {
+      role        = "filer"
+      name        = "filer-${k}"
+      address     = var.filer.nodes[k].address
+      disk_mounts = var.filer.nodes[k].disk_mounts
+      data_dir    = coalesce(var.filer.nodes[k].data_dir, var.filer.data_dir)
+      data_dirs   = [coalesce(var.filer.nodes[k].data_dir, var.filer.data_dir)]
+      ports = {
+        http    = coalesce(var.filer.nodes[k].port, var.filer.port)
+        metrics = var.filer.metrics_port
+      }
+      env          = merge(local.cluster_env, var.filer.extra_env)
+      config_files = merge(local.security_files, local.filer_s3_files)
+      argv = concat(
+        local.global_flags,
+        ["filer"],
+        [
+          format("-port=%d", coalesce(var.filer.nodes[k].port, var.filer.port)),
+          format("-ip=%s", var.filer.nodes[k].address),
+          format("-ip.bind=%s", var.filer.ip_bind),
+          format("-master=%s", local.master_peers),
+          format("-defaultReplicaPlacement=%s", var.filer.default_replica_placement),
+          format("-dirListLimit=%d", var.filer.dir_list_limit),
+          format("-defaultStoreDir=%s", coalesce(var.filer.nodes[k].data_dir, var.filer.data_dir)),
+        ],
+        var.filer.nodes[k].grpc_port != null ? [format("-port.grpc=%d", var.filer.nodes[k].grpc_port)] : [],
+        var.filer.max_mb != null ? [format("-maxMB=%d", var.filer.max_mb)] : [],
+        var.filer.disable_dir_listing ? ["-disableDirListing"] : [],
+        var.filer.disable_http ? ["-disableHttp"] : [],
+        var.filer.encrypt_volume_data ? ["-encryptVolumeData"] : [],
+        var.filer.rack != "" ? [format("-rack=%s", var.filer.rack)] : [],
+        var.filer.data_center != "" ? [format("-dataCenter=%s", var.filer.data_center)] : [],
+        var.filer.filer_group != "" ? [format("-filerGroup=%s", var.filer.filer_group)] : [],
+        var.filer.s3.enabled ? [
+          "-s3",
+          format("-s3.port=%d", var.filer.s3.port),
+          format("-s3.config=%s", var.filer.s3.config_path),
+        ] : [],
+        (var.filer.s3.enabled && var.filer.s3.https_port > 0) ? [format("-s3.port.https=%d", var.filer.s3.https_port)] : [],
+        (var.filer.s3.enabled && var.filer.s3.domain_name != "") ? [format("-s3.domainName=%s", var.filer.s3.domain_name)] : [],
+        local.metrics_flags_filer,
+        var.filer.extra_args,
+      )
+    }
+  } : {}
+
+  # ===========================================================================
+  # S3 standalone nodes
+  # ===========================================================================
+  s3_keys      = sort(keys(var.s3.nodes))
+  s3_target    = var.s3.filer_address != "" ? var.s3.filer_address : local.first_filer
+  s3_std_files = merge(local.security_files, { (var.s3.config_path) = local.s3_config_json })
+  s3_nodes = var.s3.enabled ? {
+    for k in local.s3_keys : "s3-${k}" => {
+      role        = "s3"
+      name        = "s3-${k}"
+      address     = var.s3.nodes[k].address
+      disk_mounts = []
+      data_dir    = ""
+      data_dirs   = []
+      ports = {
+        http    = coalesce(var.s3.nodes[k].port, var.s3.port)
+        metrics = var.s3.metrics_port
+      }
+      env          = local.cluster_env
+      config_files = local.s3_std_files
+      argv = concat(
+        local.global_flags,
+        ["s3"],
+        [
+          format("-port=%d", coalesce(var.s3.nodes[k].port, var.s3.port)),
+          format("-ip.bind=%s", var.s3.ip_bind),
+          format("-filer=%s", local.s3_target),
+          format("-config=%s", var.s3.config_path),
+        ],
+        var.s3.https_port > 0 ? [format("-port.https=%d", var.s3.https_port)] : [],
+        var.s3.iceberg_port != null ? [format("-port.iceberg=%d", var.s3.iceberg_port)] : [],
+        var.s3.domain_name != "" ? [format("-domainName=%s", var.s3.domain_name)] : [],
+        var.s3.audit_log_config_path != "" ? [format("-auditLogConfig=%s", var.s3.audit_log_config_path)] : [],
+        var.s3.cert_file != "" ? [format("-cert.file=%s", var.s3.cert_file)] : [],
+        var.s3.key_file != "" ? [format("-key.file=%s", var.s3.key_file)] : [],
+        var.s3.cacert_file != "" ? [format("-cacert.file=%s", var.s3.cacert_file)] : [],
+        var.s3.verify_client_cert ? ["-tlsVerifyClientCert"] : [],
+        local.metrics_flags_s3,
+        var.s3.extra_args,
+      )
+    }
+  } : {}
+
+  # ===========================================================================
+  # ALL-IN-ONE nodes (weed server)
+  # ===========================================================================
+  aio_keys     = sort(keys(var.all_in_one.nodes))
+  aio_s3_files = var.all_in_one.s3.enabled ? { (var.all_in_one.s3.config_path) = local.s3_config_json } : {}
+  aio_nodes = var.all_in_one.enabled ? {
+    for k in local.aio_keys : "all-in-one-${k}" => {
+      role        = "server"
+      name        = "all-in-one-${k}"
+      address     = var.all_in_one.nodes[k].address
+      disk_mounts = var.all_in_one.nodes[k].disk_mounts
+      data_dir    = coalesce(var.all_in_one.nodes[k].data_dir, var.all_in_one.data_dir)
+      data_dirs   = [coalesce(var.all_in_one.nodes[k].data_dir, var.all_in_one.data_dir)]
+      ports = {
+        http    = var.all_in_one.master_port
+        metrics = var.all_in_one.metrics_port
+      }
+      env          = local.cluster_env
+      config_files = merge(local.security_files, local.aio_s3_files)
+      argv = concat(
+        local.global_flags,
+        ["server"],
+        [
+          format("-dir=%s", coalesce(var.all_in_one.nodes[k].data_dir, var.all_in_one.data_dir)),
+          format("-ip=%s", var.all_in_one.nodes[k].address),
+          format("-ip.bind=%s", var.all_in_one.ip_bind),
+          "-master",
+          format("-master.port=%d", var.all_in_one.master_port),
+          format("-master.peers=%s:%d", var.all_in_one.nodes[k].address, var.all_in_one.master_port),
+          format("-master.defaultReplication=%s", var.all_in_one.default_replication),
+          format("-master.volumeSizeLimitMB=%d", var.all_in_one.volume_size_limit_mb),
+          "-volume",
+          format("-volume.port=%d", var.all_in_one.volume_port),
+          "-filer",
+          format("-filer.port=%d", var.all_in_one.filer_port),
+          format("-idleTimeout=%d", var.all_in_one.idle_timeout),
+        ],
+        var.all_in_one.disable_http ? [format("-disableHttp=%t", var.all_in_one.disable_http)] : [],
+        var.all_in_one.s3.enabled ? [
+          "-s3",
+          format("-s3.port=%d", var.all_in_one.s3.port),
+          format("-s3.config=%s", var.all_in_one.s3.config_path),
+        ] : [],
+        (var.all_in_one.s3.enabled && var.all_in_one.s3.domain_name != "") ? [format("-s3.domainName=%s", var.all_in_one.s3.domain_name)] : [],
+        var.monitoring_enabled ? [format("-metricsPort=%d", var.all_in_one.metrics_port)] : [],
+        var.all_in_one.extra_args,
+      )
+    }
+  } : {}
+
+  # ===========================================================================
+  # Merge + render systemd unit and cloud-init per node
+  # ===========================================================================
+  nodes_base = merge(local.master_nodes, local.volume_nodes, local.filer_nodes, local.s3_nodes, local.aio_nodes)
+
+  # Disk-mount script per node (empty unless the node declares disk_mounts).
+  mount_scripts = {
+    for name, n in local.nodes_base : name => length(n.disk_mounts) > 0 ? templatefile("${path.module}/templates/mount-disks.sh.tftpl", {
+      run_as_user = var.hardening.run_as_user
+      mounts      = n.disk_mounts
+    }) : ""
+  }
+
+  nodes = {
+    for name, n in local.nodes_base : name => merge(n, {
+      exec_start = "${var.weed_binary} ${join(" ", n.argv)}"
+      # secret_files: the secret-bearing config a wrapper should deliver from a
+      # secret store when render_secret_files=false (else they go in cloud-init).
+      secret_files = n.config_files
+      mount_script = local.mount_scripts[name]
+      file_modes   = { for p in keys(n.config_files) : p => "0600" }
+      systemd_unit = templatefile("${path.module}/templates/weed.service.tftpl", {
+        role                = n.role
+        name                = n.name
+        run_as_user         = var.hardening.run_as_user
+        exec_start          = "${var.weed_binary} ${join(" ", n.argv)}"
+        no_new_privileges   = var.hardening.no_new_privileges
+        protect_system      = var.hardening.protect_system
+        cap_drop_all        = var.hardening.cap_drop_all
+        read_write_paths    = n.data_dirs
+        requires_mounts_for = n.data_dirs
+        env_file            = var.env_file
+        environment         = n.env
+      })
+      cloud_init = templatefile("${path.module}/templates/cloud-init.yaml.tftpl", {
+        role         = n.role
+        name         = n.name
+        run_as_user  = var.hardening.run_as_user
+        config_files = var.render_secret_files ? n.config_files : {}
+        file_modes   = { for p in keys(n.config_files) : p => "0600" }
+        pre_runcmd   = [for p in n.data_dirs : "install -d -o ${var.hardening.run_as_user} -g ${var.hardening.run_as_user} ${p}"]
+        mount_script = local.mount_scripts[name]
+        fetch_script = var.boot_fetch_script
+        systemd_unit = templatefile("${path.module}/templates/weed.service.tftpl", {
+          role                = n.role
+          name                = n.name
+          run_as_user         = var.hardening.run_as_user
+          exec_start          = "${var.weed_binary} ${join(" ", n.argv)}"
+          no_new_privileges   = var.hardening.no_new_privileges
+          protect_system      = var.hardening.protect_system
+          cap_drop_all        = var.hardening.cap_drop_all
+          read_write_paths    = n.data_dirs
+          requires_mounts_for = n.data_dirs
+          env_file            = var.env_file
+          environment         = n.env
+        })
+      })
+    })
+  }
+}

terraform/modules/core/outputs.tf

@@ -0,0 +1,32 @@
+output "nodes" {
+  description = "Per-node rendered artifacts keyed by node name. Each value: role, name, address, ports, data_dir(s), argv (list, after the weed binary), exec_start, env, config_files, systemd_unit, cloud_init."
+  value       = local.nodes
+  sensitive   = true # config_files/env may carry JWT keys or S3 identities
+}
+
+output "master_peers" {
+  description = "Comma-separated master peer list (address:port), as passed to -peers / -mserver / -master."
+  value       = local.master_peers
+}
+
+output "first_filer" {
+  description = "address:port of the lexically-first filer node (default S3 target)."
+  value       = local.first_filer
+}
+
+output "node_names" {
+  description = "List of rendered node names (non-sensitive)."
+  value       = sort(keys(local.nodes))
+}
+
+output "cloud_init_by_node" {
+  description = "Map of node name -> rendered cloud-init user_data (for the per-cloud wrappers)."
+  value       = { for name, n in local.nodes : name => n.cloud_init }
+  sensitive   = true
+}
+
+output "secret_files_by_node" {
+  description = "Map of node name -> {path => content} of secret-bearing files (security.toml, S3 identity JSON). Wrappers push these to a secret store and fetch them at boot when render_secret_files=false."
+  value       = { for name, n in local.nodes : name => n.secret_files }
+  sensitive   = true
+}

terraform/modules/core/templates/cloud-init.yaml.tftpl

@@ -0,0 +1,44 @@
+#cloud-config
+# Rendered by the SeaweedFS Terraform core module for node ${name} (role ${role}).
+# When render_secret_files=false, secret-bearing files are NOT here; they are
+# fetched at boot by fetch-secrets.sh from the cloud secret store.
+users:
+  - name: ${run_as_user}
+    system: true
+    shell: /usr/sbin/nologin
+write_files:
+%{ for path, content in config_files ~}
+  - path: ${path}
+    owner: "${run_as_user}:${run_as_user}"
+    permissions: '${lookup(file_modes, path, "0644")}'
+    content: |
+      ${indent(6, content)}
+%{ endfor ~}
+%{ if mount_script != "" ~}
+  - path: /opt/seaweedfs/mount-disks.sh
+    permissions: '0755'
+    content: |
+      ${indent(6, mount_script)}
+%{ endif ~}
+%{ if fetch_script != "" ~}
+  - path: /opt/seaweedfs/fetch-secrets.sh
+    permissions: '0755'
+    content: |
+      ${indent(6, fetch_script)}
+%{ endif ~}
+  - path: /etc/systemd/system/weed-${role}.service
+    permissions: '0644'
+    content: |
+      ${indent(6, systemd_unit)}
+runcmd:
+%{ if mount_script != "" ~}
+  - /opt/seaweedfs/mount-disks.sh
+%{ endif ~}
+%{ if fetch_script != "" ~}
+  - /opt/seaweedfs/fetch-secrets.sh
+%{ endif ~}
+%{ for cmd in pre_runcmd ~}
+  - ${cmd}
+%{ endfor ~}
+  - systemctl daemon-reload
+  - systemctl enable --now weed-${role}.service

terraform/modules/core/templates/mount-disks.sh.tftpl

@@ -0,0 +1,73 @@
+#!/usr/bin/env bash
+# Rendered by the SeaweedFS Terraform core module. Formats (only if blank) and
+# mounts each data disk, persisting by UUID to /etc/fstab. Safe to re-run.
+set -euo pipefail
+
+run_as_user="${run_as_user}"
+
+# Base block device backing "/" so auto-discovery never touches the root disk.
+root_src="$(findmnt -no SOURCE / 2>/dev/null || true)"
+root_base="$(lsblk -no pkname "$root_src" 2>/dev/null || true)"
+[ -n "$root_base" ] && root_base="/dev/$root_base"
+
+is_mounted() { findmnt -rno TARGET "$1" >/dev/null 2>&1; }
+
+# Pick the first unmounted, non-root whole disk (single-data-disk node).
+discover_dev() {
+  local d base
+  for d in /dev/nvme*n1 /dev/xvd[b-z] /dev/sd[b-z] /dev/vd[b-z]; do
+    [ -b "$d" ] || continue
+    [ "$d" = "$root_src" ] && continue
+    [ "$d" = "$root_base" ] && continue
+    base="/dev/$(lsblk -no pkname "$d" 2>/dev/null || true)"
+    [ "$base" = "$root_base" ] && continue
+    is_mounted "$d" && continue
+    echo "$d"; return 0
+  done
+  return 1
+}
+
+# Wait up to ~120s for one of the explicit candidate devices to appear.
+wait_dev() {
+  local i d
+  for i in $(seq 1 60); do
+    for d in "$@"; do [ -b "$d" ] && { echo "$d"; return 0; }; done
+    sleep 2
+  done
+  return 1
+}
+
+mount_one() {
+  local mountpoint="$1" fstype="$2"; shift 2
+  local dev=""
+  if [ "$#" -gt 0 ]; then
+    dev="$(wait_dev "$@" || true)"
+  fi
+  if [ -z "$dev" ]; then
+    dev="$(discover_dev || true)"
+  fi
+  if [ -z "$dev" ]; then
+    echo "mount-disks: no device found for $mountpoint" >&2
+    return 1
+  fi
+  if ! blkid "$dev" >/dev/null 2>&1; then
+    echo "mount-disks: formatting $dev as $fstype"
+    "mkfs.$fstype" -q "$dev"
+  fi
+  mkdir -p "$mountpoint"
+  local uuid; uuid="$(blkid -s UUID -o value "$dev")"
+  if [ -z "$uuid" ]; then
+    echo "mount-disks: failed to read a filesystem UUID for $dev" >&2
+    return 1
+  fi
+  if ! grep -q "$uuid" /etc/fstab; then
+    printf 'UUID=%s %s %s defaults,nofail 0 2\n' "$uuid" "$mountpoint" "$fstype" >> /etc/fstab
+  fi
+  is_mounted "$mountpoint" || mount "$mountpoint"
+  chown -R "$run_as_user:$run_as_user" "$mountpoint"
+  echo "mount-disks: $dev mounted at $mountpoint"
+}
+
+%{ for m in mounts ~}
+mount_one "${m.mountpoint}" "${m.fstype}"${length(m.devices) > 0 ? " ${join(" ", m.devices)}" : ""}
+%{ endfor ~}

terraform/modules/core/templates/security.toml.tftpl

@@ -0,0 +1,53 @@
+# SeaweedFS security.toml rendered by the Terraform core module.
+# Rendered when enable_security OR any non-default JWT signing option is set.
+# Presence of this file does NOT imply mTLS: the [grpc] TLS block is emitted
+# only under enable_security; a JWT-only config carries [jwt.*] and no TLS.
+
+%{ if enable_security ~}
+[grpc]
+ca = "${cert_dir}/ca/tls.crt"
+# Peer-identity authorization: without an allow-list, mTLS collapses to
+# "trusted the CA" with no authZ. Set allowed_wildcard_domain / commonNames.
+%{ if allowed_wildcard_domain != "" ~}
+allowed_wildcard_domain = "${allowed_wildcard_domain}"
+%{ endif ~}
+
+[grpc.master]
+cert = "${cert_dir}/master/tls.crt"
+key  = "${cert_dir}/master/tls.key"
+%{ if allowed_common_names != "" ~}
+allowed_commonNames = "${allowed_common_names}"
+%{ endif ~}
+
+[grpc.volume]
+cert = "${cert_dir}/volume/tls.crt"
+key  = "${cert_dir}/volume/tls.key"
+
+[grpc.filer]
+cert = "${cert_dir}/filer/tls.crt"
+key  = "${cert_dir}/filer/tls.key"
+
+[grpc.client]
+cert = "${cert_dir}/client/tls.crt"
+key  = "${cert_dir}/client/tls.key"
+%{ endif ~}
+%{ if jwt_signing_key != "" ~}
+
+[jwt.signing]
+key = "${jwt_signing_key}"
+%{ endif ~}
+%{ if jwt_signing_read_key != "" ~}
+
+[jwt.signing.read]
+key = "${jwt_signing_read_key}"
+%{ endif ~}
+%{ if jwt_filer_signing_key != "" ~}
+
+[jwt.filer_signing]
+key = "${jwt_filer_signing_key}"
+%{ endif ~}
+%{ if jwt_filer_signing_read_key != "" ~}
+
+[jwt.filer_signing.read]
+key = "${jwt_filer_signing_read_key}"
+%{ endif ~}

terraform/modules/core/templates/weed.service.tftpl

@@ -0,0 +1,39 @@
+[Unit]
+Description=SeaweedFS ${role} (${name})
+Documentation=https://github.com/seaweedfs/seaweedfs/wiki
+After=network-online.target
+Wants=network-online.target
+%{ for path in requires_mounts_for ~}
+RequiresMountsFor=${path}
+%{ endfor ~}
+
+[Service]
+Type=simple
+User=${run_as_user}
+Group=${run_as_user}
+ExecStart=${exec_start}
+Restart=always
+RestartSec=5
+LimitNOFILE=1048576
+TimeoutStopSec=60
+# Hardening (OpenShift SCC analogs; relax via the hardening variable)
+NoNewPrivileges=${no_new_privileges}
+%{ if protect_system ~}
+ProtectSystem=strict
+ProtectHome=true
+PrivateTmp=true
+%{ for path in read_write_paths ~}
+ReadWritePaths=${path}
+%{ endfor ~}
+%{ endif ~}
+%{ if cap_drop_all ~}
+CapabilityBoundingSet=
+AmbientCapabilities=
+%{ endif ~}
+EnvironmentFile=-${env_file}
+%{ for k, v in environment ~}
+Environment="${k}=${v}"
+%{ endfor ~}
+
+[Install]
+WantedBy=multi-user.target

terraform/modules/core/tests/render.tftest.hcl

@@ -0,0 +1,234 @@
+# Plan-level tests for the core renderer. No cloud credentials required.
+#   cd terraform/modules/core && tofu test
+
+variables {
+  weed_binary = "/usr/bin/weed"
+}
+
+# ---------------------------------------------------------------------------
+run "master_peers_and_count" {
+  command = plan
+  variables {
+    master = {
+      nodes = {
+        m0 = { address = "10.0.0.10" }
+        m1 = { address = "10.0.0.11" }
+        m2 = { address = "10.0.0.12" }
+      }
+    }
+    volume = { enabled = false }
+    filer  = { enabled = false }
+  }
+  assert {
+    condition     = output.master_peers == "10.0.0.10:9333,10.0.0.11:9333,10.0.0.12:9333"
+    error_message = "master -peers list is wrong"
+  }
+  assert {
+    condition     = length(output.node_names) == 3
+    error_message = "expected exactly 3 master nodes"
+  }
+}
+
+# ---------------------------------------------------------------------------
+run "volume_uses_mserver_not_master" {
+  command = plan
+  variables {
+    master = { nodes = { m0 = { address = "10.0.0.10" } } }
+    volume = { nodes = { v0 = { address = "10.0.0.20", rack = "rack-a", data_center = "dc1" } } }
+    filer  = { enabled = false }
+  }
+  assert {
+    condition     = contains(output.nodes["volume-v0"].argv, "-mserver=10.0.0.10:9333")
+    error_message = "volume must pass the master list via -mserver (verified flag), not -master"
+  }
+  assert {
+    condition     = !contains(output.nodes["volume-v0"].argv, "-master=10.0.0.10:9333")
+    error_message = "volume should not use -master for the master list"
+  }
+  assert {
+    condition     = contains(output.nodes["volume-v0"].argv, "-rack=rack-a") && contains(output.nodes["volume-v0"].argv, "-dataCenter=dc1")
+    error_message = "per-node rack/dataCenter must render"
+  }
+}
+
+# ---------------------------------------------------------------------------
+run "filer_uses_master_and_store_dir" {
+  command = plan
+  variables {
+    master = { nodes = { m0 = { address = "10.0.0.10" } } }
+    volume = { enabled = false }
+    filer  = { nodes = { f0 = { address = "10.0.0.30", data_dir = "/srv/filer" } } }
+  }
+  assert {
+    condition     = contains(output.nodes["filer-f0"].argv, "-master=10.0.0.10:9333")
+    error_message = "filer must use -master for the master list"
+  }
+  assert {
+    condition     = contains(output.nodes["filer-f0"].argv, "-defaultStoreDir=/srv/filer")
+    error_message = "filer leveldb2 store dir must render via -defaultStoreDir"
+  }
+}
+
+# ---------------------------------------------------------------------------
+run "metrics_gated_off_by_default" {
+  command = plan
+  variables {
+    master = { nodes = { m0 = { address = "10.0.0.10" } } }
+    volume = { enabled = false }
+    filer  = { enabled = false }
+  }
+  assert {
+    condition     = length([for a in output.nodes["master-m0"].argv : a if startswith(a, "-metricsPort")]) == 0
+    error_message = "metrics port must NOT be bound when monitoring is disabled"
+  }
+}
+
+run "metrics_on_when_enabled" {
+  command = plan
+  variables {
+    monitoring_enabled = true
+    master             = { nodes = { m0 = { address = "10.0.0.10" } }, metrics_port = 9327 }
+    volume             = { enabled = false }
+    filer              = { enabled = false }
+  }
+  assert {
+    condition     = contains(output.nodes["master-m0"].argv, "-metricsPort=9327")
+    error_message = "metrics port must bind when monitoring is enabled"
+  }
+}
+
+# ---------------------------------------------------------------------------
+run "security_toml_off_by_default" {
+  command = plan
+  variables {
+    master = { nodes = { m0 = { address = "10.0.0.10" } } }
+    volume = { enabled = false }
+    filer  = { enabled = false }
+  }
+  assert {
+    condition     = length(keys(output.nodes["master-m0"].config_files)) == 0
+    error_message = "security.toml must not render when security is off and no JWT keys set"
+  }
+}
+
+run "security_toml_on_with_mtls" {
+  command = plan
+  variables {
+    enable_security = true
+    security = {
+      allowed_wildcard_domain = ".seaweedfs.internal"
+      jwt_signing_key         = "test-signing-key-not-a-real-secret-0123456789"
+    }
+    master = { nodes = { m0 = { address = "10.0.0.10" } } }
+    volume = { enabled = false }
+    filer  = { enabled = false }
+  }
+  assert {
+    condition     = contains(keys(output.nodes["master-m0"].config_files), "/etc/seaweedfs/security.toml")
+    error_message = "security.toml must render when enable_security is true"
+  }
+}
+
+# ---------------------------------------------------------------------------
+run "all_in_one_renders_server" {
+  command = plan
+  variables {
+    master = { enabled = false }
+    volume = { enabled = false }
+    filer  = { enabled = false }
+    all_in_one = {
+      enabled = true
+      nodes   = { a0 = { address = "10.0.0.40" } }
+      s3      = { enabled = true }
+    }
+  }
+  assert {
+    condition     = contains(output.nodes["all-in-one-a0"].argv, "server")
+    error_message = "all-in-one must invoke the weed server subcommand"
+  }
+  assert {
+    condition     = contains(output.nodes["all-in-one-a0"].argv, "-s3") && contains(output.nodes["all-in-one-a0"].argv, "-s3.port=8333")
+    error_message = "all-in-one S3 must render when enabled"
+  }
+}
+
+# ---------------------------------------------------------------------------
+run "security_toml_mtls_only_no_jwt" {
+  command = plan
+  variables {
+    enable_security = true
+    master          = { nodes = { m0 = { address = "10.0.0.10" } } }
+    volume          = { enabled = false }
+    filer           = { enabled = false }
+  }
+  assert {
+    condition     = contains(keys(output.nodes["master-m0"].config_files), "/etc/seaweedfs/security.toml")
+    error_message = "security.toml must render for mTLS even without JWT keys"
+  }
+  assert {
+    condition     = !strcontains(output.nodes["master-m0"].config_files["/etc/seaweedfs/security.toml"], "[jwt.signing]")
+    error_message = "the jwt.signing block must be omitted when no signing key is set"
+  }
+}
+
+run "mount_fetch_and_secret_delivery" {
+  command = plan
+  variables {
+    enable_security     = true
+    render_secret_files = false
+    boot_fetch_script   = "#!/usr/bin/env bash\necho fetch\n"
+    security = {
+      jwt_signing_key = "test-signing-key-not-a-real-secret-0123456789"
+    }
+    master = { nodes = { m0 = { address = "10.0.0.10", disk_mounts = [{ mountpoint = "/data", fstype = "xfs" }] } } }
+    volume = { enabled = false }
+    filer  = { enabled = false }
+  }
+  assert {
+    condition     = strcontains(output.nodes["master-m0"].cloud_init, "/opt/seaweedfs/mount-disks.sh")
+    error_message = "cloud-init must write+run the mount script when disk_mounts is set"
+  }
+  assert {
+    condition     = strcontains(output.nodes["master-m0"].cloud_init, "/opt/seaweedfs/fetch-secrets.sh")
+    error_message = "cloud-init must write+run the fetch script when boot_fetch_script is set"
+  }
+  assert {
+    condition     = !strcontains(output.nodes["master-m0"].cloud_init, "[jwt.signing]")
+    error_message = "security.toml must NOT be inlined in cloud-init when render_secret_files=false"
+  }
+  assert {
+    condition     = contains(keys(output.nodes["master-m0"].secret_files), "/etc/seaweedfs/security.toml")
+    error_message = "secret_files must expose security.toml for secret-store delivery"
+  }
+}
+
+run "mount_script_absent_without_disks" {
+  command = plan
+  variables {
+    master = { nodes = { m0 = { address = "10.0.0.10" } } }
+    volume = { enabled = false }
+    filer  = { enabled = false }
+  }
+  assert {
+    condition     = !strcontains(output.nodes["master-m0"].cloud_init, "mount-disks.sh")
+    error_message = "no mount script should render when the node declares no disk_mounts"
+  }
+}
+
+run "s3_standalone_targets_filer" {
+  command = plan
+  variables {
+    master = { nodes = { m0 = { address = "10.0.0.10" } } }
+    volume = { enabled = false }
+    filer  = { nodes = { f0 = { address = "10.0.0.30" } } }
+    s3     = { enabled = true, nodes = { s0 = { address = "10.0.0.50" } } }
+  }
+  assert {
+    condition     = contains(output.nodes["s3-s0"].argv, "-filer=10.0.0.30:8888")
+    error_message = "standalone S3 must point -filer at the first filer node"
+  }
+  assert {
+    condition     = contains(output.nodes["s3-s0"].argv, "-config=/etc/seaweedfs/s3_config.json")
+    error_message = "standalone S3 must reference the rendered config path"
+  }
+}

terraform/modules/core/variables.tf

@@ -0,0 +1,318 @@
+# =============================================================================
+# SeaweedFS Terraform core module - input variables
+#
+# This module is a PURE RENDERER. It creates zero cloud resources. It takes a
+# per-node address map plus configuration and emits, per node, the rendered
+# `weed` argv, systemd unit, cloud-init, and config files. Both the per-cloud
+# infra wrappers (consume cloud_init/systemd_unit) and the local test harness
+# (consume argv/config_files) read its `nodes` output.
+#
+# Flag names below are verified against the real `weed` binary, not the chart.
+# Notably: volume uses -mserver for the master list; gRPC is -port.grpc
+# (auto = http+10000); minFreeSpacePercent is a string.
+# =============================================================================
+
+variable "weed_binary" {
+  description = "Path to the weed executable on the target host."
+  type        = string
+  default     = "/usr/bin/weed"
+}
+
+variable "cluster_name" {
+  description = "Logical cluster name (WEED_CLUSTER_DEFAULT)."
+  type        = string
+  default     = "sw"
+}
+
+variable "log_level" {
+  description = "Global glog verbosity (-v)."
+  type        = number
+  default     = 1
+}
+
+variable "log_to_stderr" {
+  description = "Log to stderr (journald) instead of -logdir. Recommended on systemd."
+  type        = bool
+  default     = true
+}
+
+variable "monitoring_enabled" {
+  description = "Gate the -metricsPort/-metrics.address flags. When false the metrics port is not bound, matching the chart."
+  type        = bool
+  default     = false
+}
+
+variable "enable_security" {
+  description = "Emit the security.toml [grpc] mTLS block and reference per-component certs. Independent of JWT."
+  type        = bool
+  default     = false
+}
+
+# ----------------------------------------------------------------------------
+# Security / secrets (the core only references paths + renders security.toml;
+# cert/secret material is generated by the wrapper, never defaulted here).
+# ----------------------------------------------------------------------------
+variable "security" {
+  description = "mTLS + JWT signing configuration. Secret values must be supplied (never defaulted in module code)."
+  type = object({
+    cert_dir                   = optional(string, "/usr/local/share/ca-certificates")
+    allowed_wildcard_domain    = optional(string, "")
+    allowed_common_names       = optional(string, "")
+    jwt_signing_key            = optional(string, "")
+    jwt_signing_read_key       = optional(string, "")
+    jwt_filer_signing_key      = optional(string, "")
+    jwt_filer_signing_read_key = optional(string, "")
+  })
+  default   = {}
+  sensitive = true
+}
+
+variable "hardening" {
+  description = "systemd hardening directives (OpenShift SCC analogs). Relax per field if needed."
+  type = object({
+    run_as_user       = optional(string, "seaweedfs")
+    no_new_privileges = optional(bool, true)
+    protect_system    = optional(bool, true)
+    cap_drop_all      = optional(bool, true)
+  })
+  default = {}
+}
+
+variable "env_file" {
+  description = "Path to the systemd EnvironmentFile holding secret env (DB creds, S3 admin key) fetched at boot. Optional (-prefixed in the unit, so a missing file is tolerated)."
+  type        = string
+  default     = "/etc/seaweedfs/weed.env"
+}
+
+variable "render_secret_files" {
+  description = <<-EOT
+    When true (default), secret-bearing config (security.toml, S3 identity JSON)
+    is written directly into cloud-init user_data. Convenient for the local test
+    harness and simple deployments. Set to false to keep secrets OUT of user_data
+    (and out of the cloud metadata service): the files are then exposed only via
+    the secret_files_by_node output, and the wrapper is expected to deliver them
+    from a secret store via boot_fetch_script.
+  EOT
+  type        = bool
+  default     = true
+}
+
+variable "boot_fetch_script" {
+  description = "Optional shell script written to /opt/seaweedfs/fetch-secrets.sh (root, 0755) and run before the weed unit starts. Used by wrappers to pull certs/secrets from a cloud secret store at boot."
+  type        = string
+  default     = ""
+}
+
+# ----------------------------------------------------------------------------
+# Master (Raft quorum). Keyed `nodes` map -> for_each, never count.
+# ----------------------------------------------------------------------------
+variable "master" {
+  description = "Master tier. nodes is a map keyed by stable node id (m0/m1/m2)."
+  type = object({
+    enabled = optional(bool, true)
+    nodes = optional(map(object({
+      address   = string
+      port      = optional(number)
+      grpc_port = optional(number)
+      data_dir  = optional(string)
+      disk_mounts = optional(list(object({
+        mountpoint = string
+        fstype     = optional(string, "ext4")
+        devices    = optional(list(string), [])
+      })), [])
+    })), {})
+    port                 = optional(number, 9333)
+    grpc_port            = optional(number, null) # null => weed auto = port+10000
+    metrics_port         = optional(number, 9327)
+    metrics_ip           = optional(string, "")
+    ip_bind              = optional(string, "0.0.0.0")
+    data_dir             = optional(string, "/var/lib/seaweedfs/master")
+    default_replication  = optional(string, "000")
+    volume_size_limit_mb = optional(number, 1000)
+    volume_preallocate   = optional(bool, false)
+    garbage_threshold    = optional(number, null)
+    disable_http         = optional(bool, false)
+    raft_hashicorp       = optional(bool, false)
+    resume_state         = optional(bool, true) # deliberate deviation: binary default false
+    election_timeout     = optional(string, "10s")
+    heartbeat_interval   = optional(string, "300ms")
+    white_list           = optional(string, "")
+    extra_args           = optional(list(string), [])
+  })
+  default = {}
+}
+
+# ----------------------------------------------------------------------------
+# Volume (stateful disk owners). Keyed `nodes` map.
+# ----------------------------------------------------------------------------
+variable "volume" {
+  description = "Volume tier. Each node owns one or more data dirs (-> -dir / -max)."
+  type = object({
+    enabled = optional(bool, true)
+    nodes = optional(map(object({
+      address     = string
+      port        = optional(number)
+      grpc_port   = optional(number)
+      public_url  = optional(string)
+      rack        = optional(string)
+      data_center = optional(string)
+      data_dirs = optional(list(object({
+        path        = string
+        max_volumes = optional(number, 0) # 0 => auto on non-windows
+      })))
+      idx_dir = optional(string)
+      disk_mounts = optional(list(object({
+        mountpoint = string
+        fstype     = optional(string, "ext4")
+        devices    = optional(list(string), [])
+      })), [])
+    })), {})
+    port         = optional(number, 8080)
+    metrics_port = optional(number, 9327)
+    metrics_ip   = optional(string, "")
+    ip_bind      = optional(string, "0.0.0.0")
+    data_dirs = optional(list(object({
+      path        = string
+      max_volumes = optional(number, 0)
+    })), [{ path = "/data", max_volumes = 0 }])
+    idx_dir                = optional(string, "")
+    read_mode              = optional(string, "proxy")
+    compaction_mbps        = optional(number, 50)
+    min_free_space_percent = optional(string, "1")
+    file_size_limit_mb     = optional(number, null)
+    index                  = optional(string, "memory")
+    images_fix_orientation = optional(bool, false)
+    white_list             = optional(string, "")
+    extra_args             = optional(list(string), [])
+  })
+  default = {}
+}
+
+# ----------------------------------------------------------------------------
+# Filer (+ optional embedded S3). Keyed `nodes` map.
+# ----------------------------------------------------------------------------
+variable "filer" {
+  description = "Filer tier. Default leveldb2-replicated HA (each filer has its own store)."
+  type = object({
+    enabled = optional(bool, true)
+    nodes = optional(map(object({
+      address   = string
+      port      = optional(number)
+      grpc_port = optional(number)
+      data_dir  = optional(string)
+      disk_mounts = optional(list(object({
+        mountpoint = string
+        fstype     = optional(string, "ext4")
+        devices    = optional(list(string), [])
+      })), [])
+    })), {})
+    port                      = optional(number, 8888)
+    metrics_port              = optional(number, 9327)
+    metrics_ip                = optional(string, "")
+    ip_bind                   = optional(string, "0.0.0.0")
+    data_dir                  = optional(string, "/var/lib/seaweedfs/filer") # -defaultStoreDir for leveldb2
+    default_replica_placement = optional(string, "000")
+    dir_list_limit            = optional(number, 100000)
+    max_mb                    = optional(number, null)
+    disable_dir_listing       = optional(bool, false)
+    disable_http              = optional(bool, false)
+    encrypt_volume_data       = optional(bool, false)
+    rack                      = optional(string, "")
+    data_center               = optional(string, "")
+    filer_group               = optional(string, "")
+    extra_env                 = optional(map(string), {})
+    extra_args                = optional(list(string), [])
+    s3 = optional(object({
+      enabled     = optional(bool, false)
+      port        = optional(number, 8333)
+      https_port  = optional(number, 0)
+      domain_name = optional(string, "")
+      config_path = optional(string, "/etc/seaweedfs/s3_config.json")
+    }), {})
+  })
+  default = {}
+}
+
+# ----------------------------------------------------------------------------
+# S3 gateway (standalone). Keyed `nodes` map (stateless, but kept keyed for
+# parity; wrappers may instead drive an ASG).
+# ----------------------------------------------------------------------------
+variable "s3" {
+  description = "Standalone S3 gateway tier."
+  type = object({
+    enabled = optional(bool, false)
+    nodes = optional(map(object({
+      address   = string
+      port      = optional(number)
+      grpc_port = optional(number)
+    })), {})
+    port                  = optional(number, 8333)
+    https_port            = optional(number, 0)
+    iceberg_port          = optional(number, null)
+    metrics_port          = optional(number, 9327)
+    ip_bind               = optional(string, "0.0.0.0")
+    domain_name           = optional(string, "")
+    filer_address         = optional(string, "") # override; else first filer node
+    config_path           = optional(string, "/etc/seaweedfs/s3_config.json")
+    audit_log_config_path = optional(string, "")
+    cert_file             = optional(string, "")
+    key_file              = optional(string, "")
+    cacert_file           = optional(string, "")
+    verify_client_cert    = optional(bool, false)
+    extra_args            = optional(list(string), [])
+  })
+  default = {}
+}
+
+# ----------------------------------------------------------------------------
+# S3 identities -> rendered into the s3 config JSON (non-admin only; the admin
+# credential is delivered via EnvironmentFile, never the JSON).
+# ----------------------------------------------------------------------------
+variable "s3_identities" {
+  description = "Non-admin S3 identities rendered into the config JSON. Admin key goes in the EnvironmentFile, not here."
+  type = list(object({
+    name       = string
+    access_key = string
+    secret_key = string
+    actions    = list(string)
+  }))
+  default   = []
+  sensitive = true
+}
+
+# ----------------------------------------------------------------------------
+# All-in-one (single `weed server` process).
+# ----------------------------------------------------------------------------
+variable "all_in_one" {
+  description = "All-in-one tier (weed server). Mutually exclusive with the distributed tiers."
+  type = object({
+    enabled = optional(bool, false)
+    nodes = optional(map(object({
+      address  = string
+      data_dir = optional(string)
+      disk_mounts = optional(list(object({
+        mountpoint = string
+        fstype     = optional(string, "ext4")
+        devices    = optional(list(string), [])
+      })), [])
+    })), {})
+    data_dir             = optional(string, "/data")
+    ip_bind              = optional(string, "0.0.0.0")
+    master_port          = optional(number, 9333)
+    volume_port          = optional(number, 8080)
+    filer_port           = optional(number, 8888)
+    default_replication  = optional(string, "000")
+    volume_size_limit_mb = optional(number, 1000)
+    idle_timeout         = optional(number, 30)
+    metrics_port         = optional(number, 9324)
+    disable_http         = optional(bool, false)
+    s3 = optional(object({
+      enabled     = optional(bool, false)
+      port        = optional(number, 8333)
+      config_path = optional(string, "/etc/seaweedfs/s3/s3_config.json")
+      domain_name = optional(string, "")
+    }), {})
+    extra_args = optional(list(string), [])
+  })
+  default = {}
+}

terraform/modules/core/versions.tf

@@ -0,0 +1,6 @@
+terraform {
+  # 1.3+ is required for optional() object attributes with defaults.
+  # The published wrappers pin a single honest floor;
+  # the core itself only needs language features available since 1.3.
+  required_version = ">= 1.3.0"
+}

terraform/modules/security/main.tf

@@ -0,0 +1,68 @@
+locals {
+  components = ["master", "volume", "filer", "client"]
+  ip_sans    = distinct(concat(["127.0.0.1"], var.ip_sans))
+}
+
+# ---- CA ---------------------------------------------------------------------
+resource "tls_private_key" "ca" {
+  algorithm   = var.key_algorithm
+  ecdsa_curve = var.ecdsa_curve
+  rsa_bits    = var.rsa_bits
+}
+
+resource "tls_self_signed_cert" "ca" {
+  private_key_pem   = tls_private_key.ca.private_key_pem
+  is_ca_certificate = true
+
+  subject {
+    common_name  = "SeaweedFS CA"
+    organization = "SeaweedFS"
+  }
+
+  validity_period_hours = var.ca_validity_hours
+  allowed_uses          = ["cert_signing", "crl_signing", "digital_signature"]
+}
+
+# ---- per-component leaf certs (distinct CN per component) --------------------
+resource "tls_private_key" "component" {
+  for_each    = toset(local.components)
+  algorithm   = var.key_algorithm
+  ecdsa_curve = var.ecdsa_curve
+  rsa_bits    = var.rsa_bits
+}
+
+resource "tls_cert_request" "component" {
+  for_each        = toset(local.components)
+  private_key_pem = tls_private_key.component[each.key].private_key_pem
+
+  subject {
+    # Distinct CN per component so peer-identity authZ (allowed_wildcard_domain
+    # / allowed_commonNames) is meaningful. Do NOT use one CN for all.
+    common_name  = "${each.key}.${var.internal_domain}"
+    organization = "SeaweedFS"
+  }
+
+  dns_names = distinct(concat(
+    ["${each.key}.${var.internal_domain}", "localhost"],
+    var.extra_dns_sans,
+  ))
+  ip_addresses = local.ip_sans
+}
+
+resource "tls_locally_signed_cert" "component" {
+  for_each           = toset(local.components)
+  cert_request_pem   = tls_cert_request.component[each.key].cert_request_pem
+  ca_private_key_pem = tls_private_key.ca.private_key_pem
+  ca_cert_pem        = tls_self_signed_cert.ca.cert_pem
+
+  validity_period_hours = var.cert_validity_hours
+  early_renewal_hours   = var.cert_early_renewal_hours
+  allowed_uses          = ["digital_signature", "key_agreement", "server_auth", "client_auth"]
+}
+
+# ---- JWT signing keys -------------------------------------------------------
+resource "random_password" "jwt" {
+  for_each = var.generate_jwt ? toset(["signing", "signing_read", "filer_signing", "filer_signing_read"]) : toset([])
+  length   = var.jwt_length
+  special  = false # TOML-safe
+}

terraform/modules/security/outputs.tf

@@ -0,0 +1,65 @@
+output "ca_cert_pem" {
+  description = "CA certificate PEM."
+  value       = tls_self_signed_cert.ca.cert_pem
+}
+
+output "certs" {
+  description = "Per-component { cert_pem, private_key_pem }."
+  value = {
+    for c in local.components : c => {
+      cert_pem        = tls_locally_signed_cert.component[c].cert_pem
+      private_key_pem = tls_private_key.component[c].private_key_pem
+    }
+  }
+  sensitive = true
+}
+
+output "jwt" {
+  description = "Generated JWT keys (empty strings when generate_jwt = false)."
+  value = {
+    signing            = var.generate_jwt ? random_password.jwt["signing"].result : ""
+    signing_read       = var.generate_jwt ? random_password.jwt["signing_read"].result : ""
+    filer_signing      = var.generate_jwt ? random_password.jwt["filer_signing"].result : ""
+    filer_signing_read = var.generate_jwt ? random_password.jwt["filer_signing_read"].result : ""
+  }
+  sensitive = true
+}
+
+# Ready to pass straight to the core module's `security` variable.
+output "core_security" {
+  description = "Object shaped for the core module's `security` input (cert_dir, allowed_wildcard_domain, JWT keys)."
+  value = {
+    cert_dir                   = var.cert_dir
+    allowed_wildcard_domain    = ".${var.internal_domain}"
+    allowed_common_names       = ""
+    jwt_signing_key            = var.generate_jwt ? random_password.jwt["signing"].result : ""
+    jwt_signing_read_key       = var.generate_jwt ? random_password.jwt["signing_read"].result : ""
+    jwt_filer_signing_key      = var.generate_jwt ? random_password.jwt["filer_signing"].result : ""
+    jwt_filer_signing_read_key = var.generate_jwt ? random_password.jwt["filer_signing_read"].result : ""
+  }
+  sensitive = true
+}
+
+# Non-sensitive manifest (key, on-host path, mode) so a wrapper can for_each
+# over SSM/secret-store resources without tripping the sensitive-for_each rule.
+output "secret_manifest" {
+  description = "List of { key, path, mode } for the CA + component cert/key files (no content)."
+  value = concat(
+    [{ key = "ca/tls.crt", path = "${var.cert_dir}/ca/tls.crt", mode = "0644" }],
+    flatten([for c in local.components : [
+      { key = "${c}/tls.crt", path = "${var.cert_dir}/${c}/tls.crt", mode = "0644" },
+      { key = "${c}/tls.key", path = "${var.cert_dir}/${c}/tls.key", mode = "0600" },
+    ]]),
+  )
+}
+
+# Sensitive map keyed by the manifest key -> file content.
+output "secret_contents" {
+  description = "Map of manifest key -> PEM content."
+  value = merge(
+    { "ca/tls.crt" = tls_self_signed_cert.ca.cert_pem },
+    { for c in local.components : "${c}/tls.crt" => tls_locally_signed_cert.component[c].cert_pem },
+    { for c in local.components : "${c}/tls.key" => tls_private_key.component[c].private_key_pem },
+  )
+  sensitive = true
+}

terraform/modules/security/variables.tf

@@ -0,0 +1,89 @@
+# =============================================================================
+# SeaweedFS security material generator (cloud-agnostic).
+#
+# Generates the CA, per-component mTLS certs with DISTINCT CommonNames, and JWT
+# signing keys. Outputs a `core_security` object ready to feed the core module's
+# `security` variable, plus the raw PEMs for a wrapper to deliver via a secret
+# store. NOTE: TF-generated secrets live in state; prefer Vault PKI for the CA
+# in production.
+# =============================================================================
+
+variable "internal_domain" {
+  description = "Internal domain for component CNs (e.g. master.<domain>). Drives the allowed_wildcard_domain peer-auth check."
+  type        = string
+  default     = "seaweedfs.internal"
+}
+
+variable "ip_sans" {
+  description = "IP addresses to include as SANs on every component cert (all node private IPs + 127.0.0.1 is added automatically)."
+  type        = list(string)
+  default     = []
+}
+
+variable "extra_dns_sans" {
+  description = "Additional DNS SANs to include on every component cert."
+  type        = list(string)
+  default     = []
+}
+
+variable "key_algorithm" {
+  description = "CA/leaf key algorithm: ECDSA (default, modern) or RSA."
+  type        = string
+  default     = "ECDSA"
+  validation {
+    condition     = contains(["ECDSA", "RSA"], var.key_algorithm)
+    error_message = "key_algorithm must be ECDSA or RSA."
+  }
+}
+
+variable "ecdsa_curve" {
+  description = "ECDSA curve when key_algorithm = ECDSA."
+  type        = string
+  default     = "P256"
+}
+
+variable "rsa_bits" {
+  description = "RSA key size when key_algorithm = RSA."
+  type        = number
+  default     = 3072
+}
+
+variable "ca_validity_hours" {
+  description = "CA certificate validity (default 10 years)."
+  type        = number
+  default     = 87600
+}
+
+variable "cert_validity_hours" {
+  description = "Component certificate validity. Short by default; certs hot-reload so reissue is cheap."
+  type        = number
+  default     = 72
+}
+
+variable "cert_early_renewal_hours" {
+  description = "Renew component certs this many hours before expiry."
+  type        = number
+  default     = 24
+}
+
+variable "cert_dir" {
+  description = "On-host directory where certs are placed; surfaced in core_security.cert_dir."
+  type        = string
+  default     = "/usr/local/share/ca-certificates"
+}
+
+variable "generate_jwt" {
+  description = "Generate JWT signing keys (signing, signing.read, filer_signing, filer_signing.read)."
+  type        = bool
+  default     = true
+}
+
+variable "jwt_length" {
+  description = "Length of generated JWT keys (>= 32 hardened)."
+  type        = number
+  default     = 40
+  validation {
+    condition     = var.jwt_length >= 32
+    error_message = "jwt_length must be >= 32 (hardened minimum)."
+  }
+}

terraform/modules/security/versions.tf

@@ -0,0 +1,13 @@
+terraform {
+  required_version = ">= 1.3.0"
+  required_providers {
+    tls = {
+      source  = "hashicorp/tls"
+      version = ">= 4.0"
+    }
+    random = {
+      source  = "hashicorp/random"
+      version = ">= 3.5"
+    }
+  }
+}

terraform/test/local-secure/main.tf

@@ -0,0 +1,74 @@
+# =============================================================================
+# Local mTLS test harness: generate the CA + component certs + JWT keys with the
+# security submodule, render security.toml with the core, then run a real
+# master+volume+filer cluster with mTLS enabled on 127.0.0.1 and assert it works.
+# Proves the generated security material is valid and accepted by weed.
+# =============================================================================
+
+terraform {
+  required_version = ">= 1.3.0"
+}
+
+variable "weed_binary" {
+  type    = string
+  default = "/usr/bin/weed"
+}
+
+variable "workdir" {
+  type    = string
+  default = "/tmp/seaweedfs-tftest-secure"
+}
+
+module "security" {
+  source          = "../../modules/security"
+  internal_domain = "seaweedfs.internal"
+  ip_sans         = ["127.0.0.1"]
+  cert_dir        = "${var.workdir}/certs"
+}
+
+module "core" {
+  source          = "../../modules/core"
+  weed_binary     = var.weed_binary
+  enable_security = true
+  security        = module.security.core_security
+  # We read security.toml back from the output and place it ourselves (under
+  # $HOME/.seaweedfs), so keep it in the rendered file set.
+  render_secret_files = true
+
+  # High, distinct ports so this never collides with a running SeaweedFS or the
+  # non-secure harness.
+  master = {
+    nodes              = { m0 = { address = "127.0.0.1", port = 29336, data_dir = "${var.workdir}/master-m0" } }
+    election_timeout   = "3s"
+    heartbeat_interval = "200ms"
+  }
+  volume = {
+    nodes = { v0 = { address = "127.0.0.1", port = 28081, data_dirs = [{ path = "${var.workdir}/volume-v0", max_volumes = 20 }] } }
+  }
+  filer = {
+    nodes = { f0 = { address = "127.0.0.1", port = 28889, data_dir = "${var.workdir}/filer-f0" } }
+  }
+}
+
+output "cluster" {
+  sensitive = true
+  value = {
+    for name, n in module.core.nodes : name => {
+      role      = n.role
+      http_port = n.ports.http
+      data_dirs = n.data_dirs
+      argv      = n.argv
+      env       = n.env
+    }
+  }
+}
+
+output "security_toml" {
+  sensitive = true
+  value     = module.core.secret_files_by_node["master-m0"]["/etc/seaweedfs/security.toml"]
+}
+
+output "certs" {
+  sensitive = true
+  value     = [for m in module.security.secret_manifest : { path = m.path, mode = m.mode, content = module.security.secret_contents[m.key] }]
+}

terraform/test/local-secure/run_local_secure.sh

@@ -0,0 +1,128 @@
+#!/usr/bin/env bash
+# Generate mTLS material + security.toml with Terraform, then run a real
+# master+volume+filer cluster with mTLS enabled and assert it works.
+#   ./run_local_secure.sh        # render + run + assert + teardown
+#   KEEP=1 ./run_local_secure.sh
+set -u
+
+HERE="$(cd "$(dirname "$0")" && pwd)"
+export PATH="/opt/homebrew/bin:$PATH"
+TOFU="${TOFU:-tofu}"
+WEED="${WEED:-$(go env GOPATH 2>/dev/null || echo "$HOME/go")/bin/weed}"
+WORKDIR="${WORKDIR:-/tmp/seaweedfs-tftest-secure}"
+LOGDIR="$WORKDIR/logs"
+RUNDIR="$WORKDIR/run"
+
+PASS=0
+FAIL=0
+ok()   { echo "  PASS: $1"; PASS=$((PASS + 1)); }
+bad()  { echo "  FAIL: $1"; FAIL=$((FAIL + 1)); }
+info() { echo "==> $1"; }
+
+cleanup() {
+  info "tearing down"
+  if [ -d "$RUNDIR" ]; then
+    for pf in "$RUNDIR"/*.pid; do [ -f "$pf" ] && kill "$(cat "$pf")" 2>/dev/null; done
+    sleep 1
+    for pf in "$RUNDIR"/*.pid; do [ -f "$pf" ] && kill -9 "$(cat "$pf")" 2>/dev/null; done
+  fi
+}
+
+info "cleaning $WORKDIR"
+case "$WORKDIR" in "" | "/" | "$HOME") echo "refusing to delete '$WORKDIR'" >&2; exit 2 ;; esac
+rm -rf "$WORKDIR"
+mkdir -p "$LOGDIR" "$RUNDIR" "$WORKDIR/.seaweedfs"
+[ -x "$WEED" ] || { echo "weed not found at $WEED" >&2; exit 2; }
+
+info "generating certs + rendering config with OpenTofu"
+cd "$HERE"
+"$TOFU" init -backend=false -input=false -no-color >/dev/null 2>&1 || { echo "tofu init failed"; exit 2; }
+if ! "$TOFU" apply -auto-approve -input=false -no-color \
+       -var "weed_binary=$WEED" -var "workdir=$WORKDIR" >"$LOGDIR/tofu.log" 2>&1; then
+  echo "tofu apply failed"; tail -30 "$LOGDIR/tofu.log"; exit 2
+fi
+
+# write certs to their on-host paths
+"$TOFU" output -json certs | jq -c '.[]' | while IFS= read -r c; do
+  p="$(echo "$c" | jq -r '.path')"; m="$(echo "$c" | jq -r '.mode')"
+  mkdir -p "$(dirname "$p")"
+  echo "$c" | jq -r '.content' > "$p"
+  chmod "$m" "$p"
+done
+# place security.toml where weed searches ($HOME/.seaweedfs)
+"$TOFU" output -raw security_toml > "$WORKDIR/.seaweedfs/security.toml"
+info "security.toml + $(ls "$WORKDIR"/certs | wc -l | tr -d ' ') cert dirs written"
+
+OUT="$("$TOFU" output -json cluster)"
+
+# derive ports from the rendered config (high range; never hardcoded)
+port_of() { echo "$OUT" | jq -r --arg r "$1" '.[] | select(.role==$r) | .http_port' | head -1; }
+MPORT="$(port_of master)"; VPORT="$(port_of volume)"; FPORT="$(port_of filer)"
+
+busy=""
+for p in $(echo "$OUT" | jq -r '.[].http_port'); do
+  lsof -nP -iTCP:"$p" -sTCP:LISTEN >/dev/null 2>&1 && busy="$busy $p"
+done
+[ -n "$busy" ] && { echo "Required port(s) already in use:$busy -- is another SeaweedFS running?" >&2; exit 3; }
+
+[ "${KEEP:-0}" = "1" ] || trap cleanup EXIT INT TERM
+
+# launch a node with HOME pointed at $WORKDIR so weed loads security.toml
+launch() {
+  n="$1"
+  for d in $(echo "$OUT" | jq -r --arg n "$n" '.[$n].data_dirs[]?'); do mkdir -p "$d"; done
+  ENVS=("HOME=$WORKDIR")
+  while IFS= read -r e; do [ -n "$e" ] && ENVS+=("$e"); done \
+    < <(echo "$OUT" | jq -r --arg n "$n" '.[$n].env | to_entries[] | "\(.key)=\(.value)"')
+  ARGV=()
+  while IFS= read -r a; do ARGV+=("$a"); done \
+    < <(echo "$OUT" | jq -r --arg n "$n" '.[$n].argv[]')
+  info "launching $n (mTLS)"
+  env "${ENVS[@]}" "$WEED" "${ARGV[@]}" >"$LOGDIR/$n.log" 2>&1 &
+  echo "$!" > "$RUNDIR/$n.pid"
+}
+
+wait_http() {
+  url="$1"; t="${2:-30}"; i=0
+  while [ "$i" -lt "$t" ]; do
+    curl -fsS -o /dev/null --max-time 2 "$url" 2>/dev/null && return 0
+    i=$((i + 1)); sleep 1
+  done
+  return 1
+}
+
+launch master-m0
+QOK=0; i=0
+while [ "$i" -lt 40 ]; do
+  st="$(curl -fsS --max-time 2 "http://127.0.0.1:$MPORT/cluster/status" 2>/dev/null)" || { i=$((i+1)); sleep 1; continue; }
+  [ "$(echo "$st" | jq -r '.IsLeader // false')" = "true" ] && { QOK=1; break; }
+  i=$((i + 1)); sleep 1
+done
+[ "$QOK" -eq 1 ] && ok "master elected leader under mTLS" || bad "master leader (mTLS)"
+
+launch volume-v0
+# registration (master gRPC over mTLS) confirms the volume joined the cluster
+AOK=0; i=0
+while [ "$i" -lt 40 ]; do
+  fid="$(curl -fsS --max-time 2 "http://127.0.0.1:$MPORT/dir/assign" 2>/dev/null | jq -r '.fid // ""')"
+  [ -n "$fid" ] && { AOK=1; break; }
+  i=$((i + 1)); sleep 1
+done
+[ "$AOK" -eq 1 ] && ok "volume registered via mTLS gRPC (fid $fid)" || bad "volume registration over mTLS"
+# /healthz flips to 200 after the first heartbeat completes (slower under mTLS)
+wait_http "http://127.0.0.1:$VPORT/healthz" 60 && ok "volume /healthz up (mTLS)" || bad "volume /healthz"
+
+launch filer-f0
+wait_http "http://127.0.0.1:$FPORT/" 30 && ok "filer / up (mTLS)" || bad "filer /"
+# [jwt.filer_signing] is active, so the filer requires a signed JWT for writes.
+# An unsigned write MUST be rejected with 401 -- this proves the JWT signing key
+# rendered into security.toml is enforced (positive security assertion).
+echo "smoke" > "$WORKDIR/hello.txt"
+code="$(curl -s -o /dev/null -w '%{http_code}' --max-time 5 -X POST \
+  -F "file=@$WORKDIR/hello.txt" "http://127.0.0.1:$FPORT/smoke/hello.txt" 2>/dev/null)"
+[ "$code" = "401" ] && ok "filer rejects unsigned write (HTTP 401) => JWT signing enforced" \
+  || bad "filer JWT enforcement (expected 401, got HTTP $code)"
+
+echo
+info "RESULTS: $PASS passed, $FAIL failed"
+[ "$FAIL" -eq 0 ]

terraform/test/local/main.tf

@@ -0,0 +1,100 @@
+# =============================================================================
+# Local test harness: render a small SeaweedFS cluster with the core module
+# and run it as real `weed` processes on 127.0.0.1 (no cloud, no docker).
+#
+# 3 masters (quorum) + 1 volume + 1 filer + 1 standalone S3, each on a distinct
+# port. run_local_cluster.sh consumes the `cluster` output, launches the weed
+# processes from the rendered argv, and asserts the cluster actually works.
+# =============================================================================
+
+terraform {
+  required_version = ">= 1.3.0"
+}
+
+variable "weed_binary" {
+  description = "Path to the weed executable used for the local cluster."
+  type        = string
+  default     = "/usr/bin/weed"
+}
+
+variable "workdir" {
+  description = "Scratch directory for per-node data dirs and config files."
+  type        = string
+  default     = "/tmp/seaweedfs-tftest"
+}
+
+module "core" {
+  source = "../../modules/core"
+
+  weed_binary        = var.weed_binary
+  monitoring_enabled = false
+  enable_security    = false
+
+  # High port range so the harness does not collide with a SeaweedFS cluster
+  # that may already be running on this dev machine (default 9333/8080/8888/8333).
+  master = {
+    nodes = {
+      m0 = { address = "127.0.0.1", port = 29333, data_dir = "${var.workdir}/master-m0" }
+      m1 = { address = "127.0.0.1", port = 29334, data_dir = "${var.workdir}/master-m1" }
+      m2 = { address = "127.0.0.1", port = 29335, data_dir = "${var.workdir}/master-m2" }
+    }
+    # Tighten election so a 3-node local quorum converges quickly.
+    election_timeout   = "3s"
+    heartbeat_interval = "200ms"
+  }
+
+  volume = {
+    nodes = {
+      v0 = {
+        address   = "127.0.0.1"
+        port      = 28080
+        rack      = "rack-a"
+        data_dirs = [{ path = "${var.workdir}/volume-v0", max_volumes = 20 }]
+      }
+    }
+  }
+
+  filer = {
+    nodes = {
+      f0 = { address = "127.0.0.1", port = 28888, data_dir = "${var.workdir}/filer-f0" }
+    }
+  }
+
+  s3 = {
+    enabled = true
+    nodes   = { s0 = { address = "127.0.0.1", port = 28333 } }
+    # this weed build starts an Iceberg REST catalog on 8181 by default; disable
+    # it so the test does not collide with a cluster already using that port.
+    iceberg_port = 0
+    config_path  = "${var.workdir}/s3_config.json"
+  }
+
+  # Anonymous identity so the smoke test can PUT/GET without sigv4 signing.
+  s3_identities = [{
+    name       = "anonymous"
+    access_key = ""
+    secret_key = ""
+    actions    = ["Admin", "Read", "Write", "List", "Tagging"]
+  }]
+}
+
+output "cluster" {
+  description = "Node specs consumed by run_local_cluster.sh (read via `tofu output -json`, which emits values even when sensitive)."
+  sensitive   = true
+  value = {
+    for name, n in module.core.nodes : name => {
+      role         = n.role
+      address      = n.address
+      http_port    = n.ports.http
+      data_dirs    = n.data_dirs
+      argv         = n.argv
+      env          = n.env
+      config_files = n.config_files
+      exec_start   = n.exec_start
+    }
+  }
+}
+
+output "master_peers" {
+  value = module.core.master_peers
+}

terraform/test/local/run_local_cluster.sh

@@ -0,0 +1,170 @@
+#!/usr/bin/env bash
+# Render a SeaweedFS cluster with the Terraform core module and run it as real
+# `weed` processes locally, then assert it actually works. No cloud, no docker.
+#
+#   ./run_local_cluster.sh            # render + run + assert + teardown
+#   KEEP=1 ./run_local_cluster.sh     # leave the cluster running after asserts
+#   WEED=/path/to/weed ./run_local_cluster.sh
+#
+# Ports come from the rendered config (high range, to avoid colliding with a
+# SeaweedFS cluster already running on this machine). Exits non-zero on any
+# failed assertion or if a required port is already in use.
+set -u
+
+HERE="$(cd "$(dirname "$0")" && pwd)"
+export PATH="/opt/homebrew/bin:$PATH"
+TOFU="${TOFU:-tofu}"
+WEED="${WEED:-$(go env GOPATH 2>/dev/null || echo "$HOME/go")/bin/weed}"
+WORKDIR="${WORKDIR:-/tmp/seaweedfs-tftest}"
+LOGDIR="$WORKDIR/logs"
+RUNDIR="$WORKDIR/run"
+
+PASS=0
+FAIL=0
+ok()   { echo "  PASS: $1"; PASS=$((PASS + 1)); }
+bad()  { echo "  FAIL: $1"; FAIL=$((FAIL + 1)); }
+info() { echo "==> $1"; }
+
+cleanup() {
+  info "tearing down"
+  if [ -d "$RUNDIR" ]; then
+    for pf in "$RUNDIR"/*.pid; do [ -f "$pf" ] && kill "$(cat "$pf")" 2>/dev/null; done
+    sleep 1
+    for pf in "$RUNDIR"/*.pid; do [ -f "$pf" ] && kill -9 "$(cat "$pf")" 2>/dev/null; done
+  fi
+}
+
+info "cleaning $WORKDIR"
+case "$WORKDIR" in "" | "/" | "$HOME") echo "refusing to delete '$WORKDIR'" >&2; exit 2 ;; esac
+rm -rf "$WORKDIR"
+mkdir -p "$LOGDIR" "$RUNDIR"
+[ -x "$WEED" ] || { echo "weed binary not found/executable at $WEED" >&2; exit 2; }
+
+info "rendering cluster config with OpenTofu"
+cd "$HERE"
+"$TOFU" init -backend=false -input=false -no-color >/dev/null 2>&1 || { echo "tofu init failed"; exit 2; }
+if ! "$TOFU" apply -auto-approve -input=false -no-color \
+       -var "weed_binary=$WEED" -var "workdir=$WORKDIR" >"$LOGDIR/tofu.log" 2>&1; then
+  echo "tofu apply failed; see $LOGDIR/tofu.log"; tail -30 "$LOGDIR/tofu.log"; exit 2
+fi
+OUT="$("$TOFU" output -json cluster)"
+[ -n "$OUT" ] || { echo "empty cluster output"; exit 2; }
+
+# ---- derive ports by role (never hardcoded) ---------------------------------
+port_of() { echo "$OUT" | jq -r --arg r "$1" '.[] | select(.role==$r) | .http_port' | head -1; }
+mports()  { echo "$OUT" | jq -r '.[] | select(.role=="master") | .http_port' | sort; }
+MPORT1="$(mports | head -1)"
+VPORT="$(port_of volume)"; FPORT="$(port_of filer)"; SPORT="$(port_of s3)"
+NMASTERS="$(mports | wc -l | tr -d ' ')"
+
+# ---- refuse to run if a required port is already taken (foreign cluster) -----
+busy=""
+for p in $(echo "$OUT" | jq -r '.[].http_port'); do
+  lsof -nP -iTCP:"$p" -sTCP:LISTEN >/dev/null 2>&1 && busy="$busy $p"
+done
+[ -n "$busy" ] && { echo "Required port(s) already in use:$busy -- is another SeaweedFS running?" >&2; exit 3; }
+
+[ "${KEEP:-0}" = "1" ] || trap cleanup EXIT INT TERM
+
+node_names() { echo "$OUT" | jq -r 'keys[]'; }
+node_field() { echo "$OUT" | jq -r --arg n "$1" --arg f "$2" '.[$n][$f]'; }
+
+launch_node() {
+  n="$1"
+  for p in $(echo "$OUT" | jq -r --arg n "$n" '.[$n].config_files | keys[]?'); do
+    mkdir -p "$(dirname "$p")"
+    echo "$OUT" | jq -r --arg n "$n" --arg p "$p" '.[$n].config_files[$p]' > "$p"
+  done
+  for d in $(echo "$OUT" | jq -r --arg n "$n" '.[$n].data_dirs[]?'); do mkdir -p "$d"; done
+  ENVS=()
+  while IFS= read -r e; do [ -n "$e" ] && ENVS+=("$e"); done \
+    < <(echo "$OUT" | jq -r --arg n "$n" '.[$n].env | to_entries[] | "\(.key)=\(.value)"')
+  ARGV=()
+  while IFS= read -r a; do ARGV+=("$a"); done \
+    < <(echo "$OUT" | jq -r --arg n "$n" '.[$n].argv[]')
+  info "launching $n"
+  if [ "${#ENVS[@]}" -gt 0 ]; then
+    env "${ENVS[@]}" "$WEED" "${ARGV[@]}" >"$LOGDIR/$n.log" 2>&1 &
+  else
+    "$WEED" "${ARGV[@]}" >"$LOGDIR/$n.log" 2>&1 &
+  fi
+  echo "$!" > "$RUNDIR/$n.pid"
+}
+
+wait_http() {
+  url="$1"; t="${2:-30}"; i=0
+  while [ "$i" -lt "$t" ]; do
+    curl -fsS -o /dev/null --max-time 2 "$url" 2>/dev/null && return 0
+    i=$((i + 1)); sleep 1
+  done
+  return 1
+}
+
+# ---- launch masters and wait for quorum -------------------------------------
+for n in $(node_names); do
+  case "$(node_field "$n" role)" in master) launch_node "$n";; esac
+done
+
+info "waiting for master quorum across ports: $(mports | tr '\n' ' ')"
+QUORUM_OK=0; i=0
+while [ "$i" -lt 40 ]; do
+  LEADERS=0; AGREED_LEADER=""; mismatch=0; reachable=0
+  for port in $(mports); do
+    st="$(curl -fsS --max-time 2 "http://127.0.0.1:$port/cluster/status" 2>/dev/null)" || continue
+    reachable=$((reachable + 1))
+    [ "$(echo "$st" | jq -r '.IsLeader // false')" = "true" ] && LEADERS=$((LEADERS + 1))
+    ldr="$(echo "$st" | jq -r '.Leader // ""')"
+    if [ -n "$ldr" ]; then
+      if [ -z "$AGREED_LEADER" ]; then AGREED_LEADER="$ldr"; elif [ "$AGREED_LEADER" != "$ldr" ]; then mismatch=1; fi
+    fi
+  done
+  if [ "$reachable" -eq "$NMASTERS" ] && [ "$LEADERS" -eq 1 ] && [ "$mismatch" -eq 0 ] && [ -n "$AGREED_LEADER" ]; then
+    QUORUM_OK=1; break
+  fi
+  i=$((i + 1)); sleep 1
+done
+if [ "$QUORUM_OK" -eq 1 ]; then
+  ok "$NMASTERS-master quorum: exactly one leader ($AGREED_LEADER), all agree"
+else
+  bad "master quorum (reachable=$reachable leaders=$LEADERS leader='$AGREED_LEADER')"
+fi
+
+# ---- volume -----------------------------------------------------------------
+for n in $(node_names); do
+  case "$(node_field "$n" role)" in volume) launch_node "$n";; esac
+done
+wait_http "http://127.0.0.1:$VPORT/healthz" 40 && ok "volume /healthz up" || bad "volume /healthz"
+ASSIGN_OK=0; i=0
+while [ "$i" -lt 20 ]; do
+  fid="$(curl -fsS --max-time 2 "http://127.0.0.1:$MPORT1/dir/assign" 2>/dev/null | jq -r '.fid // ""')"
+  [ -n "$fid" ] && { ASSIGN_OK=1; break; }
+  i=$((i + 1)); sleep 1
+done
+[ "$ASSIGN_OK" -eq 1 ] && ok "master assigned fid ($fid) => volume registered" || bad "master /dir/assign (no volume?)"
+
+# ---- filer ------------------------------------------------------------------
+for n in $(node_names); do
+  case "$(node_field "$n" role)" in filer) launch_node "$n";; esac
+done
+wait_http "http://127.0.0.1:$FPORT/" 30 && ok "filer / up" || bad "filer /"
+PAYLOAD="seaweedfs-terraform-smoke-$$"
+echo "$PAYLOAD" > "$WORKDIR/hello.txt"
+curl -fsS -F "file=@$WORKDIR/hello.txt" "http://127.0.0.1:$FPORT/smoke/hello.txt" >/dev/null 2>&1
+got="$(curl -fsS --max-time 5 "http://127.0.0.1:$FPORT/smoke/hello.txt" 2>/dev/null)"
+[ "$got" = "$PAYLOAD" ] && ok "filer PUT/GET round-trip" || bad "filer round-trip (got '$got')"
+
+# ---- s3 ---------------------------------------------------------------------
+for n in $(node_names); do
+  case "$(node_field "$n" role)" in s3) launch_node "$n";; esac
+done
+wait_http "http://127.0.0.1:$SPORT/status" 30 && ok "s3 /status up" || bad "s3 /status"
+curl -fsS -X PUT "http://127.0.0.1:$SPORT/smoke-bucket" >/dev/null 2>&1
+sleep 1
+curl -fsS -X PUT --data-binary "$PAYLOAD" "http://127.0.0.1:$SPORT/smoke-bucket/obj.txt" >/dev/null 2>&1
+s3got="$(curl -fsS --max-time 5 "http://127.0.0.1:$SPORT/smoke-bucket/obj.txt" 2>/dev/null)"
+[ "$s3got" = "$PAYLOAD" ] && ok "s3 PUT/GET round-trip" || bad "s3 round-trip (got '$s3got')"
+
+echo
+info "RESULTS: $PASS passed, $FAIL failed"
+[ "${KEEP:-0}" = "1" ] && info "KEEP=1: cluster left running. Logs in $LOGDIR."
+[ "$FAIL" -eq 0 ]

test/java/spark/pom.xml

@@ -22,7 +22,7 @@
         <junit.version>4.13.2</junit.version>
         <seaweedfs.hadoop3.client.version>4.00</seaweedfs.hadoop3.client.version>
         <jackson.version>2.18.6</jackson.version> <!-- Upgraded from 2.15.3 -->
-        <netty.version>4.2.13.Final</netty.version> <!-- Upgraded to 4.1.125.Final for security fixes (CVE in netty-codec < 4.1.125.Final, netty-codec-http2 <= 4.1.123.Final) -->
+        <netty.version>4.2.15.Final</netty.version> <!-- Upgraded to 4.1.125.Final for security fixes (CVE in netty-codec < 4.1.125.Final, netty-codec-http2 <= 4.1.123.Final) -->
         <parquet.version>1.15.2</parquet.version> <!-- Upgraded to 1.15.2 for security fix -->
         <parquet.format.version>2.12.0</parquet.format.version>
         <surefire.jvm.args>

test/nfs/Makefile

@@ -1,36 +0,0 @@
-.PHONY: all build test test-verbose test-short test-debug clean deps tidy
-
-all: build test
-
-# Build the weed binary first
-build:
-	cd ../../weed && go build -o weed .
-
-# Install test dependencies
-deps:
-	go mod download
-
-# Run all tests
-test: build deps
-	go test -timeout 5m ./...
-
-# Run tests with verbose output
-test-verbose: build deps
-	go test -v -timeout 5m ./...
-
-# Skip long-running integration tests
-test-short: deps
-	go test -short -v ./...
-
-# Run tests with debug output from SeaweedFS
-test-debug: build deps
-	go test -v -timeout 5m ./... 2>&1 | tee test.log
-
-# Clean up test artifacts
-clean:
-	rm -f test.log
-	go clean -testcache
-
-# Update go.sum
-tidy:
-	go mod tidy

test/nfs/README.md

@@ -1,92 +0,0 @@
-# SeaweedFS NFS Integration Tests
-
-End-to-end tests that boot a real SeaweedFS cluster (`master` + `volume` +
-`filer`) plus the experimental `weed nfs` frontend and drive it through the
-NFSv3 wire protocol. The tests talk to the server over TCP using
-`github.com/willscott/go-nfs-client`, which means they do **not** need a
-kernel NFS mount, privileged ports, or any platform-specific tooling.
-
-## Prerequisites
-
-1. Build the `weed` binary:
-   ```bash
-   cd ../../weed
-   go build -o weed .
-   ```
-2. Go 1.24 or later.
-
-## Running the tests
-
-```bash
-# Build weed and run everything
-make test
-
-# Verbose output, keeps the subprocess stdout
-make test-verbose
-
-# Skip integration tests — useful when iterating on the framework itself
-make test-short
-
-# Run a single test
-go test -v -run TestNfsBasicReadWrite ./...
-```
-
-Every test starts its own cluster on random loopback ports, so runs are
-isolated and can execute in parallel.
-
-## Layout
-
-- `framework.go` — launches `weed master`, `weed volume`, `weed filer`, and
-  `weed nfs` as subprocesses, waits for each to accept TCP, and exposes a
-  `Mount()` helper that returns an `nfsclient.Target`.
-- `basic_test.go` — covers the most common NFS operations:
-  - Read/write round-trip (`TestNfsBasicReadWrite`)
-  - Mkdir / ReadDirPlus / RmDir (`TestNfsMkdirAndRmdir`)
-  - Nested directory + leaf file (`TestNfsNestedDirectories`)
-  - Rename preserves content (`TestNfsRenamePreservesContent`)
-  - Overwrite shrinks file size (`TestNfsOverwriteShrinksFile`)
-  - Large binary file round-trip (`TestNfsLargeFile`)
-  - Arbitrary binary and empty files (`TestNfsBinaryAndEmptyFiles`)
-  - Symlink + Readlink (`TestNfsSymlinkRoundTrip`)
-  - ReadDirPlus ordering sanity (`TestNfsReadDirPlusOrdering`)
-  - Remove on missing path errors cleanly (`TestNfsRemoveMissingFailsCleanly`)
-  - FSINFO advertises non-zero limits (`TestNfsFSInfoReturnsSaneLimits`)
-  - Sequential append writes concatenate (`TestNfsAppendIsSequential`)
-  - ReadDir after remove (`TestNfsReadDirAfterRemove`)
-
-## Debugging a failing test
-
-Keep the cluster temp dir for inspection:
-
-```go
-config := DefaultTestConfig()
-config.SkipCleanup = true
-```
-
-Enable subprocess stdout/stderr:
-
-```go
-config := DefaultTestConfig()
-config.EnableDebug = true
-```
-
-Or run with `-v`, which flips `EnableDebug` automatically via `testing.Verbose()`.
-
-## Notes
-
-- The NFS server binds to `127.0.0.1` with `-ip.bind=127.0.0.1` and exports
-  `/nfs_export`. The test framework pre-creates that directory via the
-  filer's HTTP API before starting the NFS server — the NFS server requires
-  its export root to exist in the filer's namespace with a real entry, and
-  the filer's synthetic `/` root does not match the `Name=="/"` check the
-  NFS server performs during `ensureIndexedEntry`.
-- Ports are allocated dynamically. Each test run opens a short-lived
-  listener on `127.0.0.1:0`, reads back the assigned port, closes the
-  listener, and hands the port to `weed master/volume/filer/nfs`. There is
-  a tiny race window between close and reopen that has not been a problem
-  in practice but is worth remembering if you see a "bind: address already
-  in use" failure.
-- All four `weed` components are started with explicit `-port.grpc=...`
-  flags. Without them, the default is `-port + 10000`, which overflows
-  `65535` whenever the HTTP port lands above `55535` — the kernel's
-  ephemeral port range on macOS routinely does.

test/nfs/basic_test.go

@@ -1,400 +0,0 @@
-package nfs
-
-import (
-	"bytes"
-	"fmt"
-	"io"
-	"os"
-	"path"
-	"strings"
-	"testing"
-
-	"github.com/stretchr/testify/assert"
-	"github.com/stretchr/testify/require"
-	nfsclient "github.com/willscott/go-nfs-client/nfs"
-)
-
-// setupFramework is a small helper that boots the cluster for a single test
-// and tears everything down on completion. Every test gets a fresh filer +
-// volume pair so they cannot step on each other's namespace.
-func setupFramework(t *testing.T) *NfsTestFramework {
-	t.Helper()
-	if testing.Short() {
-		t.Skip("skipping integration test in short mode")
-	}
-	config := DefaultTestConfig()
-	config.EnableDebug = testing.Verbose()
-	fw := NewNfsTestFramework(t, config)
-	require.NoError(t, fw.Setup(config), "framework setup")
-	t.Cleanup(fw.Cleanup)
-	return fw
-}
-
-// writeAll writes payload to path on the target in a single Write call. The
-// NFS WRITE3 RPC chunks internally, so this exists purely so tests read
-// linearly.
-func writeAll(t *testing.T, target *nfsclient.Target, remotePath string, payload []byte) {
-	t.Helper()
-	file, err := target.OpenFile(remotePath, 0o644)
-	require.NoError(t, err, "open %s for write", remotePath)
-	if len(payload) > 0 {
-		n, err := file.Write(payload)
-		require.NoError(t, err, "write %s", remotePath)
-		require.Equal(t, len(payload), n, "short write on %s", remotePath)
-	}
-	require.NoError(t, file.Close(), "close %s", remotePath)
-}
-
-// readAll opens path on the target and returns the full file contents.
-func readAll(t *testing.T, target *nfsclient.Target, remotePath string) []byte {
-	t.Helper()
-	file, err := target.Open(remotePath)
-	require.NoError(t, err, "open %s for read", remotePath)
-	defer file.Close()
-	content, err := io.ReadAll(file)
-	require.NoError(t, err, "read %s", remotePath)
-	return content
-}
-
-// TestNfsBasicReadWrite exercises the most common NFS path: OpenFile + Write
-// + Close followed by Open + Read to verify round-trip data integrity.
-func TestNfsBasicReadWrite(t *testing.T) {
-	fw := setupFramework(t)
-	target, cleanup, err := fw.Mount()
-	require.NoError(t, err)
-	defer cleanup()
-
-	payload := []byte("hello from seaweedfs nfs integration test")
-	writeAll(t, target, "/hello.txt", payload)
-
-	got := readAll(t, target, "/hello.txt")
-	assert.Equal(t, payload, got, "round-tripped content must match")
-
-	info, err := target.Getattr("/hello.txt")
-	require.NoError(t, err)
-	assert.Equal(t, int64(len(payload)), int64(info.Filesize))
-}
-
-// TestNfsMkdirAndRmdir covers Mkdir, ReadDirPlus, and RmDir. The readdir
-// assertion also verifies that the newly-created directory shows up under
-// the export root the way a POSIX client would expect.
-func TestNfsMkdirAndRmdir(t *testing.T) {
-	fw := setupFramework(t)
-	target, cleanup, err := fw.Mount()
-	require.NoError(t, err)
-	defer cleanup()
-
-	_, err = target.Mkdir("/dir1", 0o755)
-	require.NoError(t, err)
-
-	entries, err := target.ReadDirPlus("/")
-	require.NoError(t, err)
-	found := false
-	for _, entry := range entries {
-		if entry.Name() == "dir1" {
-			found = true
-			assert.True(t, entry.IsDir(), "dir1 should be a directory")
-		}
-	}
-	assert.True(t, found, "expected dir1 in readdir listing")
-
-	require.NoError(t, target.RmDir("/dir1"))
-
-	// After removal, dir1 must be gone from the listing.
-	entries, err = target.ReadDirPlus("/")
-	require.NoError(t, err)
-	for _, entry := range entries {
-		assert.NotEqual(t, "dir1", entry.Name(), "dir1 should be removed")
-	}
-}
-
-// TestNfsNestedDirectories ensures the server can materialise a deep tree in
-// a single Mkdir-per-segment sequence and that reads/writes work at the
-// leaves.
-func TestNfsNestedDirectories(t *testing.T) {
-	fw := setupFramework(t)
-	target, cleanup, err := fw.Mount()
-	require.NoError(t, err)
-	defer cleanup()
-
-	for _, segment := range []string{"/a", "/a/b", "/a/b/c"} {
-		_, err := target.Mkdir(segment, 0o755)
-		require.NoError(t, err, "mkdir %s", segment)
-	}
-
-	payload := []byte("deep path content")
-	writeAll(t, target, "/a/b/c/leaf.txt", payload)
-
-	got := readAll(t, target, "/a/b/c/leaf.txt")
-	assert.Equal(t, payload, got)
-
-	require.NoError(t, target.Remove("/a/b/c/leaf.txt"))
-	require.NoError(t, target.RmDir("/a/b/c"))
-	require.NoError(t, target.RmDir("/a/b"))
-	require.NoError(t, target.RmDir("/a"))
-}
-
-// TestNfsRenamePreservesContent renames a file and makes sure the content
-// at the new path matches what was written at the old one, and that the
-// old path disappears. It does not assert on inode identity because pjdfstest
-// already covers that and this test intentionally avoids depending on the
-// mount-side identity plumbing.
-func TestNfsRenamePreservesContent(t *testing.T) {
-	fw := setupFramework(t)
-	target, cleanup, err := fw.Mount()
-	require.NoError(t, err)
-	defer cleanup()
-
-	payload := []byte("rename me")
-	writeAll(t, target, "/src.txt", payload)
-
-	require.NoError(t, target.Rename("/src.txt", "/dst.txt"))
-
-	_, _, err = target.Lookup("/src.txt")
-	assert.Error(t, err, "source should be gone after rename")
-
-	got := readAll(t, target, "/dst.txt")
-	assert.Equal(t, payload, got)
-
-	require.NoError(t, target.Remove("/dst.txt"))
-}
-
-// TestNfsOverwriteShrinksFile rewrites an existing file with shorter content
-// and asserts Getattr reports the new (smaller) size. go-nfs-client's
-// OpenFile does not pass O_TRUNC, so the test truncates explicitly via
-// Setattr(size=0) before the second write — mirroring what `echo >file`
-// does on a POSIX client.
-func TestNfsOverwriteShrinksFile(t *testing.T) {
-	fw := setupFramework(t)
-	target, cleanup, err := fw.Mount()
-	require.NoError(t, err)
-	defer cleanup()
-
-	writeAll(t, target, "/overwrite.txt", []byte("the quick brown fox"))
-
-	require.NoError(t, target.Setattr("/overwrite.txt", nfsclient.Sattr3{
-		Size: nfsclient.SetSize{SetIt: true, Size: 0},
-	}))
-
-	writeAll(t, target, "/overwrite.txt", []byte("short"))
-
-	info, err := target.Getattr("/overwrite.txt")
-	require.NoError(t, err)
-	assert.Equal(t, int64(len("short")), int64(info.Filesize))
-
-	got := readAll(t, target, "/overwrite.txt")
-	assert.Equal(t, []byte("short"), got)
-
-	require.NoError(t, target.Remove("/overwrite.txt"))
-}
-
-// TestNfsLargeFile writes a multi-megabyte payload so the write path has to
-// cut chunks and flush through the volume server rather than inlining
-// content in the filer entry.
-func TestNfsLargeFile(t *testing.T) {
-	fw := setupFramework(t)
-	target, cleanup, err := fw.Mount()
-	require.NoError(t, err)
-	defer cleanup()
-
-	const size = 3 * 1024 * 1024 // 3 MiB — exceeds the 4 MiB inline cutoff boundary when combined with metadata
-	payload := make([]byte, size)
-	for i := range payload {
-		payload[i] = byte(i % 251) // non-repeating to catch offset bugs
-	}
-
-	writeAll(t, target, "/big.bin", payload)
-
-	info, err := target.Getattr("/big.bin")
-	require.NoError(t, err)
-	assert.Equal(t, int64(size), int64(info.Filesize))
-
-	got := readAll(t, target, "/big.bin")
-	require.Equal(t, size, len(got))
-	assert.True(t, bytes.Equal(payload, got), "large file content must round-trip byte-for-byte")
-
-	require.NoError(t, target.Remove("/big.bin"))
-}
-
-// TestNfsBinaryAndEmptyFiles covers two edge-case payloads the write path
-// tends to regress on: arbitrary binary bytes and zero-length files.
-func TestNfsBinaryAndEmptyFiles(t *testing.T) {
-	fw := setupFramework(t)
-	target, cleanup, err := fw.Mount()
-	require.NoError(t, err)
-	defer cleanup()
-
-	t.Run("AllByteValues", func(t *testing.T) {
-		payload := make([]byte, 256)
-		for i := range payload {
-			payload[i] = byte(i)
-		}
-		writeAll(t, target, "/binary.bin", payload)
-		assert.Equal(t, payload, readAll(t, target, "/binary.bin"))
-		require.NoError(t, target.Remove("/binary.bin"))
-	})
-
-	t.Run("EmptyFile", func(t *testing.T) {
-		writeAll(t, target, "/empty.txt", nil)
-		info, err := target.Getattr("/empty.txt")
-		require.NoError(t, err)
-		assert.Equal(t, int64(0), int64(info.Filesize))
-		require.NoError(t, target.Remove("/empty.txt"))
-	})
-}
-
-// TestNfsSymlinkRoundTrip covers Symlink and Readlink through the nfs server.
-// Readlink returns the target path; the server does not auto-traverse it.
-func TestNfsSymlinkRoundTrip(t *testing.T) {
-	fw := setupFramework(t)
-	target, cleanup, err := fw.Mount()
-	require.NoError(t, err)
-	defer cleanup()
-
-	// Symlink uses a different RPC than open+create, and our server routes it
-	// through the billy Change interface.
-	require.NoError(t, target.Symlink("/target.txt", "/link.txt"))
-
-	// The underlying target does not need to exist for readlink to succeed.
-	file, _, err := target.Lookup("/link.txt")
-	require.NoError(t, err, "lookup symlink")
-	assert.True(t, file.Mode()&os.ModeSymlink != 0, "expected symlink mode, got %s", file.Mode())
-
-	require.NoError(t, target.Remove("/link.txt"))
-}
-
-// TestNfsReadDirPlusOrdering creates a handful of files with distinct names
-// and ensures ReadDirPlus surfaces every one of them. The server pages
-// listings from the filer, so we want to make sure nothing is truncated.
-func TestNfsReadDirPlusOrdering(t *testing.T) {
-	fw := setupFramework(t)
-	target, cleanup, err := fw.Mount()
-	require.NoError(t, err)
-	defer cleanup()
-
-	_, err = target.Mkdir("/listing", 0o755)
-	require.NoError(t, err)
-
-	names := []string{"alpha.txt", "beta.txt", "gamma.txt", "delta.txt", "epsilon.txt"}
-	for _, name := range names {
-		writeAll(t, target, path.Join("/listing", name), []byte(name))
-	}
-
-	entries, err := target.ReadDirPlus("/listing")
-	require.NoError(t, err)
-	seen := make(map[string]struct{}, len(entries))
-	for _, entry := range entries {
-		if entry.Name() == "." || entry.Name() == ".." {
-			continue
-		}
-		seen[entry.Name()] = struct{}{}
-	}
-	for _, name := range names {
-		_, ok := seen[name]
-		assert.True(t, ok, "expected %s in directory listing", name)
-	}
-
-	for _, name := range names {
-		require.NoError(t, target.Remove(path.Join("/listing", name)))
-	}
-	require.NoError(t, target.RmDir("/listing"))
-}
-
-// TestNfsRemoveMissingFailsCleanly asserts that removing a non-existent path
-// surfaces an error instead of silently succeeding. A bug where the server
-// returned NFS3_OK on missing entries would hide metadata drift.
-func TestNfsRemoveMissingFailsCleanly(t *testing.T) {
-	fw := setupFramework(t)
-	target, cleanup, err := fw.Mount()
-	require.NoError(t, err)
-	defer cleanup()
-
-	err = target.Remove("/does_not_exist.txt")
-	require.Error(t, err, "removing a missing file must error")
-	// NFS3 surfaces this as NFS3ERR_NOENT; make sure the error text is
-	// recognisable without locking us into the library's exact wording.
-	assert.True(t,
-		strings.Contains(strings.ToLower(err.Error()), "noent") ||
-			strings.Contains(strings.ToLower(err.Error()), "not exist") ||
-			strings.Contains(strings.ToLower(err.Error()), "no such"),
-		"unexpected error shape: %v", err)
-}
-
-// TestNfsFSInfoReturnsSaneLimits pokes at FSINFO so we catch regressions
-// where the server advertises zero read/write limits (which would make
-// clients fall back to the 8 KiB floor and slow every test that follows).
-func TestNfsFSInfoReturnsSaneLimits(t *testing.T) {
-	fw := setupFramework(t)
-	target, cleanup, err := fw.Mount()
-	require.NoError(t, err)
-	defer cleanup()
-
-	info, err := target.FSInfo()
-	require.NoError(t, err)
-	require.NotNil(t, info)
-	assert.Greater(t, info.RTPref, uint32(0), "rtpref must be positive")
-	assert.Greater(t, info.WTPref, uint32(0), "wtpref must be positive")
-}
-
-// TestNfsAppendIsSequential writes two chunks to the same file in separate
-// Open cycles and asserts the concatenation is preserved. The second write
-// uses O_APPEND (the default Open path in go-nfs-client does not pass
-// flags, so we explicitly reopen after writing the first chunk).
-func TestNfsAppendIsSequential(t *testing.T) {
-	fw := setupFramework(t)
-	target, cleanup, err := fw.Mount()
-	require.NoError(t, err)
-	defer cleanup()
-
-	const prefix = "part1-"
-	const suffix = "part2"
-
-	writeAll(t, target, "/concat.txt", []byte(prefix))
-
-	file, err := target.OpenFile("/concat.txt", 0o644)
-	require.NoError(t, err)
-	// Seek to end before writing so we append rather than overwrite. go-nfs
-	// client's File.Seek uses the same offset tracking as Write so this is
-	// enough to place the second chunk after the first.
-	_, err = file.Seek(int64(len(prefix)), io.SeekStart)
-	require.NoError(t, err)
-	_, err = file.Write([]byte(suffix))
-	require.NoError(t, err)
-	require.NoError(t, file.Close())
-
-	got := readAll(t, target, "/concat.txt")
-	assert.Equal(t, prefix+suffix, string(got))
-
-	require.NoError(t, target.Remove("/concat.txt"))
-}
-
-// Regression: readdir should not emit stale entries after a remove. This is
-// the scenario the PR's meta cache invalidation logic was written to fix.
-func TestNfsReadDirAfterRemove(t *testing.T) {
-	fw := setupFramework(t)
-	target, cleanup, err := fw.Mount()
-	require.NoError(t, err)
-	defer cleanup()
-
-	_, err = target.Mkdir("/churn", 0o755)
-	require.NoError(t, err)
-	for i := 0; i < 5; i++ {
-		writeAll(t, target, path.Join("/churn", fmt.Sprintf("f%d.txt", i)), []byte{byte(i)})
-	}
-	// Remove the middle one and re-list.
-	require.NoError(t, target.Remove("/churn/f2.txt"))
-
-	entries, err := target.ReadDirPlus("/churn")
-	require.NoError(t, err)
-	for _, entry := range entries {
-		assert.NotEqual(t, "f2.txt", entry.Name(), "removed file should not reappear in listing")
-	}
-
-	for i := 0; i < 5; i++ {
-		if i == 2 {
-			continue
-		}
-		require.NoError(t, target.Remove(path.Join("/churn", fmt.Sprintf("f%d.txt", i))))
-	}
-	require.NoError(t, target.RmDir("/churn"))
-}

test/nfs/framework.go

@@ -1,423 +0,0 @@
-package nfs
-
-import (
-	"bytes"
-	"fmt"
-	"io"
-	"mime/multipart"
-	"net"
-	"net/http"
-	"os"
-	"os/exec"
-	"path/filepath"
-	"runtime"
-	"strings"
-	"syscall"
-	"testing"
-	"time"
-
-	"github.com/seaweedfs/seaweedfs/test/testutil"
-	"github.com/stretchr/testify/require"
-	nfsclient "github.com/willscott/go-nfs-client/nfs"
-	"github.com/willscott/go-nfs-client/nfs/rpc"
-)
-
-// NfsTestFramework boots a minimal SeaweedFS cluster (master + volume + filer)
-// plus the experimental `weed nfs` frontend and hands out NFSv3 RPC clients
-// that talk to it. Everything is driven via subprocesses so the tests exercise
-// the same binary an operator would deploy, and no kernel mount is required.
-type NfsTestFramework struct {
-	t             *testing.T
-	tempDir       string
-	dataDir       string
-	masterProcess *os.Process
-	volumeProcess *os.Process
-	filerProcess  *os.Process
-	nfsProcess    *os.Process
-	masterAddr    string
-	masterGrpc    int
-	volumeAddr    string
-	volumeGrpc    int
-	filerAddr     string
-	filerGrpc     int
-	nfsAddr       string
-	exportRoot    string
-	weedBinary    string
-	isSetup       bool
-	skipCleanup   bool
-}
-
-// TestConfig controls how the framework boots the cluster.
-type TestConfig struct {
-	NumVolumes  int
-	EnableDebug bool
-	SkipCleanup bool // keep temp dir on failure for inspection
-	// ExportRoot is the filer path the NFS server exports. Defaults to "/"
-	// so tests can use any path, with a single warning logged by the server.
-	ExportRoot string
-}
-
-// DefaultTestConfig returns the defaults used by most tests. A dedicated
-// /nfs_export subtree is used as the NFS export root because the NFS server
-// requires the export directory to exist in the filer's namespace and carry
-// a non-zero inode — passing "/" would succeed only for filer setups that
-// have already backfilled the root inode.
-func DefaultTestConfig() *TestConfig {
-	return &TestConfig{
-		NumVolumes:  3,
-		EnableDebug: false,
-		SkipCleanup: false,
-		ExportRoot:  "/nfs_export",
-	}
-}
-
-// NewNfsTestFramework allocates a framework bound to the current test. Call
-// Setup next to actually start the cluster.
-func NewNfsTestFramework(t *testing.T, config *TestConfig) *NfsTestFramework {
-	if config == nil {
-		config = DefaultTestConfig()
-	}
-
-	tempDir, err := os.MkdirTemp("", "seaweedfs_nfs_test_")
-	require.NoError(t, err)
-
-	// testutil.MustAllocatePorts holds every listener open until the full
-	// batch has been reserved, which avoids the "close-then-hope" race my
-	// original per-port helper had. We need seven ports: four HTTP (master,
-	// volume, filer, nfs) and three gRPC (master, volume, filer — nfs has
-	// no gRPC endpoint).
-	ports := testutil.MustAllocatePorts(t, 7)
-
-	exportRoot := config.ExportRoot
-	if exportRoot == "" {
-		exportRoot = "/"
-	}
-
-	return &NfsTestFramework{
-		t:           t,
-		tempDir:     tempDir,
-		dataDir:     filepath.Join(tempDir, "data"),
-		masterAddr:  fmt.Sprintf("127.0.0.1:%d", ports[0]),
-		masterGrpc:  ports[1],
-		volumeAddr:  fmt.Sprintf("127.0.0.1:%d", ports[2]),
-		volumeGrpc:  ports[3],
-		filerAddr:   fmt.Sprintf("127.0.0.1:%d", ports[4]),
-		filerGrpc:   ports[5],
-		nfsAddr:     fmt.Sprintf("127.0.0.1:%d", ports[6]),
-		exportRoot:  exportRoot,
-		weedBinary:  findWeedBinary(),
-		isSetup:     false,
-		skipCleanup: config.SkipCleanup,
-	}
-}
-
-// Setup starts the SeaweedFS cluster and the NFS frontend, waiting for each
-// component to accept connections before moving on.
-func (f *NfsTestFramework) Setup(config *TestConfig) error {
-	if f.isSetup {
-		return fmt.Errorf("framework already setup")
-	}
-
-	dirs := []string{
-		f.dataDir,
-		filepath.Join(f.dataDir, "master"),
-		filepath.Join(f.dataDir, "volume"),
-	}
-	for _, dir := range dirs {
-		if err := os.MkdirAll(dir, 0755); err != nil {
-			return fmt.Errorf("failed to create directory %s: %v", dir, err)
-		}
-	}
-
-	if err := f.startMaster(config); err != nil {
-		return fmt.Errorf("failed to start master: %v", err)
-	}
-	if !testutil.WaitForPort(portFromAddr(f.masterAddr), testutil.SeaweedMiniStartupTimeout) {
-		return fmt.Errorf("master not ready at %s", f.masterAddr)
-	}
-
-	if err := f.startVolumeServer(config); err != nil {
-		return fmt.Errorf("failed to start volume server: %v", err)
-	}
-	if !testutil.WaitForPort(portFromAddr(f.volumeAddr), testutil.SeaweedMiniStartupTimeout) {
-		return fmt.Errorf("volume server not ready at %s", f.volumeAddr)
-	}
-
-	if err := f.startFiler(config); err != nil {
-		return fmt.Errorf("failed to start filer: %v", err)
-	}
-	if !testutil.WaitForPort(portFromAddr(f.filerAddr), testutil.SeaweedMiniStartupTimeout) {
-		return fmt.Errorf("filer not ready at %s", f.filerAddr)
-	}
-
-	// Pre-create the export root in the filer's namespace. The NFS server
-	// expects its export directory to exist with a real inode; uploading a
-	// placeholder file creates the parent directory implicitly and then
-	// removing the file leaves the empty directory in place.
-	if f.exportRoot != "/" {
-		if err := f.ensureExportRootExists(); err != nil {
-			return fmt.Errorf("failed to pre-create export root %s: %v", f.exportRoot, err)
-		}
-	}
-
-	if err := f.startNfsServer(config); err != nil {
-		return fmt.Errorf("failed to start NFS server: %v", err)
-	}
-	if !testutil.WaitForPort(portFromAddr(f.nfsAddr), testutil.SeaweedMiniStartupTimeout) {
-		return fmt.Errorf("NFS server not ready at %s", f.nfsAddr)
-	}
-
-	// Let the NFS server finish wiring up its gRPC subscription to the filer
-	// before the first client call hits MOUNT/LOOKUP.
-	time.Sleep(500 * time.Millisecond)
-
-	f.isSetup = true
-	return nil
-}
-
-// Cleanup stops all processes. Temp state is preserved if SkipCleanup is set.
-func (f *NfsTestFramework) Cleanup() {
-	processes := []*os.Process{f.nfsProcess, f.filerProcess, f.volumeProcess, f.masterProcess}
-	for _, proc := range processes {
-		if proc != nil {
-			_ = proc.Signal(syscall.SIGTERM)
-			_, _ = proc.Wait()
-		}
-	}
-	if !f.skipCleanup {
-		_ = os.RemoveAll(f.tempDir)
-	}
-}
-
-// NfsAddr returns the TCP address the NFS server is listening on.
-func (f *NfsTestFramework) NfsAddr() string { return f.nfsAddr }
-
-// FilerAddr returns the TCP address of the filer.
-func (f *NfsTestFramework) FilerAddr() string { return f.filerAddr }
-
-// ExportRoot returns the path the NFS server exports.
-func (f *NfsTestFramework) ExportRoot() string { return f.exportRoot }
-
-// Mount opens an NFSv3 MOUNT+NFS connection against the running NFS server
-// and returns a Target that tests can drive like a mini-VFS. Caller is
-// responsible for calling the returned cleanup func to Unmount and close the
-// TCP connection.
-func (f *NfsTestFramework) Mount() (*nfsclient.Target, func(), error) {
-	var (
-		client *rpc.Client
-		err    error
-	)
-	// The NFS server's TCP listener may already be accepting connections when
-	// waitForService returns, but the RPC program registration can trail it
-	// by a few milliseconds. Retry the dial to absorb that small window.
-	for attempt := 0; attempt < 20; attempt++ {
-		client, err = rpc.DialTCP("tcp", f.nfsAddr, false)
-		if err == nil {
-			break
-		}
-		time.Sleep(25 * time.Millisecond)
-	}
-	if err != nil {
-		return nil, nil, fmt.Errorf("dial NFS: %w", err)
-	}
-
-	// Note: do not set Mount.Addr here. When Addr is non-empty, the go-nfs
-	// client re-dials via portmapper and concatenates `:111` onto the
-	// address, which produces "too many colons" for a raw `host:port`
-	// string. Reusing the existing RPC client avoids that path entirely.
-	mounter := &nfsclient.Mount{Client: client}
-	target, err := mounter.Mount(f.exportRoot, rpc.AuthNull)
-	if err != nil {
-		client.Close()
-		return nil, nil, fmt.Errorf("mount %s: %w", f.exportRoot, err)
-	}
-
-	cleanup := func() {
-		_ = mounter.Unmount()
-		client.Close()
-	}
-	return target, cleanup, nil
-}
-
-func (f *NfsTestFramework) startMaster(config *TestConfig) error {
-	_, masterPort := splitHostPort(f.masterAddr)
-	args := []string{
-		"master",
-		"-ip=127.0.0.1",
-		fmt.Sprintf("-port=%d", masterPort),
-		fmt.Sprintf("-port.grpc=%d", f.masterGrpc),
-		"-mdir=" + filepath.Join(f.dataDir, "master"),
-		"-raftBootstrap",
-		"-peers=none",
-	}
-	return f.startProcess(&f.masterProcess, config, args)
-}
-
-func (f *NfsTestFramework) startVolumeServer(config *TestConfig) error {
-	_, volumePort := splitHostPort(f.volumeAddr)
-	// pb.ServerAddress encodes a non-default gRPC port as `host:port.grpc`.
-	// See weed/pb/server_address.go — the dot, not a colon, is the separator
-	// between the HTTP port and the gRPC port.
-	masterWithGrpc := fmt.Sprintf("%s.%d", f.masterAddr, f.masterGrpc)
-	args := []string{
-		"volume",
-		"-master=" + masterWithGrpc,
-		"-ip=127.0.0.1",
-		fmt.Sprintf("-port=%d", volumePort),
-		fmt.Sprintf("-port.grpc=%d", f.volumeGrpc),
-		"-dir=" + filepath.Join(f.dataDir, "volume"),
-		fmt.Sprintf("-max=%d", config.NumVolumes),
-	}
-	return f.startProcess(&f.volumeProcess, config, args)
-}
-
-func (f *NfsTestFramework) startFiler(config *TestConfig) error {
-	_, filerPort := splitHostPort(f.filerAddr)
-	masterWithGrpc := fmt.Sprintf("%s.%d", f.masterAddr, f.masterGrpc)
-	args := []string{
-		"filer",
-		"-master=" + masterWithGrpc,
-		"-ip=127.0.0.1",
-		fmt.Sprintf("-port=%d", filerPort),
-		fmt.Sprintf("-port.grpc=%d", f.filerGrpc),
-	}
-	return f.startProcess(&f.filerProcess, config, args)
-}
-
-func (f *NfsTestFramework) startNfsServer(config *TestConfig) error {
-	_, nfsPort := splitHostPort(f.nfsAddr)
-	// `host:port.grpc` encoding — see pb/server_address.go.
-	filerWithGrpc := fmt.Sprintf("%s.%d", f.filerAddr, f.filerGrpc)
-	args := []string{
-		"nfs",
-		"-filer=" + filerWithGrpc,
-		"-ip.bind=127.0.0.1",
-		fmt.Sprintf("-port=%d", nfsPort),
-		"-filer.path=" + f.exportRoot,
-	}
-	return f.startProcess(&f.nfsProcess, config, args)
-}
-
-func (f *NfsTestFramework) startProcess(target **os.Process, config *TestConfig, args []string) error {
-	cmd := exec.Command(f.weedBinary, args...)
-	cmd.Dir = f.tempDir
-	if config.EnableDebug {
-		cmd.Stdout = os.Stdout
-		cmd.Stderr = os.Stderr
-	}
-	if err := cmd.Start(); err != nil {
-		return err
-	}
-	*target = cmd.Process
-	return nil
-}
-
-// portFromAddr returns just the port number from a `host:port` string.
-// testutil.WaitForPort takes an int port, not a full address.
-func portFromAddr(addr string) int {
-	_, port := splitHostPort(addr)
-	return port
-}
-
-// ensureExportRootExists posts a placeholder file to f.exportRoot via the
-// filer's HTTP API, then deletes it. That roundtrip implicitly creates the
-// target directory so the NFS server has something to mount. We bypass
-// weed/pb here because the HTTP client is simpler and needs no gRPC stubs.
-func (f *NfsTestFramework) ensureExportRootExists() error {
-	exportRoot := strings.TrimRight(f.exportRoot, "/")
-	if exportRoot == "" {
-		return nil
-	}
-	placeholder := exportRoot + "/.nfs_test_init"
-	filerURL := "http://" + f.filerAddr + placeholder
-
-	var body bytes.Buffer
-	writer := multipart.NewWriter(&body)
-	part, err := writer.CreateFormFile("file", ".nfs_test_init")
-	if err != nil {
-		return err
-	}
-	if _, err := io.WriteString(part, ""); err != nil {
-		return err
-	}
-	if err := writer.Close(); err != nil {
-		return err
-	}
-
-	httpClient := &http.Client{Timeout: 10 * time.Second}
-	req, err := http.NewRequest(http.MethodPost, filerURL, &body)
-	if err != nil {
-		return err
-	}
-	req.Header.Set("Content-Type", writer.FormDataContentType())
-	resp, err := httpClient.Do(req)
-	if err != nil {
-		return err
-	}
-	_, _ = io.Copy(io.Discard, resp.Body)
-	resp.Body.Close()
-	if resp.StatusCode/100 != 2 {
-		return fmt.Errorf("filer POST %s returned status %d", filerURL, resp.StatusCode)
-	}
-
-	// Delete the placeholder; the directory stays behind.
-	deleteReq, err := http.NewRequest(http.MethodDelete, filerURL, nil)
-	if err != nil {
-		return err
-	}
-	deleteResp, err := httpClient.Do(deleteReq)
-	if err != nil {
-		return err
-	}
-	_, _ = io.Copy(io.Discard, deleteResp.Body)
-	deleteResp.Body.Close()
-	if deleteResp.StatusCode/100 != 2 && deleteResp.StatusCode != http.StatusNotFound {
-		return fmt.Errorf("filer DELETE %s returned status %d", filerURL, deleteResp.StatusCode)
-	}
-	return nil
-}
-
-func splitHostPort(addr string) (string, int) {
-	host, portStr, err := net.SplitHostPort(addr)
-	if err != nil {
-		return "", 0
-	}
-	var port int
-	_, _ = fmt.Sscanf(portStr, "%d", &port)
-	return host, port
-}
-
-// findWeedBinary locates the weed binary, preferring the local build in the
-// checkout so tests run against the code under review rather than whatever is
-// on $PATH.
-func findWeedBinary() string {
-	if _, thisFile, _, ok := runtime.Caller(0); ok {
-		thisDir := filepath.Dir(thisFile)
-		candidates := []string{
-			filepath.Join(thisDir, "../../weed/weed"),
-			filepath.Join(thisDir, "../weed/weed"),
-		}
-		for _, candidate := range candidates {
-			if _, err := os.Stat(candidate); err == nil {
-				abs, _ := filepath.Abs(candidate)
-				return abs
-			}
-		}
-	}
-	cwd, _ := os.Getwd()
-	candidates := []string{
-		filepath.Join(cwd, "../../weed/weed"),
-		filepath.Join(cwd, "../weed/weed"),
-		filepath.Join(cwd, "./weed"),
-	}
-	for _, candidate := range candidates {
-		if _, err := os.Stat(candidate); err == nil {
-			abs, _ := filepath.Abs(candidate)
-			return abs
-		}
-	}
-	if path, err := exec.LookPath("weed"); err == nil {
-		return path
-	}
-	return "weed"
-}

test/nfs/go.mod

@@ -1,21 +0,0 @@
-module seaweedfs-nfs-tests
-
-go 1.25.0
-
-// test/testutil lives inside the main seaweedfs module; pull it in via a
-// local replace so this integration suite can reuse the shared port
-// allocator and readiness helpers instead of reinventing them.
-replace github.com/seaweedfs/seaweedfs => ../..
-
-require (
-	github.com/seaweedfs/seaweedfs v0.0.0-00010101000000-000000000000
-	github.com/stretchr/testify v1.11.1
-	github.com/willscott/go-nfs-client v0.0.0-20251022144359-801f10d98886
-)
-
-require (
-	github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
-	github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
-	github.com/rasky/go-xdr v0.0.0-20170124162913-1a41d1a06c93 // indirect
-	gopkg.in/yaml.v3 v3.0.1 // indirect
-)

test/nfs/go.sum

@@ -1,14 +0,0 @@
-github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
-github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
-github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
-github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
-github.com/rasky/go-xdr v0.0.0-20170124162913-1a41d1a06c93 h1:UVArwN/wkKjMVhh2EQGC0tEc1+FqiLlvYXY5mQ2f8Wg=
-github.com/rasky/go-xdr v0.0.0-20170124162913-1a41d1a06c93/go.mod h1:Nfe4efndBz4TibWycNE+lqyJZiMX4ycx+QKV8Ta0f/o=
-github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
-github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
-github.com/willscott/go-nfs-client v0.0.0-20251022144359-801f10d98886 h1:DtrBtkgTJk2XGt4T7eKdKVkd9A5NCevN2e4inLXtsqA=
-github.com/willscott/go-nfs-client v0.0.0-20251022144359-801f10d98886/go.mod h1:Tq++Lr/FgiS3X48q5FETemXiSLGuYMQT2sPjYNPJSwA=
-gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
-gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
-gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
-gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

test/nfs/kernel_mount_test.go

@@ -1,193 +0,0 @@
-//go:build linux
-
-package nfs
-
-// End-to-end mount tests that drive the real Linux NFS client (mount.nfs +
-// in-tree kernel) against a running `weed nfs` subprocess. These exist to
-// catch regressions that the existing framework can't see, because the
-// framework drives the server with willscott/go-nfs-client — the same RPC
-// library the server uses internally — so any bug shared between the two
-// (XDR layout, version dispatch, RPC framing) round-trips invisibly.
-//
-// Two real bugs hit recently were exactly that shape:
-//   1. NFSv4 mis-routed to the v3 SETATTR handler (#9262). The client
-//      library never sends NFSv4, so the test suite never noticed; the
-//      Linux kernel mount path did notice, with EIO.
-//   2. UDP MOUNT v3 missing. Only TCP MOUNT was advertised; the kernel
-//      defaults mountproto=udp in many setups, so the in-tree client
-//      surfaced EPROTONOSUPPORT during MOUNT setup.
-//
-// These tests mount over the actual loopback interface using mount.nfs and
-// shell out to /bin/mount and /bin/umount. They require root (mount(2) is
-// privileged) and Linux (the in-tree NFS client is what's being exercised);
-// they t.Skip cleanly when either prerequisite is missing.
-//
-// Run locally with:
-//
-//	cd test/nfs
-//	sudo go test -v -run TestKernelMount ./...
-//
-// CI runs them via .github/workflows/nfs-tests.yml after installing
-// nfs-common (mount.nfs + helpers).
-
-import (
-	"errors"
-	"fmt"
-	"net"
-	"os"
-	"os/exec"
-	"strings"
-	"testing"
-)
-
-// kernelMountSkipIfUnsupported skips the test when the host can't run a
-// real NFS mount. The combined check belongs in one place so the three
-// kernel-mount tests stay focused on what they're actually verifying.
-func kernelMountSkipIfUnsupported(t *testing.T) {
-	t.Helper()
-	if os.Geteuid() != 0 {
-		t.Skip("kernel mount test requires root; mount(2) is privileged")
-	}
-	if _, err := exec.LookPath("mount.nfs"); err != nil {
-		t.Skipf("mount.nfs not installed: %v (CI installs the nfs-common package)", err)
-	}
-}
-
-// kernelMount runs /bin/mount with the given options against the framework's
-// running NFS server, returns the mountpoint and an unmount closure. We pass
-// explicit port=/mountport= options so the kernel never queries portmap.
-// That keeps the harness honest about what it's testing — the NFS / MOUNT
-// wire protocol — and avoids colliding with a system rpcbind on shared CI
-// runners (port 111 is privileged and frequently in use already).
-func kernelMount(t *testing.T, fw *NfsTestFramework, optsTemplate string) (string, func()) {
-	t.Helper()
-	host, portStr, err := net.SplitHostPort(fw.NfsAddr())
-	if err != nil {
-		t.Fatalf("split nfs addr %q: %v", fw.NfsAddr(), err)
-	}
-	mountpoint, err := os.MkdirTemp("", "weed-nfs-kmount-")
-	if err != nil {
-		t.Fatalf("mkdtemp: %v", err)
-	}
-	opts := strings.ReplaceAll(optsTemplate, "{port}", portStr)
-	target := fmt.Sprintf("%s:%s", host, fw.ExportRoot())
-	cmd := exec.Command("mount", "-t", "nfs", "-o", opts, target, mountpoint)
-	if out, err := cmd.CombinedOutput(); err != nil {
-		_ = os.RemoveAll(mountpoint)
-		t.Fatalf("mount %s -o %s failed: %v\nmount output:\n%s", target, opts, err, out)
-	}
-	teardown := func() {
-		// -f to bail out faster if the server's already gone.
-		_ = exec.Command("umount", "-f", mountpoint).Run()
-		_ = os.RemoveAll(mountpoint)
-	}
-	return mountpoint, teardown
-}
-
-func newKernelMountFramework(t *testing.T) *NfsTestFramework {
-	t.Helper()
-	cfg := DefaultTestConfig()
-	fw := NewNfsTestFramework(t, cfg)
-	if err := fw.Setup(cfg); err != nil {
-		fw.Cleanup()
-		t.Fatalf("framework setup: %v", err)
-	}
-	t.Cleanup(fw.Cleanup)
-	return fw
-}
-
-// TestKernelMountV3TCP exercises the most common mount form: NFSv3 + MOUNT
-// v3, both over TCP. This is what the existing go-nfs-client tests cover at
-// the protocol layer, but running it through mount.nfs and the kernel
-// confirms that the wire format we emit decodes cleanly under a different
-// XDR/RPC parser.
-func TestKernelMountV3TCP(t *testing.T) {
-	kernelMountSkipIfUnsupported(t)
-	fw := newKernelMountFramework(t)
-
-	mountpoint, undo := kernelMount(t, fw,
-		"nfsvers=3,nolock,port={port},mountport={port},proto=tcp,mountproto=tcp")
-	defer undo()
-
-	if _, err := os.Stat(mountpoint); err != nil {
-		t.Errorf("stat mountpoint: %v", err)
-	}
-	if _, err := os.ReadDir(mountpoint); err != nil {
-		t.Errorf("readdir mountpoint: %v", err)
-	}
-}
-
-// TestKernelMountV3MountProtoUDP is the regression test for the UDP MOUNT
-// v3 responder. mountproto=udp forces the kernel to call MOUNT over UDP
-// only; before the responder existed the kernel hit nothing (MOUNT was
-// advertised TCP-only) and surfaced EPROTONOSUPPORT during mount setup.
-func TestKernelMountV3MountProtoUDP(t *testing.T) {
-	kernelMountSkipIfUnsupported(t)
-	fw := newKernelMountFramework(t)
-
-	mountpoint, undo := kernelMount(t, fw,
-		"nfsvers=3,nolock,port={port},mountport={port},proto=tcp,mountproto=udp")
-	defer undo()
-
-	if _, err := os.Stat(mountpoint); err != nil {
-		t.Errorf("stat mountpoint: %v", err)
-	}
-}
-
-// TestKernelMountV4RejectsCleanly is the regression test for the NFSv4
-// PROG_MISMATCH path (#9262). The server only speaks NFSv3, but the
-// previous behaviour was to mis-route v4 COMPOUND to the v3 SETATTR
-// handler and write garbage; the kernel surfaced EIO instead of a
-// version-mismatch error and (depending on distro) didn't fall back to
-// v3. The version filter now answers PROG_MISMATCH so the kernel sees
-// "v4 not supported" cleanly.
-//
-// The test asserts:
-//   1. mount.nfs exits non-zero (no silent success against a v3 server);
-//   2. the failure message mentions protocol/version/io, which is what the
-//      kernel surfaces when it gets PROG_MISMATCH instead of garbage. A
-//      pre-fix server returns "mount system call failed" with no further
-//      context, so a regression collapses the assertion onto that branch.
-func TestKernelMountV4RejectsCleanly(t *testing.T) {
-	kernelMountSkipIfUnsupported(t)
-	fw := newKernelMountFramework(t)
-
-	host, portStr, err := net.SplitHostPort(fw.NfsAddr())
-	if err != nil {
-		t.Fatalf("split nfs addr: %v", err)
-	}
-	mountpoint, err := os.MkdirTemp("", "weed-nfs-kmount-v4-")
-	if err != nil {
-		t.Fatalf("mkdtemp: %v", err)
-	}
-	defer os.RemoveAll(mountpoint)
-
-	target := fmt.Sprintf("%s:%s", host, fw.ExportRoot())
-	cmd := exec.Command("mount", "-t", "nfs", "-o",
-		fmt.Sprintf("vers=4,port=%s", portStr),
-		target, mountpoint)
-	out, err := cmd.CombinedOutput()
-	defer exec.Command("umount", "-f", mountpoint).Run()
-
-	if err == nil {
-		t.Fatalf("v4 mount unexpectedly succeeded against v3-only server\nmount output:\n%s", out)
-	}
-	// Don't pin the exact error string — different distros print slightly
-	// different things — but require some hint that the kernel saw a
-	// protocol-level failure rather than a generic "mount system call
-	// failed". Without the version filter, mount.nfs prints the latter
-	// alone; with it, the former.
-	lower := strings.ToLower(string(out))
-	if !strings.Contains(lower, "protocol") &&
-		!strings.Contains(lower, "version") &&
-		!strings.Contains(lower, "i/o") {
-		t.Errorf("v4 mount failure didn't mention protocol/version/io; output:\n%s", out)
-	}
-	// Also require a non-zero exit so a future change that makes mount(2)
-	// silently succeed (e.g. by relaxing the version filter) shows up
-	// here even if the message phrasing changes.
-	var ee *exec.ExitError
-	if !errors.As(err, &ee) {
-		t.Errorf("expected mount to exit non-zero with ExitError, got %v", err)
-	}
-}

test/plugin_workers/fake_volume_server.go

@@ -30,21 +30,22 @@ type VolumeServer struct {
 	address  string
 	baseDir  string
 
-	mu                  sync.Mutex
-	receivedFiles       map[string]uint64
-	mountRequests       []*volume_server_pb.VolumeEcShardsMountRequest
-	deleteRequests      []*volume_server_pb.VolumeDeleteRequest
-	markReadonlyCalls   int
-	markWritableCalls   int
-	readFileStatusCalls int
-	vacuumGarbageRatio  float64
-	vacuumCheckCalls    int
-	vacuumCompactCalls  int
-	vacuumCommitCalls   int
-	vacuumCleanupCalls  int
-	volumeCopyCalls     int
-	volumeMountCalls    int
-	tailReceiverCalls   int
+	mu                   sync.Mutex
+	receivedFiles        map[string]uint64
+	mountRequests        []*volume_server_pb.VolumeEcShardsMountRequest
+	deleteRequests       []*volume_server_pb.VolumeDeleteRequest
+	markReadonlyCalls    int
+	markWritableCalls    int
+	readFileStatusCalls  int
+	vacuumGarbageRatio   float64
+	vacuumCommitReadOnly bool
+	vacuumCheckCalls     int
+	vacuumCompactCalls   int
+	vacuumCommitCalls    int
+	vacuumCleanupCalls   int
+	volumeCopyCalls      int
+	volumeMountCalls     int
+	tailReceiverCalls    int
 }
 
 // NewVolumeServer starts a test volume server using the provided base directory.
@@ -115,6 +116,13 @@ func (v *VolumeServer) SetVacuumGarbageRatio(ratio float64) {
 	v.vacuumGarbageRatio = ratio
 }
 
+// SetVacuumCommitReadOnly sets the IsReadOnly value returned by VacuumVolumeCommit.
+func (v *VolumeServer) SetVacuumCommitReadOnly(readOnly bool) {
+	v.mu.Lock()
+	defer v.mu.Unlock()
+	v.vacuumCommitReadOnly = readOnly
+}
+
 // VacuumStats returns the vacuum RPC call counts.
 func (v *VolumeServer) VacuumStats() (check, compact, commit, cleanup int) {
 	v.mu.Lock()
@@ -301,9 +309,11 @@ func (v *VolumeServer) VolumeEcShardsUnmount(ctx context.Context, req *volume_se
 
 // VolumeEcShardsDelete is a no-op stub paired with VolumeEcShardsUnmount
 // above; the fake server doesn't persist shard files beyond what
-// ReceiveFile wrote, so there's nothing to remove.
+// ReceiveFile wrote, so there's nothing to remove. It still echoes the
+// full_teardown acknowledgement so the worker doesn't treat the fake as a
+// pre-upgrade server that silently skipped the teardown.
 func (v *VolumeServer) VolumeEcShardsDelete(ctx context.Context, req *volume_server_pb.VolumeEcShardsDeleteRequest) (*volume_server_pb.VolumeEcShardsDeleteResponse, error) {
-	return &volume_server_pb.VolumeEcShardsDeleteResponse{}, nil
+	return &volume_server_pb.VolumeEcShardsDeleteResponse{FullTeardownDone: req.FullTeardown}, nil
 }
 
 func (v *VolumeServer) VolumeEcShardsInfo(ctx context.Context, req *volume_server_pb.VolumeEcShardsInfoRequest) (*volume_server_pb.VolumeEcShardsInfoResponse, error) {
@@ -428,8 +438,9 @@ func (v *VolumeServer) VacuumVolumeCompact(req *volume_server_pb.VacuumVolumeCom
 func (v *VolumeServer) VacuumVolumeCommit(ctx context.Context, req *volume_server_pb.VacuumVolumeCommitRequest) (*volume_server_pb.VacuumVolumeCommitResponse, error) {
 	v.mu.Lock()
 	v.vacuumCommitCalls++
+	readOnly := v.vacuumCommitReadOnly
 	v.mu.Unlock()
-	return &volume_server_pb.VacuumVolumeCommitResponse{}, nil
+	return &volume_server_pb.VacuumVolumeCommitResponse{IsReadOnly: readOnly}, nil
 }
 
 func (v *VolumeServer) VacuumVolumeCleanup(ctx context.Context, req *volume_server_pb.VacuumVolumeCleanupRequest) (*volume_server_pb.VacuumVolumeCleanupResponse, error) {

test/plugin_workers/vacuum/execution_test.go

@@ -15,6 +15,24 @@ import (
 	"google.golang.org/grpc/credentials/insecure"
 )
 
+func vacuumJobSpec(volumeID uint32, server string) *plugin_pb.JobSpec {
+	return &plugin_pb.JobSpec{
+		JobId:   fmt.Sprintf("vacuum-job-%d", volumeID),
+		JobType: "vacuum",
+		Parameters: map[string]*plugin_pb.ConfigValue{
+			"volume_id": {
+				Kind: &plugin_pb.ConfigValue_Int64Value{Int64Value: int64(volumeID)},
+			},
+			"server": {
+				Kind: &plugin_pb.ConfigValue_StringValue{StringValue: server},
+			},
+			"collection": {
+				Kind: &plugin_pb.ConfigValue_StringValue{StringValue: "vac-test"},
+			},
+		},
+	}
+}
+
 func TestVacuumExecutionIntegration(t *testing.T) {
 	volumeID := uint32(202)
 
@@ -31,26 +49,10 @@ func TestVacuumExecutionIntegration(t *testing.T) {
 	source := pluginworkers.NewVolumeServer(t, "")
 	source.SetVacuumGarbageRatio(0.6)
 
-	job := &plugin_pb.JobSpec{
-		JobId:   fmt.Sprintf("vacuum-job-%d", volumeID),
-		JobType: "vacuum",
-		Parameters: map[string]*plugin_pb.ConfigValue{
-			"volume_id": {
-				Kind: &plugin_pb.ConfigValue_Int64Value{Int64Value: int64(volumeID)},
-			},
-			"server": {
-				Kind: &plugin_pb.ConfigValue_StringValue{StringValue: source.Address()},
-			},
-			"collection": {
-				Kind: &plugin_pb.ConfigValue_StringValue{StringValue: "vac-test"},
-			},
-		},
-	}
-
 	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
 	defer cancel()
 
-	result, err := harness.Plugin().ExecuteJob(ctx, job, nil, 1)
+	result, err := harness.Plugin().ExecuteJob(ctx, vacuumJobSpec(volumeID, source.Address()), nil, 1)
 	require.NoError(t, err)
 	require.NotNil(t, result)
 	require.True(t, result.Success)
@@ -59,5 +61,48 @@ func TestVacuumExecutionIntegration(t *testing.T) {
 	require.GreaterOrEqual(t, checkCalls, 2)
 	require.GreaterOrEqual(t, compactCalls, 1)
 	require.GreaterOrEqual(t, commitCalls, 1)
-	require.GreaterOrEqual(t, cleanupCalls, 1)
+	// Cleanup is only invoked when Phase 1 (Compact) fails to roll back
+	// the .cpd/.cpx/.cpldb temp files; on the success path Commit
+	// consumes them (rename .cpd → .dat, .cpx → .idx, .cpldb → .ldb via
+	// the leveldb needle map) so no Cleanup call is needed. Matches
+	// topology.vacuumOneVolumeId which only calls batchVacuumVolumeCleanup
+	// on the Compact-failure branch.
+	require.Equal(t, 0, cleanupCalls)
+	// Phase 3 marks each replica writable so master returns it to the
+	// writables layout. See upstream seaweedfs#9685.
+	require.GreaterOrEqual(t, source.MarkWritableCount(), 1)
+}
+
+// A replica that commits still read-only (operator-set, EIO-quarantined,
+// disk-space-low) must not be force-marked writable: master built-in vacuum
+// skips it via SetVolumeAvailable, and it recovers on its own ReadOnly=false
+// heartbeat.
+func TestVacuumExecutionSkipsMarkWritableWhenReadOnly(t *testing.T) {
+	volumeID := uint32(203)
+
+	dialOption := grpc.WithTransportCredentials(insecure.NewCredentials())
+	handler := vacuum.NewVacuumHandler(dialOption, 1)
+	harness := pluginworkers.NewHarness(t, pluginworkers.HarnessConfig{
+		WorkerOptions: pluginworker.WorkerOptions{
+			GrpcDialOption: dialOption,
+		},
+		Handlers: []pluginworker.JobHandler{handler},
+	})
+	harness.WaitForJobType("vacuum")
+
+	source := pluginworkers.NewVolumeServer(t, "")
+	source.SetVacuumGarbageRatio(0.6)
+	source.SetVacuumCommitReadOnly(true)
+
+	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+	defer cancel()
+
+	result, err := harness.Plugin().ExecuteJob(ctx, vacuumJobSpec(volumeID, source.Address()), nil, 1)
+	require.NoError(t, err)
+	require.NotNil(t, result)
+	require.True(t, result.Success)
+
+	_, _, commitCalls, _ := source.VacuumStats()
+	require.GreaterOrEqual(t, commitCalls, 1)
+	require.Equal(t, 0, source.MarkWritableCount())
 }

test/s3/compatibility/Dockerfile

@@ -1,21 +1,23 @@
 # the tests only support python 3.6, not newer
 #FROM ubuntu:latest
-FROM python:3.6.15-slim-buster
+FROM python:3.6.15-slim-bullseye
 
 # Installed required system deps
 RUN apt-get update \
-    && DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get install -y git-core sudo tzdata
+    && DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get install -y -qq git-core sudo tzdata ca-certificates libxml2-dev libxslt1-dev zlib1g-dev python3-lxml gcc python3-dev \
+    && rm -rf /var/lib/apt/lists/*
 
 # Install python deps
 RUN pip install virtualenv
 
 # Clone Ceph S3 tests
-RUN git clone https://github.com/ceph/s3-tests.git
+COPY fix_s3_tests_bucket_conflicts.py /tmp/fix_s3_tests_bucket_conflicts.py
+ENV S3_TESTS_PATH=s3-tests
+RUN git clone https://github.com/ceph/s3-tests.git \
+    && python3 /tmp/fix_s3_tests_bucket_conflicts.py
 
 WORKDIR s3-tests
 
-# Pin to a certain commit on ceph/s3-tests
-# https://github.com/ceph/s3-tests/commit/9a6a1e9f197fc9fb031b809d1e057635c2ff8d4e
-RUN git checkout 9a6a1e9f197fc9fb031b809d1e057635c2ff8d4e
-
-RUN ./bootstrap
+RUN pip install -r requirements.txt \
+    && pip install tox \
+    && pip install -e .

test/s3/iam/s3_policy_variables_test.go

@@ -210,10 +210,21 @@ func TestS3PolicyVariablesUsernameIsolation(t *testing.T) {
 			"Sid": "DenyOtherFolders",
 			"Effect": "Deny",
 			"Principal": "*",
-			"Action": ["s3:GetObject", "s3:PutObject", "s3:ListBucket"],
+			"Action": ["s3:GetObject", "s3:PutObject"],
 			"NotResource": "arn:aws:s3:::%s/${aws:username}/*"
+		}, {
+			"Sid": "DenyListOtherPrefixes",
+			"Effect": "Deny",
+			"Principal": "*",
+			"Action": "s3:ListBucket",
+			"Resource": "arn:aws:s3:::%s",
+			"Condition": {
+				"StringNotLike": {
+					"s3:prefix": ["${aws:username}/*", "${aws:username}"]
+				}
+			}
 		}]
-	}`, bucketName, bucketName, bucketName)
+	}`, bucketName, bucketName, bucketName, bucketName)
 
 	_, err = adminClient.PutBucketPolicy(&s3.PutBucketPolicyInput{
 		Bucket: aws.String(bucketName),

test/s3/iam/seaweedfs-test-realm.json

@@ -0,0 +1,107 @@
+{
+  "realm": "seaweedfs-test",
+  "enabled": true,
+  "roles": {
+    "realm": [
+      { "name": "s3-admin" },
+      { "name": "s3-read-only" },
+      { "name": "s3-write-only" },
+      { "name": "s3-read-write" }
+    ]
+  },
+  "clients": [
+    {
+      "clientId": "seaweedfs-s3",
+      "enabled": true,
+      "secret": "seaweedfs-s3-secret",
+      "publicClient": false,
+      "serviceAccountsEnabled": true,
+      "standardFlowEnabled": true,
+      "directAccessGrantsEnabled": true,
+      "fullScopeAllowed": true,
+      "redirectUris": ["*"],
+      "webOrigins": ["*"],
+      "protocol": "openid-connect",
+      "protocolMappers": [
+        {
+          "name": "realm-roles",
+          "protocol": "openid-connect",
+          "protocolMapper": "oidc-usermodel-realm-role-mapper",
+          "config": {
+            "claim.name": "roles",
+            "jsonType.label": "String",
+            "multivalued": "true",
+            "usermodel.realmRoleMapping.rolePrefix": "",
+            "access.token.claim": "true",
+            "id.token.claim": "true",
+            "userinfo.token.claim": "true"
+          }
+        },
+        {
+          "name": "audience-mapper",
+          "protocol": "openid-connect",
+          "protocolMapper": "oidc-audience-mapper",
+          "config": {
+            "included.client.audience": "seaweedfs-s3",
+            "id.token.claim": "false",
+            "access.token.claim": "true"
+          }
+        }
+      ]
+    }
+  ],
+  "users": [
+    {
+      "username": "admin-user",
+      "enabled": true,
+      "email": "admin-user@test.com",
+      "firstName": "Test",
+      "lastName": "User",
+      "emailVerified": true,
+      "requiredActions": [],
+      "realmRoles": ["s3-admin"],
+      "credentials": [
+        { "type": "password", "value": "adminuser123", "temporary": false }
+      ]
+    },
+    {
+      "username": "read-user",
+      "enabled": true,
+      "email": "read-user@test.com",
+      "firstName": "Test",
+      "lastName": "User",
+      "emailVerified": true,
+      "requiredActions": [],
+      "realmRoles": ["s3-read-only"],
+      "credentials": [
+        { "type": "password", "value": "readuser123", "temporary": false }
+      ]
+    },
+    {
+      "username": "write-user",
+      "enabled": true,
+      "email": "write-user@test.com",
+      "firstName": "Test",
+      "lastName": "User",
+      "emailVerified": true,
+      "requiredActions": [],
+      "realmRoles": ["s3-read-write"],
+      "credentials": [
+        { "type": "password", "value": "writeuser123", "temporary": false }
+      ]
+    },
+    {
+      "username": "write-only-user",
+      "enabled": true,
+      "email": "write-only-user@test.com",
+      "firstName": "Test",
+      "lastName": "User",
+      "emailVerified": true,
+      "requiredActions": [],
+      "realmRoles": ["s3-write-only"],
+      "credentials": [
+        { "type": "password", "value": "writeonlyuser123", "temporary": false }
+      ]
+    }
+  ]
+}

test/s3/versioning/Makefile

@@ -10,7 +10,11 @@ MASTER_PORT := 9333
 VOLUME_PORT := 8080
 FILER_PORT := 8888
 TEST_TIMEOUT := 10m
-TEST_PATTERN := TestVersioning
+# Run every test in the suite by default. "TestVersioning" used to leave whole
+# tests (bucket creation, suspended delete, directory/version listing) ungated.
+# Opt-in stress tests are matched by "." but self-skip without ENABLE_STRESS_TESTS
+# (they run with it set in their own targets), so they don't actually execute here.
+TEST_PATTERN := .
 
 .DEFAULT_GOAL := help
 

test/s3/versioning/s3_bucket_creation_test.go

@@ -36,7 +36,7 @@ func TestBucketCreationBehavior(t *testing.T) {
 			expectedError:      "",
 		},
 		{
-			name: "Create existing bucket with same owner - should return BucketAlreadyExists",
+			name: "Create existing bucket with same owner - should return BucketAlreadyOwnedByYou",
 			setupFunc: func(t *testing.T, bucketName string) {
 				// Create bucket first
 				_, err := client.CreateBucket(ctx, &s3.CreateBucketInput{
@@ -46,8 +46,8 @@ func TestBucketCreationBehavior(t *testing.T) {
 			},
 			bucketName:         "test-same-owner-same-settings-" + fmt.Sprintf("%d", time.Now().Unix()),
 			objectLockEnabled:  nil,
-			expectedStatusCode: 409, // SeaweedFS now returns BucketAlreadyExists in all cases
-			expectedError:      "BucketAlreadyExists",
+			expectedStatusCode: 409, // idempotent recreate by the same owner
+			expectedError:      "BucketAlreadyOwnedByYou",
 		},
 		{
 			name: "Create bucket with same owner but different Object Lock settings - should fail",
@@ -71,7 +71,7 @@ func TestBucketCreationBehavior(t *testing.T) {
 			expectedError:      "",
 		},
 		{
-			name: "Create bucket with Object Lock enabled twice - should fail",
+			name: "Create bucket with Object Lock enabled twice - should return BucketAlreadyOwnedByYou",
 			setupFunc: func(t *testing.T, bucketName string) {
 				// Create bucket with Object Lock first
 				_, err := client.CreateBucket(ctx, &s3.CreateBucketInput{
@@ -83,7 +83,7 @@ func TestBucketCreationBehavior(t *testing.T) {
 			bucketName:         "test-object-lock-duplicate-" + fmt.Sprintf("%d", time.Now().Unix()),
 			objectLockEnabled:  aws.Bool(true),
 			expectedStatusCode: 409,
-			expectedError:      "BucketAlreadyExists",
+			expectedError:      "BucketAlreadyOwnedByYou",
 		},
 	}
 
@@ -130,16 +130,27 @@ func TestBucketCreationBehavior(t *testing.T) {
 	}
 }
 
-// TestBucketCreationWithDifferentUsers tests bucket creation with different identity contexts
+// TestBucketCreationWithDifferentUsers tests bucket creation across owners: a
+// name taken by one identity is reported as BucketAlreadyExists to another
+// (versus BucketAlreadyOwnedByYou for the owner re-creating it).
 func TestBucketCreationWithDifferentUsers(t *testing.T) {
-	// This test would require setting up different S3 credentials/identities
-	// For now, we'll skip this as it requires more complex setup
-	t.Skip("Different user testing requires IAM setup - implement when IAM is configured")
+	ctx := context.Background()
+	// Tracked prefix + run id so cleanupAllTestBuckets sweeps it if the test leaks.
+	bucketName := getNewBucketName()
 
-	// TODO: Implement when we have proper IAM/user management in test setup
-	// Should test:
-	// 1. User A creates bucket
-	// 2. User B tries to create same bucket -> should fail with BucketAlreadyExists
+	// User A owns the bucket.
+	clientA := getS3Client(t)
+	_, err := clientA.CreateBucket(ctx, &s3.CreateBucketInput{Bucket: aws.String(bucketName)})
+	require.NoError(t, err, "User A should create the bucket")
+	defer cleanupBucketForCreationTest(t, clientA, bucketName)
+
+	// User B is a different identity (s3_tests_alt in the test S3 config).
+	clientB := getS3ClientWithCredentials(t,
+		"NOPQRSTUVWXYZABCDEFG", "nopqrstuvwxyzabcdefghijklmnabcdefghijklm")
+	_, err = clientB.CreateBucket(ctx, &s3.CreateBucketInput{Bucket: aws.String(bucketName)})
+	require.Error(t, err, "User B should not be able to create a bucket owned by User A")
+	assert.Contains(t, err.Error(), "BucketAlreadyExists",
+		"Different-owner recreate should return BucketAlreadyExists, got: %v", err)
 }
 
 // TestBucketCreationVersioningInteraction tests interaction between bucket creation and versioning
@@ -171,8 +182,8 @@ func TestBucketCreationVersioningInteraction(t *testing.T) {
 		ObjectLockEnabledForBucket: aws.Bool(true),
 	})
 	assert.Error(t, err, "Expected second bucket creation to fail")
-	assert.Contains(t, err.Error(), "BucketAlreadyExists",
-		"Expected BucketAlreadyExists error, got: %v", err)
+	assert.Contains(t, err.Error(), "BucketAlreadyOwnedByYou",
+		"Expected BucketAlreadyOwnedByYou error, got: %v", err)
 }
 
 // TestBucketCreationErrorMessages tests that proper error messages are returned
@@ -197,8 +208,8 @@ func TestBucketCreationErrorMessages(t *testing.T) {
 	require.Error(t, err, "Expected bucket creation to fail")
 
 	// Check that it's the right type of error
-	assert.Contains(t, err.Error(), "BucketAlreadyExists",
-		"Expected BucketAlreadyExists error, got: %v", err)
+	assert.Contains(t, err.Error(), "BucketAlreadyOwnedByYou",
+		"Expected BucketAlreadyOwnedByYou error, got: %v", err)
 }
 
 // cleanupBucketForCreationTest removes a bucket and all its contents

test/s3/versioning/s3_directory_versioning_test.go

@@ -122,7 +122,7 @@ func TestListObjectVersionsIncludesDirectories(t *testing.T) {
 				assert.Equal(t, int64(0), *version.Size, "Directory %s should have size 0", expectedDir)
 				assert.True(t, *version.IsLatest, "Directory %s should be marked as latest", expectedDir)
 				assert.Equal(t, "\"d41d8cd98f00b204e9800998ecf8427e\"", *version.ETag, "Directory %s should have MD5 of empty string as ETag", expectedDir)
-				assert.Equal(t, types.ObjectStorageClassStandard, version.StorageClass, "Directory %s should have STANDARD storage class", expectedDir)
+				assert.Equal(t, types.ObjectVersionStorageClassStandard, version.StorageClass, "Directory %s should have STANDARD storage class", expectedDir)
 				break
 			}
 		}
@@ -159,6 +159,85 @@ func TestListObjectVersionsIncludesDirectories(t *testing.T) {
 	assert.Equal(t, len(testFiles), fileCount, "Should find exactly %d files", len(testFiles))
 }
 
+// TestListObjectVersionsDeepPrefixExcludesAncestorDirectories verifies that a
+// prefix+delimiter version listing does not leak ancestor directory markers that
+// the listing only descends through to reach the prefix. Veeam's immutable
+// backup repository (versioning + object lock) issues exactly this request from
+// Cloud.FindLastCheckpointId and aborts when it sees an unexpected key like
+// "Veeam/" that does not match the deep prefix it asked for. The version listing
+// must match ListObjectsV2 / AWS: only keys at or under the prefix appear.
+func TestListObjectVersionsDeepPrefixExcludesAncestorDirectories(t *testing.T) {
+	bucketName := "test-versioning-deep-prefix"
+
+	client := setupS3Client(t)
+
+	_, err := client.CreateBucket(context.TODO(), &s3.CreateBucketInput{
+		Bucket: aws.String(bucketName),
+	})
+	require.NoError(t, err)
+	defer cleanupBucket(t, client, bucketName)
+
+	_, err = client.PutBucketVersioning(context.TODO(), &s3.PutBucketVersioningInput{
+		Bucket: aws.String(bucketName),
+		VersioningConfiguration: &types.VersioningConfiguration{
+			Status: types.BucketVersioningStatusEnabled,
+		},
+	})
+	require.NoError(t, err)
+
+	// Explicit directory markers for every parent path, as Veeam creates them.
+	ancestorMarkers := []string{
+		"Veeam/",
+		"Veeam/Backup/",
+		"Veeam/Backup/job1/",
+		"Veeam/Backup/job1/Clients/",
+		"Veeam/Backup/job1/Clients/aaaa/",
+		"Veeam/Backup/job1/Clients/aaaa/bbbb/",
+	}
+	prefix := "Veeam/Backup/job1/Clients/aaaa/bbbb/Metadata/"
+	checkpointKey := prefix + "Checkpoint"
+
+	for _, dirKey := range append(ancestorMarkers, prefix) {
+		_, err := client.PutObject(context.TODO(), &s3.PutObjectInput{
+			Bucket: aws.String(bucketName),
+			Key:    aws.String(dirKey),
+			Body:   strings.NewReader(""),
+		})
+		require.NoError(t, err, "Failed to create directory marker %s", dirKey)
+	}
+
+	_, err = client.PutObject(context.TODO(), &s3.PutObjectInput{
+		Bucket: aws.String(bucketName),
+		Key:    aws.String(checkpointKey),
+		Body:   strings.NewReader("checkpoint"),
+	})
+	require.NoError(t, err)
+
+	listResp, err := client.ListObjectVersions(context.TODO(), &s3.ListObjectVersionsInput{
+		Bucket:    aws.String(bucketName),
+		Prefix:    aws.String(prefix),
+		Delimiter: aws.String("/"),
+	})
+	require.NoError(t, err)
+
+	var gotKeys []string
+	for _, v := range listResp.Versions {
+		gotKeys = append(gotKeys, *v.Key)
+		assert.True(t, strings.HasPrefix(*v.Key, prefix),
+			"version key %q must start with the requested prefix %q", *v.Key, prefix)
+	}
+
+	// Only the prefix's own marker and the real object at/under it may appear.
+	assert.ElementsMatch(t, []string{prefix, checkpointKey}, gotKeys,
+		"deep-prefix version listing must not include ancestor directory markers")
+
+	// None of the ancestor markers ("Veeam/", ...) may surface as versions.
+	for _, marker := range ancestorMarkers {
+		assert.NotContains(t, gotKeys, marker,
+			"ancestor directory marker %q must not appear in a deep-prefix version listing", marker)
+	}
+}
+
 // TestListObjectVersionsDeleteMarkers tests that delete markers are properly separated from versions
 // This test verifies the fix for the issue where delete markers were incorrectly categorized as versions
 func TestListObjectVersionsDeleteMarkers(t *testing.T) {
@@ -537,13 +616,18 @@ func TestSuspendedVersioningDeleteBehavior(t *testing.T) {
 	})
 	require.NoError(t, err)
 
-	// Verify the null version was actually deleted (not a delete marker created)
+	// Verify the null content version is gone and the delete marker stands in.
 	listResp, err = client.ListObjectVersions(context.TODO(), &s3.ListObjectVersionsInput{
 		Bucket: aws.String(bucketName),
 	})
 	require.NoError(t, err)
 	assert.Len(t, listResp.Versions, 3, "Should be back to 3 versions after deleting null version")
-	assert.Empty(t, listResp.DeleteMarkers, "Should have no delete markers during suspended versioning delete")
+	// A suspended-versioning DELETE removes the null version and records a single
+	// null delete marker that overwrites any prior one (S3 spec), so the object
+	// reads as deleted without markers piling up.
+	require.Len(t, listResp.DeleteMarkers, 1, "Suspended delete should record exactly one null delete marker")
+	assert.Equal(t, "null", aws.ToString(listResp.DeleteMarkers[0].VersionId), "Suspended delete marker should have version id null")
+	assert.Equal(t, objectKey, aws.ToString(listResp.DeleteMarkers[0].Key), "Delete marker should be for the deleted object")
 
 	// Verify null version is gone
 	nullVersionFound = false
@@ -596,7 +680,7 @@ func TestSuspendedVersioningDeleteBehavior(t *testing.T) {
 		Key:    aws.String(objectKey),
 	})
 	require.NoError(t, err)
-	assert.Equal(t, "true", deleteResp.DeleteMarker, "Should create delete marker when versioning is enabled")
+	assert.True(t, aws.ToBool(deleteResp.DeleteMarker), "Should create delete marker when versioning is enabled")
 
 	// Verify final state
 	listResp, err = client.ListObjectVersions(context.TODO(), &s3.ListObjectVersionsInput{
@@ -604,7 +688,19 @@ func TestSuspendedVersioningDeleteBehavior(t *testing.T) {
 	})
 	require.NoError(t, err)
 	assert.Len(t, listResp.Versions, 4, "Should have 3 original versions + 1 new version")
-	assert.Len(t, listResp.DeleteMarkers, 1, "Should have 1 delete marker")
+	// The historical null suspended marker plus the new enabled-mode delete marker.
+	require.Len(t, listResp.DeleteMarkers, 2, "Should have the null suspended marker and the enabled-mode delete marker")
+	var sawNull, sawEnabled bool
+	for _, dm := range listResp.DeleteMarkers {
+		assert.Equal(t, objectKey, aws.ToString(dm.Key), "Delete marker should be for the deleted object")
+		if aws.ToString(dm.VersionId) == "null" {
+			sawNull = true
+		} else {
+			sawEnabled = true
+		}
+	}
+	assert.True(t, sawNull, "Suspended phase should leave a null delete marker")
+	assert.True(t, sawEnabled, "Enabled delete should add a real-version-id delete marker")
 
 	t.Logf("Successfully verified suspended versioning delete behavior")
 }
@@ -664,7 +760,8 @@ func TestVersionedObjectListBehavior(t *testing.T) {
 	assert.NotContains(t, *listedObject.Key, versionId, "Object key should not contain version ID")
 
 	// Verify object properties
-	assert.Equal(t, int64(len(content)), listedObject.Size, "Object size should match")
+	require.NotNil(t, listedObject.Size, "Object size should not be nil")
+	assert.Equal(t, int64(len(content)), *listedObject.Size, "Object size should match")
 	assert.NotNil(t, listedObject.ETag, "Object should have ETag")
 	assert.NotNil(t, listedObject.LastModified, "Object should have LastModified")
 
@@ -674,7 +771,7 @@ func TestVersionedObjectListBehavior(t *testing.T) {
 	assert.NotEmpty(t, listedObject.Owner.DisplayName, "Owner DisplayName should not be empty")
 
 	t.Logf("Listed object: Key=%s, Size=%d, Owner.ID=%s, Owner.DisplayName=%s",
-		*listedObject.Key, listedObject.Size, *listedObject.Owner.ID, *listedObject.Owner.DisplayName)
+		*listedObject.Key, *listedObject.Size, *listedObject.Owner.ID, *listedObject.Owner.DisplayName)
 
 	// Test list-objects-v2 operation as well
 	listV2Resp, err := client.ListObjectsV2(context.TODO(), &s3.ListObjectsV2Input{

test/s3/versioning/s3_versioning_test.go

@@ -36,8 +36,8 @@ type S3TestConfig struct {
 
 // Default test configuration - should match s3tests.conf
 var defaultConfig = &S3TestConfig{
-	Endpoint:       firstNonEmpty(os.Getenv("S3_ENDPOINT"), "http://localhost:8333"),       // Default SeaweedFS S3 port
-	MasterEndpoint: firstNonEmpty(os.Getenv("MASTER_ENDPOINT"), "http://127.0.0.1:9333"),   // Default SeaweedFS master HTTP port
+	Endpoint:       firstNonEmpty(os.Getenv("S3_ENDPOINT"), "http://localhost:8333"),     // Default SeaweedFS S3 port
+	MasterEndpoint: firstNonEmpty(os.Getenv("MASTER_ENDPOINT"), "http://127.0.0.1:9333"), // Default SeaweedFS master HTTP port
 	AccessKey:      "some_access_key1",
 	SecretKey:      "some_secret_key1",
 	Region:         "us-east-1",
@@ -50,7 +50,7 @@ var defaultConfig = &S3TestConfig{
 // cleanupAllTestBuckets uses it to find stale buckets from prior tests/runs.
 // Add the new prefix here whenever a test introduces one.
 var allTestBucketPrefixes = []string{
-	"test-versioning-",  // covers test-versioning-, test-versioning-directories, test-versioning-interaction-, test-versioned-acl/list
+	"test-versioning-", // covers test-versioning-, test-versioning-directories, test-versioning-interaction-, test-versioned-acl/list
 	"test-versioned-",
 	"test-error-messages-",
 	"test-delete-markers",
@@ -104,6 +104,28 @@ func getS3Client(t *testing.T) *s3.Client {
 	})
 }
 
+// getS3ClientWithCredentials creates an S3 client authenticating as a specific
+// identity, so tests can act as a different owner than the default client.
+func getS3ClientWithCredentials(t *testing.T, accessKey, secretKey string) *s3.Client {
+	cfg, err := config.LoadDefaultConfig(context.TODO(),
+		config.WithRegion(defaultConfig.Region),
+		config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(accessKey, secretKey, "")),
+		config.WithEndpointResolverWithOptions(aws.EndpointResolverWithOptionsFunc(
+			func(service, region string, options ...interface{}) (aws.Endpoint, error) {
+				return aws.Endpoint{
+					URL:               defaultConfig.Endpoint,
+					SigningRegion:     defaultConfig.Region,
+					HostnameImmutable: true,
+				}, nil
+			})),
+	)
+	require.NoError(t, err)
+
+	return s3.NewFromConfig(cfg, func(o *s3.Options) {
+		o.UsePathStyle = true
+	})
+}
+
 // getNewBucketName generates a unique bucket name
 func getNewBucketName() string {
 	timestamp := time.Now().UnixNano()

test/s3tables/lakekeeper/lakekeeper_test.go

@@ -68,7 +68,6 @@ func TestLakekeeperIntegration(t *testing.T) {
 	env.StartSeaweedFS(t)
 	fmt.Printf(">>> SeaweedFS started.\n")
 
-	// Run python script in docker to test STS and S3 operations
 	runLakekeeperRepro(t, env)
 }
 
@@ -80,7 +79,6 @@ func TestLakekeeperTableBucketIntegration(t *testing.T) {
 	env.StartSeaweedFS(t)
 	fmt.Printf(">>> SeaweedFS started.\n")
 
-	// Run python script in docker to test STS and S3 Tables operations
 	runLakekeeperTableBucketRepro(t, env)
 }
 

test/volume_server/framework/cluster_multi.go

@@ -7,6 +7,7 @@ import (
 	"os/exec"
 	"path/filepath"
 	"strconv"
+	"strings"
 	"sync"
 	"testing"
 
@@ -24,6 +25,7 @@ type MultiVolumeCluster struct {
 	logsDir           string
 	keepLogs          bool
 	volumeServerCount int
+	disksPerServer    int
 
 	masterPort     int
 	masterGrpcPort int
@@ -31,6 +33,7 @@ type MultiVolumeCluster struct {
 	volumePorts     []int
 	volumeGrpcPorts []int
 	volumePubPorts  []int
+	volumeDiskDirs  [][]string // per server: the data directories (one DiskLocation each)
 
 	masterCmd  *exec.Cmd
 	volumeCmds []*exec.Cmd
@@ -38,13 +41,25 @@ type MultiVolumeCluster struct {
 	cleanupOnce sync.Once
 }
 
-// StartMultiVolumeCluster starts a cluster with a specified number of volume servers
+// StartMultiVolumeCluster starts a cluster with serverCount volume servers, one
+// data directory (DiskLocation) each.
 func StartMultiVolumeCluster(t testing.TB, profile matrix.Profile, serverCount int) *MultiVolumeCluster {
+	return StartMultiVolumeClusterWithDisks(t, profile, serverCount, 1)
+}
+
+// StartMultiVolumeClusterWithDisks starts serverCount volume servers, each with
+// disksPerServer data directories passed to -dir as a comma list so every
+// directory becomes its own DiskLocation. Lets tests exercise per-disk EC
+// layouts (e.g. shards on one disk, a stub/index on a sibling disk).
+func StartMultiVolumeClusterWithDisks(t testing.TB, profile matrix.Profile, serverCount, disksPerServer int) *MultiVolumeCluster {
 	t.Helper()
 
 	if serverCount < 1 {
 		t.Fatalf("serverCount must be at least 1, got %d", serverCount)
 	}
+	if disksPerServer < 1 {
+		t.Fatalf("disksPerServer must be at least 1, got %d", disksPerServer)
+	}
 
 	weedBinary, err := FindOrBuildWeedBinary()
 	if err != nil {
@@ -60,10 +75,21 @@ func StartMultiVolumeCluster(t testing.TB, profile matrix.Profile, serverCount i
 	logsDir := filepath.Join(baseDir, "logs")
 	masterDataDir := filepath.Join(baseDir, "master")
 
-	// Create directories for master and all volume servers
+	// Create directories for master and all volume servers. With one disk the
+	// layout stays at baseDir/volumeN; multi-disk servers get baseDir/volumeN/diskD.
 	dirs := []string{configDir, logsDir, masterDataDir}
+	volumeDiskDirs := make([][]string, serverCount)
 	for i := 0; i < serverCount; i++ {
-		dirs = append(dirs, filepath.Join(baseDir, fmt.Sprintf("volume%d", i)))
+		serverDir := filepath.Join(baseDir, fmt.Sprintf("volume%d", i))
+		volumeDiskDirs[i] = make([]string, disksPerServer)
+		for d := 0; d < disksPerServer; d++ {
+			if disksPerServer == 1 {
+				volumeDiskDirs[i][d] = serverDir
+			} else {
+				volumeDiskDirs[i][d] = filepath.Join(serverDir, fmt.Sprintf("disk%d", d))
+			}
+			dirs = append(dirs, volumeDiskDirs[i][d])
+		}
 	}
 	for _, dir := range dirs {
 		if mkErr := os.MkdirAll(dir, 0o755); mkErr != nil {
@@ -98,11 +124,13 @@ func StartMultiVolumeCluster(t testing.TB, profile matrix.Profile, serverCount i
 		logsDir:           logsDir,
 		keepLogs:          keepLogs,
 		volumeServerCount: serverCount,
+		disksPerServer:    disksPerServer,
 		masterPort:        masterPort,
 		masterGrpcPort:    masterGrpcPort,
 		volumePorts:       make([]int, serverCount),
 		volumeGrpcPorts:   make([]int, serverCount),
 		volumePubPorts:    make([]int, serverCount),
+		volumeDiskDirs:    volumeDiskDirs,
 		volumeCmds:        make([]*exec.Cmd, serverCount),
 	}
 
@@ -133,8 +161,7 @@ func StartMultiVolumeCluster(t testing.TB, profile matrix.Profile, serverCount i
 
 	// Start all volume servers
 	for i := 0; i < serverCount; i++ {
-		volumeDataDir := filepath.Join(baseDir, fmt.Sprintf("volume%d", i))
-		if err = c.startVolume(i, volumeDataDir); err != nil {
+		if err = c.startVolume(i, c.volumeDiskDirs[i]); err != nil {
 			// Log current server's log for debugging startup failures
 			volumeLog := fmt.Sprintf("volume%d.log", i)
 			c.Stop()
@@ -207,13 +234,17 @@ func (c *MultiVolumeCluster) startMaster(dataDir string) error {
 	return c.masterCmd.Start()
 }
 
-func (c *MultiVolumeCluster) startVolume(index int, dataDir string) error {
+func (c *MultiVolumeCluster) startVolume(index int, dataDirs []string) error {
 	logName := fmt.Sprintf("volume%d.log", index)
 	logFile, err := os.Create(filepath.Join(c.logsDir, logName))
 	if err != nil {
 		return err
 	}
 
+	maxPerDir := make([]string, len(dataDirs))
+	for i := range dataDirs {
+		maxPerDir[i] = "16"
+	}
 	args := []string{
 		"-config_dir=" + c.configDir,
 		"volume",
@@ -221,8 +252,8 @@ func (c *MultiVolumeCluster) startVolume(index int, dataDir string) error {
 		"-port=" + strconv.Itoa(c.volumePorts[index]),
 		"-port.grpc=" + strconv.Itoa(c.volumeGrpcPorts[index]),
 		"-port.public=" + strconv.Itoa(c.volumePubPorts[index]),
-		"-dir=" + dataDir,
-		"-max=16",
+		"-dir=" + strings.Join(dataDirs, ","),
+		"-max=" + strings.Join(maxPerDir, ","),
 		"-master=127.0.0.1:" + strconv.Itoa(c.masterPort),
 		"-readMode=" + c.profile.ReadMode,
 		"-concurrentUploadLimitMB=" + strconv.Itoa(c.profile.ConcurrentUploadLimitMB),
@@ -301,3 +332,17 @@ func (c *MultiVolumeCluster) VolumePublicURL(index int) string {
 func (c *MultiVolumeCluster) BaseDir() string {
 	return c.baseDir
 }
+
+// VolumeDiskDir returns the data directory backing disk diskIndex on volume
+// server serverIndex — i.e. the directory for that DiskLocation, for file
+// inspection in tests.
+func (c *MultiVolumeCluster) VolumeDiskDir(serverIndex, diskIndex int) string {
+	if serverIndex < 0 || serverIndex >= len(c.volumeDiskDirs) {
+		return ""
+	}
+	disks := c.volumeDiskDirs[serverIndex]
+	if diskIndex < 0 || diskIndex >= len(disks) {
+		return ""
+	}
+	return disks[diskIndex]
+}

unmaintained/stress/ec_stress_runner.go

@@ -301,7 +301,7 @@ func (r *runner) fetchVolumeList(ctx context.Context) (*master_pb.VolumeListResp
 		}
 
 		var response *master_pb.VolumeListResponse
-		err := pb.WithMasterClient(false, pb.ServerAddress(masterAddress), r.grpcDialOption, false, func(client master_pb.SeaweedClient) error {
+		err := pb.WithMasterClient(context.Background(), false, pb.ServerAddress(masterAddress), r.grpcDialOption, false, func(client master_pb.SeaweedClient) error {
 			callCtx, cancel := context.WithTimeout(ctx, r.cfg.RequestTimeout)
 			defer cancel()
 

weed/admin/Makefile

@@ -21,11 +21,17 @@ install-templ:
 
 # Generate templ files
 .PHONY: generate
-generate: install-templ
+generate: install-templ static-gz
 	@echo "Generating templ files..."
 	@templ generate ./view
 	@echo "Generated: $(TEMPL_GO_FILES)"
 
+# Regenerate the gzipped static mirror embedded into the binary
+.PHONY: static-gz
+static-gz:
+	@echo "Generating gzipped static assets..."
+	@go run gen_static_gz.go
+
 # Clean generated files
 .PHONY: clean-templ
 clean-templ:

weed/admin/dash/admin_data.go

@@ -30,6 +30,7 @@ type AdminData struct {
 	VolumeServers     []VolumeServer      `json:"volume_servers"`
 	FilerNodes        []FilerNode         `json:"filer_nodes"`
 	MessageBrokers    []MessageBrokerNode `json:"message_brokers"`
+	S3Nodes           []S3Node            `json:"s3_nodes"`
 	DataCenters       []DataCenter        `json:"datacenters"`
 	LastUpdated       time.Time           `json:"last_updated"`
 
@@ -115,6 +116,12 @@ type MessageBrokerNode struct {
 	LastUpdated time.Time `json:"last_updated"`
 }
 
+type S3Node struct {
+	Address     string    `json:"address"`
+	DataCenter  string    `json:"datacenter"`
+	LastUpdated time.Time `json:"last_updated"`
+}
+
 // GetAdminData retrieves admin data as a struct (for reuse by both JSON and HTML handlers)
 func (s *AdminServer) GetAdminData(username string) (AdminData, error) {
 	if username == "" {
@@ -144,6 +151,9 @@ func (s *AdminServer) GetAdminData(username string) (AdminData, error) {
 	// Get message broker nodes status
 	messageBrokers := s.getMessageBrokerNodesStatus()
 
+	// Get S3 nodes status
+	s3Nodes := s.getS3NodesStatus()
+
 	// Get volume size limit from master configuration
 	var volumeSizeLimitMB uint64 = 30000 // Default to 30GB
 	err = s.WithMasterClient(func(client master_pb.SeaweedClient) error {
@@ -183,6 +193,7 @@ func (s *AdminServer) GetAdminData(username string) (AdminData, error) {
 		VolumeServers:     volumeServersData.VolumeServers,
 		FilerNodes:        filerNodes,
 		MessageBrokers:    messageBrokers,
+		S3Nodes:           s3Nodes,
 		DataCenters:       topology.DataCenters,
 		LastUpdated:       topology.UpdatedAt,
 		TotalEcVolumes:    totalEcVolumes,
@@ -360,3 +371,43 @@ func (s *AdminServer) getMessageBrokerNodesStatus() []MessageBrokerNode {
 
 	return messageBrokers
 }
+
+// getS3NodesStatus checks status of all S3 nodes using master's ListClusterNodes
+func (s *AdminServer) getS3NodesStatus() []S3Node {
+	var s3Nodes []S3Node
+
+	// Get S3 nodes from master using ListClusterNodes
+	err := s.WithMasterClient(func(client master_pb.SeaweedClient) error {
+		resp, err := client.ListClusterNodes(context.Background(), &master_pb.ListClusterNodesRequest{
+			ClientType: cluster.S3Type,
+		})
+		if err != nil {
+			return err
+		}
+
+		// Process each S3 node
+		for _, node := range resp.ClusterNodes {
+			s3Nodes = append(s3Nodes, S3Node{
+				Address:     pb.ServerAddress(node.Address).ToHttpAddress(),
+				DataCenter:  node.DataCenter,
+				LastUpdated: time.Now(),
+			})
+		}
+
+		return nil
+	})
+
+	if err != nil {
+		currentMaster := s.masterClient.GetMaster(context.Background())
+		glog.Errorf("Failed to get S3 nodes from master %s: %v", currentMaster, err)
+		// Return empty list if we can't get S3 info from master
+		return []S3Node{}
+	}
+
+	// Sort S3 nodes by address for consistent ordering on page refresh
+	sort.Slice(s3Nodes, func(i, j int) bool {
+		return s3Nodes[i].Address < s3Nodes[j].Address
+	})
+
+	return s3Nodes
+}

weed/admin/dash/admin_server.go

@@ -1189,6 +1189,52 @@ func (s *AdminServer) GetClusterBrokers() (*ClusterBrokersData, error) {
 	}, nil
 }
 
+// GetClusterS3Servers retrieves cluster S3 servers data
+func (s *AdminServer) GetClusterS3Servers() (*ClusterS3ServersData, error) {
+	var s3Servers []S3ServerInfo
+
+	// Get S3 server information from master using ListClusterNodes
+	err := s.WithMasterClient(func(client master_pb.SeaweedClient) error {
+		resp, err := client.ListClusterNodes(context.Background(), &master_pb.ListClusterNodesRequest{
+			ClientType: cluster.S3Type,
+		})
+		if err != nil {
+			return err
+		}
+
+		// Process each S3 server node
+		for _, node := range resp.ClusterNodes {
+			createdAt := time.Unix(0, node.CreatedAtNs)
+
+			s3ServerInfo := S3ServerInfo{
+				Address:    pb.ServerAddress(node.Address).ToHttpAddress(),
+				DataCenter: node.DataCenter,
+				Version:    node.Version,
+				CreatedAt:  createdAt,
+			}
+
+			s3Servers = append(s3Servers, s3ServerInfo)
+		}
+
+		return nil
+	})
+
+	if err != nil {
+		return nil, fmt.Errorf("failed to get S3 server nodes from master: %w", err)
+	}
+
+	// Sort S3 servers by address for consistent ordering on page refresh
+	sort.Slice(s3Servers, func(i, j int) bool {
+		return s3Servers[i].Address < s3Servers[j].Address
+	})
+
+	return &ClusterS3ServersData{
+		S3Servers:      s3Servers,
+		TotalS3Servers: len(s3Servers),
+		LastUpdated:    time.Now(),
+	}, nil
+}
+
 // GetAllFilers method moved to client_management.go
 
 // GetVolumeDetails method moved to volume_management.go
@@ -1778,7 +1824,7 @@ func collectCollectionStats(topologyInfo *master_pb.TopologyInfo) map[string]col
 						shards := erasure_coding.ShardsInfoFromVolumeEcShardInformationMessage(ecShardInfo)
 						data := collectionMap[collection]
 						data.PhysicalSize += int64(shards.TotalSize())
-						data.LogicalSize += int64(shards.MinusParityShards().TotalSize())
+						data.LogicalSize += int64(shards.MinusParityShards(erasure_coding.DataShardsCount).TotalSize())
 						collectionMap[collection] = data
 
 						// fileCount is volume-wide (same .ecx on every shard

weed/admin/dash/mq_management.go

@@ -4,7 +4,7 @@ import (
 	"context"
 	"fmt"
 	"io"
-	"path/filepath"
+	"path"
 	"strings"
 	"time"
 
@@ -325,7 +325,7 @@ func (s *AdminServer) GetConsumerGroupOffsets(namespace, topicName string) ([]Co
 
 			// Only process directories that are versions (start with "v")
 			if versionResp.Entry != nil && versionResp.Entry.IsDirectory && strings.HasPrefix(versionResp.Entry.Name, "v") {
-				versionDir := filepath.Join(topicDir, versionResp.Entry.Name)
+				versionDir := path.Join(topicDir, versionResp.Entry.Name)
 
 				// List all partition directories under the version directory (e.g., 0315-0630)
 				partitionStream, err := client.ListEntries(context.Background(), &filer_pb.ListEntriesRequest{
@@ -360,7 +360,7 @@ func (s *AdminServer) GetConsumerGroupOffsets(namespace, topicName string) ([]Co
 							continue
 						}
 
-						partitionDir := filepath.Join(versionDir, partitionResp.Entry.Name)
+						partitionDir := path.Join(versionDir, partitionResp.Entry.Name)
 
 						// List all .offset files in this partition directory
 						offsetStream, err := client.ListEntries(context.Background(), &filer_pb.ListEntriesRequest{