docs: regenerate star history chart

* feat(filer): option to store system metadata logs in their own collection

The filer's internal /topics/.system/log chunks are assigned to the
filer's default collection (-collection). In a multi-filer deployment
that default is often empty, so every restart flap, full-sync, or
event-buffered flush grows the default collection with system chunks that
are indistinguishable from user data in collection.list. This is a large
part of what makes the default collection balloon and confuses orphan
analysis.

This keeps the internal log in a dedicated collection when the operator
asks for one, without changing where user data goes:

- New optional override, filer.options.metaLog.collection (and
  .replication), read in NewFiler so both `weed filer` and
  `weed server -filer` honour it. Default "" => exactly today's
  behaviour (log follows the filer default), fully backward compatible.
- Resolution is a small helper: override first, then the filer default,
  then a storage rule matched on the log path. Kept separate from the
  user write path so the internal log targets itself.
- bucketCollection() is hardened the same way it already protects the
  filer's default collection: a bucket that happens to resolve to the
  redirected meta-log collection must not drop it on delete, because it
  backs internal log volumes.
- Scaffold filer.toml documents the new knobs under [filer.options].

Related to the persisted deletion ledger branch (fix/persist-deletion-queue):
together they cut the two sources of post-flap junk in the default
collection — that PR stops orphaned user-chunk leak on filer crash,
this one stops the internal log from living in default at all. They are
independent: no file overlap, no functional dependency; either can merge
first. They are paired only in the narrative of cleaning up default.

Adds unit tests for the collection/replication resolution chain, the
viper keys, and the bucket-delete guard (run green under -race).

Co-Authored-By: Athena 🏛️ <hermes-agent@local> (custom / Qwen3.8-Flash-Next-ROCmFP4)

* filer: collect bucket chunks when its collection survives the delete

bucketCollection returning "" preserves the collection, but the bucket
path still skipped per-entry chunk collection and could skip listing the
children entirely, so a bucket sharing the meta-log (or any preserved)
collection left its object chunks orphaned with no entry pointing at
them. Only the wholesale drop of a deleted collection skips those now.

Note in filer.toml that the meta-log target should stay stable: chunks
written under an older collection are not migrated.

* filer: exercise the metaLog override wiring through NewFiler

The viper test only echoed back the keys it set, so a wrong key in
NewFiler would still pass. It now asserts the fields NewFiler fills
from those keys.

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer: tighten comments around the metaLog collection override

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Chris Lu <chris.lu@gmail.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

.codespellrc

@@ -19,4 +19,5 @@ ignore-regex = \b[a-z]+[A-Z]\w*\b|\b[A-Z][a-z]+[A-Z]\w*\b
 # thirdparty: literal Maven groupId `org.apache.hadoop.thirdparty` (external, cannot rename)
 # unknwon: GitHub username / Go module path (`github.com/unknwon/goconfig`)
 # atleast: CLI mode literal string in test/benchmark/fuse_db/bin/sqlite_verify.py
-ignore-words-list = visibles,fo,te,ser,bject,unparseable,keep-alives,tread,anc,ue,auther,thirdparty,unknwon,atleast
+# sme: local variable for a *streamMutateError in mount tests
+ignore-words-list = visibles,fo,te,ser,bject,unparseable,keep-alives,tread,anc,ue,auther,thirdparty,unknwon,atleast,sme

.github/actions/sign-image/action.yml

@@ -0,0 +1,44 @@
+name: Sign container images
+description: >
+  Keyless cosign signature on each image, then a verification pass against the
+  identity the signature should carry, so a misconfigured job fails here and not
+  on someone's cluster. That identity is the calling workflow's own,
+  https://github.com/<owner>/<repo>/.github/workflows/<file>@<ref>.
+  The calling job needs `id-token: write` and a registry login for every image.
+
+inputs:
+  images:
+    description: Image references by digest (name@sha256:...), whitespace separated.
+    required: true
+
+runs:
+  using: composite
+  steps:
+    - name: Install cosign
+      uses: sigstore/cosign-installer@v4.1.2
+
+    - name: Sign
+      shell: bash
+      env:
+        IMAGES: ${{ inputs.images }}
+      run: |
+        set -euo pipefail
+        # The .sig tag layout: the OCI-referrer bundle cosign 3 writes by default
+        # is not read by the Kyverno and policy-controller releases in use today.
+        # Cosign 3 also defaults to --use-signing-config, which insists on a
+        # bundle for its output; turning it off falls back to the default
+        # Fulcio and Rekor URLs, which is all the .sig layout ever used.
+        cosign sign --yes --recursive \
+          --new-bundle-format=false --use-signing-config=false \
+          $IMAGES
+
+    - name: Verify
+      shell: bash
+      env:
+        IMAGES: ${{ inputs.images }}
+      run: |
+        set -euo pipefail
+        cosign verify \
+          --certificate-oidc-issuer https://token.actions.githubusercontent.com \
+          --certificate-identity "https://github.com/$GITHUB_WORKFLOW_REF" \
+          $IMAGES

.github/dependabot.yml

@@ -1,7 +1,9 @@
 version: 2
 updates:
 - package-ecosystem: "github-actions"
-  directory: "/"
+  directories:
+    - "/"
+    - "/.github/actions/sign-image"
   schedule:
       interval: "weekly"
 - package-ecosystem: gomod

.github/scripts/star_history.py

@@ -0,0 +1,87 @@
+#!/usr/bin/env python3
+"""Render the repository's star history to note/star_history.svg.
+
+Uses the GitHub REST stargazers endpoint with the starred-at accept header,
+which caps at 40,000 entries (400 pages of 100). The chart is regenerated on
+a schedule; if the repo grows past that cap the script stops at 40,000 and
+logs a warning rather than under-reporting.
+"""
+import json
+import os
+import sys
+import urllib.request
+from datetime import datetime
+
+import matplotlib
+
+matplotlib.use("Agg")
+import matplotlib.pyplot as plt  # noqa: E402
+from matplotlib.dates import AutoDateLocator, DateFormatter  # noqa: E402
+
+REPO = "seaweedfs/seaweedfs"
+TOKEN = os.environ["GITHUB_TOKEN"]
+OUT = os.environ.get("OUT", "note/star_history.svg")
+PAGE_CAP = 400  # GitHub's hard limit on stargazer pagination
+
+
+def fetch_stargazers():
+    stars = []
+    page = 1
+    while page <= PAGE_CAP:
+        url = f"https://api.github.com/repos/{REPO}/stargazers?per_page=100&page={page}"
+        req = urllib.request.Request(
+            url,
+            headers={
+                "Accept": "application/vnd.github.star+json",
+                "Authorization": f"Bearer {TOKEN}",
+                "X-GitHub-Api-Version": "2022-11-28",
+                "User-Agent": "seaweedfs-star-history",
+            },
+        )
+        with urllib.request.urlopen(req) as resp:
+            batch = json.load(resp)
+        if not batch:
+            break
+        for u in batch:
+            sa = u.get("starred_at")
+            if sa:
+                stars.append(datetime.fromisoformat(sa.replace("Z", "+00:00")))
+        if len(batch) < 100:
+            break
+        page += 1
+    if page > PAGE_CAP:
+        print(
+            f"::warning::Hit the {PAGE_CAP}-page stargazer pagination cap; "
+            "chart reflects the first 40,000 stars only."
+        )
+    return stars
+
+
+def render(stars, out):
+    stars.sort()
+    counts = list(range(1, len(stars) + 1))
+    fig, ax = plt.subplots(figsize=(10, 6), dpi=130)
+    ax.plot(stars, counts, color="#0969da", linewidth=1.6)
+    ax.set_xlabel("Date")
+    ax.set_ylabel("Stars")
+    ax.set_title(f"{REPO} star history")
+    ax.grid(True, linestyle="--", alpha=0.3)
+    ax.xaxis.set_major_locator(AutoDateLocator())
+    ax.xaxis.set_major_formatter(DateFormatter("%Y-%m"))
+    fig.autofmt_xdate()
+    fig.tight_layout()
+    fig.savefig(out, format="svg", transparent=False)
+    plt.close(fig)
+
+
+def main():
+    stars = fetch_stargazers()
+    if not stars:
+        print("::error::No stargazers fetched; not updating the chart.")
+        sys.exit(1)
+    render(stars, OUT)
+    print(f"Rendered {len(stars)} stars to {OUT}")
+
+
+if __name__ == "__main__":
+    main()

.github/workflows/codeql.yml

@@ -27,7 +27,7 @@ jobs:
 
       # Initializes the CodeQL tools for scanning.
       - name: Initialize CodeQL
-        uses: github/codeql-action/init@v4.37.4
+        uses: github/codeql-action/init@v4.38.2
         # Override language selection by uncommenting this and choosing your languages
         with:
           languages: go
@@ -35,7 +35,7 @@ jobs:
       # Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
       # If this step fails, then you should remove it and run the build manually (see below).
       - name: Autobuild
-        uses: github/codeql-action/autobuild@v4.37.4
+        uses: github/codeql-action/autobuild@v4.38.2
 
       # ℹ️ Command-line programs to run using the OS shell.
       # 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
@@ -49,4 +49,4 @@ jobs:
       #     make release
 
       - name: Perform CodeQL Analysis
-        uses: github/codeql-action/analyze@v4.37.4
+        uses: github/codeql-action/analyze@v4.38.2

.github/workflows/container_dev.yml

@@ -6,6 +6,8 @@ on:
     paths:
       - 'weed/**'
       - 'seaweed-volume/**'
+      - 'seaweed-common/**'
+      - 'seaweed-worker/**'
       - 'docker/**'
       - 'go.mod'
       - 'go.sum'
@@ -16,7 +18,7 @@ permissions:
 
 jobs:
 
-  # ── Pre-build Rust volume server binaries natively ──────────────────
+  # ── Pre-build the Rust binaries natively ────────────────────────────
   build-rust-binaries:
     runs-on: ubuntu-22.04
     strategy:
@@ -31,9 +33,6 @@ jobs:
       - name: Checkout
         uses: actions/checkout@v7
 
-      - name: Install protobuf compiler
-        run: sudo apt-get update && sudo apt-get install -y protobuf-compiler
-
       - name: Install Rust toolchain
         uses: dtolnay/rust-toolchain@stable
         with:
@@ -58,10 +57,26 @@ jobs:
             ~/.cargo/registry
             ~/.cargo/git
             seaweed-volume/target
-          key: rust-docker-dev-${{ matrix.target }}-${{ hashFiles('seaweed-volume/Cargo.lock') }}
+            seaweed-worker/target/${{ matrix.target }}/release
+          key: rust-docker-dev-${{ matrix.target }}-${{ hashFiles('seaweed-volume/Cargo.lock', 'seaweed-worker/Cargo.lock') }}
           restore-keys: |
             rust-docker-dev-${{ matrix.target }}-
 
+      # lance's build scripts compile their own protos and look for a protoc.
+      # Point them at the one protoc-bin-vendored ships, which seaweed-worker's
+      # own build already uses, so no job depends on a system package and every
+      # build sees the same version.
+      - name: Use the vendored protoc
+        run: |
+          cd seaweed-worker
+          cargo fetch
+          # The version from the lock, not whatever else a restored cache holds.
+          version=$(awk '/^name = "protoc-bin-vendored-linux-x86_64"$/{found=1; next} found && /^version = /{gsub(/"/,"",$3); print $3; exit}' Cargo.lock)
+          test -n "$version" || { echo "protoc-bin-vendored-linux-x86_64 is not in Cargo.lock" >&2; exit 1; }
+          protoc=$(find ~/.cargo/registry/src -path "*protoc-bin-vendored-linux-x86_64-$version/bin/protoc" | head -1)
+          test -x "$protoc" || { echo "no vendored protoc $version in the registry" >&2; exit 1; }
+          echo "PROTOC=$protoc" >> "$GITHUB_ENV"
+
       - name: Build normal variant
         env:
           SEAWEEDFS_COMMIT: ${{ github.sha }}
@@ -70,15 +85,26 @@ jobs:
           cargo build --release --target ${{ matrix.target }} --no-default-features
           cp target/${{ matrix.target }}/release/weed-volume ../weed-volume-normal-${{ matrix.arch }}
 
+      - name: Build the Rust maintenance worker
+        run: |
+          cd seaweed-worker
+          cargo build --release -p weed-lance-worker --target ${{ matrix.target }}
+          cp target/${{ matrix.target }}/release/weed-worker ../weed-worker-${{ matrix.arch }}
+
       - name: Upload artifacts
         uses: actions/upload-artifact@v7
         with:
-          name: rust-volume-${{ matrix.arch }}
-          path: weed-volume-normal-${{ matrix.arch }}
+          name: rust-bins-${{ matrix.arch }}
+          path: |
+            weed-volume-normal-${{ matrix.arch }}
+            weed-worker-${{ matrix.arch }}
 
   build-dev-containers:
     needs: [build-rust-binaries]
     runs-on: [ubuntu-latest]
+    permissions:
+      contents: read
+      id-token: write
 
     steps:
       - name: Checkout
@@ -87,7 +113,7 @@ jobs:
       - name: Download pre-built Rust binaries
         uses: actions/download-artifact@v8
         with:
-          pattern: rust-volume-*
+          pattern: rust-bins-*
           merge-multiple: true
           path: ./rust-bins
 
@@ -101,7 +127,16 @@ jobs:
               echo "Placed pre-built Rust binary for ${arch}"
             fi
           done
+          mkdir -p docker/weed-worker-prebuilt
+          for arch in amd64 arm64; do
+            src="./rust-bins/weed-worker-${arch}"
+            if [ -f "$src" ]; then
+              cp "$src" "docker/weed-worker-prebuilt/weed-worker-${arch}"
+              echo "Placed pre-built Rust worker for ${arch}"
+            fi
+          done
           ls -la docker/weed-volume-prebuilt/
+          ls -la docker/weed-worker-prebuilt/
 
       - name: Docker meta
         id: docker_meta
@@ -118,7 +153,7 @@ jobs:
             org.opencontainers.image.vendor=Chris Lu
 
       - name: Set up QEMU
-        uses: docker/setup-qemu-action@v4.2.0
+        uses: docker/setup-qemu-action@v4.4.0
 
       - name: Create BuildKit config
         run: |
@@ -149,6 +184,7 @@ jobs:
           password: ${{ secrets.GHCR_TOKEN }}
 
       - name: Build
+        id: build
         uses: docker/build-push-action@v7
         with:
           context: ./docker
@@ -157,3 +193,11 @@ jobs:
           platforms: linux/amd64, linux/arm64
           tags: ${{ steps.docker_meta.outputs.tags }}
           labels: ${{ steps.docker_meta.outputs.labels }}
+
+      - name: Sign
+        if: github.event_name != 'pull_request'
+        uses: ./.github/actions/sign-image
+        with:
+          images: >-
+            chrislusf/seaweedfs@${{ steps.build.outputs.digest }}
+            ghcr.io/chrislusf/seaweedfs@${{ steps.build.outputs.digest }}

.github/workflows/container_foundationdb_version.yml

@@ -30,6 +30,9 @@ permissions:
 jobs:
   build-foundationdb-image:
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      id-token: write
 
     steps:
       - name: Checkout
@@ -126,7 +129,7 @@ jobs:
           echo "seaweedfs_ref=$seaweed" >> "$GITHUB_OUTPUT"
 
       - name: Set up QEMU
-        uses: docker/setup-qemu-action@v4.2.0
+        uses: docker/setup-qemu-action@v4.4.0
 
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v4
@@ -154,6 +157,7 @@ jobs:
           fi
 
       - name: Build and push image
+        id: build
         uses: docker/build-push-action@v7
         with:
           context: ./docker
@@ -171,3 +175,8 @@ jobs:
             org.opencontainers.image.description=SeaweedFS is a distributed storage system for blobs, objects, files, and data lake, to store and serve billions of files fast!
             org.opencontainers.image.vendor=Chris Lu
 
+      - name: Sign
+        if: github.event_name != 'pull_request'
+        uses: ./.github/actions/sign-image
+        with:
+          images: chrislusf/seaweedfs@${{ steps.build.outputs.digest }}

.github/workflows/container_latest.yml

@@ -59,7 +59,7 @@ jobs:
             echo "publish=true" >> "$GITHUB_OUTPUT"
           fi
 
-  # ── Pre-build Rust volume server binaries natively ──────────────────
+  # ── Pre-build the Rust binaries natively ────────────────────────────
   build-rust-binaries:
     runs-on: ubuntu-22.04
     strategy:
@@ -76,9 +76,6 @@ jobs:
         with:
           ref: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.source_ref || github.ref }}
 
-      - name: Install protobuf compiler
-        run: sudo apt-get update && sudo apt-get install -y protobuf-compiler
-
       - name: Install Rust toolchain
         uses: dtolnay/rust-toolchain@stable
         with:
@@ -103,10 +100,26 @@ jobs:
             ~/.cargo/registry
             ~/.cargo/git
             seaweed-volume/target
-          key: rust-docker-${{ matrix.target }}-${{ hashFiles('seaweed-volume/Cargo.lock') }}
+            seaweed-worker/target/${{ matrix.target }}/release
+          key: rust-docker-${{ matrix.target }}-${{ hashFiles('seaweed-volume/Cargo.lock', 'seaweed-worker/Cargo.lock') }}
           restore-keys: |
             rust-docker-${{ matrix.target }}-
 
+      # lance's build scripts compile their own protos and look for a protoc.
+      # Point them at the one protoc-bin-vendored ships, which seaweed-worker's
+      # own build already uses, so no job depends on a system package and every
+      # build sees the same version.
+      - name: Use the vendored protoc
+        run: |
+          cd seaweed-worker
+          cargo fetch
+          # The version from the lock, not whatever else a restored cache holds.
+          version=$(awk '/^name = "protoc-bin-vendored-linux-x86_64"$/{found=1; next} found && /^version = /{gsub(/"/,"",$3); print $3; exit}' Cargo.lock)
+          test -n "$version" || { echo "protoc-bin-vendored-linux-x86_64 is not in Cargo.lock" >&2; exit 1; }
+          protoc=$(find ~/.cargo/registry/src -path "*protoc-bin-vendored-linux-x86_64-$version/bin/protoc" | head -1)
+          test -x "$protoc" || { echo "no vendored protoc $version in the registry" >&2; exit 1; }
+          echo "PROTOC=$protoc" >> "$GITHUB_ENV"
+
       - name: Build large-disk variant
         env:
           SEAWEEDFS_COMMIT: ${{ github.sha }}
@@ -123,20 +136,27 @@ jobs:
           cargo build --release --target ${{ matrix.target }} --no-default-features
           cp target/${{ matrix.target }}/release/weed-volume ../weed-volume-normal-${{ matrix.arch }}
 
+      - name: Build the Rust maintenance worker
+        run: |
+          cd seaweed-worker
+          cargo build --release -p weed-lance-worker --target ${{ matrix.target }}
+          cp target/${{ matrix.target }}/release/weed-worker ../weed-worker-${{ matrix.arch }}
+
       - name: Upload artifacts
         uses: actions/upload-artifact@v7
         with:
-          name: rust-volume-${{ matrix.arch }}
+          name: rust-bins-${{ matrix.arch }}
           path: |
             weed-volume-large-disk-${{ matrix.arch }}
             weed-volume-normal-${{ matrix.arch }}
+            weed-worker-${{ matrix.arch }}
 
   build:
     needs: [setup, build-rust-binaries]
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        platform: [amd64, arm64, arm, 386]
+        platform: [amd64, arm64, arm, 386, ppc64le, s390x]
         variant: ${{ fromJSON(needs.setup.outputs.variants) }}
 
     steps:
@@ -177,7 +197,7 @@ jobs:
       - name: Download pre-built Rust binaries
         uses: actions/download-artifact@v8
         with:
-          pattern: rust-volume-*
+          pattern: rust-bins-*
           merge-multiple: true
           path: ./rust-bins
 
@@ -191,7 +211,16 @@ jobs:
               echo "Placed pre-built Rust binary for ${arch}"
             fi
           done
+          mkdir -p docker/weed-worker-prebuilt
+          for arch in amd64 arm64; do
+            src="./rust-bins/weed-worker-${arch}"
+            if [ -f "$src" ]; then
+              cp "$src" "docker/weed-worker-prebuilt/weed-worker-${arch}"
+              echo "Placed pre-built Rust worker for ${arch}"
+            fi
+          done
           ls -la docker/weed-volume-prebuilt/
+          ls -la docker/weed-worker-prebuilt/
 
       - name: Docker meta
         id: docker_meta
@@ -207,7 +236,7 @@ jobs:
             org.opencontainers.image.vendor=Chris Lu
       - name: Set up QEMU
         if: matrix.platform != 'amd64'
-        uses: docker/setup-qemu-action@v4.2.0
+        uses: docker/setup-qemu-action@v4.4.0
       - name: Create BuildKit config
         run: |
           cat > /tmp/buildkitd.toml <<EOF
@@ -289,7 +318,7 @@ jobs:
         if: needs.setup.outputs.publish != 'true'
         uses: actions/download-artifact@v8
         with:
-          pattern: rust-volume-*
+          pattern: rust-bins-*
           merge-multiple: true
           path: ./rust-bins
       - name: Place Rust binaries in Docker context for local scan
@@ -307,7 +336,16 @@ jobs:
               echo "Placed pre-built Rust binary for ${arch}"
             fi
           done
+          mkdir -p docker/weed-worker-prebuilt
+          for arch in amd64 arm64; do
+            src="./rust-bins/weed-worker-${arch}"
+            if [ -f "$src" ]; then
+              cp "$src" "docker/weed-worker-prebuilt/weed-worker-${arch}"
+              echo "Placed pre-built Rust worker for ${arch}"
+            fi
+          done
           ls -la docker/weed-volume-prebuilt/
+          ls -la docker/weed-worker-prebuilt/
       - name: Create BuildKit config for local scan build
         if: needs.setup.outputs.publish != 'true'
         run: |
@@ -367,7 +405,7 @@ jobs:
           output: trivy-results.sarif
           exit-code: '0'
       - name: Upload Trivy scan results to GitHub Security
-        uses: github/codeql-action/upload-sarif@v4.37.4
+        uses: github/codeql-action/upload-sarif@v4.38.2
         if: always()
         with:
           sarif_file: trivy-results.sarif
@@ -403,15 +441,19 @@ jobs:
     runs-on: ubuntu-latest
     needs: [setup, build, trivy-scan]
     if: needs.setup.outputs.publish == 'true' && github.event_name != 'pull_request'
+    permissions:
+      contents: read
+      id-token: write
     strategy:
       matrix:
         variant: ${{ fromJSON(needs.setup.outputs.variants) }}
     steps:
-      - name: Checkout
+      - name: Checkout the signing action
         uses: actions/checkout@v7
         with:
-          ref: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.source_ref || github.ref }}
-      
+          sparse-checkout: .github/actions
+          persist-credentials: false
+
       - name: Configure variant
         id: config
         run: |
@@ -444,21 +486,31 @@ jobs:
         run: |
           # Install crane for efficient multi-arch image copying
           cd $(mktemp -d)
-          curl -sL "https://github.com/google/go-containerregistry/releases/latest/download/go-containerregistry_Linux_x86_64.tar.gz" | tar xz
+          curl -sLO https://github.com/google/go-containerregistry/releases/download/v0.22.0/go-containerregistry_Linux_x86_64.tar.gz
+          echo "edb74d53fad9a596860f59d1c5d04a43dfb5f441dc71f57060dd0bf39483c833  go-containerregistry_Linux_x86_64.tar.gz" | sha256sum -c -
+          tar xzf go-containerregistry_Linux_x86_64.tar.gz crane
           sudo mv crane /usr/local/bin/
           crane version
       - name: Create and push manifest
+        id: manifest
+        env:
+          BASE_TAG: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.image_tag || 'latest' }}
         run: |
           SUFFIX="${{ steps.config.outputs.tag_suffix }}"
-          BASE_TAG="${{ github.event_name == 'workflow_dispatch' && github.event.inputs.image_tag || 'latest' }}"
           
           # Create manifest on GHCR first (no rate limits)
           echo "Creating GHCR manifest (no rate limits)..."
           docker buildx imagetools create -t ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX} \
+            --metadata-file /tmp/manifest.json \
             ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-amd64 \
             ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-arm64 \
             ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-arm \
-            ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-386
+            ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-386 \
+            ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-ppc64le \
+            ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-s390x
+          # The copy and the signature below use the digest this run pushed, not whatever the tag points at by then.
+          DIGEST=$(jq -er '."containerimage.descriptor".digest' /tmp/manifest.json)
+          echo "digest=${DIGEST}" >> "$GITHUB_OUTPUT"
           
           # Copy the complete multi-arch image from GHCR to Docker Hub
           # This only requires one pull from GHCR (no rate limit) and one push to Docker Hub
@@ -494,16 +546,25 @@ jobs:
           # Use crane or skopeo to copy, fallback to docker if not available
           if command -v crane &> /dev/null; then
             echo "Using crane to copy..."
-            retry_with_backoff crane copy ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX} chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}
+            retry_with_backoff crane copy ghcr.io/chrislusf/seaweedfs@${DIGEST} chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}
           elif command -v skopeo &> /dev/null; then
             echo "Using skopeo to copy..."
-            retry_with_backoff skopeo copy --all docker://ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX} docker://chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}
+            retry_with_backoff skopeo copy --all docker://ghcr.io/chrislusf/seaweedfs@${DIGEST} docker://chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}
           else
-            echo "Using docker buildx imagetools (pulling 4 images from Docker Hub)..."
+            echo "Using docker buildx imagetools (pulling 6 images from Docker Hub)..."
             # Fallback: create manifest directly on Docker Hub (pulls from Docker Hub - rate limited)
             retry_with_backoff docker buildx imagetools create -t chrislusf/seaweedfs:${BASE_TAG}${SUFFIX} \
               ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-amd64 \
               ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-arm64 \
               ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-arm \
-              ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-386
+              ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-386 \
+              ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-ppc64le \
+              ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-s390x
           fi
+
+      - name: Sign
+        uses: ./.github/actions/sign-image
+        with:
+          images: >-
+            ghcr.io/chrislusf/seaweedfs@${{ steps.manifest.outputs.digest }}
+            chrislusf/seaweedfs@${{ steps.manifest.outputs.digest }}

.github/workflows/container_release_foundationdb.yml

@@ -21,6 +21,9 @@ jobs:
 
   build-large-release-container_foundationdb:
     runs-on: [ubuntu-latest]
+    permissions:
+      contents: read
+      id-token: write
 
     steps:
       -
@@ -43,7 +46,7 @@ jobs:
             org.opencontainers.image.vendor=Chris Lu
       -
         name: Set up QEMU
-        uses: docker/setup-qemu-action@v4.2.0
+        uses: docker/setup-qemu-action@v4.4.0
       -
         name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v4
@@ -65,6 +68,7 @@ jobs:
           fi
       -
         name: Build
+        id: build
         uses: docker/build-push-action@v7
         with:
           context: ./docker
@@ -76,4 +80,10 @@ jobs:
           platforms: linux/amd64
           tags: ${{ steps.docker_meta.outputs.tags }}
           labels: ${{ steps.docker_meta.outputs.labels }}
+      -
+        name: Sign
+        if: github.event_name != 'pull_request'
+        uses: ./.github/actions/sign-image
+        with:
+          images: chrislusf/seaweedfs@${{ steps.build.outputs.digest }}
 

.github/workflows/container_release_unified.yml

@@ -42,9 +42,10 @@ concurrency:
 
 jobs:
 
-  # ── Pre-build Rust volume server binaries natively ──────────────────
-  # Cross-compiles for amd64 and arm64 without QEMU, turning a 5-hour
-  # emulated cargo build into ~15 minutes of native compilation.
+  # ── Pre-build the Rust binaries natively ────────────────────────────
+  # The volume server and the Rust maintenance worker, cross-compiled for
+  # amd64 and arm64 without QEMU, turning a 5-hour emulated cargo build into
+  # ~15 minutes of native compilation.
   build-rust-binaries:
     runs-on: ubuntu-22.04
     strategy:
@@ -59,9 +60,6 @@ jobs:
       - name: Checkout
         uses: actions/checkout@v7
 
-      - name: Install protobuf compiler
-        run: sudo apt-get update && sudo apt-get install -y protobuf-compiler
-
       - name: Install Rust toolchain
         uses: dtolnay/rust-toolchain@stable
         with:
@@ -86,10 +84,26 @@ jobs:
             ~/.cargo/registry
             ~/.cargo/git
             seaweed-volume/target
-          key: rust-docker-${{ matrix.target }}-${{ hashFiles('seaweed-volume/Cargo.lock') }}
+            seaweed-worker/target/${{ matrix.target }}/release
+          key: rust-docker-${{ matrix.target }}-${{ hashFiles('seaweed-volume/Cargo.lock', 'seaweed-worker/Cargo.lock') }}
           restore-keys: |
             rust-docker-${{ matrix.target }}-
 
+      # lance's build scripts compile their own protos and look for a protoc.
+      # Point them at the one protoc-bin-vendored ships, which seaweed-worker's
+      # own build already uses, so no job depends on a system package and every
+      # build sees the same version.
+      - name: Use the vendored protoc
+        run: |
+          cd seaweed-worker
+          cargo fetch
+          # The version from the lock, not whatever else a restored cache holds.
+          version=$(awk '/^name = "protoc-bin-vendored-linux-x86_64"$/{found=1; next} found && /^version = /{gsub(/"/,"",$3); print $3; exit}' Cargo.lock)
+          test -n "$version" || { echo "protoc-bin-vendored-linux-x86_64 is not in Cargo.lock" >&2; exit 1; }
+          protoc=$(find ~/.cargo/registry/src -path "*protoc-bin-vendored-linux-x86_64-$version/bin/protoc" | head -1)
+          test -x "$protoc" || { echo "no vendored protoc $version in the registry" >&2; exit 1; }
+          echo "PROTOC=$protoc" >> "$GITHUB_ENV"
+
       - name: Build large-disk variant
         env:
           SEAWEEDFS_COMMIT: ${{ github.sha }}
@@ -106,13 +120,20 @@ jobs:
           cargo build --release --target ${{ matrix.target }} --no-default-features
           cp target/${{ matrix.target }}/release/weed-volume ../weed-volume-normal-${{ matrix.arch }}
 
+      - name: Build the Rust maintenance worker
+        run: |
+          cd seaweed-worker
+          cargo build --release -p weed-lance-worker --target ${{ matrix.target }}
+          cp target/${{ matrix.target }}/release/weed-worker ../weed-worker-${{ matrix.arch }}
+
       - name: Upload artifacts
         uses: actions/upload-artifact@v7
         with:
-          name: rust-volume-${{ matrix.arch }}
+          name: rust-bins-${{ matrix.arch }}
           path: |
             weed-volume-large-disk-${{ matrix.arch }}
             weed-volume-normal-${{ matrix.arch }}
+            weed-worker-${{ matrix.arch }}
 
   # One job per (variant, platform) on a native runner, pushed by digest;
   # the merge job stitches the digests into one multi-arch tag.
@@ -128,12 +149,16 @@ jobs:
           - { variant: normal, tag_suffix: "",             dockerfile: ./docker/Dockerfile.go_build, build_args: "",               rust_variant: normal,     platform: linux/arm64,  arch: arm64, runner: ubuntu-24.04-arm, qemu: false }
           - { variant: normal, tag_suffix: "",             dockerfile: ./docker/Dockerfile.go_build, build_args: "",               rust_variant: normal,     platform: linux/arm/v7, arch: armv7, runner: ubuntu-latest,    qemu: true }
           - { variant: normal, tag_suffix: "",             dockerfile: ./docker/Dockerfile.go_build, build_args: "",               rust_variant: normal,     platform: linux/386,    arch: i386,  runner: ubuntu-latest,    qemu: false }
+          - { variant: normal, tag_suffix: "",             dockerfile: ./docker/Dockerfile.go_build, build_args: "",               rust_variant: normal,     platform: linux/ppc64le, arch: ppc64le, runner: ubuntu-latest,  qemu: true }
+          - { variant: normal, tag_suffix: "",             dockerfile: ./docker/Dockerfile.go_build, build_args: "",               rust_variant: normal,     platform: linux/s390x,   arch: s390x, runner: ubuntu-latest,    qemu: true }
 
           # Large disk - multi-arch
           - { variant: large_disk, tag_suffix: _large_disk, dockerfile: ./docker/Dockerfile.go_build, build_args: TAGS=5BytesOffset, rust_variant: large-disk, platform: linux/amd64,  arch: amd64, runner: ubuntu-latest,    qemu: false }
           - { variant: large_disk, tag_suffix: _large_disk, dockerfile: ./docker/Dockerfile.go_build, build_args: TAGS=5BytesOffset, rust_variant: large-disk, platform: linux/arm64,  arch: arm64, runner: ubuntu-24.04-arm, qemu: false }
           - { variant: large_disk, tag_suffix: _large_disk, dockerfile: ./docker/Dockerfile.go_build, build_args: TAGS=5BytesOffset, rust_variant: large-disk, platform: linux/arm/v7, arch: armv7, runner: ubuntu-latest,    qemu: true }
           - { variant: large_disk, tag_suffix: _large_disk, dockerfile: ./docker/Dockerfile.go_build, build_args: TAGS=5BytesOffset, rust_variant: large-disk, platform: linux/386,    arch: i386,  runner: ubuntu-latest,    qemu: false }
+          - { variant: large_disk, tag_suffix: _large_disk, dockerfile: ./docker/Dockerfile.go_build, build_args: TAGS=5BytesOffset, rust_variant: large-disk, platform: linux/ppc64le, arch: ppc64le, runner: ubuntu-latest,  qemu: true }
+          - { variant: large_disk, tag_suffix: _large_disk, dockerfile: ./docker/Dockerfile.go_build, build_args: TAGS=5BytesOffset, rust_variant: large-disk, platform: linux/s390x,   arch: s390x, runner: ubuntu-latest,    qemu: true }
 
           # Full tags - multi-arch
           - { variant: full, tag_suffix: _full, dockerfile: ./docker/Dockerfile.go_build, build_args: "TAGS=elastic,gocdk,rclone,sqlite,tarantool,tikv,ydb", rust_variant: normal, platform: linux/amd64, arch: amd64, runner: ubuntu-latest,    qemu: false }
@@ -158,7 +183,7 @@ jobs:
         if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
         uses: actions/download-artifact@v8
         with:
-          pattern: rust-volume-*
+          pattern: rust-bins-*
           merge-multiple: true
           path: ./rust-bins
 
@@ -173,7 +198,16 @@ jobs:
               echo "Placed pre-built Rust binary for ${arch}"
             fi
           done
+          mkdir -p docker/weed-worker-prebuilt
+          for arch in amd64 arm64; do
+            src="./rust-bins/weed-worker-${arch}"
+            if [ -f "$src" ]; then
+              cp "$src" "docker/weed-worker-prebuilt/weed-worker-${arch}"
+              echo "Placed pre-built Rust worker for ${arch}"
+            fi
+          done
           ls -la docker/weed-volume-prebuilt/
+          ls -la docker/weed-worker-prebuilt/
 
       - name: Free Disk Space
         if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
@@ -201,7 +235,7 @@ jobs:
 
       - name: Set up QEMU
         if: (github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant) && matrix.qemu
-        uses: docker/setup-qemu-action@v4.2.0
+        uses: docker/setup-qemu-action@v4.4.0
 
       - name: Create BuildKit config
         if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
@@ -265,10 +299,14 @@ jobs:
           if-no-files-found: error
           retention-days: 1
 
-  # Assemble each variant's per-platform digests into one tag, then mirror to Docker Hub.
+  # Assemble each variant's per-platform digests into one tag, mirror it to
+  # Docker Hub, and sign the result on both registries.
   merge:
     needs: [build]
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      id-token: write
     strategy:
       fail-fast: false
       matrix:
@@ -279,6 +317,13 @@ jobs:
           - { variant: large_disk_full, tag_suffix: _large_disk_full }
           - { variant: rocksdb,         tag_suffix: _large_disk_rocksdb }
     steps:
+      - name: Checkout the signing action
+        if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
+        uses: actions/checkout@v7
+        with:
+          sparse-checkout: .github/actions
+          persist-credentials: false
+
       - name: Download digests
         if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
         uses: actions/download-artifact@v8
@@ -301,12 +346,17 @@ jobs:
 
       - name: Create multi-arch tag on GHCR
         if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
+        id: manifest
         working-directory: /tmp/digests
         run: |
           docker buildx imagetools create \
             -t ${{ env.IMAGE }}:${{ env.RELEASE_TAG }}${{ matrix.tag_suffix }} \
+            --metadata-file /tmp/manifest.json \
             $(printf '${{ env.IMAGE }}@sha256:%s ' *)
           docker buildx imagetools inspect ${{ env.IMAGE }}:${{ env.RELEASE_TAG }}${{ matrix.tag_suffix }}
+          # The copy and the signature below use the digest this run pushed, not whatever the tag points at by then.
+          digest=$(jq -er '."containerimage.descriptor".digest' /tmp/manifest.json)
+          echo "digest=${digest}" >> "$GITHUB_OUTPUT"
 
       - name: Login to Docker Hub
         if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
@@ -319,7 +369,9 @@ jobs:
         if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
         run: |
           cd $(mktemp -d)
-          curl -sL "https://github.com/google/go-containerregistry/releases/latest/download/go-containerregistry_Linux_x86_64.tar.gz" | tar xz
+          curl -sLO https://github.com/google/go-containerregistry/releases/download/v0.22.0/go-containerregistry_Linux_x86_64.tar.gz
+          echo "edb74d53fad9a596860f59d1c5d04a43dfb5f441dc71f57060dd0bf39483c833  go-containerregistry_Linux_x86_64.tar.gz" | sha256sum -c -
+          tar xzf go-containerregistry_Linux_x86_64.tar.gz crane
           sudo mv crane /usr/local/bin/
           crane version
 
@@ -350,10 +402,18 @@ jobs:
 
           echo "Copying ${{ matrix.variant }} from GHCR to Docker Hub..."
           retry_with_backoff crane copy \
-            ${{ env.IMAGE }}:${{ env.RELEASE_TAG }}${{ matrix.tag_suffix }} \
+            ${{ env.IMAGE }}@${{ steps.manifest.outputs.digest }} \
             chrislusf/seaweedfs:${{ env.RELEASE_TAG }}${{ matrix.tag_suffix }}
           echo "Copied ${{ matrix.variant }} to Docker Hub"
 
+      - name: Sign ${{ matrix.variant }}
+        if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
+        uses: ./.github/actions/sign-image
+        with:
+          images: >-
+            ${{ env.IMAGE }}@${{ steps.manifest.outputs.digest }}
+            chrislusf/seaweedfs@${{ steps.manifest.outputs.digest }}
+
   # Report-only trivy scan: uploads fixable HIGH/CRITICAL findings to GitHub
   # Security for visibility, but never blocks the release. Releases (including
   # `latest`) ship regardless — vulnerabilities are tracked, not gated, since
@@ -400,7 +460,7 @@ jobs:
 
       - name: Upload Trivy scan results to GitHub Security
         if: always()
-        uses: github/codeql-action/upload-sarif@v4.37.4
+        uses: github/codeql-action/upload-sarif@v4.38.2
         with:
           sarif_file: trivy-results.sarif
           category: trivy-${{ matrix.variant }}
@@ -409,7 +469,8 @@ jobs:
   # image. crane tag adds an extra tag to an existing manifest — no rebuild,
   # no QEMU, no separate workflow. Replaces the old container_latest.yml
   # rebuild that often failed or lagged behind the release. Independent of
-  # trivy-scan: vuln findings are reported but do not block `latest`.
+  # trivy-scan: vuln findings are reported but do not block `latest`. The
+  # cosign signature is attached to the digest, so `latest` carries it too.
   tag-latest:
     runs-on: ubuntu-latest
     needs: [merge]
@@ -438,7 +499,9 @@ jobs:
       - name: Install crane
         run: |
           cd $(mktemp -d)
-          curl -sL "https://github.com/google/go-containerregistry/releases/latest/download/go-containerregistry_Linux_x86_64.tar.gz" | tar xz
+          curl -sLO https://github.com/google/go-containerregistry/releases/download/v0.22.0/go-containerregistry_Linux_x86_64.tar.gz
+          echo "edb74d53fad9a596860f59d1c5d04a43dfb5f441dc71f57060dd0bf39483c833  go-containerregistry_Linux_x86_64.tar.gz" | sha256sum -c -
+          tar xzf go-containerregistry_Linux_x86_64.tar.gz crane
           sudo mv crane /usr/local/bin/
           crane version
 

.github/workflows/container_rocksdb_version.yml

@@ -22,6 +22,9 @@ permissions:
 jobs:
   build-rocksdb-image:
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      id-token: write
 
     steps:
       - name: Checkout
@@ -82,7 +85,7 @@ jobs:
           echo "seaweedfs_ref=$seaweed" >> "$GITHUB_OUTPUT"
 
       - name: Set up QEMU
-        uses: docker/setup-qemu-action@96fe6ef7f33517b61c61be40b68a1882f3264fb8 # v1
+        uses: docker/setup-qemu-action@99012661954931238ded8c8b007157a8430204e1 # v1
 
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v1
@@ -94,6 +97,7 @@ jobs:
           password: ${{ secrets.DOCKER_PASSWORD }}
 
       - name: Build and push image
+        id: build
         uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v2
         with:
           context: ./docker
@@ -108,3 +112,8 @@ jobs:
             org.opencontainers.image.title=seaweedfs
             org.opencontainers.image.description=SeaweedFS is a distributed storage system for blobs, objects, files, and data lake, to store and serve billions of files fast!
             org.opencontainers.image.vendor=Chris Lu
+
+      - name: Sign
+        uses: ./.github/actions/sign-image
+        with:
+          images: chrislusf/seaweedfs@${{ steps.build.outputs.digest }}

.github/workflows/deploy_telemetry.yml

@@ -35,11 +35,48 @@ jobs:
           cd telemetry/server
           go mod tidy
           echo "Building telemetry server..."
-          GOOS=linux GOARCH=amd64 go build -o ../../telemetry-server .
+          CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o ../../telemetry-server .
           cd ../..
           ls -la telemetry-server
           echo "Build completed successfully"
 
+      - name: Generate Service Configuration
+        if: github.event_name == 'workflow_dispatch' && (inputs.setup || inputs.deploy)
+        env:
+          REMOTE_USER: ${{ secrets.TELEMETRY_USER }}
+        run: |
+          # Create systemd service file
+          echo "
+          [Unit]
+          Description=SeaweedFS Telemetry Server
+          After=network.target
+
+          [Service]
+          Type=simple
+          User=$REMOTE_USER
+          WorkingDirectory=/home/$REMOTE_USER/seaweedfs-telemetry
+          ExecStart=/bin/sh -c 'exec /home/$REMOTE_USER/seaweedfs-telemetry/bin/telemetry-server -port=8353 >>/home/$REMOTE_USER/seaweedfs-telemetry/logs/telemetry.log 2>>/home/$REMOTE_USER/seaweedfs-telemetry/logs/telemetry.error.log'
+          Restart=always
+          RestartSec=5
+
+          [Install]
+          WantedBy=multi-user.target" > telemetry.service
+
+          # Setup logrotate configuration
+          echo "# SeaweedFS Telemetry service log rotation
+          /home/$REMOTE_USER/seaweedfs-telemetry/logs/*.log {
+              daily
+              rotate 30
+              compress
+              delaycompress
+              missingok
+              notifempty
+              create 644 $REMOTE_USER $REMOTE_USER
+              postrotate
+                  systemctl restart telemetry.service
+              endscript
+          }" > telemetry_logrotate
+
       - name: First-time Server Setup
         if: github.event_name == 'workflow_dispatch' && inputs.setup
         env:
@@ -61,40 +98,6 @@ jobs:
             touch ~/seaweedfs-telemetry/logs/telemetry.log ~/seaweedfs-telemetry/logs/telemetry.error.log && \
             chmod 644 ~/seaweedfs-telemetry/logs/*.log"
 
-          # Create systemd service file
-          echo "
-          [Unit]
-          Description=SeaweedFS Telemetry Server
-          After=network.target
-
-          [Service]
-          Type=simple
-          User=$REMOTE_USER
-          WorkingDirectory=/home/$REMOTE_USER/seaweedfs-telemetry
-          ExecStart=/home/$REMOTE_USER/seaweedfs-telemetry/bin/telemetry-server -port=8353
-          Restart=always
-          RestartSec=5
-          StandardOutput=append:/home/$REMOTE_USER/seaweedfs-telemetry/logs/telemetry.log
-          StandardError=append:/home/$REMOTE_USER/seaweedfs-telemetry/logs/telemetry.error.log
-
-          [Install]
-          WantedBy=multi-user.target" > telemetry.service
-
-          # Setup logrotate configuration
-          echo "# SeaweedFS Telemetry service log rotation
-          /home/$REMOTE_USER/seaweedfs-telemetry/logs/*.log {
-              daily
-              rotate 30
-              compress
-              delaycompress
-              missingok
-              notifempty
-              create 644 $REMOTE_USER $REMOTE_USER
-              postrotate
-                  systemctl restart telemetry.service
-              endscript
-          }" > telemetry_logrotate
-
           # Copy configuration files
           scp -i ~/.ssh/deploy_key telemetry/grafana-dashboard.json $REMOTE_USER@$REMOTE_HOST:~/seaweedfs-telemetry/
           scp -i ~/.ssh/deploy_key telemetry/prometheus.yml $REMOTE_USER@$REMOTE_HOST:~/seaweedfs-telemetry/
@@ -137,11 +140,18 @@ jobs:
           scp -i ~/.ssh/deploy_key telemetry/grafana-dashboard.json $REMOTE_USER@$REMOTE_HOST:~/seaweedfs-telemetry/
           scp -i ~/.ssh/deploy_key telemetry/prometheus.yml $REMOTE_USER@$REMOTE_HOST:~/seaweedfs-telemetry/
 
+          # Copy updated service and logrotate files
+          scp -i ~/.ssh/deploy_key telemetry.service telemetry_logrotate $REMOTE_USER@$REMOTE_HOST:~/seaweedfs-telemetry/
+
           # Check if service exists and deploy accordingly
           ssh -i ~/.ssh/deploy_key $REMOTE_USER@$REMOTE_HOST "
             if systemctl list-unit-files telemetry.service >/dev/null 2>&1; then
               echo 'Service exists, performing update...'
+              set -e
               sudo systemctl stop telemetry.service
+              sudo mv ~/seaweedfs-telemetry/telemetry.service /etc/systemd/system/
+              sudo mv ~/seaweedfs-telemetry/telemetry_logrotate /etc/logrotate.d/seaweedfs-telemetry
+              sudo systemctl daemon-reload
               mkdir -p ~/seaweedfs-telemetry/bin
               mv ~/seaweedfs-telemetry/tmp/telemetry-server ~/seaweedfs-telemetry/bin/
               chmod +x ~/seaweedfs-telemetry/bin/telemetry-server

.github/workflows/e2e.yml

@@ -61,13 +61,10 @@ jobs:
 
     - name: Install dependencies
       run: |
-        # Use faster mirrors and install with timeout
         sudo rm -f /etc/apt/sources.list.d/azure-cli.list /etc/apt/sources.list.d/microsoft-prod.list
-        echo "deb http://azure.archive.ubuntu.com/ubuntu/ $(lsb_release -cs) main restricted universe multiverse" | sudo tee /etc/apt/sources.list
-        echo "deb http://azure.archive.ubuntu.com/ubuntu/ $(lsb_release -cs)-updates main restricted universe multiverse" | sudo tee -a /etc/apt/sources.list
-        
-        sudo apt-get update --fix-missing
-        sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends fuse
+        # Same helper the e2e image installs through: the runner's own list is
+        # azure-only too, and an outage there fails this step outright.
+        sudo ./apt-install fuse
         
         # Verify FUSE installation
         echo "FUSE version: $(fusermount --version 2>&1 || echo 'fusermount not found')"

.github/workflows/ec-integration-tests.yml

@@ -30,7 +30,7 @@ jobs:
     - name: Set up Go 1.x
       uses: actions/setup-go@v7
       with:
-        go-version: ^1.25
+        go-version: ^1.26
       id: go
 
     - name: Check out code into the Go module directory
@@ -43,7 +43,10 @@ jobs:
     - name: Run EC Integration Tests
       working-directory: test/erasure_coding
       run: |
-        go test -v
+        # The suite now includes the interruption matrix and runs close to Go's
+        # default 10m binary timeout on slower runners; bound it by the job's
+        # 30m budget instead.
+        go test -v -timeout 25m
 
     - name: Collect server logs on failure
       if: failure()

.github/workflows/fuse-dlm-integration.yml

@@ -40,10 +40,11 @@ jobs:
         with:
           go-version-file: 'go.mod'
 
-      - name: Install FUSE dependencies
+      - name: Configure FUSE
         run: |
-          sudo apt-get update
-          sudo apt-get install -y libfuse3-dev
+          # Nothing to install: fuse3 ships fusermount3 and is pre-installed,
+          # and go-fuse is pure Go, so the libfuse headers were never linked
+          # against.
           echo 'user_allow_other' | sudo tee -a /etc/fuse.conf
           sudo chmod 644 /etc/fuse.conf
 

.github/workflows/fuse-failover.yml

@@ -0,0 +1,76 @@
+name: "FUSE Volume Server Failover Tests"
+
+on:
+  pull_request:
+    paths:
+      - 'weed/command/mount*.go'
+      - 'weed/mount/**'
+      - 'weed/filer/**'
+      - 'weed/wdclient/**'
+      - 'weed/operation/upload_content.go'
+      - 'test/fuse_failover/**'
+      - '.github/workflows/fuse-failover.yml'
+      - '.github/actions/fix-fusermount-setuid/**'
+  push:
+    branches: [master]
+    paths:
+      - 'weed/command/mount*.go'
+      - 'weed/mount/**'
+      - 'weed/filer/**'
+      - 'weed/wdclient/**'
+      - 'weed/operation/upload_content.go'
+      - 'test/fuse_failover/**'
+      - '.github/workflows/fuse-failover.yml'
+      - '.github/actions/fix-fusermount-setuid/**'
+
+concurrency:
+  group: ${{ github.head_ref || github.ref }}/fuse-failover
+  cancel-in-progress: true
+
+permissions:
+  contents: read
+
+jobs:
+  fuse-failover:
+    name: FUSE Volume Server Failover
+    runs-on: ubuntu-22.04
+    timeout-minutes: 40
+
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v7
+        with:
+          persist-credentials: false
+
+      - name: Set up Go
+        uses: actions/setup-go@v7
+        with:
+          go-version-file: 'go.mod'
+
+      - name: Configure FUSE
+        run: |
+          # Nothing to install: fuse3 ships fusermount3 and is pre-installed,
+          # and go-fuse is pure Go, so the libfuse headers were never linked
+          # against.
+          echo 'user_allow_other' | sudo tee -a /etc/fuse.conf
+          sudo chmod 644 /etc/fuse.conf
+
+      - name: Repair the fusermount3 setuid bit
+        uses: ./.github/actions/fix-fusermount-setuid
+
+      - name: Build SeaweedFS
+        run: go build -o weed/weed -buildvcs=false ./weed
+
+      - name: Run failover integration tests
+        timeout-minutes: 35
+        env:
+          WEED_BINARY: ${{ github.workspace }}/weed/weed
+        run: go test -v -count=1 -timeout=30m ./test/fuse_failover/...
+
+      - name: Upload logs on failure
+        if: failure()
+        uses: actions/upload-artifact@v7
+        with:
+          name: fuse-failover-test-logs
+          path: /tmp/seaweedfs-fuse-failover-logs/
+          retention-days: 3

.github/workflows/fuse-integration.yml

@@ -38,12 +38,10 @@ jobs:
       with:
         go-version-file: 'go.mod'
 
-    - name: Install FUSE and dependencies
+    - name: Configure FUSE
       run: |
-        sudo apt-get update
-        # fuse3 is pre-installed on ubuntu-22.04 runners and conflicts
-        # with the legacy fuse package, so only install the dev headers.
-        sudo apt-get install -y libfuse3-dev
+        # Nothing to install: fuse3 ships fusermount3 and is pre-installed, and
+        # go-fuse is pure Go, so the libfuse headers were never linked against.
         # Allow non-root FUSE mounts with allow_other
         echo 'user_allow_other' | sudo tee -a /etc/fuse.conf
         sudo chmod 644 /etc/fuse.conf

.github/workflows/fuse-p2p-integration.yml

@@ -46,10 +46,11 @@ jobs:
         with:
           go-version-file: 'go.mod'
 
-      - name: Install FUSE dependencies
+      - name: Configure FUSE
         run: |
-          sudo apt-get update
-          sudo apt-get install -y libfuse3-dev
+          # Nothing to install: fuse3 ships fusermount3 and is pre-installed,
+          # and go-fuse is pure Go, so the libfuse headers were never linked
+          # against.
           echo 'user_allow_other' | sudo tee -a /etc/fuse.conf
           sudo chmod 644 /etc/fuse.conf
 

.github/workflows/go.yml

@@ -111,6 +111,16 @@ jobs:
   test:
     name: Test
     runs-on: ubuntu-latest
+    services:
+      redis:
+        image: redis:8
+        ports:
+          - 6379:6379
+        options: >-
+          --health-cmd "redis-cli ping"
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 5
     steps:
     - name: Check out code into the Go module directory
       uses: actions/checkout@v7
@@ -119,6 +129,8 @@ jobs:
       with:
         go-version-file: 'go.mod'
     - name: Test
+      env:
+        RUN_REDIS_TESTS: "1"
       run: cd weed; go test -tags "elastic gocdk sqlite ydb tarantool tikv rclone" -v ./...
 
   test-32bit:

.github/workflows/helm_ci.yml

@@ -3,10 +3,10 @@ name: "helm: lint and test charts"
 on:
   push:
     branches: [ master ]
-    paths: ['k8s/**']
+    paths: ['k8s/**', '.github/workflows/helm_ci.yml']
   pull_request:
     branches: [ master ]
-    paths: ['k8s/**']
+    paths: ['k8s/**', '.github/workflows/helm_ci.yml']
 
 permissions:
   contents: read
@@ -58,16 +58,78 @@ jobs:
           grep -q "kind: Deployment" /tmp/s3.yaml && grep -q "seaweedfs-s3" /tmp/s3.yaml
           echo "S3 deployment renders correctly"
           
+          echo "=== Testing S3 credentials from an existing secret ==="
+          credential_args=(
+            --set s3.credentials.admin.existingSecret=minio-root
+            --set s3.credentials.admin.accessKeyKey=root-user
+            --set s3.credentials.admin.secretKeyKey=root-password
+            --set s3.credentials.read.existingSecret=minio-root
+          )
+          for workload in \
+            "s3.enabled=true,s3.enableAuth=true" \
+            "filer.s3.enabled=true,filer.s3.enableAuth=true" \
+            "allInOne.enabled=true,allInOne.s3.enabled=true,allInOne.s3.enableAuth=true"
+          do
+            helm template test $CHART_DIR --set "$workload" "${credential_args[@]}" > /tmp/s3-existing-credentials.yaml
+            # The identities file names the variables, and each is bound to the key it was pointed at.
+            grep -q 'accessKey":"${SEAWEEDFS_S3_ADMIN_ACCESS_KEY_ID}' /tmp/s3-existing-credentials.yaml
+            grep -q 'secretKey":"${SEAWEEDFS_S3_ADMIN_SECRET_ACCESS_KEY}' /tmp/s3-existing-credentials.yaml
+            grep -q 'accessKey":"${SEAWEEDFS_S3_READ_ACCESS_KEY_ID}' /tmp/s3-existing-credentials.yaml
+            for pair in \
+              "SEAWEEDFS_S3_ADMIN_ACCESS_KEY_ID root-user" \
+              "SEAWEEDFS_S3_ADMIN_SECRET_ACCESS_KEY root-password" \
+              "SEAWEEDFS_S3_READ_ACCESS_KEY_ID read_access_key_id" \
+              "SEAWEEDFS_S3_READ_SECRET_ACCESS_KEY read_secret_access_key"
+            do
+              set -- $pair
+              grep -A 4 -- "- name: $1\$" /tmp/s3-existing-credentials.yaml | grep -q "name: \"minio-root\""
+              grep -A 4 -- "- name: $1\$" /tmp/s3-existing-credentials.yaml | grep -q "key: \"$2\""
+            done
+            # The keys stay in the user's secret rather than being copied into the chart's.
+            ! grep -qE "^  (admin|read)_(access_key_id|secret_access_key):" /tmp/s3-existing-credentials.yaml
+            echo "S3 credentials reference the existing secret for $workload"
+          done
+
+          echo "=== Testing ingress labels ==="
+          helm template test $CHART_DIR --set s3.enabled=true,s3.ingress.enabled=true \
+            --set s3.ingress.labels.external-dns=s3 > /tmp/s3-ingress-labels.yaml
+          grep -A 12 "^kind: Ingress" /tmp/s3-ingress-labels.yaml | grep -q "^    external-dns: s3"
+          echo "S3 ingress renders custom labels"
+
           echo "=== Testing with all-in-one mode ==="
           helm template test $CHART_DIR --set allInOne.enabled=true > /tmp/allinone.yaml
           grep -q "seaweedfs-all-in-one" /tmp/allinone.yaml
           echo "All-in-one deployment renders correctly"
+
+          echo "=== Testing the all-in-one s3 secret is created by every flag that mounts it ==="
+          for auth in allInOne.s3.enableAuth s3.enableAuth filer.s3.enableAuth; do
+            helm template test $CHART_DIR \
+              --set allInOne.enabled=true --set allInOne.s3.enabled=true --set "$auth=true" \
+              > /tmp/allinone-s3-auth.yaml
+            grep -q "secretName: test-seaweedfs-s3-secret" /tmp/allinone-s3-auth.yaml
+            grep -q "name: test-seaweedfs-s3-secret" /tmp/allinone-s3-auth.yaml
+            echo "All-in-one s3 secret renders for $auth"
+          done
           
           echo "=== Testing with security enabled ==="
           helm template test $CHART_DIR --set global.seaweedfs.enableSecurity=true > /tmp/security.yaml
           grep -q "security-config" /tmp/security.yaml
           echo "Security configuration renders correctly"
 
+          echo ""
+          echo "=== Testing admin.allowInsecureBind satisfies the admin auth render guard ==="
+          helm template test $CHART_DIR --set admin.enabled=true --set admin.allowInsecureBind=true \
+            > /tmp/admin-allow-insecure-bind.yaml
+          grep -q -- "-allowInsecureBind" /tmp/admin-allow-insecure-bind.yaml
+          echo "admin.allowInsecureBind renders -allowInsecureBind and passes the render guard"
+
+          if helm template test $CHART_DIR --set admin.enabled=true > /tmp/admin-no-auth.yaml 2>/tmp/admin-no-auth.err; then
+            echo "FAIL: admin.enabled=true with no auth configured should fail to render"
+            exit 1
+          fi
+          grep -q "admin.allowInsecureBind" /tmp/admin-no-auth.err
+          echo "admin with no auth configured still fails the render guard, and the guard mentions admin.allowInsecureBind"
+
           echo ""
           echo "=== Testing JWT expiration overrides ==="
           helm template test $CHART_DIR \
@@ -179,6 +241,9 @@ jobs:
           out = render({
               "global.seaweedfs.securityConfig.jwtSigning.filerWrite": "true",
               "admin.enabled": "true",
+              # admin.ip defaults to 0.0.0.0 (non-loopback), which weed admin 4.46
+              # refuses to bind without authentication.
+              "admin.secret.adminPassword": "ci-admin-password",
           })
           cm = configmap(out, "test-seaweedfs-security-config")
           if cm is None:
@@ -440,6 +505,41 @@ jobs:
             --set filer.s3.enableAuth=true > /tmp/filer-s3.yaml
           echo "Filer S3 gateway renders correctly"
           
+          echo "=== Testing the mysql filer store gates its secret and env ==="
+          helm template test $CHART_DIR \
+            --set-string filer.extraEnvironmentVars.WEED_MONGODB_ENABLED=true \
+            --set-string filer.extraEnvironmentVars.WEED_LEVELDB2_ENABLED=false > /tmp/filer-mongodb.yaml
+          ! grep -q "db-secret" /tmp/filer-mongodb.yaml
+          ! grep -q "WEED_MYSQL" /tmp/filer-mongodb.yaml
+          grep -q "name: WEED_MONGODB_ENABLED" /tmp/filer-mongodb.yaml
+          helm template test $CHART_DIR \
+            --set-string filer.extraEnvironmentVars.WEED_MYSQL_ENABLED=true > /tmp/filer-mysql.yaml
+          grep -q "name: test-seaweedfs-db-secret" /tmp/filer-mysql.yaml
+          grep -q "name: WEED_MYSQL_USERNAME" /tmp/filer-mysql.yaml
+          grep -q "name: WEED_MYSQL_PASSWORD" /tmp/filer-mysql.yaml
+          grep -q "name: WEED_MYSQL_HOSTNAME" /tmp/filer-mysql.yaml
+          # Secret-backed keys follow the same rule as the plain ones.
+          helm template test $CHART_DIR \
+            --set filer.secretExtraEnvironmentVars.WEED_MYSQL_PASSWORD.secretKeyRef.name=db \
+            --set filer.secretExtraEnvironmentVars.WEED_MYSQL_PASSWORD.secretKeyRef.key=password > /tmp/filer-mysql-off-secret.yaml
+          ! grep -q "WEED_MYSQL" /tmp/filer-mysql-off-secret.yaml
+          helm template test $CHART_DIR \
+            --set-string filer.extraEnvironmentVars.WEED_MYSQL_ENABLED=true \
+            --set filer.secretExtraEnvironmentVars.WEED_MYSQL_PASSWORD.secretKeyRef.name=db \
+            --set filer.secretExtraEnvironmentVars.WEED_MYSQL_PASSWORD.secretKeyRef.key=password > /tmp/filer-mysql-on-secret.yaml
+          grep -A 4 -- "- name: WEED_MYSQL_PASSWORD$" /tmp/filer-mysql-on-secret.yaml | grep -q "name: db"
+          # A flag the chart cannot read counts as selected, not as off.
+          helm template test $CHART_DIR \
+            --set filer.secretExtraEnvironmentVars.WEED_MYSQL_ENABLED.secretKeyRef.name=store \
+            --set filer.secretExtraEnvironmentVars.WEED_MYSQL_ENABLED.secretKeyRef.key=enabled > /tmp/filer-mysql-secret.yaml
+          grep -q "name: test-seaweedfs-db-secret" /tmp/filer-mysql-secret.yaml
+          grep -q "name: WEED_MYSQL_HOSTNAME" /tmp/filer-mysql-secret.yaml
+          helm template test $CHART_DIR \
+            --set-string filer.extraEnvironmentVars.WEED_MYSQL2_HOSTNAME=other > /tmp/filer-mysql2.yaml
+          grep -q "name: WEED_MYSQL2_HOSTNAME" /tmp/filer-mysql2.yaml
+          ! grep -q "WEED_MYSQL_" /tmp/filer-mysql2.yaml
+          echo "The mysql secret and env follow the selected filer store"
+
           echo "=== Testing SFTP enabled ==="
           helm template test $CHART_DIR --set sftp.enabled=true > /tmp/sftp.yaml
           grep -q "seaweedfs-sftp" /tmp/sftp.yaml
@@ -618,6 +718,141 @@ jobs:
           helm template test $CHART_DIR --set cosi.enabled=true > /tmp/cosi.yaml
           grep -q "seaweedfs-cosi" /tmp/cosi.yaml
           echo "COSI driver renders correctly"
+
+          echo ""
+          echo "=== Testing configurable pod and container security contexts ==="
+          helm template test $CHART_DIR > /tmp/security-context-defaults.yaml
+
+          security_context_args=()
+          for component in master volume filer s3 sftp admin worker allInOne cosi; do
+            security_context_args+=(
+              --set "$component.podSecurityContext.enabled=true"
+              --set "$component.podSecurityContext.seccompProfile.type=RuntimeDefault"
+              --set "$component.containerSecurityContext.enabled=true"
+              --set "$component.containerSecurityContext.privileged=false"
+              --set "$component.containerSecurityContext.allowPrivilegeEscalation=false"
+              --set "$component.containerSecurityContext.capabilities.drop[0]=ALL"
+              --set "$component.containerSecurityContext.seccompProfile.type=RuntimeDefault"
+            )
+          done
+
+          helm template test $CHART_DIR \
+            "${security_context_args[@]}" \
+            --set s3.enabled=true \
+            --set s3.createBuckets[0].name=test \
+            --set sftp.enabled=true \
+            --set admin.enabled=true \
+            --set admin.secret.adminPassword=ci-admin-password \
+            --set worker.enabled=true \
+            --set volume.idx.type=hostPath \
+            --set volume.idx.hostPathPrefix=/tmp \
+            --set cosi.enabled=true > /tmp/security-contexts.yaml
+          helm template test $CHART_DIR \
+            "${security_context_args[@]}" \
+            --set allInOne.enabled=true > /tmp/security-contexts-aio.yaml
+          python3 - /tmp/security-context-defaults.yaml /tmp/security-contexts.yaml /tmp/security-contexts-aio.yaml <<'PYEOF'
+          import sys
+
+          import yaml
+
+          errors = []
+          workloads = 0
+          containers = 0
+          chart_managed_init_containers = 0
+          components = set()
+
+          def validate_container(workload_name, container):
+              context = container.get("securityContext", {})
+              prefix = f"{workload_name}/{container['name']}"
+              if "enabled" in context:
+                  errors.append(f"{prefix}: internal enabled flag leaked into container securityContext")
+              if context.get("privileged") is not False:
+                  errors.append(f"{prefix}: privileged is not false")
+              if context.get("allowPrivilegeEscalation") is not False:
+                  errors.append(f"{prefix}: allowPrivilegeEscalation is not false")
+              if context.get("capabilities", {}).get("drop") != ["ALL"]:
+                  errors.append(f"{prefix}: capabilities.drop is not exactly [ALL]")
+              if context.get("seccompProfile", {}).get("type") != "RuntimeDefault":
+                  errors.append(f"{prefix}: seccompProfile is not RuntimeDefault")
+
+          with open(sys.argv[1]) as stream:
+              default_documents = [document for document in yaml.safe_load_all(stream) if document]
+          for document in default_documents:
+              if document.get("kind") not in ("Deployment", "StatefulSet", "Job"):
+                  continue
+              name = document["metadata"]["name"]
+              pod = document["spec"]["template"]["spec"]
+              if "securityContext" in pod:
+                  errors.append(f"{name}: pod securityContext should be absent by default")
+              for container in pod.get("containers", []):
+                  if "securityContext" in container:
+                      errors.append(
+                          f"{name}/{container['name']}: container securityContext "
+                          f"should be absent by default"
+                      )
+
+          for path in sys.argv[2:]:
+              with open(path) as stream:
+                  documents = [document for document in yaml.safe_load_all(stream) if document]
+              for document in documents:
+                  if document.get("kind") not in ("Deployment", "StatefulSet", "Job"):
+                      continue
+                  workloads += 1
+                  name = document["metadata"]["name"]
+                  pod = document["spec"]["template"]["spec"]
+                  component = document["spec"]["template"]["metadata"]["labels"].get("app.kubernetes.io/component")
+                  if component:
+                      components.add(component)
+                  else:
+                      errors.append(f"{name}: app.kubernetes.io/component label is missing")
+                  pod_context = pod.get("securityContext", {})
+                  if "enabled" in pod_context:
+                      errors.append(f"{name}: internal enabled flag leaked into pod securityContext")
+                  if pod_context.get("seccompProfile", {}).get("type") != "RuntimeDefault":
+                      errors.append(f"{name}: pod seccompProfile is not RuntimeDefault")
+                  for container in pod.get("containers", []):
+                      containers += 1
+                      validate_container(name, container)
+                  for container in pod.get("initContainers", []):
+                      if container["name"] != "seaweedfs-vol-move-idx":
+                          continue
+                      chart_managed_init_containers += 1
+                      validate_container(name, container)
+
+          expected_components = {
+              "master", "volume", "filer", "s3", "sftp", "admin", "worker",
+              "objectstorage-provisioner", "bucket-hook", "seaweedfs-all-in-one",
+          }
+          if components != expected_components:
+              errors.append(
+                  f"security context workload coverage is incomplete: "
+                  f"expected {sorted(expected_components)}, got {sorted(components)}"
+              )
+          if workloads != 10 or containers != 11:
+              errors.append(
+                  f"expected 10 workloads and 11 containers, got "
+                  f"{workloads} workloads and {containers} containers"
+              )
+          if chart_managed_init_containers != 1:
+              errors.append(
+                  f"expected one chart-managed seaweedfs-vol-move-idx init container, "
+                  f"got {chart_managed_init_containers}"
+              )
+
+          if errors:
+              print("\n".join(f"FAIL: {error}" for error in errors), file=sys.stderr)
+              sys.exit(1)
+          print(f"Validated security contexts on {workloads} workloads and {containers} containers")
+          PYEOF
+
+          # The resize hook depends on lookup finding a live StatefulSet and
+          # therefore cannot render during helm template. Keep static coverage
+          # for both security-context blocks.
+          grep -Fqx '      securityContext: {{- omit .Values.volume.podSecurityContext "enabled" | toYaml | nindent 8 }}' \
+            "$CHART_DIR/templates/volume/volume-resize-hook.yaml"
+          grep -Fqx '          securityContext: {{- omit .Values.volume.containerSecurityContext "enabled" | toYaml | nindent 12 }}' \
+            "$CHART_DIR/templates/volume/volume-resize-hook.yaml"
+          echo "Volume resize hook security contexts are covered"
           
           echo ""
           echo "=== Testing long release name: service names match DNS references ==="
@@ -1058,6 +1293,9 @@ jobs:
               "s3.enabled": "true",
               "sftp.enabled": "true",
               "admin.enabled": "true",
+              # admin.ip defaults to 0.0.0.0 (non-loopback), which weed admin 4.46
+              # refuses to bind without authentication.
+              "admin.secret.adminPassword": "ci-admin-password",
               "worker.enabled": "true",
               "cosi.enabled": "true",
               "s3.createBuckets[0].name": "b",
@@ -1254,7 +1492,7 @@ jobs:
           # Which means egress on its own must render for a release that runs
           # neither COSI nor a resize: no component of it reaches the API server,
           # so nothing may demand a CIDR for one.
-          for label, values in {"defaults": {}, "admin": {"admin.enabled": "true"}}.items():
+          for label, values in {"defaults": {}, "admin": {"admin.enabled": "true", "admin.secret.adminPassword": "ci-admin-password"}}.items():
               try:
                   render(dict(values, **{"networkPolicy.enabled": "true",
                                          "networkPolicy.egress.enabled": "true"}))
@@ -1315,6 +1553,9 @@ jobs:
 
           ALL_ON = {
               "admin.enabled": "true",
+              # admin.ip defaults to 0.0.0.0 (non-loopback), which weed admin 4.46
+              # refuses to bind without authentication.
+              "admin.secret.adminPassword": "ci-admin-password",
               "s3.enabled": "true",
               "sftp.enabled": "true",
               "worker.enabled": "true",
@@ -1472,11 +1713,100 @@ jobs:
 
           echo "All template rendering tests passed!"
 
+      - name: Resolve an image tag that is published
+        run: |
+          set -e
+          # A release bumps appVersion on master ~40 minutes before the container
+          # build publishes that tag, and this workflow runs on the bump commit.
+          # Install the last released image for the length of that window instead
+          # of failing on ImagePullBackOff.
+          IMAGE=$(helm template test k8s/charts/seaweedfs \
+            -s templates/master/master-statefulset.yaml | awk '$1 == "image:" {print $2; exit}')
+          REPO=${IMAGE%:*}
+          TAG=${IMAGE##*:}
+          # Anything but a published tag installs latest, which between releases
+          # is the same digest as the chart's own appVersion, so a registry blip
+          # costs nothing while failing the job on one would cost a red build.
+          STATUS=$(curl -sSL --connect-timeout 5 --max-time 15 -o /dev/null \
+            -w '%{http_code}' "https://hub.docker.com/v2/repositories/$REPO/tags/$TAG" || true)
+          if [ "$STATUS" = 200 ]; then
+            echo "installing $IMAGE"
+          else
+            echo "$IMAGE is unavailable (HTTP ${STATUS:-none}), installing $REPO:latest"
+            TAG=latest
+          fi
+          echo "IMAGE_TAG=$TAG" >> $GITHUB_ENV
+
       - name: Create kind cluster
-        uses: helm/kind-action@v1.14.0
+        uses: helm/kind-action@v1.15.0
+
+      - name: Verify volume resize hook security contexts
+        run: |
+          set -e
+          CHART_DIR="k8s/charts/seaweedfs"
+          NS="resize-hook-security"
+          kubectl create namespace "$NS"
+          kubectl apply -n "$NS" -f - <<'EOF'
+          apiVersion: v1
+          kind: PersistentVolumeClaim
+          metadata:
+            name: data1-resize-seaweedfs-volume-0
+          spec:
+            accessModes:
+              - ReadWriteOnce
+            resources:
+              requests:
+                storage: 1Gi
+          EOF
+
+          helm template resize "$CHART_DIR" -n "$NS" --dry-run=server \
+            --set volume.dataDirs[0].name=data1 \
+            --set volume.dataDirs[0].type=persistentVolumeClaim \
+            --set volume.dataDirs[0].size=2Gi \
+            --set volume.podSecurityContext.enabled=true \
+            --set volume.podSecurityContext.seccompProfile.type=RuntimeDefault \
+            --set volume.containerSecurityContext.enabled=true \
+            --set volume.containerSecurityContext.privileged=false \
+            --set volume.containerSecurityContext.allowPrivilegeEscalation=false \
+            --set volume.containerSecurityContext.capabilities.drop[0]=ALL \
+            --set volume.containerSecurityContext.seccompProfile.type=RuntimeDefault \
+            > /tmp/security-context-resize-hook.yaml
+
+          python3 - /tmp/security-context-resize-hook.yaml <<'PYEOF'
+          import sys
+
+          import yaml
+
+          with open(sys.argv[1]) as stream:
+              documents = [document for document in yaml.safe_load_all(stream) if document]
+
+          jobs = [
+              document for document in documents
+              if document.get("kind") == "Job"
+              and document["metadata"]["name"].endswith("-volume-resize-hook")
+          ]
+          if len(jobs) != 1:
+              raise AssertionError(f"expected one volume resize hook Job, got {len(jobs)}")
+
+          pod = jobs[0]["spec"]["template"]["spec"]
+          assert pod["securityContext"] == {
+              "seccompProfile": {"type": "RuntimeDefault"},
+          }
+          assert len(pod["containers"]) == 1
+          assert pod["containers"][0]["securityContext"] == {
+              "allowPrivilegeEscalation": False,
+              "capabilities": {"drop": ["ALL"]},
+              "privileged": False,
+              "seccompProfile": {"type": "RuntimeDefault"},
+          }
+          PYEOF
+          kubectl delete namespace "$NS"
+          echo "Volume resize hook security contexts render correctly"
 
       - name: Run chart-testing (install)
-        run: ct install --target-branch ${{ github.event.repository.default_branch }} --all --chart-dirs k8s/charts
+        run: |
+          ct install --target-branch ${{ github.event.repository.default_branch }} --all --chart-dirs k8s/charts \
+            --helm-extra-set-args "--set=image.tag=$IMAGE_TAG"
 
       - name: Verify SFTP host key secret lifecycle
         run: |
@@ -1484,7 +1814,7 @@ jobs:
           CHART_DIR="k8s/charts/seaweedfs"
           NS="sftp-hostkey"
           SECRET="hk-seaweedfs-sftp-ssh-secret"
-          SFTP_ARGS="--set sftp.enabled=true --set master.enabled=false --set volume.enabled=false --set filer.enabled=false"
+          SFTP_ARGS="--set image.tag=$IMAGE_TAG --set sftp.enabled=true --set master.enabled=false --set volume.enabled=false --set filer.enabled=false"
           kubectl create namespace "$NS"
 
           echo "=== install generates a host key, upgrade keeps it ==="
@@ -1561,6 +1891,7 @@ jobs:
           # release if the hook Job does not finish, so a clean install is the
           # assertion.
           helm install np $CHART_DIR -n "$NS" --wait --timeout 8m \
+            --set image.tag=$IMAGE_TAG \
             --set s3.enabled=true \
             --set s3.createBuckets[0].name=testbucket \
             --set networkPolicy.enabled=true \

.github/workflows/java_integration_tests.yml

@@ -34,7 +34,7 @@ jobs:
         id: go
       
       - name: Set up Java
-        uses: actions/setup-java@v5
+        uses: actions/setup-java@v6
         with:
           java-version: ${{ matrix.java }}
           distribution: 'temurin'

.github/workflows/java_release.yml

@@ -73,7 +73,7 @@ jobs:
           echo "version=${VERSION}" >> "$GITHUB_OUTPUT"
 
       - name: Set up JDK 17
-        uses: actions/setup-java@v5
+        uses: actions/setup-java@v6
         with:
           java-version: '17'
           distribution: 'temurin'

.github/workflows/java_unit_tests.yml

@@ -26,7 +26,7 @@ jobs:
         uses: actions/checkout@v7
       
       - name: Set up Java
-        uses: actions/setup-java@v5
+        uses: actions/setup-java@v6
         with:
           java-version: ${{ matrix.java }}
           distribution: 'temurin'

.github/workflows/kafka-quicktest.yml

@@ -42,7 +42,7 @@ jobs:
     - name: Set up Go 1.x
       uses: actions/setup-go@v7
       with:
-        go-version: ^1.25
+        go-version: ^1.26
         cache: true
         cache-dependency-path: |
           **/go.sum

.github/workflows/kafka-tests.yml

@@ -43,7 +43,7 @@ jobs:
       matrix:
         container-id: [unit-tests-1]
     container:
-      image: golang:1.24-alpine
+      image: golang:1.26-alpine
       options: --cpus 1.0 --memory 1g --hostname kafka-unit-${{ matrix.container-id }}
     env:
       GOMAXPROCS: 1
@@ -53,7 +53,7 @@ jobs:
     - name: Set up Go 1.x
       uses: actions/setup-go@v7
       with:
-        go-version: ^1.25
+        go-version: ^1.26
       id: go
 
     - name: Check out code
@@ -87,7 +87,7 @@ jobs:
       matrix:
         container-id: [integration-1]
     container:
-      image: golang:1.24-alpine
+      image: golang:1.26-alpine
       options: --cpus 2.0 --memory 2g --ulimit nofile=1024:1024 --hostname kafka-integration-${{ matrix.container-id }}
     env:
       GOMAXPROCS: 2
@@ -98,7 +98,7 @@ jobs:
     - name: Set up Go 1.x
       uses: actions/setup-go@v7
       with:
-        go-version: ^1.25
+        go-version: ^1.26
       id: go
 
     - name: Check out code
@@ -134,7 +134,7 @@ jobs:
       matrix:
         container-id: [e2e-1]
     container:
-      image: golang:1.24-alpine
+      image: golang:1.26-alpine
       options: --cpus 2.0 --memory 2g --hostname kafka-e2e-${{ matrix.container-id }}
     env:
       GOMAXPROCS: 2
@@ -148,7 +148,7 @@ jobs:
     - name: Set up Go 1.x
       uses: actions/setup-go@v7
       with:
-        go-version: ^1.25
+        go-version: ^1.26
         cache: true
         cache-dependency-path: |
           **/go.sum
@@ -313,7 +313,7 @@ jobs:
       matrix:
         container-id: [consumer-group-1]
     container:
-      image: golang:1.24-alpine
+      image: golang:1.26-alpine
       options: --cpus 1.0 --memory 2g --ulimit nofile=512:512 --hostname kafka-consumer-${{ matrix.container-id }}
     env:
       GOMAXPROCS: 1
@@ -327,7 +327,7 @@ jobs:
     - name: Set up Go 1.x
       uses: actions/setup-go@v7
       with:
-        go-version: ^1.25
+        go-version: ^1.26
         cache: true
         cache-dependency-path: |
           **/go.sum
@@ -475,7 +475,7 @@ jobs:
       matrix:
         container-id: [client-compat-1]
     container:
-      image: golang:1.24-alpine
+      image: golang:1.26-alpine
       options: --cpus 1.0 --memory 1.5g --shm-size 256m --hostname kafka-client-${{ matrix.container-id }}
     env:
       GOMAXPROCS: 1
@@ -489,7 +489,7 @@ jobs:
     - name: Set up Go 1.x
       uses: actions/setup-go@v7
       with:
-        go-version: ^1.25
+        go-version: ^1.26
         cache: true
         cache-dependency-path: |
           **/go.sum
@@ -633,7 +633,7 @@ jobs:
       matrix:
         container-id: [smq-integration-1]
     container:
-      image: golang:1.24-alpine
+      image: golang:1.26-alpine
       options: --cpus 1.0 --memory 2g --hostname kafka-smq-${{ matrix.container-id }}
     env:
       GOMAXPROCS: 1
@@ -647,7 +647,7 @@ jobs:
     - name: Set up Go 1.x
       uses: actions/setup-go@v7
       with:
-        go-version: ^1.25
+        go-version: ^1.26
         cache: true
         cache-dependency-path: |
           **/go.sum
@@ -794,7 +794,7 @@ jobs:
       matrix:
         container-id: [protocol-1]
     container:
-      image: golang:1.24-alpine
+      image: golang:1.26-alpine
       options: --cpus 1.0 --memory 1g --tmpfs /tmp:exec --hostname kafka-protocol-${{ matrix.container-id }}
     env:
       GOMAXPROCS: 1
@@ -805,7 +805,7 @@ jobs:
     - name: Set up Go 1.x
       uses: actions/setup-go@v7
       with:
-        go-version: ^1.25
+        go-version: ^1.26
       id: go
 
     - name: Check out code

.github/workflows/mount-benchmark.yml

@@ -0,0 +1,203 @@
+name: "mount: benchmark"
+
+# Manual benchmark: native WinFsp mount vs rclone+WebDAV on the same Windows
+# runner, with a Linux FUSE mount of the same build as a reference. Numbers
+# from shared runners are noisy; this is for finding factor-of-N gaps, not
+# regressions of a few percent.
+on:
+  workflow_dispatch:
+  push:
+    branches: [ 'winfsp-bench**' ]
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+permissions:
+  contents: read
+
+jobs:
+  bench-windows:
+    name: Windows native vs rclone
+    runs-on: windows-latest
+    timeout-minutes: 60
+    env:
+      CGO_ENABLED: 0
+
+    steps:
+    - uses: actions/checkout@v7
+      with:
+        persist-credentials: false
+
+    - uses: actions/setup-go@v7
+      with:
+        go-version-file: 'go.mod'
+
+    - name: Install WinFsp and rclone
+      run: choco install winfsp rclone -y --no-progress
+
+    - name: Build weed.exe
+      run: go build -o weed.exe ./weed
+
+    - name: Benchmark both mounts
+      shell: pwsh
+      run: |
+        $ErrorActionPreference = 'Stop'
+
+        function Test-Port($port) {
+          $client = New-Object System.Net.Sockets.TcpClient
+          try { $client.Connect('127.0.0.1', $port); return $client.Connected }
+          catch { return $false }
+          finally { $client.Dispose() }
+        }
+
+        function Wait-Drive($drive, $what) {
+          $deadline = (Get-Date).AddMinutes(2)
+          while ((Get-Date) -lt $deadline) {
+            if (Test-Path "${drive}\") { Write-Host "$what is mounted on $drive"; return }
+            Start-Sleep -Seconds 2
+          }
+          throw "$what never appeared on $drive"
+        }
+
+        function Invoke-Bench($dir, $label, $out) {
+          Write-Host "::group::bench $label"
+          & go run ./test/mount_bench -dir $dir -label $label -filer 127.0.0.1:8888 -out $out
+          $code = $LASTEXITCODE
+          Write-Host "::endgroup::"
+          if ($code -ne 0) { throw "bench $label failed with exit $code" }
+        }
+
+        New-Item -ItemType Directory -Force -Path C:\seaweed-data | Out-Null
+        Start-Process -FilePath .\weed.exe `
+          -ArgumentList '-logtostderr','mini','-dir=C:\seaweed-data','-ip=127.0.0.1' `
+          -RedirectStandardOutput C:\seaweed-mini.log -RedirectStandardError C:\seaweed-mini.err.log
+
+        $deadline = (Get-Date).AddMinutes(3)
+        while ((Get-Date) -lt $deadline) {
+          if ((Test-Port 8888) -and (Test-Port 18888) -and (Test-Port 7333)) { break }
+          Start-Sleep -Seconds 3
+        }
+        if (-not ((Test-Port 8888) -and (Test-Port 18888) -and (Test-Port 7333))) {
+          Get-Content C:\seaweed-mini.log, C:\seaweed-mini.err.log -ErrorAction SilentlyContinue
+          throw "mini cluster never came up"
+        }
+        Write-Host "filer on 8888/18888, webdav on 7333"
+
+        # --- native WinFsp mount ---
+        Start-Process -FilePath .\weed.exe `
+          -ArgumentList '-logtostderr','mount','-filer=127.0.0.1:8888','-dir=S:' `
+          -RedirectStandardOutput C:\seaweed-mount.log -RedirectStandardError C:\seaweed-mount.err.log
+        Wait-Drive 'S:' 'weed mount'
+
+        Invoke-Bench 'S:\bench-native' 'winfsp-native' 'C:\results-native.json'
+
+        Get-CimInstance Win32_Process -Filter "Name = 'weed.exe'" |
+          Where-Object { $_.CommandLine -like '*mount*' } |
+          ForEach-Object { Stop-Process -Id $_.ProcessId -Force }
+        $deadline = (Get-Date).AddMinutes(1)
+        while ((Get-Date) -lt $deadline -and (Test-Path S:\)) { Start-Sleep -Seconds 2 }
+
+        # --- rclone + WebDAV on the same WinFsp ---
+        # rclone serves listings from a directory cache it fills lazily, so the
+        # big-listing files have to exist before it mounts or it never sees them.
+        & go run ./test/mount_bench -filer 127.0.0.1:8888 -seed bench-rclone/biglist
+        if ($LASTEXITCODE -ne 0) { throw "seeding failed" }
+
+        $env:RCLONE_CONFIG_SEAWEED_TYPE = 'webdav'
+        $env:RCLONE_CONFIG_SEAWEED_URL = 'http://127.0.0.1:7333'
+        $env:RCLONE_CONFIG_SEAWEED_VENDOR = 'other'
+        Start-Process -FilePath rclone `
+          -ArgumentList 'mount','seaweed:','T:','--vfs-cache-mode=writes','-v','--log-file=C:\rclone.log'
+        Wait-Drive 'T:' 'rclone mount'
+
+        Invoke-Bench 'T:\bench-rclone' 'rclone-webdav' 'C:\results-rclone.json'
+
+        Stop-Process -Name rclone -Force -ErrorAction SilentlyContinue
+
+        # --- comparison ---
+        $table = & go run ./test/mount_bench -compare C:\results-native.json,C:\results-rclone.json
+        $table | Write-Host
+        "## Windows: native WinFsp vs rclone+WebDAV" | Out-File -Append $env:GITHUB_STEP_SUMMARY
+        $table | Out-File -Append $env:GITHUB_STEP_SUMMARY
+
+    - name: Logs
+      if: always()
+      shell: pwsh
+      run: |
+        foreach ($f in 'C:\seaweed-mount.log','C:\seaweed-mount.err.log','C:\rclone.log','C:\seaweed-mini.log','C:\seaweed-mini.err.log') {
+          if (Test-Path $f) { Write-Host "===== $f"; Get-Content $f -Tail 100 }
+        }
+
+    - name: Results
+      if: always()
+      uses: actions/upload-artifact@v7
+      with:
+        name: results-windows
+        path: C:\results-*.json
+        if-no-files-found: ignore
+
+  bench-linux:
+    name: Linux FUSE reference
+    runs-on: ubuntu-latest
+    timeout-minutes: 60
+
+    steps:
+    - uses: actions/checkout@v7
+      with:
+        persist-credentials: false
+
+    - uses: actions/setup-go@v7
+      with:
+        go-version-file: 'go.mod'
+
+    - name: Repair the fusermount3 setuid bit
+      uses: ./.github/actions/fix-fusermount-setuid
+
+    - name: Allow non-root FUSE mounts with allow_other
+      run: |
+        echo 'user_allow_other' | sudo tee -a /etc/fuse.conf
+        sudo chmod 644 /etc/fuse.conf
+
+    - name: Build weed
+      run: go build -o /tmp/weed ./weed
+
+    - name: Benchmark FUSE mount
+      run: |
+        set -e
+        mkdir -p /tmp/seaweed-data
+        /tmp/weed -logtostderr mini -dir=/tmp/seaweed-data -ip=127.0.0.1 > /tmp/mini.log 2>&1 &
+
+        for i in $(seq 1 60); do
+          if nc -z 127.0.0.1 8888 && nc -z 127.0.0.1 18888; then break; fi
+          sleep 3
+        done
+        nc -z 127.0.0.1 8888 || { cat /tmp/mini.log; echo "filer never came up"; exit 1; }
+
+        mkdir -p "$HOME/mnt"
+        /tmp/weed -logtostderr mount -filer=127.0.0.1:8888 -dir="$HOME/mnt" > /tmp/mount.log 2>&1 &
+        for i in $(seq 1 60); do
+          if mountpoint -q "$HOME/mnt"; then break; fi
+          sleep 2
+        done
+        mountpoint -q "$HOME/mnt" || { cat /tmp/mount.log; echo "mount never appeared"; exit 1; }
+
+        go run ./test/mount_bench -dir "$HOME/mnt/bench-linux" -label linux-fuse -filer 127.0.0.1:8888 -out /tmp/results-linux.json
+
+        {
+          echo "## Linux FUSE reference"
+          go run ./test/mount_bench -compare /tmp/results-linux.json
+        } >> "$GITHUB_STEP_SUMMARY"
+
+    - name: Logs
+      if: always()
+      run: |
+        tail -n 100 /tmp/mount.log /tmp/mini.log 2>/dev/null || true
+
+    - name: Results
+      if: always()
+      uses: actions/upload-artifact@v7
+      with:
+        name: results-linux
+        path: /tmp/results-linux.json
+        if-no-files-found: ignore

.github/workflows/mount-windows.yml

@@ -185,8 +185,19 @@ jobs:
         if (Test-Path S:\walk) { throw "directory survived recursive delete" }
         Write-Host "::endgroup::"
 
+        # Tear down everything we started so the runner's later steps (Logs,
+        # post-checkout) launch into a clean environment. A WinFsp mount left
+        # active and a weed.exe holding winfsp-x64.dll have been seen to make
+        # the next step's pwsh.exe fail with STATUS_DLL_INIT_FAILED
+        # (0xC0000142), which fails a job whose actual test step passed.
+        try { Stop-Mount } catch { Write-Host "no mount to stop at teardown: $_" }
+        Get-CimInstance Win32_Process -Filter "Name = 'weed.exe'" |
+          ForEach-Object { Stop-Process -Id $_.ProcessId -Force -ErrorAction SilentlyContinue }
+        Start-Sleep -Seconds 3
+
     - name: Logs
       if: always()
+      continue-on-error: true
       shell: pwsh
       run: |
         foreach ($f in 'C:\seaweed-mount.log','C:\seaweed-mount.err.log','C:\seaweed-remount.log','C:\seaweed-remount.err.log','C:\seaweed-remount2.log','C:\seaweed-remount2.err.log','C:\seaweed-dirmount.log','C:\seaweed-dirmount.err.log','C:\seaweed-mini.log','C:\seaweed-mini.err.log') {

.github/workflows/performance.yml

@@ -8,6 +8,7 @@ on:
       - 'go.mod'
       - 'go.sum'
       - 'seaweed-volume/**'
+      - 'seaweed-common/**'
       - 'test/perf/**'
       - '.github/workflows/performance.yml'
   workflow_dispatch:
@@ -175,10 +176,6 @@ jobs:
         with:
           go-version-file: 'go.mod'
 
-      - name: Install protobuf compiler
-        if: matrix.impl == 'rust'
-        run: sudo apt-get update && sudo apt-get install -y protobuf-compiler
-
       - name: Install Rust toolchain
         if: matrix.impl == 'rust'
         uses: dtolnay/rust-toolchain@stable
@@ -331,10 +328,6 @@ jobs:
         with:
           go-version-file: 'go.mod'
 
-      - name: Install protobuf compiler
-        if: matrix.impl == 'rust'
-        run: sudo apt-get update && sudo apt-get install -y protobuf-compiler
-
       - name: Install Rust toolchain
         if: matrix.impl == 'rust'
         uses: dtolnay/rust-toolchain@stable

.github/workflows/postgres-tests.yml

@@ -41,7 +41,7 @@ jobs:
     - name: Set up Go 1.x
       uses: actions/setup-go@v7
       with:
-        go-version: ^1.25
+        go-version: ^1.26
       id: go
 
     - name: Check out code

.github/workflows/release_version_bump.yml

@@ -1,4 +1,17 @@
-name: "release: bump version"
+name: "release: bump version and cut the release"
+
+# One entry point for a SeaweedFS release:
+#   1. bump MAJOR/MINOR in constants.go and the Helm Chart.yaml, commit to master
+#   2. push the <appVersion> tag, which fans out to the workflows that trigger on
+#      `push: tags` (binaries_release*, container_release_unified, helm_manual_release)
+#   3. create the GitHub release, with GitHub's generated notes
+#   4. dispatch "Prepare release" in seaweedfs-csi-driver and seaweedfs-operator,
+#      which pick up the new master through `go get -u`, and wait for both
+#
+# Events raised by the default GITHUB_TOKEN do not start other workflows, and it
+# cannot reach the other two repositories at all. Add a repo secret RELEASE_PAT
+# with `contents: write` here and `actions: write` on the csi-driver and operator
+# repos. Without it the tag is still pushed, but nothing downstream of it runs.
 
 on:
   workflow_dispatch:
@@ -14,15 +27,37 @@ on:
         description: "Explicit MAJOR.MINOR to set, e.g. 4.36 (overrides 'bump')"
         type: string
         required: false
-
-permissions:
-  contents: write
+      downstream:
+        description: "Also release the CSI driver and the operator"
+        type: boolean
+        default: true
+      dry_run:
+        description: "Show the version bump, but change nothing"
+        type: boolean
+        default: false
 
 jobs:
-  bump-version:
+  release:
     runs-on: ubuntu-latest
+    permissions:
+      contents: write
+    outputs:
+      app_version: ${{ steps.compute.outputs.app_version }}
+      sha: ${{ steps.tag.outputs.sha }}
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: master
+          fetch-depth: 0
+          token: ${{ secrets.RELEASE_PAT || secrets.GITHUB_TOKEN }}
+
+      - name: Check the release token
+        env:
+          HAS_PAT: ${{ secrets.RELEASE_PAT != '' }}
+        run: |
+          if [ "$HAS_PAT" != "true" ]; then
+            echo "::warning::RELEASE_PAT is not set. The tag will be pushed with GITHUB_TOKEN, so the binary, container and helm workflows will not start on their own."
+          fi
 
       - name: Compute new version
         id: compute
@@ -99,17 +134,145 @@ jobs:
           sed -i -E "s/^version:.*/version: ${CHART_VERSION}/" "$CHART"
           cat "$CHART"
 
-      - name: Commit and push
+      - name: Commit, and push the tag
+        id: tag
         env:
-          APP_VERSION: ${{ steps.compute.outputs.app_version }}
+          TAG: ${{ steps.compute.outputs.app_version }}
+          DRY_RUN: ${{ inputs.dry_run }}
         run: |
           set -euo pipefail
-          if git diff --quiet; then
-            echo "No version change to commit."
+          if git ls-remote --exit-code --tags origin "refs/tags/${TAG}" >/dev/null 2>&1; then
+            echo "::error::Tag ${TAG} already exists."
+            exit 1
+          fi
+          if [ "$DRY_RUN" = "true" ]; then
+            git --no-pager diff --stat
+            echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
             exit 0
           fi
+
           git config user.name "github-actions[bot]"
           git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
-          git add weed/util/version/constants.go k8s/charts/seaweedfs/Chart.yaml
-          git commit -m "${APP_VERSION}"
-          git push
+          if git diff --quiet; then
+            echo "::warning::Version files are already at ${TAG}; tagging the current HEAD."
+          else
+            git add weed/util/version/constants.go k8s/charts/seaweedfs/Chart.yaml
+            git commit -m "${TAG}"
+            git push
+          fi
+
+          # Push the tag with git so the `push: tags` triggers fire. Creating the
+          # tag through the release API alone would only emit a `create` event.
+          git tag "$TAG"
+          git push origin "$TAG"
+          echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
+
+      - name: Create the release
+        if: ${{ !inputs.dry_run }}
+        env:
+          GH_TOKEN: ${{ secrets.RELEASE_PAT || secrets.GITHUB_TOKEN }}
+          TAG: ${{ steps.compute.outputs.app_version }}
+        run: gh release create "$TAG" --title "$TAG" --generate-notes --verify-tag
+
+  downstream:
+    needs: release
+    if: ${{ inputs.downstream && !inputs.dry_run }}
+    runs-on: ubuntu-latest
+    # The wait below puts no bound of its own on runner-queue time; this does.
+    timeout-minutes: 120
+    permissions: {}
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - repo: seaweedfs/seaweedfs-csi-driver
+            workflow: prepare_release.yaml
+          - repo: seaweedfs/seaweedfs-operator
+            workflow: prepare_release.yml
+    steps:
+      - name: Release ${{ matrix.repo }}
+        env:
+          GH_TOKEN: ${{ secrets.RELEASE_PAT }}
+          REPO: ${{ matrix.repo }}
+          WORKFLOW: ${{ matrix.workflow }}
+          SHA: ${{ needs.release.outputs.sha }}
+          MODULE: github.com/seaweedfs/seaweedfs
+        run: |
+          set -euo pipefail
+          if [ -z "${GH_TOKEN}" ]; then
+            echo "::error::RELEASE_PAT with actions:write on ${REPO} is required to release it"
+            exit 1
+          fi
+
+          # The dispatched workflow pins seaweedfs with `go get -u ...@latest`, so
+          # wait until the proxy serves the release commit as the tip. Asking for
+          # the commit by name is what makes the proxy fetch it. The proxy can
+          # take longer than five minutes to refresh @latest after a new tag.
+          for _ in $(seq 120); do
+            curl -sf "https://proxy.golang.org/${MODULE}/@v/${SHA}.info" >/dev/null || true
+            TIP=$(curl -sf "https://proxy.golang.org/${MODULE}/@latest" | jq -r '.Origin.Hash // ""' || true)
+            [ "$TIP" = "$SHA" ] && break
+            sleep 10
+          done
+          if [ "$TIP" != "$SHA" ]; then
+            echo "::error::the module proxy still serves ${TIP} as the tip, so ${REPO} would pin a pre-release commit. Run ${WORKFLOW} there once it catches up."
+            exit 1
+          fi
+
+          # The dispatched workflow publishes the release as its last step, so
+          # its conclusion decides success. Wait on the run, not on a wall
+          # clock: time it spends queued for a runner must not count against
+          # the budget. A dispatch does not return its run id, so take the
+          # newest workflow_dispatch run created since ours; a concurrent
+          # dispatch would be performing this same release, and waiting on it
+          # is just as good.
+          released() { gh api "repos/${REPO}/releases?per_page=30" --jq '[.[].tag_name]'; }
+
+          BEFORE=$(released)
+          # A minute early, so runner clock skew cannot hide the run.
+          DISPATCHED_AT=$(date -u -d '1 minute ago' '+%Y-%m-%dT%H:%M:%SZ')
+          gh workflow run -R "$REPO" "$WORKFLOW" --ref master -f bump=patch -f update_seaweedfs=true
+
+          RUN_ID=""
+          for _ in $(seq 12); do
+            sleep 10
+            RUN_ID=$(gh api -X GET "repos/${REPO}/actions/workflows/${WORKFLOW}/runs" \
+              -f event=workflow_dispatch -f "created=>=${DISPATCHED_AT}" \
+              --jq '(.workflow_runs | sort_by(.created_at) | last | .id) // empty' || true)
+            [ -n "$RUN_ID" ] && break
+          done
+          if [ -z "$RUN_ID" ]; then
+            echo "::error::the dispatch created no ${WORKFLOW} run in ${REPO}; see https://github.com/${REPO}/actions/workflows/${WORKFLOW}"
+            exit 1
+          fi
+          RUN_URL="https://github.com/${REPO}/actions/runs/${RUN_ID}"
+          echo "waiting on ${RUN_URL}"
+
+          # 20 minutes of execution; polls that find the run still queued do
+          # not consume it.
+          RUNNING=0
+          STATE=""
+          while :; do
+            sleep 15
+            STATE=$(gh api "repos/${REPO}/actions/runs/${RUN_ID}" \
+              --jq '.status + "/" + (.conclusion // "")' || true)
+            case "$STATE" in
+              completed/*) break ;;
+              in_progress/*) RUNNING=$((RUNNING + 1)) ;;
+            esac
+            if [ "$RUNNING" -gt 80 ]; then
+              echo "::error::${RUN_URL} has been executing for over 20 minutes; giving up on it"
+              exit 1
+            fi
+          done
+          if [ "$STATE" != "completed/success" ]; then
+            echo "::error::${RUN_URL} concluded '${STATE#completed/}'"
+            exit 1
+          fi
+
+          NEW=$(released | jq -c --argjson before "$BEFORE" '. - $before')
+          if [ "$(jq length <<<"$NEW")" -eq 0 ]; then
+            echo "::error::${RUN_URL} succeeded but ${REPO} shows no new release"
+            exit 1
+          fi
+          echo "${REPO} released $(jq -r 'join(", ")' <<<"$NEW")"

.github/workflows/rust-volume-server-tests.yml

@@ -5,6 +5,7 @@ on:
     branches: [ master ]
     paths:
       - 'seaweed-volume/**'
+      - 'seaweed-common/**'
       - 'test/volume_server/**'
       - 'weed/pb/volume_server.proto'
       - 'weed/pb/volume_server_pb/**'
@@ -13,6 +14,7 @@ on:
     branches: [ master, main ]
     paths:
       - 'seaweed-volume/**'
+      - 'seaweed-common/**'
       - 'test/volume_server/**'
       - 'weed/pb/volume_server.proto'
       - 'weed/pb/volume_server_pb/**'
@@ -27,6 +29,29 @@ permissions:
 
 
 jobs:
+  changes:
+    name: Detect changed paths
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    permissions:
+      contents: read
+    outputs:
+      rust: ${{ steps.filter.outputs.rust }}
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v7
+        with:
+          fetch-depth: 0
+
+      - name: Filter changed paths
+        id: filter
+        uses: dorny/paths-filter@v4
+        with:
+          filters: |
+            rust:
+              - 'seaweed-volume/**'
+              - '.github/workflows/rust-volume-server-tests.yml'
+
   rust-unit-tests:
     name: Rust Unit Tests
     runs-on: ubuntu-22.04
@@ -36,9 +61,6 @@ jobs:
       - name: Checkout code
         uses: actions/checkout@v7
 
-      - name: Install protobuf compiler
-        run: sudo apt-get update && sudo apt-get install -y protobuf-compiler
-
       - name: Install Rust toolchain
         uses: dtolnay/rust-toolchain@stable
 
@@ -55,16 +77,79 @@ jobs:
             ~/.cargo/registry
             ~/.cargo/git
             seaweed-volume/target
-          key: rust-${{ steps.toolchain.outputs.fingerprint }}-${{ hashFiles('seaweed-volume/Cargo.lock') }}
+          key: rust-${{ steps.toolchain.outputs.fingerprint }}-${{ hashFiles('seaweed-volume/Cargo.lock', 'seaweed-common/Cargo.lock') }}
           restore-keys: |
             rust-${{ steps.toolchain.outputs.fingerprint }}-
 
       - name: Build Rust volume server
         run: cd seaweed-volume && cargo build --release
 
+      # The crate is warning-free under clippy as of the sweep that added
+      # this step. Uncomment to make that a gate; `[lints.clippy]` in
+      # seaweed-volume/Cargo.toml is where crate-wide exceptions live.
+      # - name: Clippy
+      #   run: cd seaweed-volume && cargo clippy --all-targets -- -D warnings
+
+      # The crate is rustfmt-clean as of the PR that added this step.
+      # Uncomment to keep it that way.
+      # - name: Check formatting
+      #   run: cd seaweed-volume && cargo fmt --check
+
+      # seaweed-common is a path dependency of this crate, not a member of its
+      # workspace, so the run below does not reach its own tests. It builds into
+      # this job's cached target directory, and the cache key above covers the
+      # shared crate's lock, so the aws-lc-sys that rustls pulls in is restored
+      # with the cache instead of compiled from scratch on every run.
+      - name: Run shared-crate unit tests
+        env:
+          CARGO_TARGET_DIR: ${{ github.workspace }}/seaweed-volume/target
+        run: cd seaweed-common && cargo test
+
       - name: Run Rust unit tests
         run: cd seaweed-volume && cargo test
 
+      - name: Run Rust unit tests (redb experimental cursor)
+        run: cd seaweed-volume && cargo test --features redb-experimental-cursor --lib storage::needle_map
+
+  rust-unit-tests-windows:
+    name: Rust Unit Tests (Windows)
+    runs-on: windows-latest
+    timeout-minutes: 30
+    needs: [changes]
+    if: needs.changes.outputs.rust == 'true'
+    defaults:
+      run:
+        shell: bash
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v7
+
+      - name: Install Rust toolchain
+        uses: dtolnay/rust-toolchain@stable
+
+      # No glibc on Windows: key the cache on the toolchain and OS only.
+      - name: Fingerprint build toolchain
+        id: toolchain
+        run: echo "fingerprint=windows-rustc-$(rustc -V | awk '{print $2}')" >> "$GITHUB_OUTPUT"
+
+      - name: Cache cargo registry and target
+        uses: actions/cache@v6
+        with:
+          path: |
+            ~/.cargo/registry
+            ~/.cargo/git
+            seaweed-volume/target
+          key: rust-windows-${{ steps.toolchain.outputs.fingerprint }}-${{ hashFiles('seaweed-volume/Cargo.lock') }}
+          restore-keys: |
+            rust-windows-${{ steps.toolchain.outputs.fingerprint }}-
+
+      - name: Run Rust unit tests
+        run: cd seaweed-volume && cargo test
+
+      - name: Run Rust unit tests (redb experimental cursor)
+        run: cd seaweed-volume && cargo test --features redb-experimental-cursor --lib storage::needle_map
+
   rust-integration-tests:
     name: Rust Integration Tests
     runs-on: ubuntu-22.04
@@ -79,9 +164,6 @@ jobs:
         with:
           go-version-file: 'go.mod'
 
-      - name: Install protobuf compiler
-        run: sudo apt-get update && sudo apt-get install -y protobuf-compiler
-
       - name: Install Rust toolchain
         uses: dtolnay/rust-toolchain@stable
 
@@ -98,7 +180,7 @@ jobs:
             ~/.cargo/registry
             ~/.cargo/git
             seaweed-volume/target
-          key: rust-${{ steps.toolchain.outputs.fingerprint }}-${{ hashFiles('seaweed-volume/Cargo.lock') }}
+          key: rust-${{ steps.toolchain.outputs.fingerprint }}-${{ hashFiles('seaweed-volume/Cargo.lock', 'seaweed-common/Cargo.lock') }}
           restore-keys: |
             rust-${{ steps.toolchain.outputs.fingerprint }}-
 
@@ -164,9 +246,6 @@ jobs:
         with:
           go-version-file: 'go.mod'
 
-      - name: Install protobuf compiler
-        run: sudo apt-get update && sudo apt-get install -y protobuf-compiler
-
       - name: Install Rust toolchain
         uses: dtolnay/rust-toolchain@stable
 
@@ -183,7 +262,7 @@ jobs:
             ~/.cargo/registry
             ~/.cargo/git
             seaweed-volume/target
-          key: rust-${{ steps.toolchain.outputs.fingerprint }}-${{ hashFiles('seaweed-volume/Cargo.lock') }}
+          key: rust-${{ steps.toolchain.outputs.fingerprint }}-${{ hashFiles('seaweed-volume/Cargo.lock', 'seaweed-common/Cargo.lock') }}
           restore-keys: |
             rust-${{ steps.toolchain.outputs.fingerprint }}-
 

.github/workflows/rust-worker-tests.yml

@@ -0,0 +1,106 @@
+name: "Rust Plugin Worker Tests"
+
+on:
+  pull_request:
+    branches: [ master ]
+    paths:
+      - 'seaweed-worker/**'
+      - 'seaweed-common/**'
+      - 'weed/pb/plugin.proto'
+      - '.github/workflows/rust-worker-tests.yml'
+  push:
+    branches: [ master, main ]
+    paths:
+      - 'seaweed-worker/**'
+      - 'seaweed-common/**'
+      - 'weed/pb/plugin.proto'
+      - '.github/workflows/rust-worker-tests.yml'
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}
+  cancel-in-progress: true
+
+permissions:
+  contents: read
+
+jobs:
+  rust-worker-build:
+    name: Rust Plugin Worker Build and Unit Tests
+    runs-on: ubuntu-22.04
+    timeout-minutes: 45
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v7
+        with:
+          persist-credentials: false
+
+      - name: Install Rust toolchain
+        uses: dtolnay/rust-toolchain@stable
+
+      # cargo tracks its own inputs but not the runner's C toolchain, so a cached
+      # target/ can carry C objects built against a different glibc than we link against.
+      - name: Fingerprint build toolchain
+        id: toolchain
+        run: echo "fingerprint=$(getconf GNU_LIBC_VERSION | tr ' ' '-')-rustc-$(rustc -V | awk '{print $2}')" >> "$GITHUB_OUTPUT"
+
+      - name: Cache cargo registry and target
+        uses: actions/cache@v6
+        with:
+          path: |
+            ~/.cargo/registry
+            ~/.cargo/git
+            seaweed-worker/target/release
+          key: rust-worker-${{ steps.toolchain.outputs.fingerprint }}-${{ hashFiles('seaweed-worker/Cargo.lock', 'seaweed-common/Cargo.lock') }}
+          restore-keys: |
+            rust-worker-${{ steps.toolchain.outputs.fingerprint }}-
+
+      # lance's build scripts compile their own protos and look for a protoc.
+      # Point them at the one protoc-bin-vendored ships, which seaweed-worker's
+      # own build already uses, so no job depends on a system package and every
+      # build sees the same version.
+      - name: Use the vendored protoc
+        run: |
+          cd seaweed-worker
+          cargo fetch
+          # The version from the lock, not whatever else a restored cache holds.
+          version=$(awk '/^name = "protoc-bin-vendored-linux-x86_64"$/{found=1; next} found && /^version = /{gsub(/"/,"",$3); print $3; exit}' Cargo.lock)
+          test -n "$version" || { echo "protoc-bin-vendored-linux-x86_64 is not in Cargo.lock" >&2; exit 1; }
+          protoc=$(find ~/.cargo/registry/src -path "*protoc-bin-vendored-linux-x86_64-$version/bin/protoc" | head -1)
+          test -x "$protoc" || { echo "no vendored protoc $version in the registry" >&2; exit 1; }
+          echo "PROTOC=$protoc" >> "$GITHUB_ENV"
+
+      # The release profile is what ships, and it is where the release and the
+      # container builds would otherwise discover a break for the first time.
+      - name: Build the plugin workers
+        run: cd seaweed-worker && cargo build --release
+
+      # The workspace is warning-free under clippy as of the sweep that added
+      # this step. Uncomment to make that a gate; `[workspace.lints.clippy]`
+      # in seaweed-worker/Cargo.toml is where crate-wide exceptions live.
+      # - name: Clippy
+      #   run: cd seaweed-worker && cargo clippy --workspace --all-targets -- -D warnings
+
+      # The workspace is rustfmt-clean as of the PR that added this step.
+      # Uncomment to keep it that way.
+      # - name: Check formatting
+      #   run: cd seaweed-worker && cargo fmt --all --check
+
+      # seaweed-common is a path dependency of core and lance, not a member of
+      # this workspace, so `--workspace` below does not reach its own tests.
+      # Release and this job's cached target directory, and the cache key above
+      # covers the shared crate's lock. That lock pins the same rustls and
+      # aws-lc-sys this workspace resolves, so the release build above has
+      # already paid for them.
+      - name: Run shared-crate unit tests
+        env:
+          CARGO_TARGET_DIR: ${{ github.workspace }}/seaweed-worker/target
+        run: cd seaweed-common && cargo test --release
+
+      # The tests that need a live gateway skip themselves without one, the way
+      # the Go integration tests skip without Docker; the lifecycle suite in
+      # test/s3tables/lifecycle is what runs them against a real cluster.
+      # Release, so this reuses the build above rather than compiling lance,
+      # arrow and datafusion a second time in another profile.
+      - name: Run unit tests
+        run: cd seaweed-worker && cargo test --release --workspace

.github/workflows/rust_binaries_dev.yml

@@ -5,6 +5,7 @@ on:
     branches: [ master ]
     paths:
       - 'seaweed-volume/**'
+      - 'seaweed-common/**'
       - '.github/workflows/rust_binaries_dev.yml'
 
 permissions:
@@ -41,9 +42,6 @@ jobs:
     steps:
       - uses: actions/checkout@v7
 
-      - name: Install protobuf compiler
-        run: sudo apt-get update && sudo apt-get install -y protobuf-compiler
-
       - name: Install Rust toolchain
         uses: dtolnay/rust-toolchain@stable
 
@@ -110,9 +108,6 @@ jobs:
     steps:
       - uses: actions/checkout@v7
 
-      - name: Install protobuf compiler
-        run: brew install protobuf
-
       - name: Install Rust toolchain
         uses: dtolnay/rust-toolchain@stable
         with:

.github/workflows/rust_binaries_release.yml

@@ -1,4 +1,4 @@
-name: "rust: build versioned volume server binaries"
+name: "rust: build versioned binaries"
 
 on:
   push:
@@ -28,9 +28,6 @@ jobs:
     steps:
       - uses: actions/checkout@v7
 
-      - name: Install protobuf compiler
-        run: sudo apt-get update && sudo apt-get install -y protobuf-compiler
-
       - name: Install Rust toolchain
         uses: dtolnay/rust-toolchain@stable
         with:
@@ -116,6 +113,102 @@ jobs:
             weed-volume_${{ matrix.asset_suffix }}.tar.gz
             weed-volume_${{ matrix.asset_suffix }}.tar.gz.md5
 
+  # The Rust maintenance worker: Linux only, because it runs beside the cluster
+  # it maintains rather than on a laptop, and its dependency tree (lance, arrow,
+  # datafusion) makes every extra target an expensive build.
+  build-rust-worker-linux:
+    permissions:
+      contents: write
+    runs-on: ubuntu-22.04
+    strategy:
+      matrix:
+        include:
+          - target: x86_64-unknown-linux-gnu
+            asset_suffix: linux_amd64
+          - target: aarch64-unknown-linux-gnu
+            asset_suffix: linux_arm64
+            cross: true
+
+    steps:
+      - uses: actions/checkout@v7
+        with:
+          # The upload step is handed a token explicitly; a cargo build script
+          # should not find another one sitting in the checkout's git config.
+          persist-credentials: false
+
+      - name: Install Rust toolchain
+        uses: dtolnay/rust-toolchain@stable
+        with:
+          targets: ${{ matrix.target }}
+
+      - name: Install cross-compilation tools
+        if: matrix.cross
+        run: |
+          sudo dpkg --add-architecture arm64
+          sudo sed -i 's/^deb /deb [arch=amd64] /' /etc/apt/sources.list
+          echo "deb [arch=arm64] http://ports.ubuntu.com/ jammy main restricted universe multiverse" | sudo tee /etc/apt/sources.list.d/arm64.list
+          echo "deb [arch=arm64] http://ports.ubuntu.com/ jammy-updates main restricted universe multiverse" | sudo tee -a /etc/apt/sources.list.d/arm64.list
+          sudo apt-get update
+          sudo apt-get install -y gcc-aarch64-linux-gnu
+          echo "CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc" >> "$GITHUB_ENV"
+
+      - name: Cache cargo registry and target
+        uses: actions/cache@v6
+        with:
+          path: |
+            ~/.cargo/registry
+            ~/.cargo/git
+            seaweed-worker/target/${{ matrix.target }}/release
+          key: rust-worker-release-${{ matrix.target }}-${{ hashFiles('seaweed-worker/Cargo.lock') }}
+          restore-keys: |
+            rust-worker-release-${{ matrix.target }}-
+
+      # lance's build scripts compile their own protos and look for a protoc.
+      # Point them at the one protoc-bin-vendored ships, which seaweed-worker's
+      # own build already uses, so no job depends on a system package and every
+      # build sees the same version.
+      - name: Use the vendored protoc
+        run: |
+          cd seaweed-worker
+          cargo fetch
+          # The version from the lock, not whatever else a restored cache holds.
+          version=$(awk '/^name = "protoc-bin-vendored-linux-x86_64"$/{found=1; next} found && /^version = /{gsub(/"/,"",$3); print $3; exit}' Cargo.lock)
+          test -n "$version" || { echo "protoc-bin-vendored-linux-x86_64 is not in Cargo.lock" >&2; exit 1; }
+          protoc=$(find ~/.cargo/registry/src -path "*protoc-bin-vendored-linux-x86_64-$version/bin/protoc" | head -1)
+          test -x "$protoc" || { echo "no vendored protoc $version in the registry" >&2; exit 1; }
+          echo "PROTOC=$protoc" >> "$GITHUB_ENV"
+
+      - name: Build the Rust maintenance worker
+        run: |
+          cd seaweed-worker
+          cargo build --release -p weed-lance-worker --target ${{ matrix.target }}
+
+      - name: Package binary
+        run: |
+          cp seaweed-worker/target/${{ matrix.target }}/release/weed-worker weed-worker
+          tar czf weed-worker_${{ matrix.asset_suffix }}.tar.gz weed-worker
+          rm weed-worker
+          md5sum weed-worker_${{ matrix.asset_suffix }}.tar.gz > weed-worker_${{ matrix.asset_suffix }}.tar.gz.md5
+
+      - name: Upload release assets
+        if: startsWith(github.ref, 'refs/tags/')
+        uses: softprops/action-gh-release@v3
+        with:
+          files: |
+            weed-worker_${{ matrix.asset_suffix }}.tar.gz
+            weed-worker_${{ matrix.asset_suffix }}.tar.gz.md5
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Upload artifacts
+        if: ${{ !startsWith(github.ref, 'refs/tags/') }}
+        uses: actions/upload-artifact@v7
+        with:
+          name: rust-worker-${{ matrix.asset_suffix }}
+          path: |
+            weed-worker_${{ matrix.asset_suffix }}.tar.gz
+            weed-worker_${{ matrix.asset_suffix }}.tar.gz.md5
+
   build-rust-volume-darwin:
     permissions:
       contents: write
@@ -131,9 +224,6 @@ jobs:
     steps:
       - uses: actions/checkout@v7
 
-      - name: Install protobuf compiler
-        run: brew install protobuf
-
       - name: Install Rust toolchain
         uses: dtolnay/rust-toolchain@stable
         with:
@@ -211,9 +301,6 @@ jobs:
     steps:
       - uses: actions/checkout@v7
 
-      - name: Install protobuf compiler
-        run: choco install protoc -y
-
       - name: Install Rust toolchain
         uses: dtolnay/rust-toolchain@stable
 

.github/workflows/s3-example-integration-tests.yml

@@ -64,6 +64,14 @@ jobs:
           echo "=== Running S3 Empty Directory Marker Tests ==="
           go test -v -timeout=180s -run TestS3ListObjectsEmptyDirectoryMarkers ./...
 
+      - name: Run S3 Prefix Object Tests
+        timeout-minutes: 15
+        working-directory: test/s3/normal
+        run: |
+          set -x
+          echo "=== Running S3 Prefix Object Tests ==="
+          go test -v -timeout=180s -run TestS3PrefixObjectKeys ./...
+
       - name: Run IAM Integration Tests
         timeout-minutes: 15
         working-directory: test/s3/normal

.github/workflows/s3-parquet-tests.yml

@@ -41,7 +41,7 @@ jobs:
       - name: Set up Go
         uses: actions/setup-go@v7
         with:
-          go-version: ^1.25
+          go-version: ^1.26
           cache: true
       
       - name: Set up Python ${{ matrix.python-version }}
@@ -148,7 +148,7 @@ jobs:
       - name: Set up Go
         uses: actions/setup-go@v7
         with:
-          go-version: ^1.25
+          go-version: ^1.26
           cache: true
       
       - name: Run Go unit tests

.github/workflows/s3-snowflake-tests.yml

@@ -0,0 +1,106 @@
+name: "Snowflake S3Compat API tests"
+
+on:
+  push:
+    branches: [ master ]
+    paths:
+      - 'weed/s3api/**'
+      - 'weed/filer/**'
+      - 'weed/server/**'
+      - 'weed/iam/**'
+      - 'weed/command/**'
+      - 'weed/storage/**'
+      - 'weed/operation/**'
+      - 'weed/wdclient/**'
+      - 'weed/cluster/**'
+      - 'weed/pb/**'
+      - 'test/s3/snowflake/**'
+      - 'go.mod'
+      - 'go.sum'
+      - '.github/workflows/s3-snowflake-tests.yml'
+  pull_request:
+    branches: [ master ]
+    paths:
+      - 'weed/s3api/**'
+      - 'weed/filer/**'
+      - 'weed/server/**'
+      - 'weed/iam/**'
+      - 'weed/command/**'
+      - 'weed/storage/**'
+      - 'weed/operation/**'
+      - 'weed/wdclient/**'
+      - 'weed/cluster/**'
+      - 'weed/pb/**'
+      - 'test/s3/snowflake/**'
+      - 'go.mod'
+      - 'go.sum'
+      - '.github/workflows/s3-snowflake-tests.yml'
+
+concurrency:
+  group: ${{ github.event.pull_request.number || github.ref }}/s3-snowflake-tests
+  cancel-in-progress: true
+
+permissions:
+  contents: read
+
+jobs:
+  snowflake-s3compat-tests:
+    name: Snowflake S3Compat API tests
+    runs-on: ubuntu-22.04
+    timeout-minutes: 30
+    env:
+      WORK_DIR: /tmp/seaweedfs-snowflake-tests
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v7
+        with:
+          persist-credentials: false
+
+      - name: Set up Go
+        uses: actions/setup-go@v7
+        with:
+          go-version-file: 'go.mod'
+        id: go
+
+      - name: Set up Java
+        uses: actions/setup-java@v6
+        with:
+          java-version: '17'
+          distribution: 'temurin'
+          cache: 'maven'
+
+      - name: Install SeaweedFS
+        run: |
+          cd weed
+          go install -buildvcs=false
+          weed version
+
+      - name: Run Snowflake S3Compat API tests
+        timeout-minutes: 20
+        run: |
+          # Starts weed server, creates the buckets/objects the suite needs,
+          # clones the upstream suite, and runs mvn -Dtest=S3CompatApiTest.
+          bash test/s3/snowflake/run.sh
+
+      - name: Show logs on failure
+        if: failure()
+        run: |
+          echo "=== SeaweedFS Server Log ==="
+          tail -200 "$WORK_DIR/weed.log" || echo "No server log"
+          echo ""
+          echo "=== Surefire results ==="
+          cat "$WORK_DIR"/snowflake-s3compat-api-test-suite/s3compatapi/target/surefire-reports/*.txt 2>/dev/null || echo "No surefire reports"
+
+      - name: Upload test results
+        if: always()
+        uses: actions/upload-artifact@v7
+        with:
+          name: snowflake-s3compat-surefire-reports
+          path: /tmp/seaweedfs-snowflake-tests/snowflake-s3compat-api-test-suite/s3compatapi/target/surefire-reports/
+          retention-days: 14
+
+      - name: Cleanup
+        if: always()
+        run: |
+          pkill -9 -f "weed server" || true
+          rm -rf "$WORK_DIR" || true

.github/workflows/s3-tables-tests.yml

@@ -6,6 +6,7 @@ on:
       - 'weed/s3api/**'
       - 'weed/filer/**'
       - 'weed/server/**'
+      - 'weed/worker/tasks/iceberg/**'
       - 'test/s3tables/**'
       - 'go.mod'
       - 'go.sum'
@@ -354,9 +355,19 @@ jobs:
           retention-days: 3
 
   clickhouse-iceberg-catalog-tests:
-    name: ClickHouse Iceberg Catalog Integration Tests
+    name: ClickHouse Iceberg Catalog Integration Tests (${{ matrix.tag }})
     runs-on: ubuntu-22.04
     timeout-minutes: 30
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          # Pinned baseline, and latest so new ClickHouse releases are
+          # exercised without a code change.
+          - clickhouse-image: clickhouse/clickhouse-server:25.8
+            tag: "25.8"
+          - clickhouse-image: clickhouse/clickhouse-server:latest
+            tag: latest
 
     steps:
       - name: Check out code
@@ -376,7 +387,7 @@ jobs:
       - name: Pre-pull images
         run: |
           pull() { for i in 1 2 3; do docker pull "$1" && return 0; sleep 15; done; return 1; }
-          pull clickhouse/clickhouse-server:25.8
+          pull ${{ matrix.clickhouse-image }}
           pull python:3.11-slim
 
       - name: Run go mod tidy
@@ -389,6 +400,8 @@ jobs:
       - name: Run ClickHouse Iceberg Catalog Integration Tests
         timeout-minutes: 25
         working-directory: test/s3tables/catalog_clickhouse
+        env:
+          CLICKHOUSE_IMAGE: ${{ matrix.clickhouse-image }}
         run: |
           set -x
           set -o pipefail
@@ -422,10 +435,121 @@ jobs:
         if: failure()
         uses: actions/upload-artifact@v7
         with:
-          name: clickhouse-iceberg-catalog-test-logs
+          name: clickhouse-iceberg-catalog-test-logs-${{ matrix.tag }}
           path: test/s3tables/catalog_clickhouse/test-output.log
           retention-days: 3
 
+  olake-iceberg-catalog-tests:
+    name: OLake Iceberg Catalog Integration Tests (${{ matrix.tag }})
+    runs-on: ubuntu-22.04
+    timeout-minutes: 30
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          # Pinned baseline, and latest so new OLake releases are exercised
+          # without a code change. OLake's Iceberg writer is a Java sidecar
+          # whose Iceberg version moves independently of the Go release, so
+          # the latest leg is the one that catches library drift.
+          - olake-image: olakego/source-postgres:v0.10.1
+            tag: "v0.10.1"
+          - olake-image: olakego/source-postgres:latest
+            tag: latest
+
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v7
+
+      - name: Set up Go
+        uses: actions/setup-go@v7
+        with:
+          go-version-file: 'go.mod'
+        id: go
+
+      - name: Configure Docker Hub mirror
+        run: |
+          echo '{"registry-mirrors": ["https://mirror.gcr.io"]}' | sudo tee /etc/docker/daemon.json
+          sudo systemctl restart docker
+
+      - name: Pre-pull images
+        run: |
+          pull() { for i in 1 2 3; do docker pull "$1" && return 0; sleep 15; done; return 1; }
+          pull ${{ matrix.olake-image }}
+          pull postgres:16
+          pull python:3.11-slim
+
+      - name: Run go mod tidy
+        run: go mod tidy
+
+      - name: Install SeaweedFS
+        run: |
+          go install -buildvcs=false ./weed
+
+      - name: Run OLake Iceberg Catalog Integration Tests
+        timeout-minutes: 25
+        working-directory: test/s3tables/catalog_olake
+        env:
+          OLAKE_IMAGE: ${{ matrix.olake-image }}
+        run: |
+          set -x
+          set -o pipefail
+          echo "=== System Information ==="
+          uname -a
+          free -h
+          df -h
+          docker info
+          echo "=== Starting OLake Iceberg Catalog Tests ==="
+
+          go test -v -timeout 20m . 2>&1 | tee test-output.log || {
+            echo "OLake Iceberg catalog integration tests failed"
+            exit 1
+          }
+
+      # The suite skips itself when Docker is unavailable, so a green job is not
+      # by itself evidence that anything ran. Assert execution explicitly.
+      - name: Assert the suite actually ran
+        working-directory: test/s3tables/catalog_olake
+        run: |
+          log=test-output.log
+          if [ ! -f "$log" ]; then
+            echo "::error::no test-output.log; the suite did not run"
+            exit 1
+          fi
+          passes=$(grep -c '^--- PASS' "$log" || true)
+          skips=$(grep -c '^--- SKIP' "$log" || true)
+          echo "top-level PASS=$passes SKIP=$skips"
+          if [ "$skips" -gt 0 ]; then
+            echo "::error::the OLake suite skipped $skips top-level test(s); the environment it needs was not provisioned, so this job proves nothing"
+            grep '^--- SKIP' "$log" | head -20
+            exit 1
+          fi
+          if [ "$passes" -lt 1 ]; then
+            echo "::error::the OLake suite recorded no passing top-level test"
+            exit 1
+          fi
+
+      - name: Show test output on failure
+        if: failure()
+        working-directory: test/s3tables/catalog_olake
+        run: |
+          echo "=== Test Output ==="
+          if [ -f test-output.log ]; then
+            tail -200 test-output.log
+          fi
+
+          echo "=== Process information ==="
+          ps aux | grep -E "(weed|test|docker|olake|postgres)" || true
+          echo "=== Containers ==="
+          docker ps -a | head -30 || true
+
+      - name: Upload test logs on failure
+        if: failure()
+        uses: actions/upload-artifact@v7
+        with:
+          name: olake-iceberg-catalog-test-logs-${{ matrix.tag }}
+          path: test/s3tables/catalog_olake/test-output.log
+          retention-days: 3
+
   polaris-integration-tests:
     name: Polaris Integration Tests
     runs-on: ubuntu-22.04
@@ -862,6 +986,293 @@ jobs:
           path: test/s3tables/unity_catalog/test-output.log
           retention-days: 3
 
+  lancedb-namespace-tests:
+    name: LanceDB Namespace Integration Tests
+    runs-on: ubuntu-22.04
+    timeout-minutes: 30
+
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v7
+
+      - name: Set up Go
+        uses: actions/setup-go@v7
+        with:
+          go-version-file: 'go.mod'
+        id: go
+
+      - name: Configure Docker Hub mirror
+        run: |
+          echo '{"registry-mirrors": ["https://mirror.gcr.io"]}' | sudo tee /etc/docker/daemon.json
+          sudo systemctl restart docker
+
+      - name: Pre-pull images
+        run: |
+          pull() { for i in 1 2 3; do docker pull "$1" && return 0; sleep 15; done; return 1; }
+          pull python:3.11-slim
+
+      - name: Run go mod tidy
+        run: go mod tidy
+
+      - name: Build SeaweedFS
+        run: |
+          cd weed && go build -buildvcs=false .
+
+      - name: Run LanceDB Namespace Integration Tests
+        timeout-minutes: 25
+        working-directory: test/s3tables/catalog_lancedb
+        run: |
+          set -x
+          set -o pipefail
+          echo "=== System Information ==="
+          uname -a
+          free -h
+          df -h
+          docker info
+          echo "=== Starting LanceDB Namespace Tests ==="
+
+          go test -v -timeout 20m . 2>&1 | tee test-output.log || {
+            echo "LanceDB namespace integration tests failed"
+            exit 1
+          }
+
+      - name: Show test output on failure
+        if: failure()
+        working-directory: test/s3tables/catalog_lancedb
+        run: |
+          echo "=== Test Output ==="
+          if [ -f test-output.log ]; then
+            tail -200 test-output.log
+          fi
+
+          echo "=== Process information ==="
+          ps aux | grep -E "(weed|test|docker)" || true
+
+      - name: Upload test logs on failure
+        if: failure()
+        uses: actions/upload-artifact@v7
+        with:
+          name: lancedb-namespace-test-logs
+          path: test/s3tables/catalog_lancedb/test-output.log
+          retention-days: 3
+
+  table-lifecycle-tests:
+    name: Table Lifecycle Integration Tests
+    runs-on: ubuntu-22.04
+    timeout-minutes: 40
+
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v7
+
+      - name: Set up Go
+        uses: actions/setup-go@v7
+        with:
+          go-version-file: 'go.mod'
+        id: go
+
+      - name: Configure Docker Hub mirror
+        run: |
+          echo '{"registry-mirrors": ["https://mirror.gcr.io"]}' | sudo tee /etc/docker/daemon.json
+          sudo systemctl restart docker
+
+      - name: Pre-pull images
+        run: |
+          pull() { for i in 1 2 3; do docker pull "$1" && return 0; sleep 15; done; return 1; }
+          pull python:3.11-slim
+          pull duckdb/duckdb:latest
+
+      - name: Run go mod tidy
+        run: go mod tidy
+
+      - name: Build SeaweedFS
+        run: |
+          cd weed && go build -buildvcs=false .
+
+      - name: Run Table Lifecycle Integration Tests
+        timeout-minutes: 35
+        working-directory: test/s3tables/lifecycle
+        env:
+          # The Rust worker's own tests cover its handlers; a cold build of the
+          # lance crate costs more here than the layer it would be checking.
+          WEED_LANCE_MAINTENANCE: library
+        run: |
+          set -x
+          set -o pipefail
+          go test -v -timeout 30m . 2>&1 | tee test-output.log || {
+            echo "Table lifecycle integration tests failed"
+            exit 1
+          }
+
+      - name: Show test output on failure
+        if: failure()
+        working-directory: test/s3tables/lifecycle
+        run: |
+          echo "=== Test Output ==="
+          if [ -f test-output.log ]; then
+            tail -200 test-output.log
+          fi
+
+          echo "=== Process information ==="
+          ps aux | grep -E "(weed|test|docker)" || true
+
+      - name: Upload test logs on failure
+        if: failure()
+        uses: actions/upload-artifact@v7
+        with:
+          name: table-lifecycle-test-logs
+          path: test/s3tables/lifecycle/test-output.log
+          retention-days: 3
+
+  duckdb-lance-tests:
+    name: DuckDB Lance Integration Tests
+    runs-on: ubuntu-22.04
+    timeout-minutes: 30
+
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v7
+        with:
+          # The job uploads a test log on failure; nothing here needs to push,
+          # so do not leave a token in the checkout for it to pick up.
+          persist-credentials: false
+
+      - name: Set up Go
+        uses: actions/setup-go@v7
+        with:
+          go-version-file: 'go.mod'
+        id: go
+
+      - name: Configure Docker Hub mirror
+        run: |
+          echo '{"registry-mirrors": ["https://mirror.gcr.io"]}' | sudo tee /etc/docker/daemon.json
+          sudo systemctl restart docker
+
+      - name: Pre-pull images
+        run: |
+          pull() { for i in 1 2 3; do docker pull "$1" && return 0; sleep 15; done; return 1; }
+          pull duckdb/duckdb:latest
+          pull python:3.11-slim
+
+      - name: Run go mod tidy
+        run: go mod tidy
+
+      - name: Build SeaweedFS
+        run: |
+          cd weed && go build -buildvcs=false .
+
+      - name: Run DuckDB Lance Integration Tests
+        timeout-minutes: 25
+        working-directory: test/s3tables/catalog_duckdb_lance
+        run: |
+          set -x
+          set -o pipefail
+          echo "=== System Information ==="
+          uname -a
+          free -h
+          df -h
+          docker info
+          echo "=== Starting DuckDB Lance Tests ==="
+
+          go test -v -timeout 20m . 2>&1 | tee test-output.log || {
+            echo "DuckDB Lance integration tests failed"
+            exit 1
+          }
+
+      - name: Show test output on failure
+        if: failure()
+        working-directory: test/s3tables/catalog_duckdb_lance
+        run: |
+          echo "=== Test Output ==="
+          if [ -f test-output.log ]; then
+            tail -200 test-output.log
+          fi
+
+          echo "=== Process information ==="
+          ps aux | grep -E "(weed|test|docker|duckdb)" || true
+
+      - name: Upload test logs on failure
+        if: failure()
+        uses: actions/upload-artifact@v7
+        with:
+          name: duckdb-lance-test-logs
+          path: test/s3tables/catalog_duckdb_lance/test-output.log
+          retention-days: 3
+
+  spark-lance-namespace-tests:
+    name: Spark Lance Namespace Integration Tests
+    runs-on: ubuntu-22.04
+    timeout-minutes: 40
+
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v7
+        with:
+          # The job uploads a test log on failure; nothing here needs to push,
+          # so do not leave a token in the checkout for it to pick up.
+          persist-credentials: false
+
+      - name: Set up Go
+        uses: actions/setup-go@v7
+        with:
+          go-version-file: 'go.mod'
+        id: go
+
+      - name: Configure Docker Hub mirror
+        run: |
+          echo '{"registry-mirrors": ["https://mirror.gcr.io"]}' | sudo tee /etc/docker/daemon.json
+          sudo systemctl restart docker
+
+      - name: Pre-pull images
+        run: |
+          pull() { for i in 1 2 3; do docker pull "$1" && return 0; sleep 15; done; return 1; }
+          pull apache/spark:3.5.1
+
+      - name: Run go mod tidy
+        run: go mod tidy
+
+      - name: Build SeaweedFS
+        run: |
+          cd weed && go build -buildvcs=false .
+
+      - name: Run Spark Lance Namespace Integration Tests
+        timeout-minutes: 35
+        working-directory: test/s3tables/catalog_spark_lance
+        run: |
+          set -x
+          set -o pipefail
+          echo "=== System Information ==="
+          uname -a
+          free -h
+          df -h
+          docker info
+          echo "=== Starting Spark Lance Namespace Tests ==="
+
+          go test -v -timeout 30m . 2>&1 | tee test-output.log || {
+            echo "Spark Lance namespace integration tests failed"
+            exit 1
+          }
+
+      - name: Show test output on failure
+        if: failure()
+        working-directory: test/s3tables/catalog_spark_lance
+        run: |
+          echo "=== Test Output ==="
+          if [ -f test-output.log ]; then
+            tail -200 test-output.log
+          fi
+
+          echo "=== Process information ==="
+          ps aux | grep -E "(weed|test|docker|spark)" || true
+
+      - name: Upload test logs on failure
+        if: failure()
+        uses: actions/upload-artifact@v7
+        with:
+          name: spark-lance-namespace-test-logs
+          path: test/s3tables/catalog_spark_lance/test-output.log
+          retention-days: 3
+
   s3-tables-build-verification:
     name: S3 Tables Build Verification
     runs-on: ubuntu-22.04

.github/workflows/s3tests.yml

@@ -128,6 +128,9 @@ jobs:
           echo "All SeaweedFS components are ready!"
           cd ../s3-tests
           sed -i "s/assert prefixes == \['foo%2B1\/', 'foo\/', 'quux%20ab\/'\]/assert prefixes == \['foo\/', 'foo%2B1\/', 'quux%20ab\/'\]/" s3tests/functional/test_s3.py
+          # The suite expects RGW's 400 InvalidPart for a partNumber past the last
+          # part; AWS answers 416 InvalidPartNumber, which is what we return.
+          sed -i "/# request PartNumber out of range/,+4{s/assert status == 400/assert status == 416/; s/assert error_code == 'InvalidPart'/assert error_code == 'InvalidPartNumber'/}" s3tests/functional/test_s3.py
           
           # Debug: Show the config file contents
           echo "=== S3 Config File Contents ==="
@@ -286,6 +289,7 @@ jobs:
           s3tests/functional/test_s3.py::test_object_write_check_etag \
           s3tests/functional/test_s3.py::test_object_write_cache_control \
           s3tests/functional/test_s3.py::test_object_write_expires \
+          s3tests/functional/test_s3.py::test_object_content_encoding_aws_chunked \
           s3tests/functional/test_s3.py::test_object_write_read_update_read_delete \
           s3tests/functional/test_s3.py::test_object_metadata_replaced_on_put \
           s3tests/functional/test_s3.py::test_object_write_file \
@@ -308,6 +312,7 @@ jobs:
           s3tests/functional/test_s3.py::test_get_object_ifmodifiedsince_good \
           s3tests/functional/test_s3.py::test_get_object_ifmodifiedsince_failed \
           s3tests/functional/test_s3.py::test_get_object_ifunmodifiedsince_failed \
+          s3tests/functional/test_s3.py::test_get_checksum_object_attributes \
           s3tests/functional/test_s3.py::test_bucket_head \
           s3tests/functional/test_s3.py::test_bucket_head_notexist \
           s3tests/functional/test_s3.py::test_object_raw_authenticated \
@@ -968,6 +973,9 @@ jobs:
           echo "All SeaweedFS components are ready!"
           cd ../s3-tests
           sed -i "s/assert prefixes == \['foo%2B1\/', 'foo\/', 'quux%20ab\/'\]/assert prefixes == \['foo\/', 'foo%2B1\/', 'quux%20ab\/'\]/" s3tests/functional/test_s3.py
+          # The suite expects RGW's 400 InvalidPart for a partNumber past the last
+          # part; AWS answers 416 InvalidPartNumber, which is what we return.
+          sed -i "/# request PartNumber out of range/,+4{s/assert status == 400/assert status == 416/; s/assert error_code == 'InvalidPart'/assert error_code == 'InvalidPartNumber'/}" s3tests/functional/test_s3.py
           # Create and update s3tests.conf to use port 8004
           cp ../docker/compose/s3tests.conf ../docker/compose/s3tests-sql.conf
           sed -i 's/port = 8000/port = 8004/g' ../docker/compose/s3tests-sql.conf
@@ -1143,6 +1151,7 @@ jobs:
           s3tests/functional/test_s3.py::test_object_write_check_etag \
           s3tests/functional/test_s3.py::test_object_write_cache_control \
           s3tests/functional/test_s3.py::test_object_write_expires \
+          s3tests/functional/test_s3.py::test_object_content_encoding_aws_chunked \
           s3tests/functional/test_s3.py::test_object_write_read_update_read_delete \
           s3tests/functional/test_s3.py::test_object_metadata_replaced_on_put \
           s3tests/functional/test_s3.py::test_object_write_file \
@@ -1165,6 +1174,7 @@ jobs:
           s3tests/functional/test_s3.py::test_get_object_ifmodifiedsince_good \
           s3tests/functional/test_s3.py::test_get_object_ifmodifiedsince_failed \
           s3tests/functional/test_s3.py::test_get_object_ifunmodifiedsince_failed \
+          s3tests/functional/test_s3.py::test_get_checksum_object_attributes \
           s3tests/functional/test_s3.py::test_bucket_head \
           s3tests/functional/test_s3.py::test_bucket_head_notexist \
           s3tests/functional/test_s3.py::test_object_raw_authenticated \

.github/workflows/spark-integration-tests.yml

@@ -34,7 +34,7 @@ jobs:
       uses: actions/checkout@v7
 
     - name: Set up JDK 11
-      uses: actions/setup-java@v5
+      uses: actions/setup-java@v6
       with:
         java-version: '11'
         distribution: 'temurin'

.github/workflows/star_history.yml

@@ -0,0 +1,74 @@
+---
+name: Star History
+
+on:
+  schedule:
+    - cron: "0 0 * * *"  # daily, 00:00 UTC
+  workflow_dispatch:
+
+permissions:
+  contents: write
+
+concurrency:
+  # Only one chart regeneration per branch at a time; a newer run on the same
+  # branch cancels an in-flight one so overlapping runs never conflict on
+  # note/star_history.svg during rebase. Scoped by ref so a manual run on
+  # another branch can't cancel the daily master update.
+  group: star-history-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  render:
+    name: Regenerate star history chart
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v7
+        with:
+          # Full history so the chart commit can rebase onto a moved master.
+          fetch-depth: 0
+
+      - name: Set up Python
+        uses: actions/setup-python@v7
+        with:
+          python-version: "3.x"
+          cache: pip
+          cache-dependency-path: .github/scripts/star_history.py
+
+      - name: Install matplotlib
+        run: pip install matplotlib
+
+      - name: Render chart
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: python .github/scripts/star_history.py
+
+      - name: Commit if changed
+        run: |
+          git config user.name "github-actions[bot]"
+          git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
+          if git diff --quiet -- note/star_history.svg; then
+            echo "No changes to the chart."
+            exit 0
+          fi
+          git add note/star_history.svg
+          git commit -m "docs: regenerate star history chart"
+          # Rebase and retry so a concurrent push to master doesn't lose the chart.
+          for attempt in 1 2 3 4 5; do
+            if [ "$attempt" -gt 1 ]; then
+              # Guard the rebase: a transient fetch error or conflict must not
+              # abort the fail-fast shell before the remaining attempts run.
+              if ! git pull --rebase origin "$GITHUB_REF_NAME"; then
+                echo "rebase failed (attempt ${attempt}); aborting and retrying"
+                git rebase --abort || true
+                continue
+              fi
+            fi
+            if git push origin HEAD:"$GITHUB_REF_NAME"; then
+              exit 0
+            fi
+            echo "push rejected (attempt ${attempt}); will rebase and retry"
+          done
+          echo "::error::could not push star history chart after retries"
+          exit 1

.github/workflows/test-s3-over-https-using-awscli.yml

@@ -36,7 +36,7 @@ jobs:
 
       - uses: actions/setup-go@v7
         with:
-          go-version: ^1.25
+          go-version: ^1.26
 
       - name: Build SeaweedFS
         run: |

.github/workflows/tls-rotation-tests.yml

@@ -30,7 +30,7 @@ jobs:
     - name: Set up Go 1.x
       uses: actions/setup-go@v7
       with:
-        go-version: ^1.25
+        go-version: ^1.26
       id: go
 
     - name: Check out code into the Go module directory

.github/workflows/vacuum-integration-tests.yml

@@ -30,7 +30,7 @@ jobs:
     - name: Set up Go 1.x
       uses: actions/setup-go@v7
       with:
-        go-version: ^1.25
+        go-version: ^1.26
       id: go
 
     - name: Check out code into the Go module directory

.gitignore

@@ -116,7 +116,6 @@ test/s3/versioning/weed-test.log
 /docker/admin_integration/data
 docker/agent_pub_record
 docker/admin_integration/weed-local
-/seaweedfs-rdma-sidecar/bin
 /test/s3/encryption/filerldb2
 /test/s3/sse/filerldb2
 test/s3/sse/weed-test.log

README.md

@@ -12,425 +12,258 @@
 
 ![SeaweedFS Logo](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/seaweedfs.png)
 
-<h2 align="center"><a href="https://www.patreon.com/seaweedfs">Sponsor SeaweedFS via Patreon</a></h2>
+SeaweedFS is a simple and highly scalable distributed file system. There are two objectives:
 
-SeaweedFS is an independent Apache-licensed open source project with its ongoing development made
-possible entirely thanks to the support of these awesome [backers](https://github.com/seaweedfs/seaweedfs/blob/master/backers.md).
-If you'd like to grow SeaweedFS even stronger, please consider joining our
-<a href="https://www.patreon.com/seaweedfs">sponsors on Patreon</a>.
+1. to store billions of files!
+2. to serve the files fast!
 
-Your support will be really appreciated by me and other supporters!
-
-<!--
-<h4 align="center">Platinum</h4>
-
-<p align="center">
-  <a href="" target="_blank">
-    Add your name or icon here
-  </a>
-</p>
--->
-
-### Gold Sponsors
-[![nodion](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/sponsor_nodion.png)](https://www.nodion.com)
-[![piknik](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/piknik.png)](https://www.piknik.com)
-[![keepsec](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/keepsec.png)](https://www.keepsec.ca)
-[![zyner](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/sponsor_zyner.png)](https://zyner.org)
-
----
+One `weed` binary serves an S3 object store, a POSIX file system, and a lakehouse with S3 Tables, all over the same data. Each blob is one disk read away, capacity grows by starting another volume server, and cloud storage can be cached or tiered transparently. Both read and write operations have O(1) complexity and can run at the full speed supported by the underlying hardware.
 
 - [Download Binaries for different platforms](https://github.com/seaweedfs/seaweedfs/releases/latest)
-- [SeaweedFS on Slack](https://join.slack.com/t/seaweedfs/shared_invite/enQtMzI4MTMwMjU2MzA3LTEyYzZmZWYzOGQ3MDJlZWMzYmI0OTE4OTJiZjJjODBmMzUxNmYwODg0YjY3MTNlMjBmZDQ1NzQ5NDJhZWI2ZmY)
-- [SeaweedFS on Twitter](https://twitter.com/SeaweedFS)
-- [SeaweedFS on Telegram](https://t.me/Seaweedfs) 
-- [SeaweedFS on Reddit](https://www.reddit.com/r/SeaweedFS/)
-- [SeaweedFS Mailing List](https://groups.google.com/d/forum/seaweedfs)
 - [Wiki Documentation](https://github.com/seaweedfs/seaweedfs/wiki)
-- [SeaweedFS White Paper](https://github.com/seaweedfs/seaweedfs/wiki/SeaweedFS_Architecture.pdf)
-- [SeaweedFS Introduction Slides 2025.5](https://docs.google.com/presentation/d/1tdkp45J01oRV68dIm4yoTXKJDof-EhainlA0LMXexQE/edit?usp=sharing)
-- [SeaweedFS Introduction Slides 2021.5](https://docs.google.com/presentation/d/1DcxKWlINc-HNCjhYeERkpGXXm6nTCES8mi2W5G0Z4Ts/edit?usp=sharing)
-- [SeaweedFS Introduction Slides 2019.3](https://www.slideshare.net/chrislusf/seaweedfs-introduction)
+- [HTTP REST API](REST_API.md) for the filer, master, and volume servers
+- Community: [Slack](https://join.slack.com/t/seaweedfs/shared_invite/enQtMzI4MTMwMjU2MzA3LTEyYzZmZWYzOGQ3MDJlZWMzYmI0OTE4OTJiZjJjODBmMzUxNmYwODg0YjY3MTNlMjBmZDQ1NzQ5NDJhZWI2ZmY), [Twitter](https://twitter.com/SeaweedFS), [Telegram](https://t.me/Seaweedfs), [Reddit](https://www.reddit.com/r/SeaweedFS/), [Mailing List](https://groups.google.com/d/forum/seaweedfs)
+- [SeaweedFS White Paper](https://github.com/seaweedfs/seaweedfs/wiki/SeaweedFS_Architecture.pdf) and introduction slides: [2025.5](https://docs.google.com/presentation/d/1tdkp45J01oRV68dIm4yoTXKJDof-EhainlA0LMXexQE/edit?usp=sharing), [2021.5](https://docs.google.com/presentation/d/1DcxKWlINc-HNCjhYeERkpGXXm6nTCES8mi2W5G0Z4Ts/edit?usp=sharing), [2019.3](https://www.slideshare.net/chrislusf/seaweedfs-introduction)
 
 Table of Contents
 =================
 
 * [Quick Start](#quick-start)
-    * [Quick Start with weed mini](#quick-start-with-weed-mini)
-    * [Quick Start for S3 API on Docker](#quick-start-for-s3-api-on-docker)
-* [Introduction](#introduction)
-* [Features](#features)
-    * [Additional Features](#additional-features)
-    * [Filer Features](#filer-features)
-* [Example: Using Seaweed Object Store](#example-using-seaweed-object-store)
-* [Architecture](#object-store-architecture)
-* [Compared to Other File Systems](#compared-to-other-file-systems)
+    * [One command](#one-command)
+    * [Docker](#docker)
+    * [Docker Compose](#docker-compose)
+    * [Kubernetes with Helm](#kubernetes-with-helm)
+    * [Build from source](#build-from-source)
+    * [Scale out](#scale-out)
+* [Why SeaweedFS](#why-seaweedfs)
+    * [Fast](#fast)
+    * [Scalable](#scalable)
+    * [The most complete S3 API](#the-most-complete-s3-api)
+    * [A data warehouse with S3 Tables](#a-data-warehouse-with-s3-tables)
+    * [A fast cache for cloud storage](#a-fast-cache-for-cloud-storage)
+    * [Active-active replication and more](#active-active-replication-and-more)
+* [Architecture](#architecture)
+* [Compared to Other Systems](#compared-to-other-systems)
     * [Compared to HDFS](#compared-to-hdfs)
     * [Compared to GlusterFS, Ceph](#compared-to-glusterfs-ceph)
-    * [Compared to GlusterFS](#compared-to-glusterfs)
+    * [Compared to MooseFS](#compared-to-moosefs)
     * [Compared to Ceph](#compared-to-ceph)
-    * [Compared to Minio](#compared-to-minio)
-* [Dev Plan](#dev-plan)
-* [Installation Guide](#installation-guide)
-* [Disk Related Topics](#disk-related-topics)
+    * [Compared to MinIO, RustFS](#compared-to-minio-rustfs)
 * [Benchmark](#benchmark)
 * [Enterprise](#enterprise)
 * [License](#license)
+* [Sponsors](#sponsors)
 
 # Quick Start #
 
+## One command ##
 
-## Quick Start with weed mini ##
+Download the latest binary from the [releases](https://github.com/seaweedfs/seaweedfs/releases/latest) page and unzip the single `weed` (or `weed.exe`) file, or let the install script put it in `/usr/local/bin`:
 
-Download the latest binary from https://github.com/seaweedfs/seaweedfs/releases and unzip the single `weed` (or `weed.exe`) file, or run `go install github.com/seaweedfs/seaweedfs/weed@latest`. Then start a ready-to-use S3 object store with credentials and a pre-created bucket in one command:
+```bash
+curl -fsSL https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/install.sh | bash
+```
+
+Then start a ready-to-use S3 object store:
 
 ```bash
 AWS_ACCESS_KEY_ID=admin \
 AWS_SECRET_ACCESS_KEY=secret \
 S3_BUCKET=my-bucket \
-./weed mini -dir=/data
+./weed mini -dir=./data
 ```
 
-That's it — the S3 endpoint is at http://localhost:8333, `my-bucket` already exists, and `admin`/`secret` are valid credentials. `S3_BUCKET` accepts a comma-separated list (e.g. `raw,processed`); use `S3_TABLE_BUCKET` for S3 Tables (Iceberg) buckets. Drop any of the env vars to skip that piece (no AWS keys → S3 runs in unauthenticated "Allow All" mode for development).
+That's it. The S3 endpoint is at http://localhost:8333, `my-bucket` exists, and `admin`/`secret` are valid credentials:
 
-The same command starts everything else too:
-- **S3 Endpoint**: http://localhost:8333
-- **Master UI**: http://localhost:9333
-- **Volume Server**: http://localhost:9340
-- **Filer UI**: http://localhost:8888
-- **WebDAV**: http://localhost:7333
-- **Admin UI**: http://localhost:23646
+```bash
+AWS_ACCESS_KEY_ID=admin AWS_SECRET_ACCESS_KEY=secret \
+  aws --endpoint-url http://localhost:8333 s3 cp README.md s3://my-bucket/
+```
+
+The same process also runs the master, a volume server, the filer, WebDAV, the Iceberg REST catalog, and the Admin UI. Add `S3_TABLE_BUCKET=warehouse` to also create an Iceberg table bucket, or `warehouse:LANCE` for a Lance one. Drop the AWS keys to run without authentication for development.
 
 > macOS: if the binary is quarantined, run `xattr -d com.apple.quarantine ./weed` first.
 
-Perfect for development, testing, learning SeaweedFS, and single-node deployments. To scale out, add more volume servers by running `weed volume -dir="/some/data/dir2" -master="<master_host>:9333" -port=8081` locally, on another machine, or on thousands of machines.
+`weed mini` is auto-tuned for one node and is fine for single-node production, such as an S3 gateway that issues presigned URLs. See [Quick Start with weed mini][WeedMini].
 
-## Quick Start for S3 API on Docker ##
+## Docker ##
 
 ```bash
-docker run -p 8333:8333 \
+docker run -p 8333:8333 -v weed-data:/data \
   -e AWS_ACCESS_KEY_ID=admin \
   -e AWS_SECRET_ACCESS_KEY=secret \
   -e S3_BUCKET=my-bucket \
   chrislusf/seaweedfs
 ```
 
-Same behavior as the `weed mini` command above — the S3 endpoint is at http://localhost:8333 with `my-bucket` pre-created. Drop the env vars to run anonymously for development.
+Same behavior as the `weed mini` command above.
 
-# Introduction #
+## Docker Compose ##
 
-SeaweedFS is a simple and highly scalable distributed file system. There are two objectives:
+To run master, volume server, filer, S3, and WebDAV as separate services:
 
-1. to store billions of files!
-2. to serve the files fast!
+```bash
+wget https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/docker/seaweedfs-compose.yml
+wget -P prometheus https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/docker/prometheus/prometheus.yml
+docker compose -f seaweedfs-compose.yml -p seaweedfs up
+```
 
-SeaweedFS started as a blob store to handle small files efficiently. 
-Instead of managing all file metadata in a central master, 
-the central master only manages volumes on volume servers, 
-and these volume servers manage files and their metadata. 
-This relieves concurrency pressure from the central master and spreads file metadata into volume servers, 
-allowing faster file access (O(1), usually just one disk read operation).
+[Docker Compose for S3][DockerComposeS3] adds credentials, and the [docker/compose](docker/compose) folder has variants for replication, mounts, message queues, and more.
 
-There is only 40 bytes of disk storage overhead for each file's metadata. 
-It is so simple with O(1) disk reads that you are welcome to challenge the performance with your actual use cases.
+## Kubernetes with Helm ##
 
-SeaweedFS started by implementing [Facebook's Haystack design paper](http://www.usenix.org/event/osdi10/tech/full_papers/Beaver.pdf). 
-Also, SeaweedFS implements erasure coding with ideas from 
-[f4: Facebook’s Warm BLOB Storage System](https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-muralidhar.pdf), and has a lot of similarities with [Facebook’s Tectonic Filesystem](https://www.usenix.org/system/files/fast21-pan.pdf) and [Google's Colossus File System](https://cloud.google.com/blog/products/storage-data-transfer/a-peek-behind-colossus-googles-file-system)
+```bash
+helm repo add seaweedfs https://seaweedfs.github.io/seaweedfs/helm
+helm install seaweedfs seaweedfs/seaweedfs -n seaweedfs --create-namespace -f values.yaml
+```
 
-On top of the blob store, optional [Filer] can support directories and POSIX attributes. 
-Filer is a separate linearly-scalable stateless server with customizable metadata stores, 
-e.g., MySql, Postgres, Redis, Cassandra, HBase, Mongodb, Elastic Search, LevelDB, RocksDB, Sqlite, MemSql, TiDB, Etcd, CockroachDB, YDB, etc.
+A production-shaped `values.yaml` for a three-node cluster: two copies of every write, three masters, and an S3 endpoint with credentials and a bucket.
 
-SeaweedFS can transparently integrate with the cloud. 
-With hot data on local cluster, and warm data on the cloud with O(1) access time, 
-SeaweedFS can achieve both fast local access time and elastic cloud storage capacity.
-What's more, the cloud storage access API cost is minimized. 
-Faster and cheaper than direct cloud storage!
+```yaml
+global:
+  seaweedfs:
+    enableReplication: true
+    replicationPlacement: "001"   # one extra copy on another server; "002" for two
 
-SeaweedFS also ships a built-in **Iceberg REST Catalog**, turning the same cluster into a self-contained lakehouse.
-Spark, Trino, Dremio, DuckDB, and RisingWave can query Iceberg tables directly — no Hive Metastore, Glue, or
-external catalog service required. Storage and table metadata live in one system, simplifying on-prem and
-small-team analytics stacks.
+master:
+  replicas: 3
+  data:
+    type: persistentVolumeClaim   # the cluster's default storage class; add storageClass to pick one
+    size: 1Gi
+
+volume:
+  replicas: 3                     # at least 1 + the sum of the replication digits
+  dataDirs:
+    - name: data
+      type: persistentVolumeClaim
+      size: 500Gi
+      maxVolumes: 0               # size the volume count from the disk
+
+filer:
+  replicas: 2
+  data:
+    type: persistentVolumeClaim
+    size: 20Gi
+
+s3:
+  enabled: true
+  replicas: 2
+  enableAuth: true
+  credentials:
+    admin:
+      accessKey: admin
+      secretKey: change-me
+  createBuckets:
+    - name: app-storage
+```
+
+The S3 endpoint is the `seaweedfs-s3` service on port 8333. [Helm Chart Recipes][HelmRecipes] has values for a development cluster, a lakehouse with the Iceberg catalog exposed, filer metadata on PostgreSQL, and node-local disks. The [SeaweedFS Operator][Operator] and the [CSI driver][SeaweedFsCsiDriver] are the other Kubernetes paths.
+
+## Build from source ##
+
+```bash
+git clone https://github.com/seaweedfs/seaweedfs.git
+cd seaweedfs/weed && make install
+```
+
+`weed` lands in `$GOPATH/bin`. [Getting Started][GettingStarted] covers running master, volume, filer, and S3 as separate processes.
+
+## Scale out ##
+
+Capacity is a volume server. Start one on any machine with disk and point it at the master:
+
+```bash
+weed volume -dir=/data -master=<master_host>:9333
+```
+
+Nothing rebalances until you ask it to. Throughput is a filer or S3 gateway; they are stateless, so run as many as you need behind a load balancer. [Production Setup][ProductionSetup] walks through a multi-node cluster.
 
 [Back to TOC](#table-of-contents)
 
-# Features #
-## Additional Blob Store Features ##
-* Support different replication levels, with rack and data center aware.
-* Automatic master servers failover - no single point of failure (SPOF).
-* Automatic compression depending on file MIME type.
-* Automatic compaction to reclaim disk space after deletion or update.
-* [Automatic entry TTL expiration][VolumeServerTTL].
-* Flexible Capacity Expansion: Any server with some disk space can add to the total storage space.
-* Adding/Removing servers does **not** cause any data re-balancing unless triggered by admin commands.
-* Optional picture resizing.
-* Support ETag, Accept-Range, Last-Modified, etc.
-* Support in-memory/leveldb/readonly mode tuning for memory/performance balance.
-* Support rebalancing the writable and readonly volumes.
-* [Customizable Multiple Storage Tiers][TieredStorage]: Customizable storage disk types to balance performance and cost.
-* [Transparent cloud integration][CloudTier]: unlimited capacity via tiered cloud storage for warm data.
-* [Erasure Coding for warm storage][ErasureCoding]  Rack-Aware 10.4 erasure coding reduces storage cost and increases availability. Enterprise version can customize EC ratio.
+# Why SeaweedFS #
+
+## Fast ##
+
+* One disk read per blob. A small file is one blob; a large file is split into chunks of a few MB, each its own blob. A volume server keeps a 16-byte index entry per blob in memory and reads it in a single seek, also for erasure-coded data.
+* The master is not in the read path. Clients cache the volume-to-server mapping and talk to volume servers directly.
+* 40 bytes of metadata per file on disk. Small files are packed into append-only volume files, so there is no per-file inode, no per-file metadata file, no fragmentation, and writes are SSD friendly.
+* Hot data is replicated; [erasure coding][ErasureCoding] is applied to warm data in the background, so writes never pay the encoding cost.
+* The [Rust volume server][RustVolume] is a drop-in for higher throughput and lower tail latency on the same on-disk format.
+
+On one laptop, [`weed benchmark`][Benchmarks] writes 1KB files at 15,700 per second and reads them back at 47,000 per second, and a mixed S3 [warp][S3Benchmark] run totals 3.2 GiB/s. Numbers are in the [Benchmark](#benchmark) section; throughput grows with volume servers and gateways.
+
+## Scalable ##
+
+* The master tracks volumes, not files. A cluster with billions of files has a few thousand volumes, so the master stays small. One master is enough for most clusters; run three for [Raft failover][FailoverMaster].
+* Adding a server adds capacity with no data reshuffle. Balancing, vacuum, erasure coding, and repair run on demand from [`weed shell`][WeedShell] or the [maintenance worker][Worker].
+* Filer and S3 gateways are stateless and scale linearly. Directory metadata lives in a [store you already run][FilerStores]: LevelDB, RocksDB, SQLite, MySQL, PostgreSQL, Cassandra, HBase, MongoDB, Redis, Elasticsearch, etcd, TiKV, FoundationDB, YDB, ArangoDB, Tarantool, and MySQL or PostgreSQL compatible databases such as TiDB, CockroachDB, and MemSQL.
+* Rack and data center aware [replication][Replication], [tiered storage][TieredStorage] across disk types, and [transparent cloud tiering][CloudTier] for unlimited capacity.
+* Files from a byte to [tens of TB][SuperLargeFiles]. Volumes up to 8TB with the large-disk build.
+
+## The most complete S3 API ##
+
+The S3 gateway implements the object, bucket, S3 Tables, IAM, and STS APIs on one endpoint, so the AWS SDKs and CLI, rclone, restic, Spark, and Trino work unchanged.
+
+| API | Operations |
+| --- | --- |
+| S3 bucket and object | 73 |
+| S3 Tables | 36 |
+| IAM | 39 |
+| STS | 5 |
+
+* [Versioning][Versioning], [Object Lock][ObjectLock] with retention and legal hold, [lifecycle][Lifecycle] rules, tagging, [CORS][CORS], [conditional reads and writes][ConditionalOps], checksums, presigned URLs, browser POST uploads, multipart uploads, and an atomic [RenameObject][RenameObject].
+* [Bucket policies][BucketPolicies] with [conditions][PolicyConditions] and [variables][PolicyVariables]; IAM users, groups, and policies; STS with [OIDC][OIDC], LDAP, and [Kubernetes service accounts][K8sSA].
+* [SSE-S3, SSE-KMS, and SSE-C][SSE] server-side encryption, with OpenBao and Vault, AWS KMS, Azure Key Vault, and GCP KMS as key providers.
+* [Audit log][AuditLog], [bucket quota][BucketQuota], and [rate limiting][RateLimiting].
+* Each bucket is its own collection, so deleting a bucket is instant.
+
+The full operation list is in [Amazon S3 API][AmazonS3API], and [Supported APIs vs MinIO][S3vsMinio] compares. The S3 compatibility suite and the SDK, IAM, SSE, policy, and Spark integration tests run in CI on every change.
+
+## A data warehouse with S3 Tables ##
+
+SeaweedFS is a lakehouse in one system. [S3 Table Buckets][S3TableBucket] hold Apache Iceberg tables by default, or [Lance][LanceCatalog] tables for vectors and multimodal data, and the built-in [Iceberg REST Catalog][IcebergCatalog] and Lance namespace serve them directly. There is no Hive Metastore, Glue, or separate catalog service to deploy, secure, and back up.
+
+* Query engines operate on the same tables at the same time: [Spark][SparkIceberg], [Trino][TrinoIceberg], [Dremio][DremioIceberg], [DuckDB][DuckDBIceberg], [Apache Doris][DorisIceberg], [RisingWave][RisingWaveIceberg], ClickHouse, and [LanceDB][LanceDB]. Catalog commits are atomic compare-and-swap, so concurrent writers are safe. [Lakekeeper][Lakekeeper] can front the same storage with STS-vended credentials.
+* [Automated table maintenance][IcebergMaintenance]: compaction, snapshot expiration, orphan file removal, and manifest rewriting, configured per bucket or table through the S3 Tables maintenance APIs, and the same for [Lance][LanceMaintenance].
+* IAM at the bucket, namespace, and table level with standard bucket policies, see [S3 Tables Security][S3TablesSecurity].
+* A [Hadoop compatible file system][Hadoop] for Spark, Flink, and HBase.
+
+`S3_TABLE_BUCKET=warehouse ./weed mini -dir=./data` brings the whole stack up on a laptop.
+
+## A fast cache for cloud storage ##
+
+[Cloud Drive][CloudDrive] mounts a bucket from S3, Google Cloud Storage, Azure, Backblaze B2, Wasabi, Storj, or any S3-compatible store into SeaweedFS and serves it at local speed:
+
+* Metadata is pulled once, so listing, stat, and directory walks cost no cloud API calls.
+* File content is downloaded once, on first read or [warmed][CacheRemote] by folder, name pattern, size, or age, and cached with the capacity of the whole cluster: cache everything, no churn.
+* Local writes complete at local latency and are written back to the cloud asynchronously in the cloud's native layout, so other tools keep reading the bucket directly.
+* Uncache by the same rules to free local disk while keeping the metadata.
+
+[Cloud Tier][CloudTier] goes the other direction, moving whole warm volumes to cloud storage while keeping one-read access, and the [Gateway to Remote Object Storage][GatewayToRemoteObjectStore] mirrors every bucket to a remote store. Faster and cheaper than reading the cloud directly.
+
+## Active-active replication and more ##
+
+* [Active-active or active-passive replication][ActiveActiveAsyncReplication] between clusters, continuous and resumable, for the whole tree or chosen folders, across data centers.
+* [Filer store replication][FilerStoreReplication] for metadata HA, [async backup][AsyncBackup] to cloud storage, [metadata backup][MetaBackup], and [change data capture][CDC] with [webhooks][Webhook] on every metadata event.
+* The same data as a [FUSE mount][Mount] on Linux, macOS, and [Windows][MountWindows], over [WebDAV][WebDAV], [SFTP][SFTP], HDFS, HTTP, and [TUS resumable uploads][TUS]; on Kubernetes through the [CSI driver][SeaweedFsCsiDriver] and [Operator][Operator].
+* [AES256-GCM encryption at rest][FilerDataEncryption], TLS and mTLS between components, JWT-signed volume access, and [FIPS][FIPS] builds.
+* [Admin UI][AdminUI], Prometheus [metrics][Metrics], [TTL][VolumeServerTTL] per file or volume, automatic compression and compaction, and [seaweed-up][SeaweedUp] for bare-metal clusters.
 
 [Back to TOC](#table-of-contents)
 
-## Filer Features ##
-* [Filer server][Filer] provides "normal" directories and files via HTTP.
-* [File TTL][FilerTTL] automatically expires file metadata and actual file data.
-* [Mount filer][Mount] reads and writes files directly as a local directory via FUSE.
-* [Filer Store Replication][FilerStoreReplication] enables HA for filer meta data stores.
-* [Active-Active Replication][ActiveActiveAsyncReplication] enables asynchronous one-way or two-way cross cluster continuous replication.
-* [Amazon S3 compatible API][AmazonS3API] accesses files with S3 tooling.
-* [Hadoop Compatible File System][Hadoop] accesses files from Hadoop/Spark/Flink/etc or even runs HBase.
-* [Async Replication To Cloud][BackupToCloud] has extremely fast local access and backups to Amazon S3, Google Cloud Storage, Azure, BackBlaze.
-* [WebDAV] accesses as a mapped drive on Mac and Windows, or from mobile devices.
-* [AES256-GCM Encrypted Storage][FilerDataEncryption] safely stores the encrypted data.
-* [Super Large Files][SuperLargeFiles] stores large or super large files in tens of TB.
-* [Cloud Drive][CloudDrive] mounts cloud storage to local cluster, cached for fast read and write with asynchronous write back.
-* [Gateway to Remote Object Store][GatewayToRemoteObjectStore] mirrors bucket operations to remote object storage, in addition to [Cloud Drive][CloudDrive]
+# Architecture #
 
-## Data Lakehouse Features ##
-* [S3 Table Buckets][S3TableBucket] expose a dedicated namespace for Iceberg tables with strict layout validation.
-* Built-in [Iceberg REST Catalog][IcebergCatalog] runs alongside the S3 endpoint — no external metastore needed.
-* Native integrations with [Apache Spark][SparkIceberg], [Trino][TrinoIceberg], [Dremio][DremioIceberg], [DuckDB][DuckDBIceberg], and [RisingWave][RisingWaveIceberg].
-* [Automated table maintenance][IcebergMaintenance]: compaction, snapshot expiration, orphan removal, manifest rewriting.
-* Granular IAM at the bucket, namespace, and table level via standard S3 bucket policies.
+![SeaweedFS Architecture](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/SeaweedFS_Architecture.png)
 
-## Kubernetes ##
-* [Kubernetes CSI Driver][SeaweedFsCsiDriver] A Container Storage Interface (CSI) Driver. [![Docker Pulls](https://img.shields.io/docker/pulls/chrislusf/seaweedfs-csi-driver.svg?maxAge=4800)](https://hub.docker.com/r/chrislusf/seaweedfs-csi-driver/)
-* [SeaweedFS Operator](https://github.com/seaweedfs/seaweedfs-operator)
-
-[Filer]: https://github.com/seaweedfs/seaweedfs/wiki/Directories-and-Files
-[SuperLargeFiles]: https://github.com/seaweedfs/seaweedfs/wiki/Data-Structure-for-Large-Files
-[Mount]: https://github.com/seaweedfs/seaweedfs/wiki/FUSE-Mount
-[AmazonS3API]: https://github.com/seaweedfs/seaweedfs/wiki/Amazon-S3-API
-[BackupToCloud]: https://github.com/seaweedfs/seaweedfs/wiki/Async-Replication-to-Cloud
-[Hadoop]: https://github.com/seaweedfs/seaweedfs/wiki/Hadoop-Compatible-File-System
-[WebDAV]: https://github.com/seaweedfs/seaweedfs/wiki/WebDAV
-[ErasureCoding]: https://github.com/seaweedfs/seaweedfs/wiki/Erasure-coding-for-warm-storage
-[TieredStorage]: https://github.com/seaweedfs/seaweedfs/wiki/Tiered-Storage
-[CloudTier]: https://github.com/seaweedfs/seaweedfs/wiki/Cloud-Tier
-[FilerDataEncryption]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-Data-Encryption
-[FilerTTL]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-Stores
-[VolumeServerTTL]: https://github.com/seaweedfs/seaweedfs/wiki/Store-file-with-a-Time-To-Live
-[SeaweedFsCsiDriver]: https://github.com/seaweedfs/seaweedfs-csi-driver
-[ActiveActiveAsyncReplication]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-Active-Active-cross-cluster-continuous-synchronization
-[FilerStoreReplication]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-Store-Replication
-[KeyLargeValueStore]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-as-a-Key-Large-Value-Store
-[CloudDrive]: https://github.com/seaweedfs/seaweedfs/wiki/Cloud-Drive-Architecture
-[GatewayToRemoteObjectStore]: https://github.com/seaweedfs/seaweedfs/wiki/Gateway-to-Remote-Object-Storage
-[S3TableBucket]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Table-Bucket
-[IcebergCatalog]: https://github.com/seaweedfs/seaweedfs/wiki/SeaweedFS-Iceberg-Catalog
-[IcebergMaintenance]: https://github.com/seaweedfs/seaweedfs/wiki/Iceberg-Table-Maintenance
-[SparkIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/Spark-Iceberg-Integration
-[TrinoIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/Trino-Iceberg-Integration
-[DremioIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/Dremio-Iceberg-Integration
-[DuckDBIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/DuckDB-Iceberg-Integration
-[RisingWaveIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/RisingWave-Iceberg-Integration
+* **Master** servers, one or a Raft group of three, track which volume lives on which volume server and hand out file ids. They are not in the read path.
+* **Volume** servers store blobs in append-only volume files, keep a 16-byte in-memory index per blob, and replicate or erasure-code at the volume level.
+* **Filer** servers add directories and files on top, with metadata in a store of your choice, and expose HTTP, S3, WebDAV, SFTP, FUSE, and the table catalogs.
 
+The blob store started from [Facebook's Haystack](http://www.usenix.org/event/osdi10/tech/full_papers/Beaver.pdf), erasure coding takes ideas from [f4](https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-muralidhar.pdf), and the whole has a lot in common with [Tectonic](https://www.usenix.org/system/files/fast21-pan.pdf) and [Colossus](https://cloud.google.com/blog/products/storage-data-transfer/a-peek-behind-colossus-googles-file-system). How file ids are assigned, written, and looked up, and why a master that tracks volumes scales, is in [Blob Store Architecture][BlobStoreArchitecture]; the services are in [Components][Components] and the [white paper][WhitePaper].
 
 [Back to TOC](#table-of-contents)
 
-## Example: Using Seaweed Blob Store ##
-
-By default, the master node runs on port 9333, and the volume nodes run on port 8080.
-Let's start one master node, and two volume nodes on port 8080 and 8081. Ideally, they should be started from different machines. We'll use localhost as an example.
-
-SeaweedFS uses HTTP REST operations to read, write, and delete. The responses are in JSON or JSONP format.
-
-### Start Master Server ###
-
-```
-> ./weed master
-```
-
-### Start Volume Servers ###
-
-```
-> weed volume -dir="/tmp/data1" -max=5  -master="localhost:9333" -port=8080 &
-> weed volume -dir="/tmp/data2" -max=10 -master="localhost:9333" -port=8081 &
-```
-
-### Write A Blob ###
-
-A blob, also referred as a needle, a chunk, or mistakenly as a file, is just a byte array. It can have attributes, such as name, mime type, create or update time, etc. But basically it is just a byte array of a relatively small size, such as 2 MB ~ 64 MB. The size is not fixed.
-
-To upload a blob: first, send a HTTP POST, PUT, or GET request to `/dir/assign` to get an `fid` and a volume server URL:
-
-```
-> curl http://localhost:9333/dir/assign
-{"count":1,"fid":"3,01637037d6","url":"127.0.0.1:8080","publicUrl":"localhost:8080"}
-```
-
-Second, to store the blob content, send a HTTP multi-part POST request to `url + '/' + fid` from the response:
-
-```
-> curl -F file=@/home/chris/myphoto.jpg http://127.0.0.1:8080/3,01637037d6
-{"name":"myphoto.jpg","size":43234,"eTag":"1cc0118e"}
-```
-
-To update, send another POST request with updated blob content.
-
-For deletion, send an HTTP DELETE request to the same `url + '/' + fid` URL:
-
-```
-> curl -X DELETE http://127.0.0.1:8080/3,01637037d6
-```
-
-### Save Blob Id ###
-
-Now, you can save the `fid`, 3,01637037d6 in this case, to a database field.
-
-The number 3 at the start represents a volume id. After the comma, it's one file key, 01, and a file cookie, 637037d6.
-
-The volume id is an unsigned 32-bit integer. The file key is an unsigned 64-bit integer. The file cookie is an unsigned 32-bit integer, used to prevent URL guessing.
-
-The file key and file cookie are both coded in hex. You can store the <volume id, file key, file cookie> tuple in your own format, or simply store the `fid` as a string.
-
-If stored as a string, in theory, you would need 8+1+16+8=33 bytes. A char(33) would be enough, if not more than enough, since most uses will not need 2^32 volumes.
-
-If space is really a concern, you can store the file id in the binary format. You would need one 4-byte integer for volume id, 8-byte long number for file key, and a 4-byte integer for the file cookie. So 16 bytes are more than enough.
-
-### Read a Blob ###
-
-Here is an example of how to render the URL.
-
-First look up the volume server's URLs by the file's volumeId:
-
-```
-> curl http://localhost:9333/dir/lookup?volumeId=3
-{"volumeId":"3","locations":[{"publicUrl":"localhost:8080","url":"localhost:8080"}]}
-```
-
-Since (usually) there are not too many volume servers, and volumes don't move often, you can cache the results most of the time. Depending on the replication type, one volume can have multiple replica locations. Just randomly pick one location to read.
-
-Now you can take the public URL, render the URL or directly read from the volume server via URL:
-
-```
- http://localhost:8080/3,01637037d6.jpg
-```
-
-Notice we add a file extension ".jpg" here. It's optional and just one way for the client to specify the file content type.
-
-If you want a nicer URL, you can use one of these alternative URL formats:
-
-```
- http://localhost:8080/3/01637037d6/my_preferred_name.jpg
- http://localhost:8080/3/01637037d6.jpg
- http://localhost:8080/3,01637037d6.jpg
- http://localhost:8080/3/01637037d6
- http://localhost:8080/3,01637037d6
-```
-
-If you want to get a scaled version of an image, you can add some params:
-
-```
-http://localhost:8080/3/01637037d6.jpg?height=200&width=200
-http://localhost:8080/3/01637037d6.jpg?height=200&width=200&mode=fit
-http://localhost:8080/3/01637037d6.jpg?height=200&width=200&mode=fill
-```
-
-### Rack-Aware and Data Center-Aware Replication ###
-
-SeaweedFS applies the replication strategy at a volume level. So, when you are getting a blob id, you can specify the replication strategy. For example:
-
-```
-curl http://localhost:9333/dir/assign?replication=001
-```
-
-The replication parameter options are:
-
-```
-000: no replication
-001: replicate once on the same rack
-010: replicate once on a different rack, but same data center
-100: replicate once on a different data center
-200: replicate twice on two different data center
-110: replicate once on a different rack, and once on a different data center
-```
-
-More details about replication can be found [on the wiki][Replication].
-
-[Replication]: https://github.com/seaweedfs/seaweedfs/wiki/Replication
-
-You can also set the default replication strategy when starting the master server.
-
-### Allocate Blob Key on Specific Data Center ###
-
-Volume servers can be started with a specific data center name:
-
-```
- weed volume -dir=/tmp/1 -port=8080 -dataCenter=dc1
- weed volume -dir=/tmp/2 -port=8081 -dataCenter=dc2
-```
-
-When requesting a blob key, an optional "dataCenter" parameter can limit the assigned volume to the specific data center. For example, this specifies that the assigned volume should be limited to 'dc1':
-
-```
- http://localhost:9333/dir/assign?dataCenter=dc1
-```
-
-### Other Features ###
-  * [No Single Point of Failure][feat-1]
-  * [Insert with your own keys][feat-2]
-  * [Chunking large files][feat-3]
-  * [Collection as a Simple Name Space][feat-4]
-
-[feat-1]: https://github.com/seaweedfs/seaweedfs/wiki/Failover-Master-Server
-[feat-2]: https://github.com/seaweedfs/seaweedfs/wiki/Optimization#insert-with-your-own-keys
-[feat-3]: https://github.com/seaweedfs/seaweedfs/wiki/Optimization#upload-large-files
-[feat-4]: https://github.com/seaweedfs/seaweedfs/wiki/Optimization#collection-as-a-simple-name-space
-
-[Back to TOC](#table-of-contents)
-
-## Blob Store Architecture ##
-
-Usually distributed file systems split each file into chunks. A central server keeps a mapping of filenames to chunks, and also which chunks each chunk server has.
-
-The main drawback is that the central server can't handle many small files efficiently, and since all read requests need to go through the central master, so it might not scale well for many concurrent users.
-
-Instead of managing chunks, SeaweedFS manages data volumes in the master server. Each data volume is 32GB in size, and can hold a lot of blobs. And each storage node can have many data volumes. So the master node only needs to store the metadata about the volumes, which is a fairly small amount of data and is generally stable.
-
-The actual blob metadata, which are the blob volume, offset, and size, is stored in each volume on volume servers. Since each volume server only manages metadata of blobs on its own disk, with only 16 bytes for each blob, all access can read the metadata just from memory and only needs one disk operation to actually read file data.
-
-For comparison, consider that an xfs inode structure in Linux is 536 bytes.
-
-### Master Server and Volume Server ###
-
-The architecture is fairly simple. The actual data is stored in volumes on storage nodes. One volume server can have multiple volumes, and can both support read and write access with basic authentication.
-
-All volumes are managed by a master server. The master server contains the volume id to volume server mapping. This is fairly static information, and can be easily cached.
-
-On each write request, the master server also generates a file key, which is a growing 64-bit unsigned integer. Since write requests are not generally as frequent as read requests, one master server should be able to handle the concurrency well.
-
-### Write and Read files ###
-
-When a client sends a write request, the master server returns (volume id, file key, file cookie, volume node URL) for the blob. The client then contacts the volume node and POSTs the blob content.
-
-When a client needs to read a blob based on (volume id, file key, file cookie), it asks the master server by the volume id for the (volume node URL, volume node public URL), or retrieves this from a cache. Then the client can GET the content, or just render the URL on web pages and let browsers fetch the content.
-
-### Saving memory ###
-
-All blob metadata stored on a volume server is readable from memory without disk access. Each file takes just a 16-byte map entry of <64bit key, 32bit offset, 32bit size>. Of course, each map entry has its own space cost for the map. But usually the disk space runs out before the memory does.
-
-### Tiered Storage to the cloud ###
-
-The local volume servers are much faster, while cloud storages have elastic capacity and are actually more cost-efficient if not accessed often (usually free to upload, but relatively costly to access). With the append-only structure and O(1) access time, SeaweedFS can take advantage of both local and cloud storage by offloading the warm data to the cloud.
-
-Usually hot data are fresh and warm data are old. SeaweedFS puts the newly created volumes on local servers, and optionally upload the older volumes on the cloud. If the older data are accessed less often, this literally gives you unlimited capacity with limited local servers, and still fast for new data. 
-
-With the O(1) access time, the network latency cost is kept at minimum. 
-
-If the hot/warm data is split as 20/80, with 20 servers, you can achieve storage capacity of 100 servers. That's a cost saving of 80%! Or you can repurpose the 80 servers to store new data also, and get 5X storage throughput.
-
-[Back to TOC](#table-of-contents)
-
-## SeaweedFS Filer ##
-
-Built on top of the blob store, SeaweedFS Filer adds directory structure to create a file system. The directory structure is an interface that is implemented in many key-value stores or databases.
-
-The content of a file is mapped to one or many blobs, distributed to multiple volumes on multiple volume servers.
-
-## Compared to Other File Systems ##
+# Compared to Other Systems #
 
 Most other distributed file systems seem more complicated than necessary.
 
@@ -438,9 +271,7 @@ SeaweedFS is meant to be fast and simple, in both setup and operation. If you do
 
 SeaweedFS is constantly moving forward. Same with other systems. These comparisons can be outdated quickly. Please help to keep them updated.
 
-[Back to TOC](#table-of-contents)
-
-### Compared to HDFS ###
+## Compared to HDFS ##
 
 HDFS uses the chunk approach for each file, and is ideal for storing large files.
 
@@ -448,9 +279,7 @@ SeaweedFS is ideal for serving relatively smaller files quickly and concurrently
 
 SeaweedFS can also store extra large files by splitting them into manageable data chunks, and store the file ids of the data chunks into a meta chunk. This is managed by "weed upload/download" tool, and the weed master or volume servers are agnostic about it.
 
-[Back to TOC](#table-of-contents)
-
-### Compared to GlusterFS, Ceph ###
+## Compared to GlusterFS, Ceph ##
 
 The architectures are mostly the same. SeaweedFS aims to store and read files fast, with a simple and flat architecture. The main differences are
 
@@ -466,27 +295,18 @@ The architectures are mostly the same. SeaweedFS aims to store and read files fa
 | GlusterFS      | hashing          |                  | FUSE, NFS          |          |                           |
 | Ceph           | hashing + rules  |                  | FUSE               | Yes      |                           |
 | MooseFS        | in memory        |                  | FUSE               |       | No                          |
-| MinIO          | separate meta file for each file  |                  |         | Yes   | No                          |
+| MinIO          | separate meta file per drive for each file  |                  |         | Yes   | No                          |
+| RustFS         | separate meta file per drive for each file  |                  |         | Yes   | No                          |
 
-[Back to TOC](#table-of-contents)
+GlusterFS stores files, both directories and content, in configurable volumes called "bricks". It hashes the path and filename into ids, and assigned to virtual volumes, and then mapped to "bricks".
 
-### Compared to GlusterFS ###
-
-GlusterFS stores files, both directories and content, in configurable volumes called "bricks".
-
-GlusterFS hashes the path and filename into ids, and assigned to virtual volumes, and then mapped to "bricks".
-
-[Back to TOC](#table-of-contents)
-
-### Compared to MooseFS ###
+## Compared to MooseFS ##
 
 MooseFS chooses to neglect small file issue. From moosefs 3.0 manual, "even a small file will occupy 64KiB plus additionally 4KiB of checksums and 1KiB for the header", because it "was initially designed for keeping large amounts (like several thousands) of very big files"
 
-MooseFS Master Server keeps all meta data in memory. Same issue as HDFS namenode. 
+MooseFS Master Server keeps all meta data in memory. Same issue as HDFS namenode.
 
-[Back to TOC](#table-of-contents)
-
-### Compared to Ceph ###
+## Compared to Ceph ##
 
 Ceph can be setup similar to SeaweedFS as a key->blob store. It is much more complicated, with the need to support layers on top of it. [Here is a more detailed comparison](https://github.com/seaweedfs/seaweedfs/issues/120)
 
@@ -506,134 +326,39 @@ SeaweedFS Filer uses off-the-shelf stores, such as MySql, Postgres, Sqlite, Mong
 | Volume  | OSD | optimized for small files |
 | Filer  | Ceph FS | linearly scalable, Customizable, O(1) or O(logN) |
 
-[Back to TOC](#table-of-contents)
+## Compared to MinIO, RustFS ##
 
-### Compared to MinIO ###
+Please note, as Apr 25, 2026 MinIO ceased development. It's strongly discouraged to use that unmaintained software with multiple security bugs. RustFS is a MinIO reimplementation in Rust, Apache 2.0 licensed and still developed, keeping MinIO's storage model down to a byte-compatible on-disk format. So the points below apply to both.
 
-Please note, as Apr 25, 2026 MinIO ceased development. It's strongly discouraged to use that unmaintained software with multiple security bugs.
+MinIO followed AWS S3 closely and was ideal for testing for S3 API. It had good UI, policies, versionings, etc. SeaweedFS is trying to catch up here.
 
-MinIO followed AWS S3 closely and was ideal for testing for S3 API. It had good UI, policies, versionings, etc. SeaweedFS is trying to catch up here. 
+The metadata are in simple files. Each file write incurs extra writes to the corresponding meta file, on every drive of the erasure set. Changing only tags or retention rewrites that meta file on all of them, so the write amplification does not shrink with object size.
 
-MinIO metadata were in simple files. Each file write will incur extra writes to corresponding meta file.
-
-MinIO did not have optimization for lots of small files. The files were simply stored as is to local disks.
+There is no optimization for lots of small files. The files are simply stored as is to local disks.
 Plus the extra meta file and shards for erasure coding, it only amplifies the LOSF problem.
 
-MinIO had multiple disk IO to read one file. SeaweedFS has O(1) disk reads, even for erasure coded files.
+Multiple disk IO are needed to read one file. SeaweedFS has O(1) disk reads, even for erasure coded files.
 
-MinIO had full-time erasure coding. SeaweedFS uses replication on hot data for faster speed and optionally applies erasure coding on warm data.
+Erasure coding is full-time. SeaweedFS uses replication on hot data for faster speed and optionally applies erasure coding on warm data.
 
-MinIO did not have POSIX-like API support.
+No POSIX-like API support.
 
-MinIO had specific requirements on storage layout. It is not flexible to adjust capacity. In SeaweedFS, just start one volume server pointing to the master. That's all.
-
-## Dev Plan ##
-
-* More tools and documentation, on how to manage and scale the system.
-* Read and write stream data.
-* Support structured data.
-
-This is a super exciting project! And we need helpers and [support](https://www.patreon.com/seaweedfs)!
+There are specific requirements on storage layout, which makes it hard to scale out and to maintain. An erasure set must be 2 to 16 drives and must divide the drive list symmetrically, and capacity grows or shrinks a whole pool at a time. In SeaweedFS, just start one volume server pointing to the master. That's all.
 
 [Back to TOC](#table-of-contents)
 
-## Installation Guide ##
+# Benchmark #
 
-> Installation guide for users who are not familiar with golang
+Unscientific single-machine numbers from a MacBook with an SSD. [`weed benchmark`][Benchmarks], 1 million 1KB files, concurrency 16:
 
-Step 1: install go on your machine and setup the environment by following the instructions at:
+| | Requests per second | p50 | p99 |
+| --- | --- | --- | --- |
+| Write | 15,708 | 0.8 ms | 2.6 ms |
+| Random read | 47,019 | 0.3 ms | 0.7 ms |
 
-https://golang.org/doc/install
-
-make sure to define your $GOPATH
-
-
-Step 2: checkout this repo:
-```bash
-git clone https://github.com/seaweedfs/seaweedfs.git
-```
-Step 3: download, compile, and install the project by executing the following command
-
-```bash
-cd seaweedfs/weed && make install
-```
-
-Once this is done, you will find the executable "weed" in your `$GOPATH/bin` directory
-
-For more installation options, including how to run with Docker, see the [Getting Started guide](https://github.com/seaweedfs/seaweedfs/wiki/Getting-Started).
-
-[Back to TOC](#table-of-contents)
-
-## Disk Related Topics ##
-
-### Hard Drive Performance ###
-
-When testing read performance on SeaweedFS, it basically becomes a performance test of your hard drive's random read speed. Hard drives usually get 100MB/s~200MB/s.
-
-### Solid State Disk ###
-
-To modify or delete small files, SSD must delete a whole block at a time, and move content in existing blocks to a new block. SSD is fast when brand new, but will get fragmented over time and you have to garbage collect, compacting blocks. SeaweedFS is friendly to SSD since it is append-only. Deletion and compaction are done on volume level in the background, not slowing reading and not causing fragmentation.
-
-[Back to TOC](#table-of-contents)
-
-## Benchmark ##
-
-My Own Unscientific Single Machine Results on Mac Book with Solid State Disk, CPU: 1 Intel Core i7 2.6GHz.
-
-Write 1 million 1KB file:
-```
-Concurrency Level:      16
-Time taken for tests:   66.753 seconds
-Completed requests:      1048576
-Failed requests:        0
-Total transferred:      1106789009 bytes
-Requests per second:    15708.23 [#/sec]
-Transfer rate:          16191.69 [Kbytes/sec]
-
-Connection Times (ms)
-              min      avg        max      std
-Total:        0.3      1.0       84.3      0.9
-
-Percentage of the requests served within a certain time (ms)
-   50%      0.8 ms
-   66%      1.0 ms
-   75%      1.1 ms
-   80%      1.2 ms
-   90%      1.4 ms
-   95%      1.7 ms
-   98%      2.1 ms
-   99%      2.6 ms
-  100%     84.3 ms
-```
-
-Randomly read 1 million files:
-```
-Concurrency Level:      16
-Time taken for tests:   22.301 seconds
-Completed requests:      1048576
-Failed requests:        0
-Total transferred:      1106812873 bytes
-Requests per second:    47019.38 [#/sec]
-Transfer rate:          48467.57 [Kbytes/sec]
-
-Connection Times (ms)
-              min      avg        max      std
-Total:        0.0      0.3       54.1      0.2
-
-Percentage of the requests served within a certain time (ms)
-   50%      0.3 ms
-   90%      0.4 ms
-   98%      0.6 ms
-   99%      0.7 ms
-  100%     54.1 ms
-```
-
-### Run WARP and launch a mixed benchmark. ###
+`make benchmark` runs [warp][S3Benchmark] mixed S3 traffic against a local `weed server`:
 
 ```
-make benchmark
-warp: Benchmark data written to "warp-mixed-2025-12-05[194844]-kBpU.csv.zst"
-
 Mixed operations.
 Operation: DELETE, 10%, Concurrency: 20, Ran 42s.
  * Throughput: 55.13 obj/s
@@ -650,17 +375,19 @@ Operation: STAT, 30%, Concurrency: 20, Ran 42s.
 Cluster Total: 3302.88 MiB/s, 550.51 obj/s over 43s.
 ```
 
+Read throughput is bounded by the random read speed of the disks, and grows with every volume server added. More numbers, including multi-node, FUSE, and Hadoop, are in [Benchmarks][Benchmarks], [S3 API Benchmark][S3Benchmark], [FIO benchmark][FIO], and [Independent Benchmarks][IndependentBenchmarks].
+
 [Back to TOC](#table-of-contents)
 
-## Enterprise ##
+# Enterprise #
 
-For enterprise users, please visit [seaweedfs.com](https://seaweedfs.com) for the SeaweedFS Enterprise Edition, 
-which has advanced features, including data recovery, self-healing storage, 
+For enterprise users, please visit [seaweedfs.com](https://seaweedfs.com) for the SeaweedFS Enterprise Edition,
+which has advanced features, including data recovery, self-healing storage,
 customizable erasure coding, EC vacuum and repair, etc.
 
 [Back to TOC](#table-of-contents)
 
-## License ##
+# License #
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -674,9 +401,114 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
 
-The text of this page is available for modification and reuse under the terms of the Creative Commons Attribution-Sharealike 3.0 Unported License and the GNU Free Documentation License (unversioned, with no invariant sections, front-cover texts, or back-cover texts).
+[Back to TOC](#table-of-contents)
+
+# Sponsors #
+
+<h3 align="center"><a href="https://www.patreon.com/seaweedfs">Sponsor SeaweedFS via Patreon</a></h3>
+
+SeaweedFS is an independent Apache-licensed open source project with its ongoing development made
+possible entirely thanks to the support of these awesome [backers](https://github.com/seaweedfs/seaweedfs/blob/master/backers.md).
+If you'd like to grow SeaweedFS even stronger, please consider joining our
+<a href="https://www.patreon.com/seaweedfs">sponsors on Patreon</a>.
+
+Your support will be really appreciated by me and other supporters!
+
+<!--
+<h4 align="center">Platinum</h4>
+
+<p align="center">
+  <a href="" target="_blank">
+    <img src="https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/sponsor_nodion.png" width="200" alt="nodion">
+  </a>
+</p>
+-->
+
+### Gold Sponsors
+[![nodion](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/sponsor_nodion.png)](https://www.nodion.com)
+[![piknik](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/piknik.png)](https://www.piknik.com)
+[![keepsec](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/keepsec.png)](https://www.keepsec.ca)
+[![zyner](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/sponsor_zyner.png)](https://zyner.org)
 
 [Back to TOC](#table-of-contents)
 
-## Stargazers over time
-[![Stargazers over time](https://starchart.cc/seaweedfs/seaweedfs.svg?variant=adaptive)](https://starchart.cc/seaweedfs/seaweedfs)
+## Star History
+
+![Star History](note/star_history.svg)
+
+[WeedMini]: https://github.com/seaweedfs/seaweedfs/wiki/Quick-Start-with-weed-mini
+[DockerComposeS3]: https://github.com/seaweedfs/seaweedfs/wiki/Docker-Compose-for-S3
+[HelmRecipes]: https://github.com/seaweedfs/seaweedfs/wiki/Helm-Chart-Recipes
+[Operator]: https://github.com/seaweedfs/seaweedfs-operator
+[SeaweedFsCsiDriver]: https://github.com/seaweedfs/seaweedfs-csi-driver
+[GettingStarted]: https://github.com/seaweedfs/seaweedfs/wiki/Getting-Started
+[ProductionSetup]: https://github.com/seaweedfs/seaweedfs/wiki/Production-Setup
+[ErasureCoding]: https://github.com/seaweedfs/seaweedfs/wiki/Erasure-Coding-for-warm-storage
+[RustVolume]: https://github.com/seaweedfs/seaweedfs/wiki/Rust-Volume-Server
+[Benchmarks]: https://github.com/seaweedfs/seaweedfs/wiki/Benchmarks
+[S3Benchmark]: https://github.com/seaweedfs/seaweedfs/wiki/S3-API-Benchmark
+[FIO]: https://github.com/seaweedfs/seaweedfs/wiki/FIO-benchmark
+[IndependentBenchmarks]: https://github.com/seaweedfs/seaweedfs/wiki/Independent-Benchmarks
+[FailoverMaster]: https://github.com/seaweedfs/seaweedfs/wiki/Failover-Master-Server
+[WeedShell]: https://github.com/seaweedfs/seaweedfs/wiki/weed-shell
+[Worker]: https://github.com/seaweedfs/seaweedfs/wiki/Worker
+[FilerStores]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-Stores
+[Replication]: https://github.com/seaweedfs/seaweedfs/wiki/Replication
+[TieredStorage]: https://github.com/seaweedfs/seaweedfs/wiki/Tiered-Storage
+[CloudTier]: https://github.com/seaweedfs/seaweedfs/wiki/Cloud-Tier
+[SuperLargeFiles]: https://github.com/seaweedfs/seaweedfs/wiki/Data-Structure-for-Large-Files
+[Versioning]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Object-Versioning
+[ObjectLock]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Object-Lock-and-Retention
+[Lifecycle]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Lifecycle
+[CORS]: https://github.com/seaweedfs/seaweedfs/wiki/S3-CORS
+[ConditionalOps]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Conditional-Operations
+[RenameObject]: https://github.com/seaweedfs/seaweedfs/wiki/S3-RenameObject
+[BucketPolicies]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Bucket-Policies
+[PolicyConditions]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Policy-Conditions
+[PolicyVariables]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Policy-Variables
+[OIDC]: https://github.com/seaweedfs/seaweedfs/wiki/OIDC-Integration
+[K8sSA]: https://github.com/seaweedfs/seaweedfs/wiki/Kubernetes-ServiceAccount-Authentication
+[SSE]: https://github.com/seaweedfs/seaweedfs/wiki/Server-Side-Encryption
+[AuditLog]: https://github.com/seaweedfs/seaweedfs/wiki/S3-API-Audit-log
+[BucketQuota]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Bucket-Quota
+[RateLimiting]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Rate-Limiting
+[AmazonS3API]: https://github.com/seaweedfs/seaweedfs/wiki/Amazon-S3-API
+[S3vsMinio]: https://github.com/seaweedfs/seaweedfs/wiki/Supported-APIs-vs-Minio
+[S3TableBucket]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Table-Bucket
+[LanceCatalog]: https://github.com/seaweedfs/seaweedfs/wiki/SeaweedFS-Lance-Catalog
+[IcebergCatalog]: https://github.com/seaweedfs/seaweedfs/wiki/SeaweedFS-Iceberg-Catalog
+[SparkIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/Spark-Iceberg-Integration
+[TrinoIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/Trino-Iceberg-Integration
+[DremioIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/Dremio-Iceberg-Integration
+[DuckDBIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/DuckDB-Iceberg-Integration
+[DorisIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/Doris-Iceberg-Integration
+[RisingWaveIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/RisingWave-Iceberg-Integration
+[LanceDB]: https://github.com/seaweedfs/seaweedfs/wiki/LanceDB-Integration
+[Lakekeeper]: https://github.com/seaweedfs/seaweedfs/wiki/Lakekeeper-Iceberg-Integration
+[IcebergMaintenance]: https://github.com/seaweedfs/seaweedfs/wiki/Iceberg-Table-Maintenance
+[LanceMaintenance]: https://github.com/seaweedfs/seaweedfs/wiki/Lance-Maintenance-Worker
+[S3TablesSecurity]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Tables-Security
+[Hadoop]: https://github.com/seaweedfs/seaweedfs/wiki/Hadoop-Compatible-File-System
+[CloudDrive]: https://github.com/seaweedfs/seaweedfs/wiki/Cloud-Drive-Architecture
+[CacheRemote]: https://github.com/seaweedfs/seaweedfs/wiki/Cache-Remote-Storage
+[GatewayToRemoteObjectStore]: https://github.com/seaweedfs/seaweedfs/wiki/Gateway-to-Remote-Object-Storage
+[ActiveActiveAsyncReplication]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-Active-Active-cross-cluster-continuous-synchronization
+[FilerStoreReplication]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-Store-Replication
+[AsyncBackup]: https://github.com/seaweedfs/seaweedfs/wiki/Async-Backup
+[MetaBackup]: https://github.com/seaweedfs/seaweedfs/wiki/Async-Filer-Metadata-Backup
+[CDC]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-Change-Data-Capture
+[Webhook]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-Notification-Webhook
+[Mount]: https://github.com/seaweedfs/seaweedfs/wiki/FUSE-Mount
+[MountWindows]: https://github.com/seaweedfs/seaweedfs/wiki/Mount-on-Windows
+[WebDAV]: https://github.com/seaweedfs/seaweedfs/wiki/WebDAV
+[SFTP]: https://github.com/seaweedfs/seaweedfs/wiki/SFTP-Server
+[TUS]: https://github.com/seaweedfs/seaweedfs/wiki/TUS-Resumable-Uploads
+[FilerDataEncryption]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-Data-Encryption
+[FIPS]: https://github.com/seaweedfs/seaweedfs/wiki/Cryptography-and-FIPS-Compliance
+[AdminUI]: https://github.com/seaweedfs/seaweedfs/wiki/Admin-UI
+[Metrics]: https://github.com/seaweedfs/seaweedfs/wiki/System-Metrics
+[VolumeServerTTL]: https://github.com/seaweedfs/seaweedfs/wiki/Store-file-with-a-Time-To-Live
+[SeaweedUp]: https://github.com/seaweedfs/seaweedfs/wiki/Deployment-with-seaweed-up
+[BlobStoreArchitecture]: https://github.com/seaweedfs/seaweedfs/wiki/Blob-Store-Architecture
+[Components]: https://github.com/seaweedfs/seaweedfs/wiki/Components
+[WhitePaper]: https://github.com/seaweedfs/seaweedfs/wiki/SeaweedFS_Architecture.pdf

REST_API.md

@@ -0,0 +1,344 @@
+# SeaweedFS HTTP REST API
+
+SeaweedFS exposes three HTTP surfaces:
+
+| Service | Default port | Addressing |
+|---------|--------------|------------|
+| Filer | 8888 | File system paths (`/dir/name`) |
+| Master | 9333 | File id assignment and cluster topology |
+| Volume server | 8080 | File content by file id (`vid,fid`) |
+
+Most clients only need the filer API (paths) or the S3 API. The master and
+volume APIs are the lower-level blob store interface.
+
+Conventions applying to all three:
+
+- Responses are JSON unless noted otherwise. Append `&pretty=y` to pretty-print.
+- A file id (`fid`) has the form `volumeId,fileKeyCookie`, e.g. `3,01637037d6`.
+  An optional suffix selects a reserved id from a `count` assignment
+  (`3,01637037d6_1`, `_2`, ...), and an optional extension
+  (`3,01637037d6.jpg`) sets the content type on reads.
+- `replication` is a 3-digit replica placement `xyz`: `x` copies in other
+  data centers, `y` on other racks in the same data center, `z` on other
+  volume servers on the same rack. `000` = no replication, `001` = one copy
+  on the same rack, `010` = one copy on a different rack, `100` = one copy in
+  another data center, `200` = two copies in two other data centers, `110` =
+  one copy in another data center plus one on another rack.
+- `ttl` units: `m` minute, `h` hour, `d` day, `w` week, `M` month, `y` year.
+
+## Filer API (port 8888)
+
+The filer presents a POSIX-like namespace over the volume servers.
+
+### Upload a file
+
+```bash
+# PUT the raw body to the target path
+curl -T /home/chris/myphoto.jpg "http://localhost:8888/dir/myphoto.jpg"
+
+# or POST as multipart form (the part filename becomes the entry name)
+curl -F file=@/home/chris/myphoto.jpg "http://localhost:8888/dir/"
+```
+
+Response `201 Created`:
+
+```json
+{"name":"myphoto.jpg","size":43234,"eTag":"0x6c656...","mtime":"...","chunks":[...]}
+```
+
+Query parameters:
+
+| Parameter | Description | Default |
+|-----------|-------------|---------|
+| `collection` | collection name | empty |
+| `replication` | replica placement code | filer default |
+| `ttl` | file expiration, e.g. `3d` | never |
+| `disk` | disk type to store on | filer default |
+| `fsync` | `true` fsyncs on the volume server | false |
+| `dataCenter` | preferred data center | empty |
+| `rack` | preferred rack | empty |
+| `dataNode` | preferred volume server | empty |
+| `saveInside` | store small content inside the metadata instead of a volume | false |
+| `maxMB` | split the upload into chunks of this many MB | filer `-maxMB` |
+| `mode` | unix permission bits, e.g. `0644` | `0660` |
+| `op` | `append` appends to an existing file | overwrite |
+| `skipCheckParentDir` | `true` skips the parent-directory existence check | false |
+
+### Create a directory
+
+```bash
+curl -X POST "http://localhost:8888/dir/newdir/"
+```
+
+A POST to a path ending in `/` with no content creates the directory,
+including missing parents.
+
+### Read a file
+
+```bash
+curl "http://localhost:8888/dir/myphoto.jpg"
+```
+
+Supports `Range` requests (`Accept-Ranges: bytes`), `ETag`, and the
+`If-None-Match` / `If-Modified-Since` conditional headers. `HEAD` returns
+headers only. Entry headers stored as extended attributes are echoed back,
+minus internal `Seaweed-` and `xattr-` keys.
+
+Entry metadata instead of content:
+
+```bash
+curl "http://localhost:8888/dir/myphoto.jpg?metadata=true"
+```
+
+`metadata=true&resolveManifest=true` additionally resolves chunked-manifest
+entries into their real chunk list.
+
+### List a directory
+
+```bash
+curl -H "Accept: application/json" "http://localhost:8888/dir/?limit=10&lastFileName=a.jpg"
+```
+
+| Parameter | Description | Default |
+|-----------|-------------|---------|
+| `limit` | max entries per page | filer `-dirListLimit` |
+| `lastFileName` | resume listing after this entry name | empty |
+| `namePattern` | include only names matching the wildcard | empty |
+| `namePatternExclude` | exclude names matching the wildcard | empty |
+
+The JSON response carries `Path`, `Entries`, `Limit`, `LastFileName`,
+`ShouldDisplayLoadMore`, and `EmptyFolder`. Without the `Accept` header the
+filer renders its HTML browser.
+
+### Move and copy
+
+```bash
+curl -X POST "http://localhost:8888/dir/newname.jpg?mv.from=/dir/myphoto.jpg"
+curl -X POST "http://localhost:8888/dir/copy.jpg?cp.from=/dir/myphoto.jpg"
+```
+
+`mv.from` renames or moves the source to the request path (`204 No Content`).
+`cp.from` copies it.
+
+### Append
+
+```bash
+curl -T chunk2.bin "http://localhost:8888/dir/file.bin?op=append"
+```
+
+### Delete
+
+```bash
+curl -X DELETE "http://localhost:8888/dir/myphoto.jpg"
+curl -X DELETE "http://localhost:8888/dir/?recursive=true"
+```
+
+| Parameter | Description | Default |
+|-----------|-------------|---------|
+| `recursive` | delete a non-empty directory tree | false; when the filer runs with `filer.options.recursive_delete=true`, deletes are recursive unless `recursive=false` |
+| `ignoreRecursiveError` | keep deleting remaining entries after an error | false |
+| `skipChunkDeletion` | remove only the metadata, keep volume data | false |
+
+### Tagging
+
+Tags are carried as `Seaweed-`-prefixed request headers, not query
+parameters; `?tagging` selects the tagging handler and `?tagging=K1,K2`
+lists the keys to remove. Header names are canonicalized on write
+(`Seaweed-k1` is stored as `Seaweed-K1`), and the delete list is matched
+case-sensitively against the stored names.
+
+```bash
+curl -X PUT -H "Seaweed-k1: v1" -H "Seaweed-k2: v2" "http://localhost:8888/dir/file.jpg?tagging"
+curl -X DELETE "http://localhost:8888/dir/file.jpg?tagging=K1,K2"
+```
+
+### Read by file id
+
+```bash
+curl "http://localhost:8888/?proxyChunkId=3,01637037d6"
+```
+
+The filer proxies the chunk read to the right volume server, so only the
+filer port needs to be exposed.
+
+### Resumable uploads
+
+The filer serves the [TUS protocol](https://tus.io/) for resumable uploads
+(`POST`, `PATCH`, `HEAD` on upload URLs). It is enabled by default at
+`/.tus`; `-tusBasePath` changes the endpoint base path.
+
+### Health
+
+`GET /healthz` and `GET /readyz` return `200 OK`.
+
+## Master API (port 9333)
+
+Write-affecting endpoints are automatically proxied to the current leader, so
+any master in the quorum can serve them.
+
+### Assign a file id
+
+```bash
+curl "http://localhost:9333/dir/assign?count=1&replication=001&collection=turbo&dataCenter=dc1&ttl=3d&disk=ssd"
+{"count":1,"fid":"3,01637037d6","url":"127.0.0.1:8080","publicUrl":"localhost:8080"}
+```
+
+Upload the file content to `http://<url>/<fid>` afterwards. With `count>1`,
+use `<fid>_1`, `<fid>_2`, ... for the additional ids.
+
+| Parameter | Description | Default |
+|-----------|-------------|---------|
+| `count` | file ids to reserve | 1 |
+| `collection` | collection name | empty |
+| `dataCenter` | preferred data center | empty |
+| `rack` | preferred rack | empty |
+| `dataNode` | preferred volume server | empty |
+| `replication` | replica placement | master `-defaultReplication` |
+| `ttl` | file expiration, e.g. `3d` | never |
+| `disk` | disk type | empty |
+| `dataSize` | expected file size in bytes | 0 |
+| `preallocate` | bytes to preallocate for new volumes | master `-volumePreallocate` |
+| `writableVolumeCount` | grow this many volumes when none are writable | master default |
+| `memoryMapMaxSizeMb` | memory-mapped file size (Windows) | 0 |
+
+### Look up a volume or file id
+
+```bash
+curl "http://localhost:9333/dir/lookup?volumeId=3"
+{"locations":[{"url":"localhost:8080","publicUrl":"localhost:8080"}]}
+```
+
+| Parameter | Description | Default |
+|-----------|-------------|---------|
+| `volumeId` | volume id; a full `vid,fid` is accepted too | required |
+| `fileId` | like `volumeId`, but also returns a write JWT when security is on | empty |
+| `collection` | speeds up the lookup | empty |
+| `read` | `yes` generates a read JWT instead of a write JWT | empty |
+
+### Store a file in one call
+
+```bash
+curl -F file=@/home/chris/report.pdf "http://localhost:9333/submit?collection=turbo&replication=001"
+{"fileName":"report.pdf","fid":"3,01637037d6","fileUrl":"localhost:8080/3,01637037d6","size":43234,"eTag":"0x6c656..."}
+```
+
+`POST /submit` accepts multipart file data plus the `dir/assign` placement
+parameters (`count`, `collection`, `dataCenter`, `rack`, `replication`,
+`ttl`, `disk`), assigns a file id, uploads to the volume server, and returns
+the result.
+
+### Redirect to a file
+
+```bash
+curl -v "http://localhost:9333/3,01637037d6"
+```
+
+`GET /{fileId}` answers `308 Permanent Redirect` to a volume server holding
+the file, preserving the query string (e.g. image-resize parameters).
+
+### Cluster status
+
+```bash
+curl "http://localhost:9333/dir/status?pretty=y"   # full topology tree
+curl "http://localhost:9333/vol/status?pretty=y"   # every volume on every node
+curl "http://localhost:9333/collection/info?collection=turbo"
+curl "http://localhost:9333/collection/info?collection=turbo&detail=true"
+```
+
+`collection/info` returns aggregated `TotalSize`, `FileCount`, `UsedSize`,
+`VolumeCount`; `detail=true` splits them per volume layout.
+
+### Grow volumes
+
+```bash
+curl "http://localhost:9333/vol/grow?count=4&replication=001&collection=turbo&ttl=5d&disk=ssd&dataCenter=dc1&rack=rack1"
+{"count":4}
+```
+
+`count` is required; the placement parameters match `dir/assign`. One volume
+serves one write at a time, so pre-allocated volumes raise write concurrency.
+
+### Vacuum deleted space
+
+```bash
+curl "http://localhost:9333/vol/vacuum?garbageThreshold=0.4"
+```
+
+| Parameter | Description | Default |
+|-----------|-------------|---------|
+| `garbageThreshold` | minimum deleted-bytes ratio before a volume is compacted | master `-garbageThreshold` (0.3) |
+
+Vacuuming makes a volume read-only, copies live needles to a new volume, and
+swaps it in.
+
+### Delete a collection
+
+```bash
+curl "http://localhost:9333/col/delete?collection=benchmark"
+```
+
+Deletes all volumes of the collection, including erasure-coded shards.
+`204 No Content` on success.
+
+### Health
+
+```bash
+curl -I "http://localhost:9333/healthz"     # liveness
+curl -I "http://localhost:9333/readyz"      # readiness
+curl "http://localhost:9333/"               # web UI
+```
+
+## Volume server API (port 8080)
+
+The volume server stores file content by file id. Clients normally get the
+volume URL from `dir/assign` or `dir/lookup`.
+
+### Upload
+
+```bash
+curl -F file=@/home/chris/myphoto.jpg "http://127.0.0.1:8080/3,01637037d6"
+{"name":"myphoto.jpg","size":43234,"eTag":"0x6c656...","mime":"image/jpeg","contentMd5":"..."}
+```
+
+PUT or POST the body (or a multipart `file` part) to `/{vid},{fid}`.
+`204 No Content` is returned when the content is unchanged. `?ts=<unix>`
+sets the stored modification time.
+
+### Read
+
+```bash
+curl "http://127.0.0.1:8080/3,01637037d6"
+curl "http://127.0.0.1:8080/3,01637037d6.jpg"        # sets Content-Type from the extension
+```
+
+Supports `Range` and `HEAD`. Image files can be resized server-side:
+
+| Parameter | Description |
+|-----------|-------------|
+| `width`, `height` | resize bounds in pixels |
+| `mode` | `fit` (contain) or `fill` (cover); omitted resizes to `width`/`height` |
+| `crop_x1`, `crop_y1`, `crop_x2`, `crop_y2` | explicit crop rectangle |
+| `cm` | `false` returns the chunk-manifest blob instead of resolving it |
+| `readDeleted` | `true` reads soft-deleted needles |
+| `collection` | passed through redirects for the right volume |
+
+### Delete
+
+```bash
+curl -X DELETE "http://127.0.0.1:8080/3,01637037d6"
+{"size":43234}
+```
+
+`?ts=<unix>` sets the deletion timestamp. Replicated volumes propagate the
+delete to every replica.
+
+### Status
+
+```bash
+curl "http://localhost:8080/status?pretty=y"  # disk and volume inventory
+curl -I "http://localhost:8080/healthz"       # liveness/readiness
+```
+
+`OPTIONS` preflights answer CORS headers. When `-port.public` differs from
+`-port`, the volume server opens a separate read-only public listener on
+that port; `-publicUrl` sets the address it advertises to clients.

design-cloudstack-provider.md

@@ -0,0 +1,418 @@
+# SeaweedFS as an Apache CloudStack Object Storage Provider
+
+A CloudStack ObjectStore plugin that makes SeaweedFS a first-class object storage
+backend inside Apache CloudStack, alongside the existing MinIO and Ceph RGW
+providers. This is a collaboration with proIO (Swen), who builds private clouds on
+CloudStack and wants SeaweedFS as a storage option.
+
+## The request
+
+> We can only add MinIO and Ceph as object storage [in CloudStack] today. I want
+> to get SeaweedFS into this project... What we need is to build a provider which
+> does the communication between Cloudstack and SeaweedFS.
+
+This is **not** a SeaweedFS-side feature. The work lives in the Apache CloudStack
+repo (Java): a new plugin under `plugins/storage/object/seaweedfs/` that implements
+CloudStack's ObjectStore plugin framework and talks to SeaweedFS over its S3 and
+IAM APIs. SeaweedFS itself needs no changes for the core to work — its S3 API
+already covers every bucket operation CloudStack requires, and its IAM API covers
+user/credential management.
+
+## How the CloudStack ObjectStore framework works
+
+CloudStack 4.18+ introduced an Object Storage framework. An admin registers an
+object storage pool via `addObjectStoragePool` (URL + provider + credentials);
+tenants then create and manage buckets on it through CloudStack APIs. CloudStack
+manages pool and bucket lifecycle; the underlying provider handles the actual
+object protocol.
+
+A provider is a plugin module implementing three interfaces:
+
+### 1. `ObjectStoreProvider` — registration
+
+`MinIOObjectStoreProviderImpl` is the reference. It is a Spring `@Component` that:
+- Returns a provider name (`"MinIO"`)
+- Returns `DataStoreProviderType.OBJECT`
+- In `configure()`, injects the lifecycle and driver implementations and calls
+  `storeMgr.registerDriver(name, driver)`
+
+### 2. `ObjectStoreLifeCycle` — pool add/remove
+
+`MinIOObjectStoreLifeCycleImpl.initialize()` reads the URL, name, and
+`accesskey`/`secretkey` details from the `addObjectStoragePool` call, tests the
+connection by listing buckets, and persists an `ObjectStoreVO` via
+`ObjectStoreHelper`. The other methods (attachCluster/Host/Zone, maintain,
+deleteDataStore) are no-ops for object storage.
+
+### 3. `ObjectStoreDriver` — bucket + user operations
+
+`ObjectStoreDriver` (in `engine/storage/.../object/ObjectStoreDriver.java`) extends
+`DataStoreDriver` and defines the bucket/user contract. Every provider must
+implement:
+
+| Method | Purpose |
+| --- | --- |
+| `createBucket(Bucket, boolean objectLock)` | Create a bucket |
+| `listBuckets(long storeId)` | List all buckets |
+| `deleteBucket(BucketTO, long storeId)` | Delete a bucket |
+| `createUser(long accountId, long storeId)` | Provision a user + credentials for a CloudStack account |
+| `setBucketPolicy` / `getBucketPolicy` / `deleteBucketPolicy` | Bucket policy CRUD |
+| `setBucketEncryption` / `deleteBucketEncryption` | SSE config |
+| `setBucketVersioning` / `deleteBucketVersioning` | Versioning enable/suspend |
+| `setBucketQuota(BucketTO, long storeId, long size)` | Per-bucket quota |
+| `getAllBucketsUsage(long storeId)` | Usage map for billing/accounting |
+| `getBucketAcl` / `setBucketAcl` | ACLs (MinIO/Ceph return null / no-op) |
+
+`BaseObjectStoreDriverImpl` provides no-op defaults for the `DataStoreDriver`
+methods (`createAsync`, `deleteAsync`, `copyAsync`, `canCopy`, `resize`,
+`getTO`, `getStoreTO`), so object-store providers only implement the bucket/user
+methods above.
+
+## How the four existing providers differ (and where SeaweedFS lands)
+
+CloudStack ships four object-store providers. Three are relevant; the simulator
+is a test stub.
+
+| Concern | MinIO | Ceph RGW | Cloudian HyperStore | SeaweedFS |
+| --- | --- | --- | --- | --- |
+| Bucket CRUD | `MinioClient` (S3) | `AmazonS3` (AWS SDK v1) | `AmazonS3` (AWS SDK v1) | `AmazonS3` (AWS SDK v1) |
+| Bucket policy | `MinioClient` | `AmazonS3` | `AmazonS3` | `AmazonS3` |
+| Versioning | `MinioClient` | `AmazonS3` | `AmazonS3` | `AmazonS3` |
+| Encryption | `MinioClient` | not implemented | `AmazonS3` | `AmazonS3` |
+| **User creation** | `MinioAdminClient` | `RgwAdmin` | **`AmazonIdentityManagement`** | **`AmazonIdentityManagement`** |
+| **Per-bucket quota** | `MinioAdminClient` | `RgwAdmin` | **not supported** (throws) | **S3 extension** (`PUT /{bucket}?seaweedfs-quota`, SigV4, `s3:PutBucketQuota`) |
+| **Usage reporting** | `MinioAdminClient` | `RgwAdmin` | Cloudian admin API | S3 `ListObjectsV2` (MVP); Prometheus / SOSAPI `capacity.xml` (recommended) |
+
+**Cloudian HyperStore is the direct precedent.** It is an S3-compatible store
+that, like SeaweedFS, manages users via the **standard AWS IAM API** using the
+AWS IAM Java SDK (`com.amazonaws.services.identitymanagement`). Its driver
+(`CloudianHyperStoreObjectStoreDriverImpl`) and util
+(`CloudianHyperStoreUtil`) are the template this design follows almost line for
+line. Cloudian even validates the quota limitation the same way this design
+proposes for the MVP: `setBucketQuota` throws for any non-zero size and only
+accepts `0` (no quota).
+
+The SeaweedFS plugin is therefore a **simpler Cloudian** — same AWS S3 + IAM SDK
+clients, same store-details keys (`s3Url`, `iamUrl`, `accesskey`, `secretkey`),
+same IAM-user-with-restricted-policy pattern, but with no proprietary admin
+client at all (Cloudian has its own `CloudianClient` for its admin API; SeaweedFS
+needs only S3 + IAM). For quota, the plugin uses a narrow SeaweedFS S3 extension
+(see below); for usage reporting, it falls back to S3 `ListObjectsV2` in the MVP
+and recommends Prometheus or SOSAPI `capacity.xml` for production scale.
+
+### Quota via the S3 `?seaweedfs-quota` extension
+
+SeaweedFS supports bucket quota natively (server-side enforcement via a
+read-only flag when usage exceeds the limit). Rather than exposing the broad
+admin REST API (which would require a global bearer token and grant cluster-wide
+admin access), the integration uses a **narrow, scoped S3 subresource**:
+
+- `PUT /{bucket}?seaweedfs-quota` — set bucket quota (IAM permission `s3:PutBucketQuota`)
+- `GET /{bucket}?seaweedfs-quota` — get bucket quota (IAM permission `s3:GetBucketQuota`)
+
+**PUT request body** (JSON):
+```json
+{"quota_size": 100, "quota_unit": "GB", "quota_enabled": true}
+```
+
+**GET response body** (JSON):
+```json
+{"quota_size": 107374182400, "quota_unit": "B", "quota_enabled": true}
+```
+
+Note: GET always returns `quota_unit: "B"` and the absolute byte count, not
+the original unit. A disabled-but-retained quota returns a positive
+`quota_size` with `quota_enabled: false`.
+
+Quota is stored on the bucket's filer entry (positive = enabled, negative =
+disabled but retained, zero = no quota), matching the existing admin REST API
+behavior. When quota is cleared, the bucket's read-only flag is also lifted.
+
+**Authentication** uses the existing S3 SigV4 flow — no new global secret is
+needed. The CloudStack service credential (the `accesskey`/`secretkey` on the
+object store) is the admin credential used for all driver operations: bucket
+CRUD, IAM user provisioning, and quota management. It must have broad S3 and
+IAM permissions. The per-account IAM users created by `createUser` are the
+ones with restricted permissions (full S3 access except bucket
+creation/deletion). A future hardening could split quota management onto a
+separate credential scoped to only `s3:PutBucketQuota`/`s3:GetBucketQuota`,
+but the MVP uses the single admin credential for simplicity, matching how
+the MinIO and Ceph providers work.
+
+The plugin's `setBucketQuota` signs and sends the `PUT /{bucket}?seaweedfs-quota`
+request using the AWS SDK v1 `AWSS3V4Signer` for SigV4 signing, then sends the
+signed request via `java.net.http.HttpClient` (the AWS S3 SDK doesn't natively
+support custom subresources, so we sign manually and send the request
+ourselves). The `seaweedfs-quota` query parameter is included in the signed
+canonical query string.
+
+### Usage reporting
+
+`getAllBucketsUsage` must return a `Map<String, Long>` of bucket name → size.
+MinIO uses `MinioAdminClient.getDataUsageInfo`; Ceph uses
+`RgwAdmin.listBucketInfo`. SeaweedFS has no admin rollup endpoint, so the MVP
+plugin computes it by listing buckets and summing object sizes via S3
+`ListObjectsV2` — expensive for large stores.
+
+For production scale, SeaweedFS already exposes per-bucket size in:
+- **Prometheus metrics** (`bucket_size_bytes` gauge, refreshed every minute)
+- **SOSAPI `capacity.xml`** (reports capacity, available space, and usage
+  through the S3 endpoint)
+
+Operators should consume one of those instead of S3 list-based aggregation for
+large deployments. The MVP's list-based approach is correct but slow; flag it as
+a known limitation.
+
+## SeaweedFS API surface (what the plugin relies on)
+
+SeaweedFS exposes two relevant APIs, both AWS-compatible:
+
+### S3 API (`weed s3`)
+Full S3-compatible surface. Confirmed against the SeaweedFS S3 wiki and code:
+- `CreateBucket`, `HeadBucket`, `ListBuckets`, `DeleteBucket`
+- `PutBucketPolicy`, `GetBucketPolicy`, `DeleteBucketPolicy`
+- `PutBucketVersioning` (Enabled / Suspended), `GetBucketVersioning`
+- `PutBucketEncryption`, `GetBucketEncryption`, `DeleteBucketEncryption`
+- `PutBucketAcl`, `GetBucketAcl`
+- `ListObjectsV2`, `HeadObject`, `GetObject`, `PutObject`, `DeleteObject`
+- Bucket quota via extended attributes / `s3.bucket.quota` (enforced server-side,
+  surfaced as a read-only state when exceeded — see PR #10224)
+
+### IAM API (`weed iam` / `iamapi`)
+AWS IAM-compatible REST endpoints, implemented in `weed/iamapi/`. Confirmed by
+the test suite which uses the **AWS IAM SDK** (`aws-sdk-go/service/iam`) against
+the same handlers CloudStack would call:
+- `CreateUser`, `DeleteUser`, `ListUsers`, `GetUser`
+- `CreateAccessKey`, `DeleteAccessKey`, `ListAccessKeys`
+- `PutUserPolicy`, `GetUserPolicy`, `DeleteUserPolicy`
+- `AttachUserPolicy`, `ListAttachedUserPolicies`
+
+This means the CloudStack plugin can manage SeaweedFS users with the **AWS IAM
+Java SDK** (`com.amazonaws.services.identitymanagement.AmazonIdentityManagement`),
+exactly the way the AWS IAM Go SDK is used in SeaweedFS's own tests. No proprietary
+admin client is needed. **Cloudian HyperStore already does exactly this** in the
+CloudStack tree — the SeaweedFS plugin follows the same pattern.
+
+## Design
+
+### Module layout
+
+New CloudStack plugin module, mirroring `plugins/storage/object/cloudian/`
+(the closest precedent — same AWS S3 + IAM SDK approach):
+
+```
+plugins/storage/object/seaweedfs/
+  pom.xml
+  src/main/java/org/apache/cloudstack/storage/datastore/
+    driver/SeaweedFSObjectStoreDriverImpl.java
+    lifecycle/SeaweedFSObjectStoreLifeCycleImpl.java
+    provider/SeaweedFSObjectStoreProviderImpl.java
+    util/SeaweedFSObjectStoreUtil.java
+  src/test/java/org/apache/cloudstack/storage/datastore/
+    driver/SeaweedFSObjectStoreDriverImplTest.java
+    provider/SeaweedFSObjectStoreProviderImplTest.java
+  src/main/resources/META-INF/cloudstack/storage-object-seaweedfs/
+    module.properties
+    spring-storage-object-seaweedfs-context.xml
+```
+
+### `SeaweedFSObjectStoreProviderImpl`
+
+Direct copy of `MinIOObjectStoreProviderImpl` with `providerName = "SeaweedFS"`,
+injecting the SeaweedFS lifecycle and driver. Registers via
+`storeMgr.registerDriver`.
+
+### `SeaweedFSObjectStoreLifeCycleImpl`
+
+Copy of `MinIOObjectStoreLifeCycleImpl`. `initialize()` reads `url`, `name`,
+`accesskey`, `secretkey` from the `addObjectStoragePool` details map, tests the
+connection by calling `AmazonS3.listBuckets()` against the SeaweedFS S3 endpoint,
+and persists the `ObjectStoreVO`. No proprietary client needed — the AWS S3 SDK
+is enough for the health check.
+
+### `SeaweedFSObjectStoreDriverImpl`
+
+The substantive class. Uses two AWS SDK v1 clients (same dependency Ceph already
+pulls in, so no new CloudStack dependency):
+
+- `AmazonS3` for bucket operations (path-style, endpoint-pinned, `us-east-1`
+  region placeholder — same as Ceph's `getS3Client`)
+- `AmazonIdentityManagement` for user/credential operations, pointed at the
+  SeaweedFS IAM endpoint
+
+#### Bucket operations — straightforward S3
+
+| Interface method | Implementation |
+| --- | --- |
+| `createBucket` | `s3.createBucket(name)`; reject if `doesBucketExistV2`; persist access/secret key + URL on `BucketVO` (same as Ceph) |
+| `listBuckets` | `s3.listBuckets()` → wrap as `BucketObject` (same as Ceph) |
+| `deleteBucket` | `s3.deleteBucket(name)` (same as Ceph) |
+| `setBucketPolicy` | `s3.setBucketPolicy(...)` with the same public/private JSON the MinIO/Ceph drivers build |
+| `getBucketPolicy` / `deleteBucketPolicy` | `s3.getBucketPolicy` / `s3.deleteBucketPolicy` |
+| `setBucketVersioning` | `s3.setBucketVersioningConfiguration(Enabled)` |
+| `deleteBucketVersioning` | `s3.setBucketVersioningConfiguration(Suspended)` |
+| `setBucketEncryption` | `s3.setBucketEncryptionConfiguration(SSE-S3 rule)` |
+| `deleteBucketEncryption` | `s3.deleteBucketEncryptionConfiguration` |
+| `getBucketAcl` / `setBucketAcl` | no-op / null (same as MinIO and Ceph) |
+
+#### User creation — the key difference
+
+MinIO calls `MinioAdminClient.addUser`; Ceph calls `RgwAdmin.createUser`. SeaweedFS
+exposes the standard AWS IAM API, so the plugin calls:
+
+```java
+AmazonIdentityManagement iam = getIamClient(storeId);
+String userName = "acs-" + account.getUuid();
+
+// CreateUser (idempotent — check GetUser first, like Ceph does)
+iam.createUser(new CreateUserRequest(userName));
+
+// CreateAccessKey → returns the access key + secret key to persist
+CreateAccessKeyResult result = iam.createAccessKey(
+    new CreateAccessKeyRequest().withUserName(userName));
+AccessKey key = result.getAccessKey();
+
+// Persist per-account, same pattern as Ceph's CEPH_ACCESS_KEY/CEPH_SECRET_KEY
+details.put(SEAWEEDFS_ACCESS_KEY, key.getAccessKeyId());
+details.put(SEAWEEDFS_SECRET_KEY, key.getSecretAccessKey());
+_accountDetailsDao.persist(accountId, details);
+```
+
+This is the cleanest mapping of the three providers: no proprietary admin client,
+just the AWS IAM SDK that CloudStack already has access to. The IAM endpoint URL
+is provided as `iamUrl` in the store details. If `iamUrl` is omitted, the driver
+defaults it to `s3Url` — SeaweedFS registers its embedded IAM API at `POST /` on
+the same S3 endpoint (`UnifiedPostHandler` in `s3api_server.go`), so the IAM
+endpoint is the same as the S3 endpoint unless the deployment runs a separate
+`weed iam` server.
+
+#### Bucket quota — S3 `?seaweedfs-quota` extension
+
+This is the one genuine gap. MinIO and Ceph both have an admin API to set a
+per-bucket quota that the backend enforces. SeaweedFS enforces bucket quota
+server-side, but the configuration path was **not exposed over a standard S3 or
+IAM API** — it was only set via the admin REST API or shell commands.
+
+The integration adds a **narrow S3 subresource** to SeaweedFS:
+- `PUT /{bucket}?seaweedfs-quota` — set bucket quota (IAM permission `s3:PutBucketQuota`)
+- `GET /{bucket}?seaweedfs-quota` — get bucket quota (IAM permission `s3:GetBucketQuota`)
+
+This is implemented in SeaweedFS PR #11279. It uses SigV4 authentication and
+dedicated IAM permissions, so the CloudStack service credential can be scoped
+to quota management only — no global admin token, no cluster-wide admin access.
+The enforcement already exists (PR #10224); this PR only adds the HTTP
+configuration surface.
+
+An earlier approach (PR #11278, closed) added bearer-token auth to the broad
+admin REST API. After review, that was unnecessary for this integration —
+static S3 config plus standard S3 APIs plus one scoped quota mutation API is
+sufficient and far safer.
+
+> **Note on AWS tools compatibility.** `?seaweedfs-quota` is a SeaweedFS-specific
+> S3 subresource, not part of the AWS S3 API. Standard AWS tools (`aws s3api`,
+> `s3cmd`, `rclone`) cannot call it directly. This is the same limitation MinIO
+> and Ceph have — MinIO quota lives behind a separate admin API (`mc admin
+> bucket quota`), and Ceph quota lives behind the Admin Ops API
+> (`radosgw-admin quota set`). Neither is callable via `aws s3api` either.
+> SeaweedFS's approach is the closest to standard S3 because it uses the same
+> endpoint and same SigV4 credentials, just with a custom query parameter.
+> Interactive quota management remains available via `weed shell`; the S3
+> extension exists for programmatic integration (CloudStack) where the
+> integrator can sign SigV4 requests but cannot run shell commands.
+
+#### Usage reporting
+
+`getAllBucketsUsage` must return a `Map<String, Long>` of bucket name → size.
+MinIO uses `MinioAdminClient.getDataUsageInfo`; Ceph uses
+`RgwAdmin.listBucketInfo`. SeaweedFS has no admin rollup endpoint, so the MVP
+plugin computes it by listing buckets and summing object sizes via S3
+`ListObjectsV2` — expensive for large stores. Better options exist in
+SeaweedFS already:
+- **Prometheus metrics** (`bucket_size_bytes` gauge, refreshed every minute)
+- **SOSAPI `capacity.xml`** (reports capacity, available space, and usage
+  through the S3 endpoint — note: the current "return zero on backend error"
+  behavior should be validated before using it for billing)
+
+For the MVP, `listBuckets` + per-bucket size via the S3 API is correct but slow;
+flag it as a known limitation. Operators should consume Prometheus or SOSAPI
+for production-scale usage reporting.
+
+### Spring wiring
+
+`spring-storage-object-seaweedfs-context.xml` registers the provider bean,
+identical to the MinIO one. `module.properties` sets
+`name=storage-object-seaweedfs`, `parent=storage`.
+
+### `pom.xml`
+
+Depends on `aws-java-sdk-s3` and `aws-java-sdk-iam` — both already in the
+CloudStack dependency tree (Ceph uses the S3 SDK; the IAM SDK is the standard AWS
+bundle). No new third-party dependency, unlike MinIO which pulls in the MinIO
+Java client.
+
+## What changes on the SeaweedFS side
+
+**One narrow S3 extension is required for quota management.** SeaweedFS PR #11279
+adds the `?seaweedfs-quota` S3 subresource:
+
+- `PUT /{bucket}?seaweedfs-quota` — set bucket quota (IAM permission `s3:PutBucketQuota`)
+- `GET /{bucket}?seaweedfs-quota` — get bucket quota (IAM permission `s3:GetBucketQuota`)
+
+This is authenticated via standard S3 SigV4 and authorized via dedicated IAM
+permissions, so no global admin token is needed. The enforcement already exists
+(PR #10224); this PR only adds the HTTP configuration surface.
+
+One follow-up improvement on the SeaweedFS side would close the usage reporting
+gap:
+
+1. **Validate SOSAPI `capacity.xml` usage calculation** — the current "return
+   zero on backend error" behavior should be validated before using it for
+   billing. If reliable, CloudStack can consume it directly instead of
+   list-based aggregation.
+
+## Open questions for proIO / Swen
+
+1. **IAM endpoint path.** ~~Where does `weed iam` listen relative to the S3
+   endpoint in a typical proIO deployment?~~ **Resolved.** SeaweedFS registers
+   its embedded IAM API at `POST /` on the same S3 endpoint
+   (`UnifiedPostHandler`), so the driver defaults `iamUrl` to `s3Url`. A
+   separate `iamUrl` is only needed if the deployment runs a standalone
+   `weed iam` server on a different host/port.
+2. **Quota requirements.** Do proIO's customers need server-enforced per-bucket
+   quotas, or is CloudStack-side accounting sufficient for the first release?
+   The `?seaweedfs-quota` S3 extension (PR #11279) provides server-enforced
+   quotas via a scoped credential; this is the recommended path.
+3. **Object Lock.** `createBucket` takes an `objectLock` boolean. MinIO supports
+   it; Ceph ignores it. SeaweedFS has Object Lock support. Should the plugin pass
+   it through?
+4. **Contribution model.** Does proIO want to submit the PR to
+   `apache/cloudstack` themselves (with SeaweedFS maintainers as reviewers), or
+   the reverse? Apache CloudStack requires an ICLA for non-trivial contributions.
+
+## Files
+
+All in the `apache/cloudstack` repo (new module):
+
+| File | Purpose |
+| --- | --- |
+| `plugins/storage/object/seaweedfs/pom.xml` | Maven module |
+| `.../datastore/util/SeaweedFSObjectStoreUtil.java` | S3 + IAM client builders, constants, URL validators |
+| `.../datastore/provider/SeaweedFSObjectStoreProviderImpl.java` | Spring provider registration |
+| `.../datastore/lifecycle/SeaweedFSObjectStoreLifeCycleImpl.java` | Pool add/health-check |
+| `.../datastore/driver/SeaweedFSObjectStoreDriverImpl.java` | Bucket + user ops via S3 + IAM SDK |
+| `.../resources/META-INF/cloudstack/storage-object-seaweedfs/module.properties` | Module name |
+| `.../resources/META-INF/cloudstack/storage-object-seaweedfs/spring-storage-object-seaweedfs-context.xml` | Spring bean |
+| `plugins/pom.xml` | Register `storage/object/seaweedfs` module |
+
+No files in `seaweedfs/seaweedfs` for the MVP.
+
+### SeaweedFS-side changes (PR #11279)
+
+| File | Purpose |
+| --- | --- |
+| `weed/s3api/s3_constants/s3_action_strings.go` | Add `S3_ACTION_PUT_BUCKET_QUOTA` and `S3_ACTION_GET_BUCKET_QUOTA` |
+| `weed/s3api/s3_constants/s3_actions.go` | Add coarse-grained `ACTION_PUT_BUCKET_QUOTA` and `ACTION_GET_BUCKET_QUOTA` |
+| `weed/s3api/s3_action_resolver.go` | Map `seaweedfs-quota` query param to fine-grained s3: actions |
+| `weed/s3api/s3api_bucket_quota_handlers.go` | New — `PutBucketQuotaHandler` and `GetBucketQuotaHandler` |
+| `weed/s3api/s3api_bucket_quota_handlers_test.go` | New — tests for unit conversion, validation, and error paths |
+| `weed/s3api/s3api_server.go` | Register the two routes in the bucket subrouter |

design-lance-catalog.md

@@ -0,0 +1,696 @@
+# Lance Catalog for SeaweedFS
+
+A second catalog surface next to the Iceberg REST catalog, speaking the Lance Namespace
+REST spec, over the same table buckets and the same filer.
+
+## Why
+
+Gravitino 1.1 added a Lance REST service and 1.3 ships it as a standalone server; Lakekeeper
+added Lance in the same window by a completely different route. That is the useful signal:
+two unrelated catalogs decided independently that Lance had to be first-class, not a niche.
+The client side is already there — `lance-spark` (`LanceNamespaceSparkCatalog`
+with `impl=rest`), `lance-ray`, and the generated Python/Java/Rust clients all talk the same
+OpenAPI. Implementing the spec means those engines work against SeaweedFS with no
+SeaweedFS-specific code on the client.
+
+The second reason is that Gravitino's own documentation names the gap it cannot close:
+DuckDB, pandas and DataFusion "do not support Lance REST natively yet" and have to fetch a
+location from the catalog and then open the dataset directly. Gravitino cannot help there,
+because it does not own the storage. SeaweedFS does. That is the whole design opportunity
+below.
+
+## Prior art: three families
+
+Upstream lists twelve catalog implementations, and they fall into three shapes. Knowing
+which one we are building matters more than any individual API decision.
+
+**1. Storage-native, no service.** The Lance Directory Catalog. V1 is a directory listing
+where every `<name>.lance/` child of a prefix is a table; V2 adds a `__manifest` table —
+itself a Lance table — holding `object_id`/`object_type`/`location` rows, with nested
+namespaces, hash-prefixed table directories, and optional managed versioning. No server, no
+credentials, no governance. This is the floor every other implementation has to beat.
+
+**2. Protocol-native server.** Someone implements the Lance Namespace REST OpenAPI and
+clients connect with `impl=rest`. Gravitino is the only one of the twelve that does this,
+and it is what this design proposes.
+
+**3. Client-side adapters onto an existing catalog.** Nine of the twelve. The Lance client
+translates namespace operations into whatever the backing catalog already speaks: Apache
+Polaris, Unity Catalog, AWS Glue, Hive Metastore v2 and v3, Google BigLake, Dataproc,
+Microsoft OneLake — and Apache Iceberg REST. Two flavors:
+
+- Catalogs with a real non-Iceberg table concept mark the format directly. Polaris uses its
+  Generic Table API with `format = lance`; Unity uses an `EXTERNAL` table with
+  `table_type=lance` in properties and the path in `storage_location`; Glue uses
+  `EXTERNAL_TABLE` plus `table_type=lance` in `Parameters`, path in
+  `StorageDescriptor.Location`.
+- Catalogs with no such concept fake one. The Iceberg REST adapter registers **a regular
+  Iceberg table with a dummy schema — a single nullable string column named `dummy`** —
+  carrying the property `table_type=lance`, and treats the Iceberg table location as the
+  Lance dataset root.
+
+Every adapter in family 3 lands in the same place: `DeclareTable`/`ListTables`/
+`DescribeTable`/`DeregisterTable` only, `DropNamespace` in RESTRICT mode only,
+`load_detailed_metadata=false` only, and `managed_versioning=false`. They are a name-to-
+location map and nothing more.
+
+Lakekeeper is the instructive outlier. It has the same generic-table concept Polaris has,
+but no upstream adapter exists for it — there is no `lance-namespace` reference anywhere in
+its repository and no page for it in the supported-catalogs list. So Polaris's generic tables
+are reachable from a stock Lance client and Lakekeeper's are not, despite being the same
+idea. Shipping the concept is not the same as shipping the integration.
+
+## Gravitino and Lakekeeper: the two opposite bets
+
+Both shipped Lance support in the same window and did not build the same thing.
+
+**Gravitino implements the protocol.** Its `lance/` module serves the Lance Namespace REST
+spec on its own port (`:9101/lance`), so stock `lance-spark` and `lance-ray` connect with
+`impl=rest` and no vendor-specific client. The cost is governance: storage credentials are
+static properties on the catalog (`lance.storage.access_key_id`, `secret_access_key`,
+`endpoint`, `region`, `allow_http`), optionally overridden per table, handed to the engine
+as-is. No STS, no expiry, no per-table scoping.
+
+**Lakekeeper refuses the protocol and governs the object instead.** There is no
+`lance-namespace` anywhere in the repository; Lance arrived in 0.13.0 (2026-06-30, issue
+#1673 `Generic Table API with Lance`) as one `format` string on a Lakekeeper-native Generic
+Table API:
+
+```
+POST/GET/DELETE /lakekeeper/v1/{prefix}/namespaces/{ns}/generic-tables[/{table}]
+GET             /lakekeeper/v1/{prefix}/namespaces/{ns}/generic-tables/{table}/credentials
+POST            /lakekeeper/v1/{prefix}/generic-tables/rename
+```
+
+`format` is opaque, `schema` and `statistics` are stored but never validated, and the
+catalog writes no format-specific metadata — engines go straight to the location. In
+exchange Lance tables get everything Iceberg tables get: STS-vended prefix-scoped
+credentials, OpenFGA per-action permissions (16 actions), soft-delete with undrop, a
+protection flag, rename, pagination, and name uniqueness across Iceberg tables, views and
+generic tables in one namespace. The price is that no stock Lance client can talk to it —
+you need `pylakekeeper`, which exists mainly to translate vended credentials into
+`lance_storage_options`.
+
+So: protocol fidelity and weak governance, or strong governance and client lock-in. Both
+documented their limit honestly, and it is the same limit. Lakekeeper's capability table
+says it outright — "Commit coordination: the catalog does not arbitrate writes — engines
+write directly." Gravitino does not claim it either. Neither of them coordinates a Lance
+commit, which is exactly the thing a store can do and a control plane cannot.
+
+We do not have to choose. Serve the Lance protocol natively the way Gravitino does, over
+the `s3tables` entries that already carry ARNs, policies, tags and maintenance config, and
+the governance comes from the layer underneath rather than from a proprietary API on top.
+That is only available to us because we are the store, which is also what makes the third
+option — arbitrating the commit — available.
+
+## We are probably already a Lance catalog, and that is a problem
+
+The Iceberg REST adapter does not care whose Iceberg catalog it is talking to. It needs
+`/v1/config?warehouse=`, `/v1/{prefix}/namespaces`, `/v1/{prefix}/namespaces/{ns}/tables`
+and unit-separator (`\x1F`) multi-level namespaces. We serve all of those, and
+`parseNamespace` in `weed/s3api/iceberg/utils.go:22` already splits on `\x1F`. So a stock
+Lance client pointed at our Iceberg catalog on :8181 with the Iceberg impl should already
+create, list, describe and deregister Lance tables today, with no SeaweedFS change at all.
+
+That is worth testing before writing a line of the design above, for two reasons. It is a
+free baseline — and possibly a free announcement. And it is a data-loss hazard.
+
+A Lance table registered this way is an Iceberg table whose metadata references no data
+files, sitting on top of a Lance dataset that uses `data/` for its fragments — the same
+subdirectory name Iceberg uses. The maintenance worker's orphan cleaner walks exactly
+`<table>/metadata` and `<table>/data`, and deletes every file not referenced by a snapshot
+and older than `orphan_older_than_hours`
+(`weed/worker/tasks/iceberg/operations.go:331`, default 72). Against an adapter-registered
+Lance table, every fragment is unreferenced by construction. Run maintenance and the
+dataset is deleted.
+
+Maintenance is disabled by default (`handler.go:334`), so this is a latent hazard rather
+than a live one: it needs an operator to enable Iceberg maintenance on a bucket that also
+holds adapter-registered Lance tables. But it costs nothing to close — detection should
+skip any table carrying a non-Iceberg format marker (`table_type` property, or
+`Format != "ICEBERG"` once the format field is honest), and that guard is worth landing on
+its own regardless of whether the rest of this design ever gets built. It is the same
+"catalog-only, no maintenance" marker the generic-format question needs.
+
+## Where we differ from Gravitino
+
+Gravitino is a metadata service in front of somebody else's object store:
+
+```
+  Spark / Ray                  Spark / Ray / pandas / duckdb
+      |                              |
+  Lance REST                    Lance REST          (direct S3)
+      |                              |                   |
+  Gravitino                     SeaweedFS S3 gateway ----+
+      |                              |
+   S3 keys handed out           SeaweedFS filer + volumes
+      |
+  somebody else's S3
+```
+
+It resolves a name to a location plus `lance.storage.*` credentials, and steps out of the
+way. Everything a Lance table actually is — `_versions/`, `data/`, `_indices/` — is opaque
+to it.
+
+We are the store. Three things follow that Gravitino cannot do:
+
+1. The catalog and a plain directory listing can be made to agree, so a client with no
+   catalog at all still sees the right tables.
+2. `_versions/` is a filer directory listing, not an object-store `LIST`. Version history
+   is cheap and can back the admin UI.
+3. We can offer a genuinely atomic commit reservation. Lance's commit protocol needs
+   put-if-not-exists; our S3 layer does not currently provide one (see
+   [Commit safety](#commit-safety)). The filer does.
+
+## Placement
+
+The Iceberg catalog is a thin HTTP shell over `s3tables.Manager`; the storage work lives in
+`weed/s3api/s3tables`. Table buckets live under `TablesPath = s3_constants.DefaultBucketsPath`,
+i.e. the same filer tree the S3 gateway serves, so `s3://bucket/ns/table/` is simultaneously
+a catalog entry and an S3 prefix. Catalog entries are filer directories carrying `s3tables.*`
+extended attributes. `Table.Format` already exists and is hard-checked against `"ICEBERG"`
+in `weed/s3api/s3tables/handler_table.go:48`.
+
+So:
+
+```
+weed/s3api/lance/          new: HTTP surface, id codec, error model
+weed/s3api/s3tables/       extended: Format "LANCE", lance state xattr, version entries
+weed/command/s3.go         new: -port.lance (default 9101), startLanceServer
+```
+
+`Format: "LANCE"` on the table entry is the whole storage-model change for phase 1.
+Everything else — namespaces, ARNs, policies, tags, ownership — is shared verbatim.
+
+```
+                        s3tables.Manager (filer)
+                                 |
+        +------------------------+------------------------+
+        |                                                 |
+  weed/s3api/iceberg                              weed/s3api/lance
+   Iceberg REST :8181                              Lance REST :9101
+        |                                                 |
+  Iceberg tables                                   Lance datasets
+        \                                                 /
+         +-------------------- s3 :8333 -----------------+
+                                 |
+                         SeaweedFS volumes
+```
+
+## Identifier mapping
+
+Lance identifiers are `["ns", ..., "table"]`, encoded in the URL as a single string joined
+by a delimiter that defaults to `$`. The delimiter alone means the root namespace, so
+`/v1/namespace/$/list` lists the root's children.
+
+Iceberg had to invent a warehouse selector because its identifier is flat and every table
+bucket is a separate catalog. Lance does not need that — its identifier is already
+hierarchical, and Gravitino uses exactly three levels (`["lance_catalog", "sales", "orders"]`).
+That maps onto us without inventing anything:
+
+```
+  $                            root      -> list of table buckets
+  $analytics                   level 1   -> a table bucket
+  $analytics$sales             level 2   -> a namespace in that bucket
+  $analytics$sales$orders      table
+```
+
+`spark.sql.catalog.lance.parent = analytics` then makes `sales.orders` resolve, which is the
+same shape Gravitino's Spark example uses.
+
+Levels 2..N join into one `s3tables` namespace with `.`, matching what the Iceberg catalog
+already does with `flattenNamespacePath`. The flattened form is only the directory name —
+`namespaceMetadata.Namespace []string` in the xattr keeps the authoritative parts, so the
+mapping stays invertible even though `.` is a legal character inside a namespace part.
+Reject `$` in any name part with `InvalidInput`; our charsets already exclude it, so no
+escaping scheme is needed.
+
+Root-level `ListNamespaces` returning table buckets means an unauthenticated or
+broadly-scoped caller can enumerate buckets. Filter it through the same
+`s3tables/permissions.go` check `ListTableBuckets` uses, not a separate path.
+
+`CreateNamespace` on a one-part identifier creates a table bucket, and it does so only if
+the caller is permitted to — the namespace never creates a bucket as a side effect of
+creating something inside it. A table bucket is a tenant resource with its own policy, ARN
+and lifecycle, and conjuring one because a client said `CREATE SCHEMA` is a privilege
+escalation dressed as a convenience. Lakekeeper draws the same line explicitly: its client
+creates tables, not warehouses.
+
+## Storage layout
+
+Lay tables out as:
+
+```
+s3://<table-bucket>/<flattened-namespace>/<table>/
+    data/
+    _versions/
+    _indices/
+```
+
+**Built without the `.lance` suffix this design originally proposed.** The suffix would have
+made every namespace prefix a valid Lance Directory Catalog V1 root, since V1 recognises a
+table by exactly that naming. It does not survive contact with the storage layer: the
+catalog entry *is* the dataset directory, `validateTableName` excludes `.` from the charset,
+and a suffixed entry name would leak into ARNs, policy documents and the S3 Tables API,
+where the same table would answer to two different names. Making `GetTablePath` format-aware
+instead spreads an "unless it is Lance" branch through code that has no business knowing —
+the exact cross-cutting cost this design rejects family 3 for.
+
+So one name, one directory. What survives is direct access by URI, which is the larger half
+of the story and needs no naming convention at all:
+
+```python
+# with the catalog
+spark.sql("SELECT * FROM lance.sales.orders")
+
+# without it, same bytes
+lance.dataset("s3://analytics/sales/orders")
+```
+
+DuckDB, pandas and DataFusion still reach the data with no catalog running, which is the gap
+Gravitino's documentation admits to. What they no longer get for free is *enumeration* — a
+directory-catalog client pointed at the namespace prefix will not list these as tables. If
+that turns out to matter, the cheapest fix is a repair-style tool that materialises `.lance`
+aliases, not a rename of the catalog entry.
+
+Note also that the directory catalog's own V2 mode puts child-namespace tables in
+`<hash>_<ns$table>` directories at the root and creates no physical subdirectories for
+namespaces, so full directory-catalog fidelity was never on offer anyway. We are a
+server-backed catalog; the human-readable prefix layout is worth more than partial V1
+lookalike behaviour.
+
+## The table bucket was not a neutral container
+
+This design assumed a table bucket is a place to put a table's files. It is
+not: `validateTableBucketObjectPath` runs on every S3 write into one and
+validated the path against Iceberg's layout, so a Lance client got 403 on
+`data/*.lance`, on `_versions/`, and on `_transactions/` — a directory Lance
+writes that neither the spec documentation nor this design anticipated. Nothing
+about the catalog worked end to end until that changed.
+
+The layout guard now admits the union of what the supported formats write, and
+treats any underscore-prefixed top-level directory as belonging to the format,
+checking only that the path stays inside the table. Enumerating Lance's
+internal directories by name is exactly the mistake that missed
+`_transactions`. Iceberg writes none of them, so it loses nothing.
+
+Found by pointing the real Python client at a running gateway, not by reading
+the spec. Worth remembering for the next format: the premise to check first is
+whether the storage layer will accept its files at all.
+
+## Table lifecycle
+
+Lance has three table states, and the spec pins them to marker files:
+
+| State | Marker | Created by | Visible in ListTables |
+| --- | --- | --- | --- |
+| declared | `.lance-reserved` | `DeclareTable` | yes, when `include_declared=true` |
+| created | `_versions/` present | client writes, or `CreateTable` | yes |
+| deregistered | `.lance-deregistered` | `DeregisterTable` | no; data preserved |
+
+Record the state in an xattr (`s3tables.lanceState`) on the catalog entry *and* write the
+marker file into the table directory. The xattr is what the catalog reads; the marker is
+what keeps a directory-catalog client honest. Dual-write is the price of the interop claim
+above, and it is one extra filer write on three rarely-called operations.
+
+`DeclareTable` is the operation `lance-spark` actually calls on `CREATE TABLE` (it replaced
+the legacy `create-empty`), so it is not optional in practice even though the spec marks
+only a subset as required.
+
+`DeregisterTable` preserving data is the same shape as our Iceberg rename, where the catalog
+entry moves and the data stays put — reuse `TableDataDirFromMetadataLocation`'s idea rather
+than re-deriving the data path from the catalog name.
+
+## Commit safety
+
+This is the part I got wrong, and the correction removed a feature rather than adding one.
+
+Lance commits a version by writing `_versions/{v}.manifest` with put-if-not-exists: exactly
+one writer is supposed to win, and the loser rebases. In lance 10 that path is not optional
+and needs nothing bolted on — `commit_handler_from_url` hands every `s3://` dataset a
+`ConditionalPutCommitHandler`, which calls `put_opts` with `PutMode::Create`, which
+object_store's S3 backend sends as `If-None-Match: *`.
+
+I originally read our gateway as evaluating that header check-then-act, and designed around
+it. That was already out of date. `buildWriteCondition`
+(`weed/s3api/s3api_object_routed_write.go`) reduces `If-None-Match: *` to a filer
+`WriteCondition{IF_NOT_EXISTS}`, and `putToFiler` routes the create to the object's owner
+filer, which evaluates the precondition under its per-path lock; when routing is not
+available it falls back to the object write lock, which evaluates it under the lock too.
+Either way it is atomic. Sixteen concurrent writers of the same fresh key get one 200 and
+fifteen 412s, repeatedly.
+
+So the store already has the primitive Lance needs, cluster-wide, for every conditional-PUT
+client and not just this one.
+
+### What that removed
+
+An earlier draft of this design offered the catalog as an **external manifest store**:
+`managed_versioning: true` plus `CreateTableVersion` and friends, with the reserve step as a
+filer `CreateEntry` with `o_excl`. It was implemented, tested, and shipped behind a default-off
+flag — and it should not exist.
+
+- It solves a problem this store does not have. The spec offers that path for stores that
+  cannot order commits themselves.
+- It moves a table's version history out of the dataset and into the catalog, so a reader
+  that does not go through this namespace no longer sees the whole picture. That is a real
+  cost paid for nothing.
+- lance 10 cannot even use it past the first commit: `NamespaceManifestStore::put_if_not_exists`
+  answers "put_if_not_exists is not supported for namespace-backed stores", which is exactly
+  what a second `append` needs.
+
+The version operations now answer `Unsupported` alongside the other operations the catalog
+does not serve, and `managed_versioning` is answered `false`. The property they were
+protecting is covered instead by a test that races eight writers at the manifest key through
+S3 and asserts one wins — testing the path Lance actually takes.
+
+## Credential vending
+
+Iceberg needed a header (`X-Iceberg-Access-Delegation: vended-credentials`) and a bespoke
+response shape. Lance has it in the spec: `vend_credentials: true` on the request,
+`storage_options` on the response, with `expires_at_millis` as the well-known expiry key.
+
+Reuse the existing vendor interface unchanged — `iceberg.CredentialVendor` /
+`STSService.AssumeRoleForPrincipal` scoped to the table prefix (#10777) — and map its output
+to the storage options Lance passes through to `object_store`:
+
+```
+aws_access_key_id, aws_secret_access_key, aws_session_token,
+aws_region, aws_endpoint, allow_http, expires_at_millis
+```
+
+Those are the names `pylakekeeper` emits as `lance_storage_options`, which is the shape
+Lakekeeper's tested S3 path actually feeds to Lance. `object_store` also accepts the
+un-prefixed aliases (`endpoint`, `region`) that the directory catalog's `storage.` prefix
+strips down to and that Gravitino's `lance.storage.endpoint` resolves to, but the `aws_`
+forms are the ones with a tested integration behind them, so emit those. `aws_endpoint`
+should come from `deriveS3AdvertisedEndpoint()`, the same source the Iceberg `FileIO` config
+uses, and `allow_http` must be set when that endpoint is plain HTTP or every read fails with
+a TLS error that looks like a credential problem — Lakekeeper vends both automatically for
+exactly this reason, and calls out that there is then no per-vendor branch in client code.
+
+We emit this server-side, in the `storage_options` field the Lance spec already defines,
+which is strictly better than Lakekeeper's arrangement: no client library has to translate
+anything, so vending works from any stock Lance client rather than only from theirs.
+
+Guard the same way #10777 had to after review: bucket-scoped list grants need an `s3:prefix`
+condition, and a location containing `*` or `?` must be refused rather than widened into a
+resource pattern.
+
+## Auth and authorization
+
+Authentication reuses `S3Authenticator` and `CredentialValidator` as-is. The Lance spec maps
+identity to headers — `api_key` to `x-api-key`, `auth_token` to `Authorization: Bearer` — and
+SigV4 keeps working because it is the same authenticator the Iceberg catalog already fronts.
+
+Authorization needs nothing new. A Lance table gets the same ARN shape,
+`arn:aws:s3tables:...:bucket/B/table/NS/T`, so every existing table-bucket policy covers
+Lance tables with no new policy language and no second permission model. Route it through
+`s3tables/permissions.go` and inherit the `DefaultAllow` semantics the Iceberg server already
+mirrors from the S3 port.
+
+One spec quirk worth honoring: request context entries prefixed `header.` become request
+headers, and every response header comes back as a `header.`-prefixed context entry. Echoing
+`x-request-id` through it costs nothing and makes tracing work.
+
+## What to take from Lakekeeper
+
+Rejecting Lakekeeper's API shape does not mean rejecting what it learned building it.
+
+**Deregister is soft-delete, so implement it as one.** Lakekeeper gives generic tables
+soft-deletion with undrop and a `protected` flag that makes a drop require `force=true`.
+Lance already has the concept — `DeregisterTable` preserves the data and hides the table —
+so the `.lance-deregistered` marker is a soft-delete by another name, and a re-register is
+an undrop. A protection flag on table-bucket entries is worth having regardless of Lance:
+it is a few lines against the existing xattrs and it applies to Iceberg tables too.
+
+**Enforce one identifier space across entry kinds.** Lakekeeper rejects a generic table
+whose name collides with an Iceberg table or view in the same namespace. Our catalog entries
+already share one filer directory and already carry `s3tables.entryType`, so this is
+structurally true — but it has to be enforced deliberately on every path, or a Lance handler
+happily loads an Iceberg table's directory and vice versa. That is the same crossover bug
+class as the view/table rename authorization fixed in #10776; the `catalogEntryKind` pattern
+from that change is the thing to reuse rather than re-derive.
+
+**A re-vend path matters more than it looks.** Lakekeeper exposes `/credentials` separately
+from load, because STS credentials expire in the middle of long jobs and re-loading the
+whole table to refresh them is wasteful. In Lance the spec's answer is another
+`DescribeTable` with `vend_credentials: true`, which is fine — but it means `DescribeTable`
+must stay cheap when `load_detailed_metadata` is false, which is another reason not to open
+the dataset on that path.
+
+**Generic tables are a cheap orthogonal win.** Lakekeeper's real insight is that Delta,
+Parquet, CSV, Vortex and Paimon all get governance for free once the catalog stops caring
+what the format is. Our `Table.Format` field already exists and the only thing stopping it
+is the hard `"ICEBERG"` check in `handler_table.go:48`. Loosening that and letting the S3
+Tables API register a table with an arbitrary format and a location — no metadata, no
+commits — is a small change that makes every format cataloguable. It is independent of this
+design and probably worth doing first, since `Format: "LANCE"` is then just a value rather
+than a special case.
+
+**Skip remote signing.** It is Lakekeeper's fallback for S3-compatible stores with no STS,
+and their own documentation notes that Lance will not use it — format libraries with their
+own S3 client expect static credentials and do not implement the Iceberg signer protocol. We
+have STS, so vended credentials are the path, and the signer is not worth building for a
+client that cannot consume it.
+
+## Errors
+
+Lance uses `{code, error, detail, instance}` with numeric codes, not Iceberg's exception-type
+strings. The mapping is mechanical:
+
+| HTTP | code | when |
+| --- | --- | --- |
+| 400 | 13 InvalidInput | charset violations, malformed id, route/body mismatch |
+| 401 | 16 Unauthenticated | |
+| 403 | 15 PermissionDenied | |
+| 404 | 1 NamespaceNotFound, 4 TableNotFound, 11 TableVersionNotFound | |
+| 409 | 2/5 AlreadyExists, 3 NamespaceNotEmpty, 14 ConcurrentModification | |
+| 501 | 0 Unsupported | every phase-3 data operation |
+
+Route/body mismatch is a spec requirement, not a nicety: when the identifier appears in both
+the path and the body and they disagree, the server must return 400. Cheap to get right at
+the decode step, annoying to retrofit.
+
+## Route surface
+
+Phase 0 is not in this table: point a stock Lance client at the existing Iceberg catalog
+with the Iceberg impl, see how far it gets, and land the maintenance guard either way. That
+tells us what the native server actually has to beat.
+
+Phase 1, the whole `lance-spark` and `lance-ray` contract:
+
+```
+POST /v1/namespace/{id}/create         CreateNamespace       mode: Create|ExistOk|Overwrite
+GET  /v1/namespace/{id}/list           ListNamespaces
+POST /v1/namespace/{id}/describe       DescribeNamespace
+POST /v1/namespace/{id}/drop           DropNamespace         mode: Fail|Skip, behavior: Restrict|Cascade
+POST /v1/namespace/{id}/exists         NamespaceExists
+GET  /v1/namespace/{id}/table/list     ListTables            ?include_declared, ?page_token, ?limit
+GET  /v1/table                         ListAllTables
+POST /v1/table/{id}/declare            DeclareTable
+POST /v1/table/{id}/describe           DescribeTable         ?with_table_uri, ?load_detailed_metadata, ?check_declared
+POST /v1/table/{id}/exists             TableExists
+POST /v1/table/{id}/register           RegisterTable         mode: Create|Overwrite
+POST /v1/table/{id}/deregister         DeregisterTable
+POST /v1/table/{id}/drop               DropTable
+POST /v1/table/{id}/rename             RenameTable
+```
+
+`DescribeTable` with `load_detailed_metadata=false` needs only `location`, which is the
+common case and which we can answer from xattrs alone. With `load_detailed_metadata=true`
+the spec wants `version`, `schema` and `stats`, which means reading the Lance manifest. For
+phase 1, return the fields we can derive from the filer — `version` from the highest entry in
+`_versions/`, given V2 naming is `{u64::MAX - version:020}.manifest` and V1 is
+`{version}.manifest` — and omit `schema`/`stats` rather than fabricating them. The spec
+tolerates a partial response here; it does not tolerate a wrong one.
+
+Phase 2 was the five version operations plus `managed_versioning`; it was built and then
+removed, for the reasons under Commit safety.
+
+Phase 3 is the data plane: `CreateTable`, `InsertIntoTable`, `MergeInsertIntoTable`,
+`UpdateTable`, `DeleteFromTable`, `QueryTable`, `CountTableRows`, and the index and tag
+operations. These exchange Arrow IPC, and more to the point they require reading and writing
+the Lance file format, for which no Go implementation exists. Return `Unsupported` (code 0)
+and say so in the docs. `arrow-go/v18` is already an indirect dependency, so Arrow framing is
+not the blocker — Lance is.
+
+## Does a Lance table need maintenance?
+
+Yes, and one part of it has no Iceberg equivalent. The client exposes three jobs:
+
+- `optimize.compact_files()` — Lance writes a fragment per write batch, so a table fed by
+  small appends accumulates small files exactly the way an Iceberg table does.
+- `optimize.optimize_indices()` — **rows written after an index was built are not covered by
+  it.** A vector search against a stale index silently misses recent data. That is a
+  correctness-shaped failure, not a slow query, and it is specific to what people use Lance
+  for.
+- `cleanup_old_versions()` — every version is retained until something removes it. Lance can
+  do this itself: `optimize.enable_auto_cleanup()` sets it on the dataset, so this one need
+  not be an external job at all.
+
+None of it can run in the Go worker. All three read and rewrite Lance files, which needs
+Lance format code that does not exist in Go, and there is no useful subset either: deciding
+which fragments an old version still references means parsing Lance manifests.
+
+So the maintenance worker must not touch a Lance table, and it declines by reading the format
+the catalog recorded rather than by failing to parse Iceberg metadata.
+
+## The worker can be Rust, and it is not a sidecar
+
+The Go worker is not the only worker. `weed/pb/plugin.proto` defines `PluginControlService`,
+a language-agnostic gRPC stream that external maintenance workers connect on: the worker
+opens `WorkerStream`, sends `WorkerHello` with the job types it can `detect` and `execute`,
+answers `RequestConfigSchema` with a `JobTypeDescriptor`, replies to `RunDetectionRequest`
+with `JobProposal`s and to `ExecuteJobRequest` with `JobProgressUpdate`s and `JobCompleted`.
+`weed worker -admin=host:23646` is the Go reference implementation of exactly that contract,
+from outside the admin process.
+
+Nothing in it is Go-specific, and the Rust toolchain is already in the tree.
+`seaweed-volume/build.rs` compiles protos straight out of `../weed/pb/` with `tonic_build`,
+including `filer.proto`, on tonic 0.12 and prost 0.13. A Lance worker is that same build
+with `plugin.proto` added and the `lance` crate as a dependency — the real one, no FFI and
+no Python.
+
+Three job types, one per real maintenance operation:
+
+| Job type | Calls | Detected from |
+| --- | --- | --- |
+| `lance_compact` | `optimize.compact_files` | fragment count and sizes |
+| `lance_optimize_indices` | `optimize.optimize_indices` | rows an index does not cover |
+| `lance_cleanup_versions` | `cleanup_old_versions` | version count and age |
+
+What the existing machinery then supplies for free is the part worth noticing. Scheduling,
+retries, dedupe by `dedupe_key`, progress reporting, per-job concurrency limits and the
+admin settings page all come from the protocol: a worker that answers `RequestConfigSchema`
+with a descriptor gets its configuration form rendered in the admin UI without a line of Go
+or templ. A Rust worker is a first-class maintenance worker, not an appendage.
+
+The remaining wiring is small and mostly decided already. `RunDetectionRequest` carries a
+`ClusterContext` with filer and S3 addresses plus a free-form `metadata` map, which is where
+the Lance namespace URL goes; the worker lists Lance tables from the namespace, which is the
+catalog of record and already filters by format. It gets at the data by asking
+`DescribeTable` for `storage_options` with `vend_credentials`, so the worker is just another
+client of the STS path rather than a component with its own credentials. And when it commits
+a compaction it goes through `CreateTableVersion` like any other writer, which is what
+managed versioning was for.
+
+## The worker is also the only thing that can describe the table
+
+Admin can render an Iceberg table because it can read Iceberg metadata. It cannot read
+Lance: it knows the dataset's location and its format string, and that is the whole of it.
+The details page showed a location and two empty panels, which is an honest answer and a
+useless one.
+
+The worker already knows. Detection opens every dataset to decide whether it needs
+compacting, so at that moment it holds the schema, the row count, the fragment count and
+the version count. It just had no way to say so — every message on the stream was about
+work.
+
+So `WorkerObservations` is a body on `WorkerToAdminMessage`: a repeated `ObjectObservation`
+of `object_id`, `object_kind`, `format`, and a `ConfigValue` map the worker fills with
+whatever it can cheaply say. Admin keeps the last observation per object and serves it back
+with the time it was taken and the worker that took it. Nothing schedules from it, and it is
+not authoritative — it is a cache with its staleness on the label, which is why the page
+badges it rather than presenting it as metadata it read itself.
+
+The keys are the worker's to choose, which keeps the protocol out of the business of knowing
+what a Lance table is. A worker for any other format admin cannot parse describes itself the
+same way.
+
+## A bucket declares its format
+
+Format was recorded per table, which is enough for the storage layer and not enough for
+anything that has to answer a question about a bucket. The admin UI printed one Iceberg
+endpoint for every bucket, including the ones holding Lance datasets, where that endpoint
+serves nothing; an empty bucket had no format at all.
+
+So `CreateTableBucket` takes an optional `format`, stored with the rest of the bucket
+metadata. Empty means `ICEBERG` - what AWS S3 Tables serves, and therefore what an SDK
+that has never heard of the field means. `CreateTable` refuses another format, and
+`CreateView` refuses outright outside an Iceberg bucket, a view being Iceberg metadata.
+The Lance namespace declares `LANCE` for the buckets it creates.
+
+**Enforced rather than defaulted**, because the point of showing a format at all is the
+endpoint that follows from it, and that endpoint is only truthful if the bucket holds one
+format. **Buckets that already exist stay undeclared** and keep taking anything: nothing is
+migrated, and the UI shows "unset" as a fact about the bucket's age rather than a fault.
+That state is also the only way to hold both formats at once, which is what the
+Iceberg-REST adapter path produces.
+
+## Sample rows are fetched, not cached
+
+The same asymmetry has a second half. Admin renders an Iceberg table's rows by
+reading its Parquet files directly; for Lance it has nothing to read with, so the data
+page offered a Browse Data button that led to an empty grid.
+
+`RequestObjectPreview` / `ObjectPreviewResponse` mirror the config-schema round trip
+already on the stream: admin asks, the worker scans the dataset and hands back rows it
+has already rendered as text, because it is the only side that knows the types. Admin
+picks the worker from the observation store, so the one that last described a table is
+the one asked to read it.
+
+The rows are deliberately not cached, and that is the line between the two channels. An
+observation describes an object, so a copy with a timestamp on it is useful. Rows are the
+object's contents: a copy held in admin would be stale, larger, and nobody's business.
+The page fetches on load, bounded, or says why it cannot.
+
+## The sidecar question
+
+The data plane is a different problem, and this design previously conflated the two.
+Maintenance rides the worker protocol; `QueryTable` and `InsertIntoTable` do not, because
+they are synchronous REST operations on the namespace's own surface. Serving those means a
+Rust process that answers HTTP, either behind the Go namespace as a proxy target or in front
+of it. It would make SeaweedFS a store you can run vector search *in* rather than one you
+read vectors *out of*, which is the larger prize and the reason to keep the option open.
+
+Neither should gate phase 1. Phases 1 and 2 are pure Go over the filer and are worth
+shipping on their own — they are what makes Spark and Ray work.
+
+## Testing
+
+Mirror the Iceberg package: `httptest` plus a fake filer client for the handler tests, in
+`weed/s3api/lance`. Then an integration suite under `test/s3tables/catalog/` next to the
+existing `pyiceberg_test.go`, driving the generated Python `lance-namespace` client against
+a live gateway. Three things that suite must cover and unit tests cannot:
+
+- the storage-options key names actually work, i.e. a client that gets `storage_options` from
+  `DescribeTable` can open the dataset;
+- a table created through the catalog is visible to `lance.dataset()` by URI and to a V1
+  directory-catalog client rooted at the namespace prefix;
+- concurrent writers do not lose a commit, which is the phase-2 acceptance test and the
+  thing that justifies the external manifest store.
+
+Phase 1 is validated: `lance_namespace` 0.11.1 with `impl=rest` drives the namespace,
+`lance.write_dataset` writes to the vended location with the vended `storage_options`, and
+the rows read back. Note that this client version drops `check_declared` and
+`include_declared` on the wire, so `is_only_declared` reads null through it however the
+server behaves.
+
+The commit path is validated at both levels. The mechanism: eight writers race the same
+manifest key through S3 with `If-None-Match: *`, and exactly one wins. The property that
+actually matters, which single-winner exclusivity does not by itself establish: eight
+writers append to one dataset concurrently through lance, and afterwards every batch is
+still there — the losers saw the conflict, rebased, and committed again. That second test
+is also the sequence managed versioning could not complete at all, since its store answers
+"put_if_not_exists is not supported" to the second commit.
+
+One more that belongs in the Iceberg suite, not this one: a Lance dataset registered through
+the Iceberg adapter must survive a full maintenance pass. Reading the code, that test should
+fail today; it has not been run.
+
+## Open questions
+
+- Root-level `ListNamespaces` enumerating table buckets is convenient and is a listing
+  surface we do not have on the Iceberg side. Decide whether it is gated behind a flag.
+- Whether the `.lance` directory suffix is worth the divergence from the Iceberg layout. I
+  think yes — it is what makes the catalog optional — but it means the two catalogs' tables
+  do not look alike on disk, and the admin UI has to know that.
+- Names: our charsets are lowercase-only and Lance identifiers are arbitrary strings. Reject
+  and document, as Iceberg does, or case-fold. Rejecting is right, but see #10734 for how
+  case handling bites when only one side normalizes.
+- Whether to land generic-format registration first. Dropping the `"ICEBERG"` check and
+  letting a table carry an arbitrary format plus a location is smaller than this whole
+  design, gets Delta and Parquet catalogued as a side effect, and turns `Format: "LANCE"`
+  into an ordinary value. The argument against is that it invites tables the maintenance
+  worker cannot service, so it needs a "catalog-only, no maintenance" marker to be honest.

docker/Dockerfile.e2e

@@ -2,25 +2,21 @@ FROM ubuntu:22.04
 
 LABEL author="Chris Lu"
 
-# Use Azure's Ubuntu mirror — much faster than archive.ubuntu.com from GitHub-hosted runners,
-# which have been hanging long enough on Ign:/retry to trip the 10-min step timeout.
 # Note: This e2e test image intentionally runs as root for simplicity and compatibility.
 # Production images (Dockerfile.go_build) use proper user isolation with su-exec.
 # For testing purposes, running as root avoids permission complexities and dependency
 # on Alpine-specific tools like su-exec (not available in Ubuntu repos).
-RUN sed -i 's|http://archive.ubuntu.com/ubuntu|http://azure.archive.ubuntu.com/ubuntu|g; s|http://security.ubuntu.com/ubuntu|http://azure.archive.ubuntu.com/ubuntu|g' /etc/apt/sources.list && \
-    apt-get -o Acquire::http::Timeout=15 update && \
-    DEBIAN_FRONTEND=noninteractive apt-get -o Acquire::http::Timeout=15 install -y \
-    --no-install-recommends \
-    --no-install-suggests \
-    curl \
-    fio \
-    fuse \
-    ca-certificates \
-    && apt-get clean \
-    && rm -rf /var/lib/apt/lists/* \
-    && rm -rf /tmp/* \
-    && rm -rf /var/tmp/*
+
+# apt-install prefers Azure's mirror, which archive.ubuntu.com is slow enough from
+# GitHub-hosted runners to justify, and falls through to archive.ubuntu.com when
+# Azure is unreachable - which it periodically is, and Acquire::Retries against a
+# single mirror just retries a dead host. Images built FROM this one install
+# through it for the same reason.
+COPY apt-install /usr/local/bin/apt-install
+RUN chmod +x /usr/local/bin/apt-install && \
+    apt-install curl fio fuse ca-certificates && \
+    rm -rf /tmp/* /var/tmp/*
+
 RUN mkdir -p /etc/seaweedfs /data/filerldb2
 
 COPY ./weed /usr/bin/

docker/Dockerfile.foundationdb_large

@@ -1,4 +1,4 @@
-FROM golang:1.25 AS builder
+FROM golang:1.26 AS builder
 
 RUN apt-get update && \
     apt-get install -y build-essential wget ca-certificates && \

docker/Dockerfile.go_build

@@ -1,6 +1,6 @@
 # Pin the builder to the host arch and cross-compile the (CGO-free) Go binary,
 # so arm64/arm/386 targets skip QEMU emulation of the whole compile.
-FROM --platform=$BUILDPLATFORM golang:1.25-alpine AS builder
+FROM --platform=$BUILDPLATFORM golang:1.26-alpine AS builder
 RUN apk add git g++ fuse
 RUN mkdir -p /go/src/github.com/seaweedfs/
 ARG BRANCH=${BRANCH:-master}
@@ -14,6 +14,9 @@ RUN cd /go/src/github.com/seaweedfs/seaweedfs && \
       git checkout $BRANCH) || \
      (echo "ERROR: Branch/commit $BRANCH not found in repository" && \
       echo "Available branches:" && git branch -a && exit 1))
+# seaweed-common only exists on revisions that have it; a BRANCH predating it
+# still needs the directory so the COPY into rust_builder below never fails.
+RUN mkdir -p /go/src/github.com/seaweedfs/seaweedfs/seaweed-common
 ARG TARGETOS TARGETARCH TARGETVARIANT
 RUN cd /go/src/github.com/seaweedfs/seaweedfs/weed \
   && export LDFLAGS="-X github.com/seaweedfs/seaweedfs/weed/util/version.COMMIT=$(git rev-parse --short HEAD)" \
@@ -29,7 +32,11 @@ FROM alpine:3.23 as rust_builder
 ARG TARGETARCH
 ARG TAGS
 COPY weed-volume-prebuilt/ /prebuilt/
+COPY weed-worker-prebuilt/ /prebuilt-worker/
 COPY --from=builder /go/src/github.com/seaweedfs/seaweedfs/seaweed-volume /build/seaweed-volume
+# seaweed-common is a path dependency of seaweed-volume that lives beside it,
+# so the source build below needs it in the same relative position.
+COPY --from=builder /go/src/github.com/seaweedfs/seaweedfs/seaweed-common /build/seaweed-common
 COPY --from=builder /go/src/github.com/seaweedfs/seaweedfs/weed /build/weed
 WORKDIR /build/seaweed-volume
 RUN if [ -f "/prebuilt/weed-volume-${TARGETARCH}" ]; then \
@@ -47,16 +54,30 @@ RUN if [ -f "/prebuilt/weed-volume-${TARGETARCH}" ]; then \
       echo "Skipping Rust build for $TARGETARCH (unsupported)" && \
       touch /weed-volume; \
     fi
+# The Rust maintenance worker is taken pre-built or not at all: the lance jobs
+# it carries pull in arrow and datafusion, a far larger dependency tree than
+# the image build can carry, so an architecture CI did not build for gets the
+# same empty placeholder the entrypoint refuses to exec.
+RUN if [ -f "/prebuilt-worker/weed-worker-${TARGETARCH}" ]; then \
+      echo "Using pre-built Rust worker for ${TARGETARCH}" && \
+      cp "/prebuilt-worker/weed-worker-${TARGETARCH}" /weed-worker; \
+    else \
+      echo "No pre-built Rust worker for ${TARGETARCH}" && \
+      touch /weed-worker; \
+    fi
+
 # Pre-built binaries arrive via GitHub Actions artifacts, which drop the
 # executable bit, so the copied file is 0644 and exec fails with "Permission
-# denied". Restore it (no-op for the empty placeholder, which stays size 0).
-RUN chmod 0755 /weed-volume
+# denied". Restore it (no-op for the empty placeholders, which stay size 0).
+RUN chmod 0755 /weed-volume /weed-worker
 
 FROM alpine AS final
 LABEL author="Chris Lu"
 COPY --from=builder /go/bin/weed /usr/bin/
 # Copy Rust volume server binary (real binary on amd64/arm64, empty placeholder on other platforms)
 COPY --from=rust_builder /weed-volume /usr/bin/weed-volume
+# Same for the Rust maintenance worker, which serves Lance table buckets
+COPY --from=rust_builder /weed-worker /usr/bin/weed-worker
 RUN mkdir -p /etc/seaweedfs
 COPY --from=builder /go/src/github.com/seaweedfs/seaweedfs/docker/filer.toml /etc/seaweedfs/filer.toml
 COPY --from=builder /go/src/github.com/seaweedfs/seaweedfs/docker/entrypoint.sh /entrypoint.sh

docker/Dockerfile.rocksdb_dev_env

@@ -1,4 +1,4 @@
-FROM golang:1.25 AS builder
+FROM golang:1.26 AS builder
 
 RUN apt-get update
 RUN apt-get install -y build-essential libsnappy-dev zlib1g-dev libbz2-dev libgflags-dev liblz4-dev libzstd-dev

docker/Dockerfile.rocksdb_large

@@ -1,4 +1,4 @@
-FROM golang:1.25 AS builder
+FROM golang:1.26 AS builder
 
 RUN apt-get update
 RUN apt-get install -y build-essential libsnappy-dev zlib1g-dev libbz2-dev libgflags-dev liblz4-dev libzstd-dev

docker/README.md

@@ -31,6 +31,49 @@ docker compose -f seaweedfs-dev-compose.yml -p seaweedfs up
 
 ```
 
+## Verify an image signature
+
+Every image CI pushes to `chrislusf/seaweedfs` and `ghcr.io/chrislusf/seaweedfs` is signed with [cosign](https://docs.sigstore.dev/cosign/verifying/verify/), keyless, by the GitHub Actions workflow that built it, so there is no key to fetch or pin. The signature is attached to the image digest and covers the multi-arch index and each platform image in it; `latest` is the release image under another tag and verifies the same way. Images published before September 2026 predate signing.
+
+```bash
+cosign verify \
+  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
+  --certificate-identity-regexp '^https://github.com/seaweedfs/seaweedfs/\.github/workflows/container_release_unified\.yml@' \
+  chrislusf/seaweedfs:latest
+```
+
+cosign prints the digest it verified. Deploy by that digest, or let an admission controller resolve the tag, so what runs is what was checked.
+
+The identity is `https://github.com/seaweedfs/seaweedfs/.github/workflows/<workflow>@<ref>`. The ref is `refs/tags/<version>` for a release and `refs/heads/master` when a variant was republished by hand. The workflow is `container_release_unified.yml` for the release images, `container_dev.yml` for `dev`, `container_latest.yml` for a `latest` rebuilt by hand, `container_release_foundationdb.yml` for the `_large_disk_foundationdb` release image, and `container_foundationdb_version.yml` or `container_rocksdb_version.yml` for the per-version builds. The regexp above accepts release images only; `container_[a-z_]+\.yml@` accepts everything this repository publishes, `dev` included.
+
+The same check as a Kyverno policy, release images only:
+
+```yaml
+apiVersion: kyverno.io/v1
+kind: ClusterPolicy
+metadata:
+  name: verify-seaweedfs-images
+spec:
+  validationFailureAction: Enforce
+  webhookTimeoutSeconds: 30
+  rules:
+    - name: signed-by-the-release-workflow
+      match:
+        any:
+          - resources:
+              kinds:
+                - Pod
+      verifyImages:
+        - imageReferences:
+            - "docker.io/chrislusf/seaweedfs:*"
+            - "ghcr.io/chrislusf/seaweedfs:*"
+          attestors:
+            - entries:
+                - keyless:
+                    issuer: https://token.actions.githubusercontent.com
+                    subject: https://github.com/seaweedfs/seaweedfs/.github/workflows/container_release_unified.yml@refs/tags/*
+```
+
 ## Local Development
 
 ```bash

docker/admin_integration/Dockerfile.local

@@ -1,11 +1,14 @@
 FROM alpine:latest
 
 # Install required packages
-RUN apk add --no-cache \
+RUN apk upgrade --no-cache && \
+    apk add --no-cache \
     ca-certificates \
     fuse \
     curl \
-    jq
+    jq \
+    libcrypto3 \
+    libssl3
 
 # Copy our locally built binary
 COPY weed-local /usr/bin/weed

docker/apt-install

@@ -0,0 +1,25 @@
+#!/bin/sh
+# Install packages, falling through to the next Ubuntu mirror when one is
+# unreachable. See docker/Dockerfile.e2e.
+set -e
+
+# Every rewrite starts from the pristine list, so a mirror that just failed does
+# not become the pattern the next rewrite has to match.
+[ -f /etc/apt/sources.list.orig ] || cp /etc/apt/sources.list /etc/apt/sources.list.orig
+
+for mirror in azure.archive.ubuntu.com archive.ubuntu.com; do
+	# Any archive host, so this works whether the pristine list came from the
+	# base image (archive.ubuntu.com) or a CI runner (azure.archive.ubuntu.com).
+	sed "s|http://[a-z0-9.]*archive\.ubuntu\.com/ubuntu|http://$mirror/ubuntu|g; s|http://security\.ubuntu\.com/ubuntu|http://$mirror/ubuntu|g" \
+		/etc/apt/sources.list.orig > /etc/apt/sources.list
+	if apt-get -o Acquire::Retries=3 -o Acquire::http::Timeout=15 update && \
+		DEBIAN_FRONTEND=noninteractive apt-get -o Acquire::Retries=3 -o Acquire::http::Timeout=15 install -y \
+			--no-install-recommends --no-install-suggests "$@"; then
+		apt-get clean
+		rm -rf /var/lib/apt/lists/*
+		exit 0
+	fi
+	echo "apt: $mirror unreachable, trying the next mirror" >&2
+done
+
+exit 1

docker/compose/master-cloud.toml

@@ -30,3 +30,5 @@ sleep_minutes = 17          # sleep minutes between each script execution
 	bucket = "volume_bucket"    # an existing bucket
 	endpoint = "http://server2:8333"
 	storage_class = "STANDARD_IA"
+	# upload_concurrency = 5     # concurrent multipart part uploads per volume (volume.tier.upload -concurrent overrides)
+	# download_concurrency = 5   # concurrent multipart part downloads per volume (volume.tier.download -concurrent overrides)

docker/entrypoint.sh

@@ -90,6 +90,16 @@ case "$1" in
   	exec /usr/bin/weed-volume $ARGS $@
 	;;
 
+  'worker-rust')
+  	shift
+  	if [ ! -s /usr/bin/weed-worker ]; then
+  	  echo "Error: Rust maintenance worker is not available on this platform ($(uname -m))." >&2
+  	  echo "Use 'worker' for the Go maintenance worker instead." >&2
+  	  exit 1
+  	fi
+  	exec /usr/bin/weed-worker "$@"
+	;;
+
   'server')
   	ARGS="-dir=/data -volume.max=0 -master.volumeSizeLimitMB=1024"
   	if isArgPassed "-volume.max" "$@"; then

docker/prometheus/prometheus.yml

@@ -11,4 +11,8 @@ scrape_configs:
           - 'master:9324'
           - 'volume:9325'
           - 'filer:9326'
-          - 's3:9327'
+          - 's3:9327'
+          # A plugin worker publishes its own metrics when started with
+          # -metricsPort (Go) or --metrics-port (Rust). 9328 continues the
+          # series, since 9327 is already the S3 gateway's.
+          # - 'worker:9328'

go.mod

@@ -1,20 +1,20 @@
 module github.com/seaweedfs/seaweedfs
 
-go 1.25.8
+go 1.26.6
 
 require (
 	cloud.google.com/go v0.123.0 // indirect
-	cloud.google.com/go/pubsub v1.51.0
-	cloud.google.com/go/storage v1.64.0
+	cloud.google.com/go/pubsub v1.51.1
+	cloud.google.com/go/storage v1.67.0
 	github.com/Shopify/sarama v1.38.1
 	github.com/aws/aws-sdk-go v1.55.8
 	github.com/beorn7/perks v1.0.1 // indirect
 	github.com/bwmarrin/snowflake v0.3.0
 	github.com/cenkalti/backoff/v4 v4.3.0
 	github.com/coreos/go-semver v0.3.1 // indirect
-	github.com/coreos/go-systemd/v22 v22.6.0 // indirect
+	github.com/coreos/go-systemd/v22 v22.7.0 // indirect
 	github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
-	github.com/dustin/go-humanize v1.0.1
+	github.com/dustin/go-humanize v1.1.0
 	github.com/eapache/go-resiliency v1.6.0 // indirect
 	github.com/eapache/go-xerial-snappy v0.0.0-20230731223053-c322873962e3 // indirect
 	github.com/eapache/queue v1.1.0 // indirect
@@ -25,14 +25,14 @@ require (
 	github.com/facebookgo/subset v0.0.0-20200203212716-c811ad88dec4 // indirect
 	github.com/fsnotify/fsnotify v1.9.0 // indirect
 	github.com/go-redsync/redsync/v4 v4.17.0
-	github.com/go-sql-driver/mysql v1.10.0
+	github.com/go-sql-driver/mysql v1.10.1
 	github.com/go-zookeeper/zk v1.0.4 // indirect
 	github.com/golang/protobuf v1.5.4
 	github.com/golang/snappy v1.0.0
 	github.com/google/btree v1.1.3
 	github.com/google/uuid v1.6.0
 	github.com/google/wire v0.7.0 // indirect
-	github.com/googleapis/gax-go/v2 v2.23.0 // indirect
+	github.com/googleapis/gax-go/v2 v2.24.0 // indirect
 	github.com/gorilla/mux v1.8.1
 	github.com/hashicorp/errwrap v1.1.0 // indirect
 	github.com/hashicorp/go-multierror v1.1.1 // indirect
@@ -44,12 +44,12 @@ require (
 	github.com/jmespath/go-jmespath v0.4.0 // indirect
 	github.com/json-iterator/go v1.1.12
 	github.com/karlseguin/ccache/v2 v2.0.8
-	github.com/klauspost/compress v1.19.1
-	github.com/klauspost/reedsolomon v1.14.1
+	github.com/klauspost/compress v1.19.2
+	github.com/klauspost/reedsolomon v1.14.2
 	github.com/kurin/blazer v0.5.3
 	github.com/linxGnu/grocksdb v1.10.8
 	github.com/mailru/easyjson v0.9.2 // indirect
-	github.com/mattn/go-isatty v0.0.23 // indirect
+	github.com/mattn/go-isatty v0.0.24 // indirect
 	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
 	github.com/modern-go/reflect2 v1.0.2 // indirect
 	github.com/olivere/elastic/v7 v7.0.32
@@ -59,18 +59,18 @@ require (
 	github.com/posener/complete v1.2.3
 	github.com/pquerna/cachecontrol v0.2.0
 	github.com/prometheus/client_golang v1.24.1
-	github.com/prometheus/client_model v0.6.2
+	github.com/prometheus/client_model v0.6.3
 	github.com/prometheus/common v0.70.1 // indirect
-	github.com/prometheus/procfs v0.21.1
+	github.com/prometheus/procfs v0.22.0
 	github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 // indirect
 	github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
-	github.com/seaweedfs/goexif v1.0.3
-	github.com/seaweedfs/raft v1.2.0
+	github.com/seaweedfs/goexif v2.0.0+incompatible
+	github.com/seaweedfs/raft v1.2.1
 	github.com/sirupsen/logrus v1.9.4 // indirect
 	github.com/spf13/afero v1.15.0 // indirect
 	github.com/spf13/cast v1.10.0 // indirect
 	github.com/spf13/viper v1.21.0
-	github.com/stretchr/testify v1.11.1
+	github.com/stretchr/testify v1.12.1
 	github.com/stvp/tempredis v0.0.0-20181119212430-b82af8480203
 	github.com/syndtr/goleveldb v1.0.1-0.20190318030020-c3a204f8e965
 	github.com/tidwall/gjson v1.18.0
@@ -84,53 +84,53 @@ require (
 	github.com/xdg-go/scram v1.2.0
 	github.com/xdg-go/stringprep v1.0.4 // indirect
 	github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 // indirect
-	go.etcd.io/etcd/client/v3 v3.6.12
+	go.etcd.io/etcd/client/v3 v3.7.1
 	go.mongodb.org/mongo-driver v1.17.9
 	go.opencensus.io v0.24.0 // indirect
 	gocloud.dev v0.46.0
 	gocloud.dev/pubsub/natspubsub v0.46.0
 	gocloud.dev/pubsub/rabbitpubsub v0.46.0
-	golang.org/x/crypto v0.54.0
+	golang.org/x/crypto v0.57.0
 	golang.org/x/exp v0.0.0-20260709172345-9ea1abe57597
-	golang.org/x/image v0.44.0
-	golang.org/x/net v0.57.0
-	golang.org/x/oauth2 v0.36.0
-	golang.org/x/sys v0.47.0
-	golang.org/x/text v0.40.0 // indirect
-	golang.org/x/tools v0.48.0 // indirect
+	golang.org/x/image v0.46.0
+	golang.org/x/net v0.58.0
+	golang.org/x/oauth2 v0.37.0
+	golang.org/x/sys v0.48.0
+	golang.org/x/text v0.42.0 // indirect
+	golang.org/x/tools v0.49.0 // indirect
 	golang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da // indirect
-	google.golang.org/api v0.289.0
-	google.golang.org/genproto v0.0.0-20260519071638-aa98bba5eb94 // indirect
-	google.golang.org/grpc v1.84.0-dev.0.20260723093437-b6eac429d7b6
-	google.golang.org/protobuf v1.36.11
+	google.golang.org/api v0.297.0
+	google.golang.org/genproto v0.0.0-20260715232425-e75dac1f907d // indirect
+	google.golang.org/grpc v1.85.0-dev.0.20260915183914-4e49413dcab7
+	google.golang.org/protobuf v1.36.12
 	gopkg.in/inf.v0 v0.9.1 // indirect
 	modernc.org/b v1.0.0 // indirect
 	modernc.org/mathutil v1.7.1 // indirect
 	modernc.org/memory v1.11.0 // indirect
-	modernc.org/sqlite v1.53.0
+	modernc.org/sqlite v1.57.0
 )
 
 require (
-	cloud.google.com/go/kms v1.31.0
+	cloud.google.com/go/kms v1.35.0
 	github.com/Azure/azure-sdk-for-go/sdk/keyvault/azkeys v0.10.0
 	github.com/DATA-DOG/go-sqlmock v1.5.2
 	github.com/Jille/raft-grpc-transport v1.6.1
 	github.com/ThreeDotsLabs/watermill v1.5.2
 	github.com/a-h/templ v0.3.1020
 	github.com/apache/cassandra-gocql-driver/v2 v2.1.2
-	github.com/apache/iceberg-go v0.6.0
+	github.com/apache/iceberg-go v0.6.1-0.20260817192109-c2105090c9e2
 	github.com/apple/foundationdb/bindings/go v0.0.0-20250911184653-27f7192f47c3
 	github.com/arangodb/go-driver v1.6.9
 	github.com/armon/go-metrics v0.4.1
-	github.com/aws/aws-sdk-go-v2 v1.43.2
-	github.com/aws/aws-sdk-go-v2/config v1.32.33
-	github.com/aws/aws-sdk-go-v2/credentials v1.19.32
-	github.com/aws/aws-sdk-go-v2/service/s3 v1.105.2
+	github.com/aws/aws-sdk-go-v2 v1.47.0
+	github.com/aws/aws-sdk-go-v2/config v1.32.35
+	github.com/aws/aws-sdk-go-v2/credentials v1.20.4
+	github.com/aws/aws-sdk-go-v2/service/s3 v1.107.3
 	github.com/cespare/xxhash/v2 v2.3.0
 	github.com/cognusion/imaging v1.0.4
 	github.com/fluent/fluent-logger-golang v1.10.1
-	github.com/getsentry/sentry-go v0.44.1
-	github.com/go-ldap/ldap/v3 v3.4.13
+	github.com/getsentry/sentry-go v0.49.0
+	github.com/go-ldap/ldap/v3 v3.4.14
 	github.com/golang-jwt/jwt/v5 v5.3.1
 	github.com/google/flatbuffers/go v0.0.0-20230108230133-3b8644d32c50
 	github.com/hashicorp/golang-lru/v2 v2.0.7
@@ -141,25 +141,26 @@ require (
 	github.com/linkedin/goavro/v2 v2.15.0
 	github.com/minio/crc64nvme v1.1.1
 	github.com/orcaman/concurrent-map/v2 v2.0.1
-	github.com/parquet-go/parquet-go v0.30.1
+	github.com/parquet-go/parquet-go v0.32.0
 	github.com/pkg/sftp v1.13.11
-	github.com/rabbitmq/amqp091-go v1.11.0
-	github.com/rclone/rclone v1.75.0
+	github.com/rabbitmq/amqp091-go v1.14.0
+	github.com/rclone/rclone v1.75.1
 	github.com/rdleal/intervalst v1.5.0
-	github.com/redis/go-redis/v9 v9.21.0
+	github.com/redis/go-redis/v9 v9.22.0
 	github.com/schollz/progressbar/v3 v3.19.1
 	github.com/seaweedfs/go-fuse/v2 v2.9.4
-	github.com/shirou/gopsutil/v4 v4.26.6
+	github.com/shirou/gopsutil/v4 v4.26.7
 	github.com/tarantool/go-option v1.1.0
-	github.com/tarantool/go-tarantool/v3 v3.0.0
-	github.com/testcontainers/testcontainers-go v0.43.0
+	github.com/tarantool/go-tarantool/v3 v3.0.2
+	github.com/testcontainers/testcontainers-go v0.44.0
 	github.com/tikv/client-go/v2 v2.0.7
+	github.com/twmb/avro v1.9.0
 	github.com/xeipuuv/gojsonschema v1.2.0
 	github.com/ydb-platform/ydb-go-sdk-auth-environ v0.5.2
-	github.com/ydb-platform/ydb-go-sdk/v3 v3.146.3
-	go.etcd.io/etcd/client/pkg/v3 v3.6.12
-	go.uber.org/atomic v1.11.0
-	golang.org/x/sync v0.22.0
+	github.com/ydb-platform/ydb-go-sdk/v3 v3.151.1
+	go.etcd.io/etcd/client/pkg/v3 v3.7.1
+	go.uber.org/atomic v1.12.0
+	golang.org/x/sync v0.23.0
 	golang.org/x/tools/godoc v0.1.0-deprecated
 	google.golang.org/grpc/security/advancedtls v1.0.0
 )
@@ -177,23 +178,25 @@ require (
 	github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect
 	github.com/FilenCloudDienste/filen-sdk-go v0.0.39 // indirect
 	github.com/ProtonMail/gopenpgp/v3 v3.4.1 // indirect
+	github.com/RoaringBitmap/roaring/v2 v2.24.0 // indirect
 	github.com/a1ex3/zstd-seekable-format-go/pkg v0.10.0 // indirect
 	github.com/adrg/xdg v0.5.3 // indirect
 	github.com/anchore/go-lzo v0.1.1 // indirect
 	github.com/antlr4-go/antlr/v4 v4.13.1 // indirect
 	github.com/apache/arrow-go/v18 v18.7.0 // indirect
 	github.com/apache/thrift v0.24.0 // indirect
-	github.com/aws/aws-sdk-go-v2/service/signin v1.5.2 // indirect
+	github.com/aws/aws-sdk-go-v2/service/signin v1.10.0 // indirect
 	github.com/bahlo/generic-list-go v0.2.0 // indirect
 	github.com/bazelbuild/rules_go v0.46.0 // indirect
 	github.com/biogo/store v0.0.0-20201120204734-aad293a2328f // indirect
+	github.com/bits-and-blooms/bitset v1.24.4 // indirect
 	github.com/blevesearch/snowballstem v0.9.0 // indirect
 	github.com/boombuler/barcode v1.1.0 // indirect
 	github.com/buger/jsonparser v1.2.0 // indirect
 	github.com/cenkalti/backoff/v5 v5.0.3 // indirect
 	github.com/clipperhouse/uax29/v2 v2.7.0 // indirect
 	github.com/cockroachdb/apd/v3 v3.2.1 // indirect
-	github.com/cockroachdb/errors v1.11.3 // indirect
+	github.com/cockroachdb/errors v1.14.0 // indirect
 	github.com/cockroachdb/logtags v0.0.0-20241215232642-bb51bb14a506 // indirect
 	github.com/cockroachdb/redact v1.1.5 // indirect
 	github.com/cockroachdb/version v0.0.0-20250314144055-3860cd14adf2 // indirect
@@ -206,11 +209,12 @@ require (
 	github.com/dave/dst v0.27.2 // indirect
 	github.com/diskfs/go-diskfs v1.9.4 // indirect
 	github.com/distribution/reference v0.6.0 // indirect
-	github.com/docker/go-connections v0.6.0 // indirect
+	github.com/docker/go-connections v0.7.0 // indirect
 	github.com/docker/go-units v0.5.0 // indirect
 	github.com/dromara/dongle v1.0.1 // indirect
+	github.com/geoarrow/geoarrow-go v0.0.0-20260403143023-f54751c3e3a1 // indirect
 	github.com/gin-gonic/gin v1.11.0 // indirect
-	github.com/go-asn1-ber/asn1-ber v1.5.8-0.20250403174932-29230038a667 // indirect
+	github.com/go-asn1-ber/asn1-ber v1.5.8 // indirect
 	github.com/go-git/go-billy/v5 v5.9.0 // indirect
 	github.com/goccy/go-yaml v1.18.0 // indirect
 	github.com/golang/geo v0.0.0-20210211234256-740aa86cb551 // indirect
@@ -238,14 +242,15 @@ require (
 	github.com/lpar/calendar v0.2.0 // indirect
 	github.com/magiconair/properties v1.8.10 // indirect
 	github.com/moby/docker-image-spec v1.3.1 // indirect
-	github.com/moby/go-archive v0.2.0 // indirect
-	github.com/moby/moby/api v1.54.2 // indirect
-	github.com/moby/moby/client v0.4.0 // indirect
+	github.com/moby/go-archive v0.3.0 // indirect
+	github.com/moby/moby/api v1.55.0 // indirect
+	github.com/moby/moby/client v0.5.0 // indirect
 	github.com/moby/patternmatcher v0.6.1 // indirect
-	github.com/moby/sys/sequential v0.6.0 // indirect
-	github.com/moby/sys/user v0.4.0 // indirect
+	github.com/moby/sys/sequential v0.7.0 // indirect
+	github.com/moby/sys/user v0.4.1 // indirect
 	github.com/moby/sys/userns v0.1.0 // indirect
 	github.com/moby/term v0.5.2 // indirect
+	github.com/mschoch/smat v0.2.0 // indirect
 	github.com/oklog/ulid/v2 v2.1.1 // indirect
 	github.com/opencontainers/go-digest v1.0.0 // indirect
 	github.com/opencontainers/image-spec v1.1.1 // indirect
@@ -257,9 +262,10 @@ require (
 	github.com/pquerna/otp v1.5.0 // indirect
 	github.com/pterm/pterm v0.12.83 // indirect
 	github.com/quic-go/qpack v0.6.0 // indirect
-	github.com/rclone/Proton-API-Bridge v1.0.4 // indirect
-	github.com/rclone/go-proton-api v1.0.3 // indirect
+	github.com/rclone/Proton-API-Bridge v1.0.5 // indirect
+	github.com/rclone/go-proton-api v1.0.4 // indirect
 	github.com/rogpeppe/go-internal v1.15.0 // indirect
+	github.com/rwcarlsen/goexif v0.0.0-20190401172101-9e8deecbddbd // indirect
 	github.com/ryanuber/go-glob v1.0.0 // indirect
 	github.com/sasha-s/go-deadlock v0.3.1 // indirect
 	github.com/smarty/assertions v1.15.0 // indirect
@@ -268,7 +274,6 @@ require (
 	github.com/substrait-io/substrait v0.87.0 // indirect
 	github.com/substrait-io/substrait-go/v8 v8.1.1 // indirect
 	github.com/substrait-io/substrait-protobuf/go v0.85.0 // indirect
-	github.com/twmb/avro v1.7.2 // indirect
 	github.com/twpayne/go-geom v1.6.1 // indirect
 	github.com/twpayne/go-kml/v3 v3.2.1 // indirect
 	github.com/tyler-smith/go-bip39 v1.1.0 // indirect
@@ -278,39 +283,39 @@ require (
 	github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
 	github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
 	github.com/zeebo/xxh3 v1.1.0 // indirect
-	go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0 // indirect
-	go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.42.0 // indirect
-	go.opentelemetry.io/otel/exporters/zipkin v1.36.0 // indirect
-	go.opentelemetry.io/proto/otlp v1.10.0 // indirect
+	go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.45.0 // indirect
+	go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.45.0 // indirect
+	go.opentelemetry.io/otel/exporters/zipkin v1.45.0 // indirect
+	go.opentelemetry.io/proto/otlp v1.11.0 // indirect
 	go.uber.org/mock v0.5.2 // indirect
 	go.yaml.in/yaml/v2 v2.4.4 // indirect
-	go.yaml.in/yaml/v3 v3.0.4 // indirect
-	golang.org/x/mod v0.38.0 // indirect
+	go.yaml.in/yaml/v3 v3.0.5 // indirect
+	golang.org/x/mod v0.41.0 // indirect
 	gonum.org/v1/gonum v0.17.0 // indirect
 )
 
 require (
-	cel.dev/expr v0.25.2 // indirect
-	cloud.google.com/go/auth v0.20.0 // indirect
+	cel.dev/expr v0.25.3 // indirect
+	cloud.google.com/go/auth v0.23.2 // indirect
 	cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
 	cloud.google.com/go/compute/metadata v0.9.0 // indirect
-	cloud.google.com/go/iam v1.11.0 // indirect
-	cloud.google.com/go/monitoring v1.29.0 // indirect
+	cloud.google.com/go/iam v1.12.0 // indirect
+	cloud.google.com/go/monitoring v1.30.0 // indirect
 	filippo.io/edwards25519 v1.2.0 // indirect
-	github.com/Azure/azure-sdk-for-go/sdk/azcore v1.22.0
-	github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.14.0
+	github.com/Azure/azure-sdk-for-go/sdk/azcore v1.23.1
+	github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.14.1
 	github.com/Azure/azure-sdk-for-go/sdk/internal v1.12.0 // indirect
 	github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.8.0
 	github.com/Azure/azure-sdk-for-go/sdk/storage/azfile v1.7.0 // indirect
 	github.com/Azure/go-ntlmssp v0.1.1 // indirect
-	github.com/AzureAD/microsoft-authentication-library-for-go v1.7.2 // indirect
+	github.com/AzureAD/microsoft-authentication-library-for-go v1.8.0 // indirect
 	github.com/Files-com/files-sdk-go/v3 v3.3.194 // indirect
-	github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.34.0 // indirect
+	github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.35.0 // indirect
 	github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.57.0 // indirect
 	github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.57.0 // indirect
 	github.com/IBM/go-sdk-core/v5 v5.23.1 // indirect
 	github.com/Max-Sum/base32768 v0.0.0-20230304063302-18e6ce5945fd // indirect
-	github.com/Microsoft/go-winio v0.6.2 // indirect
+	github.com/Microsoft/go-winio v0.6.3-0.20251027160822-ad3df93bed29 // indirect
 	github.com/ProtonMail/bcrypt v0.0.0-20211005172633-e235017c1baf // indirect
 	github.com/ProtonMail/gluon v0.17.1-0.20230724134000-308be39be96e // indirect
 	github.com/ProtonMail/go-crypto v1.4.1 // indirect
@@ -321,22 +326,22 @@ require (
 	github.com/andybalholm/cascadia v1.3.4 // indirect
 	github.com/appscode/go-querystring v0.0.0-20170504095604-0126cfb3f1dc // indirect
 	github.com/arangodb/go-velocypack v0.0.0-20200318135517-5af53c29c67e // indirect
-	github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.14 // indirect
-	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.33 // indirect
+	github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.18 // indirect
+	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.20.0 // indirect
 	github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.22.34 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.33 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.33 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.34 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.14 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.23 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.33 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.31 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/configsources v1.5.3 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.8.3 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/v4a v1.5.3 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.19 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.31 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.14.3 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.39 // indirect
 	github.com/aws/aws-sdk-go-v2/service/sns v1.39.14 // indirect
 	github.com/aws/aws-sdk-go-v2/service/sqs v1.42.24 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sso v1.33.2 // indirect
-	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.38.2 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sts v1.45.2
-	github.com/aws/smithy-go v1.27.5
+	github.com/aws/aws-sdk-go-v2/service/sso v1.38.0 // indirect
+	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.43.0 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sts v1.51.0
+	github.com/aws/smithy-go v1.28.1
 	github.com/boltdb/bolt v1.3.1 // indirect
 	github.com/bradenaw/juniper v0.15.3 // indirect
 	github.com/buengese/sgzip v0.1.1 // indirect
@@ -354,11 +359,11 @@ require (
 	github.com/d4l3k/messagediff v1.2.1 // indirect
 	github.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13 // indirect
 	github.com/dropbox/dropbox-sdk-go-unofficial/v6 v6.4.0 // indirect
-	github.com/ebitengine/purego v0.10.1 // indirect
+	github.com/ebitengine/purego v0.10.2 // indirect
 	github.com/elastic/gosigar v0.14.3 // indirect
 	github.com/emersion/go-message v0.18.2 // indirect
 	github.com/emersion/go-vcard v0.0.0-20260618161152-d854b7e0e2d3 // indirect
-	github.com/envoyproxy/go-control-plane/envoy v1.37.0 // indirect
+	github.com/envoyproxy/go-control-plane/envoy v1.39.1-0.20260819172001-e6e3fd93e4be // indirect
 	github.com/envoyproxy/protoc-gen-validate v1.3.3 // indirect
 	github.com/fatih/color v1.18.0 // indirect
 	github.com/felixge/httpsnoop v1.1.0 // indirect
@@ -368,7 +373,7 @@ require (
 	github.com/go-chi/chi/v5 v5.3.1 // indirect
 	github.com/go-darwin/apfs v0.0.0-20211011131704-f84b94dbf348 // indirect
 	github.com/go-jose/go-jose/v4 v4.1.4 // indirect
-	github.com/go-logr/logr v1.4.3 // indirect
+	github.com/go-logr/logr v1.4.4 // indirect
 	github.com/go-logr/stdr v1.2.2 // indirect
 	github.com/go-ole/go-ole v1.3.0 // indirect
 	github.com/go-openapi/errors v0.22.8 // indirect
@@ -383,12 +388,12 @@ require (
 	github.com/gogo/protobuf v1.3.2 // indirect
 	github.com/golang-jwt/jwt/v4 v4.5.2 // indirect
 	github.com/google/s2a-go v0.1.9 // indirect
-	github.com/googleapis/enterprise-certificate-proxy v0.3.18 // indirect
+	github.com/googleapis/enterprise-certificate-proxy v0.3.21 // indirect
 	github.com/gorilla/schema v1.4.1 // indirect
 	github.com/gorilla/securecookie v1.1.2 // indirect
 	github.com/gorilla/sessions v1.4.0
 	github.com/grpc-ecosystem/go-grpc-middleware v1.4.0 // indirect
-	github.com/grpc-ecosystem/grpc-gateway/v2 v2.28.0 // indirect
+	github.com/grpc-ecosystem/grpc-gateway/v2 v2.29.0 // indirect
 	github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
 	github.com/hashicorp/go-hclog v1.6.3 // indirect
 	github.com/hashicorp/go-immutable-radix v1.3.1 // indirect
@@ -433,10 +438,10 @@ require (
 	github.com/oracle/oci-go-sdk/v65 v65.121.0 // indirect
 	github.com/panjf2000/ants/v2 v2.12.1 // indirect
 	github.com/patrickmn/go-cache v2.1.0+incompatible // indirect
-	github.com/pelletier/go-toml/v2 v2.2.4 // indirect
+	github.com/pelletier/go-toml/v2 v2.4.3 // indirect
 	github.com/pengsrc/go-shared v0.2.1-0.20190131101655-1999055a4a14 // indirect
 	github.com/philhofer/fwd v1.2.0 // indirect
-	github.com/pierrec/lz4/v4 v4.1.27
+	github.com/pierrec/lz4/v4 v4.1.29
 	github.com/pingcap/errors v0.11.5-0.20211224045212-9687c2b0f87c // indirect
 	github.com/pingcap/failpoint v0.0.0-20220801062533-2eaa32854a6c // indirect
 	github.com/pingcap/kvproto v0.0.0-20230403051650-e166ae588106 // indirect
@@ -474,7 +479,7 @@ require (
 	github.com/winfsp/cgofuse v1.6.1-0.20260126094232-f2c4fccdb286
 	github.com/xanzy/ssh-agent v0.3.3 // indirect
 	github.com/yandex-cloud/go-genproto v0.0.0-20211115083454-9ca41db5ed9e // indirect
-	github.com/ydb-platform/ydb-go-genproto v0.0.0-20260428144813-1c07baab7f7b // indirect
+	github.com/ydb-platform/ydb-go-genproto v0.0.0-20260810122915-65bfd5c4b705 // indirect
 	github.com/ydb-platform/ydb-go-yc v0.12.1 // indirect
 	github.com/ydb-platform/ydb-go-yc-metadata v0.6.1 // indirect
 	github.com/yunify/qingstor-sdk-go/v3 v3.2.0 // indirect
@@ -482,27 +487,27 @@ require (
 	github.com/zeebo/blake3 v0.2.4 // indirect
 	github.com/zeebo/errs v1.4.0 // indirect
 	go.etcd.io/bbolt v1.5.0 // indirect
-	go.etcd.io/etcd/api/v3 v3.6.12 // indirect
+	go.etcd.io/etcd/api/v3 v3.7.1 // indirect
 	go.opentelemetry.io/auto/sdk v1.2.1 // indirect
-	go.opentelemetry.io/contrib/detectors/gcp v1.44.0 // indirect
-	go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.68.0 // indirect
-	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.69.0 // indirect
-	go.opentelemetry.io/otel v1.44.0 // indirect
-	go.opentelemetry.io/otel/metric v1.44.0 // indirect
-	go.opentelemetry.io/otel/sdk v1.44.0 // indirect
-	go.opentelemetry.io/otel/sdk/metric v1.44.0 // indirect
-	go.opentelemetry.io/otel/trace v1.44.0 // indirect
+	go.opentelemetry.io/contrib/detectors/gcp v1.45.0 // indirect
+	go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.69.0 // indirect
+	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.70.0 // indirect
+	go.opentelemetry.io/otel v1.45.0 // indirect
+	go.opentelemetry.io/otel/metric v1.45.0 // indirect
+	go.opentelemetry.io/otel/sdk v1.45.0 // indirect
+	go.opentelemetry.io/otel/sdk/metric v1.45.0 // indirect
+	go.opentelemetry.io/otel/trace v1.45.0 // indirect
 	go.uber.org/multierr v1.11.0 // indirect
 	go.uber.org/zap v1.27.1 // indirect
-	golang.org/x/term v0.45.0
+	golang.org/x/term v0.46.0
 	golang.org/x/time v0.15.0
-	google.golang.org/genproto/googleapis/api v0.0.0-20260706201446-f0a921348800 // indirect
-	google.golang.org/genproto/googleapis/rpc v0.0.0-20260715232425-e75dac1f907d // indirect
+	google.golang.org/genproto/googleapis/api v0.0.0-20260817212433-ac3dfec99bb1 // indirect
+	google.golang.org/genproto/googleapis/rpc v0.0.0-20260819154853-08b0e4226688 // indirect
 	gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect
 	gopkg.in/validator.v2 v2.0.1 // indirect
 	gopkg.in/yaml.v2 v2.4.0 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
-	modernc.org/libc v1.73.4 // indirect
+	modernc.org/libc v1.74.4 // indirect
 	moul.io/http2curl/v2 v2.3.0 // indirect
 	sigs.k8s.io/yaml v1.6.0 // indirect
 	storj.io/common v0.0.0-20260629224719-ba1bff0a7846 // indirect
@@ -515,13 +520,6 @@ require (
 
 // replace github.com/seaweedfs/raft => /Users/chrislu/go/src/github.com/seaweedfs/raft
 
-// apache/thrift v0.23.0 fixes CVE-2026-41602 but compares int against the
-// untyped math.MaxUint32 in lib/go/thrift/framed_transport.go, which overflows
-// int on 32-bit GOARCHes (e.g. openbsd/arm, linux/arm) and fails to compile.
-// Upstream fixed the range check post-release; pin to that commit until the
-// next tagged release carries both the CVE fix and the 32-bit fix.
-replace github.com/apache/thrift => github.com/apache/thrift v0.23.1-0.20260429145742-d2acd3c49e58
-
 // tyler-smith/go-bip39 was deleted from GitHub, so `go mod download` fails for
 // anyone resolving it directly (GOPROXY=direct). It only reaches us transitively
 // through rclone's internxt backend, which calls IsMnemonicValid and NewSeed.

install.sh

@@ -9,7 +9,7 @@
 #   curl -fsSL ... | bash -s -- --version 4.34 --dir /usr/local/bin
 #
 # Options:
-#   --component COMP   Which binary to install: weed, volume-rust, all (default: weed)
+#   --component COMP   Which binary to install: weed, volume-rust, worker-rust, all (default: weed)
 #   --version VER      Release version tag (default: latest)
 #   --large-disk       Use large disk variant (5-byte offset, 8TB max volume)
 #   --dir DIR          Installation directory (default: /usr/local/bin)
@@ -22,6 +22,7 @@ COMPONENT="weed"
 VERSION=""
 LARGE_DISK=false
 INSTALL_DIR="/usr/local/bin"
+WORKER_INSTALLED=false
 
 # Colors (if terminal supports them)
 if [ -t 1 ]; then
@@ -128,6 +129,33 @@ rust_asset_name() {
     fi
 }
 
+# Does a release carry this asset? Answers 0 for yes and 1 for a 404, and stops
+# the installer on anything else: a rate limit or a network blip must not read
+# as "this release predates the binary" and quietly skip it.
+asset_exists() {
+    local url="$1" code=""
+    if command -v curl &>/dev/null; then
+        code="$(curl -sL -o /dev/null -I -w '%{http_code}' "$url" || true)"
+    elif command -v wget &>/dev/null; then
+        # --spider's exit status folds 404 in with every other server error, so
+        # read the status line itself; -S prints one per redirect hop.
+        code="$(wget -S --spider -q -O /dev/null "$url" 2>&1 | awk '/^ *HTTP\// {c=$2} END {print c}')"
+    fi
+
+    case "$code" in
+        200) return 0 ;;
+        404) return 1 ;;
+        *)   error "Could not check ${url} (HTTP ${code:-none}). Retry, or install components one at a time." ;;
+    esac
+}
+
+# Build Rust maintenance worker asset name. No large-disk variant: the worker
+# maintains tables through the namespace and never opens a volume file.
+worker_asset_name() {
+    local os="$1" arch="$2"
+    echo "weed-worker_${os}_${arch}.tar.gz"
+}
+
 # Install a single component
 install_component() {
     local component="$1" os="$2" arch="$3"
@@ -204,10 +232,48 @@ install_component() {
             ok "Installed weed-volume to ${INSTALL_DIR}/${dest_name}"
             ;;
 
+        worker-rust)
+            # Published for linux only: the worker runs beside the cluster it
+            # maintains, and its dependency tree makes every extra target an
+            # expensive build.
+            case "$os" in
+                linux) ;;
+                *) error "Rust maintenance worker is not available for ${os}. Supported: linux" ;;
+            esac
+            case "$arch" in
+                amd64|arm64) ;;
+                *) error "Rust maintenance worker is not available for ${arch}. Supported: amd64, arm64" ;;
+            esac
+
+            asset_name="$(worker_asset_name "$os" "$arch")"
+            download_url="https://github.com/${REPO}/releases/download/${VERSION}/${asset_name}"
+            download "$download_url" "${tmpdir}/${asset_name}"
+
+            info "Extracting ${asset_name}..."
+            tar xzf "${tmpdir}/${asset_name}" -C "$tmpdir"
+
+            local worker_bin
+            worker_bin="$(find "$tmpdir" -name 'weed-worker' -type f | head -1)"
+            if [ -z "$worker_bin" ]; then
+                error "Could not find weed-worker binary in archive"
+            fi
+
+            chmod +x "$worker_bin"
+            install_binary "$worker_bin" "weed-worker"
+            WORKER_INSTALLED=true
+            ok "Installed weed-worker to ${INSTALL_DIR}/weed-worker"
+            ;;
+
         *)
-            error "Unknown component: ${component}. Use: weed, volume-rust, all"
+            error "Unknown component: ${component}. Use: weed, volume-rust, worker-rust, all"
             ;;
     esac
+
+    # The trap is per-process, so a later component's would replace this one and
+    # leave the earlier extraction behind. Clean up here and hand the trap back;
+    # the error paths above exit, which still fires it.
+    rm -rf "$tmpdir"
+    trap - EXIT
 }
 
 # Copy binary to install dir, using sudo if needed
@@ -251,6 +317,16 @@ main() {
         all)
             install_component "weed" "$os" "$arch"
             install_component "volume-rust" "$os" "$arch"
+            # The worker is published for linux amd64/arm64 only, and only by
+            # releases new enough to carry it; skip either case rather than fail
+            # an install that has already put two binaries in place.
+            if [ "$os" != "linux" ] || { [ "$arch" != "amd64" ] && [ "$arch" != "arm64" ]; }; then
+                warn "Skipping the Rust maintenance worker: no build for ${os}/${arch}"
+            elif ! asset_exists "https://github.com/${REPO}/releases/download/${VERSION}/$(worker_asset_name "$os" "$arch")"; then
+                warn "Skipping the Rust maintenance worker: ${VERSION} does not carry one"
+            else
+                install_component "worker-rust" "$os" "$arch"
+            fi
             ;;
         *)
             install_component "$COMPONENT" "$os" "$arch"
@@ -265,11 +341,17 @@ main() {
     if [ "$COMPONENT" = "volume-rust" ] || [ "$COMPONENT" = "all" ]; then
         info "  weed-volume:  ${INSTALL_DIR}/weed-volume"
     fi
+    if [ "$WORKER_INSTALLED" = true ]; then
+        info "  weed-worker:     ${INSTALL_DIR}/weed-worker"
+    fi
     echo ""
     info "Quick start:"
     info "  weed master                          # Start master server"
     info "  weed volume -mserver=localhost:9333   # Start Go volume server"
     info "  weed-volume -mserver localhost:9333 # Start Rust volume server"
+    if [ "$WORKER_INSTALLED" = true ]; then
+        info "  weed-worker --admin localhost:23646  # Start the Rust maintenance worker"
+    fi
 }
 
 main

k8s/charts/seaweedfs/Chart.yaml

@@ -1,6 +1,6 @@
 apiVersion: v1
 description: SeaweedFS
 name: seaweedfs
-appVersion: "4.41"
+appVersion: "4.48"
 # Dev note: Trigger a helm chart release by `git tag -a helm-<version>`
-version: 4.41.0
+version: 4.48.1

k8s/charts/seaweedfs/README.md

@@ -22,8 +22,8 @@ helm install --values=values.yaml seaweedfs seaweedfs/seaweedfs
 ## Info:
 * master/filer/volume are stateful sets with anti-affinity on the hostname,
 so your deployment will be spread/HA.
-* chart is using memsql(mysql) as the filer backend to enable HA (multiple filer instances) and backup/HA memsql can provide.
-* mysql user/password are created in a k8s secret (default: `<release>-seaweedfs-db-secret`) and injected to the filer with ENV.
+* leveldb2 is the default filer backend; a mysql-compatible database (memsql, ...) enables HA (multiple filer instances) and the backup/HA it can provide.
+* with `filer.extraEnvironmentVars.WEED_MYSQL_ENABLED` set to `"true"`, mysql user/password are created in a k8s secret (default: `<release>-seaweedfs-db-secret`) and injected to the filer with ENV. On any other store neither the secret nor the `WEED_MYSQL_*` env, plain or secret-backed, is rendered.
 * cert config exists and can be enabled, but not been tested, requires cert-manager to be installed.
 
 ## Prerequisites
@@ -286,9 +286,35 @@ metadata:
     app.kubernetes.io/component: s3
 stringData:
   # this key must be an inline json config file
-  seaweedfs_s3_config: '{"identities":[{"name":"anvAdmin","credentials":[{"accessKey":"snu8yoP6QAlY0ne4","secretKey":"PNzBcmeLNEdR0oviwm04NQAicOrDH1Km"}],"actions":["Admin","Read","Write"]},{"name":"anvReadOnly","credentials":[{"accessKey":"SCigFee6c5lbi04A","secretKey":"kgFhbT38R8WUYVtiFQ1OiSVOrYr3NKku"}],"actions":["Read"]}]}'
+  seaweedfs_s3_config: '{"identities":[{"name":"anvAdmin","credentials":[{"accessKey":"snu8yoP6QAlY0ne4","secretKey":"PNzBcmeLNEdR0oviwm04NQAicOrDH1Km"}],"actions":["Admin","Read","Write"]},{"name":"anvReadOnly","credentials":[{"accessKey":"SCigFee6c5lbi04A","secretKey":"kgFhbT38R8WUYVtiFQ1OiSVOrYr3NKku"}],"actions":["Read","List"]}]}'
 ```
 
+#### Source S3 credentials from an existing Secret
+
+To keep the keys out of `values.yaml` while still letting the chart generate the
+identities file, point an identity at an existing Secret:
+
+```yaml
+s3:
+  enabled: true
+  enableAuth: true
+  credentials:
+    admin:
+      existingSecret: minio-root
+      accessKeyKey: root-user
+      secretKeyKey: root-password
+```
+
+`accessKeyKey` and `secretKeyKey` default to the chart's own key names
+(`admin_access_key_id`, `admin_secret_access_key`, and the `read_` pair). The
+generated `seaweedfs_s3_config` references the keys as `${SEAWEEDFS_S3_ADMIN_ACCESS_KEY_ID}`
+and the gateway resolves them from the environment, which the chart wires up
+from the Secret. Nothing is read from the cluster at render time, so
+`helm template`, `--dry-run` and an Argo CD diff all render what an install
+applies. Rotating a key in the Secret takes effect on the next pod restart, as
+with any other environment variable. The COSI driver parses the config itself
+and does not resolve these references.
+
 ## Admin Component
 
 The admin component provides a modern web-based administration interface for managing SeaweedFS clusters. It includes:
@@ -337,6 +363,30 @@ If `adminPassword` is empty or not set, the admin interface runs without authent
 
 As an alternative, a kubernetes Secret can be used (`admin.secret.existingSecret`).
 
+### Admin listen address
+
+Since SeaweedFS 4.46, `weed admin` defaults to listening on loopback (`127.0.0.1`).
+The chart's httpGet readiness/liveness probes dial the pod IP, so the admin
+server must bind a non-loopback address for the probes to succeed. The chart
+therefore passes `-ip={{ .Values.admin.ip }}`, defaulting `admin.ip` to `0.0.0.0`
+(the pre-4.46 behaviour of listening on all interfaces).
+
+Binding a non-loopback address requires authentication: `weed admin` refuses to
+start on a non-loopback address without `-adminPassword`, so the chart fails at
+render time if `admin.ip` is non-loopback and authentication is not configured via
+`admin.secret.adminPassword`, `admin.secret.existingSecret`, or
+`WEED_ADMIN_PASSWORD` supplied through `admin.extraEnvironmentVars` /
+`admin.secretExtraEnvironmentVars`. Setting `admin.allowInsecureBind` renders
+`-allowInsecureBind` and bypasses this guard; it leaves the admin API
+unauthenticated on the network, so use it only when access is otherwise
+restricted (e.g. network policies). The whole `127.0.0.0/8` range and `::1` are
+treated as loopback (matching `weed admin`); `localhost` is treated as
+non-loopback. Set `admin.ip` to a loopback address only if you also replace the
+httpGet probes (e.g. with an `exec` probe that checks `127.0.0.1`).
+
+The `-ip` flag requires SeaweedFS 4.46 or newer; pinning `admin.imageOverride`
+to an older image is not supported with this chart version.
+
 ### Admin Data Persistence
 
 The admin component can store configuration and maintenance data. You can configure storage in several ways:
@@ -520,6 +570,14 @@ Two things worth knowing before you turn this on:
 
 The DNS selectors default to CoreDNS as kubeadm, kind and the managed offerings from AWS, Google and Azure install it. On OpenShift, override `egress.dnsNamespaceSelector` and `egress.dnsPodSelector` to match `openshift-dns`; see the comment in `values.yaml`.
 
+## Pod and container security contexts
+
+Pod and container security contexts are configurable independently for every built-in workload and remain empty by default for backwards compatibility. The examples in `values.yaml` show how to enable a `RuntimeDefault` seccomp profile, disable privilege escalation and privileged mode, and drop all Linux capabilities.
+
+The chart does not enable `runAsNonRoot` by default because its default `hostPath` storage may be owned by root. To enforce the Kubernetes `restricted` Pod Security Standard, use storage that is writable by a non-root user and configure `runAsNonRoot` or use the OpenShift overrides below.
+
+Security contexts configured for a component also apply to the chart-managed helper containers for that component. User-provided init containers and sidecars must define their own container security context.
+
 ## OpenShift Support
 
 SeaweedFS can be deployed on OpenShift or any cluster enforcing the Kubernetes "restricted" Pod Security Standard. By default, OpenShift blocks containers that run as root or use `hostPath` volumes.

k8s/charts/seaweedfs/ci/admin-values.yaml

@@ -0,0 +1,7 @@
+# Admin install: exercises the admin StatefulSet, which passes -ip (default
+# 0.0.0.0) and therefore requires authentication to bind a non-loopback address.
+admin:
+  enabled: true
+  secret:
+    adminUser: "admin"
+    adminPassword: "ci-admin-password"

k8s/charts/seaweedfs/templates/admin/admin-ingress.yaml

@@ -23,6 +23,9 @@ metadata:
     app.kubernetes.io/managed-by: {{ .Release.Service }}
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/component: admin
+    {{- with .Values.admin.ingress.labels }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
 spec:
   {{- if and (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion) .Values.admin.ingress.className }}
   ingressClassName: {{ .Values.admin.ingress.className | quote }}

k8s/charts/seaweedfs/templates/admin/admin-service.yaml

@@ -21,6 +21,7 @@ spec:
   {{- $nodePorts = .Values.admin.service.nodePorts | default dict }}
   {{- end }}
   type: {{ $serviceType }}
+  {{- include "seaweedfs.service.loadBalancerFields" .Values.admin.service }}
   ports:
   - name: "http"
     port: {{ .Values.admin.port }}

k8s/charts/seaweedfs/templates/admin/admin-statefulset.yaml

@@ -6,6 +6,11 @@
 {{- if and (not .Values.admin.masters) (not .Values.global.seaweedfs.masterServer) (not .Values.master.enabled) }}
 {{- fail "admin.masters or global.seaweedfs.masterServer must be set if master.enabled is false" -}}
 {{- end }}
+{{- $adminAuthEnabled := include "seaweedfs.admin.authEnabled" . }}
+{{- $adminIp := .Values.admin.ip | default "0.0.0.0" }}
+{{- if and (not (include "seaweedfs.admin.isLoopbackIp" $adminIp)) (ne $adminAuthEnabled "true") }}
+{{- fail (printf "admin.ip is set to %q (non-loopback) but admin authentication is not configured. Since `weed admin` 4.46 refuses to bind a non-loopback address without authentication, the admin container would exit on startup. Set admin.secret.adminPassword or admin.secret.existingSecret, or supply WEED_ADMIN_PASSWORD via admin.extraEnvironmentVars / admin.secretExtraEnvironmentVars, or set admin.ip to a loopback address such as 127.0.0.1 (note: a loopback bind makes the chart's httpGet readiness/liveness probes fail), or set admin.allowInsecureBind to true to opt out via -allowInsecureBind (INSECURE: exposes the admin API unauthenticated on the network)." $adminIp) -}}
+{{- end }}
 apiVersion: apps/v1
 kind: StatefulSet
 metadata:
@@ -69,6 +74,9 @@ spec:
       {{- if .Values.admin.priorityClassName }}
       priorityClassName: {{ .Values.admin.priorityClassName | quote }}
       {{- end }}
+      {{- if .Values.admin.schedulerName }}
+      schedulerName: {{ .Values.admin.schedulerName | quote }}
+      {{- end }}
       enableServiceLinks: false
       {{- if .Values.admin.serviceAccountName }}
       serviceAccountName: {{ .Values.admin.serviceAccountName | quote }}
@@ -159,6 +167,7 @@ spec:
               -v={{ .Values.global.seaweedfs.loggingLevel }} \
               {{- end }}
               admin \
+              -ip={{ .Values.admin.ip | default "0.0.0.0" }} \
               -port={{ .Values.admin.port }} \
               -port.grpc={{ .Values.admin.grpcPort }} \
               {{- if or (eq .Values.admin.data.type "hostPath") (eq .Values.admin.data.type "persistentVolumeClaim") (eq .Values.admin.data.type "emptyDir") (eq .Values.admin.data.type "existingClaim") }}
@@ -167,14 +176,17 @@ spec:
               -dataDir={{ .Values.admin.dataDir }} \
               {{- end }}
               {{- if .Values.admin.masters }}
-              -masters={{ .Values.admin.masters }}{{- if or $urlPrefix .Values.admin.extraArgs }} \{{ end }}
+              -masters={{ .Values.admin.masters }} \
               {{- else if .Values.global.seaweedfs.masterServer }}
-              -masters={{ .Values.global.seaweedfs.masterServer }}{{- if or $urlPrefix .Values.admin.extraArgs }} \{{ end }}
+              -masters={{ .Values.global.seaweedfs.masterServer }} \
               {{- else }}
-              -masters={{ range $index := until (.Values.master.replicas | int) }}${SEAWEEDFS_FULLNAME}-master-{{ $index }}.${SEAWEEDFS_FULLNAME}-master.{{ $.Release.Namespace }}:{{ $.Values.master.port }}{{ if lt $index (sub ($.Values.master.replicas | int) 1) }},{{ end }}{{ end }}{{- if or $urlPrefix .Values.admin.extraArgs }} \{{ end }}
+              -masters={{ range $index := until (.Values.master.replicas | int) }}${SEAWEEDFS_FULLNAME}-master-{{ $index }}.${SEAWEEDFS_FULLNAME}-master.{{ $.Release.Namespace }}:{{ $.Values.master.port }}{{ if lt $index (sub ($.Values.master.replicas | int) 1) }},{{ end }}{{ end }} \
               {{- end }}
               {{- if $urlPrefix }}
-              -urlPrefix={{ $urlPrefix }}{{- if .Values.admin.extraArgs }} \{{ end }}
+              -urlPrefix={{ $urlPrefix }} \
+              {{- end }}
+              {{- if .Values.admin.allowInsecureBind }}
+              -allowInsecureBind \
               {{- end }}
               {{- range $index, $arg := .Values.admin.extraArgs }}
               {{ $arg }}{{- if lt $index (sub (len $.Values.admin.extraArgs) 1) }} \{{ end }}

k8s/charts/seaweedfs/templates/all-in-one/all-in-one-deployment.yaml

@@ -37,13 +37,17 @@ spec:
         {{- with .Values.allInOne.podLabels }}
         {{- toYaml . | nindent 8 }}
         {{- end }}
+      {{- $podAnnotations := mergeOverwrite (deepCopy (.Values.podAnnotations | default dict)) (.Values.allInOne.podAnnotations | default dict) }}
+      {{- $existingS3ConfigSecret := or .Values.allInOne.s3.existingConfigSecret .Values.s3.existingConfigSecret .Values.filer.s3.existingConfigSecret }}
+      {{- if $existingS3ConfigSecret }}
+      {{- $configSecret := (lookup "v1" "Secret" .Release.Namespace $existingS3ConfigSecret) | default dict }}
+      {{- $_ := set $podAnnotations "checksum/s3config" ($configSecret | toYaml | sha256sum) }}
+      {{- else }}
+      {{- $_ := set $podAnnotations "checksum/s3config" (include (print .Template.BasePath "/s3/s3-secret.yaml") . | sha256sum) }}
+      {{- end }}
+      {{- $_ := set $podAnnotations "checksum/master-config" (include (print .Template.BasePath "/master/master-configmap.yaml") . | sha256sum) }}
       annotations:
-        {{- with .Values.podAnnotations }}
-        {{- toYaml . | nindent 8 }}
-        {{- end }}
-        {{- with .Values.allInOne.podAnnotations }}
-        {{- toYaml . | nindent 8 }}
-        {{- end }}
+        {{- toYaml $podAnnotations | nindent 8 }}
     spec:
       restartPolicy: {{ default .Values.global.seaweedfs.restartPolicy .Values.allInOne.restartPolicy }}
       {{- if .Values.allInOne.affinity }}
@@ -64,6 +68,9 @@ spec:
       {{- if .Values.allInOne.priorityClassName }}
       priorityClassName: {{ .Values.allInOne.priorityClassName | quote }}
       {{- end }}
+      {{- if .Values.allInOne.schedulerName }}
+      schedulerName: {{ .Values.allInOne.schedulerName | quote }}
+      {{- end }}
       {{- if .Values.allInOne.serviceAccountName }}
       serviceAccountName: {{ .Values.allInOne.serviceAccountName | quote }}
       {{- end }}
@@ -81,6 +88,9 @@ spec:
           imagePullPolicy: {{ default "IfNotPresent" .Values.global.seaweedfs.imagePullPolicy }}
           env:
             {{- include "seaweedfs.licenseEnv" . | nindent 12 }}
+            {{- if and .Values.allInOne.s3.enabled (or .Values.allInOne.s3.enableAuth .Values.s3.enableAuth .Values.filer.s3.enableAuth) }}
+            {{- include "seaweedfs.s3.credentialEnv" . | nindent 12 }}
+            {{- end }}
             {{- /* Determine default cluster alias and the corresponding env var keys to avoid conflicts */}}
             {{- $mergedExtraEnvironmentVars := dict }}
             {{- include "seaweedfs.mergeExtraEnvironmentVars" (dict "global" .Values.global.seaweedfs "component" .Values.allInOne "target" $mergedExtraEnvironmentVars) }}

k8s/charts/seaweedfs/templates/all-in-one/all-in-one-service.yml

@@ -21,6 +21,7 @@ spec:
   {{- $nodePorts = .Values.allInOne.service.nodePorts | default dict }}
   {{- end }}
   type: {{ $serviceType }}
+  {{- include "seaweedfs.service.loadBalancerFields" .Values.allInOne.service }}
   internalTrafficPolicy: {{ .Values.allInOne.service.internalTrafficPolicy | default "Cluster" }}
   {{- if and (semverCompare ">=1.31-0" .Capabilities.KubeVersion.GitVersion) .Values.allInOne.s3.trafficDistribution }}
   trafficDistribution: {{ include "seaweedfs.trafficDistribution" (dict "value" .Values.allInOne.s3.trafficDistribution "Capabilities" .Capabilities) }}

k8s/charts/seaweedfs/templates/cert/admin-cert.yaml

@@ -25,6 +25,10 @@ spec:
     organizations:
     - "SeaweedFS CA"
   dnsNames:
+    - '{{ include "seaweedfs.fullname" . }}-admin'
+    - '{{ include "seaweedfs.fullname" . }}-admin.{{ .Release.Namespace }}'
+    - '{{ include "seaweedfs.fullname" . }}-admin.{{ .Release.Namespace }}.svc'
+    - '{{ include "seaweedfs.fullname" . }}-admin.{{ .Release.Namespace }}.svc.cluster.local'
     - '*.{{ include "seaweedfs.fullname" . }}-admin'
     - '*.{{ include "seaweedfs.fullname" . }}-admin.{{ .Release.Namespace }}'
     - '*.{{ include "seaweedfs.fullname" . }}-admin.{{ .Release.Namespace }}.svc'

k8s/charts/seaweedfs/templates/cosi/cosi-deployment.yaml

@@ -57,6 +57,9 @@ spec:
       {{- if .Values.cosi.priorityClassName }}
       priorityClassName: {{ .Values.cosi.priorityClassName | quote }}
       {{- end }}
+      {{- if .Values.cosi.schedulerName }}
+      schedulerName: {{ .Values.cosi.schedulerName | quote }}
+      {{- end }}
       enableServiceLinks: false
       serviceAccountName: {{ include "seaweedfs.componentName" (list . "objectstorage-provisioner") }}
       {{- if .Values.cosi.initContainers }}
@@ -145,6 +148,9 @@ spec:
           resources:
             {{- toYaml . | nindent 12 }}
           {{- end }}
+          {{- if .Values.cosi.containerSecurityContext.enabled }}
+          securityContext: {{- omit .Values.cosi.containerSecurityContext "enabled" | toYaml | nindent 12 }}
+          {{- end }}
         - name: seaweedfs-cosi-sidecar
           image: "{{ .Values.cosi.sidecar.image }}"
           imagePullPolicy: {{ default "IfNotPresent" .Values.global.seaweedfs.imagePullPolicy }}

k8s/charts/seaweedfs/templates/filer/filer-grpc-ingressroutetcp.yaml

@@ -24,6 +24,9 @@ metadata:
     app.kubernetes.io/managed-by: {{ .Release.Service }}
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/component: filer
+    {{- with .Values.filer.ingresses.grpc.labels }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
 spec:
   {{- if $securityEnabled }}
   tls:

k8s/charts/seaweedfs/templates/filer/filer-ingress.yaml

@@ -29,6 +29,9 @@ metadata:
     app.kubernetes.io/managed-by: {{ .Release.Service }}
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/component: filer
+    {{- with .Values.filer.ingresses.http.labels }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
 spec:
   {{- if and (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion) .Values.filer.ingresses.http.className }}
   ingressClassName: {{ .Values.filer.ingresses.http.className | quote }}
@@ -87,6 +90,9 @@ metadata:
     app.kubernetes.io/managed-by: {{ .Release.Service }}
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/component: filer
+    {{- with .Values.filer.ingresses.grpc.labels }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
 spec:
   {{- if and (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion) .Values.filer.ingresses.grpc.className }}
   ingressClassName: {{ .Values.filer.ingresses.grpc.className | quote }}

k8s/charts/seaweedfs/templates/filer/filer-service-client.yaml

@@ -10,9 +10,6 @@ metadata:
     app.kubernetes.io/managed-by: {{ .Release.Service }}
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/component: filer
-{{- if .Values.filer.metricsPort }}
-    monitoring: "true"
-{{- end }}
 {{- if .Values.filer.annotations }}
   annotations:
     {{- toYaml .Values.filer.annotations | nindent 4 }}

k8s/charts/seaweedfs/templates/filer/filer-service.yaml

@@ -10,6 +10,9 @@ metadata:
     app.kubernetes.io/managed-by: {{ .Release.Service }}
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/component: filer
+{{- if .Values.filer.metricsPort }}
+    monitoring: "true"
+{{- end }}
   annotations:
     service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
 {{- if .Values.filer.annotations }}

k8s/charts/seaweedfs/templates/filer/filer-servicemonitor.yaml

@@ -30,6 +30,7 @@ spec:
       app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
       app.kubernetes.io/instance: {{ .Release.Name }}
       app.kubernetes.io/component: filer
+      monitoring: "true"
 {{- end }}
 {{- end }}
 {{- end }}

k8s/charts/seaweedfs/templates/filer/filer-statefulset.yaml

@@ -43,19 +43,15 @@ spec:
       {{- with .Values.filer.podLabels }}
       {{- toYaml . | nindent 8 }}
       {{- end }}
-      annotations:
-      {{- with .Values.podAnnotations }}
-      {{- toYaml . | nindent 8 }}
-      {{- end }}
-      {{- with .Values.filer.podAnnotations }}
-      {{- toYaml . | nindent 8 }}
-      {{- end }}
+      {{- $podAnnotations := mergeOverwrite (deepCopy (.Values.podAnnotations | default dict)) (.Values.filer.podAnnotations | default dict) }}
       {{- if .Values.filer.s3.existingConfigSecret }}
-        {{- $configSecret := (lookup "v1" "Secret" .Release.Namespace .Values.filer.s3.existingConfigSecret) | default dict }}
-        checksum/s3config: {{ $configSecret | toYaml | sha256sum }}
+      {{- $configSecret := (lookup "v1" "Secret" .Release.Namespace .Values.filer.s3.existingConfigSecret) | default dict }}
+      {{- $_ := set $podAnnotations "checksum/s3config" ($configSecret | toYaml | sha256sum) }}
       {{- else }}
-        checksum/s3config: {{ include (print .Template.BasePath "/s3/s3-secret.yaml") . | sha256sum }}
+      {{- $_ := set $podAnnotations "checksum/s3config" (include (print .Template.BasePath "/s3/s3-secret.yaml") . | sha256sum) }}
       {{- end }}
+      annotations:
+        {{- toYaml $podAnnotations | nindent 8 }}
     spec:
       restartPolicy: {{ default .Values.global.seaweedfs.restartPolicy .Values.filer.restartPolicy }}
       {{- if .Values.filer.affinity }}
@@ -76,6 +72,9 @@ spec:
       {{- if .Values.filer.priorityClassName }}
       priorityClassName: {{ .Values.filer.priorityClassName | quote }}
       {{- end }}
+      {{- if .Values.filer.schedulerName }}
+      schedulerName: {{ .Values.filer.schedulerName | quote }}
+      {{- end }}
       enableServiceLinks: false
       {{- if .Values.filer.initContainers }}
       initContainers:
@@ -89,6 +88,7 @@ spec:
           image: {{ template "seaweedfs.filer.image" . }}
           imagePullPolicy: {{ default "IfNotPresent" .Values.global.seaweedfs.imagePullPolicy }}
           env:
+            {{- $mysqlEnabled := include "seaweedfs.filer.mysqlEnabled" . }}
             - name: POD_IP
               valueFrom:
                 fieldRef:
@@ -101,6 +101,7 @@ spec:
               valueFrom:
                 fieldRef:
                   fieldPath: metadata.namespace
+            {{- if $mysqlEnabled }}
             - name: WEED_MYSQL_USERNAME
               valueFrom:
                 secretKeyRef:
@@ -113,10 +114,21 @@ spec:
                   name: {{ include "seaweedfs.fullname" . }}-db-secret
                   key: password
                   optional: true
+            {{- end }}
             - name: SEAWEEDFS_FULLNAME
               value: "{{ include "seaweedfs.fullname" . }}"
+            {{- if and .Values.filer.s3.enabled .Values.filer.s3.enableAuth }}
+            {{- include "seaweedfs.s3.credentialEnv" . | nindent 12 }}
+            {{- end }}
             {{- $mergedExtraEnvironmentVars := dict }}
             {{- include "seaweedfs.mergeExtraEnvironmentVars" (dict "global" .Values.global.seaweedfs "component" .Values.filer "target" $mergedExtraEnvironmentVars) }}
+            {{- if not $mysqlEnabled }}
+            {{- range $key := keys $mergedExtraEnvironmentVars }}
+            {{- if hasPrefix "WEED_MYSQL_" $key }}
+            {{- $_ := unset $mergedExtraEnvironmentVars $key }}
+            {{- end }}
+            {{- end }}
+            {{- end }}
             {{- range $key := keys $mergedExtraEnvironmentVars | sortAlpha }}
             {{- $value := index $mergedExtraEnvironmentVars $key }}
             - name: {{ $key }}
@@ -129,10 +141,12 @@ spec:
             {{- end }}
             {{- if .Values.filer.secretExtraEnvironmentVars }}
             {{- range $key, $value := .Values.filer.secretExtraEnvironmentVars }}
+            {{- if or $mysqlEnabled (not (hasPrefix "WEED_MYSQL_" $key)) }}
             - name: {{ $key }}
               valueFrom: {{ toYaml $value | nindent 16 }}
             {{- end }}
             {{- end }}
+            {{- end }}
           command:
             - "/bin/sh"
             - "-ec"

k8s/charts/seaweedfs/templates/master/master-ingress.yaml

@@ -21,6 +21,9 @@ metadata:
     app.kubernetes.io/managed-by: {{ .Release.Service }}
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/component: master
+    {{- with .Values.master.ingress.labels }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
 spec:
   ingressClassName: {{ .Values.master.ingress.className | quote }}
   tls:

k8s/charts/seaweedfs/templates/master/master-statefulset.yaml

@@ -43,13 +43,10 @@ spec:
       {{- with .Values.master.podLabels }}
       {{- toYaml . | nindent 8 }}
       {{- end }}
+      {{- $podAnnotations := mergeOverwrite (deepCopy (.Values.podAnnotations | default dict)) (.Values.master.podAnnotations | default dict) }}
+      {{- $_ := set $podAnnotations "checksum/master-config" (include (print .Template.BasePath "/master/master-configmap.yaml") . | sha256sum) }}
       annotations:
-      {{ with .Values.podAnnotations }}
-      {{- toYaml . | nindent 8 }}
-      {{- end }}
-      {{- with .Values.master.podAnnotations }}
-      {{- toYaml . | nindent 8 }}
-      {{- end }}
+        {{- toYaml $podAnnotations | nindent 8 }}
     spec:
       restartPolicy: {{ default .Values.global.seaweedfs.restartPolicy .Values.master.restartPolicy }}
       {{- if .Values.master.affinity }}
@@ -69,6 +66,9 @@ spec:
       {{- if .Values.master.priorityClassName }}
       priorityClassName: {{ .Values.master.priorityClassName | quote }}
       {{- end }}
+      {{- if .Values.master.schedulerName }}
+      schedulerName: {{ .Values.master.schedulerName | quote }}
+      {{- end }}
       enableServiceLinks: false
       serviceAccountName: {{ .Values.master.serviceAccountName | default (include "seaweedfs.serviceAccountName" .) | quote }} # for deleting statefulset pods after migration
       {{- if .Values.master.initContainers }}

k8s/charts/seaweedfs/templates/s3/s3-deployment.yaml

@@ -35,13 +35,15 @@ spec:
       {{- with .Values.s3.podLabels }}
       {{- toYaml . | nindent 8 }}
       {{- end }}
+      {{- $podAnnotations := mergeOverwrite (deepCopy (.Values.podAnnotations | default dict)) (.Values.s3.podAnnotations | default dict) }}
+      {{- if .Values.s3.existingConfigSecret }}
+      {{- $configSecret := (lookup "v1" "Secret" .Release.Namespace .Values.s3.existingConfigSecret) | default dict }}
+      {{- $_ := set $podAnnotations "checksum/s3config" ($configSecret | toYaml | sha256sum) }}
+      {{- else }}
+      {{- $_ := set $podAnnotations "checksum/s3config" (include (print .Template.BasePath "/s3/s3-secret.yaml") . | sha256sum) }}
+      {{- end }}
       annotations:
-      {{ with .Values.podAnnotations }}
-      {{- toYaml . | nindent 8 }}
-      {{- end }}
-      {{- with .Values.s3.podAnnotations }}
-      {{- toYaml . | nindent 8 }}
-      {{- end }}
+        {{- toYaml $podAnnotations | nindent 8 }}
     spec:
       restartPolicy: {{ default .Values.global.seaweedfs.restartPolicy .Values.s3.restartPolicy }}
       {{- if .Values.s3.affinity }}
@@ -61,6 +63,9 @@ spec:
       {{- if .Values.s3.priorityClassName }}
       priorityClassName: {{ .Values.s3.priorityClassName | quote }}
       {{- end }}
+      {{- if .Values.s3.schedulerName }}
+      schedulerName: {{ .Values.s3.schedulerName | quote }}
+      {{- end }}
       enableServiceLinks: false
       {{- if .Values.s3.serviceAccountName }}
       serviceAccountName: {{ .Values.s3.serviceAccountName | quote }}
@@ -91,6 +96,9 @@ spec:
                   fieldPath: metadata.namespace
             - name: SEAWEEDFS_FULLNAME
               value: "{{ include "seaweedfs.fullname" . }}"
+            {{- if .Values.s3.enableAuth }}
+            {{- include "seaweedfs.s3.credentialEnv" . | nindent 12 }}
+            {{- end }}
             {{- $mergedExtraEnvironmentVars := dict }}
             {{- include "seaweedfs.mergeExtraEnvironmentVars" (dict "global" .Values.global.seaweedfs "component" .Values.s3 "target" $mergedExtraEnvironmentVars) }}
             {{- range $key := keys $mergedExtraEnvironmentVars | sortAlpha }}
@@ -143,6 +151,8 @@ spec:
               {{- if .Values.s3.icebergPort }}
               -port.iceberg={{ .Values.s3.icebergPort }} \
               {{- end }}
+              {{- /* rendered even when 0: an if would drop the flag and weed would serve its default */}}
+              -port.lance={{ .Values.s3.lancePort | default 0 }} \
               {{- range .Values.s3.extraArgs }}
               {{ . }} \
               {{- end }}
@@ -192,6 +202,10 @@ spec:
             - containerPort: {{ .Values.s3.icebergPort }}
               name: swfs-iceberg
             {{- end }}
+            {{- if .Values.s3.lancePort }}
+            - containerPort: {{ .Values.s3.lancePort }}
+              name: swfs-lance
+            {{- end }}
             {{- if .Values.s3.metricsPort }}
             - containerPort: {{ .Values.s3.metricsPort }}
               name: metrics

k8s/charts/seaweedfs/templates/s3/s3-iceberg-ingress.yaml

@@ -41,6 +41,9 @@ metadata:
     app.kubernetes.io/managed-by: {{ .Release.Service }}
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/component: s3-iceberg
+    {{- with .Values.s3.icebergIngress.labels }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
 spec:
   {{- if .Values.s3.icebergIngress.className }}
   ingressClassName: {{ .Values.s3.icebergIngress.className | quote }}

k8s/charts/seaweedfs/templates/s3/s3-ingress.yaml

@@ -32,6 +32,9 @@ metadata:
     app.kubernetes.io/managed-by: {{ .Release.Service }}
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/component: s3
+    {{- with .Values.s3.ingress.labels }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
 spec:
   ingressClassName: {{ .Values.s3.ingress.className | quote }}
   tls:

k8s/charts/seaweedfs/templates/s3/s3-lance-ingress.yaml

@@ -0,0 +1,64 @@
+{{- define "seaweedfs.s3.lance.ingress.paths" -}}
+paths:
+- path: {{ .Values.s3.lanceIngress.path | quote }}
+  pathType: {{ .Values.s3.lanceIngress.pathType | quote }}
+  backend:
+    {{- if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion }}
+    service:
+      name: {{ include "seaweedfs.componentName" (list . "s3") }}
+      port:
+        number: {{ .Values.s3.lancePort }}
+    {{- else }}
+    serviceName: {{ include "seaweedfs.componentName" (list . "s3") }}
+    servicePort: {{ .Values.s3.lancePort }}
+    {{- end }}
+{{- end -}}
+{{- if and .Values.s3.enabled .Values.s3.lancePort .Values.s3.lanceIngress.enabled }}
+{{- $hosts := list }}
+{{- if kindIs "slice" .Values.s3.lanceIngress.host }}
+  {{- $hosts = .Values.s3.lanceIngress.host }}
+{{- else if .Values.s3.lanceIngress.host }}
+  {{- $hosts = list .Values.s3.lanceIngress.host }}
+{{- end }}
+{{- if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion }}
+apiVersion: networking.k8s.io/v1
+{{- else if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion }}
+apiVersion: networking.k8s.io/v1beta1
+{{- else }}
+apiVersion: extensions/v1beta1
+{{- end }}
+kind: Ingress
+metadata:
+  name: ingress-{{ include "seaweedfs.fullname" . }}-s3-lance
+  namespace: {{ .Release.Namespace }}
+  {{- with .Values.s3.lanceIngress.annotations }}
+  annotations:
+    {{- toYaml . | nindent 4 }}
+  {{- end }}
+  labels:
+    app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
+    helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
+    app.kubernetes.io/managed-by: {{ .Release.Service }}
+    app.kubernetes.io/instance: {{ .Release.Name }}
+    app.kubernetes.io/component: s3-lance
+    {{- with .Values.s3.lanceIngress.labels }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
+spec:
+  {{- if .Values.s3.lanceIngress.className }}
+  ingressClassName: {{ .Values.s3.lanceIngress.className | quote }}
+  {{- end }}
+  tls:
+    {{ .Values.s3.lanceIngress.tls | default list | toYaml | nindent 6}}
+  rules:
+{{- if $hosts }}
+{{- range $host := $hosts }}
+  - host: {{ $host | quote }}
+    http:
+      {{- include "seaweedfs.s3.lance.ingress.paths" $ | nindent 6 }}
+{{- end }}
+{{- else }}
+  - http:
+      {{- include "seaweedfs.s3.lance.ingress.paths" . | nindent 4 }}
+{{- end }}
+{{- end }}

k8s/charts/seaweedfs/templates/s3/s3-secret.yaml

@@ -1,4 +1,7 @@
-{{- if or (and (or .Values.s3.enabled .Values.allInOne.enabled) .Values.s3.enableAuth (not .Values.s3.existingConfigSecret)) (and .Values.filer.s3.enabled .Values.filer.s3.enableAuth (not .Values.filer.s3.existingConfigSecret)) }}
+{{- /* Mirrors the condition the all-in-one deployment mounts this secret under,
+       so the flags that make it mount are exactly the flags that create it. */}}
+{{- $allInOneAuth := and .Values.allInOne.enabled .Values.allInOne.s3.enabled (or .Values.allInOne.s3.enableAuth .Values.s3.enableAuth .Values.filer.s3.enableAuth) (not (or .Values.allInOne.s3.existingConfigSecret .Values.s3.existingConfigSecret .Values.filer.s3.existingConfigSecret)) }}
+{{- if or (and (or .Values.s3.enabled .Values.allInOne.enabled) .Values.s3.enableAuth (not .Values.s3.existingConfigSecret)) (and .Values.filer.s3.enabled .Values.filer.s3.enableAuth (not .Values.filer.s3.existingConfigSecret)) $allInOneAuth }}
 {{- $secretName := printf "%s-s3-secret" (include "seaweedfs.fullname" .) }}
 {{- $legacySecretName := "seaweedfs-s3-secret" }}
 {{- $lookupName := $secretName }}
@@ -14,14 +17,20 @@
 {{- $adminCreds := $creds.admin | default dict -}}
 {{- $access_key_admin := $adminCreds.accessKey -}}
 {{- $secret_key_admin := $adminCreds.secretKey -}}
-{{- if not (and $access_key_admin $secret_key_admin) -}}
+{{- if $adminCreds.existingSecret -}}
+  {{- $access_key_admin = printf "${%s}" (include "seaweedfs.s3.credentialEnvName" (list "admin" "accessKey")) -}}
+  {{- $secret_key_admin = printf "${%s}" (include "seaweedfs.s3.credentialEnvName" (list "admin" "secretKey")) -}}
+{{- else if not (and $access_key_admin $secret_key_admin) -}}
   {{- $access_key_admin = include "seaweedfs.getOrGeneratePassword" (dict "namespace" .Release.Namespace "secretName" $secretName "key" "admin_access_key_id" "length" 20 "existingSecret" (ternary $existingSecret nil $reuse)) -}}
   {{- $secret_key_admin = include "seaweedfs.getOrGeneratePassword" (dict "namespace" .Release.Namespace "secretName" $secretName "key" "admin_secret_access_key" "length" 40 "existingSecret" (ternary $existingSecret nil $reuse)) -}}
 {{- end -}}
 {{- $readCreds := $creds.read | default dict -}}
 {{- $access_key_read := $readCreds.accessKey -}}
 {{- $secret_key_read := $readCreds.secretKey -}}
-{{- if not (and $access_key_read $secret_key_read) -}}
+{{- if $readCreds.existingSecret -}}
+  {{- $access_key_read = printf "${%s}" (include "seaweedfs.s3.credentialEnvName" (list "read" "accessKey")) -}}
+  {{- $secret_key_read = printf "${%s}" (include "seaweedfs.s3.credentialEnvName" (list "read" "secretKey")) -}}
+{{- else if not (and $access_key_read $secret_key_read) -}}
   {{- $access_key_read = include "seaweedfs.getOrGeneratePassword" (dict "namespace" .Release.Namespace "secretName" $secretName "key" "read_access_key_id" "length" 20 "existingSecret" (ternary $existingSecret nil $reuse)) -}}
   {{- $secret_key_read = include "seaweedfs.getOrGeneratePassword" (dict "namespace" .Release.Namespace "secretName" $secretName "key" "read_secret_access_key" "length" 40 "existingSecret" (ternary $existingSecret nil $reuse)) -}}
 {{- end -}}
@@ -41,11 +50,17 @@ metadata:
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/component: s3
 stringData:
+  {{- /* An identity read from an existing Secret keeps its keys there; the
+         config below names the environment variables carrying them. */}}
+  {{- if not $adminCreds.existingSecret }}
   admin_access_key_id: {{ $access_key_admin }}
   admin_secret_access_key: {{ $secret_key_admin }}
+  {{- end }}
+  {{- if not $readCreds.existingSecret }}
   read_access_key_id: {{ $access_key_read }}
   read_secret_access_key: {{ $secret_key_read }}
-  seaweedfs_s3_config: '{"identities":[{"name":"anvAdmin","credentials":[{"accessKey":"{{ $access_key_admin }}","secretKey":"{{ $secret_key_admin }}"}],"actions":["Admin","Read","Write"]},{"name":"anvReadOnly","credentials":[{"accessKey":"{{ $access_key_read }}","secretKey":"{{ $secret_key_read }}"}],"actions":["Read"]}]}'
+  {{- end }}
+  seaweedfs_s3_config: '{"identities":[{"name":"anvAdmin","credentials":[{"accessKey":"{{ $access_key_admin }}","secretKey":"{{ $secret_key_admin }}"}],"actions":["Admin","Read","Write"]},{"name":"anvReadOnly","credentials":[{"accessKey":"{{ $access_key_read }}","secretKey":"{{ $secret_key_read }}"}],"actions":["Read","List"]}]}'
   {{- if .Values.filer.s3.auditLogConfig }}
   filer_s3_auditLogConfig.json: |
     {{ toJson .Values.filer.s3.auditLogConfig | nindent 4 }}

k8s/charts/seaweedfs/templates/s3/s3-service.yaml

@@ -21,6 +21,7 @@ spec:
   {{- $nodePorts = .Values.s3.service.nodePorts | default dict }}
   {{- end }}
   type: {{ $serviceType }}
+  {{- include "seaweedfs.service.loadBalancerFields" .Values.s3.service }}
   internalTrafficPolicy: {{ .Values.s3.internalTrafficPolicy | default "Cluster" }}
   {{- $td := .Values.s3.trafficDistribution | default .Values.filer.s3.trafficDistribution }}
   {{- if and (semverCompare ">=1.31-0" .Capabilities.KubeVersion.GitVersion) $td }}
@@ -43,6 +44,15 @@ spec:
 {{- end }}
     protocol: TCP
 {{- end }}
+{{- if and .Values.s3.enabled .Values.s3.lancePort }}
+  - name: "swfs-lance"
+    port: {{ .Values.s3.lancePort }}
+    targetPort: {{ .Values.s3.lancePort }}
+{{- if $nodePorts.lance }}
+    nodePort: {{ $nodePorts.lance }}
+{{- end }}
+    protocol: TCP
+{{- end }}
 {{- if and .Values.s3.enabled .Values.s3.httpsPort }}
   - name: "swfs-s3-tls"
     port: {{ .Values.s3.httpsPort }}

k8s/charts/seaweedfs/templates/sftp/sftp-deployment.yaml

@@ -61,6 +61,9 @@ spec:
       {{- if .Values.sftp.priorityClassName }}
       priorityClassName: {{ .Values.sftp.priorityClassName | quote }}
       {{- end }}
+      {{- if .Values.sftp.schedulerName }}
+      schedulerName: {{ .Values.sftp.schedulerName | quote }}
+      {{- end }}
       enableServiceLinks: false
       {{- if .Values.sftp.serviceAccountName }}
       serviceAccountName: {{ .Values.sftp.serviceAccountName | quote }}

k8s/charts/seaweedfs/templates/sftp/sftp-service.yaml

@@ -21,6 +21,7 @@ spec:
   {{- $nodePorts = .Values.sftp.service.nodePorts | default dict }}
   {{- end }}
   type: {{ $serviceType }}
+  {{- include "seaweedfs.service.loadBalancerFields" .Values.sftp.service }}
   internalTrafficPolicy: {{ .Values.sftp.internalTrafficPolicy | default "Cluster" }}
   ports:
   - name: "swfs-sftp"

k8s/charts/seaweedfs/templates/shared/_helpers.tpl

@@ -76,6 +76,55 @@ Inject extra environment vars in the format key:value, if populated
 {{- end }}
 {{- end -}}
 
+{{/* Whether the mysql filer store is selected; a flag the chart cannot read counts as selected. */}}
+{{- define "seaweedfs.filer.mysqlEnabled" -}}
+{{- $merged := dict -}}
+{{- $_ := include "seaweedfs.mergeExtraEnvironmentVars" (dict "global" .Values.global.seaweedfs "component" .Values.filer "target" $merged) -}}
+{{- $enabled := index $merged "WEED_MYSQL_ENABLED" -}}
+{{- if or (kindIs "map" $enabled) (hasKey (.Values.filer.secretExtraEnvironmentVars | default dict) "WEED_MYSQL_ENABLED") -}}
+true
+{{- else if and $enabled (eq (lower (toString $enabled)) "true") -}}
+true
+{{- end -}}
+{{- end -}}
+
+{{/* Classify an admin bind address as loopback, mirroring weed admin's
+     isLoopbackIp (net.ParseIP + IsLoopback). Helm templates cannot call
+     net.ParseIP, so we approximate: valid IPv4 addresses in 127.0.0.0/8
+     (validated via regex to reject malformed values like "127.not-an-ip")
+     and the IPv6 loopback "::1" / its expanded form "0:0:0:0:0:0:0:1" are
+     loopback. Hostnames (e.g. "localhost") and wildcard addresses
+     ("0.0.0.0", "::") are non-loopback, matching the binary, which
+     treats unparseable hostnames as non-loopback to be safe. Other IPv6
+     loopback representations are not matched; the binary's own runtime
+     validation is the authoritative guard. */}}
+{{- define "seaweedfs.admin.isLoopbackIp" -}}
+{{- $ip := toString . -}}
+{{- if or (regexMatch "^127\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}$" $ip) (eq $ip "::1") (eq $ip "0:0:0:0:0:0:0:1") -}}
+true
+{{- end -}}
+{{- end -}}
+
+{{/* Whether the admin non-loopback bind guard is satisfied: admin.secret
+     (adminPassword or existingSecret), WEED_ADMIN_PASSWORD via
+     extraEnvironmentVars / secretExtraEnvironmentVars, or
+     admin.allowInsecureBind. A secret-backed entry counts as enabled even
+     though the chart cannot read its value. */}}
+{{- define "seaweedfs.admin.authEnabled" -}}
+{{- if or .Values.admin.secret.existingSecret .Values.admin.secret.adminPassword .Values.admin.allowInsecureBind -}}
+true
+{{- else -}}
+{{- $merged := dict -}}
+{{- $_ := include "seaweedfs.mergeExtraEnvironmentVars" (dict "global" .Values.global.seaweedfs "component" .Values.admin "target" $merged) -}}
+{{- $envPassword := index $merged "WEED_ADMIN_PASSWORD" -}}
+{{- if or (kindIs "map" $envPassword) (hasKey (.Values.admin.secretExtraEnvironmentVars | default dict) "WEED_ADMIN_PASSWORD") -}}
+true
+{{- else if and $envPassword (ne (toString $envPassword) "") -}}
+true
+{{- end -}}
+{{- end -}}
+{{- end -}}
+
 {{/* Return the proper filer image */}}
 {{- define "seaweedfs.filer.image" -}}
 {{- if .Values.filer.imageOverride -}}
@@ -136,6 +185,15 @@ Inject extra environment vars in the format key:value, if populated
 {{- end -}}
 {{- end -}}
 
+{{/* Lance namespace URL the worker's Lance container maintains; empty when unreachable */}}
+{{- define "seaweedfs.worker.lanceNamespaceUrl" -}}
+{{- if .Values.worker.namespaceUrl -}}
+{{- .Values.worker.namespaceUrl -}}
+{{- else if and .Values.s3.enabled .Values.s3.lancePort -}}
+{{- printf "http://%s.%s:%d" (include "seaweedfs.componentName" (list . "s3")) .Release.Namespace (int .Values.s3.lancePort) -}}
+{{- end -}}
+{{- end -}}
+
 {{/* Return the proper volume image */}}
 {{- define "seaweedfs.volume.image" -}}
 {{- if .Values.volume.imageOverride -}}
@@ -496,6 +554,39 @@ true
 {{- end }}
 {{- end -}}
 
+{{/* Name of the environment variable carrying one generated S3 credential
+     field, e.g. SEAWEEDFS_S3_ADMIN_ACCESS_KEY_ID. The generated identities file
+     names it in place of the key when the key lives in an existing Secret.
+     Usage: include "seaweedfs.s3.credentialEnvName" (list "admin" "accessKey") */}}
+{{- define "seaweedfs.s3.credentialEnvName" -}}
+{{- $identity := index . 0 -}}
+{{- $field := index . 1 -}}
+{{- printf "SEAWEEDFS_S3_%s_%s" (upper $identity) (ternary "ACCESS_KEY_ID" "SECRET_ACCESS_KEY" (eq $field "accessKey")) -}}
+{{- end -}}
+
+{{/* Environment for the S3 identities the chart generates from an existing
+     Secret. The gateway resolves the ${VAR} references the identities file
+     carries, so the keys never enter the rendered manifests and a dry run
+     renders the same as an install. */}}
+{{- define "seaweedfs.s3.credentialEnv" -}}
+{{- $creds := $.Values.s3.credentials | default dict -}}
+{{- range $identity := list "admin" "read" -}}
+{{- $identityCreds := index $creds $identity | default dict -}}
+{{- if $identityCreds.existingSecret }}
+- name: {{ include "seaweedfs.s3.credentialEnvName" (list $identity "accessKey") }}
+  valueFrom:
+    secretKeyRef:
+      name: {{ $identityCreds.existingSecret | quote }}
+      key: {{ default (printf "%s_access_key_id" $identity) $identityCreds.accessKeyKey | quote }}
+- name: {{ include "seaweedfs.s3.credentialEnvName" (list $identity "secretKey") }}
+  valueFrom:
+    secretKeyRef:
+      name: {{ $identityCreds.existingSecret | quote }}
+      key: {{ default (printf "%s_secret_access_key" $identity) $identityCreds.secretKeyKey | quote }}
+{{- end -}}
+{{- end -}}
+{{- end -}}
+
 {{/* Generate a compatible trafficDistribution value due to "PreferClose" fast deprecation in k8s v1.35.
      Accepts a dict with "value" (the trafficDistribution string) and "Capabilities". */}}
 {{- define "seaweedfs.trafficDistribution" -}}
@@ -503,3 +594,23 @@ true
 {{- and (eq .value "PreferClose") (semverCompare ">=1.35-0" .Capabilities.KubeVersion.GitVersion) | ternary "PreferSameZone" .value -}}
 {{- end -}}
 {{- end -}}
+
+{{/*
+Render LoadBalancer-specific service fields (loadBalancerClass, loadBalancerIP,
+loadBalancerSourceRanges), only when the service type is LoadBalancer.
+Usage: {{ include "seaweedfs.service.loadBalancerFields" .Values.s3.service }}
+*/}}
+{{- define "seaweedfs.service.loadBalancerFields" -}}
+{{- if eq (.type | default "ClusterIP") "LoadBalancer" }}
+{{- with .loadBalancerClass }}
+  loadBalancerClass: {{ . }}
+{{- end }}
+{{- with .loadBalancerIP }}
+  loadBalancerIP: {{ . }}
+{{- end }}
+{{- with .loadBalancerSourceRanges }}
+  loadBalancerSourceRanges:
+    {{- toYaml . | nindent 4 }}
+{{- end }}
+{{- end }}
+{{- end -}}

k8s/charts/seaweedfs/templates/shared/networkpolicy.yaml

@@ -73,6 +73,9 @@
 {{-   if .Values.s3.icebergPort }}
 {{-     $ports = append $ports .Values.s3.icebergPort }}
 {{-   end }}
+{{-   if .Values.s3.lancePort }}
+{{-     $ports = append $ports .Values.s3.lancePort }}
+{{-   end }}
 {{-   if .Values.s3.metricsPort }}
 {{-     $ports = append $ports .Values.s3.metricsPort }}
 {{-   end }}
@@ -97,6 +100,9 @@
 {{-   if .Values.worker.metricsPort }}
 {{-     $ports = append $ports .Values.worker.metricsPort }}
 {{-   end }}
+{{-   if and .Values.worker.lanceMetricsPort (include "seaweedfs.worker.lanceNamespaceUrl" .) }}
+{{-     $ports = append $ports .Values.worker.lanceMetricsPort }}
+{{-   end }}
 {{-   $targets = append $targets (dict "component" "worker" "ports" $ports) }}
 {{- end }}
 

k8s/charts/seaweedfs/templates/shared/secret-seaweedfs-db.yaml

@@ -1,4 +1,4 @@
-{{- if .Values.filer.enabled }}
+{{- if and .Values.filer.enabled (include "seaweedfs.filer.mysqlEnabled" .) }}
 apiVersion: v1
 kind: Secret
 type: Opaque

k8s/charts/seaweedfs/templates/shared/security-configmap.yaml

@@ -97,6 +97,19 @@ data:
     cert = "/usr/local/share/ca-certificates/worker/tls.crt"
     key  = "/usr/local/share/ca-certificates/worker/tls.key"
 
+    {{- /* S3 gRPC server identity (weed s3 -port.grpc, default httpPort+10000).
+      Without this section, LoadServerTLS("grpc.s3") returns nil and the S3
+      server serves its internal gRPC (IAM cache propagation, s3 lifecycle
+      delete RPCs) as plaintext, while peers dial it with mTLS credentials:
+      "tls: first record does not look like a TLS handshake".
+      Always uses the internal CA-signed client certificate (already mounted
+      on s3 pods by s3-deployment.yaml). Deliberately NOT s3.tlsSecret: that
+      secret is for the public HTTPS listener and may be issued by a public
+      CA which internal gRPC peers (trusting only grpc.ca) would reject. */}}
+    [grpc.s3]
+    cert = "/usr/local/share/ca-certificates/client/tls.crt"
+    key  = "/usr/local/share/ca-certificates/client/tls.key"
+
     # use this for any place needs a grpc client
     # i.e., "weed backup|benchmark|filer.copy|filer.replicate|mount|s3|upload"
     [grpc.client]

k8s/charts/seaweedfs/templates/volume/volume-ingress.yaml

@@ -27,6 +27,9 @@ metadata:
     app.kubernetes.io/managed-by: {{ .Release.Service }}
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/component: volume
+    {{- with .Values.volume.ingress.labels }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
 spec:
   {{- if and (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion) .Values.volume.ingress.className }}
   ingressClassName: {{ .Values.volume.ingress.className | quote }}

k8s/charts/seaweedfs/templates/volume/volume-resize-hook.yaml

@@ -32,6 +32,9 @@ spec:
     spec:
       serviceAccountName: {{ $seaweedfsName }}-volume-resize-hook
       restartPolicy: Never
+      {{- if .Values.volume.podSecurityContext.enabled }}
+      securityContext: {{- omit .Values.volume.podSecurityContext "enabled" | toYaml | nindent 8 }}
+      {{- end }}
       containers:
         - name: resize
           image: {{ .Values.volume.resizeHook.image }}
@@ -39,6 +42,9 @@ spec:
           args:
             - |
 {{ $commands | indent 14 }}
+          {{- if .Values.volume.containerSecurityContext.enabled }}
+          securityContext: {{- omit .Values.volume.containerSecurityContext "enabled" | toYaml | nindent 12 }}
+          {{- end }}
 ---
 apiVersion: v1
 kind: ServiceAccount

k8s/charts/seaweedfs/templates/volume/volume-statefulset.yaml

@@ -73,6 +73,9 @@ spec:
       {{- if $volume.priorityClassName }}
       priorityClassName: {{ $volume.priorityClassName | quote }}
       {{- end }}
+      {{- if $volume.schedulerName }}
+      schedulerName: {{ $volume.schedulerName | quote }}
+      {{- end }}
       enableServiceLinks: false
       serviceAccountName: {{ $volume.serviceAccountName | default (include "seaweedfs.serviceAccountName" $) | quote }} # for deleting statefulset pods after migration
       {{- $initContainers_exists := include "seaweedfs.volume.initContainers_exists" $ -}}

k8s/charts/seaweedfs/templates/worker/worker-deployment.yaml

@@ -64,6 +64,9 @@ spec:
       {{- if .Values.worker.priorityClassName }}
       priorityClassName: {{ .Values.worker.priorityClassName | quote }}
       {{- end }}
+      {{- if .Values.worker.schedulerName }}
+      schedulerName: {{ .Values.worker.schedulerName | quote }}
+      {{- end }}
       enableServiceLinks: false
       {{- if .Values.worker.serviceAccountName }}
       serviceAccountName: {{ .Values.worker.serviceAccountName | quote }}
@@ -220,6 +223,81 @@ spec:
           {{- if .Values.worker.containerSecurityContext.enabled }}
           securityContext: {{- omit .Values.worker.containerSecurityContext "enabled" | toYaml | nindent 12 }}
           {{- end }}
+        {{- if include "seaweedfs.worker.lanceNamespaceUrl" . }}
+        - name: worker-lance
+          image: {{ template "seaweedfs.worker.image" . }}
+          imagePullPolicy: {{ default "IfNotPresent" .Values.global.seaweedfs.imagePullPolicy }}
+          env:
+            - name: POD_NAME
+              valueFrom:
+                fieldRef:
+                  fieldPath: metadata.name
+            {{- /* the URL crosses a shell line; metacharacters must arrive as data, not syntax */}}
+            - name: LANCE_NAMESPACE_URL
+              value: {{ include "seaweedfs.worker.lanceNamespaceUrl" . | quote }}
+          command:
+            - "/bin/sh"
+            - "-ec"
+            - |
+              {{- /* the armv7/386 placeholder is empty; exec of it becomes the shell and exits 0 */}}
+              if [ ! -s /usr/bin/weed-worker ]; then
+                echo "the Rust worker is not available on this platform ($(uname -m)); it ships for amd64 and arm64" >&2
+                exit 1
+              fi
+              exec /usr/bin/weed-worker \
+              --id="$POD_NAME" \
+              {{- if .Values.worker.adminServer }}
+              --admin={{ .Values.worker.adminServer }} \
+              {{- else }}
+              --admin={{ template "seaweedfs.fullname" . }}-admin.{{ .Release.Namespace }}:{{ .Values.admin.port }}{{ if .Values.admin.grpcPort }}.{{ .Values.admin.grpcPort }}{{ end }} \
+              {{- end }}
+              --namespace="$LANCE_NAMESPACE_URL" \
+              {{- if .Values.global.seaweedfs.enableSecurity }}
+              --tls-ca=/usr/local/share/ca-certificates/ca/tls.crt \
+              --tls-cert=/usr/local/share/ca-certificates/worker/tls.crt \
+              --tls-key=/usr/local/share/ca-certificates/worker/tls.key \
+              {{- end }}
+              {{- if .Values.worker.lanceMetricsPort }}
+              --metrics-port={{ .Values.worker.lanceMetricsPort }} \
+              --metrics-ip=0.0.0.0 \
+              {{- end }}
+              --max-concurrency={{ .Values.worker.maxExecute }}
+          {{- if .Values.global.seaweedfs.enableSecurity }}
+          volumeMounts:
+            - name: ca-cert
+              readOnly: true
+              mountPath: /usr/local/share/ca-certificates/ca/
+            - name: worker-cert
+              readOnly: true
+              mountPath: /usr/local/share/ca-certificates/worker/
+          {{- end }}
+          {{- if .Values.worker.lanceMetricsPort }}
+          ports:
+            - containerPort: {{ .Values.worker.lanceMetricsPort }}
+              name: lance-metrics
+          livenessProbe:
+            httpGet:
+              path: /health
+              port: lance-metrics
+            initialDelaySeconds: 30
+            periodSeconds: 60
+            successThreshold: 1
+            failureThreshold: 5
+            timeoutSeconds: 10
+          readinessProbe:
+            httpGet:
+              path: /ready
+              port: lance-metrics
+            initialDelaySeconds: 20
+            periodSeconds: 15
+            successThreshold: 1
+            failureThreshold: 3
+            timeoutSeconds: 10
+          {{- end }}
+          {{- if .Values.worker.containerSecurityContext.enabled }}
+          securityContext: {{- omit .Values.worker.containerSecurityContext "enabled" | toYaml | nindent 12 }}
+          {{- end }}
+        {{- end }}
       {{- if .Values.worker.sidecars }}
       {{- include "seaweedfs.tplvalues.render" (dict "value" .Values.worker.sidecars "context" $) | nindent 8 }}
       {{- end }}

k8s/charts/seaweedfs/templates/worker/worker-service.yaml

@@ -12,13 +12,22 @@ metadata:
     app.kubernetes.io/component: worker
 spec:
   clusterIP: None  # Headless service
-  {{- if .Values.worker.metricsPort }}
+  {{- $lanceMetrics := and .Values.worker.lanceMetricsPort (include "seaweedfs.worker.lanceNamespaceUrl" .) }}
+  {{- if or .Values.worker.metricsPort $lanceMetrics }}
   ports:
+  {{- if .Values.worker.metricsPort }}
   - name: "metrics"
     port: {{ .Values.worker.metricsPort }}
     targetPort: {{ .Values.worker.metricsPort }}
     protocol: TCP
   {{- end }}
+  {{- if $lanceMetrics }}
+  - name: "lance-metrics"
+    port: {{ .Values.worker.lanceMetricsPort }}
+    targetPort: {{ .Values.worker.lanceMetricsPort }}
+    protocol: TCP
+  {{- end }}
+  {{- end }}
   selector:
     app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
     app.kubernetes.io/instance: {{ .Release.Name }}

k8s/charts/seaweedfs/templates/worker/worker-servicemonitor.yaml

@@ -1,6 +1,7 @@
 {{- include "seaweedfs.compat" . -}}
 {{- if .Values.worker.enabled }}
-{{- if .Values.worker.metricsPort }}
+{{- $lanceMetrics := and .Values.worker.lanceMetricsPort (include "seaweedfs.worker.lanceNamespaceUrl" .) }}
+{{- if or .Values.worker.metricsPort $lanceMetrics }}
 {{- if .Values.global.seaweedfs.monitoring.enabled }}
 apiVersion: monitoring.coreos.com/v1
 kind: ServiceMonitor
@@ -22,9 +23,16 @@ metadata:
 {{- end }}
 spec:
   endpoints:
+    {{- if .Values.worker.metricsPort }}
     - interval: 30s
       port: metrics
       scrapeTimeout: 5s
+    {{- end }}
+    {{- if $lanceMetrics }}
+    - interval: 30s
+      port: lance-metrics
+      scrapeTimeout: 5s
+    {{- end }}
   selector:
     matchLabels:
       app.kubernetes.io/name: {{ template "seaweedfs.name" . }}