4.46

* iam: bind CreateServiceAccount target to caller in AuthorizeIamAction

A non-admin holding iam:CreateServiceAccount could pass an arbitrary
ParentUser and mint a service account for any identity, inheriting that
identity permissions. Add a self-target category so a granted non-admin
may only target their own identity; admins remain unrestricted.

* iam: authorize CreateServiceAccount against its ParentUser target

AuthIamManagement passed UserName as the authorization target for every
action, so CreateServiceAccount was authorized with an empty target and
the self-target binding never saw the caller-supplied ParentUser. Pass
ParentUser for that action so the binding takes effect on the live path.

* iam: test CreateServiceAccount binds target to caller

Regression test: a non-admin holding iam:CreateServiceAccount may target
itself but is denied targeting another identity; admins remain
unrestricted.

* iam: authorize CreateServiceAccount against ParentUser on the S3 port

UnifiedPostHandler passed UserName as the authorization target for every
IAM action, so CreateServiceAccount was authorized with an empty target
on the S3-port route and the self-target binding never saw the caller
ParentUser. Extract iamTargetUserName (ParentUser for CreateServiceAccount,
UserName otherwise) and use it from both IAM dispatch surfaces so the
binding applies on the live S3-port path as well as the standalone iam
server.

* iam: test CreateServiceAccount ParentUser binding on the S3 port

End-to-end regression test through UnifiedPostHandler: a non-admin
holding iam:CreateServiceAccount is denied (403) when targeting another
identity and passes authorization when targeting itself.

.codespellrc

@@ -19,4 +19,5 @@ ignore-regex = \b[a-z]+[A-Z]\w*\b|\b[A-Z][a-z]+[A-Z]\w*\b
 # thirdparty: literal Maven groupId `org.apache.hadoop.thirdparty` (external, cannot rename)
 # unknwon: GitHub username / Go module path (`github.com/unknwon/goconfig`)
 # atleast: CLI mode literal string in test/benchmark/fuse_db/bin/sqlite_verify.py
-ignore-words-list = visibles,fo,te,ser,bject,unparseable,keep-alives,tread,anc,ue,auther,thirdparty,unknwon,atleast
+# sme: local variable for a *streamMutateError in mount tests
+ignore-words-list = visibles,fo,te,ser,bject,unparseable,keep-alives,tread,anc,ue,auther,thirdparty,unknwon,atleast,sme

.github/actions/sign-image/action.yml

@@ -0,0 +1,44 @@
+name: Sign container images
+description: >
+  Keyless cosign signature on each image, then a verification pass against the
+  identity the signature should carry, so a misconfigured job fails here and not
+  on someone's cluster. That identity is the calling workflow's own,
+  https://github.com/<owner>/<repo>/.github/workflows/<file>@<ref>.
+  The calling job needs `id-token: write` and a registry login for every image.
+
+inputs:
+  images:
+    description: Image references by digest (name@sha256:...), whitespace separated.
+    required: true
+
+runs:
+  using: composite
+  steps:
+    - name: Install cosign
+      uses: sigstore/cosign-installer@v4.1.2
+
+    - name: Sign
+      shell: bash
+      env:
+        IMAGES: ${{ inputs.images }}
+      run: |
+        set -euo pipefail
+        # The .sig tag layout: the OCI-referrer bundle cosign 3 writes by default
+        # is not read by the Kyverno and policy-controller releases in use today.
+        # Cosign 3 also defaults to --use-signing-config, which insists on a
+        # bundle for its output; turning it off falls back to the default
+        # Fulcio and Rekor URLs, which is all the .sig layout ever used.
+        cosign sign --yes --recursive \
+          --new-bundle-format=false --use-signing-config=false \
+          $IMAGES
+
+    - name: Verify
+      shell: bash
+      env:
+        IMAGES: ${{ inputs.images }}
+      run: |
+        set -euo pipefail
+        cosign verify \
+          --certificate-oidc-issuer https://token.actions.githubusercontent.com \
+          --certificate-identity "https://github.com/$GITHUB_WORKFLOW_REF" \
+          $IMAGES

.github/dependabot.yml

@@ -1,7 +1,9 @@
 version: 2
 updates:
 - package-ecosystem: "github-actions"
-  directory: "/"
+  directories:
+    - "/"
+    - "/.github/actions/sign-image"
   schedule:
       interval: "weekly"
 - package-ecosystem: gomod

.github/scripts/star_history.py

@@ -0,0 +1,87 @@
+#!/usr/bin/env python3
+"""Render the repository's star history to note/star_history.svg.
+
+Uses the GitHub REST stargazers endpoint with the starred-at accept header,
+which caps at 40,000 entries (400 pages of 100). The chart is regenerated on
+a schedule; if the repo grows past that cap the script stops at 40,000 and
+logs a warning rather than under-reporting.
+"""
+import json
+import os
+import sys
+import urllib.request
+from datetime import datetime
+
+import matplotlib
+
+matplotlib.use("Agg")
+import matplotlib.pyplot as plt  # noqa: E402
+from matplotlib.dates import AutoDateLocator, DateFormatter  # noqa: E402
+
+REPO = "seaweedfs/seaweedfs"
+TOKEN = os.environ["GITHUB_TOKEN"]
+OUT = os.environ.get("OUT", "note/star_history.svg")
+PAGE_CAP = 400  # GitHub's hard limit on stargazer pagination
+
+
+def fetch_stargazers():
+    stars = []
+    page = 1
+    while page <= PAGE_CAP:
+        url = f"https://api.github.com/repos/{REPO}/stargazers?per_page=100&page={page}"
+        req = urllib.request.Request(
+            url,
+            headers={
+                "Accept": "application/vnd.github.star+json",
+                "Authorization": f"Bearer {TOKEN}",
+                "X-GitHub-Api-Version": "2022-11-28",
+                "User-Agent": "seaweedfs-star-history",
+            },
+        )
+        with urllib.request.urlopen(req) as resp:
+            batch = json.load(resp)
+        if not batch:
+            break
+        for u in batch:
+            sa = u.get("starred_at")
+            if sa:
+                stars.append(datetime.fromisoformat(sa.replace("Z", "+00:00")))
+        if len(batch) < 100:
+            break
+        page += 1
+    if page > PAGE_CAP:
+        print(
+            f"::warning::Hit the {PAGE_CAP}-page stargazer pagination cap; "
+            "chart reflects the first 40,000 stars only."
+        )
+    return stars
+
+
+def render(stars, out):
+    stars.sort()
+    counts = list(range(1, len(stars) + 1))
+    fig, ax = plt.subplots(figsize=(10, 6), dpi=130)
+    ax.plot(stars, counts, color="#0969da", linewidth=1.6)
+    ax.set_xlabel("Date")
+    ax.set_ylabel("Stars")
+    ax.set_title(f"{REPO} star history")
+    ax.grid(True, linestyle="--", alpha=0.3)
+    ax.xaxis.set_major_locator(AutoDateLocator())
+    ax.xaxis.set_major_formatter(DateFormatter("%Y-%m"))
+    fig.autofmt_xdate()
+    fig.tight_layout()
+    fig.savefig(out, format="svg", transparent=False)
+    plt.close(fig)
+
+
+def main():
+    stars = fetch_stargazers()
+    if not stars:
+        print("::error::No stargazers fetched; not updating the chart.")
+        sys.exit(1)
+    render(stars, OUT)
+    print(f"Rendered {len(stars)} stars to {OUT}")
+
+
+if __name__ == "__main__":
+    main()

.github/workflows/container_dev.yml

@@ -101,6 +101,9 @@ jobs:
   build-dev-containers:
     needs: [build-rust-binaries]
     runs-on: [ubuntu-latest]
+    permissions:
+      contents: read
+      id-token: write
 
     steps:
       - name: Checkout
@@ -149,7 +152,7 @@ jobs:
             org.opencontainers.image.vendor=Chris Lu
 
       - name: Set up QEMU
-        uses: docker/setup-qemu-action@v4.2.0
+        uses: docker/setup-qemu-action@v4.3.0
 
       - name: Create BuildKit config
         run: |
@@ -180,6 +183,7 @@ jobs:
           password: ${{ secrets.GHCR_TOKEN }}
 
       - name: Build
+        id: build
         uses: docker/build-push-action@v7
         with:
           context: ./docker
@@ -188,3 +192,11 @@ jobs:
           platforms: linux/amd64, linux/arm64
           tags: ${{ steps.docker_meta.outputs.tags }}
           labels: ${{ steps.docker_meta.outputs.labels }}
+
+      - name: Sign
+        if: github.event_name != 'pull_request'
+        uses: ./.github/actions/sign-image
+        with:
+          images: >-
+            chrislusf/seaweedfs@${{ steps.build.outputs.digest }}
+            ghcr.io/chrislusf/seaweedfs@${{ steps.build.outputs.digest }}

.github/workflows/container_foundationdb_version.yml

@@ -30,6 +30,9 @@ permissions:
 jobs:
   build-foundationdb-image:
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      id-token: write
 
     steps:
       - name: Checkout
@@ -126,7 +129,7 @@ jobs:
           echo "seaweedfs_ref=$seaweed" >> "$GITHUB_OUTPUT"
 
       - name: Set up QEMU
-        uses: docker/setup-qemu-action@v4.2.0
+        uses: docker/setup-qemu-action@v4.3.0
 
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v4
@@ -154,6 +157,7 @@ jobs:
           fi
 
       - name: Build and push image
+        id: build
         uses: docker/build-push-action@v7
         with:
           context: ./docker
@@ -171,3 +175,8 @@ jobs:
             org.opencontainers.image.description=SeaweedFS is a distributed storage system for blobs, objects, files, and data lake, to store and serve billions of files fast!
             org.opencontainers.image.vendor=Chris Lu
 
+      - name: Sign
+        if: github.event_name != 'pull_request'
+        uses: ./.github/actions/sign-image
+        with:
+          images: chrislusf/seaweedfs@${{ steps.build.outputs.digest }}

.github/workflows/container_latest.yml

@@ -236,7 +236,7 @@ jobs:
             org.opencontainers.image.vendor=Chris Lu
       - name: Set up QEMU
         if: matrix.platform != 'amd64'
-        uses: docker/setup-qemu-action@v4.2.0
+        uses: docker/setup-qemu-action@v4.3.0
       - name: Create BuildKit config
         run: |
           cat > /tmp/buildkitd.toml <<EOF
@@ -441,15 +441,19 @@ jobs:
     runs-on: ubuntu-latest
     needs: [setup, build, trivy-scan]
     if: needs.setup.outputs.publish == 'true' && github.event_name != 'pull_request'
+    permissions:
+      contents: read
+      id-token: write
     strategy:
       matrix:
         variant: ${{ fromJSON(needs.setup.outputs.variants) }}
     steps:
-      - name: Checkout
+      - name: Checkout the signing action
         uses: actions/checkout@v7
         with:
-          ref: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.source_ref || github.ref }}
-      
+          sparse-checkout: .github/actions
+          persist-credentials: false
+
       - name: Configure variant
         id: config
         run: |
@@ -482,21 +486,29 @@ jobs:
         run: |
           # Install crane for efficient multi-arch image copying
           cd $(mktemp -d)
-          curl -sL "https://github.com/google/go-containerregistry/releases/latest/download/go-containerregistry_Linux_x86_64.tar.gz" | tar xz
+          curl -sLO https://github.com/google/go-containerregistry/releases/download/v0.22.0/go-containerregistry_Linux_x86_64.tar.gz
+          echo "edb74d53fad9a596860f59d1c5d04a43dfb5f441dc71f57060dd0bf39483c833  go-containerregistry_Linux_x86_64.tar.gz" | sha256sum -c -
+          tar xzf go-containerregistry_Linux_x86_64.tar.gz crane
           sudo mv crane /usr/local/bin/
           crane version
       - name: Create and push manifest
+        id: manifest
+        env:
+          BASE_TAG: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.image_tag || 'latest' }}
         run: |
           SUFFIX="${{ steps.config.outputs.tag_suffix }}"
-          BASE_TAG="${{ github.event_name == 'workflow_dispatch' && github.event.inputs.image_tag || 'latest' }}"
           
           # Create manifest on GHCR first (no rate limits)
           echo "Creating GHCR manifest (no rate limits)..."
           docker buildx imagetools create -t ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX} \
+            --metadata-file /tmp/manifest.json \
             ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-amd64 \
             ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-arm64 \
             ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-arm \
             ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-386
+          # The copy and the signature below use the digest this run pushed, not whatever the tag points at by then.
+          DIGEST=$(jq -er '."containerimage.descriptor".digest' /tmp/manifest.json)
+          echo "digest=${DIGEST}" >> "$GITHUB_OUTPUT"
           
           # Copy the complete multi-arch image from GHCR to Docker Hub
           # This only requires one pull from GHCR (no rate limit) and one push to Docker Hub
@@ -532,10 +544,10 @@ jobs:
           # Use crane or skopeo to copy, fallback to docker if not available
           if command -v crane &> /dev/null; then
             echo "Using crane to copy..."
-            retry_with_backoff crane copy ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX} chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}
+            retry_with_backoff crane copy ghcr.io/chrislusf/seaweedfs@${DIGEST} chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}
           elif command -v skopeo &> /dev/null; then
             echo "Using skopeo to copy..."
-            retry_with_backoff skopeo copy --all docker://ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX} docker://chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}
+            retry_with_backoff skopeo copy --all docker://ghcr.io/chrislusf/seaweedfs@${DIGEST} docker://chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}
           else
             echo "Using docker buildx imagetools (pulling 4 images from Docker Hub)..."
             # Fallback: create manifest directly on Docker Hub (pulls from Docker Hub - rate limited)
@@ -545,3 +557,10 @@ jobs:
               ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-arm \
               ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-386
           fi
+
+      - name: Sign
+        uses: ./.github/actions/sign-image
+        with:
+          images: >-
+            ghcr.io/chrislusf/seaweedfs@${{ steps.manifest.outputs.digest }}
+            chrislusf/seaweedfs@${{ steps.manifest.outputs.digest }}

.github/workflows/container_release_foundationdb.yml

@@ -21,6 +21,9 @@ jobs:
 
   build-large-release-container_foundationdb:
     runs-on: [ubuntu-latest]
+    permissions:
+      contents: read
+      id-token: write
 
     steps:
       -
@@ -43,7 +46,7 @@ jobs:
             org.opencontainers.image.vendor=Chris Lu
       -
         name: Set up QEMU
-        uses: docker/setup-qemu-action@v4.2.0
+        uses: docker/setup-qemu-action@v4.3.0
       -
         name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v4
@@ -65,6 +68,7 @@ jobs:
           fi
       -
         name: Build
+        id: build
         uses: docker/build-push-action@v7
         with:
           context: ./docker
@@ -76,4 +80,10 @@ jobs:
           platforms: linux/amd64
           tags: ${{ steps.docker_meta.outputs.tags }}
           labels: ${{ steps.docker_meta.outputs.labels }}
+      -
+        name: Sign
+        if: github.event_name != 'pull_request'
+        uses: ./.github/actions/sign-image
+        with:
+          images: chrislusf/seaweedfs@${{ steps.build.outputs.digest }}
 

.github/workflows/container_release_unified.yml

@@ -231,7 +231,7 @@ jobs:
 
       - name: Set up QEMU
         if: (github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant) && matrix.qemu
-        uses: docker/setup-qemu-action@v4.2.0
+        uses: docker/setup-qemu-action@v4.3.0
 
       - name: Create BuildKit config
         if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
@@ -295,10 +295,14 @@ jobs:
           if-no-files-found: error
           retention-days: 1
 
-  # Assemble each variant's per-platform digests into one tag, then mirror to Docker Hub.
+  # Assemble each variant's per-platform digests into one tag, mirror it to
+  # Docker Hub, and sign the result on both registries.
   merge:
     needs: [build]
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      id-token: write
     strategy:
       fail-fast: false
       matrix:
@@ -309,6 +313,13 @@ jobs:
           - { variant: large_disk_full, tag_suffix: _large_disk_full }
           - { variant: rocksdb,         tag_suffix: _large_disk_rocksdb }
     steps:
+      - name: Checkout the signing action
+        if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
+        uses: actions/checkout@v7
+        with:
+          sparse-checkout: .github/actions
+          persist-credentials: false
+
       - name: Download digests
         if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
         uses: actions/download-artifact@v8
@@ -331,12 +342,17 @@ jobs:
 
       - name: Create multi-arch tag on GHCR
         if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
+        id: manifest
         working-directory: /tmp/digests
         run: |
           docker buildx imagetools create \
             -t ${{ env.IMAGE }}:${{ env.RELEASE_TAG }}${{ matrix.tag_suffix }} \
+            --metadata-file /tmp/manifest.json \
             $(printf '${{ env.IMAGE }}@sha256:%s ' *)
           docker buildx imagetools inspect ${{ env.IMAGE }}:${{ env.RELEASE_TAG }}${{ matrix.tag_suffix }}
+          # The copy and the signature below use the digest this run pushed, not whatever the tag points at by then.
+          digest=$(jq -er '."containerimage.descriptor".digest' /tmp/manifest.json)
+          echo "digest=${digest}" >> "$GITHUB_OUTPUT"
 
       - name: Login to Docker Hub
         if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
@@ -349,7 +365,9 @@ jobs:
         if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
         run: |
           cd $(mktemp -d)
-          curl -sL "https://github.com/google/go-containerregistry/releases/latest/download/go-containerregistry_Linux_x86_64.tar.gz" | tar xz
+          curl -sLO https://github.com/google/go-containerregistry/releases/download/v0.22.0/go-containerregistry_Linux_x86_64.tar.gz
+          echo "edb74d53fad9a596860f59d1c5d04a43dfb5f441dc71f57060dd0bf39483c833  go-containerregistry_Linux_x86_64.tar.gz" | sha256sum -c -
+          tar xzf go-containerregistry_Linux_x86_64.tar.gz crane
           sudo mv crane /usr/local/bin/
           crane version
 
@@ -380,10 +398,18 @@ jobs:
 
           echo "Copying ${{ matrix.variant }} from GHCR to Docker Hub..."
           retry_with_backoff crane copy \
-            ${{ env.IMAGE }}:${{ env.RELEASE_TAG }}${{ matrix.tag_suffix }} \
+            ${{ env.IMAGE }}@${{ steps.manifest.outputs.digest }} \
             chrislusf/seaweedfs:${{ env.RELEASE_TAG }}${{ matrix.tag_suffix }}
           echo "Copied ${{ matrix.variant }} to Docker Hub"
 
+      - name: Sign ${{ matrix.variant }}
+        if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
+        uses: ./.github/actions/sign-image
+        with:
+          images: >-
+            ${{ env.IMAGE }}@${{ steps.manifest.outputs.digest }}
+            chrislusf/seaweedfs@${{ steps.manifest.outputs.digest }}
+
   # Report-only trivy scan: uploads fixable HIGH/CRITICAL findings to GitHub
   # Security for visibility, but never blocks the release. Releases (including
   # `latest`) ship regardless — vulnerabilities are tracked, not gated, since
@@ -439,7 +465,8 @@ jobs:
   # image. crane tag adds an extra tag to an existing manifest — no rebuild,
   # no QEMU, no separate workflow. Replaces the old container_latest.yml
   # rebuild that often failed or lagged behind the release. Independent of
-  # trivy-scan: vuln findings are reported but do not block `latest`.
+  # trivy-scan: vuln findings are reported but do not block `latest`. The
+  # cosign signature is attached to the digest, so `latest` carries it too.
   tag-latest:
     runs-on: ubuntu-latest
     needs: [merge]
@@ -468,7 +495,9 @@ jobs:
       - name: Install crane
         run: |
           cd $(mktemp -d)
-          curl -sL "https://github.com/google/go-containerregistry/releases/latest/download/go-containerregistry_Linux_x86_64.tar.gz" | tar xz
+          curl -sLO https://github.com/google/go-containerregistry/releases/download/v0.22.0/go-containerregistry_Linux_x86_64.tar.gz
+          echo "edb74d53fad9a596860f59d1c5d04a43dfb5f441dc71f57060dd0bf39483c833  go-containerregistry_Linux_x86_64.tar.gz" | sha256sum -c -
+          tar xzf go-containerregistry_Linux_x86_64.tar.gz crane
           sudo mv crane /usr/local/bin/
           crane version
 

.github/workflows/container_rocksdb_version.yml

@@ -22,6 +22,9 @@ permissions:
 jobs:
   build-rocksdb-image:
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      id-token: write
 
     steps:
       - name: Checkout
@@ -82,7 +85,7 @@ jobs:
           echo "seaweedfs_ref=$seaweed" >> "$GITHUB_OUTPUT"
 
       - name: Set up QEMU
-        uses: docker/setup-qemu-action@96fe6ef7f33517b61c61be40b68a1882f3264fb8 # v1
+        uses: docker/setup-qemu-action@1f40c72289eff860ee54a304f1438e3cff362e0a # v1
 
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v1
@@ -94,6 +97,7 @@ jobs:
           password: ${{ secrets.DOCKER_PASSWORD }}
 
       - name: Build and push image
+        id: build
         uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v2
         with:
           context: ./docker
@@ -108,3 +112,8 @@ jobs:
             org.opencontainers.image.title=seaweedfs
             org.opencontainers.image.description=SeaweedFS is a distributed storage system for blobs, objects, files, and data lake, to store and serve billions of files fast!
             org.opencontainers.image.vendor=Chris Lu
+
+      - name: Sign
+        uses: ./.github/actions/sign-image
+        with:
+          images: chrislusf/seaweedfs@${{ steps.build.outputs.digest }}

.github/workflows/helm_ci.yml

@@ -90,6 +90,12 @@ jobs:
             echo "S3 credentials reference the existing secret for $workload"
           done
 
+          echo "=== Testing ingress labels ==="
+          helm template test $CHART_DIR --set s3.enabled=true,s3.ingress.enabled=true \
+            --set s3.ingress.labels.external-dns=s3 > /tmp/s3-ingress-labels.yaml
+          grep -A 12 "^kind: Ingress" /tmp/s3-ingress-labels.yaml | grep -q "^    external-dns: s3"
+          echo "S3 ingress renders custom labels"
+
           echo "=== Testing with all-in-one mode ==="
           helm template test $CHART_DIR --set allInOne.enabled=true > /tmp/allinone.yaml
           grep -q "seaweedfs-all-in-one" /tmp/allinone.yaml
@@ -1574,7 +1580,7 @@ jobs:
           echo "IMAGE_TAG=$TAG" >> $GITHUB_ENV
 
       - name: Create kind cluster
-        uses: helm/kind-action@v1.14.0
+        uses: helm/kind-action@v1.15.0
 
       - name: Run chart-testing (install)
         run: |

.github/workflows/mount-windows.yml

@@ -185,8 +185,19 @@ jobs:
         if (Test-Path S:\walk) { throw "directory survived recursive delete" }
         Write-Host "::endgroup::"
 
+        # Tear down everything we started so the runner's later steps (Logs,
+        # post-checkout) launch into a clean environment. A WinFsp mount left
+        # active and a weed.exe holding winfsp-x64.dll have been seen to make
+        # the next step's pwsh.exe fail with STATUS_DLL_INIT_FAILED
+        # (0xC0000142), which fails a job whose actual test step passed.
+        try { Stop-Mount } catch { Write-Host "no mount to stop at teardown: $_" }
+        Get-CimInstance Win32_Process -Filter "Name = 'weed.exe'" |
+          ForEach-Object { Stop-Process -Id $_.ProcessId -Force -ErrorAction SilentlyContinue }
+        Start-Sleep -Seconds 3
+
     - name: Logs
       if: always()
+      continue-on-error: true
       shell: pwsh
       run: |
         foreach ($f in 'C:\seaweed-mount.log','C:\seaweed-mount.err.log','C:\seaweed-remount.log','C:\seaweed-remount.err.log','C:\seaweed-remount2.log','C:\seaweed-remount2.err.log','C:\seaweed-dirmount.log','C:\seaweed-dirmount.err.log','C:\seaweed-mini.log','C:\seaweed-mini.err.log') {

.github/workflows/release_version_bump.yml

@@ -178,6 +178,8 @@ jobs:
     needs: release
     if: ${{ inputs.downstream && !inputs.dry_run }}
     runs-on: ubuntu-latest
+    # The wait below puts no bound of its own on runner-queue time; this does.
+    timeout-minutes: 120
     permissions: {}
     strategy:
       fail-fast: false
@@ -216,21 +218,60 @@ jobs:
             exit 1
           fi
 
-          # Wait on the release the dispatched workflow publishes, not on the run
-          # that publishes it: a dispatch cannot be told apart from a concurrent
-          # one through the API, and the release is what we are here for.
+          # The dispatched workflow publishes the release as its last step, so
+          # its conclusion decides success. Wait on the run, not on a wall
+          # clock: time it spends queued for a runner must not count against
+          # the budget. A dispatch does not return its run id, so take the
+          # newest workflow_dispatch run created since ours; a concurrent
+          # dispatch would be performing this same release, and waiting on it
+          # is just as good.
           released() { gh api "repos/${REPO}/releases?per_page=30" --jq '[.[].tag_name]'; }
 
           BEFORE=$(released)
+          # A minute early, so runner clock skew cannot hide the run.
+          DISPATCHED_AT=$(date -u -d '1 minute ago' '+%Y-%m-%dT%H:%M:%SZ')
           gh workflow run -R "$REPO" "$WORKFLOW" --ref master -f bump=patch -f update_seaweedfs=true
 
-          for _ in $(seq 80); do
-            sleep 15
-            NEW=$(released | jq -c --argjson before "$BEFORE" '. - $before')
-            [ "$(jq length <<<"$NEW")" -gt 0 ] && break
+          RUN_ID=""
+          for _ in $(seq 12); do
+            sleep 10
+            RUN_ID=$(gh api -X GET "repos/${REPO}/actions/workflows/${WORKFLOW}/runs" \
+              -f event=workflow_dispatch -f "created=>=${DISPATCHED_AT}" \
+              --jq '(.workflow_runs | sort_by(.created_at) | last | .id) // empty' || true)
+            [ -n "$RUN_ID" ] && break
           done
+          if [ -z "$RUN_ID" ]; then
+            echo "::error::the dispatch created no ${WORKFLOW} run in ${REPO}; see https://github.com/${REPO}/actions/workflows/${WORKFLOW}"
+            exit 1
+          fi
+          RUN_URL="https://github.com/${REPO}/actions/runs/${RUN_ID}"
+          echo "waiting on ${RUN_URL}"
+
+          # 20 minutes of execution; polls that find the run still queued do
+          # not consume it.
+          RUNNING=0
+          STATE=""
+          while :; do
+            sleep 15
+            STATE=$(gh api "repos/${REPO}/actions/runs/${RUN_ID}" \
+              --jq '.status + "/" + (.conclusion // "")' || true)
+            case "$STATE" in
+              completed/*) break ;;
+              in_progress/*) RUNNING=$((RUNNING + 1)) ;;
+            esac
+            if [ "$RUNNING" -gt 80 ]; then
+              echo "::error::${RUN_URL} has been executing for over 20 minutes; giving up on it"
+              exit 1
+            fi
+          done
+          if [ "$STATE" != "completed/success" ]; then
+            echo "::error::${RUN_URL} concluded '${STATE#completed/}'"
+            exit 1
+          fi
+
+          NEW=$(released | jq -c --argjson before "$BEFORE" '. - $before')
           if [ "$(jq length <<<"$NEW")" -eq 0 ]; then
-            echo "::error::${REPO} published no release within 20 minutes; see https://github.com/${REPO}/actions/workflows/${WORKFLOW}"
+            echo "::error::${RUN_URL} succeeded but ${REPO} shows no new release"
             exit 1
           fi
           echo "${REPO} released $(jq -r 'join(", ")' <<<"$NEW")"

.github/workflows/rust-volume-server-tests.yml

@@ -62,6 +62,9 @@ jobs:
       - name: Run Rust unit tests
         run: cd seaweed-volume && cargo test
 
+      - name: Run Rust unit tests (redb experimental cursor)
+        run: cd seaweed-volume && cargo test --features redb-experimental-cursor --lib storage::needle_map
+
   rust-integration-tests:
     name: Rust Integration Tests
     runs-on: ubuntu-22.04

.github/workflows/s3tests.yml

@@ -128,6 +128,9 @@ jobs:
           echo "All SeaweedFS components are ready!"
           cd ../s3-tests
           sed -i "s/assert prefixes == \['foo%2B1\/', 'foo\/', 'quux%20ab\/'\]/assert prefixes == \['foo\/', 'foo%2B1\/', 'quux%20ab\/'\]/" s3tests/functional/test_s3.py
+          # The suite expects RGW's 400 InvalidPart for a partNumber past the last
+          # part; AWS answers 416 InvalidPartNumber, which is what we return.
+          sed -i "/# request PartNumber out of range/,+4{s/assert status == 400/assert status == 416/; s/assert error_code == 'InvalidPart'/assert error_code == 'InvalidPartNumber'/}" s3tests/functional/test_s3.py
           
           # Debug: Show the config file contents
           echo "=== S3 Config File Contents ==="
@@ -968,6 +971,9 @@ jobs:
           echo "All SeaweedFS components are ready!"
           cd ../s3-tests
           sed -i "s/assert prefixes == \['foo%2B1\/', 'foo\/', 'quux%20ab\/'\]/assert prefixes == \['foo\/', 'foo%2B1\/', 'quux%20ab\/'\]/" s3tests/functional/test_s3.py
+          # The suite expects RGW's 400 InvalidPart for a partNumber past the last
+          # part; AWS answers 416 InvalidPartNumber, which is what we return.
+          sed -i "/# request PartNumber out of range/,+4{s/assert status == 400/assert status == 416/; s/assert error_code == 'InvalidPart'/assert error_code == 'InvalidPartNumber'/}" s3tests/functional/test_s3.py
           # Create and update s3tests.conf to use port 8004
           cp ../docker/compose/s3tests.conf ../docker/compose/s3tests-sql.conf
           sed -i 's/port = 8000/port = 8004/g' ../docker/compose/s3tests-sql.conf

.github/workflows/star_history.yml

@@ -0,0 +1,46 @@
+---
+name: Star History
+
+on:
+  schedule:
+    - cron: "0 0 * * *"  # daily, 00:00 UTC
+  workflow_dispatch:
+
+permissions:
+  contents: write
+
+jobs:
+  render:
+    name: Regenerate star history chart
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v7
+
+      - name: Set up Python
+        uses: actions/setup-python@v7
+        with:
+          python-version: "3.x"
+          cache: pip
+          cache-dependency-path: .github/scripts/star_history.py
+
+      - name: Install matplotlib
+        run: pip install matplotlib
+
+      - name: Render chart
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: python .github/scripts/star_history.py
+
+      - name: Commit if changed
+        run: |
+          git config user.name "github-actions[bot]"
+          git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
+          if git diff --quiet -- note/star_history.svg; then
+            echo "No changes to the chart."
+            exit 0
+          fi
+          git add note/star_history.svg
+          git commit -m "docs: regenerate star history chart"
+          git push

.gitignore

@@ -116,7 +116,6 @@ test/s3/versioning/weed-test.log
 /docker/admin_integration/data
 docker/agent_pub_record
 docker/admin_integration/weed-local
-/seaweedfs-rdma-sidecar/bin
 /test/s3/encryption/filerldb2
 /test/s3/sse/filerldb2
 test/s3/sse/weed-test.log

README.md

@@ -12,425 +12,257 @@
 
 ![SeaweedFS Logo](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/seaweedfs.png)
 
-<h2 align="center"><a href="https://www.patreon.com/seaweedfs">Sponsor SeaweedFS via Patreon</a></h2>
+SeaweedFS is a simple and highly scalable distributed file system. There are two objectives:
 
-SeaweedFS is an independent Apache-licensed open source project with its ongoing development made
-possible entirely thanks to the support of these awesome [backers](https://github.com/seaweedfs/seaweedfs/blob/master/backers.md).
-If you'd like to grow SeaweedFS even stronger, please consider joining our
-<a href="https://www.patreon.com/seaweedfs">sponsors on Patreon</a>.
+1. to store billions of files!
+2. to serve the files fast!
 
-Your support will be really appreciated by me and other supporters!
-
-<!--
-<h4 align="center">Platinum</h4>
-
-<p align="center">
-  <a href="" target="_blank">
-    Add your name or icon here
-  </a>
-</p>
--->
-
-### Gold Sponsors
-[![nodion](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/sponsor_nodion.png)](https://www.nodion.com)
-[![piknik](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/piknik.png)](https://www.piknik.com)
-[![keepsec](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/keepsec.png)](https://www.keepsec.ca)
-[![zyner](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/sponsor_zyner.png)](https://zyner.org)
-
----
+One `weed` binary serves an S3 object store, a POSIX file system, and a lakehouse with S3 Tables, all over the same data. Each blob is one disk read away, capacity grows by starting another volume server, and cloud storage can be cached or tiered transparently.
 
 - [Download Binaries for different platforms](https://github.com/seaweedfs/seaweedfs/releases/latest)
-- [SeaweedFS on Slack](https://join.slack.com/t/seaweedfs/shared_invite/enQtMzI4MTMwMjU2MzA3LTEyYzZmZWYzOGQ3MDJlZWMzYmI0OTE4OTJiZjJjODBmMzUxNmYwODg0YjY3MTNlMjBmZDQ1NzQ5NDJhZWI2ZmY)
-- [SeaweedFS on Twitter](https://twitter.com/SeaweedFS)
-- [SeaweedFS on Telegram](https://t.me/Seaweedfs) 
-- [SeaweedFS on Reddit](https://www.reddit.com/r/SeaweedFS/)
-- [SeaweedFS Mailing List](https://groups.google.com/d/forum/seaweedfs)
 - [Wiki Documentation](https://github.com/seaweedfs/seaweedfs/wiki)
-- [SeaweedFS White Paper](https://github.com/seaweedfs/seaweedfs/wiki/SeaweedFS_Architecture.pdf)
-- [SeaweedFS Introduction Slides 2025.5](https://docs.google.com/presentation/d/1tdkp45J01oRV68dIm4yoTXKJDof-EhainlA0LMXexQE/edit?usp=sharing)
-- [SeaweedFS Introduction Slides 2021.5](https://docs.google.com/presentation/d/1DcxKWlINc-HNCjhYeERkpGXXm6nTCES8mi2W5G0Z4Ts/edit?usp=sharing)
-- [SeaweedFS Introduction Slides 2019.3](https://www.slideshare.net/chrislusf/seaweedfs-introduction)
+- Community: [Slack](https://join.slack.com/t/seaweedfs/shared_invite/enQtMzI4MTMwMjU2MzA3LTEyYzZmZWYzOGQ3MDJlZWMzYmI0OTE4OTJiZjJjODBmMzUxNmYwODg0YjY3MTNlMjBmZDQ1NzQ5NDJhZWI2ZmY), [Twitter](https://twitter.com/SeaweedFS), [Telegram](https://t.me/Seaweedfs), [Reddit](https://www.reddit.com/r/SeaweedFS/), [Mailing List](https://groups.google.com/d/forum/seaweedfs)
+- [SeaweedFS White Paper](https://github.com/seaweedfs/seaweedfs/wiki/SeaweedFS_Architecture.pdf) and introduction slides: [2025.5](https://docs.google.com/presentation/d/1tdkp45J01oRV68dIm4yoTXKJDof-EhainlA0LMXexQE/edit?usp=sharing), [2021.5](https://docs.google.com/presentation/d/1DcxKWlINc-HNCjhYeERkpGXXm6nTCES8mi2W5G0Z4Ts/edit?usp=sharing), [2019.3](https://www.slideshare.net/chrislusf/seaweedfs-introduction)
 
 Table of Contents
 =================
 
 * [Quick Start](#quick-start)
-    * [Quick Start with weed mini](#quick-start-with-weed-mini)
-    * [Quick Start for S3 API on Docker](#quick-start-for-s3-api-on-docker)
-* [Introduction](#introduction)
-* [Features](#features)
-    * [Additional Features](#additional-features)
-    * [Filer Features](#filer-features)
-* [Example: Using Seaweed Blob Store](#example-using-seaweed-blob-store)
-* [Architecture](#object-store-architecture)
-* [Compared to Other File Systems](#compared-to-other-file-systems)
+    * [One command](#one-command)
+    * [Docker](#docker)
+    * [Docker Compose](#docker-compose)
+    * [Kubernetes with Helm](#kubernetes-with-helm)
+    * [Build from source](#build-from-source)
+    * [Scale out](#scale-out)
+* [Why SeaweedFS](#why-seaweedfs)
+    * [Fast](#fast)
+    * [Scalable](#scalable)
+    * [The most complete S3 API](#the-most-complete-s3-api)
+    * [A data warehouse with S3 Tables](#a-data-warehouse-with-s3-tables)
+    * [A fast cache for cloud storage](#a-fast-cache-for-cloud-storage)
+    * [Active-active replication and more](#active-active-replication-and-more)
+* [Architecture](#architecture)
+* [Compared to Other Systems](#compared-to-other-systems)
     * [Compared to HDFS](#compared-to-hdfs)
     * [Compared to GlusterFS, Ceph](#compared-to-glusterfs-ceph)
-    * [Compared to GlusterFS](#compared-to-glusterfs)
+    * [Compared to MooseFS](#compared-to-moosefs)
     * [Compared to Ceph](#compared-to-ceph)
     * [Compared to MinIO, RustFS](#compared-to-minio-rustfs)
-* [Dev Plan](#dev-plan)
-* [Installation Guide](#installation-guide)
-* [Disk Related Topics](#disk-related-topics)
 * [Benchmark](#benchmark)
 * [Enterprise](#enterprise)
 * [License](#license)
+* [Sponsors](#sponsors)
 
 # Quick Start #
 
+## One command ##
 
-## Quick Start with weed mini ##
+Download the latest binary from the [releases](https://github.com/seaweedfs/seaweedfs/releases/latest) page and unzip the single `weed` (or `weed.exe`) file, or let the install script put it in `/usr/local/bin`:
 
-Download the latest binary from https://github.com/seaweedfs/seaweedfs/releases and unzip the single `weed` (or `weed.exe`) file, or run `go install github.com/seaweedfs/seaweedfs/weed@latest`. Then start a ready-to-use S3 object store with credentials and a pre-created bucket in one command:
+```bash
+curl -fsSL https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/install.sh | bash
+```
+
+Then start a ready-to-use S3 object store:
 
 ```bash
 AWS_ACCESS_KEY_ID=admin \
 AWS_SECRET_ACCESS_KEY=secret \
 S3_BUCKET=my-bucket \
-./weed mini -dir=/data
+./weed mini -dir=./data
 ```
 
-That's it — the S3 endpoint is at http://localhost:8333, `my-bucket` already exists, and `admin`/`secret` are valid credentials. `S3_BUCKET` accepts a comma-separated list (e.g. `raw,processed`); use `S3_TABLE_BUCKET` for S3 Tables buckets, each `name` or `name:FORMAT` where the format is `ICEBERG` (the default) or `LANCE`. Drop any of the env vars to skip that piece (no AWS keys → S3 runs in unauthenticated "Allow All" mode for development).
+That's it. The S3 endpoint is at http://localhost:8333, `my-bucket` exists, and `admin`/`secret` are valid credentials:
 
-The same command starts everything else too:
-- **S3 Endpoint**: http://localhost:8333
-- **Master UI**: http://localhost:9333
-- **Volume Server**: http://localhost:9340
-- **Filer UI**: http://localhost:8888
-- **WebDAV**: http://localhost:7333
-- **Admin UI**: http://localhost:23646
+```bash
+AWS_ACCESS_KEY_ID=admin AWS_SECRET_ACCESS_KEY=secret \
+  aws --endpoint-url http://localhost:8333 s3 cp README.md s3://my-bucket/
+```
+
+The same process also runs the master, a volume server, the filer, WebDAV, the Iceberg REST catalog, and the Admin UI. Add `S3_TABLE_BUCKET=warehouse` to also create an Iceberg table bucket, or `warehouse:LANCE` for a Lance one. Drop the AWS keys to run without authentication for development.
 
 > macOS: if the binary is quarantined, run `xattr -d com.apple.quarantine ./weed` first.
 
-Perfect for development, testing, learning SeaweedFS, and single-node deployments. To scale out, add more volume servers by running `weed volume -dir="/some/data/dir2" -master="<master_host>:9333" -port=8081` locally, on another machine, or on thousands of machines.
+`weed mini` is auto-tuned for one node and is fine for single-node production, such as an S3 gateway that issues presigned URLs. See [Quick Start with weed mini][WeedMini].
 
-## Quick Start for S3 API on Docker ##
+## Docker ##
 
 ```bash
-docker run -p 8333:8333 \
+docker run -p 8333:8333 -v weed-data:/data \
   -e AWS_ACCESS_KEY_ID=admin \
   -e AWS_SECRET_ACCESS_KEY=secret \
   -e S3_BUCKET=my-bucket \
   chrislusf/seaweedfs
 ```
 
-Same behavior as the `weed mini` command above — the S3 endpoint is at http://localhost:8333 with `my-bucket` pre-created. Drop the env vars to run anonymously for development.
+Same behavior as the `weed mini` command above.
 
-# Introduction #
+## Docker Compose ##
 
-SeaweedFS is a simple and highly scalable distributed file system. There are two objectives:
+To run master, volume server, filer, S3, and WebDAV as separate services:
 
-1. to store billions of files!
-2. to serve the files fast!
+```bash
+wget https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/docker/seaweedfs-compose.yml
+wget -P prometheus https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/docker/prometheus/prometheus.yml
+docker compose -f seaweedfs-compose.yml -p seaweedfs up
+```
 
-SeaweedFS started as a blob store to handle small files efficiently. 
-Instead of managing all file metadata in a central master, 
-the central master only manages volumes on volume servers, 
-and these volume servers manage files and their metadata. 
-This relieves concurrency pressure from the central master and spreads file metadata into volume servers, 
-allowing faster file access (O(1), usually just one disk read operation).
+[Docker Compose for S3][DockerComposeS3] adds credentials, and the [docker/compose](docker/compose) folder has variants for replication, mounts, message queues, and more.
 
-There is only 40 bytes of disk storage overhead for each file's metadata. 
-It is so simple with O(1) disk reads that you are welcome to challenge the performance with your actual use cases.
+## Kubernetes with Helm ##
 
-SeaweedFS started by implementing [Facebook's Haystack design paper](http://www.usenix.org/event/osdi10/tech/full_papers/Beaver.pdf). 
-Also, SeaweedFS implements erasure coding with ideas from 
-[f4: Facebook’s Warm BLOB Storage System](https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-muralidhar.pdf), and has a lot of similarities with [Facebook’s Tectonic Filesystem](https://www.usenix.org/system/files/fast21-pan.pdf) and [Google's Colossus File System](https://cloud.google.com/blog/products/storage-data-transfer/a-peek-behind-colossus-googles-file-system)
+```bash
+helm repo add seaweedfs https://seaweedfs.github.io/seaweedfs/helm
+helm install seaweedfs seaweedfs/seaweedfs -n seaweedfs --create-namespace -f values.yaml
+```
 
-On top of the blob store, optional [Filer] can support directories and POSIX attributes. 
-Filer is a separate linearly-scalable stateless server with customizable metadata stores, 
-e.g., MySql, Postgres, Redis, Cassandra, HBase, Mongodb, Elastic Search, LevelDB, RocksDB, Sqlite, MemSql, TiDB, Etcd, CockroachDB, YDB, etc.
+A production-shaped `values.yaml` for a three-node cluster: two copies of every write, three masters, and an S3 endpoint with credentials and a bucket.
 
-SeaweedFS can transparently integrate with the cloud. 
-With hot data on local cluster, and warm data on the cloud with O(1) access time, 
-SeaweedFS can achieve both fast local access time and elastic cloud storage capacity.
-What's more, the cloud storage access API cost is minimized. 
-Faster and cheaper than direct cloud storage!
+```yaml
+global:
+  seaweedfs:
+    enableReplication: true
+    replicationPlacement: "001"   # one extra copy on another server; "002" for two
 
-SeaweedFS also ships a built-in **Iceberg REST Catalog**, turning the same cluster into a self-contained lakehouse.
-Spark, Trino, Dremio, DuckDB, and RisingWave can query Iceberg tables directly — no Hive Metastore, Glue, or
-external catalog service required. Storage and table metadata live in one system, simplifying on-prem and
-small-team analytics stacks.
+master:
+  replicas: 3
+  data:
+    type: persistentVolumeClaim   # the cluster's default storage class; add storageClass to pick one
+    size: 1Gi
+
+volume:
+  replicas: 3                     # at least 1 + the sum of the replication digits
+  dataDirs:
+    - name: data
+      type: persistentVolumeClaim
+      size: 500Gi
+      maxVolumes: 0               # size the volume count from the disk
+
+filer:
+  replicas: 2
+  data:
+    type: persistentVolumeClaim
+    size: 20Gi
+
+s3:
+  enabled: true
+  replicas: 2
+  enableAuth: true
+  credentials:
+    admin:
+      accessKey: admin
+      secretKey: change-me
+  createBuckets:
+    - name: app-storage
+```
+
+The S3 endpoint is the `seaweedfs-s3` service on port 8333. [Helm Chart Recipes][HelmRecipes] has values for a development cluster, a lakehouse with the Iceberg catalog exposed, filer metadata on PostgreSQL, and node-local disks. The [SeaweedFS Operator][Operator] and the [CSI driver][SeaweedFsCsiDriver] are the other Kubernetes paths.
+
+## Build from source ##
+
+```bash
+git clone https://github.com/seaweedfs/seaweedfs.git
+cd seaweedfs/weed && make install
+```
+
+`weed` lands in `$GOPATH/bin`. [Getting Started][GettingStarted] covers running master, volume, filer, and S3 as separate processes.
+
+## Scale out ##
+
+Capacity is a volume server. Start one on any machine with disk and point it at the master:
+
+```bash
+weed volume -dir=/data -master=<master_host>:9333
+```
+
+Nothing rebalances until you ask it to. Throughput is a filer or S3 gateway; they are stateless, so run as many as you need behind a load balancer. [Production Setup][ProductionSetup] walks through a multi-node cluster.
 
 [Back to TOC](#table-of-contents)
 
-# Features #
-## Additional Blob Store Features ##
-* Support different replication levels, with rack and data center aware.
-* Automatic master servers failover - no single point of failure (SPOF).
-* Automatic compression depending on file MIME type.
-* Automatic compaction to reclaim disk space after deletion or update.
-* [Automatic entry TTL expiration][VolumeServerTTL].
-* Flexible Capacity Expansion: Any server with some disk space can add to the total storage space.
-* Adding/Removing servers does **not** cause any data re-balancing unless triggered by admin commands.
-* Optional picture resizing.
-* Support ETag, Accept-Range, Last-Modified, etc.
-* Support in-memory/leveldb/readonly mode tuning for memory/performance balance.
-* Support rebalancing the writable and readonly volumes.
-* [Customizable Multiple Storage Tiers][TieredStorage]: Customizable storage disk types to balance performance and cost.
-* [Transparent cloud integration][CloudTier]: unlimited capacity via tiered cloud storage for warm data.
-* [Erasure Coding for warm storage][ErasureCoding]  Rack-Aware 10.4 erasure coding reduces storage cost and increases availability. Enterprise version can customize EC ratio.
+# Why SeaweedFS #
+
+## Fast ##
+
+* One disk read per blob. A small file is one blob; a large file is split into chunks of a few MB, each its own blob. A volume server keeps a 16-byte index entry per blob in memory and reads it in a single seek, also for erasure-coded data.
+* The master is not in the read path. Clients cache the volume-to-server mapping and talk to volume servers directly.
+* 40 bytes of metadata per file on disk. Small files are packed into append-only volume files, so there is no per-file inode, no per-file metadata file, no fragmentation, and writes are SSD friendly.
+* Hot data is replicated; [erasure coding][ErasureCoding] is applied to warm data in the background, so writes never pay the encoding cost.
+* The [Rust volume server][RustVolume] is a drop-in for higher throughput and lower tail latency on the same on-disk format.
+
+On one laptop, [`weed benchmark`][Benchmarks] writes 1KB files at 15,700 per second and reads them back at 47,000 per second, and a mixed S3 [warp][S3Benchmark] run totals 3.2 GiB/s. Numbers are in the [Benchmark](#benchmark) section; throughput grows with volume servers and gateways.
+
+## Scalable ##
+
+* The master tracks volumes, not files. A cluster with billions of files has a few thousand volumes, so the master stays small. One master is enough for most clusters; run three for [Raft failover][FailoverMaster].
+* Adding a server adds capacity with no data reshuffle. Balancing, vacuum, erasure coding, and repair run on demand from [`weed shell`][WeedShell] or the [maintenance worker][Worker].
+* Filer and S3 gateways are stateless and scale linearly. Directory metadata lives in a [store you already run][FilerStores]: LevelDB, RocksDB, SQLite, MySQL, PostgreSQL, Cassandra, HBase, MongoDB, Redis, Elasticsearch, etcd, TiKV, FoundationDB, YDB, ArangoDB, Tarantool, and MySQL or PostgreSQL compatible databases such as TiDB, CockroachDB, and MemSQL.
+* Rack and data center aware [replication][Replication], [tiered storage][TieredStorage] across disk types, and [transparent cloud tiering][CloudTier] for unlimited capacity.
+* Files from a byte to [tens of TB][SuperLargeFiles]. Volumes up to 8TB with the large-disk build.
+
+## The most complete S3 API ##
+
+The S3 gateway implements the object, bucket, S3 Tables, IAM, and STS APIs on one endpoint, so the AWS SDKs and CLI, rclone, restic, Spark, and Trino work unchanged.
+
+| API | Operations |
+| --- | --- |
+| S3 bucket and object | 73 |
+| S3 Tables | 36 |
+| IAM | 39 |
+| STS | 5 |
+
+* [Versioning][Versioning], [Object Lock][ObjectLock] with retention and legal hold, [lifecycle][Lifecycle] rules, tagging, [CORS][CORS], [conditional reads and writes][ConditionalOps], checksums, presigned URLs, browser POST uploads, multipart uploads, and an atomic [RenameObject][RenameObject].
+* [Bucket policies][BucketPolicies] with [conditions][PolicyConditions] and [variables][PolicyVariables]; IAM users, groups, and policies; STS with [OIDC][OIDC], LDAP, and [Kubernetes service accounts][K8sSA].
+* [SSE-S3, SSE-KMS, and SSE-C][SSE] server-side encryption, with OpenBao and Vault, AWS KMS, Azure Key Vault, and GCP KMS as key providers.
+* [Audit log][AuditLog], [bucket quota][BucketQuota], and [rate limiting][RateLimiting].
+* Each bucket is its own collection, so deleting a bucket is instant.
+
+The full operation list is in [Amazon S3 API][AmazonS3API], and [Supported APIs vs MinIO][S3vsMinio] compares. The S3 compatibility suite and the SDK, IAM, SSE, policy, and Spark integration tests run in CI on every change.
+
+## A data warehouse with S3 Tables ##
+
+SeaweedFS is a lakehouse in one system. [S3 Table Buckets][S3TableBucket] hold Apache Iceberg tables by default, or [Lance][LanceCatalog] tables for vectors and multimodal data, and the built-in [Iceberg REST Catalog][IcebergCatalog] and Lance namespace serve them directly. There is no Hive Metastore, Glue, or separate catalog service to deploy, secure, and back up.
+
+* Query engines operate on the same tables at the same time: [Spark][SparkIceberg], [Trino][TrinoIceberg], [Dremio][DremioIceberg], [DuckDB][DuckDBIceberg], [Apache Doris][DorisIceberg], [RisingWave][RisingWaveIceberg], ClickHouse, and [LanceDB][LanceDB]. Catalog commits are atomic compare-and-swap, so concurrent writers are safe. [Lakekeeper][Lakekeeper] can front the same storage with STS-vended credentials.
+* [Automated table maintenance][IcebergMaintenance]: compaction, snapshot expiration, orphan file removal, and manifest rewriting, configured per bucket or table through the S3 Tables maintenance APIs, and the same for [Lance][LanceMaintenance].
+* IAM at the bucket, namespace, and table level with standard bucket policies, see [S3 Tables Security][S3TablesSecurity].
+* A [Hadoop compatible file system][Hadoop] for Spark, Flink, and HBase.
+
+`S3_TABLE_BUCKET=warehouse ./weed mini -dir=./data` brings the whole stack up on a laptop.
+
+## A fast cache for cloud storage ##
+
+[Cloud Drive][CloudDrive] mounts a bucket from S3, Google Cloud Storage, Azure, Backblaze B2, Wasabi, Storj, or any S3-compatible store into SeaweedFS and serves it at local speed:
+
+* Metadata is pulled once, so listing, stat, and directory walks cost no cloud API calls.
+* File content is downloaded once, on first read or [warmed][CacheRemote] by folder, name pattern, size, or age, and cached with the capacity of the whole cluster: cache everything, no churn.
+* Local writes complete at local latency and are written back to the cloud asynchronously in the cloud's native layout, so other tools keep reading the bucket directly.
+* Uncache by the same rules to free local disk while keeping the metadata.
+
+[Cloud Tier][CloudTier] goes the other direction, moving whole warm volumes to cloud storage while keeping one-read access, and the [Gateway to Remote Object Storage][GatewayToRemoteObjectStore] mirrors every bucket to a remote store. Faster and cheaper than reading the cloud directly.
+
+## Active-active replication and more ##
+
+* [Active-active or active-passive replication][ActiveActiveAsyncReplication] between clusters, continuous and resumable, for the whole tree or chosen folders, across data centers.
+* [Filer store replication][FilerStoreReplication] for metadata HA, [async backup][AsyncBackup] to cloud storage, [metadata backup][MetaBackup], and [change data capture][CDC] with [webhooks][Webhook] on every metadata event.
+* The same data as a [FUSE mount][Mount] on Linux, macOS, and [Windows][MountWindows], over [WebDAV][WebDAV], [SFTP][SFTP], HDFS, HTTP, and [TUS resumable uploads][TUS]; on Kubernetes through the [CSI driver][SeaweedFsCsiDriver] and [Operator][Operator].
+* [AES256-GCM encryption at rest][FilerDataEncryption], TLS and mTLS between components, JWT-signed volume access, and [FIPS][FIPS] builds.
+* [Admin UI][AdminUI], Prometheus [metrics][Metrics], [TTL][VolumeServerTTL] per file or volume, automatic compression and compaction, and [seaweed-up][SeaweedUp] for bare-metal clusters.
 
 [Back to TOC](#table-of-contents)
 
-## Filer Features ##
-* [Filer server][Filer] provides "normal" directories and files via HTTP.
-* [File TTL][FilerTTL] automatically expires file metadata and actual file data.
-* [Mount filer][Mount] reads and writes files directly as a local directory via FUSE.
-* [Filer Store Replication][FilerStoreReplication] enables HA for filer meta data stores.
-* [Active-Active Replication][ActiveActiveAsyncReplication] enables asynchronous one-way or two-way cross cluster continuous replication.
-* [Amazon S3 compatible API][AmazonS3API] accesses files with S3 tooling.
-* [Hadoop Compatible File System][Hadoop] accesses files from Hadoop/Spark/Flink/etc or even runs HBase.
-* [Async Replication To Cloud][BackupToCloud] has extremely fast local access and backups to Amazon S3, Google Cloud Storage, Azure, BackBlaze.
-* [WebDAV] accesses as a mapped drive on Mac and Windows, or from mobile devices.
-* [AES256-GCM Encrypted Storage][FilerDataEncryption] safely stores the encrypted data.
-* [Super Large Files][SuperLargeFiles] stores large or super large files in tens of TB.
-* [Cloud Drive][CloudDrive] mounts cloud storage to local cluster, cached for fast read and write with asynchronous write back.
-* [Gateway to Remote Object Store][GatewayToRemoteObjectStore] mirrors bucket operations to remote object storage, in addition to [Cloud Drive][CloudDrive]
+# Architecture #
 
-## Data Lakehouse Features ##
-* [S3 Table Buckets][S3TableBucket] expose a dedicated namespace for Iceberg tables with strict layout validation.
-* Built-in [Iceberg REST Catalog][IcebergCatalog] runs alongside the S3 endpoint — no external metastore needed.
-* Native integrations with [Apache Spark][SparkIceberg], [Trino][TrinoIceberg], [Dremio][DremioIceberg], [DuckDB][DuckDBIceberg], and [RisingWave][RisingWaveIceberg].
-* [Automated table maintenance][IcebergMaintenance]: compaction, snapshot expiration, orphan removal, manifest rewriting.
-* Granular IAM at the bucket, namespace, and table level via standard S3 bucket policies.
+![SeaweedFS Architecture](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/SeaweedFS_Architecture.png)
 
-## Kubernetes ##
-* [Kubernetes CSI Driver][SeaweedFsCsiDriver] A Container Storage Interface (CSI) Driver. [![Docker Pulls](https://img.shields.io/docker/pulls/chrislusf/seaweedfs-csi-driver.svg?maxAge=4800)](https://hub.docker.com/r/chrislusf/seaweedfs-csi-driver/)
-* [SeaweedFS Operator](https://github.com/seaweedfs/seaweedfs-operator)
-
-[Filer]: https://github.com/seaweedfs/seaweedfs/wiki/Directories-and-Files
-[SuperLargeFiles]: https://github.com/seaweedfs/seaweedfs/wiki/Data-Structure-for-Large-Files
-[Mount]: https://github.com/seaweedfs/seaweedfs/wiki/FUSE-Mount
-[AmazonS3API]: https://github.com/seaweedfs/seaweedfs/wiki/Amazon-S3-API
-[BackupToCloud]: https://github.com/seaweedfs/seaweedfs/wiki/Async-Replication-to-Cloud
-[Hadoop]: https://github.com/seaweedfs/seaweedfs/wiki/Hadoop-Compatible-File-System
-[WebDAV]: https://github.com/seaweedfs/seaweedfs/wiki/WebDAV
-[ErasureCoding]: https://github.com/seaweedfs/seaweedfs/wiki/Erasure-coding-for-warm-storage
-[TieredStorage]: https://github.com/seaweedfs/seaweedfs/wiki/Tiered-Storage
-[CloudTier]: https://github.com/seaweedfs/seaweedfs/wiki/Cloud-Tier
-[FilerDataEncryption]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-Data-Encryption
-[FilerTTL]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-Stores
-[VolumeServerTTL]: https://github.com/seaweedfs/seaweedfs/wiki/Store-file-with-a-Time-To-Live
-[SeaweedFsCsiDriver]: https://github.com/seaweedfs/seaweedfs-csi-driver
-[ActiveActiveAsyncReplication]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-Active-Active-cross-cluster-continuous-synchronization
-[FilerStoreReplication]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-Store-Replication
-[KeyLargeValueStore]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-as-a-Key-Large-Value-Store
-[CloudDrive]: https://github.com/seaweedfs/seaweedfs/wiki/Cloud-Drive-Architecture
-[GatewayToRemoteObjectStore]: https://github.com/seaweedfs/seaweedfs/wiki/Gateway-to-Remote-Object-Storage
-[S3TableBucket]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Table-Bucket
-[IcebergCatalog]: https://github.com/seaweedfs/seaweedfs/wiki/SeaweedFS-Iceberg-Catalog
-[IcebergMaintenance]: https://github.com/seaweedfs/seaweedfs/wiki/Iceberg-Table-Maintenance
-[SparkIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/Spark-Iceberg-Integration
-[TrinoIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/Trino-Iceberg-Integration
-[DremioIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/Dremio-Iceberg-Integration
-[DuckDBIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/DuckDB-Iceberg-Integration
-[RisingWaveIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/RisingWave-Iceberg-Integration
+* **Master** servers, one or a Raft group of three, track which volume lives on which volume server and hand out file ids. They are not in the read path.
+* **Volume** servers store blobs in append-only volume files, keep a 16-byte in-memory index per blob, and replicate or erasure-code at the volume level.
+* **Filer** servers add directories and files on top, with metadata in a store of your choice, and expose HTTP, S3, WebDAV, SFTP, FUSE, and the table catalogs.
 
+The blob store started from [Facebook's Haystack](http://www.usenix.org/event/osdi10/tech/full_papers/Beaver.pdf), erasure coding takes ideas from [f4](https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-muralidhar.pdf), and the whole has a lot in common with [Tectonic](https://www.usenix.org/system/files/fast21-pan.pdf) and [Colossus](https://cloud.google.com/blog/products/storage-data-transfer/a-peek-behind-colossus-googles-file-system). How file ids are assigned, written, and looked up, and why a master that tracks volumes scales, is in [Blob Store Architecture][BlobStoreArchitecture]; the services are in [Components][Components] and the [white paper][WhitePaper].
 
 [Back to TOC](#table-of-contents)
 
-## Example: Using Seaweed Blob Store ##
-
-By default, the master node runs on port 9333, and the volume nodes run on port 8080.
-Let's start one master node, and two volume nodes on port 8080 and 8081. Ideally, they should be started from different machines. We'll use localhost as an example.
-
-SeaweedFS uses HTTP REST operations to read, write, and delete. The responses are in JSON or JSONP format.
-
-### Start Master Server ###
-
-```
-> ./weed master
-```
-
-### Start Volume Servers ###
-
-```
-> weed volume -dir="/tmp/data1" -max=5  -master="localhost:9333" -port=8080 &
-> weed volume -dir="/tmp/data2" -max=10 -master="localhost:9333" -port=8081 &
-```
-
-### Write A Blob ###
-
-A blob, also referred as a needle, a chunk, or mistakenly as a file, is just a byte array. It can have attributes, such as name, mime type, create or update time, etc. But basically it is just a byte array of a relatively small size, such as 2 MB ~ 64 MB. The size is not fixed.
-
-To upload a blob: first, send a HTTP POST, PUT, or GET request to `/dir/assign` to get an `fid` and a volume server URL:
-
-```
-> curl http://localhost:9333/dir/assign
-{"count":1,"fid":"3,01637037d6","url":"127.0.0.1:8080","publicUrl":"localhost:8080"}
-```
-
-Second, to store the blob content, send a HTTP multi-part POST request to `url + '/' + fid` from the response:
-
-```
-> curl -F file=@/home/chris/myphoto.jpg http://127.0.0.1:8080/3,01637037d6
-{"name":"myphoto.jpg","size":43234,"eTag":"1cc0118e"}
-```
-
-To update, send another POST request with updated blob content.
-
-For deletion, send an HTTP DELETE request to the same `url + '/' + fid` URL:
-
-```
-> curl -X DELETE http://127.0.0.1:8080/3,01637037d6
-```
-
-### Save Blob Id ###
-
-Now, you can save the `fid`, 3,01637037d6 in this case, to a database field.
-
-The number 3 at the start represents a volume id. After the comma, it's one file key, 01, and a file cookie, 637037d6.
-
-The volume id is an unsigned 32-bit integer. The file key is an unsigned 64-bit integer. The file cookie is an unsigned 32-bit integer, used to prevent URL guessing.
-
-The file key and file cookie are both coded in hex. You can store the <volume id, file key, file cookie> tuple in your own format, or simply store the `fid` as a string.
-
-If stored as a string, in theory, you would need 8+1+16+8=33 bytes. A char(33) would be enough, if not more than enough, since most uses will not need 2^32 volumes.
-
-If space is really a concern, you can store the file id in the binary format. You would need one 4-byte integer for volume id, 8-byte long number for file key, and a 4-byte integer for the file cookie. So 16 bytes are more than enough.
-
-### Read a Blob ###
-
-Here is an example of how to render the URL.
-
-First look up the volume server's URLs by the file's volumeId:
-
-```
-> curl http://localhost:9333/dir/lookup?volumeId=3
-{"volumeId":"3","locations":[{"publicUrl":"localhost:8080","url":"localhost:8080"}]}
-```
-
-Since (usually) there are not too many volume servers, and volumes don't move often, you can cache the results most of the time. Depending on the replication type, one volume can have multiple replica locations. Just randomly pick one location to read.
-
-Now you can take the public URL, render the URL or directly read from the volume server via URL:
-
-```
- http://localhost:8080/3,01637037d6.jpg
-```
-
-Notice we add a file extension ".jpg" here. It's optional and just one way for the client to specify the file content type.
-
-If you want a nicer URL, you can use one of these alternative URL formats:
-
-```
- http://localhost:8080/3/01637037d6/my_preferred_name.jpg
- http://localhost:8080/3/01637037d6.jpg
- http://localhost:8080/3,01637037d6.jpg
- http://localhost:8080/3/01637037d6
- http://localhost:8080/3,01637037d6
-```
-
-If you want to get a scaled version of an image, you can add some params:
-
-```
-http://localhost:8080/3/01637037d6.jpg?height=200&width=200
-http://localhost:8080/3/01637037d6.jpg?height=200&width=200&mode=fit
-http://localhost:8080/3/01637037d6.jpg?height=200&width=200&mode=fill
-```
-
-### Rack-Aware and Data Center-Aware Replication ###
-
-SeaweedFS applies the replication strategy at a volume level. So, when you are getting a blob id, you can specify the replication strategy. For example:
-
-```
-curl http://localhost:9333/dir/assign?replication=001
-```
-
-The replication parameter options are:
-
-```
-000: no replication
-001: replicate once on the same rack
-010: replicate once on a different rack, but same data center
-100: replicate once on a different data center
-200: replicate twice on two different data center
-110: replicate once on a different rack, and once on a different data center
-```
-
-More details about replication can be found [on the wiki][Replication].
-
-[Replication]: https://github.com/seaweedfs/seaweedfs/wiki/Replication
-
-You can also set the default replication strategy when starting the master server.
-
-### Allocate Blob Key on Specific Data Center ###
-
-Volume servers can be started with a specific data center name:
-
-```
- weed volume -dir=/tmp/1 -port=8080 -dataCenter=dc1
- weed volume -dir=/tmp/2 -port=8081 -dataCenter=dc2
-```
-
-When requesting a blob key, an optional "dataCenter" parameter can limit the assigned volume to the specific data center. For example, this specifies that the assigned volume should be limited to 'dc1':
-
-```
- http://localhost:9333/dir/assign?dataCenter=dc1
-```
-
-### Other Features ###
-  * [No Single Point of Failure][feat-1]
-  * [Insert with your own keys][feat-2]
-  * [Chunking large files][feat-3]
-  * [Collection as a Simple Name Space][feat-4]
-
-[feat-1]: https://github.com/seaweedfs/seaweedfs/wiki/Failover-Master-Server
-[feat-2]: https://github.com/seaweedfs/seaweedfs/wiki/Optimization#insert-with-your-own-keys
-[feat-3]: https://github.com/seaweedfs/seaweedfs/wiki/Optimization#upload-large-files
-[feat-4]: https://github.com/seaweedfs/seaweedfs/wiki/Optimization#collection-as-a-simple-name-space
-
-[Back to TOC](#table-of-contents)
-
-## Blob Store Architecture ##
-
-Usually distributed file systems split each file into chunks. A central server keeps a mapping of filenames to chunks, and also which chunks each chunk server has.
-
-The main drawback is that the central server can't handle many small files efficiently, and since all read requests need to go through the central master, so it might not scale well for many concurrent users.
-
-Instead of managing chunks, SeaweedFS manages data volumes in the master server. Each data volume is 32GB in size, and can hold a lot of blobs. And each storage node can have many data volumes. So the master node only needs to store the metadata about the volumes, which is a fairly small amount of data and is generally stable.
-
-The actual blob metadata, which are the blob volume, offset, and size, is stored in each volume on volume servers. Since each volume server only manages metadata of blobs on its own disk, with only 16 bytes for each blob, all access can read the metadata just from memory and only needs one disk operation to actually read file data.
-
-For comparison, consider that an xfs inode structure in Linux is 536 bytes.
-
-### Master Server and Volume Server ###
-
-The architecture is fairly simple. The actual data is stored in volumes on storage nodes. One volume server can have multiple volumes, and can both support read and write access with basic authentication.
-
-All volumes are managed by a master server. The master server contains the volume id to volume server mapping. This is fairly static information, and can be easily cached.
-
-On each write request, the master server also generates a file key, which is a growing 64-bit unsigned integer. Since write requests are not generally as frequent as read requests, one master server should be able to handle the concurrency well.
-
-### Write and Read files ###
-
-When a client sends a write request, the master server returns (volume id, file key, file cookie, volume node URL) for the blob. The client then contacts the volume node and POSTs the blob content.
-
-When a client needs to read a blob based on (volume id, file key, file cookie), it asks the master server by the volume id for the (volume node URL, volume node public URL), or retrieves this from a cache. Then the client can GET the content, or just render the URL on web pages and let browsers fetch the content.
-
-### Saving memory ###
-
-All blob metadata stored on a volume server is readable from memory without disk access. Each file takes just a 16-byte map entry of <64bit key, 32bit offset, 32bit size>. Of course, each map entry has its own space cost for the map. But usually the disk space runs out before the memory does.
-
-### Tiered Storage to the cloud ###
-
-The local volume servers are much faster, while cloud storages have elastic capacity and are actually more cost-efficient if not accessed often (usually free to upload, but relatively costly to access). With the append-only structure and O(1) access time, SeaweedFS can take advantage of both local and cloud storage by offloading the warm data to the cloud.
-
-Usually hot data are fresh and warm data are old. SeaweedFS puts the newly created volumes on local servers, and optionally upload the older volumes on the cloud. If the older data are accessed less often, this literally gives you unlimited capacity with limited local servers, and still fast for new data. 
-
-With the O(1) access time, the network latency cost is kept at minimum. 
-
-If the hot/warm data is split as 20/80, with 20 servers, you can achieve storage capacity of 100 servers. That's a cost saving of 80%! Or you can repurpose the 80 servers to store new data also, and get 5X storage throughput.
-
-[Back to TOC](#table-of-contents)
-
-## SeaweedFS Filer ##
-
-Built on top of the blob store, SeaweedFS Filer adds directory structure to create a file system. The directory structure is an interface that is implemented in many key-value stores or databases.
-
-The content of a file is mapped to one or many blobs, distributed to multiple volumes on multiple volume servers.
-
-## Compared to Other File Systems ##
+# Compared to Other Systems #
 
 Most other distributed file systems seem more complicated than necessary.
 
@@ -438,9 +270,7 @@ SeaweedFS is meant to be fast and simple, in both setup and operation. If you do
 
 SeaweedFS is constantly moving forward. Same with other systems. These comparisons can be outdated quickly. Please help to keep them updated.
 
-[Back to TOC](#table-of-contents)
-
-### Compared to HDFS ###
+## Compared to HDFS ##
 
 HDFS uses the chunk approach for each file, and is ideal for storing large files.
 
@@ -448,9 +278,7 @@ SeaweedFS is ideal for serving relatively smaller files quickly and concurrently
 
 SeaweedFS can also store extra large files by splitting them into manageable data chunks, and store the file ids of the data chunks into a meta chunk. This is managed by "weed upload/download" tool, and the weed master or volume servers are agnostic about it.
 
-[Back to TOC](#table-of-contents)
-
-### Compared to GlusterFS, Ceph ###
+## Compared to GlusterFS, Ceph ##
 
 The architectures are mostly the same. SeaweedFS aims to store and read files fast, with a simple and flat architecture. The main differences are
 
@@ -469,25 +297,15 @@ The architectures are mostly the same. SeaweedFS aims to store and read files fa
 | MinIO          | separate meta file per drive for each file  |                  |         | Yes   | No                          |
 | RustFS         | separate meta file per drive for each file  |                  |         | Yes   | No                          |
 
-[Back to TOC](#table-of-contents)
+GlusterFS stores files, both directories and content, in configurable volumes called "bricks". It hashes the path and filename into ids, and assigned to virtual volumes, and then mapped to "bricks".
 
-### Compared to GlusterFS ###
-
-GlusterFS stores files, both directories and content, in configurable volumes called "bricks".
-
-GlusterFS hashes the path and filename into ids, and assigned to virtual volumes, and then mapped to "bricks".
-
-[Back to TOC](#table-of-contents)
-
-### Compared to MooseFS ###
+## Compared to MooseFS ##
 
 MooseFS chooses to neglect small file issue. From moosefs 3.0 manual, "even a small file will occupy 64KiB plus additionally 4KiB of checksums and 1KiB for the header", because it "was initially designed for keeping large amounts (like several thousands) of very big files"
 
-MooseFS Master Server keeps all meta data in memory. Same issue as HDFS namenode. 
+MooseFS Master Server keeps all meta data in memory. Same issue as HDFS namenode.
 
-[Back to TOC](#table-of-contents)
-
-### Compared to Ceph ###
+## Compared to Ceph ##
 
 Ceph can be setup similar to SeaweedFS as a key->blob store. It is much more complicated, with the need to support layers on top of it. [Here is a more detailed comparison](https://github.com/seaweedfs/seaweedfs/issues/120)
 
@@ -507,13 +325,11 @@ SeaweedFS Filer uses off-the-shelf stores, such as MySql, Postgres, Sqlite, Mong
 | Volume  | OSD | optimized for small files |
 | Filer  | Ceph FS | linearly scalable, Customizable, O(1) or O(logN) |
 
-[Back to TOC](#table-of-contents)
-
-### Compared to MinIO, RustFS ###
+## Compared to MinIO, RustFS ##
 
 Please note, as Apr 25, 2026 MinIO ceased development. It's strongly discouraged to use that unmaintained software with multiple security bugs. RustFS is a MinIO reimplementation in Rust, Apache 2.0 licensed and still developed, keeping MinIO's storage model down to a byte-compatible on-disk format. So the points below apply to both.
 
-MinIO followed AWS S3 closely and was ideal for testing for S3 API. It had good UI, policies, versionings, etc. SeaweedFS is trying to catch up here. 
+MinIO followed AWS S3 closely and was ideal for testing for S3 API. It had good UI, policies, versionings, etc. SeaweedFS is trying to catch up here.
 
 The metadata are in simple files. Each file write incurs extra writes to the corresponding meta file, on every drive of the erasure set. Changing only tags or retention rewrites that meta file on all of them, so the write amplification does not shrink with object size.
 
@@ -530,113 +346,18 @@ There are specific requirements on storage layout, which makes it hard to scale
 
 [Back to TOC](#table-of-contents)
 
-## Dev Plan ##
+# Benchmark #
 
-* More tools and documentation, on how to manage and scale the system.
-* Read and write stream data.
-* Support structured data.
+Unscientific single-machine numbers from a MacBook with an SSD. [`weed benchmark`][Benchmarks], 1 million 1KB files, concurrency 16:
 
-This is a super exciting project! And we need helpers and [support](https://www.patreon.com/seaweedfs)!
+| | Requests per second | p50 | p99 |
+| --- | --- | --- | --- |
+| Write | 15,708 | 0.8 ms | 2.6 ms |
+| Random read | 47,019 | 0.3 ms | 0.7 ms |
 
-[Back to TOC](#table-of-contents)
-
-## Installation Guide ##
-
-> Installation guide for users who are not familiar with golang
-
-Step 1: install go on your machine and setup the environment by following the instructions at:
-
-https://golang.org/doc/install
-
-make sure to define your $GOPATH
-
-
-Step 2: checkout this repo:
-```bash
-git clone https://github.com/seaweedfs/seaweedfs.git
-```
-Step 3: download, compile, and install the project by executing the following command
-
-```bash
-cd seaweedfs/weed && make install
-```
-
-Once this is done, you will find the executable "weed" in your `$GOPATH/bin` directory
-
-For more installation options, including how to run with Docker, see the [Getting Started guide](https://github.com/seaweedfs/seaweedfs/wiki/Getting-Started).
-
-[Back to TOC](#table-of-contents)
-
-## Disk Related Topics ##
-
-### Hard Drive Performance ###
-
-When testing read performance on SeaweedFS, it basically becomes a performance test of your hard drive's random read speed. Hard drives usually get 100MB/s~200MB/s.
-
-### Solid State Disk ###
-
-To modify or delete small files, SSD must delete a whole block at a time, and move content in existing blocks to a new block. SSD is fast when brand new, but will get fragmented over time and you have to garbage collect, compacting blocks. SeaweedFS is friendly to SSD since it is append-only. Deletion and compaction are done on volume level in the background, not slowing reading and not causing fragmentation.
-
-[Back to TOC](#table-of-contents)
-
-## Benchmark ##
-
-My Own Unscientific Single Machine Results on Mac Book with Solid State Disk, CPU: 1 Intel Core i7 2.6GHz.
-
-Write 1 million 1KB file:
-```
-Concurrency Level:      16
-Time taken for tests:   66.753 seconds
-Completed requests:      1048576
-Failed requests:        0
-Total transferred:      1106789009 bytes
-Requests per second:    15708.23 [#/sec]
-Transfer rate:          16191.69 [Kbytes/sec]
-
-Connection Times (ms)
-              min      avg        max      std
-Total:        0.3      1.0       84.3      0.9
-
-Percentage of the requests served within a certain time (ms)
-   50%      0.8 ms
-   66%      1.0 ms
-   75%      1.1 ms
-   80%      1.2 ms
-   90%      1.4 ms
-   95%      1.7 ms
-   98%      2.1 ms
-   99%      2.6 ms
-  100%     84.3 ms
-```
-
-Randomly read 1 million files:
-```
-Concurrency Level:      16
-Time taken for tests:   22.301 seconds
-Completed requests:      1048576
-Failed requests:        0
-Total transferred:      1106812873 bytes
-Requests per second:    47019.38 [#/sec]
-Transfer rate:          48467.57 [Kbytes/sec]
-
-Connection Times (ms)
-              min      avg        max      std
-Total:        0.0      0.3       54.1      0.2
-
-Percentage of the requests served within a certain time (ms)
-   50%      0.3 ms
-   90%      0.4 ms
-   98%      0.6 ms
-   99%      0.7 ms
-  100%     54.1 ms
-```
-
-### Run WARP and launch a mixed benchmark. ###
+`make benchmark` runs [warp][S3Benchmark] mixed S3 traffic against a local `weed server`:
 
 ```
-make benchmark
-warp: Benchmark data written to "warp-mixed-2025-12-05[194844]-kBpU.csv.zst"
-
 Mixed operations.
 Operation: DELETE, 10%, Concurrency: 20, Ran 42s.
  * Throughput: 55.13 obj/s
@@ -653,17 +374,19 @@ Operation: STAT, 30%, Concurrency: 20, Ran 42s.
 Cluster Total: 3302.88 MiB/s, 550.51 obj/s over 43s.
 ```
 
+Read throughput is bounded by the random read speed of the disks, and grows with every volume server added. More numbers, including multi-node, FUSE, and Hadoop, are in [Benchmarks][Benchmarks], [S3 API Benchmark][S3Benchmark], [FIO benchmark][FIO], and [Independent Benchmarks][IndependentBenchmarks].
+
 [Back to TOC](#table-of-contents)
 
-## Enterprise ##
+# Enterprise #
 
-For enterprise users, please visit [seaweedfs.com](https://seaweedfs.com) for the SeaweedFS Enterprise Edition, 
-which has advanced features, including data recovery, self-healing storage, 
+For enterprise users, please visit [seaweedfs.com](https://seaweedfs.com) for the SeaweedFS Enterprise Edition,
+which has advanced features, including data recovery, self-healing storage,
 customizable erasure coding, EC vacuum and repair, etc.
 
 [Back to TOC](#table-of-contents)
 
-## License ##
+# License #
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -681,5 +404,112 @@ The text of this page is available for modification and reuse under the terms of
 
 [Back to TOC](#table-of-contents)
 
-## Stargazers over time
-[![Stargazers over time](https://starchart.cc/seaweedfs/seaweedfs.svg?variant=adaptive)](https://starchart.cc/seaweedfs/seaweedfs)
+# Sponsors #
+
+<h3 align="center"><a href="https://www.patreon.com/seaweedfs">Sponsor SeaweedFS via Patreon</a></h3>
+
+SeaweedFS is an independent Apache-licensed open source project with its ongoing development made
+possible entirely thanks to the support of these awesome [backers](https://github.com/seaweedfs/seaweedfs/blob/master/backers.md).
+If you'd like to grow SeaweedFS even stronger, please consider joining our
+<a href="https://www.patreon.com/seaweedfs">sponsors on Patreon</a>.
+
+Your support will be really appreciated by me and other supporters!
+
+<!--
+<h4 align="center">Platinum</h4>
+
+<p align="center">
+  <a href="" target="_blank">
+    <img src="https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/sponsor_nodion.png" width="200" alt="nodion">
+  </a>
+</p>
+-->
+
+### Gold Sponsors
+[![nodion](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/sponsor_nodion.png)](https://www.nodion.com)
+[![piknik](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/piknik.png)](https://www.piknik.com)
+[![keepsec](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/keepsec.png)](https://www.keepsec.ca)
+[![zyner](https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/note/sponsor_zyner.png)](https://zyner.org)
+
+[Back to TOC](#table-of-contents)
+
+## Star History
+
+![Star History](note/star_history.svg)
+
+[WeedMini]: https://github.com/seaweedfs/seaweedfs/wiki/Quick-Start-with-weed-mini
+[DockerComposeS3]: https://github.com/seaweedfs/seaweedfs/wiki/Docker-Compose-for-S3
+[HelmRecipes]: https://github.com/seaweedfs/seaweedfs/wiki/Helm-Chart-Recipes
+[Operator]: https://github.com/seaweedfs/seaweedfs-operator
+[SeaweedFsCsiDriver]: https://github.com/seaweedfs/seaweedfs-csi-driver
+[GettingStarted]: https://github.com/seaweedfs/seaweedfs/wiki/Getting-Started
+[ProductionSetup]: https://github.com/seaweedfs/seaweedfs/wiki/Production-Setup
+[ErasureCoding]: https://github.com/seaweedfs/seaweedfs/wiki/Erasure-Coding-for-warm-storage
+[RustVolume]: https://github.com/seaweedfs/seaweedfs/wiki/Rust-Volume-Server
+[Benchmarks]: https://github.com/seaweedfs/seaweedfs/wiki/Benchmarks
+[S3Benchmark]: https://github.com/seaweedfs/seaweedfs/wiki/S3-API-Benchmark
+[FIO]: https://github.com/seaweedfs/seaweedfs/wiki/FIO-benchmark
+[IndependentBenchmarks]: https://github.com/seaweedfs/seaweedfs/wiki/Independent-Benchmarks
+[FailoverMaster]: https://github.com/seaweedfs/seaweedfs/wiki/Failover-Master-Server
+[WeedShell]: https://github.com/seaweedfs/seaweedfs/wiki/weed-shell
+[Worker]: https://github.com/seaweedfs/seaweedfs/wiki/Worker
+[FilerStores]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-Stores
+[Replication]: https://github.com/seaweedfs/seaweedfs/wiki/Replication
+[TieredStorage]: https://github.com/seaweedfs/seaweedfs/wiki/Tiered-Storage
+[CloudTier]: https://github.com/seaweedfs/seaweedfs/wiki/Cloud-Tier
+[SuperLargeFiles]: https://github.com/seaweedfs/seaweedfs/wiki/Data-Structure-for-Large-Files
+[Versioning]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Object-Versioning
+[ObjectLock]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Object-Lock-and-Retention
+[Lifecycle]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Lifecycle
+[CORS]: https://github.com/seaweedfs/seaweedfs/wiki/S3-CORS
+[ConditionalOps]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Conditional-Operations
+[RenameObject]: https://github.com/seaweedfs/seaweedfs/wiki/S3-RenameObject
+[BucketPolicies]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Bucket-Policies
+[PolicyConditions]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Policy-Conditions
+[PolicyVariables]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Policy-Variables
+[OIDC]: https://github.com/seaweedfs/seaweedfs/wiki/OIDC-Integration
+[K8sSA]: https://github.com/seaweedfs/seaweedfs/wiki/Kubernetes-ServiceAccount-Authentication
+[SSE]: https://github.com/seaweedfs/seaweedfs/wiki/Server-Side-Encryption
+[AuditLog]: https://github.com/seaweedfs/seaweedfs/wiki/S3-API-Audit-log
+[BucketQuota]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Bucket-Quota
+[RateLimiting]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Rate-Limiting
+[AmazonS3API]: https://github.com/seaweedfs/seaweedfs/wiki/Amazon-S3-API
+[S3vsMinio]: https://github.com/seaweedfs/seaweedfs/wiki/Supported-APIs-vs-Minio
+[S3TableBucket]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Table-Bucket
+[LanceCatalog]: https://github.com/seaweedfs/seaweedfs/wiki/SeaweedFS-Lance-Catalog
+[IcebergCatalog]: https://github.com/seaweedfs/seaweedfs/wiki/SeaweedFS-Iceberg-Catalog
+[SparkIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/Spark-Iceberg-Integration
+[TrinoIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/Trino-Iceberg-Integration
+[DremioIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/Dremio-Iceberg-Integration
+[DuckDBIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/DuckDB-Iceberg-Integration
+[DorisIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/Doris-Iceberg-Integration
+[RisingWaveIceberg]: https://github.com/seaweedfs/seaweedfs/wiki/RisingWave-Iceberg-Integration
+[LanceDB]: https://github.com/seaweedfs/seaweedfs/wiki/LanceDB-Integration
+[Lakekeeper]: https://github.com/seaweedfs/seaweedfs/wiki/Lakekeeper-Iceberg-Integration
+[IcebergMaintenance]: https://github.com/seaweedfs/seaweedfs/wiki/Iceberg-Table-Maintenance
+[LanceMaintenance]: https://github.com/seaweedfs/seaweedfs/wiki/Lance-Maintenance-Worker
+[S3TablesSecurity]: https://github.com/seaweedfs/seaweedfs/wiki/S3-Tables-Security
+[Hadoop]: https://github.com/seaweedfs/seaweedfs/wiki/Hadoop-Compatible-File-System
+[CloudDrive]: https://github.com/seaweedfs/seaweedfs/wiki/Cloud-Drive-Architecture
+[CacheRemote]: https://github.com/seaweedfs/seaweedfs/wiki/Cache-Remote-Storage
+[GatewayToRemoteObjectStore]: https://github.com/seaweedfs/seaweedfs/wiki/Gateway-to-Remote-Object-Storage
+[ActiveActiveAsyncReplication]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-Active-Active-cross-cluster-continuous-synchronization
+[FilerStoreReplication]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-Store-Replication
+[AsyncBackup]: https://github.com/seaweedfs/seaweedfs/wiki/Async-Backup
+[MetaBackup]: https://github.com/seaweedfs/seaweedfs/wiki/Async-Filer-Metadata-Backup
+[CDC]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-Change-Data-Capture
+[Webhook]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-Notification-Webhook
+[Mount]: https://github.com/seaweedfs/seaweedfs/wiki/FUSE-Mount
+[MountWindows]: https://github.com/seaweedfs/seaweedfs/wiki/Mount-on-Windows
+[WebDAV]: https://github.com/seaweedfs/seaweedfs/wiki/WebDAV
+[SFTP]: https://github.com/seaweedfs/seaweedfs/wiki/SFTP-Server
+[TUS]: https://github.com/seaweedfs/seaweedfs/wiki/TUS-Resumable-Uploads
+[FilerDataEncryption]: https://github.com/seaweedfs/seaweedfs/wiki/Filer-Data-Encryption
+[FIPS]: https://github.com/seaweedfs/seaweedfs/wiki/Cryptography-and-FIPS-Compliance
+[AdminUI]: https://github.com/seaweedfs/seaweedfs/wiki/Admin-UI
+[Metrics]: https://github.com/seaweedfs/seaweedfs/wiki/System-Metrics
+[VolumeServerTTL]: https://github.com/seaweedfs/seaweedfs/wiki/Store-file-with-a-Time-To-Live
+[SeaweedUp]: https://github.com/seaweedfs/seaweedfs/wiki/Deployment-with-seaweed-up
+[BlobStoreArchitecture]: https://github.com/seaweedfs/seaweedfs/wiki/Blob-Store-Architecture
+[Components]: https://github.com/seaweedfs/seaweedfs/wiki/Components
+[WhitePaper]: https://github.com/seaweedfs/seaweedfs/wiki/SeaweedFS_Architecture.pdf

docker/README.md

@@ -31,6 +31,49 @@ docker compose -f seaweedfs-dev-compose.yml -p seaweedfs up
 
 ```
 
+## Verify an image signature
+
+Every image CI pushes to `chrislusf/seaweedfs` and `ghcr.io/chrislusf/seaweedfs` is signed with [cosign](https://docs.sigstore.dev/cosign/verifying/verify/), keyless, by the GitHub Actions workflow that built it, so there is no key to fetch or pin. The signature is attached to the image digest and covers the multi-arch index and each platform image in it; `latest` is the release image under another tag and verifies the same way. Images published before September 2026 predate signing.
+
+```bash
+cosign verify \
+  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
+  --certificate-identity-regexp '^https://github.com/seaweedfs/seaweedfs/\.github/workflows/container_release_unified\.yml@' \
+  chrislusf/seaweedfs:latest
+```
+
+cosign prints the digest it verified. Deploy by that digest, or let an admission controller resolve the tag, so what runs is what was checked.
+
+The identity is `https://github.com/seaweedfs/seaweedfs/.github/workflows/<workflow>@<ref>`. The ref is `refs/tags/<version>` for a release and `refs/heads/master` when a variant was republished by hand. The workflow is `container_release_unified.yml` for the release images, `container_dev.yml` for `dev`, `container_latest.yml` for a `latest` rebuilt by hand, `container_release_foundationdb.yml` for the `_large_disk_foundationdb` release image, and `container_foundationdb_version.yml` or `container_rocksdb_version.yml` for the per-version builds. The regexp above accepts release images only; `container_[a-z_]+\.yml@` accepts everything this repository publishes, `dev` included.
+
+The same check as a Kyverno policy, release images only:
+
+```yaml
+apiVersion: kyverno.io/v1
+kind: ClusterPolicy
+metadata:
+  name: verify-seaweedfs-images
+spec:
+  validationFailureAction: Enforce
+  webhookTimeoutSeconds: 30
+  rules:
+    - name: signed-by-the-release-workflow
+      match:
+        any:
+          - resources:
+              kinds:
+                - Pod
+      verifyImages:
+        - imageReferences:
+            - "docker.io/chrislusf/seaweedfs:*"
+            - "ghcr.io/chrislusf/seaweedfs:*"
+          attestors:
+            - entries:
+                - keyless:
+                    issuer: https://token.actions.githubusercontent.com
+                    subject: https://github.com/seaweedfs/seaweedfs/.github/workflows/container_release_unified.yml@refs/tags/*
+```
+
 ## Local Development
 
 ```bash

go.mod

@@ -5,7 +5,7 @@ go 1.26
 require (
 	cloud.google.com/go v0.123.0 // indirect
 	cloud.google.com/go/pubsub v1.51.1
-	cloud.google.com/go/storage v1.64.0
+	cloud.google.com/go/storage v1.67.0
 	github.com/Shopify/sarama v1.38.1
 	github.com/aws/aws-sdk-go v1.55.8
 	github.com/beorn7/perks v1.0.1 // indirect
@@ -44,8 +44,8 @@ require (
 	github.com/jmespath/go-jmespath v0.4.0 // indirect
 	github.com/json-iterator/go v1.1.12
 	github.com/karlseguin/ccache/v2 v2.0.8
-	github.com/klauspost/compress v1.19.1
-	github.com/klauspost/reedsolomon v1.14.1
+	github.com/klauspost/compress v1.19.2
+	github.com/klauspost/reedsolomon v1.14.2
 	github.com/kurin/blazer v0.5.3
 	github.com/linxGnu/grocksdb v1.10.8
 	github.com/mailru/easyjson v0.9.2 // indirect
@@ -59,9 +59,9 @@ require (
 	github.com/posener/complete v1.2.3
 	github.com/pquerna/cachecontrol v0.2.0
 	github.com/prometheus/client_golang v1.24.1
-	github.com/prometheus/client_model v0.6.2
+	github.com/prometheus/client_model v0.6.3
 	github.com/prometheus/common v0.70.1 // indirect
-	github.com/prometheus/procfs v0.21.1
+	github.com/prometheus/procfs v0.22.0
 	github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 // indirect
 	github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
 	github.com/seaweedfs/goexif v2.0.0+incompatible
@@ -92,14 +92,14 @@ require (
 	gocloud.dev/pubsub/rabbitpubsub v0.46.0
 	golang.org/x/crypto v0.55.0
 	golang.org/x/exp v0.0.0-20260709172345-9ea1abe57597
-	golang.org/x/image v0.44.0
+	golang.org/x/image v0.45.0
 	golang.org/x/net v0.58.0
 	golang.org/x/oauth2 v0.36.0
 	golang.org/x/sys v0.47.0
 	golang.org/x/text v0.41.0 // indirect
 	golang.org/x/tools v0.48.0 // indirect
 	golang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da // indirect
-	google.golang.org/api v0.294.0
+	google.golang.org/api v0.296.0
 	google.golang.org/genproto v0.0.0-20260715232425-e75dac1f907d // indirect
 	google.golang.org/grpc v1.85.0-dev
 	google.golang.org/protobuf v1.36.12
@@ -118,19 +118,19 @@ require (
 	github.com/ThreeDotsLabs/watermill v1.5.2
 	github.com/a-h/templ v0.3.1020
 	github.com/apache/cassandra-gocql-driver/v2 v2.1.2
-	github.com/apache/iceberg-go v0.6.0
+	github.com/apache/iceberg-go v0.6.1-0.20260817192109-c2105090c9e2
 	github.com/apple/foundationdb/bindings/go v0.0.0-20250911184653-27f7192f47c3
 	github.com/arangodb/go-driver v1.6.9
 	github.com/armon/go-metrics v0.4.1
 	github.com/aws/aws-sdk-go-v2 v1.45.1
-	github.com/aws/aws-sdk-go-v2/config v1.32.33
-	github.com/aws/aws-sdk-go-v2/credentials v1.19.34
-	github.com/aws/aws-sdk-go-v2/service/s3 v1.105.2
+	github.com/aws/aws-sdk-go-v2/config v1.32.35
+	github.com/aws/aws-sdk-go-v2/credentials v1.20.1
+	github.com/aws/aws-sdk-go-v2/service/s3 v1.107.0
 	github.com/cespare/xxhash/v2 v2.3.0
 	github.com/cognusion/imaging v1.0.4
 	github.com/fluent/fluent-logger-golang v1.10.1
-	github.com/getsentry/sentry-go v0.48.0
-	github.com/go-ldap/ldap/v3 v3.4.13
+	github.com/getsentry/sentry-go v0.49.0
+	github.com/go-ldap/ldap/v3 v3.4.14
 	github.com/golang-jwt/jwt/v5 v5.3.1
 	github.com/google/flatbuffers/go v0.0.0-20230108230133-3b8644d32c50
 	github.com/hashicorp/golang-lru/v2 v2.0.7
@@ -152,8 +152,9 @@ require (
 	github.com/shirou/gopsutil/v4 v4.26.7
 	github.com/tarantool/go-option v1.1.0
 	github.com/tarantool/go-tarantool/v3 v3.0.1
-	github.com/testcontainers/testcontainers-go v0.43.0
+	github.com/testcontainers/testcontainers-go v0.44.0
 	github.com/tikv/client-go/v2 v2.0.7
+	github.com/twmb/avro v1.8.0
 	github.com/xeipuuv/gojsonschema v1.2.0
 	github.com/ydb-platform/ydb-go-sdk-auth-environ v0.5.2
 	github.com/ydb-platform/ydb-go-sdk/v3 v3.151.1
@@ -177,16 +178,18 @@ require (
 	github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect
 	github.com/FilenCloudDienste/filen-sdk-go v0.0.39 // indirect
 	github.com/ProtonMail/gopenpgp/v3 v3.4.1 // indirect
+	github.com/RoaringBitmap/roaring/v2 v2.24.0 // indirect
 	github.com/a1ex3/zstd-seekable-format-go/pkg v0.10.0 // indirect
 	github.com/adrg/xdg v0.5.3 // indirect
 	github.com/anchore/go-lzo v0.1.1 // indirect
 	github.com/antlr4-go/antlr/v4 v4.13.1 // indirect
 	github.com/apache/arrow-go/v18 v18.7.0 // indirect
 	github.com/apache/thrift v0.24.0 // indirect
-	github.com/aws/aws-sdk-go-v2/service/signin v1.5.4 // indirect
+	github.com/aws/aws-sdk-go-v2/service/signin v1.7.1 // indirect
 	github.com/bahlo/generic-list-go v0.2.0 // indirect
 	github.com/bazelbuild/rules_go v0.46.0 // indirect
 	github.com/biogo/store v0.0.0-20201120204734-aad293a2328f // indirect
+	github.com/bits-and-blooms/bitset v1.24.4 // indirect
 	github.com/blevesearch/snowballstem v0.9.0 // indirect
 	github.com/boombuler/barcode v1.1.0 // indirect
 	github.com/buger/jsonparser v1.2.0 // indirect
@@ -206,11 +209,12 @@ require (
 	github.com/dave/dst v0.27.2 // indirect
 	github.com/diskfs/go-diskfs v1.9.4 // indirect
 	github.com/distribution/reference v0.6.0 // indirect
-	github.com/docker/go-connections v0.6.0 // indirect
+	github.com/docker/go-connections v0.7.0 // indirect
 	github.com/docker/go-units v0.5.0 // indirect
 	github.com/dromara/dongle v1.0.1 // indirect
+	github.com/geoarrow/geoarrow-go v0.0.0-20260403143023-f54751c3e3a1 // indirect
 	github.com/gin-gonic/gin v1.11.0 // indirect
-	github.com/go-asn1-ber/asn1-ber v1.5.8-0.20250403174932-29230038a667 // indirect
+	github.com/go-asn1-ber/asn1-ber v1.5.8 // indirect
 	github.com/go-git/go-billy/v5 v5.9.0 // indirect
 	github.com/goccy/go-yaml v1.18.0 // indirect
 	github.com/golang/geo v0.0.0-20210211234256-740aa86cb551 // indirect
@@ -239,13 +243,14 @@ require (
 	github.com/magiconair/properties v1.8.10 // indirect
 	github.com/moby/docker-image-spec v1.3.1 // indirect
 	github.com/moby/go-archive v0.3.0 // indirect
-	github.com/moby/moby/api v1.54.2 // indirect
-	github.com/moby/moby/client v0.4.0 // indirect
+	github.com/moby/moby/api v1.55.0 // indirect
+	github.com/moby/moby/client v0.5.0 // indirect
 	github.com/moby/patternmatcher v0.6.1 // indirect
 	github.com/moby/sys/sequential v0.7.0 // indirect
 	github.com/moby/sys/user v0.4.1 // indirect
 	github.com/moby/sys/userns v0.1.0 // indirect
 	github.com/moby/term v0.5.2 // indirect
+	github.com/mschoch/smat v0.2.0 // indirect
 	github.com/oklog/ulid/v2 v2.1.1 // indirect
 	github.com/opencontainers/go-digest v1.0.0 // indirect
 	github.com/opencontainers/image-spec v1.1.1 // indirect
@@ -269,7 +274,6 @@ require (
 	github.com/substrait-io/substrait v0.87.0 // indirect
 	github.com/substrait-io/substrait-go/v8 v8.1.1 // indirect
 	github.com/substrait-io/substrait-protobuf/go v0.85.0 // indirect
-	github.com/twmb/avro v1.7.2 // indirect
 	github.com/twpayne/go-geom v1.6.1 // indirect
 	github.com/twpayne/go-kml/v3 v3.2.1 // indirect
 	github.com/tyler-smith/go-bip39 v1.1.0 // indirect
@@ -279,8 +283,8 @@ require (
 	github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
 	github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
 	github.com/zeebo/xxh3 v1.1.0 // indirect
-	go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0 // indirect
-	go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.42.0 // indirect
+	go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.44.0 // indirect
+	go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.44.0 // indirect
 	go.opentelemetry.io/otel/exporters/zipkin v1.36.0 // indirect
 	go.opentelemetry.io/proto/otlp v1.10.0 // indirect
 	go.uber.org/mock v0.5.2 // indirect
@@ -311,7 +315,7 @@ require (
 	github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.57.0 // indirect
 	github.com/IBM/go-sdk-core/v5 v5.23.1 // indirect
 	github.com/Max-Sum/base32768 v0.0.0-20230304063302-18e6ce5945fd // indirect
-	github.com/Microsoft/go-winio v0.6.2 // indirect
+	github.com/Microsoft/go-winio v0.6.3-0.20251027160822-ad3df93bed29 // indirect
 	github.com/ProtonMail/bcrypt v0.0.0-20211005172633-e235017c1baf // indirect
 	github.com/ProtonMail/gluon v0.17.1-0.20230724134000-308be39be96e // indirect
 	github.com/ProtonMail/go-crypto v1.4.1 // indirect
@@ -322,21 +326,21 @@ require (
 	github.com/andybalholm/cascadia v1.3.4 // indirect
 	github.com/appscode/go-querystring v0.0.0-20170504095604-0126cfb3f1dc // indirect
 	github.com/arangodb/go-velocypack v0.0.0-20200318135517-5af53c29c67e // indirect
-	github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.14 // indirect
-	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.35 // indirect
+	github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.16 // indirect
+	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.19.1 // indirect
 	github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.22.34 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.35 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.35 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.36 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.15 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.23 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.35 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.31 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/configsources v1.5.1 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.8.1 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/v4a v1.5.1 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.19 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.28 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.14.1 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.36 // indirect
 	github.com/aws/aws-sdk-go-v2/service/sns v1.39.14 // indirect
 	github.com/aws/aws-sdk-go-v2/service/sqs v1.42.24 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sso v1.33.4 // indirect
-	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.38.4 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sts v1.45.4
+	github.com/aws/aws-sdk-go-v2/service/sso v1.35.1 // indirect
+	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.40.1 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sts v1.47.1
 	github.com/aws/smithy-go v1.28.1
 	github.com/boltdb/bolt v1.3.1 // indirect
 	github.com/bradenaw/juniper v0.15.3 // indirect
@@ -369,7 +373,7 @@ require (
 	github.com/go-chi/chi/v5 v5.3.1 // indirect
 	github.com/go-darwin/apfs v0.0.0-20211011131704-f84b94dbf348 // indirect
 	github.com/go-jose/go-jose/v4 v4.1.4 // indirect
-	github.com/go-logr/logr v1.4.3 // indirect
+	github.com/go-logr/logr v1.4.4 // indirect
 	github.com/go-logr/stdr v1.2.2 // indirect
 	github.com/go-ole/go-ole v1.3.0 // indirect
 	github.com/go-openapi/errors v0.22.8 // indirect
@@ -434,10 +438,10 @@ require (
 	github.com/oracle/oci-go-sdk/v65 v65.121.0 // indirect
 	github.com/panjf2000/ants/v2 v2.12.1 // indirect
 	github.com/patrickmn/go-cache v2.1.0+incompatible // indirect
-	github.com/pelletier/go-toml/v2 v2.2.4 // indirect
+	github.com/pelletier/go-toml/v2 v2.4.1 // indirect
 	github.com/pengsrc/go-shared v0.2.1-0.20190131101655-1999055a4a14 // indirect
 	github.com/philhofer/fwd v1.2.0 // indirect
-	github.com/pierrec/lz4/v4 v4.1.28
+	github.com/pierrec/lz4/v4 v4.1.29
 	github.com/pingcap/errors v0.11.5-0.20211224045212-9687c2b0f87c // indirect
 	github.com/pingcap/failpoint v0.0.0-20220801062533-2eaa32854a6c // indirect
 	github.com/pingcap/kvproto v0.0.0-20230403051650-e166ae588106 // indirect
@@ -486,13 +490,13 @@ require (
 	go.etcd.io/etcd/api/v3 v3.7.1 // indirect
 	go.opentelemetry.io/auto/sdk v1.2.1 // indirect
 	go.opentelemetry.io/contrib/detectors/gcp v1.44.0 // indirect
-	go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.68.0 // indirect
+	go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.69.0 // indirect
 	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.69.0 // indirect
-	go.opentelemetry.io/otel v1.44.0 // indirect
-	go.opentelemetry.io/otel/metric v1.44.0 // indirect
-	go.opentelemetry.io/otel/sdk v1.44.0 // indirect
-	go.opentelemetry.io/otel/sdk/metric v1.44.0 // indirect
-	go.opentelemetry.io/otel/trace v1.44.0 // indirect
+	go.opentelemetry.io/otel v1.45.0 // indirect
+	go.opentelemetry.io/otel/metric v1.45.0 // indirect
+	go.opentelemetry.io/otel/sdk v1.45.0 // indirect
+	go.opentelemetry.io/otel/sdk/metric v1.45.0 // indirect
+	go.opentelemetry.io/otel/trace v1.45.0 // indirect
 	go.uber.org/multierr v1.11.0 // indirect
 	go.uber.org/zap v1.27.1 // indirect
 	golang.org/x/term v0.45.0
@@ -516,13 +520,6 @@ require (
 
 // replace github.com/seaweedfs/raft => /Users/chrislu/go/src/github.com/seaweedfs/raft
 
-// apache/thrift v0.23.0 fixes CVE-2026-41602 but compares int against the
-// untyped math.MaxUint32 in lib/go/thrift/framed_transport.go, which overflows
-// int on 32-bit GOARCHes (e.g. openbsd/arm, linux/arm) and fails to compile.
-// Upstream fixed the range check post-release; pin to that commit until the
-// next tagged release carries both the CVE fix and the 32-bit fix.
-replace github.com/apache/thrift => github.com/apache/thrift v0.23.1-0.20260429145742-d2acd3c49e58
-
 // tyler-smith/go-bip39 was deleted from GitHub, so `go mod download` fails for
 // anyone resolving it directly (GOPROXY=direct). It only reaches us transitively
 // through rclone's internxt backend, which calls IsMnemonicValid and NewSeed.

go.sum

@@ -485,8 +485,8 @@ cloud.google.com/go/storage v1.22.1/go.mod h1:S8N1cAStu7BOeFfE8KAQzmyyLkK8p/vmRq
 cloud.google.com/go/storage v1.23.0/go.mod h1:vOEEDNFnciUMhBeT6hsJIn3ieU5cFRmzeLgDvXzfIXc=
 cloud.google.com/go/storage v1.27.0/go.mod h1:x9DOL8TK/ygDUMieqwfhdpQryTeEkhGKMi80i/iqR2s=
 cloud.google.com/go/storage v1.28.1/go.mod h1:Qnisd4CqDdo6BGs2AD5LLnEsmSQ80wQ5ogcBBKhU86Y=
-cloud.google.com/go/storage v1.64.0 h1:KLpxI/oX9LxeRsNqn877d2WyeT3ryiEwnGt8pwcSPZg=
-cloud.google.com/go/storage v1.64.0/go.mod h1:lWyAtwvDZHdL3k68WVKbESP6bmWaV23ZJJ/JEVw/ZaQ=
+cloud.google.com/go/storage v1.67.0 h1:8xD3NvKuHE4c+b/kmSIVBdTc0BiOjplx+KecKGR9fr8=
+cloud.google.com/go/storage v1.67.0/go.mod h1:UsS9OgFg/XHOSYakQ8ZtLWWeyGkk1WnmD/GsGfN0BHM=
 cloud.google.com/go/storagetransfer v1.5.0/go.mod h1:dxNzUopWy7RQevYFHewchb29POFv3/AaBgnhqzqiK0w=
 cloud.google.com/go/storagetransfer v1.6.0/go.mod h1:y77xm4CQV/ZhFZH75PLEXY0ROiS7Gh6pSKrM8dJyg6I=
 cloud.google.com/go/storagetransfer v1.7.0/go.mod h1:8Giuj1QNb1kfLAiWM1bN6dHzfdlDAVC9rv9abHot2W4=
@@ -587,8 +587,8 @@ github.com/DATA-DOG/go-sqlmock v1.5.2 h1:OcvFkGmslmlZibjAjaHm3L//6LiuBgolP7Oputl
 github.com/DATA-DOG/go-sqlmock v1.5.2/go.mod h1:88MAG/4G7SMwSE3CeA0ZKzrT5CiOU3OJ+JlNzwDqpNU=
 github.com/DataDog/datadog-go v3.2.0+incompatible/go.mod h1:LButxg5PwREeZtORoXG3tL4fMGNddJ+vMq1mwgfaqoQ=
 github.com/DataDog/zstd v1.5.2/go.mod h1:g4AWEaM3yOg3HYfnJ3YIawPnVdXJh9QME85blwSAmyw=
-github.com/DefangLabs/secret-detector v0.0.0-20250403165618-22662109213e h1:rd4bOvKmDIx0WeTv9Qz+hghsgyjikFiPrseXHlKepO0=
-github.com/DefangLabs/secret-detector v0.0.0-20250403165618-22662109213e/go.mod h1:blbwPQh4DTlCZEfk1BLU4oMIhLda2U+A840Uag9DsZw=
+github.com/DefangLabs/secret-detector v0.0.0-20250811234530-d4b4214cd679 h1:qNT7R4qrN+5u5ajSbqSW1opHP4LA8lzA+ASyw5MQZjs=
+github.com/DefangLabs/secret-detector v0.0.0-20250811234530-d4b4214cd679/go.mod h1:blbwPQh4DTlCZEfk1BLU4oMIhLda2U+A840Uag9DsZw=
 github.com/FilenCloudDienste/filen-sdk-go v0.0.39 h1:tgV5jYL6dsXop9TpDTIQU6UwJjws122HrwskaEE/igY=
 github.com/FilenCloudDienste/filen-sdk-go v0.0.39/go.mod h1:0cBhKXQg49XbKZZfk5TCDa3sVLP+xMxZTWL+7KY0XR0=
 github.com/Files-com/files-sdk-go/v3 v3.3.194 h1:dtOFxSTWWRpkmvXa6ycNiw8dVDu1wkgzcXyVV1VafNc=
@@ -620,8 +620,8 @@ github.com/Masterminds/semver/v3 v3.2.0/go.mod h1:qvl/7zhW3nngYb5+80sSMF+FG2BjYr
 github.com/Max-Sum/base32768 v0.0.0-20230304063302-18e6ce5945fd h1:nzE1YQBdx1bq9IlZinHa+HVffy+NmVRoKr+wHN8fpLE=
 github.com/Max-Sum/base32768 v0.0.0-20230304063302-18e6ce5945fd/go.mod h1:C8yoIfvESpM3GD07OCHU7fqI7lhwyZ2Td1rbNbTAhnc=
 github.com/Microsoft/go-winio v0.5.2/go.mod h1:WpS1mjBmmwHBEWmogvA2mj8546UReBk4v8QkMxJ6pZY=
-github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
-github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
+github.com/Microsoft/go-winio v0.6.3-0.20251027160822-ad3df93bed29 h1:0kQAzHq8vLs7Pptv+7TxjdETLf/nIqJpIB4oC6Ba4vY=
+github.com/Microsoft/go-winio v0.6.3-0.20251027160822-ad3df93bed29/go.mod h1:ZWa7ssZJT30CCDGJ7fk/2SBTq9BIQrrVjrcss0UW2s0=
 github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
 github.com/ProtonMail/bcrypt v0.0.0-20210511135022-227b4adcab57/go.mod h1:HecWFHognK8GfRDGnFQbW/LiV7A3MX3gZVs45vk5h8I=
 github.com/ProtonMail/bcrypt v0.0.0-20211005172633-e235017c1baf h1:yc9daCCYUefEs69zUkSzubzjBbL+cmOXgnmt9Fyd9ug=
@@ -637,6 +637,8 @@ github.com/ProtonMail/gopenpgp/v3 v3.4.1 h1:K7uUhSHSJxORZ+RuHpilTT6S4MA2whCRlXNw
 github.com/ProtonMail/gopenpgp/v3 v3.4.1/go.mod h1:bGdV9f6edhmd581wzXsQCTKdH8bXBbyhkgDKPjwPc6U=
 github.com/PuerkitoBio/goquery v1.12.0 h1:pAcL4g3WRXekcB9AU/y1mbKez2dbY2AajVhtkO8RIBo=
 github.com/PuerkitoBio/goquery v1.12.0/go.mod h1:802ej+gV2y7bbIhOIoPY5sT183ZW0YFofScC4q/hIpQ=
+github.com/RoaringBitmap/roaring/v2 v2.24.0 h1:zQkkBZtG3WRP4j+P3A5DO221SvL1Br88TJkhyqEQRZo=
+github.com/RoaringBitmap/roaring/v2 v2.24.0/go.mod h1:SfT3of9nYh3vis1dIbCj4Yw6KQGujTN+f345nrN/0JA=
 github.com/Sereal/Sereal/Go/sereal v0.0.0-20231009093132-b9187f1a92c6/go.mod h1:JwrycNnC8+sZPDyzM3MQ86LvaGzSpfxg885KOOwFRW4=
 github.com/Shopify/sarama v1.38.1 h1:lqqPUPQZ7zPqYlWpTh+LQ9bhYNu2xJL6k1SJN4WVe2A=
 github.com/Shopify/sarama v1.38.1/go.mod h1:iwv9a67Ha8VNa+TifujYoWGxWnu2kNVAQdSdZ4X2o5g=
@@ -690,10 +692,11 @@ github.com/apache/arrow-go/v18 v18.7.0/go.mod h1:PM6IigLJkdMwIpeHXnymo+xZ52f42a9
 github.com/apache/arrow/go/v10 v10.0.1/go.mod h1:YvhnlEePVnBS4+0z3fhPfUy7W1Ikj0Ih0vcRo/gZ1M0=
 github.com/apache/cassandra-gocql-driver/v2 v2.1.2 h1:lu/p0Db2av18enHJvWJQoChLssI0P+AR06STq4VdvCc=
 github.com/apache/cassandra-gocql-driver/v2 v2.1.2/go.mod h1:QH/asJjB3mHvY6Dot6ZKMMpTcOrWJ8i9GhsvG1g0PK4=
-github.com/apache/iceberg-go v0.6.0 h1:tOVhC5BhBOEgPTowo5AVrPnAgsDo00qEbUPprFcLd4s=
-github.com/apache/iceberg-go v0.6.0/go.mod h1:kESfDlyaW/6hK0WW6TzA4EWps+0NMhhrDYE4qajVjlo=
-github.com/apache/thrift v0.23.1-0.20260429145742-d2acd3c49e58 h1:rDLE+tSW60VzRD7v5I+DU22Mjhmm+mfLc5Xl5dHkx6w=
-github.com/apache/thrift v0.23.1-0.20260429145742-d2acd3c49e58/go.mod h1:zPt6WxgvTOM6hF92y8C+MkEM5LMxZuk4JcQOiU4Esvs=
+github.com/apache/iceberg-go v0.6.1-0.20260817192109-c2105090c9e2 h1:xRULj4L2wrlAAPAYNruyuP17lz2zq2DGLgIFDgqiBjo=
+github.com/apache/iceberg-go v0.6.1-0.20260817192109-c2105090c9e2/go.mod h1:u6gs2aFRl7QOL0FqfHX6DOzvxOyev6mmq3XT+3VUS9M=
+github.com/apache/thrift v0.16.0/go.mod h1:PHK3hniurgQaNMZYaCLEqXKsYK8upmhPbmdP2FXSqgU=
+github.com/apache/thrift v0.24.0 h1:zy31L1a49QTNB2bG1BBfMXol3yJrTH975G3pPubQVLQ=
+github.com/apache/thrift v0.24.0/go.mod h1:zPt6WxgvTOM6hF92y8C+MkEM5LMxZuk4JcQOiU4Esvs=
 github.com/apple/foundationdb/bindings/go v0.0.0-20250911184653-27f7192f47c3 h1:WZaTKNHCfcw7fWSR6/RKnCldVzvYZC+Y20Su4lffEIg=
 github.com/apple/foundationdb/bindings/go v0.0.0-20250911184653-27f7192f47c3/go.mod h1:OMVSB21p9+xQUIqlGizHPZfjK+SHws1ht+ZytVDoz9U=
 github.com/appscode/go-querystring v0.0.0-20170504095604-0126cfb3f1dc h1:LoL75er+LKDHDUfU5tRvFwxH0LjPpZN8OoG8Ll+liGU=
@@ -709,46 +712,46 @@ github.com/aws/aws-sdk-go v1.55.8 h1:JRmEUbU52aJQZ2AjX4q4Wu7t4uZjOu71uyNmaWlUkJQ
 github.com/aws/aws-sdk-go v1.55.8/go.mod h1:ZkViS9AqA6otK+JBBNH2++sx1sgxrPKcSzPPvQkUtXk=
 github.com/aws/aws-sdk-go-v2 v1.45.1 h1:iIoG3NaLhV6UZpPXyPXlDj2I9oS8tV/nMcMnITCC6Ks=
 github.com/aws/aws-sdk-go-v2 v1.45.1/go.mod h1:bttEH6JqnUL8LepvDVfdrds/fZ5bCIxzpe3abyUrhDU=
-github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.14 h1:3IZY0XAJquT3aHzbkHfPzy4ACPcEjVG0x87KOwtpqGY=
-github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.14/go.mod h1:zwM6veDkhGgQFqkBy+uT28AAYpLu+uFMlPl+rCg/73E=
-github.com/aws/aws-sdk-go-v2/config v1.32.33 h1:M1m/Q6f0OKDEDGwhiNOqx1OjTdrewe3v+GDbHmKczWk=
-github.com/aws/aws-sdk-go-v2/config v1.32.33/go.mod h1:fGj1iQj2QpIZzp7jE4aQQ+71TE8cd4z9K4+xCd6EqmE=
-github.com/aws/aws-sdk-go-v2/credentials v1.19.34 h1:y6GkSmcv5myd1ngrYbGmiLlwQqB6TQhOuN/tbSSuWDY=
-github.com/aws/aws-sdk-go-v2/credentials v1.19.34/go.mod h1:w3dTcnDVoQIewjo7JG45hduAToikiIFLC4FIO7fndvw=
-github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.35 h1:+S7kbJoLDDQ5tE+lHrUBgMkzC8NLgsaioS2F3dVoFAE=
-github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.35/go.mod h1:Ak7xXviIARfFdNUJ9Etb0bdVDt/KAvKjMGJVLWXDzik=
+github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.16 h1:aiuaKlDweRC5qExJondpWjOgyzMHpofpwspGXUtwn4c=
+github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.16/go.mod h1:nG/LOlmox9BDe9HvQnXWzgcK8uKbgBMZ/Hp5pVt/21I=
+github.com/aws/aws-sdk-go-v2/config v1.32.35 h1:UEzXuET8E42lxBPijuACu/tEK7v5lFPlk0Q+GT5WD9E=
+github.com/aws/aws-sdk-go-v2/config v1.32.35/go.mod h1:KaMtJpFa2JlL2BStjjHQVwQpzZEmw+ND/EgVrfFoo2g=
+github.com/aws/aws-sdk-go-v2/credentials v1.20.1 h1:Z8GRNEx0u9sDkZOq4PUnN8mjGwbUQGRzMSXpvt3d8xQ=
+github.com/aws/aws-sdk-go-v2/credentials v1.20.1/go.mod h1:uBIK00kFo95dnemqfFMTWx0X8YRqsh6ecIoCjjOkZqM=
+github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.19.1 h1:YIEBqcqRnpi4Pfv0YHImtgi6czGCwKHANC7SwmUAVD0=
+github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.19.1/go.mod h1:imEf0oufgAo8KAkCHhrOdqGEC0YWx1PPBQH82shSxGw=
 github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.22.34 h1:Pn7OsMwBLbkZ6OnCxWHAjf0L/22H8cnhxZC0uPwtMtg=
 github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.22.34/go.mod h1:eToXR/Gk1uqpn04eSmdgVXwfS0WvH8aG4eBFr8ygbpU=
-github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.2.3 h1:w5OoDiMN6x53ROmiIImGzmVcxXv2q1GXY+aKV4WAJYM=
-github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.2.3/go.mod h1:dAhgYp776bX3LuWvnSCFwQEjNs6fuFg7YXIy5PXcP3Q=
-github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.35 h1:kzVuGlatQtYinwBJEEyLAbggepCoavosiaHHX9+fD+c=
-github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.35/go.mod h1:0yLx0yEI+SfqeJMPvOtIEFoZbiQYXMGszBueiutQyaI=
-github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.35 h1:WK6CjihTuLisCjSKKbildJ79sGZZgbBz3iNa7VsKIhU=
-github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.35/go.mod h1:KYleN57luLoe97R7vTnx8PMcVrr9gAcRECtOjl91DNg=
-github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.36 h1:jbGY4CXLzZElOXgGsexlC3Hi+3YM0rSmk4opFXKqg/k=
-github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.36/go.mod h1:uBu/9aKsS/UQGc72RAt3y54kjgYQxmhut8ZD2dXCDNE=
-github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.15 h1:JJLBQxwY+AFwuPAi5ivGc1ChnTdUt4cXMv7e76m2c/Y=
-github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.15/go.mod h1:lQknBIe78MVL0cQOQDlag8KGflMbMEVFx9mB6O8ENvk=
-github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.23 h1:9Fjh6fi/U5JEStVZijmaMpUwE/gvBJj7x2B/PjbO9To=
-github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.23/go.mod h1:iMoT2f1tClxrWAAnKCXjZQ6LOmfLrMG14wmnWpM+F14=
-github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.35 h1:BBEElKh4a+rKshvjrfpajTe9CbpZvrbb4Jkg2PB7RzA=
-github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.35/go.mod h1:zaZk983w//8beSruBVec/mr4CmDwgZitW/qzGhAAX0g=
-github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.31 h1:uao4A3QZ5UmB326V6KF+qRpv9Tjz7IlnlnTbbANntlU=
-github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.31/go.mod h1:I/1+z0VwL1GhQyLgkoHDlygpUZ+iTAwOQ/NsftiUL2I=
-github.com/aws/aws-sdk-go-v2/service/s3 v1.105.2 h1:5C00eQYpTrgQXnp6V3P6P7zPElna3AXvlukbANE6nJI=
-github.com/aws/aws-sdk-go-v2/service/s3 v1.105.2/go.mod h1:zdmCoFO/dSI7GlrwsPqFJI+WlFnSU4Tc8TJnlXrM1Do=
-github.com/aws/aws-sdk-go-v2/service/signin v1.5.4 h1:cOJELVNrq5Q3Udry2GLuHUM7MhwpeaQRdYaoa6GI/yI=
-github.com/aws/aws-sdk-go-v2/service/signin v1.5.4/go.mod h1:f4LxzKBtaTxD7xh3PiVg3CE1tchQemfmghaJr+NbK2c=
+github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.3.11 h1:eBXB8KZgzQ8A9QB4iJS4aw/u6+4OY3i2hQXPABeAIOg=
+github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.3.11/go.mod h1:N9+5pG27Fy61GUL5YXVLXDTLmUudMrgwsuDbgBMNLxQ=
+github.com/aws/aws-sdk-go-v2/internal/configsources v1.5.1 h1:pc138gM1CW+XPc60rEwUlwwuwWFQK16CI1T7v1F9Oec=
+github.com/aws/aws-sdk-go-v2/internal/configsources v1.5.1/go.mod h1:1+koxpPIbfBdfzP6vojm5/zTpTQ/micYwlxIiNB3TxI=
+github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.8.1 h1:K0JsbZQj+1h208Ro1zHeA4l7bMp0NvRffHQ91q8Ol1s=
+github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.8.1/go.mod h1:W3/vL6EtCIatICGy9ab29QhMuae+cOKPWcMxv02CO+Q=
+github.com/aws/aws-sdk-go-v2/internal/v4a v1.5.1 h1:yhw5KD1phVyP9vijxOUzDfEtJx+bt+L63k+VfuiYFAA=
+github.com/aws/aws-sdk-go-v2/internal/v4a v1.5.1/go.mod h1:ZW2e0d7DYlRxlS9hEiMXE47gTdX5KRN4byUiNbUpG+Q=
+github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.19 h1:bAdDl/HkGCcGPoe25ToSHEw23VIxt6CT5fLcg111BKg=
+github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.19/go.mod h1:KaUzbLxv4CeSxh6ZCl9B4m7CuFenS8kUEaDs+f/DQr4=
+github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.28 h1:Q1TF1J9jVD+vFo0LzNnmNdQ9EAt52TS+MQlq9Ir+Yxo=
+github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.28/go.mod h1:4KqXXC/p1hrotmouDFbrRoWaLy962b9PMUReCG6+uWo=
+github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.14.1 h1:RmmWQPREQdk9U+PfqeHW3MqZaBaNK7TpV9W3RY+b+7g=
+github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.14.1/go.mod h1:0A3W4F+68ZnNk5XcNL/e9HFMwnP8RlEicFfy6eOEDyw=
+github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.36 h1:EUIwBoN+q7UmhAejxgD27APiRjh1vwCFo53gSqdT0BM=
+github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.36/go.mod h1:6u00gmlTGR6W0b2k9NBrld7MnOEmf1Spqx0VVt6AqyE=
+github.com/aws/aws-sdk-go-v2/service/s3 v1.107.0 h1:OkYV+1171za+ab9otU1tGxMXhx6uZvwVEtVddjLuYTg=
+github.com/aws/aws-sdk-go-v2/service/s3 v1.107.0/go.mod h1:5FTZoQxhmLEiCAtYVk6V+t0iS/B5yGZVLZ3Wq5FDJZI=
+github.com/aws/aws-sdk-go-v2/service/signin v1.7.1 h1:mdMtSVKdQ3+mzBh+l0ogrFYZVQUCg6pJZOirA2ARsYE=
+github.com/aws/aws-sdk-go-v2/service/signin v1.7.1/go.mod h1:9IqUlsJDbUPcg6cgx3WEzXdjrbWzLDQrak0aaSqlTcI=
 github.com/aws/aws-sdk-go-v2/service/sns v1.39.14 h1:p8WdWDh5AwSZdp19Haa3XMyPCICi9Z375a/Nu3IIEZY=
 github.com/aws/aws-sdk-go-v2/service/sns v1.39.14/go.mod h1:NKVY7DER6VXHkt2I/ycmHakALNboi3Rqwt4eEf/1Cnk=
 github.com/aws/aws-sdk-go-v2/service/sqs v1.42.24 h1:JP2wjWGmUp8lTCZb13Dv0Eciyc1jbO8pd0HZVMHFlrc=
 github.com/aws/aws-sdk-go-v2/service/sqs v1.42.24/go.mod h1:Ql9ziDutk8ERAN9HMaYANCW3lop451ppebkxEJMLCTM=
-github.com/aws/aws-sdk-go-v2/service/sso v1.33.4 h1:AMW7a7S8iQaHjBYZdU3PCq4GKRPijTPRAc7e6XtEThY=
-github.com/aws/aws-sdk-go-v2/service/sso v1.33.4/go.mod h1:QQNsFV1DVXoXcZt18FS8lI8rtUrlDyAuWZLQ5shunv4=
-github.com/aws/aws-sdk-go-v2/service/ssooidc v1.38.4 h1:AsbZcJAQPRmHDJG8K1N0pof/1zPWjVT8TFlTWuGLSvo=
-github.com/aws/aws-sdk-go-v2/service/ssooidc v1.38.4/go.mod h1:6imqztH0//t0mKbl6yWl7swSEl7F/w32oAmqB3vP1ag=
-github.com/aws/aws-sdk-go-v2/service/sts v1.45.4 h1:w/AryDYMjSUANSQ2uoZxJovUsMTwWJNTv3IMex30Y+4=
-github.com/aws/aws-sdk-go-v2/service/sts v1.45.4/go.mod h1:WeBiAa67azG7Su9Vf+ChGDBLiAozJCXzdjXiPBUwtbc=
+github.com/aws/aws-sdk-go-v2/service/sso v1.35.1 h1:B6WFn91tobD6gG4724ONHaqrpKsoETGnv98LHe/yIGM=
+github.com/aws/aws-sdk-go-v2/service/sso v1.35.1/go.mod h1:tWuiVBUtPBr8/rgRiYS8Uf85sHcAN+G7XS3D3CEoUh8=
+github.com/aws/aws-sdk-go-v2/service/ssooidc v1.40.1 h1:6yeYCWFvgbI2TI3K6jr9LtBNhXgJ7g4xqD+DEiaDDmM=
+github.com/aws/aws-sdk-go-v2/service/ssooidc v1.40.1/go.mod h1:naFe83jSMuYkH+QjQPX8n1MLhBkeCFM5Lsnh5m5wz3c=
+github.com/aws/aws-sdk-go-v2/service/sts v1.47.1 h1:Sv2xPnRHlThSUtVujYuUBPI/Il8si6UPHXL8DMiB/F0=
+github.com/aws/aws-sdk-go-v2/service/sts v1.47.1/go.mod h1:mKo/CzaCz8qytGW70NG4vIIGAx1HXTlb5lHNkC5k3lk=
 github.com/aws/smithy-go v1.28.1 h1:R/nXH00c8qcfCzQVELtRw+eLQWtzv+VAIEFJ1/xxXlQ=
 github.com/aws/smithy-go v1.28.1/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
 github.com/bahlo/generic-list-go v0.2.0 h1:5sz/EEAK+ls5wF+NeqDpk5+iNdMDXrh3z3nPnH1Wvgk=
@@ -762,6 +765,8 @@ github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
 github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
 github.com/biogo/store v0.0.0-20201120204734-aad293a2328f h1:+6okTAeUsUrdQr/qN7fIODzowrjjCrnJDg/gkYqcSXY=
 github.com/biogo/store v0.0.0-20201120204734-aad293a2328f/go.mod h1:z52shMwD6SGwRg2iYFjjDwX5Ene4ENTw6HfXraUy/08=
+github.com/bits-and-blooms/bitset v1.24.4 h1:95H15Og1clikBrKr/DuzMXkQzECs1M6hhoGXLwLQOZE=
+github.com/bits-and-blooms/bitset v1.24.4/go.mod h1:7hO7Gc7Pp1vODcmWvKMRA9BNmbv6a/7QIWpPxHddWR8=
 github.com/blevesearch/snowballstem v0.9.0 h1:lMQ189YspGP6sXvZQ4WZ+MLawfV8wOmPoD/iWeNXm8s=
 github.com/blevesearch/snowballstem v0.9.0/go.mod h1:PivSj3JMc8WuaFkTSRDW2SlrulNWPl4ABg1tC/hlgLs=
 github.com/boltdb/bolt v1.3.1 h1:JQmyP4ZBrce+ZQu0dY660FMfatumYDLun9hBCUVIkF4=
@@ -860,17 +865,17 @@ github.com/cognusion/imaging v1.0.4 h1:hrmWVa4S9fiLJJFKEHYYRNBUnHEKZD8/oxnC77xTU
 github.com/cognusion/imaging v1.0.4/go.mod h1:X1mPSSZpMyiva54chCO/1NROp+b6xG4uMcud8VLii+U=
 github.com/colinmarc/hdfs/v2 v2.4.0 h1:v6R8oBx/Wu9fHpdPoJJjpGSUxo8NhHIwrwsfhFvU9W0=
 github.com/colinmarc/hdfs/v2 v2.4.0/go.mod h1:0NAO+/3knbMx6+5pCv+Hcbaz4xn/Zzbn9+WIib2rKVI=
-github.com/compose-spec/compose-go/v2 v2.10.2 h1:USa1NUbDcl/cjb8T9iwnuFsnO79H+2ho2L5SjFKz3uI=
-github.com/compose-spec/compose-go/v2 v2.10.2/go.mod h1:ZU6zlcweCZKyiB7BVfCizQT9XmkEIMFE+PRZydVcsZg=
+github.com/compose-spec/compose-go/v2 v2.12.1 h1:+xBZNxcgSus4atQJwXPEdhHRgCEyZmj/BuqN5m33Ou0=
+github.com/compose-spec/compose-go/v2 v2.12.1/go.mod h1:ZU6zlcweCZKyiB7BVfCizQT9XmkEIMFE+PRZydVcsZg=
 github.com/containerd/console v1.0.3/go.mod h1:7LqA/THxQ86k76b8c/EMSiaJ3h1eZkMkXar0TQ1gf3U=
 github.com/containerd/console v1.0.5 h1:R0ymNeydRqH2DmakFNdmjR2k0t7UPuiOV/N/27/qqsc=
 github.com/containerd/console v1.0.5/go.mod h1:YynlIjWYF8myEu6sdkwKIvGQq+cOckRm6So2avqoYAk=
-github.com/containerd/containerd/api v1.10.0 h1:5n0oHYVBwN4VhoX9fFykCV9dF1/BvAXeg2F8W6UYq1o=
-github.com/containerd/containerd/api v1.10.0/go.mod h1:NBm1OAk8ZL+LG8R0ceObGxT5hbUYj7CzTmR3xh0DlMM=
-github.com/containerd/containerd/v2 v2.2.2 h1:mjVQdtfryzT7lOqs5EYUFZm8ioPVjOpkSoG1GJPxEMY=
-github.com/containerd/containerd/v2 v2.2.2/go.mod h1:5Jhevmv6/2J+Iu/A2xXAdUIdI5Ah/hfyO7okJ4AFIdY=
-github.com/containerd/continuity v0.4.5 h1:ZRoN1sXq9u7V6QoHMcVWGhOwDFqZ4B9i5H6un1Wh0x4=
-github.com/containerd/continuity v0.4.5/go.mod h1:/lNJvtJKUQStBzpVQ1+rasXO1LAWtUQssk28EZvJ3nE=
+github.com/containerd/containerd/api v1.11.1 h1:h8nfoDW9+fNsC/9TwiAHj8B1GzXKtR4eFtkhi/X5RLU=
+github.com/containerd/containerd/api v1.11.1/go.mod h1:CaQFRu+N1MtbgL6JDOJLUB1hCKESU1lD6MuTJhgtdlw=
+github.com/containerd/containerd/v2 v2.2.5 h1:KTFzB02LviYmmfRmz8r9UFd+n6YlddVFK+5lbgQXUTU=
+github.com/containerd/containerd/v2 v2.2.5/go.mod h1:5t2+xFv2dGd/iDYp9Z8DXB4cmWrWQi1XqxGJPS2gBzU=
+github.com/containerd/continuity v0.5.0 h1:7a85HZpCSs+1Zps0Ee3DPSuAWY+0SJM1JNM51nlEVDg=
+github.com/containerd/continuity v0.5.0/go.mod h1:/lNJvtJKUQStBzpVQ1+rasXO1LAWtUQssk28EZvJ3nE=
 github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI=
 github.com/containerd/errdefs v1.0.0/go.mod h1:+YBYIdtsnF4Iw6nWZhJcqGSg/dwvV7tyJ/kCkyJ2k+M=
 github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151Xdx3ZPPE=
@@ -881,8 +886,8 @@ github.com/containerd/platforms v1.0.0-rc.4 h1:M42JrUT4zfZTqtkUwkr0GzmUWbfyO5VO0
 github.com/containerd/platforms v1.0.0-rc.4/go.mod h1:lKlMXyLybmBedS/JJm11uDofzI8L2v0J2ZbYvNsbq1A=
 github.com/containerd/ttrpc v1.2.8 h1:xbVu6D4qF2jihdh9rDVOKqUMiFBQk6YctTdo1zk087Y=
 github.com/containerd/ttrpc v1.2.8/go.mod h1:wyZW2K79t4Hfcxl+GUvkZqRBzJlqFFvgEeeWXa42tyE=
-github.com/containerd/typeurl/v2 v2.2.3 h1:yNA/94zxWdvYACdYO8zofhrTVuQY73fFU1y++dYSw40=
-github.com/containerd/typeurl/v2 v2.2.3/go.mod h1:95ljDnPfD3bAbDJRugOiShd/DlAAsxGtUBhJxIn7SCk=
+github.com/containerd/typeurl/v2 v2.3.0 h1:HZHPhRWo5XMy3QGQoPrUzbW/2ckwjfweHmOwlkIrPAQ=
+github.com/containerd/typeurl/v2 v2.3.0/go.mod h1:Qk+PAdUYArVj41TnGi6rJ+48RF0PkcTc4i/taoBcK0w=
 github.com/coreos/go-semver v0.3.1 h1:yi21YpKnrx1gt5R+la8n5WgS0kCrsPp33dmEyHReZr4=
 github.com/coreos/go-semver v0.3.1/go.mod h1:irMmmIw/7yzSRPWryHsK7EYSg09caPQL03VsM8rvUec=
 github.com/coreos/go-systemd/v22 v22.7.0 h1:LAEzFkke61DFROc7zNLX/WA2i5J8gYqe0rSj9KI28KA=
@@ -926,18 +931,18 @@ github.com/djherbis/times v1.6.0 h1:w2ctJ92J8fBvWPxugmXIv7Nz7Q3iDMKNx9v5ocVH20c=
 github.com/djherbis/times v1.6.0/go.mod h1:gOHeRAz2h+VJNZ5Gmc/o7iD9k4wW7NMVqieYCY99oc0=
 github.com/dnaeon/go-vcr v1.2.0 h1:zHCHvJYTMh1N7xnV7zf1m1GPBF9Ad0Jk/whtQ1663qI=
 github.com/dnaeon/go-vcr v1.2.0/go.mod h1:R4UdLID7HZT3taECzJs4YgbbH6PIGXB6W/sc5OLb6RQ=
-github.com/docker/buildx v0.33.0 h1:xuZeuQe/C/2tvLDgiIA6+Ynq3FFWSfsGNWIHM3q1hD8=
-github.com/docker/buildx v0.33.0/go.mod h1:7JVma62htERKE5iy5YD1q64PKiAHUzXuhSBd4oq3I74=
-github.com/docker/cli v29.4.0+incompatible h1:+IjXULMetlvWJiuSI0Nbor36lcJ5BTcVpUmB21KBoVM=
-github.com/docker/cli v29.4.0+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
-github.com/docker/compose/v5 v5.1.2 h1:HxtfGZA0DESw/+hvrNJDjM8VKmCond7OkmgVJCHku48=
-github.com/docker/compose/v5 v5.1.2/go.mod h1:JJ2H+lRSugOH50/zxCbkBwN8jU91qDtRCp7gEHWZdgo=
+github.com/docker/buildx v0.35.0 h1:5r/ZcAC0s2oZ1XOJHvYFYbj44I4rxRsIdfsRBeXClks=
+github.com/docker/buildx v0.35.0/go.mod h1:kO/9ZBoZmhi1OsZbeLJqWclckB2TKu/yeW6aj6xxdt0=
+github.com/docker/cli v29.6.0+incompatible h1:nw9himxMMZ7eIeherJNlKQq+acnlzGgHd+4uf10QRSc=
+github.com/docker/cli v29.6.0+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
+github.com/docker/compose/v5 v5.2.0 h1:WuZgtAM91BjFWy7xMKcM2Huq9OksVqRgkgBKFiEXs2Q=
+github.com/docker/compose/v5 v5.2.0/go.mod h1:tjYsBMooQQZ8XGtVoUnc+/T9peXtkMmTdmMakW8/ZXE=
 github.com/docker/docker v28.5.2+incompatible h1:DBX0Y0zAjZbSrm1uzOkdr1onVghKaftjlSWt4AFexzM=
 github.com/docker/docker v28.5.2+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
-github.com/docker/docker-credential-helpers v0.9.5 h1:EFNN8DHvaiK8zVqFA2DT6BjXE0GzfLOZ38ggPTKePkY=
-github.com/docker/docker-credential-helpers v0.9.5/go.mod h1:v1S+hepowrQXITkEfw6o4+BMbGot02wiKpzWhGUZK6c=
-github.com/docker/go-connections v0.6.0 h1:LlMG9azAe1TqfR7sO+NJttz1gy6KO7VJBh+pMmjSD94=
-github.com/docker/go-connections v0.6.0/go.mod h1:AahvXYshr6JgfUJGdDCs2b5EZG/vmaMAntpSFH5BFKE=
+github.com/docker/docker-credential-helpers v0.9.8 h1:bIREROb7So6PRlq6KTtdS9MPEjC29OQRkFNlvK2OX8Q=
+github.com/docker/docker-credential-helpers v0.9.8/go.mod h1:v1S+hepowrQXITkEfw6o4+BMbGot02wiKpzWhGUZK6c=
+github.com/docker/go-connections v0.7.0 h1:6SsRfJddP22WMrCkj19x9WKjEDTB+ahsdiGYf0mN39c=
+github.com/docker/go-connections v0.7.0/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q=
 github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
 github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
 github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE=
@@ -1029,17 +1034,19 @@ github.com/fvbommel/sortorder v1.1.0 h1:fUmoe+HLsBTctBDoaBwpQo5N+nrCp8g/BjKb/6ZQ
 github.com/fvbommel/sortorder v1.1.0/go.mod h1:uk88iVf1ovNn1iLfgUVU2F9o5eO30ui720w+kxuqRs0=
 github.com/gabriel-vasile/mimetype v1.4.13 h1:46nXokslUBsAJE/wMsp5gtO500a4F3Nkz9Ufpk2AcUM=
 github.com/gabriel-vasile/mimetype v1.4.13/go.mod h1:d+9Oxyo1wTzWdyVUPMmXFvp4F9tea18J8ufA774AB3s=
+github.com/geoarrow/geoarrow-go v0.0.0-20260403143023-f54751c3e3a1 h1:8VcjuP5pKZ717K4zOLS9Lm4Zsn8JwD9X+CQGFCiRT2E=
+github.com/geoarrow/geoarrow-go v0.0.0-20260403143023-f54751c3e3a1/go.mod h1:XMOIOA5J96jEzAOpBROPyz0n14sHsooeF+y2FtGdrCY=
 github.com/geoffgarside/ber v1.2.0 h1:/loowoRcs/MWLYmGX9QtIAbA+V/FrnVLsMMPhwiRm64=
 github.com/geoffgarside/ber v1.2.0/go.mod h1:jVPKeCbj6MvQZhwLYsGwaGI52oUorHoHKNecGT85ZCc=
-github.com/getsentry/sentry-go v0.48.0 h1:FRZNr7Uk1C86ev1bSJmYlUkL9oyivQA6YOcdYfaaMmY=
-github.com/getsentry/sentry-go v0.48.0/go.mod h1:E5UkA5wp1qR2+MDydNYlVeUiNN2xEdjYMidkgf0Qoss=
+github.com/getsentry/sentry-go v0.49.0 h1:Ehejknu1l023Ub7QoRBVLAI7g3Jnhqku4oWx4B4Sh5s=
+github.com/getsentry/sentry-go v0.49.0/go.mod h1:nuMJAoCfe1u0Bts2ocyNI+TW8HT84vRMqwA5Qq/SKUI=
 github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
 github.com/gin-contrib/sse v1.1.0 h1:n0w2GMuUpWDVp7qSpvze6fAu9iRxJY4Hmj6AmBOU05w=
 github.com/gin-contrib/sse v1.1.0/go.mod h1:hxRZ5gVpWMT7Z0B0gSNYqqsSCNIJMjzvm6fqCz9vjwM=
 github.com/gin-gonic/gin v1.11.0 h1:OW/6PLjyusp2PPXtyxKHU0RbX6I/l28FTdDlae5ueWk=
 github.com/gin-gonic/gin v1.11.0/go.mod h1:+iq/FyxlGzII0KHiBGjuNn4UNENUlKbGlNmc+W50Dls=
-github.com/go-asn1-ber/asn1-ber v1.5.8-0.20250403174932-29230038a667 h1:BP4M0CvQ4S3TGls2FvczZtj5Re/2ZzkV9VwqPHH/3Bo=
-github.com/go-asn1-ber/asn1-ber v1.5.8-0.20250403174932-29230038a667/go.mod h1:hEBeB/ic+5LoWskz+yKT7vGhhPYkProFKoKdwZRWMe0=
+github.com/go-asn1-ber/asn1-ber v1.5.8 h1:H9AZkK22UOmfX8J84ubyaZxKJZ3FMHVwn8swoMML7iQ=
+github.com/go-asn1-ber/asn1-ber v1.5.8/go.mod h1:hEBeB/ic+5LoWskz+yKT7vGhhPYkProFKoKdwZRWMe0=
 github.com/go-chi/chi/v5 v5.3.1 h1:3j4HZLGZQ3JpMCrPJF/Jl3mYJfWLKBfNJ6quurUGCf8=
 github.com/go-chi/chi/v5 v5.3.1/go.mod h1:R+tYY2hNuVUUjxoPtqUdgBqevM9s9njzkTLutVsOCto=
 github.com/go-darwin/apfs v0.0.0-20211011131704-f84b94dbf348 h1:JnrjqG5iR07/8k7NqrLNilRsl3s1EPRQEGvbPyOce68=
@@ -1063,14 +1070,14 @@ github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2
 github.com/go-kit/log v0.1.0/go.mod h1:zbhenjAZHb184qTLMA9ZjW7ThYL0H2mk7Q6pNt4vbaY=
 github.com/go-latex/latex v0.0.0-20210118124228-b3d85cf34e07/go.mod h1:CO1AlKB2CSIqUrmQPqA0gdRIlnLEY0gK5JGjh37zN5U=
 github.com/go-latex/latex v0.0.0-20210823091927-c0d11ff05a81/go.mod h1:SX0U8uGpxhq9o2S/CELCSUxEWWAuoCUcVCQWv7G2OCk=
-github.com/go-ldap/ldap/v3 v3.4.13 h1:+x1nG9h+MZN7h/lUi5Q3UZ0fJ1GyDQYbPvbuH38baDQ=
-github.com/go-ldap/ldap/v3 v3.4.13/go.mod h1:LxsGZV6vbaK0sIvYfsv47rfh4ca0JXokCoKjZxsszv0=
+github.com/go-ldap/ldap/v3 v3.4.14 h1:D6PYdEgsaVzsXyr6w/yDC06Ria4uUhWm+Rb+er8lfAs=
+github.com/go-ldap/ldap/v3 v3.4.14/go.mod h1:S4eJUMUNjDkE0ZJtIZdybwyb03sGGLW6gxXT1Hs8VKA=
 github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=
 github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
 github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A=
 github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
-github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
-github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
+github.com/go-logr/logr v1.4.4 h1:tG4xh9yMsRCAiodLVTxyrkzSZ9+o0L1Kg/+cPVcbP/8=
+github.com/go-logr/logr v1.4.4/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
 github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
 github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
 github.com/go-ole/go-ole v1.2.6/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0=
@@ -1360,8 +1367,8 @@ github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpO
 github.com/iancoleman/strcase v0.2.0/go.mod h1:iwCmte+B7n89clKwxIoIXy/HfoL7AsD47ZCWhYzw7ho=
 github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
 github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
-github.com/in-toto/attestation v1.1.2 h1:MBFn6lsMq6dptQZJBhalXTcWMb/aJy3V+GX3VYj/V1E=
-github.com/in-toto/attestation v1.1.2/go.mod h1:gYFddHMZj3DiQ0b62ltNi1Vj5rC879bTmBbrv9CRHpM=
+github.com/in-toto/attestation v1.2.0 h1:aPRUZ3azbqD7yEBD5fP3TD8Dszf+YHo284SOcpahjQk=
+github.com/in-toto/attestation v1.2.0/go.mod h1:r79G45gOmzPismgObLSL+rZTFxUgZLOQJI6LofTZgXk=
 github.com/in-toto/in-toto-golang v0.11.0 h1:nfidMYBFx+E0lnmX5KUnN2Pdm8zdNKal1ayjJuzzRoA=
 github.com/in-toto/in-toto-golang v0.11.0/go.mod h1:u3PjTnwFKjp5a1YCcw8SJg0G+tMeKfVoWsWeFMDCMtw=
 github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
@@ -1446,15 +1453,15 @@ github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+o
 github.com/kisielk/sqlstruct v0.0.0-20201105191214-5f3e10d3ab46/go.mod h1:yyMNCyc/Ib3bDTKd379tNMpB/7/H5TjM2Y9QJ5THLbE=
 github.com/klauspost/asmfmt v1.3.2/go.mod h1:AG8TuvYojzulgDAMCnYn50l/5QV3Bs/tp6j0HLHbNSE=
 github.com/klauspost/compress v1.15.9/go.mod h1:PhcZ0MbTNciWF3rruxRgKxI5NkcHHrHUDtV4Yw2GlzU=
-github.com/klauspost/compress v1.19.1 h1:VsB4HPswih7mmZ8WleSFQ75c/Ui1M4trX5oAsJnhSlk=
-github.com/klauspost/compress v1.19.1/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
+github.com/klauspost/compress v1.19.2 h1:hMRETovs/pu/dVWN7zIT1PGG8t509MwT6bO7XSi26R8=
+github.com/klauspost/compress v1.19.2/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
 github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
 github.com/klauspost/cpuid/v2 v2.0.10/go.mod h1:g2LTdtYhdyuGPqyWyv7qRAmj1WBqxuObKfj5c0PQa7c=
 github.com/klauspost/cpuid/v2 v2.0.12/go.mod h1:g2LTdtYhdyuGPqyWyv7qRAmj1WBqxuObKfj5c0PQa7c=
 github.com/klauspost/cpuid/v2 v2.4.0 h1:S6Hrbc7+ywsr0r+RLapfGBHfyefhCTwEh3A0tV913Dw=
 github.com/klauspost/cpuid/v2 v2.4.0/go.mod h1:19jmZ9mjzoF//ddRSUsv0zfBTJWh3QJh9FNxZTMrGxU=
-github.com/klauspost/reedsolomon v1.14.1 h1:swE9kzyWXD/wVG+l5Pe8bWnQ0giIY7D1GjCBKk3kG2U=
-github.com/klauspost/reedsolomon v1.14.1/go.mod h1:yjqqjgMTQkBUHSG97/rm4zipffCNbCiZcB3kTqr++sQ=
+github.com/klauspost/reedsolomon v1.14.2 h1:SafJYwpBBQBI6amHUygcjxZjXeN2HpiENHQDwuPWCCQ=
+github.com/klauspost/reedsolomon v1.14.2/go.mod h1:yjqqjgMTQkBUHSG97/rm4zipffCNbCiZcB3kTqr++sQ=
 github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
 github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
 github.com/koofr/go-httpclient v0.0.0-20240520111329-e20f8f203988 h1:CjEMN21Xkr9+zwPmZPaJJw+apzVbjGL5uK/6g9Q2jGU=
@@ -1517,8 +1524,8 @@ github.com/mattn/go-runewidth v0.0.3/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzp
 github.com/mattn/go-runewidth v0.0.13/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
 github.com/mattn/go-runewidth v0.0.24 h1:cpokDiIn0MGnhdHwuWnJBITySJ20QyNGnY2kR/ay2DU=
 github.com/mattn/go-runewidth v0.0.24/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
-github.com/mattn/go-shellwords v1.0.12 h1:M2zGm7EW6UQJvDeQxo4T51eKPurbeFbe8WtebGE2xrk=
-github.com/mattn/go-shellwords v1.0.12/go.mod h1:EZzvwXDESEeg03EKmM+RmDnNOPKG4lLtQsUlTZDWQ8Y=
+github.com/mattn/go-shellwords v1.0.13 h1:DC0OMEpGjm6LfNFU4ckYcvbQKyp2vE8atyFGXNtDcf4=
+github.com/mattn/go-shellwords v1.0.13/go.mod h1:EZzvwXDESEeg03EKmM+RmDnNOPKG4lLtQsUlTZDWQ8Y=
 github.com/mattn/go-sqlite3 v1.14.14/go.mod h1:NyWgC/yNuGj7Q9rpYnZvas74GogHl5/Z4A/KQRfk6bU=
 github.com/mattn/go-sqlite3 v1.14.34 h1:3NtcvcUnFBPsuRcno8pUtupspG/GM+9nZ88zgJcp6Zk=
 github.com/mattn/go-sqlite3 v1.14.34/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
@@ -1539,18 +1546,18 @@ github.com/mitchellh/mapstructure v1.5.1-0.20220423185008-bf980b35cac4 h1:BpfhmL
 github.com/mitchellh/mapstructure v1.5.1-0.20220423185008-bf980b35cac4/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
 github.com/mmcloughlin/geohash v0.9.0 h1:FihR004p/aE1Sju6gcVq5OLDqGcMnpBY+8moBqIsVOs=
 github.com/mmcloughlin/geohash v0.9.0/go.mod h1:oNZxQo5yWJh0eMQEP/8hwQuVx9Z9tjwFUqcTB1SmG0c=
-github.com/moby/buildkit v0.29.0 h1:wxLEFbCOJntEDjSNNN2YWd8zxltZxT5muDQ0LzpbtpU=
-github.com/moby/buildkit v0.29.0/go.mod h1:Dmv2FeDe34t75QuzeU87rBoZpAAkcpT5zeu4hXzmASc=
+github.com/moby/buildkit v0.31.0 h1:hMUAbQGgjtzJDDOZ6o7MQk5XBZkBTyzLWEvnjguHHQI=
+github.com/moby/buildkit v0.31.0/go.mod h1:YM5iNEbNCc6L1Zt3YWFB/aXNLufvf4Rcu0DPlc9HwQg=
 github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0=
 github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
 github.com/moby/go-archive v0.3.0 h1:nos4BtzzUIqB406BgQnWGMI4qib9BZ8XUHU+ucv/n1c=
 github.com/moby/go-archive v0.3.0/go.mod h1:Npdv43fFqlhZW7Xo8fbm3ZMYFvAGNviUPqX21VERbcE=
 github.com/moby/locker v1.0.1 h1:fOXqR41zeveg4fFODix+1Ch4mj/gT0NE1XJbp/epuBg=
 github.com/moby/locker v1.0.1/go.mod h1:S7SDdo5zpBK84bzzVlKr2V0hz+7x9hWbYC/kq7oQppc=
-github.com/moby/moby/api v1.54.2 h1:wiat9QAhnDQjA7wk1kh/TqHz2I1uUA7M7t9SAl/JNXg=
-github.com/moby/moby/api v1.54.2/go.mod h1:+RQ6wluLwtYaTd1WnPLykIDPekkuyD/ROWQClE83pzs=
-github.com/moby/moby/client v0.4.0 h1:S+2XegzHQrrvTCvF6s5HFzcrywWQmuVnhOXe2kiWjIw=
-github.com/moby/moby/client v0.4.0/go.mod h1:QWPbvWchQbxBNdaLSpoKpCdf5E+WxFAgNHogCWDoa7g=
+github.com/moby/moby/api v1.55.0 h1:2/sexvQyqIWS8pRSCFddBfpW2qE7vR7FCL+vN8pxwMc=
+github.com/moby/moby/api v1.55.0/go.mod h1:+RQ6wluLwtYaTd1WnPLykIDPekkuyD/ROWQClE83pzs=
+github.com/moby/moby/client v0.5.0 h1:5XhyPk2fuOWf6RlSFa3MkIIgDZkF25xToXW8Q/BH7cc=
+github.com/moby/moby/client v0.5.0/go.mod h1:rcVpF8ncl9vo5gaIBdol6CnbEtSj1uxMvEV/UrykF/s=
 github.com/moby/patternmatcher v0.6.1 h1:qlhtafmr6kgMIJjKJMDmMWq7WLkKIo23hsrpR3x084U=
 github.com/moby/patternmatcher v0.6.1/go.mod h1:hDPoyOpDY7OrrMDLaYoY3hf52gNCR/YOUYxkhApJIxc=
 github.com/moby/sys/atomicwriter v0.1.0 h1:kw5D/EqkBwsBFi0ss9v1VG3wIkVhzGvLklJ+w3A14Sw=
@@ -1582,6 +1589,8 @@ github.com/montanaflynn/stats v0.7.1 h1:etflOAAHORrCC44V+aR6Ftzort912ZU+YLiSTuV8
 github.com/montanaflynn/stats v0.7.1/go.mod h1:etXPPgVO6n31NxCd9KQUMvCM+ve0ruNzt6R8Bnaayow=
 github.com/morikuni/aec v1.1.0 h1:vBBl0pUnvi/Je71dsRrhMBtreIqNMYErSAbEeb8jrXQ=
 github.com/morikuni/aec v1.1.0/go.mod h1:xDRgiq/iw5l+zkao76YTKzKttOp2cwPEne25HDkJnBw=
+github.com/mschoch/smat v0.2.0 h1:8imxQsjDm8yFEAVBe7azKmKSgzSkZXDuKkSq9374khM=
+github.com/mschoch/smat v0.2.0/go.mod h1:kc9mz7DoBKqDyiRL7VZN8KvXQMWeTaVnttLRXOlotKw=
 github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
 github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
 github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
@@ -1643,8 +1652,8 @@ github.com/pascaldekloe/goe v0.1.0/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144T
 github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc=
 github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ=
 github.com/pborman/getopt v0.0.0-20170112200414-7148bc3a4c30/go.mod h1:85jBQOZwpVEaDAr341tbn15RS4fCAsIst0qp7i8ex1o=
-github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
-github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
+github.com/pelletier/go-toml/v2 v2.4.1 h1:j5OMOImsH+j2k7GJ5YO+RxfWwohNiH6t5zB/+h3bagc=
+github.com/pelletier/go-toml/v2 v2.4.1/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
 github.com/pengsrc/go-shared v0.2.1-0.20190131101655-1999055a4a14 h1:XeOYlK9W1uCmhjJSsY78Mcuh7MVkNjTzmHx1yBzizSU=
 github.com/pengsrc/go-shared v0.2.1-0.20190131101655-1999055a4a14/go.mod h1:jVblp62SafmidSkvWrXyxAme3gaTfEtWwRPGz5cpvHg=
 github.com/peterh/liner v1.2.2 h1:aJ4AOodmL+JxOZZEL2u9iJf8omNRpqHc/EbrK+3mAXw=
@@ -1659,8 +1668,8 @@ github.com/phpdave11/gofpdf v1.4.2/go.mod h1:zpO6xFn9yxo3YLyMvW8HcKWVdbNqgIfOOp2
 github.com/phpdave11/gofpdi v1.0.12/go.mod h1:vBmVV0Do6hSBHC8uKUQ71JGW+ZGQq74llk/7bXwjDoI=
 github.com/phpdave11/gofpdi v1.0.13/go.mod h1:vBmVV0Do6hSBHC8uKUQ71JGW+ZGQq74llk/7bXwjDoI=
 github.com/pierrec/lz4/v4 v4.1.15/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
-github.com/pierrec/lz4/v4 v4.1.28 h1:pPEPwRJ4kybBTfGt28q7lQsRJQHhC08axprdLD5Ppio=
-github.com/pierrec/lz4/v4 v4.1.28/go.mod h1:EoQMVJgeeEOMsCqCzqFm2O0cJvljX2nGZjcRIPL34O4=
+github.com/pierrec/lz4/v4 v4.1.29 h1:CDQY6qZOLI4DW0Nx6R1vRrifrCeQHnNXkMb0hZWXFjg=
+github.com/pierrec/lz4/v4 v4.1.29/go.mod h1:EoQMVJgeeEOMsCqCzqFm2O0cJvljX2nGZjcRIPL34O4=
 github.com/pierrre/compare v1.0.2 h1:k4IUsHgh+dbcAOIWCfxVa/7G6STjADH2qmhomv+1quc=
 github.com/pierrre/compare v1.0.2/go.mod h1:8UvyRHH+9HS8Pczdd2z5x/wvv67krDwVxoOndaIIDVU=
 github.com/pierrre/geohash v1.0.0 h1:f/zfjdV4rVofTCz1FhP07T+EMQAvcMM2ioGZVt+zqjI=
@@ -1717,8 +1726,8 @@ github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:
 github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
 github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
 github.com/prometheus/client_model v0.3.0/go.mod h1:LDGWKZIo7rky3hgvBe+caln+Dr3dPggB5dvjtD7w9+w=
-github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
-github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=
+github.com/prometheus/client_model v0.6.3 h1:O0jaTVAYNxTHYInEPFJt5I3+sN8zqBtVMPTB1qyxiEo=
+github.com/prometheus/client_model v0.6.3/go.mod h1:gpN5P9S7Rr6Yr92PiQ+Ixvhf6JZEkF1dnxsYL2aPBEM=
 github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
 github.com/prometheus/common v0.9.1/go.mod h1:yhUN8i9wzaXS3w1O07YhxHEBxD+W35wd8bs7vj7HSQ4=
 github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo=
@@ -1730,8 +1739,8 @@ github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsT
 github.com/prometheus/procfs v0.0.8/go.mod h1:7Qr8sr6344vo1JqZ6HhLceV9o3AJ1Ff+GxbHq6oeK9A=
 github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU=
 github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA=
-github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
-github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
+github.com/prometheus/procfs v0.22.0 h1:6q9+/JL9IKAPbCmBrv9n5O5Ty3NKnciV5X7YGw0oics=
+github.com/prometheus/procfs v0.22.0/go.mod h1:CvmFr/GVhIjIvWJZW3tgkODBQMRIf0EyWMQLHCHab58=
 github.com/pterm/pterm v0.12.27/go.mod h1:PhQ89w4i95rhgE+xedAoqous6K9X+r6aSOI2eFF7DZI=
 github.com/pterm/pterm v0.12.29/go.mod h1:WI3qxgvoQFFGKGjGnJR849gU0TsEOvKn5Q8LlY1U7lg=
 github.com/pterm/pterm v0.12.30/go.mod h1:MOqLIyMOgmTDz9yorcYbcw+HsgoZo3BQfg2wtl3HEFE=
@@ -1800,8 +1809,8 @@ github.com/sagikazarmark/locafero v0.11.0 h1:1iurJgmM9G3PA/I+wWYIOw/5SyBtxapeHDc
 github.com/sagikazarmark/locafero v0.11.0/go.mod h1:nVIGvgyzw595SUSUE6tvCp3YYTeHs15MvlmU87WwIik=
 github.com/samber/lo v1.53.0 h1:t975lj2py4kJPQ6haz1QMgtId2gtmfktACxIXArw3HM=
 github.com/samber/lo v1.53.0/go.mod h1:4+MXEGsJzbKGaUEQFKBq2xtfuznW9oz/WrgyzMzRoM0=
-github.com/santhosh-tekuri/jsonschema/v6 v6.0.1 h1:PKK9DyHxif4LZo+uQSgXNqs0jj5+xZwwfKHgph2lxBw=
-github.com/santhosh-tekuri/jsonschema/v6 v6.0.1/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=
+github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 h1:KRzFb2m7YtdldCEkzs6KqmJw4nqEVZGK7IN2kJkjTuQ=
+github.com/santhosh-tekuri/jsonschema/v6 v6.0.2/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=
 github.com/sasha-s/go-deadlock v0.3.1 h1:sqv7fDNShgjcaxkO0JNcOAlr8B9+cV5Ey/OB71efZx0=
 github.com/sasha-s/go-deadlock v0.3.1/go.mod h1:F73l+cr82YSh10GxyRI6qZiCgK64VaZjwesgfQ1/iLM=
 github.com/schollz/progressbar/v3 v3.19.1 h1:iv8BgwOvdML/S3p84uBpy/IMigv4U9594vPZYa2EdrU=
@@ -1814,8 +1823,8 @@ github.com/seaweedfs/goexif v2.0.0+incompatible h1:x8pckiT12QQhifwhDQpeISgDfsqmQ
 github.com/seaweedfs/goexif v2.0.0+incompatible/go.mod h1:Oni780Z236sXpIQzk1XoJlTwqrJ02smEin9zQeff7Fk=
 github.com/seaweedfs/raft v1.2.0 h1:Ez4Hw9ifBbTT7wg54DvGHBjw1vRlTb4roH0TKl0Oj9Y=
 github.com/seaweedfs/raft v1.2.0/go.mod h1:fgs/rAVEzjQ7e04XMzG3eJhwZZRmBW+2uRtjakeCGeU=
-github.com/secure-systems-lab/go-securesystemslib v0.10.0 h1:l+H5ErcW0PAehBNrBxoGv1jjNpGYdZ9RcheFkB2WI14=
-github.com/secure-systems-lab/go-securesystemslib v0.10.0/go.mod h1:MRKONWmRoFzPNQ9USRF9i1mc7MvAVvF1LlW8X5VWDvk=
+github.com/secure-systems-lab/go-securesystemslib v0.11.0 h1:iuCR9kcMFD4QurdKrGvPLoKZLv9YvwPYVr0473BdtFs=
+github.com/secure-systems-lab/go-securesystemslib v0.11.0/go.mod h1:+PMOTjUGwHj2vcZ+TFKlb1tXRbrdWE1LYDT5i9JC80Q=
 github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo=
 github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
 github.com/sergi/go-diff v1.2.0 h1:XU+rvMAioB0UC3q1MFrIQy4Vo5/4VsRDQQXHsEya6xQ=
@@ -1824,10 +1833,10 @@ github.com/shibumi/go-pathspec v1.3.0 h1:QUyMZhFo0Md5B8zV8x2tesohbb5kfbpTi9rBnKh
 github.com/shibumi/go-pathspec v1.3.0/go.mod h1:Xutfslp817l2I1cZvgcfeMQJG5QnU2lh5tVaaMCl3jE=
 github.com/shirou/gopsutil/v4 v4.26.7 h1:IXzpHz/dkMRYAhKkOXr1HB6SuzWU3eoyyeWe7g3bNZc=
 github.com/shirou/gopsutil/v4 v4.26.7/go.mod h1:5O9FjBiXoTDFatIWjZZosqj4pV0DRtLx598xGbBehzM=
-github.com/sigstore/sigstore v1.10.4 h1:ytOmxMgLdcUed3w1SbbZOgcxqwMG61lh1TmZLN+WeZE=
-github.com/sigstore/sigstore v1.10.4/go.mod h1:tDiyrdOref3q6qJxm2G+JHghqfmvifB7hw+EReAfnbI=
-github.com/sigstore/sigstore-go v1.1.4 h1:wTTsgCHOfqiEzVyBYA6mDczGtBkN7cM8mPpjJj5QvMg=
-github.com/sigstore/sigstore-go v1.1.4/go.mod h1:2U/mQOT9cjjxrtIUeKDVhL+sHBKsnWddn8URlswdBsg=
+github.com/sigstore/sigstore v1.10.8 h1:1Mgkxvkw4AXMfIP1DOjc6kw0GkUgA8pGVpveN/EfOq4=
+github.com/sigstore/sigstore v1.10.8/go.mod h1:f9+B/4iaYimvUkySyb2mvc73n3RLqNn24grHZM/ET8M=
+github.com/sigstore/sigstore-go v1.2.1 h1:YWP/rDbBaEBvtbkj6xtwsSj38ZCFEhTVVadNOXjVe3A=
+github.com/sigstore/sigstore-go v1.2.1/go.mod h1:I8BqVwAb/SaQJ5pBu5IDFY+ksq8O/1/kCag8XUgrsko=
 github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
 github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
 github.com/sirupsen/logrus v1.5.0/go.mod h1:+F7Ogzej0PZc/94MaYx/nvG9jOFMD2osvC3s+Squfpo=
@@ -1911,10 +1920,10 @@ github.com/tarantool/go-option v1.1.0 h1:ShoOhNsdL41sRpm4hXCRDjV8H0WzPkd4UnKhLKb
 github.com/tarantool/go-option v1.1.0/go.mod h1:hMr9z2JXOWlgdCBpCPSL2nwp8718GKYvNBJ+ZuzJbCo=
 github.com/tarantool/go-tarantool/v3 v3.0.1 h1:vaUX4xmVmXh2dIJ/LqlX1MXK3iYqAqV6YiE54Wwl/qg=
 github.com/tarantool/go-tarantool/v3 v3.0.1/go.mod h1:TXxLWhUCgdxXFfelnTSkq+goKRTRj660zxq4/WXPe8k=
-github.com/testcontainers/testcontainers-go v0.43.0 h1:oEQx5MW2DGd9z3AeEQfB2lPM0eLs7ztyaGRu75bFo5A=
-github.com/testcontainers/testcontainers-go v0.43.0/go.mod h1:+VxkT2NQnKOZPKi6praMuMKYHYyOGXr0XSBSlSMCzFo=
-github.com/testcontainers/testcontainers-go/modules/compose v0.42.0 h1:+t1ZN31TD36cwxmeLqGwe7wIdvblBm0Z+vlj4SX8Mv0=
-github.com/testcontainers/testcontainers-go/modules/compose v0.42.0/go.mod h1:CfMpouDHqNTCHC8CijEURU2ZotTV3QhH6pXd48s6ofk=
+github.com/testcontainers/testcontainers-go v0.44.0 h1:/Fwh6HY1mIikhnm9e7HwoxGycx0lzRAE0f5VQpjFxzI=
+github.com/testcontainers/testcontainers-go v0.44.0/go.mod h1:IcnwQrYTO86xHXu5bvMaBH7ATlbS3Qn1M1QWW3c66rE=
+github.com/testcontainers/testcontainers-go/modules/compose v0.44.0 h1:8YcW51jhgpkkiRVe10Wj9TCBthJmoNpU2fK5WSf7TQ8=
+github.com/testcontainers/testcontainers-go/modules/compose v0.44.0/go.mod h1:0du5YaH5n8Of4bDM/RwIt421Q14kvqrXZ0mB8EjuMoo=
 github.com/the42/cartconvert v0.0.0-20131203171324-aae784c392b8 h1:I4DY8wLxJXCrMYzDM6lKCGc3IQwJX0PlTLsd3nQqI3c=
 github.com/the42/cartconvert v0.0.0-20131203171324-aae784c392b8/go.mod h1:fWO/msnJVhHqN1yX6OBoxSyfj7TEj1hHiL8bJSQsK30=
 github.com/tiancaiamao/gp v0.0.0-20221230034425-4025bc8a4d4a h1:J/YdBZ46WKpXsxsW93SG+q0F8KI+yFrcIDT4c/RNoc4=
@@ -1943,8 +1952,8 @@ github.com/tmthrgd/go-hex v0.0.0-20190904060850-447a3041c3bc h1:9lRDQMhESg+zvGYm
 github.com/tmthrgd/go-hex v0.0.0-20190904060850-447a3041c3bc/go.mod h1:bciPuU6GHm1iF1pBvUfxfsH0Wmnc2VbpgvbI9ZWuIRs=
 github.com/tonistiigi/dchapes-mode v0.0.0-20250318174251-73d941a28323 h1:r0p7fK56l8WPequOaR3i9LBqfPtEdXIQbUTzT55iqT4=
 github.com/tonistiigi/dchapes-mode v0.0.0-20250318174251-73d941a28323/go.mod h1:3Iuxbr0P7D3zUzBMAZB+ois3h/et0shEz0qApgHYGpY=
-github.com/tonistiigi/fsutil v0.0.0-20251211185533-a2aa163d723f h1:Z4NEQ86qFl1mHuCu9gwcE+EYCwDKfXAYXZbdIXyxmEA=
-github.com/tonistiigi/fsutil v0.0.0-20251211185533-a2aa163d723f/go.mod h1:BKdcez7BiVtBvIcef90ZPc6ebqIWr4JWD7+EvLm6J98=
+github.com/tonistiigi/fsutil v0.0.0-20260609174605-b61e79c0c046 h1:j29MScUISj0S98EHdM6/pzKgqppfswl9OZ7OVpnQdzE=
+github.com/tonistiigi/fsutil v0.0.0-20260609174605-b61e79c0c046/go.mod h1:K5zrLch9UaSGNiek5XHZeqZUf1zPWJHqDfLIcnpquQ4=
 github.com/tonistiigi/go-csvvalue v0.0.0-20240814133006-030d3b2625d0 h1:2f304B10LaZdB8kkVEaoXvAMVan2tl9AiK4G0odjQtE=
 github.com/tonistiigi/go-csvvalue v0.0.0-20240814133006-030d3b2625d0/go.mod h1:278M4p8WsNh3n4a1eqiFcV2FGk7wE5fwUpUom9mK9lE=
 github.com/tonistiigi/units v0.0.0-20180711220420-6950e57a87ea h1:SXhTLE6pb6eld/v/cCndK0AMpt1wiVFb/YYmqB3/QG0=
@@ -1956,8 +1965,8 @@ github.com/tsuna/gohbase v0.0.0-20201125011725-348991136365/go.mod h1:zj0GJHGvyf
 github.com/tv42/httpunix v0.0.0-20150427012821-b75d8614f926/go.mod h1:9ESjWnEqriFuLhtthL60Sar/7RFoluCcXsuvEwTV5KM=
 github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
 github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
-github.com/twmb/avro v1.7.2 h1:cmrEBRSbELRqsg/dRkQvVWuOaR2EfGifHIt/2iJ9lfI=
-github.com/twmb/avro v1.7.2/go.mod h1:X0fT1dY2xcbV4YuCE4mYro+qljHl4kUF5uA/2z1rgSk=
+github.com/twmb/avro v1.8.0 h1:UMWLg+nH4P3yad5Om7yFSohYLy2RG1s7BcFFiOvmK9Q=
+github.com/twmb/avro v1.8.0/go.mod h1:X0fT1dY2xcbV4YuCE4mYro+qljHl4kUF5uA/2z1rgSk=
 github.com/twmb/murmur3 v1.1.8 h1:8Yt9taO/WN3l08xErzjeschgZU2QSrwm1kclYq+0aRg=
 github.com/twmb/murmur3 v1.1.8/go.mod h1:Qq/R7NUyOfr65zD+6Q5IHKsJLwP7exErjN6lyyq3OSQ=
 github.com/twpayne/go-geom v1.6.1 h1:iLE+Opv0Ihm/ABIcvQFGIiFBXd76oBIar9drAwHFhR4=
@@ -2096,38 +2105,38 @@ go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ
 go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
 go.opentelemetry.io/contrib/detectors/gcp v1.44.0 h1:NmLfL734pJhM0JKaYd2Y28+nY9dPRWYAAbxhRCrKXPw=
 go.opentelemetry.io/contrib/detectors/gcp v1.44.0/go.mod h1:tNAsgd8avTGke1+MndXlU5Cru4PQ9Ai/cCNWQv/ZJ/s=
-go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.68.0 h1:0Qx7VGBacMm9ZENQ7TnNObTYI4ShC+lHI16seduaxZo=
-go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.68.0/go.mod h1:Sje3i3MjSPKTSPvVWCaL8ugBzJwik3u4smCjUeuupqg=
-go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.63.0 h1:2pn7OzMewmYRiNtv1doZnLo3gONcnMHlFnmOR8Vgt+8=
-go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.63.0/go.mod h1:rjbQTDEPQymPE0YnRQp9/NuPwwtL0sesz/fnqRW/v84=
+go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.69.0 h1:2yEATaop1/a1I4psnSLgWVPLWwCzkqWakgJy7xTDVy0=
+go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.69.0/go.mod h1:D7J12YRapIekYyPWgGPlA/23pRmpSEZC5xJC/TTLI9U=
+go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.69.0 h1:MCcYL7J6Vt/X0kjqbMZkekCmwsurbQRbL69vkiye2lk=
+go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.69.0/go.mod h1:3jnStNwSufK+f5ktjL4EPcwtig4rtd81NS70lqHuXl8=
 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.69.0 h1:8tvICD4vSTOOsNrsI4Ljf6C+6UKvpTEH5XY3JMoyPoo=
 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.69.0/go.mod h1:z9+yiacE0IHRqM4qFfkbt/JYlmYXgss8GY/jXoNuPJI=
-go.opentelemetry.io/otel v1.44.0 h1:JjwHmHpA4iZ3wBxluu2fbbE7j4kqlE8jXyAyPXH7HqU=
-go.opentelemetry.io/otel v1.44.0/go.mod h1:BMgjTHL9WPRlRjL2oZCBTL4whCGtXch2H4BhOPIAyYc=
-go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.42.0 h1:MdKucPl/HbzckWWEisiNqMPhRrAOQX8r4jTuGr636gk=
-go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.42.0/go.mod h1:RolT8tWtfHcjajEH5wFIZ4Dgh5jpPdFXYV9pTAk/qjc=
-go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.43.0 h1:w1K+pCJoPpQifuVpsKamUdn9U0zM3xUziVOqsGksUrY=
-go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.43.0/go.mod h1:HBy4BjzgVE8139ieRI75oXm3EcDN+6GhD88JT1Kjvxg=
-go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0 h1:88Y4s2C8oTui1LGM6bTWkw0ICGcOLCAI5l6zsD1j20k=
-go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0/go.mod h1:Vl1/iaggsuRlrHf/hfPJPvVag77kKyvrLeD10kpMl+A=
-go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.42.0 h1:zWWrB1U6nqhS/k6zYB74CjRpuiitRtLLi68VcgmOEto=
-go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.42.0/go.mod h1:2qXPNBX1OVRC0IwOnfo1ljoid+RD0QK3443EaqVlsOU=
-go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.43.0 h1:3iZJKlCZufyRzPzlQhUIWVmfltrXuGyfjREgGP3UUjc=
-go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.43.0/go.mod h1:/G+nUPfhq2e+qiXMGxMwumDrP5jtzU+mWN7/sjT2rak=
+go.opentelemetry.io/otel v1.45.0 h1:pdrWmLHofpubmArBv1LgFSv1Z0Ie/ppdZzu+kUN5EeU=
+go.opentelemetry.io/otel v1.45.0/go.mod h1:XZxIqPapzEYnhNSScF5DIqXhm/rYi0FzCe2XddAwZfQ=
+go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.44.0 h1:SUplec5dp06reu1zaXmOXdvqH398taqrDXqUl99jxSc=
+go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.44.0/go.mod h1:ho2g4N+ane+swq5I/VBkKWnRDY4kUINH3FuqyZqX/Ug=
+go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.44.0 h1:RuynHbfU8JUEw7DyONgkVYg2SVtsoF28y0LGIr69jgA=
+go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.44.0/go.mod h1:qZF+/lBs71APw8mlnEZcqZHMzqrYrsFiJOv83lX1OGo=
+go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.44.0 h1:4YsVu3B8+3qtWYYrsUYgn0OG78pN0rnNPRGX4SbokQI=
+go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.44.0/go.mod h1:+wnlSn0mD1ADVMe3v9Z/WIaiz6q6gL2J/ejaAmdmv80=
+go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.44.0 h1:qazEJlUOQzhCpzQpFETGby7EdqjI1wsd0W+6Gg1SCTU=
+go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.44.0/go.mod h1:fOD2Yefuxixkx3ahVNf0O/PERb6r4OlbxfATVnYvzCo=
+go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.44.0 h1:lgh3PiVrRUWMLOVSkQicxzZll5NjF1r+AtsX1XRIHw0=
+go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.44.0/go.mod h1:5Cnhth3m/AgOeTgE3ex12pPmiu/gGtZit03kSzx9X7s=
 go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.44.0 h1:hqxVTu/GtBF+vJ8d1fzW7fRxZFvgoDjWcxwwCaFDYpU=
 go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.44.0/go.mod h1:z5fVEF4X5v0ESvlJqBrrFlBVoj5EQuefZpzsu7R+x5Q=
 go.opentelemetry.io/otel/exporters/zipkin v1.36.0 h1:s0n95ya5tOG03exJ5JySOdJFtwGo4ZQ+KeY7Zro4CLI=
 go.opentelemetry.io/otel/exporters/zipkin v1.36.0/go.mod h1:m9wRxtKA2MZ1HcnNC4BKI+9aYe434qRZTCvI7QGUN7Y=
-go.opentelemetry.io/otel/metric v1.44.0 h1:1w0gILTcHdr3YI+ixLyjemwrVnsMURbTZFrSYCdDdmc=
-go.opentelemetry.io/otel/metric v1.44.0/go.mod h1:8O7hanEPBNgEMmybD3s2VBKcgWOCsA6tzHBPODAiquo=
-go.opentelemetry.io/otel/metric/x v0.66.0 h1:YkCrx1zLOChi9ZcZ6euupOcsgzbVlec7D/xoEU1+cTA=
-go.opentelemetry.io/otel/metric/x v0.66.0/go.mod h1:d1+BDj9t96do0/1LoU1ayfCv79ZgNE41qbhBvnMOBZk=
-go.opentelemetry.io/otel/sdk v1.44.0 h1:nHYwb9lK+fJPU/dnT6s7W7Z8itMWyqrnVfbheVYrZ58=
-go.opentelemetry.io/otel/sdk v1.44.0/go.mod h1:Osuydd3Se74nqjAKxid74N5eC+jfEqfTegHRnq58oK0=
-go.opentelemetry.io/otel/sdk/metric v1.44.0 h1:3LlKgI+VjbVsjNRFZJZAJ30WjXC5VkNRks6si09iEfI=
-go.opentelemetry.io/otel/sdk/metric v1.44.0/go.mod h1:5B5pMARnXxKhltooO4xUuCBorl65a4EpnTalObqOigA=
-go.opentelemetry.io/otel/trace v1.44.0 h1:jxF5CsGYCe74MCRx2X4g7WsY/VBKRqqpNvXlX/6gtIk=
-go.opentelemetry.io/otel/trace v1.44.0/go.mod h1:oLl1jrMQAVo6v3GAggN+1VH9VIz9iUSvW53sW1Q8PIE=
+go.opentelemetry.io/otel/metric v1.45.0 h1:7Eg1uH7CJ5cXv9is6tnBe1FI6rj1nwUdbFypRm3br/M=
+go.opentelemetry.io/otel/metric v1.45.0/go.mod h1:HAPbm1nd3p1PmFH7v2dR+6BjXxw+Lq4a2+pndMAm08s=
+go.opentelemetry.io/otel/metric/x v0.67.0 h1:PcicCNZFkZ4bXfSooXdo3WN7RBOVOtjVdo1wD358Uns=
+go.opentelemetry.io/otel/metric/x v0.67.0/go.mod h1:FBjCWZe6wgcqxcMtjdGiClDKXb2YxxXii0CXftE4QtI=
+go.opentelemetry.io/otel/sdk v1.45.0 h1:4VVSMgQ83dUgW2aoX5f6JgLvHwIvzcuLnF9lUdCSpCw=
+go.opentelemetry.io/otel/sdk v1.45.0/go.mod h1:Sr40LgXV7DsKMMJMKOhUWOgMWTfAaqvm2kF0g7ilwuA=
+go.opentelemetry.io/otel/sdk/metric v1.45.0 h1:oVFszMfyj1Am6s24Vtc7wBb8BKLcwepJjNEYILuiE3o=
+go.opentelemetry.io/otel/sdk/metric v1.45.0/go.mod h1:vUWUxDZvu1WVRj8JA8S0AdhsPrZoDpA2DdZauIh4mDA=
+go.opentelemetry.io/otel/trace v1.45.0 h1:l/mP6Uv7oNO7/TblbhpbgMidxhq1uO/rPsikOyVhxag=
+go.opentelemetry.io/otel/trace v1.45.0/go.mod h1:qoJJA2xNMnxRrdISU/kLtfUH2wNeQbiv+jhs/CxI8bc=
 go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI=
 go.opentelemetry.io/proto/otlp v0.15.0/go.mod h1:H7XAot3MsfNsj7EXtrA2q5xSNQ10UqI405h3+duxN4U=
 go.opentelemetry.io/proto/otlp v0.19.0/go.mod h1:H7XAot3MsfNsj7EXtrA2q5xSNQ10UqI405h3+duxN4U=
@@ -2156,8 +2165,8 @@ go.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ=
 go.yaml.in/yaml/v2 v2.4.4/go.mod h1:gMZqIpDtDqOfM0uNfy0SkpRhvUryYH0Z6wdMYcacYXQ=
 go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
 go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
-go.yaml.in/yaml/v4 v4.0.0-rc.4 h1:UP4+v6fFrBIb1l934bDl//mmnoIZEDK0idg1+AIvX5U=
-go.yaml.in/yaml/v4 v4.0.0-rc.4/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0=
+go.yaml.in/yaml/v4 v4.0.0-rc.6 h1:1h7H1ohdUh93/FyE4YaDa1Zh64K6VVbjF4K6WUxMtH4=
+go.yaml.in/yaml/v4 v4.0.0-rc.6/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0=
 gocloud.dev v0.46.0 h1:niIuZwSjMtBx8K+ITB2s5kZullB13PGOS2ZoQPZxQ4Q=
 gocloud.dev v0.46.0/go.mod h1:ACQe+2qO+hEO+pdcvvsM+RB63r8TyGD1W3ESCLFyzvM=
 gocloud.dev/pubsub/natspubsub v0.46.0 h1:cTmJnWOmhb3dYRzX7PvNkq2m5mQdBLFFxZbl/21pfjk=
@@ -2216,8 +2225,8 @@ golang.org/x/image v0.0.0-20210607152325-775e3b0c77b9/go.mod h1:023OzeP/+EPmXeap
 golang.org/x/image v0.0.0-20210628002857-a66eb6448b8d/go.mod h1:023OzeP/+EPmXeapQh35lcL3II3LrY8Ic+EFFKVhULM=
 golang.org/x/image v0.0.0-20211028202545-6944b10bf410/go.mod h1:023OzeP/+EPmXeapQh35lcL3II3LrY8Ic+EFFKVhULM=
 golang.org/x/image v0.0.0-20220302094943-723b81ca9867/go.mod h1:023OzeP/+EPmXeapQh35lcL3II3LrY8Ic+EFFKVhULM=
-golang.org/x/image v0.44.0 h1:+tDekMZED9+LrtB3G5xzRggpVh9CARjZqROla3R3R+I=
-golang.org/x/image v0.44.0/go.mod h1:V8K3KE9KKKE+pLpQDOeN18w9oacNSvy1tDOirTu4xtY=
+golang.org/x/image v0.45.0 h1:FMb1nTbH5H9vF55SriQHgFw5GnNL9Jg6L25BwXKzhB0=
+golang.org/x/image v0.45.0/go.mod h1:n62x/7RqlwXDvGsSU4u6IUTUf6KghUZ9Bt7cG/T9Fx4=
 golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
 golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
 golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
@@ -2659,8 +2668,8 @@ google.golang.org/api v0.106.0/go.mod h1:2Ts0XTHNVWxypznxWOYUeI4g3WdP9Pk2Qk58+a/
 google.golang.org/api v0.107.0/go.mod h1:2Ts0XTHNVWxypznxWOYUeI4g3WdP9Pk2Qk58+a/O9MY=
 google.golang.org/api v0.108.0/go.mod h1:2Ts0XTHNVWxypznxWOYUeI4g3WdP9Pk2Qk58+a/O9MY=
 google.golang.org/api v0.110.0/go.mod h1:7FC4Vvx1Mooxh8C5HWjzZHcavuS2f6pmJpZx60ca7iI=
-google.golang.org/api v0.294.0 h1:8gASjJxdtcIieB3OqbkLcF0FfbXVNqKtU5iozD1ssvA=
-google.golang.org/api v0.294.0/go.mod h1:02qB8+Ox1ZFzcaKFMguy1nQLJmSIyvV6Ff4txJEXtl4=
+google.golang.org/api v0.296.0 h1:Nn5EHeKdGx70MFClaV/II0gsWUm6xhEjb0xYLylVvaA=
+google.golang.org/api v0.296.0/go.mod h1:02qB8+Ox1ZFzcaKFMguy1nQLJmSIyvV6Ff4txJEXtl4=
 google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
 google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
 google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=

k8s/charts/seaweedfs/Chart.yaml

@@ -1,6 +1,6 @@
 apiVersion: v1
 description: SeaweedFS
 name: seaweedfs
-appVersion: "4.44"
+appVersion: "4.46"
 # Dev note: Trigger a helm chart release by `git tag -a helm-<version>`
-version: 4.44.0
+version: 4.46.0

k8s/charts/seaweedfs/templates/admin/admin-ingress.yaml

@@ -23,6 +23,9 @@ metadata:
     app.kubernetes.io/managed-by: {{ .Release.Service }}
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/component: admin
+    {{- with .Values.admin.ingress.labels }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
 spec:
   {{- if and (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion) .Values.admin.ingress.className }}
   ingressClassName: {{ .Values.admin.ingress.className | quote }}

k8s/charts/seaweedfs/templates/filer/filer-grpc-ingressroutetcp.yaml

@@ -24,6 +24,9 @@ metadata:
     app.kubernetes.io/managed-by: {{ .Release.Service }}
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/component: filer
+    {{- with .Values.filer.ingresses.grpc.labels }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
 spec:
   {{- if $securityEnabled }}
   tls:

k8s/charts/seaweedfs/templates/filer/filer-ingress.yaml

@@ -29,6 +29,9 @@ metadata:
     app.kubernetes.io/managed-by: {{ .Release.Service }}
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/component: filer
+    {{- with .Values.filer.ingresses.http.labels }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
 spec:
   {{- if and (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion) .Values.filer.ingresses.http.className }}
   ingressClassName: {{ .Values.filer.ingresses.http.className | quote }}
@@ -87,6 +90,9 @@ metadata:
     app.kubernetes.io/managed-by: {{ .Release.Service }}
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/component: filer
+    {{- with .Values.filer.ingresses.grpc.labels }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
 spec:
   {{- if and (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion) .Values.filer.ingresses.grpc.className }}
   ingressClassName: {{ .Values.filer.ingresses.grpc.className | quote }}

k8s/charts/seaweedfs/templates/master/master-ingress.yaml

@@ -21,6 +21,9 @@ metadata:
     app.kubernetes.io/managed-by: {{ .Release.Service }}
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/component: master
+    {{- with .Values.master.ingress.labels }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
 spec:
   ingressClassName: {{ .Values.master.ingress.className | quote }}
   tls:

k8s/charts/seaweedfs/templates/s3/s3-iceberg-ingress.yaml

@@ -41,6 +41,9 @@ metadata:
     app.kubernetes.io/managed-by: {{ .Release.Service }}
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/component: s3-iceberg
+    {{- with .Values.s3.icebergIngress.labels }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
 spec:
   {{- if .Values.s3.icebergIngress.className }}
   ingressClassName: {{ .Values.s3.icebergIngress.className | quote }}

k8s/charts/seaweedfs/templates/s3/s3-ingress.yaml

@@ -32,6 +32,9 @@ metadata:
     app.kubernetes.io/managed-by: {{ .Release.Service }}
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/component: s3
+    {{- with .Values.s3.ingress.labels }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
 spec:
   ingressClassName: {{ .Values.s3.ingress.className | quote }}
   tls:

k8s/charts/seaweedfs/templates/s3/s3-lance-ingress.yaml

@@ -41,6 +41,9 @@ metadata:
     app.kubernetes.io/managed-by: {{ .Release.Service }}
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/component: s3-lance
+    {{- with .Values.s3.lanceIngress.labels }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
 spec:
   {{- if .Values.s3.lanceIngress.className }}
   ingressClassName: {{ .Values.s3.lanceIngress.className | quote }}

k8s/charts/seaweedfs/templates/shared/security-configmap.yaml

@@ -97,6 +97,19 @@ data:
     cert = "/usr/local/share/ca-certificates/worker/tls.crt"
     key  = "/usr/local/share/ca-certificates/worker/tls.key"
 
+    {{- /* S3 gRPC server identity (weed s3 -port.grpc, default httpPort+10000).
+      Without this section, LoadServerTLS("grpc.s3") returns nil and the S3
+      server serves its internal gRPC (IAM cache propagation, s3 lifecycle
+      delete RPCs) as plaintext, while peers dial it with mTLS credentials:
+      "tls: first record does not look like a TLS handshake".
+      Always uses the internal CA-signed client certificate (already mounted
+      on s3 pods by s3-deployment.yaml). Deliberately NOT s3.tlsSecret: that
+      secret is for the public HTTPS listener and may be issued by a public
+      CA which internal gRPC peers (trusting only grpc.ca) would reject. */}}
+    [grpc.s3]
+    cert = "/usr/local/share/ca-certificates/client/tls.crt"
+    key  = "/usr/local/share/ca-certificates/client/tls.key"
+
     # use this for any place needs a grpc client
     # i.e., "weed backup|benchmark|filer.copy|filer.replicate|mount|s3|upload"
     [grpc.client]

k8s/charts/seaweedfs/templates/volume/volume-ingress.yaml

@@ -27,6 +27,9 @@ metadata:
     app.kubernetes.io/managed-by: {{ .Release.Service }}
     app.kubernetes.io/instance: {{ .Release.Name }}
     app.kubernetes.io/component: volume
+    {{- with .Values.volume.ingress.labels }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
 spec:
   {{- if and (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion) .Values.volume.ingress.className }}
   ingressClassName: {{ .Values.volume.ingress.className | quote }}

k8s/charts/seaweedfs/values.yaml

@@ -55,7 +55,7 @@ global:
       gatewayHost: null
       gatewayPort: null
       additionalLabels: {}
-    # if enabled will use global.seaweedfs.replicationPlacement and override master & filer defaultReplicaPlacement config
+    # if enabled will use global.seaweedfs.replicationPlacement and override master.defaultReplication & filer.defaultReplicaPlacement config
     enableReplication: false
     #  replication type is XYZ:
     # X number of replica in other data centers
@@ -276,6 +276,7 @@ master:
     host: "master.seaweedfs.local"
     path: "/sw-master/?(.*)"
     pathType: ImplementationSpecific
+    labels: {}
     annotations: {}
       # nginx.ingress.kubernetes.io/auth-type: "basic"
       # nginx.ingress.kubernetes.io/auth-secret: "default/ingress-basic-auth-secret"
@@ -600,6 +601,7 @@ volume:
     host: "volume.seaweedfs.local"
     path: "/"
     pathType: Prefix
+    labels: {}
     annotations:
       nginx.ingress.kubernetes.io/app-root: /ui/index.html
       # nginx.ingress.kubernetes.io/use-regex: "true"
@@ -858,6 +860,7 @@ filer:
       host: "seaweedfs.cluster.local"
       path: "/sw-filer/?(.*)"
       pathType: ImplementationSpecific
+      labels: {}
       annotations: {}
         # nginx.ingress.kubernetes.io/backend-protocol: GRPC
         # nginx.ingress.kubernetes.io/auth-type: "basic"
@@ -884,6 +887,7 @@ filer:
       # whole host, not the HTTP UI's regex path.
       path: "/"
       pathType: Prefix
+      labels: {}
       annotations:
         # Ingress terminates TLS and re-originates gRPC (HTTP/2) to the filer.
         nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
@@ -1180,6 +1184,7 @@ s3:
     host: "seaweedfs.cluster.local"
     path: "/"
     pathType: Prefix
+    labels: {}
     # additional ingress annotations for the s3 endpoint
     annotations: {}
     tls: []
@@ -1205,6 +1210,7 @@ s3:
     host: "seaweedfs-iceberg.cluster.local"
     path: "/"
     pathType: Prefix
+    labels: {}
     annotations: {}
     tls: []
 
@@ -1214,6 +1220,7 @@ s3:
     host: "seaweedfs-lance.cluster.local"
     path: "/"
     pathType: Prefix
+    labels: {}
     annotations: {}
     tls: []
 
@@ -1445,6 +1452,7 @@ admin:
     host: "admin.seaweedfs.local"
     path: "/"
     pathType: Prefix
+    labels: {}
     annotations: {}
     tls: []
 

other/java/client/src/main/proto/filer.proto

@@ -13,6 +13,9 @@ service SeaweedFiler {
     rpc LookupDirectoryEntry (LookupDirectoryEntryRequest) returns (LookupDirectoryEntryResponse) {
     }
 
+    rpc LookupDirectoryEntries (LookupDirectoryEntriesRequest) returns (LookupDirectoryEntriesResponse) {
+    }
+
     rpc ListEntries (ListEntriesRequest) returns (stream ListEntriesResponse) {
     }
 
@@ -250,9 +253,11 @@ message CreateEntryRequest {
     repeated int32 signatures = 5;
     bool skip_check_parent_directory = 6;
     // Optional precondition evaluated against the current entry atomically with
-    // the write, under the filer's per-path lock. The caller must route the
-    // key's writes to this entry's owner filer for the check to be authoritative.
+    // the write, under the filer's per-path lock.
     WriteCondition condition = 7;
+    // Set on a create a filer forwarded to the entry's ring owner, so the owner
+    // applies it locally instead of forwarding again. Clients leave it unset.
+    bool is_moved = 8;
 }
 
 // WriteCondition is the precondition the filer evaluates against the existing
@@ -561,6 +566,9 @@ message AssignVolumeResponse {
     string error = 8;
     Location location = 9;
     repeated Location replicas = 10;
+    // fsync is the storage rule's fsync decision for the assigned path, so the
+    // client can carry it onto the volume server upload request.
+    bool fsync = 11;
 }
 
 message LookupVolumeRequest {
@@ -576,6 +584,7 @@ message Location {
     string public_url = 2;
     uint32 grpc_port = 3;
     string data_center = 4;
+    bool data_in_remote = 5;
 }
 message LookupVolumeResponse {
     map<string, Locations> locations_map = 1;
@@ -662,6 +671,7 @@ message SubscribeMetadataResponse {
     int64 ts_ns = 3;
     repeated SubscribeMetadataResponse events = 4; // batch of additional events (backlog catch-up)
     repeated LogFileChunkRef log_file_refs = 5;    // log file chunk refs for client direct-read
+    int64 flushed_ts_ns = 6; // local log-buffer flush watermark: everything at or below it is on disk
 }
 message ListMetadataSubscribersRequest {
     repeated string client_types = 1; // optional filter by client type, e.g. "mount"; empty = all
@@ -906,3 +916,40 @@ message MountInfo {
     int64 last_seen_ns = 3;
     string data_center = 4;
 }
+
+// LookupDirectoryEntriesRequest batches independent exact-path lookups into
+// one bounded RPC. Results preserve this order. Empty batches and batches over
+// 4096 requests are rejected. Declared last to keep generated message indices
+// stable.
+message LookupDirectoryEntriesRequest {
+    repeated LookupDirectoryEntryRequest requests = 1;
+    // Cache callers may treat an Entry whose Volume the master no longer
+    // reports as a miss. A lookup the filer could not complete still marks the
+    // Entry with an error. Ordinary filesystem callers keep the fail-closed
+    // error in both cases.
+    bool unavailable_volume_is_miss = 2;
+}
+
+message LookupDirectoryEntryResult {
+    // found reports metadata presence. An error makes this item unusable even
+    // when entry is present, for example when a chunk volume cannot be located.
+    bool found = 1;
+    Entry entry = 2;
+    string error = 3;
+    // Per-entry read fence. Every filer event at or below this position is
+    // reflected in found/entry/error for this item.
+    int64 log_ts_ns = 4;
+    int32 log_signature = 5;
+}
+
+message LookupDirectoryEntriesResponse {
+    repeated LookupDirectoryEntryResult results = 1;
+    // One deduplicated location set for every volume referenced by returned
+    // Entry.chunks. A missing/unavailable volume has an empty Locations value
+    // and marks each affected result with an error.
+    map<string, Locations> locations_map = 2;
+    // Short-lived, exact-FID read capabilities. Direct data-plane clients must
+    // present the matching token to a Volume instead of treating a shared
+    // service credential as authority to read arbitrary needles.
+    map<string, string> read_auth = 3;
+}

other/metrics/grafana_seaweedfs.json

@@ -5869,8 +5869,8 @@
             "type": "prometheus",
             "uid": "${DS_PROMETHEUS}"
           },
-          "title": "Filer Sync Offset Lag",
-          "description": "Seconds behind source (filer.sync); only present when filer sync runs",
+          "title": "Filer Sync Lag",
+          "description": "Seconds the replicated watermark trails the source filer (filer.sync); only present when filer sync runs",
           "type": "timeseries",
           "id": 155,
           "gridPos": {
@@ -5952,10 +5952,10 @@
                 "uid": "${DS_PROMETHEUS}"
               },
               "editorMode": "code",
-              "expr": "time() - (SeaweedFS_filerSync_sync_offset{cluster=~\"$cluster\"} > 0) / 1e9",
+              "expr": "SeaweedFS_filerSync_lag_seconds{cluster=~\"$cluster\"}",
               "range": true,
               "refId": "A",
-              "legendFormat": "{{clientName}} {{path}}"
+              "legendFormat": "{{sourceFiler}} -> {{targetFiler}} {{path}}"
             }
           ],
           "pluginVersion": "10.3.1"
@@ -6055,6 +6055,423 @@
             }
           ],
           "pluginVersion": "10.3.1"
+        },
+        {
+          "datasource": {
+            "type": "prometheus",
+            "uid": "${DS_PROMETHEUS}"
+          },
+          "title": "Filer Sync Event Rate",
+          "description": "Metadata events read off the source stream, replicated, and failed after retries (filer.sync)",
+          "type": "timeseries",
+          "id": 230,
+          "gridPos": {
+            "h": 8,
+            "w": 12,
+            "x": 0,
+            "y": 40
+          },
+          "fieldConfig": {
+            "defaults": {
+              "color": {
+                "mode": "palette-classic"
+              },
+              "custom": {
+                "axisBorderShow": false,
+                "axisCenteredZero": false,
+                "axisColorMode": "text",
+                "axisLabel": "",
+                "axisPlacement": "auto",
+                "barAlignment": 0,
+                "drawStyle": "line",
+                "fillOpacity": 10,
+                "gradientMode": "none",
+                "hideFrom": {
+                  "legend": false,
+                  "tooltip": false,
+                  "viz": false
+                },
+                "insertNulls": false,
+                "lineInterpolation": "linear",
+                "lineWidth": 1,
+                "pointSize": 4,
+                "scaleDistribution": {
+                  "type": "linear"
+                },
+                "showPoints": "never",
+                "spanNulls": false,
+                "stacking": {
+                  "group": "A",
+                  "mode": "none"
+                },
+                "thresholdsStyle": {
+                  "mode": "off"
+                }
+              },
+              "mappings": [],
+              "unit": "ops",
+              "thresholds": {
+                "mode": "absolute",
+                "steps": [
+                  {
+                    "color": "green",
+                    "value": null
+                  }
+                ]
+              }
+            },
+            "overrides": []
+          },
+          "options": {
+            "legend": {
+              "calcs": [
+                "lastNotNull",
+                "max"
+              ],
+              "displayMode": "table",
+              "placement": "bottom",
+              "showLegend": true
+            },
+            "tooltip": {
+              "mode": "multi",
+              "sort": "desc"
+            }
+          },
+          "targets": [
+            {
+              "datasource": {
+                "type": "prometheus",
+                "uid": "${DS_PROMETHEUS}"
+              },
+              "editorMode": "code",
+              "expr": "rate(SeaweedFS_filerSync_events_received_total{cluster=~\"$cluster\"}[$__rate_interval])",
+              "range": true,
+              "refId": "A",
+              "legendFormat": "{{sourceFiler}} -> {{targetFiler}} {{path}} received"
+            },
+            {
+              "datasource": {
+                "type": "prometheus",
+                "uid": "${DS_PROMETHEUS}"
+              },
+              "editorMode": "code",
+              "expr": "rate(SeaweedFS_filerSync_events_processed_total{cluster=~\"$cluster\"}[$__rate_interval])",
+              "range": true,
+              "refId": "B",
+              "legendFormat": "{{sourceFiler}} -> {{targetFiler}} {{path}} processed"
+            },
+            {
+              "datasource": {
+                "type": "prometheus",
+                "uid": "${DS_PROMETHEUS}"
+              },
+              "editorMode": "code",
+              "expr": "rate(SeaweedFS_filerSync_events_failed_total{cluster=~\"$cluster\"}[$__rate_interval])",
+              "range": true,
+              "refId": "C",
+              "legendFormat": "{{sourceFiler}} -> {{targetFiler}} {{path}} failed"
+            }
+          ],
+          "pluginVersion": "10.3.1"
+        },
+        {
+          "datasource": {
+            "type": "prometheus",
+            "uid": "${DS_PROMETHEUS}"
+          },
+          "title": "Filer Sync Throughput",
+          "description": "Chunk data bytes carried by sync events: the chunk delta, so deletes, renames, and attribute-only updates count zero (filer.sync)",
+          "type": "timeseries",
+          "id": 231,
+          "gridPos": {
+            "h": 8,
+            "w": 12,
+            "x": 12,
+            "y": 40
+          },
+          "fieldConfig": {
+            "defaults": {
+              "color": {
+                "mode": "palette-classic"
+              },
+              "custom": {
+                "axisBorderShow": false,
+                "axisCenteredZero": false,
+                "axisColorMode": "text",
+                "axisLabel": "",
+                "axisPlacement": "auto",
+                "barAlignment": 0,
+                "drawStyle": "line",
+                "fillOpacity": 10,
+                "gradientMode": "none",
+                "hideFrom": {
+                  "legend": false,
+                  "tooltip": false,
+                  "viz": false
+                },
+                "insertNulls": false,
+                "lineInterpolation": "linear",
+                "lineWidth": 1,
+                "pointSize": 4,
+                "scaleDistribution": {
+                  "type": "linear"
+                },
+                "showPoints": "never",
+                "spanNulls": false,
+                "stacking": {
+                  "group": "A",
+                  "mode": "none"
+                },
+                "thresholdsStyle": {
+                  "mode": "off"
+                }
+              },
+              "mappings": [],
+              "unit": "Bps",
+              "thresholds": {
+                "mode": "absolute",
+                "steps": [
+                  {
+                    "color": "green",
+                    "value": null
+                  }
+                ]
+              }
+            },
+            "overrides": []
+          },
+          "options": {
+            "legend": {
+              "calcs": [
+                "lastNotNull",
+                "max"
+              ],
+              "displayMode": "table",
+              "placement": "bottom",
+              "showLegend": true
+            },
+            "tooltip": {
+              "mode": "multi",
+              "sort": "desc"
+            }
+          },
+          "targets": [
+            {
+              "datasource": {
+                "type": "prometheus",
+                "uid": "${DS_PROMETHEUS}"
+              },
+              "editorMode": "code",
+              "expr": "rate(SeaweedFS_filerSync_received_bytes_total{cluster=~\"$cluster\"}[$__rate_interval])",
+              "range": true,
+              "refId": "A",
+              "legendFormat": "{{sourceFiler}} -> {{targetFiler}} {{path}} received"
+            },
+            {
+              "datasource": {
+                "type": "prometheus",
+                "uid": "${DS_PROMETHEUS}"
+              },
+              "editorMode": "code",
+              "expr": "rate(SeaweedFS_filerSync_processed_bytes_total{cluster=~\"$cluster\"}[$__rate_interval])",
+              "range": true,
+              "refId": "B",
+              "legendFormat": "{{sourceFiler}} -> {{targetFiler}} {{path}} processed"
+            }
+          ],
+          "pluginVersion": "10.3.1"
+        },
+        {
+          "datasource": {
+            "type": "prometheus",
+            "uid": "${DS_PROMETHEUS}"
+          },
+          "title": "Filer Sync In-flight Jobs",
+          "description": "Jobs currently replicating; pinned at -concurrency means the sync itself is the bottleneck, near zero means caught up or starved by the source (filer.sync)",
+          "type": "timeseries",
+          "id": 232,
+          "gridPos": {
+            "h": 8,
+            "w": 12,
+            "x": 0,
+            "y": 48
+          },
+          "fieldConfig": {
+            "defaults": {
+              "color": {
+                "mode": "palette-classic"
+              },
+              "custom": {
+                "axisBorderShow": false,
+                "axisCenteredZero": false,
+                "axisColorMode": "text",
+                "axisLabel": "",
+                "axisPlacement": "auto",
+                "barAlignment": 0,
+                "drawStyle": "line",
+                "fillOpacity": 10,
+                "gradientMode": "none",
+                "hideFrom": {
+                  "legend": false,
+                  "tooltip": false,
+                  "viz": false
+                },
+                "insertNulls": false,
+                "lineInterpolation": "linear",
+                "lineWidth": 1,
+                "pointSize": 4,
+                "scaleDistribution": {
+                  "type": "linear"
+                },
+                "showPoints": "never",
+                "spanNulls": false,
+                "stacking": {
+                  "group": "A",
+                  "mode": "none"
+                },
+                "thresholdsStyle": {
+                  "mode": "off"
+                }
+              },
+              "mappings": [],
+              "unit": "short",
+              "thresholds": {
+                "mode": "absolute",
+                "steps": [
+                  {
+                    "color": "green",
+                    "value": null
+                  }
+                ]
+              }
+            },
+            "overrides": []
+          },
+          "options": {
+            "legend": {
+              "calcs": [
+                "lastNotNull",
+                "max"
+              ],
+              "displayMode": "table",
+              "placement": "bottom",
+              "showLegend": true
+            },
+            "tooltip": {
+              "mode": "multi",
+              "sort": "desc"
+            }
+          },
+          "targets": [
+            {
+              "datasource": {
+                "type": "prometheus",
+                "uid": "${DS_PROMETHEUS}"
+              },
+              "editorMode": "code",
+              "expr": "SeaweedFS_filerSync_in_flight_jobs{cluster=~\"$cluster\"}",
+              "range": true,
+              "refId": "A",
+              "legendFormat": "{{sourceFiler}} -> {{targetFiler}} {{path}}"
+            }
+          ],
+          "pluginVersion": "10.3.1"
+        },
+        {
+          "datasource": {
+            "type": "prometheus",
+            "uid": "${DS_PROMETHEUS}"
+          },
+          "title": "Filer Sync In-flight Bytes",
+          "description": "Chunk data carried by the jobs currently replicating; distinguishes workers stuck on a few large files from many small ones (filer.sync)",
+          "type": "timeseries",
+          "id": 233,
+          "gridPos": {
+            "h": 8,
+            "w": 12,
+            "x": 12,
+            "y": 48
+          },
+          "fieldConfig": {
+            "defaults": {
+              "color": {
+                "mode": "palette-classic"
+              },
+              "custom": {
+                "axisBorderShow": false,
+                "axisCenteredZero": false,
+                "axisColorMode": "text",
+                "axisLabel": "",
+                "axisPlacement": "auto",
+                "barAlignment": 0,
+                "drawStyle": "line",
+                "fillOpacity": 10,
+                "gradientMode": "none",
+                "hideFrom": {
+                  "legend": false,
+                  "tooltip": false,
+                  "viz": false
+                },
+                "insertNulls": false,
+                "lineInterpolation": "linear",
+                "lineWidth": 1,
+                "pointSize": 4,
+                "scaleDistribution": {
+                  "type": "linear"
+                },
+                "showPoints": "never",
+                "spanNulls": false,
+                "stacking": {
+                  "group": "A",
+                  "mode": "none"
+                },
+                "thresholdsStyle": {
+                  "mode": "off"
+                }
+              },
+              "mappings": [],
+              "unit": "bytes",
+              "thresholds": {
+                "mode": "absolute",
+                "steps": [
+                  {
+                    "color": "green",
+                    "value": null
+                  }
+                ]
+              }
+            },
+            "overrides": []
+          },
+          "options": {
+            "legend": {
+              "calcs": [
+                "lastNotNull",
+                "max"
+              ],
+              "displayMode": "table",
+              "placement": "bottom",
+              "showLegend": true
+            },
+            "tooltip": {
+              "mode": "multi",
+              "sort": "desc"
+            }
+          },
+          "targets": [
+            {
+              "datasource": {
+                "type": "prometheus",
+                "uid": "${DS_PROMETHEUS}"
+              },
+              "editorMode": "code",
+              "expr": "SeaweedFS_filerSync_in_flight_bytes{cluster=~\"$cluster\"}",
+              "range": true,
+              "refId": "A",
+              "legendFormat": "{{sourceFiler}} -> {{targetFiler}} {{path}}"
+            }
+          ],
+          "pluginVersion": "10.3.1"
         }
       ]
     },

seaweed-volume/Cargo.lock

@@ -3160,9 +3160,9 @@ dependencies = [
 
 [[package]]
 name = "redb"
-version = "3.1.3"
+version = "4.2.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "4ba239c1c1693315d3cc0e601db3b3965543afbf48c41730fdca2f069f510f4a"
+checksum = "de6c3b63e007e90ce536ec2ae4690826136a20ec8dbbbb400daef1bb999d2e36"
 dependencies = [
  "libc",
 ]

seaweed-volume/Cargo.toml

@@ -16,6 +16,9 @@ path = "src/main.rs"
 # Disable with --no-default-features for 4-byte offsets (32GB max volume size).
 default = ["5bytes"]
 5bytes = []
+# Unstable redb cursor bulk-load for full_rebuild. Off in production.
+# Pulls redb's experimental_cursor (and therefore experimental-api-5).
+redb-experimental-cursor = ["redb/experimental_cursor"]
 
 [dependencies]
 # Async runtime
@@ -57,7 +60,7 @@ rustls-pemfile = "2"
 rusty-leveldb = "3"
 
 # Disk-backed needle map (alternative to in-memory HashMap)
-redb = "3"
+redb = "4"
 
 # Reed-Solomon erasure coding
 reed-solomon-erasure = "6"

seaweed-volume/proto/master.proto

@@ -145,6 +145,7 @@ message VolumeInformationMessage {
   uint64 delete_count = 5;
   uint64 deleted_byte_count = 6;
   bool read_only = 7;
+  bool read_only_can_delete = 17;
   uint32 replica_placement = 8;
   uint32 version = 9;
   uint32 ttl = 10;
@@ -164,6 +165,8 @@ message VolumeShortInformationMessage {
   uint32 ttl = 10;
   string disk_type = 15;
   uint32 disk_id = 16;
+  bool read_only = 17;
+  bool read_only_can_delete = 18;
 }
 
 message VolumeEcShardInformationMessage {
@@ -222,6 +225,9 @@ message VolumeLocation {
   uint32 grpc_port = 7;
   repeated uint32 new_ec_vids = 8;
   repeated uint32 deleted_ec_vids = 9;
+  repeated uint32 remote_vids = 10;
+  repeated uint32 read_only_vids = 11;
+  repeated uint32 read_only_can_delete_vids = 12;
 }
 
 message ClusterNodeUpdate {
@@ -267,6 +273,9 @@ message Location {
   string public_url = 2;
   uint32 grpc_port = 3;
   string data_center = 4;
+  bool data_in_remote = 5;
+  bool read_only = 6;
+  bool read_only_can_delete = 7;
 }
 
 message AssignRequest {

seaweed-volume/proto/remote.proto

@@ -75,4 +75,6 @@ message RemoteStorageLocation {
   string name = 1;
   string bucket = 2;
   string path = 3;
+  int32 listing_cache_ttl_seconds = 4; // 0 = disabled; >0 enables on-demand directory listing with this TTL in seconds
+  optional int32 cache_wait_ms = 5; // unset = size based default; 0 = read straight from the remote without caching
 }

seaweed-volume/src/config.rs

@@ -64,6 +64,8 @@ pub struct Cli {
     pub rack: String,
 
     /// Choose [memory|redb|redbMedium|redbLarge] mode for memory~performance balance.
+    /// The redb tiers give each volume's on-disk index a 4, 8, or 16 MiB page
+    /// cache respectively; total index memory is roughly (volumes x cache).
     /// `leveldb`/`leveldbMedium`/`leveldbLarge` are accepted as aliases for the
     /// corresponding redb backends (Rust volume server uses redb under the hood).
     #[arg(long = "index", default_value = "memory")]

seaweed-volume/src/remote_storage/s3_tier.rs

@@ -81,13 +81,17 @@ impl S3TierBackend {
     /// Returns (s3_key, file_size) on success.
     /// The progress callback receives (bytes_uploaded, percentage).
     /// Uses 64MB part size and 5 concurrent uploads (matches Go s3manager).
+    /// `progress_fn` returning `Err` aborts the upload, mirroring Go's
+    /// `fn(progressed, percentage) error` in `s3_upload.go`, where the error
+    /// surfaces out of `ReadAt` and fails the transfer. It is what lets a
+    /// caller that has hung up stop the work it is no longer waiting for.
     pub async fn upload_file<F>(
         &self,
         file_path: &str,
         progress_fn: F,
     ) -> Result<(String, u64), String>
     where
-        F: FnMut(i64, f32) + Send + Sync + 'static,
+        F: FnMut(i64, f32) -> Result<(), String> + Send + Sync + 'static,
     {
         let key = uuid::Uuid::new_v4().to_string();
 
@@ -184,8 +188,10 @@ impl S3TierBackend {
 
                 let e_tag = upload_part_resp.e_tag().unwrap_or_default().to_string();
 
-                // Report progress
-                {
+                // Report progress. The lock is released before the result is
+                // propagated so an aborting callback cannot poison the mutex
+                // for the other parts still in flight.
+                let progress_result = {
                     let mut guard = progress.lock().unwrap();
                     guard.0 += size as u64;
                     let uploaded = guard.0;
@@ -194,8 +200,9 @@ impl S3TierBackend {
                     } else {
                         100.0
                     };
-                    (guard.1)(uploaded as i64, pct);
-                }
+                    (guard.1)(uploaded as i64, pct)
+                };
+                progress_result?;
 
                 Ok::<_, String>(
                     CompletedPart::builder()
@@ -206,29 +213,58 @@ impl S3TierBackend {
             }));
         }
 
-        // Collect results, preserving part order
-        let mut completed_parts = Vec::with_capacity(handles.len());
-        for handle in handles {
-            let part = handle
+        let finish = async {
+            // Collect results, preserving part order
+            let mut completed_parts = Vec::with_capacity(handles.len());
+            for handle in handles {
+                let part = handle
+                    .await
+                    .map_err(|e| format!("upload task panicked: {}", e))??;
+                completed_parts.push(part);
+            }
+
+            // Complete multipart upload
+            let completed_upload = CompletedMultipartUpload::builder()
+                .set_parts(Some(completed_parts))
+                .build();
+
+            self.client
+                .complete_multipart_upload()
+                .bucket(&self.bucket)
+                .key(&key)
+                .upload_id(&upload_id)
+                .multipart_upload(completed_upload)
+                .send()
                 .await
-                .map_err(|e| format!("upload task panicked: {}", e))??;
-            completed_parts.push(part);
+                .map_err(|e| format!("failed to complete multipart upload: {}", e))?;
+
+            Ok::<(), String>(())
         }
+        .await;
 
-        // Complete multipart upload
-        let completed_upload = CompletedMultipartUpload::builder()
-            .set_parts(Some(completed_parts))
-            .build();
-
-        self.client
-            .complete_multipart_upload()
-            .bucket(&self.bucket)
-            .key(&key)
-            .upload_id(&upload_id)
-            .multipart_upload(completed_upload)
-            .send()
-            .await
-            .map_err(|e| format!("failed to complete multipart upload: {}", e))?;
+        if let Err(e) = finish {
+            // An abandoned multipart upload does not appear in an ordinary
+            // object listing but still accrues storage charges until a
+            // lifecycle rule reaps it. Now that a departing caller aborts the
+            // transfer this is a routine path, not a rare one.
+            if let Err(abort_err) = self
+                .client
+                .abort_multipart_upload()
+                .bucket(&self.bucket)
+                .key(&key)
+                .upload_id(&upload_id)
+                .send()
+                .await
+            {
+                tracing::warn!(
+                    "failed to abort multipart upload {} for key {}: {}",
+                    upload_id,
+                    key,
+                    abort_err
+                );
+            }
+            return Err(e);
+        }
 
         Ok((key, file_size))
     }
@@ -238,6 +274,8 @@ impl S3TierBackend {
     ///
     /// Returns the file size on success.
     /// Uses 64MB part size and 5 concurrent downloads (matches Go s3manager).
+    /// `progress_fn` returning `Err` aborts the download, mirroring Go's
+    /// `fn(progressed, percentage) error` in `s3_download.go`.
     pub async fn download_file<F>(
         &self,
         dest_path: &str,
@@ -245,7 +283,7 @@ impl S3TierBackend {
         progress_fn: F,
     ) -> Result<u64, String>
     where
-        F: FnMut(i64, f32) + Send + Sync + 'static,
+        F: FnMut(i64, f32) -> Result<(), String> + Send + Sync + 'static,
     {
         // Get file size first
         let head_resp = self
@@ -340,8 +378,10 @@ impl S3TierBackend {
                     .await
                     .map_err(|e| format!("failed to write to {}: {}", dp, e))?;
 
-                // Report progress
-                {
+                // Report progress. The lock is released before the result is
+                // propagated so an aborting callback cannot poison the mutex
+                // for the other parts still in flight.
+                let progress_result = {
                     let mut guard = progress.lock().unwrap();
                     guard.0 += bytes.len() as u64;
                     let downloaded = guard.0;
@@ -350,8 +390,9 @@ impl S3TierBackend {
                     } else {
                         100.0
                     };
-                    (guard.1)(downloaded as i64, pct);
-                }
+                    (guard.1)(downloaded as i64, pct)
+                };
+                progress_result?;
 
                 Ok::<_, String>(())
             }));

seaweed-volume/src/server/handlers.rs

@@ -356,6 +356,10 @@ fn parse_url_path(path: &str) -> Option<(VolumeId, NeedleId, Cookie)> {
 #[derive(Clone, Debug, Deserialize)]
 struct VolumeLocation {
     url: String,
+    #[serde(rename = "readOnly", default)]
+    read_only: bool,
+    #[serde(rename = "readOnlyCanDelete", default)]
+    read_only_can_delete: bool,
     // Master often omits publicUrl when it matches url (Go json omitempty).
     #[serde(rename = "publicUrl", default)]
     public_url: String,
@@ -546,27 +550,39 @@ async fn do_replicated_request(
     .await
     .map_err(|e| format!("lookup volume failed: {}", e))?;
 
-    // Mirror Go's GetWritableRemoteReplications: reject when the master reports fewer replicas than
-    // the copy count. lookup_volume is uncached, so recovery is immediate once the replica re-registers.
     let copy_count = {
         let store = state.store.read().unwrap();
-        store.find_volume(VolumeId(vid)).map_or(1, |(_, v)| {
-            v.super_block.replica_placement.get_copy_count()
-        })
+        store
+            .find_volume(VolumeId(vid))
+            .map_or(1, |(_, v)| v.super_block.replica_placement.get_copy_count())
     };
-    if locations.len() < copy_count as usize {
+    let allow_delete = method == axum::http::Method::DELETE;
+    let eligible_locations: Vec<_> = locations
+        .into_iter()
+        .filter(|loc| {
+            (!loc.read_only && allow_delete)
+                || (!loc.read_only && !allow_delete)
+                || (allow_delete && loc.read_only_can_delete)
+        })
+        .collect();
+    if eligible_locations.len() < copy_count as usize {
         return Err(format!(
             "replicating operations [{}] is less than volume {} replication copy count [{}]",
-            locations.len(),
+            eligible_locations.len(),
             vid,
             copy_count
         ));
     }
 
     let self_http = to_http_address(&state.self_url);
-    let remote_locations: Vec<_> = locations
+    let remote_locations: Vec<_> = eligible_locations
         .into_iter()
         .filter(|loc| {
+            if (!allow_delete && loc.read_only)
+                || (allow_delete && loc.read_only && !loc.read_only_can_delete)
+            {
+                return false;
+            }
             to_http_address(&loc.url) != self_http
                 && to_http_address(loc.public_or_url()) != self_http
         })
@@ -1047,8 +1063,8 @@ async fn get_or_head_handler_inner(
     let has_range = headers.contains_key(header::RANGE);
     let ext = extract_extension_from_path(&path);
     // Go checks resize and crop extensions separately: resize supports .webp, crop does not.
-    let has_resize_ops =
-        is_image_resize_ext(&ext) && (query.width.unwrap_or(0) > 0 || query.height.unwrap_or(0) > 0);
+    let has_resize_ops = is_image_resize_ext(&ext)
+        && (query.width.unwrap_or(0) > 0 || query.height.unwrap_or(0) > 0);
     // Go's shouldCropImages (L410) requires x2 > x1 && y2 > y1 (x1/y1 default 0).
     // Only disable streaming when a real crop will actually happen.
     let has_crop_ops = is_image_crop_ext(&ext) && {
@@ -1077,10 +1093,8 @@ async fn get_or_head_handler_inner(
         // serves both the "all shards local" fast case and the
         // "some intervals need peer fetch + reconstruct" general
         // case without paying for the local interval reads twice.
-        match crate::server::store_ec::read_ec_shard_needle_distributed(
-            &state, vid, needle_id,
-        )
-        .await
+        match crate::server::store_ec::read_ec_shard_needle_distributed(&state, vid, needle_id)
+            .await
         {
             Ok(Some(ec_needle)) => {
                 n = ec_needle;
@@ -1101,10 +1115,7 @@ async fn get_or_head_handler_inner(
                 if e.kind() == std::io::ErrorKind::NotFound {
                     return StatusCode::NOT_FOUND.into_response();
                 }
-                return (
-                    StatusCode::INTERNAL_SERVER_ERROR,
-                    format!("ec read: {}", e),
-                )
+                return (StatusCode::INTERNAL_SERVER_ERROR, format!("ec read: {}", e))
                     .into_response();
             }
         }
@@ -2243,7 +2254,10 @@ pub async fn post_handler(
             // With a limit configured, an error here means the body exceeded it
             // before we buffered the whole thing; report it like the size check.
             let msg = if state.file_size_limit_bytes > 0 {
-                format!("file over the limited {} bytes", state.file_size_limit_bytes)
+                format!(
+                    "file over the limited {} bytes",
+                    state.file_size_limit_bytes
+                )
             } else {
                 format!("read body: {}", e)
             };
@@ -3464,10 +3478,7 @@ async fn try_expand_chunk_manifest(
 /// (reconstruct-on-read from surviving shards), or a peer resolved via the
 /// master. Mirrors Go's ChunkedFileReader, which looks every chunk up through
 /// the master instead of assuming a local regular needle.
-async fn read_chunk_needle(
-    state: &Arc<VolumeServerState>,
-    fid: &str,
-) -> Result<Vec<u8>, String> {
+async fn read_chunk_needle(state: &Arc<VolumeServerState>, fid: &str) -> Result<Vec<u8>, String> {
     let (vid, nid, cookie) =
         parse_url_path(fid).ok_or_else(|| format!("invalid chunk fid: {}", fid))?;
 
@@ -4178,6 +4189,8 @@ mod tests {
             url: "volume.internal:8080".to_string(),
             public_url: "volume.public:8080".to_string(),
             grpc_port: 18080,
+            read_only: false,
+            read_only_can_delete: false,
         };
 
         let response = redirect_request(&info, &target, "https");
@@ -4204,6 +4217,8 @@ mod tests {
             url: "volume.internal:8080.18080".to_string(),
             public_url: "volume.public:8080.18080".to_string(),
             grpc_port: 18080,
+            read_only: false,
+            read_only_can_delete: false,
         };
 
         let response = redirect_request(&info, &target, "http");
@@ -4250,15 +4265,19 @@ mod tests {
 
         let app = Router::new().route(
             "/dir/lookup",
-            get(|axum::extract::Query(params): axum::extract::Query<std::collections::HashMap<String, String>>| async move {
-                assert_eq!(params.get("volumeId").map(String::as_str), Some("31"));
-                axum::Json(serde_json::json!({
-                    "volumeOrFileId": "31",
-                    "locations": [
-                        {"url": "10.0.0.2:5301", "publicUrl": "10.0.0.2:5301", "grpcPort": 5311}
-                    ]
-                }))
-            }),
+            get(
+                |axum::extract::Query(params): axum::extract::Query<
+                    std::collections::HashMap<String, String>,
+                >| async move {
+                    assert_eq!(params.get("volumeId").map(String::as_str), Some("31"));
+                    axum::Json(serde_json::json!({
+                        "volumeOrFileId": "31",
+                        "locations": [
+                            {"url": "10.0.0.2:5301", "publicUrl": "10.0.0.2:5301", "grpcPort": 5311}
+                        ]
+                    }))
+                },
+            ),
         );
 
         let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();

seaweed-volume/src/server/heartbeat.rs

@@ -19,9 +19,9 @@ use crate::pb::master_pb::seaweed_client::SeaweedClient;
 use crate::pb::volume_server_pb;
 use crate::remote_storage::s3_tier::{S3TierBackend, S3TierConfig};
 use crate::storage::store::Store;
+use crate::storage::types::NeedleId;
 use crate::storage::volume_report::VolumeReportKey;
 use crate::storage::volume_report_hash::report_hash;
-use crate::storage::types::NeedleId;
 
 const DUPLICATE_UUID_RETRY_MESSAGE: &str = "duplicate UUIDs detected, retrying connection";
 const MAX_DUPLICATE_UUID_RETRIES: u32 = 3;
@@ -92,8 +92,15 @@ pub async fn run_heartbeat_with_state(
                 SleepDuplicate(Duration),
                 SleepPulse,
             }
-            let action = match do_heartbeat(&config, &state, &grpc_addr, &target_addr, pulse, &mut shutdown_rx)
-                .await
+            let action = match do_heartbeat(
+                &config,
+                &state,
+                &grpc_addr,
+                &target_addr,
+                pulse,
+                &mut shutdown_rx,
+            )
+            .await
             {
                 Ok(Some(leader)) => {
                     info!("Master leader changed to {}", leader);
@@ -418,8 +425,7 @@ async fn do_heartbeat(
     // form so Ping admission can recognise it once a leader change moves us
     // off the seed list. Mirrors Go's vs.setCurrentMaster(masterAddress).
     {
-        let normalised =
-            super::volume_server::to_http_address(current_master).into_owned();
+        let normalised = super::volume_server::to_http_address(current_master).into_owned();
         let mut guard = state.current_master_url.write().await;
         *guard = normalised;
     }
@@ -538,7 +544,12 @@ async fn do_heartbeat(
                 let mut del_vols = Vec::new();
 
                 for (id, vol) in &current_volumes {
-                    if !last_volumes.contains_key(id) {
+                    if let Some(previous) = last_volumes.get(id) {
+                        if previous != vol {
+                            del_vols.push(previous.to_short_message(*id));
+                            new_vols.push(vol.to_short_message(*id));
+                        }
+                    } else {
                         new_vols.push(vol.to_short_message(*id));
                     }
                 }
@@ -730,7 +741,7 @@ fn parse_bool_property(value: Option<&String>) -> bool {
 /// information message the heartbeat carries. A server holding millions of
 /// volumes cannot keep a whole message for each just to notice one leave; the
 /// Go report state keeps the same fields for the same reason.
-#[derive(Clone)]
+#[derive(Clone, PartialEq)]
 struct VolumeIdentity {
     collection: String,
     disk_type: String,
@@ -738,6 +749,8 @@ struct VolumeIdentity {
     replica_placement: u32,
     ttl: u32,
     disk_id: u32,
+    read_only: bool,
+    read_only_can_delete: bool,
 }
 
 impl VolumeIdentity {
@@ -749,6 +762,8 @@ impl VolumeIdentity {
             replica_placement: v.replica_placement,
             ttl: v.ttl,
             disk_id: v.disk_id,
+            read_only: v.read_only,
+            read_only_can_delete: v.read_only_can_delete,
         }
     }
 
@@ -761,6 +776,8 @@ impl VolumeIdentity {
             ttl: self.ttl,
             disk_type: self.disk_type.clone(),
             disk_id: self.disk_id,
+            read_only: self.read_only,
+            read_only_can_delete: self.read_only_can_delete,
         }
     }
 }
@@ -778,7 +795,10 @@ fn volume_identities(
 fn collect_heartbeat_with_snapshot(
     config: &HeartbeatConfig,
     state: &Arc<VolumeServerState>,
-) -> (master_pb::Heartbeat, Vec<master_pb::VolumeInformationMessage>) {
+) -> (
+    master_pb::Heartbeat,
+    Vec<master_pb::VolumeInformationMessage>,
+) {
     let mut store = state.store.write().unwrap();
     let (ec_shards, deleted_ec_shards) = store.delete_expired_ec_volumes();
     build_heartbeat_with_ec_status(
@@ -856,7 +876,10 @@ fn build_heartbeat_with_ec_status(
     deleted_ec_shards: Vec<master_pb::VolumeEcShardInformationMessage>,
     has_no_ec_shards: bool,
     commit_report: bool,
-) -> (master_pb::Heartbeat, Vec<master_pb::VolumeInformationMessage>) {
+) -> (
+    master_pb::Heartbeat,
+    Vec<master_pb::VolumeInformationMessage>,
+) {
     const MAX_TTL_VOLUME_REMOVAL_DELAY: u32 = 10;
 
     #[derive(Default)]
@@ -939,7 +962,9 @@ fn build_heartbeat_with_ec_status(
                         .duration_since(UNIX_EPOCH)
                         .unwrap_or(Duration::ZERO)
                         .as_nanos() as i64;
-                    if now_ns - vol.last_disk_check_ns.load(Ordering::Relaxed) > DISK_CHECK_INTERVAL_NS {
+                    if now_ns - vol.last_disk_check_ns.load(Ordering::Relaxed)
+                        > DISK_CHECK_INTERVAL_NS
+                    {
                         if !Path::new(&vol.file_name(".dat")).exists() {
                             warn!("Volume {}: data file {} missing (held open as deleted FD) - not reporting to master", vol.id.0, vol.file_name(".dat"));
                             continue;
@@ -957,11 +982,15 @@ fn build_heartbeat_with_ec_status(
                     delete_count: vol.deleted_count() as u64,
                     deleted_byte_count: vol.deleted_size(),
                     read_only: vol.is_read_only(),
+                    read_only_can_delete: vol.is_no_write_can_delete(),
                     replica_placement: vol.super_block.replica_placement.to_byte() as u32,
                     version: vol.super_block.version.0 as u32,
                     ttl: vol.super_block.ttl.to_u32(),
                     compact_revision: vol.super_block.compaction_revision as u32,
-                    modified_at_second: vol.last_modified_ts() as i64,
+                    // The .dat mtime, as Go reports: the shell's quiet-period
+                    // gates read this as "last touched", which a delete has to
+                    // count towards even though the TTL clock ignores it.
+                    modified_at_second: vol.dat_file_mod_time() as i64,
                     disk_type: loc.disk_type.to_string(),
                     disk_id: disk_id as u32,
                     remote_storage_name,
@@ -1012,7 +1041,6 @@ fn build_heartbeat_with_ec_status(
                     }
                 }
             }
-
         }
 
         for vid in delete_vids {
@@ -1143,7 +1171,10 @@ fn collect_live_ec_shards(
 }
 
 /// Collect EC shard information into a Heartbeat message.
-fn collect_ec_heartbeat(config: &HeartbeatConfig, state: &Arc<VolumeServerState>) -> master_pb::Heartbeat {
+fn collect_ec_heartbeat(
+    config: &HeartbeatConfig,
+    state: &Arc<VolumeServerState>,
+) -> master_pb::Heartbeat {
     let store = state.store.read().unwrap();
     let ec_shards = collect_live_ec_shards(&store, true);
 
@@ -1166,10 +1197,10 @@ mod tests {
     use crate::config::MinFreeSpace;
     use crate::config::ReadMode;
     use crate::metrics::{
-        DISK_SIZE_GAUGE, DISK_SIZE_LABEL_DELETED_BYTES, DISK_SIZE_LABEL_EC,
-        DISK_SIZE_LABEL_NORMAL, READ_ONLY_LABEL_IS_DISK_SPACE_LOW,
-        READ_ONLY_LABEL_IS_READ_ONLY, READ_ONLY_LABEL_NO_WRITE_CAN_DELETE,
-        READ_ONLY_LABEL_NO_WRITE_OR_DELETE, READ_ONLY_VOLUME_GAUGE,
+        DISK_SIZE_GAUGE, DISK_SIZE_LABEL_DELETED_BYTES, DISK_SIZE_LABEL_EC, DISK_SIZE_LABEL_NORMAL,
+        READ_ONLY_LABEL_IS_DISK_SPACE_LOW, READ_ONLY_LABEL_IS_READ_ONLY,
+        READ_ONLY_LABEL_NO_WRITE_CAN_DELETE, READ_ONLY_LABEL_NO_WRITE_OR_DELETE,
+        READ_ONLY_VOLUME_GAUGE,
     };
     use crate::remote_storage::s3_tier::S3TierRegistry;
     use crate::security::{Guard, SigningKey};
@@ -1254,7 +1285,10 @@ mod tests {
     fn test_to_grpc_address_explicit_grpc_port() {
         // host:port.grpcPort form — gRPC port is what's after the dot.
         assert_eq!(to_grpc_address("10.85.183.6:5300.6300"), "10.85.183.6:6300");
-        assert_eq!(to_grpc_address("master.local:9333.19333"), "master.local:19333");
+        assert_eq!(
+            to_grpc_address("master.local:9333.19333"),
+            "master.local:19333"
+        );
     }
 
     #[test]
@@ -1308,7 +1342,10 @@ mod tests {
             heartbeat.disk_tags[0].tags,
             vec!["fast".to_string(), "ssd".to_string()]
         );
-        assert_eq!(heartbeat.disk_tags[0].r#type, DiskType::HardDrive.to_string());
+        assert_eq!(
+            heartbeat.disk_tags[0].r#type,
+            DiskType::HardDrive.to_string()
+        );
         assert_eq!(heartbeat.disk_tags[0].max_volume_count, 3);
     }
 
@@ -1347,7 +1384,10 @@ mod tests {
 
         let heartbeat = build_heartbeat(&test_config(), &mut store);
         assert_eq!(heartbeat.disk_tags[0].max_volume_count, 1);
-        assert_eq!(heartbeat.max_volume_counts[&DiskType::HardDrive.to_string()], 1);
+        assert_eq!(
+            heartbeat.max_volume_counts[&DiskType::HardDrive.to_string()],
+            1
+        );
     }
 
     #[test]
@@ -1565,10 +1605,9 @@ mod tests {
         let heartbeat = build_heartbeat(&test_config(), &mut store);
         assert_eq!(heartbeat.volumes.len(), 2);
 
-        let expected = heartbeat
-            .volumes
-            .iter()
-            .fold(0u64, |acc, m| acc ^ crate::storage::volume_report_hash::report_hash(m));
+        let expected = heartbeat.volumes.iter().fold(0u64, |acc, m| {
+            acc ^ crate::storage::volume_report_hash::report_hash(m)
+        });
         assert_eq!(heartbeat.volume_digest, Some(expected));
         assert_ne!(heartbeat.volume_digest, Some(0));
     }
@@ -1663,10 +1702,9 @@ mod tests {
             .iter()
             .flat_map(|family| family.get_metric().to_vec())
             .filter(|metric| {
-                metric
-                    .get_label()
-                    .iter()
-                    .any(|label| label.get_name() == "collection" && label.get_value() == collection)
+                metric.get_label().iter().any(|label| {
+                    label.get_name() == "collection" && label.get_value() == collection
+                })
             })
             .count()
     }
@@ -1800,7 +1838,9 @@ mod tests {
         assert_eq!(heartbeat.ec_shards[0].disk_id, 0);
         assert_eq!(
             heartbeat.ec_shards[0].disk_type,
-            state.store.read().unwrap().locations[0].disk_type.to_string()
+            state.store.read().unwrap().locations[0]
+                .disk_type
+                .to_string()
         );
         assert_eq!(heartbeat.ec_shards[0].ec_index_bits, 1);
         assert_eq!(heartbeat.ec_shards[0].shard_sizes, vec![8]);
@@ -1968,12 +2008,15 @@ mod tests {
             )
             .unwrap();
         let (_, volume) = store.find_volume_mut(VolumeId(71)).unwrap();
-        volume.volume_info.files.push(crate::storage::volume::PbRemoteFile {
-            backend_type: "s3".to_string(),
-            backend_id: "archive".to_string(),
-            key: "volumes/71.dat".to_string(),
-            ..Default::default()
-        });
+        volume
+            .volume_info
+            .files
+            .push(crate::storage::volume::PbRemoteFile {
+                backend_type: "s3".to_string(),
+                backend_id: "archive".to_string(),
+                key: "volumes/71.dat".to_string(),
+                ..Default::default()
+            });
         volume.refresh_remote_write_mode().unwrap();
 
         let heartbeat = build_heartbeat(&test_config(), &mut store);
@@ -2137,8 +2180,7 @@ mod tests {
             .mount_ec_shards(VolumeId(81), "ec_delta_case", &[0], "")
             .unwrap();
         let current = collect_ec_shard_delta_messages(&store);
-        let (new_ec_shards, deleted_ec_shards) =
-            diff_ec_shard_delta_messages(&previous, &current);
+        let (new_ec_shards, deleted_ec_shards) = diff_ec_shard_delta_messages(&previous, &current);
 
         assert_eq!(new_ec_shards.len(), 1);
         assert!(deleted_ec_shards.is_empty());

seaweed-volume/src/storage/erasure_coding/ec_shard.rs

@@ -4,6 +4,7 @@ use std::fs::{self, File, OpenOptions};
 use std::io::{self, Write};
 
 use crate::storage::types::*;
+use crate::storage::volume_open::open_volume_file;
 
 pub const DATA_SHARDS_COUNT: usize = 10;
 pub const PARITY_SHARDS_COUNT: usize = 4;
@@ -50,7 +51,7 @@ impl EcVolumeShard {
     /// Open the shard file for reading.
     pub fn open(&mut self) -> io::Result<()> {
         let path = self.file_name();
-        let file = File::open(&path)?;
+        let file = open_volume_file(OpenOptions::new().read(true), &path)?;
         self.ecd_file_size = file.metadata()?.len() as i64;
         self.ecd_file = Some(file);
         Ok(())
@@ -59,12 +60,14 @@ impl EcVolumeShard {
     /// Create the shard file for writing.
     pub fn create(&mut self) -> io::Result<()> {
         let path = self.file_name();
-        let file = OpenOptions::new()
-            .read(true)
-            .write(true)
-            .create(true)
-            .truncate(true)
-            .open(&path)?;
+        let file = open_volume_file(
+            OpenOptions::new()
+                .read(true)
+                .write(true)
+                .create(true)
+                .truncate(true),
+            &path,
+        )?;
         self.ecd_file = Some(file);
         self.ecd_file_size = 0;
         Ok(())

seaweed-volume/src/storage/erasure_coding/ec_volume.rs

@@ -14,6 +14,7 @@ use crate::storage::erasure_coding::ec_locate;
 use crate::storage::erasure_coding::ec_shard::*;
 use crate::storage::needle::needle::{get_actual_size, Needle, NeedleError};
 use crate::storage::types::*;
+use crate::storage::volume_open::open_volume_file;
 
 /// An erasure-coded volume managing its local shards and index.
 pub struct EcVolume {
@@ -435,7 +436,7 @@ impl EcVolume {
         // Matches Go which opens ecx for writing via MarkNeedleDeleted.
         let ecx_path = vol.ecx_file_name();
         if std::path::Path::new(&ecx_path).exists() {
-            let file = OpenOptions::new().read(true).write(true).open(&ecx_path)?;
+            let file = open_volume_file(OpenOptions::new().read(true).write(true), &ecx_path)?;
             vol.ecx_file_size = file.metadata()?.len() as i64;
             vol.ecx_file = Some(file);
         } else if dir_idx != dir {
@@ -447,7 +448,8 @@ impl EcVolume {
                     volume_id = volume_id.0,
                     "ecx file not found in idx dir, falling back to data dir"
                 );
-                let file = OpenOptions::new().read(true).write(true).open(&fallback_ecx)?;
+                let file =
+                    open_volume_file(OpenOptions::new().read(true).write(true), &fallback_ecx)?;
                 vol.ecx_file_size = file.metadata()?.len() as i64;
                 vol.ecx_file = Some(file);
                 vol.ecx_actual_dir = dir.to_string();
@@ -463,12 +465,14 @@ impl EcVolume {
         let ecj_base =
             crate::storage::volume::volume_file_name(&vol.ecx_actual_dir, collection, volume_id);
         let ecj_path = format!("{}.ecj", ecj_base);
-        let ecj_file = OpenOptions::new()
-            .read(true)
-            .write(true)
-            .create(true)
-            .append(true)
-            .open(&ecj_path)?;
+        let ecj_file = open_volume_file(
+            OpenOptions::new()
+                .read(true)
+                .write(true)
+                .create(true)
+                .append(true),
+            &ecj_path,
+        )?;
         vol.ecj_file_size = ecj_file.metadata()?.len() as i64;
         vol.ecj_file = Some(ecj_file);
 
@@ -1559,12 +1563,14 @@ impl EcVolume {
         // in-memory deleted set (all of its contents are now materialized
         // in .ecx), and reset the cached size.
         fs::remove_file(&ecj_path)?;
-        let ecj_file = OpenOptions::new()
-            .read(true)
-            .write(true)
-            .create(true)
-            .append(true)
-            .open(&ecj_path)?;
+        let ecj_file = open_volume_file(
+            OpenOptions::new()
+                .read(true)
+                .write(true)
+                .create(true)
+                .append(true),
+            &ecj_path,
+        )?;
         self.ecj_file = Some(ecj_file);
         self.ecj_file_size = 0;
         if let Ok(mut set) = self.deleted_needles.write() {

seaweed-volume/src/storage/idx/mod.rs

@@ -6,7 +6,7 @@ use crate::storage::needle::needle::get_actual_size;
 use crate::storage::types::*;
 use std::io::{self, Read, Seek, SeekFrom};
 
-const ROWS_TO_READ: usize = 1024;
+pub(crate) const ROWS_TO_READ: usize = 1024;
 
 /// Walk all entries in an .idx file, calling `f` for each.
 /// Mirrors Go's `WalkIndexFile()`.

seaweed-volume/src/storage/mod.rs

@@ -9,6 +9,8 @@ pub mod store_ec_reconcile;
 pub mod super_block;
 pub mod types;
 pub mod volume;
+pub mod volume_idx_rebuild;
 pub mod volume_idx_repair;
+pub mod volume_open;
 pub mod volume_report;
 pub mod volume_report_hash;

seaweed-volume/src/storage/needle_map/file_pool.rs

@@ -18,6 +18,8 @@ use std::fs::{File, OpenOptions};
 use std::io;
 use std::sync::{Arc, Mutex, OnceLock};
 
+use crate::storage::volume_open::open_volume_file;
+
 /// Descriptors the pool keeps open. Matches Go's `maxPooledIndexFiles`.
 pub const MAX_POOLED_INDEX_FILES: usize = 1024;
 
@@ -68,7 +70,10 @@ impl IndexFilePool {
 
         // Opened outside the lock: a cold open blocks on disk, and holding a
         // process-wide mutex across it would serialize every volume's lookups.
-        let file = Arc::new(OpenOptions::new().read(true).write(writable).open(path)?);
+        let file = Arc::new(open_volume_file(
+            OpenOptions::new().read(true).write(writable),
+            path,
+        )?);
         Ok(self.insert(key, file))
     }
 

seaweed-volume/src/storage/needle_map/idx_metric.rs

@@ -0,0 +1,161 @@
+//! Rebuild a needle map's counters from the whole `.idx` history.
+//!
+//! The volume's garbage ratio is `deleted_size / content_size`, and both are
+//! additive over the life of the volume: an overwritten needle keeps its
+//! bytes in `content_size` and adds them to `deleted_size`. A backend whose
+//! table only holds the final value per key (redb) cannot recover that from
+//! the table, so on load the counters come from the `.idx` file instead.
+//!
+//! This mirrors Go's `needleMapMetricFromIndexFile`: walk the index newest
+//! entry first with a bloom filter of the keys already seen, so the memory
+//! cost is a few bits per entry instead of a map of every key. The counting
+//! rule reproduces what the live `on_put`/`on_delete` path accumulates:
+//!
+//! - every live entry is one put: `file_count`, `file_byte_count`;
+//! - a live entry with a newer entry for the same key was overwritten or
+//!   deleted later, so it is also one deletion: `deletion_count`,
+//!   `deletion_byte_count`;
+//! - a tombstone only marks its key as seen.
+//!
+//! A bloom false positive (0.1%) can only add a spurious deletion, which
+//! over-reports garbage slightly; it never hides any.
+
+use std::io::{self, Read, Seek, SeekFrom};
+use std::sync::atomic::Ordering;
+
+use xxhash_rust::xxh64::xxh64;
+
+use super::NeedleMapMetric;
+use crate::storage::types::*;
+
+/// Entries read per batch while walking backwards (64 KiB of index).
+const BATCH_ENTRIES: usize = 4096;
+/// Same target false-positive rate as the Go server's filter.
+const FALSE_POSITIVE_RATE: f64 = 0.001;
+
+/// Minimal bloom filter over needle ids, double hashing with xxh64.
+struct SeenKeys {
+    bits: Vec<u64>,
+    bit_count: u64,
+    hashes: u64,
+}
+
+impl SeenKeys {
+    fn new(expected: u64, false_positive_rate: f64) -> Self {
+        let n = expected.max(1) as f64;
+        let ln2 = std::f64::consts::LN_2;
+        let bit_count = (-(n * false_positive_rate.ln()) / (ln2 * ln2))
+            .ceil()
+            .max(64.0) as u64;
+        let hashes = ((bit_count as f64 / n) * ln2).round().clamp(1.0, 16.0) as u64;
+        SeenKeys {
+            bits: vec![0u64; bit_count.div_ceil(64) as usize],
+            bit_count,
+            hashes,
+        }
+    }
+
+    /// Whether `key` was (probably) seen before; marks it seen either way.
+    fn test_and_add(&mut self, key: u64) -> bool {
+        let bytes = key.to_le_bytes();
+        let h1 = xxh64(&bytes, 0);
+        let h2 = xxh64(&bytes, 0x9E37_79B9_7F4A_7C15) | 1;
+        let mut seen = true;
+        for i in 0..self.hashes {
+            let bit = h1.wrapping_add(i.wrapping_mul(h2)) % self.bit_count;
+            let word = (bit / 64) as usize;
+            let mask = 1u64 << (bit % 64);
+            if self.bits[word] & mask == 0 {
+                seen = false;
+                self.bits[word] |= mask;
+            }
+        }
+        seen
+    }
+}
+
+/// Walk `reader` (an `.idx` file) newest entry first and return the counters
+/// a live volume would hold after applying the same history. A torn partial
+/// entry at the tail is ignored, as `walk_index_file` does. The reader is
+/// left positioned at the start of the file.
+pub(super) fn metrics_from_idx<R: Read + Seek>(
+    reader: &mut R,
+    version: Version,
+) -> io::Result<NeedleMapMetric> {
+    let metric = NeedleMapMetric::default();
+    let file_size = reader.seek(SeekFrom::End(0))?;
+    let entry_count = file_size / NEEDLE_MAP_ENTRY_SIZE as u64;
+    let mut seen = SeenKeys::new(entry_count, FALSE_POSITIVE_RATE);
+    let mut buf = vec![0u8; NEEDLE_MAP_ENTRY_SIZE * BATCH_ENTRIES];
+
+    let mut remaining = entry_count;
+    while remaining > 0 {
+        let batch = remaining.min(BATCH_ENTRIES as u64) as usize;
+        let first_entry = remaining - batch as u64;
+        let len = batch * NEEDLE_MAP_ENTRY_SIZE;
+        reader.seek(SeekFrom::Start(first_entry * NEEDLE_MAP_ENTRY_SIZE as u64))?;
+        reader.read_exact(&mut buf[..len])?;
+        for i in (0..batch).rev() {
+            let entry = &buf[i * NEEDLE_MAP_ENTRY_SIZE..(i + 1) * NEEDLE_MAP_ENTRY_SIZE];
+            let (key, offset, size) = idx_entry_from_bytes(entry);
+            metric.maybe_set_max_needle_end(offset, size, version);
+            let superseded = seen.test_and_add(key.into());
+            if offset.is_zero() || size.is_deleted() {
+                // Tombstone: reserves no bytes, only marks the key as seen.
+                continue;
+            }
+            metric.maybe_set_max_file_key(key);
+            metric.file_count.fetch_add(1, Ordering::Relaxed);
+            metric
+                .file_byte_count
+                .fetch_add(size.0 as u64, Ordering::Relaxed);
+            if superseded && size.0 > 0 {
+                metric.deletion_count.fetch_add(1, Ordering::Relaxed);
+                metric
+                    .deletion_byte_count
+                    .fetch_add(size.0 as u64, Ordering::Relaxed);
+            }
+        }
+        remaining = first_entry;
+    }
+
+    reader.seek(SeekFrom::Start(0))?;
+    Ok(metric)
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_seen_keys_reports_repeats_and_not_fresh_keys() {
+        let mut seen = SeenKeys::new(10_000, FALSE_POSITIVE_RATE);
+        // Fresh keys may occasionally collide (that is the false-positive
+        // rate), but only rarely.
+        let fresh_reported_seen = (0..10_000u64)
+            .filter(|&key| seen.test_and_add(key))
+            .count();
+        assert!(
+            fresh_reported_seen < 50,
+            "fresh keys reported seen: {fresh_reported_seen}"
+        );
+        // A repeated key is never reported fresh: no false negatives.
+        for key in 0..10_000u64 {
+            assert!(seen.test_and_add(key), "repeated key {key} reported fresh");
+        }
+        // Over 100k never-inserted keys the false-positive rate stays near
+        // the 0.1% target; allow a generous margin.
+        let false_positives = (1_000_000..1_100_000u64)
+            .filter(|k| {
+                let bytes = k.to_le_bytes();
+                let h1 = xxh64(&bytes, 0);
+                let h2 = xxh64(&bytes, 0x9E37_79B9_7F4A_7C15) | 1;
+                (0..seen.hashes).all(|i| {
+                    let bit = h1.wrapping_add(i.wrapping_mul(h2)) % seen.bit_count;
+                    seen.bits[(bit / 64) as usize] & (1u64 << (bit % 64)) != 0
+                })
+            })
+            .count();
+        assert!(false_positives < 500, "false positives: {false_positives}");
+    }
+}

seaweed-volume/src/storage/volume.rs

@@ -19,13 +19,13 @@ use std::time::{SystemTime, UNIX_EPOCH};
 
 use tracing::{error, info, warn};
 
-#[cfg(test)]
 use crate::storage::idx;
 use crate::storage::needle::needle::{self, get_actual_size, Needle, NeedleError};
 use crate::storage::needle_map::sorted_file::SortedFileNeedleMap;
 use crate::storage::needle_map::{CompactNeedleMap, NeedleMap, NeedleMapKind, RedbNeedleMap};
 use crate::storage::super_block::{ReplicaPlacement, SuperBlock, SUPER_BLOCK_SIZE};
 use crate::storage::types::*;
+use crate::storage::volume_open::open_volume_file;
 
 // ============================================================================
 // Errors
@@ -736,12 +736,12 @@ impl Volume {
             let metadata = fs::metadata(&dat_path)?;
 
             // Try to open read-write; fall back to read-only
-            match OpenOptions::new().read(true).write(true).open(&dat_path) {
+            match open_volume_file(OpenOptions::new().read(true).write(true), &dat_path) {
                 Ok(file) => {
                     self.dat_file = Some(file);
                 }
                 Err(e) if e.kind() == io::ErrorKind::PermissionDenied => {
-                    self.dat_file = Some(File::open(&dat_path)?);
+                    self.dat_file = Some(open_volume_file(OpenOptions::new().read(true), &dat_path)?);
                     self.no_write_or_delete = true;
                 }
                 Err(e) => return Err(e.into()),
@@ -762,11 +762,10 @@ impl Volume {
             if let Some(parent) = Path::new(&dat_path).parent() {
                 fs::create_dir_all(parent)?;
             }
-            let file = OpenOptions::new()
-                .read(true)
-                .write(true)
-                .create(true)
-                .open(&dat_path)?;
+            let file = open_volume_file(
+                OpenOptions::new().read(true).write(true).create(true),
+                &dat_path,
+            )?;
             if preallocate > 0 {
                 preallocate_file(&file, preallocate);
             }
@@ -801,6 +800,30 @@ impl Volume {
         }
 
         if also_load_index {
+            // Adjust for existing volumes with .idx together with .dat files:
+            // an index already beside the data keeps serving after --dir.idx
+            // named a different directory.
+            if self.dir_idx != self.dir
+                && Path::new(&format!("{}.idx", self.data_file_name())).exists()
+            {
+                self.dir_idx = self.dir.clone();
+            }
+
+            // A changed --dir.idx leaves the new directory without an index.
+            // The .dat still holds every row, so rebuild rather than mount the
+            // volume with every needle invisible.
+            if !self.has_remote_file
+                && !Path::new(&self.file_name(".idx")).exists()
+                && self.current_dat_file_size()? > SUPER_BLOCK_SIZE as u64
+            {
+                self.rebuild_idx_file()?;
+                info!(
+                    volume_id = self.id.0,
+                    idx = %self.file_name(".idx"),
+                    "rebuilt the index from the data file"
+                );
+            }
+
             // Recover rows that deletes on a tiered read-only volume overwrote
             // at the front of .idx. Best effort: a volume that cannot be
             // repaired is still servable for everything the surviving rows
@@ -831,6 +854,7 @@ impl Volume {
                         "volumeDataIntegrityChecking failed"
                     );
                 }
+                self.recover_last_modified_ts();
 
                 // Structural check: no .idx entry may reference bytes past the
                 // end of .dat. The needle map's load walk above already
@@ -982,7 +1006,7 @@ impl Volume {
         if self.no_write_or_delete {
             // Open read-only
             if Path::new(&idx_path).exists() {
-                let mut idx_file = File::open(&idx_path)?;
+                let mut idx_file = open_volume_file(OpenOptions::new().read(true), idx_path)?;
                 let nm = CompactNeedleMap::load_from_idx(&mut idx_file, self.version())?;
                 self.nm = Some(NeedleMap::InMemory(nm));
             } else {
@@ -1001,11 +1025,10 @@ impl Volume {
             }
         } else {
             // Open read-write (create if missing)
-            let idx_file = OpenOptions::new()
-                .read(true)
-                .write(true)
-                .create(true)
-                .open(&idx_path)?;
+            let idx_file = open_volume_file(
+                OpenOptions::new().read(true).write(true).create(true),
+                idx_path,
+            )?;
 
             let idx_size = trim_torn_idx_tail(&idx_file, idx_path)?;
             let mut idx_reader = io::BufReader::new(&idx_file);
@@ -1027,12 +1050,20 @@ impl Volume {
     fn load_index_redb(&mut self, idx_path: &str) -> Result<(), VolumeError> {
         // The redb database file is stored alongside the volume files
         let rdb_path = self.file_name(".rdb");
+        // One redb database per volume: keep its page cache small, or the
+        // process grows by up to redb's 1 GiB default per volume (#11179).
+        let cache_bytes = self.needle_map_kind.redb_cache_bytes();
 
         if self.no_write_or_delete {
             // Open read-only
             if Path::new(&idx_path).exists() {
-                let mut idx_file = File::open(&idx_path)?;
-                let nm = RedbNeedleMap::load_from_idx(&rdb_path, &mut idx_file, self.version())?;
+                let mut idx_file = open_volume_file(OpenOptions::new().read(true), idx_path)?;
+                let nm = RedbNeedleMap::load_from_idx(
+                    &rdb_path,
+                    &mut idx_file,
+                    self.version(),
+                    cache_bytes,
+                )?;
                 self.nm = Some(NeedleMap::Redb(nm));
             } else {
                 // Missing .idx with existing .dat could orphan needles
@@ -1046,19 +1077,23 @@ impl Volume {
                         );
                     }
                 }
-                self.nm = Some(NeedleMap::Redb(RedbNeedleMap::new(&rdb_path)?));
+                self.nm = Some(NeedleMap::Redb(RedbNeedleMap::new(&rdb_path, cache_bytes)?));
             }
         } else {
             // Open read-write (create if missing)
-            let idx_file = OpenOptions::new()
-                .read(true)
-                .write(true)
-                .create(true)
-                .open(&idx_path)?;
+            let idx_file = open_volume_file(
+                OpenOptions::new().read(true).write(true).create(true),
+                idx_path,
+            )?;
 
             let idx_size = trim_torn_idx_tail(&idx_file, idx_path)?;
             let mut idx_reader = io::BufReader::new(&idx_file);
-            let mut nm = RedbNeedleMap::load_from_idx(&rdb_path, &mut idx_reader, self.version())?;
+            let mut nm = RedbNeedleMap::load_from_idx(
+                &rdb_path,
+                &mut idx_reader,
+                self.version(),
+                cache_bytes,
+            )?;
 
             // Re-open for append-only writes
             let write_file = OpenOptions::new()
@@ -1839,10 +1874,45 @@ impl Volume {
             self.last_modified_ts_seconds = n.last_modified;
         }
 
+        self.maybe_checkpoint_index(fsync);
+
         // Return Size(n.DataSize) as the logical size, matching Go's doWriteRequest
         Ok((offset, Size(n.data_size as i32), false))
     }
 
+    /// Take the index checkpoint the needle map asked for, data first: the
+    /// checkpoint makes the index durable, and an index row that outlives
+    /// the bytes it points at loads read-only (see load()). A failed .dat
+    /// flush therefore skips the checkpoint; the map keeps asking, so it is
+    /// retried on the next write.
+    ///
+    /// When `idx_already_synced` is true the .idx has already been fsynced by
+    /// `flush_idx` on the fsync=true write path, so the checkpoint skips its
+    /// own .idx fsync to avoid a redundant one.
+    fn maybe_checkpoint_index(&mut self, idx_already_synced: bool) {
+        let due = self.nm.as_ref().is_some_and(|nm| nm.checkpoint_due());
+        if !due {
+            return;
+        }
+        if let Err(e) = self.flush_dat() {
+            self.check_read_write_error(Some(&e));
+            tracing::warn!(
+                "volume {}: skipping index checkpoint, .dat flush failed: {}",
+                self.id.0,
+                e
+            );
+            return;
+        }
+        let checkpointed = match self.nm.as_mut() {
+            Some(nm) => nm.checkpoint(!idx_already_synced),
+            None => Ok(()),
+        };
+        if let Err(e) = checkpointed {
+            self.check_read_write_error(Some(&e));
+            tracing::warn!("volume {}: index checkpoint failed: {}", self.id.0, e);
+        }
+    }
+
     fn read_needle_header_unlocked(&self, n: &mut Needle, offset: i64) -> Result<(), VolumeError> {
         let mut header = [0u8; NEEDLE_HEADER_SIZE];
         self.read_exact_at_backend(&mut header, offset as u64)?;
@@ -1979,6 +2049,7 @@ impl Volume {
         if let Some(nm) = &mut self.nm {
             nm.delete(n.id, Offset::from_actual_offset(offset as i64))?;
         }
+        self.maybe_checkpoint_index(false);
 
         Ok(size)
     }
@@ -2047,6 +2118,26 @@ impl Volume {
         (ttl_minutes as u64) < lived_minutes
     }
 
+    /// When this volume's data becomes garbage, counted from its last write.
+    /// Counting from the current time instead let every .vif rewrite — a
+    /// read-only mark, a tier upload, an EC encode — hand an already expiring
+    /// volume another full TTL. Zero when the volume has no TTL. Mirrors Go's
+    /// ExpireAtSec.
+    pub fn expire_at_sec(&self) -> u64 {
+        let ttl_seconds = self.super_block.ttl.to_seconds();
+        if ttl_seconds == 0 {
+            return 0;
+        }
+        let mut last_write_sec = self.last_modified_ts_seconds;
+        if last_write_sec == 0 {
+            last_write_sec = SystemTime::now()
+                .duration_since(UNIX_EPOCH)
+                .unwrap_or_default()
+                .as_secs();
+        }
+        last_write_sec + ttl_seconds
+    }
+
     pub fn is_expired_long_enough(&self, max_delay_minutes: u32) -> bool {
         let ttl_minutes = self.super_block.ttl.minutes();
         if ttl_minutes == 0 {
@@ -2176,6 +2267,128 @@ impl Volume {
         }
     }
 
+    /// Bounds the work a vacuumed volume's recovery does, where key order makes
+    /// every write a candidate for the newest one. A volume with more live
+    /// needles than this keeps the .dat mtime: reading a subset could recover a
+    /// timestamp older than the newest write and expire data still inside its
+    /// TTL, so a scan that will not fit declines instead of guessing.
+    const VACUUMED_LAST_WRITE_SCAN_ENTRIES: usize = 1 << 16;
+
+    /// Point the TTL clock at the newest write recorded in the volume, replacing
+    /// the .dat mtime the loader starts from. A delete appends a tombstone and
+    /// vacuum rewrites the .dat wholesale, so the mtime moves without any write
+    /// ever landing: every restart of a volume taking delete traffic re-armed
+    /// is_expired() for another full TTL and the volume was never reclaimed.
+    /// Mirrors Go's recoverLastModifiedTs.
+    fn recover_last_modified_ts(&mut self) {
+        if self.super_block.ttl.minutes() == 0 {
+            return;
+        }
+        match self.find_last_write_append_at_ns() {
+            Ok(0) => {}
+            Ok(append_at_ns) => self.last_modified_ts_seconds = append_at_ns / 1_000_000_000,
+            Err(e) => warn!(
+                volume_id = self.id.0,
+                error = %e,
+                "recover the last write from the index"
+            ),
+        }
+    }
+
+    /// Scan the .idx backwards for the newest write — an entry that is not a
+    /// deletion tombstone — and return that needle's append timestamp. The .idx
+    /// and the .dat share an order, so an append-ordered volume answers with the
+    /// first write the scan reaches. Vacuum rewrites both in key order, which
+    /// tracks write order only because the master issues keys increasing: an
+    /// overwrite keeps its original, lower key, so a vacuumed volume has to take
+    /// the maximum over every write it indexes. Returns 0 when the .idx holds
+    /// nothing but tombstones, when a vacuumed volume holds more needles than
+    /// the scan budget, or for a volume older than version 3, whose needles
+    /// carry no append timestamp. Mirrors Go's findLastWriteAppendAtNs.
+    fn find_last_write_append_at_ns(&self) -> Result<u64, VolumeError> {
+        let version = self.version();
+        if version != VERSION_3 {
+            return Ok(0);
+        }
+        let idx_path = self.file_name(".idx");
+        let idx_size = fs::metadata(&idx_path).map(|m| m.len()).unwrap_or(0) as i64;
+        if idx_size == 0 || idx_size % NEEDLE_MAP_ENTRY_SIZE as i64 != 0 {
+            return Ok(0);
+        }
+        let scan_every_write = self.super_block.compaction_revision > 0;
+        let mut entry_budget = Self::VACUUMED_LAST_WRITE_SCAN_ENTRIES;
+        let mut last_write_append_at_ns = 0u64;
+        let mut idx_file = File::open(&idx_path)?;
+        let mut block = vec![0u8; NEEDLE_MAP_ENTRY_SIZE * idx::ROWS_TO_READ];
+        let mut end = idx_size;
+        while end > 0 {
+            let start = (end - block.len() as i64).max(0);
+            let entries = &mut block[..(end - start) as usize];
+            idx_file.seek(SeekFrom::Start(start as u64))?;
+            idx_file.read_exact(entries)?;
+            for entry in entries.chunks_exact(NEEDLE_MAP_ENTRY_SIZE).rev() {
+                let (key, offset, size) = idx_entry_from_bytes(entry);
+                if offset.is_zero() || size.is_deleted() {
+                    continue;
+                }
+                let Some(needle_offset) =
+                    self.find_needle_offset(offset.to_actual_offset(), key, size)
+                else {
+                    continue;
+                };
+                last_write_append_at_ns = last_write_append_at_ns
+                    .max(self.read_needle_append_at_ns(needle_offset, size)?);
+                if !scan_every_write {
+                    return Ok(last_write_append_at_ns);
+                }
+                entry_budget -= 1;
+                if entry_budget == 0 {
+                    warn!(
+                        volume_id = self.id.0,
+                        budget = Self::VACUUMED_LAST_WRITE_SCAN_ENTRIES,
+                        "too many needles to scan for the last write, keeping the .dat mtime"
+                    );
+                    return Ok(0);
+                }
+            }
+            end = start;
+        }
+        Ok(last_write_append_at_ns)
+    }
+
+    /// The .dat offset holding the needle an .idx entry describes, or None when
+    /// no needle there matches it. A .dat past MAX_POSSIBLE_VOLUME_SIZE wraps the
+    /// offsets in its .idx, so the needle can sit one volume size further in;
+    /// verify_needle_integrity retries the same way. Mirrors Go's
+    /// findNeedleOffset.
+    fn find_needle_offset(&self, actual_offset: i64, key: NeedleId, size: Size) -> Option<i64> {
+        for at in [
+            actual_offset,
+            actual_offset + MAX_POSSIBLE_VOLUME_SIZE as i64,
+        ] {
+            let mut header = [0u8; NEEDLE_HEADER_SIZE];
+            if self.read_exact_at_backend(&mut header, at as u64).is_err() {
+                continue;
+            }
+            let (_, needle_id, needle_size) = Needle::parse_header(&header);
+            if needle_id == key && needle_size == size {
+                return Some(at);
+            }
+        }
+        None
+    }
+
+    /// Read the append timestamp a version 3 needle carries past its checksum.
+    fn read_needle_append_at_ns(&self, actual_offset: i64, size: Size) -> Result<u64, VolumeError> {
+        let ts_offset = actual_offset as u64
+            + NEEDLE_HEADER_SIZE as u64
+            + size.0 as u64
+            + NEEDLE_CHECKSUM_SIZE as u64;
+        let mut ts = [0u8; TIMESTAMP_SIZE];
+        self.read_exact_at_backend(&mut ts, ts_offset)?;
+        Ok(u64::from_be_bytes(ts))
+    }
+
     /// .idx file position of the entry whose needle is physically last in the
     /// .dat (highest offset). The common case — an append-ordered .idx — is
     /// resolved in O(1): the last entry's on-disk end equals the .dat size. A
@@ -2275,10 +2488,7 @@ impl Volume {
             return Ok(());
         }
 
-        let ts_offset = checked_offset as u64 + NEEDLE_HEADER_SIZE as u64 + size.0 as u64 + 4; // skip checksum
-        let mut ts_buf = [0u8; 8];
-        self.read_exact_at_backend(&mut ts_buf, ts_offset)?;
-        let ts = u64::from_be_bytes(ts_buf);
+        let ts = self.read_needle_append_at_ns(checked_offset, size)?;
         if ts > 0 {
             self.last_append_at_ns = ts;
         }
@@ -2552,6 +2762,7 @@ impl Volume {
         if let Some(ref mut nm) = self.nm {
             nm.put(key, offset, size).map_err(VolumeError::Io)?;
         }
+        self.maybe_checkpoint_index(false);
         Ok(())
     }
 
@@ -2696,10 +2907,7 @@ impl Volume {
     /// Open the local .dat as the data backend, dropping any remote backend, so reads
     /// are served from local disk. Mirrors Go's swapToLocalDatBackend after a tier-down.
     pub(crate) fn open_local_dat_backend(&mut self) -> Result<(), VolumeError> {
-        let file = OpenOptions::new()
-            .read(true)
-            .write(true)
-            .open(self.dat_path())?;
+        let file = open_volume_file(OpenOptions::new().read(true).write(true), self.dat_path())?;
         self.remote_dat_file = None;
         self.dat_file = Some(file);
         Ok(())
@@ -2816,13 +3024,9 @@ impl Volume {
         let mut vif = VifVolumeInfo::from_pb(&self.volume_info);
 
         // Match Go's SaveVolumeInfo: compute ExpireAtSec from TTL
-        let ttl_seconds = self.super_block.ttl.to_seconds();
-        if ttl_seconds > 0 {
-            let now = SystemTime::now()
-                .duration_since(UNIX_EPOCH)
-                .unwrap_or_default()
-                .as_secs();
-            vif.expire_at_sec = now + ttl_seconds;
+        let expire_at_sec = self.expire_at_sec();
+        if expire_at_sec > 0 {
+            vif.expire_at_sec = expire_at_sec;
         }
 
         let content = serde_json::to_string_pretty(&vif)
@@ -2840,13 +3044,9 @@ impl Volume {
         self.volume_info.read_only_can_delete = marked_can_delete;
 
         // Compute ExpireAtSec from TTL (matches Go's SaveVolumeInfo)
-        let ttl_seconds = self.super_block.ttl.to_seconds();
-        if ttl_seconds > 0 {
-            let now = SystemTime::now()
-                .duration_since(UNIX_EPOCH)
-                .unwrap_or_default()
-                .as_secs();
-            self.volume_info.expire_at_sec = now + ttl_seconds;
+        let expire_at_sec = self.expire_at_sec();
+        if expire_at_sec > 0 {
+            self.volume_info.expire_at_sec = expire_at_sec;
         }
 
         let vif = VifVolumeInfo::from_pb(&self.volume_info);
@@ -3197,6 +3397,7 @@ impl Volume {
         if let Some(ref mut nm) = self.nm {
             nm.put(needle_id, offset, size)?;
         }
+        self.maybe_checkpoint_index(false);
 
         Ok(())
     }
@@ -3804,13 +4005,29 @@ impl Volume {
     }
 
     pub fn close(&mut self) {
-        if let Some(ref dat_file) = self.dat_file {
-            let _ = dat_file.sync_all();
-        }
+        let dat_synced = if let Some(ref dat_file) = self.dat_file {
+            dat_file.sync_all().is_ok()
+        } else {
+            true
+        };
         self.dat_file = None;
         self.remote_dat_file = None;
-        if let Some(ref nm) = self.nm {
-            let _ = nm.sync();
+        // When the .dat flushed, checkpoint the index so the next load starts
+        // from the recorded .idx size. When it did not, skip the checkpoint:
+        // a durable index pointing past an unflushed .dat tail would make the
+        // reload's max_needle_end check mark the volume read-only. Without the
+        // checkpoint, META_IDX_SIZE stays at the last successful one and the
+        // reload replays the uncheckpointed tail (redb flushes on drop).
+        if let Some(ref mut nm) = self.nm {
+            if dat_synced {
+                nm.close();
+            } else {
+                tracing::warn!(
+                    "volume {}: .dat sync failed on close, skipping index checkpoint",
+                    self.id.0
+                );
+                nm.close_without_checkpoint();
+            }
         }
         self.nm = None;
     }
@@ -4000,7 +4217,7 @@ impl Volume {
 /// Byte offset just past the needle's on-disk record. Deletion tombstones
 /// carry TombstoneFileSize (-1) in the .idx but are written with DataSize=0,
 /// so their on-disk record is sized as 0. Mirrors Go's needleDiskEnd.
-fn needle_disk_end(offset: Offset, size: Size, version: Version) -> i64 {
+pub(crate) fn needle_disk_end(offset: Offset, size: Size, version: Version) -> i64 {
     let on_disk_size = if size.is_deleted() { Size(0) } else { size };
     offset.to_actual_offset() + get_actual_size(on_disk_size, version)
 }
@@ -4171,6 +4388,11 @@ pub fn scan_volume_file(
         if size.0 == 0 && _id.is_empty() {
             break; // end of valid data
         }
+        // A negative size is a corrupt header, and body_length would advance the
+        // walk backwards from it. Go's scanners stop here by returning io.EOF.
+        if size.0 < 0 {
+            break;
+        }
 
         let body_length = needle::needle_body_length(size, version);
         let total_size = NEEDLE_HEADER_SIZE as i64 + body_length;
@@ -4751,6 +4973,165 @@ mod tests {
         );
     }
 
+    // Reproduce the delete-traffic TTL bug: deletes append a tombstone to the
+    // .dat, which moves the file's mtime, and the loader read the TTL clock
+    // back from that mtime. A volume taking delete traffic therefore had
+    // is_expired() re-armed for another full TTL on every restart and was
+    // never reclaimed.
+    #[test]
+    fn test_ttl_clock_survives_deletes() {
+        let tmp = TempDir::new().unwrap();
+        let dir = tmp.path().to_str().unwrap();
+        let ttl = crate::storage::needle::ttl::TTL::read("5m").unwrap();
+        let last_write_ns = (SystemTime::now()
+            .duration_since(UNIX_EPOCH)
+            .unwrap()
+            .as_secs()
+            - 2 * 60 * 60)
+            * 1_000_000_000;
+
+        let content_size = {
+            let mut v = make_ttl_volume(dir, ttl);
+            let mut written = Vec::new();
+            for i in 1..=3u64 {
+                let data = format!("data {}", i);
+                let mut n = Needle {
+                    id: NeedleId(i),
+                    cookie: Cookie(i as u32),
+                    data: data.as_bytes().to_vec(),
+                    data_size: data.len() as u32,
+                    ..Needle::default()
+                };
+                let (offset, _, _) = v.write_needle(&mut n, true, false).unwrap();
+                written.push((offset, n.size));
+            }
+            // More than one tombstone: the scan has to walk back over the whole
+            // run of them to reach a write.
+            for i in [2u64, 3] {
+                v.delete_needle(&mut Needle {
+                    id: NeedleId(i),
+                    cookie: Cookie(i as u32),
+                    ..Needle::default()
+                })
+                .unwrap();
+            }
+            v.sync_to_disk().unwrap();
+            // Backdate the writes on disk so the last one sits well outside the
+            // TTL, while the tombstones leave the .dat mtime at now.
+            for (offset, size) in written {
+                backdate_append_at_ns(&v.dat_path(), offset, size, last_write_ns);
+            }
+            v.content_size()
+        };
+
+        let v = make_ttl_volume(dir, ttl);
+        assert_eq!(v.last_modified_ts(), last_write_ns / 1_000_000_000);
+        assert!(
+            v.is_expired(content_size, 1024 * 1024),
+            "a TTL volume whose last write is 2h old must be expired after a reload"
+        );
+    }
+
+    // The one layout where a .dat's order does not track its write order:
+    // vacuum rewrites it by key, and an overwrite keeps its original, lower key.
+    // Reading the position rather than the timestamps would recover the
+    // highest-key needle's older write time and expire the volume before the
+    // overwrite has lived out its TTL.
+    #[test]
+    fn test_ttl_clock_after_vacuum_takes_newest_write() {
+        let tmp = TempDir::new().unwrap();
+        let dir = tmp.path().to_str().unwrap();
+        let ttl = crate::storage::needle::ttl::TTL::read("5m").unwrap();
+        let now = SystemTime::now()
+            .duration_since(UNIX_EPOCH)
+            .unwrap()
+            .as_secs();
+        let old_write_ns = (now - 2 * 60 * 60) * 1_000_000_000;
+        let new_write_ns = (now - 60) * 1_000_000_000;
+
+        let content_size = {
+            let mut v = make_ttl_volume(dir, ttl);
+            // Needle 1 is overwritten last but sorts first, so vacuum leaves it
+            // at the head of the .dat with the newest timestamp of the three.
+            for (id, append_at_ns) in [(2u64, old_write_ns), (3, old_write_ns), (1, new_write_ns)] {
+                let data = format!("data {}", id);
+                let mut n = Needle {
+                    id: NeedleId(id),
+                    cookie: Cookie(id as u32),
+                    data: data.as_bytes().to_vec(),
+                    data_size: data.len() as u32,
+                    ..Needle::default()
+                };
+                let (offset, _, _) = v.write_needle(&mut n, true, false).unwrap();
+                v.sync_to_disk().unwrap();
+                backdate_append_at_ns(&v.dat_path(), offset, n.size, append_at_ns);
+            }
+            v.compact_by_index(0, 0, |_| true).unwrap();
+            v.commit_compact().unwrap();
+            v.content_size()
+        };
+
+        let v = make_ttl_volume(dir, ttl);
+        assert_eq!(v.last_modified_ts(), new_write_ns / 1_000_000_000);
+        assert!(
+            !v.is_expired(content_size, 1024 * 1024),
+            "a volume overwritten a minute ago must not be expired after a vacuum and reload"
+        );
+    }
+
+    // Guard the destroy time an EC volume is reclaimed on: it was recomputed as
+    // now+TTL every time the .vif was written, so a read-only mark, a tier
+    // upload or an EC encode handed an already expiring volume another full TTL.
+    #[test]
+    fn test_expire_at_sec_counts_from_last_write() {
+        let tmp = TempDir::new().unwrap();
+        let dir = tmp.path().to_str().unwrap();
+        let ttl = crate::storage::needle::ttl::TTL::read("5m").unwrap();
+        let now = SystemTime::now()
+            .duration_since(UNIX_EPOCH)
+            .unwrap()
+            .as_secs();
+
+        let mut v = make_ttl_volume(dir, ttl);
+        // A volume with nothing written yet has no last write to count from, and
+        // must not land in 1970 with its data due for destruction on sight.
+        assert!(v.expire_at_sec() >= now);
+
+        v.set_last_modified_ts_for_test(now - 3600);
+        let want = now - 3600 + ttl.to_seconds();
+        for pass in 0..2 {
+            v.save_volume_info().unwrap();
+            assert_eq!(
+                v.volume_info.expire_at_sec, want,
+                ".vif save {} moved the destroy time off the last write",
+                pass
+            );
+        }
+    }
+
+    fn make_ttl_volume(dir: &str, ttl: crate::storage::needle::ttl::TTL) -> Volume {
+        Volume::new(
+            dir,
+            dir,
+            "",
+            VolumeId(1),
+            NeedleMapKind::InMemory,
+            None,
+            Some(ttl),
+            0,
+            Version::current(),
+        )
+        .unwrap()
+    }
+
+    fn backdate_append_at_ns(dat_path: &str, offset: u64, size: Size, append_at_ns: u64) {
+        let ts_offset =
+            offset + NEEDLE_HEADER_SIZE as u64 + size.0 as u64 + NEEDLE_CHECKSUM_SIZE as u64;
+        let mut f = OpenOptions::new().write(true).open(dat_path).unwrap();
+        f.seek(SeekFrom::Start(ts_offset)).unwrap();
+        f.write_all(&append_at_ns.to_be_bytes()).unwrap();
+    }
+
     fn write_three_needles(dir: &str) {
         let mut v = make_test_volume(dir);
         for i in 1..=3u64 {
@@ -5180,6 +5561,106 @@ mod tests {
         assert_eq!(std::str::from_utf8(&n.data).unwrap(), "data 2");
     }
 
+    #[test]
+    fn test_redb_volume_close_then_reload_keeps_counters_exact() {
+        let tmp = TempDir::new().unwrap();
+        let dir = tmp.path().to_str().unwrap();
+        let open = || {
+            Volume::new(
+                dir,
+                dir,
+                "",
+                VolumeId(1),
+                NeedleMapKind::Redb,
+                None,
+                None,
+                0,
+                Version::current(),
+            )
+            .unwrap()
+        };
+
+        {
+            let mut v = open();
+            for i in 1..=3 {
+                let data = format!("data {}", i);
+                let mut n = Needle {
+                    id: NeedleId(i),
+                    cookie: Cookie(i as u32),
+                    data: data.as_bytes().to_vec(),
+                    data_size: data.len() as u32,
+                    ..Needle::default()
+                };
+                v.write_needle(&mut n, true, false).unwrap();
+            }
+            // Volume::close is the shutdown path (store -> disk location ->
+            // volume). It must checkpoint the redb index, or the reload
+            // replays rows the table already holds and inflates the counters.
+            v.close();
+        }
+
+        let v = open();
+        assert_eq!(v.file_count(), 3);
+        assert_eq!(v.deleted_count(), 0);
+        let mut n = Needle {
+            id: NeedleId(2),
+            ..Needle::default()
+        };
+        v.read_needle(&mut n).unwrap();
+        assert_eq!(std::str::from_utf8(&n.data).unwrap(), "data 2");
+    }
+
+    #[test]
+    fn test_redb_volume_checkpoint_flushes_dat_before_index() {
+        use crate::storage::needle_map::test_support::durable_idx_size;
+
+        let tmp = TempDir::new().unwrap();
+        let dir = tmp.path().to_str().unwrap();
+        let mut v = Volume::new(
+            dir,
+            dir,
+            "",
+            VolumeId(1),
+            NeedleMapKind::Redb,
+            None,
+            None,
+            0,
+            Version::current(),
+        )
+        .unwrap();
+        let rdb_path = std::path::PathBuf::from(v.file_name(".rdb"));
+        let write = |v: &mut Volume, i: u64| {
+            let mut n = Needle {
+                id: NeedleId(i),
+                cookie: Cookie(1),
+                data: b"x".to_vec(),
+                data_size: 1,
+                ..Needle::default()
+            };
+            // fsync=false: the .dat append is not flushed by the write itself.
+            v.write_needle(&mut n, true, false).unwrap();
+        };
+        for i in 1..1000 {
+            write(&mut v, i);
+        }
+        assert_eq!(durable_idx_size(&rdb_path), None);
+
+        // The 1000th write makes an index checkpoint due. A checkpoint makes
+        // the index durable, so the volume has to flush the .dat first, and
+        // when that flush fails nothing may be checkpointed: a durable index
+        // row pointing past the end of an unflushed .dat loads read-only.
+        v.fail_next_fsync_for_test(true);
+        write(&mut v, 1000);
+        assert_eq!(durable_idx_size(&rdb_path), None);
+
+        v.fail_next_fsync_for_test(false);
+        write(&mut v, 1001);
+        assert_eq!(
+            durable_idx_size(&rdb_path),
+            Some(1001 * NEEDLE_MAP_ENTRY_SIZE as u64)
+        );
+    }
+
     #[test]
     fn test_relocate_index_to_moves_index_and_serves_reads() {
         let root = TempDir::new().unwrap();
@@ -5244,6 +5725,71 @@ mod tests {
         assert!(Path::new(&idx_dir_idx).exists());
     }
 
+    #[test]
+    fn test_load_keeps_index_co_located_with_the_data() {
+        let root = TempDir::new().unwrap();
+        let data_dir = root.path().join("data");
+        let idx_dir = root.path().join("idx");
+        fs::create_dir_all(&data_dir).unwrap();
+        fs::create_dir_all(&idx_dir).unwrap();
+        let data = data_dir.to_str().unwrap();
+        let idx = idx_dir.to_str().unwrap();
+
+        let mut v = Volume::new(
+            data,
+            data,
+            "",
+            VolumeId(7),
+            NeedleMapKind::InMemory,
+            None,
+            None,
+            0,
+            Version::current(),
+        )
+        .unwrap();
+        let payload = b"payload-beside-the-data".to_vec();
+        let mut n = Needle {
+            id: NeedleId(42),
+            cookie: Cookie(0x55),
+            data: payload.clone(),
+            data_size: payload.len() as u32,
+            ..Needle::default()
+        };
+        v.write_needle(&mut n, true, false).unwrap();
+        v.sync_to_disk().unwrap();
+        drop(v);
+
+        // --dir.idx now names an empty directory: the index already beside the
+        // data keeps serving, and nothing lands in the new directory.
+        let reopened = Volume::new(
+            data,
+            idx,
+            "",
+            VolumeId(7),
+            NeedleMapKind::InMemory,
+            None,
+            None,
+            0,
+            Version::current(),
+        )
+        .unwrap();
+        assert!(
+            Path::new(&format!("{data}/7.idx")).exists(),
+            "index stays with the data"
+        );
+        assert!(
+            !Path::new(&format!("{idx}/7.idx")).exists(),
+            "nothing written to the new idx dir"
+        );
+
+        let mut got = Needle {
+            id: NeedleId(42),
+            ..Needle::default()
+        };
+        reopened.read_needle(&mut got).unwrap();
+        assert_eq!(got.data, payload);
+    }
+
     #[test]
     fn test_relocate_index_to_noop_when_already_in_place() {
         let tmp = TempDir::new().unwrap();

seaweed-volume/src/storage/volume_idx_rebuild.rs

@@ -0,0 +1,376 @@
+//! Rebuild a missing .idx from the .dat it indexes. Mirrors
+//! `weed/storage/volume_idx_rebuild.go`.
+
+use std::fs::{self, OpenOptions};
+use std::io::{BufWriter, Write};
+use std::path::Path;
+
+use crate::storage::idx;
+use crate::storage::needle::Needle;
+use crate::storage::super_block::SuperBlock;
+use crate::storage::types::*;
+use crate::storage::volume::{
+    fsync_dir, needle_disk_end, scan_volume_file, Volume, VolumeError, VolumeFileVisitor,
+};
+
+/// Writes one .idx row per .dat record, in .dat append order, which is the
+/// shape the volume server's own writes leave behind.
+struct VolumeFileScanner4RebuildIdx<W: Write> {
+    writer: W,
+    dat_size: i64,
+    version: Version,
+    stopped: bool,
+}
+
+impl<W: Write> VolumeFileVisitor for VolumeFileScanner4RebuildIdx<W> {
+    fn visit_super_block(&mut self, _sb: &SuperBlock) -> Result<(), VolumeError> {
+        Ok(())
+    }
+
+    fn read_needle_body(&self) -> bool {
+        false
+    }
+
+    fn visit_needle(&mut self, n: &Needle, offset: i64) -> Result<(), VolumeError> {
+        // A record reaching past the end of .dat is a torn append or a corrupt
+        // header: nothing beyond it is indexable, and a row pointing past EOF
+        // would fail every read of that needle. The all-zero header case ends
+        // the walk upstream.
+        if self.stopped {
+            return Ok(());
+        }
+        if needle_disk_end(Offset::from_actual_offset(offset), n.size, self.version) > self.dat_size
+        {
+            self.stopped = true;
+            return Ok(());
+        }
+        let size = if n.size.is_valid() {
+            n.size
+        } else {
+            TOMBSTONE_FILE_SIZE
+        };
+        idx::write_index_entry(
+            &mut self.writer,
+            n.id,
+            Offset::from_actual_offset(offset),
+            size,
+        )?;
+        Ok(())
+    }
+}
+
+impl Volume {
+    /// Regenerate the volume's .idx from its .dat. The whole index is derivable
+    /// from the data file, so a volume whose index directory has no .idx -- a
+    /// --dir.idx pointed at an empty directory, or a lost index -- comes back on
+    /// its own instead of mounting with every needle invisible. The rows go to a
+    /// temp file that is renamed in, so an interrupted rebuild leaves no partial
+    /// index behind.
+    pub(crate) fn rebuild_idx_file(&self) -> Result<(), VolumeError> {
+        let idx_path = self.file_name(".idx");
+        let dat_path = self.file_name(".dat");
+        let tmp_path = format!("{idx_path}.tmp");
+
+        let rebuild = || -> Result<(), VolumeError> {
+            if let Some(parent) = Path::new(&idx_path).parent() {
+                fs::create_dir_all(parent)?;
+            }
+            let tmp_file = OpenOptions::new()
+                .write(true)
+                .create(true)
+                .truncate(true)
+                .open(&tmp_path)?;
+            let mut scanner = VolumeFileScanner4RebuildIdx {
+                writer: BufWriter::new(&tmp_file),
+                dat_size: fs::metadata(&dat_path)?.len() as i64,
+                version: self.version(),
+                stopped: false,
+            };
+            scan_volume_file(&dat_path, &mut scanner)?;
+            scanner.writer.flush()?;
+            drop(scanner);
+            tmp_file.sync_all()?;
+            fs::rename(&tmp_path, &idx_path)?;
+            fsync_dir(&idx_path)?;
+            Ok(())
+        };
+
+        let result = rebuild();
+        if result.is_err() {
+            let _ = fs::remove_file(&tmp_path);
+        }
+        result
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use crate::storage::needle::crc::CRC;
+    use crate::storage::needle::Needle;
+    use crate::storage::needle_map::NeedleMapKind;
+    use crate::storage::types::*;
+    use crate::storage::volume::Volume;
+    use std::fs;
+    use std::path::Path;
+    use tempfile::TempDir;
+
+    fn needle(id: u64) -> Needle {
+        let data = format!("payload-{id}").into_bytes();
+        Needle {
+            id: NeedleId(id),
+            cookie: Cookie(0x55),
+            data_size: data.len() as u32,
+            checksum: CRC::new(&data),
+            data,
+            ..Needle::default()
+        }
+    }
+
+    // Pointing --dir.idx at a directory with no .idx used to mount the volume on
+    // an empty index; the index is derivable from the .dat, so it must be
+    // rebuilt in place instead.
+    #[test]
+    fn test_load_moved_idx_directory_rebuilds_idx() {
+        let root = TempDir::new().unwrap();
+        let data_dir = root.path().join("data");
+        let old_idx_dir = root.path().join("idxA");
+        let new_idx_dir = root.path().join("idxB");
+        for dir in [&data_dir, &old_idx_dir, &new_idx_dir] {
+            fs::create_dir_all(dir).unwrap();
+        }
+        let data = data_dir.to_str().unwrap();
+        let old_idx = old_idx_dir.to_str().unwrap();
+        let new_idx = new_idx_dir.to_str().unwrap();
+
+        let mut v = Volume::new(
+            data,
+            old_idx,
+            "",
+            VolumeId(1),
+            NeedleMapKind::InMemory,
+            None,
+            None,
+            0,
+            Version::current(),
+        )
+        .unwrap();
+        for id in 1..=3 {
+            v.write_needle(&mut needle(id), true, false).unwrap();
+        }
+        v.delete_needle(&mut needle(2)).unwrap();
+        v.sync_to_disk().unwrap();
+        let (want_count, want_deleted) = (v.file_count(), v.deleted_count());
+        drop(v);
+
+        let seeded = fs::read(format!("{old_idx}/1.idx")).unwrap();
+
+        let reopened = Volume::new(
+            data,
+            new_idx,
+            "",
+            VolumeId(1),
+            NeedleMapKind::InMemory,
+            None,
+            None,
+            0,
+            Version::current(),
+        )
+        .unwrap();
+
+        assert!(
+            Path::new(&format!("{new_idx}/1.idx")).exists(),
+            "idx not rebuilt in the new idx dir"
+        );
+        assert_eq!(reopened.file_count(), want_count);
+        assert_eq!(reopened.deleted_count(), want_deleted);
+        assert_eq!(
+            fs::read(format!("{new_idx}/1.idx")).unwrap(),
+            seeded,
+            "rebuilt idx differs from the one the server wrote"
+        );
+
+        for id in [1, 3] {
+            let mut got = needle(id);
+            got.data.clear();
+            reopened.read_needle(&mut got).unwrap();
+            assert_eq!(got.data, format!("payload-{id}").into_bytes());
+        }
+    }
+
+    // A .dat padded with zeros must not be indexed as needle 0 rows: the walk
+    // stops where the records do.
+    #[test]
+    fn test_rebuild_idx_stops_at_zero_padded_dat_tail() {
+        let root = TempDir::new().unwrap();
+        let dir = root.path().to_str().unwrap();
+
+        let mut v = Volume::new(
+            dir,
+            dir,
+            "",
+            VolumeId(1),
+            NeedleMapKind::InMemory,
+            None,
+            None,
+            0,
+            Version::current(),
+        )
+        .unwrap();
+        v.write_needle(&mut needle(1), true, false).unwrap();
+        v.sync_to_disk().unwrap();
+        drop(v);
+
+        let seeded = fs::read(format!("{dir}/1.idx")).unwrap();
+        let dat = fs::OpenOptions::new()
+            .write(true)
+            .open(format!("{dir}/1.dat"))
+            .unwrap();
+        let dat_size = dat.metadata().unwrap().len();
+        dat.set_len(dat_size + 4096).unwrap();
+        drop(dat);
+        fs::remove_file(format!("{dir}/1.idx")).unwrap();
+
+        let reopened = Volume::new(
+            dir,
+            dir,
+            "",
+            VolumeId(1),
+            NeedleMapKind::InMemory,
+            None,
+            None,
+            0,
+            Version::current(),
+        )
+        .unwrap();
+        drop(reopened);
+
+        assert_eq!(
+            fs::read(format!("{dir}/1.idx")).unwrap(),
+            seeded,
+            "the zero-padded tail leaked into the rebuilt idx"
+        );
+    }
+
+    // A .dat whose last append was torn mid-body must not gain an index row
+    // that points past the end of the file.
+    #[test]
+    fn test_rebuild_idx_skips_truncated_dat_tail() {
+        let root = TempDir::new().unwrap();
+        let dir = root.path().to_str().unwrap();
+
+        let mut v = Volume::new(
+            dir,
+            dir,
+            "",
+            VolumeId(1),
+            NeedleMapKind::InMemory,
+            None,
+            None,
+            0,
+            Version::current(),
+        )
+        .unwrap();
+        v.write_needle(&mut needle(1), true, false).unwrap();
+        v.sync_to_disk().unwrap();
+        let kept = fs::read(format!("{dir}/1.idx")).unwrap();
+        v.write_needle(&mut needle(2), true, false).unwrap();
+        v.sync_to_disk().unwrap();
+        drop(v);
+
+        // Chop the second needle's body, leaving its header intact.
+        let dat = fs::OpenOptions::new()
+            .write(true)
+            .open(format!("{dir}/1.dat"))
+            .unwrap();
+        let torn_size = dat.metadata().unwrap().len() - 8;
+        dat.set_len(torn_size).unwrap();
+        drop(dat);
+        fs::remove_file(format!("{dir}/1.idx")).unwrap();
+
+        let reopened = Volume::new(
+            dir,
+            dir,
+            "",
+            VolumeId(1),
+            NeedleMapKind::InMemory,
+            None,
+            None,
+            0,
+            Version::current(),
+        )
+        .unwrap();
+        drop(reopened);
+
+        assert_eq!(
+            fs::read(format!("{dir}/1.idx")).unwrap(),
+            kept,
+            "the torn record leaked into the rebuilt idx"
+        );
+    }
+
+    // A corrupt header carrying a negative size advances the .dat walk
+    // backwards, which cycles forever between it and the record before it.
+    #[test]
+    fn test_rebuild_idx_stops_at_negative_size_header() {
+        let root = TempDir::new().unwrap();
+        let dir = root.path().to_str().unwrap();
+
+        let mut v = Volume::new(
+            dir,
+            dir,
+            "",
+            VolumeId(1),
+            NeedleMapKind::InMemory,
+            None,
+            None,
+            0,
+            Version::current(),
+        )
+        .unwrap();
+        v.write_needle(&mut needle(1), true, false).unwrap();
+        v.sync_to_disk().unwrap();
+        let kept = fs::read(format!("{dir}/1.idx")).unwrap();
+        v.write_needle(&mut needle(2), true, false).unwrap();
+        v.sync_to_disk().unwrap();
+        drop(v);
+
+        let rows = fs::read(format!("{dir}/1.idx")).unwrap();
+        let (_, offset, _) = idx_entry_from_bytes(&rows[NEEDLE_MAP_ENTRY_SIZE..]);
+        let corrupt_at = offset.to_actual_offset();
+
+        // Overwrite the second needle's size field with a negative i32.
+        use std::io::{Seek, SeekFrom, Write};
+        let mut dat = fs::OpenOptions::new()
+            .write(true)
+            .open(format!("{dir}/1.dat"))
+            .unwrap();
+        dat.seek(SeekFrom::Start(
+            corrupt_at as u64 + COOKIE_SIZE as u64 + NEEDLE_ID_SIZE as u64,
+        ))
+        .unwrap();
+        dat.write_all(&[0xff, 0xff, 0xf0, 0x00]).unwrap();
+        drop(dat);
+        fs::remove_file(format!("{dir}/1.idx")).unwrap();
+
+        // Pre-fix the rebuild walked backwards from here and never terminated.
+        let reopened = Volume::new(
+            dir,
+            dir,
+            "",
+            VolumeId(1),
+            NeedleMapKind::InMemory,
+            None,
+            None,
+            0,
+            Version::current(),
+        )
+        .unwrap();
+        drop(reopened);
+
+        assert_eq!(
+            fs::read(format!("{dir}/1.idx")).unwrap(),
+            kept,
+            "the corrupt record leaked into the rebuilt idx"
+        );
+    }
+}

seaweed-volume/src/storage/volume_open.rs

@@ -0,0 +1,27 @@
+//! Opening volume data and index files without access-time updates.
+
+use std::fs::{File, OpenOptions};
+use std::io;
+use std::path::Path;
+
+/// Open a volume data or index file with `O_NOATIME`. Nothing reads these
+/// files' atime, but without the flag every needle read dirties the inode —
+/// even relatime writes atime on the first read after each write, so an
+/// actively written volume pays a metadata write per read/write cycle.
+/// Matches Go's `backend.OpenVolumeFile`.
+pub fn open_volume_file(opts: &OpenOptions, path: impl AsRef<Path>) -> io::Result<File> {
+    let path = path.as_ref();
+    #[cfg(target_os = "linux")]
+    {
+        use std::os::unix::fs::OpenOptionsExt;
+        let mut noatime = opts.clone();
+        noatime.custom_flags(libc::O_NOATIME);
+        match noatime.open(path) {
+            // O_NOATIME is refused unless we own the file or hold CAP_FOWNER.
+            Err(e) if e.raw_os_error() == Some(libc::EPERM) => opts.open(path),
+            result => result,
+        }
+    }
+    #[cfg(not(target_os = "linux"))]
+    opts.open(path)
+}

seaweed-volume/src/storage/volume_report_hash.rs

@@ -30,6 +30,9 @@ pub fn report_hash(m: &master_pb::VolumeInformationMessage) -> u64 {
     if m.read_only {
         buf[56] = 1;
     }
+    if m.read_only_can_delete {
+        buf[56] |= 2;
+    }
     let mut h = xxh64(&buf, 0);
 
     h = fold(h, xxh64(&(m.modified_at_second as u64).to_le_bytes(), 0));
@@ -82,6 +85,7 @@ mod tests {
             delete_count: 2,
             deleted_byte_count: 99,
             read_only: true,
+            read_only_can_delete: false,
             replica_placement: 10,
             version: 3,
             ttl: 3 << 8,

seaweed-worker/Cargo.lock

@@ -5451,6 +5451,14 @@ dependencies = [
  "tracing",
 ]
 
+[[package]]
+name = "seaweed-worker-sort"
+version = "0.1.0"
+dependencies = [
+ "anyhow",
+ "seaweed-worker-core",
+]
+
 [[package]]
 name = "security-framework"
 version = "3.7.0"
@@ -6542,14 +6550,19 @@ dependencies = [
  "clap",
  "futures",
  "lance",
+ "lance-datafusion",
  "lance-index",
  "lance-linalg",
+ "lance-table",
  "prometheus",
  "reqwest 0.12.28",
+ "rustls",
  "seaweed-worker-core",
+ "seaweed-worker-sort",
  "serde",
  "serde_json",
  "tokio",
+ "tonic",
  "tracing",
  "tracing-subscriber",
 ]

seaweed-worker/Cargo.toml

@@ -2,10 +2,11 @@
 #
 # `core` is the plugin.proto contract and nothing else; a worker crate beside it
 # supplies job handlers and a binary. Adding a worker means adding a member here,
-# not touching the protocol.
+# not touching the protocol. `sort` is neither: it is the sort specification the
+# sorting jobs share, so that two of them cannot drift on what an order means.
 [workspace]
 resolver = "2"
-members = ["crates/core", "crates/lance"]
+members = ["crates/core", "crates/lance", "crates/sort"]
 
 [workspace.package]
 version = "0.1.0"

seaweed-worker/crates/core/src/config_form.rs

@@ -64,6 +64,17 @@ pub fn number_field(name: &str, label: &str, description: &str, min: i64, max: i
     }
 }
 
+pub fn text_field(name: &str, label: &str, description: &str, placeholder: &str) -> ConfigField {
+    ConfigField {
+        name: name.to_string(),
+        label: label.to_string(),
+        description: description.to_string(),
+        field_type: ConfigFieldType::String as i32,
+        placeholder: placeholder.to_string(),
+        ..Default::default()
+    }
+}
+
 pub fn bool_field(name: &str, label: &str, description: &str) -> ConfigField {
     ConfigField {
         name: name.to_string(),

seaweed-worker/crates/lance/Cargo.toml

@@ -16,11 +16,17 @@ path = "src/main.rs"
 
 [dependencies]
 seaweed-worker-core = { path = "../core" }
+seaweed-worker-sort = { path = "../sort" }
 prometheus.workspace = true
 # Only the S3 backend: the other object stores lance enables by default are
 # build time this worker never spends.
 lance = { version = "10", default-features = false, features = ["aws"] }
 lance-index = "10"
+# For the manifest's own fragment type, which the sort marker summarises.
+lance-table = "10"
+# For the execution options the sort needs; the scanner's own stream helper
+# takes the default, which has spilling off.
+lance-datafusion = "10"
 arrow-schema = "58"
 arrow-cast = "58"
 chrono = "0.4"
@@ -29,6 +35,7 @@ anyhow.workspace = true
 async-trait.workspace = true
 clap = { version = "4", features = ["derive", "env"] }
 reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] }
+rustls = "0.23"
 serde = { version = "1", features = ["derive"] }
 serde_json = "1"
 tokio.workspace = true
@@ -37,6 +44,7 @@ tracing-subscriber.workspace = true
 
 [dev-dependencies]
 tokio = { workspace = true }
+tonic.workspace = true
 arrow-array = "58"
 arrow-schema = "58"
 arrow-cast = "58"

seaweed-worker/crates/lance/src/jobs/mod.rs

@@ -9,6 +9,7 @@
 pub mod cleanup;
 pub mod compact;
 pub mod indices;
+pub mod sort;
 
 use std::collections::HashMap;
 use std::sync::Arc;
@@ -54,6 +55,11 @@ pub fn handlers(
                 .with_fallback(fallback.clone())
                 .with_metrics(metrics.clone()),
         ),
+        Arc::new(
+            sort::SortHandler::new(namespace_url.clone())
+                .with_fallback(fallback.clone())
+                .with_metrics(metrics.clone()),
+        ),
         Arc::new(
             cleanup::CleanupVersionsHandler::new(namespace_url)
                 .with_fallback(fallback)

seaweed-worker/crates/lance/src/jobs/sort.rs

@@ -0,0 +1,488 @@
+use std::collections::HashMap;
+use std::sync::Arc;
+
+use anyhow::{anyhow, Context, Result};
+use async_trait::async_trait;
+use lance::dataset::scanner::ColumnOrdering;
+use lance::dataset::transaction::Operation;
+use lance::dataset::write::{CommitBuilder, InsertBuilder};
+use lance::dataset::{WriteDestination, WriteMode, WriteParams};
+use lance::index::DatasetIndexExt;
+use lance_datafusion::exec::{execute_plan, LanceExecutionOptions};
+use seaweed_worker_core::config_form::{form, int_or, int_value, string_or, string_value};
+use seaweed_worker_core::pb::{
+    ConfigValue, DetectionComplete, DetectionProposals, ExecuteJobRequest, JobCompleted,
+    JobProgressUpdate, JobProposal, JobResult, JobTypeCapability, JobTypeDescriptor,
+    RunDetectionRequest, WorkerObservations,
+};
+use seaweed_worker_core::{DetectionSender, ExecutionSender, JobHandler};
+use seaweed_worker_sort::{
+    config_fields, resolve, verdict, FragmentSummary, SortSpec, SortState,
+    CONFIG_MAX_ROWS_PER_FILE, CONFIG_MEMORY_BUDGET_MB, CONFIG_MIN_UNSORTED_ROWS,
+    CONFIG_SORT_FIELDS, DECLARED_FIELDS_KEY,
+};
+use tracing::warn;
+
+use crate::catalog::{parse_id, NamespaceClient};
+use crate::dataset;
+use crate::jobs::{clamp, observation, string_list, table_id, FORMAT};
+
+pub const JOB_TYPE: &str = "lance_sort";
+
+const DEFAULT_MIN_UNSORTED_ROWS: i64 = 1_048_576;
+const DEFAULT_MEMORY_BUDGET_MB: i64 = 512;
+const DEFAULT_MAX_ROWS_PER_FILE: i64 = 1_048_576;
+
+// The ranges the shared form offers, repeated here because a value from outside
+// one is a value the UI could not have produced and every one of these is cast
+// to an unsigned type on the way in.
+const MIN_UNSORTED_ROWS_FLOOR: i64 = 1;
+const MIN_UNSORTED_ROWS_CEILING: i64 = 1_000_000_000;
+const MEMORY_BUDGET_FLOOR: i64 = 64;
+const MEMORY_BUDGET_CEILING: i64 = 1_048_576;
+const MAX_ROWS_PER_FILE_FLOOR: i64 = 1024;
+const MAX_ROWS_PER_FILE_CEILING: i64 = 16_777_216;
+
+const BYTES_PER_MB: u64 = 1024 * 1024;
+
+/// Rewrites a table so its rows are stored in the order its fields declare,
+/// which is what lets a range scan read a few files instead of all of them.
+///
+/// Lance appends fragments in write order and has no notion of a sorted table,
+/// so nothing but a rewrite establishes that order, and nothing but another
+/// rewrite restores it once rows have been appended.
+pub struct SortHandler {
+    namespace_url: String,
+    fallback: dataset::FallbackOptions,
+    metrics: Option<crate::metrics::LanceMetrics>,
+}
+
+impl SortHandler {
+    pub fn new(namespace_url: String) -> Self {
+        Self {
+            namespace_url,
+            fallback: dataset::FallbackOptions::new(),
+            metrics: None,
+        }
+    }
+
+    pub fn with_metrics(mut self, metrics: Option<crate::metrics::LanceMetrics>) -> Self {
+        self.metrics = metrics;
+        self
+    }
+
+    /// Storage options to use where the namespace vends none.
+    pub fn with_fallback(mut self, fallback: dataset::FallbackOptions) -> Self {
+        self.fallback = fallback;
+        self
+    }
+
+    fn client(&self) -> NamespaceClient {
+        NamespaceClient::new(self.namespace_url.clone())
+    }
+}
+
+/// The fragments a dataset holds, in fragment order, as the sort crate reads
+/// them: the data files each one names, and the live rows it holds.
+///
+/// Appends only add fragments after the existing ones and lance hands out
+/// fragment ids monotonically even across an overwrite, so ordering by id makes
+/// "the fragments the sort wrote are still the first ones" a prefix test.
+fn fragment_summaries(dataset: &lance::Dataset) -> Vec<FragmentSummary> {
+    let mut fragments = dataset.get_fragments();
+    fragments.sort_by_key(|fragment| fragment.id());
+    fragments
+        .iter()
+        .map(|fragment| summarize(fragment.metadata()))
+        .collect()
+}
+
+/// The manifest's own account of one fragment. `num_rows` is physical rows less
+/// deletions, and None where the manifest does not record them.
+fn summarize(fragment: &lance_table::format::Fragment) -> FragmentSummary {
+    FragmentSummary {
+        files: fragment
+            .files
+            .iter()
+            .map(|file| file.path.clone())
+            .collect(),
+        rows: fragment.num_rows().map(|rows| rows as u64),
+    }
+}
+
+/// The scan ordering for a spec. Lance validates the column names against the
+/// schema when the ordering is set, so a spec naming a column the table does
+/// not have fails before anything is written.
+fn orderings(spec: &SortSpec) -> Vec<ColumnOrdering> {
+    spec.fields
+        .iter()
+        .map(|field| ColumnOrdering {
+            ascending: !field.descending,
+            nulls_first: field.nulls_first,
+            column_name: field.path.clone(),
+        })
+        .collect()
+}
+
+#[async_trait]
+impl JobHandler for SortHandler {
+    fn capability(&self) -> JobTypeCapability {
+        JobTypeCapability {
+            job_type: JOB_TYPE.to_string(),
+            can_detect: true,
+            can_execute: true,
+            max_detection_concurrency: 1,
+            max_execution_concurrency: 1,
+            display_name: "Lance Sort".to_string(),
+            description: "Rewrite Lance tables in the order their fields declare. The rewrite replaces every fragment, so indices do not survive it and want rebuilding afterwards.".to_string(),
+            weight: 40,
+        }
+    }
+
+    fn descriptor(&self) -> JobTypeDescriptor {
+        let mut defaults: HashMap<String, ConfigValue> = HashMap::new();
+        defaults.insert(CONFIG_SORT_FIELDS.to_string(), string_value(""));
+        defaults.insert(
+            CONFIG_MIN_UNSORTED_ROWS.to_string(),
+            int_value(DEFAULT_MIN_UNSORTED_ROWS),
+        );
+        defaults.insert(
+            CONFIG_MEMORY_BUDGET_MB.to_string(),
+            int_value(DEFAULT_MEMORY_BUDGET_MB),
+        );
+        defaults.insert(
+            CONFIG_MAX_ROWS_PER_FILE.to_string(),
+            int_value(DEFAULT_MAX_ROWS_PER_FILE),
+        );
+
+        JobTypeDescriptor {
+            job_type: JOB_TYPE.to_string(),
+            display_name: "Lance Sort".to_string(),
+            description: "Sort Lance tables by their declared fields".to_string(),
+            icon: "fas fa-sort".to_string(),
+            descriptor_version: 1,
+            worker_config_form: Some(form(
+                "lance-sort-worker",
+                "Sort",
+                config_fields(),
+                defaults.clone(),
+            )),
+            worker_default_values: defaults,
+            ..Default::default()
+        }
+    }
+
+    /// Propose a job for every table whose declared order the data no longer
+    /// follows. Opening a dataset reads its manifest and its configuration, not
+    /// its data, so this stays cheap across a catalog.
+    async fn detect(
+        &self,
+        request: &RunDetectionRequest,
+        sender: &dyn DetectionSender,
+    ) -> Result<()> {
+        let configured = string_or(&request.worker_config_values, CONFIG_SORT_FIELDS, "");
+        let min_unsorted_rows = clamp(
+            int_or(
+                &request.worker_config_values,
+                CONFIG_MIN_UNSORTED_ROWS,
+                DEFAULT_MIN_UNSORTED_ROWS,
+            ),
+            MIN_UNSORTED_ROWS_FLOOR,
+            MIN_UNSORTED_ROWS_CEILING,
+        ) as u64;
+
+        let client = self.client();
+        let tables = client.list_all_tables().await?;
+
+        let mut proposals = Vec::new();
+        let mut observations = Vec::new();
+        for encoded in &tables {
+            let id = parse_id(encoded);
+            let table = match dataset::open(&client, &id, &self.fallback).await {
+                Ok(table) => table,
+                Err(err) => {
+                    // One unreadable table must not end the sweep: the tables
+                    // already read would lose their proposals.
+                    if let Some(counters) = &self.metrics {
+                        counters.worker.object_skipped(JOB_TYPE, "open");
+                    }
+                    warn!("skipping {encoded}: {err:#}");
+                    continue;
+                }
+            };
+            if let Some(counters) = &self.metrics {
+                counters.worker.object_seen(JOB_TYPE);
+            }
+            let stats = match table.stats().await {
+                Ok(stats) => stats,
+                Err(err) => {
+                    if let Some(counters) = &self.metrics {
+                        counters.worker.object_skipped(JOB_TYPE, "stats");
+                    }
+                    warn!("skipping {encoded}: reading its stats failed: {err:#}");
+                    continue;
+                }
+            };
+
+            let config = table.dataset.config().clone();
+            let spec = match resolve(
+                config.get(DECLARED_FIELDS_KEY).map(String::as_str),
+                &configured,
+            ) {
+                Ok(Some(spec)) => spec,
+                Ok(None) => {
+                    // Not every table is one an operator wants sorted, and a
+                    // table with no order named anywhere is not a failure.
+                    if let Some(counters) = &self.metrics {
+                        counters.worker.object_skipped(JOB_TYPE, "no_sort_order");
+                    }
+                    continue;
+                }
+                Err(err) => {
+                    if let Some(counters) = &self.metrics {
+                        counters.worker.object_skipped(JOB_TYPE, "sort_order");
+                    }
+                    warn!("skipping {encoded}: {err:#}");
+                    continue;
+                }
+            };
+
+            let state = SortState::from_config(&config);
+            let decision = verdict(
+                &spec,
+                &state,
+                &fragment_summaries(&table.dataset),
+                min_unsorted_rows,
+            );
+
+            let mut attributes: HashMap<String, ConfigValue> = HashMap::new();
+            attributes.insert("rows".to_string(), int_value(stats.rows as i64));
+            attributes.insert("version".to_string(), int_value(stats.version as i64));
+            attributes.insert("fragments".to_string(), int_value(stats.fragments as i64));
+            attributes.insert("sort_fields".to_string(), string_value(spec.to_string()));
+            attributes.insert("sort_state".to_string(), string_value(decision.reason()));
+            if let Some(schema) = stats.schema.clone() {
+                attributes.insert("schema".to_string(), string_value(schema));
+            }
+            observations.push(observation(&id, FORMAT, attributes));
+
+            // Logged because "detection proposed nothing" is otherwise
+            // indistinguishable from a table the worker could not read.
+            tracing::info!(
+                "sort detection: {encoded} has {} rows in {spec} order: {}",
+                stats.rows,
+                decision.reason()
+            );
+
+            if !decision.needs_sort() {
+                continue;
+            }
+            let mut parameters: HashMap<String, ConfigValue> = HashMap::new();
+            parameters.insert("table_id".to_string(), string_list(&id));
+            proposals.push(JobProposal {
+                proposal_id: format!("{JOB_TYPE}:{encoded}"),
+                dedupe_key: format!("{JOB_TYPE}:{encoded}"),
+                job_type: JOB_TYPE.to_string(),
+                summary: format!("Sort {encoded} by {spec}"),
+                detail: format!("{} rows, {}", stats.rows, decision.reason()),
+                parameters,
+                ..Default::default()
+            });
+        }
+
+        if !observations.is_empty() {
+            sender.send_observations(WorkerObservations {
+                job_type: JOB_TYPE.to_string(),
+                observations,
+            })?;
+        }
+
+        let total = proposals.len() as i32;
+        sender.send_proposals(DetectionProposals {
+            request_id: request.request_id.clone(),
+            job_type: JOB_TYPE.to_string(),
+            proposals,
+            has_more: false,
+        })?;
+        sender.send_complete(DetectionComplete {
+            request_id: request.request_id.clone(),
+            job_type: JOB_TYPE.to_string(),
+            success: true,
+            error_message: String::new(),
+            total_proposals: total,
+        })?;
+        Ok(())
+    }
+
+    async fn execute(
+        &self,
+        request: &ExecuteJobRequest,
+        sender: &dyn ExecutionSender,
+    ) -> Result<()> {
+        let job = request
+            .job
+            .as_ref()
+            .ok_or_else(|| anyhow!("execute request carried no job"))?;
+        let id = table_id(&job.parameters)
+            .ok_or_else(|| anyhow!("job {} carried no table_id", job.job_id))?;
+
+        let configured = string_or(&request.worker_config_values, CONFIG_SORT_FIELDS, "");
+        let memory_budget_mb = clamp(
+            int_or(
+                &request.worker_config_values,
+                CONFIG_MEMORY_BUDGET_MB,
+                DEFAULT_MEMORY_BUDGET_MB,
+            ),
+            MEMORY_BUDGET_FLOOR,
+            MEMORY_BUDGET_CEILING,
+        ) as u64;
+        let max_rows_per_file = clamp(
+            int_or(
+                &request.worker_config_values,
+                CONFIG_MAX_ROWS_PER_FILE,
+                DEFAULT_MAX_ROWS_PER_FILE,
+            ),
+            MAX_ROWS_PER_FILE_FLOOR,
+            MAX_ROWS_PER_FILE_CEILING,
+        ) as usize;
+
+        let client = self.client();
+        // Re-resolve rather than trusting what detection saw: the table may have
+        // been repointed, its declared order changed, and the vended credentials
+        // have expired.
+        let table = dataset::open(&client, &id, &self.fallback).await?;
+        let before = table.stats().await?;
+        let config = table.dataset.config().clone();
+        let spec = resolve(
+            config.get(DECLARED_FIELDS_KEY).map(String::as_str),
+            &configured,
+        )?
+        .ok_or_else(|| anyhow!("neither the table nor this worker names an order to sort by"))?;
+
+        sender.send_progress(JobProgressUpdate {
+            request_id: request.request_id.clone(),
+            job_id: job.job_id.clone(),
+            job_type: JOB_TYPE.to_string(),
+            progress_percent: 10.0,
+            stage: format!("sorting {} rows by {spec}", before.rows),
+            ..Default::default()
+        })?;
+
+        let mut scanner = table.dataset.scan();
+        scanner
+            .order_by(Some(orderings(&spec)))
+            .with_context(|| format!("order the scan of {} by {spec}", table.location))?;
+        let plan = scanner
+            .create_plan()
+            .await
+            .context("plan the ordered scan")?;
+
+        // Spilling is off in LanceExecutionOptions::default(), and the scanner's
+        // own stream helper takes that default, so the options are built here
+        // instead: a sort that cannot spill is one bounded by memory, which is
+        // the whole thing this job exists to avoid.
+        let stream = execute_plan(
+            plan,
+            LanceExecutionOptions {
+                use_spilling: true,
+                mem_pool_size: Some(memory_budget_mb * BYTES_PER_MB),
+                ..Default::default()
+            },
+        )
+        .context("run the ordered scan")?;
+
+        let params = WriteParams {
+            mode: WriteMode::Overwrite,
+            max_rows_per_file,
+            ..Default::default()
+        };
+        let destination = Arc::new(table.dataset.clone());
+        let mut transaction = InsertBuilder::new(WriteDestination::Dataset(destination.clone()))
+            .with_params(&params)
+            .execute_uncommitted_stream(stream)
+            .await
+            .context("write the sorted fragments")?;
+
+        // The marker rides in the same commit as the data it describes:
+        // Operation::Overwrite is the one place lance takes configuration
+        // values alongside fragments, and a marker written as a second commit
+        // would be lost if the worker died in between, re-sorting the whole
+        // table on the next sweep.
+        match &mut transaction.operation {
+            Operation::Overwrite {
+                config_upsert_values,
+                fragments,
+                ..
+            } => {
+                // The files these fragments name were written a moment ago and
+                // keep their names whatever version this commit becomes, which
+                // is what lets the marker travel inside the commit it describes.
+                let written: Vec<FragmentSummary> = fragments.iter().map(summarize).collect();
+                *config_upsert_values =
+                    Some(SortState::record(&spec, &written, before.rows as u64));
+            }
+            other => {
+                return Err(anyhow!(
+                    "a sorted rewrite produced {other} instead of an overwrite"
+                ))
+            }
+        }
+
+        let sorted = CommitBuilder::new(WriteDestination::Dataset(destination))
+            .execute(transaction)
+            .await
+            .context("commit the sorted rewrite")?;
+
+        if let Some(counters) = &self.metrics {
+            counters.rows_sorted.inc_by(before.rows as u64);
+        }
+
+        // An overwrite carries fragments, schema and configuration — not index
+        // metadata — so a table that had indices no longer does. Saying so is
+        // the difference between a slow table and a mystery.
+        let dropped_indices = match table.dataset.load_indices().await {
+            Ok(indices) => indices.len(),
+            Err(err) => {
+                warn!("could not read the indices of {}: {err:#}", table.location);
+                0
+            }
+        };
+        if dropped_indices > 0 {
+            warn!(
+                "{} lost {dropped_indices} index/indices to the sorted rewrite; run the index job to rebuild them",
+                table.location
+            );
+        }
+
+        let fragments_after = sorted.get_fragments().len();
+        let mut output: HashMap<String, ConfigValue> = HashMap::new();
+        output.insert("rows_sorted".to_string(), int_value(before.rows as i64));
+        output.insert("sort_fields".to_string(), string_value(spec.to_string()));
+        output.insert(
+            "fragments_before".to_string(),
+            int_value(before.fragments as i64),
+        );
+        output.insert(
+            "fragments_after".to_string(),
+            int_value(fragments_after as i64),
+        );
+        output.insert(
+            "indices_dropped".to_string(),
+            int_value(dropped_indices as i64),
+        );
+
+        sender.send_completed(JobCompleted {
+            request_id: request.request_id.clone(),
+            job_id: job.job_id.clone(),
+            job_type: JOB_TYPE.to_string(),
+            success: true,
+            result: Some(JobResult {
+                output_values: output,
+                summary: format!("{} rows rewritten in {spec} order", before.rows),
+                ..Default::default()
+            }),
+            ..Default::default()
+        })?;
+        Ok(())
+    }
+}

seaweed-worker/crates/lance/src/lib.rs

@@ -10,5 +10,6 @@ pub mod dataset;
 pub mod jobs;
 pub mod metrics;
 pub mod preview;
+pub mod tls;
 
 pub use jobs::handlers;

seaweed-worker/crates/lance/src/main.rs

@@ -12,10 +12,7 @@ use weed_lance_worker::metrics::LanceMetrics;
 /// Mirrors `weed worker`'s flags, because this is the same contract from another
 /// language and an operator should not have to learn a second set of names.
 #[derive(Parser, Debug)]
-#[command(
-    name = "weed-worker",
-    about = "SeaweedFS maintenance worker"
-)]
+#[command(name = "weed-worker", about = "SeaweedFS maintenance worker")]
 struct Args {
     /// Admin server gRPC address.
     #[arg(long, default_value = "localhost:23646", env = "WEED_ADMIN")]
@@ -116,6 +113,8 @@ fn metrics_address(ip: &str, port: u16) -> Result<SocketAddr> {
 
 #[tokio::main]
 async fn main() -> Result<()> {
+    weed_lance_worker::tls::install_default_crypto_provider();
+
     tracing_subscriber::fmt()
         .with_env_filter(
             tracing_subscriber::EnvFilter::try_from_default_env()

seaweed-worker/crates/lance/src/metrics.rs

@@ -18,6 +18,7 @@ pub struct LanceMetrics {
     pub worker: Metrics,
     pub fragments_removed: IntCounter,
     pub rows_indexed: IntCounter,
+    pub rows_sorted: IntCounter,
     pub versions_removed: IntCounter,
     pub bytes_reclaimed: IntCounter,
 }
@@ -34,6 +35,10 @@ impl LanceMetrics {
                 "lance_rows_indexed_total",
                 "Rows brought under an index that did not cover them.",
             )?,
+            rows_sorted: metrics.counter(
+                "lance_rows_sorted_total",
+                "Rows rewritten into their table's declared order.",
+            )?,
             versions_removed: metrics.counter(
                 "lance_versions_removed_total",
                 "Dataset versions removed by cleanup.",

seaweed-worker/crates/lance/src/tls.rs

@@ -0,0 +1,35 @@
+use rustls::crypto::aws_lc_rs;
+
+// aws-lc-rs and ring both get linked transitively (lance's aws backend pulls
+// aws-lc-rs, reqwest's rustls-tls pulls ring), so rustls can't auto-select a
+// provider and tonic's client TLS panics on first use. Pin the default to
+// aws-lc-rs, matching the Rust volume server. Idempotent.
+pub fn install_default_crypto_provider() {
+    let _ = aws_lc_rs::default_provider().install_default();
+}
+
+#[cfg(test)]
+mod tests {
+    use tonic::transport::{Certificate, Channel, ClientTlsConfig, Identity};
+
+    use super::install_default_crypto_provider;
+
+    const TEST_CERT_PEM: &str = "-----BEGIN CERTIFICATE-----\nMIIBPDCB76ADAgECAhRuRPQgeAu43BT/M7EfAWSdapVdYDAFBgMrZXAwFDESMBAG\nA1UEAwwJbG9jYWxob3N0MB4XDTI2MDcwNTE2MTUyOVoXDTM2MDcwMjE2MTUyOVow\nFDESMBAGA1UEAwwJbG9jYWxob3N0MCowBQYDK2VwAyEAr/3bNIFI+8V32oCiY6y+\nXRFmZpdNQ2g//VtRkT+nQg+jUzBRMB0GA1UdDgQWBBTsy9tLf1zPiXCQfgci6zNi\ndEzRSjAfBgNVHSMEGDAWgBTsy9tLf1zPiXCQfgci6zNidEzRSjAPBgNVHRMBAf8E\nBTADAQH/MAUGAytlcANBAIvsdw0IbvOBBkb9cd7BfMJfIP9pQQrAL03pCRWJFnFh\nSysaLVgFXI4T078IiaM874oO+iB+5vNbWEpc7CkGow4=\n-----END CERTIFICATE-----\n";
+    const TEST_KEY_PEM: &str = "-----BEGIN PRIVATE KEY-----\nMC4CAQAwBQYDK2VwBCIEIHbyn71Kk+Y7KT3sBctit7uZpErpoH6qDbFj6P8qGaZH\n-----END PRIVATE KEY-----\n";
+
+    // Without install_default_crypto_provider this panics in the lance crate,
+    // where both aws-lc-rs and ring are linked. Mirrors the volume server's
+    // test_build_grpc_endpoint_with_tls_resolves_crypto_provider.
+    #[tokio::test]
+    async fn tls_channel_builds_after_crypto_provider_installed() {
+        install_default_crypto_provider();
+        let config = ClientTlsConfig::new()
+            .ca_certificate(Certificate::from_pem(TEST_CERT_PEM))
+            .identity(Identity::from_pem(TEST_CERT_PEM, TEST_KEY_PEM));
+        let endpoint = Channel::from_shared("https://127.0.0.1:9")
+            .unwrap()
+            .tls_config(config)
+            .unwrap();
+        assert_eq!(endpoint.uri().scheme_str(), Some("https"));
+    }
+}

seaweed-worker/crates/lance/tests/sort.rs

@@ -0,0 +1,504 @@
+//! Drives the sort handler against a live namespace.
+//!
+//! Skipped unless WEED_LANCE_NAMESPACE names one, like the other integration
+//! tests here: the job rewrites every fragment of a real table and commits, and
+//! the commit is the half worth testing.
+
+use anyhow::Result;
+use seaweed_worker_core::pb::{
+    config_value::Kind, ConfigValue, ExecuteJobRequest, JobSpec, RunDetectionRequest,
+};
+use seaweed_worker_core::JobHandler;
+use weed_lance_worker::catalog::NamespaceClient;
+use weed_lance_worker::jobs::sort::{SortHandler, JOB_TYPE};
+
+mod common;
+use common::{fallback, namespace_url, Recorder};
+
+/// One live gateway and one shared catalog, and `list_all_tables` sweeps
+/// everything, so these tests take a lock the way the compaction ones do.
+static GATEWAY: tokio::sync::Mutex<()> = tokio::sync::Mutex::const_new(());
+
+fn config(values: &[(&str, ConfigValue)]) -> std::collections::HashMap<String, ConfigValue> {
+    values
+        .iter()
+        .map(|(name, value)| (name.to_string(), value.clone()))
+        .collect()
+}
+
+fn int(value: i64) -> ConfigValue {
+    ConfigValue {
+        kind: Some(Kind::Int64Value(value)),
+    }
+}
+
+fn text(value: &str) -> ConfigValue {
+    ConfigValue {
+        kind: Some(Kind::StringValue(value.to_string())),
+    }
+}
+
+/// Declares a table and writes `batches` appends whose ids descend, so the rows
+/// land in an order no sort would produce. `declared` is written into the
+/// dataset's own configuration, which is where a table says how it wants to be
+/// sorted.
+async fn seed_unsorted_table(
+    url: &str,
+    name: &str,
+    batches: i64,
+    rows_each: i64,
+    declared: Option<&str>,
+) -> Result<String> {
+    use arrow_array::{Int64Array, RecordBatch, RecordBatchIterator};
+    use arrow_schema::{DataType, Field, Schema};
+    use lance::dataset::{Dataset, WriteMode, WriteParams};
+    use lance::io::{ObjectStoreParams, StorageOptionsAccessor};
+    use std::sync::Arc;
+
+    let http = reqwest::Client::new();
+    for parent in ["vec", "vec$ml"] {
+        http.post(format!("{url}/v1/namespace/{parent}/create"))
+            .json(&serde_json::json!({"mode": "EXIST_OK"}))
+            .send()
+            .await?
+            .error_for_status()?;
+    }
+    let encoded = format!("vec$ml${name}");
+    http.post(format!("{url}/v1/table/{encoded}/declare"))
+        .json(&serde_json::json!({}))
+        .send()
+        .await?
+        .error_for_status()?;
+
+    let client = NamespaceClient::new(url.to_string());
+    let id = vec!["vec".to_string(), "ml".to_string(), name.to_string()];
+    let description = client.describe_table(&id).await?;
+    let mut options = description.storage_options.clone();
+    options.extend(fallback());
+
+    let schema = Arc::new(Schema::new(vec![Field::new("id", DataType::Int64, false)]));
+    let total = batches * rows_each;
+    let mut dataset = None;
+    for batch_index in 0..batches {
+        // Ids descend across and within the batches, so the physical order is
+        // the reverse of the order the table declares.
+        let ids: Vec<i64> = (0..rows_each)
+            .map(|row| total - 1 - (batch_index * rows_each + row))
+            .collect();
+        let batch = RecordBatch::try_new(schema.clone(), vec![Arc::new(Int64Array::from(ids))])?;
+        let params = WriteParams {
+            mode: if batch_index == 0 {
+                WriteMode::Overwrite
+            } else {
+                WriteMode::Append
+            },
+            store_params: Some(ObjectStoreParams {
+                storage_options_accessor: Some(Arc::new(
+                    StorageOptionsAccessor::with_static_options(options.clone()),
+                )),
+                ..Default::default()
+            }),
+            ..Default::default()
+        };
+        dataset = Some(
+            Dataset::write(
+                RecordBatchIterator::new(vec![Ok(batch)], schema.clone()),
+                description.location.as_str(),
+                Some(params),
+            )
+            .await?,
+        );
+    }
+
+    if let Some(declared) = declared {
+        let mut dataset = dataset.expect("at least one batch was written");
+        dataset
+            .update_config([("seaweedfs.sort.fields", declared)])
+            .await?;
+    }
+    Ok(encoded)
+}
+
+/// Reads the ids back in the order they are stored, which is what a sort is
+/// supposed to change.
+async fn stored_ids(url: &str, name: &str) -> Result<Vec<i64>> {
+    use arrow_array::Int64Array;
+    use futures::TryStreamExt;
+
+    let client = NamespaceClient::new(url.to_string());
+    let id = vec!["vec".to_string(), "ml".to_string(), name.to_string()];
+    let table = weed_lance_worker::dataset::open(&client, &id, &fallback()).await?;
+    let stream = table.dataset.scan().try_into_stream().await?;
+    let batches: Vec<_> = stream.try_collect().await?;
+    let mut ids = Vec::new();
+    for batch in batches {
+        let column = batch
+            .column_by_name("id")
+            .expect("the seeded table has an id column")
+            .as_any()
+            .downcast_ref::<Int64Array>()
+            .expect("id is an Int64 column")
+            .clone();
+        ids.extend(column.values().iter().copied());
+    }
+    Ok(ids)
+}
+
+/// The whole loop: a table declaring an order its rows do not follow is
+/// proposed, sorting it stores the rows in that order, and the marker the same
+/// commit carries is what stops the next sweep proposing it again.
+#[tokio::test]
+async fn sorts_a_table_and_stops_proposing_it() {
+    let _gateway = GATEWAY.lock().await;
+    let Some(url) = namespace_url() else {
+        eprintln!("WEED_LANCE_NAMESPACE is unset, skipping");
+        return;
+    };
+    let encoded = seed_unsorted_table(&url, "sortme", 4, 250, Some("id asc"))
+        .await
+        .expect("seed an unsorted table");
+
+    let before = stored_ids(&url, "sortme").await.expect("read the ids back");
+    assert!(
+        before.windows(2).any(|pair| pair[0] > pair[1]),
+        "the seeded table was already sorted, so the test proves nothing"
+    );
+
+    let handler = SortHandler::new(url.clone()).with_fallback(fallback());
+    let recorder = Recorder::default();
+    let detection = RunDetectionRequest {
+        request_id: "detect-sort".to_string(),
+        job_type: JOB_TYPE.to_string(),
+        worker_config_values: config(&[("min_unsorted_rows", int(1))]),
+        ..Default::default()
+    };
+    handler
+        .detect(&detection, &recorder)
+        .await
+        .expect("detection failed");
+
+    let proposal = recorder
+        .proposals
+        .lock()
+        .unwrap()
+        .iter()
+        .find(|p| p.summary.contains(encoded.as_str()))
+        .cloned()
+        .expect("the unsorted table was not proposed");
+    assert!(
+        proposal.summary.contains("id asc"),
+        "the proposal does not say what order it would use: {}",
+        proposal.summary
+    );
+
+    let execute = ExecuteJobRequest {
+        request_id: "execute-sort".to_string(),
+        job: Some(JobSpec {
+            job_id: "job-sort".to_string(),
+            job_type: JOB_TYPE.to_string(),
+            parameters: proposal.parameters.clone(),
+            ..Default::default()
+        }),
+        worker_config_values: config(&[
+            ("memory_budget_mb", int(64)),
+            ("max_rows_per_file", int(1024)),
+        ]),
+        ..Default::default()
+    };
+    handler
+        .execute(&execute, &recorder)
+        .await
+        .expect("sorting failed");
+
+    let completed = recorder.completed.lock().unwrap().clone();
+    let result = completed.last().expect("no completion reported");
+    assert!(
+        result.success,
+        "the sort reported failure: {}",
+        result.error_message
+    );
+
+    let after = stored_ids(&url, "sortme")
+        .await
+        .expect("read the sorted ids back");
+    assert_eq!(after.len(), before.len(), "the sort lost or invented rows");
+    assert!(
+        after.windows(2).all(|pair| pair[0] <= pair[1]),
+        "the rows are not stored in ascending id order"
+    );
+
+    // The marker rode in the same commit as the data, so a second sweep finds
+    // the table up to date without anything else having written it.
+    let second = Recorder::default();
+    handler
+        .detect(&detection, &second)
+        .await
+        .expect("second detection failed");
+    assert!(
+        !second
+            .proposals
+            .lock()
+            .unwrap()
+            .iter()
+            .any(|p| p.summary.contains(encoded.as_str())),
+        "a table that was just sorted must not be proposed again"
+    );
+    eprintln!("sort result: {}", result.result.as_ref().unwrap().summary);
+}
+
+/// A table that declares no order, with a worker configuring none either, is
+/// not something to rewrite on a guess.
+#[tokio::test]
+async fn leaves_a_table_that_declares_no_order_alone() {
+    let _gateway = GATEWAY.lock().await;
+    let Some(url) = namespace_url() else {
+        eprintln!("WEED_LANCE_NAMESPACE is unset, skipping");
+        return;
+    };
+    let encoded = seed_unsorted_table(&url, "unsorted", 2, 16, None)
+        .await
+        .expect("seed a table with no declared order");
+
+    let handler = SortHandler::new(url.clone()).with_fallback(fallback());
+    let recorder = Recorder::default();
+    handler
+        .detect(
+            &RunDetectionRequest {
+                request_id: "detect-none".to_string(),
+                job_type: JOB_TYPE.to_string(),
+                worker_config_values: config(&[
+                    ("min_unsorted_rows", int(1)),
+                    ("sort_fields", text("")),
+                ]),
+                ..Default::default()
+            },
+            &recorder,
+        )
+        .await
+        .expect("detection failed");
+
+    assert!(
+        !recorder
+            .proposals
+            .lock()
+            .unwrap()
+            .iter()
+            .any(|p| p.summary.contains(encoded.as_str())),
+        "a table with no order named anywhere must not be proposed"
+    );
+}
+
+/// The case a row-count threshold cannot see: the table replaced by a write of
+/// its own, growing by far less than `min_unsorted_rows`. Every row moved, so
+/// the table is unsorted, and detection has to say so however small the delta.
+#[tokio::test]
+async fn proposes_a_replacement_that_barely_changes_the_row_count() {
+    let _gateway = GATEWAY.lock().await;
+    let Some(url) = namespace_url() else {
+        eprintln!("WEED_LANCE_NAMESPACE is unset, skipping");
+        return;
+    };
+    let encoded = seed_unsorted_table(&url, "replaceme", 4, 250, Some("id asc"))
+        .await
+        .expect("seed an unsorted table");
+
+    let handler = SortHandler::new(url.clone()).with_fallback(fallback());
+    // A threshold no realistic append would cross, so nothing but the identity
+    // of the data can be what proposes this table.
+    let detection = RunDetectionRequest {
+        request_id: "detect-replace".to_string(),
+        job_type: JOB_TYPE.to_string(),
+        worker_config_values: config(&[("min_unsorted_rows", int(1_000_000))]),
+        ..Default::default()
+    };
+
+    let first = Recorder::default();
+    handler.detect(&detection, &first).await.expect("detection");
+    let proposal = first
+        .proposals
+        .lock()
+        .unwrap()
+        .iter()
+        .find(|p| p.summary.contains(encoded.as_str()))
+        .cloned()
+        .expect("a never-sorted table was not proposed");
+
+    handler
+        .execute(
+            &ExecuteJobRequest {
+                request_id: "execute-replace".to_string(),
+                job: Some(JobSpec {
+                    job_id: "job-replace".to_string(),
+                    job_type: JOB_TYPE.to_string(),
+                    parameters: proposal.parameters.clone(),
+                    ..Default::default()
+                }),
+                worker_config_values: config(&[("max_rows_per_file", int(1024))]),
+                ..Default::default()
+            },
+            &first,
+        )
+        .await
+        .expect("sorting failed");
+
+    // Sorted, and left alone by the next sweep.
+    let after_sort = Recorder::default();
+    handler
+        .detect(&detection, &after_sort)
+        .await
+        .expect("detection");
+    assert!(
+        !after_sort
+            .proposals
+            .lock()
+            .unwrap()
+            .iter()
+            .any(|p| p.summary.contains(encoded.as_str())),
+        "a table that was just sorted must not be proposed again"
+    );
+
+    // Now replace the data: 1002 rows where there were 1000, shuffled again.
+    seed_unsorted_table(&url, "replaceme", 3, 334, Some("id asc"))
+        .await
+        .expect("replace the table's data");
+
+    let after_replace = Recorder::default();
+    handler
+        .detect(&detection, &after_replace)
+        .await
+        .expect("detection");
+    assert!(
+        after_replace
+            .proposals
+            .lock()
+            .unwrap()
+            .iter()
+            .any(|p| p.summary.contains(encoded.as_str())),
+        "a replaced table must be proposed even when the row count barely moved"
+    );
+}
+
+/// Deletions must not hide appended rows. This is the arithmetic against a real
+/// manifest rather than a hand-built one: the sorted fragments keep their files
+/// and lose most of their rows, a smaller batch is appended after them, and the
+/// table has to be proposed *because rows were appended* — not because it looks
+/// replaced, and not left alone because it now holds fewer rows than when it
+/// was sorted.
+#[tokio::test]
+async fn deletions_do_not_hide_appended_rows() {
+    let _gateway = GATEWAY.lock().await;
+    let Some(url) = namespace_url() else {
+        eprintln!("WEED_LANCE_NAMESPACE is unset, skipping");
+        return;
+    };
+    let encoded = seed_unsorted_table(&url, "deleteme", 4, 250, Some("id asc"))
+        .await
+        .expect("seed an unsorted table");
+
+    let handler = SortHandler::new(url.clone()).with_fallback(fallback());
+    let detection = RunDetectionRequest {
+        request_id: "detect-delete".to_string(),
+        job_type: JOB_TYPE.to_string(),
+        worker_config_values: config(&[("min_unsorted_rows", int(100))]),
+        ..Default::default()
+    };
+
+    let recorder = Recorder::default();
+    handler
+        .detect(&detection, &recorder)
+        .await
+        .expect("detection");
+    let proposal = recorder
+        .proposals
+        .lock()
+        .unwrap()
+        .iter()
+        .find(|p| p.summary.contains(encoded.as_str()))
+        .cloned()
+        .expect("a never-sorted table was not proposed");
+    handler
+        .execute(
+            &ExecuteJobRequest {
+                request_id: "execute-delete".to_string(),
+                job: Some(JobSpec {
+                    job_id: "job-delete".to_string(),
+                    job_type: JOB_TYPE.to_string(),
+                    parameters: proposal.parameters.clone(),
+                    ..Default::default()
+                }),
+                worker_config_values: config(&[("max_rows_per_file", int(1024))]),
+                ..Default::default()
+            },
+            &recorder,
+        )
+        .await
+        .expect("sorting failed");
+
+    // Delete 900 of the 1000 sorted rows, then append 200 unsorted ones. The
+    // table is smaller than when it was sorted, so net growth reads as zero.
+    let client = NamespaceClient::new(url.clone());
+    let id = vec!["vec".to_string(), "ml".to_string(), "deleteme".to_string()];
+    let mut table = weed_lance_worker::dataset::open(&client, &id, &fallback())
+        .await
+        .expect("reopen the sorted table");
+    table
+        .dataset
+        .delete("id < 900")
+        .await
+        .expect("delete most of the sorted rows");
+    append_ids(&url, "deleteme", 5_000, 200)
+        .await
+        .expect("append rows after the sorted ones");
+
+    let after = Recorder::default();
+    handler.detect(&detection, &after).await.expect("detection");
+    let proposal = after
+        .proposals
+        .lock()
+        .unwrap()
+        .iter()
+        .find(|p| p.summary.contains(encoded.as_str()))
+        .cloned()
+        .expect("a table with 200 unsorted rows appended was not proposed");
+    assert!(
+        proposal.detail.contains("200 rows appended"),
+        "expected the appended rows to be counted, got {:?}",
+        proposal.detail
+    );
+}
+
+/// Appends one batch of ascending ids to an existing table.
+async fn append_ids(url: &str, name: &str, first_id: i64, rows: i64) -> Result<()> {
+    use arrow_array::{Int64Array, RecordBatch, RecordBatchIterator};
+    use arrow_schema::{DataType, Field, Schema};
+    use lance::dataset::{Dataset, WriteMode, WriteParams};
+    use lance::io::{ObjectStoreParams, StorageOptionsAccessor};
+    use std::sync::Arc;
+
+    let client = NamespaceClient::new(url.to_string());
+    let id = vec!["vec".to_string(), "ml".to_string(), name.to_string()];
+    let description = client.describe_table(&id).await?;
+    let mut options = description.storage_options.clone();
+    options.extend(fallback());
+
+    let schema = Arc::new(Schema::new(vec![Field::new("id", DataType::Int64, false)]));
+    let ids: Vec<i64> = (0..rows).map(|row| first_id + row).collect();
+    let batch = RecordBatch::try_new(schema.clone(), vec![Arc::new(Int64Array::from(ids))])?;
+    let params = WriteParams {
+        mode: WriteMode::Append,
+        store_params: Some(ObjectStoreParams {
+            storage_options_accessor: Some(Arc::new(StorageOptionsAccessor::with_static_options(
+                options,
+            ))),
+            ..Default::default()
+        }),
+        ..Default::default()
+    };
+    Dataset::write(
+        RecordBatchIterator::new(vec![Ok(batch)], schema.clone()),
+        description.location.as_str(),
+        Some(params),
+    )
+    .await?;
+    Ok(())
+}

seaweed-worker/crates/sort/Cargo.toml

@@ -0,0 +1,12 @@
+[package]
+name = "seaweed-worker-sort"
+version.workspace = true
+edition.workspace = true
+description = "The sort specification shared by SeaweedFS sorting jobs"
+
+[lib]
+name = "seaweed_worker_sort"
+
+[dependencies]
+seaweed-worker-core = { path = "../core" }
+anyhow.workspace = true

seaweed-worker/crates/sort/src/lib.rs

@@ -0,0 +1,612 @@
+//! The sort specification the sorting jobs share.
+//!
+//! A worker sorts a table by the fields the table itself declares, and by the
+//! fields its own configuration names for a table that declares none. Both
+//! halves of that rule, and the spelling of a field list, live here rather than
+//! in each job: a Lance job and an Iceberg job that disagreed about what
+//! "id desc nulls-first" means would be two features wearing one name.
+//!
+//! The spelling is deliberately the one `weed/worker/tasks/iceberg` already
+//! writes into an Iceberg snapshot's `sort-fields` summary, so the same order
+//! reads the same way whichever format holds the table and whichever language
+//! sorted it.
+
+use std::collections::{HashMap, HashSet};
+use std::fmt;
+
+use anyhow::{bail, Context, Result};
+use seaweed_worker_core::config_form::{number_field, text_field};
+use seaweed_worker_core::pb::ConfigField;
+
+/// Worker configuration keys. Shared so the Iceberg and Lance forms offer an
+/// operator the same names for the same settings.
+pub const CONFIG_SORT_FIELDS: &str = "sort_fields";
+pub const CONFIG_MIN_UNSORTED_ROWS: &str = "min_unsorted_rows";
+pub const CONFIG_MEMORY_BUDGET_MB: &str = "memory_budget_mb";
+pub const CONFIG_MAX_ROWS_PER_FILE: &str = "max_rows_per_file";
+
+/// The order an operator declares on a table. The worker only ever reads this
+/// one.
+pub const DECLARED_FIELDS_KEY: &str = "seaweedfs.sort.fields";
+
+/// What the worker recorded about the sort it last performed. Kept apart from
+/// the declaration above so that "the operator asked for this order" and "the
+/// worker achieved this order" stay distinguishable — comparing them is how a
+/// changed order is noticed.
+pub const SORTED_FIELDS_KEY: &str = "seaweedfs.sort.sorted_fields";
+pub const SORTED_ROWS_KEY: &str = "seaweedfs.sort.sorted_rows";
+/// How many fragments the sort wrote, and a digest of their data file names.
+/// Together they identify the data the sort produced — see [`verdict`] for why
+/// a version number cannot.
+pub const SORTED_FRAGMENTS_KEY: &str = "seaweedfs.sort.sorted_fragments";
+pub const SORTED_DIGEST_KEY: &str = "seaweedfs.sort.sorted_digest";
+
+/// One field of a sort order.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct SortField {
+    pub path: String,
+    pub descending: bool,
+    pub nulls_first: bool,
+}
+
+/// A whole sort order, in the order the fields are compared.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct SortSpec {
+    pub fields: Vec<SortField>,
+}
+
+impl SortSpec {
+    /// Reads "user_id asc nulls-last, ts desc" into a spec. An empty string is
+    /// not an error: it is how an operator says nothing, and the caller decides
+    /// what that means.
+    ///
+    /// Direction defaults to ascending and null order to Iceberg's default for
+    /// the direction — nulls first ascending, nulls last descending — because
+    /// this spelling is shared with tables that already have that rule.
+    pub fn parse(text: &str) -> Result<Option<Self>> {
+        let mut fields = Vec::new();
+        let mut seen = HashSet::new();
+        for entry in text.split(',') {
+            let entry = entry.trim();
+            if entry.is_empty() {
+                continue;
+            }
+            let field =
+                parse_field(entry).with_context(|| format!("read the sort field {entry:?}"))?;
+            // Compared exactly, not case-folded: Arrow schemas are
+            // case-sensitive, so `id` and `ID` can be two real columns and
+            // folding them together would reject a valid order. The Iceberg
+            // job, whose format resolves names case-insensitively, does its own
+            // check against the table schema.
+            if !seen.insert(field.path.clone()) {
+                bail!("the sort field {:?} is named twice", field.path);
+            }
+            fields.push(field);
+        }
+        if fields.is_empty() {
+            return Ok(None);
+        }
+        Ok(Some(Self { fields }))
+    }
+}
+
+/// Renders a spec back, always spelling out direction and null order so that a
+/// recorded order round-trips to the same thing it was parsed from.
+impl fmt::Display for SortSpec {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        for (i, field) in self.fields.iter().enumerate() {
+            if i > 0 {
+                write!(f, ", ")?;
+            }
+            write!(
+                f,
+                "{} {} {}",
+                field.path,
+                if field.descending { "desc" } else { "asc" },
+                if field.nulls_first {
+                    "nulls-first"
+                } else {
+                    "nulls-last"
+                }
+            )?;
+        }
+        Ok(())
+    }
+}
+
+fn parse_field(entry: &str) -> Result<SortField> {
+    let mut tokens = entry.split_whitespace();
+    let path = tokens
+        .next()
+        .expect("split_whitespace yields one token for a non-empty entry")
+        .to_string();
+
+    let mut descending = None;
+    let mut nulls_first = None;
+    for token in tokens {
+        // Underscores are accepted because that is how the same words are
+        // spelled in configuration keys, and refusing them would only teach an
+        // operator that the two spellings are different things.
+        match token.to_ascii_lowercase().replace('_', "-").as_str() {
+            "asc" | "ascending" if descending.is_none() => descending = Some(false),
+            "desc" | "descending" if descending.is_none() => descending = Some(true),
+            "nulls-first" if nulls_first.is_none() => nulls_first = Some(true),
+            "nulls-last" if nulls_first.is_none() => nulls_first = Some(false),
+            other => bail!("{other:?} is not a direction or a null order"),
+        }
+    }
+
+    let descending = descending.unwrap_or(false);
+    Ok(SortField {
+        path,
+        descending,
+        nulls_first: nulls_first.unwrap_or(!descending),
+    })
+}
+
+/// The order to sort a table by: what the table declares wins, and the worker's
+/// configuration is the fallback for a table that declares nothing. Neither one
+/// is not an error — it means this is not a table the operator wants sorted.
+///
+/// A declaration that cannot be read is an error rather than a reason to fall
+/// back: sorting by the worker's default order instead of the one the table
+/// asked for would silently rewrite the table the wrong way.
+pub fn resolve(declared: Option<&str>, configured: &str) -> Result<Option<SortSpec>> {
+    if let Some(declared) = declared {
+        if let Some(spec) = SortSpec::parse(declared).context("read the table's declared order")? {
+            return Ok(Some(spec));
+        }
+    }
+    SortSpec::parse(configured).context("read the configured sort order")
+}
+
+/// One fragment as detection sees it.
+///
+/// `rows` is the live row count the manifest records — physical rows less the
+/// deletions — and is None when the manifest does not say, which older
+/// fragments do not.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct FragmentSummary {
+    pub files: Vec<String>,
+    pub rows: Option<u64>,
+}
+
+/// Every data file of these fragments, in order.
+pub fn data_files(fragments: &[FragmentSummary]) -> Vec<&str> {
+    fragments
+        .iter()
+        .flat_map(|fragment| fragment.files.iter().map(String::as_str))
+        .collect()
+}
+
+/// A digest of data file names, in the order the fragments hold them.
+///
+/// FNV-1a rather than the standard library's hasher, whose output is explicitly
+/// not stable across releases: a marker that hashed differently after a
+/// toolchain upgrade would re-sort every table once, silently.
+pub fn digest<S: AsRef<str>>(files: &[S]) -> String {
+    let mut hash: u64 = 0xcbf2_9ce4_8422_2325;
+    for file in files {
+        for byte in file.as_ref().as_bytes().iter().chain(std::iter::once(&0u8)) {
+            hash ^= *byte as u64;
+            hash = hash.wrapping_mul(0x1000_0000_01b3);
+        }
+    }
+    format!("{hash:016x}")
+}
+
+/// What the worker recorded on a table the last time it sorted it.
+#[derive(Debug, Clone, Default, PartialEq, Eq)]
+pub struct SortState {
+    pub fields: Option<String>,
+    pub rows: Option<u64>,
+    /// The fragments the sort wrote: how many, and a digest of their data file
+    /// names. Data file names are chosen before the commit, which is what makes
+    /// them usable in a marker the same commit carries.
+    pub fragments: Option<usize>,
+    pub digest: Option<String>,
+}
+
+impl SortState {
+    /// Reads the marker out of a table's key/value configuration. A key that
+    /// will not parse is treated as absent, which asks for a sort rather than
+    /// skipping the table: the cost of a needless sort is time, and the cost of
+    /// skipping is a table that silently never gets sorted again.
+    pub fn from_config(config: &HashMap<String, String>) -> Self {
+        Self {
+            fields: config.get(SORTED_FIELDS_KEY).cloned(),
+            rows: config.get(SORTED_ROWS_KEY).and_then(|v| v.parse().ok()),
+            fragments: config
+                .get(SORTED_FRAGMENTS_KEY)
+                .and_then(|v| v.parse().ok()),
+            digest: config.get(SORTED_DIGEST_KEY).cloned(),
+        }
+    }
+
+    /// The marker to write for a sort that just finished.
+    pub fn record(
+        spec: &SortSpec,
+        fragments: &[FragmentSummary],
+        rows: u64,
+    ) -> HashMap<String, String> {
+        HashMap::from([
+            (SORTED_FIELDS_KEY.to_string(), spec.to_string()),
+            (SORTED_ROWS_KEY.to_string(), rows.to_string()),
+            (
+                SORTED_FRAGMENTS_KEY.to_string(),
+                fragments.len().to_string(),
+            ),
+            (
+                SORTED_DIGEST_KEY.to_string(),
+                digest(&data_files(fragments)),
+            ),
+        ])
+    }
+}
+
+/// Why a table does or does not need sorting.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub enum SortVerdict {
+    NeverSorted,
+    OrderChanged,
+    RowsAppended(u64),
+    /// The table was committed to since the sort, and the row count did not
+    /// grow — the data is not the data that was sorted.
+    Rewritten,
+    UpToDate,
+}
+
+impl SortVerdict {
+    pub fn needs_sort(&self) -> bool {
+        !matches!(self, Self::UpToDate)
+    }
+
+    pub fn reason(&self) -> String {
+        match self {
+            Self::NeverSorted => "never sorted".to_string(),
+            Self::OrderChanged => "the declared order changed since the last sort".to_string(),
+            Self::RowsAppended(rows) => format!("{rows} rows appended since the last sort"),
+            Self::Rewritten => "the data was replaced since the last sort".to_string(),
+            Self::UpToDate => "sorted".to_string(),
+        }
+    }
+}
+
+/// Decides whether a table is worth sorting.
+///
+/// The question detection actually has to answer is "is this still the data the
+/// sort wrote?", and neither the row count nor the dataset version can answer
+/// it. A rewrite that leaves the row count where it was is invisible to the
+/// first, and the second cannot be recorded at all: the commit carrying the
+/// marker is the commit that creates the version, and a commit that rebases
+/// past a conflict lands on a different number again.
+///
+/// Data file names can answer it. They are chosen before the commit, so the
+/// marker the same commit carries can name them, and they are stable no matter
+/// which version the commit ends up as. So:
+///
+/// - the same files → the table is exactly what the sort left;
+/// - the sorted files still there, with more after them → rows were appended,
+///   and only then is `min_unsorted_rows` consulted, which is the churn that
+///   threshold exists to damp;
+/// - anything else → the data was replaced, whatever the row count says.
+///
+/// Appends only ever add fragments after the existing ones, and lance hands out
+/// fragment ids monotonically even across an overwrite, so "the sorted files
+/// are still there" is a prefix test on the files in fragment order.
+///
+/// Deletes leave the data files alone and write a deletion file beside them, so
+/// they read as unchanged here — which is right: deleting rows does not unsort
+/// the ones that remain.
+pub fn verdict(
+    spec: &SortSpec,
+    state: &SortState,
+    fragments: &[FragmentSummary],
+    min_unsorted_rows: u64,
+) -> SortVerdict {
+    let (Some(recorded_digest), Some(recorded_fragments)) =
+        (state.digest.as_deref(), state.fragments)
+    else {
+        return SortVerdict::NeverSorted;
+    };
+    if state.fields.as_deref() != Some(spec.to_string().as_str()) {
+        return SortVerdict::OrderChanged;
+    }
+    if digest(&data_files(fragments)) == recorded_digest {
+        return SortVerdict::UpToDate;
+    }
+
+    // Not the same files. Only a prefix match means the sorted data survived
+    // and the rest was appended after it.
+    if recorded_fragments > fragments.len()
+        || digest(&data_files(&fragments[..recorded_fragments])) != recorded_digest
+    {
+        return SortVerdict::Rewritten;
+    }
+
+    // How many rows are in the fragments appended after the sorted ones.
+    //
+    // Not the table's net growth against the recorded row count: rows deleted
+    // from the sorted fragments hide appended rows one for one, and enough of
+    // them hide every appended row, leaving a table that reads as sorted with
+    // an unsorted tail. Counting the appended fragments themselves is the same
+    // arithmetic the threshold was always meant to do.
+    let mut appended = 0u64;
+    for fragment in &fragments[recorded_fragments..] {
+        // A fragment whose length the manifest does not record cannot be
+        // counted, and a table that cannot be judged is one to sort rather than
+        // one to leave alone forever.
+        let Some(rows) = fragment.rows else {
+            return SortVerdict::Rewritten;
+        };
+        appended += rows;
+    }
+    if appended >= min_unsorted_rows.max(1) {
+        return SortVerdict::RowsAppended(appended);
+    }
+    SortVerdict::UpToDate
+}
+
+/// The settings every sorting job offers, so admin renders one form for the
+/// same behaviour whichever format the job sorts.
+pub fn config_fields() -> Vec<ConfigField> {
+    vec![
+        text_field(
+            CONFIG_SORT_FIELDS,
+            "Sort fields",
+            "Order for tables that declare none of their own, as \"user_id asc, ts desc nulls-last\". Empty sorts only the tables that declare an order.",
+            "user_id asc, ts desc",
+        ),
+        number_field(
+            CONFIG_MIN_UNSORTED_ROWS,
+            "Minimum unsorted rows",
+            "Rows appended since the last sort before a table is worth sorting again",
+            1,
+            1_000_000_000,
+        ),
+        number_field(
+            CONFIG_MEMORY_BUDGET_MB,
+            "Memory budget (MB)",
+            "Memory the sort may hold before it spills runs to disk. A table larger than this sorts more slowly rather than failing.",
+            64,
+            1_048_576,
+        ),
+        number_field(
+            CONFIG_MAX_ROWS_PER_FILE,
+            "Maximum rows per file",
+            "Rows to write into each output file of a sorted rewrite",
+            1024,
+            16_777_216,
+        ),
+    ]
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn parses_direction_and_null_order() {
+        let spec = SortSpec::parse("user_id, ts desc, name asc nulls-last")
+            .unwrap()
+            .unwrap();
+        assert_eq!(
+            spec.fields,
+            vec![
+                // Ascending defaults to nulls first, descending to nulls last,
+                // which is the rule the Iceberg tables sharing this spelling use.
+                SortField {
+                    path: "user_id".into(),
+                    descending: false,
+                    nulls_first: true
+                },
+                SortField {
+                    path: "ts".into(),
+                    descending: true,
+                    nulls_first: false
+                },
+                SortField {
+                    path: "name".into(),
+                    descending: false,
+                    nulls_first: false
+                },
+            ]
+        );
+    }
+
+    #[test]
+    fn renders_back_to_something_that_parses_the_same() {
+        let spec = SortSpec::parse("a, b desc nulls_first").unwrap().unwrap();
+        let rendered = spec.to_string();
+        assert_eq!(rendered, "a asc nulls-first, b desc nulls-first");
+        assert_eq!(SortSpec::parse(&rendered).unwrap().unwrap(), spec);
+    }
+
+    #[test]
+    fn empty_is_no_spec_rather_than_an_error() {
+        assert!(SortSpec::parse("").unwrap().is_none());
+        assert!(SortSpec::parse("  ,  ").unwrap().is_none());
+    }
+
+    #[test]
+    fn rejects_nonsense_and_repeats() {
+        assert!(SortSpec::parse("a sideways").is_err());
+        assert!(SortSpec::parse("a asc desc").is_err());
+        assert!(SortSpec::parse("a, a desc").is_err());
+    }
+
+    #[test]
+    fn the_table_wins_and_configuration_fills_in() {
+        let configured = "b desc";
+        let from_table = resolve(Some("a asc"), configured).unwrap().unwrap();
+        assert_eq!(from_table.fields[0].path, "a");
+
+        let from_config = resolve(None, configured).unwrap().unwrap();
+        assert_eq!(from_config.fields[0].path, "b");
+
+        let from_config_again = resolve(Some("  "), configured).unwrap().unwrap();
+        assert_eq!(from_config_again.fields[0].path, "b");
+
+        assert!(resolve(None, "").unwrap().is_none());
+    }
+
+    // Falling back would sort the table by an order nobody asked for.
+    #[test]
+    fn a_broken_declaration_is_an_error_not_a_fallback() {
+        assert!(resolve(Some("a sideways"), "b desc").is_err());
+    }
+
+    /// One fragment per name, one file each, with a known row count.
+    fn frags(names: &[(&str, u64)]) -> Vec<FragmentSummary> {
+        names
+            .iter()
+            .map(|(name, rows)| FragmentSummary {
+                files: vec![name.to_string()],
+                rows: Some(*rows),
+            })
+            .collect()
+    }
+
+    #[test]
+    fn verdicts_follow_the_marker() {
+        let spec = SortSpec::parse("a asc").unwrap().unwrap();
+        let sorted = frags(&[("a.lance", 500), ("b.lance", 500)]);
+
+        let fresh = SortState::default();
+        assert_eq!(
+            verdict(&spec, &fresh, &sorted, 100),
+            SortVerdict::NeverSorted
+        );
+
+        let state = SortState {
+            fields: Some(spec.to_string()),
+            rows: Some(1_000),
+            fragments: Some(sorted.len()),
+            digest: Some(digest(&data_files(&sorted))),
+        };
+
+        // The same files, whatever version the commit ended up as.
+        assert_eq!(verdict(&spec, &state, &sorted, 100), SortVerdict::UpToDate);
+
+        // Appended: the sorted fragments are still the first ones.
+        let appended_50 = frags(&[("a.lance", 500), ("b.lance", 500), ("c.lance", 50)]);
+        assert_eq!(
+            verdict(&spec, &state, &appended_50, 100),
+            SortVerdict::UpToDate
+        );
+        let appended_100 = frags(&[("a.lance", 500), ("b.lance", 500), ("c.lance", 100)]);
+        assert_eq!(
+            verdict(&spec, &state, &appended_100, 100),
+            SortVerdict::RowsAppended(100)
+        );
+
+        // Replaced: different files, and the row count is no defence — this is
+        // the case a row-count threshold lets through, whatever the delta.
+        let replaced = frags(&[("x.lance", 500), ("y.lance", 501)]);
+        assert_eq!(
+            verdict(&spec, &state, &replaced, 100),
+            SortVerdict::Rewritten
+        );
+        // Compacted into fewer files is a replacement too.
+        assert_eq!(
+            verdict(&spec, &state, &frags(&[("merged.lance", 1_000)]), 100),
+            SortVerdict::Rewritten
+        );
+
+        // Deleting rows rewrites no data file, so the table still reads as
+        // sorted — which it is.
+        let after_deletes = frags(&[("a.lance", 500), ("b.lance", 400)]);
+        assert_eq!(
+            verdict(&spec, &state, &after_deletes, 100),
+            SortVerdict::UpToDate
+        );
+
+        let other = SortSpec::parse("b desc").unwrap().unwrap();
+        assert_eq!(
+            verdict(&other, &state, &sorted, 100),
+            SortVerdict::OrderChanged
+        );
+    }
+
+    /// Deletions must not hide appended rows. Net growth against the recorded
+    /// row count would say this table shrank, and it would stay unsorted with
+    /// an unsorted tail for as long as the deletes kept pace with the appends.
+    #[test]
+    fn deletions_do_not_mask_appended_rows() {
+        let spec = SortSpec::parse("a asc").unwrap().unwrap();
+        let sorted = frags(&[("a.lance", 500), ("b.lance", 500)]);
+        let state = SortState {
+            fields: Some(spec.to_string()),
+            rows: Some(1_000),
+            fragments: Some(sorted.len()),
+            digest: Some(digest(&data_files(&sorted))),
+        };
+
+        // 800 rows deleted from the sorted fragments, 300 appended after them:
+        // the table is smaller than it was, and 300 rows of it are unsorted.
+        let mixed = frags(&[("a.lance", 100), ("b.lance", 100), ("c.lance", 300)]);
+        assert_eq!(
+            verdict(&spec, &state, &mixed, 100),
+            SortVerdict::RowsAppended(300)
+        );
+
+        // The threshold still damps a small append beside heavy deletion.
+        let small = frags(&[("a.lance", 100), ("b.lance", 100), ("c.lance", 20)]);
+        assert_eq!(verdict(&spec, &state, &small, 100), SortVerdict::UpToDate);
+
+        // And net growth is still detected where there are no deletions.
+        let grown = frags(&[("a.lance", 500), ("b.lance", 500), ("c.lance", 700)]);
+        assert_eq!(
+            verdict(&spec, &state, &grown, 100),
+            SortVerdict::RowsAppended(700)
+        );
+    }
+
+    // An appended fragment whose length the manifest does not record cannot be
+    // measured against the threshold, and a table that cannot be judged is one
+    // to sort rather than one to leave alone forever.
+    #[test]
+    fn an_uncountable_appended_fragment_is_stale() {
+        let spec = SortSpec::parse("a asc").unwrap().unwrap();
+        let sorted = frags(&[("a.lance", 500)]);
+        let state = SortState {
+            fields: Some(spec.to_string()),
+            rows: Some(500),
+            fragments: Some(1),
+            digest: Some(digest(&data_files(&sorted))),
+        };
+
+        let mut appended = sorted.clone();
+        appended.push(FragmentSummary {
+            files: vec!["b.lance".to_string()],
+            rows: None,
+        });
+        assert_eq!(
+            verdict(&spec, &state, &appended, 100),
+            SortVerdict::Rewritten
+        );
+        // The untouched table is still up to date: the files answer that
+        // without needing to count anything.
+        assert_eq!(verdict(&spec, &state, &sorted, 100), SortVerdict::UpToDate);
+    }
+
+    // Distinct columns whose names differ only in case are two columns in an
+    // Arrow schema, not one named twice.
+    #[test]
+    fn case_distinguishes_two_fields() {
+        let spec = SortSpec::parse("id asc, ID desc").unwrap().unwrap();
+        assert_eq!(spec.fields.len(), 2);
+        assert!(SortSpec::parse("id asc, id desc").is_err());
+    }
+
+    #[test]
+    fn the_marker_round_trips_through_a_config_map() {
+        let spec = SortSpec::parse("a asc").unwrap().unwrap();
+        let written = frags(&[("one.lance", 20), ("two.lance", 22)]);
+        let recorded = SortState::record(&spec, &written, 42);
+        let state = SortState::from_config(&recorded);
+        assert_eq!(state.rows, Some(42));
+        assert_eq!(state.fragments, Some(2));
+        assert_eq!(verdict(&spec, &state, &written, 1), SortVerdict::UpToDate);
+    }
+}

seaweedfs-rdma-sidecar/.dockerignore

@@ -1,65 +0,0 @@
-# Git
-.git
-.gitignore
-.gitmodules
-
-# Documentation
-*.md
-docs/
-
-# Development files
-.vscode/
-.idea/
-*.swp
-*.swo
-*~
-
-# OS generated files
-.DS_Store
-.DS_Store?
-._*
-.Spotlight-V100
-.Trashes
-ehthumbs.db
-Thumbs.db
-
-# Build artifacts
-# bin/ (commented out for Docker build - needed for mount container)
-# target/ (commented out for Docker build)
-*.exe
-*.dll
-*.so
-*.dylib
-
-# Go specific
-vendor/
-*.test
-*.prof
-go.work
-go.work.sum
-
-# Rust specific  
-Cargo.lock
-# rdma-engine/target/ (commented out for Docker build)
-*.pdb
-
-# Docker
-Dockerfile*
-docker-compose*.yml
-.dockerignore
-
-# Test files (tests/ needed for integration test container)
-# tests/
-# scripts/ (commented out for Docker build - needed for mount container)
-*.log
-
-# Temporary files
-tmp/
-temp/
-*.tmp
-*.temp
-
-# IDE and editor files
-*.sublime-*
-.vscode/
-.idea/

seaweedfs-rdma-sidecar/CORRECT-SIDECAR-APPROACH.md

@@ -1,196 +0,0 @@
-# ✅ Correct RDMA Sidecar Approach - Simple Parameter-Based
-
-## 🎯 **You're Right - Simplified Architecture**
-
-The RDMA sidecar should be **simple** and just take the volume server address as a parameter. The volume lookup complexity should stay in `weed mount`, not in the sidecar.
-
-## 🏗️ **Correct Architecture**
-
-### **1. weed mount (Client Side) - Does Volume Lookup**
-```go
-// File: weed/mount/filehandle_read.go (integration point)
-func (fh *FileHandle) tryRDMARead(ctx context.Context, buff []byte, offset int64) (int64, int64, error) {
-    entry := fh.GetEntry()
-    
-    for _, chunk := range entry.GetEntry().Chunks {
-        if offset >= chunk.Offset && offset < chunk.Offset+int64(chunk.Size) {
-            // Parse chunk info
-            volumeID, needleID, cookie, err := ParseFileId(chunk.FileId)
-            if err != nil {
-                return 0, 0, err
-            }
-            
-            // 🔍 VOLUME LOOKUP (in weed mount, not sidecar)
-            volumeServerAddr, err := fh.wfs.lookupVolumeServer(ctx, volumeID)
-            if err != nil {
-                return 0, 0, err
-            }
-            
-            // 🚀 SIMPLE RDMA REQUEST WITH VOLUME SERVER PARAMETER
-            data, isRDMA, err := fh.wfs.rdmaClient.ReadNeedleFromServer(
-                ctx, volumeServerAddr, volumeID, needleID, cookie, chunkOffset, readSize)
-            
-            return int64(copy(buff, data)), time.Now().UnixNano(), nil
-        }
-    }
-}
-```
-
-### **2. RDMA Mount Client - Passes Volume Server Address**
-```go
-// File: weed/mount/rdma_client.go (modify existing)
-func (c *RDMAMountClient) ReadNeedleFromServer(ctx context.Context, volumeServerAddr string, volumeID uint32, needleID uint64, cookie uint32, offset, size uint64) ([]byte, bool, error) {
-    // Simple HTTP request with volume server as parameter
-    reqURL := fmt.Sprintf("http://%s/rdma/read", c.sidecarAddr)
-    
-    requestBody := map[string]interface{}{
-        "volume_server": volumeServerAddr,  // ← KEY: Pass volume server address
-        "volume_id":     volumeID,
-        "needle_id":     needleID,
-        "cookie":        cookie,
-        "offset":        offset,
-        "size":          size,
-    }
-    
-    // POST request with volume server parameter
-    jsonBody, err := json.Marshal(requestBody)
-    if err != nil {
-        return nil, false, fmt.Errorf("failed to marshal request body: %w", err)
-    }
-    resp, err := c.httpClient.Post(reqURL, "application/json", bytes.NewBuffer(jsonBody))
-    if err != nil {
-        return nil, false, fmt.Errorf("http post to sidecar: %w", err)
-    }
-}
-```
-
-### **3. RDMA Sidecar - Simple, No Lookup Logic**
-```go
-// File: seaweedfs-rdma-sidecar/cmd/demo-server/main.go
-func (s *DemoServer) rdmaReadHandler(w http.ResponseWriter, r *http.Request) {
-    if r.Method != http.MethodPost {
-        http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
-        return
-    }
-    
-    // Parse request body
-    var req struct {
-        VolumeServer string `json:"volume_server"`  // ← Receive volume server address
-        VolumeID     uint32 `json:"volume_id"`
-        NeedleID     uint64 `json:"needle_id"`
-        Cookie       uint32 `json:"cookie"`
-        Offset       uint64 `json:"offset"`
-        Size         uint64 `json:"size"`
-    }
-    
-    if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
-        http.Error(w, "Invalid request", http.StatusBadRequest)
-        return
-    }
-    
-    s.logger.WithFields(logrus.Fields{
-        "volume_server": req.VolumeServer,  // ← Use provided volume server
-        "volume_id":     req.VolumeID,
-        "needle_id":     req.NeedleID,
-    }).Info("📖 Processing RDMA read with volume server parameter")
-    
-    // 🚀 SIMPLE: Use the provided volume server address
-    // No complex lookup logic needed!
-    resp, err := s.rdmaClient.ReadFromVolumeServer(r.Context(), req.VolumeServer, req.VolumeID, req.NeedleID, req.Cookie, req.Offset, req.Size)
-    
-    if err != nil {
-        http.Error(w, fmt.Sprintf("RDMA read failed: %v", err), http.StatusInternalServerError)
-        return
-    }
-    
-    // Return binary data
-    w.Header().Set("Content-Type", "application/octet-stream")
-    w.Header().Set("X-RDMA-Used", "true")
-    w.Write(resp.Data)
-}
-```
-
-### **4. Volume Lookup in weed mount (Where it belongs)**
-```go
-// File: weed/mount/weedfs.go (add method)
-func (wfs *WFS) lookupVolumeServer(ctx context.Context, volumeID uint32) (string, error) {
-    // Use existing SeaweedFS volume lookup logic
-    vid := fmt.Sprintf("%d", volumeID)
-    
-    // Query master server for volume location
-    locations, err := operation.LookupVolumeId(wfs.getMasterFn(), wfs.option.GrpcDialOption, vid)
-    if err != nil {
-        return "", fmt.Errorf("volume lookup failed: %w", err)
-    }
-    
-    if len(locations.Locations) == 0 {
-        return "", fmt.Errorf("no locations found for volume %d", volumeID)
-    }
-    
-    // Return first available location (or implement smart selection)
-    return locations.Locations[0].Url, nil
-}
-```
-
-## 🎯 **Key Differences from Over-Complicated Approach**
-
-### **❌ Over-Complicated (What I Built Before):**
-- ❌ Sidecar does volume lookup
-- ❌ Sidecar has master client integration  
-- ❌ Sidecar has volume location caching
-- ❌ Sidecar forwards requests to remote sidecars
-- ❌ Complex distributed logic in sidecar
-
-### **✅ Correct Simple Approach:**
-- ✅ **weed mount** does volume lookup (where it belongs)
-- ✅ **weed mount** passes volume server address to sidecar
-- ✅ **Sidecar** is simple and stateless
-- ✅ **Sidecar** just does local RDMA read for given server
-- ✅ **No complex distributed logic in sidecar**
-
-## 🚀 **Request Flow (Corrected)**
-
-1. **User Application** → `read()` system call
-2. **FUSE** → `weed mount` WFS.Read()
-3. **weed mount** → Volume lookup: "Where is volume 7?"
-4. **SeaweedFS Master** → "Volume 7 is on server-B:8080"
-5. **weed mount** → HTTP POST to sidecar: `{volume_server: "server-B:8080", volume: 7, needle: 12345}`
-6. **RDMA Sidecar** → Connect to server-B:8080, do local RDMA read
-7. **RDMA Engine** → Direct memory access to volume file
-8. **Response** → Binary data back to weed mount → user
-
-## 📝 **Implementation Changes Needed**
-
-### **1. Simplify Sidecar (Remove Complex Logic)**
-- Remove `DistributedRDMAClient`
-- Remove volume lookup logic
-- Remove master client integration
-- Keep simple RDMA engine communication
-
-### **2. Add Volume Lookup to weed mount**
-- Add `lookupVolumeServer()` method to WFS
-- Modify `RDMAMountClient` to accept volume server parameter
-- Integrate with existing SeaweedFS volume lookup
-
-### **3. Simple Sidecar API**
-```
-POST /rdma/read
-{
-  "volume_server": "server-B:8080",
-  "volume_id": 7,
-  "needle_id": 12345,
-  "cookie": 0,
-  "offset": 0,
-  "size": 4096
-}
-```
-
-## ✅ **Benefits of Simple Approach**
-
-- **🎯 Single Responsibility**: Sidecar only does RDMA, weed mount does lookup
-- **🔧 Maintainable**: Less complex logic in sidecar
-- **⚡ Performance**: No extra network hops for volume lookup
-- **🏗️ Clean Architecture**: Separation of concerns
-- **🐛 Easier Debugging**: Clear responsibility boundaries
-
-You're absolutely right - this is much cleaner! The sidecar should be a simple RDMA accelerator, not a distributed system coordinator.

seaweedfs-rdma-sidecar/CURRENT-STATUS.md

@@ -1,165 +0,0 @@
-# SeaweedFS RDMA Sidecar - Current Status Summary
-
-## 🎉 **IMPLEMENTATION COMPLETE** 
-**Status**: ✅ **READY FOR PRODUCTION** (Mock Mode) / 🔄 **READY FOR HARDWARE INTEGRATION**
-
----
-
-## 📊 **What's Working Right Now**
-
-### ✅ **Complete Integration Pipeline**
-- **SeaweedFS Mount** → **Go Sidecar** → **Rust Engine** → **Mock RDMA**
-- End-to-end data flow with proper error handling
-- Zero-copy page cache optimization
-- Connection pooling for performance
-
-### ✅ **Production-Ready Components**
-- HTTP API with RESTful endpoints
-- Robust health checks and monitoring
-- Docker multi-service orchestration
-- Comprehensive error handling and fallback
-- Volume lookup and server discovery
-
-### ✅ **Performance Features**
-- **Zero-Copy**: Direct kernel page cache population
-- **Connection Pooling**: Reused IPC connections
-- **Async Operations**: Non-blocking I/O throughout
-- **Metrics**: Detailed performance monitoring
-
-### ✅ **Code Quality**
-- All GitHub PR review comments addressed
-- Memory-safe operations (no dangerous channel closes)
-- Proper file ID parsing using SeaweedFS functions
-- RESTful API design with correct HTTP methods
-
----
-
-## 🔄 **What's Mock/Simulated**
-
-### 🟡 **Mock RDMA Engine** (Rust)
-- **Location**: `rdma-engine/src/rdma.rs`
-- **Function**: Simulates RDMA hardware operations
-- **Data**: Generates pattern data (0,1,2...255,0,1,2...)
-- **Performance**: Realistic latency simulation (150ns reads)
-
-### 🟡 **Simulated Hardware**
-- **Device Info**: Mock Mellanox ConnectX-5 capabilities
-- **Memory Regions**: Fake registration without HCA
-- **Transfers**: Pattern generation instead of network transfer
-- **Completions**: Synthetic work completions
-
----
-
-## 📈 **Current Performance**
-- **Throughput**: ~403 operations/second
-- **Latency**: ~2.48ms average (mock overhead)
-- **Success Rate**: 100% in integration tests
-- **Memory Usage**: Optimized with zero-copy
-
----
-
-## 🏗️ **Architecture Overview**
-
-```
-┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
-│   SeaweedFS     │────▶│   Go Sidecar    │────▶│  Rust Engine    │
-│   Mount Client  │     │   HTTP Server   │     │  Mock RDMA      │
-│   (REAL)        │     │   (REAL)        │     │  (MOCK)         │
-└─────────────────┘     └─────────────────┘     └─────────────────┘
-         │                       │                       │
-         ▼                       ▼                       ▼
-┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
-│ - File ID Parse │     │ - Zero-Copy     │     │ - UCX Ready     │
-│ - Volume Lookup │     │ - Conn Pooling  │     │ - Memory Mgmt   │
-│ - HTTP Fallback │     │ - Health Checks │     │ - IPC Protocol  │
-│ - Error Handling│     │ - REST API      │     │ - Async Ops     │
-└─────────────────┘     └─────────────────┘     └─────────────────┘
-```
-
----
-
-## 🔧 **Key Files & Locations**
-
-### **Core Integration**
-- `weed/mount/filehandle_read.go` - RDMA read integration in FUSE
-- `weed/mount/rdma_client.go` - Mount client RDMA communication
-- `cmd/demo-server/main.go` - Main RDMA sidecar HTTP server
-
-### **RDMA Engine**
-- `rdma-engine/src/rdma.rs` - Mock RDMA implementation
-- `rdma-engine/src/ipc.rs` - IPC protocol with Go sidecar
-- `pkg/rdma/client.go` - Go client for RDMA engine
-
-### **Configuration**
-- `docker-compose.mount-rdma.yml` - Complete integration test setup
-- `go.mod` - Dependencies with local SeaweedFS replacement
-
----
-
-## 🚀 **Ready For Next Steps**
-
-### **Immediate Capability**
-- ✅ **Development**: Full testing without RDMA hardware
-- ✅ **Integration Testing**: Complete pipeline validation
-- ✅ **Performance Benchmarking**: Baseline metrics
-- ✅ **CI/CD**: Mock mode for automated testing
-
-### **Production Transition**
-- 🔄 **Hardware Integration**: Replace mock with UCX library
-- 🔄 **Real Data Transfer**: Remove pattern generation
-- 🔄 **Device Detection**: Enumerate actual RDMA NICs
-- 🔄 **Performance Optimization**: Hardware-specific tuning
-
----
-
-## 📋 **Commands to Resume Work**
-
-### **Start Development Environment**
-```bash
-# Navigate to your seaweedfs-rdma-sidecar directory
-cd /path/to/your/seaweedfs/seaweedfs-rdma-sidecar
-
-# Build components
-go build -o bin/demo-server ./cmd/demo-server
-cargo build --manifest-path rdma-engine/Cargo.toml
-
-# Run integration tests
-docker-compose -f docker-compose.mount-rdma.yml up
-```
-
-### **Test Current Implementation**
-```bash
-# Test sidecar HTTP API
-curl http://localhost:8081/health
-curl http://localhost:8081/stats
-
-# Test RDMA read
-curl "http://localhost:8081/read?volume=1&needle=123&cookie=456&offset=0&size=1024&volume_server=http://localhost:8080"
-```
-
----
-
-## 🎯 **Success Metrics Achieved**
-
-- ✅ **Functional**: Complete RDMA integration pipeline
-- ✅ **Reliable**: Robust error handling and fallback
-- ✅ **Performant**: Zero-copy and connection pooling
-- ✅ **Testable**: Comprehensive mock implementation
-- ✅ **Maintainable**: Clean code with proper documentation
-- ✅ **Scalable**: Async operations and pooling
-- ✅ **Production-Ready**: All review comments addressed
-
----
-
-## 📚 **Documentation**
-
-- `FUTURE-WORK-TODO.md` - Next steps for hardware integration
-- `DOCKER-TESTING.md` - Integration testing guide
-- `docker-compose.mount-rdma.yml` - Complete test environment
-- GitHub PR reviews - All issues addressed and documented
-
----
-
-**🏆 ACHIEVEMENT**: Complete RDMA sidecar architecture with production-ready infrastructure and seamless mock-to-real transition path!
-
-**Next**: Follow `FUTURE-WORK-TODO.md` to replace mock with real UCX hardware integration.

seaweedfs-rdma-sidecar/DOCKER-TESTING.md

@@ -1,290 +0,0 @@
-# 🐳 Docker Integration Testing Guide
-
-This guide provides comprehensive Docker-based integration testing for the SeaweedFS RDMA sidecar system.
-
-## 🏗️ Architecture
-
-The Docker Compose setup includes:
-
-```
-┌─────────────────────┐    ┌─────────────────────┐    ┌─────────────────────┐
-│   SeaweedFS Master  │    │  SeaweedFS Volume   │    │    Rust RDMA       │
-│     :9333          │◄──►│     :8080          │    │    Engine           │
-└─────────────────────┘    └─────────────────────┘    └─────────────────────┘
-                                    │                           │
-                                    ▼                           ▼
-┌─────────────────────┐    ┌─────────────────────┐    ┌─────────────────────┐
-│   Go RDMA Sidecar  │◄──►│    Unix Socket      │◄──►│   Integration       │
-│     :8081          │    │   /tmp/rdma.sock    │    │   Test Suite        │
-└─────────────────────┘    └─────────────────────┘    └─────────────────────┘
-```
-
-## 🚀 Quick Start
-
-### 1. Start All Services
-
-```bash
-# Using the helper script (recommended)
-./tests/docker-test-helper.sh start
-
-# Or using docker-compose directly
-docker-compose up -d
-```
-
-### 2. Run Integration Tests
-
-```bash
-# Run the complete test suite
-./tests/docker-test-helper.sh test
-
-# Or run tests manually
-docker-compose run --rm integration-tests
-```
-
-### 3. Interactive Testing
-
-```bash
-# Open a shell in the test container
-./tests/docker-test-helper.sh shell
-
-# Inside the container, you can run:
-./test-rdma ping
-./test-rdma capabilities  
-./test-rdma read --volume 1 --needle 12345 --size 1024
-curl http://rdma-sidecar:8081/health
-curl http://rdma-sidecar:8081/stats
-```
-
-## 📋 Test Helper Commands
-
-The `docker-test-helper.sh` script provides convenient commands:
-
-```bash
-# Service Management
-./tests/docker-test-helper.sh start      # Start all services
-./tests/docker-test-helper.sh stop       # Stop all services  
-./tests/docker-test-helper.sh clean      # Stop and clean volumes
-
-# Testing
-./tests/docker-test-helper.sh test       # Run integration tests
-./tests/docker-test-helper.sh shell      # Interactive testing shell
-
-# Monitoring
-./tests/docker-test-helper.sh status     # Check service health
-./tests/docker-test-helper.sh logs       # Show all logs
-./tests/docker-test-helper.sh logs rdma-engine  # Show specific service logs
-```
-
-## 🧪 Test Coverage
-
-The integration test suite covers:
-
-### ✅ Core Components
-- **SeaweedFS Master**: Cluster leadership and status
-- **SeaweedFS Volume Server**: Volume operations and health
-- **Rust RDMA Engine**: Socket communication and operations
-- **Go RDMA Sidecar**: HTTP API and RDMA integration
-
-### ✅ Integration Points
-- **IPC Communication**: Unix socket + MessagePack protocol
-- **RDMA Operations**: Ping, capabilities, read operations
-- **HTTP API**: All sidecar endpoints and error handling
-- **Fallback Logic**: RDMA → HTTP fallback behavior
-
-### ✅ Performance Testing
-- **Direct RDMA Benchmarks**: Engine-level performance
-- **Sidecar Benchmarks**: End-to-end performance
-- **Latency Measurements**: Operation timing validation
-- **Throughput Testing**: Operations per second
-
-## 🔧 Service Details
-
-### SeaweedFS Master
-- **Port**: 9333
-- **Health Check**: `/cluster/status`
-- **Data**: Persistent volume `master-data`
-
-### SeaweedFS Volume Server
-- **Port**: 8080  
-- **Health Check**: `/status`
-- **Data**: Persistent volume `volume-data`
-- **Depends on**: SeaweedFS Master
-
-### Rust RDMA Engine
-- **Socket**: `/tmp/rdma-engine.sock`
-- **Mode**: Mock RDMA (development)
-- **Health Check**: Socket existence
-- **Privileged**: Yes (for RDMA access)
-
-### Go RDMA Sidecar
-- **Port**: 8081
-- **Health Check**: `/health`
-- **API Endpoints**: `/stats`, `/read`, `/benchmark`
-- **Depends on**: RDMA Engine, Volume Server
-
-### Test Client
-- **Purpose**: Integration testing and interactive debugging
-- **Tools**: curl, jq, test-rdma binary
-- **Environment**: All service URLs configured
-
-## 📊 Expected Test Results
-
-### ✅ Successful Output Example
-
-```
-===============================================
-🚀 SEAWEEDFS RDMA INTEGRATION TEST SUITE  
-===============================================
-
-🔵 Waiting for SeaweedFS Master to be ready...
-✅ SeaweedFS Master is ready
-✅ SeaweedFS Master is leader and ready
-
-🔵 Waiting for SeaweedFS Volume Server to be ready...
-✅ SeaweedFS Volume Server is ready
-Volume Server Version: 3.60
-
-🔵 Checking RDMA engine socket...
-✅ RDMA engine socket exists
-🔵 Testing RDMA engine ping...
-✅ RDMA engine ping successful
-
-🔵 Waiting for RDMA Sidecar to be ready...
-✅ RDMA Sidecar is ready
-✅ RDMA Sidecar is healthy
-RDMA Status: true
-
-🔵 Testing needle read via sidecar...
-✅ Sidecar needle read successful
-⚠️  HTTP fallback used. Duration: 2.48ms
-
-🔵 Running sidecar performance benchmark...
-✅ Sidecar benchmark completed
-Benchmark Results:
-  RDMA Operations: 5
-  HTTP Operations: 0  
-  Average Latency: 2.479ms
-  Operations/sec: 403.2
-
-===============================================
-🎉 ALL INTEGRATION TESTS COMPLETED!
-===============================================
-```
-
-## 🐛 Troubleshooting
-
-### Service Not Starting
-
-```bash
-# Check service logs
-./tests/docker-test-helper.sh logs [service-name]
-
-# Check container status
-docker-compose ps
-
-# Restart specific service
-docker-compose restart [service-name]
-```
-
-### RDMA Engine Issues
-
-```bash
-# Check socket permissions
-docker-compose exec rdma-engine ls -la /tmp/rdma/rdma-engine.sock
-
-# Check RDMA engine logs
-./tests/docker-test-helper.sh logs rdma-engine
-
-# Test socket directly
-docker-compose exec test-client ./test-rdma ping
-```
-
-### Sidecar Connection Issues  
-
-```bash
-# Test sidecar health directly
-curl http://localhost:8081/health
-
-# Check sidecar logs
-./tests/docker-test-helper.sh logs rdma-sidecar
-
-# Verify environment variables
-docker-compose exec rdma-sidecar env | grep RDMA
-```
-
-### Volume Server Issues
-
-```bash
-# Check SeaweedFS status
-curl http://localhost:9333/cluster/status
-curl http://localhost:8080/status
-
-# Check volume server logs  
-./tests/docker-test-helper.sh logs seaweedfs-volume
-```
-
-## 🔍 Manual Testing Examples
-
-### Test RDMA Engine Directly
-
-```bash
-# Enter test container
-./tests/docker-test-helper.sh shell
-
-# Test RDMA operations
-./test-rdma ping --socket /tmp/rdma-engine.sock
-./test-rdma capabilities --socket /tmp/rdma-engine.sock
-./test-rdma read --socket /tmp/rdma-engine.sock --volume 1 --needle 12345
-./test-rdma bench --socket /tmp/rdma-engine.sock --iterations 10
-```
-
-### Test Sidecar HTTP API
-
-```bash
-# Health and status
-curl http://rdma-sidecar:8081/health | jq '.'
-curl http://rdma-sidecar:8081/stats | jq '.'
-
-# Needle operations
-curl "http://rdma-sidecar:8081/read?volume=1&needle=12345&size=1024" | jq '.'
-
-# Benchmarking
-curl "http://rdma-sidecar:8081/benchmark?iterations=5&size=2048" | jq '.benchmark_results'
-```
-
-### Test SeaweedFS Integration
-
-```bash
-# Check cluster status
-curl http://seaweedfs-master:9333/cluster/status | jq '.'
-
-# Check volume status  
-curl http://seaweedfs-volume:8080/status | jq '.'
-
-# List volumes
-curl http://seaweedfs-master:9333/vol/status | jq '.'
-```
-
-## 🚀 Production Deployment
-
-This Docker setup can be adapted for production by:
-
-1. **Replacing Mock RDMA**: Switch to `real-ucx` feature in Rust
-2. **RDMA Hardware**: Add RDMA device mappings and capabilities
-3. **Security**: Remove privileged mode, add proper user/group mapping  
-4. **Scaling**: Use Docker Swarm or Kubernetes for orchestration
-5. **Monitoring**: Add Prometheus metrics and Grafana dashboards
-6. **Persistence**: Configure proper volume management
-
-## 📚 Additional Resources
-
-- [Main README](README.md) - Complete project overview
-- [Docker Compose Reference](https://docs.docker.com/compose/)
-- [SeaweedFS Documentation](https://github.com/seaweedfs/seaweedfs/wiki)
-- [UCX Documentation](https://github.com/openucx/ucx)
-
----
-
-**🐳 Happy Docker Testing!** 
-
-For issues or questions, please check the logs first and refer to the troubleshooting section above.

seaweedfs-rdma-sidecar/Dockerfile.integration-test

@@ -1,25 +0,0 @@
-# Dockerfile for RDMA Mount Integration Tests
-FROM ubuntu:22.04
-
-# Install dependencies
-RUN apt-get update && apt-get install -y \
-    curl \
-    wget \
-    ca-certificates \
-    jq \
-    bc \
-    time \
-    util-linux \
-    coreutils \
-    && rm -rf /var/lib/apt/lists/*
-
-# Create test directories
-RUN mkdir -p /usr/local/bin /test-results
-
-# Copy test scripts
-COPY scripts/run-integration-tests.sh /usr/local/bin/run-integration-tests.sh
-COPY scripts/test-rdma-mount.sh /usr/local/bin/test-rdma-mount.sh
-RUN chmod +x /usr/local/bin/*.sh
-
-# Default command
-CMD ["/usr/local/bin/run-integration-tests.sh"]

seaweedfs-rdma-sidecar/Dockerfile.mount-rdma

@@ -1,40 +0,0 @@
-# Dockerfile for SeaweedFS Mount with RDMA support
-FROM ubuntu:22.04
-
-# Install dependencies
-RUN apt-get update && apt-get install -y \
-    fuse3 \
-    curl \
-    wget \
-    ca-certificates \
-    procps \
-    util-linux \
-    jq \
-    && rm -rf /var/lib/apt/lists/*
-
-# Create necessary directories
-RUN mkdir -p /usr/local/bin /mnt/seaweedfs /var/log/seaweedfs
-
-# Copy SeaweedFS binary (will be built from context)
-COPY bin/weed /usr/local/bin/weed
-RUN chmod +x /usr/local/bin/weed
-
-# Copy mount helper scripts
-COPY scripts/mount-helper.sh /usr/local/bin/mount-helper.sh
-RUN chmod +x /usr/local/bin/mount-helper.sh
-
-# Create mount point
-RUN mkdir -p /mnt/seaweedfs
-
-# Set up FUSE permissions
-RUN echo 'user_allow_other' >> /etc/fuse.conf
-
-# Health check script
-COPY scripts/mount-health-check.sh /usr/local/bin/mount-health-check.sh
-RUN chmod +x /usr/local/bin/mount-health-check.sh
-
-# Expose mount point as volume
-VOLUME ["/mnt/seaweedfs"]
-
-# Default command
-CMD ["/usr/local/bin/mount-helper.sh"]

seaweedfs-rdma-sidecar/Dockerfile.performance-test

@@ -1,26 +0,0 @@
-# Dockerfile for RDMA Mount Performance Tests
-FROM ubuntu:22.04
-
-# Install dependencies
-RUN apt-get update && apt-get install -y \
-    curl \
-    wget \
-    ca-certificates \
-    jq \
-    bc \
-    time \
-    util-linux \
-    coreutils \
-    fio \
-    iozone3 \
-    && rm -rf /var/lib/apt/lists/*
-
-# Create test directories
-RUN mkdir -p /usr/local/bin /performance-results
-
-# Copy test scripts
-COPY scripts/run-performance-tests.sh /usr/local/bin/run-performance-tests.sh
-RUN chmod +x /usr/local/bin/*.sh
-
-# Default command
-CMD ["/usr/local/bin/run-performance-tests.sh"]

seaweedfs-rdma-sidecar/Dockerfile.rdma-engine

@@ -1,63 +0,0 @@
-# Multi-stage build for Rust RDMA Engine
-FROM rust:1.80-slim AS builder
-
-# Install build dependencies
-RUN apt-get update && apt-get install -y \
-    pkg-config \
-    libssl-dev \
-    libudev-dev \
-    build-essential \
-    libc6-dev \
-    linux-libc-dev \
-    && rm -rf /var/lib/apt/lists/*
-
-# Set work directory
-WORKDIR /app
-
-# Copy Rust project files
-COPY rdma-engine/Cargo.toml ./
-COPY rdma-engine/Cargo.lock ./
-COPY rdma-engine/src ./src
-
-# Build the release binary
-RUN cargo build --release
-
-# Runtime stage
-FROM debian:bookworm-slim
-
-# Install runtime dependencies
-RUN apt-get update && apt-get install -y \
-    ca-certificates \
-    libssl3 \
-    curl \
-    && rm -rf /var/lib/apt/lists/*
-
-# Create app user
-RUN useradd -m -u 1001 appuser
-
-# Set work directory
-WORKDIR /app
-
-# Copy binary from builder stage
-COPY --from=builder /app/target/release/rdma-engine-server .
-
-# Change ownership
-RUN chown -R appuser:appuser /app
-
-# Set default socket path (can be overridden)
-ENV RDMA_SOCKET_PATH=/tmp/rdma/rdma-engine.sock
-
-# Create socket directory with proper permissions (before switching user)
-RUN mkdir -p /tmp/rdma && chown -R appuser:appuser /tmp/rdma
-
-USER appuser
-
-# Expose any needed ports (none for this service as it uses Unix sockets)
-# EXPOSE 18515
-
-# Health check - verify both process and socket using environment variable
-HEALTHCHECK --interval=5s --timeout=3s --start-period=10s --retries=3 \
-    CMD pgrep rdma-engine-server >/dev/null && test -S "$RDMA_SOCKET_PATH"
-
-# Default command using environment variable
-CMD sh -c "./rdma-engine-server --debug --ipc-socket \"$RDMA_SOCKET_PATH\""

seaweedfs-rdma-sidecar/Dockerfile.rdma-engine.simple

@@ -1,36 +0,0 @@
-# Simplified Dockerfile for Rust RDMA Engine (using pre-built binary)
-FROM debian:bookworm-slim
-
-# Install runtime dependencies
-RUN apt-get update && apt-get install -y \
-    ca-certificates \
-    libssl3 \
-    curl \
-    procps \
-    && rm -rf /var/lib/apt/lists/*
-
-# Create app user
-RUN useradd -m -u 1001 appuser
-
-# Set work directory
-WORKDIR /app
-
-# Copy pre-built binary from local build
-COPY ./rdma-engine/target/release/rdma-engine-server .
-
-# Change ownership
-RUN chown -R appuser:appuser /app
-USER appuser
-
-# Set default socket path (can be overridden)
-ENV RDMA_SOCKET_PATH=/tmp/rdma-engine.sock
-
-# Create socket directory
-RUN mkdir -p /tmp
-
-# Health check - verify both process and socket using environment variable
-HEALTHCHECK --interval=5s --timeout=3s --start-period=10s --retries=3 \
-    CMD pgrep rdma-engine-server >/dev/null && test -S "$RDMA_SOCKET_PATH"
-
-# Default command using environment variable
-CMD sh -c "./rdma-engine-server --debug --ipc-socket \"$RDMA_SOCKET_PATH\""

seaweedfs-rdma-sidecar/Dockerfile.sidecar

@@ -1,55 +0,0 @@
-# Multi-stage build for Go Sidecar
-FROM golang:1.26-alpine AS builder
-
-# Install build dependencies
-RUN apk add --no-cache git ca-certificates tzdata
-
-# Set work directory
-WORKDIR /app
-
-# Copy go mod files
-COPY go.mod go.sum ./
-
-# Download dependencies
-RUN go mod download
-
-# Copy source code
-COPY cmd/ ./cmd/
-COPY pkg/ ./pkg/
-
-# Build the binaries
-RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o demo-server ./cmd/demo-server
-RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o sidecar ./cmd/sidecar
-RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o test-rdma ./cmd/test-rdma
-
-# Runtime stage
-FROM alpine:3.18
-
-# Install runtime dependencies
-RUN apk --no-cache add ca-certificates curl jq
-
-# Create app user
-RUN addgroup -g 1001 appgroup && \
-    adduser -D -s /bin/sh -u 1001 -G appgroup appuser
-
-# Set work directory
-WORKDIR /app
-
-# Copy binaries from builder stage
-COPY --from=builder /app/demo-server .
-COPY --from=builder /app/sidecar .
-COPY --from=builder /app/test-rdma .
-
-# Change ownership
-RUN chown -R appuser:appgroup /app
-USER appuser
-
-# Expose the demo server port
-EXPOSE 8081
-
-# Health check
-HEALTHCHECK --interval=10s --timeout=5s --start-period=15s --retries=3 \
-    CMD curl -f http://localhost:8081/health || exit 1
-
-# Default command (demo server)
-CMD ["./demo-server", "--port", "8081", "--enable-rdma", "--debug"]

seaweedfs-rdma-sidecar/Dockerfile.test-client

@@ -1,59 +0,0 @@
-# Multi-stage build for Test Client
-FROM golang:1.26-alpine AS builder
-
-# Install build dependencies
-RUN apk add --no-cache git ca-certificates tzdata
-
-# Set work directory
-WORKDIR /app
-
-# Copy go mod files
-COPY go.mod go.sum ./
-
-# Download dependencies
-RUN go mod download
-
-# Copy source code
-COPY cmd/ ./cmd/
-COPY pkg/ ./pkg/
-
-# Build the test binaries
-RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o test-rdma ./cmd/test-rdma
-RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o demo-server ./cmd/demo-server
-
-# Runtime stage
-FROM alpine:3.18
-
-# Install runtime dependencies and testing tools
-RUN apk --no-cache add \
-    ca-certificates \
-    curl \
-    jq \
-    bash \
-    wget \
-    netcat-openbsd \
-    && rm -rf /var/cache/apk/*
-
-# Create app user
-RUN addgroup -g 1001 appgroup && \
-    adduser -D -s /bin/bash -u 1001 -G appgroup appuser
-
-# Set work directory
-WORKDIR /app
-
-# Copy binaries from builder stage
-COPY --from=builder /app/test-rdma .
-COPY --from=builder /app/demo-server .
-
-# Copy test scripts
-COPY tests/ ./tests/
-RUN chmod +x ./tests/*.sh
-
-# Change ownership
-RUN chown -R appuser:appgroup /app
-
-# Switch to app user
-USER appuser
-
-# Default command
-CMD ["/bin/bash"]

seaweedfs-rdma-sidecar/FUTURE-WORK-TODO.md

@@ -1,276 +0,0 @@
-# SeaweedFS RDMA Sidecar - Future Work TODO
-
-## 🎯 **Current Status (✅ COMPLETED)**
-
-### **Phase 1: Architecture & Integration - DONE**
-- ✅ **Complete Go ↔ Rust IPC Pipeline**: Unix sockets + MessagePack
-- ✅ **SeaweedFS Integration**: Mount client with RDMA acceleration
-- ✅ **Docker Orchestration**: Multi-service setup with proper networking
-- ✅ **Error Handling**: Robust fallback and recovery mechanisms
-- ✅ **Performance Optimizations**: Zero-copy page cache + connection pooling
-- ✅ **Code Quality**: All GitHub PR review comments addressed
-- ✅ **Testing Framework**: Integration tests and benchmarking tools
-
-### **Phase 2: Mock Implementation - DONE**
-- ✅ **Mock RDMA Engine**: Complete Rust implementation for development
-- ✅ **Pattern Data Generation**: Predictable test data for validation
-- ✅ **Simulated Performance**: Realistic latency and throughput modeling
-- ✅ **Development Environment**: Full testing without hardware requirements
-
----
-
-## 🚀 **PHASE 3: REAL RDMA IMPLEMENTATION**
-
-### **3.1 Hardware Abstraction Layer** 🔴 **HIGH PRIORITY**
-
-#### **Replace Mock RDMA Context**
-**File**: `rdma-engine/src/rdma.rs`
-**Current**:
-```rust
-RdmaContextImpl::Mock(MockRdmaContext::new(config).await?)
-```
-**TODO**:
-```rust
-// Enable UCX feature and implement
-RdmaContextImpl::Ucx(UcxRdmaContext::new(config).await?)
-```
-
-**Tasks**:
-- [ ] Implement `UcxRdmaContext` struct
-- [ ] Add UCX FFI bindings for Rust
-- [ ] Handle UCX initialization and cleanup
-- [ ] Add feature flag: `real-ucx` vs `mock`
-
-#### **Real Memory Management**
-**File**: `rdma-engine/src/rdma.rs` lines 245-270
-**Current**: Fake memory regions in vector
-**TODO**:
-- [ ] Integrate with UCX memory registration APIs
-- [ ] Implement HugePage support for large transfers
-- [ ] Add memory region caching for performance
-- [ ] Handle registration/deregistration errors
-
-#### **Actual RDMA Operations**
-**File**: `rdma-engine/src/rdma.rs` lines 273-335
-**Current**: Pattern data + artificial latency
-**TODO**:
-- [ ] Replace `post_read()` with real UCX RDMA operations
-- [ ] Implement `post_write()` with actual memory transfers
-- [ ] Add completion polling from hardware queues
-- [ ] Handle partial transfers and retries
-
-### **3.2 Data Path Replacement** 🟡 **MEDIUM PRIORITY**
-
-#### **Real Data Transfer**
-**File**: `pkg/rdma/client.go` lines 420-442
-**Current**:
-```go
-// MOCK: Pattern generation
-mockData[i] = byte(i % 256)
-```
-**TODO**:
-```go
-// Get actual data from RDMA buffer
-realData := getRdmaBufferContents(startResp.LocalAddr, startResp.TransferSize)
-validateDataIntegrity(realData, completeResp.ServerCrc)
-```
-
-**Tasks**:
-- [ ] Remove mock data generation
-- [ ] Access actual RDMA transferred data
-- [ ] Implement CRC validation: `completeResp.ServerCrc`
-- [ ] Add data integrity error handling
-
-#### **Hardware Device Detection**
-**File**: `rdma-engine/src/rdma.rs` lines 222-233
-**Current**: Hardcoded Mellanox device info
-**TODO**:
-- [ ] Enumerate real RDMA devices using UCX
-- [ ] Query actual device capabilities
-- [ ] Handle multiple device scenarios
-- [ ] Add device selection logic
-
-### **3.3 Performance Optimization** 🟢 **LOW PRIORITY**
-
-#### **Memory Registration Caching**
-**TODO**:
-- [ ] Implement MR (Memory Region) cache
-- [ ] Add LRU eviction for memory pressure
-- [ ] Optimize for frequently accessed regions
-- [ ] Monitor cache hit rates
-
-#### **Advanced RDMA Features**
-**TODO**:
-- [ ] Implement RDMA Write operations
-- [ ] Add Immediate Data support
-- [ ] Implement RDMA Write with Immediate
-- [ ] Add Atomic operations (if needed)
-
-#### **Multi-Transport Support**
-**TODO**:
-- [ ] Leverage UCX's automatic transport selection
-- [ ] Add InfiniBand support
-- [ ] Add RoCE (RDMA over Converged Ethernet) support
-- [ ] Implement TCP fallback via UCX
-
----
-
-## 🔧 **PHASE 4: PRODUCTION HARDENING**
-
-### **4.1 Error Handling & Recovery**
-- [ ] Add RDMA-specific error codes
-- [ ] Implement connection recovery
-- [ ] Add retry logic for transient failures
-- [ ] Handle device hot-plug scenarios
-
-### **4.2 Monitoring & Observability**
-- [ ] Add RDMA-specific metrics (bandwidth, latency, errors)
-- [ ] Implement tracing for RDMA operations
-- [ ] Add health checks for RDMA devices
-- [ ] Create performance dashboards
-
-### **4.3 Configuration & Tuning**
-- [ ] Add RDMA-specific configuration options
-- [ ] Implement auto-tuning based on workload
-- [ ] Add support for multiple RDMA ports
-- [ ] Create deployment guides for different hardware
-
----
-
-## 📋 **IMMEDIATE NEXT STEPS**
-
-### **Step 1: UCX Integration Setup**
-1. **Add UCX dependencies to Rust**:
-   ```toml
-   [dependencies]
-   ucx-sys = "0.1"  # UCX FFI bindings
-   ```
-
-2. **Create UCX wrapper module**:
-   ```bash
-   touch rdma-engine/src/ucx.rs
-   ```
-
-3. **Implement basic UCX context**:
-   ```rust
-   pub struct UcxRdmaContext {
-       context: *mut ucx_sys::ucp_context_h,
-       worker: *mut ucx_sys::ucp_worker_h,
-   }
-   ```
-
-### **Step 2: Development Environment**
-1. **Install UCX library**:
-   ```bash
-   # Ubuntu/Debian
-   sudo apt-get install libucx-dev
-   
-   # CentOS/RHEL  
-   sudo yum install ucx-devel
-   ```
-
-2. **Update Cargo.toml features**:
-   ```toml
-   [features]
-   default = ["mock"]
-   mock = []
-   real-ucx = ["ucx-sys"]
-   ```
-
-### **Step 3: Testing Strategy**
-1. **Add hardware detection tests**
-2. **Create UCX initialization tests**
-3. **Implement gradual feature migration**
-4. **Maintain mock fallback for CI/CD**
-
----
-
-## 🏗️ **ARCHITECTURE NOTES**
-
-### **Current Working Components**
-- ✅ **Go Sidecar**: Production-ready HTTP API
-- ✅ **IPC Layer**: Robust Unix socket + MessagePack
-- ✅ **SeaweedFS Integration**: Complete mount client integration
-- ✅ **Docker Setup**: Multi-service orchestration
-- ✅ **Error Handling**: Comprehensive fallback mechanisms
-
-### **Mock vs Real Boundary**
-```
-┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
-│   SeaweedFS     │────▶│   Go Sidecar    │────▶│  Rust Engine    │
-│   (REAL)        │     │   (REAL)        │     │   (MOCK)        │
-└─────────────────┘     └─────────────────┘     └─────────────────┘
-                                                          │
-                                                          ▼
-                                                 ┌─────────────────┐
-                                                 │ RDMA Hardware   │
-                                                 │ (TO IMPLEMENT)  │
-                                                 └─────────────────┘
-```
-
-### **Performance Expectations**
-- **Current Mock**: ~403 ops/sec, 2.48ms latency
-- **Target Real**: ~4000 ops/sec, 250μs latency (UCX optimized)
-- **Bandwidth Goal**: 25-100 Gbps (depending on hardware)
-
----
-
-## 📚 **REFERENCE MATERIALS**
-
-### **UCX Documentation**
-- **GitHub**: https://github.com/openucx/ucx
-- **API Reference**: https://openucx.readthedocs.io/
-- **Rust Bindings**: https://crates.io/crates/ucx-sys
-
-### **RDMA Programming**
-- **InfiniBand Architecture**: Volume 1 Specification
-- **RoCE Standards**: IBTA Annex A17
-- **Performance Tuning**: UCX Performance Guide
-
-### **SeaweedFS Integration**
-- **File ID Format**: `weed/storage/needle/file_id.go`
-- **Volume Server**: `weed/server/volume_server_handlers_read.go`
-- **Mount Client**: `weed/mount/filehandle_read.go`
-
----
-
-## ⚠️ **IMPORTANT NOTES**
-
-### **Breaking Changes to Avoid**
-- **Keep IPC Protocol Stable**: Don't change MessagePack format
-- **Maintain HTTP API**: Existing endpoints must remain compatible
-- **Preserve Configuration**: Environment variables should work unchanged
-
-### **Testing Requirements**
-- **Hardware Tests**: Require actual RDMA NICs
-- **CI/CD Compatibility**: Must fallback to mock for automated testing
-- **Performance Benchmarks**: Compare mock vs real performance
-
-### **Security Considerations**
-- **Memory Protection**: Ensure RDMA regions are properly isolated
-- **Access Control**: Validate remote memory access permissions
-- **Data Validation**: Always verify CRC checksums
-
----
-
-## 🎯 **SUCCESS CRITERIA**
-
-### **Phase 3 Complete When**:
-- [ ] Real RDMA data transfers working
-- [ ] Hardware device detection functional
-- [ ] Performance exceeds mock implementation
-- [ ] All integration tests passing with real hardware
-
-### **Phase 4 Complete When**:
-- [ ] Production deployment successful
-- [ ] Monitoring and alerting operational
-- [ ] Performance targets achieved
-- [ ] Error handling validated under load
-
----
-
-**📅 Last Updated**: December 2024
-**👤 Contact**: Resume from `seaweedfs-rdma-sidecar/` directory
-**🏷️ Version**: v1.0 (Mock Implementation Complete)
-
-**🚀 Ready to resume**: All infrastructure is in place, just need to replace the mock RDMA layer with UCX integration!

seaweedfs-rdma-sidecar/Makefile

@@ -1,205 +0,0 @@
-# SeaweedFS RDMA Sidecar Makefile
-
-.PHONY: help build test clean docker-build docker-test docker-clean integration-test
-
-# Default target
-help: ## Show this help message
-	@echo "SeaweedFS RDMA Sidecar - Available Commands:"
-	@echo ""
-	@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "  \033[36m%-20s\033[0m %s\n", $$1, $$2}'
-	@echo ""
-	@echo "Examples:"
-	@echo "  make build           # Build all components locally"
-	@echo "  make docker-test     # Run complete Docker integration tests"
-	@echo "  make test            # Run unit tests"
-
-# Local Build Targets
-build: build-go build-rust ## Build all components locally
-
-build-go: ## Build Go components (sidecar, demo-server, test-rdma)
-	@echo "🔨 Building Go components..."
-	go build -o bin/sidecar ./cmd/sidecar
-	go build -o bin/demo-server ./cmd/demo-server  
-	go build -o bin/test-rdma ./cmd/test-rdma
-	@echo "✅ Go build complete"
-
-build-rust: ## Build Rust RDMA engine
-	@echo "🦀 Building Rust RDMA engine..."
-	cd rdma-engine && cargo build --release
-	@echo "✅ Rust build complete"
-
-# Testing Targets
-test: test-go test-rust ## Run all unit tests
-
-test-go: ## Run Go tests
-	@echo "🧪 Running Go tests..."
-	go test ./...
-	@echo "✅ Go tests complete"
-
-test-rust: ## Run Rust tests
-	@echo "🧪 Running Rust tests..."
-	cd rdma-engine && cargo test
-	@echo "✅ Rust tests complete"
-
-integration-test: build ## Run local integration test
-	@echo "🔗 Running local integration test..."
-	./scripts/demo-e2e.sh
-	@echo "✅ Local integration test complete"
-
-# Docker Targets
-docker-build: ## Build all Docker images
-	@echo "🐳 Building Docker images..."
-	docker-compose build
-	@echo "✅ Docker images built"
-
-docker-start: ## Start Docker services
-	@echo "🚀 Starting Docker services..."
-	./tests/docker-test-helper.sh start
-	@echo "✅ Docker services started"
-
-docker-test: ## Run Docker integration tests
-	@echo "🧪 Running Docker integration tests..."
-	./tests/docker-test-helper.sh test
-	@echo "✅ Docker integration tests complete"
-
-docker-stop: ## Stop Docker services
-	@echo "🛑 Stopping Docker services..."
-	./tests/docker-test-helper.sh stop
-	@echo "✅ Docker services stopped"
-
-docker-clean: ## Clean Docker services and volumes
-	@echo "🧹 Cleaning Docker environment..."
-	./tests/docker-test-helper.sh clean
-	docker system prune -f
-	@echo "✅ Docker cleanup complete"
-
-docker-logs: ## Show Docker logs
-	./tests/docker-test-helper.sh logs
-
-docker-status: ## Show Docker service status
-	./tests/docker-test-helper.sh status
-
-docker-shell: ## Open interactive shell in test container
-	./tests/docker-test-helper.sh shell
-
-# RDMA Simulation Targets
-rdma-sim-build: ## Build RDMA simulation environment
-	@echo "🚀 Building RDMA simulation environment..."
-	docker-compose -f docker-compose.rdma-sim.yml build
-	@echo "✅ RDMA simulation images built"
-
-rdma-sim-start: ## Start RDMA simulation environment
-	@echo "🚀 Starting RDMA simulation environment..."
-	docker-compose -f docker-compose.rdma-sim.yml up -d
-	@echo "✅ RDMA simulation environment started"
-
-rdma-sim-test: ## Run RDMA simulation tests
-	@echo "🧪 Running RDMA simulation tests..."
-	docker-compose -f docker-compose.rdma-sim.yml run --rm integration-tests-rdma
-	@echo "✅ RDMA simulation tests complete"
-
-rdma-sim-stop: ## Stop RDMA simulation environment
-	@echo "🛑 Stopping RDMA simulation environment..."
-	docker-compose -f docker-compose.rdma-sim.yml down
-	@echo "✅ RDMA simulation environment stopped"
-
-rdma-sim-clean: ## Clean RDMA simulation environment
-	@echo "🧹 Cleaning RDMA simulation environment..."
-	docker-compose -f docker-compose.rdma-sim.yml down -v --remove-orphans
-	docker system prune -f
-	@echo "✅ RDMA simulation cleanup complete"
-
-rdma-sim-status: ## Check RDMA simulation status
-	@echo "📊 RDMA simulation status:"
-	docker-compose -f docker-compose.rdma-sim.yml ps
-	@echo ""
-	@echo "🔍 RDMA device status:"
-	docker-compose -f docker-compose.rdma-sim.yml exec rdma-simulation /opt/rdma-sim/test-rdma.sh || true
-
-rdma-sim-shell: ## Open shell in RDMA simulation container
-	@echo "🐚 Opening RDMA simulation shell..."
-	docker-compose -f docker-compose.rdma-sim.yml exec rdma-simulation /bin/bash
-
-rdma-sim-logs: ## Show RDMA simulation logs
-	docker-compose -f docker-compose.rdma-sim.yml logs
-
-rdma-sim-ucx: ## Show UCX information in simulation
-	@echo "📋 UCX information in simulation:"
-	docker-compose -f docker-compose.rdma-sim.yml exec rdma-simulation /opt/rdma-sim/ucx-info.sh
-
-# Development Targets
-dev-setup: ## Set up development environment
-	@echo "🛠️  Setting up development environment..."
-	go mod tidy
-	cd rdma-engine && cargo check
-	chmod +x scripts/*.sh tests/*.sh
-	@echo "✅ Development environment ready"
-
-format: ## Format code
-	@echo "✨ Formatting code..."
-	go fmt ./...
-	cd rdma-engine && cargo fmt
-	@echo "✅ Code formatted"
-
-lint: ## Run linters
-	@echo "🔍 Running linters..."
-	go vet ./...
-	cd rdma-engine && cargo clippy -- -D warnings
-	@echo "✅ Linting complete"
-
-# Cleanup Targets
-clean: clean-go clean-rust ## Clean all build artifacts
-
-clean-go: ## Clean Go build artifacts
-	@echo "🧹 Cleaning Go artifacts..."
-	rm -rf bin/
-	go clean -testcache
-	@echo "✅ Go artifacts cleaned"
-
-clean-rust: ## Clean Rust build artifacts  
-	@echo "🧹 Cleaning Rust artifacts..."
-	cd rdma-engine && cargo clean
-	@echo "✅ Rust artifacts cleaned"
-
-# Full Workflow Targets
-check: format lint test ## Format, lint, and test everything
-
-ci: check integration-test docker-test ## Complete CI workflow
-
-demo: build ## Run local demo
-	@echo "🎮 Starting local demo..."
-	./scripts/demo-e2e.sh
-
-# Docker Development Workflow
-docker-dev: docker-clean docker-build docker-test ## Complete Docker development cycle
-
-# Quick targets
-quick-test: build ## Quick local test
-	./bin/test-rdma --help
-
-quick-docker: ## Quick Docker test
-	docker-compose up -d rdma-engine rdma-sidecar
-	sleep 5
-	curl -s http://localhost:8081/health | jq '.'
-	docker-compose down
-
-# Help and Documentation
-docs: ## Generate/update documentation
-	@echo "📚 Documentation ready:"
-	@echo "  README.md - Main project documentation"
-	@echo "  DOCKER-TESTING.md - Docker integration testing guide"
-	@echo "  Use 'make help' for available commands"
-
-# Environment Info
-info: ## Show environment information
-	@echo "🔍 Environment Information:"
-	@echo "  Go Version: $$(go version)"
-	@echo "  Rust Version: $$(cd rdma-engine && cargo --version)"
-	@echo "  Docker Version: $$(docker --version)"
-	@echo "  Docker Compose Version: $$(docker-compose --version)"
-	@echo ""
-	@echo "🏗️  Project Structure:"
-	@echo "  Go Components: cmd/ pkg/"
-	@echo "  Rust Engine: rdma-engine/"  
-	@echo "  Tests: tests/"
-	@echo "  Scripts: scripts/"

seaweedfs-rdma-sidecar/README.md

@@ -1,385 +0,0 @@
-# 🚀 SeaweedFS RDMA Sidecar
-
-**High-Performance RDMA Acceleration for SeaweedFS using UCX and Rust**
-
-[![Build Status](https://img.shields.io/badge/build-passing-brightgreen)](#) 
-[![Go Version](https://img.shields.io/badge/go-1.23+-blue)](#)
-[![Rust Version](https://img.shields.io/badge/rust-1.70+-orange)](#)
-[![License](https://img.shields.io/badge/license-MIT-green)](#)
-
-## 🎯 Overview
-
-This project implements a **high-performance RDMA (Remote Direct Memory Access) sidecar** for SeaweedFS that provides significant performance improvements for data-intensive read operations. The sidecar uses a **hybrid Go + Rust architecture** with the [UCX (Unified Communication X)](https://github.com/openucx/ucx) framework to deliver up to **44x performance improvement** over traditional HTTP-based reads.
-
-### 🏗️ Architecture
-
-```
-┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
-│   SeaweedFS     │    │   Go Sidecar    │    │  Rust Engine    │
-│  Volume Server  │◄──►│ (Control Plane) │◄──►│  (Data Plane)   │
-└─────────────────┘    └─────────────────┘    └─────────────────┘
-         │                       │                       │
-         │                       │                       │
-         ▼                       ▼                       ▼
-   HTTP/gRPC API         RDMA Client API         UCX/RDMA Hardware
-```
-
-**Components:**
-- **🟢 Go Sidecar**: Control plane handling SeaweedFS integration, client API, and fallback logic
-- **🦀 Rust Engine**: High-performance data plane with UCX framework for RDMA operations  
-- **🔗 IPC Bridge**: Unix domain socket communication with MessagePack serialization
-
-## 🌟 Key Features
-
-### ⚡ Performance
-- **44x faster** than HTTP reads (theoretical max based on RDMA vs TCP overhead)
-- **Sub-microsecond latency** for memory-mapped operations
-- **Zero-copy data transfers** directly to/from SeaweedFS volume files
-- **Concurrent session management** with up to 1000+ simultaneous operations
-
-### 🛡️ Reliability
-- **Automatic HTTP fallback** when RDMA unavailable
-- **Graceful degradation** under failure conditions
-- **Session timeout and cleanup** to prevent resource leaks
-- **Comprehensive error handling** with structured logging
-
-### 🔧 Production Ready
-- **Container-native deployment** with Kubernetes support
-- **RDMA device plugin integration** for hardware resource management
-- **HugePages optimization** for memory efficiency
-- **Prometheus metrics** and structured logging for observability
-
-### 🎚️ Flexibility
-- **Mock RDMA implementation** for development and testing
-- **Configurable transport selection** (RDMA, TCP, shared memory via UCX)
-- **Multi-device support** with automatic failover
-- **Authentication and authorization** support
-
-## 🚀 Quick Start
-
-### Prerequisites
-
-```bash
-# Required dependencies
-- Go 1.26+
-- Rust 1.70+
-- UCX libraries (for hardware RDMA)
-- Linux with RDMA-capable hardware (InfiniBand/RoCE)
-
-# Optional for development
-- Docker
-- Kubernetes
-- jq (for demo scripts)
-```
-
-### 🏗️ Build
-
-```bash
-# Clone the repository
-git clone <repository-url>
-cd seaweedfs-rdma-sidecar
-
-# Build Go components
-go build -o bin/sidecar ./cmd/sidecar
-go build -o bin/test-rdma ./cmd/test-rdma
-go build -o bin/demo-server ./cmd/demo-server
-
-# Build Rust engine
-cd rdma-engine
-cargo build --release
-cd ..
-```
-
-### 🎮 Demo
-
-Run the complete end-to-end demonstration:
-
-```bash
-# Interactive demo with all components
-./scripts/demo-e2e.sh
-
-# Or run individual components
-./rdma-engine/target/release/rdma-engine-server --debug &
-./bin/demo-server --port 8080 --enable-rdma
-```
-
-## 📊 Performance Results
-
-### Mock RDMA Performance (Development)
-```
-Average Latency:    2.48ms per operation  
-Throughput:         403.2 operations/sec
-Success Rate:       100%
-Session Management: ✅ Working
-IPC Communication:  ✅ Working
-```
-
-### Expected Hardware RDMA Performance
-```
-Average Latency:    < 10µs per operation (440x improvement)
-Throughput:         > 1M operations/sec (2500x improvement)  
-Bandwidth:          > 100 Gbps (theoretical InfiniBand limit)
-CPU Utilization:    < 5% (vs 60%+ for HTTP)
-```
-
-## 🧩 Components
-
-### 1️⃣ Rust RDMA Engine (`rdma-engine/`)
-
-High-performance data plane built with:
-
-- **🔧 UCX Integration**: Production-grade RDMA framework
-- **⚡ Async Operations**: Tokio-based async runtime  
-- **🧠 Memory Management**: Pooled buffers with HugePage support
-- **📡 IPC Server**: Unix domain socket with MessagePack
-- **📊 Session Management**: Thread-safe lifecycle handling
-
-```rust
-// Example: Starting the RDMA engine
-let config = RdmaEngineConfig {
-    device_name: "auto".to_string(),
-    port: 18515,
-    max_sessions: 1000,
-    // ... other config
-};
-
-let engine = RdmaEngine::new(config).await?;
-engine.start().await?;
-```
-
-### 2️⃣ Go Sidecar (`pkg/`, `cmd/`)
-
-Control plane providing:
-
-- **🔌 SeaweedFS Integration**: Native needle read/write support
-- **🔄 HTTP Fallback**: Automatic degradation when RDMA unavailable
-- **📈 Performance Monitoring**: Metrics and benchmarking
-- **🌐 HTTP API**: RESTful interface for management
-
-```go
-// Example: Using the RDMA client
-client := seaweedfs.NewSeaweedFSRDMAClient(&seaweedfs.Config{
-    RDMASocketPath: "/tmp/rdma-engine.sock",
-    Enabled:        true,
-})
-
-resp, err := client.ReadNeedle(ctx, &seaweedfs.NeedleReadRequest{
-    VolumeID: 1,
-    NeedleID: 12345,
-    Size:     4096,
-})
-```
-
-### 3️⃣ Integration Examples (`cmd/demo-server/`)
-
-Production-ready integration examples:
-
-- **🌐 HTTP Server**: Demonstrates SeaweedFS integration
-- **📊 Benchmarking**: Performance testing utilities
-- **🔍 Health Checks**: Monitoring and diagnostics
-- **📱 Web Interface**: Browser-based demo and testing
-
-## 🐳 Deployment
-
-### Kubernetes
-
-```yaml
-apiVersion: v1
-kind: Pod
-metadata:
-  name: seaweedfs-with-rdma
-spec:
-  containers:
-  - name: volume-server
-    image: chrislusf/seaweedfs:latest
-    # ... volume server config
-    
-  - name: rdma-sidecar
-    image: seaweedfs-rdma-sidecar:latest
-    resources:
-      limits:
-        rdma/hca: 1  # RDMA device
-        hugepages-2Mi: 1Gi
-    volumeMounts:
-    - name: rdma-socket
-      mountPath: /tmp/rdma-engine.sock
-```
-
-### Docker Compose
-
-```yaml
-version: '3.8'
-services:
-  rdma-engine:
-    build:
-      context: .
-      dockerfile: rdma-engine/Dockerfile
-    privileged: true
-    volumes:
-      - /tmp/rdma-engine.sock:/tmp/rdma-engine.sock
-      
-  seaweedfs-sidecar:
-    build: .
-    depends_on:
-      - rdma-engine
-    ports:
-      - "8080:8080"
-    volumes:
-      - /tmp/rdma-engine.sock:/tmp/rdma-engine.sock
-```
-
-## 🧪 Testing
-
-### Unit Tests
-```bash
-# Go tests
-go test ./...
-
-# Rust tests  
-cd rdma-engine && cargo test
-```
-
-### Integration Tests
-```bash
-# Full end-to-end testing
-./scripts/demo-e2e.sh
-
-# Direct RDMA engine testing
-./bin/test-rdma ping
-./bin/test-rdma capabilities
-./bin/test-rdma read --volume 1 --needle 12345
-./bin/test-rdma bench --iterations 100
-```
-
-### Performance Benchmarking
-```bash
-# HTTP vs RDMA comparison
-./bin/demo-server --enable-rdma &
-curl "http://localhost:8080/benchmark?iterations=1000&size=1048576"
-```
-
-## 🔧 Configuration
-
-### RDMA Engine Configuration
-
-```toml
-# rdma-engine/config.toml
-[rdma]
-device_name = "mlx5_0"  # or "auto"
-port = 18515
-max_sessions = 1000
-buffer_size = "1GB"
-
-[ipc]
-socket_path = "/tmp/rdma-engine.sock"
-max_connections = 100
-
-[logging]
-level = "info"
-```
-
-### Go Sidecar Configuration
-
-```yaml
-# config.yaml
-rdma:
-  socket_path: "/tmp/rdma-engine.sock"
-  enabled: true
-  timeout: "30s"
-
-seaweedfs:
-  volume_server_url: "http://localhost:8080"
-  
-http:
-  port: 8080
-  enable_cors: true
-```
-
-## 📈 Monitoring
-
-### Metrics
-
-The sidecar exposes Prometheus-compatible metrics:
-
-- `rdma_operations_total{type="read|write", result="success|error"}`
-- `rdma_operation_duration_seconds{type="read|write"}`
-- `rdma_sessions_active`
-- `rdma_bytes_transferred_total{direction="tx|rx"}`
-
-### Health Checks
-
-```bash
-# Sidecar health
-curl http://localhost:8080/health
-
-# RDMA engine health  
-curl http://localhost:8080/stats
-```
-
-### Logging
-
-Structured logging with configurable levels:
-
-```json
-{
-  "timestamp": "2025-08-16T20:55:17Z",
-  "level": "INFO", 
-  "message": "✅ RDMA read completed successfully",
-  "session_id": "db152578-bfad-4cb3-a50f-a2ac66eecc6a",
-  "bytes_read": 1024,
-  "duration": "2.48ms",
-  "transfer_rate": 800742.88
-}
-```
-
-## 🛠️ Development
-
-### Mock RDMA Mode
-
-For development without RDMA hardware:
-
-```bash
-# Enable mock mode (default)
-cargo run --features mock-ucx
-
-# All operations simulate RDMA with realistic latencies
-```
-
-### UCX Hardware Mode
-
-For production with real RDMA hardware:
-
-```bash  
-# Enable hardware UCX
-cargo run --features real-ucx
-
-# Requires UCX libraries and RDMA-capable hardware
-```
-
-### Adding New Operations
-
-1. **Define protobuf messages** in `rdma-engine/src/ipc.rs`
-2. **Implement Go client** in `pkg/ipc/client.go`
-3. **Add Rust handler** in `rdma-engine/src/ipc.rs`
-4. **Update tests** in both languages
-
-## 🙏 Acknowledgments
-
-- **[UCX Project](https://github.com/openucx/ucx)** - Unified Communication X framework
-- **[SeaweedFS](https://github.com/seaweedfs/seaweedfs)** - Distributed file system
-- **Rust Community** - Excellent async/await and FFI capabilities
-- **Go Community** - Robust networking and gRPC libraries
-
-## 📞 Support
-
-- 🐛 **Bug Reports**: [Create an issue](../../issues/new?template=bug_report.md)
-- 💡 **Feature Requests**: [Create an issue](../../issues/new?template=feature_request.md)  
-- 📚 **Documentation**: See [docs/](docs/) folder
-- 💬 **Discussions**: [GitHub Discussions](../../discussions)
-
----
-
-**🚀 Ready to accelerate your SeaweedFS deployment with RDMA?**
-
-Get started with the [Quick Start Guide](#-quick-start) or explore the [Demo Server](cmd/demo-server/) for hands-on experience!
-

seaweedfs-rdma-sidecar/REVIEW_FEEDBACK.md

@@ -1,55 +0,0 @@
-# PR #7140 Review Feedback Summary
-
-## Positive Feedback Received ✅
-
-### Source: [GitHub PR #7140 Review](https://github.com/seaweedfs/seaweedfs/pull/7140#pullrequestreview-3126580539)
-**Reviewer**: Gemini Code Assist (Automated Review Bot)  
-**Date**: August 18, 2025
-
-## Comments Analysis
-
-### 🏆 Binary Search Optimization - PRAISED
-**File**: `weed/mount/filehandle_read.go`  
-**Implementation**: Efficient chunk lookup using binary search with cached cumulative offsets
-
-**Reviewer Comment**: 
-> "The `tryRDMARead` function efficiently finds the target chunk for a given offset by using a binary search on cached cumulative chunk offsets. This is an effective optimization that will perform well even for files with a large number of chunks."
-
-**Technical Merit**:
-- ✅ O(log N) performance vs O(N) linear search
-- ✅ Cached cumulative offsets prevent repeated calculations  
-- ✅ Scales well for large fragmented files
-- ✅ Memory-efficient implementation
-
-### 🏆 Resource Management - PRAISED
-**File**: `weed/mount/weedfs.go`  
-**Implementation**: Proper RDMA client initialization and cleanup
-
-**Reviewer Comment**:
-> "The RDMA client is now correctly initialized and attached to the `WFS` struct when RDMA is enabled. The shutdown logic in the `grace.OnInterrupt` handler has also been updated to properly close the RDMA client, preventing resource leaks."
-
-**Technical Merit**:
-- ✅ Proper initialization with error handling
-- ✅ Clean shutdown in interrupt handler
-- ✅ No resource leaks
-- ✅ Graceful degradation on failure
-
-## Summary
-
-**All review comments are positive acknowledgments of excellent implementation practices.**
-
-### Key Strengths Recognized:
-1. **Performance Optimization**: Binary search algorithm implementation
-2. **Memory Safety**: Proper resource lifecycle management  
-3. **Code Quality**: Clean, efficient, and maintainable code
-4. **Production Readiness**: Robust error handling and cleanup
-
-### Build Status: ✅ PASSING
-- ✅ `go build ./...` - All packages compile successfully
-- ✅ `go vet ./...` - No linting issues
-- ✅ All tests passing
-- ✅ Docker builds working
-
-## Conclusion
-
-The RDMA sidecar implementation has received positive feedback from automated code review, confirming high code quality and adherence to best practices. **No action items required** - these are endorsements of excellent work.

seaweedfs-rdma-sidecar/WEED-MOUNT-CODE-PATH.md

@@ -1,260 +0,0 @@
-# 📋 Weed Mount RDMA Integration - Code Path Analysis
-
-## Current Status
-
-The RDMA client (`RDMAMountClient`) exists in `weed/mount/rdma_client.go` but is **not yet integrated** into the actual file read path. The integration points are identified but not implemented.
-
-## 🔍 Complete Code Path
-
-### **1. FUSE Read Request Entry Point**
-```go
-// File: weed/mount/weedfs_file_read.go:41
-func (wfs *WFS) Read(cancel <-chan struct{}, in *fuse.ReadIn, buff []byte) (fuse.ReadResult, fuse.Status) {
-    fh := wfs.GetHandle(FileHandleId(in.Fh))
-    // ...
-    offset := int64(in.Offset)
-    totalRead, err := readDataByFileHandleWithContext(ctx, buff, fh, offset)
-    // ...
-    return fuse.ReadResultData(buff[:totalRead]), fuse.OK
-}
-```
-
-### **2. File Handle Read Coordination**
-```go
-// File: weed/mount/weedfs_file_read.go:103
-func readDataByFileHandleWithContext(ctx context.Context, buff []byte, fhIn *FileHandle, offset int64) (int64, error) {
-    size := len(buff)
-    fhIn.lockForRead(offset, size)
-    defer fhIn.unlockForRead(offset, size)
-
-    // KEY INTEGRATION POINT: This is where RDMA should be attempted
-    n, tsNs, err := fhIn.readFromChunksWithContext(ctx, buff, offset)
-    // ...
-    return n, err
-}
-```
-
-### **3. Chunk Reading (Current Implementation)**
-```go
-// File: weed/mount/filehandle_read.go:29
-func (fh *FileHandle) readFromChunksWithContext(ctx context.Context, buff []byte, offset int64) (int64, int64, error) {
-    // ...
-    
-    // CURRENT: Direct chunk reading without RDMA
-    totalRead, ts, err := fh.entryChunkGroup.ReadDataAt(ctx, fileSize, buff, offset)
-    
-    // MISSING: RDMA integration should happen here
-    return int64(totalRead), ts, err
-}
-```
-
-### **4. RDMA Integration Point (What Needs to Be Added)**
-
-The integration should happen in `readFromChunksWithContext` like this:
-
-```go
-func (fh *FileHandle) readFromChunksWithContext(ctx context.Context, buff []byte, offset int64) (int64, int64, error) {
-    // ... existing code ...
-    
-    // NEW: Try RDMA acceleration first
-    if fh.wfs.rdmaClient != nil && fh.wfs.rdmaClient.IsHealthy() {
-        if totalRead, ts, err := fh.tryRDMARead(ctx, buff, offset); err == nil {
-            glog.V(4).Infof("RDMA read successful: %d bytes", totalRead)
-            return totalRead, ts, nil
-        }
-        glog.V(2).Infof("RDMA read failed, falling back to HTTP")
-    }
-    
-    // FALLBACK: Original HTTP-based chunk reading
-    totalRead, ts, err := fh.entryChunkGroup.ReadDataAt(ctx, fileSize, buff, offset)
-    return int64(totalRead), ts, err
-}
-```
-
-## 🚀 RDMA Client Integration
-
-### **5. RDMA Read Implementation (Already Exists)**
-```go
-// File: weed/mount/rdma_client.go:129
-func (c *RDMAMountClient) ReadNeedle(ctx context.Context, volumeID uint32, needleID uint64, cookie uint32, offset, size uint64) ([]byte, bool, error) {
-    // Prepare request URL
-    reqURL := fmt.Sprintf("http://%s/read?volume=%d&needle=%d&cookie=%d&offset=%d&size=%d",
-        c.sidecarAddr, volumeID, needleID, cookie, offset, size)
-    
-    // Execute HTTP request to RDMA sidecar
-    resp, err := c.httpClient.Do(req)
-    // ...
-    
-    // Return data with RDMA metadata
-    return data, isRDMA, nil
-}
-```
-
-### **6. RDMA Sidecar Processing**
-```go
-// File: seaweedfs-rdma-sidecar/cmd/demo-server/main.go:375
-func (s *DemoServer) readHandler(w http.ResponseWriter, r *http.Request) {
-    // Parse volume, needle, cookie from URL parameters
-    volumeID, _ := strconv.ParseUint(query.Get("volume"), 10, 32)
-    needleID, _ := strconv.ParseUint(query.Get("needle"), 10, 64)
-    
-    // Use distributed client for volume lookup + RDMA
-    if s.useDistributed && s.distributedClient != nil {
-        resp, err = s.distributedClient.ReadNeedle(ctx, req)
-    } else {
-        resp, err = s.rdmaClient.ReadNeedle(ctx, req)  // Local RDMA
-    }
-    
-    // Return binary data or JSON metadata
-    w.Write(resp.Data)
-}
-```
-
-### **7. Volume Lookup & RDMA Engine**
-```go
-// File: seaweedfs-rdma-sidecar/pkg/seaweedfs/distributed_client.go:45
-func (c *DistributedRDMAClient) ReadNeedle(ctx context.Context, req *NeedleReadRequest) (*NeedleReadResponse, error) {
-    // Step 1: Lookup volume location from master
-    locations, err := c.locationService.LookupVolume(ctx, req.VolumeID)
-    
-    // Step 2: Find best server (local preferred)
-    bestLocation := c.locationService.FindBestLocation(locations)
-    
-    // Step 3: Make HTTP request to target server's RDMA sidecar
-    return c.makeRDMARequest(ctx, req, bestLocation, start)
-}
-```
-
-### **8. Rust RDMA Engine (Final Data Access)**
-```rust
-// File: rdma-engine/src/ipc.rs:403
-async fn handle_start_read(req: StartReadRequest, ...) -> RdmaResult<StartReadResponse> {
-    // Create RDMA session
-    let session_id = Uuid::new_v4().to_string();
-    let buffer = vec![0u8; transfer_size as usize];
-    
-    // Register memory for RDMA
-    let memory_region = rdma_context.register_memory(local_addr, transfer_size).await?;
-    
-    // Perform RDMA read (mock implementation)
-    rdma_context.post_read(local_addr, remote_addr, remote_key, size, wr_id).await?;
-    let completions = rdma_context.poll_completion(1).await?;
-    
-    // Return session info
-    Ok(StartReadResponse { session_id, local_addr, ... })
-}
-```
-
-## 🔧 Missing Integration Components
-
-### **1. WFS Struct Extension**
-```go
-// File: weed/mount/weedfs.go (needs modification)
-type WFS struct {
-    // ... existing fields ...
-    rdmaClient *RDMAMountClient  // ADD THIS
-}
-```
-
-### **2. RDMA Client Initialization**
-```go
-// File: weed/command/mount.go (needs modification)
-func runMount(cmd *cobra.Command, args []string) bool {
-    // ... existing code ...
-    
-    // NEW: Initialize RDMA client if enabled
-    var rdmaClient *mount.RDMAMountClient
-    if *mountOptions.rdmaEnabled && *mountOptions.rdmaSidecarAddr != "" {
-        rdmaClient, err = mount.NewRDMAMountClient(
-            *mountOptions.rdmaSidecarAddr,
-            *mountOptions.rdmaMaxConcurrent,
-            *mountOptions.rdmaTimeoutMs,
-        )
-        if err != nil {
-            glog.Warningf("Failed to initialize RDMA client: %v", err)
-        }
-    }
-    
-    // Pass RDMA client to WFS
-    wfs := mount.NewSeaweedFileSystem(&mount.Option{
-        // ... existing options ...
-        RDMAClient: rdmaClient,  // ADD THIS
-    })
-}
-```
-
-### **3. Chunk-to-Needle Mapping**
-```go
-// File: weed/mount/filehandle_read.go (needs new method)
-func (fh *FileHandle) tryRDMARead(ctx context.Context, buff []byte, offset int64) (int64, int64, error) {
-    entry := fh.GetEntry()
-    
-    // Find which chunk contains the requested offset
-    for _, chunk := range entry.GetEntry().Chunks {
-        if offset >= chunk.Offset && offset < chunk.Offset+int64(chunk.Size) {
-            // Parse chunk.FileId to get volume, needle, cookie
-            volumeID, needleID, cookie, err := ParseFileId(chunk.FileId)
-            if err != nil {
-                return 0, 0, err
-            }
-            
-            // Calculate offset within the chunk
-            chunkOffset := uint64(offset - chunk.Offset)
-            readSize := uint64(min(len(buff), int(chunk.Size-chunkOffset)))
-            
-            // Make RDMA request
-            data, isRDMA, err := fh.wfs.rdmaClient.ReadNeedle(
-                ctx, volumeID, needleID, cookie, chunkOffset, readSize)
-            if err != nil {
-                return 0, 0, err
-            }
-            
-            // Copy data to buffer
-            copied := copy(buff, data)
-            return int64(copied), time.Now().UnixNano(), nil
-        }
-    }
-    
-    return 0, 0, fmt.Errorf("chunk not found for offset %d", offset)
-}
-```
-
-## 📊 Request Flow Summary
-
-1. **User Application** → `read()` system call
-2. **FUSE Kernel** → Routes to `WFS.Read()`
-3. **WFS.Read()** → Calls `readDataByFileHandleWithContext()`
-4. **readDataByFileHandleWithContext()** → Calls `fh.readFromChunksWithContext()`
-5. **readFromChunksWithContext()** → **[INTEGRATION POINT]** Try RDMA first
-6. **tryRDMARead()** → Parse chunk info, call `RDMAMountClient.ReadNeedle()`
-7. **RDMAMountClient** → HTTP request to RDMA sidecar
-8. **RDMA Sidecar** → Volume lookup + RDMA engine call
-9. **RDMA Engine** → Direct memory access via RDMA hardware
-10. **Response Path** → Data flows back through all layers to user
-
-## ✅ What's Working vs Missing
-
-### **✅ Already Implemented:**
-- ✅ `RDMAMountClient` with HTTP communication
-- ✅ RDMA sidecar with volume lookup
-- ✅ Rust RDMA engine with mock hardware
-- ✅ File ID parsing utilities
-- ✅ Health checks and statistics
-- ✅ Command-line flags for RDMA options
-
-### **❌ Missing Integration:**
-- ❌ RDMA client not added to WFS struct
-- ❌ RDMA client not initialized in mount command
-- ❌ `tryRDMARead()` method not implemented
-- ❌ Chunk-to-needle mapping logic missing
-- ❌ RDMA integration not wired into read path
-
-## 🎯 Next Steps
-
-1. **Add RDMA client to WFS struct and Option**
-2. **Initialize RDMA client in mount command**
-3. **Implement `tryRDMARead()` method**
-4. **Wire RDMA integration into `readFromChunksWithContext()`**
-5. **Test end-to-end RDMA acceleration**
-
-The architecture is sound and most components exist - only the final integration wiring is needed!

seaweedfs-rdma-sidecar/cmd/demo-server/main.go

@@ -1,663 +0,0 @@
-// Package main provides a demonstration server showing SeaweedFS RDMA integration
-package main
-
-import (
-	"context"
-	"encoding/json"
-	"fmt"
-	"net/http"
-	"os"
-	"os/signal"
-	"strconv"
-	"strings"
-	"syscall"
-	"time"
-
-	"seaweedfs-rdma-sidecar/pkg/seaweedfs"
-
-	"github.com/seaweedfs/seaweedfs/weed/storage/needle"
-	"github.com/sirupsen/logrus"
-	"github.com/spf13/cobra"
-)
-
-var (
-	port            int
-	rdmaSocket      string
-	volumeServerURL string
-	enableRDMA      bool
-	enableZeroCopy  bool
-	tempDir         string
-	enablePooling   bool
-	maxConnections  int
-	maxIdleTime     time.Duration
-	debug           bool
-)
-
-func main() {
-	var rootCmd = &cobra.Command{
-		Use:   "demo-server",
-		Short: "SeaweedFS RDMA integration demonstration server",
-		Long: `Demonstration server that shows how SeaweedFS can integrate with the RDMA sidecar
-for accelerated read operations. This server provides HTTP endpoints that demonstrate
-the RDMA fast path with HTTP fallback capabilities.`,
-		RunE: runServer,
-	}
-
-	rootCmd.Flags().IntVarP(&port, "port", "p", 8080, "Demo server HTTP port")
-	rootCmd.Flags().StringVarP(&rdmaSocket, "rdma-socket", "r", "/tmp/rdma-engine.sock", "Path to RDMA engine Unix socket")
-	rootCmd.Flags().StringVarP(&volumeServerURL, "volume-server", "v", "http://localhost:8080", "SeaweedFS volume server URL for HTTP fallback")
-	rootCmd.Flags().BoolVarP(&enableRDMA, "enable-rdma", "e", true, "Enable RDMA acceleration")
-	rootCmd.Flags().BoolVarP(&enableZeroCopy, "enable-zerocopy", "z", true, "Enable zero-copy optimization via temp files")
-	rootCmd.Flags().StringVarP(&tempDir, "temp-dir", "t", "/tmp/rdma-cache", "Temp directory for zero-copy files")
-	rootCmd.Flags().BoolVar(&enablePooling, "enable-pooling", true, "Enable RDMA connection pooling")
-	rootCmd.Flags().IntVar(&maxConnections, "max-connections", 10, "Maximum connections in RDMA pool")
-	rootCmd.Flags().DurationVar(&maxIdleTime, "max-idle-time", 5*time.Minute, "Maximum idle time for pooled connections")
-	rootCmd.Flags().BoolVarP(&debug, "debug", "d", false, "Enable debug logging")
-
-	if err := rootCmd.Execute(); err != nil {
-		fmt.Fprintf(os.Stderr, "Error: %v\n", err)
-		os.Exit(1)
-	}
-}
-
-func runServer(cmd *cobra.Command, args []string) error {
-	// Setup logging
-	logger := logrus.New()
-	if debug {
-		logger.SetLevel(logrus.DebugLevel)
-		logger.SetFormatter(&logrus.TextFormatter{
-			FullTimestamp: true,
-			ForceColors:   true,
-		})
-	} else {
-		logger.SetLevel(logrus.InfoLevel)
-	}
-
-	logger.WithFields(logrus.Fields{
-		"port":              port,
-		"rdma_socket":       rdmaSocket,
-		"volume_server_url": volumeServerURL,
-		"enable_rdma":       enableRDMA,
-		"enable_zerocopy":   enableZeroCopy,
-		"temp_dir":          tempDir,
-		"enable_pooling":    enablePooling,
-		"max_connections":   maxConnections,
-		"max_idle_time":     maxIdleTime,
-		"debug":             debug,
-	}).Info("🚀 Starting SeaweedFS RDMA Demo Server")
-
-	// Create SeaweedFS RDMA client
-	config := &seaweedfs.Config{
-		RDMASocketPath:  rdmaSocket,
-		VolumeServerURL: volumeServerURL,
-		Enabled:         enableRDMA,
-		DefaultTimeout:  30 * time.Second,
-		Logger:          logger,
-		TempDir:         tempDir,
-		UseZeroCopy:     enableZeroCopy,
-		EnablePooling:   enablePooling,
-		MaxConnections:  maxConnections,
-		MaxIdleTime:     maxIdleTime,
-	}
-
-	rdmaClient, err := seaweedfs.NewSeaweedFSRDMAClient(config)
-	if err != nil {
-		return fmt.Errorf("failed to create RDMA client: %w", err)
-	}
-
-	// Start RDMA client
-	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
-	if err := rdmaClient.Start(ctx); err != nil {
-		logger.WithError(err).Error("Failed to start RDMA client")
-	}
-	cancel()
-
-	// Create demo server
-	server := &DemoServer{
-		rdmaClient: rdmaClient,
-		logger:     logger,
-	}
-
-	// Setup HTTP routes
-	mux := http.NewServeMux()
-	mux.HandleFunc("/", server.homeHandler)
-	mux.HandleFunc("/health", server.healthHandler)
-	mux.HandleFunc("/stats", server.statsHandler)
-	mux.HandleFunc("/read", server.readHandler)
-	mux.HandleFunc("/benchmark", server.benchmarkHandler)
-	mux.HandleFunc("/cleanup", server.cleanupHandler)
-
-	httpServer := &http.Server{
-		Addr:    fmt.Sprintf(":%d", port),
-		Handler: mux,
-	}
-
-	// Handle graceful shutdown
-	sigChan := make(chan os.Signal, 1)
-	signal.Notify(sigChan, syscall.SIGINT, syscall.SIGTERM)
-
-	go func() {
-		logger.WithField("port", port).Info("🌐 Demo server starting")
-		if err := httpServer.ListenAndServe(); err != nil && err != http.ErrServerClosed {
-			logger.WithError(err).Fatal("HTTP server failed")
-		}
-	}()
-
-	// Wait for shutdown signal
-	<-sigChan
-	logger.Info("📡 Received shutdown signal, gracefully shutting down...")
-
-	// Shutdown HTTP server
-	shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), 10*time.Second)
-	defer shutdownCancel()
-
-	if err := httpServer.Shutdown(shutdownCtx); err != nil {
-		logger.WithError(err).Error("HTTP server shutdown failed")
-	} else {
-		logger.Info("🌐 HTTP server shutdown complete")
-	}
-
-	// Stop RDMA client
-	rdmaClient.Stop()
-	logger.Info("🛑 Demo server shutdown complete")
-
-	return nil
-}
-
-// DemoServer demonstrates SeaweedFS RDMA integration
-type DemoServer struct {
-	rdmaClient *seaweedfs.SeaweedFSRDMAClient
-	logger     *logrus.Logger
-}
-
-// homeHandler provides information about the demo server
-func (s *DemoServer) homeHandler(w http.ResponseWriter, r *http.Request) {
-	if r.Method != http.MethodGet {
-		http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
-		return
-	}
-
-	w.Header().Set("Content-Type", "text/html")
-	fmt.Fprintf(w, `<!DOCTYPE html>
-<html>
-<head>
-    <title>SeaweedFS RDMA Demo Server</title>
-    <style>
-        body { font-family: Arial, sans-serif; margin: 40px; background-color: #f5f5f5; }
-        .container { max-width: 800px; margin: 0 auto; background: white; padding: 20px; border-radius: 8px; box-shadow: 0 2px 4px rgba(0,0,0,0.1); }
-        h1 { color: #2c3e50; }
-        .endpoint { margin: 20px 0; padding: 15px; background: #ecf0f1; border-radius: 4px; }
-        .endpoint h3 { margin: 0 0 10px 0; color: #34495e; }
-        .endpoint a { color: #3498db; text-decoration: none; }
-        .endpoint a:hover { text-decoration: underline; }
-        .status { padding: 10px; border-radius: 4px; margin: 10px 0; }
-        .status.enabled { background: #d5f4e6; color: #27ae60; }
-        .status.disabled { background: #fadbd8; color: #e74c3c; }
-    </style>
-</head>
-<body>
-    <div class="container">
-        <h1>🚀 SeaweedFS RDMA Demo Server</h1>
-        <p>This server demonstrates SeaweedFS integration with RDMA acceleration for high-performance reads.</p>
-        
-        <div class="status %s">
-            <strong>RDMA Status:</strong> %s
-        </div>
-
-        <h2>📋 Available Endpoints</h2>
-        
-        <div class="endpoint">
-            <h3>🏥 Health Check</h3>
-            <p><a href="/health">/health</a> - Check server and RDMA engine health</p>
-        </div>
-
-        <div class="endpoint">
-            <h3>📊 Statistics</h3>
-            <p><a href="/stats">/stats</a> - Get RDMA client statistics and capabilities</p>
-        </div>
-
-        <div class="endpoint">
-            <h3>📖 Read Needle</h3>
-            <p><a href="/read?file_id=3,01637037d6&size=1024&volume_server=http://localhost:8080">/read</a> - Read a needle with RDMA fast path</p>
-            <p><strong>Parameters:</strong> file_id OR (volume, needle, cookie), volume_server, offset (optional), size (optional)</p>
-        </div>
-
-        <div class="endpoint">
-            <h3>🏁 Benchmark</h3>
-            <p><a href="/benchmark?iterations=10&size=4096">/benchmark</a> - Run performance benchmark</p>
-            <p><strong>Parameters:</strong> iterations (default: 10), size (default: 4096)</p>
-        </div>
-
-        <h2>📝 Example Usage</h2>
-        <pre>
-# Read a needle using file ID (recommended)
-curl "http://localhost:%d/read?file_id=3,01637037d6&size=1024&volume_server=http://localhost:8080"
-
-# Read a needle using individual parameters (legacy)
-curl "http://localhost:%d/read?volume=1&needle=12345&cookie=305419896&size=1024&volume_server=http://localhost:8080"
-
-# Read a needle (hex cookie)
-curl "http://localhost:%d/read?volume=1&needle=12345&cookie=0x12345678&size=1024&volume_server=http://localhost:8080"
-
-# Run benchmark
-curl "http://localhost:%d/benchmark?iterations=5&size=2048"
-
-# Check health
-curl "http://localhost:%d/health"
-        </pre>
-    </div>
-</body>
-</html>`,
-		map[bool]string{true: "enabled", false: "disabled"}[s.rdmaClient.IsEnabled()],
-		map[bool]string{true: "RDMA Enabled ✅", false: "RDMA Disabled (HTTP Fallback Only) ⚠️"}[s.rdmaClient.IsEnabled()],
-		port, port, port, port)
-}
-
-// healthHandler checks server and RDMA health
-func (s *DemoServer) healthHandler(w http.ResponseWriter, r *http.Request) {
-	if r.Method != http.MethodGet {
-		http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
-		return
-	}
-
-	ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second)
-	defer cancel()
-
-	health := map[string]interface{}{
-		"status":    "healthy",
-		"timestamp": time.Now().Format(time.RFC3339),
-		"rdma": map[string]interface{}{
-			"enabled":   false,
-			"connected": false,
-		},
-	}
-
-	if s.rdmaClient != nil {
-		health["rdma"].(map[string]interface{})["enabled"] = s.rdmaClient.IsEnabled()
-		health["rdma"].(map[string]interface{})["type"] = "local"
-
-		if s.rdmaClient.IsEnabled() {
-			if err := s.rdmaClient.HealthCheck(ctx); err != nil {
-				s.logger.WithError(err).Warn("RDMA health check failed")
-				health["rdma"].(map[string]interface{})["error"] = err.Error()
-			} else {
-				health["rdma"].(map[string]interface{})["connected"] = true
-			}
-		}
-	}
-
-	w.Header().Set("Content-Type", "application/json")
-	json.NewEncoder(w).Encode(health)
-}
-
-// statsHandler returns RDMA statistics
-func (s *DemoServer) statsHandler(w http.ResponseWriter, r *http.Request) {
-	if r.Method != http.MethodGet {
-		http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
-		return
-	}
-
-	var stats map[string]interface{}
-
-	if s.rdmaClient != nil {
-		stats = s.rdmaClient.GetStats()
-		stats["client_type"] = "local"
-	} else {
-		stats = map[string]interface{}{
-			"client_type": "none",
-			"error":       "no RDMA client available",
-		}
-	}
-
-	stats["timestamp"] = time.Now().Format(time.RFC3339)
-
-	w.Header().Set("Content-Type", "application/json")
-	json.NewEncoder(w).Encode(stats)
-}
-
-// readHandler demonstrates needle reading with RDMA
-func (s *DemoServer) readHandler(w http.ResponseWriter, r *http.Request) {
-	if r.Method != http.MethodGet {
-		http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
-		return
-	}
-
-	// Parse parameters - support both file_id and individual parameters for backward compatibility
-	query := r.URL.Query()
-	volumeServer := query.Get("volume_server")
-	fileID := query.Get("file_id")
-
-	var volumeID, cookie uint64
-	var needleID uint64
-	var err error
-
-	if fileID != "" {
-		// Use file ID format (e.g., "3,01637037d6")
-		// Extract individual components using existing SeaweedFS parsing
-		fid, parseErr := needle.ParseFileIdFromString(fileID)
-		if parseErr != nil {
-			http.Error(w, fmt.Sprintf("invalid 'file_id' parameter: %v", parseErr), http.StatusBadRequest)
-			return
-		}
-		volumeID = uint64(fid.VolumeId)
-		needleID = uint64(fid.Key)
-		cookie = uint64(fid.Cookie)
-	} else {
-		// Use individual parameters (backward compatibility)
-		volumeID, err = strconv.ParseUint(query.Get("volume"), 10, 32)
-		if err != nil {
-			http.Error(w, "invalid 'volume' parameter", http.StatusBadRequest)
-			return
-		}
-
-		needleID, err = strconv.ParseUint(query.Get("needle"), 10, 64)
-		if err != nil {
-			http.Error(w, "invalid 'needle' parameter", http.StatusBadRequest)
-			return
-		}
-
-		// Parse cookie parameter - support both decimal and hexadecimal formats
-		cookieStr := query.Get("cookie")
-		if strings.HasPrefix(strings.ToLower(cookieStr), "0x") {
-			// Parse as hexadecimal (remove "0x" prefix)
-			cookie, err = strconv.ParseUint(cookieStr[2:], 16, 32)
-		} else {
-			// Parse as decimal (default)
-			cookie, err = strconv.ParseUint(cookieStr, 10, 32)
-		}
-		if err != nil {
-			http.Error(w, "invalid 'cookie' parameter (expected decimal or hex with 0x prefix)", http.StatusBadRequest)
-			return
-		}
-	}
-
-	var offset uint64
-	if offsetStr := query.Get("offset"); offsetStr != "" {
-		var parseErr error
-		offset, parseErr = strconv.ParseUint(offsetStr, 10, 64)
-		if parseErr != nil {
-			http.Error(w, "invalid 'offset' parameter", http.StatusBadRequest)
-			return
-		}
-	}
-
-	var size uint64
-	if sizeStr := query.Get("size"); sizeStr != "" {
-		var parseErr error
-		size, parseErr = strconv.ParseUint(sizeStr, 10, 64)
-		if parseErr != nil {
-			http.Error(w, "invalid 'size' parameter", http.StatusBadRequest)
-			return
-		}
-	}
-
-	if volumeServer == "" {
-		http.Error(w, "volume_server parameter is required", http.StatusBadRequest)
-		return
-	}
-
-	if volumeID == 0 || needleID == 0 {
-		http.Error(w, "volume and needle parameters are required", http.StatusBadRequest)
-		return
-	}
-
-	// Note: cookie and size can have defaults for demo purposes when user provides empty values,
-	// but invalid parsing is caught above with proper error responses
-	if cookie == 0 {
-		cookie = 0x12345678 // Default cookie for demo
-	}
-
-	if size == 0 {
-		size = 4096 // Default size
-	}
-
-	logFields := logrus.Fields{
-		"volume_server": volumeServer,
-		"volume_id":     volumeID,
-		"needle_id":     needleID,
-		"cookie":        fmt.Sprintf("0x%x", cookie),
-		"offset":        offset,
-		"size":          size,
-	}
-	if fileID != "" {
-		logFields["file_id"] = fileID
-	}
-	s.logger.WithFields(logFields).Info("📖 Processing needle read request")
-
-	ctx, cancel := context.WithTimeout(r.Context(), 30*time.Second)
-	defer cancel()
-
-	start := time.Now()
-	req := &seaweedfs.NeedleReadRequest{
-		VolumeID:     uint32(volumeID),
-		NeedleID:     needleID,
-		Cookie:       uint32(cookie),
-		Offset:       offset,
-		Size:         size,
-		VolumeServer: volumeServer,
-	}
-
-	resp, err := s.rdmaClient.ReadNeedle(ctx, req)
-
-	if err != nil {
-		s.logger.WithError(err).Error("❌ Needle read failed")
-		http.Error(w, fmt.Sprintf("Read failed: %v", err), http.StatusInternalServerError)
-		return
-	}
-
-	duration := time.Since(start)
-
-	s.logger.WithFields(logrus.Fields{
-		"volume_id": volumeID,
-		"needle_id": needleID,
-		"is_rdma":   resp.IsRDMA,
-		"source":    resp.Source,
-		"duration":  duration,
-		"data_size": len(resp.Data),
-	}).Info("✅ Needle read completed")
-
-	// Return metadata and first few bytes
-	result := map[string]interface{}{
-		"success":       true,
-		"volume_id":     volumeID,
-		"needle_id":     needleID,
-		"cookie":        fmt.Sprintf("0x%x", cookie),
-		"is_rdma":       resp.IsRDMA,
-		"source":        resp.Source,
-		"session_id":    resp.SessionID,
-		"duration":      duration.String(),
-		"data_size":     len(resp.Data),
-		"timestamp":     time.Now().Format(time.RFC3339),
-		"use_temp_file": resp.UseTempFile,
-		"temp_file":     resp.TempFilePath,
-	}
-
-	// Set headers for zero-copy optimization
-	if resp.UseTempFile && resp.TempFilePath != "" {
-		w.Header().Set("X-Use-Temp-File", "true")
-		w.Header().Set("X-Temp-File", resp.TempFilePath)
-		w.Header().Set("X-Source", resp.Source)
-		w.Header().Set("X-RDMA-Used", fmt.Sprintf("%t", resp.IsRDMA))
-
-		// For zero-copy, return minimal JSON response and let client read from temp file
-		w.Header().Set("Content-Type", "application/json")
-		json.NewEncoder(w).Encode(result)
-		return
-	}
-
-	// Regular response with data
-	w.Header().Set("X-Source", resp.Source)
-	w.Header().Set("X-RDMA-Used", fmt.Sprintf("%t", resp.IsRDMA))
-
-	// Include first 32 bytes as hex for verification
-	if len(resp.Data) > 0 {
-		displayLen := 32
-		if len(resp.Data) < displayLen {
-			displayLen = len(resp.Data)
-		}
-		result["data_preview"] = fmt.Sprintf("%x", resp.Data[:displayLen])
-	}
-
-	w.Header().Set("Content-Type", "application/json")
-	json.NewEncoder(w).Encode(result)
-}
-
-// benchmarkHandler runs performance benchmarks
-func (s *DemoServer) benchmarkHandler(w http.ResponseWriter, r *http.Request) {
-	if r.Method != http.MethodGet {
-		http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
-		return
-	}
-
-	// Parse parameters
-	query := r.URL.Query()
-
-	iterations := 10 // default value
-	if iterationsStr := query.Get("iterations"); iterationsStr != "" {
-		var parseErr error
-		iterations, parseErr = strconv.Atoi(iterationsStr)
-		if parseErr != nil {
-			http.Error(w, "invalid 'iterations' parameter", http.StatusBadRequest)
-			return
-		}
-	}
-
-	size := uint64(4096) // default value
-	if sizeStr := query.Get("size"); sizeStr != "" {
-		var parseErr error
-		size, parseErr = strconv.ParseUint(sizeStr, 10, 64)
-		if parseErr != nil {
-			http.Error(w, "invalid 'size' parameter", http.StatusBadRequest)
-			return
-		}
-	}
-
-	if iterations <= 0 {
-		iterations = 10
-	}
-	if size == 0 {
-		size = 4096
-	}
-
-	s.logger.WithFields(logrus.Fields{
-		"iterations": iterations,
-		"size":       size,
-	}).Info("🏁 Starting benchmark")
-
-	ctx, cancel := context.WithTimeout(r.Context(), 60*time.Second)
-	defer cancel()
-
-	var rdmaSuccessful, rdmaFailed, httpSuccessful, httpFailed int
-	var totalDuration time.Duration
-	var totalBytes uint64
-
-	startTime := time.Now()
-
-	for i := 0; i < iterations; i++ {
-		req := &seaweedfs.NeedleReadRequest{
-			VolumeID: 1,
-			NeedleID: uint64(i + 1),
-			Cookie:   0x12345678,
-			Offset:   0,
-			Size:     size,
-		}
-
-		opStart := time.Now()
-		resp, err := s.rdmaClient.ReadNeedle(ctx, req)
-		opDuration := time.Since(opStart)
-
-		if err != nil {
-			httpFailed++
-			continue
-		}
-
-		totalDuration += opDuration
-		totalBytes += uint64(len(resp.Data))
-
-		if resp.IsRDMA {
-			rdmaSuccessful++
-		} else {
-			httpSuccessful++
-		}
-	}
-
-	benchDuration := time.Since(startTime)
-
-	// Calculate statistics
-	totalOperations := rdmaSuccessful + httpSuccessful
-	avgLatency := time.Duration(0)
-	if totalOperations > 0 {
-		avgLatency = totalDuration / time.Duration(totalOperations)
-	}
-
-	throughputMBps := float64(totalBytes) / benchDuration.Seconds() / (1024 * 1024)
-	opsPerSec := float64(totalOperations) / benchDuration.Seconds()
-
-	result := map[string]interface{}{
-		"benchmark_results": map[string]interface{}{
-			"iterations":      iterations,
-			"size_per_op":     size,
-			"total_duration":  benchDuration.String(),
-			"successful_ops":  totalOperations,
-			"failed_ops":      rdmaFailed + httpFailed,
-			"rdma_ops":        rdmaSuccessful,
-			"http_ops":        httpSuccessful,
-			"avg_latency":     avgLatency.String(),
-			"throughput_mbps": fmt.Sprintf("%.2f", throughputMBps),
-			"ops_per_sec":     fmt.Sprintf("%.1f", opsPerSec),
-			"total_bytes":     totalBytes,
-		},
-		"rdma_enabled": s.rdmaClient.IsEnabled(),
-		"timestamp":    time.Now().Format(time.RFC3339),
-	}
-
-	s.logger.WithFields(logrus.Fields{
-		"iterations":      iterations,
-		"successful_ops":  totalOperations,
-		"rdma_ops":        rdmaSuccessful,
-		"http_ops":        httpSuccessful,
-		"avg_latency":     avgLatency,
-		"throughput_mbps": throughputMBps,
-		"ops_per_sec":     opsPerSec,
-	}).Info("📊 Benchmark completed")
-
-	w.Header().Set("Content-Type", "application/json")
-	json.NewEncoder(w).Encode(result)
-}
-
-// cleanupHandler handles temp file cleanup requests from mount clients
-func (s *DemoServer) cleanupHandler(w http.ResponseWriter, r *http.Request) {
-	if r.Method != http.MethodDelete {
-		http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
-		return
-	}
-
-	// Get temp file path from query parameters
-	tempFilePath := r.URL.Query().Get("temp_file")
-	if tempFilePath == "" {
-		http.Error(w, "missing 'temp_file' parameter", http.StatusBadRequest)
-		return
-	}
-
-	s.logger.WithField("temp_file", tempFilePath).Debug("🗑️ Processing cleanup request")
-
-	// Use the RDMA client's cleanup method (which delegates to seaweedfs client)
-	err := s.rdmaClient.CleanupTempFile(tempFilePath)
-	if err != nil {
-		s.logger.WithError(err).WithField("temp_file", tempFilePath).Warn("Failed to cleanup temp file")
-		http.Error(w, fmt.Sprintf("cleanup failed: %v", err), http.StatusInternalServerError)
-		return
-	}
-
-	s.logger.WithField("temp_file", tempFilePath).Debug("🧹 Temp file cleanup successful")
-
-	// Return success response
-	w.Header().Set("Content-Type", "application/json")
-	response := map[string]interface{}{
-		"success":   true,
-		"message":   "temp file cleaned up successfully",
-		"temp_file": tempFilePath,
-		"timestamp": time.Now().Format(time.RFC3339),
-	}
-	json.NewEncoder(w).Encode(response)
-}

seaweedfs-rdma-sidecar/cmd/sidecar/main.go

@@ -1,345 +0,0 @@
-// Package main provides the main RDMA sidecar service that integrates with SeaweedFS
-package main
-
-import (
-	"context"
-	"encoding/json"
-	"fmt"
-	"net/http"
-	"os"
-	"os/signal"
-	"strconv"
-	"syscall"
-	"time"
-
-	"seaweedfs-rdma-sidecar/pkg/rdma"
-
-	"github.com/sirupsen/logrus"
-	"github.com/spf13/cobra"
-)
-
-var (
-	port         int
-	engineSocket string
-	debug        bool
-	timeout      time.Duration
-)
-
-// Response structs for JSON encoding
-type HealthResponse struct {
-	Status              string `json:"status"`
-	RdmaEngineConnected bool   `json:"rdma_engine_connected"`
-	RdmaEngineLatency   string `json:"rdma_engine_latency"`
-	Timestamp           string `json:"timestamp"`
-}
-
-type CapabilitiesResponse struct {
-	Version         string   `json:"version"`
-	DeviceName      string   `json:"device_name"`
-	VendorId        uint32   `json:"vendor_id"`
-	MaxSessions     uint32   `json:"max_sessions"`
-	MaxTransferSize uint64   `json:"max_transfer_size"`
-	ActiveSessions  uint32   `json:"active_sessions"`
-	RealRdma        bool     `json:"real_rdma"`
-	PortGid         string   `json:"port_gid"`
-	PortLid         uint16   `json:"port_lid"`
-	SupportedAuth   []string `json:"supported_auth"`
-}
-
-type PingResponse struct {
-	Success       bool   `json:"success"`
-	EngineLatency string `json:"engine_latency"`
-	TotalLatency  string `json:"total_latency"`
-	Timestamp     string `json:"timestamp"`
-}
-
-func main() {
-	var rootCmd = &cobra.Command{
-		Use:   "rdma-sidecar",
-		Short: "SeaweedFS RDMA acceleration sidecar",
-		Long: `RDMA sidecar that accelerates SeaweedFS read/write operations using UCX and Rust RDMA engine.
-
-This sidecar acts as a bridge between SeaweedFS volume servers and the high-performance
-Rust RDMA engine, providing significant performance improvements for data-intensive workloads.`,
-		RunE: runSidecar,
-	}
-
-	// Flags
-	rootCmd.Flags().IntVarP(&port, "port", "p", 8081, "HTTP server port")
-	rootCmd.Flags().StringVarP(&engineSocket, "engine-socket", "e", "/tmp/rdma-engine.sock", "Path to RDMA engine Unix socket")
-	rootCmd.Flags().BoolVarP(&debug, "debug", "d", false, "Enable debug logging")
-	rootCmd.Flags().DurationVarP(&timeout, "timeout", "t", 30*time.Second, "RDMA operation timeout")
-
-	if err := rootCmd.Execute(); err != nil {
-		fmt.Fprintf(os.Stderr, "Error: %v\n", err)
-		os.Exit(1)
-	}
-}
-
-func runSidecar(cmd *cobra.Command, args []string) error {
-	// Setup logging
-	logger := logrus.New()
-	if debug {
-		logger.SetLevel(logrus.DebugLevel)
-		logger.SetFormatter(&logrus.TextFormatter{
-			FullTimestamp: true,
-			ForceColors:   true,
-		})
-	} else {
-		logger.SetLevel(logrus.InfoLevel)
-	}
-
-	logger.WithFields(logrus.Fields{
-		"port":          port,
-		"engine_socket": engineSocket,
-		"debug":         debug,
-		"timeout":       timeout,
-	}).Info("🚀 Starting SeaweedFS RDMA Sidecar")
-
-	// Create RDMA client
-	rdmaConfig := &rdma.Config{
-		EngineSocketPath: engineSocket,
-		DefaultTimeout:   timeout,
-		Logger:           logger,
-	}
-
-	rdmaClient := rdma.NewClient(rdmaConfig)
-
-	// Connect to RDMA engine
-	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
-	defer cancel()
-
-	logger.Info("🔗 Connecting to RDMA engine...")
-	if err := rdmaClient.Connect(ctx); err != nil {
-		return fmt.Errorf("failed to connect to RDMA engine: %w", err)
-	}
-	logger.Info("✅ Connected to RDMA engine successfully")
-
-	// Create HTTP server
-	sidecar := &Sidecar{
-		rdmaClient: rdmaClient,
-		logger:     logger,
-	}
-
-	mux := http.NewServeMux()
-
-	// Health check endpoint
-	mux.HandleFunc("/health", sidecar.healthHandler)
-
-	// RDMA operations endpoints
-	mux.HandleFunc("/rdma/read", sidecar.rdmaReadHandler)
-	mux.HandleFunc("/rdma/capabilities", sidecar.capabilitiesHandler)
-	mux.HandleFunc("/rdma/ping", sidecar.pingHandler)
-
-	server := &http.Server{
-		Addr:    fmt.Sprintf(":%d", port),
-		Handler: mux,
-	}
-
-	// Handle graceful shutdown
-	sigChan := make(chan os.Signal, 1)
-	signal.Notify(sigChan, syscall.SIGINT, syscall.SIGTERM)
-
-	go func() {
-		logger.WithField("port", port).Info("🌐 HTTP server starting")
-		if err := server.ListenAndServe(); err != nil && err != http.ErrServerClosed {
-			logger.WithError(err).Fatal("HTTP server failed")
-		}
-	}()
-
-	// Wait for shutdown signal
-	<-sigChan
-	logger.Info("📡 Received shutdown signal, gracefully shutting down...")
-
-	// Shutdown HTTP server
-	shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), 10*time.Second)
-	defer shutdownCancel()
-
-	if err := server.Shutdown(shutdownCtx); err != nil {
-		logger.WithError(err).Error("HTTP server shutdown failed")
-	} else {
-		logger.Info("🌐 HTTP server shutdown complete")
-	}
-
-	// Disconnect from RDMA engine
-	rdmaClient.Disconnect()
-	logger.Info("🛑 RDMA sidecar shutdown complete")
-
-	return nil
-}
-
-// Sidecar represents the main sidecar service
-type Sidecar struct {
-	rdmaClient *rdma.Client
-	logger     *logrus.Logger
-}
-
-// Health check handler
-func (s *Sidecar) healthHandler(w http.ResponseWriter, r *http.Request) {
-	if r.Method != http.MethodGet {
-		http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
-		return
-	}
-
-	ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second)
-	defer cancel()
-
-	// Test RDMA engine connectivity
-	if !s.rdmaClient.IsConnected() {
-		s.logger.Warn("⚠️  RDMA engine not connected")
-		http.Error(w, "RDMA engine not connected", http.StatusServiceUnavailable)
-		return
-	}
-
-	// Ping RDMA engine
-	latency, err := s.rdmaClient.Ping(ctx)
-	if err != nil {
-		s.logger.WithError(err).Error("❌ RDMA engine ping failed")
-		http.Error(w, "RDMA engine ping failed", http.StatusServiceUnavailable)
-		return
-	}
-
-	w.Header().Set("Content-Type", "application/json")
-	response := HealthResponse{
-		Status:              "healthy",
-		RdmaEngineConnected: true,
-		RdmaEngineLatency:   latency.String(),
-		Timestamp:           time.Now().Format(time.RFC3339),
-	}
-	json.NewEncoder(w).Encode(response)
-}
-
-// RDMA capabilities handler
-func (s *Sidecar) capabilitiesHandler(w http.ResponseWriter, r *http.Request) {
-	if r.Method != http.MethodGet {
-		http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
-		return
-	}
-
-	caps := s.rdmaClient.GetCapabilities()
-	if caps == nil {
-		http.Error(w, "No capabilities available", http.StatusServiceUnavailable)
-		return
-	}
-
-	w.Header().Set("Content-Type", "application/json")
-	response := CapabilitiesResponse{
-		Version:         caps.Version,
-		DeviceName:      caps.DeviceName,
-		VendorId:        caps.VendorId,
-		MaxSessions:     uint32(caps.MaxSessions),
-		MaxTransferSize: caps.MaxTransferSize,
-		ActiveSessions:  uint32(caps.ActiveSessions),
-		RealRdma:        caps.RealRdma,
-		PortGid:         caps.PortGid,
-		PortLid:         caps.PortLid,
-		SupportedAuth:   caps.SupportedAuth,
-	}
-	json.NewEncoder(w).Encode(response)
-}
-
-// RDMA ping handler
-func (s *Sidecar) pingHandler(w http.ResponseWriter, r *http.Request) {
-	if r.Method != http.MethodGet {
-		http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
-		return
-	}
-
-	ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
-	defer cancel()
-
-	start := time.Now()
-	latency, err := s.rdmaClient.Ping(ctx)
-	totalLatency := time.Since(start)
-
-	if err != nil {
-		s.logger.WithError(err).Error("❌ RDMA ping failed")
-		http.Error(w, fmt.Sprintf("Ping failed: %v", err), http.StatusInternalServerError)
-		return
-	}
-
-	w.Header().Set("Content-Type", "application/json")
-	response := PingResponse{
-		Success:       true,
-		EngineLatency: latency.String(),
-		TotalLatency:  totalLatency.String(),
-		Timestamp:     time.Now().Format(time.RFC3339),
-	}
-	json.NewEncoder(w).Encode(response)
-}
-
-// RDMA read handler - uses GET method with query parameters for RESTful read operations
-func (s *Sidecar) rdmaReadHandler(w http.ResponseWriter, r *http.Request) {
-	if r.Method != http.MethodGet {
-		http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
-		return
-	}
-
-	// Parse query parameters
-	query := r.URL.Query()
-
-	// Get file ID (e.g., "3,01637037d6") - this is the natural SeaweedFS identifier
-	fileID := query.Get("file_id")
-	if fileID == "" {
-		http.Error(w, "missing 'file_id' parameter", http.StatusBadRequest)
-		return
-	}
-
-	// Parse optional offset and size parameters
-	offset := uint64(0) // default value
-	if offsetStr := query.Get("offset"); offsetStr != "" {
-		val, err := strconv.ParseUint(offsetStr, 10, 64)
-		if err != nil {
-			http.Error(w, "invalid 'offset' parameter", http.StatusBadRequest)
-			return
-		}
-		offset = val
-	}
-
-	size := uint64(4096) // default value
-	if sizeStr := query.Get("size"); sizeStr != "" {
-		val, err := strconv.ParseUint(sizeStr, 10, 64)
-		if err != nil {
-			http.Error(w, "invalid 'size' parameter", http.StatusBadRequest)
-			return
-		}
-		size = val
-	}
-
-	s.logger.WithFields(logrus.Fields{
-		"file_id": fileID,
-		"offset":  offset,
-		"size":    size,
-	}).Info("📖 Processing RDMA read request")
-
-	ctx, cancel := context.WithTimeout(r.Context(), timeout)
-	defer cancel()
-
-	start := time.Now()
-	resp, err := s.rdmaClient.ReadFileRange(ctx, fileID, offset, size)
-	duration := time.Since(start)
-
-	if err != nil {
-		s.logger.WithError(err).Error("❌ RDMA read failed")
-		http.Error(w, fmt.Sprintf("RDMA read failed: %v", err), http.StatusInternalServerError)
-		return
-	}
-
-	s.logger.WithFields(logrus.Fields{
-		"file_id":       fileID,
-		"bytes_read":    resp.BytesRead,
-		"duration":      duration,
-		"transfer_rate": resp.TransferRate,
-		"session_id":    resp.SessionID,
-	}).Info("✅ RDMA read completed successfully")
-
-	// Set response headers
-	w.Header().Set("Content-Type", "application/octet-stream")
-	w.Header().Set("X-RDMA-Session-ID", resp.SessionID)
-	w.Header().Set("X-RDMA-Duration", duration.String())
-	w.Header().Set("X-RDMA-Transfer-Rate", fmt.Sprintf("%.2f", resp.TransferRate))
-	w.Header().Set("X-RDMA-Bytes-Read", fmt.Sprintf("%d", resp.BytesRead))
-
-	// Write the data
-	w.Write(resp.Data)
-}

seaweedfs-rdma-sidecar/cmd/test-rdma/main.go

@@ -1,295 +0,0 @@
-// Package main provides a test client for the RDMA engine integration
-package main
-
-import (
-	"context"
-	"fmt"
-	"os"
-	"time"
-
-	"seaweedfs-rdma-sidecar/pkg/rdma"
-
-	"github.com/sirupsen/logrus"
-	"github.com/spf13/cobra"
-)
-
-var (
-	socketPath string
-	debug      bool
-	timeout    time.Duration
-	volumeID   uint32
-	needleID   uint64
-	cookie     uint32
-	offset     uint64
-	size       uint64
-)
-
-func main() {
-	var rootCmd = &cobra.Command{
-		Use:   "test-rdma",
-		Short: "Test client for SeaweedFS RDMA engine integration",
-		Long: `Test client that demonstrates communication between Go sidecar and Rust RDMA engine.
-		
-This tool allows you to test various RDMA operations including:
-- Engine connectivity and capabilities
-- RDMA read operations with mock data
-- Performance measurements
-- IPC protocol validation`,
-	}
-
-	// Global flags
-	defaultSocketPath := os.Getenv("RDMA_SOCKET_PATH")
-	if defaultSocketPath == "" {
-		defaultSocketPath = "/tmp/rdma-engine.sock"
-	}
-	rootCmd.PersistentFlags().StringVarP(&socketPath, "socket", "s", defaultSocketPath, "Path to RDMA engine Unix socket (env: RDMA_SOCKET_PATH)")
-	rootCmd.PersistentFlags().BoolVarP(&debug, "debug", "d", false, "Enable debug logging")
-	rootCmd.PersistentFlags().DurationVarP(&timeout, "timeout", "t", 30*time.Second, "Operation timeout")
-
-	// Subcommands
-	rootCmd.AddCommand(pingCmd())
-	rootCmd.AddCommand(capsCmd())
-	rootCmd.AddCommand(readCmd())
-	rootCmd.AddCommand(benchCmd())
-
-	if err := rootCmd.Execute(); err != nil {
-		fmt.Fprintf(os.Stderr, "Error: %v\n", err)
-		os.Exit(1)
-	}
-}
-
-func pingCmd() *cobra.Command {
-	return &cobra.Command{
-		Use:   "ping",
-		Short: "Test connectivity to RDMA engine",
-		Long:  "Send a ping message to the RDMA engine and measure latency",
-		RunE: func(cmd *cobra.Command, args []string) error {
-			client := createClient()
-			defer client.Disconnect()
-
-			ctx, cancel := context.WithTimeout(context.Background(), timeout)
-			defer cancel()
-
-			fmt.Printf("🏓 Pinging RDMA engine at %s...\n", socketPath)
-
-			if err := client.Connect(ctx); err != nil {
-				return fmt.Errorf("failed to connect: %w", err)
-			}
-
-			latency, err := client.Ping(ctx)
-			if err != nil {
-				return fmt.Errorf("ping failed: %w", err)
-			}
-
-			fmt.Printf("✅ Ping successful! Latency: %v\n", latency)
-			return nil
-		},
-	}
-}
-
-func capsCmd() *cobra.Command {
-	return &cobra.Command{
-		Use:   "capabilities",
-		Short: "Get RDMA engine capabilities",
-		Long:  "Query the RDMA engine for its current capabilities and status",
-		RunE: func(cmd *cobra.Command, args []string) error {
-			client := createClient()
-			defer client.Disconnect()
-
-			ctx, cancel := context.WithTimeout(context.Background(), timeout)
-			defer cancel()
-
-			fmt.Printf("🔍 Querying RDMA engine capabilities...\n")
-
-			if err := client.Connect(ctx); err != nil {
-				return fmt.Errorf("failed to connect: %w", err)
-			}
-
-			caps := client.GetCapabilities()
-			if caps == nil {
-				return fmt.Errorf("no capabilities received")
-			}
-
-			fmt.Printf("\n📊 RDMA Engine Capabilities:\n")
-			fmt.Printf("  Version: %s\n", caps.Version)
-			fmt.Printf("  Max Sessions: %d\n", caps.MaxSessions)
-			fmt.Printf("  Max Transfer Size: %d bytes (%.1f MB)\n", caps.MaxTransferSize, float64(caps.MaxTransferSize)/(1024*1024))
-			fmt.Printf("  Active Sessions: %d\n", caps.ActiveSessions)
-			fmt.Printf("  Real RDMA: %t\n", caps.RealRdma)
-			fmt.Printf("  Port GID: %s\n", caps.PortGid)
-			fmt.Printf("  Port LID: %d\n", caps.PortLid)
-			fmt.Printf("  Supported Auth: %v\n", caps.SupportedAuth)
-
-			if caps.RealRdma {
-				fmt.Printf("🚀 Hardware RDMA enabled!\n")
-			} else {
-				fmt.Printf("🟡 Using mock RDMA (development mode)\n")
-			}
-
-			return nil
-		},
-	}
-}
-
-func readCmd() *cobra.Command {
-	cmd := &cobra.Command{
-		Use:   "read",
-		Short: "Test RDMA read operation",
-		Long:  "Perform a test RDMA read operation with specified parameters",
-		RunE: func(cmd *cobra.Command, args []string) error {
-			client := createClient()
-			defer client.Disconnect()
-
-			ctx, cancel := context.WithTimeout(context.Background(), timeout)
-			defer cancel()
-
-			fmt.Printf("📖 Testing RDMA read operation...\n")
-			fmt.Printf("  Volume ID: %d\n", volumeID)
-			fmt.Printf("  Needle ID: %d\n", needleID)
-			fmt.Printf("  Cookie: 0x%x\n", cookie)
-			fmt.Printf("  Offset: %d\n", offset)
-			fmt.Printf("  Size: %d bytes\n", size)
-
-			if err := client.Connect(ctx); err != nil {
-				return fmt.Errorf("failed to connect: %w", err)
-			}
-
-			start := time.Now()
-			resp, err := client.ReadRange(ctx, volumeID, needleID, cookie, offset, size)
-			if err != nil {
-				return fmt.Errorf("read failed: %w", err)
-			}
-
-			duration := time.Since(start)
-
-			fmt.Printf("\n✅ RDMA read completed successfully!\n")
-			fmt.Printf("  Session ID: %s\n", resp.SessionID)
-			fmt.Printf("  Bytes Read: %d\n", resp.BytesRead)
-			fmt.Printf("  Duration: %v\n", duration)
-			fmt.Printf("  Transfer Rate: %.2f MB/s\n", resp.TransferRate)
-			fmt.Printf("  Success: %t\n", resp.Success)
-			fmt.Printf("  Message: %s\n", resp.Message)
-
-			// Show first few bytes of data for verification
-			if len(resp.Data) > 0 {
-				displayLen := 32
-				if len(resp.Data) < displayLen {
-					displayLen = len(resp.Data)
-				}
-				fmt.Printf("  Data (first %d bytes): %x\n", displayLen, resp.Data[:displayLen])
-			}
-
-			return nil
-		},
-	}
-
-	cmd.Flags().Uint32VarP(&volumeID, "volume", "v", 1, "Volume ID")
-	cmd.Flags().Uint64VarP(&needleID, "needle", "n", 100, "Needle ID")
-	cmd.Flags().Uint32VarP(&cookie, "cookie", "c", 0x12345678, "Needle cookie")
-	cmd.Flags().Uint64VarP(&offset, "offset", "o", 0, "Read offset")
-	cmd.Flags().Uint64VarP(&size, "size", "z", 4096, "Read size in bytes")
-
-	return cmd
-}
-
-func benchCmd() *cobra.Command {
-	var (
-		iterations int
-		readSize   uint64
-	)
-
-	cmd := &cobra.Command{
-		Use:   "bench",
-		Short: "Benchmark RDMA read performance",
-		Long:  "Run multiple RDMA read operations and measure performance statistics",
-		RunE: func(cmd *cobra.Command, args []string) error {
-			client := createClient()
-			defer client.Disconnect()
-
-			ctx, cancel := context.WithTimeout(context.Background(), timeout)
-			defer cancel()
-
-			fmt.Printf("🏁 Starting RDMA read benchmark...\n")
-			fmt.Printf("  Iterations: %d\n", iterations)
-			fmt.Printf("  Read Size: %d bytes\n", readSize)
-			fmt.Printf("  Socket: %s\n", socketPath)
-
-			if err := client.Connect(ctx); err != nil {
-				return fmt.Errorf("failed to connect: %w", err)
-			}
-
-			// Warmup
-			fmt.Printf("🔥 Warming up...\n")
-			for i := 0; i < 5; i++ {
-				_, err := client.ReadRange(ctx, 1, uint64(i+1), 0x12345678, 0, readSize)
-				if err != nil {
-					return fmt.Errorf("warmup read %d failed: %w", i+1, err)
-				}
-			}
-
-			// Benchmark
-			fmt.Printf("📊 Running benchmark...\n")
-			var totalDuration time.Duration
-			var totalBytes uint64
-			successful := 0
-
-			startTime := time.Now()
-			for i := 0; i < iterations; i++ {
-				opStart := time.Now()
-				resp, err := client.ReadRange(ctx, 1, uint64(i+1), 0x12345678, 0, readSize)
-				opDuration := time.Since(opStart)
-
-				if err != nil {
-					fmt.Printf("❌ Read %d failed: %v\n", i+1, err)
-					continue
-				}
-
-				totalDuration += opDuration
-				totalBytes += resp.BytesRead
-				successful++
-
-				if (i+1)%10 == 0 || i == iterations-1 {
-					fmt.Printf("  Completed %d/%d reads\n", i+1, iterations)
-				}
-			}
-			benchDuration := time.Since(startTime)
-
-			// Calculate statistics
-			avgLatency := totalDuration / time.Duration(successful)
-			throughputMBps := float64(totalBytes) / benchDuration.Seconds() / (1024 * 1024)
-			opsPerSec := float64(successful) / benchDuration.Seconds()
-
-			fmt.Printf("\n📈 Benchmark Results:\n")
-			fmt.Printf("  Total Duration: %v\n", benchDuration)
-			fmt.Printf("  Successful Operations: %d/%d (%.1f%%)\n", successful, iterations, float64(successful)/float64(iterations)*100)
-			fmt.Printf("  Total Bytes Transferred: %d (%.1f MB)\n", totalBytes, float64(totalBytes)/(1024*1024))
-			fmt.Printf("  Average Latency: %v\n", avgLatency)
-			fmt.Printf("  Throughput: %.2f MB/s\n", throughputMBps)
-			fmt.Printf("  Operations/sec: %.1f\n", opsPerSec)
-
-			return nil
-		},
-	}
-
-	cmd.Flags().IntVarP(&iterations, "iterations", "i", 100, "Number of read operations")
-	cmd.Flags().Uint64VarP(&readSize, "read-size", "r", 4096, "Size of each read in bytes")
-
-	return cmd
-}
-
-func createClient() *rdma.Client {
-	logger := logrus.New()
-	if debug {
-		logger.SetLevel(logrus.DebugLevel)
-	} else {
-		logger.SetLevel(logrus.InfoLevel)
-	}
-
-	config := &rdma.Config{
-		EngineSocketPath: socketPath,
-		DefaultTimeout:   timeout,
-		Logger:           logger,
-	}
-
-	return rdma.NewClient(config)
-}

seaweedfs-rdma-sidecar/docker-compose.mount-rdma.yml

@@ -1,267 +0,0 @@
-services:
-  # SeaweedFS Master
-  seaweedfs-master:
-    image: chrislusf/seaweedfs:latest
-    container_name: seaweedfs-master
-    ports:
-      - "9333:9333"
-      - "19333:19333"
-    command: >
-      master
-      -port=9333
-      -mdir=/data
-      -volumeSizeLimitMB=1024
-      -defaultReplication=000
-    volumes:
-      - seaweedfs_master_data:/data
-    networks:
-      - seaweedfs-rdma
-    healthcheck:
-      test: ["CMD", "wget", "--timeout=10", "--quiet", "--tries=1", "--spider", "http://127.0.0.1:9333/cluster/status"]
-      interval: 10s
-      timeout: 10s
-      retries: 6
-      start_period: 60s
-
-  # SeaweedFS Volume Server
-  seaweedfs-volume:
-    image: chrislusf/seaweedfs:latest
-    container_name: seaweedfs-volume
-    ports:
-      - "8080:8080"
-      - "18080:18080"
-    command: >
-      volume
-      -master=seaweedfs-master:9333
-      -port=8080
-      -dir=/data
-      -max=100
-    volumes:
-      - seaweedfs_volume_data:/data
-    networks:
-      - seaweedfs-rdma
-    depends_on:
-      seaweedfs-master:
-        condition: service_healthy
-    healthcheck:
-      test: ["CMD", "sh", "-c", "pgrep weed && netstat -tln | grep :8080"]
-      interval: 10s
-      timeout: 10s
-      retries: 6
-      start_period: 30s
-
-  # SeaweedFS Filer
-  seaweedfs-filer:
-    image: chrislusf/seaweedfs:latest
-    container_name: seaweedfs-filer
-    ports:
-      - "8888:8888"
-      - "18888:18888"
-    command: >
-      filer
-      -master=seaweedfs-master:9333
-      -port=8888
-      -defaultReplicaPlacement=000
-    networks:
-      - seaweedfs-rdma
-    depends_on:
-      seaweedfs-master:
-        condition: service_healthy
-      seaweedfs-volume:
-        condition: service_healthy
-    healthcheck:
-      test: ["CMD", "sh", "-c", "pgrep weed && netstat -tln | grep :8888"]
-      interval: 10s
-      timeout: 10s
-      retries: 6
-      start_period: 45s
-
-  # RDMA Engine (Rust)
-  rdma-engine:
-    build:
-      context: .
-      dockerfile: Dockerfile.rdma-engine
-    container_name: rdma-engine
-    volumes:
-      - rdma_socket:/tmp/rdma
-    networks:
-      - seaweedfs-rdma
-    environment:
-      - RUST_LOG=debug
-      - RDMA_SOCKET_PATH=/tmp/rdma/rdma-engine.sock
-      - RDMA_DEVICE=auto
-      - RDMA_PORT=18515
-      - RDMA_GID_INDEX=0
-      - DEBUG=true
-    command: >
-      ./rdma-engine-server
-      --ipc-socket ${RDMA_SOCKET_PATH}
-      --device ${RDMA_DEVICE}
-      --port ${RDMA_PORT}
-      --debug
-    healthcheck:
-      test: ["CMD", "sh", "-c", "pgrep rdma-engine-server >/dev/null && test -S /tmp/rdma/rdma-engine.sock"]
-      interval: 5s
-      timeout: 3s
-      retries: 5
-      start_period: 10s
-
-  # RDMA Sidecar (Go)
-  rdma-sidecar:
-    build:
-      context: .
-      dockerfile: Dockerfile.sidecar
-    container_name: rdma-sidecar
-    ports:
-      - "8081:8081"
-    volumes:
-      - rdma_socket:/tmp/rdma
-    networks:
-      - seaweedfs-rdma
-    environment:
-      - RDMA_SOCKET_PATH=/tmp/rdma/rdma-engine.sock
-      - VOLUME_SERVER_URL=http://seaweedfs-volume:8080
-      - SIDECAR_PORT=8081
-      - ENABLE_RDMA=true
-      - ENABLE_ZEROCOPY=true
-      - ENABLE_POOLING=true
-      - MAX_CONNECTIONS=10
-      - MAX_IDLE_TIME=5m
-      - DEBUG=true
-    command: >
-      ./demo-server
-      --port ${SIDECAR_PORT}
-      --rdma-socket ${RDMA_SOCKET_PATH}
-      --volume-server ${VOLUME_SERVER_URL}
-      --enable-rdma
-      --enable-zerocopy
-      --enable-pooling
-      --max-connections ${MAX_CONNECTIONS}
-      --max-idle-time ${MAX_IDLE_TIME}
-      --debug
-    depends_on:
-      rdma-engine:
-        condition: service_healthy
-      seaweedfs-volume:
-        condition: service_healthy
-    healthcheck:
-      test: ["CMD", "curl", "-f", "http://localhost:8081/health"]
-      interval: 10s
-      timeout: 5s
-      retries: 3
-      start_period: 15s
-
-  # SeaweedFS Mount with RDMA
-  seaweedfs-mount:
-    build:
-      context: .
-      dockerfile: Dockerfile.mount-rdma
-    platform: linux/amd64
-    container_name: seaweedfs-mount
-    privileged: true  # Required for FUSE
-    devices:
-      - /dev/fuse:/dev/fuse
-    cap_add:
-      - SYS_ADMIN
-    volumes:
-      - seaweedfs_mount:/mnt/seaweedfs
-      - /tmp/seaweedfs-mount-logs:/var/log/seaweedfs
-    networks:
-      - seaweedfs-rdma
-    environment:
-      - FILER_ADDR=seaweedfs-filer:8888
-      - RDMA_SIDECAR_ADDR=rdma-sidecar:8081
-      - MOUNT_POINT=/mnt/seaweedfs
-      - RDMA_ENABLED=true
-      - RDMA_FALLBACK=true
-      - RDMA_MAX_CONCURRENT=64
-      - RDMA_TIMEOUT_MS=5000
-      - DEBUG=true
-    command: /usr/local/bin/mount-helper.sh
-    depends_on:
-      seaweedfs-filer:
-        condition: service_healthy
-      rdma-sidecar:
-        condition: service_healthy
-    healthcheck:
-      test: ["CMD", "mountpoint", "-q", "/mnt/seaweedfs"]
-      interval: 15s
-      timeout: 10s
-      retries: 3
-      start_period: 45s
-
-  # Integration Test Runner
-  integration-test:
-    build:
-      context: .
-      dockerfile: Dockerfile.integration-test
-    container_name: integration-test
-    volumes:
-      - seaweedfs_mount:/mnt/seaweedfs
-      - ./test-results:/test-results
-    networks:
-      - seaweedfs-rdma
-    environment:
-      - MOUNT_POINT=/mnt/seaweedfs
-      - FILER_ADDR=seaweedfs-filer:8888
-      - RDMA_SIDECAR_ADDR=rdma-sidecar:8081
-      - TEST_RESULTS_DIR=/test-results
-    depends_on:
-      seaweedfs-mount:
-        condition: service_healthy
-    command: >
-      sh -c "
-        echo 'Starting RDMA Mount Integration Tests...' &&
-        sleep 10 &&
-        /usr/local/bin/run-integration-tests.sh
-      "
-    profiles:
-      - test
-
-  # Performance Test Runner
-  performance-test:
-    build:
-      context: .
-      dockerfile: Dockerfile.performance-test
-    container_name: performance-test
-    volumes:
-      - seaweedfs_mount:/mnt/seaweedfs
-      - ./performance-results:/performance-results
-    networks:
-      - seaweedfs-rdma
-    environment:
-      - MOUNT_POINT=/mnt/seaweedfs
-      - RDMA_SIDECAR_ADDR=rdma-sidecar:8081
-      - PERFORMANCE_RESULTS_DIR=/performance-results
-    depends_on:
-      seaweedfs-mount:
-        condition: service_healthy
-    command: >
-      sh -c "
-        echo 'Starting RDMA Mount Performance Tests...' &&
-        sleep 10 &&
-        /usr/local/bin/run-performance-tests.sh
-      "
-    profiles:
-      - performance
-
-volumes:
-  seaweedfs_master_data:
-    driver: local
-  seaweedfs_volume_data:
-    driver: local
-  seaweedfs_mount:
-    driver: local
-    driver_opts:
-      type: tmpfs
-      device: tmpfs
-      o: size=1g
-  rdma_socket:
-    driver: local
-
-networks:
-  seaweedfs-rdma:
-    driver: bridge
-    ipam:
-      config:
-        - subnet: 172.20.0.0/16

seaweedfs-rdma-sidecar/docker-compose.rdma-sim.yml

@@ -1,209 +0,0 @@
-services:
-  # SeaweedFS Master Server
-  seaweedfs-master:
-    image: chrislusf/seaweedfs:latest
-    container_name: seaweedfs-master
-    command: master -ip=seaweedfs-master -port=9333 -mdir=/data
-    ports:
-      - "9333:9333"
-    volumes:
-      - master-data:/data
-    networks:
-      - seaweedfs-rdma
-    healthcheck:
-      test: ["CMD", "pgrep", "-f", "weed"]
-      interval: 15s
-      timeout: 10s
-      retries: 5
-      start_period: 30s
-
-  # SeaweedFS Volume Server
-  seaweedfs-volume:
-    image: chrislusf/seaweedfs:latest
-    container_name: seaweedfs-volume
-    command: volume -master=seaweedfs-master:9333 -ip=seaweedfs-volume -port=8080 -dir=/data
-    ports:
-      - "8080:8080"
-    volumes:
-      - volume-data:/data
-    depends_on:
-      seaweedfs-master:
-        condition: service_healthy
-    networks:
-      - seaweedfs-rdma
-    healthcheck:
-      test: ["CMD", "pgrep", "-f", "weed"]
-      interval: 15s
-      timeout: 10s
-      retries: 5
-      start_period: 30s
-
-  # RDMA Simulation Environment
-  rdma-simulation:
-    build:
-      context: .
-      dockerfile: docker/Dockerfile.rdma-simulation
-    container_name: rdma-simulation
-    privileged: true  # Required for RDMA kernel module loading
-    environment:
-      - RDMA_DEVICE=rxe0
-      - UCX_TLS=rc_verbs,ud_verbs,tcp
-      - UCX_LOG_LEVEL=info
-    volumes:
-      - /lib/modules:/lib/modules:ro  # Host kernel modules
-      - /sys:/sys  # Required for sysfs access
-      - rdma-simulation-data:/opt/rdma-sim/data
-    networks:
-      - seaweedfs-rdma
-    ports:
-      - "18515:18515"  # RDMA application port
-      - "4791:4791"    # RDMA CM port
-      - "4792:4792"    # Additional RDMA port
-    command: |
-      bash -c "
-        echo '🚀 Setting up RDMA simulation environment...'
-        sudo /opt/rdma-sim/setup-soft-roce.sh || echo 'RDMA setup failed, continuing...'
-        echo '📋 RDMA environment status:'
-        /opt/rdma-sim/test-rdma.sh || true
-        echo '🔧 UCX information:'
-        /opt/rdma-sim/ucx-info.sh || true
-        echo '✅ RDMA simulation ready - keeping container alive...'
-        tail -f /dev/null
-      "
-    healthcheck:
-      test: ["CMD", "test", "-f", "/opt/rdma-sim/setup-soft-roce.sh"]
-      interval: 30s
-      timeout: 10s
-      retries: 3
-      start_period: 30s
-
-  # Rust RDMA Engine (with RDMA simulation support)
-  rdma-engine:
-    build:
-      context: .
-      dockerfile: Dockerfile.rdma-engine
-    container_name: rdma-engine
-    environment:
-      - RUST_LOG=debug
-      - RDMA_SOCKET_PATH=/tmp/rdma-engine.sock
-      # UCX configuration for real RDMA
-      - UCX_TLS=rc_verbs,ud_verbs,tcp,shm
-      - UCX_NET_DEVICES=all
-      - UCX_LOG_LEVEL=info
-      - UCX_RNDV_SCHEME=put_zcopy
-      - UCX_RNDV_THRESH=8192
-    volumes:
-      - rdma-socket:/tmp
-      # Share network namespace with RDMA simulation for device access
-    network_mode: "container:rdma-simulation"
-    depends_on:
-      rdma-simulation:
-        condition: service_healthy
-    command: ["./rdma-engine-server", "--debug", "--ipc-socket", "/tmp/rdma-engine.sock"]
-    healthcheck:
-      test: ["CMD", "test", "-S", "/tmp/rdma-engine.sock"]
-      interval: 10s
-      timeout: 5s
-      retries: 3
-      start_period: 15s
-
-  # Go RDMA Sidecar / Demo Server
-  rdma-sidecar:
-    build:
-      context: .
-      dockerfile: Dockerfile.sidecar
-    container_name: rdma-sidecar
-    ports:
-      - "8081:8081"
-    environment:
-      - RDMA_SOCKET_PATH=/tmp/rdma-engine.sock
-      - VOLUME_SERVER_URL=http://seaweedfs-volume:8080
-      - DEBUG=true
-    volumes:
-      - rdma-socket:/tmp
-    depends_on:
-      rdma-engine:
-        condition: service_healthy
-      seaweedfs-volume:
-        condition: service_healthy
-    networks:
-      - seaweedfs-rdma
-    command: [
-      "./demo-server",
-      "--port", "8081",
-      "--rdma-socket", "/tmp/rdma-engine.sock",
-      "--volume-server", "http://seaweedfs-volume:8080",
-      "--enable-rdma",
-      "--debug"
-    ]
-    healthcheck:
-      test: ["CMD", "curl", "-f", "http://localhost:8081/health"]
-      interval: 10s
-      timeout: 5s
-      retries: 3
-      start_period: 20s
-
-  # Test Client for Integration Testing
-  test-client:
-    build:
-      context: .
-      dockerfile: Dockerfile.test-client
-    container_name: test-client
-    environment:
-      - RDMA_SOCKET_PATH=/tmp/rdma-engine.sock
-      - SIDECAR_URL=http://rdma-sidecar:8081
-      - SEAWEEDFS_MASTER=http://seaweedfs-master:9333
-      - SEAWEEDFS_VOLUME=http://seaweedfs-volume:8080
-    volumes:
-      - rdma-socket:/tmp
-    depends_on:
-      rdma-sidecar:
-        condition: service_healthy
-    networks:
-      - seaweedfs-rdma
-    profiles:
-      - testing
-    command: ["tail", "-f", "/dev/null"]  # Keep container running for manual testing
-
-  # Integration Test Runner with RDMA
-  integration-tests-rdma:
-    build:
-      context: .
-      dockerfile: Dockerfile.test-client
-    container_name: integration-tests-rdma
-    environment:
-      - RDMA_SOCKET_PATH=/tmp/rdma-engine.sock
-      - SIDECAR_URL=http://rdma-sidecar:8081
-      - SEAWEEDFS_MASTER=http://seaweedfs-master:9333
-      - SEAWEEDFS_VOLUME=http://seaweedfs-volume:8080
-      - RDMA_SIMULATION=true
-    volumes:
-      - rdma-socket:/tmp
-      - ./tests:/tests
-    depends_on:
-      rdma-sidecar:
-        condition: service_healthy
-      rdma-simulation:
-        condition: service_healthy
-    networks:
-      - seaweedfs-rdma
-    profiles:
-      - testing
-    command: ["/tests/run-integration-tests.sh"]
-
-volumes:
-  master-data:
-    driver: local
-  volume-data:
-    driver: local
-  rdma-socket:
-    driver: local
-  rdma-simulation-data:
-    driver: local
-
-networks:
-  seaweedfs-rdma:
-    driver: bridge
-    ipam:
-      config:
-        - subnet: 172.20.0.0/16

seaweedfs-rdma-sidecar/docker-compose.yml

@@ -1,157 +0,0 @@
-services:
-  # SeaweedFS Master Server
-  seaweedfs-master:
-    image: chrislusf/seaweedfs:latest
-    container_name: seaweedfs-master
-    command: master -ip=seaweedfs-master -port=9333 -mdir=/data
-    ports:
-      - "9333:9333"
-    volumes:
-      - master-data:/data
-    networks:
-      - seaweedfs-rdma
-    healthcheck:
-      test: ["CMD", "curl", "-f", "http://localhost:9333/cluster/status"]
-      interval: 10s
-      timeout: 5s
-      retries: 3
-      start_period: 10s
-
-  # SeaweedFS Volume Server
-  seaweedfs-volume:
-    image: chrislusf/seaweedfs:latest
-    container_name: seaweedfs-volume
-    command: volume -master=seaweedfs-master:9333 -ip=seaweedfs-volume -port=8080 -dir=/data
-    ports:
-      - "8080:8080"
-    volumes:
-      - volume-data:/data
-    depends_on:
-      seaweedfs-master:
-        condition: service_healthy
-    networks:
-      - seaweedfs-rdma
-    healthcheck:
-      test: ["CMD", "curl", "-f", "http://localhost:8080/status"]
-      interval: 10s
-      timeout: 5s
-      retries: 3
-      start_period: 15s
-
-  # Rust RDMA Engine
-  rdma-engine:
-    build:
-      context: .
-      dockerfile: Dockerfile.rdma-engine.simple
-    container_name: rdma-engine
-    environment:
-      - RUST_LOG=debug
-      - RDMA_SOCKET_PATH=/tmp/rdma-engine.sock
-    volumes:
-      - rdma-socket:/tmp
-      # Note: hugepages mount commented out to avoid host system requirements
-      # - /dev/hugepages:/dev/hugepages
-    # Privileged mode for RDMA access (in production, use specific capabilities)
-    privileged: true
-    networks:
-      - seaweedfs-rdma
-    command: ["./rdma-engine-server", "--debug", "--ipc-socket", "/tmp/rdma-engine.sock"]
-    healthcheck:
-      test: ["CMD", "test", "-S", "/tmp/rdma-engine.sock"]
-      interval: 5s
-      timeout: 3s
-      retries: 5
-      start_period: 10s
-
-  # Go RDMA Sidecar / Demo Server
-  rdma-sidecar:
-    build:
-      context: .
-      dockerfile: Dockerfile.sidecar
-    container_name: rdma-sidecar
-    ports:
-      - "8081:8081"
-    environment:
-      - RDMA_SOCKET_PATH=/tmp/rdma-engine.sock
-      - VOLUME_SERVER_URL=http://seaweedfs-volume:8080
-      - DEBUG=true
-    volumes:
-      - rdma-socket:/tmp
-    depends_on:
-      rdma-engine:
-        condition: service_healthy
-      seaweedfs-volume:
-        condition: service_healthy
-    networks:
-      - seaweedfs-rdma
-    command: [
-      "./demo-server",
-      "--port", "8081",
-      "--rdma-socket", "/tmp/rdma-engine.sock",
-      "--volume-server", "http://seaweedfs-volume:8080",
-      "--enable-rdma",
-      "--debug"
-    ]
-    healthcheck:
-      test: ["CMD", "curl", "-f", "http://localhost:8081/health"]
-      interval: 10s
-      timeout: 5s
-      retries: 3
-      start_period: 15s
-
-  # Test Client for Integration Testing
-  test-client:
-    build:
-      context: .
-      dockerfile: Dockerfile.test-client
-    container_name: test-client
-    environment:
-      - RDMA_SOCKET_PATH=/tmp/rdma-engine.sock
-      - SIDECAR_URL=http://rdma-sidecar:8081
-      - SEAWEEDFS_MASTER=http://seaweedfs-master:9333
-      - SEAWEEDFS_VOLUME=http://seaweedfs-volume:8080
-    volumes:
-      - rdma-socket:/tmp
-    depends_on:
-      rdma-sidecar:
-        condition: service_healthy
-    networks:
-      - seaweedfs-rdma
-    profiles:
-      - testing
-    command: ["tail", "-f", "/dev/null"]  # Keep container running for manual testing
-
-  # Integration Test Runner
-  integration-tests:
-    build:
-      context: .
-      dockerfile: Dockerfile.test-client
-    container_name: integration-tests
-    environment:
-      - RDMA_SOCKET_PATH=/tmp/rdma-engine.sock
-      - SIDECAR_URL=http://rdma-sidecar:8081
-      - SEAWEEDFS_MASTER=http://seaweedfs-master:9333
-      - SEAWEEDFS_VOLUME=http://seaweedfs-volume:8080
-    volumes:
-      - rdma-socket:/tmp
-      - ./tests:/tests
-    depends_on:
-      rdma-sidecar:
-        condition: service_healthy
-    networks:
-      - seaweedfs-rdma
-    profiles:
-      - testing
-    command: ["/tests/run-integration-tests.sh"]
-
-volumes:
-  master-data:
-    driver: local
-  volume-data:
-    driver: local
-  rdma-socket:
-    driver: local
-
-networks:
-  seaweedfs-rdma:
-    driver: bridge

seaweedfs-rdma-sidecar/docker/Dockerfile.rdma-simulation

@@ -1,77 +0,0 @@
-# RDMA Simulation Container with Soft-RoCE (RXE)
-# This container enables software RDMA over regular Ethernet
-
-FROM ubuntu:22.04
-
-# Install RDMA and networking tools
-RUN apt-get update && apt-get install -y \
-    # System utilities
-    sudo \
-    # RDMA core libraries
-    libibverbs1 \
-    libibverbs-dev \
-    librdmacm1 \
-    librdmacm-dev \
-    rdma-core \
-    ibverbs-utils \
-    infiniband-diags \
-    # Network tools  
-    iproute2 \
-    iputils-ping \
-    net-tools \
-    # Build tools
-    build-essential \
-    pkg-config \
-    cmake \
-    # UCX dependencies
-    libnuma1 \
-    libnuma-dev \
-    # UCX library (pre-built) - try to install but don't fail if not available
-    # libucx0 \
-    # libucx-dev \
-    # Debugging tools
-    strace \
-    gdb \
-    valgrind \
-    # Utilities
-    curl \
-    wget \
-    vim \
-    htop \
-    && rm -rf /var/lib/apt/lists/*
-
-# Try to install UCX tools (optional, may not be available in all repositories)
-RUN apt-get update && \
-    (apt-get install -y ucx-tools || echo "UCX tools not available in repository") && \
-    rm -rf /var/lib/apt/lists/*
-
-# Create rdmauser for security (avoid conflict with system rdma group)
-RUN useradd -m -s /bin/bash -G sudo,rdma rdmauser && \
-    echo "rdmauser ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
-
-# Create directories for RDMA setup
-RUN mkdir -p /opt/rdma-sim /var/log/rdma
-
-# Copy RDMA simulation scripts
-COPY docker/scripts/setup-soft-roce.sh /opt/rdma-sim/
-COPY docker/scripts/test-rdma.sh /opt/rdma-sim/
-COPY docker/scripts/ucx-info.sh /opt/rdma-sim/
-
-# Make scripts executable
-RUN chmod +x /opt/rdma-sim/*.sh
-
-# Set working directory
-WORKDIR /opt/rdma-sim
-
-# Switch to rdmauser
-USER rdmauser
-
-# Default command
-CMD ["/bin/bash"]
-
-# Health check for RDMA devices
-HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
-    CMD /opt/rdma-sim/test-rdma.sh || exit 1
-
-# Expose common RDMA ports
-EXPOSE 18515 4791 4792

seaweedfs-rdma-sidecar/docker/scripts/setup-soft-roce.sh

@@ -1,183 +0,0 @@
-#!/bin/bash
-
-# Setup Soft-RoCE (RXE) for RDMA simulation
-# This script enables RDMA over Ethernet using the RXE kernel module
-
-set -e
-
-echo "🔧 Setting up Soft-RoCE (RXE) RDMA simulation..."
-
-# Function to check if running with required privileges
-check_privileges() {
-    if [ "$EUID" -ne 0 ]; then
-        echo "❌ This script requires root privileges"
-        echo "Run with: sudo $0 or inside a privileged container"
-        exit 1
-    fi
-}
-
-# Function to load RXE kernel module
-load_rxe_module() {
-    echo "📦 Loading RXE kernel module..."
-    
-    # Try to load the rdma_rxe module
-    if modprobe rdma_rxe 2>/dev/null; then
-        echo "✅ rdma_rxe module loaded successfully"
-    else
-        echo "⚠️  Failed to load rdma_rxe module, trying alternative approach..."
-        
-        # Alternative: Try loading rxe_net (older kernels)
-        if modprobe rxe_net 2>/dev/null; then
-            echo "✅ rxe_net module loaded successfully"
-        else
-            echo "❌ Failed to load RXE modules. Possible causes:"
-            echo "  - Kernel doesn't support RXE (needs CONFIG_RDMA_RXE=m)"
-            echo "  - Running in unprivileged container"
-            echo "  - Missing kernel modules"
-            echo ""
-            echo "🔧 Workaround: Run container with --privileged flag"
-            exit 1
-        fi
-    fi
-    
-    # Verify module is loaded
-    if lsmod | grep -q "rdma_rxe\|rxe_net"; then
-        echo "✅ RXE module verification successful"
-    else
-        echo "❌ RXE module verification failed"
-        exit 1
-    fi
-}
-
-# Function to setup virtual RDMA device
-setup_rxe_device() {
-    echo "🌐 Setting up RXE device over Ethernet interface..."
-    
-    # Find available network interface (prefer eth0, fallback to others)
-    local interface=""
-    for iface in eth0 enp0s3 enp0s8 lo; do
-        if ip link show "$iface" >/dev/null 2>&1; then
-            interface="$iface"
-            break
-        fi
-    done
-    
-    if [ -z "$interface" ]; then
-        echo "❌ No suitable network interface found"
-        echo "Available interfaces:"
-        ip link show | grep "^[0-9]" | cut -d':' -f2 | tr -d ' '
-        exit 1
-    fi
-    
-    echo "📡 Using network interface: $interface"
-    
-    # Create RXE device
-    echo "🔨 Creating RXE device on $interface..."
-    
-    # Try modern rxe_cfg approach first
-    if command -v rxe_cfg >/dev/null 2>&1; then
-        rxe_cfg add "$interface" || {
-            echo "⚠️  rxe_cfg failed, trying manual approach..."
-            setup_rxe_manual "$interface"
-        }
-    else
-        echo "⚠️  rxe_cfg not available, using manual setup..."
-        setup_rxe_manual "$interface"
-    fi
-}
-
-# Function to manually setup RXE device
-setup_rxe_manual() {
-    local interface="$1"
-    
-    # Use sysfs interface to create RXE device
-    if [ -d /sys/module/rdma_rxe ]; then
-        echo "$interface" > /sys/module/rdma_rxe/parameters/add 2>/dev/null || {
-            echo "❌ Failed to add RXE device via sysfs"
-            exit 1
-        }
-    else
-        echo "❌ RXE sysfs interface not found"
-        exit 1
-    fi
-}
-
-# Function to verify RDMA devices
-verify_rdma_devices() {
-    echo "🔍 Verifying RDMA devices..."
-    
-    # Check for RDMA devices
-    if [ -d /sys/class/infiniband ]; then
-        local devices=$(ls /sys/class/infiniband/ 2>/dev/null | wc -l)
-        if [ "$devices" -gt 0 ]; then
-            echo "✅ Found $devices RDMA device(s):"
-            ls /sys/class/infiniband/
-            
-            # Show device details
-            for device in /sys/class/infiniband/*; do
-                if [ -d "$device" ]; then
-                    local dev_name=$(basename "$device")
-                    echo "  📋 Device: $dev_name"
-                    
-                    # Try to get device info
-                    if command -v ibv_devinfo >/dev/null 2>&1; then
-                        ibv_devinfo -d "$dev_name" | head -10
-                    fi
-                fi
-            done
-        else
-            echo "❌ No RDMA devices found in /sys/class/infiniband/"
-            exit 1
-        fi
-    else
-        echo "❌ /sys/class/infiniband directory not found"
-        exit 1
-    fi
-}
-
-# Function to test basic RDMA functionality
-test_basic_rdma() {
-    echo "🧪 Testing basic RDMA functionality..."
-    
-    # Test libibverbs
-    if command -v ibv_devinfo >/dev/null 2>&1; then
-        echo "📋 RDMA device information:"
-        ibv_devinfo | head -20
-    else
-        echo "⚠️  ibv_devinfo not available"
-    fi
-    
-    # Test UCX if available
-    if command -v ucx_info >/dev/null 2>&1; then
-        echo "📋 UCX information:"
-        ucx_info -d | head -10
-    else
-        echo "⚠️  UCX tools not available"
-    fi
-}
-
-# Main execution
-main() {
-    echo "🚀 Starting Soft-RoCE RDMA simulation setup..."
-    echo "======================================"
-    
-    check_privileges
-    load_rxe_module
-    setup_rxe_device  
-    verify_rdma_devices
-    test_basic_rdma
-    
-    echo ""
-    echo "🎉 Soft-RoCE setup completed successfully!"
-    echo "======================================"
-    echo "✅ RDMA simulation is ready for testing"
-    echo "📡 You can now run RDMA applications"
-    echo ""
-    echo "Next steps:"
-    echo "  - Test with: /opt/rdma-sim/test-rdma.sh"
-    echo "  - Check UCX: /opt/rdma-sim/ucx-info.sh"
-    echo "  - Run your RDMA applications"
-}
-
-# Execute main function
-main "$@"

seaweedfs-rdma-sidecar/docker/scripts/test-rdma.sh

@@ -1,253 +0,0 @@
-#!/bin/bash
-
-# Test RDMA functionality in simulation environment
-# This script validates that RDMA devices and libraries are working
-
-set -e
-
-echo "🧪 Testing RDMA simulation environment..."
-
-# Colors for output
-RED='\033[0;31m'
-GREEN='\033[0;32m'
-YELLOW='\033[1;33m'
-BLUE='\033[0;34m'
-NC='\033[0m' # No Color
-
-# Function to print colored output
-print_status() {
-    local status="$1"
-    local message="$2"
-    
-    case "$status" in
-        "success")
-            echo -e "${GREEN}✅ $message${NC}"
-            ;;
-        "warning") 
-            echo -e "${YELLOW}⚠️  $message${NC}"
-            ;;
-        "error")
-            echo -e "${RED}❌ $message${NC}"
-            ;;
-        "info")
-            echo -e "${BLUE}📋 $message${NC}"
-            ;;
-    esac
-}
-
-# Function to test RDMA devices
-test_rdma_devices() {
-    print_status "info" "Testing RDMA devices..."
-    
-    # Check for InfiniBand/RDMA devices
-    if [ -d /sys/class/infiniband ]; then
-        local device_count=$(ls /sys/class/infiniband/ 2>/dev/null | wc -l)
-        if [ "$device_count" -gt 0 ]; then
-            print_status "success" "Found $device_count RDMA device(s)"
-            
-            # List devices
-            for device in /sys/class/infiniband/*; do
-                if [ -d "$device" ]; then
-                    local dev_name=$(basename "$device")
-                    print_status "info" "Device: $dev_name"
-                fi
-            done
-            return 0
-        else
-            print_status "error" "No RDMA devices found"
-            return 1
-        fi
-    else
-        print_status "error" "/sys/class/infiniband directory not found"
-        return 1
-    fi
-}
-
-# Function to test libibverbs
-test_libibverbs() {
-    print_status "info" "Testing libibverbs..."
-    
-    if command -v ibv_devinfo >/dev/null 2>&1; then
-        # Get device info
-        local device_info=$(ibv_devinfo 2>/dev/null)
-        if [ -n "$device_info" ]; then
-            print_status "success" "libibverbs working - devices detected"
-            
-            # Show basic info
-            echo "$device_info" | head -5
-            
-            # Test device capabilities
-            if echo "$device_info" | grep -q "transport.*InfiniBand\|transport.*Ethernet"; then
-                print_status "success" "RDMA transport layer detected"
-            else
-                print_status "warning" "Transport layer information unclear"
-            fi
-            
-            return 0
-        else
-            print_status "error" "ibv_devinfo found no devices"
-            return 1
-        fi
-    else
-        print_status "error" "ibv_devinfo command not found"
-        return 1
-    fi
-}
-
-# Function to test UCX
-test_ucx() {
-    print_status "info" "Testing UCX..."
-    
-    if command -v ucx_info >/dev/null 2>&1; then
-        # Test UCX device detection
-        local ucx_output=$(ucx_info -d 2>/dev/null)
-        if [ -n "$ucx_output" ]; then
-            print_status "success" "UCX detecting devices"
-            
-            # Show UCX device info
-            echo "$ucx_output" | head -10
-            
-            # Check for RDMA transports
-            if echo "$ucx_output" | grep -q "rc\|ud\|dc"; then
-                print_status "success" "UCX RDMA transports available" 
-            else
-                print_status "warning" "UCX RDMA transports not detected"
-            fi
-            
-            return 0
-        else
-            print_status "warning" "UCX not detecting devices"
-            return 1
-        fi
-    else
-        print_status "warning" "UCX tools not available"
-        return 1
-    fi
-}
-
-# Function to test RDMA CM (Connection Manager)
-test_rdma_cm() {
-    print_status "info" "Testing RDMA Connection Manager..."
-    
-    # Check for RDMA CM device
-    if [ -e /dev/infiniband/rdma_cm ]; then
-        print_status "success" "RDMA CM device found"
-        return 0
-    else
-        print_status "warning" "RDMA CM device not found"
-        return 1
-    fi
-}
-
-# Function to test basic RDMA operations
-test_rdma_operations() {
-    print_status "info" "Testing basic RDMA operations..."
-    
-    # Try to run a simple RDMA test if tools are available
-    if command -v ibv_rc_pingpong >/dev/null 2>&1; then
-        # This would need a client/server setup, so just check if binary exists
-        print_status "success" "RDMA test tools available (ibv_rc_pingpong)"
-    else
-        print_status "warning" "RDMA test tools not available"
-    fi
-    
-    # Check for other useful RDMA utilities
-    local tools_found=0
-    for tool in ibv_asyncwatch ibv_read_lat ibv_write_lat; do
-        if command -v "$tool" >/dev/null 2>&1; then
-            tools_found=$((tools_found + 1))
-        fi
-    done
-    
-    if [ "$tools_found" -gt 0 ]; then
-        print_status "success" "Found $tools_found additional RDMA test tools"
-    else
-        print_status "warning" "No additional RDMA test tools found"
-    fi
-}
-
-# Function to generate test summary
-generate_summary() {
-    echo ""
-    print_status "info" "RDMA Simulation Test Summary"
-    echo "======================================"
-    
-    # Re-run key tests for summary
-    local devices_ok=0
-    local libibverbs_ok=0
-    local ucx_ok=0
-    
-    if [ -d /sys/class/infiniband ] && [ "$(ls /sys/class/infiniband/ 2>/dev/null | wc -l)" -gt 0 ]; then
-        devices_ok=1
-    fi
-    
-    if command -v ibv_devinfo >/dev/null 2>&1 && ibv_devinfo >/dev/null 2>&1; then
-        libibverbs_ok=1
-    fi
-    
-    if command -v ucx_info >/dev/null 2>&1 && ucx_info -d >/dev/null 2>&1; then
-        ucx_ok=1
-    fi
-    
-    echo "📊 Test Results:"
-    [ "$devices_ok" -eq 1 ] && print_status "success" "RDMA Devices: PASS" || print_status "error" "RDMA Devices: FAIL"
-    [ "$libibverbs_ok" -eq 1 ] && print_status "success" "libibverbs: PASS" || print_status "error" "libibverbs: FAIL"
-    [ "$ucx_ok" -eq 1 ] && print_status "success" "UCX: PASS" || print_status "warning" "UCX: FAIL/WARNING"
-    
-    echo ""
-    if [ "$devices_ok" -eq 1 ] && [ "$libibverbs_ok" -eq 1 ]; then
-        print_status "success" "RDMA simulation environment is ready! 🎉"
-        echo ""
-        print_status "info" "You can now:"
-        echo "  - Run RDMA applications"
-        echo "  - Test SeaweedFS RDMA engine with real RDMA"
-        echo "  - Use UCX for high-performance transfers"
-        return 0
-    else
-        print_status "error" "RDMA simulation setup needs attention"
-        echo ""
-        print_status "info" "Troubleshooting:"
-        echo "  - Run setup script: sudo /opt/rdma-sim/setup-soft-roce.sh"
-        echo "  - Check container privileges (--privileged flag)"
-        echo "  - Verify kernel RDMA support"
-        return 1
-    fi
-}
-
-# Main test execution
-main() {
-    echo "🚀 RDMA Simulation Test Suite"
-    echo "======================================"
-    
-    # Run tests
-    test_rdma_devices || true
-    echo ""
-    
-    test_libibverbs || true
-    echo ""
-    
-    test_ucx || true
-    echo ""
-    
-    test_rdma_cm || true
-    echo ""
-    
-    test_rdma_operations || true
-    echo ""
-    
-    # Generate summary
-    generate_summary
-}
-
-# Health check mode (for Docker healthcheck)
-if [ "$1" = "healthcheck" ]; then
-    # Quick health check - just verify devices exist
-    if [ -d /sys/class/infiniband ] && [ "$(ls /sys/class/infiniband/ 2>/dev/null | wc -l)" -gt 0 ]; then
-        exit 0
-    else
-        exit 1
-    fi
-fi
-
-# Execute main function
-main "$@"

seaweedfs-rdma-sidecar/docker/scripts/ucx-info.sh

@@ -1,269 +0,0 @@
-#!/bin/bash
-
-# UCX Information and Testing Script
-# Provides detailed information about UCX configuration and capabilities
-
-set -e
-
-echo "📋 UCX (Unified Communication X) Information"
-echo "============================================="
-
-# Colors for output
-GREEN='\033[0;32m'
-YELLOW='\033[1;33m'
-BLUE='\033[0;34m'
-NC='\033[0m' # No Color
-
-print_section() {
-    echo -e "\n${BLUE}📌 $1${NC}"
-    echo "----------------------------------------"
-}
-
-print_info() {
-    echo -e "${GREEN}ℹ️  $1${NC}"
-}
-
-print_warning() {
-    echo -e "${YELLOW}⚠️  $1${NC}"
-}
-
-# Function to check UCX installation
-check_ucx_installation() {
-    print_section "UCX Installation Status"
-    
-    if command -v ucx_info >/dev/null 2>&1; then
-        print_info "UCX tools are installed"
-        
-        # Get UCX version
-        if ucx_info -v >/dev/null 2>&1; then
-            local version=$(ucx_info -v 2>/dev/null | head -1)
-            print_info "Version: $version"
-        fi
-    else
-        print_warning "UCX tools not found"
-        echo "Install with: apt-get install ucx-tools libucx-dev"
-        return 1
-    fi
-    
-    # Check UCX libraries
-    local libs_found=0
-    for lib in libucp.so libucs.so libuct.so; do
-        if ldconfig -p | grep -q "$lib"; then
-            libs_found=$((libs_found + 1))
-        fi
-    done
-    
-    if [ "$libs_found" -eq 3 ]; then
-        print_info "All UCX libraries found (ucp, ucs, uct)"
-    else
-        print_warning "Some UCX libraries may be missing ($libs_found/3 found)"
-    fi
-}
-
-# Function to show UCX device information
-show_ucx_devices() {
-    print_section "UCX Transport Devices"
-    
-    if command -v ucx_info >/dev/null 2>&1; then
-        echo "Available UCX transports and devices:"
-        ucx_info -d 2>/dev/null || {
-            print_warning "Failed to get UCX device information"
-            return 1
-        }
-    else
-        print_warning "ucx_info command not available"
-        return 1
-    fi
-}
-
-# Function to show UCX configuration
-show_ucx_config() {
-    print_section "UCX Configuration"
-    
-    if command -v ucx_info >/dev/null 2>&1; then
-        echo "UCX configuration parameters:"
-        ucx_info -c 2>/dev/null | head -20 || {
-            print_warning "Failed to get UCX configuration"
-            return 1
-        }
-        
-        echo ""
-        print_info "Key UCX environment variables:"
-        echo "  UCX_TLS - Transport layers to use"
-        echo "  UCX_NET_DEVICES - Network devices to use"
-        echo "  UCX_LOG_LEVEL - Logging level (error, warn, info, debug, trace)"
-        echo "  UCX_MEMTYPE_CACHE - Memory type caching (y/n)"
-    else
-        print_warning "ucx_info command not available"
-        return 1
-    fi
-}
-
-# Function to test UCX capabilities
-test_ucx_capabilities() {
-    print_section "UCX Capability Testing"
-    
-    if command -v ucx_info >/dev/null 2>&1; then
-        print_info "Testing UCX transport capabilities..."
-        
-        # Check for RDMA transports
-        local ucx_transports=$(ucx_info -d 2>/dev/null | grep -i "transport\|tl:" || true)
-        
-        if echo "$ucx_transports" | grep -q "rc\|dc\|ud"; then
-            print_info "✅ RDMA transports detected (RC/DC/UD)"
-        else
-            print_warning "No RDMA transports detected"
-        fi
-        
-        if echo "$ucx_transports" | grep -q "tcp"; then
-            print_info "✅ TCP transport available"
-        else
-            print_warning "TCP transport not detected"
-        fi
-        
-        if echo "$ucx_transports" | grep -q "shm\|posix"; then
-            print_info "✅ Shared memory transport available"
-        else
-            print_warning "Shared memory transport not detected"
-        fi
-        
-        # Memory types
-        print_info "Testing memory type support..."
-        local memory_info=$(ucx_info -d 2>/dev/null | grep -i "memory\|md:" || true)
-        if [ -n "$memory_info" ]; then
-            echo "$memory_info" | head -5
-        fi
-        
-    else
-        print_warning "Cannot test UCX capabilities - ucx_info not available"
-        return 1
-    fi
-}
-
-# Function to show recommended UCX settings for RDMA
-show_rdma_settings() {
-    print_section "Recommended UCX Settings for RDMA"
-    
-    print_info "For optimal RDMA performance with SeaweedFS:"
-    echo ""
-    echo "Environment Variables:"
-    echo "  export UCX_TLS=rc_verbs,ud_verbs,rc_mlx5_dv,dc_mlx5_dv"
-    echo "  export UCX_NET_DEVICES=all"
-    echo "  export UCX_LOG_LEVEL=info"
-    echo "  export UCX_RNDV_SCHEME=put_zcopy"
-    echo "  export UCX_RNDV_THRESH=8192"
-    echo ""
-    
-    print_info "For development/debugging:"
-    echo "  export UCX_LOG_LEVEL=debug"
-    echo "  export UCX_LOG_FILE=/tmp/ucx.log"
-    echo ""
-    
-    print_info "For Soft-RoCE (RXE) specifically:"
-    echo "  export UCX_TLS=rc_verbs,ud_verbs"
-    echo "  export UCX_IB_DEVICE_SPECS=rxe0:1"  
-    echo ""
-}
-
-# Function to test basic UCX functionality
-test_ucx_basic() {
-    print_section "Basic UCX Functionality Test"
-    
-    if command -v ucx_hello_world >/dev/null 2>&1; then
-        print_info "UCX hello_world test available"
-        echo "You can test UCX with:"
-        echo "  Server: UCX_TLS=tcp ucx_hello_world -l"
-        echo "  Client: UCX_TLS=tcp ucx_hello_world <server_ip>"
-    else
-        print_warning "UCX hello_world test not available"
-    fi
-    
-    # Check for other UCX test utilities
-    local test_tools=0
-    for tool in ucx_perftest ucp_hello_world; do
-        if command -v "$tool" >/dev/null 2>&1; then
-            test_tools=$((test_tools + 1))
-            print_info "UCX test tool available: $tool"
-        fi
-    done
-    
-    if [ "$test_tools" -eq 0 ]; then
-        print_warning "No UCX test tools found"
-        echo "Consider installing: ucx-tools package"
-    fi
-}
-
-# Function to generate UCX summary
-generate_summary() {
-    print_section "UCX Status Summary"
-    
-    local ucx_ok=0
-    local devices_ok=0
-    local rdma_ok=0
-    
-    # Check UCX availability
-    if command -v ucx_info >/dev/null 2>&1; then
-        ucx_ok=1
-    fi
-    
-    # Check devices
-    if command -v ucx_info >/dev/null 2>&1 && ucx_info -d >/dev/null 2>&1; then
-        devices_ok=1
-        
-        # Check for RDMA
-        if ucx_info -d 2>/dev/null | grep -q "rc\|dc\|ud"; then
-            rdma_ok=1
-        fi
-    fi
-    
-    echo "📊 UCX Status:"
-    [ "$ucx_ok" -eq 1 ] && print_info "✅ UCX Installation: OK" || print_warning "❌ UCX Installation: Missing"
-    [ "$devices_ok" -eq 1 ] && print_info "✅ UCX Devices: Detected" || print_warning "❌ UCX Devices: Not detected"
-    [ "$rdma_ok" -eq 1 ] && print_info "✅ RDMA Support: Available" || print_warning "⚠️  RDMA Support: Limited/Missing"
-    
-    echo ""
-    if [ "$ucx_ok" -eq 1 ] && [ "$devices_ok" -eq 1 ]; then
-        print_info "🎉 UCX is ready for SeaweedFS RDMA integration!"
-        
-        if [ "$rdma_ok" -eq 1 ]; then
-            print_info "🚀 Real RDMA acceleration is available"
-        else
-            print_warning "💡 Only TCP/shared memory transports available"
-        fi
-    else
-        print_warning "🔧 UCX setup needs attention for optimal performance"
-    fi
-}
-
-# Main execution
-main() {
-    check_ucx_installation
-    echo ""
-    
-    show_ucx_devices
-    echo ""
-    
-    show_ucx_config
-    echo ""
-    
-    test_ucx_capabilities  
-    echo ""
-    
-    show_rdma_settings
-    echo ""
-    
-    test_ucx_basic
-    echo ""
-    
-    generate_summary
-    
-    echo ""
-    print_info "For SeaweedFS RDMA engine integration:"
-    echo "  1. Use UCX with your Rust engine"
-    echo "  2. Configure appropriate transport layers"
-    echo "  3. Test with SeaweedFS RDMA sidecar"
-    echo "  4. Monitor performance and adjust settings"
-}
-
-# Execute main function
-main "$@"

seaweedfs-rdma-sidecar/go.mod

@@ -1,49 +0,0 @@
-module seaweedfs-rdma-sidecar
-
-go 1.25.8
-
-require (
-	github.com/seaweedfs/seaweedfs v0.0.0-00010101000000-000000000000
-	github.com/sirupsen/logrus v1.9.4
-	github.com/spf13/cobra v1.8.0
-	github.com/vmihailenco/msgpack/v5 v5.4.1
-)
-
-require (
-	github.com/beorn7/perks v1.0.1 // indirect
-	github.com/cespare/xxhash/v2 v2.3.0 // indirect
-	github.com/cognusion/imaging v1.0.4 // indirect
-	github.com/fsnotify/fsnotify v1.9.0 // indirect
-	github.com/go-viper/mapstructure/v2 v2.5.0 // indirect
-	github.com/inconshreveable/mousetrap v1.1.0 // indirect
-	github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
-	github.com/pelletier/go-toml/v2 v2.2.4 // indirect
-	github.com/prometheus/client_golang v1.24.0 // indirect
-	github.com/prometheus/client_model v0.6.2 // indirect
-	github.com/prometheus/common v0.70.0 // indirect
-	github.com/prometheus/procfs v0.21.1 // indirect
-	github.com/sagikazarmark/locafero v0.11.0 // indirect
-	github.com/seaweedfs/goexif v1.0.3 // indirect
-	github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 // indirect
-	github.com/spf13/afero v1.15.0 // indirect
-	github.com/spf13/cast v1.10.0 // indirect
-	github.com/spf13/pflag v1.0.10 // indirect
-	github.com/spf13/viper v1.21.0 // indirect
-	github.com/subosito/gotenv v1.6.0 // indirect
-	github.com/vmihailenco/tagparser/v2 v2.0.0 // indirect
-	go.yaml.in/yaml/v3 v3.0.4 // indirect
-	golang.org/x/image v0.43.0 // indirect
-	golang.org/x/net v0.57.0 // indirect
-	golang.org/x/sys v0.47.0 // indirect
-	golang.org/x/text v0.40.0 // indirect
-	google.golang.org/genproto/googleapis/rpc v0.0.0-20260706201446-f0a921348800 // indirect
-	google.golang.org/grpc v1.82.1 // indirect
-	google.golang.org/protobuf v1.36.11 // indirect
-)
-
-// For local development, this replace directive is required to build the sidecar
-// against the parent SeaweedFS module in this monorepo.
-//
-// To build this module, ensure the main SeaweedFS repository is checked out
-// as a sibling directory to this `seaweedfs-rdma-sidecar` directory.
-replace github.com/seaweedfs/seaweedfs => ../

seaweedfs-rdma-sidecar/go.sum

@@ -1,118 +0,0 @@
-github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
-github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
-github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
-github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
-github.com/cognusion/imaging v1.0.4 h1:hrmWVa4S9fiLJJFKEHYYRNBUnHEKZD8/oxnC77xTUa0=
-github.com/cognusion/imaging v1.0.4/go.mod h1:X1mPSSZpMyiva54chCO/1NROp+b6xG4uMcud8VLii+U=
-github.com/cpuguy83/go-md2man/v2 v2.0.3/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
-github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
-github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
-github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
-github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
-github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
-github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
-github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
-github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
-github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
-github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
-github.com/go-viper/mapstructure/v2 v2.5.0 h1:vM5IJoUAy3d7zRSVtIwQgBj7BiWtMPfmPEgAXnvj1Ro=
-github.com/go-viper/mapstructure/v2 v2.5.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
-github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
-github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
-github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
-github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
-github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
-github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
-github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
-github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
-github.com/klauspost/compress v1.19.0 h1:sXLILfc9jV2QYWkzFOPWStmcUVH2RHEB1JCdY2oVvCQ=
-github.com/klauspost/compress v1.19.0/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
-github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
-github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
-github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
-github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
-github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
-github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
-github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
-github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
-github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
-github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
-github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
-github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
-github.com/prometheus/client_golang v1.24.0 h1:5XStIklKuAtJSNpdD3s8XJj/Yv78IQmE1kbNk87JrAI=
-github.com/prometheus/client_golang v1.24.0/go.mod h1:QcsNdotprC2nS4BTM2ucbcqxd2CeXTEa9jW7zHO9iDE=
-github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
-github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=
-github.com/prometheus/common v0.70.0 h1:bcpru3tWPVnxGnETLgOV5jbp/JRXgYEyv65CuBLAMMI=
-github.com/prometheus/common v0.70.0/go.mod h1:S/SFasQmgGiYH6C81LKCtYa8QACgthGg5zxL2udV7SY=
-github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
-github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
-github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
-github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
-github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
-github.com/sagikazarmark/locafero v0.11.0 h1:1iurJgmM9G3PA/I+wWYIOw/5SyBtxapeHDcg+AAIFXc=
-github.com/sagikazarmark/locafero v0.11.0/go.mod h1:nVIGvgyzw595SUSUE6tvCp3YYTeHs15MvlmU87WwIik=
-github.com/seaweedfs/goexif v1.0.3 h1:ve/OjI7dxPW8X9YQsv3JuVMaxEyF9Rvfd04ouL+Bz30=
-github.com/seaweedfs/goexif v1.0.3/go.mod h1:Oni780Z236sXpIQzk1XoJlTwqrJ02smEin9zQeff7Fk=
-github.com/sirupsen/logrus v1.9.4 h1:TsZE7l11zFCLZnZ+teH4Umoq5BhEIfIzfRDZ1Uzql2w=
-github.com/sirupsen/logrus v1.9.4/go.mod h1:ftWc9WdOfJ0a92nsE2jF5u5ZwH8Bv2zdeOC42RjbV2g=
-github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 h1:+jumHNA0Wrelhe64i8F6HNlS8pkoyMv5sreGx2Ry5Rw=
-github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8/go.mod h1:3n1Cwaq1E1/1lhQhtRK2ts/ZwZEhjcQeJQ1RuC6Q/8U=
-github.com/spf13/afero v1.15.0 h1:b/YBCLWAJdFWJTN9cLhiXXcD7mzKn9Dm86dNnfyQw1I=
-github.com/spf13/afero v1.15.0/go.mod h1:NC2ByUVxtQs4b3sIUphxK0NioZnmxgyCrfzeuq8lxMg=
-github.com/spf13/cast v1.10.0 h1:h2x0u2shc1QuLHfxi+cTJvs30+ZAHOGRic8uyGTDWxY=
-github.com/spf13/cast v1.10.0/go.mod h1:jNfB8QC9IA6ZuY2ZjDp0KtFO2LZZlg4S/7bzP6qqeHo=
-github.com/spf13/cobra v1.8.0 h1:7aJaZx1B85qltLMc546zn58BxxfZdR/W22ej9CFoEf0=
-github.com/spf13/cobra v1.8.0/go.mod h1:WXLWApfZ71AjXPya3WOlMsY9yMs7YeiHhFVlvLyhcho=
-github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
-github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
-github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
-github.com/spf13/viper v1.21.0 h1:x5S+0EU27Lbphp4UKm1C+1oQO+rKx36vfCoaVebLFSU=
-github.com/spf13/viper v1.21.0/go.mod h1:P0lhsswPGWD/1lZJ9ny3fYnVqxiegrlNrEmgLjbTCAY=
-github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
-github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
-github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
-github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
-github.com/vmihailenco/msgpack/v5 v5.4.1 h1:cQriyiUvjTwOHg8QZaPihLWeRAAVoCpE00IUPn0Bjt8=
-github.com/vmihailenco/msgpack/v5 v5.4.1/go.mod h1:GaZTsDaehaPpQVyxrf5mtQlH+pc21PIudVV/E3rRQok=
-github.com/vmihailenco/tagparser/v2 v2.0.0 h1:y09buUbR+b5aycVFQs/g70pqKVZNBmxwAhO7/IwNM9g=
-github.com/vmihailenco/tagparser/v2 v2.0.0/go.mod h1:Wri+At7QHww0WTrCBeu4J6bNtoV6mEfg5OIWRZA9qds=
-go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
-go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
-go.opentelemetry.io/otel v1.44.0 h1:JjwHmHpA4iZ3wBxluu2fbbE7j4kqlE8jXyAyPXH7HqU=
-go.opentelemetry.io/otel v1.44.0/go.mod h1:BMgjTHL9WPRlRjL2oZCBTL4whCGtXch2H4BhOPIAyYc=
-go.opentelemetry.io/otel/metric v1.44.0 h1:1w0gILTcHdr3YI+ixLyjemwrVnsMURbTZFrSYCdDdmc=
-go.opentelemetry.io/otel/metric v1.44.0/go.mod h1:8O7hanEPBNgEMmybD3s2VBKcgWOCsA6tzHBPODAiquo=
-go.opentelemetry.io/otel/sdk v1.44.0 h1:nHYwb9lK+fJPU/dnT6s7W7Z8itMWyqrnVfbheVYrZ58=
-go.opentelemetry.io/otel/sdk v1.44.0/go.mod h1:Osuydd3Se74nqjAKxid74N5eC+jfEqfTegHRnq58oK0=
-go.opentelemetry.io/otel/sdk/metric v1.44.0 h1:3LlKgI+VjbVsjNRFZJZAJ30WjXC5VkNRks6si09iEfI=
-go.opentelemetry.io/otel/sdk/metric v1.44.0/go.mod h1:5B5pMARnXxKhltooO4xUuCBorl65a4EpnTalObqOigA=
-go.opentelemetry.io/otel/trace v1.44.0 h1:jxF5CsGYCe74MCRx2X4g7WsY/VBKRqqpNvXlX/6gtIk=
-go.opentelemetry.io/otel/trace v1.44.0/go.mod h1:oLl1jrMQAVo6v3GAggN+1VH9VIz9iUSvW53sW1Q8PIE=
-go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
-go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
-go.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ=
-go.yaml.in/yaml/v2 v2.4.4/go.mod h1:gMZqIpDtDqOfM0uNfy0SkpRhvUryYH0Z6wdMYcacYXQ=
-go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
-go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
-golang.org/x/image v0.43.0 h1:FLxcP4ec2350nTfOC8ysKtqYSIFbk/QGjw1ZHNP4tsY=
-golang.org/x/image v0.43.0/go.mod h1:rrpelvGFt+kLPAjPM4HeWPgrl0FtafueU//e5N0qk/Q=
-golang.org/x/net v0.57.0 h1:K5+3DljvIuDG9/Jv9rvyMywYNFCQ9RSUY6OOTTkT+tE=
-golang.org/x/net v0.57.0/go.mod h1:KpXc8iv+r3XplLAG/f7Jsf9RPszJzdR0f58q9vGOuEU=
-golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
-golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
-golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs=
-golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
-gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=
-gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=
-google.golang.org/genproto/googleapis/rpc v0.0.0-20260706201446-f0a921348800 h1:qEHAMpSaUhtD0p3NbEEI83HwNGFxEwaSJ1G9PLnCBZE=
-google.golang.org/genproto/googleapis/rpc v0.0.0-20260706201446-f0a921348800/go.mod h1:4Hqkh8ycfw05ld/3BWL7rJOSfebL2Q+DVDeRgYgxUU8=
-google.golang.org/grpc v1.82.1 h1:NnAxzGRA0677vCa4BUkOAnO5+FfQqVl9iUXeD0IqcGE=
-google.golang.org/grpc v1.82.1/go.mod h1:yzTZ1TB1Z3SG+LIYaI+WiE8D5+PZ3ArnrSp8zF3+/ZA=
-google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
-google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
-gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
-gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
-gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
-gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
-gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

seaweedfs-rdma-sidecar/pkg/ipc/client.go

@@ -1,331 +0,0 @@
-package ipc
-
-import (
-	"context"
-	"encoding/binary"
-	"fmt"
-	"net"
-	"sync"
-	"time"
-
-	"github.com/sirupsen/logrus"
-	"github.com/vmihailenco/msgpack/v5"
-)
-
-// Client provides IPC communication with the Rust RDMA engine
-type Client struct {
-	socketPath string
-	conn       net.Conn
-	mu         sync.RWMutex
-	logger     *logrus.Logger
-	connected  bool
-}
-
-// NewClient creates a new IPC client
-func NewClient(socketPath string, logger *logrus.Logger) *Client {
-	if logger == nil {
-		logger = logrus.New()
-		logger.SetLevel(logrus.InfoLevel)
-	}
-
-	return &Client{
-		socketPath: socketPath,
-		logger:     logger,
-	}
-}
-
-// Connect establishes connection to the Rust RDMA engine
-func (c *Client) Connect(ctx context.Context) error {
-	c.mu.Lock()
-	defer c.mu.Unlock()
-
-	if c.connected {
-		return nil
-	}
-
-	c.logger.WithField("socket", c.socketPath).Info("🔗 Connecting to Rust RDMA engine")
-
-	dialer := &net.Dialer{}
-	conn, err := dialer.DialContext(ctx, "unix", c.socketPath)
-	if err != nil {
-		c.logger.WithError(err).Error("❌ Failed to connect to RDMA engine")
-		return fmt.Errorf("failed to connect to RDMA engine at %s: %w", c.socketPath, err)
-	}
-
-	c.conn = conn
-	c.connected = true
-	c.logger.Info("✅ Connected to Rust RDMA engine")
-
-	return nil
-}
-
-// Disconnect closes the connection
-func (c *Client) Disconnect() {
-	c.mu.Lock()
-	defer c.mu.Unlock()
-
-	if c.conn != nil {
-		c.conn.Close()
-		c.conn = nil
-		c.connected = false
-		c.logger.Info("🔌 Disconnected from Rust RDMA engine")
-	}
-}
-
-// IsConnected returns connection status
-func (c *Client) IsConnected() bool {
-	c.mu.RLock()
-	defer c.mu.RUnlock()
-	return c.connected
-}
-
-// SendMessage sends an IPC message and waits for response
-func (c *Client) SendMessage(ctx context.Context, msg *IpcMessage) (*IpcMessage, error) {
-	c.mu.RLock()
-	conn := c.conn
-	connected := c.connected
-	c.mu.RUnlock()
-
-	if !connected || conn == nil {
-		return nil, fmt.Errorf("not connected to RDMA engine")
-	}
-
-	// Set write timeout
-	if deadline, ok := ctx.Deadline(); ok {
-		conn.SetWriteDeadline(deadline)
-	} else {
-		conn.SetWriteDeadline(time.Now().Add(30 * time.Second))
-	}
-
-	c.logger.WithField("type", msg.Type).Debug("📤 Sending message to Rust engine")
-
-	// Serialize message with MessagePack
-	data, err := msgpack.Marshal(msg)
-	if err != nil {
-		c.logger.WithError(err).Error("❌ Failed to marshal message")
-		return nil, fmt.Errorf("failed to marshal message: %w", err)
-	}
-
-	// Send message length (4 bytes) + message data
-	lengthBytes := make([]byte, 4)
-	binary.LittleEndian.PutUint32(lengthBytes, uint32(len(data)))
-
-	if _, err := conn.Write(lengthBytes); err != nil {
-		c.logger.WithError(err).Error("❌ Failed to send message length")
-		return nil, fmt.Errorf("failed to send message length: %w", err)
-	}
-
-	if _, err := conn.Write(data); err != nil {
-		c.logger.WithError(err).Error("❌ Failed to send message data")
-		return nil, fmt.Errorf("failed to send message data: %w", err)
-	}
-
-	c.logger.WithFields(logrus.Fields{
-		"type": msg.Type,
-		"size": len(data),
-	}).Debug("📤 Message sent successfully")
-
-	// Read response
-	return c.readResponse(ctx, conn)
-}
-
-// readResponse reads and deserializes the response message
-func (c *Client) readResponse(ctx context.Context, conn net.Conn) (*IpcMessage, error) {
-	// Set read timeout
-	if deadline, ok := ctx.Deadline(); ok {
-		conn.SetReadDeadline(deadline)
-	} else {
-		conn.SetReadDeadline(time.Now().Add(30 * time.Second))
-	}
-
-	// Read message length (4 bytes)
-	lengthBytes := make([]byte, 4)
-	if _, err := conn.Read(lengthBytes); err != nil {
-		c.logger.WithError(err).Error("❌ Failed to read response length")
-		return nil, fmt.Errorf("failed to read response length: %w", err)
-	}
-
-	length := binary.LittleEndian.Uint32(lengthBytes)
-	if length > 64*1024*1024 { // 64MB sanity check
-		c.logger.WithField("length", length).Error("❌ Response message too large")
-		return nil, fmt.Errorf("response message too large: %d bytes", length)
-	}
-
-	// Read message data
-	data := make([]byte, length)
-	if _, err := conn.Read(data); err != nil {
-		c.logger.WithError(err).Error("❌ Failed to read response data")
-		return nil, fmt.Errorf("failed to read response data: %w", err)
-	}
-
-	c.logger.WithField("size", length).Debug("📥 Response received")
-
-	// Deserialize with MessagePack
-	var response IpcMessage
-	if err := msgpack.Unmarshal(data, &response); err != nil {
-		c.logger.WithError(err).Error("❌ Failed to unmarshal response")
-		return nil, fmt.Errorf("failed to unmarshal response: %w", err)
-	}
-
-	c.logger.WithField("type", response.Type).Debug("📥 Response deserialized successfully")
-
-	return &response, nil
-}
-
-// High-level convenience methods
-
-// Ping sends a ping message to test connectivity
-func (c *Client) Ping(ctx context.Context, clientID *string) (*PongResponse, error) {
-	msg := NewPingMessage(clientID)
-
-	response, err := c.SendMessage(ctx, msg)
-	if err != nil {
-		return nil, err
-	}
-
-	if response.Type == MsgError {
-		errorData, err := msgpack.Marshal(response.Data)
-		if err != nil {
-			return nil, fmt.Errorf("failed to marshal engine error data: %w", err)
-		}
-		var errorResp ErrorResponse
-		if err := msgpack.Unmarshal(errorData, &errorResp); err != nil {
-			return nil, fmt.Errorf("failed to unmarshal engine error response: %w", err)
-		}
-		return nil, fmt.Errorf("engine error: %s - %s", errorResp.Code, errorResp.Message)
-	}
-
-	if response.Type != MsgPong {
-		return nil, fmt.Errorf("unexpected response type: %s", response.Type)
-	}
-
-	// Convert response data to PongResponse
-	pongData, err := msgpack.Marshal(response.Data)
-	if err != nil {
-		return nil, fmt.Errorf("failed to marshal pong data: %w", err)
-	}
-
-	var pong PongResponse
-	if err := msgpack.Unmarshal(pongData, &pong); err != nil {
-		return nil, fmt.Errorf("failed to unmarshal pong response: %w", err)
-	}
-
-	return &pong, nil
-}
-
-// GetCapabilities requests engine capabilities
-func (c *Client) GetCapabilities(ctx context.Context, clientID *string) (*GetCapabilitiesResponse, error) {
-	msg := NewGetCapabilitiesMessage(clientID)
-
-	response, err := c.SendMessage(ctx, msg)
-	if err != nil {
-		return nil, err
-	}
-
-	if response.Type == MsgError {
-		errorData, err := msgpack.Marshal(response.Data)
-		if err != nil {
-			return nil, fmt.Errorf("failed to marshal engine error data: %w", err)
-		}
-		var errorResp ErrorResponse
-		if err := msgpack.Unmarshal(errorData, &errorResp); err != nil {
-			return nil, fmt.Errorf("failed to unmarshal engine error response: %w", err)
-		}
-		return nil, fmt.Errorf("engine error: %s - %s", errorResp.Code, errorResp.Message)
-	}
-
-	if response.Type != MsgGetCapabilitiesResponse {
-		return nil, fmt.Errorf("unexpected response type: %s", response.Type)
-	}
-
-	// Convert response data to GetCapabilitiesResponse
-	capsData, err := msgpack.Marshal(response.Data)
-	if err != nil {
-		return nil, fmt.Errorf("failed to marshal capabilities data: %w", err)
-	}
-
-	var caps GetCapabilitiesResponse
-	if err := msgpack.Unmarshal(capsData, &caps); err != nil {
-		return nil, fmt.Errorf("failed to unmarshal capabilities response: %w", err)
-	}
-
-	return &caps, nil
-}
-
-// StartRead initiates an RDMA read operation
-func (c *Client) StartRead(ctx context.Context, req *StartReadRequest) (*StartReadResponse, error) {
-	msg := NewStartReadMessage(req)
-
-	response, err := c.SendMessage(ctx, msg)
-	if err != nil {
-		return nil, err
-	}
-
-	if response.Type == MsgError {
-		errorData, err := msgpack.Marshal(response.Data)
-		if err != nil {
-			return nil, fmt.Errorf("failed to marshal engine error data: %w", err)
-		}
-		var errorResp ErrorResponse
-		if err := msgpack.Unmarshal(errorData, &errorResp); err != nil {
-			return nil, fmt.Errorf("failed to unmarshal engine error response: %w", err)
-		}
-		return nil, fmt.Errorf("engine error: %s - %s", errorResp.Code, errorResp.Message)
-	}
-
-	if response.Type != MsgStartReadResponse {
-		return nil, fmt.Errorf("unexpected response type: %s", response.Type)
-	}
-
-	// Convert response data to StartReadResponse
-	startData, err := msgpack.Marshal(response.Data)
-	if err != nil {
-		return nil, fmt.Errorf("failed to marshal start read data: %w", err)
-	}
-
-	var startResp StartReadResponse
-	if err := msgpack.Unmarshal(startData, &startResp); err != nil {
-		return nil, fmt.Errorf("failed to unmarshal start read response: %w", err)
-	}
-
-	return &startResp, nil
-}
-
-// CompleteRead completes an RDMA read operation
-func (c *Client) CompleteRead(ctx context.Context, sessionID string, success bool, bytesTransferred uint64, clientCrc *uint32) (*CompleteReadResponse, error) {
-	msg := NewCompleteReadMessage(sessionID, success, bytesTransferred, clientCrc, nil)
-
-	response, err := c.SendMessage(ctx, msg)
-	if err != nil {
-		return nil, err
-	}
-
-	if response.Type == MsgError {
-		errorData, err := msgpack.Marshal(response.Data)
-		if err != nil {
-			return nil, fmt.Errorf("failed to marshal engine error data: %w", err)
-		}
-		var errorResp ErrorResponse
-		if err := msgpack.Unmarshal(errorData, &errorResp); err != nil {
-			return nil, fmt.Errorf("failed to unmarshal engine error response: %w", err)
-		}
-		return nil, fmt.Errorf("engine error: %s - %s", errorResp.Code, errorResp.Message)
-	}
-
-	if response.Type != MsgCompleteReadResponse {
-		return nil, fmt.Errorf("unexpected response type: %s", response.Type)
-	}
-
-	// Convert response data to CompleteReadResponse
-	completeData, err := msgpack.Marshal(response.Data)
-	if err != nil {
-		return nil, fmt.Errorf("failed to marshal complete read data: %w", err)
-	}
-
-	var completeResp CompleteReadResponse
-	if err := msgpack.Unmarshal(completeData, &completeResp); err != nil {
-		return nil, fmt.Errorf("failed to unmarshal complete read response: %w", err)
-	}
-
-	return &completeResp, nil
-}

seaweedfs-rdma-sidecar/pkg/ipc/messages.go

@@ -1,160 +0,0 @@
-// Package ipc provides communication between Go sidecar and Rust RDMA engine
-package ipc
-
-import "time"
-
-// IpcMessage represents the tagged union of all IPC messages
-// This matches the Rust enum: #[serde(tag = "type", content = "data")]
-type IpcMessage struct {
-	Type string      `msgpack:"type"`
-	Data interface{} `msgpack:"data"`
-}
-
-// Request message types
-const (
-	MsgStartRead       = "StartRead"
-	MsgCompleteRead    = "CompleteRead"
-	MsgGetCapabilities = "GetCapabilities"
-	MsgPing            = "Ping"
-)
-
-// Response message types
-const (
-	MsgStartReadResponse       = "StartReadResponse"
-	MsgCompleteReadResponse    = "CompleteReadResponse"
-	MsgGetCapabilitiesResponse = "GetCapabilitiesResponse"
-	MsgPong                    = "Pong"
-	MsgError                   = "Error"
-)
-
-// StartReadRequest corresponds to Rust StartReadRequest
-type StartReadRequest struct {
-	VolumeID    uint32  `msgpack:"volume_id"`
-	NeedleID    uint64  `msgpack:"needle_id"`
-	Cookie      uint32  `msgpack:"cookie"`
-	Offset      uint64  `msgpack:"offset"`
-	Size        uint64  `msgpack:"size"`
-	RemoteAddr  uint64  `msgpack:"remote_addr"`
-	RemoteKey   uint32  `msgpack:"remote_key"`
-	TimeoutSecs uint64  `msgpack:"timeout_secs"`
-	AuthToken   *string `msgpack:"auth_token,omitempty"`
-}
-
-// StartReadResponse corresponds to Rust StartReadResponse
-type StartReadResponse struct {
-	SessionID    string `msgpack:"session_id"`
-	LocalAddr    uint64 `msgpack:"local_addr"`
-	LocalKey     uint32 `msgpack:"local_key"`
-	TransferSize uint64 `msgpack:"transfer_size"`
-	ExpectedCrc  uint32 `msgpack:"expected_crc"`
-	ExpiresAtNs  uint64 `msgpack:"expires_at_ns"`
-}
-
-// CompleteReadRequest corresponds to Rust CompleteReadRequest
-type CompleteReadRequest struct {
-	SessionID        string  `msgpack:"session_id"`
-	Success          bool    `msgpack:"success"`
-	BytesTransferred uint64  `msgpack:"bytes_transferred"`
-	ClientCrc        *uint32 `msgpack:"client_crc,omitempty"`
-	ErrorMessage     *string `msgpack:"error_message,omitempty"`
-}
-
-// CompleteReadResponse corresponds to Rust CompleteReadResponse
-type CompleteReadResponse struct {
-	Success   bool    `msgpack:"success"`
-	ServerCrc *uint32 `msgpack:"server_crc,omitempty"`
-	Message   *string `msgpack:"message,omitempty"`
-}
-
-// GetCapabilitiesRequest corresponds to Rust GetCapabilitiesRequest
-type GetCapabilitiesRequest struct {
-	ClientID *string `msgpack:"client_id,omitempty"`
-}
-
-// GetCapabilitiesResponse corresponds to Rust GetCapabilitiesResponse
-type GetCapabilitiesResponse struct {
-	DeviceName      string   `msgpack:"device_name"`
-	VendorId        uint32   `msgpack:"vendor_id"`
-	MaxTransferSize uint64   `msgpack:"max_transfer_size"`
-	MaxSessions     usize    `msgpack:"max_sessions"`
-	ActiveSessions  usize    `msgpack:"active_sessions"`
-	PortGid         string   `msgpack:"port_gid"`
-	PortLid         uint16   `msgpack:"port_lid"`
-	SupportedAuth   []string `msgpack:"supported_auth"`
-	Version         string   `msgpack:"version"`
-	RealRdma        bool     `msgpack:"real_rdma"`
-}
-
-// usize corresponds to Rust's usize type (platform dependent, but typically uint64 on 64-bit systems)
-type usize uint64
-
-// PingRequest corresponds to Rust PingRequest
-type PingRequest struct {
-	TimestampNs uint64  `msgpack:"timestamp_ns"`
-	ClientID    *string `msgpack:"client_id,omitempty"`
-}
-
-// PongResponse corresponds to Rust PongResponse
-type PongResponse struct {
-	ClientTimestampNs uint64 `msgpack:"client_timestamp_ns"`
-	ServerTimestampNs uint64 `msgpack:"server_timestamp_ns"`
-	ServerRttNs       uint64 `msgpack:"server_rtt_ns"`
-}
-
-// ErrorResponse corresponds to Rust ErrorResponse
-type ErrorResponse struct {
-	Code    string  `msgpack:"code"`
-	Message string  `msgpack:"message"`
-	Details *string `msgpack:"details,omitempty"`
-}
-
-// Helper functions for creating messages
-func NewStartReadMessage(req *StartReadRequest) *IpcMessage {
-	return &IpcMessage{
-		Type: MsgStartRead,
-		Data: req,
-	}
-}
-
-func NewCompleteReadMessage(sessionID string, success bool, bytesTransferred uint64, clientCrc *uint32, errorMessage *string) *IpcMessage {
-	return &IpcMessage{
-		Type: MsgCompleteRead,
-		Data: &CompleteReadRequest{
-			SessionID:        sessionID,
-			Success:          success,
-			BytesTransferred: bytesTransferred,
-			ClientCrc:        clientCrc,
-			ErrorMessage:     errorMessage,
-		},
-	}
-}
-
-func NewGetCapabilitiesMessage(clientID *string) *IpcMessage {
-	return &IpcMessage{
-		Type: MsgGetCapabilities,
-		Data: &GetCapabilitiesRequest{
-			ClientID: clientID,
-		},
-	}
-}
-
-func NewPingMessage(clientID *string) *IpcMessage {
-	return &IpcMessage{
-		Type: MsgPing,
-		Data: &PingRequest{
-			TimestampNs: uint64(time.Now().UnixNano()),
-			ClientID:    clientID,
-		},
-	}
-}
-
-func NewErrorMessage(code, message string, details *string) *IpcMessage {
-	return &IpcMessage{
-		Type: MsgError,
-		Data: &ErrorResponse{
-			Code:    code,
-			Message: message,
-			Details: details,
-		},
-	}
-}

seaweedfs-rdma-sidecar/pkg/rdma/client.go

@@ -1,630 +0,0 @@
-// Package rdma provides high-level RDMA operations for SeaweedFS integration
-package rdma
-
-import (
-	"context"
-	"fmt"
-	"sync"
-	"time"
-
-	"seaweedfs-rdma-sidecar/pkg/ipc"
-
-	"github.com/seaweedfs/seaweedfs/weed/storage/needle"
-	"github.com/sirupsen/logrus"
-)
-
-// PooledConnection represents a pooled RDMA connection
-type PooledConnection struct {
-	ipcClient *ipc.Client
-	lastUsed  time.Time
-	inUse     bool
-	sessionID string
-	created   time.Time
-}
-
-// ConnectionPool manages a pool of RDMA connections
-type ConnectionPool struct {
-	connections    []*PooledConnection
-	mutex          sync.RWMutex
-	maxConnections int
-	maxIdleTime    time.Duration
-	enginePath     string
-	logger         *logrus.Logger
-}
-
-// Client provides high-level RDMA operations with connection pooling
-type Client struct {
-	pool           *ConnectionPool
-	logger         *logrus.Logger
-	enginePath     string
-	capabilities   *ipc.GetCapabilitiesResponse
-	connected      bool
-	defaultTimeout time.Duration
-
-	// Legacy single connection (for backward compatibility)
-	ipcClient *ipc.Client
-}
-
-// Config holds configuration for the RDMA client
-type Config struct {
-	EngineSocketPath string
-	DefaultTimeout   time.Duration
-	Logger           *logrus.Logger
-
-	// Connection pooling options
-	EnablePooling  bool          // Enable connection pooling (default: true)
-	MaxConnections int           // Max connections in pool (default: 10)
-	MaxIdleTime    time.Duration // Max idle time before connection cleanup (default: 5min)
-}
-
-// ReadRequest represents a SeaweedFS needle read request
-type ReadRequest struct {
-	VolumeID  uint32
-	NeedleID  uint64
-	Cookie    uint32
-	Offset    uint64
-	Size      uint64
-	AuthToken *string
-}
-
-// ReadResponse represents the result of an RDMA read operation
-type ReadResponse struct {
-	Data         []byte
-	BytesRead    uint64
-	Duration     time.Duration
-	TransferRate float64
-	SessionID    string
-	Success      bool
-	Message      string
-}
-
-// NewConnectionPool creates a new connection pool
-func NewConnectionPool(enginePath string, maxConnections int, maxIdleTime time.Duration, logger *logrus.Logger) *ConnectionPool {
-	if maxConnections <= 0 {
-		maxConnections = 10 // Default
-	}
-	if maxIdleTime <= 0 {
-		maxIdleTime = 5 * time.Minute // Default
-	}
-
-	return &ConnectionPool{
-		connections:    make([]*PooledConnection, 0, maxConnections),
-		maxConnections: maxConnections,
-		maxIdleTime:    maxIdleTime,
-		enginePath:     enginePath,
-		logger:         logger,
-	}
-}
-
-// getConnection gets an available connection from the pool or creates a new one
-func (p *ConnectionPool) getConnection(ctx context.Context) (*PooledConnection, error) {
-	p.mutex.Lock()
-	defer p.mutex.Unlock()
-
-	// Look for an available connection
-	for _, conn := range p.connections {
-		if !conn.inUse && time.Since(conn.lastUsed) < p.maxIdleTime {
-			conn.inUse = true
-			conn.lastUsed = time.Now()
-			p.logger.WithField("session_id", conn.sessionID).Debug("🔌 Reusing pooled RDMA connection")
-			return conn, nil
-		}
-	}
-
-	// Create new connection if under limit
-	if len(p.connections) < p.maxConnections {
-		ipcClient := ipc.NewClient(p.enginePath, p.logger)
-		if err := ipcClient.Connect(ctx); err != nil {
-			return nil, fmt.Errorf("failed to create new pooled connection: %w", err)
-		}
-
-		conn := &PooledConnection{
-			ipcClient: ipcClient,
-			lastUsed:  time.Now(),
-			inUse:     true,
-			sessionID: fmt.Sprintf("pool-%d-%d", len(p.connections), time.Now().Unix()),
-			created:   time.Now(),
-		}
-
-		p.connections = append(p.connections, conn)
-		p.logger.WithFields(logrus.Fields{
-			"session_id": conn.sessionID,
-			"pool_size":  len(p.connections),
-		}).Info("🚀 Created new pooled RDMA connection")
-
-		return conn, nil
-	}
-
-	// Pool is full, wait for an available connection
-	return nil, fmt.Errorf("connection pool exhausted (max: %d)", p.maxConnections)
-}
-
-// releaseConnection returns a connection to the pool
-func (p *ConnectionPool) releaseConnection(conn *PooledConnection) {
-	p.mutex.Lock()
-	defer p.mutex.Unlock()
-
-	conn.inUse = false
-	conn.lastUsed = time.Now()
-
-	p.logger.WithField("session_id", conn.sessionID).Debug("🔄 Released RDMA connection back to pool")
-}
-
-// cleanup removes idle connections from the pool
-func (p *ConnectionPool) cleanup() {
-	p.mutex.Lock()
-	defer p.mutex.Unlock()
-
-	now := time.Now()
-	activeConnections := make([]*PooledConnection, 0, len(p.connections))
-
-	for _, conn := range p.connections {
-		if conn.inUse || now.Sub(conn.lastUsed) < p.maxIdleTime {
-			activeConnections = append(activeConnections, conn)
-		} else {
-			// Close idle connection
-			conn.ipcClient.Disconnect()
-			p.logger.WithFields(logrus.Fields{
-				"session_id": conn.sessionID,
-				"idle_time":  now.Sub(conn.lastUsed),
-			}).Debug("🧹 Cleaned up idle RDMA connection")
-		}
-	}
-
-	p.connections = activeConnections
-}
-
-// Close closes all connections in the pool
-func (p *ConnectionPool) Close() {
-	p.mutex.Lock()
-	defer p.mutex.Unlock()
-
-	for _, conn := range p.connections {
-		conn.ipcClient.Disconnect()
-	}
-	p.connections = nil
-	p.logger.Info("🔌 Connection pool closed")
-}
-
-// NewClient creates a new RDMA client
-func NewClient(config *Config) *Client {
-	if config.Logger == nil {
-		config.Logger = logrus.New()
-		config.Logger.SetLevel(logrus.InfoLevel)
-	}
-
-	if config.DefaultTimeout == 0 {
-		config.DefaultTimeout = 30 * time.Second
-	}
-
-	client := &Client{
-		logger:         config.Logger,
-		enginePath:     config.EngineSocketPath,
-		defaultTimeout: config.DefaultTimeout,
-	}
-
-	// Initialize connection pooling if enabled (default: true)
-	enablePooling := config.EnablePooling
-	if config.MaxConnections == 0 && config.MaxIdleTime == 0 {
-		// Default to enabled if not explicitly configured
-		enablePooling = true
-	}
-
-	if enablePooling {
-		client.pool = NewConnectionPool(
-			config.EngineSocketPath,
-			config.MaxConnections,
-			config.MaxIdleTime,
-			config.Logger,
-		)
-
-		// Start cleanup goroutine
-		go client.startCleanupRoutine()
-
-		config.Logger.WithFields(logrus.Fields{
-			"max_connections": client.pool.maxConnections,
-			"max_idle_time":   client.pool.maxIdleTime,
-		}).Info("🔌 RDMA connection pooling enabled")
-	} else {
-		// Legacy single connection mode
-		client.ipcClient = ipc.NewClient(config.EngineSocketPath, config.Logger)
-		config.Logger.Info("🔌 RDMA single connection mode (pooling disabled)")
-	}
-
-	return client
-}
-
-// startCleanupRoutine starts a background goroutine to clean up idle connections
-func (c *Client) startCleanupRoutine() {
-	ticker := time.NewTicker(1 * time.Minute) // Cleanup every minute
-	go func() {
-		defer ticker.Stop()
-		for range ticker.C {
-			if c.pool != nil {
-				c.pool.cleanup()
-			}
-		}
-	}()
-}
-
-// Connect establishes connection to the Rust RDMA engine and queries capabilities
-func (c *Client) Connect(ctx context.Context) error {
-	c.logger.Info("🚀 Connecting to RDMA engine")
-
-	if c.pool != nil {
-		// Connection pooling mode - connections are created on-demand
-		c.connected = true
-		c.logger.Info("✅ RDMA client ready (connection pooling enabled)")
-		return nil
-	}
-
-	// Single connection mode
-	if err := c.ipcClient.Connect(ctx); err != nil {
-		return fmt.Errorf("failed to connect to IPC: %w", err)
-	}
-
-	// Test connectivity with ping
-	clientID := "rdma-client"
-	pong, err := c.ipcClient.Ping(ctx, &clientID)
-	if err != nil {
-		c.ipcClient.Disconnect()
-		return fmt.Errorf("failed to ping RDMA engine: %w", err)
-	}
-
-	latency := time.Duration(pong.ServerRttNs)
-	c.logger.WithFields(logrus.Fields{
-		"latency":    latency,
-		"server_rtt": time.Duration(pong.ServerRttNs),
-	}).Info("📡 RDMA engine ping successful")
-
-	// Get capabilities
-	caps, err := c.ipcClient.GetCapabilities(ctx, &clientID)
-	if err != nil {
-		c.ipcClient.Disconnect()
-		return fmt.Errorf("failed to get engine capabilities: %w", err)
-	}
-
-	c.capabilities = caps
-	c.connected = true
-
-	c.logger.WithFields(logrus.Fields{
-		"version":           caps.Version,
-		"device_name":       caps.DeviceName,
-		"vendor_id":         caps.VendorId,
-		"max_sessions":      caps.MaxSessions,
-		"max_transfer_size": caps.MaxTransferSize,
-		"active_sessions":   caps.ActiveSessions,
-		"real_rdma":         caps.RealRdma,
-		"port_gid":          caps.PortGid,
-		"port_lid":          caps.PortLid,
-	}).Info("✅ RDMA engine connected and ready")
-
-	return nil
-}
-
-// Disconnect closes the connection to the RDMA engine
-func (c *Client) Disconnect() {
-	if c.connected {
-		if c.pool != nil {
-			// Connection pooling mode
-			c.pool.Close()
-			c.logger.Info("🔌 Disconnected from RDMA engine (pool closed)")
-		} else {
-			// Single connection mode
-			c.ipcClient.Disconnect()
-			c.logger.Info("🔌 Disconnected from RDMA engine")
-		}
-		c.connected = false
-	}
-}
-
-// IsConnected returns true if connected to the RDMA engine
-func (c *Client) IsConnected() bool {
-	if c.pool != nil {
-		// Connection pooling mode - always connected if pool exists
-		return c.connected
-	} else {
-		// Single connection mode
-		return c.connected && c.ipcClient.IsConnected()
-	}
-}
-
-// GetCapabilities returns the RDMA engine capabilities
-func (c *Client) GetCapabilities() *ipc.GetCapabilitiesResponse {
-	return c.capabilities
-}
-
-// Read performs an RDMA read operation for a SeaweedFS needle
-func (c *Client) Read(ctx context.Context, req *ReadRequest) (*ReadResponse, error) {
-	if !c.IsConnected() {
-		return nil, fmt.Errorf("not connected to RDMA engine")
-	}
-
-	startTime := time.Now()
-
-	c.logger.WithFields(logrus.Fields{
-		"volume_id": req.VolumeID,
-		"needle_id": req.NeedleID,
-		"offset":    req.Offset,
-		"size":      req.Size,
-	}).Debug("📖 Starting RDMA read operation")
-
-	if c.pool != nil {
-		// Connection pooling mode
-		return c.readWithPool(ctx, req, startTime)
-	}
-
-	// Single connection mode
-	// Create IPC request
-	ipcReq := &ipc.StartReadRequest{
-		VolumeID:    req.VolumeID,
-		NeedleID:    req.NeedleID,
-		Cookie:      req.Cookie,
-		Offset:      req.Offset,
-		Size:        req.Size,
-		RemoteAddr:  0, // Will be set by engine (mock for now)
-		RemoteKey:   0, // Will be set by engine (mock for now)
-		TimeoutSecs: uint64(c.defaultTimeout.Seconds()),
-		AuthToken:   req.AuthToken,
-	}
-
-	// Start RDMA read
-	startResp, err := c.ipcClient.StartRead(ctx, ipcReq)
-	if err != nil {
-		c.logger.WithError(err).Error("❌ Failed to start RDMA read")
-		return nil, fmt.Errorf("failed to start RDMA read: %w", err)
-	}
-
-	// In the new protocol, if we got a StartReadResponse, the operation was successful
-
-	c.logger.WithFields(logrus.Fields{
-		"session_id":    startResp.SessionID,
-		"local_addr":    fmt.Sprintf("0x%x", startResp.LocalAddr),
-		"local_key":     startResp.LocalKey,
-		"transfer_size": startResp.TransferSize,
-		"expected_crc":  fmt.Sprintf("0x%x", startResp.ExpectedCrc),
-		"expires_at":    time.Unix(0, int64(startResp.ExpiresAtNs)).Format(time.RFC3339),
-	}).Debug("📖 RDMA read session started")
-
-	// Complete the RDMA read
-	completeResp, err := c.ipcClient.CompleteRead(ctx, startResp.SessionID, true, startResp.TransferSize, &startResp.ExpectedCrc)
-	if err != nil {
-		c.logger.WithError(err).Error("❌ Failed to complete RDMA read")
-		return nil, fmt.Errorf("failed to complete RDMA read: %w", err)
-	}
-
-	duration := time.Since(startTime)
-
-	if !completeResp.Success {
-		errorMsg := "unknown error"
-		if completeResp.Message != nil {
-			errorMsg = *completeResp.Message
-		}
-		c.logger.WithFields(logrus.Fields{
-			"session_id":    startResp.SessionID,
-			"error_message": errorMsg,
-		}).Error("❌ RDMA read completion failed")
-		return nil, fmt.Errorf("RDMA read completion failed: %s", errorMsg)
-	}
-
-	// Calculate transfer rate (bytes/second)
-	transferRate := float64(startResp.TransferSize) / duration.Seconds()
-
-	c.logger.WithFields(logrus.Fields{
-		"session_id":    startResp.SessionID,
-		"bytes_read":    startResp.TransferSize,
-		"duration":      duration,
-		"transfer_rate": transferRate,
-		"server_crc":    completeResp.ServerCrc,
-	}).Info("✅ RDMA read completed successfully")
-
-	// MOCK DATA IMPLEMENTATION - FOR DEVELOPMENT/TESTING ONLY
-	//
-	// This section generates placeholder data for the mock RDMA implementation.
-	// In a production RDMA implementation, this should be replaced with:
-	//
-	// 1. The actual data transferred via RDMA from the remote memory region
-	// 2. Data validation using checksums/CRC from the RDMA completion
-	// 3. Proper error handling for RDMA transfer failures
-	// 4. Memory region cleanup and deregistration
-	//
-	// TODO for real RDMA implementation:
-	// - Replace mockData with actual RDMA buffer contents
-	// - Validate data integrity using server CRC: completeResp.ServerCrc
-	// - Handle partial transfers and retry logic
-	// - Implement proper memory management for RDMA regions
-	//
-	// Current mock behavior: Generates a simple pattern (0,1,2...255,0,1,2...)
-	// This allows testing of the integration pipeline without real hardware
-	mockData := make([]byte, startResp.TransferSize)
-	for i := range mockData {
-		mockData[i] = byte(i % 256) // Simple repeating pattern for verification
-	}
-	// END MOCK DATA IMPLEMENTATION
-
-	return &ReadResponse{
-		Data:         mockData,
-		BytesRead:    startResp.TransferSize,
-		Duration:     duration,
-		TransferRate: transferRate,
-		SessionID:    startResp.SessionID,
-		Success:      true,
-		Message:      "RDMA read completed successfully",
-	}, nil
-}
-
-// ReadRange performs an RDMA read for a specific range within a needle
-func (c *Client) ReadRange(ctx context.Context, volumeID uint32, needleID uint64, cookie uint32, offset, size uint64) (*ReadResponse, error) {
-	req := &ReadRequest{
-		VolumeID: volumeID,
-		NeedleID: needleID,
-		Cookie:   cookie,
-		Offset:   offset,
-		Size:     size,
-	}
-	return c.Read(ctx, req)
-}
-
-// ReadFileRange performs an RDMA read using SeaweedFS file ID format
-func (c *Client) ReadFileRange(ctx context.Context, fileID string, offset, size uint64) (*ReadResponse, error) {
-	// Parse file ID (e.g., "3,01637037d6" -> volume=3, needle=0x01637037d6, cookie extracted)
-	volumeID, needleID, cookie, err := parseFileID(fileID)
-	if err != nil {
-		return nil, fmt.Errorf("invalid file ID %s: %w", fileID, err)
-	}
-
-	req := &ReadRequest{
-		VolumeID: volumeID,
-		NeedleID: needleID,
-		Cookie:   cookie,
-		Offset:   offset,
-		Size:     size,
-	}
-	return c.Read(ctx, req)
-}
-
-// parseFileID extracts volume ID, needle ID, and cookie from a SeaweedFS file ID
-// Uses existing SeaweedFS parsing logic to ensure compatibility
-func parseFileID(fileId string) (volumeID uint32, needleID uint64, cookie uint32, err error) {
-	// Use existing SeaweedFS file ID parsing
-	fid, err := needle.ParseFileIdFromString(fileId)
-	if err != nil {
-		return 0, 0, 0, fmt.Errorf("failed to parse file ID %s: %w", fileId, err)
-	}
-
-	volumeID = uint32(fid.VolumeId)
-	needleID = uint64(fid.Key)
-	cookie = uint32(fid.Cookie)
-
-	return volumeID, needleID, cookie, nil
-}
-
-// ReadFull performs an RDMA read for an entire needle
-func (c *Client) ReadFull(ctx context.Context, volumeID uint32, needleID uint64, cookie uint32) (*ReadResponse, error) {
-	req := &ReadRequest{
-		VolumeID: volumeID,
-		NeedleID: needleID,
-		Cookie:   cookie,
-		Offset:   0,
-		Size:     0, // 0 means read entire needle
-	}
-	return c.Read(ctx, req)
-}
-
-// Ping tests connectivity to the RDMA engine
-func (c *Client) Ping(ctx context.Context) (time.Duration, error) {
-	if !c.IsConnected() {
-		return 0, fmt.Errorf("not connected to RDMA engine")
-	}
-
-	clientID := "health-check"
-	start := time.Now()
-	pong, err := c.ipcClient.Ping(ctx, &clientID)
-	if err != nil {
-		return 0, err
-	}
-
-	totalLatency := time.Since(start)
-	serverRtt := time.Duration(pong.ServerRttNs)
-
-	c.logger.WithFields(logrus.Fields{
-		"total_latency": totalLatency,
-		"server_rtt":    serverRtt,
-		"client_id":     clientID,
-	}).Debug("🏓 RDMA engine ping successful")
-
-	return totalLatency, nil
-}
-
-// readWithPool performs RDMA read using connection pooling
-func (c *Client) readWithPool(ctx context.Context, req *ReadRequest, startTime time.Time) (*ReadResponse, error) {
-	// Get connection from pool
-	conn, err := c.pool.getConnection(ctx)
-	if err != nil {
-		return nil, fmt.Errorf("failed to get pooled connection: %w", err)
-	}
-	defer c.pool.releaseConnection(conn)
-
-	c.logger.WithField("session_id", conn.sessionID).Debug("🔌 Using pooled RDMA connection")
-
-	// Create IPC request
-	ipcReq := &ipc.StartReadRequest{
-		VolumeID:    req.VolumeID,
-		NeedleID:    req.NeedleID,
-		Cookie:      req.Cookie,
-		Offset:      req.Offset,
-		Size:        req.Size,
-		RemoteAddr:  0, // Will be set by engine (mock for now)
-		RemoteKey:   0, // Will be set by engine (mock for now)
-		TimeoutSecs: uint64(c.defaultTimeout.Seconds()),
-		AuthToken:   req.AuthToken,
-	}
-
-	// Start RDMA read
-	startResp, err := conn.ipcClient.StartRead(ctx, ipcReq)
-	if err != nil {
-		c.logger.WithError(err).Error("❌ Failed to start RDMA read (pooled)")
-		return nil, fmt.Errorf("failed to start RDMA read: %w", err)
-	}
-
-	c.logger.WithFields(logrus.Fields{
-		"session_id":    startResp.SessionID,
-		"local_addr":    fmt.Sprintf("0x%x", startResp.LocalAddr),
-		"local_key":     startResp.LocalKey,
-		"transfer_size": startResp.TransferSize,
-		"expected_crc":  fmt.Sprintf("0x%x", startResp.ExpectedCrc),
-		"expires_at":    time.Unix(0, int64(startResp.ExpiresAtNs)).Format(time.RFC3339),
-		"pooled":        true,
-	}).Debug("📖 RDMA read session started (pooled)")
-
-	// Complete the RDMA read
-	completeResp, err := conn.ipcClient.CompleteRead(ctx, startResp.SessionID, true, startResp.TransferSize, &startResp.ExpectedCrc)
-	if err != nil {
-		c.logger.WithError(err).Error("❌ Failed to complete RDMA read (pooled)")
-		return nil, fmt.Errorf("failed to complete RDMA read: %w", err)
-	}
-
-	duration := time.Since(startTime)
-
-	if !completeResp.Success {
-		errorMsg := "unknown error"
-		if completeResp.Message != nil {
-			errorMsg = *completeResp.Message
-		}
-		c.logger.WithFields(logrus.Fields{
-			"session_id":    conn.sessionID,
-			"error_message": errorMsg,
-			"pooled":        true,
-		}).Error("❌ RDMA read completion failed (pooled)")
-		return nil, fmt.Errorf("RDMA read completion failed: %s", errorMsg)
-	}
-
-	// Calculate transfer rate (bytes/second)
-	transferRate := float64(startResp.TransferSize) / duration.Seconds()
-
-	c.logger.WithFields(logrus.Fields{
-		"session_id":    conn.sessionID,
-		"bytes_read":    startResp.TransferSize,
-		"duration":      duration,
-		"transfer_rate": transferRate,
-		"server_crc":    completeResp.ServerCrc,
-		"pooled":        true,
-	}).Info("✅ RDMA read completed successfully (pooled)")
-
-	// For the mock implementation, we'll return placeholder data
-	// In the real implementation, this would be the actual RDMA transferred data
-	mockData := make([]byte, startResp.TransferSize)
-	for i := range mockData {
-		mockData[i] = byte(i % 256) // Simple pattern for testing
-	}
-
-	return &ReadResponse{
-		Data:         mockData,
-		BytesRead:    startResp.TransferSize,
-		Duration:     duration,
-		TransferRate: transferRate,
-		SessionID:    conn.sessionID,
-		Success:      true,
-		Message:      "RDMA read successful (pooled)",
-	}, nil
-}