Build the Rust worker against the protoc that ships with the build (#10881)

* worker: compile plugin.proto with the protoc that ships with the build

seaweed-volume already does this: protoc-bin-vendored carries the binary, so
the build needs no package manager and every build sees the same version. An
explicit PROTOC still wins, which is what lets the lance crates - whose own
build scripts read the same variable - share it.

Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm

* ci: point the worker builds at the vendored protoc

The jobs installed protobuf-compiler for lance's build scripts. They read
PROTOC, so pointing it at the binary protoc-bin-vendored already puts in the
registry serves them without a system package - one less apt call on the way
to a release, and the same protoc a developer's build uses.

Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm

* docs: say what the worker build needs from protoc

The lance crates' build scripts are the ones that need it, not ours, and they
take the same vendored binary.

Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm

.github/workflows/container_dev.yml

@@ -49,11 +49,6 @@ jobs:
           # Disable glibc fortify source — its __memcpy_chk etc. symbols don't exist in musl
           echo "CFLAGS_aarch64_unknown_linux_musl=-U_FORTIFY_SOURCE" >> "$GITHUB_ENV"
 
-      # lance's build scripts compile their own protos, and unlike seaweed-volume
-      # they do not vendor a protoc to do it with.
-      - name: Install protoc
-        run: sudo apt-get install -y protobuf-compiler
-
       - name: Cache cargo registry and target
         uses: actions/cache@v6
         with:
@@ -66,6 +61,21 @@ jobs:
           restore-keys: |
             rust-docker-dev-${{ matrix.target }}-
 
+      # lance's build scripts compile their own protos and look for a protoc.
+      # Point them at the one protoc-bin-vendored ships, which seaweed-worker's
+      # own build already uses, so no job depends on a system package and every
+      # build sees the same version.
+      - name: Use the vendored protoc
+        run: |
+          cd seaweed-worker
+          cargo fetch
+          # The version from the lock, not whatever else a restored cache holds.
+          version=$(awk '/^name = "protoc-bin-vendored-linux-x86_64"$/{found=1; next} found && /^version = /{gsub(/"/,"",$3); print $3; exit}' Cargo.lock)
+          test -n "$version" || { echo "protoc-bin-vendored-linux-x86_64 is not in Cargo.lock" >&2; exit 1; }
+          protoc=$(find ~/.cargo/registry/src -path "*protoc-bin-vendored-linux-x86_64-$version/bin/protoc" | head -1)
+          test -x "$protoc" || { echo "no vendored protoc $version in the registry" >&2; exit 1; }
+          echo "PROTOC=$protoc" >> "$GITHUB_ENV"
+
       - name: Build normal variant
         env:
           SEAWEEDFS_COMMIT: ${{ github.sha }}

.github/workflows/container_latest.yml

@@ -93,11 +93,6 @@ jobs:
           # Disable glibc fortify source — its __memcpy_chk etc. symbols don't exist in musl
           echo "CFLAGS_aarch64_unknown_linux_musl=-U_FORTIFY_SOURCE" >> "$GITHUB_ENV"
 
-      # lance's build scripts compile their own protos, and unlike seaweed-volume
-      # they do not vendor a protoc to do it with.
-      - name: Install protoc
-        run: sudo apt-get install -y protobuf-compiler
-
       - name: Cache cargo registry and target
         uses: actions/cache@v6
         with:
@@ -110,6 +105,21 @@ jobs:
           restore-keys: |
             rust-docker-${{ matrix.target }}-
 
+      # lance's build scripts compile their own protos and look for a protoc.
+      # Point them at the one protoc-bin-vendored ships, which seaweed-worker's
+      # own build already uses, so no job depends on a system package and every
+      # build sees the same version.
+      - name: Use the vendored protoc
+        run: |
+          cd seaweed-worker
+          cargo fetch
+          # The version from the lock, not whatever else a restored cache holds.
+          version=$(awk '/^name = "protoc-bin-vendored-linux-x86_64"$/{found=1; next} found && /^version = /{gsub(/"/,"",$3); print $3; exit}' Cargo.lock)
+          test -n "$version" || { echo "protoc-bin-vendored-linux-x86_64 is not in Cargo.lock" >&2; exit 1; }
+          protoc=$(find ~/.cargo/registry/src -path "*protoc-bin-vendored-linux-x86_64-$version/bin/protoc" | head -1)
+          test -x "$protoc" || { echo "no vendored protoc $version in the registry" >&2; exit 1; }
+          echo "PROTOC=$protoc" >> "$GITHUB_ENV"
+
       - name: Build large-disk variant
         env:
           SEAWEEDFS_COMMIT: ${{ github.sha }}

.github/workflows/container_release_unified.yml

@@ -77,11 +77,6 @@ jobs:
           # Disable glibc fortify source — its __memcpy_chk etc. symbols don't exist in musl
           echo "CFLAGS_aarch64_unknown_linux_musl=-U_FORTIFY_SOURCE" >> "$GITHUB_ENV"
 
-      # lance's build scripts compile their own protos, and unlike seaweed-volume
-      # they do not vendor a protoc to do it with.
-      - name: Install protoc
-        run: sudo apt-get install -y protobuf-compiler
-
       - name: Cache cargo registry and target
         uses: actions/cache@v6
         with:
@@ -94,6 +89,21 @@ jobs:
           restore-keys: |
             rust-docker-${{ matrix.target }}-
 
+      # lance's build scripts compile their own protos and look for a protoc.
+      # Point them at the one protoc-bin-vendored ships, which seaweed-worker's
+      # own build already uses, so no job depends on a system package and every
+      # build sees the same version.
+      - name: Use the vendored protoc
+        run: |
+          cd seaweed-worker
+          cargo fetch
+          # The version from the lock, not whatever else a restored cache holds.
+          version=$(awk '/^name = "protoc-bin-vendored-linux-x86_64"$/{found=1; next} found && /^version = /{gsub(/"/,"",$3); print $3; exit}' Cargo.lock)
+          test -n "$version" || { echo "protoc-bin-vendored-linux-x86_64 is not in Cargo.lock" >&2; exit 1; }
+          protoc=$(find ~/.cargo/registry/src -path "*protoc-bin-vendored-linux-x86_64-$version/bin/protoc" | head -1)
+          test -x "$protoc" || { echo "no vendored protoc $version in the registry" >&2; exit 1; }
+          echo "PROTOC=$protoc" >> "$GITHUB_ENV"
+
       - name: Build large-disk variant
         env:
           SEAWEEDFS_COMMIT: ${{ github.sha }}

.github/workflows/rust-worker-tests.yml

@@ -36,11 +36,6 @@ jobs:
       - name: Install Rust toolchain
         uses: dtolnay/rust-toolchain@stable
 
-      # lance's build scripts compile their own protos, and unlike seaweed-volume
-      # they do not vendor a protoc to do it with.
-      - name: Install protoc
-        run: sudo apt-get install -y protobuf-compiler
-
       # cargo tracks its own inputs but not the runner's C toolchain, so a cached
       # target/ can carry C objects built against a different glibc than we link against.
       - name: Fingerprint build toolchain
@@ -58,6 +53,21 @@ jobs:
           restore-keys: |
             rust-worker-${{ steps.toolchain.outputs.fingerprint }}-
 
+      # lance's build scripts compile their own protos and look for a protoc.
+      # Point them at the one protoc-bin-vendored ships, which seaweed-worker's
+      # own build already uses, so no job depends on a system package and every
+      # build sees the same version.
+      - name: Use the vendored protoc
+        run: |
+          cd seaweed-worker
+          cargo fetch
+          # The version from the lock, not whatever else a restored cache holds.
+          version=$(awk '/^name = "protoc-bin-vendored-linux-x86_64"$/{found=1; next} found && /^version = /{gsub(/"/,"",$3); print $3; exit}' Cargo.lock)
+          test -n "$version" || { echo "protoc-bin-vendored-linux-x86_64 is not in Cargo.lock" >&2; exit 1; }
+          protoc=$(find ~/.cargo/registry/src -path "*protoc-bin-vendored-linux-x86_64-$version/bin/protoc" | head -1)
+          test -x "$protoc" || { echo "no vendored protoc $version in the registry" >&2; exit 1; }
+          echo "PROTOC=$protoc" >> "$GITHUB_ENV"
+
       # The release profile is what ships, and it is where the release and the
       # container builds would otherwise discover a break for the first time.
       - name: Build the plugin workers

.github/workflows/rust_binaries_release.yml

@@ -152,11 +152,6 @@ jobs:
           sudo apt-get install -y gcc-aarch64-linux-gnu
           echo "CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc" >> "$GITHUB_ENV"
 
-      # lance's build scripts compile their own protos, and unlike seaweed-volume
-      # they do not vendor a protoc to do it with.
-      - name: Install protoc
-        run: sudo apt-get install -y protobuf-compiler
-
       - name: Cache cargo registry and target
         uses: actions/cache@v6
         with:
@@ -168,6 +163,21 @@ jobs:
           restore-keys: |
             rust-worker-release-${{ matrix.target }}-
 
+      # lance's build scripts compile their own protos and look for a protoc.
+      # Point them at the one protoc-bin-vendored ships, which seaweed-worker's
+      # own build already uses, so no job depends on a system package and every
+      # build sees the same version.
+      - name: Use the vendored protoc
+        run: |
+          cd seaweed-worker
+          cargo fetch
+          # The version from the lock, not whatever else a restored cache holds.
+          version=$(awk '/^name = "protoc-bin-vendored-linux-x86_64"$/{found=1; next} found && /^version = /{gsub(/"/,"",$3); print $3; exit}' Cargo.lock)
+          test -n "$version" || { echo "protoc-bin-vendored-linux-x86_64 is not in Cargo.lock" >&2; exit 1; }
+          protoc=$(find ~/.cargo/registry/src -path "*protoc-bin-vendored-linux-x86_64-$version/bin/protoc" | head -1)
+          test -x "$protoc" || { echo "no vendored protoc $version in the registry" >&2; exit 1; }
+          echo "PROTOC=$protoc" >> "$GITHUB_ENV"
+
       - name: Build the Rust maintenance worker
         run: |
           cd seaweed-worker