s3-gateway/docs/multi-version-pipeline.md

Multi-version build pipeline

How s3-gateway turns an upstream versitygw release into a de-branded build, and how to keep it tracking new upstream versions. This mirrors the scoutfs-build / scoutfs-notify model, collapsed into one repo because the patch set is small.

The flow

Each build is four steps, run per upstream tag:

1. ci/clone-source.sh — shallow-clones upstream versitygw at a tag into src/. Tag selection priority:

   1. VGW_TAG (set by the CI matrix or a manual run),
   2. the pushed tag (GITHUB_REF_NAME when GITHUB_REF_TYPE=tag),
   3. the latest remote vX.Y.Z tag. The checkout forces core.autocrlf=false so the tree is byte-for- byte upstream (LF) and the LF patches apply on any host.

2. ci/apply-patches.sh — resolves the patch directory for the tag from ci/support-matrix.yaml (patches: field, default = tag name) and applies patches/<dir>/*.patch with git am --whitespace=nowarn --3way, in filename order. The --3way fallback is what lets a series rebase across minor upstream drift.

3. ci/build.sh — runs goreleaser release --clean --skip=validate,publish in src/, producing binaries + deb/rpm in src/dist/. Because the patch commits sit on top of the tag, HEAD is no longer exactly the tag, so we set GORELEASER_CURRENT_TAG to pin the stamped version and --skip=validate to bypass the dirty/tag checks. Upstream's .goreleaser.yaml is used unmodified — artifact names stay versitygw.

4. ci/publish-release.sh — creates (or reuses) a Gitea release for the tag in this repo and uploads src/dist/* as assets. Idempotent: re-running replaces existing assets, so daily rebuilds are safe.

The source of truth: ci/support-matrix.yaml

versitygw:
  - tag: v1.5.0
    patches: v1.5.0

• tag — upstream git tag to build.
• patches — directory under patches/ to apply (defaults to tag). Point several tags at one directory if the same series applies to a range of upstream releases.

The highest-version entry is "latest". bash ci/matrix.sh prints the matrix this file implies.

Act Runner caveat. Gitea's Act Runner does not support dynamic matrices (matrix: ${{ fromJSON(...) }}). The workflow therefore carries a static matrix: that you must keep in sync with this file by hand — same compromise scoutfs-build makes. ci/matrix.sh exists to diff/preview, not to feed CI.

Adding a new upstream version

When upstream releases vX.Y.Z:

1. Rebase the patch series onto the new tag in a throwaway clone:

   git clone https://github.com/versity/versitygw vgw && cd vgw
   git -c core.autocrlf=false checkout -b debrand vX.Y.Z
   git config user.email you@example.com && git config user.name you
   git am --3way /path/to/s3-gateway/patches/<previous-tag>/*.patch
   # If a hunk conflicts: fix files, `git add -A`, `git am --continue`.
   mkdir -p /path/to/s3-gateway/patches/vX.Y.Z
   git format-patch --binary vX.Y.Z..HEAD \
     -o /path/to/s3-gateway/patches/vX.Y.Z
   

   --binary is required because the series deletes PNG logo assets.

2. Register it in ci/support-matrix.yaml:

   versitygw:
     - tag: vX.Y.Z
       patches: vX.Y.Z
     - tag: v1.5.0
       patches: v1.5.0
   

3. Mirror it into the static matrix.include in .gitea/workflows/build.yml.

4. Commit and push. The next CI run builds and publishes the new version alongside the existing ones.

Sanity-check a series before committing

VGW_REPO_URL="/path/to/local/versitygw" VGW_TAG=vX.Y.Z bash ci/run.sh

A clean git am plus a populated src/dist/ means the series is good.

Aging out a version

Remove its entry from ci/support-matrix.yaml and the workflow matrix. CI stops building it; releases already published in this repo stay until you delete them manually. You can keep the patches/<tag>/ directory for history or remove it.

CI configuration

• Runner: runs-on: s3-gateway-ci. Add that label to your existing scoutfs-ci Act Runner, or change the label.
• Container: golang:latest — pure-Go build, GoReleaser's bundled nfpm makes deb/rpm without rpmbuild/dpkg.
• Secret TOKEN_GITEA: Gitea PAT with write:repository. Used both to clone this repo (the workflow does a manual token-rewritten clone, mirroring scoutfs-build, since a stock container has no Node for actions/checkout) and to publish releases.
• Variable VGW_REPO_URL (optional): override the upstream URL, e.g. to build from a mirror.

Release tagging

Releases are created in this repo with the upstream tag name (v1.5.0). If you need to ship a rebuild of the same upstream version (e.g. a patch fix), set RELEASE_TAG in ci/publish-release.sh's environment to something like v1.5.0-acg2 to avoid clobbering.