alphacentri/s3-gateway
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

Initial s3-gateway de-brand build pipeline

Browse Source 
ScoutFS-Build-style pipeline that pulls upstream versitygw release tags, applies the web-UI de-brand patch series, and builds GoReleaser releases (binaries + deb/rpm) for publishing to Gitea. Artifact names stay versitygw; only the embedded admin UI is rebranded to S3 Gateway.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
William Gill
2026-06-15 13:20:15 -05:00
co-authored by Claude Opus 4.8
commit 6b35d863c9
13 changed files with 3704 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitattributes
+9
View File
@@ -0,0 +1,9 @@
# Authored on Windows, built on a Linux CI runner. Force LF on every
# text file so shell scripts (#!/bin/bash) and the .patch series work
# on the runner regardless of the committer's core.autocrlf setting.
* text=auto eol=lf
# Safety net for any binary assets added later.
*.png binary
*.woff2 binary
*.ico binary
.gitea/workflows/build.yml
+100
View File
@@ -0,0 +1,100 @@
name: build
on:
push:
tags:
- 'v*'
schedule:
# Daily — rebuilds the supported matrix (cheap; publish is idempotent).
- cron: '0 6 * * *'
workflow_dispatch:
jobs:
build:
# Self-hosted Gitea Act Runner. Reuse your scoutfs-ci runner by
# adding an `s3-gateway-ci` label to it, or change this to a label
# your runner already advertises.
runs-on: s3-gateway-ci
# Deterministic build env: upstream is pure Go (CGO disabled), and
# GoReleaser builds deb/rpm via its bundled nfpm — no rpmbuild/dpkg
# needed — so a stock golang image is enough.
container:
image: golang:latest
options: --user 0:0
strategy:
fail-fast: false
# STATIC matrix — mirror ci/support-matrix.yaml by hand. Gitea
# Act Runner does not support `matrix: ${{ fromJSON(...) }}`
# (dynamic matrices): the parser bails and downstream jobs see an
# empty matrix context. When you edit one, edit the other.
# `bash ci/matrix.sh` prints what this should contain.
matrix:
include:
- tag: v1.5.0
patches: v1.5.0
steps:
- name: Checkout CI scripts + patches
env:
# Stored Gitea PAT. read:repository is enough to clone this
# repo; the publish step below needs write:repository.
GITEA_TOKEN: ${{ secrets.TOKEN_GITEA }}
run: |
set -eu
apt-get update -y >/dev/null
apt-get install -y --no-install-recommends git curl jq ca-certificates >/dev/null
if [ -z "${GITEA_TOKEN:-}" ]; then
echo "ERROR: secrets.TOKEN_GITEA is empty or not visible to this run." >&2
echo "Create it under: Repo -> Settings -> Actions -> Secrets" >&2
exit 1
fi
SERVER_HOST="${GITHUB_SERVER_URL#http://}"; SERVER_HOST="${SERVER_HOST#https://}"
SCHEME="${GITHUB_SERVER_URL%%://*}"
git config --global \
url."${SCHEME}://x-access-token:${GITEA_TOKEN}@${SERVER_HOST}/".insteadOf \
"${SCHEME}://${SERVER_HOST}/"
git clone --depth 1 --branch "${GITHUB_REF_NAME:-main}" \
"${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git" /tmp/ci-repo
cp -a /tmp/ci-repo/ci /tmp/ci-repo/patches .
chmod +x ci/*.sh
- name: Install GoReleaser
run: |
set -eu
curl -fsSL https://github.com/goreleaser/goreleaser/releases/latest/download/goreleaser_Linux_x86_64.tar.gz \
| tar -xz -C /usr/local/bin goreleaser
goreleaser --version | head -1
- name: Clone upstream versitygw
env:
VGW_REPO_URL: ${{ vars.VGW_REPO_URL || 'https://github.com/versity/versitygw.git' }}
VGW_TAG: ${{ matrix.tag }}
run: bash ci/clone-source.sh
- name: Apply de-brand patches
env:
VGW_TAG: ${{ matrix.tag }}
run: bash ci/apply-patches.sh
- name: Build release artifacts
env:
VGW_TAG: ${{ matrix.tag }}
run: bash ci/build.sh
- name: Publish to Gitea release
env:
VGW_TAG: ${{ matrix.tag }}
GITEA_API_URL: ${{ github.server_url }}/api/v1
GITEA_OWNER: ${{ github.repository_owner }}
GITEA_REPO: ${{ github.event.repository.name }}
# Needs write:repository scope to create releases + upload assets.
GITEA_TOKEN: ${{ secrets.TOKEN_GITEA }}
run: bash ci/publish-release.sh
- name: Summary
if: always()
run: |
echo "=== ${{ matrix.tag }} ==="
ls -1 src/dist/*.tar.gz src/dist/*.deb src/dist/*.rpm 2>/dev/null || true
.gitignore
+11
View File
@@ -0,0 +1,11 @@
# Cloned upstream source trees
/src/
/*-src/
# GoReleaser output
/dist/
src/dist/
# Local scratch
/tmp/
*.log
README.md
+130
View File
@@ -0,0 +1,130 @@
# s3-gateway
Downstream build pipeline that produces **de-branded
[versitygw](https://github.com/versity/versitygw) releases** — the
upstream Versity S3 Gateway with Versity trademarks and logos removed
from the admin web UI.
Same pattern as [`scoutfs-build`](../scoutfs-build): pull an upstream
release **tag**, apply our **patch series** scoped to that tag, build
with upstream's GoReleaser, and publish to **Gitea Releases**.
Artifact names are intentionally **unchanged** — the binary stays
`versitygw` and the deb/rpm packages stay `versitygw`, so builds are
drop-in compatible with existing configs, systemd units, and the
`versitygw` CLI. Only the embedded web UI is rebranded to "S3 Gateway".
```
upstream tag (vX.Y.Z) this repo
github.com/versity/versitygw patches/<tag>/*.patch
│ │
▼ ▼
ci/clone-source.sh ───► ci/apply-patches.sh ───► ci/build.sh ───► ci/publish-release.sh
(shallow clone tag) (git am --3way) (goreleaser) (Gitea release assets)
```
`ci/support-matrix.yaml` is the single source of truth for which
upstream versions get built.
## Layout
```
s3-gateway/
├── ci/
│ ├── support-matrix.yaml # which upstream tags we build (source of truth)
│ ├── clone-source.sh # shallow-clone upstream versitygw at a tag
│ ├── apply-patches.sh # git am --3way patches/<tag>/*.patch
│ ├── build.sh # goreleaser release (binaries + deb/rpm)
│ ├── publish-release.sh # upload dist/* to a Gitea release
│ ├── matrix.sh # print the CI matrix from support-matrix.yaml (local/preview)
│ └── run.sh # local end-to-end: clone + patch + build one tag
├── patches/
│ └── v1.5.0/
│ └── 0001-webui-debrand.patch
├── .gitea/workflows/build.yml
└── docs/multi-version-pipeline.md
```
## Build locally
Prerequisites: `git`, `go`, and [`goreleaser`](https://goreleaser.com).
```bash
# Build one upstream tag end-to-end → artifacts in src/dist/
VGW_TAG=v1.5.0 bash ci/run.sh
# Fast offline build against a local upstream clone:
VGW_REPO_URL="/path/to/versitygw" VGW_TAG=v1.5.0 bash ci/run.sh
```
Or run the steps individually:
```bash
VGW_TAG=v1.5.0 bash ci/clone-source.sh # → src/
VGW_TAG=v1.5.0 bash ci/apply-patches.sh # git am the series onto src/
VGW_TAG=v1.5.0 bash ci/build.sh # goreleaser → src/dist/
```
`src/` and `src/dist/` are gitignored.
## Patches
Patch sets live in `patches/<dir>/` as `git format-patch` files and are
applied with `git am --3way`, in filename order. The directory for a
tag comes from the `patches:` field in `ci/support-matrix.yaml`
(defaults to the tag name).
Currently maintained:
| Upstream tag | Patch set | Contents |
|--------------|---------------|-----------------------------------|
| `v1.5.0` | `v1.5.0/` | `0001-webui-debrand.patch` |
### What the de-brand patch does
- Sidebar/login logos → a plain **"S3 Gateway"** wordmark.
- Page titles: "VersityGW Dashboard" → "Dashboard" (Buckets/Users/Explorer too).
- Browser tab titles → "S3 Gateway – …".
- Removes the sidebar **Resources** section (external `github.com/versity` links).
- Replaces the Versity favicon with a neutral SVG; deletes the Versity logo PNGs.
Deliberately **kept** (license compliance): the per-file Apache-2.0
headers, the `NOTICE` file, and the "© Versity Software Inc." login
footer. Upstream is Apache-2.0; these de-brand the *trademark*, not the
copyright.
## Adding / refreshing a version
When upstream cuts a new release you want a build of, you rebase the
patch series onto it and add a matrix row. See
[docs/multi-version-pipeline.md](docs/multi-version-pipeline.md) for the
full recipe. In short:
```bash
# Rebase the series onto the new tag, re-export, drop into patches/vX.Y.Z/
git clone https://github.com/versity/versitygw && cd versitygw
git checkout -b debrand vX.Y.Z
git am --3way /path/to/s3-gateway/patches/<prev>/*.patch # resolve any conflicts
git format-patch vX.Y.Z..HEAD -o /path/to/s3-gateway/patches/vX.Y.Z/
```
Then add the tag to **both** `ci/support-matrix.yaml` and the static
`matrix:` in `.gitea/workflows/build.yml` (the Gitea Act Runner can't
read a dynamic matrix — see the comment in the workflow).
## CI
`.gitea/workflows/build.yml` runs on tag push, daily, and manual
dispatch. It builds every entry in the static matrix and publishes each
to a Gitea release in this repo.
- **Runner**: targets the `s3-gateway-ci` label — add that label to your
existing `scoutfs-ci` runner, or edit `runs-on`.
- **Secret**: `TOKEN_GITEA` — a Gitea PAT with **write:repository**
(clones this repo + creates releases / uploads assets).
- **Variable (optional)**: `VGW_REPO_URL` to override the upstream URL.
## License
Apache-2.0, matching versitygw upstream. Upstream copyright and
`NOTICE` are retained in every build.
ci/apply-patches.sh
+62
View File
@@ -0,0 +1,62 @@
#!/bin/bash
# Apply the s3-gateway patch series for the target versitygw tag onto
# the checked-out source tree (run ci/clone-source.sh first).
#
# Patch sets live in patches/<dir>/. The directory for a given tag
# comes from ci/support-matrix.yaml (the `patches:` field, defaulting
# to the tag name). Patches are git-format-patch files applied with
# `git am --3way` in filename order — same model as scoutfs-notify.
#
# Environment:
# VGW_SRC_DIR source tree (default: src; set by clone-source.sh)
# VGW_TAG upstream tag being built (required)
# PATCH_DIR override the patch directory (optional)
set -euo pipefail
VGW_SRC_DIR="${VGW_SRC_DIR:-src}"
: "${VGW_TAG:?VGW_TAG must be set}"
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
SRC="$(cd "${VGW_SRC_DIR}" && pwd)"
# Resolve the patch directory: explicit override, else matrix lookup,
# else the tag name.
if [[ -z "${PATCH_DIR:-}" ]]; then
PATCH_NAME=""
if command -v yq >/dev/null 2>&1; then
PATCH_NAME="$(yq -r \
".versitygw[] | select(.tag == \"${VGW_TAG}\") | .patches // \"\"" \
"${REPO_ROOT}/ci/support-matrix.yaml" 2>/dev/null || true)"
fi
[[ -n "${PATCH_NAME}" && "${PATCH_NAME}" != "null" ]] || PATCH_NAME="${VGW_TAG}"
PATCH_DIR="${REPO_ROOT}/patches/${PATCH_NAME}"
fi
echo ">>> target versitygw tag: ${VGW_TAG}"
echo ">>> patch directory: ${PATCH_DIR}"
shopt -s nullglob
patches=( "${PATCH_DIR}"/*.patch )
shopt -u nullglob
if (( ${#patches[@]} == 0 )); then
echo ">>> WARNING: no patches in ${PATCH_DIR} — building vanilla upstream"
exit 0
fi
echo ">>> applying ${#patches[@]} patch(es) with git am --3way:"
printf ' %s\n' "${patches[@]##*/}"
cd "${SRC}"
git am --whitespace=nowarn --3way "${patches[@]}"
echo ">>> applied commits:"
git log --oneline "${VGW_TAG}..HEAD"
if [[ -n "${GITHUB_ENV:-}" ]]; then
{
echo "S3GW_PATCH_DIR=${PATCH_DIR}"
echo "S3GW_PATCH_COUNT=${#patches[@]}"
} >> "${GITHUB_ENV}"
fi
echo ">>> patches applied"
ci/build.sh
+42
View File
@@ -0,0 +1,42 @@
#!/bin/bash
# Build de-branded versitygw release artifacts from the patched source
# tree using upstream's GoReleaser config. Artifacts land in
# <src>/dist/. Run ci/clone-source.sh and ci/apply-patches.sh first.
#
# We keep upstream's artifact names: the binary stays `versitygw` and
# the deb/rpm packages stay `versitygw` — only the embedded web UI is
# de-branded. So upstream's .goreleaser.yaml is used unmodified.
#
# Environment:
# VGW_SRC_DIR patched source tree (default: src)
# VGW_TAG upstream tag -> release version (required)
# GORELEASER goreleaser binary (default: goreleaser)
set -euo pipefail
VGW_SRC_DIR="${VGW_SRC_DIR:-src}"
: "${VGW_TAG:?VGW_TAG must be set}"
GORELEASER="${GORELEASER:-goreleaser}"
SRC="$(cd "${VGW_SRC_DIR}" && pwd)"
cd "${SRC}"
if ! command -v "${GORELEASER}" >/dev/null 2>&1; then
echo "ERROR: '${GORELEASER}' not found in PATH." >&2
echo " Install from https://goreleaser.com or set \$GORELEASER." >&2
exit 1
fi
# The tree carries downstream patch commits on top of the tag, so HEAD
# is no longer exactly the tag. Pin the version GoReleaser stamps and
# skip the clean-tree / tag validations.
export GORELEASER_CURRENT_TAG="${VGW_TAG}"
echo ">>> building ${VGW_TAG} with $("${GORELEASER}" --version | head -1)..."
"${GORELEASER}" release --clean --skip=validate,publish
echo ">>> artifacts in ${SRC}/dist:"
ls -1 dist/*.tar.gz dist/*.zip dist/*.deb dist/*.rpm dist/checksums.txt 2>/dev/null || ls -1 dist/
if [[ -n "${GITHUB_ENV:-}" ]]; then
echo "VGW_DIST_DIR=${SRC}/dist" >> "${GITHUB_ENV}"
fi
ci/clone-source.sh
+63
View File
@@ -0,0 +1,63 @@
#!/bin/bash
# Clone a versitygw release tag into a local source tree, ready for
# patch application. Mirrors the scoutfs-build clone-source.sh tag
# selection logic.
#
# Environment:
# VGW_REPO_URL upstream git URL (default github.com/versity/versitygw)
# VGW_TAG explicit tag to build (set by the CI matrix from
# ci/support-matrix.yaml). Highest priority.
# VGW_SRC_DIR where to put the checkout (default: src)
#
# Tag selection priority:
# 1. VGW_TAG env (matrix / manual single-version run)
# 2. GITHUB_REF_NAME when this run was triggered by a tag push
# 3. latest remote vX.Y.Z tag
#
# Exports VGW_VERSION / VGW_TAG / VGW_SRC_DIR to GITHUB_ENV when run
# under Gitea Actions.
set -euo pipefail
VGW_REPO_URL="${VGW_REPO_URL:-https://github.com/versity/versitygw.git}"
VGW_SRC_DIR="${VGW_SRC_DIR:-src}"
if [[ -n "${VGW_TAG:-}" ]]; then
TAG="${VGW_TAG}"
echo ">>> Using VGW_TAG from env: ${TAG}"
elif [[ -n "${GITHUB_REF_NAME:-}" && "${GITHUB_REF_TYPE:-}" == "tag" ]]; then
TAG="${GITHUB_REF_NAME}"
echo ">>> Using tag from push event: ${TAG}"
else
echo ">>> Discovering latest release tag from ${VGW_REPO_URL}..."
TAG=$(git ls-remote --tags --sort=-v:refname "${VGW_REPO_URL}" 'refs/tags/v*' \
| grep -v '\^{}' \
| head -1 \
| sed 's|.*refs/tags/||')
[[ -n "$TAG" ]] || { echo "ERROR: no release tags found in ${VGW_REPO_URL}" >&2; exit 1; }
echo ">>> Latest release tag: ${TAG}"
fi
echo ">>> Cloning ${VGW_REPO_URL} at ${TAG}..."
rm -rf "${VGW_SRC_DIR}"
# autocrlf=false keeps the checkout byte-identical to upstream (LF) so
# the LF patches in patches/ apply cleanly regardless of host platform.
git -c core.autocrlf=false clone --depth 1 --branch "${TAG}" \
"${VGW_REPO_URL}" "${VGW_SRC_DIR}"
# git am needs a clean worktree, LF line endings, and a committer
# identity inside the source tree. CI runners don't set these.
git -C "${VGW_SRC_DIR}" config core.autocrlf false
git -C "${VGW_SRC_DIR}" config user.email "ci@s3-gateway.local"
git -C "${VGW_SRC_DIR}" config user.name "s3-gateway CI"
VGW_VERSION="${TAG#v}"
echo ">>> versitygw version: ${VGW_VERSION}"
echo ">>> source directory: ${VGW_SRC_DIR}"
if [[ -n "${GITHUB_ENV:-}" ]]; then
{
echo "VGW_VERSION=${VGW_VERSION}"
echo "VGW_TAG=${TAG}"
echo "VGW_SRC_DIR=${VGW_SRC_DIR}"
} >> "${GITHUB_ENV}"
fi
ci/matrix.sh
+23
View File
@@ -0,0 +1,23 @@
#!/bin/bash
# Emit the CI build matrix as JSON from ci/support-matrix.yaml.
#
# If VGW_TAG is set (manual single-version run) the matrix collapses to
# that one tag. Otherwise every entry in support-matrix.yaml is built.
#
# Prints the matrix object to stdout and, under Gitea/GitHub Actions,
# writes `matrix=...` to GITHUB_OUTPUT for use with fromJSON().
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
MATRIX_FILE="${REPO_ROOT}/ci/support-matrix.yaml"
if [[ -n "${VGW_TAG:-}" ]]; then
json="$(jq -cn --arg tag "$VGW_TAG" '{include:[{tag:$tag, patches:$tag}]}')"
else
arr="$(yq -o=json '[.versitygw[] | {tag: .tag, patches: .patches}]' "${MATRIX_FILE}")"
json="$(echo "$arr" | jq -c '{include: [.[] | {tag, patches: (.patches // .tag)}]}')"
fi
echo "${json}"
[[ -n "${GITHUB_OUTPUT:-}" ]] && echo "matrix=${json}" >> "${GITHUB_OUTPUT}"
exit 0
ci/publish-release.sh
+64
View File
@@ -0,0 +1,64 @@
#!/bin/bash
# Publish built artifacts to a Gitea release in the s3-gateway repo.
# Idempotent: re-running for the same tag replaces existing assets.
#
# Environment:
# GITEA_API_URL e.g. https://git.anomalous.dev/api/v1 (required)
# GITEA_OWNER e.g. alphacentri (required)
# GITEA_REPO e.g. s3-gateway (required)
# GITEA_TOKEN API token with repo write scope (required)
# VGW_TAG upstream tag being published (required)
# VGW_DIST_DIR artifact dir (default: <VGW_SRC_DIR>/dist)
# RELEASE_TAG git tag for the release (default: VGW_TAG)
# RELEASE_TARGET target commitish for the tag (default: main)
# RELEASE_NOTE release body (default: auto)
set -euo pipefail
: "${GITEA_API_URL:?}"; : "${GITEA_OWNER:?}"; : "${GITEA_REPO:?}"; : "${GITEA_TOKEN:?}"
: "${VGW_TAG:?VGW_TAG must be set}"
DIST="${VGW_DIST_DIR:-${VGW_SRC_DIR:-src}/dist}"
RELEASE_TAG="${RELEASE_TAG:-${VGW_TAG}}"
RELEASE_TARGET="${RELEASE_TARGET:-main}"
RELEASE_NOTE="${RELEASE_NOTE:-De-branded versitygw ${VGW_TAG} build.}"
API="${GITEA_API_URL%/}/repos/${GITEA_OWNER}/${GITEA_REPO}"
auth=(-H "Authorization: token ${GITEA_TOKEN}")
echo ">>> ensuring release ${RELEASE_TAG} in ${GITEA_OWNER}/${GITEA_REPO}..."
rel_id="$(curl -fsSL "${auth[@]}" "${API}/releases/tags/${RELEASE_TAG}" 2>/dev/null \
| jq -r '.id // empty' || true)"
if [[ -z "${rel_id}" ]]; then
echo ">>> creating release ${RELEASE_TAG}"
rel_id="$(curl -fsSL "${auth[@]}" -H 'Content-Type: application/json' \
-X POST "${API}/releases" \
-d "$(jq -n --arg tag "$RELEASE_TAG" --arg tgt "$RELEASE_TARGET" \
--arg name "$RELEASE_TAG" --arg body "$RELEASE_NOTE" \
'{tag_name:$tag, target_commitish:$tgt, name:$name, body:$body, draft:false, prerelease:false}')" \
| jq -r '.id')"
else
echo ">>> release ${RELEASE_TAG} already exists (id ${rel_id})"
fi
shopt -s nullglob
assets=( "${DIST}"/*.tar.gz "${DIST}"/*.zip "${DIST}"/*.deb "${DIST}"/*.rpm "${DIST}"/checksums.txt )
shopt -u nullglob
(( ${#assets[@]} )) || { echo "ERROR: no artifacts in ${DIST}" >&2; exit 1; }
existing="$(curl -fsSL "${auth[@]}" "${API}/releases/${rel_id}/assets" || echo '[]')"
for f in "${assets[@]}"; do
name="$(basename "$f")"
old="$(echo "$existing" | jq -r --arg n "$name" '.[] | select(.name==$n) | .id' 2>/dev/null || true)"
if [[ -n "$old" ]]; then
echo ">>> replacing asset ${name}"
curl -fsSL "${auth[@]}" -X DELETE "${API}/releases/${rel_id}/assets/${old}" >/dev/null
else
echo ">>> uploading asset ${name}"
fi
curl -fsSL "${auth[@]}" -F "attachment=@${f}" \
"${API}/releases/${rel_id}/assets?name=${name}" >/dev/null
done
echo ">>> published ${#assets[@]} artifact(s) to release ${RELEASE_TAG}"
ci/run.sh
+20
View File
@@ -0,0 +1,20 @@
#!/bin/bash
# Local end-to-end build of one versitygw tag: clone -> patch -> build.
# Runs the same ci/ steps CI does, in a single shell so VGW_* env flows
# between them.
#
# VGW_TAG=v1.5.0 bash ci/run.sh
#
# Optional: point VGW_REPO_URL at a local clone for fast offline builds.
set -euo pipefail
: "${VGW_TAG:?set VGW_TAG, e.g. VGW_TAG=v1.5.0 bash ci/run.sh}"
export VGW_TAG
export VGW_SRC_DIR="${VGW_SRC_DIR:-src}"
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
bash "${DIR}/clone-source.sh"
bash "${DIR}/apply-patches.sh"
bash "${DIR}/build.sh"
echo ">>> done — artifacts in ${VGW_SRC_DIR}/dist/"
ci/support-matrix.yaml
+16
View File
@@ -0,0 +1,16 @@
# s3-gateway build support matrix — single source of truth for CI.
#
# Each entry maps an upstream versitygw release tag to the patch set
# (a directory under patches/) applied on top of that tag before
# building. Add a row when upstream cuts a release you want a
# de-branded build of; remove one to stop building it. See
# docs/multi-version-pipeline.md for the patch-refresh workflow.
#
# The highest-version entry is treated as "latest".
#
# tag upstream versitygw git tag (github.com/versity/versitygw)
# patches directory under patches/ to apply (defaults to `tag`)
versitygw:
- tag: v1.5.0
patches: v1.5.0
docs/multi-version-pipeline.md
+128
View File
@@ -0,0 +1,128 @@
# Multi-version build pipeline
How s3-gateway turns an upstream [versitygw](https://github.com/versity/versitygw)
release into a de-branded build, and how to keep it tracking new
upstream versions. This mirrors the scoutfs-build / scoutfs-notify
model, collapsed into one repo because the patch set is small.
## The flow
Each build is four steps, run per upstream tag:
1. **`ci/clone-source.sh`** — shallow-clones upstream versitygw at a
tag into `src/`. Tag selection priority:
1. `VGW_TAG` (set by the CI matrix or a manual run),
2. the pushed tag (`GITHUB_REF_NAME` when `GITHUB_REF_TYPE=tag`),
3. the latest remote `vX.Y.Z` tag.
The checkout forces `core.autocrlf=false` so the tree is byte-for-
byte upstream (LF) and the LF patches apply on any host.
2. **`ci/apply-patches.sh`** — resolves the patch directory for the tag
from `ci/support-matrix.yaml` (`patches:` field, default = tag name)
and applies `patches/<dir>/*.patch` with `git am --whitespace=nowarn
--3way`, in filename order. The `--3way` fallback is what lets a
series rebase across minor upstream drift.
3. **`ci/build.sh`** — runs `goreleaser release --clean
--skip=validate,publish` in `src/`, producing binaries + deb/rpm in
`src/dist/`. Because the patch commits sit on top of the tag, HEAD is
no longer exactly the tag, so we set `GORELEASER_CURRENT_TAG` to pin
the stamped version and `--skip=validate` to bypass the dirty/tag
checks. Upstream's `.goreleaser.yaml` is used unmodified — artifact
names stay `versitygw`.
4. **`ci/publish-release.sh`** — creates (or reuses) a Gitea release for
the tag in this repo and uploads `src/dist/*` as assets. Idempotent:
re-running replaces existing assets, so daily rebuilds are safe.
## The source of truth: `ci/support-matrix.yaml`
```yaml
versitygw:
- tag: v1.5.0
patches: v1.5.0
```
- `tag` — upstream git tag to build.
- `patches` — directory under `patches/` to apply (defaults to `tag`).
Point several tags at one directory if the same series applies to a
range of upstream releases.
The highest-version entry is "latest". `bash ci/matrix.sh` prints the
matrix this file implies.
> **Act Runner caveat.** Gitea's Act Runner does not support dynamic
> matrices (`matrix: ${{ fromJSON(...) }}`). The workflow therefore
> carries a **static** `matrix:` that you must keep in sync with this
> file by hand — same compromise scoutfs-build makes. `ci/matrix.sh`
> exists to diff/preview, not to feed CI.
## Adding a new upstream version
When upstream releases `vX.Y.Z`:
1. **Rebase the patch series** onto the new tag in a throwaway clone:
```bash
git clone https://github.com/versity/versitygw vgw && cd vgw
git -c core.autocrlf=false checkout -b debrand vX.Y.Z
git config user.email you@example.com && git config user.name you
git am --3way /path/to/s3-gateway/patches/<previous-tag>/*.patch
# If a hunk conflicts: fix files, `git add -A`, `git am --continue`.
mkdir -p /path/to/s3-gateway/patches/vX.Y.Z
git format-patch --binary vX.Y.Z..HEAD \
-o /path/to/s3-gateway/patches/vX.Y.Z
```
`--binary` is required because the series deletes PNG logo assets.
2. **Register it** in `ci/support-matrix.yaml`:
```yaml
versitygw:
- tag: vX.Y.Z
patches: vX.Y.Z
- tag: v1.5.0
patches: v1.5.0
```
3. **Mirror it** into the static `matrix.include` in
`.gitea/workflows/build.yml`.
4. Commit and push. The next CI run builds and publishes the new
version alongside the existing ones.
### Sanity-check a series before committing
```bash
VGW_REPO_URL="/path/to/local/versitygw" VGW_TAG=vX.Y.Z bash ci/run.sh
```
A clean `git am` plus a populated `src/dist/` means the series is good.
## Aging out a version
Remove its entry from `ci/support-matrix.yaml` and the workflow matrix.
CI stops building it; releases already published in this repo stay until
you delete them manually. You can keep the `patches/<tag>/` directory
for history or remove it.
## CI configuration
- **Runner**: `runs-on: s3-gateway-ci`. Add that label to your existing
scoutfs-ci Act Runner, or change the label.
- **Container**: `golang:latest` — pure-Go build, GoReleaser's bundled
nfpm makes deb/rpm without rpmbuild/dpkg.
- **Secret `TOKEN_GITEA`**: Gitea PAT with **write:repository**. Used
both to clone this repo (the workflow does a manual token-rewritten
clone, mirroring scoutfs-build, since a stock container has no Node
for `actions/checkout`) and to publish releases.
- **Variable `VGW_REPO_URL`** (optional): override the upstream URL,
e.g. to build from a mirror.
## Release tagging
Releases are created in this repo with the **upstream** tag name
(`v1.5.0`). If you need to ship a rebuild of the same upstream version
(e.g. a patch fix), set `RELEASE_TAG` in `ci/publish-release.sh`'s
environment to something like `v1.5.0-acg2` to avoid clobbering.
patches/v1.5.0/0001-webui-debrand.patch
+3036
View File
File diff suppressed because it is too large Load Diff