Ergo keeps history in RAM by default, so it is lost on every restart --
including the ones this stack's updater performs. HISTORY now selects a backend
at first deploy: sqlite (default, a file beside ircd.db, no extra container),
postgres (a pinned container via docker-compose.postgres.yml, loopback-only,
POSTGRES_PASSWORD_FILE so no secret lands in .env), or off. Both SQL backends
need Ergo 2.18.0+, which deploy.sh enforces. HISTORY_EXPIRE (default 30d) sets
retention, because upstream's 1w expire-time DELETES from persistent storage --
persistence with the shipped default would buy only a week.
Ergo opens the history backend only at startup and, unlike MySQL, has no
"after launching the server" guard for sqlite/postgresql: a rehash that enables
one reports success and then silently discards every message. So `ergoctl
history` restarts, and `ergoctl edit` detects a backend change and refuses to
apply it by rehash. The config validator runs with no network by design, so it
neutralises postgresql in its copy and reachability is checked separately.
Audit fixes (six lenses over first-deploy, runtime, lifecycle, security, docs):
- SECURITY (blocker): ircd/ is writable by the container uid while the 15-minute
cert sync and every ergoctl config edit run as root. cp/install/> follow a
symlink, so code execution in Ergo could redirect a root write onto any host
file. All such writes now stage under $STACK_DIR (0700 root) and land via mv
(rename(2) replaces a symlink instead of following it); root reads use cp -P
or refuse. Regression-tested with real symlinks.
- deploy.sh no longer swaps the image when ERGO_TAG changes on a deployed stack
(that bypassed pre-flight, snapshot, user warning and rollback); it points at
`TARGET_VERSION=... ergoctl update update`.
- restore was lossy and could lock you out: it now snapshots the current config
as well as the database, restores message history, re-points the postgres
password and re-hashes the local admin password into the restored config, and
validates before starting.
- `install -d -m` re-modes existing directories: `ergoctl backup /var/backups`
no longer chmods it 0700 root, and the updater no longer re-modes /var/log
(0775 root:syslog on Debian, which rsyslog needs).
- The admin oper password is printed only on a first run at a TTY, so it stays
out of cloud-init serial-console logs.
- A failed update is remembered, so the daily job stops repeating a disruptive
warn/stop/swap/roll-back cycle every night; postgres readiness gates an update
that would otherwise stop a healthy server it cannot restart.
- certsync no longer sends "TLS recovered" for outcomes that synced nothing.
- ergoctl history writes .env only after the restart is healthy, and returns 0.
- CR/LF is stripped before IRC framing, so a multi-line argument cannot inject a
second command; the ntfy token moves out of curl's argv.
- ufw/firewalld are additive, so 6667 is now explicitly revoked when PLAINTEXT=0.
- build.sh refuses to build a deploy.sh whose archive is missing a file the
script reads -- the failure mode that would have shipped a stack aborting on
every host.
Docs corrected against the code throughout, including retention, the pre-connect
account-registration default, encrypted-restore (AGE_IDENTITY), what a re-run
really does to .env, and what the update log does and does not contain.
Verified locally: all six suites pass (config render for each backend against the
real 2.19.1 template, yaml/oper/version/env helpers, the IRC client against a
fake server, and the audit fixes including the symlink escalation). Still not
exercised on a Docker host: the containers, ACME issuance, cert sync and
PostgreSQL itself.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
New deployments/ergo/: the Ergo IRC server behind Caddy for Let's Encrypt TLS
and the IRC-over-WebSocket endpoint. Docker rather than a native OpenRC/systemd
service, because Alpine's apk ergo trails upstream (3.24 ships 2.18.0 against a
2.19.1 security release) and Debian/Alma package it at all -- so native would
mean three install paths plus a per-distro ACME client.
Both containers run with network_mode: host. IRC bans, throttling and cloaking
key on the client's address, and Docker's userland proxy would hide every IPv6
client behind the bridge gateway; host mode also makes the repo's INPUT firewall
genuinely govern 80/443/6697. Caddy reaches Ergo over loopback, which is what
lets Ergo honour X-Forwarded-For (proxy-allowed-from defaults to localhost) and
mark web sessions secure.
- deploy.sh generates ircd.yaml ONCE from the pulled image's own default.yaml
(version-matched), rewriting the listeners/websockets blocks wholesale rather
than patching lines, then asserts hard post-conditions and validates with
`ergo run --smoke` in a throwaway container before anything starts.
- update.sh: pinned vX.Y.Z tags, GHSA + "### Security" release-note policies,
pre-flight against the new image, user NOTICE + grace, stop-consistent DB
snapshot, health check (IRC-level, not a bare TCP connect) and rollback that
restores the DB only when the schema actually moved. Compatibility-break
releases are held for review. certsync copies Caddy's cert pairwise-atomically
and verifies the fingerprint served on 6697 after the rehash.
- ergoctl: status/users/logs, validated edit+rehash, oper add/passwd/rm,
moderation, backup/restore, cert and update passthrough. Talks IRC to the
loopback listener over bash /dev/tcp and strips control characters from
replies.
- Ergo runs as a non-root system user, read-only rootfs, all caps dropped;
Caddy keeps only NET_BIND_SERVICE, with admin API and HTTP/3 off.
Reviewed adversarially across six lenses; 20 confirmed findings fixed, notably
a dead SIGHUP fallback (`rc=$?` after an `if` is always 0), several `set -e`
aborts from non-total pipelines, a release-list cache that only ever populated
in a subshell, and re-runs that used shell defaults instead of the deployed
.env. Verified locally: bash -n, LF endings, the ircd.yaml render against the
real 2.19.1 template in both PLAINTEXT modes, the yaml/oper/version/env helpers,
and the IRC client against a fake server (registration, oper, rehash success and
400-failure, control-character stripping, server-down paths). Not yet exercised
on a Docker host: the containers themselves, ACME issuance and cert sync.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
automations.sh was #!/usr/bin/env bash and written in bash, but a stock Alpine
box has busybox ash and no bash at all -- so the launcher died before it could
install anything:
./automations.sh -> env: 'bash': No such file or directory
curl ... | bash -> bash: not found
sh automations.sh -> syntax errors
cloud-init/base.yml already installs bash/git/curl before touching the repo;
the launcher never got the same treatment.
Give it a #!/bin/sh shebang and a strictly-POSIX prologue that ash can parse:
self-locate via ${0%/*} (busybox dirname takes `--` as the filename, so
`dirname -- "$0"` would answer "."), clone on the piped path as before, install
git/bash via apk/apt-get/dnf/yum, then exec bash on this same file. The bash
pass skips the prologue via BASH_VERSION, and an exported _AUTOMATIONS_REEXEC
guard rules out an exec loop. Everything below the handoff is the unchanged
bash launcher.
The generated bundle stub had the identical bug -- it's the other fresh-host
entry point -- so make it POSIX sh too and have it install bash before
extracting.
Docs: the one-liner now pipes to `sh`, bundle examples use `sh`.
Still bash-only on a bare Alpine host: the scp'd deployments/*/deploy.sh
artifacts.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A multi-agent sanity audit of the freshly-merged deployment found four
end-to-end blockers (and several smaller issues); all fixed here.
HIGH (were blocking):
- deploy.sh never called install_docker(), so `docker compose pull` hit
command-not-found on any host without Docker. Now called before the
compose steps.
- The container's server process runs as the image's own (often non-root)
user but the mounted config/TLS were root-owned 0640/0600 and the raft
volume root-owned -> vault crash-looped, never binding :8200. deploy.sh
now detects the image UID after pull and aligns ownership of config.hcl,
./tls and the data volume (a no-op when the image runs as root);
config.hcl is installed 0644 (holds no secrets).
- Docs told operators to set the daemon key `openbao_ca_cert`, but the
Kanrisha daemon's key is `ca_cert` (config.go, mapstructure:"ca_cert").
The wrong key is fatal on strict unmarshal / leaves TLS unverified.
Renamed in all 5 places (config.hcl, gen-tls.sh, deploy.sh x2, README).
- DR backup used `docker compose cp openbao:… -`, which emits a TAR stream,
so the age-encrypted snapshot was tar-wrapped and would not restore.
Switched to `docker compose exec -T openbao cat` for the raw bytes, wrote
the snapshot to a scratch path (not the live raft dir), and documented the
matching restore.
MEDIUM:
- Swap detection used `swapon --show` (absent on BusyBox) and `\s` (GNU-only)
-> silently no-op on Alpine, leaving swap on. Now uses /proc/swaps and
[[:space:]] so mlock hardening actually holds on musl.
- A Docker-published port bypasses the host INPUT firewall, so the source
rule was illusory. deploy.sh now narrows OPENBAO_BIND to OPENBAO_ADDR when
it is an IP, the compose/README/.env comments state the reality, and a new
Exposure section + an init-immediately warning were added.
- Fixed broken ../kanrisha/ and deployments/kanrisha/ links (separate repo).
LOW:
- OPENBAO_TLS_SANS is now honored (folded into the SAN list from the env).
- .gitignore excludes *.snap / *.snap.age.
- Bootstrap note clarifies bootstrap.sh needs the `bao` CLI (run it from the
Kanrisha host/workstation, not this Docker-only vault host).
- README multi-OS count corrected (eight stacks) + automations.sh header
lists openbao.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A dedicated OpenBao deployment, kept OFF the Kanrisha tape host so a compromise
of the tape node can't reach the vault.
- Native TLS on the listener (self-signed by default via gen-tls.sh, or a
CA-signed cert from a Smallstep CA over ACME) — no Caddy/Let's Encrypt;
reached over the LAN, not the public internet.
- Integrated raft storage (clean snapshot-based DR).
- mlock on (cap_add IPC_LOCK + memlock unlimited + host swapoff in deploy.sh).
- Manual unseal by default; optional PKCS#11 HSM auto-unseal.
- deploy.sh: Docker install (Alpine/Debian/Alma), self-signed cert, .env seed,
swapoff, firewall 8200/tcp, compose up; then prints init/unseal + the KV-v2 +
AppRole bootstrap for Kanrisha + the raft-snapshot DR flow. Self-contained
(config payload embedded by build.sh).
- Registered in automations.sh + the README deployment table.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New deployments/copyparty/: copyparty (copyparty/ac) behind Caddy/LE for the
web UI/WebDAV, plus its own SFTP (password auth) and FTPS listeners published
directly. Ships update.sh, which drives container updates off copyparty's
security-advisories API (api.copyparty.eu/advisories) -- policies latest|security|off.
- Real client IP end-to-end: Caddy XFF/X-Real-IP + copyparty xff-src: lan.
- SFTP host key + self-signed FTPS cert generated/persisted in /cfg; admin
password generated on first deploy; conf auto-included via the image's % /cfg.
- Firewall opens 80/443 + SFTP/FTPS + passive range (colon form for ports.d).
- Wired into automations.sh, README, .gitignore; cloud-init for fresh VMs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New deployments/squid/: an explicit forward proxy with SSL-bump TLS
interception (local CA, generated on first deploy) and hostname-targeted
static-content caching. Unlike the other stacks it is a forward proxy, not
a Caddy/Let's-Encrypt inbound site.
- Self-built minimal Alpine image (apk squid ships ssl-bump); entrypoint
renders squid.conf and generates the cache policy from the domain lists.
- Wildcard hostname caching (cache-domains.txt leading-dot + optional
cache-domains.regex); boost vs strict-allowlist toggle (CACHE_ONLY_LISTED).
- Storage gate never caches HTML or dynamic content; query strings exempt on
boosted domains so versioned static assets still cache.
- splice-domains.txt passthrough for pinned/banking domains.
- Deny-by-default http_access (TRUSTED_CIDR) + BIND_ADDR pinning; CA key 0600
on host, never embedded, git-ignored.
- Wired into automations.sh, README, .gitignore; cloud-init for fresh VMs.
The one-liner bootstrap only auto-installed git on Alpine (apk), so piping it
onto a fresh Debian/Alma host with no git fell straight into 'git clone' and
died with 'git: command not found'. oslib's pkg_install can't help here -- the
repo isn't on disk yet. Install git inline via apk/apt-get/dnf/yum, and fail
with a clear message if it still can't.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Integrate headplane (ghcr.io/tale/headplane) into the headscale stack, served
by Caddy at /admin. API-only (no Docker socket); deploy.sh mints a headscale
API key on first run, generates headplane.yaml, and wires optional OIDC login
via pocket-id (second client, /admin/oidc/callback). Adds HEADPLANE_* env,
compose service, Caddy routing; rebuild embedded archive.
Restructure around a single entry point (automations.sh) with a Gum wizard and
a self-extracting bundle for repo-less installs. Add scripts/oslib.sh so the
provisioning scripts (setup-host, harden-ssh, harden-jumphost, sshuser) run on
Alpine/Debian/Alma; seed root keys from globals/.
- ntfy SSH-login alerts (user, source IP, key, region, jump target) via pam_exec
- daily auto-updates: AUTO_REBOOT=idle reboots only when no SSH active; opt-in
Alpine stable-branch upgrades (ALLOW_RELEASE_UPGRADE)
- cloud-init: generic base/jumphost + per-deployment, which harden SSH by
default on fresh VMs
- pocket-id: optional WebFinger block (BASE_DOMAIN), tag v2.8.0
- headscale: fix oidc.expiry schema for 0.28 so the container starts
- Gitea release workflow on tag (TOKEN_GITEA); repo URLs -> Gitea
- README/LICENSE/.gitignore/.gitattributes (force LF)
Restructure around a single entry point (automations.sh) with a Gum wizard and
a self-extracting bundle for repo-less installs. Add scripts/oslib.sh so the
provisioning scripts (setup-host, harden-ssh, harden-jumphost, sshuser) run on
Alpine/Debian/Alma; seed root keys from globals/.
- ntfy SSH-login alerts (user, source IP, key, region, jump target) via pam_exec
- daily auto-updates with AUTO_REBOOT=idle (reboots only when no SSH active) and
opt-in Alpine stable-branch upgrades
- generic + per-deployment cloud-init; Gitea release workflow on tag
- README/LICENSE/.gitignore/.gitattributes (force LF); repo URLs -> Gitea
57_WolveandClaude Opus 5