feat(ergo): add Ergo IRC server deployment with ergoctl admin CLI

New deployments/ergo/: the Ergo IRC server behind Caddy for Let's Encrypt TLS
and the IRC-over-WebSocket endpoint. Docker rather than a native OpenRC/systemd
service, because Alpine's apk ergo trails upstream (3.24 ships 2.18.0 against a
2.19.1 security release) and Debian/Alma package it at all -- so native would
mean three install paths plus a per-distro ACME client.

Both containers run with network_mode: host. IRC bans, throttling and cloaking
key on the client's address, and Docker's userland proxy would hide every IPv6
client behind the bridge gateway; host mode also makes the repo's INPUT firewall
genuinely govern 80/443/6697. Caddy reaches Ergo over loopback, which is what
lets Ergo honour X-Forwarded-For (proxy-allowed-from defaults to localhost) and
mark web sessions secure.

- deploy.sh generates ircd.yaml ONCE from the pulled image's own default.yaml
  (version-matched), rewriting the listeners/websockets blocks wholesale rather
  than patching lines, then asserts hard post-conditions and validates with
  `ergo run --smoke` in a throwaway container before anything starts.
- update.sh: pinned vX.Y.Z tags, GHSA + "### Security" release-note policies,
  pre-flight against the new image, user NOTICE + grace, stop-consistent DB
  snapshot, health check (IRC-level, not a bare TCP connect) and rollback that
  restores the DB only when the schema actually moved. Compatibility-break
  releases are held for review. certsync copies Caddy's cert pairwise-atomically
  and verifies the fingerprint served on 6697 after the rehash.
- ergoctl: status/users/logs, validated edit+rehash, oper add/passwd/rm,
  moderation, backup/restore, cert and update passthrough. Talks IRC to the
  loopback listener over bash /dev/tcp and strips control characters from
  replies.
- Ergo runs as a non-root system user, read-only rootfs, all caps dropped;
  Caddy keeps only NET_BIND_SERVICE, with admin API and HTTP/3 off.

Reviewed adversarially across six lenses; 20 confirmed findings fixed, notably
a dead SIGHUP fallback (`rc=$?` after an `if` is always 0), several `set -e`
aborts from non-total pipelines, a release-list cache that only ever populated
in a subshell, and re-runs that used shell defaults instead of the deployed
.env. Verified locally: bash -n, LF endings, the ircd.yaml render against the
real 2.19.1 template in both PLAINTEXT modes, the yaml/oper/version/env helpers,
and the IRC client against a fake server (registration, oper, rehash success and
400-failure, control-character stripping, server-down paths). Not yet exercised
on a Docker host: the containers themselves, ACME issuance and cert sync.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-02 13:19:31 -05:00
co-authored by Claude Opus 5
parent 1c1c41ac17
commit 4cbd808eb1
16 changed files with 3269 additions and 6 deletions
+5
View File
@@ -13,3 +13,8 @@ Caddyfile text eol=lf
# Treat the embedded-archive deploy scripts as text too (they're base64+shell).
deployments/*/deploy.sh text eol=lf
# Extension-less / unusual-extension scripts and configs that must stay LF.
deployments/ergo/ergoctl text eol=lf
*.caddy text eol=lf
*.motd text eol=lf
+9
View File
@@ -28,6 +28,15 @@ deployments/squid/ssl/
# copyparty.conf.example template stays tracked.
deployments/copyparty/cfg/
# Ergo -- generated on the host at deploy time: ircd/ (config with the oper
# hash, database, TLS key), caddy/ (ACME account + certs), secrets/ (admin oper
# password), backups/, templates/. Only the loose sources stay tracked.
deployments/ergo/ircd/
deployments/ergo/caddy/
deployments/ergo/secrets/
deployments/ergo/backups/
deployments/ergo/templates/
# ── Backups ─────────────────────────────────────────────────────────────────
*.tar.gz.age
*-backup-*.tar.gz*
+8 -3
View File
@@ -120,12 +120,15 @@ deployments/<name>/ # one folder per stack
| [`copyparty`](deployments/copyparty/) | Portable file server — web UI/WebDAV behind Caddy, plus direct **SFTP** + **FTPS**. Ships a security-notices-aware updater. | — |
| [`simplex`](deployments/simplex/) | SimpleX SMP + XFTP relay with Tor hidden services + encrypted backups. | globals/age-pubkey.txt |
| [`openbao`](deployments/openbao/) | Hardened tape-encryption key store for **Kanrisha** (the LTO tape-archive system — separate repo/host). **Exception: native TLS on the LAN, no Caddy/LE.** raft storage, mlock, self-signed or Smallstep-ACME cert, manual/HSM unseal. | globals/age-pubkey.txt (backups) |
| [`ergo`](deployments/ergo/) | IRC server ([Ergo](https://ergo.chat)) with Caddy for Let's Encrypt + the websocket endpoint. **Both containers use host networking** (real client IPs for bans/cloaking; the host firewall applies). Ships `ergoctl` (admin CLI) and a health-checked updater that also syncs Caddy's cert into Ergo. | — |
## Conventions
- **Alpine + Docker Compose + Caddy/Let's Encrypt** across every stack, with two
exceptions: `squid` (a forward proxy with a local TLS-interception CA) and
`openbao` (a same-LAN secrets store with native TLS) — neither uses Caddy/LE.
`ergo` keeps Caddy/LE but runs both containers in the **host network
namespace** (IRC needs real client IPs), so it has no published ports.
- **`build.sh``deploy.sh`**: each stack's `build.sh` embeds its
`docker-compose.yml` / `Caddyfile` / `.env.example` into a single
self-contained `deploy.sh` (base64 tar.gz). That one file can be `scp`'d to a
@@ -154,8 +157,8 @@ instance user-data, and the host configures itself on first boot.
## Multi-OS notes
The host-provisioning scripts (`setup-host`, `harden-ssh`, `harden-jumphost`,
`sshuser`) and the eight Docker stacks (pocket-id, beszel, headscale, webfinger,
squid, copyparty, simplex, openbao) run on Alpine, Debian, and Alma. Distro differences live in
`sshuser`) and the nine Docker stacks (pocket-id, beszel, headscale, webfinger,
squid, copyparty, simplex, openbao, ergo) run on Alpine, Debian, and Alma. Distro differences live in
[`scripts/oslib.sh`](scripts/oslib.sh) — package manager (`apk`/`apt`/`dnf`),
init system (OpenRC/systemd), sshd service name, the per-distro `sftp-server`
path, hostname, boot hooks, and the sshguard log source/backend.
@@ -199,7 +202,9 @@ networking is unaffected. The harden scripts and `cloud-init/base.yml` /
- **Docker caveat** — containers published with `-p` (e.g. Caddy's 80/443)
reach the host through nat/`FORWARD` and **bypass `INPUT`**, so the firewall
neither blocks nor needs to open them; the per-stack rule files are
belt-and-braces for any host-bound bind and self-documentation.
belt-and-braces for any host-bound bind and self-documentation. The one
stack that runs with `network_mode: host` (`ergo`) binds on the host
directly, so there the registered ports are the real gate.
- **Recovery** — `harden-firewall.sh disable` un-locks you: on iptables it
flushes the rules and sets `INPUT` back to `ACCEPT` (persisted); on firewalld it
re-opens SSH (the `ssh` service + the configured port) and leaves firewalld
+9 -3
View File
@@ -14,8 +14,8 @@
# It opens a Gum wizard (auto-installed) that lets you:
# • Mode: deploy on THIS host, or build deploy.sh artifacts locally.
# • Pick any deployment (pocket-id, beszel, headscale, webfinger, squid,
# copyparty, simplex, openbao) or any generic script (setup-host, harden-ssh,
# harden-jumphost, sshuser, auto-update).
# copyparty, simplex, openbao, ergo) or any generic script (setup-host,
# harden-ssh, harden-jumphost, sshuser, auto-update).
# Shared defaults come from globals/ (see globals/README.md).
#
# Non-interactive: set SKIP_PROMPTS=1 plus the needed vars and pipe the menu
@@ -118,7 +118,7 @@ set -euo pipefail
. "$ROOT/scripts/lib.sh"
load_globals
DEPLOYMENTS=(pocket-id beszel headscale webfinger squid copyparty simplex openbao)
DEPLOYMENTS=(pocket-id beszel headscale webfinger squid copyparty simplex openbao ergo)
SCRIPTS=(setup-host harden-ssh harden-jumphost sshuser auto-update)
# ----------------------------------------------------------------------------
@@ -184,6 +184,12 @@ ask_deployment_vars() {
openbao)
ask OPENBAO_ADDR "LAN address the Kanrisha tape host reaches the vault at (IP or DNS)"
ask OPENBAO_BIND "Host IP to bind the API on (blank = 0.0.0.0)" optional ;;
ergo)
ask ERGO_DOMAIN "IRC server hostname (e.g. irc.example.com)"
ask ACME_EMAIL "Let's Encrypt email"
ask NETWORK_NAME "IRC network name, no spaces (blank = the hostname)" optional
ask PLAINTEXT "Also serve PUBLIC plaintext IRC on 6667? (1 = yes, blank = no)" optional
ask UPDATE_POLICY "Auto-update policy: latest | security | off (blank = latest)" optional ;;
esac
}
+63
View File
@@ -0,0 +1,63 @@
# Copy to .env and fill in. docker compose picks .env up automatically.
# deploy.sh seeds this file on first run and never rewrites it afterwards.
#
# NOTE: values are NOT quoted and must not contain a dollar sign (Compose
# interpolates .env). Secrets never live here: the admin oper password is in
# secrets/admin.pass (0600) and only its bcrypt hash is in ircd/ircd.yaml.
# --- Identity -----------------------------------------------------------------
# Public hostname (lower-case). Becomes Ergo's server.name, the Let's Encrypt
# cert subject, the Caddy site and the allowed websocket origin. Immutable after
# the first deploy (Ergo refuses to rehash a changed server name).
ERGO_DOMAIN=irc.example.com
# Email for Let's Encrypt registration / expiry notices.
ACME_EMAIL=admin@example.com
# IRC network name (ISUPPORT NETWORK=, welcome line). Letters, digits, . _ -
# only -- no spaces. deploy.sh defaults it to ERGO_DOMAIN.
NETWORK_NAME=irc.example.com
# --- Image --------------------------------------------------------------------
# ghcr.io tags carry the leading 'v' (v2.19.1). deploy.sh pins the newest
# release on first deploy; update.sh bumps this when it updates. Never pin
# 'stable'/'latest' here -- those are separate builds and defeat rollback.
ERGO_IMAGE=ghcr.io/ergochat/ergo
ERGO_TAG=v2.19.1
# uid:gid Ergo runs as inside the container = the host's `ergo` system user
# (created by deploy.sh; owns ./ircd). Filled in by deploy.sh.
ERGO_UID=
ERGO_GID=
# --- Listeners ----------------------------------------------------------------
# 0 (default): plaintext 6667 on 127.0.0.1 only (healthcheck + ergoctl).
# 1: ALSO serve plaintext IRC on :6667 publicly, with STS advertised so capable
# clients pin TLS. Credentials sent over 6667 cross the internet in clear.
# Chosen at the FIRST deploy: it selects a listener in ircd/ircd.yaml, which
# deploy.sh never rewrites. To change it later, edit that listener with
# 'ergoctl edit', set the value here, 'ergoctl restart', then re-run deploy.sh
# so the host firewall matches.
PLAINTEXT=0
# --- Updates (update.sh) ------------------------------------------------------
# This file is where the scheduled jobs read these four knobs from; edit them
# here. (An explicit environment variable, or a value pinned in
# /etc/ergo-update.conf, overrides what is set here for that run.)
# latest - update to the newest release whenever one exists (default)
# security - update ONLY when a published GitHub security advisory covers the
# running version, or a release in range has a "### Security"
# section in its notes
# off - never change the running version (check/notify only)
UPDATE_POLICY=latest
# Releases whose notes announce "Compatibility breaks" are held for review and
# only notified, unless FORCE_UPDATE=1.
FORCE_UPDATE=0
# Seconds of warning (NOTICE to all users) before an update restarts Ergo.
UPDATE_GRACE=60
# 1 = the daily run also pulls a newer Caddy image and recreates it (drops every
# web-client websocket at that moment). 0 = Caddy updates via `ergoctl update caddy`.
CADDY_AUTOUPDATE=0
# --- Image tags ---------------------------------------------------------------
CADDY_TAG=2-alpine
+76
View File
@@ -0,0 +1,76 @@
# Caddyfile for the Ergo stack. Caddy runs in the HOST network namespace.
#
# Roles:
# 1. Obtain and renew the Let's Encrypt certificate for {$ERGO_DOMAIN}. Ergo's
# :6697 listener reuses that cert -- `update.sh certsync` copies it into
# ./ircd and rehashes Ergo (runs every 15 minutes, silent when unchanged).
# 2. Terminate HTTPS for IRC-over-WebSocket: wss://{$ERGO_DOMAIN}/webirc ->
# Ergo's plaintext websocket listener on 127.0.0.1:8097. Caddy adds
# X-Forwarded-For / X-Forwarded-Proto and Ergo trusts them from loopback
# (proxy-allowed-from: localhost), so web users keep their real IP and are
# marked secure (+Z).
# 3. A plain-text landing page with connection details at /.
#
# Add your own site config (e.g. a Gamja web client, see README) as
# conf.d/*.caddy -- deploy.sh installs this file but never touches conf.d/.
# Apply changes with: docker compose restart caddy
{
email {$ACME_EMAIL}
# Host networking would put the admin API on the HOST's 127.0.0.1:2019,
# reachable by every local process and container. The config is static, so
# turn it off; changes are applied by restarting the container.
admin off
# No HTTP/3: it would bind udp/443 on the host (not in the firewall's port
# list) and browsers do not run WebSockets over h3 anyway.
servers {
protocols h1 h2
}
}
# Loopback-only health endpoint for the compose healthcheck (no admin API to ask).
http://127.0.0.1 {
respond "ok" 200
}
{$ERGO_DOMAIN} {
encode zstd gzip
# IRC over WebSocket. Ergo ignores the request path; Caddy proxies the
# Upgrade transparently and supplies X-Forwarded-For/-Proto itself.
handle_path /webirc* {
reverse_proxy 127.0.0.1:8097
}
# Operator additions (web client, redirects, ...). See conf.d/00-readme.caddy.
import conf.d/*.caddy
handle {
header Content-Type "text/plain; charset=utf-8"
respond <<TXT
{$NETWORK_NAME} -- IRC server
Connect with any IRC client:
server: {$ERGO_DOMAIN}
port: 6697 (TLS)
web: wss://{$ERGO_DOMAIN}/webirc (IRC-over-WebSocket endpoint)
Register a nickname: /msg NickServ REGISTER <password>
Powered by Ergo (https://ergo.chat).
TXT 200
}
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains"
X-Content-Type-Options "nosniff"
Referrer-Policy "strict-origin-when-cross-origin"
-Server
}
log {
output stdout
format console
}
}
+281
View File
@@ -0,0 +1,281 @@
# ergo
[Ergo](https://ergo.chat) — a modern IRC server with built-in services
(NickServ/ChanServ/HostServ), message history, always-on "bouncer" clients and
IRCv3 — behind Caddy for automatic Let's Encrypt TLS. Ships a host CLI
(**`ergoctl`**) for day-to-day administration and a health-checked updater
(**`update.sh`**) that also keeps Ergo's TLS certificate in sync with Caddy.
## Docker, not an OpenRC/systemd service
Ergo is a single static Go binary, so a native install is perfectly possible.
This stack still uses Docker Compose because it matches the repo's goals better:
| | Docker (this stack) | Native OpenRC / systemd |
|---|---|---|
| Alpine, Debian, Alma | one compose file, one image | Alpine packages `ergo` in community, but a stable branch keeps the version it shipped with (3.24 → 2.18.0, 3.23/3.22 → 2.16.0) while upstream is at 2.19.1, a *security* release; only edge tracks upstream. Debian/Alma have no package at all → hand-rolled tarball install + a unit per init system |
| Upgrade / rollback | `ERGO_TAG` in `.env`, health-checked, DB snapshot, rollback | replace a binary + hand-restore the DB |
| ACME TLS | stock Caddy (already used by every other stack) | certbot/acme.sh per distro + deploy hook |
| Real client IPs | yes — **host networking** (see below) | yes |
The usual objection to containerised IRC — bans, throttling and IP cloaking
need the client's real address, and Docker's userland proxy hides IPv6 clients
behind the bridge gateway — is solved by running **both** containers with
`network_mode: host`. That also means the host's deny-by-default firewall
genuinely governs the ports (the "Docker bypasses INPUT" caveat from the root
README does not apply here).
## Topology
| Port | Who listens | Purpose |
|---|---|---|
| `6697/tcp` | Ergo | IRC over TLS (Let's Encrypt cert, copied from Caddy) |
| `443/tcp` | Caddy | `wss://DOMAIN/webirc` → Ergo's websocket listener on `127.0.0.1:8097`; text landing page at `/` |
| `80/tcp` | Caddy | ACME HTTP-01 + redirect to HTTPS |
| `127.0.0.1:6667` | Ergo | loopback plaintext: compose healthcheck + `ergoctl` (exempt from ip-limits/bans, treated as secure) |
| `6667/tcp` public | Ergo | only with `PLAINTEXT=1` (STS is then advertised) — **not recommended** |
No HTTP/3 (would bind udp/443), Caddy's admin API is **off** (it would sit on
the host's loopback), and there is no `ports:` section — host mode.
## Required `.env` values
| Variable | Notes |
|---|---|
| `ERGO_DOMAIN` | Public hostname (lower-case). Server name, cert subject, Caddy site and websocket origin. **Immutable** after the first deploy. |
| `ACME_EMAIL` | Let's Encrypt registration email. |
| `NETWORK_NAME` | IRC network name (letters, digits, `. _ -`; no spaces). Defaults to `ERGO_DOMAIN`. |
| `ERGO_TAG` | Pinned image tag **with the `v`** (`v2.19.1`). Set by deploy.sh to the newest release; bumped by update.sh. Never `stable`/`latest` — if a first deploy could not reach GitHub it falls back to `stable`, and the next update run pins it. |
| `PLAINTEXT` | `0` (default) or `1` — public plaintext 6667. **Fixed at the first deploy** (it selects a listener in `ircd.yaml`, which deploy.sh never rewrites); see below to change it. |
| `UPDATE_POLICY` | `latest` (default) / `security` / `off` — see [Updates](#updates). |
See [`.env.example`](.env.example) for the rest (`UPDATE_GRACE`,
`FORCE_UPDATE`, `CADDY_AUTOUPDATE`, `CADDY_TAG`). `.env` must not contain `$`
(Compose interpolates it) — secrets never live there. The four update knobs are
read from `.env` by the scheduled jobs; an environment variable, or a value
pinned in `/etc/ergo-update.conf`, overrides it for that run.
Deploy-time-only options (not stored in `.env`): `ERGO_AUTOUPDATE=0` installs
`update.sh` without scheduling the daily update (cert sync is still scheduled),
and `CERT_WAIT` (default 180s) bounds the wait for the first Let's Encrypt cert.
To switch `PLAINTEXT` after the first deploy: change the plaintext listener with
`ergoctl edit` (`"127.0.0.1:6667":``":6667":`, and `server.sts.enabled` with
it), set `PLAINTEXT` in `.env`, run `ergoctl restart`, then re-run `deploy.sh`
so the host firewall matches. Passing a conflicting `PLAINTEXT=` to a re-run
stops with those instructions rather than half-applying the change.
## Deploy
```bash
./automations.sh # Deploy on this host → deploy: ergo
```
Or build + run the self-contained artifact:
```bash
./build.sh
scp deploy.sh root@host:
ssh root@host 'bash deploy.sh'
# non-interactive:
# ERGO_DOMAIN=irc.example.com ACME_EMAIL=me@example.com NETWORK_NAME=MyNet SKIP_PROMPTS=1 bash deploy.sh
```
Unattended provisioning: [`cloud-init.yml`](cloud-init.yml). On first run
deploy.sh:
1. installs Docker (+ `openssl`, `jq`, `curl`) and creates the `ergo` system
user the container runs as;
2. registers `80,443,6697/tcp` with the host firewall
(`/etc/firewall/ports.d/ergo.rule`, or ufw/firewalld);
3. pins `ERGO_TAG` to the newest release and seeds `.env`;
4. generates **`ircd/ircd.yaml` once**, from the pulled image's *own*
`default.yaml` (so it always matches the running version): server/network
name, the listeners above, the websocket origin, the cloak suffix, and an
`admin` oper whose random password is saved to `secrets/admin.pass` (0600;
only the bcrypt hash goes into the yaml). Hard post-checks refuse to
continue if the upstream template layout ever changes under the edits;
5. validates the config with `ergo run --smoke` in a throwaway container
**before** anything starts, validates the Caddyfile, then starts the stack
and waits for both healthchecks (Ergo's is IRC-level: a `QUIT` must be
answered with `ERROR`);
6. waits for Caddy's Let's Encrypt cert, copies it into `ircd/` and rehashes
Ergo (until then Ergo serves a self-signed cert from its own `mkcerts`);
7. schedules `update.sh certsync` every 15 min and `update.sh run` daily
(`ERGO_AUTOUPDATE=0` schedules only the cert sync).
Re-runs are idempotent: they never touch `ircd/ircd.yaml`, `secrets/` or
`caddy/etc/conf.d/`, and they adopt the deployed settings from `.env` rather
than this shell's defaults, so a plain `bash deploy.sh` cannot silently close a
`PLAINTEXT=1` firewall port or reset your update policy. A setting you do pass
explicitly is written back to `.env`. A changed `Caddyfile` triggers a Caddy
restart, since Caddy has no admin API here.
## Administer: `ergoctl`
Installed at `/usr/local/bin/ergoctl`. It talks to Ergo over the loopback
listener as the `admin` oper.
| Command | What it does |
|---|---|
| `ergoctl status` / `users` / `logs [-f]` / `version` | health, versions, TLS state, user count; server log with health-probe noise filtered |
| `ergoctl edit` / `motd` | `$EDITOR` on `ircd.yaml` / `ergo.motd`, then **validate** (throwaway container) and **REHASH**; the previous file is restored if either step fails |
| `ergoctl rehash` | reload config, MOTD and TLS certs without disconnecting anyone |
| `ergoctl restart` / `stop` / `start` | restart the container (drops every user; needed for settings Ergo fixes at startup, such as `server.name`, `datastore.path`, `casemapping`, `enforce-utf8`, `max-line-len`, `idle-timeouts`). Changes to `docker-compose.yml` or `.env` need `docker compose up -d` instead |
| `ergoctl oper list\|add\|passwd\|rm` | manage operators in `ircd.yaml`; passwords are generated (24 chars) and printed once |
| `ergoctl passwd [show\|rotate]` | the `admin` oper password |
| `ergoctl announce <text>` | NOTICE to everyone |
| `ergoctl kill <nick> [reason]` · `ban add\|del\|list\|info` · `defcon [1-5]` | moderation (`UBAN` under the hood). `ban add` takes an **IP, CIDR, `nick!user@host` mask, or account name** — a bare name is an *account* to suspend, not a connected nick, so run `ban info <nick>` first to get their IP |
| `ergoctl cmd <raw IRC line>` | anything else as the admin oper, e.g. `ergoctl cmd NS SAREGISTER alice hunter2`, `ergoctl cmd CS PURGE #spam`; replies are printed with control characters stripped |
| `ergoctl cert [show\|sync]` | cert on disk vs the one served on 6697; force a sync |
| `ergoctl backup [--live] [dir]` / `restore <file>` | see [Backups](#backups) |
| `ergoctl update …` | passthrough to `update.sh` |
| `ergoctl debug on\|off` · `caddy-restart` · `shell` | log level; apply Caddyfile/conf.d changes; a shell in the container |
Editing by hand works too: change `ircd/ircd.yaml`, then `ergoctl rehash` — a
rejected config leaves the old one running (Ergo's rehash is transactional), but
the *file* on disk would then be unbootable, which is why `ergoctl edit`
validates first. Ergo's [manual](https://github.com/ergochat/ergo/blob/master/docs/MANUAL.md)
covers every option; the pristine template for your version is in `templates/`.
## TLS
Caddy owns the ACME account and renews the cert on its own schedule (ARI-driven).
Ergo terminates TLS itself on 6697, so the cert has to reach `ircd/fullchain.pem`
+ `privkey.pem`. `update.sh certsync` (every 15 minutes, via busybox crond or a
systemd timer) does that the way upstream's certbot deploy-hook recipe does,
plus safety checks:
- picks the newest `<domain>.crt` under `caddy/data/caddy/certificates/*/` (Let's
Encrypt or the ZeroSSL fallback), skipping a renewal that is still mid-write;
- verifies the key matches the cert (certmagic writes them as separate files and
generates a new key on every renewal), stages both in `ircd/` and installs
them with two atomic `mv`s — Ergo never sees a half pair, which would be
fatal at its next start;
- `SIGHUP`s Ergo (rehash swaps the TLS config for new connections only) and
confirms the fingerprint served on 6697 changed; if not, the previous pair is
restored and you get an ntfy alert.
Silent when nothing changed. The daily run also warns when the served cert is
still self-signed (DNS/80/443 not right yet) or expires within 14 days
(certsync not running). `ergoctl cert` shows both sides.
## Updates
```bash
ergoctl update check # running vs latest, published security advisories, cert state; changes nothing
ergoctl update update # update now (TARGET_VERSION=2.19.1 to pin); honours the safety rails
ergoctl update install # (re)schedule / uninstall to stop
```
`UPDATE_POLICY` (in `.env`) drives the daily `run`:
| Policy | Behaviour |
|---|---|
| `latest` *(default)* | Update to the newest release whenever one exists. |
| `security` | Update **only** when a published GitHub security advisory covers the running version, or a release since it has a `### Security` section in its notes (upstream only opened GHSAs from 2.19.1). |
| `off` | Never change the running version (check/notify only). |
Every update, scheduled or manual:
1. pulls the pinned `ghcr.io/ergochat/ergo:vX.Y.Z` and **pre-flights** it against
a copy of `ircd/` (`ergo run --smoke` — also dry-runs a DB schema upgrade);
2. `NOTICE`s connected users and waits `UPDATE_GRACE` seconds (default 60);
3. stops Ergo and snapshots `ircd.db` to `backups/` (consistent copy; buntdb is
append-only, so a live copy could miss the last second), keeps the last 5;
4. flips `ERGO_TAG`, starts, waits for the healthcheck **and** a registration
handshake;
5. on failure: previous tag back; if the old version refuses the upgraded
database ("Database requires update") the snapshot is restored too. Cause
and snapshot path go into the ntfy message;
6. on success: saves the new image's `default.yaml` under `templates/` and, if it
differs from the previous version's, writes `templates/diff-vA-vB.txt` so
new/renamed options are visible.
Releases whose notes announce **compatibility breaks** are held (one ntfy, not
daily) until you run `ergoctl update update` or set `FORCE_UPDATE=1`. Restarting
Ergo disconnects every user (there is no hot restart), so pick `security` if
that matters more than being current. Caddy is only updated by
`ergoctl update caddy` (or daily with `CADDY_AUTOUPDATE=1`); recreating it drops
web-client websockets. Update results reuse the ntfy config at
`/etc/ssh-notify.conf` when present.
## Backups
`ergoctl backup` writes `backups/ergo-backup-<ts>.tar.gz` (0600) with
`ircd.yaml`, `ircd.db`, `ergo.motd` and a `meta` file (version, date). By default
it stops Ergo for a few seconds for a consistent copy (`--live` skips that).
Put an age public key in `$STACK_DIR/age-recipients.txt` (e.g. from
`globals/age-pubkey.txt`) **and install `age`** (`apk add age`, `apt install age`,
`dnf install age` — deploy.sh does not) and backups are encrypted with it; with
the recipients file present but `age` missing, `ergoctl backup` refuses rather
than writing the account and oper hashes out in the clear. TLS files are not
included — certsync regenerates them. `ergoctl restore <file>` stops Ergo,
warns if the backup came from a newer version (older Ergo cannot read a newer
schema), restores, and waits for health. Ergo's own `datastore.autoupgrade`
leaves `ircd/ircd.db.v<N>.<ts>.bak` files behind on schema upgrades; prune them
once you are happy with a release.
## Web client (optional)
The `wss://DOMAIN/webirc` endpoint is ready; Ergo only accepts browser
connections whose `Origin` is `https://DOMAIN` (`server.websockets.allowed-origins`).
To host [Gamja](https://codeberg.org/emersion/gamja) on the same origin, unpack a
release tarball into `caddy/www/gamja`, add
`{"server": {"url": "/webirc", "autojoin": "#lobby"}}` as `config.json`, mount
`./caddy/www:/www:ro` in the caddy service and drop a `handle { root * /www/gamja
file_server }` block into `caddy/etc/conf.d/` (see `00-readme.caddy`), then
`ergoctl caddy-restart`.
## Security model
- **Host loopback is Ergo's trust boundary.** Loopback peers are exempt from
bans and ip-limits, are treated as secure, and (because
`proxy-allowed-from: [localhost]` is what lets Caddy's `X-Forwarded-For`
through) may assert a client IP. Every local process and every other
host-network container can reach `127.0.0.1:6667`/`:8097`. This stack is for
a single-purpose container host without untrusted local users; deploy.sh
warns about other host-network containers.
- **Ergo runs unprivileged**: as the host's `ergo` system user, read-only root
filesystem, all capabilities dropped, `no-new-privileges`. `ircd/` (database,
config with the oper hash, TLS key) is `0700 ergo:ergo`; `secrets/`,
`backups/`, `caddy/data` are `0700 root`.
- **Caddy runs as root** (it needs `NET_BIND_SERVICE` for 80/443) with every
other capability dropped, admin API off, HTTP/3 off.
- **Loopback plaintext is the control plane by design**: OPER over loopback is
not a downgrade (Ergo marks it secure), and it lets `ergoctl` work even when a
ban or throttle would lock out a remote client. `PLAINTEXT=1` puts real users'
credentials on the wire in clear — leave it at `0`.
- The admin password is sent to Ergo only over loopback and stored only in
`secrets/admin.pass`. `ergo genpasswd` hashes at bcrypt cost 4, so oper
passwords are always generated (24 random chars), never typed.
## Files
| File | Purpose |
|---|---|
| `docker-compose.yml` | caddy + ergo, both `network_mode: host`, hardened; IRC-level healthcheck. |
| `Caddyfile` | ACME, `/webirc` websocket proxy, text landing page, `admin off`, no h3; imports `conf.d/*.caddy`. |
| `conf.d-readme.caddy` | Installed once as `caddy/etc/conf.d/00-readme.caddy` (operator drop-ins; Gamja example). |
| `ergo.motd` | MOTD template (`__NETWORK_NAME__`, `__DOMAIN__`). |
| `ergolib.sh` | Shared helpers: `.env`, compose, health/IRC probes, `--smoke` validator, oper edits, certsync, ntfy. |
| `update.sh` | Updater + certsync + Caddy update + scheduling. |
| `ergoctl` | Host admin CLI. |
| `.env.example` | Stack tunables. |
| `deploy.sh` / `build.sh` | Self-contained installer + archive embedder. |
| `cloud-init.yml` | Fresh-VM bootstrap (harden SSH, then deploy). |
On the host: `/srv/ergo/{ircd,caddy/{etc,data,config},secrets,backups,templates}`
plus `.state/` (notification de-duplication) and `.env.bak.<ts>` copies from each
update (last 5 kept); `/etc/ergo-update.conf` (where the jobs find the stack);
`/usr/local/bin/ergoctl`; `/var/log/ergo-update.log` (scheduled runs log there,
since busybox crond has nowhere else to put their output).
## Notes
- Only one host-networked Caddy fits on a box: don't co-locate this stack with
another Caddy/80/443 stack (deploy.sh checks the ports).
- DNS for `ERGO_DOMAIN` must resolve to the host and 80/443 be reachable before
deploy for the cert to issue; Ergo stays up on a self-signed cert meanwhile.
- Ergo's built-in registration is open by default (`accounts.registration.enabled`);
edit `ircd.yaml` if you want an invite-only network.
+55
View File
@@ -0,0 +1,55 @@
#!/usr/bin/env bash
#
# build.sh -- (re)embed the loose deployment files into deploy.sh as a base64
# tar.gz payload after __ARCHIVE_BELOW__. Idempotent: strips any existing
# payload first.
#
# Run this after editing ANY embedded file below, then re-stage deploy.sh --
# the deployed stack uses the EMBEDDED copies, not the loose files.
set -euo pipefail
DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
SCRIPT="$DIR/deploy.sh"
MARKER="__ARCHIVE_BELOW__"
# README, cloud-init and the build/deploy scripts are NOT embedded. ircd.yaml is
# generated on the host from the pulled image's own default config.
FILES=(
docker-compose.yml
Caddyfile
conf.d-readme.caddy
ergo.motd
.env.example
ergolib.sh
update.sh
ergoctl
)
[[ -f "$SCRIPT" ]] || { echo "deploy.sh not found at $SCRIPT" >&2; exit 1; }
for f in "${FILES[@]}"; do
[[ -f "$DIR/$f" ]] || { echo "Missing $DIR/$f" >&2; exit 1; }
done
# Syntax-check the bash payload before embedding it.
for f in ergolib.sh update.sh ergoctl deploy.sh; do
bash -n "$DIR/$f" || { echo "bash -n failed: $f" >&2; exit 1; }
done
PAYLOAD=$(tar -czf - -C "$DIR" "${FILES[@]}" | base64)
TMP=$(mktemp)
trap 'rm -f "$TMP"' EXIT
sed "/^${MARKER}\$/,\$d" "$SCRIPT" > "$TMP"
{
echo "$MARKER"
echo "$PAYLOAD"
} >> "$TMP"
mv "$TMP" "$SCRIPT"
chmod +x "$SCRIPT"
trap - EXIT
size=$(wc -c < "$SCRIPT")
echo "Built $SCRIPT (${size} bytes)"
+42
View File
@@ -0,0 +1,42 @@
#cloud-config
#
# Ergo (IRC server) -- harden SSH, then deploy, on a fresh host.
#
# Fill in REPO_URL and the values in the runcmd block, then paste this as the
# instance user-data. DNS for ERGO_DOMAIN must point at this host and ports
# 80/443 must be reachable before boot, or the Let's Encrypt cert request fails
# (Ergo then keeps serving a self-signed cert until certsync catches up).
# 6697/tcp (IRC over TLS) is opened on the host firewall as well.
packages:
- git
runcmd:
- hostnamectl set-hostname irc || true
- |
set -e
REPO_URL=https://git.anomalous.dev/57_Wolve/automations.git
REPO_BRANCH=main
HARDEN_SSH=1 # harden SSH on this fresh VM (set 0 to skip)
SSH_PORT=22
ALLOWED_IP= # optional: whitelist your client IP in sshguard
git clone --depth 1 --branch "$REPO_BRANCH" "$REPO_URL" /opt/automations
cd /opt/automations
# Harden SSH first (PQ KEX, key-only auth, sshguard + deny-by-default
# firewall). Because the Ergo stack uses host networking, that firewall
# genuinely governs 80/443/6697 -- deploy.sh registers them with it.
if [ "$HARDEN_SSH" = 1 ]; then
SSH_PORT="$SSH_PORT" ALLOWED_IP="$ALLOWED_IP" SKIP_PROMPTS=1 FORCE=1 \
bash scripts/harden-ssh.sh
fi
# Deploy Ergo. The admin oper password is generated and stored in
# /srv/ergo/secrets/admin.pass (also printed to this log once).
ERGO_DOMAIN=irc.example.com \
ACME_EMAIL=admin@example.com \
NETWORK_NAME=ExampleNet \
PLAINTEXT=0 \
UPDATE_POLICY=latest \
SKIP_PROMPTS=1 \
bash deployments/ergo/deploy.sh
+20
View File
@@ -0,0 +1,20 @@
# Operator drop-ins for the Ergo stack's Caddy site block.
#
# Every conf.d/*.caddy file is imported INSIDE the {$ERGO_DOMAIN} site block, after
# the /webirc websocket route and before the text landing page. deploy.sh installs
# this readme once and never touches conf.d/ again. Apply with:
#
# docker compose restart caddy (or: ergoctl caddy-restart)
#
# Example -- serve the Gamja web client (https://codeberg.org/emersion/gamja) from
# ./caddy/www/gamja (unpack a release tarball there and write a config.json with
# {"server": {"url": "/webirc", "autojoin": "#lobby"}}):
#
# handle {
# root * /www/gamja
# file_server
# }
#
# ...and add `- ./caddy/www:/www:ro` to the caddy service volumes in
# docker-compose.yml. Gamja's Origin (https://{$ERGO_DOMAIN}) already matches
# server.websockets.allowed-origins in ircd.yaml.
File diff suppressed because it is too large Load Diff
+80
View File
@@ -0,0 +1,80 @@
# Ergo stack -- Caddy (Let's Encrypt + WebSocket front) + Ergo IRC server.
#
# Topology -- BOTH services run in the HOST network namespace (no Docker NAT):
#
# IRC clients --> :6697 TLS ----------------------> ergo (sees the real IPv4/IPv6)
# web clients --> :443 caddy --/webirc--> 127.0.0.1:8097 ergo websocket (X-Forwarded-For)
# Let's Encrypt --> :80/:443 caddy (ACME); the issued cert is copied into ./ircd for
# Ergo's :6697 listener by `update.sh certsync` (+ SIGHUP rehash)
# loopback --> 127.0.0.1:6667 plaintext: compose healthcheck + `ergoctl` ONLY
# (loopback is exempt from ip-limits/bans and counts as secure)
#
# Why host networking: IRC bans, throttling and IP cloaking key on the client's
# address, and Docker's userland proxy would hide every IPv6 client behind the
# bridge gateway. Host mode also means the host's deny-by-default INPUT firewall
# really governs these ports (deploy.sh registers 80/443/6697). Consequences:
# no `ports:`/`networks:` here (Compose rejects them in host mode), Caddy reaches
# Ergo over 127.0.0.1, and the host's loopback is Ergo's trust boundary -- see the
# README ("Security model").
name: ergo
services:
# ---------------------------------------------------------------------------
# Caddy -- obtains/renews the Let's Encrypt cert, terminates HTTPS for the
# IRC-over-WebSocket path and serves a text landing page. Static config:
# apply Caddyfile edits with `docker compose restart caddy` (admin API is off).
# ---------------------------------------------------------------------------
caddy:
image: caddy:${CADDY_TAG:-2-alpine}
container_name: ergo-caddy
restart: unless-stopped
network_mode: host
cap_drop: [ALL]
cap_add: [NET_BIND_SERVICE] # :80/:443 (root in-container needs only this)
security_opt: [no-new-privileges:true]
volumes:
- ./caddy/etc:/etc/caddy # Caddyfile + conf.d/ (directory mount, not a file)
- ./caddy/data:/data # ACME account + certs; host-readable for certsync
- ./caddy/config:/config
environment:
ERGO_DOMAIN: "${ERGO_DOMAIN}"
ACME_EMAIL: "${ACME_EMAIL}"
NETWORK_NAME: "${NETWORK_NAME}"
healthcheck:
# Loopback-only health site defined in the Caddyfile (the admin API is off).
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1/ | grep -q '^ok'"]
interval: 30s
timeout: 5s
retries: 3
start_period: 10s
# ---------------------------------------------------------------------------
# Ergo -- the IRC server. Runs as the host's `ergo` system user (deploy.sh
# creates it and chowns ./ircd), read-only rootfs, no capabilities. Every
# file it needs lives in ./ircd: ircd.yaml, ircd.db, ergo.motd, fullchain.pem,
# privkey.pem. `init: true` puts tini at PID 1 so `docker compose kill -s HUP`
# (rehash: reloads config, MOTD and TLS certs) reaches the ergo process.
# ---------------------------------------------------------------------------
ergo:
image: ${ERGO_IMAGE:-ghcr.io/ergochat/ergo}:${ERGO_TAG:-stable}
container_name: ergo
restart: unless-stopped
network_mode: host
init: true
user: "${ERGO_UID:-1000}:${ERGO_GID:-1000}"
cap_drop: [ALL]
security_opt: [no-new-privileges:true]
read_only: true
tmpfs: [/tmp]
volumes:
- ./ircd:/ircd
healthcheck:
# IRC-level probe: Ergo answers a pre-registration QUIT with "ERROR :Quit"
# and closes. A bare TCP connect would pass even with a wedged server.
# busybox nc exits 0 after a timeout, hence the grep.
test: ["CMD-SHELL", "printf 'QUIT\r\n' | nc -w 3 127.0.0.1 6667 | grep -q '^ERROR'"]
interval: 60s
timeout: 6s
retries: 3
start_period: 20s
+11
View File
@@ -0,0 +1,11 @@
$b__NETWORK_NAME__$r -- __DOMAIN__
Welcome! This network runs Ergo (https://ergo.chat).
Register your nick: /msg NickServ REGISTER <password>
Log in on connect: SASL PLAIN, or /msg NickServ IDENTIFY <password>
Register a channel: /msg ChanServ REGISTER #channel
Help: /msg NickServ HELP and /msg ChanServ HELP
Be kind to each other. Operators are listed in /ADMIN.
(Edit this file with: ergoctl motd)
+402
View File
@@ -0,0 +1,402 @@
#!/usr/bin/env bash
#
# ergoctl -- administer the Ergo IRC stack from the host.
#
# Installed by deploy.sh as $STACK_DIR/ergoctl with a wrapper at
# /usr/local/bin/ergoctl. Talks to Ergo over the loopback plaintext listener
# (127.0.0.1:6667) as the 'admin' oper, using the password in secrets/admin.pass.
# Config changes are validated in a throwaway container (ergo run --smoke on a
# copy) and applied with REHASH; on failure the previous file is restored.
#
# Run `ergoctl help` for the command list.
set -euo pipefail
SELF_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
log() { printf '\033[1;32m[+]\033[0m %s\n' "$*"; }
warn() { printf '\033[1;33m[!]\033[0m %s\n' "$*" >&2; }
die() { printf '\033[1;31m[x]\033[0m %s\n' "$*" >&2; exit 1; }
# shellcheck source=ergolib.sh
. "$SELF_DIR/ergolib.sh"
load_conf
: "${STACK_DIR:=$SELF_DIR}"
ergo_set_paths
CONF="$IRCD_DIR/ircd.yaml"
MOTD="$IRCD_DIR/ergo.motd"
usage() {
cat <<EOF
ergoctl -- administer the Ergo IRC server (stack: $STACK_DIR)
Status & logs
status containers, versions, TLS cert, user count, schedule
users LUSERS summary
logs [-f] [N] ergo log (health-probe noise filtered; default last 100)
version running and pinned version, latest release
cert [show|sync] cert on disk vs served on :6697; 'sync' pulls Caddy's now
Configuration (validated, then REHASH; reverted on failure)
edit \$EDITOR ircd/ircd.yaml
motd \$EDITOR ircd/ergo.motd
rehash reload config, MOTD and TLS certs (no disconnects)
restart | stop | start the ergo container (restart drops every user)
caddy-restart apply Caddyfile / conf.d changes
debug on|off switch the log level (info <-> debug)
Operators
oper list
oper add <name> [class] class: chat-moderator (default) | server-admin; prints the password once
oper passwd <name> new random password for an oper
oper rm <name>
passwd [show|rotate] the 'admin' oper password (secrets/admin.pass)
Moderation (as the admin oper)
announce <text> NOTICE to all users
kill <nick> [reason]
ban add <ip|cidr|nick!user@host|account> [duration] [reason] e.g. ban add 203.0.113.9 1d spam
(a bare name is treated as an ACCOUNT to suspend, not a
connected nick -- use 'ban info <nick>' to get their IP first)
ban del <target> | ban list | ban info <target>
defcon [1-5] show or set the DEFCON level
cmd <raw IRC line> anything else, e.g. cmd NS SAREGISTER alice hunter2
(replies are printed; control characters stripped)
Data
backup [--live] [dir] tar.gz of ircd.yaml, ircd.db, ergo.motd (default: stop Ergo
briefly for a consistent copy; --live = crash-consistent).
Encrypted with age when $STACK_DIR/age-recipients.txt exists.
restore <file> restore a backup (stops Ergo, restarts, waits for health)
Updates (update.sh)
update [check|update|run|certsync|caddy|install|uninstall]
shell sh inside the ergo container
EOF
}
require_root() { [[ $EUID -eq 0 ]] || die "Run as root."; }
require_stack() { [[ -f "$STACK_DIR/docker-compose.yml" && -f "$CONF" ]] || die "No deployed stack at $STACK_DIR (set STACK_DIR)."; }
# Validate the current ./ircd with the running image; on failure print the log.
validate_or_fail() { # <what>
if ergo_validate_config; then rm -f "$VALIDATE_LOG"; return 0; fi
warn "$1 did not load in $(ergo_image):"
tail -n 15 "$VALIDATE_LOG" >&2 || true
rm -f "$VALIDATE_LOG"
return 1
}
# Apply a config change: validate, then REHASH; revert the file(s) on failure.
# $1 = description; $2.. = files that were changed (each has a .last-good copy).
apply_change() {
local what="$1"; shift
local f rc=0
if ! validate_or_fail "$what"; then
for f in "$@"; do [[ -f "$f.last-good" ]] && cat "$f.last-good" > "$f"; done
die "$what rejected; previous version restored."
fi
if ! ergo_running; then
log "$what saved (Ergo is not running; it will load on start)."
for f in "$@"; do rm -f "$f.last-good"; done
return 0
fi
# Take the status from the call itself: `$?` after an `if` whose condition
# failed and that has no else branch is 0, which would make the SIGHUP
# fallback below unreachable.
irc_rehash || rc=$?
if (( rc == 0 )); then
log "$what applied (rehash complete)."
for f in "$@"; do rm -f "$f.last-good"; done
return 0
fi
# Ergo refuses to rehash settings that are fixed at startup. It phrases them
# all as "... after launching the server", so match that, not one variant.
if [[ "$REHASH_MSG" == *"after launching the server"* || "$REHASH_MSG" == *"cannot be changed after launching"* ]]; then
warn "Rehash refused: $REHASH_MSG"
warn "The file is valid and saved; apply it with: ergoctl restart (disconnects everyone)"
for f in "$@"; do rm -f "$f.last-good"; done
return 0
fi
if (( rc == 2 )); then
warn "Could not REHASH over IRC ($REHASH_MSG); sending SIGHUP instead."
dc kill -s HUP ergo >/dev/null 2>&1 || true
sleep 2
if dc logs --since 15s ergo 2>/dev/null | grep -qiE 'rehash.*(complete|success)'; then
log "$what applied (rehash via SIGHUP)."; for f in "$@"; do rm -f "$f.last-good"; done; return 0
fi
fi
for f in "$@"; do [[ -f "$f.last-good" ]] && cat "$f.last-good" > "$f"; done
die "Rehash failed: ${REHASH_MSG}. Previous version restored (Ergo kept running on the old config)."
}
snapshot_file() { cp -p "$1" "$1.last-good"; }
edit_file() { # <file> <what>
local f="$1" what="$2" tmp
require_root; require_stack
tmp="$(mktemp)"; cat "$f" > "$tmp"
"${EDITOR:-vi}" "$tmp"
if cmp -s "$f" "$tmp"; then rm -f "$tmp"; log "No changes."; return 0; fi
snapshot_file "$f"
cat "$tmp" > "$f"; rm -f "$tmp"
apply_change "$what" "$f"
}
oper_name_ok() { [[ "$1" =~ ^[a-z][a-z0-9_-]{1,31}$ ]]; }
cmd_status() {
require_stack
echo "Stack: $STACK_DIR (domain $(ergo_domain), network $(env_get NETWORK_NAME))"
dc ps 2>/dev/null || true
echo
local st cur latest
st="$(svc_state ergo || true)"; cur="$(ergo_version_running || true)"
latest="$(normver "$(ergo_latest_tag || true)")"
printf 'Ergo: %s | running %s | pinned %s | latest %s\n' "${st:-not created}" "${cur:-?}" "$(ergo_tag)" "${latest:-?}"
if [[ "$(env_get PLAINTEXT)" == "1" ]]; then echo " PUBLIC PLAINTEXT LISTENER on :6667 (PLAINTEXT=1)"; fi
cmd_cert show
if ergo_running; then
echo; cmd_users || true
fi
echo
printf 'Policy: %s | schedule: ' "${UPDATE_POLICY:-$(env_get UPDATE_POLICY)}"
if [[ -x /etc/periodic/15min/ergo-certsync || -f /etc/systemd/system/ergo-certsync.timer ]]; then printf 'certsync 15min'; else printf 'certsync MISSING'; fi
if [[ -x /etc/periodic/daily/ergo-update || -f /etc/systemd/system/ergo-update.timer ]]; then printf ', update daily\n'; else printf ', update NOT scheduled\n'; fi
local s; s="$(state_get certsync)"; [[ -z "$s" ]] || echo "certsync: LAST FAILURE: $s"
}
cmd_users() {
local out
out="$(irc_raw --quiet 1 -- LUSERS)" || die "Could not talk to Ergo on 127.0.0.1:6667 (is it running?)."
# ":server 251 nick :There are N users..." -> "There are N users..."
printf '%s\n' "$out" | awk '$2=="251"||$2=="252"||$2=="254"||$2=="265"||$2=="266" {
sub(/^:[^ ]+ [0-9]+ [^ ]+ /, ""); sub(/^:/, ""); sub(/ :/, " "); print " " $0 }'
}
cmd_logs() {
require_stack
local follow="" n=100
while [[ $# -gt 0 ]]; do
case "$1" in -f|--follow) follow="-f" ;; *[0-9]*) n="$1" ;; esac; shift
done
# Each healthcheck probe logs a 'Client connecting: real IP 127.0.0.1' line; hide those.
dc logs --tail "$n" $follow ergo 2>&1 | grep -vE 'real IP 127\.0\.0\.1, proxied IP <nil>|Preexisting TLS cert' || true
}
cmd_cert() {
require_stack
local f="$IRCD_DIR/fullchain.pem" served
case "${1:-show}" in
sync) require_root; bash "$STACK_DIR/update.sh" certsync ;;
*)
if [[ ! -f "$f" ]]; then echo "TLS: no cert in ircd/ yet"; return 0; fi
if cert_is_selfsigned "$f"; then
echo "TLS: SELF-SIGNED (Caddy has not issued a cert yet -- DNS/80/443?) expires $(cert_enddate "$f")"
else
echo "TLS: $(cert_subject_cn "$f") expires $(cert_enddate "$f") (from Caddy)"
fi
if ergo_running; then
served="$(served_fingerprint || true)"
if [[ -z "$served" ]]; then echo " :6697 did not complete a TLS handshake"
elif [[ "$served" == "$(cert_fingerprint "$f")" ]]; then echo " :6697 serves this cert"
else echo " :6697 serves a DIFFERENT cert -- run: ergoctl rehash"; fi
fi ;;
esac
}
cmd_oper() {
require_root; require_stack
local sub="${1:-list}" name="${2:-}" class="${3:-chat-moderator}" pw hash
case "$sub" in
list) yaml_oper_list "$CONF" | awk '{printf " %-20s %s\n", $1, $2}' ;;
add)
[[ -n "$name" ]] || die "Usage: ergoctl oper add <name> [chat-moderator|server-admin]"
oper_name_ok "$name" || die "Oper names: lowercase letters, digits, _ - (2-32 chars)."
yaml_oper_has "$CONF" "$name" && die "Oper '$name' already exists."
grep -qE "^ \"?${class}\"?:" "$CONF" || warn "Class '$class' is not defined under oper-classes in ircd.yaml -- the rehash will fail unless you add it."
pw="$(ergo_random_password 24)"
hash="$(ergo_genpasswd "$(ergo_image)" "$pw")" || die "genpasswd failed."
snapshot_file "$CONF"
yaml_oper_add "$CONF" "$name" "$class" "$hash" || { cat "$CONF.last-good" > "$CONF"; die "Could not insert the oper block."; }
apply_change "oper '$name'" "$CONF"
echo
echo " Oper '$name' ($class) added. Log in with: /OPER $name $pw"
echo " (shown once; rotate with: ergoctl oper passwd $name)" ;;
passwd)
[[ -n "$name" ]] || die "Usage: ergoctl oper passwd <name>"
yaml_oper_has "$CONF" "$name" || die "No oper '$name' in ircd.yaml."
pw="$(ergo_random_password 24)"
hash="$(ergo_genpasswd "$(ergo_image)" "$pw")" || die "genpasswd failed."
snapshot_file "$CONF"
yaml_oper_set_password "$CONF" "$name" "$hash" || { cat "$CONF.last-good" > "$CONF"; die "Oper '$name' has no password line to replace."; }
apply_change "oper '$name' password" "$CONF"
if [[ "$name" == "admin" ]]; then ( umask 077; printf '%s\n' "$pw" > "$ADMIN_PASS_FILE" ); fi
echo
echo " New password for '$name': /OPER $name $pw" ;;
rm|remove|del)
[[ -n "$name" ]] || die "Usage: ergoctl oper rm <name>"
[[ "$name" != "admin" ]] || die "Refusing to remove 'admin' (ergoctl uses it). Rotate its password instead."
yaml_oper_has "$CONF" "$name" || die "No oper '$name' in ircd.yaml."
snapshot_file "$CONF"
yaml_oper_rm "$CONF" "$name" || { cat "$CONF.last-good" > "$CONF"; die "Could not remove the oper block."; }
apply_change "removal of oper '$name'" "$CONF" ;;
*) die "Usage: ergoctl oper list|add|passwd|rm" ;;
esac
}
cmd_passwd() {
require_root; require_stack
case "${1:-show}" in
show) [[ -f "$ADMIN_PASS_FILE" ]] || die "No $ADMIN_PASS_FILE -- run: ergoctl passwd rotate"
echo "/OPER admin $(head -n1 "$ADMIN_PASS_FILE")" ;;
rotate) cmd_oper passwd admin
if irc_raw --oper --quiet 1 -- >/dev/null 2>&1; then log "Verified: the new admin password works."; else warn "Could not verify the new password over IRC (is Ergo running?)."; fi ;;
*) die "Usage: ergoctl passwd [show|rotate]" ;;
esac
}
cmd_cmd() { # raw line as oper
require_root; require_stack
[[ $# -gt 0 ]] || die "Usage: ergoctl cmd <raw IRC line>"
local out rc=0
out="$(irc_raw --oper --quiet 2 -- "$*")" || rc=$?
case "$rc" in
0) ;;
3) die "OPER failed (wrong admin password? see secrets/admin.pass; the server logs details under type 'opers')." ;;
4) die "Cannot connect to 127.0.0.1:6667 -- is Ergo running?" ;;
*) die "Registration with the server failed (rc=$rc)." ;;
esac
# Drop the routine registration burst and show only what the command produced.
# The MODE filter is anchored to our own throwaway nick so that MODE replies
# caused by the command itself (e.g. SAMODE #chan +m) still print.
printf '%s\n' "$out" | grep -vE '^:[^ ]+ (00[1-5]|25[0-9]|26[56]|37[256]|422|381) |^:[^ ]+ MODE ergoctl[0-9]+ ' || true
}
cmd_backup() {
require_root; require_stack
local live=0 dest="$BACKUP_DIR" ts stage out was_running=0 users=""
while [[ $# -gt 0 ]]; do case "$1" in --live) live=1 ;; *) dest="$1" ;; esac; shift; done
install -d -m 0700 "$dest"
# A recipients file means "encrypt these" -- refuse rather than silently
# writing account hashes and the oper hash out in the clear.
if [[ -f "$STACK_DIR/age-recipients.txt" ]] && ! command -v age >/dev/null 2>&1; then
die "$STACK_DIR/age-recipients.txt exists but 'age' is not installed, so the backup would be UNENCRYPTED. Install it (apk add age / apt install age / dnf install age) or move the recipients file away."
fi
ts="$(date -u +%Y%m%d-%H%M%S)"
stage="$(mktemp -d)"
if ergo_running; then
was_running=1
if [[ "$live" == 0 ]]; then
users="$(irc_raw --quiet 1 -- LUSERS 2>/dev/null | awk '$2=="251" {print}' | sed 's/.*:There are //' || true)"
log "Stopping Ergo for a consistent snapshot (${users:-users unknown})..."
irc_raw --oper --quiet 1 -- 'NOTICE $$* :Brief restart for a backup -- back in a few seconds.' >/dev/null 2>&1 || true
dc stop -t 15 ergo >/dev/null 2>&1 || true
else
warn "Live backup: ircd.db is copied while Ergo writes to it (crash-consistent; may miss the last second)."
fi
fi
cp -p "$CONF" "$stage/ircd.yaml"
[[ -f "$IRCD_DIR/ircd.db" ]] && cp -p "$IRCD_DIR/ircd.db" "$stage/ircd.db"
[[ -f "$MOTD" ]] && cp -p "$MOTD" "$stage/ergo.motd"
cat > "$stage/meta" <<EOF
ERGO_TAG=$(ergo_tag)
ERGO_DOMAIN=$(ergo_domain)
DATE=$(date -u +%FT%TZ)
HOST=$(hostname -f 2>/dev/null || hostname)
CONSISTENT=$(( live == 0 ))
EOF
if [[ "$was_running" == 1 && "$live" == 0 ]]; then dc start ergo >/dev/null 2>&1 || true; fi
out="$dest/ergo-backup-${ts}.tar.gz"
( umask 077; tar -czf "$out" -C "$stage" . )
rm -rf "$stage"
if [[ -f "$STACK_DIR/age-recipients.txt" ]]; then
age -R "$STACK_DIR/age-recipients.txt" -o "$out.age" "$out" || { rm -f "$out.age"; die "age encryption failed; the plaintext tar is still at $out."; }
rm -f "$out"; out="$out.age"
fi
chmod 0600 "$out"
log "Backup written: $out ($(wc -c < "$out") bytes). Copy it off this host."
[[ "$out" == *.age ]] || warn "Unencrypted: it holds account hashes and the oper hash. Put an age public key in $STACK_DIR/age-recipients.txt (and install 'age') to encrypt future backups."
}
cmd_restore() {
require_root; require_stack
local file="${1:-}" stage tag
[[ -f "$file" ]] || die "Usage: ergoctl restore <backup.tar.gz[.age]>"
stage="$(mktemp -d)"
if [[ "$file" == *.age ]]; then
command -v age >/dev/null 2>&1 || die "age is required to decrypt $file."
age -d -i "${AGE_IDENTITY:?set AGE_IDENTITY=/path/to/age/key}" "$file" | tar -xzf - -C "$stage"
else
tar -xzf "$file" -C "$stage"
fi
[[ -f "$stage/ircd.yaml" && -f "$stage/ircd.db" ]] || { rm -rf "$stage"; die "Backup lacks ircd.yaml/ircd.db."; }
tag="$(grep '^ERGO_TAG=' "$stage/meta" 2>/dev/null | cut -d= -f2 || true)"
if [[ -n "$tag" ]] && ver_gt "$(normver "$tag")" "$(normver "$(ergo_tag)")"; then
warn "Backup was taken with Ergo $tag but $(ergo_tag) is pinned -- an older Ergo cannot read a newer database schema. Update first (ergoctl update update) or pin ERGO_TAG=$tag."
read -r -p "Continue anyway? [y/N] " ans; [[ "${ans,,}" == y* ]] || { rm -rf "$stage"; die "Aborted."; }
fi
log "Stopping Ergo and restoring from $file..."
dc stop -t 15 ergo >/dev/null 2>&1 || true
install -d -m 0700 "$BACKUP_DIR"
[[ -f "$IRCD_DIR/ircd.db" ]] && cp -p "$IRCD_DIR/ircd.db" "$BACKUP_DIR/ircd.db.pre-restore.$(date -u +%Y%m%d%H%M%S)"
install -m 0600 -o "$(ergo_uid)" -g "$(ergo_gid)" "$stage/ircd.yaml" "$CONF"
install -m 0600 -o "$(ergo_uid)" -g "$(ergo_gid)" "$stage/ircd.db" "$IRCD_DIR/ircd.db"
[[ -f "$stage/ergo.motd" ]] && install -m 0644 -o "$(ergo_uid)" -g "$(ergo_gid)" "$stage/ergo.motd" "$MOTD"
rm -rf "$stage"
dc up -d --no-deps ergo >/dev/null 2>&1 || true
if wait_healthy ergo 120 && irc_probe; then log "Restored; Ergo is healthy."; else dc logs --tail 20 ergo >&2 || true; die "Ergo is not healthy after the restore."; fi
}
cmd_debug() {
require_root; require_stack
local level
case "${1:-}" in on) level=debug ;; off) level=info ;; *) die "Usage: ergoctl debug on|off" ;; esac
snapshot_file "$CONF"
yaml_set_log_level "$CONF" "$level" || { cat "$CONF.last-good" > "$CONF"; die "No 'level:' line found in the logging block."; }
apply_change "log level $level" "$CONF"
}
case "${1:-help}" in
help|-h|--help) usage ;;
status) cmd_status ;;
users) require_stack; cmd_users ;;
logs) shift; cmd_logs "$@" ;;
version) require_stack; printf 'running %s | pinned %s | latest %s\n' "$(ergo_version_running)" "$(ergo_tag)" "$(ergo_latest_tag || echo '?')" ;;
cert) shift; cmd_cert "$@" ;;
edit) edit_file "$CONF" "ircd.yaml" ;;
motd) edit_file "$MOTD" "MOTD" ;;
rehash) require_root; require_stack
if irc_rehash; then log "Rehash complete."; else
if [[ "$REHASH_MSG" == *"could not reach"* ]]; then warn "$REHASH_MSG -- sending SIGHUP."; dc kill -s HUP ergo; else die "Rehash failed: $REHASH_MSG"; fi
fi ;;
restart) require_root; require_stack; irc_raw --oper --quiet 1 -- 'NOTICE $$* :Server restarting now.' >/dev/null 2>&1 || true
dc restart -t 15 ergo; wait_healthy ergo 120 && log "Ergo is healthy." || die "Ergo did not become healthy (ergoctl logs)." ;;
stop) require_root; require_stack; dc stop -t 15 ergo ;;
start) require_root; require_stack; dc start ergo; wait_healthy ergo 120 && log "Ergo is healthy." || die "Ergo did not become healthy (ergoctl logs)." ;;
caddy-restart) require_root; require_stack
dc run --rm --no-deps -T caddy caddy validate --config /etc/caddy/Caddyfile --adapter caddyfile >/dev/null || die "Caddyfile invalid (see caddy/etc/Caddyfile, conf.d/)."
dc restart caddy; wait_healthy caddy 60 && log "Caddy is healthy." || die "Caddy did not become healthy." ;;
debug) shift; cmd_debug "$@" ;;
oper) shift; cmd_oper "$@" ;;
passwd) shift; cmd_passwd "$@" ;;
cmd) shift; cmd_cmd "$@" ;;
announce) shift; [[ $# -gt 0 ]] || die "Usage: ergoctl announce <text>"; cmd_cmd "NOTICE \$\$* :$*" ;;
kill) shift; [[ $# -ge 1 ]] || die "Usage: ergoctl kill <nick> [reason]"; n="$1"; shift; cmd_cmd "KILL $n :${*:-Killed by an operator}" ;;
ban) shift; sub="${1:-list}"; shift || true
case "$sub" in
add) [[ $# -ge 1 ]] || die "Usage: ergoctl ban add <ip|cidr|nick!user@host|account> [duration e.g. 1d] [reason] -- a bare name means an ACCOUNT; use 'ban info <nick>' for a connected user's IP"
t="$1"; shift; d=""; if [[ "${1:-}" =~ ^[0-9]+[ymwdhs]$|^[0-9]+mo$ ]]; then d="DURATION $1"; shift; fi
cmd_cmd "UBAN ADD $t $d ${*:+:$*}" ;;
del) [[ $# -ge 1 ]] || die "Usage: ergoctl ban del <target>"; cmd_cmd "UBAN DEL $1" ;;
info) [[ $# -ge 1 ]] || die "Usage: ergoctl ban info <target>"; cmd_cmd "UBAN INFO $1" ;;
list|*) cmd_cmd "UBAN LIST" ;;
esac ;;
defcon) shift; cmd_cmd "DEFCON ${1:-}" ;;
backup) shift; cmd_backup "$@" ;;
restore) shift; cmd_restore "$@" ;;
update) shift; require_root; require_stack; exec bash "$STACK_DIR/update.sh" "${1:-check}" ;;
shell) require_root; require_stack; dc exec ergo sh ;;
*) usage; exit 1 ;;
esac
+554
View File
@@ -0,0 +1,554 @@
#!/usr/bin/env bash
#
# ergolib.sh -- shared helpers for the Ergo stack. Sourced by deploy.sh (from the
# extracted archive), update.sh and ergoctl (from $STACK_DIR). Not run directly.
#
# Everything that touches the running stack lives here so the three scripts stay
# consistent: .env access, the compose wrapper, health + IRC probes over the
# loopback listener, the throwaway-container config validator, Caddy->Ergo cert
# sync, oper edits in ircd.yaml, and ntfy notifications.
#
# Conventions: functions return non-zero instead of exiting (callers decide),
# and everything is portable to busybox (Alpine) -- POSIX awk, no GNU date -d,
# no sort -V, no grep -P.
# ---------------------------------------------------------------------------
# Logging (reuse the caller's if defined)
# ---------------------------------------------------------------------------
if ! declare -f log >/dev/null 2>&1; then
log() { printf '\033[1;32m[+]\033[0m %s\n' "$*"; }
warn() { printf '\033[1;33m[!]\033[0m %s\n' "$*" >&2; }
die() { printf '\033[1;31m[x]\033[0m %s\n' "$*" >&2; exit 1; }
fi
# ---------------------------------------------------------------------------
# Config file (/etc/ergo-update.conf): KEY=VALUE defaults for the scheduled
# jobs. Environment wins over the file, the file over built-ins.
# ---------------------------------------------------------------------------
: "${ERGO_UPDATE_CONF:=/etc/ergo-update.conf}"
load_conf() {
[[ -r "$ERGO_UPDATE_CONF" ]] || return 0
local _line _k _v
while IFS= read -r _line || [[ -n "$_line" ]]; do
[[ "$_line" =~ ^[[:space:]]*# || -z "${_line//[[:space:]]/}" ]] && continue
_k="${_line%%=*}"; _v="${_line#*=}"; _k="${_k//[[:space:]]/}"
[[ "$_k" =~ ^[A-Za-z_][A-Za-z0-9_]*$ ]] || continue
[[ -n "${!_k:-}" ]] && continue
_v="${_v%\"}"; _v="${_v#\"}"
printf -v "$_k" '%s' "$_v"
done < "$ERGO_UPDATE_CONF"
return 0
}
# Derived paths. Call after STACK_DIR is final.
ergo_set_paths() {
: "${STACK_DIR:=/srv/ergo}"
ENV_FILE="$STACK_DIR/.env"
IRCD_DIR="$STACK_DIR/ircd"
CADDY_DIR="$STACK_DIR/caddy"
SECRETS_DIR="$STACK_DIR/secrets"
BACKUP_DIR="$STACK_DIR/backups"
TEMPLATES_DIR="$STACK_DIR/templates"
STATE_DIR="$STACK_DIR/.state"
ADMIN_PASS_FILE="$SECRETS_DIR/admin.pass"
: "${GH_REPO:=ergochat/ergo}"
: "${NOTIFY:=1}"
: "${SSH_NOTIFY_CONF:=/etc/ssh-notify.conf}"
: "${DRY_RUN:=0}"
}
osfam() {
local id="" like=""
if [[ -r /etc/os-release ]]; then
id="$(. /etc/os-release 2>/dev/null && echo "${ID:-}")"
like="$(. /etc/os-release 2>/dev/null && echo "${ID_LIKE:-}")"
fi
case " $id $like " in
*" alpine "*) echo alpine ;;
*" debian "*|*" ubuntu "*) echo debian ;;
*" rhel "*|*" fedora "*|*" centos "*) echo rhel ;;
*) echo "${id:-unknown}" ;;
esac
}
# fetch a URL to stdout (curl or wget); rc 1 on failure.
fetch() {
if command -v curl >/dev/null 2>&1; then curl -fsSL -m 20 "$1" 2>/dev/null
elif command -v wget >/dev/null 2>&1; then wget -qO- -T 20 "$1" 2>/dev/null
else return 1
fi
}
# ---------------------------------------------------------------------------
# .env access
# ---------------------------------------------------------------------------
env_get() { [[ -f "$ENV_FILE" ]] && grep -E "^$1=" "$ENV_FILE" | head -n1 | cut -d= -f2- || true; }
env_set() { # <KEY> <value>: update KEY in .env, or append if absent
local key="$1" val="$2" esc
esc=${val//\\/\\\\}; esc=${esc//|/\\|}; esc=${esc//&/\\&}
if grep -qE "^${key}=" "$ENV_FILE" 2>/dev/null; then
sed -i -e "s|^${key}=.*|${key}=${esc}|" "$ENV_FILE"
else
printf '%s=%s\n' "$key" "$val" >> "$ENV_FILE"
fi
}
ergo_domain() { env_get ERGO_DOMAIN | tr 'A-Z' 'a-z' || true; }
# Fill the named variables from .env when they are not already set in the
# environment. Used by update.sh/ergoctl so the knobs .env documents really are
# the ones the scheduled jobs obey (environment > /etc/ergo-update.conf > .env).
env_defaults() { # env_defaults <VAR>...
local k v
for k in "$@"; do
[[ -n "${!k:-}" ]] && continue
v="$(env_get "$k")"
[[ -n "$v" ]] && printf -v "$k" '%s' "$v"
done
return 0
}
ergo_uid() { local v; v="$(env_get ERGO_UID)"; printf '%s' "${v:-1000}"; }
ergo_gid() { local v; v="$(env_get ERGO_GID)"; printf '%s' "${v:-1000}"; }
ergo_tag() { local v; v="$(env_get ERGO_TAG)"; printf '%s' "${v:-stable}"; }
ergo_image() { local i; i="$(env_get ERGO_IMAGE)"; printf '%s:%s' "${i:-ghcr.io/ergochat/ergo}" "$(ergo_tag)"; }
# docker compose, scoped to the stack dir (so ./ircd, ./caddy resolve and .env
# is auto-loaded).
dc() { ( cd "$STACK_DIR" && docker compose "$@" ); }
# ---------------------------------------------------------------------------
# Version helpers (pure bash; busybox sort has no -V)
# ---------------------------------------------------------------------------
# NOTE the trailing `|| true` here and in the cert/state helpers below: these are
# assigned in `set -euo pipefail` scripts, where a no-match grep would abort the
# caller instead of yielding "". Callers test for an empty result.
normver() { printf '%s' "$1" | grep -oE '[0-9]+\.[0-9]+(\.[0-9]+)?' | head -n1 || true; }
ver_cmp() { # <a> <b> -> prints -1 / 0 / 1
local a b i x y
IFS=. read -r -a a <<< "$(normver "$1")"
IFS=. read -r -a b <<< "$(normver "$2")"
for i in 0 1 2; do
x="${a[$i]:-0}"; y="${b[$i]:-0}"
if (( x > y )); then echo 1; return; fi
if (( x < y )); then echo -1; return; fi
done
echo 0
}
ver_gt() { [[ "$(ver_cmp "$1" "$2")" == "1" ]]; }
ver_ge() { [[ "$(ver_cmp "$1" "$2")" != "-1" ]]; }
ver_le() { [[ "$(ver_cmp "$1" "$2")" != "1" ]]; }
# Running version as reported by the binary ('ergo-v2.19.1' or 'ergo-2.19.1[-hash]');
# falls back to the pinned tag when the container is down.
ergo_version_running() {
local v=""
if ergo_running; then
v="$(dc exec -T ergo /ircd-bin/ergo --version 2>/dev/null | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -n1 || true)"
fi
[[ -n "$v" ]] || v="$(normver "$(ergo_tag)")"
printf '%s' "$v"
}
# Newest non-prerelease tag from GitHub (e.g. v2.19.1). rc 1 if unreachable.
ergo_latest_tag() {
local t
t="$(fetch "https://api.github.com/repos/${GH_REPO}/releases/latest" \
| grep -oE '"tag_name"[[:space:]]*:[[:space:]]*"[^"]+"' | head -n1 \
| sed -E 's/.*"tag_name"[[:space:]]*:[[:space:]]*"([^"]+)".*/\1/')"
[[ -n "$t" ]] || return 1
printf '%s' "$t"
}
# ---------------------------------------------------------------------------
# Container state
# ---------------------------------------------------------------------------
# "<state> <health>" for one compose service, e.g. "running healthy", "exited ".
# Empty when the container does not exist or the docker daemon is unreachable.
svc_state() {
dc ps -a --format '{{.Service}} {{.State}} {{.Health}}' 2>/dev/null | awk -v s="$1" '$1==s {print $2" "$3; exit}' || true
}
ergo_running() { [[ "$(svc_state ergo)" == running* ]]; }
# Wait for a service to be "running healthy"; rc 1 on timeout. Exited/restarting
# containers never count (docker compose ps hides them without -a).
wait_healthy() { # <service> [timeout-seconds]
local svc="$1" deadline st
deadline=$(( $(date +%s) + ${2:-120} ))
while (( $(date +%s) < deadline )); do
st="$(svc_state "$svc")"
[[ "$st" == "running healthy" ]] && return 0
sleep 3
done
return 1
}
# ---------------------------------------------------------------------------
# Talking IRC to the loopback listener (127.0.0.1:6667) with bash /dev/tcp.
#
# irc_raw [--oper] [--quiet SECS] -- <line>...
#
# Registers a throwaway nick, optionally opers up as 'admin' (password from
# $ADMIN_PASS_FILE or $ERGO_OPER_PASS), sends each line, prints the server's
# replies (control characters stripped) until the server has been quiet for
# --quiet seconds, then QUITs. IRC_LAST holds the raw transcript, but only when
# irc_raw is called directly -- inside $(...) it stays in the subshell, so parse
# the printed output instead.
#
# rc: 0 ok | 3 OPER refused | 4 connect failed | 5 registration failed
# Loopback is exempt from ip-limits/bans, but fakelag applies until opered:
# NICK+USER+OPER is 3 of the 5-line burst, so keep pre-oper payloads short.
# ---------------------------------------------------------------------------
IRC_LAST=""
_irc_send() { printf '%s\r\n' "$1" >&3 2>/dev/null; }
_irc_cmd() { # echo the command/numeric token of a server line
local l="$1"
if [[ "$l" == :* ]]; then l="${l#* }"; fi
printf '%s' "${l%% *}"
}
irc_raw() {
local oper=0 quiet=2 line cmd nick tries=0 pass="" registered=0 opered=0 out=""
while [[ $# -gt 0 ]]; do
case "$1" in
--oper) oper=1; shift ;;
--quiet) quiet="$2"; shift 2 ;;
--) shift; break ;;
*) break ;;
esac
done
IRC_LAST=""
if [[ "$oper" == 1 ]]; then
pass="${ERGO_OPER_PASS:-}"
[[ -n "$pass" ]] || pass="$(head -n1 "$ADMIN_PASS_FILE" 2>/dev/null || true)"
[[ -n "$pass" ]] || { warn "No admin oper password (expected $ADMIN_PASS_FILE)."; return 3; }
fi
if ! { exec 3<>/dev/tcp/127.0.0.1/6667; } 2>/dev/null; then
return 4
fi
nick="ergoctl$RANDOM"
_irc_send "NICK $nick"
_irc_send "USER ergoctl 0 * :ergoctl"
while IFS= read -r -t 15 -u 3 line; do
line="${line%$'\r'}"
IRC_LAST+="$line"$'\n'
cmd="$(_irc_cmd "$line")"
case "$cmd" in
PING) _irc_send "PONG ${line#PING }" ;;
376|422) registered=1; break ;;
433) # nick in use -- pick another (up to 3 times)
tries=$((tries + 1)); (( tries > 3 )) && break
nick="ergoctl$RANDOM"; _irc_send "NICK $nick" ;;
432|464|465|ERROR|FAIL) break ;;
esac
done
if [[ "$registered" != 1 ]]; then
exec 3>&- 2>/dev/null; return 5
fi
if [[ "$oper" == 1 ]]; then
_irc_send "OPER admin $pass"
while IFS= read -r -t 15 -u 3 line; do
line="${line%$'\r'}"
IRC_LAST+="$line"$'\n'
cmd="$(_irc_cmd "$line")"
case "$cmd" in
PING) _irc_send "PONG ${line#PING }" ;;
381) opered=1; break ;;
491|464|ERROR) break ;;
esac
done
if [[ "$opered" != 1 ]]; then
_irc_send "QUIT :ergoctl"; exec 3>&- 2>/dev/null; return 3
fi
fi
for line in "$@"; do _irc_send "$line"; done
while IFS= read -r -t "$quiet" -u 3 line; do
line="${line%$'\r'}"
IRC_LAST+="$line"$'\n'
cmd="$(_irc_cmd "$line")"
case "$cmd" in
PING) _irc_send "PONG ${line#PING }"; continue ;;
ERROR) out+="$line"$'\n'; break ;;
esac
out+="$line"$'\n'
done
_irc_send "QUIT :ergoctl"
exec 3>&- 2>/dev/null
# Never let user-controlled IRC text drive the terminal: drop control chars
# (this also strips IRC formatting bytes 0x02/0x03/0x1d-0x1f).
[[ -n "$out" ]] && printf '%s' "$out" | tr -d '\000-\010\013\014\016-\037\177'
return 0
}
# Registration-only probe (no oper): proves the config loaded and the server
# completes a handshake. rc 0/1.
irc_probe() { irc_raw --quiet 1 -- >/dev/null 2>&1; }
# REHASH as the admin oper; sets REHASH_MSG. rc 0 = "Rehash complete",
# 1 = server refused (message in REHASH_MSG), 2 = could not connect/oper.
REHASH_MSG=""
irc_rehash() {
local out rc
REHASH_MSG=""
out="$(irc_raw --oper --quiet 3 -- REHASH)"; rc=$?
if (( rc != 0 )); then
REHASH_MSG="could not reach Ergo as oper (rc=$rc)"; return 2
fi
# $out holds every reply after the REHASH was sent (the $() subshell keeps
# IRC_LAST from us, so parse the printed transcript).
if printf '%s' "$out" | grep -qi 'Rehash complete'; then return 0; fi
REHASH_MSG="$(printf '%s\n' "$out" | awk '$2=="400" {sub(/^[^ ]+ 400 [^ ]+ REHASH :?/,""); print; exit}')"
[[ -n "$REHASH_MSG" ]] || REHASH_MSG="no confirmation from the server: $(printf '%s' "$out" | tail -n 3 | tr '\n' ' ')"
return 1
}
# ---------------------------------------------------------------------------
# Helper containers: never on a network, never privileged, run as the ergo uid.
# ---------------------------------------------------------------------------
ergo_helper() { # ergo_helper [docker-run-opts...] -- <image> <args...>
local opts=()
while [[ $# -gt 0 && "$1" != "--" ]]; do opts+=("$1"); shift; done
[[ "${1:-}" == "--" ]] && shift
docker run --rm --network none --cap-drop ALL --security-opt no-new-privileges \
--user "$(ergo_uid):$(ergo_gid)" "${opts[@]}" "$@"
}
# The image's own default config (already patched by Ergo's Dockerfile: public
# ":6667", no [::1]). Writes to <dest>; rc 1 if nothing came out.
ergo_extract_template() { # <image> <dest>
local image="$1" dest="$2"
ergo_helper --entrypoint cat -- "$image" /ircd-bin/default.yaml > "$dest" 2>/dev/null || true
if ! grep -q '^server:' "$dest" 2>/dev/null; then
ergo_helper --entrypoint /ircd-bin/ergo -- "$image" defaultconfig > "$dest" 2>/dev/null || true
fi
grep -q '^server:' "$dest" 2>/dev/null
}
# bcrypt hash for a password (stdin -> stdout, one line). rc 1 if it doesn't
# look like a bcrypt hash.
ergo_genpasswd() { # <image> <password>
local h
h="$(printf '%s\n' "$2" | ergo_helper -i --entrypoint /ircd-bin/ergo -- "$1" genpasswd 2>/dev/null | tr -d '\r\n')"
[[ "$h" == \$2a\$* || "$h" == \$2b\$* ]] || return 1
printf '%s' "$h"
}
# Random password from [A-Za-z0-9_-]: never trips Ergo's passphrase checks.
ergo_random_password() { head -c 256 /dev/urandom | tr -dc '_A-Za-z0-9-' | head -c "${1:-24}"; }
# ---------------------------------------------------------------------------
# Config validation: `ergo run --smoke` in a throwaway container against a COPY
# of ./ircd. Its own network namespace means the wildcard listeners bind fine,
# the copied ircd.db/lock are disposable, and a pending schema upgrade is
# dry-run on the copy. rc 0 = config (and certs) load; output in $VALIDATE_LOG.
# Ergo has no offline config checker and --smoke on the live dir would fight
# the running server for the flock and the ports.
# ---------------------------------------------------------------------------
VALIDATE_LOG=""
ergo_validate_config() { # [image]
local image="${1:-$(ergo_image)}" tmp rc
tmp="$(mktemp -d "$STACK_DIR/.validate.XXXXXX")" || return 1
VALIDATE_LOG="$(mktemp)"
if ! cp -a "$IRCD_DIR"/. "$tmp"/ 2>>"$VALIDATE_LOG"; then rm -rf "$tmp"; return 1; fi
rm -f "$tmp/ircd.lock"
chown -R "$(ergo_uid):$(ergo_gid)" "$tmp"
ergo_helper -v "$tmp:/ircd" -w /ircd --entrypoint /bin/sh -- "$image" \
-c '/ircd-bin/ergo mkcerts >/dev/null 2>&1; exec /ircd-bin/ergo run --smoke' >>"$VALIDATE_LOG" 2>&1
rc=$?
rm -rf "$tmp"
return $rc
}
# ---------------------------------------------------------------------------
# ircd.yaml oper block edits (POSIX awk). The opers block is
# opers:
# <name>:
# class: ...
# password: "..."
# Each helper rewrites in place via a temp file and verifies the result.
# ---------------------------------------------------------------------------
yaml_server_name() {
awk '/^[a-z]/ {top=$1} top=="server:" && /^ name:/ {sub(/^ name:[ \t]*/,""); gsub(/["\047]/,""); print; exit}' "$1"
}
yaml_oper_list() { # <file> -> "name class" per oper
awk '
/^[a-z]/ { top=$1 }
top=="opers:" && /^ [A-Za-z0-9_.-]+:/ { if (name!="") print name, cls; name=$1; sub(/:$/,"",name); cls="?" }
top=="opers:" && name!="" && /^ class:/ { cls=$2; gsub(/"/,"",cls) }
END { if (name!="") print name, cls }' "$1"
}
yaml_oper_has() { yaml_oper_list "$1" | awk -v n="$2" '$1==n {f=1} END {exit !f}'; }
yaml_oper_set_password() { # <file> <name> <hash>
local tmp; tmp="$(mktemp)"
NAME="$2" HASH="$3" awk '
/^[a-z]/ { top=$1; cur="" }
top=="opers:" && /^ [A-Za-z0-9_.-]+:/ { cur=$1; sub(/:$/,"",cur) }
top=="opers:" && cur==ENVIRON["NAME"] && /^ password:/ { print " password: \"" ENVIRON["HASH"] "\""; done=1; next }
{ print }
END { exit done ? 0 : 1 }' "$1" > "$tmp" || { rm -f "$tmp"; return 1; }
cat "$tmp" > "$1"; rm -f "$tmp"
}
yaml_oper_add() { # <file> <name> <class> <hash> [whois-line]
local tmp; tmp="$(mktemp)"
local block
block=" $2:
class: \"$3\"
hidden: true
whois-line: ${5:-is a network operator}
password: \"$4\"
"
BLOCK="$block" awk '
/^[a-z]/ { if (inopers && !done) { printf "%s\n", ENVIRON["BLOCK"]; done=1 } inopers=($1=="opers:") }
{ print }
END { if (inopers && !done) printf "%s", ENVIRON["BLOCK"] }' "$1" > "$tmp" || { rm -f "$tmp"; return 1; }
cat "$tmp" > "$1"; rm -f "$tmp"
yaml_oper_has "$1" "$2"
}
yaml_oper_rm() { # <file> <name>
local tmp; tmp="$(mktemp)"
NAME="$2" awk '
/^[a-z]/ { top=$1; skip=0 }
top=="opers:" && /^ [A-Za-z0-9_.-]+:/ { n=$1; sub(/:$/,"",n); skip=(n==ENVIRON["NAME"]); if (skip) next }
top=="opers:" && skip && (/^ / || /^[ \t]*$/) { next }
top=="opers:" && skip { skip=0 }
{ print }' "$1" > "$tmp" || { rm -f "$tmp"; return 1; }
cat "$tmp" > "$1"; rm -f "$tmp"
! yaml_oper_has "$1" "$2"
}
# The first logging block's level (info <-> debug), for `ergoctl debug on|off`.
yaml_set_log_level() { # <file> <level>
local tmp; tmp="$(mktemp)"
LEVEL="$2" awk '
/^[a-z]/ { top=$1 }
top=="logging:" && !done && /^ level:/ { print " level: " ENVIRON["LEVEL"]; done=1; next }
{ print }
END { exit done ? 0 : 1 }' "$1" > "$tmp" || { rm -f "$tmp"; return 1; }
cat "$tmp" > "$1"; rm -f "$tmp"
}
# ---------------------------------------------------------------------------
# TLS: Caddy's cert -> ./ircd/{fullchain,privkey}.pem -> SIGHUP (rehash)
# ---------------------------------------------------------------------------
# Newest <domain>.crt across issuer dirs (Let's Encrypt, ZeroSSL fallback, ...):
# certmagic writes .key, then .crt, then .json per issuance, each atomically.
ergo_cert_source() {
local d best="" f
d="$(ergo_domain)"
for f in "$CADDY_DIR"/data/caddy/certificates/*/"$d"/"$d".crt; do
[[ -f "$f" ]] || continue
if [[ -z "$best" || "$f" -nt "$best" ]]; then best="$f"; fi
done
[[ -n "$best" ]] || return 1
printf '%s' "$best"
}
cert_fingerprint() { openssl x509 -in "$1" -noout -fingerprint -sha256 2>/dev/null | cut -d= -f2 || true; }
cert_enddate() { openssl x509 -in "$1" -noout -enddate 2>/dev/null | cut -d= -f2 || true; }
cert_subject_cn() { openssl x509 -in "$1" -noout -subject 2>/dev/null | sed -E 's/.*CN ?= ?([^,\/]+).*/\1/' || true; }
cert_expires_within() { ! openssl x509 -in "$1" -noout -checkend "$2" >/dev/null 2>&1; } # <file> <seconds>
cert_is_selfsigned() {
[[ "$(openssl x509 -in "$1" -noout -issuer 2>/dev/null | cut -d= -f2-)" == "$(openssl x509 -in "$1" -noout -subject 2>/dev/null | cut -d= -f2-)" ]]
}
served_fingerprint() { # what :6697 currently presents ("" if the handshake fails)
local d; d="$(ergo_domain)"
timeout 10 openssl s_client -connect 127.0.0.1:6697 -servername "$d" </dev/null 2>/dev/null \
| openssl x509 -noout -fingerprint -sha256 2>/dev/null | cut -d= -f2 || true
}
# Sync Caddy's cert into ./ircd if it changed. Pairwise + atomic: both files are
# verified to match (pubkey) and staged on the same filesystem before two mv's,
# so Ergo can never see a half-renewed pair (which is fatal at the next start).
# rc: 0 synced or nothing to do | 2 no cert from Caddy yet | 1 failed (caller notifies)
CERTSYNC_MSG=""
ergo_certsync() {
local src_crt src_key src_json dst_crt dst_key tmpc tmpk want got i uid gid
CERTSYNC_MSG=""
command -v openssl >/dev/null 2>&1 || { CERTSYNC_MSG="openssl missing on the host"; return 1; }
src_crt="$(ergo_cert_source)" || { CERTSYNC_MSG="no certificate from Caddy yet"; return 2; }
src_key="${src_crt%.crt}.key"; src_json="${src_crt%.crt}.json"
[[ -f "$src_key" ]] || { CERTSYNC_MSG="renewal in progress (key missing)"; return 0; }
# .json is written last: if it is older than .crt/.key a renewal is mid-flight.
if [[ -f "$src_json" ]] && { [[ "$src_crt" -nt "$src_json" ]] || [[ "$src_key" -nt "$src_json" ]]; }; then
CERTSYNC_MSG="renewal in progress; retrying later"; return 0
fi
dst_crt="$IRCD_DIR/fullchain.pem"; dst_key="$IRCD_DIR/privkey.pem"
if [[ -f "$dst_crt" && -f "$dst_key" ]] && cmp -s "$src_crt" "$dst_crt" && cmp -s "$src_key" "$dst_key"; then
CERTSYNC_MSG="up to date"; return 0
fi
# cert and key must belong together
want="$(openssl x509 -in "$src_crt" -noout -pubkey 2>/dev/null | sha256sum | cut -d' ' -f1)"
got="$(openssl pkey -in "$src_key" -pubout 2>/dev/null | sha256sum | cut -d' ' -f1)"
if [[ -z "$want" || "$want" != "$got" ]]; then
CERTSYNC_MSG="cert/key mismatch in Caddy storage (renewal in progress?); retrying later"; return 0
fi
uid="$(ergo_uid)"; gid="$(ergo_gid)"
tmpc="$IRCD_DIR/.fullchain.pem.new"; tmpk="$IRCD_DIR/.privkey.pem.new"
if ! install -m 0600 -o "$uid" -g "$gid" "$src_crt" "$tmpc" || ! install -m 0600 -o "$uid" -g "$gid" "$src_key" "$tmpk"; then
rm -f "$tmpc" "$tmpk"; CERTSYNC_MSG="could not stage the new pair in $IRCD_DIR"; return 1
fi
# source must not have moved under us mid-copy
if ! cmp -s "$src_crt" "$tmpc" || ! cmp -s "$src_key" "$tmpk"; then
rm -f "$tmpc" "$tmpk"; CERTSYNC_MSG="source changed during copy; retrying later"; return 0
fi
[[ -f "$dst_crt" ]] && cp -p "$dst_crt" "$dst_crt.prev"
[[ -f "$dst_key" ]] && cp -p "$dst_key" "$dst_key.prev"
mv -f "$tmpc" "$dst_crt"; mv -f "$tmpk" "$dst_key"
if ergo_running; then
dc kill -s HUP ergo >/dev/null 2>&1 || true
want="$(cert_fingerprint "$dst_crt")"; got=""
for i in 1 2 3 4 5 6 7 8 9 10; do
got="$(served_fingerprint)"
[[ -n "$got" && "$got" == "$want" ]] && break
sleep 1
done
if [[ "$got" != "$want" ]]; then
# Ergo kept the previous tls config (a failed rehash leaves the old one
# active). Put the old pair back so the next start is not fatal.
if [[ -f "$dst_crt.prev" && -f "$dst_key.prev" ]]; then
mv -f "$dst_crt.prev" "$dst_crt"; mv -f "$dst_key.prev" "$dst_key"
dc kill -s HUP ergo >/dev/null 2>&1 || true
fi
CERTSYNC_MSG="Ergo did not present the new cert after SIGHUP (see: docker compose logs ergo); previous pair restored"
return 1
fi
fi
rm -f "$dst_crt.prev" "$dst_key.prev"
CERTSYNC_MSG="installed cert for $(cert_subject_cn "$dst_crt") (expires $(cert_enddate "$dst_crt")) from ${src_crt#"$CADDY_DIR"/data/caddy/certificates/}"
return 0
}
# ---------------------------------------------------------------------------
# Small persistent state for de-duplicating notifications (.state/<key>)
# ---------------------------------------------------------------------------
state_get() { cat "$STATE_DIR/$1" 2>/dev/null || true; }
state_set() { install -d -m 0700 "$STATE_DIR" 2>/dev/null; printf '%s\n' "$2" > "$STATE_DIR/$1"; }
state_rm() { rm -f "$STATE_DIR/$1"; }
# ---------------------------------------------------------------------------
# Notify (reuse the login-notifier's ntfy config, like scripts/auto-update.sh)
# ---------------------------------------------------------------------------
send_notice() { # <title> <priority> <body>
[[ "${NOTIFY:-1}" == "1" ]] || return 0
[[ -r "$SSH_NOTIFY_CONF" ]] || return 0
# shellcheck disable=SC1090
. "$SSH_NOTIFY_CONF"
[[ -n "${NTFY_URL:-}" ]] || return 0
command -v curl >/dev/null 2>&1 || return 0
local title="$1" prio="$2" body="$3" host
host="$(hostname -f 2>/dev/null || hostname 2>/dev/null || echo unknown)"
set -- -fsS -m 5 -H "X-Title: ${title}" -H "X-Priority: ${prio}"
[[ -n "${NTFY_TOKEN:-}" ]] && set -- "$@" -H "Authorization: Bearer ${NTFY_TOKEN}"
[[ -n "${NTFY_EMAIL:-}" ]] && set -- "$@" -H "X-Email: ${NTFY_EMAIL}"
local t="ergo"; [[ -n "${NTFY_REGION:-}" ]] && t="${t},${NTFY_REGION}"
set -- "$@" -H "X-Tags: ${t}"
if [[ "${DRY_RUN:-0}" == "1" ]]; then
echo "DRY: curl ntfy ($prio): $body"
else
curl "$@" -d "${body} [${host}]" "$NTFY_URL" >/dev/null 2>&1 || true
fi
return 0
}
+570
View File
@@ -0,0 +1,570 @@
#!/usr/bin/env bash
#
# update.sh -- keep the Ergo stack current: container updates with a DB snapshot
# and health-checked rollback, plus the Caddy -> Ergo TLS certificate sync.
# Companion to deploy.sh; installed in $STACK_DIR next to ergolib.sh.
#
# Subcommands:
# check (default) current vs latest, advisories, cert state; changes nothing
# run what the daily schedule runs: apply UPDATE_POLICY, cert sync, expiry check
# update update now (to latest, or TARGET_VERSION=x.y.z); honours FORCE_UPDATE
# certsync copy Caddy's cert into ircd/ and rehash if it changed (15-min schedule)
# caddy pull a newer Caddy image and recreate it (health-checked)
# install write /etc/ergo-update.conf + schedule the jobs
# uninstall remove the schedule
#
# Policy (UPDATE_POLICY):
# latest update to the newest release whenever one exists (default)
# security update ONLY when a published GitHub security advisory covers the
# running version, or a release between current and latest has a
# "### Security" section in its notes
# off never change the running version (check/notify only)
#
# Safety rails (all policies):
# * releases whose notes announce "Compatibility breaks" are HELD (notify
# only) unless FORCE_UPDATE=1 -- review the notes, then `update update`
# * the new image must load the current ircd/ (ergo run --smoke on a copy,
# which also dry-runs any DB schema upgrade) before the live server is touched
# * users get a NOTICE and UPDATE_GRACE seconds before the restart
# * ircd.db is snapshotted with Ergo stopped (buntdb is append-only; a live
# copy can miss the last second of writes)
# * health = compose healthcheck (IRC-level) + a registration handshake; on
# failure the previous tag is restored and, if the new version bumped the
# DB schema, the snapshot is put back
#
# Env, in precedence order: environment > /etc/ergo-update.conf > the stack's
# .env > the built-in default.
# STACK_DIR=/srv/ergo UPDATE_POLICY=latest FORCE_UPDATE=0 UPDATE_GRACE=60
# CADDY_AUTOUPDATE=0 NOTIFY=1 SSH_NOTIFY_CONF=/etc/ssh-notify.conf DRY_RUN=0
# TARGET_VERSION= GH_REPO=ergochat/ergo LOG=/var/log/ergo-update.log
# SCHEDULE_UPDATES=1 (install: 0 schedules certsync only, no daily update)
# VERBOSE= (certsync: also log "nothing to do" outcomes)
set -euo pipefail
SELF="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/$(basename "${BASH_SOURCE[0]}")"
SELF_DIR="$(dirname "$SELF")"
log() { printf '\033[1;32m[+]\033[0m %s\n' "$*"; }
warn() { printf '\033[1;33m[!]\033[0m %s\n' "$*" >&2; }
die() { printf '\033[1;31m[x]\033[0m %s\n' "$*" >&2; exit 1; }
# shellcheck source=ergolib.sh
. "$SELF_DIR/ergolib.sh"
load_conf
: "${STACK_DIR:=$SELF_DIR}"
ergo_set_paths
# .env is the stack's own settings file and documents these knobs, so honour it
# for anything the environment and /etc/ergo-update.conf did not already set.
env_defaults UPDATE_POLICY FORCE_UPDATE UPDATE_GRACE CADDY_AUTOUPDATE
: "${UPDATE_POLICY:=latest}"
: "${FORCE_UPDATE:=0}"
: "${UPDATE_GRACE:=60}"
: "${CADDY_AUTOUPDATE:=0}"
: "${TARGET_VERSION:=}"
: "${SCHEDULE_UPDATES:=1}"
: "${LOG:=/var/log/ergo-update.log}"
# ---------------------------------------------------------------------------
# Release metadata (GitHub). Unauthenticated: 60 requests/hour -- a run uses 2-3.
# ---------------------------------------------------------------------------
RELEASES_JSON=""
# Fetch the release list ONCE per process. This must be called as a plain
# statement, never inside a pipeline or $(): the assignment would land in a
# subshell and every use would re-download the list (and burn the 60/hour
# anonymous API budget).
load_releases() {
[[ -n "$RELEASES_JSON" ]] && return 0
RELEASES_JSON="$(fetch "https://api.github.com/repos/${GH_REPO}/releases?per_page=50" || true)"
printf '%s' "$RELEASES_JSON" | jq -e 'type=="array"' >/dev/null 2>&1 || RELEASES_JSON="[]"
return 0
}
# Bodies of stable releases with cur < version <= upto, newest first:
# "=== <tag>" header lines followed by the body.
release_notes_between() { # <cur> <upto>
local cur="$1" upto="$2" tag v
load_releases
while IFS= read -r tag; do
[[ -n "$tag" ]] || continue
v="$(normver "$tag")"; [[ -n "$v" ]] || continue
ver_gt "$v" "$cur" || continue
ver_le "$v" "$upto" || continue
printf '=== %s\n' "$tag"
printf '%s' "$RELEASES_JSON" | jq -r --arg t "$tag" '.[] | select(.tag_name==$t) | .body // ""'
printf '\n'
done < <(printf '%s' "$RELEASES_JSON" | jq -r '.[] | select(.prerelease==false and .draft==false) | .tag_name' 2>/dev/null)
}
# Published GitHub security advisories covering <cur>. Prints "GHSA-... <patched>"
# lines. Ranges look like "<= v2.19.0" or ">= v2.15.0, < v2.19.1".
ghsa_covering() { # <cur>
local cur="$1" json id rng patched part op ver ok
json="$(fetch "https://api.github.com/repos/${GH_REPO}/security-advisories" || true)"
printf '%s' "$json" | jq -e 'type=="array"' >/dev/null 2>&1 || return 0
while IFS=$'\t' read -r id rng patched; do
[[ -n "$rng" ]] || continue
ok=1
IFS=',' read -r -a parts <<< "$rng"
for part in "${parts[@]}"; do
part="$(printf '%s' "$part" | tr -d ' ')"
op="$(printf '%s' "$part" | grep -oE '^(<=|>=|==|<|>|=)' || echo '=')"
ver="$(normver "$part")"; [[ -n "$ver" ]] || { ok=0; break; }
case "$op" in
'<') ver_gt "$ver" "$cur" || ok=0 ;;
'<=') ver_ge "$ver" "$cur" || ok=0 ;;
'>') ver_gt "$cur" "$ver" || ok=0 ;;
'>=') ver_ge "$cur" "$ver" || ok=0 ;;
*) [[ "$(ver_cmp "$ver" "$cur")" == "0" ]] || ok=0 ;;
esac
(( ok )) || break
done
(( ok )) && printf '%s %s\n' "$id" "$(normver "$patched")"
done < <(printf '%s' "$json" | jq -r '.[] | select(.state=="published") | .ghsa_id as $id | (.vulnerabilities // [])[] | [$id, (.vulnerable_version_range // ""), ((.patched_versions // "") | tostring)] | @tsv' 2>/dev/null)
}
# ---------------------------------------------------------------------------
# Decide what to do. Sets globals (not via stdout -- a $() would lose them):
# TARGET (x.y.z or ""), REASON, HOLD (1 = compat break, notify only),
# NOTES (one-line summary for the notification), DBCHANGE (0/1)
# ---------------------------------------------------------------------------
TARGET=""; REASON=""; HOLD=0; NOTES=""; DBCHANGE=0
resolve_target() { # <current x.y.z>
local cur="$1" latest notes ghsa ids
TARGET=""; REASON=""; HOLD=0; NOTES=""; DBCHANGE=0
latest="$(normver "$(ergo_latest_tag || true)")"
if [[ -n "$TARGET_VERSION" ]]; then
TARGET="$(normver "$TARGET_VERSION")"; REASON="target override ${TARGET}"
else
case "$UPDATE_POLICY" in
off) REASON="policy=off (no changes)"; return 0 ;;
security)
ghsa="$(ghsa_covering "$cur")"
if [[ -n "$ghsa" ]]; then
ids="$(printf '%s\n' "$ghsa" | awk '{print $1}' | paste -sd, - || true)"
# NF>=2: an advisory with no patched version yields only an id.
TARGET="$(printf '%s\n' "$ghsa" | awk 'NF>=2 {print $2}' | tail -n1 || true)"
[[ -n "$TARGET" ]] || TARGET="$latest"
REASON="security advisory covers ${cur}: ${ids}"
elif [[ -n "$latest" ]] && ver_gt "$latest" "$cur" \
&& grep -qiE '^#+ *security|security vulnerabilit' <<< "$(release_notes_between "$cur" "$latest")"; then
TARGET="$latest"; REASON="a release since ${cur} carries a Security section"
else
REASON="no security advisory or security release covers ${cur}"; return 0
fi ;;
latest|*)
[[ -n "$latest" ]] || { REASON="could not resolve the latest release"; return 0; }
TARGET="$latest"; REASON="policy=latest" ;;
esac
fi
[[ -n "$TARGET" ]] || return 0
notes="$(release_notes_between "$cur" "$TARGET")"
# Feed grep from a here-string, not a pipe: `printf | grep -q` on notes
# larger than the pipe buffer makes printf die of SIGPIPE, and pipefail
# then reports "no match" for what was in fact a match.
if grep -qiE 'compatibility break|not backwards[- ]compatible|backwards[- ]incompatible' <<< "$notes"; then
HOLD=1
fi
if grep -qiE 'database change|database file format changes|schema' <<< "$notes" \
&& ! grep -qiE 'no changes to the config file format or database file format' <<< "$notes"; then
DBCHANGE=1
fi
NOTES="$(grep -iE 'config file format|database (file )?format|database change|compatibility break' <<< "$notes" \
| head -n 3 | sed 's/^[ *-]*//' | tr '\n' ' ' | cut -c1-300 || true)"
return 0
}
# ---------------------------------------------------------------------------
# Apply an update: pre-flight on a copy, warn users, stop, snapshot, up, verify;
# roll back (tag + DB snapshot when needed) on failure.
# ---------------------------------------------------------------------------
notify_users() { # <text> -- best effort, only when Ergo is up and we can oper
ergo_running || return 0
irc_raw --oper --quiet 1 -- "NOTICE \$\$* :$1" >/dev/null 2>&1 || true
}
apply_update() { # <from x.y.z> <to x.y.z> <reason>
local from="$1" to="$2" reason="$3" image_base new_image snap ts cause prev_tag
image_base="$(env_get ERGO_IMAGE)"; image_base="${image_base:-ghcr.io/ergochat/ergo}"
new_image="${image_base}:v${to}"
# Roll back to the tag that is actually pinned right now. Reconstructing
# "v${from}" would write a bare "v" when the running version is unknown
# (e.g. .env still holds the 'stable' fallback and the container is down).
prev_tag="$(ergo_tag)"
log "Updating Ergo ${from:-?} -> ${to} (${reason})..."
if [[ "$DRY_RUN" == "1" ]]; then
echo "DRY: pull ${new_image}; validate on a copy; NOTICE users; stop; snapshot ircd.db; ERGO_TAG=v${to}; up -d --no-deps ergo"
return 0
fi
cp -a "$ENV_FILE" "${ENV_FILE}.bak.$(date -u +%Y%m%d%H%M%S)" 2>/dev/null || true
log "Pulling ${new_image}..."
if ! docker pull -q "$new_image" >/dev/null 2>&1; then
send_notice "Ergo update FAILED" "high" "pull of ${new_image} failed; staying on ${from}"
return 1
fi
log "Pre-flight: loading the current ircd/ with ${new_image} (throwaway container)..."
if ! ergo_validate_config "$new_image"; then
cause="$(tail -n 5 "$VALIDATE_LOG" 2>/dev/null | tr '\n' ' ' | cut -c1-300)"
warn "Pre-flight failed: $cause"
send_notice "Ergo update HELD" "high" "${to} cannot load the current config/DB: ${cause}. Staying on ${from}; run 'ergoctl update check'."
return 1
fi
rm -f "$VALIDATE_LOG"
if (( UPDATE_GRACE > 0 )) && ergo_running; then
log "Warning connected users (${UPDATE_GRACE}s grace)..."
notify_users "Server restarting for an upgrade to Ergo ${to} in ${UPDATE_GRACE} seconds -- you will be disconnected briefly."
sleep "$UPDATE_GRACE"
fi
log "Stopping Ergo and snapshotting ircd.db..."
dc stop -t 15 ergo >/dev/null 2>&1 || true
ts="$(date -u +%Y%m%d%H%M%S)"
install -d -m 0700 "$BACKUP_DIR"
snap="$BACKUP_DIR/ircd.db.${from:-unknown}.${ts}"
if [[ -f "$IRCD_DIR/ircd.db" ]]; then
cp -p "$IRCD_DIR/ircd.db" "$snap" && chmod 0600 "$snap"
else
snap=""
fi
env_set ERGO_TAG "v${to}"
dc up -d --no-deps ergo >/dev/null 2>&1 || true
if wait_healthy ergo 120 && irc_probe; then
log "Ergo ${to} is healthy."
record_template "$new_image" "$from" "$to"
prune_snapshots
local b="updated ${from:-?} -> ${to} (${reason})"
[[ -n "$NOTES" ]] && b="${b}. Notes: ${NOTES}"
[[ -n "$TEMPLATE_NOTE" ]] && b="${b}. ${TEMPLATE_NOTE}"
send_notice "Ergo updated" "default" "$b"
return 0
fi
cause="$(dc logs --tail 60 ergo 2>/dev/null | grep -E 'Config file did not load|Database requires update|Could not load server|failed to|fatal|panic' | tail -n 2 | cut -c1-200 | tr '\n' ' ' || true)"
warn "Ergo ${to} did not become healthy (${cause:-no diagnostic in the log}); rolling back to ${prev_tag}."
dc stop -t 10 ergo >/dev/null 2>&1 || true
env_set ERGO_TAG "$prev_tag"
dc up -d --no-deps ergo >/dev/null 2>&1 || true
if ! wait_healthy ergo 90; then
# The old binary refuses a newer schema ("Database requires update") --
# only then does the snapshot go back (otherwise keep the live DB).
if [[ -n "$snap" ]] && { [[ "$DBCHANGE" == 1 ]] || dc logs --tail 40 ergo 2>/dev/null | grep -q 'Database requires update'; }; then
warn "Old version rejects the upgraded database; restoring the snapshot ${snap}."
dc stop -t 10 ergo >/dev/null 2>&1 || true
install -m 0600 -o "$(ergo_uid)" -g "$(ergo_gid)" "$snap" "$IRCD_DIR/ircd.db"
dc up -d --no-deps ergo >/dev/null 2>&1 || true
wait_healthy ergo 90 || true
fi
fi
if [[ "$(svc_state ergo)" == "running healthy" ]]; then
send_notice "Ergo update FAILED" "high" "${to} unhealthy (${cause:-see docker compose logs ergo}); rolled back to ${prev_tag}. Snapshot: ${snap:-none}"
else
send_notice "Ergo DOWN after failed update" "urgent" "${to} failed (${cause:-?}) and ${prev_tag} did not come back healthy. Snapshot: ${snap:-none}. Run 'ergoctl status' / 'docker compose logs ergo'."
fi
return 1
}
# Keep the image's default.yaml per version and diff it against the previous one
# so template drift (new options, renamed keys) is visible after an upgrade.
TEMPLATE_NOTE=""
record_template() { # <image> <from> <to>
local image="$1" from="$2" to="$3" new old diff added removed
TEMPLATE_NOTE=""
install -d -m 0700 "$TEMPLATES_DIR"
new="$TEMPLATES_DIR/default.yaml.v${to}"
ergo_extract_template "$image" "$new" 2>/dev/null || { rm -f "$new"; return 0; }
chmod 0600 "$new"
old="$TEMPLATES_DIR/default.yaml.v${from}"
[[ -f "$old" ]] || return 0
diff="$TEMPLATES_DIR/diff-v${from}-v${to}.txt"
if diff -u "$old" "$new" > "$diff" 2>/dev/null; then rm -f "$diff"; return 0; fi
chmod 0600 "$diff" 2>/dev/null || true
added="$(grep -cE '^\+[^+]' "$diff" || true)"; removed="$(grep -cE '^-[^-]' "$diff" || true)"
TEMPLATE_NOTE="default config changed upstream (+${added}/-${removed} lines): review ${diff} against ircd/ircd.yaml"
log "$TEMPLATE_NOTE"
}
prune_snapshots() { # keep the last 5 DB snapshots and .env backups
local f n=0
for f in $(ls -1t "$BACKUP_DIR"/ircd.db.* 2>/dev/null); do
n=$((n + 1)); (( n > 5 )) && rm -f "$f"
done
n=0
for f in $(ls -1t "$STACK_DIR"/.env.bak.* 2>/dev/null); do
n=$((n + 1)); (( n > 5 )) && rm -f "$f"
done
return 0
}
# The pinned tag must be an immutable vX.Y.Z: deploy.sh falls back to 'stable'
# when GitHub is unreachable, and a floating tag silently changes the running
# version on any pull, with no pre-flight, snapshot or rollback. Pin it to
# whatever is running as soon as we can see a version (no restart needed).
pin_floating_tag() { # <current x.y.z>
local cur="$1" tag
tag="$(ergo_tag)"
[[ -n "$cur" ]] || return 0
case "$tag" in v[0-9]*) return 0 ;; esac
env_set ERGO_TAG "v${cur}"
log "Pinned ERGO_TAG=v${cur} (was '${tag}', which floats)."
}
# ---------------------------------------------------------------------------
# Cert sync + expiry watch
# ---------------------------------------------------------------------------
do_certsync() {
local rc=0 prev
ergo_certsync || rc=$?
prev="$(state_get certsync)"
case "$rc" in
0) [[ "$CERTSYNC_MSG" == "up to date" ]] || log "certsync: $CERTSYNC_MSG"
if [[ -n "$prev" ]]; then state_rm certsync; send_notice "Ergo TLS recovered" "default" "cert sync ok: ${CERTSYNC_MSG}"; fi ;;
2) [[ -n "${VERBOSE:-}" ]] && log "certsync: $CERTSYNC_MSG"
return 0 ;;
*) warn "certsync: $CERTSYNC_MSG"
if [[ "$prev" != "$CERTSYNC_MSG" ]]; then state_set certsync "$CERTSYNC_MSG"; send_notice "Ergo TLS sync FAILED" "high" "$CERTSYNC_MSG"; fi
return 1 ;;
esac
return 0
}
cert_expiry_check() { # daily: warn once per day if the served cert is self-signed or expiring
local f="$IRCD_DIR/fullchain.pem" today msg=""
[[ -f "$f" ]] || return 0
today="$(date -u +%F)"
if cert_is_selfsigned "$f"; then
msg="Ergo is serving a SELF-SIGNED certificate on 6697 (Caddy has not issued one yet: check DNS for $(ergo_domain) and ports 80/443)."
elif cert_expires_within "$f" $((14 * 86400)); then
msg="Ergo's TLS cert expires soon ($(cert_enddate "$f")) and certsync has not replaced it -- is the 15-minute job running? ($SELF certsync)"
fi
[[ -n "$msg" ]] || { state_rm certexpiry; return 0; }
warn "$msg"
[[ "$(state_get certexpiry)" == "$today" ]] && return 0
state_set certexpiry "$today"
send_notice "Ergo TLS attention" "high" "$msg"
}
# ---------------------------------------------------------------------------
# Caddy: pull + recreate when the image moved (drops web-client websockets)
# ---------------------------------------------------------------------------
do_caddy() {
preflight
local tag before after
tag="$(env_get CADDY_TAG)"; tag="${tag:-2-alpine}"
before="$(docker inspect --format '{{.Image}}' ergo-caddy 2>/dev/null | head -n1 || true)"
log "Pulling caddy:${tag}..."
docker pull -q "caddy:${tag}" >/dev/null 2>&1 || { warn "pull failed"; return 1; }
after="$(docker image inspect --format '{{.Id}}' "caddy:${tag}" 2>/dev/null | head -n1 || true)"
if [[ -n "$before" && "$before" == "$after" ]]; then log "Caddy is current (${tag})."; return 0; fi
[[ "$DRY_RUN" == "1" ]] && { echo "DRY: docker compose up -d --no-deps caddy"; return 0; }
log "Recreating Caddy (web-client websockets will reconnect)..."
dc up -d --no-deps caddy >/dev/null 2>&1 || true
if wait_healthy caddy 60; then
send_notice "Caddy updated" "min" "caddy:${tag} recreated (image $(printf '%s' "$after" | cut -c8-19))"
return 0
fi
send_notice "Caddy update FAILED" "high" "caddy:${tag} is not healthy after recreate; check docker compose logs caddy"
return 1
}
# ---------------------------------------------------------------------------
# Subcommands
# ---------------------------------------------------------------------------
preflight() {
[[ $EUID -eq 0 ]] || die "Run as root."
[[ -f "$STACK_DIR/docker-compose.yml" ]] || die "No stack at $STACK_DIR (set STACK_DIR)."
command -v docker >/dev/null 2>&1 || die "docker not found."
command -v jq >/dev/null 2>&1 || die "jq is required (release metadata); install it."
}
do_check() {
preflight
local cur latest ghsa held="" prio="min" note
cur="$(ergo_version_running)"
pin_floating_tag "$cur"
latest="$(normver "$(ergo_latest_tag || true)")"
log "Running: ${cur:-unknown} (pinned $(ergo_tag)) | latest release: ${latest:-unknown} | policy: ${UPDATE_POLICY}"
ghsa="$(ghsa_covering "$cur" || true)"
if [[ -n "$ghsa" ]]; then
note="VULNERABLE: $(printf '%s\n' "$ghsa" | awk '{print $1}' | paste -sd, -) covers ${cur}"; prio="high"; warn "$note"
else
note="no published advisory covers ${cur}"; log "$note"
fi
if [[ -n "$cur" && -n "$latest" ]] && ver_gt "$latest" "$cur"; then
resolve_target "$cur"
log "A newer release is available: ${cur} -> ${latest}"
[[ "$prio" == "min" ]] && prio="default"
note="${note}; newer release ${latest} available"
if [[ "$HOLD" == 1 ]]; then held=" -- HELD: release notes announce compatibility breaks (review, then FORCE_UPDATE=1 or 'ergoctl update update')"; warn "${held# -- }"; fi
[[ -n "$NOTES" ]] && log "Release notes: $NOTES"
send_notice "Ergo check" "$prio" "${note}${held}"
fi
local f="$IRCD_DIR/fullchain.pem"
if [[ -f "$f" ]]; then
if cert_is_selfsigned "$f"; then warn "TLS: serving a SELF-SIGNED cert (no Let's Encrypt cert from Caddy yet)."
else log "TLS: $(cert_subject_cn "$f"), expires $(cert_enddate "$f") (served: $( [[ "$(served_fingerprint)" == "$(cert_fingerprint "$f")" ]] && echo matches || echo DIFFERS -- rehash needed ))"; fi
fi
}
do_run() {
preflight
if [[ "$DRY_RUN" != "1" ]]; then
install -d -m 0755 "$(dirname "$LOG")" 2>/dev/null || true
echo "=== ergo-update $(date -u +%FT%TZ) ===" >> "$LOG"
# Scheduled runs have nowhere to send stdout (busybox crond would try to
# mail it and these hosts have no MTA), so keep the whole run in the log.
[[ -t 1 ]] || exec >>"$LOG" 2>&1
fi
do_certsync || true
cert_expiry_check || true
local cur
cur="$(ergo_version_running)"
pin_floating_tag "$cur"
resolve_target "$cur"
log "current=${cur:-?} | ${REASON}"
if [[ -n "$TARGET" && -n "$cur" ]] && ver_gt "$TARGET" "$cur"; then
if [[ "$HOLD" == 1 && "$FORCE_UPDATE" != "1" ]]; then
warn "HELD: ${TARGET} announces compatibility breaks. Review the release notes, then 'ergoctl update update' (or FORCE_UPDATE=1)."
if [[ "$(state_get held)" != "$TARGET" ]]; then
state_set held "$TARGET"
send_notice "Ergo update HELD" "default" "${cur} -> ${TARGET} announces compatibility breaks; not applied automatically. Notes: ${NOTES:-see GitHub release}. Apply with 'ergoctl update update'."
fi
else
state_rm held
apply_update "$cur" "$TARGET" "$REASON" || true
fi
else
log "No update to apply."
fi
if [[ "$CADDY_AUTOUPDATE" == "1" ]]; then do_caddy || true; fi
}
do_update() {
preflight
local cur
cur="$(ergo_version_running)"
[[ -n "$cur" ]] || die "Cannot tell which version is running (container down and ERGO_TAG='$(ergo_tag)' is not a version). Start Ergo, or pin ERGO_TAG=vX.Y.Z in $ENV_FILE first."
pin_floating_tag "$cur"
if [[ -z "$TARGET_VERSION" && "$UPDATE_POLICY" == "off" ]]; then UPDATE_POLICY=latest; fi
resolve_target "$cur"
[[ -n "$TARGET" ]] || die "Could not determine a target version (${REASON})."
if [[ -n "$cur" ]] && ! ver_gt "$TARGET" "$cur" && [[ -z "$TARGET_VERSION" ]]; then
log "Already on ${cur} (latest ${TARGET}); nothing to do."
return 0
fi
if [[ "$HOLD" == 1 && "$FORCE_UPDATE" != "1" ]]; then
warn "Release notes between ${cur} and ${TARGET} announce compatibility breaks: ${NOTES}"
warn "Proceeding because you asked explicitly (this is 'update update'). Set FORCE_UPDATE=1 to also auto-apply such releases."
fi
apply_update "$cur" "$TARGET" "${REASON} (manual)"
}
# Only what the scheduled job needs to FIND the stack lives here. The update
# knobs (UPDATE_POLICY, FORCE_UPDATE, UPDATE_GRACE, CADDY_AUTOUPDATE) are read
# from the stack's .env so there is a single place to edit them; setting one
# here (or in the environment) still overrides .env for that run.
write_conf() {
cat > "$ERGO_UPDATE_CONF" <<CONF
# Where the scheduled Ergo jobs find the stack (deployments/ergo/update.sh).
# The update knobs live in \${STACK_DIR}/.env -- edit them there. Anything set
# here overrides .env, so leave it alone unless you mean to pin a value.
STACK_DIR="${STACK_DIR}"
NOTIFY="${NOTIFY}"
CONF
chmod 644 "$ERGO_UPDATE_CONF"
log "Wrote $ERGO_UPDATE_CONF (update knobs come from ${STACK_DIR}/.env)"
}
do_install() {
[[ $EUID -eq 0 ]] || die "Run as root."
write_conf
case "$(osfam)" in
alpine)
install -d -m 0755 /etc/periodic/15min /etc/periodic/daily
printf '#!/bin/sh\nexec bash "%s" certsync\n' "$SELF" > /etc/periodic/15min/ergo-certsync
chmod +x /etc/periodic/15min/ergo-certsync
if [[ "$SCHEDULE_UPDATES" == "1" ]]; then
printf '#!/bin/sh\nexec bash "%s" run\n' "$SELF" > /etc/periodic/daily/ergo-update
chmod +x /etc/periodic/daily/ergo-update
else
rm -f /etc/periodic/daily/ergo-update
fi
if command -v rc-update >/dev/null 2>&1; then
rc-update add crond default >/dev/null 2>&1 || true
rc-service crond status >/dev/null 2>&1 || rc-service crond start >/dev/null 2>&1 || true
fi
log "Scheduled: /etc/periodic/15min/ergo-certsync$( [[ "$SCHEDULE_UPDATES" == "1" ]] && printf ', /etc/periodic/daily/ergo-update (policy=%s)' "$UPDATE_POLICY" )." ;;
*)
cat > /etc/systemd/system/ergo-certsync.service <<EOF
[Unit]
Description=Ergo: sync Caddy's TLS cert into the IRC server
After=docker.service
[Service]
Type=oneshot
ExecStart=/usr/bin/env bash $SELF certsync
EOF
cat > /etc/systemd/system/ergo-certsync.timer <<EOF
[Unit]
Description=Ergo cert sync every 15 minutes
[Timer]
OnCalendar=*:0/15
Persistent=true
[Install]
WantedBy=timers.target
EOF
cat > /etc/systemd/system/ergo-update.service <<EOF
[Unit]
Description=Ergo container updater
After=docker.service
[Service]
Type=oneshot
ExecStart=/usr/bin/env bash $SELF run
EOF
cat > /etc/systemd/system/ergo-update.timer <<EOF
[Unit]
Description=Daily Ergo update check
[Timer]
OnCalendar=daily
Persistent=true
RandomizedDelaySec=1h
[Install]
WantedBy=timers.target
EOF
systemctl daemon-reload
systemctl enable --now ergo-certsync.timer >/dev/null 2>&1 || true
if [[ "$SCHEDULE_UPDATES" == "1" ]]; then systemctl enable --now ergo-update.timer >/dev/null 2>&1 || true
else systemctl disable --now ergo-update.timer >/dev/null 2>&1 || true; fi
log "Scheduled: ergo-certsync.timer (15 min)$( [[ "$SCHEDULE_UPDATES" == "1" ]] && printf ', ergo-update.timer (daily, policy=%s)' "$UPDATE_POLICY" )." ;;
esac
}
do_uninstall() {
[[ $EUID -eq 0 ]] || die "Run as root."
rm -f /etc/periodic/15min/ergo-certsync /etc/periodic/daily/ergo-update
if command -v systemctl >/dev/null 2>&1; then
systemctl disable --now ergo-certsync.timer ergo-update.timer >/dev/null 2>&1 || true
rm -f /etc/systemd/system/ergo-certsync.{timer,service} /etc/systemd/system/ergo-update.{timer,service}
systemctl daemon-reload >/dev/null 2>&1 || true
fi
log "Removed the scheduled Ergo jobs (config kept at $ERGO_UPDATE_CONF)."
}
case "${1:-check}" in
check) do_check ;;
run) do_run ;;
update) do_update ;;
certsync) preflight; do_certsync ;;
caddy) do_caddy ;;
install) do_install ;;
uninstall) do_uninstall ;;
*) die "Usage: update.sh [check|run|update|certsync|caddy|install|uninstall]" ;;
esac