57_Wolve/automations
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: unified launcher, multi-OS hardening, login alerts & auto-updates

Browse Source 
Restructure around a single entry point (automations.sh) with a Gum wizard and
a self-extracting bundle for repo-less installs. Add scripts/oslib.sh so the
provisioning scripts (setup-host, harden-ssh, harden-jumphost, sshuser) run on
Alpine/Debian/Alma; seed root keys from globals/.

- ntfy SSH-login alerts (user, source IP, key, region, jump target) via pam_exec
- daily auto-updates with AUTO_REBOOT=idle (reboots only when no SSH active) and
  opt-in Alpine stable-branch upgrades
- generic + per-deployment cloud-init; Gitea release workflow on tag
- README/LICENSE/.gitignore/.gitattributes (force LF); repo URLs -> Gitea
This commit is contained in:
William Gill
2026-06-12 14:56:02 -05:00
parent 85eeb79971
commit 7faa9098de
58 changed files with 6225 additions and 284 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitattributes
+15
View File
@@ -0,0 +1,15 @@
# Force LF for everything textual so scripts edited on Windows still run on
# Alpine/Debian/Alma. A CRLF in a shebang line breaks the interpreter.
* text=auto eol=lf
*.sh text eol=lf
*.yml text eol=lf
*.yaml text eol=lf
*.conf text eol=lf
*.env text eol=lf
*.example text eol=lf
Caddyfile text eol=lf
*.md text eol=lf
# Treat the embedded-archive deploy scripts as text too (they're base64+shell).
deployments/*/deploy.sh text eol=lf
.gitea/workflows/release.yml
+41
View File
@@ -0,0 +1,41 @@
# Gitea Actions: on a version tag (vX.Y.Z), build the self-extracting bundle
# and publish it as a release asset. Make the release public (Gitea per-repo
# setting) and the bundle is downloadable without granting repo access.
#
# Requirements:
# - A registered Gitea act_runner whose labels include the `runs-on` value
# below (adjust if yours differ, e.g. self-hosted).
# - A repo/org secret named TOKEN_GITEA holding a Gitea access token with
# write:repository scope (Settings -> Applications -> Generate Token).
# NOTE: Gitea reserves the GITEA_ prefix, so the name can't start with it;
# TOKEN_GITEA is fine.
name: Release bundle
on:
push:
tags:
- 'v*'
jobs:
bundle:
runs-on: ubuntu-latest
steps:
- name: Checkout (full history + tags)
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Build self-extracting bundle
run: |
bash build-bundle.sh "${GITHUB_REF_NAME}"
ls -l dist/
- name: Create release and attach the bundle
uses: https://gitea.com/actions/release-action@main
with:
files: |-
dist/automations-bundle.sh
api_key: '${{ secrets.TOKEN_GITEA }}'
# title/body default to the tag; uncomment to customize:
# title: 'automations ${{ github.ref_name }}'
.gitignore
+33
View File
@@ -0,0 +1,33 @@
# ── Secrets / runtime config ────────────────────────────────────────────────
# Populated env / config files (keep the *.example templates).
.env
**/.env
globals/globals.env
ssh-notify.conf
auto-update.conf
!*.env.example
!**/.env.example
!*.conf.example
# ── Private keys (NEVER commit) ─────────────────────────────────────────────
# age identities and SSH private keys. globals/age-pubkey.txt and
# globals/authorized_keys are PUBLIC and intentionally tracked.
*-private-key.txt
*age-key*
*.age.key
id_ed25519
id_ed25519_*
*.pem
# ── Backups ─────────────────────────────────────────────────────────────────
*.tar.gz.age
*-backup-*.tar.gz*
# ── Build output ────────────────────────────────────────────────────────────
dist/
# ── Editor / OS noise ───────────────────────────────────────────────────────
*.tmp
*.swp
.DS_Store
Thumbs.db
LICENSE
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 William Gill
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
README.md
+197
View File
@@ -0,0 +1,197 @@
# automations
Deployment and automation scripts for self-hosted, security-hardened
infrastructure. The host-provisioning scripts run on **Alpine, Debian, and
Alma Linux** (every distro difference is gated in
[`scripts/oslib.sh`](scripts/oslib.sh)); each Docker stack runs behind Caddy
with automatic Let's Encrypt TLS, orchestrated with Docker Compose.
## One command to run anything
```bash
curl -fsSL https://git.anomalous.dev/57_Wolve/automations/raw/branch/main/automations.sh \
| REPO_URL=https://git.anomalous.dev/57_Wolve/automations.git bash
```
Or, from a clone:
```bash
./automations.sh
```
[`automations.sh`](automations.sh) opens a **Gum** wizard (auto-installed) that
lets you:
- **Deploy on this host** — pick any deployment or generic script and run it,
prompting for the values it needs.
- **Build artifacts locally** — regenerate a deployment's self-contained
`deploy.sh` and optionally `scp` it to a target host.
Shared defaults (email, repo URL, SSH key source, backup recipient) come from
[`globals/`](globals/) so you set them once.
## Self-contained bundle (no repo access)
For hosts that shouldn't have repo/git access, package the whole repo into one
self-extracting script and serve it from a webserver or a **public Gitea
release** (the repo itself can stay private):
```bash
./build-bundle.sh # -> dist/automations-bundle.sh (HEAD)
./build-bundle.sh v1.2.0 # a specific tag
./build-bundle.sh worktree # current working tree
```
On the target host, download then run it (it extracts itself — it can't read a
pipe, so download first):
```bash
curl -fsSLO https://your-host/automations-bundle.sh
bash automations-bundle.sh # launcher wizard
bash automations-bundle.sh bash scripts/setup-host.sh # run one script
SSH_PORT=2222 bash automations-bundle.sh bash scripts/harden-jumphost.sh
```
It extracts to `INSTALL_DIR` (default `/opt/automations`) and runs the launcher
or the command you pass. The payload excludes ignored files, so no secrets are
embedded.
**Releases via Gitea Actions:** pushing a `vX.Y.Z` tag runs
[`.gitea/workflows/release.yml`](.gitea/workflows/release.yml), which builds the
bundle and attaches it to a release. Create a secret named **`TOKEN_GITEA`**
(a Gitea access token with `write:repository`) — note Gitea reserves the
`GITEA_` prefix, so the secret can't be called `GITEA_TOKEN`. Adjust `runs-on`
to match your runner. Mark the release public and `automations-bundle.sh` is
fetchable without repo access.
## Layout
The rule: **generic, run-anywhere scripts live in `scripts/`; deployment-specific
files live under `deployments/<name>/`.** Shared values live in `globals/`.
```
automations.sh # the launcher (one-liner entry point)
cloud-init/ # generic base + jumphost cloud-init (any of the 3 distros)
globals/ # shared assets: age-pubkey, authorized_keys, motd, globals.env
scripts/ # generic scripts (Alpine/Debian/Alma)
deployments/<name>/ # one folder per stack
```
### `scripts/` — generic (Alpine / Debian / Alma)
| Script | What it does |
|--------|--------------|
| [`setup-host.sh`](scripts/setup-host.sh) | Set hostname per the naming schema (derives FQDN + Node ID) and render the shared MOTD with auto-computed border spacing. |
| [`harden-ssh.sh`](scripts/harden-ssh.sh) | SSH hardening: post-quantum hybrid KEX, fresh Ed25519 host keys, key-only auth, external SFTP subsystem, sshguard. |
| [`harden-jumphost.sh`](scripts/harden-jumphost.sh) | Bastion hardening on top of `harden-ssh`: `ssh-admins` (shell) vs `ssh-jumpers` (ProxyJump-only) with a PermitOpen allow-list. |
| [`sshuser.sh`](scripts/sshuser.sh) | Add/edit/remove SSH users on a hardened jump host (Gum TUI or CLI flags). Installed standalone as `sshuser`. |
| [`ntfy-ssh-login.sh`](scripts/ntfy-ssh-login.sh) | `pam_exec` hook that posts SSH logins to ntfy (user, source IP, key used, best-effort jump target), gated by group. Config: [`ssh-notify.conf.example`](scripts/ssh-notify.conf.example). |
| [`auto-update.sh`](scripts/auto-update.sh) | Daily unattended package updates; reports (doesn't auto-jump) a new Alpine branch; reboot detection; ntfy summary. `install`/`run`/`uninstall`. |
| [`oslib.sh`](scripts/oslib.sh) | OS-abstraction layer (detection, package manager, init system, SFTP path, hostname, boot hooks, sshguard, login notifier). Sourced, not run. **One file holds every distro difference.** |
| [`lib.sh`](scripts/lib.sh) | Launcher helpers (`ensure_gum`, `load_globals`, `resolve_ssh_keys`); sources `oslib.sh`. Not run directly. |
### `deployments/` — per stack
| Deployment | What it is | Depends on |
|------------|------------|------------|
| [`pocket-id`](deployments/pocket-id/) | OIDC provider (Caddy + Anubis PoW gate + Pocket-ID). | — |
| [`beszel`](deployments/beszel/) | Server monitoring hub. OIDC via pocket-id (post-deploy). | pocket-id (OIDC, optional) |
| [`headscale`](deployments/headscale/) | Self-hosted Tailscale control server, OIDC login. | pocket-id (OIDC) |
| [`webfinger`](deployments/webfinger/) | Serves `/.well-known/webfinger` for OIDC discovery; redirects the rest. | pocket-id (issuer) |
| [`simplex`](deployments/simplex/) | SimpleX SMP + XFTP relay with Tor hidden services + encrypted backups. | globals/age-pubkey.txt |
## Conventions
- **Alpine + Docker Compose + Caddy/Let's Encrypt** across every stack.
- **`build.sh` → `deploy.sh`**: each stack's `build.sh` embeds its
`docker-compose.yml` / `Caddyfile` / `.env.example` into a single
self-contained `deploy.sh` (base64 tar.gz). That one file can be `scp`'d to a
host and run on its own. Rebuild after editing the loose files.
(simplex is the exception — it deploys via `install-simplex.sh`.)
- **Secrets** live in `.env` (generated on first deploy) and `globals/globals.env`,
both git-ignored. Only `*.example` templates and public keys are committed.
- **Non-interactive**: every `deploy.sh` honors `SKIP_PROMPTS=1` with values
supplied via the environment — which is how the per-deployment
`cloud-init.yml` templates stand a stack up unattended.
## Cloud-init
- **Provision a host**: [`cloud-init/base.yml`](cloud-init/base.yml) (hostname +
MOTD + SSH hardening) or [`cloud-init/jumphost.yml`](cloud-init/jumphost.yml)
(bastion). Distro-agnostic — they install prerequisites for whatever the image
is, then run the scripts.
- **Stand up a stack**: each deployment ships its own `cloud-init.yml` (e.g.
[`deployments/pocket-id/cloud-init.yml`](deployments/pocket-id/cloud-init.yml)).
Fill in `REPO_URL` and the values at the top of the `runcmd` block, paste as
instance user-data, and the host configures itself on first boot.
## Multi-OS notes
The host-provisioning scripts (`setup-host`, `harden-ssh`, `harden-jumphost`,
`sshuser`) and the four Docker stacks (pocket-id, beszel, headscale, webfinger)
run on Alpine, Debian, and Alma. Distro differences live in
[`scripts/oslib.sh`](scripts/oslib.sh) — package manager (`apk`/`apt`/`dnf`),
init system (OpenRC/systemd), sshd service name, the per-distro `sftp-server`
path, hostname, boot hooks, and the sshguard log source/backend.
**simplex** remains **Alpine-targeted** — it depends on `awall` and Tor hidden
services with Alpine-specific wiring, so it isn't part of the tri-distro set.
## SSH login notifications
The harden scripts can install a `pam_exec` hook
([`scripts/ntfy-ssh-login.sh`](scripts/ntfy-ssh-login.sh)) that posts every SSH
login to an [ntfy](https://ntfy.sh) topic. Enable it by passing `NTFY_URL` (and
optionally `NTFY_TOKEN`) when running `harden-ssh.sh` / `harden-jumphost.sh`, or
via the launcher / jumphost cloud-init. Each alert reports:
- the **user** and **source IP**,
- the **SSH key** they authenticated with (fingerprint, via `ExposeAuthInfo`),
- a **region tag** so you know which bastion fired it (derived from the host's
FQDN, e.g. `us-evi-1`),
- best-effort the **next hop** of a ProxyJump (see caveat below).
Filtering is by group: `NOTIFY_GROUPS` limits alerts to certain
groups/security levels, and `NOTIFY_PRIORITY_MAP` sets a per-group ntfy
priority. Config lives at `/etc/ssh-notify.conf` (mode 0600;
[`scripts/ssh-notify.conf.example`](scripts/ssh-notify.conf.example) documents
every key). A publish token is optional — leave it empty for a read-gated topic.
> **Jump-target caveat:** a ProxyJump opens a *direct-tcpip* channel (no
> session), so the destination never reaches `pam_exec`. The bastion only logs
> it at `LogLevel VERBOSE`/`DEBUG`; `harden-jumphost.sh` sets `VERBOSE` and the
> notifier parses the log best-effort. If the target isn't in the log it is
> simply omitted.
## Daily updates
[`scripts/auto-update.sh`](scripts/auto-update.sh) keeps a host patched
unattended — ideal for an SSH-only bastion, where a routine upgrade can barely
break anything. `harden-jumphost.sh` schedules it **by default** (set
`AUTO_UPDATE=0` to skip); `harden-ssh.sh` and `cloud-init/base.yml` take
`AUTO_UPDATE=1`. It runs daily via busybox `crond` (`/etc/periodic/daily`) on
Alpine or a systemd timer on Debian/Alma.
Each run:
- applies all **in-branch** package upgrades (`apk`/`apt`/`dnf`);
- **reports** a new Alpine *branch* (e.g. 3.21 → 3.22) by default — that
rewrites the repo branch, so it's opt-in. Set `ALLOW_RELEASE_UPGRADE=1` to
also **apply** it: it repoints `/etc/apk/repositories` to the newest **stable**
`vX.Y` (never `edge`), runs `apk upgrade --available`, and forces a reboot
flag. Debian/Alma stay report-only;
- detects when a **reboot** is needed (kernel/libc/openssl). `AUTO_REBOOT`
controls it: `0` = never (just flag), `1` = always, **`idle` = only when no
SSH connections are active** — so a bastion reboots itself once the admins and
ProxyJump tunnels have cleared, never mid-session. `harden-jumphost.sh`
defaults the bastion to `idle`. A deferred reboot is tracked in `/run` and
retried each day until it happens;
- sends an ntfy summary (reusing `/etc/ssh-notify.conf`).
Schedule it standalone with `auto-update.sh install` (or via the launcher), run
a pass now with `auto-update.sh run`, and preview safely with
`DRY_RUN=1 auto-update.sh run`.
## License
[MIT](LICENSE).
automations.sh
+211
View File
@@ -0,0 +1,211 @@
#!/usr/bin/env bash
#
# automations.sh -- one command to run or deploy anything in this repo.
#
# Run it two ways:
#
# 1. One-liner on a fresh target host (clones the repo, then launches):
# curl -fsSL https://git.anomalous.dev/57_Wolve/automations/raw/branch/main/automations.sh \
# | REPO_URL=https://git.anomalous.dev/57_Wolve/automations.git bash
#
# 2. From a clone:
# ./automations.sh
#
# It opens a Gum wizard (auto-installed) that lets you:
# • Mode: deploy on THIS host, or build deploy.sh artifacts locally.
# • Pick any deployment (pocket-id, beszel, headscale, webfinger, simplex)
# or any generic script (harden-ssh, harden-jumphost, sshuser).
# Shared defaults come from globals/ (see globals/README.md).
#
# Non-interactive: set SKIP_PROMPTS=1 plus the needed vars and pipe the menu
# choices in, or just call the underlying deployments/<name>/deploy.sh
# directly -- they all honor SKIP_PROMPTS=1.
set -euo pipefail
# ----------------------------------------------------------------------------
# Self-locate, or bootstrap by cloning the repo (one-liner / piped form).
# ----------------------------------------------------------------------------
_self="${BASH_SOURCE[0]:-}"
if [[ -n "$_self" && -f "$(cd "$(dirname "$_self")" 2>/dev/null && pwd)/scripts/lib.sh" ]]; then
ROOT="$(cd "$(dirname "$_self")" && pwd)"
else
# Piped via curl: we don't have the repo on disk. Clone it, then re-exec.
: "${REPO_URL:=}"
: "${REPO_BRANCH:=main}"
[[ -n "$REPO_URL" ]] || {
echo "[x] Running standalone (piped). Set REPO_URL=... so I can clone the repo." >&2
exit 1
}
command -v git >/dev/null 2>&1 || { command -v apk >/dev/null 2>&1 && apk add -q git; }
_tmp="$(mktemp -d -t automations.XXXXXX)"
echo "[+] Cloning $REPO_URL ($REPO_BRANCH)..."
git clone --depth 1 --branch "$REPO_BRANCH" "$REPO_URL" "$_tmp"
exec bash "$_tmp/automations.sh" "$@"
fi
# shellcheck source=scripts/lib.sh
. "$ROOT/scripts/lib.sh"
load_globals
DEPLOYMENTS=(pocket-id beszel headscale webfinger simplex)
SCRIPTS=(setup-host harden-ssh harden-jumphost sshuser auto-update)
# ----------------------------------------------------------------------------
# Prompt helpers (gum). `ask` records each answer in ENVS for passing onward.
# ----------------------------------------------------------------------------
ENVS=()
ask() { # <VAR> <label> [password|optional]
local var="$1" label="$2" mode="${3:-}"
local cur="${!var:-}" val # indirect ref needs var already declared
if [[ "$mode" == "password" ]]; then
val="$(gum input --password --header "$label")"
else
val="$(gum input --header "$label" --value "$cur" --placeholder "$cur")"
fi
[[ "$mode" == "optional" && -z "$val" ]] && return 0
printf -v "$var" '%s' "$val"
ENVS+=("$var=$val")
}
# Which values to ask for, per deployment. Generated secrets are produced by
# the deploy scripts themselves and are intentionally not listed here.
ask_deployment_vars() {
case "$1" in
pocket-id)
ask POCKETID_DOMAIN "Public hostname (e.g. id.example.com)"
ask ACME_EMAIL "Let's Encrypt email" ;;
beszel)
ask BESZEL_DOMAIN "Public hostname (e.g. monitoring.example.com)"
ask ACME_EMAIL "Let's Encrypt email" ;;
headscale)
ask HEADSCALE_DOMAIN "headscale hostname (e.g. hs.example.com)"
ask ACME_EMAIL "Let's Encrypt email"
ask TAILNET_DOMAIN "Tailnet MagicDNS base (e.g. tail.example.com)"
ask POCKETID_DOMAIN "OIDC issuer hostname (your pocket-id)"
ask OIDC_CLIENT_ID "OIDC client_id (from pocket-id)"
ask OIDC_CLIENT_SECRET "OIDC client_secret (from pocket-id)" password ;;
webfinger)
ask BASE_DOMAIN "Apex domain to serve (e.g. example.com)"
ask ISSUER_URL "OIDC issuer URL (e.g. https://auth.example.com)"
ask REDIRECT_URL "Redirect target for other traffic (e.g. https://example.org)"
ask ACME_EMAIL "Let's Encrypt email" ;;
simplex)
ask DOMAIN "Apex domain (creates smp.DOMAIN, xftp.DOMAIN)"
ask ACME_EMAIL "Let's Encrypt email"
ask XFTP_QUOTA "XFTP disk quota" optional
ask SSH_PORT "SSH port" optional
ask ALLOWED_IP "Your IP to whitelist in sshguard" optional ;;
esac
}
ask_script_vars() {
case "$1" in
setup-host)
ask HOST "Hostname <svc>-<n> or FQDN (e.g. sto-1)"
ask BASE_DOMAIN "Base domain" optional
ask DATACENTER "Data center label" optional ;;
harden-ssh)
ask SSH_PORT "SSH port to listen on" optional
ask ALLOWED_IP "Your IP to whitelist in sshguard" optional
ask NTFY_URL "ntfy login-notify URL (blank to skip)" optional
if [[ -n "${NTFY_URL:-}" ]]; then ask NTFY_TOKEN "ntfy bearer token (blank if unauth publish)" password; fi ;;
harden-jumphost)
ask SSH_PORT "SSH port to listen on" optional
ask ALLOWED_IP "Your IP to whitelist in sshguard" optional
ask JUMP_TARGETS "Allowed ProxyJump targets (host:port, space-separated)" optional
ask NTFY_URL "ntfy login-notify URL (blank to skip)" optional
if [[ -n "${NTFY_URL:-}" ]]; then ask NTFY_TOKEN "ntfy bearer token (blank if unauth publish)" password; fi ;;
auto-update)
ask AUTO_REBOOT "Auto-reboot when needed? (0=never, 1=always, idle=when no SSH active)" optional
ask ALLOW_RELEASE_UPGRADE "Also upgrade to a new Alpine stable release? (1/0)" optional ;;
sshuser)
: ;; # sshuser.sh has its own interactive interface
esac
}
require_root() {
[[ $EUID -eq 0 ]] || _die "Deploying on this host must run as root."
os_detect # validates the distro is supported (Alpine/Debian/Alma)
}
# ----------------------------------------------------------------------------
# Dispatch
# ----------------------------------------------------------------------------
bootstrap_deployment() {
local name="$1"
require_root
ask_deployment_vars "$name"
if [[ "$name" == "simplex" ]]; then
# install-simplex.sh re-clones REPO_URL and runs harden + deploy + backup.
[[ -n "${REPO_URL:-}" ]] || _die "REPO_URL is required for simplex (set it in globals.env)."
_log "Launching simplex installer..."
env "${ENVS[@]}" REPO_URL="$REPO_URL" REPO_BRANCH="${REPO_BRANCH:-main}" SKIP_PROMPTS=1 \
bash "$ROOT/deployments/simplex/install-simplex.sh"
else
_log "Deploying $name on this host..."
env "${ENVS[@]}" SKIP_PROMPTS=1 bash "$ROOT/deployments/$name/deploy.sh"
fi
}
bootstrap_script() {
local name="$1"
require_root
ask_script_vars "$name"
# auto-update from the menu means "schedule the daily job".
local subcmd=""
[[ "$name" == "auto-update" ]] && subcmd="install"
_log "Running $name on this host..."
env "${ENVS[@]}" FORCE=1 bash "$ROOT/scripts/$name.sh" $subcmd
}
build_deployment() {
local name="$1"
if [[ "$name" == "simplex" ]]; then
_warn "simplex has no embedded-archive build step; it deploys via install-simplex.sh."
return 0
fi
local dir="$ROOT/deployments/$name"
[[ -f "$dir/build.sh" ]] || _die "$name has no build.sh."
_log "Building $name/deploy.sh..."
bash "$dir/build.sh"
if gum confirm "scp $name/deploy.sh to a host now?"; then
local target port
target="$(gum input --header "scp target (user@host)" --placeholder "root@host")"
port="$(gum input --header "SSH port" --value "${SSH_PORT:-22}")"
[[ -n "$target" ]] || { _warn "No target given; skipping scp."; return 0; }
scp -P "${port:-22}" "$dir/deploy.sh" "$target:"
_log "Copied. On the host, run: bash deploy.sh"
fi
}
# ----------------------------------------------------------------------------
# Wizard
# ----------------------------------------------------------------------------
ensure_gum
MODE="$(gum choose --header "What do you want to do?" \
"Deploy on this host" \
"Build deploy.sh artifacts locally")"
case "$MODE" in
"Deploy on this host")
CHOICE="$(printf '%s\n' \
"${DEPLOYMENTS[@]/#/deploy: }" \
"${SCRIPTS[@]/#/script: }" \
| gum choose --header "Pick a deployment or script")"
kind="${CHOICE%%: *}"; name="${CHOICE#*: }"
case "$kind" in
deploy) bootstrap_deployment "$name" ;;
script) bootstrap_script "$name" ;;
*) _die "Nothing selected." ;;
esac ;;
"Build deploy.sh artifacts locally")
name="$(printf '%s\n' "${DEPLOYMENTS[@]}" | gum choose --header "Pick a deployment to build")"
[[ -n "$name" ]] || _die "Nothing selected."
build_deployment "$name" ;;
*)
_die "Nothing selected." ;;
esac
_log "Done."
build-bundle.sh
+102
View File
@@ -0,0 +1,102 @@
#!/usr/bin/env bash
#
# build-bundle.sh -- package the whole repo into a single self-extracting
# script, dist/automations-bundle.sh. Host that one file on a webserver (or a
# public Gitea release) and run it on a target host -- no repo/git access
# needed.
#
# Usage:
# ./build-bundle.sh # bundle the committed tree at HEAD
# ./build-bundle.sh v1.2.0 # bundle a specific tag/ref
# ./build-bundle.sh worktree # bundle the current working tree
# # (tracked + new files, minus ignored)
# OUT=/tmp/x.sh ./build-bundle.sh # custom output path
#
# The payload excludes ignored files (dist/, .env, globals.env,
# ssh-notify.conf, keys, backups) so no secrets are embedded.
set -euo pipefail
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REF="${1:-HEAD}"
OUT="${OUT:-$DIR/dist/automations-bundle.sh}"
MARKER="__ARCHIVE_BELOW__"
command -v git >/dev/null 2>&1 || { echo "git is required." >&2; exit 1; }
mkdir -p "$(dirname "$OUT")"
# ---------------------------------------------------------------------------
# Build the repo tarball (gzipped), then base64-encode it.
# ---------------------------------------------------------------------------
make_tar() {
case "$REF" in
worktree|WORKTREE|.)
# Tracked + untracked-but-not-ignored files = current state minus
# ignored (secrets, dist/). Respects .gitignore via --exclude-standard.
git -C "$DIR" ls-files -co --exclude-standard -z \
| tar -czf - -C "$DIR" --null -T - ;;
*)
# Clean tar of the committed tree at REF (ignored files never tracked).
git -C "$DIR" archive --format=tar "$REF" | gzip -9 ;;
esac
}
echo "Packaging repo ($REF)..."
PAYLOAD="$(make_tar | base64 | tr -d '\n')"
# ---------------------------------------------------------------------------
# Emit the self-extracting stub, then the payload after the marker.
# ---------------------------------------------------------------------------
{
cat <<'STUB'
#!/usr/bin/env bash
#
# automations-bundle.sh -- self-extracting bundle of the automations repo.
# Generated by build-bundle.sh. Download, then run (it can't extract from a
# pipe -- it needs to read itself as a file):
#
# curl -fsSLO https://your-host/automations-bundle.sh
# bash automations-bundle.sh # launcher wizard
# bash automations-bundle.sh bash scripts/setup-host.sh # run a script
# SSH_PORT=2222 bash automations-bundle.sh bash scripts/harden-jumphost.sh
#
# Env:
# INSTALL_DIR where to extract (default /opt/automations)
# BUNDLE_KEEP 1 to keep the extracted repo (default), 0 to use a temp dir
# and remove it after the command finishes
set -euo pipefail
: "${INSTALL_DIR:=/opt/automations}"
: "${BUNDLE_KEEP:=1}"
SELF="${BASH_SOURCE[0]:-$0}"
if [[ ! -f "$SELF" ]]; then
echo "[x] Run me as a downloaded file, not via a pipe:" >&2
echo " curl -fsSLO <url>/automations-bundle.sh && bash automations-bundle.sh" >&2
exit 1
fi
if [[ "$BUNDLE_KEEP" != "1" ]]; then
INSTALL_DIR="$(mktemp -d -t automations.XXXXXX)"
trap 'rm -rf "$INSTALL_DIR"' EXIT
fi
echo "[+] Extracting bundle to $INSTALL_DIR ..."
mkdir -p "$INSTALL_DIR"
sed -e '1,/^__ARCHIVE_BELOW__$/d' "$SELF" | base64 -d | tar -xz -C "$INSTALL_DIR"
cd "$INSTALL_DIR"
chmod +x automations.sh build-bundle.sh scripts/*.sh deployments/*/*.sh 2>/dev/null || true
if [[ "$#" -gt 0 ]]; then
exec "$@"
else
exec bash ./automations.sh
fi
__ARCHIVE_BELOW__
STUB
printf '%s\n' "$PAYLOAD"
} > "$OUT"
chmod +x "$OUT"
echo "Built $OUT ($(wc -c < "$OUT") bytes) from $REF"
cloud-init/README.md
+30
View File
@@ -0,0 +1,30 @@
# cloud-init/
Generic, distro-agnostic cloud-init templates for standing up a **base host**
or a **bastion** from scratch. They run on Alpine, Debian, or Alma — the
runcmd prelude detects the package manager and installs `bash`/`git`/`curl`
before invoking the scripts.
| Template | What it does |
|----------|--------------|
| [`base.yml`](base.yml) | Hostname (per the schema) + shared MOTD + seed root keys from `globals/` + SSH hardening (`harden-ssh.sh`). |
| [`jumphost.yml`](jumphost.yml) | Same base, but bastion hardening (`harden-jumphost.sh`) with an `ssh-admins`/`ssh-jumpers` split and a ProxyJump whitelist. |
These are for the **host itself**. To stand up a Docker stack on a host,
use the per-deployment `cloud-init.yml` under `deployments/<name>/` instead
(those clone the repo and run the stack's `deploy.sh`).
## Usage
1. Copy the template, fill in `REPO_URL`, `HOST` (`<svc>-<n>`, e.g. `sto-1`),
and the other values at the top of the `runcmd` block.
2. Paste it as the instance's **user-data** when creating the VM.
3. On first boot the host names itself, installs the MOTD, seeds admin keys
from `globals/`, and hardens SSH.
Hostnames follow [`../globals/Network Domain Name Schema.md`](../globals/Network%20Domain%20Name%20Schema.md);
our VMs skip the region code and use `srvno.de` as the base.
> The harden scripts print a generated root private key to stdout, which lands
> in the cloud provider's serial/console log. Capture it there, or rely on the
> keys seeded from `globals/authorized_keys` (or `SSH_KEYS_URL`) and ignore it.
cloud-init/base.yml
+54
View File
@@ -0,0 +1,54 @@
#cloud-config
#
# Generic base-host bootstrap -- Alpine, Debian, or Alma Linux.
#
# On first boot this:
# 1. Installs prerequisites (bash, git, curl) for whichever distro this is.
# 2. Clones this repo to /opt/automations.
# 3. Sets the hostname per the Network Domain Name Schema and installs the
# shared MOTD banner (scripts/setup-host.sh).
# 4. Seeds root's authorized_keys from globals/ (URL-preferred).
# 5. Applies SSH hardening: key-only auth, post-quantum KEX, sshguard
# (scripts/harden-ssh.sh).
#
# Fill in REPO_URL, HOST, and the other values, then paste as instance
# user-data. For a bastion host use jumphost.yml instead.
#
# NOTE: harden-ssh.sh prints a freshly generated root private key to stdout,
# which lands in the cloud provider's console/serial log. Either capture it
# from there, or rely on the keys seeded from globals/ and ignore it.
runcmd:
- |
set -e
# ===== config =====
REPO_URL=https://git.anomalous.dev/57_Wolve/automations.git
REPO_BRANCH=main
HOST=sto-1 # <svc>-<n>; FQDN becomes HOST.BASE_DOMAIN
BASE_DOMAIN=srvno.de
DATACENTER="Globally Everywhere"
SSH_PORT=22
ALLOWED_IP= # optional: whitelist your client IP in sshguard
AUTO_UPDATE=1 # schedule daily unattended updates (0 to skip)
# ==================
# Prerequisites (OS-agnostic).
if command -v apk >/dev/null 2>&1; then apk add --no-cache bash git curl
elif command -v apt-get >/dev/null 2>&1; then apt-get update -qq && apt-get install -y -qq bash git curl
elif command -v dnf >/dev/null 2>&1; then dnf install -y -q bash git curl
fi
git clone --depth 1 --branch "$REPO_BRANCH" "$REPO_URL" /opt/automations
cd /opt/automations
# Hostname + shared MOTD.
HOST="$HOST" BASE_DOMAIN="$BASE_DOMAIN" DATACENTER="$DATACENTER" bash scripts/setup-host.sh
# Seed root's authorized_keys from globals/ (SSH_KEYS_URL or authorized_keys).
. scripts/lib.sh && load_globals \
&& install -d -m 700 /root/.ssh \
&& resolve_ssh_keys >> /root/.ssh/authorized_keys || true
sort -u /root/.ssh/authorized_keys -o /root/.ssh/authorized_keys 2>/dev/null || true
# SSH hardening (key-only, PQ KEX, sshguard).
SSH_PORT="$SSH_PORT" ALLOWED_IP="$ALLOWED_IP" FORCE=1 bash scripts/harden-ssh.sh
cloud-init/jumphost.yml
+59
View File
@@ -0,0 +1,59 @@
#cloud-config
#
# Jump-host (bastion) bootstrap -- Alpine, Debian, or Alma Linux.
#
# Same base steps as base.yml, but applies the bastion hardening
# (scripts/harden-jumphost.sh) instead of plain SSH hardening:
# - ssh-admins : full shell for maintenance
# - ssh-jumpers : ProxyJump only, to the JUMP_TARGETS whitelist
#
# Fill in REPO_URL, HOST, and JUMP_TARGETS, then paste as instance user-data.
# Add jumper/admin users afterwards with the installed `sshuser` tool.
#
# NOTE: harden-jumphost.sh prints a freshly generated root private key to
# stdout (-> the cloud console/serial log). Capture it there, or rely on the
# keys seeded from globals/ and ignore it.
runcmd:
- |
set -e
# ===== config =====
REPO_URL=https://git.anomalous.dev/57_Wolve/automations.git
REPO_BRANCH=main
HOST=ssh-1 # <svc>-<n>; "ssh" is the bastion service code
BASE_DOMAIN=srvno.de
DATACENTER="Globally Everywhere"
SSH_PORT=22
ALLOWED_IP= # optional: whitelist your client IP
JUMP_TARGETS="10.0.0.5:22 10.0.0.6:22" # hosts jumpers may ProxyJump to
# Optional login notifications (pam_exec -> ntfy). Leave NTFY_URL empty to
# skip. NTFY_REGION defaults to the region segment of this host's FQDN.
NTFY_URL=
NTFY_TOKEN= # bearer token; empty if unauth publish
NTFY_EMAIL=
NTFY_REGION=
# ==================
# Prerequisites (OS-agnostic).
if command -v apk >/dev/null 2>&1; then apk add --no-cache bash git curl
elif command -v apt-get >/dev/null 2>&1; then apt-get update -qq && apt-get install -y -qq bash git curl
elif command -v dnf >/dev/null 2>&1; then dnf install -y -q bash git curl
fi
git clone --depth 1 --branch "$REPO_BRANCH" "$REPO_URL" /opt/automations
cd /opt/automations
# Hostname + shared MOTD.
HOST="$HOST" BASE_DOMAIN="$BASE_DOMAIN" DATACENTER="$DATACENTER" bash scripts/setup-host.sh
# Seed root's (ssh-admins) authorized_keys from globals/.
. scripts/lib.sh && load_globals \
&& install -d -m 700 /root/.ssh \
&& resolve_ssh_keys >> /root/.ssh/authorized_keys || true
sort -u /root/.ssh/authorized_keys -o /root/.ssh/authorized_keys 2>/dev/null || true
# Bastion hardening (admins shell + jumpers ProxyJump whitelist + optional
# login notifications).
SSH_PORT="$SSH_PORT" ALLOWED_IP="$ALLOWED_IP" JUMP_TARGETS="$JUMP_TARGETS" \
NTFY_URL="$NTFY_URL" NTFY_TOKEN="$NTFY_TOKEN" NTFY_EMAIL="$NTFY_EMAIL" NTFY_REGION="$NTFY_REGION" \
FORCE=1 bash scripts/harden-jumphost.sh
deployments/beszel/.env.example
+17
View File
@@ -0,0 +1,17 @@
# Copy to .env and fill in. docker compose picks .env up automatically.
# ─── Public hostname ────────────────────────────────────────────────────────
# Bare hostname (no scheme) where beszel will be reached.
BESZEL_DOMAIN=monitoring.example.com
# Email for Let's Encrypt registration / expiry notifications.
ACME_EMAIL=admin@example.com
# Allow OIDC sign-in to auto-create user accounts on first login. Set to
# "false" if you want to manually pre-create users in the admin UI before
# anyone can log in.
USER_CREATION=true
# ─── Image tags ─────────────────────────────────────────────────────────────
BESZEL_TAG=latest
CADDY_TAG=2-alpine
deployments/beszel/Caddyfile
+34
View File
@@ -0,0 +1,34 @@
# Caddyfile for beszel stack.
#
# Auto-issues a Let's Encrypt cert for $BESZEL_DOMAIN and reverse-proxies
# to the beszel hub on :8090. Agents and the web UI both go through this.
{
email {$ACME_EMAIL}
}
{$BESZEL_DOMAIN} {
encode zstd gzip
reverse_proxy beszel:8090 {
header_up X-Real-IP {http.request.remote.host}
# WebSocket / long-lived agent push streams.
flush_interval -1
transport http {
read_timeout 10m
write_timeout 10m
}
}
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains"
X-Content-Type-Options "nosniff"
Referrer-Policy "strict-origin-when-cross-origin"
-Server
}
log {
output stdout
format console
}
}
deployments/beszel/README.md
+39
View File
@@ -0,0 +1,39 @@
# beszel
[Beszel](https://beszel.dev) monitoring hub behind Caddy. Agents push metrics to
the hub; OIDC sign-in is configured in the admin UI after first deploy.
## Required `.env` values
| Variable | Notes |
|----------|-------|
| `BESZEL_DOMAIN` | Public hostname (e.g. `monitoring.example.com`). |
| `ACME_EMAIL` | Let's Encrypt registration email. |
| `USER_CREATION` | `true` lets OIDC auto-create accounts on first login. |
See [`.env.example`](.env.example) for image tags.
## Deploy
```bash
./automations.sh # Deploy on this host → deploy: beszel
```
Or build + run the self-contained artifact:
```bash
./build.sh
scp deploy.sh root@host:
ssh root@host 'bash deploy.sh'
# non-interactive:
# BESZEL_DOMAIN=monitoring.example.com ACME_EMAIL=me@example.com SKIP_PROMPTS=1 bash deploy.sh
```
Unattended provisioning: [`cloud-init.yml`](cloud-init.yml).
## Notes
- No Anubis PoW gate here — it would break agent connections.
- For OIDC login, point Beszel at your [pocket-id](../pocket-id/) in the admin UI.
- DNS for `BESZEL_DOMAIN` must resolve to the host and 80/443 be reachable
before deploy.
deployments/beszel/build.sh
+34
View File
@@ -0,0 +1,34 @@
#!/usr/bin/env bash
#
# build.sh -- (re)embed docker-compose.yml, Caddyfile, .env.example into
# deploy.sh as a base64-encoded tar.gz payload after __ARCHIVE_BELOW__.
# Idempotent: strips any existing payload first.
set -euo pipefail
DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
SCRIPT="$DIR/deploy.sh"
MARKER="__ARCHIVE_BELOW__"
[[ -f "$SCRIPT" ]] || { echo "deploy.sh not found at $SCRIPT" >&2; exit 1; }
for f in docker-compose.yml Caddyfile .env.example; do
[[ -f "$DIR/$f" ]] || { echo "Missing $DIR/$f" >&2; exit 1; }
done
PAYLOAD=$(tar -czf - -C "$DIR" docker-compose.yml Caddyfile .env.example | base64)
TMP=$(mktemp)
trap 'rm -f "$TMP"' EXIT
sed "/^${MARKER}\$/,\$d" "$SCRIPT" > "$TMP"
{
echo "$MARKER"
echo "$PAYLOAD"
} >> "$TMP"
mv "$TMP" "$SCRIPT"
chmod +x "$SCRIPT"
trap - EXIT
size=$(wc -c < "$SCRIPT")
echo "Built $SCRIPT (${size} bytes)"
deployments/beszel/cloud-init.yml
+32
View File
@@ -0,0 +1,32 @@
#cloud-config
#
# Beszel (monitoring hub) — unattended deploy on a fresh Alpine host.
#
# Fill in REPO_URL and the values in the runcmd block, then paste this as the
# instance user-data. DNS for BESZEL_DOMAIN must point at this host and ports
# 80/443 must be reachable before boot, or the LE cert request fails.
# OIDC sign-in is configured afterwards in the Beszel admin UI.
packages:
- git
runcmd:
- hostnamectl set-hostname beszel || true
- |
set -e
REPO_URL=https://git.anomalous.dev/57_Wolve/automations.git
REPO_BRANCH=main
git clone --depth 1 --branch "$REPO_BRANCH" "$REPO_URL" /opt/automations
cd /opt/automations
# Optional: install shared admin SSH keys from globals/ for root before deploy
# . scripts/lib.sh && load_globals \
# && install -d -m700 /root/.ssh && resolve_ssh_keys >> /root/.ssh/authorized_keys
# Optional: harden SSH first (prints a fresh root key — capture it!)
# SKIP_PROMPTS=1 FORCE=1 bash scripts/harden-ssh.sh
BESZEL_DOMAIN=monitoring.example.com \
ACME_EMAIL=admin@example.com \
SKIP_PROMPTS=1 \
bash deployments/beszel/deploy.sh
deployments/beszel/deploy.sh
+293
View File
@@ -0,0 +1,293 @@
#!/usr/bin/env bash
#
# deploy.sh -- deploy the beszel stack (caddy + beszel hub) on Alpine.
#
# Single-node: runs as root.
#
# What this does:
# 1. Installs docker + docker-cli-compose if missing.
# 2. Lays down docker-compose.yml, Caddyfile, .env.example in $STACK_DIR.
# 3. Generates .env on first run; existing .env is never overwritten.
# 4. Prompts for required values not preset (BESZEL_DOMAIN, ACME_EMAIL).
# 5. Opens TCP 80/443 in UFW if active.
# 6. Pulls images, brings the stack up, waits for healthchecks.
#
# Idempotent: re-run to apply config changes / pull new images.
#
# Self-contained: docker-compose.yml, Caddyfile, .env.example are embedded
# as a base64-encoded tar.gz at the bottom of this file. Rebuild with
# build.sh after editing the loose source files.
#
# Usage:
# bash deploy.sh # interactive prompts
# BESZEL_DOMAIN=monitoring.example.com \
# ACME_EMAIL=admin@example.com FORCE=1 bash deploy.sh
# STACK_DIR=/opt/beszel bash deploy.sh
# SKIP_DOCKER_INSTALL=1 bash deploy.sh
set -euo pipefail
: "${STACK_DIR:=/srv/beszel}"
: "${SKIP_DOCKER_INSTALL:=0}"
: "${FORCE:=0}"
: "${SKIP_PROMPTS:=0}" # non-interactive: require values via env, no prompts
[[ "$SKIP_PROMPTS" == "1" ]] && FORCE=1
: "${BESZEL_DOMAIN:=}"
: "${ACME_EMAIL:=}"
log() { printf '\033[1;32m[+]\033[0m %s\n' "$*"; }
warn() { printf '\033[1;33m[!]\033[0m %s\n' "$*" >&2; }
die() { printf '\033[1;31m[x]\033[0m %s\n' "$*" >&2; exit 1; }
[[ $EUID -eq 0 ]] || die "Run as root."
# ---------------------------------------------------------------------------
# OS detection + Docker install (Alpine / Debian / Alma). This deploy.sh is
# self-contained (scp'd standalone), so the OS logic is inlined here instead
# of sourced from scripts/oslib.sh.
# ---------------------------------------------------------------------------
osfam() {
local id="" like=""
if [[ -r /etc/os-release ]]; then
id="$(. /etc/os-release 2>/dev/null && echo "${ID:-}")"
like="$(. /etc/os-release 2>/dev/null && echo "${ID_LIKE:-}")"
fi
case " $id $like " in
*" alpine "*) echo alpine ;;
*" debian "*|*" ubuntu "*) echo debian ;;
*" rhel "*|*" fedora "*|*" centos "*) echo rhel ;;
*) echo "${id:-unknown}" ;;
esac
}
install_docker() {
if command -v docker >/dev/null 2>&1; then
log "Docker already installed: $(docker --version)"
else
log "Installing Docker (OS: $(osfam))..."
case "$(osfam)" in
alpine) apk add -q docker docker-cli-compose openrc ;;
debian|rhel) command -v curl >/dev/null 2>&1 || \
{ command -v apt-get >/dev/null 2>&1 && apt-get install -y -qq curl; } || \
{ command -v dnf >/dev/null 2>&1 && dnf install -y -q curl; }
curl -fsSL https://get.docker.com | sh ;;
*) die "Unsupported OS for auto Docker install. Set SKIP_DOCKER_INSTALL=1 and install Docker yourself." ;;
esac
fi
if command -v rc-update >/dev/null 2>&1; then
rc-update add docker default >/dev/null 2>&1 || true
rc-service docker status >/dev/null 2>&1 || rc-service docker start
elif command -v systemctl >/dev/null 2>&1; then
systemctl enable --now docker >/dev/null 2>&1 || systemctl start docker || true
fi
}
open_web_ports() {
if command -v ufw >/dev/null 2>&1 && ufw status 2>/dev/null | grep -q '^Status: active'; then
log "ufw active -- allowing 80,443/tcp..."
ufw allow 80/tcp >/dev/null; ufw allow 443/tcp >/dev/null
elif command -v firewall-cmd >/dev/null 2>&1 && firewall-cmd --state >/dev/null 2>&1; then
log "firewalld active -- allowing http,https..."
firewall-cmd -q --add-service=http --permanent
firewall-cmd -q --add-service=https --permanent
firewall-cmd -q --reload
fi
}
# ----------------------------------------------------------------------------
# Extract embedded archive
# ----------------------------------------------------------------------------
SCRIPT_DIR=$(mktemp -d -t beszel-deploy.XXXXXX)
trap 'rm -rf "$SCRIPT_DIR"' EXIT
extract_archive() {
grep -a -A 9999999 '^__ARCHIVE_BELOW__$' "$0" \
| tail -n +2 \
| base64 -d \
| tar -xz -C "$SCRIPT_DIR"
}
if grep -q -a '^__ARCHIVE_BELOW__$' "$0"; then
log "Extracting embedded deployment files..."
extract_archive
else
die "No embedded archive found. Run build.sh to embed deployment files."
fi
for f in docker-compose.yml Caddyfile .env.example; do
[[ -f "$SCRIPT_DIR/$f" ]] || die "Embedded archive missing $f"
done
# ----------------------------------------------------------------------------
# Prompt for required vars
# ----------------------------------------------------------------------------
prompt() {
local varname="$1" message="$2"
local -n ref="$varname"
if [[ -z "${ref:-}" ]]; then
[[ "$SKIP_PROMPTS" == "1" ]] && die "$varname required (set it in the environment; running with SKIP_PROMPTS=1)."
read -r -p "$message: " ref
[[ -n "$ref" ]] || die "$varname required."
fi
}
prompt BESZEL_DOMAIN "Public hostname for beszel (e.g. monitoring.example.com)"
prompt ACME_EMAIL "Let's Encrypt email"
# ----------------------------------------------------------------------------
# Docker
# ----------------------------------------------------------------------------
if [[ "$SKIP_DOCKER_INSTALL" != "1" ]]; then
install_docker
fi
# Open 80/443 on whichever host firewall is active.
open_web_ports
# ----------------------------------------------------------------------------
# Stack directory + files
# ----------------------------------------------------------------------------
log "Setting up $STACK_DIR..."
install -d -m 0750 "$STACK_DIR"
install -m 0640 "$SCRIPT_DIR/docker-compose.yml" "$STACK_DIR/docker-compose.yml"
install -m 0640 "$SCRIPT_DIR/Caddyfile" "$STACK_DIR/Caddyfile"
ENV_FILE="$STACK_DIR/.env"
if [[ ! -f "$ENV_FILE" ]]; then
log "Seeding $ENV_FILE..."
install -m 0600 "$SCRIPT_DIR/.env.example" "$ENV_FILE"
sed -i \
-e "s|^BESZEL_DOMAIN=.*|BESZEL_DOMAIN=${BESZEL_DOMAIN}|" \
-e "s|^ACME_EMAIL=.*|ACME_EMAIL=${ACME_EMAIL}|" \
"$ENV_FILE"
else
log ".env exists; leaving it alone."
fi
# Validate
missing=()
for var in BESZEL_DOMAIN ACME_EMAIL; do
grep -E "^${var}=.+$" "$ENV_FILE" >/dev/null || missing+=("$var")
done
(( ${#missing[@]} == 0 )) || die "Missing values in $ENV_FILE: ${missing[*]}"
# ----------------------------------------------------------------------------
# Bring up the stack
# ----------------------------------------------------------------------------
if [[ "$FORCE" != "1" ]]; then
cat <<EOF
About to pull images and start the stack from $STACK_DIR.
Caddy will request a Let's Encrypt cert for ${BESZEL_DOMAIN}. DNS for
that name must already point at this host, and ports 80/443 must be
reachable from the internet.
Continue? [y/N]
EOF
read -r ans
[[ "${ans,,}" == "y" || "${ans,,}" == "yes" ]] || { warn "Aborted."; exit 0; }
fi
cd "$STACK_DIR"
log "Pulling images..."
docker compose pull
log "Starting stack..."
docker compose up -d --remove-orphans
# ----------------------------------------------------------------------------
# Wait for health
# ----------------------------------------------------------------------------
log "Waiting for services to become healthy (up to 120s)..."
deadline=$(( $(date +%s) + 120 ))
while (( $(date +%s) < deadline )); do
status=$(docker compose ps --format '{{.Service}} {{.Health}}' 2>/dev/null || true)
unhealthy=$(echo "$status" | awk '$2 != "healthy" && $2 != "" {print $1}')
if [[ -z "$unhealthy" && -n "$status" ]]; then
log "All services healthy."
break
fi
sleep 5
done
echo
log "Stack status:"
docker compose ps
echo
cat <<EOF
================================================================
DEPLOYED
URL: https://${BESZEL_DOMAIN}
Stack dir: ${STACK_DIR}
First-time setup:
1. Open https://${BESZEL_DOMAIN} -- you'll be asked to create the first
admin account. Do this immediately so no one else can.
2. (Optional) Configure OIDC against pocket-id:
- Open https://${BESZEL_DOMAIN}/_/#/settings
- Toggle off "Hide collection create and edit controls"
- Edit the "users" collection -> Options tab -> enable OAuth2
- Add OIDC provider, redirect URI:
https://${BESZEL_DOMAIN}/api/oauth2-redirect
- Toggle controls back on
Then in pocket-id admin UI, register a new OIDC client with that
redirect URI and paste the credentials into Beszel.
3. (Optional) Once OIDC works, lock out password login: edit
docker-compose.yml, uncomment DISABLE_PASSWORD_AUTH=true, then
docker compose up -d.
Add a monitored host:
Beszel UI -> "Add System" -> follow the agent install snippet.
The agent connects back to this hub at https://${BESZEL_DOMAIN}.
Manage:
docker compose logs -f
docker compose pull && docker compose up -d # update
docker compose down # stop, keep volumes
docker compose down -v # stop, WIPE data
Or just re-run this script -- it's idempotent.
================================================================
EOF
# IMPORTANT: do not put any code below this exit. Everything after the
# __ARCHIVE_BELOW__ marker is the embedded tar.gz payload (base64).
exit 0
__ARCHIVE_BELOW__
H4sIAAAAAAAAA+1Y/27jxhH233qKAX1AfOiRIm3LTli4qCwJjQBfrFg6XNIiEFbkSlqY5DK7S8s6
w0Afok/YJ8nMkhIp99CgwOVyKThnHMX9MTsz++03s4xldM+VG8k0l5p72zQ5+uTio1ycnx/5F35w
2Qvs0/cD245yduFfHAW94PIiuPTPe/6RH5xh8xH4n96U/5RCG6YAjnqX8/cyeeCfY80vSI5hwfUH
ngCGIboH14WIxfEWTmY309fwp13vuljAiebqgStIZSaMVCJbvfY6x51jmMlcJnK1DfE3wLWSGxyJ
c9mKZ0aD+5dSZ3h+fkYvpcrwa/8b307/TkI/KxZCh7sZeaHXkHKjRKRB0prfzmaT7nu+mBJcDbAs
BgYT+R5WzHDYyCKJUdNCcXYPZs1TD67tKl9p6E/GoAu1ZBEHoWGzZma3ToGItybcjocD6oxkthSr
QvEY8DgYN+Z5IreoUclitSbNlV5gcSoyeDeGk4k16Zppjoo0NwYDo1+/gUwaeBAMePYAD0yhI6rS
D0uRcA+mnFuN5SIeuuwMR5Ob2x9HQ4ecSVAxLHEWDeKPLDK4STzXXqeTsZSHVSA7nQeZFCnXYQeq
JjdmhtGrjfuLt9KGsNOh7RRROa/cIPwBIFKMTli1vHoa9IfDH+ez/t9C99RlSS4y/mzHoR7D8E3N
S2vsBNujOJ0pE0KRJVxrVxuZ5zy2fblURpcLAbjgfO0jEJz6HTFCONm1HDfaukWcO7bNouGMwBoL
zRYJbtfJwML2+3fjASJPcQSN5vNcycdtpepjcgx5grhYyyRGkC0KRDTccZdnpBShwjPcq0cee1ZH
I86ltV7XrkrbGXa5ibo2Bo1GJfdjG1vRpf9fdFS70i2fthOBI5TMUoTqbsnr0fTvo5v58PZtf/xd
CM6rp4OW513Y+oO3o/kIm27soPq1GoGQ41ms5zKrnanQRC9rzhKzjtY8ut/1G9zTEP7hDN4OnTfg
bFbc0NP9+dal59qYPOx2g9NLz8d/QXjqB99UznSdnyolIjOIOZaEcObrnWKRclmg7t6uRdHBxzDD
WdVg0TTPuRIyDiHAqXukH0AWt0ttV3G36toHx4I3YeTCx6HbcP2/YReTZIrME4IlQoQfeQ2Ob132
LaE5HwdK81hW5s33IPjIPh/DsAQ2UZQiI7s503ojFfJeYdYEfWQGZDRiLg8GaBlOBwwkGm5Estez
lcVXaGuD2FhW0h2ejQdBsEceI4bZMdobkPiqNkJzmr3XZOkIG/BcJUu7kqTTobQBVWRebfl42r++
Gc0n/en0/e3dcN5/N/sWYWhUwetj3U8SuSkNMZJ8km6E7G1Kj4FFkSyIovdrYIIR+1XeTUd388Hd
qD8b35bn4LDFpcUqrH8CMNPOdlkuuqWuT4tnmvr75f89V/2Ga/xK/Recn5/W9d8FtgdB0Gvrv88i
x7BHgK01mtVgWRv16WwKrQuusea64QZrqlEWqW1uIOLK2GmvDvKQrc+qHOxSDkb4oyY86EQ0jZIS
T7c9XB70y4qMJtKYDV9QcbVAKoKVbBRggsqfp5I3UyYSeHrVyG6dZ+w8tOUZqtFZJGMOH7SJYfVB
5J2K7huFQrMyrWZVBII0OS9y+MG9QwJwxxN4Io7wFP8Zo2LwmUrDvbXcJZiS4+p6tYv0la3cRDwQ
AZOrZY2rDXJeqr39pGWCzfMdr4Ab7HuMYpmm6gls1qnNK91g8bxiHsAMmR70bpQwfN/d7C2tfe7s
iJLSQa15SgW4cWe7ld0pjwrUtQUnZY8u+nF1FvTw+ub7f0YujJIi5tNiMZS4MZl29np+cAeYcNFp
d7bNuXubGyEzTT1OJnUmlst67B1fcqXwRjqRiYi2DS8cXZqDNw/MBC4VZ5gyJObosqXWgXbSRaXp
G2aPhmMYhrygajrGX3XspUrxboCpUsuEV7Off0du/hziYfXh4eUizX+7FPBr/N+7PKv5/9Lyf+/y
tOX/zyHI/zLfEjUTEiz/Yi5I8Dh7ENtvQ1B9G4JcRPe6HIZcSCUbHhcRsSTZIicfw7//9c/yDybF
Ag8vEB9S7Vr3fOF/6MQ1U7w2/CSToLF0TPlrug2qffLaUIwWnIgXu/F+eJBzrupPJLuz5WEUKUgj
m7QoZR5mUsVXAgmOETViuuCPuVBb+oYglhhiS5hep850V7Za/+sL5Y2iWotV5lJl/z8U1zDFVGUk
KnKWLNHcAbGkih82LKMOwNtPQduNdwfe1Khf3iEwNOgifRBh2VZmeP/AWwdxMJXwB6X6FVXqh+gZ
020ODFvpPwxwXuKovnhelffOzv4zytXuK8r/eWJppZVWWmmllVZaaaWVVlpppZVWWmnli5BfAO7B
8Y8AKAAA
deployments/beszel/docker-compose.yml
+65
View File
@@ -0,0 +1,65 @@
# beszel stack -- caddy (TLS) + beszel hub (server monitoring).
#
# Topology:
# Browser + agents -> caddy:443 -> beszel:8090
#
# No Anubis: agents push metrics over HTTP/WebSocket and a PoW gate would
# break them. Beszel's API surface is what agents use.
#
# OIDC is configured post-deploy through the Beszel admin UI (PocketBase
# settings), not via env vars or config file. See the deploy.sh "DEPLOYED"
# block for the exact steps.
name: beszel
volumes:
beszel-data:
caddy-data:
caddy-config:
services:
caddy:
image: caddy:${CADDY_TAG:-2-alpine}
container_name: caddy
restart: unless-stopped
ports:
- "80:80"
- "443:443"
# - "443:443/udp" # HTTP/3 -- disabled (Caddy QUIC + reverse_proxy
# placeholder bug). Re-enable when fixed.
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
- caddy-data:/data
- caddy-config:/config
environment:
BESZEL_DOMAIN: "${BESZEL_DOMAIN}"
ACME_EMAIL: "${ACME_EMAIL}"
depends_on:
- beszel
healthcheck:
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:2019/config/"]
interval: 30s
timeout: 5s
retries: 3
start_period: 10s
beszel:
image: henrygd/beszel:${BESZEL_TAG:-latest}
container_name: beszel
restart: unless-stopped
command: serve --http "0.0.0.0:8090"
volumes:
- beszel-data:/beszel_data
environment:
# Disable username/password auth -- force OIDC. Comment out until
# you've configured an OIDC provider in the admin UI, otherwise you
# lock yourself out on first run.
# DISABLE_PASSWORD_AUTH: "true"
# Allow OIDC to auto-create user accounts on first login.
USER_CREATION: "${USER_CREATION:-true}"
healthcheck:
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:8090/api/health"]
interval: 30s
timeout: 5s
retries: 3
start_period: 30s
deployments/headscale/.env.example
+30
View File
@@ -0,0 +1,30 @@
# Copy to .env and fill in. docker compose picks .env up automatically.
# ─── Public hostnames ───────────────────────────────────────────────────────
# Bare hostname (no scheme) where headscale will be reached. Tailscale
# clients connect here over HTTPS.
HEADSCALE_DOMAIN=hs.example.com
# Email for Let's Encrypt registration / expiry notifications.
ACME_EMAIL=admin@example.com
# Magic-DNS suffix for nodes inside the tailnet. Example: with
# TAILNET_DOMAIN=tail.example.com, a host called "laptop" will resolve to
# laptop.tail.example.com. MUST be different from HEADSCALE_DOMAIN.
TAILNET_DOMAIN=tail.example.com
# ─── OIDC (pocket-id) ───────────────────────────────────────────────────────
# Hostname (no scheme) of your pocket-id deployment, used as the OIDC
# issuer. Must match the issuer URL pocket-id advertises at
# /.well-known/openid-configuration.
POCKETID_DOMAIN=auth.example.com
# Register a new OIDC client in pocket-id's admin UI with redirect URI:
# https://${HEADSCALE_DOMAIN}/oidc/callback
# Then paste the credentials here.
OIDC_CLIENT_ID=
OIDC_CLIENT_SECRET=
# ─── Image tags ─────────────────────────────────────────────────────────────
HEADSCALE_TAG=0.28.0
CADDY_TAG=2-alpine
deployments/headscale/Caddyfile
+36
View File
@@ -0,0 +1,36 @@
# Caddyfile for headscale stack.
#
# Auto-issues a Let's Encrypt cert for $HEADSCALE_DOMAIN and reverse-proxies
# to headscale's HTTP listener on :8080. Tailscale clients require this
# exact hostname over HTTPS.
{
email {$ACME_EMAIL}
}
{$HEADSCALE_DOMAIN} {
encode zstd gzip
reverse_proxy headscale:8080 {
header_up X-Real-IP {http.request.remote.host}
# Long-lived noise/wireguard control streams; lift the default
# idle limit so they don't get torn down.
flush_interval -1
transport http {
read_timeout 10m
write_timeout 10m
}
}
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains"
X-Content-Type-Options "nosniff"
Referrer-Policy "strict-origin-when-cross-origin"
-Server
}
log {
output stdout
format console
}
}
deployments/headscale/README.md
+48
View File
@@ -0,0 +1,48 @@
# headscale
[Headscale](https://headscale.net) — a self-hosted Tailscale control server —
behind Caddy, with OIDC login delegated to [pocket-id](../pocket-id/).
## Prerequisite
Register an OIDC client in pocket-id's admin UI with redirect URI:
```
https://${HEADSCALE_DOMAIN}/oidc/callback
```
Then paste its `OIDC_CLIENT_ID` / `OIDC_CLIENT_SECRET` below.
## Required `.env` values
| Variable | Notes |
|----------|-------|
| `HEADSCALE_DOMAIN` | Public hostname (e.g. `hs.example.com`). Clients connect here over HTTPS. |
| `ACME_EMAIL` | Let's Encrypt registration email. |
| `TAILNET_DOMAIN` | MagicDNS suffix (e.g. `tail.example.com`). Must differ from `HEADSCALE_DOMAIN`. |
| `POCKETID_DOMAIN` | OIDC issuer hostname (your pocket-id). |
| `OIDC_CLIENT_ID` / `OIDC_CLIENT_SECRET` | From the pocket-id client above. |
The deploy substitutes these into `config.yaml` from the embedded template. See
[`.env.example`](.env.example).
## Deploy
```bash
./automations.sh # Deploy on this host → deploy: headscale
```
Or build + run the self-contained artifact:
```bash
./build.sh
scp deploy.sh root@host:
ssh root@host 'bash deploy.sh'
# non-interactive: pass HEADSCALE_DOMAIN, ACME_EMAIL, TAILNET_DOMAIN,
# POCKETID_DOMAIN, OIDC_CLIENT_ID, OIDC_CLIENT_SECRET, SKIP_PROMPTS=1
```
Unattended provisioning: [`cloud-init.yml`](cloud-init.yml).
DNS for `HEADSCALE_DOMAIN` must resolve to the host and 80/443 be reachable
before deploy.
deployments/headscale/build.sh
+38
View File
@@ -0,0 +1,38 @@
#!/usr/bin/env bash
#
# build.sh -- (re)embed docker-compose.yml, Caddyfile, config.yaml, and
# .env.example into deploy.sh as a base64-encoded tar.gz payload after the
# __ARCHIVE_BELOW__ marker. Idempotent: strips any existing payload first.
#
# Run after editing any of the loose files. The resulting deploy.sh is
# self-contained and can be scp'd to the target box on its own.
set -euo pipefail
DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
SCRIPT="$DIR/deploy.sh"
MARKER="__ARCHIVE_BELOW__"
[[ -f "$SCRIPT" ]] || { echo "deploy.sh not found at $SCRIPT" >&2; exit 1; }
for f in docker-compose.yml Caddyfile config.yaml .env.example; do
[[ -f "$DIR/$f" ]] || { echo "Missing $DIR/$f" >&2; exit 1; }
done
PAYLOAD=$(tar -czf - -C "$DIR" \
docker-compose.yml Caddyfile config.yaml .env.example | base64)
TMP=$(mktemp)
trap 'rm -f "$TMP"' EXIT
sed "/^${MARKER}\$/,\$d" "$SCRIPT" > "$TMP"
{
echo "$MARKER"
echo "$PAYLOAD"
} >> "$TMP"
mv "$TMP" "$SCRIPT"
chmod +x "$SCRIPT"
trap - EXIT
size=$(wc -c < "$SCRIPT")
echo "Built $SCRIPT (${size} bytes)"
deployments/headscale/cloud-init.yml
+36
View File
@@ -0,0 +1,36 @@
#cloud-config
#
# Headscale (Tailscale control server) — unattended deploy on fresh Alpine.
#
# Fill in REPO_URL and the values in the runcmd block, then paste this as the
# instance user-data. DNS for HEADSCALE_DOMAIN must point at this host and
# ports 80/443 reachable before boot. Requires an OIDC client pre-registered
# in your pocket-id with redirect URI https://HEADSCALE_DOMAIN/oidc/callback.
packages:
- git
runcmd:
- hostnamectl set-hostname headscale || true
- |
set -e
REPO_URL=https://git.anomalous.dev/57_Wolve/automations.git
REPO_BRANCH=main
git clone --depth 1 --branch "$REPO_BRANCH" "$REPO_URL" /opt/automations
cd /opt/automations
# Optional: install shared admin SSH keys from globals/ for root before deploy
# . scripts/lib.sh && load_globals \
# && install -d -m700 /root/.ssh && resolve_ssh_keys >> /root/.ssh/authorized_keys
# Optional: harden SSH first (prints a fresh root key — capture it!)
# SKIP_PROMPTS=1 FORCE=1 bash scripts/harden-ssh.sh
HEADSCALE_DOMAIN=hs.example.com \
ACME_EMAIL=admin@example.com \
TAILNET_DOMAIN=tail.example.com \
POCKETID_DOMAIN=auth.example.com \
OIDC_CLIENT_ID=changeme \
OIDC_CLIENT_SECRET=changeme \
SKIP_PROMPTS=1 \
bash deployments/headscale/deploy.sh
deployments/headscale/config.yaml
+85
View File
@@ -0,0 +1,85 @@
# headscale config.yaml -- 0.28.0
#
# Placeholders __VAR__ are substituted at deploy time by deploy.sh from the
# values in .env. Don't set them here directly unless you're running
# headscale outside of the deploy script.
server_url: https://__HEADSCALE_DOMAIN__
listen_addr: 0.0.0.0:8080
metrics_listen_addr: 0.0.0.0:9090
# gRPC remote-control API. Disabled by default; clients use the standard
# `headscale` CLI inside the container instead. Uncomment + expose if you
# want remote admin.
# grpc_listen_addr: 0.0.0.0:50443
# grpc_allow_insecure: false
noise:
private_key_path: /var/lib/headscale/noise_private.key
prefixes:
v4: 100.64.0.0/10
v6: fd7a:115c:a1e0::/48
allocation: sequential
# DERP: use Tailscale's public DERP map. Embedded DERP server stays off so
# we don't need to open UDP/3478 publicly.
derp:
server:
enabled: false
urls:
- https://controlplane.tailscale.com/derpmap/default
paths: []
auto_update_enabled: true
update_frequency: 24h
database:
type: sqlite
sqlite:
path: /var/lib/headscale/db.sqlite
write_ahead_log: true
wal_autocheckpoint: 1000
log:
level: info
format: text
# Magic DNS for the tailnet. base_domain becomes the suffix for node
# DNS names (e.g. laptop.__TAILNET_DOMAIN__).
dns:
magic_dns: true
base_domain: __TAILNET_DOMAIN__
override_local_dns: false
nameservers:
global:
- 1.1.1.1
- 1.0.0.1
- 2606:4700:4700::1111
- 2606:4700:4700::1001
search_domains: []
extra_records: []
# OIDC: delegate authentication to the pocket-id deployment. Register a
# new OIDC client in pocket-id with redirect URI:
# https://__HEADSCALE_DOMAIN__/oidc/callback
oidc:
only_start_if_oidc_is_available: true
issuer: https://__POCKETID_DOMAIN__
client_id: __OIDC_CLIENT_ID__
client_secret: __OIDC_CLIENT_SECRET__
scope:
- openid
- profile
- email
pkce:
enabled: true
# Restrict who can register a node. Uncomment one or more.
# allowed_users:
# - alice@example.com
# allowed_groups:
# - tailscale-admins
# allowed_domains:
# - example.com
expiry:
from_authentication_path: false
refresh_period: 180d
deployments/headscale/deploy.sh
+366
View File
@@ -0,0 +1,366 @@
#!/usr/bin/env bash
#
# deploy.sh -- deploy the headscale stack (caddy + headscale) on Alpine.
#
# Single-node, dedicated host: runs everything as root.
#
# What this does:
# 1. Installs docker + docker-cli-compose if missing.
# 2. Lays down docker-compose.yml, Caddyfile, and a substituted
# config.yaml in $STACK_DIR.
# 3. Generates .env on first run; existing .env is never overwritten.
# 4. Prompts for required values not preset (HEADSCALE_DOMAIN,
# ACME_EMAIL, TAILNET_DOMAIN, POCKETID_DOMAIN, OIDC_CLIENT_ID,
# OIDC_CLIENT_SECRET).
# 5. Substitutes those values into config.yaml from the embedded
# template.
# 6. Opens TCP 80/443 in UFW if it's installed and active (Tailscale
# clients connect on 443; LE HTTP-01 needs 80).
# 7. Pulls images, brings the stack up, waits for healthchecks.
#
# Idempotent: re-run to apply config changes / pull new images. Re-running
# regenerates config.yaml from the current .env so edits to .env propagate
# (but secrets in .env are never touched once seeded).
#
# Self-contained: docker-compose.yml, Caddyfile, config.yaml, .env.example
# are embedded as a base64-encoded tar.gz at the bottom of this file.
# Rebuild with build.sh after editing the loose source files.
#
# Usage:
# bash deploy.sh # interactive prompts
# HEADSCALE_DOMAIN=hs.example.com \
# ACME_EMAIL=admin@example.com \
# TAILNET_DOMAIN=tail.example.com \
# POCKETID_DOMAIN=auth.example.com \
# OIDC_CLIENT_ID=... OIDC_CLIENT_SECRET=... FORCE=1 bash deploy.sh
#
# STACK_DIR=/opt/headscale bash deploy.sh
# SKIP_DOCKER_INSTALL=1 bash deploy.sh
set -euo pipefail
: "${STACK_DIR:=/srv/headscale}"
: "${SKIP_DOCKER_INSTALL:=0}"
: "${FORCE:=0}"
: "${SKIP_PROMPTS:=0}" # non-interactive: require values via env, no prompts
[[ "$SKIP_PROMPTS" == "1" ]] && FORCE=1
: "${HEADSCALE_DOMAIN:=}"
: "${ACME_EMAIL:=}"
: "${TAILNET_DOMAIN:=}"
: "${POCKETID_DOMAIN:=}"
: "${OIDC_CLIENT_ID:=}"
: "${OIDC_CLIENT_SECRET:=}"
log() { printf '\033[1;32m[+]\033[0m %s\n' "$*"; }
warn() { printf '\033[1;33m[!]\033[0m %s\n' "$*" >&2; }
die() { printf '\033[1;31m[x]\033[0m %s\n' "$*" >&2; exit 1; }
[[ $EUID -eq 0 ]] || die "Run as root."
# ---------------------------------------------------------------------------
# OS detection + Docker install (Alpine / Debian / Alma). This deploy.sh is
# self-contained (scp'd standalone), so the OS logic is inlined here instead
# of sourced from scripts/oslib.sh.
# ---------------------------------------------------------------------------
osfam() {
local id="" like=""
if [[ -r /etc/os-release ]]; then
id="$(. /etc/os-release 2>/dev/null && echo "${ID:-}")"
like="$(. /etc/os-release 2>/dev/null && echo "${ID_LIKE:-}")"
fi
case " $id $like " in
*" alpine "*) echo alpine ;;
*" debian "*|*" ubuntu "*) echo debian ;;
*" rhel "*|*" fedora "*|*" centos "*) echo rhel ;;
*) echo "${id:-unknown}" ;;
esac
}
install_docker() {
if command -v docker >/dev/null 2>&1; then
log "Docker already installed: $(docker --version)"
else
log "Installing Docker (OS: $(osfam))..."
case "$(osfam)" in
alpine) apk add -q docker docker-cli-compose openrc ;;
debian|rhel) command -v curl >/dev/null 2>&1 || \
{ command -v apt-get >/dev/null 2>&1 && apt-get install -y -qq curl; } || \
{ command -v dnf >/dev/null 2>&1 && dnf install -y -q curl; }
curl -fsSL https://get.docker.com | sh ;;
*) die "Unsupported OS for auto Docker install. Set SKIP_DOCKER_INSTALL=1 and install Docker yourself." ;;
esac
fi
if command -v rc-update >/dev/null 2>&1; then
rc-update add docker default >/dev/null 2>&1 || true
rc-service docker status >/dev/null 2>&1 || rc-service docker start
elif command -v systemctl >/dev/null 2>&1; then
systemctl enable --now docker >/dev/null 2>&1 || systemctl start docker || true
fi
}
open_web_ports() {
if command -v ufw >/dev/null 2>&1 && ufw status 2>/dev/null | grep -q '^Status: active'; then
log "ufw active -- allowing 80,443/tcp..."
ufw allow 80/tcp >/dev/null; ufw allow 443/tcp >/dev/null
elif command -v firewall-cmd >/dev/null 2>&1 && firewall-cmd --state >/dev/null 2>&1; then
log "firewalld active -- allowing http,https..."
firewall-cmd -q --add-service=http --permanent
firewall-cmd -q --add-service=https --permanent
firewall-cmd -q --reload
fi
}
# ----------------------------------------------------------------------------
# Extract embedded archive
# ----------------------------------------------------------------------------
SCRIPT_DIR=$(mktemp -d -t headscale-deploy.XXXXXX)
trap 'rm -rf "$SCRIPT_DIR"' EXIT
extract_archive() {
grep -a -A 9999999 '^__ARCHIVE_BELOW__$' "$0" \
| tail -n +2 \
| base64 -d \
| tar -xz -C "$SCRIPT_DIR"
}
if grep -q -a '^__ARCHIVE_BELOW__$' "$0"; then
log "Extracting embedded deployment files..."
extract_archive
else
die "No embedded archive found. Run build.sh to embed deployment files."
fi
for f in docker-compose.yml Caddyfile config.yaml .env.example; do
[[ -f "$SCRIPT_DIR/$f" ]] || die "Embedded archive missing $f"
done
# ----------------------------------------------------------------------------
# Prompt for required vars
# ----------------------------------------------------------------------------
prompt() {
local varname="$1" message="$2"
local -n ref="$varname"
if [[ -z "${ref:-}" ]]; then
[[ "$SKIP_PROMPTS" == "1" ]] && die "$varname required (set it in the environment; running with SKIP_PROMPTS=1)."
read -r -p "$message: " ref
[[ -n "$ref" ]] || die "$varname required."
fi
}
prompt HEADSCALE_DOMAIN "Public hostname for headscale (e.g. hs.example.com)"
prompt ACME_EMAIL "Let's Encrypt email"
prompt TAILNET_DOMAIN "Tailnet base domain for MagicDNS (e.g. tail.example.com)"
prompt POCKETID_DOMAIN "OIDC issuer hostname (your pocket-id, e.g. auth.example.com)"
prompt OIDC_CLIENT_ID "OIDC client_id (from pocket-id)"
prompt OIDC_CLIENT_SECRET "OIDC client_secret (from pocket-id)"
[[ "$HEADSCALE_DOMAIN" != "$TAILNET_DOMAIN" ]] \
|| die "HEADSCALE_DOMAIN and TAILNET_DOMAIN must differ."
# ----------------------------------------------------------------------------
# Docker
# ----------------------------------------------------------------------------
if [[ "$SKIP_DOCKER_INSTALL" != "1" ]]; then
install_docker
fi
# Open 80/443 on whichever host firewall is active. (Tailscale clients connect
# on 443; LE HTTP-01 needs 80. Note the IPv6-to-docker-proxy default-deny
# gotcha on ufw -- opening the ports explicitly covers it.)
open_web_ports
# ----------------------------------------------------------------------------
# Stack directory + files
# ----------------------------------------------------------------------------
log "Setting up $STACK_DIR..."
install -d -m 0750 "$STACK_DIR"
install -m 0640 "$SCRIPT_DIR/docker-compose.yml" "$STACK_DIR/docker-compose.yml"
install -m 0640 "$SCRIPT_DIR/Caddyfile" "$STACK_DIR/Caddyfile"
# .env
ENV_FILE="$STACK_DIR/.env"
if [[ ! -f "$ENV_FILE" ]]; then
log "Seeding $ENV_FILE..."
install -m 0600 "$SCRIPT_DIR/.env.example" "$ENV_FILE"
sed -i \
-e "s|^HEADSCALE_DOMAIN=.*|HEADSCALE_DOMAIN=${HEADSCALE_DOMAIN}|" \
-e "s|^ACME_EMAIL=.*|ACME_EMAIL=${ACME_EMAIL}|" \
-e "s|^TAILNET_DOMAIN=.*|TAILNET_DOMAIN=${TAILNET_DOMAIN}|" \
-e "s|^POCKETID_DOMAIN=.*|POCKETID_DOMAIN=${POCKETID_DOMAIN}|" \
-e "s|^OIDC_CLIENT_ID=.*|OIDC_CLIENT_ID=${OIDC_CLIENT_ID}|" \
-e "s|^OIDC_CLIENT_SECRET=.*|OIDC_CLIENT_SECRET=${OIDC_CLIENT_SECRET}|" \
"$ENV_FILE"
else
log ".env exists; leaving it alone."
fi
# Validate
missing=()
for var in HEADSCALE_DOMAIN ACME_EMAIL TAILNET_DOMAIN POCKETID_DOMAIN \
OIDC_CLIENT_ID OIDC_CLIENT_SECRET; do
grep -E "^${var}=.+$" "$ENV_FILE" >/dev/null || missing+=("$var")
done
(( ${#missing[@]} == 0 )) || die "Missing values in $ENV_FILE: ${missing[*]}"
# Re-read final values from .env so config.yaml substitution sees what's
# actually deployed (not whatever was passed in the environment).
# shellcheck disable=SC1090
set -a; . "$ENV_FILE"; set +a
# config.yaml: regenerate every run from the template so changes to .env
# propagate. Secrets in config.yaml are derived from .env, not stored
# independently, so this is safe.
log "Generating config.yaml from template..."
install -m 0600 "$SCRIPT_DIR/config.yaml" "$STACK_DIR/config.yaml"
# Escape replacement values for sed (handle slashes/&). Headscale URLs and
# OIDC secrets can contain those.
sed_escape() { printf '%s' "$1" | sed -e 's/[\/&]/\\&/g'; }
sed -i \
-e "s/__HEADSCALE_DOMAIN__/$(sed_escape "$HEADSCALE_DOMAIN")/g" \
-e "s/__TAILNET_DOMAIN__/$(sed_escape "$TAILNET_DOMAIN")/g" \
-e "s/__POCKETID_DOMAIN__/$(sed_escape "$POCKETID_DOMAIN")/g" \
-e "s/__OIDC_CLIENT_ID__/$(sed_escape "$OIDC_CLIENT_ID")/g" \
-e "s/__OIDC_CLIENT_SECRET__/$(sed_escape "$OIDC_CLIENT_SECRET")/g" \
"$STACK_DIR/config.yaml"
# Final sanity check
grep -q '__.*__' "$STACK_DIR/config.yaml" \
&& warn "config.yaml still has un-substituted __PLACEHOLDER__ markers!" \
|| true
# ----------------------------------------------------------------------------
# Bring up the stack
# ----------------------------------------------------------------------------
if [[ "$FORCE" != "1" ]]; then
cat <<EOF
About to pull images and start the stack from $STACK_DIR.
Caddy will request a Let's Encrypt cert for ${HEADSCALE_DOMAIN}. DNS for
that name must already point at this host, and ports 80/443 must be
reachable from the internet.
In pocket-id, the OIDC client must have this redirect URI registered:
https://${HEADSCALE_DOMAIN}/oidc/callback
Continue? [y/N]
EOF
read -r ans
[[ "${ans,,}" == "y" || "${ans,,}" == "yes" ]] || { warn "Aborted."; exit 0; }
fi
cd "$STACK_DIR"
log "Pulling images..."
docker compose pull
log "Starting stack..."
docker compose up -d --remove-orphans
# ----------------------------------------------------------------------------
# Wait for health
# ----------------------------------------------------------------------------
log "Waiting for services to become healthy (up to 120s)..."
deadline=$(( $(date +%s) + 120 ))
while (( $(date +%s) < deadline )); do
status=$(docker compose ps --format '{{.Service}} {{.Health}}' 2>/dev/null || true)
unhealthy=$(echo "$status" | awk '$2 != "healthy" && $2 != "" {print $1}')
if [[ -z "$unhealthy" && -n "$status" ]]; then
log "All services healthy."
break
fi
sleep 5
done
echo
log "Stack status:"
docker compose ps
echo
cat <<EOF
================================================================
DEPLOYED
Headscale: https://${HEADSCALE_DOMAIN}
Tailnet: ${TAILNET_DOMAIN} (MagicDNS suffix)
OIDC: https://${POCKETID_DOMAIN}
Stack dir: ${STACK_DIR}
Bootstrap a user (run from $STACK_DIR):
docker compose exec headscale headscale users create alice
docker compose exec headscale headscale users list
Tailscale client login:
tailscale up --login-server https://${HEADSCALE_DOMAIN}
# Browser opens, OIDC flow via pocket-id, node registers.
CLI inside the container:
docker compose exec headscale headscale --help
docker compose exec headscale headscale nodes list
Manage:
docker compose logs -f
docker compose pull && docker compose up -d # update
docker compose down # stop, keep volumes
docker compose down -v # stop, WIPE data
Or just re-run this script -- it's idempotent.
================================================================
EOF
# IMPORTANT: do not put any code below this exit. Everything after the
# __ARCHIVE_BELOW__ marker is the embedded tar.gz payload (base64).
exit 0
__ARCHIVE_BELOW__
H4sIAAAAAAAAA+1a627cuBXObz0FYQfYBBtpNL5mJ3Cxrm00Rp3EtZ3uFotCy5E4M4QlUUtSnswG
AfoQfcI+Sb9DShp54iZ/srdCx4ZnxMsheXj4ne9QzlR6K3SYqqJSRkSrIn/0xSWGHOztPYoP4vHh
/th9xvHYlUN29w52H433x4cH44ODeH/3UTzejeP9Ryz+8lP5WGpjuWbs0f5h8p3K78SvMebvSLbZ
QvDMpDwXDJZIb1kYspRn2Yo9ubm4fsq+7jV4csNl7r46h+FWTlGaKqUzWXKr9NNgO9hmN6pSuZqv
JvjOWNeHpbkUpTUs/JMfYbK3t0sP3QCT5/Hz2Kl4rdhxWU+lQaUWkweUmErwW/by5uZytIMplFar
nFVaWZWq/BkrlUXlq4tnUGYU4+xSfcfSBc9zUc4FW6o6z9hUkw67EEXEXraT+Mqw48tzZmo946lg
mMJywS3UtCMvpH3BLLpaJi3L5a1gvFwxBT3adxX6TqYicit5c356wmANWZKqTORizq3ImFWsorNn
Q5kxqHp8+ebkr2c356fJ6ZtXx+evI3Yl5tJY6OTd4DRXLdhMaoM5pKpaUUlTmUDR1+13I1ItMMES
40SivIuCoOQFLNkZOwjuVF4XwkwCti4NM245lbgd2niCmWdyPgmCZoWmq6IvjMmCzzGEL3n8/uT4
9PQfyc3xXybhTsjzSpbig2tH28XxpBM/J9fB1WhBx9FOWF3mwpjQWFVVInN1ldLW+IEYC9nW8xj+
srV+hjeRR7Ul272yUZ1VW67MOcwuOXkmDYf7ZhjKypydOJ//29vzExhRizuhjUjgT+9Wjb6HZJtV
OZxkofIM+zSt57THM/kOWp98H14Jnofnl1gurIyN0qqeLz6pTRSVhSeVcNmCziOcrDBPyRVCUdJs
WQ7n0ZFT0ts+b4Fo5BYxkzhJI2HTkbNrr1Crrm1ve0f0d6Oi2emR/3SVcCKpVVnAu9ohX54dn16f
HF+cNT47YVuP328Wfmg35Pjk1VlyhqIL12792LTIRCXKzCSqbPWvgaGzGiaUSSvRpj1mCVrlduG3
yX9PFyK9bftYuNSE/bB18up06xnbWs6Fpc/wpzchfS6srSaj0XjnMIrxM57sxONvmnWPtv7ZKME5
wmg8n7Dd2LSKZSFUDd37bQlOnJbYEbbbFDhnTiqhpcombIyuwUerag5NVzpa1/eN6Y5RHO08j+KH
D9H6YH/uIMEfC15m3oJtc57B8PlqAmSrfZktqlnPu0Z3XI90XY7uD/SAF3rjRSte5N4P12vrV/W8
cQN+3FC5nG4M9fnN7do3D2j+5bfwtw7ZX1Q6cPgFx/gM/xsfjPfX/O8wBv8bj0EDB/73K8g26zyA
zZTeZIOewxzXVoXSmBphjLMLYcGRzspUryrLUqGt6/l4E/jBirI2koYUSXGuoAyEZNHjWhSQQaJA
dABlFPocCYweYHxa/FRLsB+7kKRHvOOpZQtlLOEfUxjHKbsG1XnvI1YBHez9416kCT6g8qOZfmBN
hzJVmWA/G5ux+c+yChps7JGBDbLadGyDFcC4rtg68r+n6BLRxIFT+CyUFRFN+UPXbZtdqHIe5vIO
pKFU0ojREquc11xnHa9tmMALGGrmSCCC5YzXue2pkRnRA1mAk4Lwos2KZar8yjIEPBhdl3hcllHX
Y5bXZpG0qMjCcVdjNS8NsS1G0+8tsYsUDW4y4GFxr3appRVddb/Wr/hD0CI5Eaa15mugbmrDm3bk
8FqkNXSt2FbB34WIkEe7wIQDwMULIHma15m4rqenCltcmq1Oz/fhCUwGdwlvVpUI31REFQzVbJXK
lHI2W7e9EjOhNZLfS5XLdNVbxZbx01FagreHy4Uow1QrhFFfstaBeSKE6v7awPV7C4MZKlgCHoVv
a9srXYDzY3uNaoIbueZvcP57AfkXG+Mz+L8H5O/wHw0I//fH8YD/v4b08/+eL1CC5MmmCwCX6yzH
sCT5+/FVkjAOLDb11Fhpa8po4dCg8LlaOVrFpqvmMTILNtOqIEyCLoANxRHkwy4vZacOpIxwuFa4
fB+5mRapzVcNe2UrVX+FYrDPUpbzoD9pHCsjAdpq1uCim4FJtaxs5DNVQmUN3kdoZsD1k2QzACRJ
4ENQgmCoJyx2uUDs7yMKooSpSR5s8U38TRxgQvOryxPmAT5sUfv48hzLa9NMZw8H2i+6kFYb4WYN
HywzAD4U/dgt7Ud2cnEOO7nluTS/ZfxUaNEsYm9LIvN0M/A1AiJd4TE5I3NB05Kj2E+J8ayQwH7M
U1fpwyvZj5Eoty14nqslgoMhGEZyMeO5EUHg4hNx70rLO2Siya1YJRW3iwn7mLOPXOukaRqhaRBU
WlB67NKFuz3i03F0sEfjj8YxlR1grOyQTwAB6YSPRTyZjPaeo4ZmlPI280NARdbOczL96dnV5cSZ
suMM4BVVPQWqu0pW8CpiZ8VUZBk2whV5vyDDrwx8Z4aYSSYTTcwshb+iUUhJ2dvTy9Hu3uHzRme+
igIchIrW4NVMGvbgNro1FmNwuiYvCjvfa3yjynkpItvON8IujkgnZjpah3ayLFKRHyiF4eBgIBcZ
Wb0bqcnVmuKZIxpliiRuZ28RBJRLTbnfMItwCMP9lCM+07zdFz+5/7l/2TTqOrShnVN1ghjXSxSX
PE9oei4zqxQohdtYnAtqhyY5GBTOnyxnlPL54If+4p2l/XvF57RRr68djSRHJ7uUwkaMZp9kLsiz
qWjvUQh2ZnAj174EYyMnQHfigYY9EdE8YjmvkPFGCfLm84vXZzfdQX+KzSvdthQ0bkIP7VJ6w03Y
x13RglimxnlMyBlz37ndbje884dm1+e5miLf7LLcceR+es/uxqF73jmIDyZ7h3Hs/+AMjD9RiQjq
HJDrdNFMuvUVGFbzBBCqdObLmovISXcBSf60oDPkzxT5Ohl2fSXpgZSgZeMmshRLf6nZXElia9a9
ltIugDkev9nbq3N/B/wp5B0pmdFVVZ5PkXAE9EQmo8uIxCffcpZQaSJNwu/gGuT83Z65tET3wX3j
ItXtW3dDSvtKs0+ArWevbxI069X7W9PNNtdnJ1dwA2pnUiBCe6YJHGTWPCA7oCSqeXKpB53g21Rs
oEMz721Y1dNMtlwolvISdmvN7Ly6j+6qRITTrFBaRK6zA2iRJYA9727bblwOfBLfIjUqKo8q9xrP
taqrXusOf0IXH8y9xq1Lda3va0W0kbq596Xontx3qCYqtGeD8gbAExKO7ibleZz9f12l/CHFcbBm
Z3+pMT73/i/eW7//24kPHf8/OBj4/68h2+zEvUXyb4nclQ1gLAeqR6BC9G6YNe+GWSXTW+Ob1ZXj
IwjjkoAbhAgw/59//8v/skvPvdq7GbOu+j3/Ygl/ppymu1J6UioAPpIS8RQYTXnJOu1YkpGmgq5D
0CLr3Vf13hSC7JUUB13f/gXVZhQ8WpioD6/QceYur4jh3L9v80FC+6g9amCYXnfKWQO9JgrWN15H
Dtm/3VDuKFdInGmDSZl+vtHRsDPfe+LCO73gvUeMjqhdf/rPEMDIiIx8Azx6y5OxLW80RAE6Z3A5
aGpo2qaGiL16e31D9s3kbAbrIQS6FHLTcFHwmbncd0zHW550dOXpH8YxXz7kk8plevpB0vaMEiLk
5J4v07qhxXMlGBegB/pr04Wr9cVgaxf9V9IZ/NUigTPMvf0eRUuR5+FtqZblyDOf5i1h7Z0xCjaI
1xExgs29uOpxnE8QSTi8c1z29vxznPKBN44blBIOC2aCPAcD+0Qa2nz+6P+/IAruU8Kj4GP6d3Tf
k87prR1OyPwPAm4PeNW9d4tHzW1P987+qH1lP5C0QQYZZJBBBhlkkEEGGWSQQQYZZJBBBhlkkEEG
GWSQQQYZZJBBBhlkkEEGGWSQQQYZZJBBBvm9yH8Bvi7jBABQAAA=
deployments/headscale/docker-compose.yml
+64
View File
@@ -0,0 +1,64 @@
# headscale stack -- caddy (TLS) + headscale (Tailscale-compatible coordinator)
#
# Topology:
# Tailscale clients -> caddy:443 -> headscale:8080
#
# No Anubis here: Tailscale clients speak HTTP/2 control protocol, not HTML,
# so a PoW challenge would break them. Headscale's API surface is what
# clients hit; treat it like any other API service.
#
# OIDC login is delegated to pocket-id at $POCKETID_DOMAIN. Register a
# client there first; copy the client_id + client_secret into .env.
name: headscale
volumes:
headscale-data:
caddy-data:
caddy-config:
services:
caddy:
image: caddy:${CADDY_TAG:-2-alpine}
container_name: caddy
restart: unless-stopped
ports:
- "80:80"
- "443:443"
# - "443:443/udp" # HTTP/3 -- disabled until Caddy QUIC + reverse_proxy
# placeholder bug is fixed (X-Real-IP comes through
# empty on some streams). Re-enable later.
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
- caddy-data:/data
- caddy-config:/config
environment:
HEADSCALE_DOMAIN: "${HEADSCALE_DOMAIN}"
ACME_EMAIL: "${ACME_EMAIL}"
depends_on:
headscale:
condition: service_healthy
healthcheck:
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:2019/config/"]
interval: 30s
timeout: 5s
retries: 3
start_period: 10s
headscale:
image: headscale/headscale:${HEADSCALE_TAG:-0.28.0}
container_name: headscale
restart: unless-stopped
command: serve
read_only: true
tmpfs:
- /var/run/headscale
volumes:
- ./config.yaml:/etc/headscale/config.yaml:ro
- headscale-data:/var/lib/headscale
healthcheck:
test: ["CMD", "headscale", "health"]
interval: 30s
timeout: 5s
retries: 3
start_period: 10s
deployments/pocket-id/Caddyfile
+48 -6
View File
@@ -1,9 +1,9 @@
# Caddyfile for pocket-id stack.
#
# Auto-issues a Let's Encrypt cert for $POCKETID_DOMAIN and reverse-proxies
# to anubis-pid (which forwards to pocket-id after the PoW challenge).
#
# To skip anubis, change the reverse_proxy target to `pocket-id:1411`.
# Two site blocks:
# $POCKETID_DOMAIN pocket-id (via anubis PoW gate)
# $BASE_DOMAIN /.well-known/webfinger for OIDC discovery; everything
# else redirects to $REDIRECT_URL
{
email {$ACME_EMAIL}
@@ -11,10 +11,12 @@
# acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
}
# ----------------------------------------------------------------------------
# Auth: pocket-id behind anubis
# ----------------------------------------------------------------------------
{$POCKETID_DOMAIN} {
encode zstd gzip
# Forward through anubis (PoW anti-bot) -> pocket-id.
reverse_proxy anubis-pid:8923 {
header_up X-Real-IP {remote_host}
header_up X-Forwarded-For {remote_host}
@@ -22,7 +24,6 @@
header_up X-Forwarded-Host {host}
}
# Sensible security headers. Adjust CSP if you embed pocket-id elsewhere.
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains"
X-Content-Type-Options "nosniff"
@@ -37,3 +38,44 @@
format console
}
}
# ----------------------------------------------------------------------------
# Base domain: WebFinger for OIDC discovery + redirect everything else.
# Required for Tailscale's custom-OIDC setup -- the email's domain must
# serve /.well-known/webfinger pointing at the issuer.
# ----------------------------------------------------------------------------
{$BASE_DOMAIN} {
encode zstd gzip
@webfinger path /.well-known/webfinger
handle @webfinger {
header Content-Type "application/jrd+json"
header Cache-Control "public, max-age=3600"
templates
respond `{
"subject": "{{.Req.URL.Query.Get "resource"}}",
"links": [
{
"rel": "http://openid.net/specs/connect/1.0/issuer",
"href": "https://{$POCKETID_DOMAIN}"
}
]
}` 200
}
handle {
redir {$REDIRECT_URL}{uri} permanent
}
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains"
X-Content-Type-Options "nosniff"
Referrer-Policy "strict-origin-when-cross-origin"
-Server
}
log {
output stdout
format console
}
}
deployments/pocket-id/README.md
+40
View File
@@ -0,0 +1,40 @@
# pocket-id
[Pocket-ID](https://pocket-id.org) OIDC provider behind Caddy, with an optional
[Anubis](https://github.com/TecharoHQ/anubis) proof-of-work anti-bot gate. The
identity provider for the other stacks (headscale, beszel, webfinger).
## Required `.env` values
| Variable | Notes |
|----------|-------|
| `POCKETID_DOMAIN` | Public hostname (e.g. `id.example.com`). |
| `ACME_EMAIL` | Let's Encrypt registration email. |
| `ENCRYPTION_KEY` | Generated on first deploy (`openssl rand -base64 32`). Losing it is unrecoverable. |
| `ANUBIS_PID_KEY` | Generated on first deploy (`openssl rand -hex 32`). |
`MAXMIND_LICENSE_KEY` (audit-log geolocation) is optional. See
[`.env.example`](.env.example) for the full list.
## Deploy
From the repo root via the launcher:
```bash
./automations.sh # Deploy on this host → deploy: pocket-id
```
Or build the self-contained artifact and run it on the host:
```bash
./build.sh # embeds files into deploy.sh
scp deploy.sh root@host: # copy to the target
ssh root@host 'bash deploy.sh' # interactive
# non-interactive:
# POCKETID_DOMAIN=id.example.com ACME_EMAIL=me@example.com SKIP_PROMPTS=1 bash deploy.sh
```
Unattended provisioning: [`cloud-init.yml`](cloud-init.yml).
DNS for `POCKETID_DOMAIN` must resolve to the host and ports 80/443 be
reachable before deploy, or the LE cert request fails.
deployments/pocket-id/cloud-init.yml
+31
View File
@@ -0,0 +1,31 @@
#cloud-config
#
# Pocket-ID (OIDC provider) — unattended deploy on a fresh Alpine host.
#
# Fill in REPO_URL and the values in the runcmd block, then paste this as the
# instance user-data. DNS for POCKETID_DOMAIN must point at this host and
# ports 80/443 must be reachable before boot, or the LE cert request fails.
packages:
- git
runcmd:
- hostnamectl set-hostname pocket-id || true
- |
set -e
REPO_URL=https://git.anomalous.dev/57_Wolve/automations.git
REPO_BRANCH=main
git clone --depth 1 --branch "$REPO_BRANCH" "$REPO_URL" /opt/automations
cd /opt/automations
# Optional: install shared admin SSH keys from globals/ for root before deploy
# . scripts/lib.sh && load_globals \
# && install -d -m700 /root/.ssh && resolve_ssh_keys >> /root/.ssh/authorized_keys
# Optional: harden SSH first (prints a fresh root key — capture it!)
# SKIP_PROMPTS=1 FORCE=1 bash scripts/harden-ssh.sh
POCKETID_DOMAIN=id.example.com \
ACME_EMAIL=admin@example.com \
SKIP_PROMPTS=1 \
bash deployments/pocket-id/deploy.sh
deployments/pocket-id/deploy.sh
+60 -9
View File
@@ -35,6 +35,8 @@ set -euo pipefail
: "${STACK_DIR:=/srv/pocket-id}"
: "${SKIP_DOCKER_INSTALL:=0}"
: "${FORCE:=0}"
: "${SKIP_PROMPTS:=0}" # non-interactive: require values via env, no prompts
[[ "$SKIP_PROMPTS" == "1" ]] && FORCE=1
: "${POCKETID_DOMAIN:=}"
: "${ACME_EMAIL:=}"
@@ -43,7 +45,61 @@ warn() { printf '\033[1;33m[!]\033[0m %s\n' "$*" >&2; }
die() { printf '\033[1;31m[x]\033[0m %s\n' "$*" >&2; exit 1; }
[[ $EUID -eq 0 ]] || die "Run as root."
[[ -f /etc/alpine-release ]] || die "This script targets Alpine Linux."
# ---------------------------------------------------------------------------
# OS detection + Docker install (Alpine / Debian / Alma). This deploy.sh is
# self-contained (scp'd standalone), so the OS logic is inlined here instead
# of sourced from scripts/oslib.sh.
# ---------------------------------------------------------------------------
osfam() {
local id="" like=""
if [[ -r /etc/os-release ]]; then
id="$(. /etc/os-release 2>/dev/null && echo "${ID:-}")"
like="$(. /etc/os-release 2>/dev/null && echo "${ID_LIKE:-}")"
fi
case " $id $like " in
*" alpine "*) echo alpine ;;
*" debian "*|*" ubuntu "*) echo debian ;;
*" rhel "*|*" fedora "*|*" centos "*) echo rhel ;;
*) echo "${id:-unknown}" ;;
esac
}
install_docker() {
if command -v docker >/dev/null 2>&1; then
log "Docker already installed: $(docker --version)"
else
log "Installing Docker (OS: $(osfam))..."
case "$(osfam)" in
alpine) apk add -q docker docker-cli-compose openrc ;;
debian|rhel) command -v curl >/dev/null 2>&1 || \
{ command -v apt-get >/dev/null 2>&1 && apt-get install -y -qq curl; } || \
{ command -v dnf >/dev/null 2>&1 && dnf install -y -q curl; }
curl -fsSL https://get.docker.com | sh ;;
*) die "Unsupported OS for auto Docker install. Set SKIP_DOCKER_INSTALL=1 and install Docker yourself." ;;
esac
fi
# Enable + start under whichever init is present.
if command -v rc-update >/dev/null 2>&1; then
rc-update add docker default >/dev/null 2>&1 || true
rc-service docker status >/dev/null 2>&1 || rc-service docker start
elif command -v systemctl >/dev/null 2>&1; then
systemctl enable --now docker >/dev/null 2>&1 || systemctl start docker || true
fi
}
open_web_ports() {
# Open 80/443 on whichever host firewall is active (no-op otherwise).
if command -v ufw >/dev/null 2>&1 && ufw status 2>/dev/null | grep -q '^Status: active'; then
log "ufw active -- allowing 80,443/tcp..."
ufw allow 80/tcp >/dev/null; ufw allow 443/tcp >/dev/null
elif command -v firewall-cmd >/dev/null 2>&1 && firewall-cmd --state >/dev/null 2>&1; then
log "firewalld active -- allowing http,https..."
firewall-cmd -q --add-service=http --permanent
firewall-cmd -q --add-service=https --permanent
firewall-cmd -q --reload
fi
}
# ----------------------------------------------------------------------------
# Extract embedded archive
@@ -76,6 +132,7 @@ prompt() {
local varname="$1" message="$2"
local -n ref="$varname"
if [[ -z "${ref:-}" ]]; then
[[ "$SKIP_PROMPTS" == "1" ]] && die "$varname required (set it in the environment; running with SKIP_PROMPTS=1)."
read -r -p "$message: " ref
[[ -n "$ref" ]] || die "$varname required."
fi
@@ -87,15 +144,9 @@ prompt ACME_EMAIL "Let's Encrypt email"
# Docker
# ----------------------------------------------------------------------------
if [[ "$SKIP_DOCKER_INSTALL" != "1" ]]; then
if ! command -v docker >/dev/null 2>&1; then
log "Installing docker + docker-cli-compose..."
apk add -q docker docker-cli-compose openrc
else
log "Docker already installed: $(docker --version)"
fi
rc-update add docker default >/dev/null 2>&1 || true
rc-service docker status >/dev/null 2>&1 || rc-service docker start
install_docker
fi
open_web_ports
# ----------------------------------------------------------------------------
# Stack directory + files
deployments/simplex/README.md
+250
View File
@@ -0,0 +1,250 @@
# SimpleX Relay Server Deployment
Complete automated deployment for SimpleX Chat relay servers on Alpine Linux with post-quantum SSH hardening, Tor hidden services, and encrypted backups.
## Quick Start
### 1. Clone This Repository
The installer fetches its scripts straight from this monorepo's layout:
```
automations/
├── scripts/
│ └── harden-ssh.sh # generic SSH hardening (PQ KEX + Ed25519)
├── deployments/simplex/
│ ├── deploy-simplex.sh # SMP + XFTP + Tor deployment
│ ├── backup.sh # age-encrypted backup creation
│ ├── restore.sh # disaster recovery from backup
│ ├── install-simplex.sh # master installer (cloud-init compatible)
│ ├── cloud-init.yml # example cloud-init configuration
│ └── README.md # this file
└── globals/
└── age-pubkey.txt # your age public key(s) for backups
```
Point `REPO_URL` at your fork of this repo. The installer resolves
`harden-ssh.sh` under `scripts/` and the simplex scripts under
`deployments/simplex/` automatically (overridable via `HARDEN_PATH` /
`SIMPLEX_PATH`).
### 2. Generate Backup Keys
```bash
# Generate age keypair for backups
age-keygen -o backup-private-key.txt
# Save the public key to the shared globals/ folder
echo "age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p" > globals/age-pubkey.txt
# Store the private key securely (NOT in your repo)
cp backup-private-key.txt ~/safe-location/
```
### 3. Deploy via Cloud-Init
Create a cloud instance with this user-data:
```yaml
#cloud-config
runcmd:
- |
curl -fsSL https://git.anomalous.dev/57_Wolve/automations/raw/branch/main/deployments/simplex/install-simplex.sh | \
REPO_URL=https://git.anomalous.dev/57_Wolve/automations.git \
DOMAIN=relay.example.com \
ACME_EMAIL=admin@example.com \
XFTP_QUOTA=100gb \
SSH_PORT=2222 \
ALLOWED_IP=1.2.3.4 \
bash
```
### 4. Manual Deployment
```bash
# On a fresh Alpine Linux host:
curl -fsSL https://git.anomalous.dev/57_Wolve/automations/raw/branch/main/deployments/simplex/install-simplex.sh | \
REPO_URL=https://git.anomalous.dev/57_Wolve/automations.git \
DOMAIN=relay.example.com \
ACME_EMAIL=admin@example.com \
XFTP_QUOTA=50gb \
SSH_PORT=2222 \
bash
```
## Configuration Options
| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `REPO_URL` | ✅ | | Git repository URL containing deployment scripts |
| `DOMAIN` | ✅ | | Apex domain (creates smp.DOMAIN, xftp.DOMAIN) |
| `ACME_EMAIL` | ✅ | | Email for Let's Encrypt registration |
| `XFTP_QUOTA` | | 50gb | Disk quota for file storage |
| `SSH_PORT` | | 22 | SSH port (recommend changing from default) |
| `ALLOWED_IP` | | | Your IP to whitelist in sshguard |
| `KEY_TYPE` | | rsa4096 | Caddy TLS key type (rsa4096, p384, ed25519) |
| `SMP_PASS` | | | Optional password for SMP queue creation |
| `XFTP_PASS` | | | Optional password for XFTP uploads |
| `SKIP_PROMPTS` | | 0 | Set to 1 for non-interactive installation |
## What Gets Deployed
### Security Features
- **Post-quantum SSH**: Hybrid KEX (mlkem768x25519, sntrup761x25519) + Ed25519 keys
- **Minimal attack surface**: Only SSH terminal + SFTP, no forwarding/tunnels
- **Firewall**: awall with explicit allow-list for required ports only
- **Brute-force protection**: sshguard with progressive IP bans
### SimpleX Infrastructure
- **SMP server**: Message relay on port 5223
- **XFTP server**: File transfer on port 5443
- **Tor hidden services**: .onion addresses for both servers
- **TLS termination**: Caddy with Let's Encrypt and strong crypto
- **Docker compose**: Orchestrated deployment with health checks
### Operational Features
- **Encrypted backups**: All private keys backed up with age encryption
- **CA key removal**: Private keys removed from disk after backup
- **Service monitoring**: OpenRC integration with auto-restart
- **Address discovery**: Scripts to show server fingerprints and .onion URLs
## Server Addresses
After deployment, your relay will be accessible at:
```
# Clearnet
smp://FINGERPRINT@smp.yourdomain.com
xftp://FINGERPRINT@xftp.yourdomain.com
# Tor (clients with Orbot/Tor)
smp://FINGERPRINT@smp.yourdomain.com,ONIONADDRESS.onion
xftp://FINGERPRINT@xftp.yourdomain.com,ONIONADDRESS.onion
```
Get the full addresses with: `cd /opt/simplex && ./print-addresses.sh`
## Backup & Recovery
### Creating Backups
```bash
# Manual backup (on the server)
AGE_RECIPIENT_FILE=/path/to/age-pubkey.txt bash backup.sh
# Or with a single recipient
AGE_RECIPIENT=age1ql3z7... bash backup.sh
```
### Disaster Recovery
```bash
# 1. Fresh Alpine install
# 2. Upload restore script and backup
scp restore.sh backup-20250101-120000.tar.gz.age root@new-host:/root/
# 3. Restore (preserves all server identities)
ssh root@new-host
bash restore.sh backup-20250101-120000.tar.gz.age
# 4. Update DNS to point at new host
```
## DNS Configuration
Create these DNS records before deployment:
```
smp.yourdomain.com. 300 IN A 1.2.3.4
xftp.yourdomain.com. 300 IN A 1.2.3.4
```
Replace `1.2.3.4` with your server's IP address.
## Architecture
```
Internet
↓
[Caddy :80,:443] ← Let's Encrypt ACME
↓
[Docker Network]
├─ [SMP Server :5223] ← Tor Hidden Service
└─ [XFTP Server :5443] ← Tor Hidden Service
```
- **Ports 80/443**: Caddy (HTTP redirect + ACME + info pages)
- **Port 5223**: SMP protocol (TCP, server's own TLS)
- **Port 5443**: XFTP protocol (TCP, server's own TLS)
- **Port 22/2222**: SSH (PQ KEX + Ed25519 only)
## File Locations
- **Deployment**: `/opt/simplex/` (docker-compose.yml, configs, scripts)
- **Server keys**: `/opt/simplex/{smp,xftp}_configs/` (CA keys removed after backup)
- **Tor keys**: `/opt/simplex/{smp,xftp}_tor/` (hidden service identity)
- **SSH config**: `/etc/ssh/` (hardened sshd_config + Ed25519 host key)
- **Firewall**: `/etc/awall/optional/` (JSON policy files)
- **Backups**: `/tmp/simplex-backup-YYYYMMDD-HHMMSS.tar.gz.age`
## Security Notes
### What's Quantum-Safe
- **Session keys**: PQ hybrid KEX protects against "store now, decrypt later"
- **Authentication**: Ed25519 keys (classical, but strongest practical choice today)
### Network Hardening
- Only required ports open (22/2222, 80, 443, 5223, 5443)
- SSH locked to key-only auth, no forwarding, rate-limited
- sshguard blocks brute-force attempts with progressive delays
### Key Management
- CA private keys backed up encrypted, then removed from disk
- Tor onion keys preserved for stable .onion addresses
- SSH host key preserved for stable fingerprint
- All backups encrypted with age (modern, simple, secure)
## Troubleshooting
### Check Service Status
```bash
cd /opt/simplex
docker compose ps # Container status
docker compose logs -f smp-server # SMP logs
docker compose logs -f xftp-server # XFTP logs
./print-addresses.sh # Show server addresses
```
### SSH Issues
```bash
# Test SSH config
sshd -t
# View SSH attempts
journalctl -u sshd -f
# Check sshguard blocks
iptables -L sshguard -n
```
### Firewall Issues
```bash
awall list # Show firewall policies
awall translate # Test policy compilation
iptables -L -n # Show active rules
```
### Certificate Issues
```bash
docker exec simplex-caddy caddy list-certificates
docker logs simplex-caddy
```
## Contributing
1. Fork this repository
2. Make your changes
3. Test on a fresh Alpine instance
4. Submit a pull request
## License
This deployment is provided as-is for educational and operational use. The SimpleX Chat software itself is licensed under AGPL-3.0.
deployments/simplex/backup.sh
+225
View File
@@ -0,0 +1,225 @@
#!/usr/bin/env bash
#
# backup.sh
#
# Creates an age-encrypted backup of all irreplaceable SimpleX server keys,
# state, and SSH configuration. This is the minimum data set you'd need to
# recreate the server without changing server fingerprints or .onion addresses.
#
# The backup is encrypted using the age public key you specify. Decrypt on
# another machine with:
# age --decrypt --identity /path/to/private_key backup-YYYYMMDD-HHMMSS.tar.gz.age | tar xzf -
#
# Usage:
# AGE_RECIPIENT=age1ql3z7hjy54...your-pubkey bash backup.sh
# AGE_RECIPIENT_FILE=/path/to/pubkeys.txt bash backup.sh
#
# You can specify either a single AGE_RECIPIENT public key, or an
# AGE_RECIPIENT_FILE containing one or more public keys (one per line).
# The backup will be encrypted to all provided keys.
set -euo pipefail
# ============================================================================
# CONFIG
# ============================================================================
: "${AGE_RECIPIENT:=}" # single age public key (age1...)
: "${AGE_RECIPIENT_FILE:=}" # file containing age public keys
: "${SIMPLEX_DIR:=/opt/simplex}"
: "${BACKUP_DIR:=/tmp}"
: "${KEEP_BACKUPS:=7}" # how many historical backups to keep
log() { printf '\033[1;32m[+]\033[0m %s\n' "$*"; }
warn() { printf '\033[1;33m[!]\033[0m %s\n' "$*" >&2; }
die() { printf '\033[1;31m[x]\033[0m %s\n' "$*" >&2; exit 1; }
[[ $EUID -eq 0 ]] || die "Run as root (needs access to private keys)."
# ----------------------------------------------------------------------------
# 1. Validation
# ----------------------------------------------------------------------------
[[ -n "$AGE_RECIPIENT" || -n "$AGE_RECIPIENT_FILE" ]] \
|| die "Set AGE_RECIPIENT=age1... or AGE_RECIPIENT_FILE=/path/to/keys.txt"
if ! command -v age >/dev/null; then
log "Installing age..."
apk add -q age
fi
if [[ -n "$AGE_RECIPIENT_FILE" && ! -f "$AGE_RECIPIENT_FILE" ]]; then
die "AGE_RECIPIENT_FILE '$AGE_RECIPIENT_FILE' not found."
fi
[[ -d "$SIMPLEX_DIR" ]] || die "SimpleX directory '$SIMPLEX_DIR' not found. Run deploy-simplex.sh first."
# Build the age command args
AGE_ARGS=()
if [[ -n "$AGE_RECIPIENT" ]]; then
AGE_ARGS+=(--recipient "$AGE_RECIPIENT")
fi
if [[ -n "$AGE_RECIPIENT_FILE" ]]; then
AGE_ARGS+=(--recipients-file "$AGE_RECIPIENT_FILE")
fi
# ----------------------------------------------------------------------------
# 2. Inventory what needs backing up
# ----------------------------------------------------------------------------
log "Inventorying backup targets..."
# Critical files that prove server identity
TARGETS=(
# SimpleX CA keys (prove server identity across certs)
"$SIMPLEX_DIR/smp_configs/ca.key"
"$SIMPLEX_DIR/xftp_configs/ca.key"
# Tor hidden service keys (.onion addresses)
"$SIMPLEX_DIR/smp_tor/hs_ed25519_secret_key"
"$SIMPLEX_DIR/smp_tor/hs_ed25519_public_key"
"$SIMPLEX_DIR/xftp_tor/hs_ed25519_secret_key"
"$SIMPLEX_DIR/xftp_tor/hs_ed25519_public_key"
# SSH host key (server fingerprint)
"/etc/ssh/ssh_host_ed25519_key"
"/etc/ssh/ssh_host_ed25519_key.pub"
# Root's SSH authorized_keys and config
"/root/.ssh/authorized_keys"
"/etc/ssh/sshd_config"
# SimpleX server configs and current certificates
"$SIMPLEX_DIR/smp_configs/smp-server.ini"
"$SIMPLEX_DIR/xftp_configs/file-server.ini"
"$SIMPLEX_DIR/smp_configs/server.crt"
"$SIMPLEX_DIR/smp_configs/server.key"
"$SIMPLEX_DIR/xftp_configs/server.crt"
"$SIMPLEX_DIR/xftp_configs/server.key"
# Current environment and docker-compose setup
"$SIMPLEX_DIR/.env"
"$SIMPLEX_DIR/docker-compose.yml"
"$SIMPLEX_DIR/print-addresses.sh"
# Tor configs
"$SIMPLEX_DIR/tor_conf/"
# Current firewall policies (so you can see what was open)
"/etc/awall/optional/"
)
# Check which targets actually exist
EXISTING_TARGETS=()
MISSING_TARGETS=()
for target in "${TARGETS[@]}"; do
if [[ -e "$target" ]]; then
EXISTING_TARGETS+=("$target")
else
MISSING_TARGETS+=("$target")
fi
done
log "Found ${#EXISTING_TARGETS[@]} backup targets"
if [[ ${#MISSING_TARGETS[@]} -gt 0 ]]; then
warn "Missing ${#MISSING_TARGETS[@]} expected files:"
printf ' %s\n' "${MISSING_TARGETS[@]}" >&2
fi
[[ ${#EXISTING_TARGETS[@]} -gt 0 ]] || die "No backup targets found."
# ----------------------------------------------------------------------------
# 3. Create the backup
# ----------------------------------------------------------------------------
TIMESTAMP=$(date -u +%Y%m%d-%H%M%S)
BACKUP_NAME="simplex-backup-${TIMESTAMP}"
BACKUP_TAR="${BACKUP_DIR}/${BACKUP_NAME}.tar.gz"
BACKUP_ENCRYPTED="${BACKUP_TAR}.age"
log "Creating backup archive..."
# Use tar to preserve permissions, ownership, and handle both files and directories.
# The --transform option puts everything under a dated directory in the tarball.
tar -czf "$BACKUP_TAR" \
--transform="s|^|${BACKUP_NAME}/|" \
--preserve-permissions \
--same-owner \
"${EXISTING_TARGETS[@]}" 2>/dev/null
if [[ ! -f "$BACKUP_TAR" ]]; then
die "Failed to create tar archive '$BACKUP_TAR'"
fi
# Encrypt the tarball
log "Encrypting with age..."
age "${AGE_ARGS[@]}" --output "$BACKUP_ENCRYPTED" "$BACKUP_TAR"
# Remove the unencrypted tar
shred -u "$BACKUP_TAR" 2>/dev/null || rm -f "$BACKUP_TAR"
# ----------------------------------------------------------------------------
# 4. Verify the backup
# ----------------------------------------------------------------------------
log "Verifying backup can be read..."
if age --decrypt "${AGE_ARGS[@]/--recipient*/--identity}" --output /dev/null "$BACKUP_ENCRYPTED" 2>/dev/null; then
: # Verification with --identity would need the private key; skip for now
else
# Just check the file isn't empty/corrupted
[[ -s "$BACKUP_ENCRYPTED" ]] || die "Backup file appears empty or corrupted"
fi
BACKUP_SIZE=$(stat -c%s "$BACKUP_ENCRYPTED" 2>/dev/null || wc -c < "$BACKUP_ENCRYPTED")
log "Backup created: $(basename "$BACKUP_ENCRYPTED") (${BACKUP_SIZE} bytes)"
# ----------------------------------------------------------------------------
# 5. Cleanup old backups
# ----------------------------------------------------------------------------
log "Cleaning up old backups (keeping ${KEEP_BACKUPS})..."
find "$BACKUP_DIR" -name 'simplex-backup-*.tar.gz.age' -type f -print0 \
| sort -z \
| head -z -n -"$KEEP_BACKUPS" \
| xargs -0 rm -f
REMAINING=$(find "$BACKUP_DIR" -name 'simplex-backup-*.tar.gz.age' -type f | wc -l)
log "Backup directory now contains ${REMAINING} backup(s)"
# ----------------------------------------------------------------------------
# 6. Final report + retrieval instructions
# ----------------------------------------------------------------------------
cat <<EOF
================================================================
BACKUP COMPLETE
================================================================
Encrypted backup: ${BACKUP_ENCRYPTED}
Size: ${BACKUP_SIZE} bytes
This backup contains:
• SimpleX CA keys (smp_configs/ca.key, xftp_configs/ca.key)
• Tor hidden service keys (*/hs_ed25519_*_key)
• SSH host key (/etc/ssh/ssh_host_ed25519_key*)
• SSH authorized_keys and sshd_config
• SimpleX server configs and certificates
• Current compose stack (.env, docker-compose.yml)
• Firewall policies (/etc/awall/optional/)
RETRIEVE THE BACKUP:
1. Copy from the server:
scp -i ~/.ssh/your_key root@host:${BACKUP_ENCRYPTED} ./
2. Decrypt and extract:
age --decrypt --identity /path/to/your/age_private_key \\
${BACKUP_NAME}.tar.gz.age | tar xzf -
WARNING: This backup contains private keys. Store it securely and
delete it from /tmp after copying off-host.
RESTORE PROCESS (if needed):
1. Fresh Alpine install + harden-ssh.sh
2. Extract backup: ${BACKUP_NAME}/
3. Copy keys back to their original paths
4. Run deploy-simplex.sh (will reuse existing keys)
5. Verify fingerprints match the backup
================================================================
EOF
deployments/simplex/cloud-init.yml
+98
View File
@@ -0,0 +1,98 @@
#cloud-config
#
# SimpleX Chat Relay Server - Cloud-Init Configuration
#
# This cloud-init configuration deploys a complete SimpleX relay server
# on Alpine Linux with:
# - Post-quantum SSH hardening
# - SMP + XFTP servers with Tor hidden services
# - Caddy reverse proxy with Let's Encrypt
# - awall firewall with minimal attack surface
# - Encrypted backup of all server keys
#
# Customize the environment variables below, then use this as user-data
# when creating your cloud instance.
# Use Alpine Linux (most cloud providers support it)
# Recommended: Alpine 3.19+ for latest OpenSSH with PQ KEX support
runcmd:
# Set a hostname (optional)
- hostnamectl set-hostname simplex-relay
# Run the master installer
- |
curl -fsSL https://git.anomalous.dev/57_Wolve/automations/raw/branch/main/deployments/simplex/install-simplex.sh | \
REPO_URL=https://git.anomalous.dev/57_Wolve/automations.git \
DOMAIN=relay.yourdomain.com \
ACME_EMAIL=admin@yourdomain.com \
XFTP_QUOTA=100gb \
SSH_PORT=2222 \
ALLOWED_IP=your.client.ip.here \
KEY_TYPE=rsa4096 \
SMP_PASS= \
XFTP_PASS= \
SKIP_PROMPTS=1 \
AUTO_BACKUP=1 \
REMOVE_CA_KEYS=1 \
DEBUG=0 \
bash
# Optional: Configure additional settings
write_files:
# Custom SSH banner (optional)
- path: /etc/ssh/banner
content: |
===============================================
SimpleX Chat Relay Server
Authorized access only.
All connections are logged and monitored.
===============================================
permissions: '0644'
# Optional: Install additional packages
packages:
- htop
- nano
- curl
- jq
# Optional: Configure automatic security updates (Alpine)
package_update: true
package_upgrade: true
# Set timezone
timezone: UTC
# Configure locale
locale: en_US.UTF-8
# Configure SSH (these will be overridden by harden-ssh.sh)
ssh_pwauth: true # Will be disabled by harden-ssh.sh
disable_root: false # Keep root enabled for harden-ssh.sh
# Optional: Add non-root user (created before SSH hardening)
users:
- name: admin
groups: wheel
sudo: ['ALL=(ALL) NOPASSWD:ALL']
shell: /bin/bash
# Note: SSH hardening will restrict to Ed25519 keys only
# Add your Ed25519 public key here if you want this user to survive hardening:
# ssh_authorized_keys:
# - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA... your-key-here
# Optional: Configure fail2ban (will be replaced by sshguard)
# runcmd will install sshguard which is lighter and more suitable
# Security note: The master installer will:
# 1. Generate fresh SSH keys and disable password auth
# 2. Create an encrypted backup containing all private keys
# 3. Remove CA keys from disk (they exist only in the backup)
# 4. Lock down the firewall to required ports only
#
# Make sure to:
# 1. Download the encrypted backup immediately after deployment
# 2. Save the SSH private key from the installer output
# 3. Test SSH access before deploying to production
deployments/simplex/deploy-simplex.sh
+541
View File
@@ -0,0 +1,541 @@
#!/usr/bin/env bash
#
# deploy-simplex.sh
#
# Single-file deployment of SimpleX SMP + XFTP + Tor hidden services on a
# fresh Alpine Linux install, behind awall.
#
# Targets Alpine 3.19+. Run as root on a machine where:
# - smp.${DOMAIN} and xftp.${DOMAIN} both resolve here (A/AAAA records set)
# - SSH is your only existing access (the script preserves whatever port
# you specify in SSH_PORT)
#
# Usage:
# 1. Copy this file onto the host: scp deploy-simplex.sh root@host:/root/
# 2. Edit the CONFIG block below (or pass via env vars)
# 3. Run: bash deploy-simplex.sh
#
# What it does:
# 1. Enables community repo, updates apk
# 2. Installs docker, docker-cli-compose, awall, iptables, curl, openrc bits
# 3. Configures awall: deny-all default + ssh + http(80) + https(443) +
# smp(5223) + xftp(5443), with rate-limited SSH
# 4. Writes docker-compose.yml + Caddyfile generator + Tor configs into /opt/simplex
# 5. Starts the stack
# 6. Prints the multi-host SimpleX server addresses once Tor publishes
#
# What it does NOT do (intentional):
# - SSH hardening beyond keeping your existing port. Set up keys yourself
# before running this. The script does not change sshd_config.
# - Backups of CA keys -- it tells you to do that, you do it.
# - Auto-update of containers. Use `docker compose pull && up -d` periodically
# or add Watchtower if you want that.
set -euo pipefail
# ============================================================================
# CONFIG -- edit these or override via env vars
# ============================================================================
: "${DOMAIN:=example.com}" # apex domain; uses smp.$DOMAIN and xftp.$DOMAIN
: "${ACME_EMAIL:=admin@example.com}" # for Let's Encrypt
: "${XFTP_QUOTA:=50gb}" # disk quota for XFTP file storage
: "${KEY_TYPE:=rsa4096}" # Caddy TLS key: rsa4096|rsa2048|p384|p256
# Note: ed25519 NOT supported by Let's Encrypt
: "${SMP_PASS:=}" # optional: queue creation password
: "${XFTP_PASS:=}" # optional: file upload password
: "${SSH_PORT:=22}" # whatever your sshd is listening on
: "${INSTALL_DIR:=/opt/simplex}" # where everything lives
: "${WAN_IFACE:=}" # autodetected if blank
: "${CERT_PATH:=acme-v02.api.letsencrypt.org-directory}"
# use acme-staging-v02.api.letsencrypt.org-directory
# to test without burning rate limits
# ============================================================================
log() { printf '\033[1;32m[+]\033[0m %s\n' "$*"; }
warn() { printf '\033[1;33m[!]\033[0m %s\n' "$*" >&2; }
die() { printf '\033[1;31m[x]\033[0m %s\n' "$*" >&2; exit 1; }
[[ $EUID -eq 0 ]] || die "Run as root."
[[ -f /etc/alpine-release ]] || die "This script targets Alpine Linux."
if [[ -z "$WAN_IFACE" ]]; then
WAN_IFACE=$(ip -o -4 route show default | awk '{print $5; exit}')
[[ -n "$WAN_IFACE" ]] || die "Could not autodetect WAN interface; set WAN_IFACE."
fi
log "WAN interface: $WAN_IFACE"
log "Domain: $DOMAIN"
log "Install dir: $INSTALL_DIR"
# ----------------------------------------------------------------------------
# 1. Repos + packages
# ----------------------------------------------------------------------------
log "Enabling community repo and updating apk..."
ALPINE_VER=$(cut -d. -f1,2 < /etc/alpine-release)
if ! grep -qE "^https?://.+/v${ALPINE_VER}/community" /etc/apk/repositories; then
sed -i -E "s|^#(https?://.+/v${ALPINE_VER}/community)|\1|" /etc/apk/repositories || true
if ! grep -qE "^https?://.+/v${ALPINE_VER}/community" /etc/apk/repositories; then
# Fallback: append the standard community URL based on existing main mirror
MAIN_MIRROR=$(awk '/main$/ {print; exit}' /etc/apk/repositories \
| sed -E "s|/v${ALPINE_VER}/main|/v${ALPINE_VER}/community|")
[[ -n "$MAIN_MIRROR" ]] && echo "$MAIN_MIRROR" >> /etc/apk/repositories
fi
fi
apk update -q
apk upgrade -q
log "Installing packages..."
apk add -q \
docker docker-cli-compose \
awall iptables ip6tables \
curl bash openssl \
ca-certificates
# ----------------------------------------------------------------------------
# 2. Kernel modules + iptables/awall
# ----------------------------------------------------------------------------
log "Loading iptables kernel modules..."
modprobe -q ip_tables || true
modprobe -q ip6_tables || true
modprobe -q iptable_nat || true
modprobe -q iptable_filter || true
log "Enabling iptables/ip6tables/docker at boot..."
rc-update add iptables default
rc-update add ip6tables default
rc-update add docker default
log "Writing awall policies..."
mkdir -p /etc/awall/optional /etc/awall/private
cat > /etc/awall/optional/base.json <<JSON
{
"description": "SimpleX relay base policy: deny-all on internet zone",
"variable": { "wan_if": "${WAN_IFACE}" },
"zone": { "internet": { "iface": "\$wan_if" } },
"policy": [
{ "in": "internet", "action": "drop" },
{ "action": "accept" }
]
}
JSON
cat > /etc/awall/optional/ssh.json <<JSON
{
"description": "Allow SSH on tcp/${SSH_PORT}, rate-limited",
"filter": [
{
"in": "internet",
"out": "_fw",
"service": { "proto": "tcp", "port": ${SSH_PORT} },
"action": "accept",
"conn-limit": { "count": 5, "interval": 60 }
}
]
}
JSON
cat > /etc/awall/optional/web.json <<JSON
{
"description": "Allow HTTP/HTTPS for Caddy info pages and ACME",
"filter": [
{ "in": "internet", "out": "_fw", "service": "http", "action": "accept" },
{ "in": "internet", "out": "_fw", "service": "https", "action": "accept" }
]
}
JSON
cat > /etc/awall/optional/simplex.json <<JSON
{
"description": "SimpleX SMP (5223) and XFTP (5443) protocol ports",
"filter": [
{ "in": "internet", "out": "_fw",
"service": { "proto": "tcp", "port": 5223 }, "action": "accept" },
{ "in": "internet", "out": "_fw",
"service": { "proto": "tcp", "port": 5443 }, "action": "accept" }
]
}
JSON
cat > /etc/awall/optional/icmp.json <<JSON
{
"description": "Allow rate-limited ICMP echo (ping)",
"filter": [
{
"in": "internet",
"service": "ping",
"action": "accept",
"flow-limit": { "count": 10, "interval": 6 }
}
]
}
JSON
awall enable base ssh web simplex icmp
awall translate --output /etc/iptables 2>/dev/null || true
awall translate --output /etc/ip6tables 2>/dev/null || true
awall activate --force
rc-service iptables restart || rc-service iptables start
rc-service ip6tables restart || rc-service ip6tables start
# ----------------------------------------------------------------------------
# 3. Sysctl hardening (modest)
# ----------------------------------------------------------------------------
log "Applying minimal sysctl hardening..."
cat > /etc/sysctl.d/90-simplex.conf <<'EOF'
# IP spoof protection
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
# Don't forward unless this is a router (Docker re-enables forwarding for the
# docker0 bridge specifically, which is fine)
net.ipv4.ip_forward = 0
# Ignore source-routed packets, ICMP redirects, broadcasts
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.icmp_ignore_bogus_error_responses = 1
# SYN flood protection
net.ipv4.tcp_syncookies = 1
EOF
sysctl -p /etc/sysctl.d/90-simplex.conf >/dev/null
# ----------------------------------------------------------------------------
# 4. Start docker
# ----------------------------------------------------------------------------
log "Starting docker..."
rc-service docker status >/dev/null 2>&1 || rc-service docker start
# Wait for socket
for _ in $(seq 1 20); do
[[ -S /var/run/docker.sock ]] && break
sleep 1
done
[[ -S /var/run/docker.sock ]] || die "Docker socket didn't appear."
# ----------------------------------------------------------------------------
# 5. Lay out /opt/simplex
# ----------------------------------------------------------------------------
log "Writing compose stack to ${INSTALL_DIR}..."
mkdir -p "$INSTALL_DIR"/tor_conf
cd "$INSTALL_DIR"
cat > .env <<EOF
DOMAIN=${DOMAIN}
ACME_EMAIL=${ACME_EMAIL}
XFTP_QUOTA=${XFTP_QUOTA}
KEY_TYPE=${KEY_TYPE}
SMP_PASS=${SMP_PASS}
XFTP_PASS=${XFTP_PASS}
CERT_PATH=${CERT_PATH}
EOF
chmod 600 .env
cat > docker-compose.yml <<'YAML'
name: simplex
services:
caddy-init:
image: alpine:latest
command: >
sh -c '
if [ ! -f /etc/caddy/Caddyfile ]; then
cat > /etc/caddy/Caddyfile <<EOF
{
email ${ACME_EMAIL}
}
http://smp.${DOMAIN} {
redir https://smp.${DOMAIN}{uri} permanent
}
smp.${DOMAIN}:8443 {
tls { key_type ${KEY_TYPE} }
reverse_proxy smp-server:8000
}
http://xftp.${DOMAIN} {
redir https://xftp.${DOMAIN}{uri} permanent
}
xftp.${DOMAIN}:8443 {
tls { key_type ${KEY_TYPE} }
reverse_proxy xftp-server:8000
}
EOF
fi
'
environment:
DOMAIN: ${DOMAIN:?}
ACME_EMAIL: ${ACME_EMAIL:?}
KEY_TYPE: ${KEY_TYPE:-rsa4096}
volumes:
- ./caddy_conf:/etc/caddy
restart: "no"
caddy:
image: caddy:2-alpine
depends_on:
caddy-init:
condition: service_completed_successfully
cap_add:
- NET_ADMIN
ports:
- "80:80"
- "443:8443"
volumes:
- ./caddy_conf:/etc/caddy
- caddy_data:/data
- caddy_config:/config
restart: unless-stopped
healthcheck:
test: >
sh -c '
test -d /data/caddy/certificates/${CERT_PATH:-acme-v02.api.letsencrypt.org-directory}/smp.${DOMAIN} &&
test -d /data/caddy/certificates/${CERT_PATH:-acme-v02.api.letsencrypt.org-directory}/xftp.${DOMAIN}
'
interval: 5s
timeout: 3s
retries: 60
start_period: 30s
environment:
DOMAIN: ${DOMAIN}
CERT_PATH: ${CERT_PATH:-acme-v02.api.letsencrypt.org-directory}
smp-server:
image: simplexchat/smp-server:latest
depends_on:
caddy:
condition: service_healthy
environment:
ADDR: smp.${DOMAIN}
PASS: ${SMP_PASS:-}
WEB_MANUAL: "0"
volumes:
- ./smp_configs:/etc/opt/simplex
- ./smp_state:/var/opt/simplex
- type: volume
source: caddy_data
target: /certificates
read_only: true
volume:
subpath: "caddy/certificates/${CERT_PATH:-acme-v02.api.letsencrypt.org-directory}/smp.${DOMAIN}"
ports:
- "5223:5223"
restart: unless-stopped
smp-tor:
image: dperson/torproxy:latest
depends_on:
- smp-server
network_mode: "service:smp-server"
volumes:
- ./smp_tor:/var/lib/tor/hidden_service
- ./tor_conf/smp-torrc:/etc/tor/torrc:ro
restart: unless-stopped
xftp-server:
image: simplexchat/xftp-server:latest
depends_on:
caddy:
condition: service_healthy
environment:
ADDR: xftp.${DOMAIN}
QUOTA: ${XFTP_QUOTA:?}
PASS: ${XFTP_PASS:-}
WEB_MANUAL: "0"
volumes:
- ./xftp_configs:/etc/opt/simplex-xftp
- ./xftp_state:/var/opt/simplex-xftp
- ./xftp_files:/srv/xftp
- type: volume
source: caddy_data
target: /certificates
read_only: true
volume:
subpath: "caddy/certificates/${CERT_PATH:-acme-v02.api.letsencrypt.org-directory}/xftp.${DOMAIN}"
ports:
- "5443:443"
restart: unless-stopped
xftp-tor:
image: dperson/torproxy:latest
depends_on:
- xftp-server
network_mode: "service:xftp-server"
volumes:
- ./xftp_tor:/var/lib/tor/hidden_service
- ./tor_conf/xftp-torrc:/etc/tor/torrc:ro
restart: unless-stopped
volumes:
caddy_data:
caddy_config:
YAML
cat > tor_conf/smp-torrc <<'EOF'
# Single-hop / non-anonymous mode: lower latency. The SERVER's location is
# already public via clearnet, so hiding it is moot. Clients remain fully
# anonymous to us. Drop these three lines for a tor-only relay where the
# server's location should also be hidden.
SOCKSPort 0
HiddenServiceNonAnonymousMode 1
HiddenServiceSingleHopMode 1
HiddenServiceDir /var/lib/tor/hidden_service/
HiddenServicePort 5223 127.0.0.1:5223
HiddenServicePort 443 127.0.0.1:8000
Log notice stdout
EOF
cat > tor_conf/xftp-torrc <<'EOF'
SOCKSPort 0
HiddenServiceNonAnonymousMode 1
HiddenServiceSingleHopMode 1
HiddenServiceDir /var/lib/tor/hidden_service/
HiddenServicePort 443 127.0.0.1:443
HiddenServicePort 8443 127.0.0.1:8000
Log notice stdout
EOF
# ----------------------------------------------------------------------------
# 6. Helper script for printing addresses post-bringup
# ----------------------------------------------------------------------------
cat > print-addresses.sh <<'BASH'
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")"
# shellcheck disable=SC1091
source .env
get_fp() {
docker compose logs "$1" 2>/dev/null \
| grep -m1 'Server address:' \
| sed -E 's|.*://([^@]+)@.*|\1|'
}
read_or_blank() { [[ -f "$1" ]] && cat "$1" || echo ""; }
SMP_FP=$(get_fp smp-server)
XFTP_FP=$(get_fp xftp-server)
SMP_ONION=$(read_or_blank ./smp_tor/hostname)
XFTP_ONION=$(read_or_blank ./xftp_tor/hostname)
build_addr() {
local scheme=$1 fp=$2 clearnet=$3 onion=$4
local hosts="$clearnet"
[[ -n "$onion" ]] && hosts="${hosts},${onion}"
echo "${scheme}://${fp}@${hosts}"
}
echo
echo "=== SMP ==="
echo " fingerprint: ${SMP_FP:-<not yet available>}"
echo " clearnet: smp.${DOMAIN}"
echo " onion: ${SMP_ONION:-<not yet available>}"
echo " full addr: $(build_addr smp "$SMP_FP" "smp.${DOMAIN}" "$SMP_ONION")"
echo
echo "=== XFTP ==="
echo " fingerprint: ${XFTP_FP:-<not yet available>}"
echo " clearnet: xftp.${DOMAIN}"
echo " onion: ${XFTP_ONION:-<not yet available>}"
echo " full addr: $(build_addr xftp "$XFTP_FP" "xftp.${DOMAIN}" "$XFTP_ONION")"
echo
BASH
chmod +x print-addresses.sh
# ----------------------------------------------------------------------------
# 7. OpenRC service for the compose stack itself
# ----------------------------------------------------------------------------
log "Creating openrc service for the stack..."
cat > /etc/init.d/simplex <<'EOF'
#!/sbin/openrc-run
name="simplex"
description="SimpleX SMP + XFTP + Tor docker compose stack"
directory="/opt/simplex"
depend() {
need docker
after net firewall
}
start() {
ebegin "Starting SimpleX stack"
cd "$directory"
docker compose up -d
eend $?
}
stop() {
ebegin "Stopping SimpleX stack"
cd "$directory"
docker compose down
eend $?
}
status() {
cd "$directory"
docker compose ps
}
EOF
chmod +x /etc/init.d/simplex
rc-update add simplex default
# ----------------------------------------------------------------------------
# 8. Bring it up
# ----------------------------------------------------------------------------
log "Pulling images and starting the stack..."
cd "$INSTALL_DIR"
docker compose pull
docker compose up -d
log "Waiting up to 90s for Tor to publish hidden services..."
for _ in $(seq 1 90); do
if [[ -f "$INSTALL_DIR/smp_tor/hostname" && -f "$INSTALL_DIR/xftp_tor/hostname" ]]; then
break
fi
sleep 1
done
# ----------------------------------------------------------------------------
# 9. Final report
# ----------------------------------------------------------------------------
echo
echo "================================================================"
echo " SimpleX deployment complete"
echo "================================================================"
"$INSTALL_DIR/print-addresses.sh" || true
cat <<EOF
NEXT STEPS (do these now, not later):
1. BACK UP CA KEYS (these prove server identity if you ever need to rotate):
$INSTALL_DIR/smp_configs/ca.key
$INSTALL_DIR/xftp_configs/ca.key
Copy them off this host, then delete the on-host copies:
rm $INSTALL_DIR/smp_configs/ca.key
rm $INSTALL_DIR/xftp_configs/ca.key
2. BACK UP TOR HIDDEN SERVICE KEYS (preserve the .onion across reinstalls):
$INSTALL_DIR/smp_tor/hs_ed25519_secret_key
$INSTALL_DIR/xftp_tor/hs_ed25519_secret_key
3. Add the server addresses above to your SimpleX app:
Settings -> Network & Servers -> SMP/XFTP servers -> Add
4. Verify the firewall is doing its job:
awall list
iptables -L -n
5. Watch the stack:
cd $INSTALL_DIR && docker compose logs -f
EOF
deployments/simplex/install-simplex.sh
+298
View File
@@ -0,0 +1,298 @@
#!/usr/bin/env bash
#
# install-simplex.sh
#
# Master installer for SimpleX relay deployment. Designed for cloud-init but
# works as a standalone script. Fetches deployment scripts from git, runs the
# complete setup sequence, and creates an initial encrypted backup.
#
# This script:
# 1. Fetches deployment scripts from your git repo
# 2. Runs harden-ssh.sh (PQ KEX, Ed25519, firewall)
# 3. Runs deploy-simplex.sh (SMP + XFTP + Tor)
# 4. Runs backup.sh with your age public key
# 5. Cleans up CA keys from disk (they're now only in the encrypted backup)
# 6. Reports final status and backup location
#
# Usage as cloud-init user-data:
# #cloud-config
# runcmd:
# - |
# curl -fsSL https://git.anomalous.dev/57_Wolve/automations/raw/branch/main/deployments/simplex/install-simplex.sh | \
# REPO_URL=https://git.anomalous.dev/57_Wolve/automations.git \
# DOMAIN=relay.example.com \
# ACME_EMAIL=admin@example.com \
# XFTP_QUOTA=100gb \
# SSH_PORT=2222 \
# bash
#
# Usage as standalone script:
# curl -fsSL https://git.anomalous.dev/57_Wolve/automations/raw/branch/main/deployments/simplex/install-simplex.sh > install-simplex.sh
# REPO_URL=https://git.anomalous.dev/57_Wolve/automations.git \
# DOMAIN=relay.example.com \
# ACME_EMAIL=admin@example.com \
# bash install-simplex.sh
#
# Required git repo structure (this monorepo):
# automations/
# ├── scripts/
# │ └── harden-ssh.sh # generic, run-anywhere
# ├── deployments/simplex/
# │ ├── deploy-simplex.sh
# │ ├── backup.sh
# │ └── restore.sh # optional
# └── globals/
# └── age-pubkey.txt # your age public key(s), one per line
set -euo pipefail
# ============================================================================
# CONFIG
# ============================================================================
# Git repository containing the deployment scripts and age public key
: "${REPO_URL:=}" # REQUIRED: git repo URL
# SimpleX deployment config
: "${DOMAIN:=}" # REQUIRED: apex domain (uses smp.DOMAIN, xftp.DOMAIN)
: "${ACME_EMAIL:=}" # REQUIRED: Let's Encrypt email
: "${XFTP_QUOTA:=50gb}" # XFTP disk quota
: "${SSH_PORT:=22}" # SSH port (recommend changing from default)
: "${KEY_TYPE:=rsa4096}" # Caddy TLS key type
: "${SMP_PASS:=}" # optional: SMP queue creation password
: "${XFTP_PASS:=}" # optional: XFTP upload password
# Git and installation options
: "${REPO_BRANCH:=main}" # git branch to fetch
: "${HARDEN_PATH:=scripts}" # path within repo to the generic harden-ssh.sh
: "${SIMPLEX_PATH:=deployments/simplex}" # path within repo to the simplex scripts
: "${AGE_PUBKEY_FILE:=globals/age-pubkey.txt}" # path within repo to age public key
: "${INSTALL_DIR:=/opt/simplex-deploy}" # where to clone the repo
: "${ALLOWED_IP:=}" # optional: IP to whitelist in sshguard
: "${AUTO_BACKUP:=1}" # set to 0 to skip initial backup
: "${REMOVE_CA_KEYS:=1}" # set to 0 to keep CA keys on disk
# Behavior flags
: "${SKIP_PROMPTS:=0}" # set to 1 for non-interactive operation
: "${DEBUG:=0}" # set to 1 for verbose output
# ============================================================================
log() { printf '\033[1;32m[+]\033[0m %s\n' "$*"; }
warn() { printf '\033[1;33m[!]\033[0m %s\n' "$*" >&2; }
die() { printf '\033[1;31m[x]\033[0m %s\n' "$*" >&2; exit 1; }
[[ $EUID -eq 0 ]] || die "Run as root."
[[ -f /etc/alpine-release ]] || die "This script targets Alpine Linux."
if [[ "$DEBUG" == "1" ]]; then
set -x
fi
# ----------------------------------------------------------------------------
# 1. Validate required parameters
# ----------------------------------------------------------------------------
[[ -n "$REPO_URL" ]] || die "Set REPO_URL=https://git.anomalous.dev/57_Wolve/automations.git"
[[ -n "$DOMAIN" ]] || die "Set DOMAIN=your.domain.com"
[[ -n "$ACME_EMAIL" ]] || die "Set ACME_EMAIL=admin@your.domain.com"
log "SimpleX relay installer"
log "Repo: $REPO_URL"
log "Domain: $DOMAIN (will create smp.$DOMAIN, xftp.$DOMAIN)"
log "SSH port: $SSH_PORT"
if [[ "$SKIP_PROMPTS" != "1" ]]; then
cat <<EOF
This will install a complete SimpleX relay server on this host:
• SSH hardening with PQ KEX and Ed25519 keys
• awall firewall with minimal attack surface
• SMP + XFTP servers with Tor hidden services
• Caddy reverse proxy with Let's Encrypt TLS
• Initial encrypted backup of all server keys
The process will generate new server keys and may change your SSH config.
Only run this on a fresh Alpine install intended as a SimpleX relay.
Continue? [y/N]
EOF
read -r ans
[[ "${ans,,}" =~ ^(y|yes)$ ]] || die "Aborted."
fi
# ----------------------------------------------------------------------------
# 2. Install git and fetch deployment repo
# ----------------------------------------------------------------------------
log "Installing git and fetching deployment scripts..."
apk add -q git
rm -rf "$INSTALL_DIR"
git clone --depth 1 --branch "$REPO_BRANCH" "$REPO_URL" "$INSTALL_DIR"
# Verify required files exist. harden-ssh.sh is generic and lives under
# scripts/; the simplex scripts live under deployments/simplex/.
HARDEN_DIR="$INSTALL_DIR/$HARDEN_PATH"
SIMPLEX_DIR="$INSTALL_DIR/$SIMPLEX_PATH"
verify_script() { # <dir> <relpath-for-messages> <name>
local path="$1/$3"
[[ -f "$path" ]] || die "Required script not found: $2/$3"
[[ -x "$path" ]] || chmod +x "$path"
}
verify_script "$HARDEN_DIR" "$HARDEN_PATH" "harden-ssh.sh"
verify_script "$SIMPLEX_DIR" "$SIMPLEX_PATH" "deploy-simplex.sh"
verify_script "$SIMPLEX_DIR" "$SIMPLEX_PATH" "backup.sh"
# Check for age public key file
AGE_PUBKEY_PATH="$INSTALL_DIR/$AGE_PUBKEY_FILE"
if [[ "$AUTO_BACKUP" == "1" ]]; then
[[ -f "$AGE_PUBKEY_PATH" ]] || die "Age public key file not found: $AGE_PUBKEY_FILE"
# Validate it contains what looks like age public keys
if ! grep -q "^age1" "$AGE_PUBKEY_PATH"; then
die "Age public key file doesn't contain valid age1... public keys"
fi
fi
log "Repository cloned to: $INSTALL_DIR"
# ----------------------------------------------------------------------------
# 3. Run SSH hardening
# ----------------------------------------------------------------------------
log "Step 1/3: SSH hardening..."
SSH_ENV=""
[[ -n "$SSH_PORT" ]] && SSH_ENV="${SSH_ENV} SSH_PORT=$SSH_PORT"
[[ -n "$ALLOWED_IP" ]] && SSH_ENV="${SSH_ENV} ALLOWED_IP=$ALLOWED_IP"
[[ "$SKIP_PROMPTS" == "1" ]] && SSH_ENV="${SSH_ENV} FORCE=1"
# Capture the private key output from harden-ssh.sh
HARDEN_OUTPUT=$(env $SSH_ENV "$HARDEN_DIR/harden-ssh.sh" 2>&1) || {
echo "$HARDEN_OUTPUT" >&2
die "SSH hardening failed"
}
# Extract the private key from the output for later display
SSH_PRIVATE_KEY=$(echo "$HARDEN_OUTPUT" | sed -n '/BEGIN ROOT PRIVATE KEY/,/END ROOT PRIVATE KEY/p')
SSH_PUBLIC_KEY=$(echo "$HARDEN_OUTPUT" | grep "Public key (already in" | sed 's/.*: //')
HOST_FINGERPRINT=$(echo "$HARDEN_OUTPUT" | grep "Host fingerprint" | tail -1)
log "SSH hardening completed"
# ----------------------------------------------------------------------------
# 4. Run SimpleX deployment
# ----------------------------------------------------------------------------
log "Step 2/3: SimpleX deployment..."
DEPLOY_ENV=""
DEPLOY_ENV="${DEPLOY_ENV} DOMAIN=$DOMAIN"
DEPLOY_ENV="${DEPLOY_ENV} ACME_EMAIL=$ACME_EMAIL"
DEPLOY_ENV="${DEPLOY_ENV} XFTP_QUOTA=$XFTP_QUOTA"
DEPLOY_ENV="${DEPLOY_ENV} SSH_PORT=$SSH_PORT"
DEPLOY_ENV="${DEPLOY_ENV} KEY_TYPE=$KEY_TYPE"
[[ -n "$SMP_PASS" ]] && DEPLOY_ENV="${DEPLOY_ENV} SMP_PASS=$SMP_PASS"
[[ -n "$XFTP_PASS" ]] && DEPLOY_ENV="${DEPLOY_ENV} XFTP_PASS=$XFTP_PASS"
env $DEPLOY_ENV "$SIMPLEX_DIR/deploy-simplex.sh" || die "SimpleX deployment failed"
log "SimpleX deployment completed"
# ----------------------------------------------------------------------------
# 5. Create initial encrypted backup
# ----------------------------------------------------------------------------
if [[ "$AUTO_BACKUP" == "1" ]]; then
log "Step 3/3: Creating initial encrypted backup..."
# Wait a bit for the services to fully initialize
sleep 10
BACKUP_ENV="AGE_RECIPIENT_FILE=$AGE_PUBKEY_PATH"
env $BACKUP_ENV "$SIMPLEX_DIR/backup.sh" || die "Backup creation failed"
log "Initial backup created"
# Remove CA keys from disk now that they're safely backed up
if [[ "$REMOVE_CA_KEYS" == "1" ]]; then
log "Removing CA private keys from disk (they're now only in the encrypted backup)..."
rm -f /opt/simplex/smp_configs/ca.key /opt/simplex/xftp_configs/ca.key
log "CA keys removed from disk"
fi
else
log "Step 3/3: Skipped (AUTO_BACKUP=0)"
fi
# ----------------------------------------------------------------------------
# 6. Final status report
# ----------------------------------------------------------------------------
sleep 5 # Let services settle
cat <<EOF
================================================================
SIMPLEX RELAY INSTALLATION COMPLETE
================================================================
Your SimpleX relay is now running at:
Domain: $DOMAIN
SMP: smp.$DOMAIN:5223
XFTP: xftp.$DOMAIN:5443
SSH: port $SSH_PORT
EOF
# Show server addresses if available
if [[ -f /opt/simplex/print-addresses.sh ]]; then
cd /opt/simplex && ./print-addresses.sh 2>/dev/null || {
log "Server addresses not yet ready; check with: cd /opt/simplex && ./print-addresses.sh"
}
else
log "Server addresses script not found"
fi
# Show backup location
LATEST_BACKUP=$(find /tmp -name 'simplex-backup-*.tar.gz.age' -type f -printf '%T@ %p\n' 2>/dev/null | sort -nr | head -1 | cut -d' ' -f2-)
if [[ -n "$LATEST_BACKUP" && "$AUTO_BACKUP" == "1" ]]; then
BACKUP_SIZE=$(stat -c%s "$LATEST_BACKUP" 2>/dev/null || echo "unknown")
cat <<EOF
BACKUP CREATED:
File: $LATEST_BACKUP
Size: $BACKUP_SIZE bytes
IMPORTANT: Download this backup immediately and store it securely.
It contains all server private keys. The CA keys have been removed
from disk and exist only in this encrypted backup.
Download with:
scp -i ~/.ssh/your_key -P $SSH_PORT root@$DOMAIN:$LATEST_BACKUP ./
EOF
fi
cat <<EOF
IMPORTANT - SAVE THIS SSH PRIVATE KEY:
$SSH_PRIVATE_KEY
Connect with:
ssh -i ~/.ssh/id_ed25519_simplex -p $SSH_PORT root@$DOMAIN
$HOST_FINGERPRINT
NEXT STEPS:
1. Save the SSH private key above to your local machine
2. Download the encrypted backup from $LATEST_BACKUP
3. Test SSH access from another terminal
4. Add the server to your SimpleX app:
Settings -> Network & Servers -> Add servers -> paste the addresses above
5. Test that contacts can reach your relay
================================================================
EOF
# Clean up the git repo (optional - contains no secrets)
if [[ "${CLEANUP_REPO:-1}" == "1" ]]; then
rm -rf "$INSTALL_DIR"
log "Cleaned up installation directory"
fi
log "Installation complete!"
deployments/simplex/restore.sh
+416
View File
@@ -0,0 +1,416 @@
#!/usr/bin/env bash
#
# restore.sh
#
# Restores a SimpleX server from an age-encrypted backup created by backup.sh.
# This script can rebuild the entire server setup from scratch while preserving:
# - Server fingerprints (SimpleX CA keys)
# - .onion addresses (Tor hidden service keys)
# - SSH host key fingerprint
# - SSH authorized_keys and hardened config
# - All server configurations
#
# IMPORTANT: Run this on a FRESH Alpine installation. This script will:
# 1. Decrypt and extract the backup
# 2. Install all required packages
# 3. Restore SSH hardening + keys
# 4. Restore firewall config
# 5. Deploy the SimpleX stack with preserved keys
# 6. Start services
#
# Usage:
# # Copy restore.sh and your backup onto fresh Alpine host
# scp restore.sh backup-YYYYMMDD-HHMMSS.tar.gz.age root@new-host:/root/
#
# # Run restore (will prompt for age private key)
# ssh root@new-host
# bash restore.sh backup-YYYYMMDD-HHMMSS.tar.gz.age
#
# # Alternative: pass age identity file
# AGE_IDENTITY=/root/backup-key.txt bash restore.sh backup.tar.gz.age
set -euo pipefail
# ============================================================================
# CONFIG
# ============================================================================
: "${AGE_IDENTITY:=}" # optional: path to age private key file
: "${FORCE_OVERWRITE:=0}" # set to 1 to overwrite existing files
: "${SKIP_VALIDATION:=0}" # set to 1 to skip "are you sure" prompts
BACKUP_FILE="${1:-}"
log() { printf '\033[1;32m[+]\033[0m %s\n' "$*"; }
warn() { printf '\033[1;33m[!]\033[0m %s\n' "$*" >&2; }
die() { printf '\033[1;31m[x]\033[0m %s\n' "$*" >&2; exit 1; }
# ----------------------------------------------------------------------------
# 1. Validation and setup
# ----------------------------------------------------------------------------
[[ $EUID -eq 0 ]] || die "Run as root."
[[ -f /etc/alpine-release ]] || die "This script targets Alpine Linux."
[[ -n "$BACKUP_FILE" ]] || die "Usage: $0 <backup-file.tar.gz.age>"
[[ -f "$BACKUP_FILE" ]] || die "Backup file '$BACKUP_FILE' not found."
if [[ "$SKIP_VALIDATION" != "1" ]]; then
cat <<EOF
WARNING: This will restore a SimpleX server backup to this host.
This script will:
1. Install packages (docker, openssh, awall, sshguard, age)
2. Restore SSH hardening (may change your SSH config)
3. Restore firewall rules (may change your network access)
4. Deploy the SimpleX docker stack
5. Restore all server keys and certificates
ONLY run this on a fresh Alpine installation intended to become
a SimpleX relay server. This will overwrite existing configs.
Continue? [y/N]
EOF
read -r ans
[[ "${ans,,}" =~ ^(y|yes)$ ]] || die "Aborted."
fi
log "Starting restore from: $(basename "$BACKUP_FILE")"
# ----------------------------------------------------------------------------
# 2. Install age and decrypt backup
# ----------------------------------------------------------------------------
if ! command -v age >/dev/null; then
log "Installing age..."
apk add -q age
fi
RESTORE_DIR="/tmp/restore-$$"
mkdir -p "$RESTORE_DIR"
cd "$RESTORE_DIR"
log "Decrypting backup..."
if [[ -n "$AGE_IDENTITY" ]]; then
if [[ ! -f "$AGE_IDENTITY" ]]; then
die "AGE_IDENTITY file '$AGE_IDENTITY' not found."
fi
age --decrypt --identity "$AGE_IDENTITY" "$BACKUP_FILE" | tar xzf -
else
# Interactive decryption - age will prompt for passphrase or identity
echo "Enter your age private key or passphrase when prompted:"
age --decrypt "$BACKUP_FILE" | tar xzf -
fi
# Find the extracted directory (should be simplex-backup-YYYYMMDD-HHMMSS)
BACKUP_EXTRACT_DIR=$(find . -maxdepth 1 -type d -name 'simplex-backup-*' | head -1)
[[ -n "$BACKUP_EXTRACT_DIR" ]] || die "Could not find extracted backup directory."
BACKUP_EXTRACT_DIR=$(realpath "$BACKUP_EXTRACT_DIR")
log "Extracted to: $BACKUP_EXTRACT_DIR"
# ----------------------------------------------------------------------------
# 3. Install required packages
# ----------------------------------------------------------------------------
log "Updating package lists and installing requirements..."
# Enable community repo if not already
ALPINE_VER=$(cut -d. -f1,2 < /etc/alpine-release)
if ! grep -qE "^https?://.+/v${ALPINE_VER}/community" /etc/apk/repositories; then
sed -i -E "s|^#(https?://.+/v${ALPINE_VER}/community)|\1|" /etc/apk/repositories || {
MAIN_MIRROR=$(awk '/main$/ {print; exit}' /etc/apk/repositories \
| sed -E "s|/v${ALPINE_VER}/main|/v${ALPINE_VER}/community|")
[[ -n "$MAIN_MIRROR" ]] && echo "$MAIN_MIRROR" >> /etc/apk/repositories
}
fi
apk update -q
apk upgrade -q
apk add -q \
docker docker-cli-compose \
openssh openssh-server \
awall iptables ip6tables \
sshguard \
curl bash openssl \
ca-certificates openrc
# ----------------------------------------------------------------------------
# 4. Restore SSH configuration and keys
# ----------------------------------------------------------------------------
log "Restoring SSH configuration and keys..."
# Back up existing SSH config if it exists
[[ -f /etc/ssh/sshd_config ]] && cp /etc/ssh/sshd_config /etc/ssh/sshd_config.pre-restore
# Restore SSH host key
if [[ -f "$BACKUP_EXTRACT_DIR/etc/ssh/ssh_host_ed25519_key" ]]; then
cp "$BACKUP_EXTRACT_DIR/etc/ssh/ssh_host_ed25519_key" /etc/ssh/
cp "$BACKUP_EXTRACT_DIR/etc/ssh/ssh_host_ed25519_key.pub" /etc/ssh/
chmod 600 /etc/ssh/ssh_host_ed25519_key
chmod 644 /etc/ssh/ssh_host_ed25519_key.pub
# Remove any other host key types
rm -f /etc/ssh/ssh_host_rsa_key* /etc/ssh/ssh_host_ecdsa_key* /etc/ssh/ssh_host_dsa_key*
log "Restored SSH host key: $(ssh-keygen -l -f /etc/ssh/ssh_host_ed25519_key.pub)"
else
warn "No SSH host key found in backup; keeping system default"
fi
# Restore sshd_config
if [[ -f "$BACKUP_EXTRACT_DIR/etc/ssh/sshd_config" ]]; then
cp "$BACKUP_EXTRACT_DIR/etc/ssh/sshd_config" /etc/ssh/
log "Restored SSH daemon configuration"
else
warn "No sshd_config found in backup"
fi
# Restore root's authorized_keys
mkdir -p /root/.ssh
chmod 700 /root/.ssh
if [[ -f "$BACKUP_EXTRACT_DIR/root/.ssh/authorized_keys" ]]; then
cp "$BACKUP_EXTRACT_DIR/root/.ssh/authorized_keys" /root/.ssh/
chmod 600 /root/.ssh/authorized_keys
KEY_COUNT=$(wc -l < /root/.ssh/authorized_keys)
log "Restored ${KEY_COUNT} authorized SSH key(s) for root"
else
warn "No authorized_keys found in backup"
fi
# Validate SSH config
if [[ -f /etc/ssh/sshd_config ]]; then
if sshd -t 2>/dev/null; then
log "SSH configuration validated"
else
warn "SSH configuration validation failed; keeping anyway"
fi
fi
# ----------------------------------------------------------------------------
# 5. Restore firewall configuration
# ----------------------------------------------------------------------------
log "Restoring firewall configuration..."
# Load iptables kernel modules
modprobe -q ip_tables || true
modprobe -q ip6_tables || true
modprobe -q iptable_nat || true
modprobe -q iptable_filter || true
# Restore awall policies if they exist
if [[ -d "$BACKUP_EXTRACT_DIR/etc/awall/optional" ]]; then
mkdir -p /etc/awall/optional
cp -r "$BACKUP_EXTRACT_DIR/etc/awall/optional/"* /etc/awall/optional/
# Enable the restored policies
cd /etc/awall/optional
for policy in *.json; do
[[ -f "$policy" ]] || continue
POLICY_NAME=$(basename "$policy" .json)
awall enable "$POLICY_NAME"
log "Enabled awall policy: $POLICY_NAME"
done
# Activate firewall
awall translate --output /etc/iptables 2>/dev/null || true
awall translate --output /etc/ip6tables 2>/dev/null || true
awall activate --force
log "Firewall rules activated"
else
warn "No awall policies found in backup"
fi
# Enable services
rc-update add iptables default || true
rc-update add ip6tables default || true
rc-update add sshd default || true
rc-update add sshguard default || true
rc-update add docker default || true
# ----------------------------------------------------------------------------
# 6. Restore SimpleX configuration and keys
# ----------------------------------------------------------------------------
SIMPLEX_DIR="/opt/simplex"
log "Restoring SimpleX configuration to $SIMPLEX_DIR..."
mkdir -p "$SIMPLEX_DIR"
# Restore the core SimpleX files
SIMPLEX_FILES=(
".env"
"docker-compose.yml"
"print-addresses.sh"
"tor_conf/"
)
for item in "${SIMPLEX_FILES[@]}"; do
SRC="$BACKUP_EXTRACT_DIR/opt/simplex/$item"
if [[ -e "$SRC" ]]; then
cp -r "$SRC" "$SIMPLEX_DIR/"
log "Restored: $item"
else
warn "Missing from backup: $item"
fi
done
# Make script executable
[[ -f "$SIMPLEX_DIR/print-addresses.sh" ]] && chmod +x "$SIMPLEX_DIR/print-addresses.sh"
# Restore SimpleX server keys and configs
mkdir -p "$SIMPLEX_DIR/smp_configs" "$SIMPLEX_DIR/xftp_configs" \
"$SIMPLEX_DIR/smp_state" "$SIMPLEX_DIR/xftp_state" \
"$SIMPLEX_DIR/xftp_files" "$SIMPLEX_DIR/smp_tor" "$SIMPLEX_DIR/xftp_tor"
# SMP server restoration
for key in ca.key server.crt server.key smp-server.ini; do
SRC="$BACKUP_EXTRACT_DIR/opt/simplex/smp_configs/$key"
if [[ -f "$SRC" ]]; then
cp "$SRC" "$SIMPLEX_DIR/smp_configs/"
log "Restored SMP: $key"
fi
done
# XFTP server restoration
for key in ca.key server.crt server.key file-server.ini; do
SRC="$BACKUP_EXTRACT_DIR/opt/simplex/xftp_configs/$key"
if [[ -f "$SRC" ]]; then
cp "$SRC" "$SIMPLEX_DIR/xftp_configs/"
log "Restored XFTP: $key"
fi
done
# Tor hidden service keys
for service in smp xftp; do
for key in hs_ed25519_secret_key hs_ed25519_public_key hostname; do
SRC="$BACKUP_EXTRACT_DIR/opt/simplex/${service}_tor/$key"
if [[ -f "$SRC" ]]; then
cp "$SRC" "$SIMPLEX_DIR/${service}_tor/"
log "Restored Tor ($service): $key"
fi
done
done
# Set proper ownership and permissions
chmod 600 "$SIMPLEX_DIR"/*_configs/ca.key 2>/dev/null || true
chmod 600 "$SIMPLEX_DIR"/*_tor/hs_ed25519_secret_key 2>/dev/null || true
# ----------------------------------------------------------------------------
# 7. Start services
# ----------------------------------------------------------------------------
log "Starting services..."
# Start firewall services
rc-service iptables start || rc-service iptables restart || true
rc-service ip6tables start || rc-service ip6tables restart || true
# Configure and start sshguard (basic setup - backup.sh doesn't save the full config)
mkdir -p /etc/sshguard
cat > /etc/sshguard/sshguard.conf <<'EOF'
BACKEND="/usr/libexec/sshg-fw-iptables"
LOGREADER="LANG=C journalctl -afb -p info -n1 -u sshd -o cat"
THRESHOLD=30
BLOCK_TIME=300
DETECTION_TIME=1800
PID_FILE=/run/sshguard.pid
EOF
# Basic sshguard iptables setup
cat > /etc/local.d/sshguard-iptables.start <<'EOF'
#!/bin/sh
SSH_PORT=$(awk '/^Port / {print $2; exit}' /etc/ssh/sshd_config)
SSH_PORT=${SSH_PORT:-22}
for ipt in iptables ip6tables; do
$ipt -N sshguard 2>/dev/null || true
$ipt -C INPUT -p tcp --dport "$SSH_PORT" -j sshguard 2>/dev/null \
|| $ipt -I INPUT -p tcp --dport "$SSH_PORT" -j sshguard
done
EOF
chmod +x /etc/local.d/sshguard-iptables.start
rc-update add local default 2>/dev/null || true
/etc/local.d/sshguard-iptables.start
rc-service sshguard start || true
# Start docker
rc-service docker start || true
# Wait for docker socket
for _ in $(seq 1 20); do
[[ -S /var/run/docker.sock ]] && break
sleep 1
done
if [[ ! -S /var/run/docker.sock ]]; then
die "Docker socket didn't appear after starting service"
fi
# ----------------------------------------------------------------------------
# 8. Deploy SimpleX stack
# ----------------------------------------------------------------------------
if [[ -f "$SIMPLEX_DIR/docker-compose.yml" && -f "$SIMPLEX_DIR/.env" ]]; then
log "Starting SimpleX docker stack..."
cd "$SIMPLEX_DIR"
docker compose pull
docker compose up -d
log "Waiting for services to stabilize..."
sleep 10
# Show status
docker compose ps
else
warn "SimpleX compose files not found; skipping docker deployment"
fi
# ----------------------------------------------------------------------------
# 9. SSH service restart (do this last so we don't kill our session early)
# ----------------------------------------------------------------------------
log "Reloading SSH service with restored configuration..."
rc-service sshd reload || rc-service sshd restart || true
# ----------------------------------------------------------------------------
# 10. Cleanup and final report
# ----------------------------------------------------------------------------
log "Cleaning up temporary files..."
rm -rf "$RESTORE_DIR"
# Display current status and next steps
cat <<EOF
================================================================
RESTORE COMPLETE
================================================================
Services Status:
$(rc-service --list | grep -E '(docker|sshd|iptables|sshguard)' || true)
Next Steps:
1. VERIFY SSH ACCESS STILL WORKS
Test from another terminal before closing this session:
ssh -i ~/.ssh/your_key root@$(hostname)
2. CHECK SIMPLEX SERVICES
cd $SIMPLEX_DIR && docker compose ps
./print-addresses.sh
3. VERIFY SERVER FINGERPRINTS
Compare these to your original backup to confirm identities were preserved:
SSH host key:
$(ssh-keygen -l -f /etc/ssh/ssh_host_ed25519_key.pub 2>/dev/null | sed 's/^/ /' || echo " [not found]")
SMP server (when ready):
$(cd "$SIMPLEX_DIR" 2>/dev/null && docker compose logs smp-server 2>/dev/null | grep -m1 'Server address:' | sed 's/^/ /' || echo " [check with: cd $SIMPLEX_DIR && docker compose logs smp-server]")
4. VERIFY .ONION ADDRESSES
These should match your backup:
SMP: $(cat "$SIMPLEX_DIR/smp_tor/hostname" 2>/dev/null || echo "[not yet available]")
XFTP: $(cat "$SIMPLEX_DIR/xftp_tor/hostname" 2>/dev/null || echo "[not yet available]")
5. UPDATE DNS
Ensure smp.yourdomain.com and xftp.yourdomain.com point to this host
6. TEST CONNECTIONS
Verify existing SimpleX contacts can still reach your server
================================================================
EOF
deployments/webfinger/.env.example
+16
View File
@@ -0,0 +1,16 @@
# Copy to .env and fill in. docker compose picks .env up automatically.
# Apex domain to serve from (no scheme). Caddy auto-issues an LE cert.
BASE_DOMAIN=example.com
# OIDC issuer URL for WebFinger discovery (your pocket-id deployment).
ISSUER_URL=https://auth.example.com
# Where to send any non-WebFinger request (preserves path + query).
REDIRECT_URL=https://your-redirect-target.example.com
# Let's Encrypt registration email.
ACME_EMAIL=admin@example.com
# Image tag.
CADDY_TAG=2-alpine
deployments/webfinger/Caddyfile
+50
View File
@@ -0,0 +1,50 @@
# Caddyfile for the webfinger + redirect stack.
#
# - /.well-known/webfinger -> dynamic JRD response, echoing the queried
# resource as `subject` and pointing OIDC discovery at $ISSUER_URL.
# - everything else -> 301 to $REDIRECT_URL (preserving path + query).
#
# To also serve www: change `{$BASE_DOMAIN}` below to
# `{$BASE_DOMAIN}, www.{$BASE_DOMAIN}`.
{
email {$ACME_EMAIL}
# Staging CA for testing without burning LE rate limits:
# acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
}
{$BASE_DOMAIN} {
encode zstd gzip
@webfinger path /.well-known/webfinger
handle @webfinger {
header Content-Type "application/jrd+json"
header Cache-Control "public, max-age=3600"
templates
respond `{
"subject": "{{.Req.URL.Query.Get "resource"}}",
"links": [
{
"rel": "http://openid.net/specs/connect/1.0/issuer",
"href": "{$ISSUER_URL}"
}
]
}` 200
}
handle {
redir {$REDIRECT_URL}{uri} permanent
}
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains"
X-Content-Type-Options "nosniff"
Referrer-Policy "strict-origin-when-cross-origin"
-Server
}
log {
output stdout
format console
}
}
deployments/webfinger/README.md
+42
View File
@@ -0,0 +1,42 @@
# webfinger
A single Caddy container that serves `/.well-known/webfinger` at an apex domain
(for OIDC issuer discovery — used by Tailscale and others) and 301-redirects
everything else to a target URL.
## Required `.env` values
| Variable | Notes |
|----------|-------|
| `BASE_DOMAIN` | Apex domain to serve from (e.g. `example.com`). |
| `ISSUER_URL` | OIDC issuer for the WebFinger response (your [pocket-id](../pocket-id/)). |
| `REDIRECT_URL` | Where non-WebFinger traffic is redirected (path + query preserved). |
| `ACME_EMAIL` | Let's Encrypt registration email. |
See [`.env.example`](.env.example).
## Deploy
```bash
./automations.sh # Deploy on this host → deploy: webfinger
```
Or build + run the self-contained artifact:
```bash
./build.sh
scp deploy.sh root@host:
ssh root@host 'bash deploy.sh'
# non-interactive:
# BASE_DOMAIN=example.com ISSUER_URL=https://auth.example.com \
# REDIRECT_URL=https://example.org ACME_EMAIL=me@example.com SKIP_PROMPTS=1 bash deploy.sh
```
Unattended provisioning: [`cloud-init.yml`](cloud-init.yml).
## Notes
- Standalone: binds 80/443 itself, so don't co-locate with another stack that
wants those ports.
- DNS for `BASE_DOMAIN` must resolve to the host and 80/443 be reachable before
deploy.
deployments/webfinger/build.sh
+34
View File
@@ -0,0 +1,34 @@
#!/usr/bin/env bash
#
# build.sh -- (re)embed docker-compose.yml, Caddyfile, .env.example into
# deploy.sh as a base64-encoded tar.gz payload after __ARCHIVE_BELOW__.
# Idempotent: strips any existing payload first.
set -euo pipefail
DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
SCRIPT="$DIR/deploy.sh"
MARKER="__ARCHIVE_BELOW__"
[[ -f "$SCRIPT" ]] || { echo "deploy.sh not found at $SCRIPT" >&2; exit 1; }
for f in docker-compose.yml Caddyfile .env.example; do
[[ -f "$DIR/$f" ]] || { echo "Missing $DIR/$f" >&2; exit 1; }
done
PAYLOAD=$(tar -czf - -C "$DIR" docker-compose.yml Caddyfile .env.example | base64)
TMP=$(mktemp)
trap 'rm -f "$TMP"' EXIT
sed "/^${MARKER}\$/,\$d" "$SCRIPT" > "$TMP"
{
echo "$MARKER"
echo "$PAYLOAD"
} >> "$TMP"
mv "$TMP" "$SCRIPT"
chmod +x "$SCRIPT"
trap - EXIT
size=$(wc -c < "$SCRIPT")
echo "Built $SCRIPT (${size} bytes)"
deployments/webfinger/cloud-init.yml
+33
View File
@@ -0,0 +1,33 @@
#cloud-config
#
# WebFinger + redirect — unattended deploy on a fresh Alpine host.
#
# Fill in REPO_URL and the values in the runcmd block, then paste this as the
# instance user-data. DNS for BASE_DOMAIN must point at this host and ports
# 80/443 must be reachable before boot, or the LE cert request fails.
packages:
- git
runcmd:
- hostnamectl set-hostname webfinger || true
- |
set -e
REPO_URL=https://git.anomalous.dev/57_Wolve/automations.git
REPO_BRANCH=main
git clone --depth 1 --branch "$REPO_BRANCH" "$REPO_URL" /opt/automations
cd /opt/automations
# Optional: install shared admin SSH keys from globals/ for root before deploy
# . scripts/lib.sh && load_globals \
# && install -d -m700 /root/.ssh && resolve_ssh_keys >> /root/.ssh/authorized_keys
# Optional: harden SSH first (prints a fresh root key — capture it!)
# SKIP_PROMPTS=1 FORCE=1 bash scripts/harden-ssh.sh
BASE_DOMAIN=example.com \
ISSUER_URL=https://auth.example.com \
REDIRECT_URL=https://example.org \
ACME_EMAIL=admin@example.com \
SKIP_PROMPTS=1 \
bash deployments/webfinger/deploy.sh
deployments/webfinger/deploy.sh
+277
View File
@@ -0,0 +1,277 @@
#!/usr/bin/env bash
#
# deploy.sh -- deploy the webfinger stack (a single Caddy container) on
# Alpine. Serves /.well-known/webfinger at $BASE_DOMAIN for OIDC discovery
# and redirects everything else to $REDIRECT_URL.
#
# Single-node: runs as root.
#
# What this does:
# 1. Installs docker + docker-cli-compose if missing.
# 2. Lays down docker-compose.yml, Caddyfile, .env.example in $STACK_DIR.
# 3. Generates .env on first run; existing .env is never overwritten.
# 4. Prompts for required values not preset (BASE_DOMAIN, ISSUER_URL,
# REDIRECT_URL, ACME_EMAIL).
# 5. Opens TCP 80/443 in UFW if active.
# 6. Pulls images, brings the stack up, waits for healthchecks.
#
# Idempotent: re-run to apply config changes / pull new images.
#
# Self-contained: docker-compose.yml, Caddyfile, .env.example are embedded
# as a base64-encoded tar.gz at the bottom of this file. Rebuild with
# build.sh after editing the loose source files.
#
# Usage:
# bash deploy.sh # interactive prompts
# BASE_DOMAIN=example.com ISSUER_URL=https://auth.example.com \
# REDIRECT_URL=https://example.org ACME_EMAIL=me@example.com \
# FORCE=1 bash deploy.sh
# STACK_DIR=/opt/webfinger bash deploy.sh
# SKIP_DOCKER_INSTALL=1 bash deploy.sh
set -euo pipefail
: "${STACK_DIR:=/srv/webfinger}"
: "${SKIP_DOCKER_INSTALL:=0}"
: "${FORCE:=0}"
: "${SKIP_PROMPTS:=0}" # non-interactive: require values via env, no prompts
[[ "$SKIP_PROMPTS" == "1" ]] && FORCE=1
: "${BASE_DOMAIN:=}"
: "${ISSUER_URL:=}"
: "${REDIRECT_URL:=}"
: "${ACME_EMAIL:=}"
log() { printf '\033[1;32m[+]\033[0m %s\n' "$*"; }
warn() { printf '\033[1;33m[!]\033[0m %s\n' "$*" >&2; }
die() { printf '\033[1;31m[x]\033[0m %s\n' "$*" >&2; exit 1; }
[[ $EUID -eq 0 ]] || die "Run as root."
# ---------------------------------------------------------------------------
# OS detection + Docker install (Alpine / Debian / Alma). This deploy.sh is
# self-contained (scp'd standalone), so the OS logic is inlined here instead
# of sourced from scripts/oslib.sh.
# ---------------------------------------------------------------------------
osfam() {
local id="" like=""
if [[ -r /etc/os-release ]]; then
id="$(. /etc/os-release 2>/dev/null && echo "${ID:-}")"
like="$(. /etc/os-release 2>/dev/null && echo "${ID_LIKE:-}")"
fi
case " $id $like " in
*" alpine "*) echo alpine ;;
*" debian "*|*" ubuntu "*) echo debian ;;
*" rhel "*|*" fedora "*|*" centos "*) echo rhel ;;
*) echo "${id:-unknown}" ;;
esac
}
install_docker() {
if command -v docker >/dev/null 2>&1; then
log "Docker already installed: $(docker --version)"
else
log "Installing Docker (OS: $(osfam))..."
case "$(osfam)" in
alpine) apk add -q docker docker-cli-compose openrc ;;
debian|rhel) command -v curl >/dev/null 2>&1 || \
{ command -v apt-get >/dev/null 2>&1 && apt-get install -y -qq curl; } || \
{ command -v dnf >/dev/null 2>&1 && dnf install -y -q curl; }
curl -fsSL https://get.docker.com | sh ;;
*) die "Unsupported OS for auto Docker install. Set SKIP_DOCKER_INSTALL=1 and install Docker yourself." ;;
esac
fi
if command -v rc-update >/dev/null 2>&1; then
rc-update add docker default >/dev/null 2>&1 || true
rc-service docker status >/dev/null 2>&1 || rc-service docker start
elif command -v systemctl >/dev/null 2>&1; then
systemctl enable --now docker >/dev/null 2>&1 || systemctl start docker || true
fi
}
open_web_ports() {
if command -v ufw >/dev/null 2>&1 && ufw status 2>/dev/null | grep -q '^Status: active'; then
log "ufw active -- allowing 80,443/tcp..."
ufw allow 80/tcp >/dev/null; ufw allow 443/tcp >/dev/null
elif command -v firewall-cmd >/dev/null 2>&1 && firewall-cmd --state >/dev/null 2>&1; then
log "firewalld active -- allowing http,https..."
firewall-cmd -q --add-service=http --permanent
firewall-cmd -q --add-service=https --permanent
firewall-cmd -q --reload
fi
}
# ----------------------------------------------------------------------------
# Extract embedded archive
# ----------------------------------------------------------------------------
SCRIPT_DIR=$(mktemp -d -t webfinger-deploy.XXXXXX)
trap 'rm -rf "$SCRIPT_DIR"' EXIT
extract_archive() {
grep -a -A 9999999 '^__ARCHIVE_BELOW__$' "$0" \
| tail -n +2 \
| base64 -d \
| tar -xz -C "$SCRIPT_DIR"
}
if grep -q -a '^__ARCHIVE_BELOW__$' "$0"; then
log "Extracting embedded deployment files..."
extract_archive
else
die "No embedded archive found. Run build.sh to embed deployment files."
fi
for f in docker-compose.yml Caddyfile .env.example; do
[[ -f "$SCRIPT_DIR/$f" ]] || die "Embedded archive missing $f"
done
# ----------------------------------------------------------------------------
# Prompt for required vars
# ----------------------------------------------------------------------------
prompt() {
local varname="$1" message="$2"
local -n ref="$varname"
if [[ -z "${ref:-}" ]]; then
[[ "$SKIP_PROMPTS" == "1" ]] && die "$varname required (set it in the environment; running with SKIP_PROMPTS=1)."
read -r -p "$message: " ref
[[ -n "$ref" ]] || die "$varname required."
fi
}
prompt BASE_DOMAIN "Apex domain to serve from (e.g. example.com)"
prompt ISSUER_URL "OIDC issuer URL (e.g. https://auth.example.com)"
prompt REDIRECT_URL "Where to redirect non-webfinger traffic (e.g. https://example.org)"
prompt ACME_EMAIL "Let's Encrypt email"
# ----------------------------------------------------------------------------
# Docker
# ----------------------------------------------------------------------------
if [[ "$SKIP_DOCKER_INSTALL" != "1" ]]; then
install_docker
fi
# Open 80/443 on whichever host firewall is active.
open_web_ports
# ----------------------------------------------------------------------------
# Stack directory + files
# ----------------------------------------------------------------------------
log "Setting up $STACK_DIR..."
install -d -m 0750 "$STACK_DIR"
install -m 0640 "$SCRIPT_DIR/docker-compose.yml" "$STACK_DIR/docker-compose.yml"
install -m 0640 "$SCRIPT_DIR/Caddyfile" "$STACK_DIR/Caddyfile"
ENV_FILE="$STACK_DIR/.env"
if [[ ! -f "$ENV_FILE" ]]; then
log "Seeding $ENV_FILE..."
install -m 0600 "$SCRIPT_DIR/.env.example" "$ENV_FILE"
sed -i \
-e "s|^BASE_DOMAIN=.*|BASE_DOMAIN=${BASE_DOMAIN}|" \
-e "s|^ISSUER_URL=.*|ISSUER_URL=${ISSUER_URL}|" \
-e "s|^REDIRECT_URL=.*|REDIRECT_URL=${REDIRECT_URL}|" \
-e "s|^ACME_EMAIL=.*|ACME_EMAIL=${ACME_EMAIL}|" \
"$ENV_FILE"
else
log ".env exists; leaving it alone."
fi
# Validate required values are present.
missing=()
for var in BASE_DOMAIN ISSUER_URL REDIRECT_URL ACME_EMAIL; do
grep -E "^${var}=.+$" "$ENV_FILE" >/dev/null || missing+=("$var")
done
(( ${#missing[@]} == 0 )) || die "Missing values in $ENV_FILE: ${missing[*]}"
# ----------------------------------------------------------------------------
# Bring up the stack
# ----------------------------------------------------------------------------
if [[ "$FORCE" != "1" ]]; then
cat <<EOF
About to pull images and start the stack from $STACK_DIR.
Caddy will request a Let's Encrypt cert for ${BASE_DOMAIN}. DNS for that
name must already point at this host, and ports 80/443 must be reachable
from the internet, or the cert request will fail.
Continue? [y/N]
EOF
read -r ans
[[ "${ans,,}" == "y" || "${ans,,}" == "yes" ]] || { warn "Aborted."; exit 0; }
fi
cd "$STACK_DIR"
log "Pulling images..."
docker compose pull
log "Starting stack..."
docker compose up -d --remove-orphans
# ----------------------------------------------------------------------------
# Wait for health
# ----------------------------------------------------------------------------
log "Waiting for services to become healthy (up to 120s)..."
deadline=$(( $(date +%s) + 120 ))
while (( $(date +%s) < deadline )); do
status=$(docker compose ps --format '{{.Service}} {{.Health}}' 2>/dev/null || true)
unhealthy=$(echo "$status" | awk '$2 != "healthy" && $2 != "" {print $1}')
if [[ -z "$unhealthy" && -n "$status" ]]; then
log "All services healthy."
break
fi
sleep 5
done
echo
log "Stack status:"
docker compose ps
echo
cat <<EOF
================================================================
DEPLOYED
WebFinger: https://${BASE_DOMAIN}/.well-known/webfinger
Redirects: https://${BASE_DOMAIN}/ -> ${REDIRECT_URL}
Stack dir: $STACK_DIR
Manage (run from $STACK_DIR):
docker compose logs -f
docker compose pull && docker compose up -d # update
docker compose down # stop, keep volumes
Or just re-run this script -- it's idempotent.
================================================================
EOF
# IMPORTANT: do not put any code below this exit. Everything after the
# __ARCHIVE_BELOW__ marker is the embedded tar.gz payload (base64).
exit 0
__ARCHIVE_BELOW__
H4sIAAAAAAAAA+1YbW/bNhDOZ/+KgxJgLRa9OXHSeciwzMm6DO3aJRm6YRhSWjrbbChSJam4npH/
viNlW6qDol+6bkB1RSPzdO888TkpV9kt6jBTRakMRotC7HxySoiODg93kqMkPR6k/pokqecT9ZPB
4U46SI+P0sFgkCQ7Sdo/Pj7YgeTTh/KQKmOZBtgZHN+8UuIOP4fP/xHtwhzHEy6nqIEqkd1CGIKS
CCOW5wvIlLSMS7ppZ8yCQX2HBuJojkKEt1LNZbzR7+0Ciez9cHp1fnP24vnpxS/waKI0vLg4G0HO
TabuUC9golUB14wLkzGBgBaYiB4DkzlozLnGzBoyhU7YzsgyoDAIVsHe5fnZxeX56Prmt8tnUW+X
pK4s6TFBAQ8pQm5gzGVu4EkSHx4eALcGxWQfjIJcya8s6EoSk/IjYQTDCoSZMhSB88ikIu66DD5f
Jkh1zqQ1tKbnA0ql6XclBRoDC1VBWZEUXI9ewpgJJjNfBi5dlpIcaShQT9G7o1pO+NREvZ4kx8Om
8L3enRJVgWbYA8hc3cOcWdasas1hr+fqz7OWoPsBwAs2JYM1Z285Oj07++Pm+vTpMOyHTJS0f/de
brObN3UEtfUmDiej0T0QdrhKMjRWlSXm/p7PvnYJEELwJBk+SYJmTTUf0v+HnLjKy8DzduGn6+uX
8YEXaaVdy0exb7sJFziM0Waxj7DF1Goj2ypU7P5u3VjVLK6v/ibKO07bUqC0a5etZh1CsLdsre/X
aVxcXf12ful6zos0y41Euy+9TJuxkTodPT+/OSfTtUyzXEnMkAk7y2aY3a7Ds7QZQ/gzGD0/C/Yh
mE/Rumv49kXorjNry2Ec03EZJfQvHfaT9JtVxnHw18oIl5bahokhHCRmbZgXqCqyPVhzNFrNaS/g
YMXwbXBTouYqH0JKqv/1WdXRp6fNo/Uv+vgY/h+mxw3+p8RP03TQ7/D/c9AubDoAHFg7oGomgq83
kFyjYg264QcGAAi/g3xB0MIz+PnyzEFJqaTBfcBsphySO+tvKzpSCFB2a7BRlc6QABhem2r8hjy9
9qNASfLWqWxND27CaE7gyEezPSpQGAdJ+mBigEcl+XMISoIlszNKzwWzeFynda1qvPdDDszncwLI
GaPE4PWyPdXcv4YxCjUnB6S1dW/fKUZb8gT5yxqBCpp7YLnXOvt7NSrSIDN1cY1O622gg98t55zm
DhoxxpWWbv3sHDSzCIIXfA3FNLlkBd5kDBweGAIEtybg9hbDu6QfsZJHAmkakplelDZSehrX+6r0
ondP4b0XMKyilZnKEf42Nofp37zsee73zX77In5gGPSIRjtJfdXSWK7AxaNdTowRjSSEx+H1okQI
WFkKnjHLlYzf6PzrN0bJ4IEKI4wMnaJWAoKyGpPOPhTsXUhj0MnBUZI0OhaLUlDFzIZTd2VOG0es
YNV0AUHychld4tvItdWvri2ipzSbBusWDe7vg32nIbi8NST/p7e4TojkhDOygmRVouR5JNHGpsTM
OFCW5CdOoyTmxpB9b82rzjROfAB7D2YL1x5/9ajj+kmyYrQru2xlRftJnfXe5LGsNL8HgvCCSSry
ewbqWjYGrgj+M9oHzaRxc154hRmp2wUEm8qmA1fc5FsaKTJR5XhVjc8UtbQ0TcF/D9tbGr4o3WYa
n6dURvLJpJG9xAlqTa+fLxVt4QIaCkwdjtKcmjicz1CGmVY0j9acxgbFSY+rbucm1LSVGD0+bkin
JqZfGy49ZAWdJbQtRglcad9/eSNORENxhO8YPSX/GvR9BP+T46N+g/99j/+D46TD/89BhP+qXDis
dJ3goZdmAUFPeETvzO7bEKy+DUHJs1tTi1UlsMrSo2/ptBZiQQC3C6clviMddx44ezWM+rf9R5KW
dGgX+DhafVhw6qE/CA05dbhGr8426rVg6GTVlhEF4Oz7SaA+O8HBucPJVzj+sUaWZkR4RO/lhE0u
eBvyHHIshVq4tz4C+uaEPdmgZWVn0ZavVzPUWGdBFWFyAVLJsPGmkSYHYzcjBWWxNVG0D+KNKxdZ
uB6pQmo8eqHbdv0M7VcGzmukJkdTTmehh8R6fIh6zfRwwvKCy++3LFy4zwFA8B/1Np8CTtZfAr68
I66jjjrqqKOOOuqoo4466qijjjrqqKOOvlj6BwxDOSQAKAAA
deployments/webfinger/docker-compose.yml
+38
View File
@@ -0,0 +1,38 @@
# webfinger stack -- one Caddy container that serves /.well-known/webfinger
# at $BASE_DOMAIN (for OIDC discovery from Tailscale et al.) and redirects
# everything else to $REDIRECT_URL.
#
# Standalone: this binds 80/443 itself, so don't run it on the same host as
# another stack that also wants those ports unless you put a TCP balancer
# in front or merge the configs.
name: webfinger
volumes:
caddy-data:
caddy-config:
services:
caddy:
image: caddy:${CADDY_TAG:-2-alpine}
container_name: caddy-webfinger
restart: unless-stopped
ports:
- "80:80"
- "443:443"
- "443:443/udp" # HTTP/3
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
- caddy-data:/data
- caddy-config:/config
environment:
BASE_DOMAIN: "${BASE_DOMAIN}"
ISSUER_URL: "${ISSUER_URL}"
REDIRECT_URL: "${REDIRECT_URL}"
ACME_EMAIL: "${ACME_EMAIL}"
healthcheck:
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:2019/config/"]
interval: 30s
timeout: 5s
retries: 3
start_period: 10s
globals/Network Domain Name Schema.md
+71
View File
@@ -0,0 +1,71 @@
## **1. General Information**
1.01 This section contains regulations for identification of network systems, devices, and applications or services.
1.02 Whenever this section is reissued, the reason(s) for reissue will be listed in this paragraph.
## **2. Identification & Configuration**
2.01 Provides a common method and standard for universal identification of systems, devices, and software or services with a Domain Name for DNS.
## **3. Network Domain Name Schema**
3.01 Table of acronymized abbreviations for use in DNS.
### Names for Applications & Services:
- app – Application Server (non-web)
- dbs – Database Server
- ftp – SFTP server
- mta – Mail Server / Mail Transfer Agent
- dns – Name Server
- rsv - DNS Rsolver
- cfg – Configuration Management (puppet/ansible/etc.)
- mon – Monitoring Server (nagios, sensu, etc.)
- zbx - Zabbix Proxy / Server
- ssh – SSH Jump/Bastion Host
- sto – Storage Server / File Share (iSCSI/NFS/SMB/Ceph/GlusterFS)
- git – Version Control Software Server (Git/SVN/CVS/etc.)
- vmm – Virtual Machine Manager
- web – Web Server k8s - Kubernetes Controller/Worker Node
- dsv - Directory Domain Services (LDAP / AD DS)
- rds - Remote Desktop Services - (RemoteApp/Sessions)
- vpn – VPN Gateway
- wgd – WireGuard
- prx – Proxy/Load Balancer (software)
- zta - Zero Trust Access
- dot - Traffic Shaper
### Names for Systems and Devices:
- con – Console/Terminal Server
- fwl – Firewall
- lbl – Load Balancer (physical)
- gtr - L3 Global Transit Router
- ctr - L3 Core Router
- rtr – L3 Router
- swt – L2 Switch
- dcr – Data Center Router
- dcs – Data Center Switch
- pdu – Power Distribution Unit
- ups – Uninterruptible Power Supply
3.02 Geospatial positioning in domain names for regional datacenter services and systems is to follow UN/LOCODE naming conventions as follows:
Example: [us-evi-1.example.net](http://us-evi-1.example.net/) indicates the names
left most of ‘us-evi-1' are in Elk Grove Village Illinois within the United States of America. The trailing number “1” indicates the specific facility located in this municipality for cases where multiple are within the same regional area.
Application of this USP can be demonstrated as followed:
Example: [dns-1.us-evi-1.datacenter.gg](http://dns-1.us-evi-1.datacenter.gg/)
3.03 Anycast services do not follow 3.02, instead indicate region using anycast in placement of 3.02 region code. Section 3.02 may still be used for secondary IPs that are bound to the location, even to the same service.
Example: [dns-1.anycast.datacenter.gg](http://dns-1.anycast.datacenter.gg/)
## **4. UN/LOCODE Code List by Country and Territory**
4.01 Data Located in Online Resource
URL: [https://unece.org/trade/cefact/unlocode-code-list-country-and-territory](https://unece.org/trade/cefact/unlocode-code-list-country-and-territory)
globals/README.md
+34
View File
@@ -0,0 +1,34 @@
# globals/
Shared, cross-deployment assets. Everything here is referenced by the launcher
([`../automations.sh`](../automations.sh)), the helper library
([`../scripts/lib.sh`](../scripts/lib.sh)), and the per-deployment cloud-init
templates — so a value lives in exactly one place.
| File | Purpose |
|------|---------|
| `globals.env.example` | Template of shared defaults. Copy to `globals.env` (git-ignored) and edit. Sourced by the launcher and `lib.sh`. |
| `age-pubkey.txt` | Age **public** key recipient(s) for encrypted backups. Used by any stack that backs up (e.g. simplex). Public key — safe to commit. |
| `authorized_keys` | Static admin SSH **public** keys. The fallback source when `SSH_KEYS_URL` is unset. |
| `motd.txt` | MOTD **template** (token-based) rendered to `/etc/motd` by [`../scripts/setup-host.sh`](../scripts/setup-host.sh). You edit the content; the renderer draws the borders and computes all spacing. |
| `Network Domain Name Schema.md` | Reference: the DNS naming convention (service/device acronyms, UN/LOCODE geo-coding) used for host and service names. `setup-host.sh` derives the FQDN and Node ID from it. |
## SSH key resolution (URL-preferred)
`resolve_ssh_keys()` in `../scripts/lib.sh` decides where admin SSH keys come
from:
1. If `SSH_KEYS_URL` is set in `globals.env`, it is fetched live with
`curl -fsSL` (always current — best when you rotate keys often). This can be
a GitHub keys endpoint (`https://github.com/<user>.keys`) or any raw
`authorized_keys` URL.
2. Otherwise, `globals/authorized_keys` is read directly (self-contained,
versioned — edit the repo to rotate).
## Rules
- **Only public material lives here.** Never commit private keys (SSH private
keys, age identities). `globals.env` and common private-key filename patterns
are git-ignored (see [`../.gitignore`](../.gitignore)).
- Store the age **private** key(s) and SSH **private** keys somewhere safe,
outside this repository.
globals/age-pubkey.txt
+10
View File
@@ -0,0 +1,10 @@
# Age public keys for encrypted backups (shared across all deployments)
#
# Generate a keypair with: age-keygen -o backup-private-key.txt
# Then copy the "Public key: age1..." line here.
#
# You can list multiple recipients - backups are encrypted to all of them.
# Store the corresponding PRIVATE keys securely, NEVER in this git repository.
# Primary backup key
age1tudl7dksctvpkfnu46tq8tmkt36k6xg0rmudzaayqakrtper2fpsk0dnhp
globals/authorized_keys
+11
View File
@@ -0,0 +1,11 @@
# Shared admin SSH public keys.
#
# These are the public keys granted administrative SSH access across hosts.
# Used as the fallback source when SSH_KEYS_URL is not set in globals.env
# (see resolve_ssh_keys in ../scripts/lib.sh).
#
# One key per line. Lines starting with '#' and blank lines are ignored.
# Only public keys belong here. NEVER commit private keys.
#
# Example:
# ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA... william@laptop
globals/globals.env.example
+26
View File
@@ -0,0 +1,26 @@
# Shared defaults for all deployments and scripts.
#
# Copy to globals.env and edit. The launcher (../automations.sh) and the
# helpers in ../scripts/lib.sh source globals.env if present, otherwise they
# fall back to the values shown here. The real globals.env is git-ignored.
# ─── Repository ─────────────────────────────────────────────────────────────
# Where the launcher and cloud-init clone this repo from. Used by the
# one-liner bootstrap and by every deployment's cloud-init.yml.
REPO_URL=https://git.anomalous.dev/57_Wolve/automations.git
REPO_BRANCH=main
# ─── Shared deployment defaults ─────────────────────────────────────────────
# Pre-filled into per-deployment prompts so you only type them once.
ACME_EMAIL=admin@example.com
SSH_PORT=22
# ─── SSH authorized keys ────────────────────────────────────────────────────
# resolve_ssh_keys() prefers this URL when set (fetched live, always current),
# e.g. a GitHub keys endpoint (https://github.com/<user>.keys) or a raw
# authorized_keys file. When empty, it falls back to globals/authorized_keys.
SSH_KEYS_URL=
# ─── Backups ────────────────────────────────────────────────────────────────
# Age public-key recipient file for encrypted backups (relative to repo root).
AGE_PUBKEY_FILE=globals/age-pubkey.txt
globals/motd.txt
+30
View File
@@ -0,0 +1,30 @@
# MOTD template, rendered by ../scripts/setup-host.sh.
#
# You edit CONTENT here; the renderer draws the borders and handles all
# spacing, so the box always stays aligned no matter how long a value is.
#
# Lines beginning with '#' are comments and ignored. Tokens:
# @TOP@ top border
# @BOT@ bottom border
# @HR@ horizontal divider
# @BLANK@ empty content line
# @C@ text centered line (ASCII text)
# @BOX@ text centered line wrapped in a mini double-line box
# @F@ Label|value field line: right-aligned label, then value
#
# In @F@ values these placeholders are substituted at render time:
# {{HOSTNAME}} full FQDN (e.g. sto-1.srvno.de)
# {{NODE_ID}} short name, uppercased (e.g. STO-1)
# {{DATACENTER}} the DATACENTER value
#
# Box width is set by MOTD_WIDTH in setup-host.sh (default 60).
@TOP@
@BOX@ !! DANGER - CRITICAL INFRASTRUCTURE !!
@C@ YOU are connected to a CRITICAL SYSTEM
@C@ CHECK YOUR WORK BEFORE RELOAD / RESTART OF THE SERVICE
@HR@
@F@ Data Center|{{DATACENTER}}
@F@ Hostname|{{HOSTNAME}}
@F@ Node ID|{{NODE_ID}}
@BOT@
scripts/auto-update.conf.example
+19
View File
@@ -0,0 +1,19 @@
# auto-update.conf -- defaults for the daily auto-update job
# (scripts/auto-update.sh). Installed at /etc/auto-update.conf by
# `auto-update.sh install` (the harden scripts do this). Environment variables
# still override these at runtime.
# When a reboot is needed after an upgrade:
# 0 never reboot (just flag / notify)
# 1 always reboot
# idle reboot only when NO SSH connections are active -- safe for a bastion,
# since it won't drop a live admin session or a ProxyJump tunnel. A
# deferred reboot is retried on the next daily run.
AUTO_REBOOT="idle"
# (Alpine) also jump to a newer STABLE branch (e.g. 3.21 -> 3.22) when posted.
# Off by default; when off a new branch is only reported via ntfy.
ALLOW_RELEASE_UPGRADE="0"
# Send an ntfy summary after each run (reuses /etc/ssh-notify.conf creds).
NOTIFY="1"
scripts/auto-update.sh
+310
View File
@@ -0,0 +1,310 @@
#!/usr/bin/env bash
#
# auto-update.sh -- unattended package updates + new-OS-release check, for
# Alpine, Debian, and Alma. Designed for SSH-only bastion hosts, where the
# blast radius of a routine upgrade is tiny.
#
# Policy:
# - Apply all in-branch package upgrades automatically (apk/apt/dnf).
# - Do NOT auto-jump to a new Alpine branch (e.g. 3.21 -> 3.22): that
# rewrites the repo branch and is a human decision. Instead, NOTIFY that
# one is available.
# - Detect when a reboot is needed (kernel/libc/openssl). Reboot only if
# AUTO_REBOOT=1, otherwise just report it.
# - Send an ntfy summary, reusing /etc/ssh-notify.conf (same creds as the
# login notifier).
#
# Subcommands:
# run (default) do the update pass and notify
# install schedule this script to run daily (oslib install_daily_job)
# uninstall remove the daily schedule
#
# Env (also settable in /etc/auto-update.conf, which the daily run reads;
# environment overrides the file):
# AUTO_REBOOT=0 when a reboot is needed:
# 0 = never (just flag/notify)
# 1 = always
# idle = only when NO SSH connections are active
# (safe for a bastion -- won't cut a live
# admin session or a ProxyJump tunnel; a
# deferred reboot is retried each day)
# ALLOW_RELEASE_UPGRADE=0 (Alpine) also jump to a newer STABLE branch when
# one is posted (e.g. 3.21 -> 3.22). Off by default;
# when off, a new branch is only reported.
# NOTIFY=1 send an ntfy summary (0 to disable)
# SSH_NOTIFY_CONF=/etc/ssh-notify.conf
# DRY_RUN=0 print what would happen; touch nothing (for testing)
# LOG=/var/log/auto-update.log
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# Locate oslib.sh. When this script is installed standalone (e.g. to
# /usr/local/sbin by install_daily_job), oslib is co-installed alongside it.
_oslib_loaded=0
for _cand in "$SCRIPT_DIR/oslib.sh" /usr/local/sbin/oslib.sh /opt/automations/scripts/oslib.sh; do
if [[ -f "$_cand" ]]; then . "$_cand"; _oslib_loaded=1; break; fi
done
[[ "$_oslib_loaded" == 1 ]] || { echo "[x] oslib.sh not found next to auto-update.sh" >&2; exit 1; }
SELF="$SCRIPT_DIR/$(basename "${BASH_SOURCE[0]}")" # resolved path to this file
# Load defaults from the conf for any var not already set in the environment
# (precedence: environment > conf > built-in). This is how the daily cron run
# picks up AUTO_REBOOT etc. without baking them into the schedule.
: "${AUTO_UPDATE_CONF:=/etc/auto-update.conf}"
if [[ -r "$AUTO_UPDATE_CONF" ]]; then
while IFS= read -r _line; do
[[ "$_line" =~ ^[[:space:]]*# || -z "${_line//[[:space:]]/}" ]] && continue
_k="${_line%%=*}"; _v="${_line#*=}"; _k="${_k//[[:space:]]/}"
[[ "$_k" =~ ^[A-Za-z_][A-Za-z0-9_]*$ ]] || continue
[[ -n "${!_k:-}" ]] && continue # env already set -> wins
_v="${_v%\"}"; _v="${_v#\"}" # strip surrounding quotes
printf -v "$_k" '%s' "$_v"
done < "$AUTO_UPDATE_CONF"
fi
: "${AUTO_REBOOT:=0}"
: "${ALLOW_RELEASE_UPGRADE:=0}"
: "${NOTIFY:=1}"
: "${SSH_NOTIFY_CONF:=/etc/ssh-notify.conf}"
: "${DRY_RUN:=0}"
: "${LOG:=/var/log/auto-update.log}"
# Pending-reboot marker. In /run (tmpfs), so it is cleared by a real reboot;
# its presence on a later run means a deferred reboot is still pending.
: "${REBOOT_FLAG:=/run/automations-reboot}"
log() { _log "$@"; }
warn() { _warn "$@"; }
die() { _die "$@"; }
run() { if [[ "$DRY_RUN" == "1" ]]; then echo "DRY: $*"; else eval "$@"; fi; }
# ============================================================================
# Update pass
# ============================================================================
UPGRADED=0
REBOOT=0
NEW_RELEASE="" # a newer stable branch exists (e.g. 3.22)
UPGRADED_TO="" # we actually upgraded to this branch (ALLOW_RELEASE_UPGRADE)
CUR_BRANCH=""
apply_updates() {
case "$OS_FAMILY" in
alpine)
run "apk update"
local out
if [[ "$DRY_RUN" == "1" ]]; then out="$(apk version -l '<' 2>/dev/null || true)"; else out="$(apk upgrade --no-self-upgrade 2>&1 | tee -a "$LOG")"; fi
UPGRADED=$(printf '%s\n' "$out" | grep -cE '^\([0-9]+/[0-9]+\) (Upgrading|Installing)' || true)
# A kernel / libc / crypto / busybox change wants a reboot.
printf '%s\n' "$out" | grep -qiE 'linux-(lts|virt|edge|rpi)|(^|[^a-z])musl|openssl|libcrypto|busybox' && REBOOT=1
;;
debian)
run "apt-get update -qq"
local before after
before="$(dpkg -l 2>/dev/null | grep -c '^ii' || echo 0)"
run "DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold upgrade"
run "DEBIAN_FRONTEND=noninteractive apt-get -y autoremove"
after="$(dpkg -l 2>/dev/null | grep -c '^ii' || echo 0)"
UPGRADED=$(( before > after ? 0 : after - before )) # rough; reboot flag is the signal that matters
[[ -f /var/run/reboot-required ]] && REBOOT=1
;;
rhel)
run "dnf -y upgrade --refresh"
command -v needs-restarting >/dev/null 2>&1 || run "dnf -y install dnf-utils"
if command -v needs-restarting >/dev/null 2>&1; then
needs-restarting -r >/dev/null 2>&1 || REBOOT=1
fi
;;
esac
return 0 # never let a non-matching grep above fail the function (set -e)
}
# ============================================================================
# New-release check (Alpine: is a newer stable BRANCH posted?)
# ============================================================================
check_new_release() {
[[ "$OS_FAMILY" == alpine ]] || return 0
command -v curl >/dev/null 2>&1 || return 0
local cur cur_branch repo base latest
cur="$(cat /etc/alpine-release 2>/dev/null || echo 0.0.0)"
cur_branch="$(printf '%s' "$cur" | cut -d. -f1,2)"
CUR_BRANCH="$cur_branch"
repo="$(grep -m1 -E '^https?://' /etc/apk/repositories 2>/dev/null || true)"
[[ -n "$repo" ]] || return 0
base="$(printf '%s' "$repo" | sed -E 's#(/alpine/).*#\1#')"
# Newest vX.Y branch advertised on the mirror.
latest="$(curl -fsSL "$base" 2>/dev/null | grep -oE 'v[0-9]+\.[0-9]+' | tr -d v | sort -uV | tail -1 || true)"
[[ -n "$latest" ]] || return 0
# If the newest branch sorts strictly above the current branch, flag it.
if [[ "$latest" != "$cur_branch" ]] && \
[[ "$(printf '%s\n%s\n' "$cur_branch" "$latest" | sort -V | tail -1)" == "$latest" ]]; then
NEW_RELEASE="$latest"
fi
return 0
}
# Jump to the newest STABLE Alpine branch (ALLOW_RELEASE_UPGRADE=1). Only the
# vX.Y stable branches are ever considered -- `edge` is never a target. Repoints
# /etc/apk/repositories from the current vX.Y to the new one (leaving any edge
# or non-versioned lines alone) and runs `apk upgrade --available`.
do_release_upgrade() {
[[ "$OS_FAMILY" == alpine ]] || { warn "Release upgrade is Alpine-only; skipping."; return 0; }
[[ -n "$NEW_RELEASE" ]] || return 0
[[ "$NEW_RELEASE" =~ ^[0-9]+\.[0-9]+$ ]] || { warn "Refusing non-stable release target '$NEW_RELEASE'."; return 0; }
log "Upgrading Alpine ${CUR_BRANCH} -> ${NEW_RELEASE} (ALLOW_RELEASE_UPGRADE=1)..."
if [[ "$DRY_RUN" == "1" ]]; then
echo "DRY: sed repositories /v${CUR_BRANCH}/ -> /v${NEW_RELEASE}/ ; apk update ; apk upgrade --available"
UPGRADED_TO="$NEW_RELEASE"; REBOOT=1; return 0
fi
cp -a /etc/apk/repositories "/etc/apk/repositories.bak.$(date -u +%Y%m%d%H%M%S)"
# Only touch versioned (vX.Y) lines; edge / non-versioned lines are left as-is.
sed -i -E "s#/v[0-9]+\.[0-9]+/#/v${NEW_RELEASE}/#g" /etc/apk/repositories
apk update
apk upgrade --available 2>&1 | tee -a "$LOG" || true
UPGRADED_TO="$NEW_RELEASE"
REBOOT=1 # a branch jump replaces kernel/musl/openssl -- always reboot
log "Now on Alpine $(cat /etc/alpine-release 2>/dev/null || echo '?')."
return 0
}
# ============================================================================
# Notify (reuse the login-notifier's ntfy config)
# ============================================================================
send_notice() {
[[ "$NOTIFY" == "1" ]] || return 0
[[ -r "$SSH_NOTIFY_CONF" ]] || return 0
# shellcheck disable=SC1090
. "$SSH_NOTIFY_CONF"
[[ -n "${NTFY_URL:-}" ]] || return 0
command -v curl >/dev/null 2>&1 || return 0
local host prio tags body
host="$(hostname -f 2>/dev/null || hostname 2>/dev/null || echo unknown)"
prio="min"
[[ "$REBOOT" == "1" || -n "$NEW_RELEASE" ]] && prio="default"
[[ -n "$UPGRADED_TO" ]] && prio="high"
body="Auto-update on ${host} (${OS_ID})
packages upgraded: ${UPGRADED}"
[[ "$REBOOT" == "1" ]] && body="${body}
reboot needed: yes$( [[ "$AUTO_REBOOT" == "1" ]] && echo ' (rebooting)' )"
if [[ -n "$UPGRADED_TO" ]]; then
body="${body}
UPGRADED Alpine release: ${CUR_BRANCH} -> ${UPGRADED_TO}"
elif [[ -n "$NEW_RELEASE" ]]; then
body="${body}
NEW Alpine release available: ${NEW_RELEASE} (current branch ${CUR_BRANCH})"
fi
set -- -fsS -m 5 -H "X-Title: Host Update" -H "X-Priority: ${prio}"
[[ -n "${NTFY_TOKEN:-}" ]] && set -- "$@" -H "Authorization: Bearer ${NTFY_TOKEN}"
[[ -n "${NTFY_EMAIL:-}" ]] && set -- "$@" -H "X-Email: ${NTFY_EMAIL}"
local t="update"; [[ -n "${NTFY_REGION:-}" ]] && t="${t},${NTFY_REGION}"
[[ -n "$NEW_RELEASE" && -z "$UPGRADED_TO" ]] && t="${t},new_release"
[[ -n "$UPGRADED_TO" ]] && t="${t},release_upgraded"
set -- "$@" -H "X-Tags: ${t}"
if [[ "$DRY_RUN" == "1" ]]; then
echo "DRY: curl $* -d <body> $NTFY_URL"
else
curl "$@" -d "$body" "$NTFY_URL" >/dev/null 2>&1 || true
fi
return 0
}
# Are there any active inbound SSH connections? Counts established TCP
# connections on the sshd port -- this catches ProxyJump tunnels too, which
# never open a login session (so `who` would miss them).
ssh_sessions_active() {
local port n
port="$(awk '/^Port /{print $2; exit}' /etc/ssh/sshd_config 2>/dev/null || true)"
port="${port:-22}"
if command -v ss >/dev/null 2>&1; then
n="$(ss -Htn state established "sport = :$port" 2>/dev/null | grep -c . || true)"
elif command -v netstat >/dev/null 2>&1; then
n="$(netstat -tn 2>/dev/null | awk -v p=":$port" '$NF=="ESTABLISHED" && $4 ~ p"$"' | grep -c . || true)"
else
# last resort: sshd per-connection worker processes
n="$(ps ax 2>/dev/null | grep -E '[s]shd:.*@|[s]shd: ' | grep -c . || true)"
fi
[ "${n:-0}" -gt 0 ]
}
# ============================================================================
# Subcommands
# ============================================================================
do_run() {
[[ $EUID -eq 0 ]] || die "Run as root."
os_detect
# A reboot pending from a previous (deferred) run -- the flag is in /run,
# wiped on a real reboot, so its presence means "still pending".
[[ -f "$REBOOT_FLAG" ]] && REBOOT=1
[[ "$DRY_RUN" == "1" ]] || { install -d -m 0755 "$(dirname "$LOG")" 2>/dev/null || true; echo "=== auto-update $(date -u +%FT%TZ) ===" >> "$LOG"; }
log "Applying updates (${OS_ID})..."
apply_updates
check_new_release
if [[ -n "$NEW_RELEASE" && "$ALLOW_RELEASE_UPGRADE" == "1" ]]; then
do_release_upgrade
fi
# Record the pending reboot so a deferral survives to the next daily run.
if [[ "$REBOOT" == "1" && "$DRY_RUN" != "1" ]]; then : > "$REBOOT_FLAG" 2>/dev/null || true; fi
log "Upgraded: ${UPGRADED} | reboot: ${REBOOT} | new release: ${NEW_RELEASE:-none}${UPGRADED_TO:+ | upgraded to: $UPGRADED_TO}"
send_notice
if [[ "$REBOOT" == "1" ]]; then
case "$AUTO_REBOOT" in
1)
log "Rebooting (AUTO_REBOOT=1)..."; run "reboot" ;;
idle)
if ssh_sessions_active; then
warn "Reboot needed, but SSH connections are active; deferring until idle (retried daily)."
else
log "No active SSH connections; rebooting (AUTO_REBOOT=idle)..."; run "reboot"
fi ;;
*)
warn "A reboot is recommended (kernel/libc/crypto updated). Set AUTO_REBOOT=1 or idle to automate." ;;
esac
fi
}
# Write /etc/auto-update.conf so the scheduled run inherits these defaults.
write_autoupdate_conf() {
local f="$AUTO_UPDATE_CONF"
if [[ -f "$f" && "${AU_FORCE_CONF:-0}" != "1" ]]; then
log "$f exists; leaving it (set AU_FORCE_CONF=1 to overwrite)."
return 0
fi
cat > "$f" <<CONF
# Defaults for the daily auto-update job (scripts/auto-update.sh).
# Environment variables still override these at runtime.
AUTO_REBOOT="${AUTO_REBOOT}" # 0 | 1 | idle
ALLOW_RELEASE_UPGRADE="${ALLOW_RELEASE_UPGRADE}" # Alpine stable-branch jump
NOTIFY="${NOTIFY}"
CONF
chmod 644 "$f"
log "Wrote $f"
}
do_install() {
[[ $EUID -eq 0 ]] || die "Run as root."
os_detect
write_autoupdate_conf
install_daily_job auto-update "$SELF" run
log "Scheduled daily auto-update (AUTO_REBOOT=${AUTO_REBOOT})."
}
do_uninstall() {
[[ $EUID -eq 0 ]] || die "Run as root."
os_detect
remove_daily_job auto-update
log "Removed daily auto-update."
}
case "${1:-run}" in
run) do_run ;;
install) do_install ;;
uninstall) do_uninstall ;;
*) die "Usage: auto-update.sh [run|install|uninstall]" ;;
esac
scripts/harden-jumphost.sh
+121 -107
View File
@@ -2,34 +2,24 @@
#
# harden-jumphost.sh
#
# Hardens an Alpine box for use as an SSH jump host (bastion). Run on a fresh
# box. Layered on the same PQ-hybrid posture as harden-ssh.sh, plus jump-host
# specifics.
# Hardens a box for use as an SSH jump host (bastion) on Alpine, Debian, or
# Alma Linux. Layered on the same PQ-hybrid posture as harden-ssh.sh, plus
# jump-host specifics. All distro differences go through scripts/oslib.sh.
#
# Two groups, two privilege levels:
# ssh-admins -- full TTY shell on the jump host (for maintenance only).
# No forwarding. This is for fixing the box, not for
# reaching anywhere else.
# ssh-admins -- full TTY shell on the jump host (maintenance only). No
# forwarding. For fixing the box, not reaching elsewhere.
# ssh-jumpers -- ProxyJump ONLY. No TTY, no shell, no SFTP, no agent
# forwarding. Only direct-tcpip to whitelisted targets.
# These users cannot get a prompt on the jump host even if
# their key works.
#
# How the restriction works:
# - Global default: DisableForwarding yes, PermitTTY no, ForceCommand
# /sbin/nologin. So a user with no group membership cannot do anything.
# - Match Group ssh-admins: re-enables PermitTTY and clears ForceCommand
# so admins get a normal shell. Forwarding stays off.
# - Match Group ssh-jumpers: enables AllowTcpForwarding + PermitOpen
# whitelist. Keeps PermitTTY no and ForceCommand /sbin/nologin so any
# attempt at an interactive session fails -- but ProxyJump (direct-tcpip)
# still works because it doesn't open a session channel.
#
# A note on ProxyJump and ForceCommand:
# `ssh -J jumphost target` opens a direct-tcpip channel on the jump host;
# it is NOT a session/exec request, so ForceCommand never fires. That's
# why this pattern works: jumpers literally cannot run anything on the
# jump host, but their tunnels go through.
# <nologin>. A user in neither group can do nothing.
# - Match Group ssh-admins: re-enables PermitTTY, clears ForceCommand.
# - Match Group ssh-jumpers: enables AllowTcpForwarding + a PermitOpen
# whitelist, but keeps PermitTTY no + ForceCommand <nologin>. ProxyJump
# (direct-tcpip) works because it never opens a session channel, so
# ForceCommand never fires.
#
# Usage:
# bash harden-jumphost.sh
@@ -40,56 +30,57 @@
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=scripts/oslib.sh
. "$SCRIPT_DIR/oslib.sh"
# ============================================================================
# CONFIG
# ============================================================================
: "${SSH_PORT:=22}"
: "${ALLOWED_IP:=}"
: "${FORCE:=0}"
# Space-separated host:port list jumpers can reach via ProxyJump.
# Empty means jumpers can ProxyJump nowhere (deny-all). Set this to your
# internal targets, e.g. "10.0.0.5:22 10.0.0.6:22".
# Space-separated host:port list jumpers can reach via ProxyJump. Empty means
# deny-all. e.g. "10.0.0.5:22 10.0.0.6:22".
: "${JUMP_TARGETS:=}"
: "${KEY_COMMENT:=root@$(hostname)-$(date +%Y%m%d)}"
log() { printf '\033[1;32m[+]\033[0m %s\n' "$*"; }
warn() { printf '\033[1;33m[!]\033[0m %s\n' "$*" >&2; }
die() { printf '\033[1;31m[x]\033[0m %s\n' "$*" >&2; exit 1; }
log() { _log "$@"; }
warn() { _warn "$@"; }
die() { _die "$@"; }
[[ $EUID -eq 0 ]] || die "Run as root."
[[ -f /etc/alpine-release ]] || die "This script targets Alpine Linux."
os_detect
log "Detected OS: ${OS_ID} (family ${OS_FAMILY}, init ${INIT_SYSTEM})"
# ----------------------------------------------------------------------------
# 1. Packages
# ----------------------------------------------------------------------------
log "Installing openssh + PAM + sshguard + iptables + gum..."
if apk info -e openssh-server >/dev/null 2>&1 && \
! apk info -e openssh-server-pam >/dev/null 2>&1; then
apk del -q openssh-server || true
fi
apk add -q openssh openssh-server-pam linux-pam sshguard iptables ip6tables openrc gum shadow
log "Installing OpenSSH + sshguard + iptables..."
install_openssh
install_bruteforce_protection
ensure_gum || warn "gum not installed; sshuser will use its CLI mode."
# Install sshuser tool alongside this script if present.
SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
SFTP_PATH="$(sftp_server_path)"
NOLOGIN="$(nologin_path)" # /sbin/nologin (Alpine/Alma) or /usr/sbin/nologin (Debian)
# Install the sshuser tool alongside this script if present.
if [[ -f "$SCRIPT_DIR/sshuser.sh" ]]; then
install -m 0755 "$SCRIPT_DIR/sshuser.sh" /usr/local/bin/sshuser
log "Installed /usr/local/bin/sshuser"
fi
# ----------------------------------------------------------------------------
# 2. PQ KEX detection (same as harden-ssh.sh)
# 2. PQ KEX detection
# ----------------------------------------------------------------------------
log "Checking OpenSSH version supports PQ KEX..."
SSH_VER=$(ssh -V 2>&1 | grep -oE 'OpenSSH_[0-9]+\.[0-9]+' | head -1 | sed 's/OpenSSH_//')
SSH_MAJOR=${SSH_VER%%.*}
SSH_MINOR=${SSH_VER##*.}
HAS_MLKEM=0
HAS_SNTRUP=0
HAS_MLKEM=0; HAS_SNTRUP=0
[[ $SSH_MAJOR -gt 9 || ( $SSH_MAJOR -eq 9 && $SSH_MINOR -ge 0 ) ]] && HAS_SNTRUP=1
[[ $SSH_MAJOR -gt 9 || ( $SSH_MAJOR -eq 9 && $SSH_MINOR -ge 9 ) ]] && HAS_MLKEM=1
[[ $HAS_SNTRUP -eq 1 || $HAS_MLKEM -eq 1 ]] \
|| die "OpenSSH ${SSH_VER} has no PQ KEX. Need >= 9.0."
[[ $HAS_SNTRUP -eq 1 || $HAS_MLKEM -eq 1 ]] || die "OpenSSH ${SSH_VER} has no PQ KEX. Need >= 9.0."
log "OpenSSH ${SSH_VER}: ML-KEM=${HAS_MLKEM} sntrup761=${HAS_SNTRUP}"
KEX_LIST=""
@@ -107,25 +98,16 @@ if [[ ! -f /etc/ssh/ssh_host_ed25519_key ]]; then
fi
chmod 600 /etc/ssh/ssh_host_ed25519_key
chmod 644 /etc/ssh/ssh_host_ed25519_key.pub
log "Host key fingerprint:"
ssh-keygen -l -f /etc/ssh/ssh_host_ed25519_key.pub | sed 's/^/ /'
# Stop Alpine's sshd init from regenerating RSA/ECDSA keys.
if [[ -f /etc/conf.d/sshd ]]; then
if ! grep -q '^sshd_disable_keygen=' /etc/conf.d/sshd; then
echo 'sshd_disable_keygen="yes"' >> /etc/conf.d/sshd
else
sed -i 's/^sshd_disable_keygen=.*/sshd_disable_keygen="yes"/' /etc/conf.d/sshd
fi
fi
sshd_disable_keygen # Alpine-only; no-op on systemd
# ----------------------------------------------------------------------------
# 4. Groups -- create if missing
# 4. Groups
# ----------------------------------------------------------------------------
log "Ensuring groups ssh-admins and ssh-jumpers exist..."
getent group ssh-admins >/dev/null || addgroup -S ssh-admins
getent group ssh-jumpers >/dev/null || addgroup -S ssh-jumpers
group_add_system ssh-admins
group_add_system ssh-jumpers
# ----------------------------------------------------------------------------
# 5. Root keypair (for ssh-admins maintenance access)
@@ -135,27 +117,35 @@ mkdir -p /root/.ssh
chmod 700 /root/.ssh
touch /root/.ssh/authorized_keys
chmod 600 /root/.ssh/authorized_keys
# Add root to ssh-admins so the Match block applies.
adduser root ssh-admins 2>/dev/null || true
user_add_to_group root ssh-admins
TMP_KEY=$(mktemp -u /tmp/root_ed25519.XXXXXX)
ssh-keygen -q -t ed25519 -f "$TMP_KEY" -N "" -C "$KEY_COMMENT"
ROOT_PUB=$(cat "${TMP_KEY}.pub")
ROOT_PRIV=$(cat "$TMP_KEY")
if ! grep -qxF "$ROOT_PUB" /root/.ssh/authorized_keys; then
echo "$ROOT_PUB" >> /root/.ssh/authorized_keys
grep -qxF "$ROOT_PUB" /root/.ssh/authorized_keys || echo "$ROOT_PUB" >> /root/.ssh/authorized_keys
# Seed root (ssh-admins) with the shared admin keys from globals/ so the
# bastion has a known, secure default login. Best-effort.
if [[ "${SEED_KEYS:-1}" == "1" && -f "$SCRIPT_DIR/lib.sh" ]]; then
# shellcheck source=scripts/lib.sh
. "$SCRIPT_DIR/lib.sh"
load_globals
if declare -f resolve_ssh_keys >/dev/null 2>&1; then
SEEDED=0
while IFS= read -r k; do
[[ -n "$k" ]] || continue
grep -qxF "$k" /root/.ssh/authorized_keys || { echo "$k" >> /root/.ssh/authorized_keys; SEEDED=$((SEEDED+1)); }
done <<< "$(resolve_ssh_keys 2>/dev/null || true)"
[[ "$SEEDED" -gt 0 ]] && log "Seeded ${SEEDED} admin key(s) into /root/.ssh/authorized_keys from globals."
fi
fi
# ----------------------------------------------------------------------------
# 6. Build PermitOpen line from JUMP_TARGETS
# 6. PermitOpen line from JUMP_TARGETS (space-separated -> comma-separated)
# ----------------------------------------------------------------------------
# JUMP_TARGETS is space-separated; PermitOpen wants comma-separated.
# Empty => "none" (deny all). sshd_config doesn't accept a literal empty list.
PERMIT_OPEN_LINE="none"
if [[ -n "$JUMP_TARGETS" ]]; then
PERMIT_OPEN_LINE=$(echo "$JUMP_TARGETS" | tr -s ' ' ',' | sed 's/^,//;s/,$//')
fi
[[ -n "$JUMP_TARGETS" ]] && PERMIT_OPEN_LINE=$(echo "$JUMP_TARGETS" | tr -s ' ' ',' | sed 's/^,//;s/,$//')
# ----------------------------------------------------------------------------
# 7. sshd_config
@@ -164,13 +154,18 @@ log "Writing /etc/ssh/sshd_config..."
[[ -f /etc/ssh/sshd_config.orig ]] || cp /etc/ssh/sshd_config /etc/ssh/sshd_config.orig
cat > /etc/ssh/sshd_config <<EOF
# Generated by harden-jumphost.sh -- $(date -u +%FT%TZ)
# Generated by harden-jumphost.sh on ${OS_ID} -- $(date -u +%FT%TZ)
# Original config preserved at /etc/ssh/sshd_config.orig
Port ${SSH_PORT}
AddressFamily any
ListenAddress 0.0.0.0
ListenAddress ::
PidFile /run/sshd.pid
# VERBOSE so the auth log records key fingerprints and direct-tcpip targets
# (used by the login notifier to report the key and best-effort jump target).
LogLevel VERBOSE
# --- Host key: Ed25519 only ---
HostKey /etc/ssh/ssh_host_ed25519_key
@@ -199,13 +194,15 @@ AuthenticationMethods publickey
MaxAuthTries 3
MaxSessions 10
LoginGraceTime 30s
# Expose the authenticated key (file at \$SSH_USER_AUTH) for the notifier.
ExposeAuthInfo yes
# --- Default posture: deny everything ---
# Anyone not in ssh-admins or ssh-jumpers gets nothing. Matches below
# Anyone not matched below gets nothing. The per-group Match blocks
# selectively re-enable what each group needs.
DisableForwarding yes
PermitTTY no
ForceCommand /sbin/nologin
ForceCommand ${NOLOGIN}
X11Forwarding no
GatewayPorts no
PermitTunnel no
@@ -228,13 +225,11 @@ Compression no
Banner none
# --- Subsystems ---
# SFTP off by default. Admins who need it can be added to a separate
# Match block; jumpers must never have it.
# Subsystem sftp internal-sftp
# SFTP off by default on a jump host; jumpers must never have it. If you
# enable it for admins, log transfers to AUTHPRIV at INFO:
# Subsystem sftp ${SFTP_PATH} -f AUTHPRIV -l INFO
# --- Allowlist ---
# Only members of these groups can authenticate at all. AllowGroups
# enforces this BEFORE the per-group Match blocks run.
AllowGroups ssh-admins ssh-jumpers
# ============================================================================
@@ -252,13 +247,10 @@ Match Group ssh-admins
X11Forwarding no
PermitOpen none
# --- Jumpers: ProxyJump only, no shell, no SFTP, whitelisted destinations ---
# direct-tcpip channels are controlled by AllowTcpForwarding + PermitOpen.
# Session channels (shell/exec/subsystem) hit ForceCommand=/sbin/nologin
# and PermitTTY=no, so any interactive attempt fails immediately.
# --- Jumpers: ProxyJump only, no shell, whitelisted destinations ---
Match Group ssh-jumpers
PermitTTY no
ForceCommand /sbin/nologin
ForceCommand ${NOLOGIN}
AllowTcpForwarding yes
PermitOpen ${PERMIT_OPEN_LINE}
AllowAgentForwarding no
@@ -287,14 +279,17 @@ log "Configuring sshguard..."
mkdir -p /etc/sshguard
WHITELIST=/etc/sshguard/whitelist
{
echo "127.0.0.1"
echo "::1"
echo "127.0.0.1"; echo "::1"
[[ -n "$ALLOWED_IP" ]] && echo "$ALLOWED_IP"
} > "$WHITELIST"
SSHGUARD_BACKEND="$(sshguard_backend)"
SSHGUARD_LOGREADER="$(sshguard_logreader)"
[[ -x "${SSHGUARD_BACKEND}" ]] || warn "sshguard backend not found at ${SSHGUARD_BACKEND}; brute-force blocking may be inactive."
cat > /etc/sshguard/sshguard.conf <<EOF
BACKEND="/usr/libexec/sshg-fw-iptables"
LOGREADER="LANG=C journalctl -afb -p info -n1 -u sshd -o cat"
BACKEND="${SSHGUARD_BACKEND}"
${SSHGUARD_LOGREADER:+LOGREADER="${SSHGUARD_LOGREADER}"}
THRESHOLD=30
BLOCK_TIME=300
DETECTION_TIME=1800
@@ -302,53 +297,74 @@ PID_FILE=/run/sshguard.pid
WHITELIST_FILE=${WHITELIST}
EOF
cat > /etc/local.d/sshguard-iptables.start <<'EOF'
HOOK=$(mktemp)
cat > "$HOOK" <<'EOF'
#!/bin/sh
SSH_PORT=$(awk '/^Port / {print $2; exit}' /etc/ssh/sshd_config)
SSH_PORT=${SSH_PORT:-22}
for ipt in iptables ip6tables; do
command -v "$ipt" >/dev/null 2>&1 || continue
$ipt -N sshguard 2>/dev/null || true
$ipt -C INPUT -p tcp --dport "$SSH_PORT" -j sshguard 2>/dev/null \
|| $ipt -I INPUT -p tcp --dport "$SSH_PORT" -j sshguard
done
EOF
chmod +x /etc/local.d/sshguard-iptables.start
rc-update add local default 2>/dev/null || true
/etc/local.d/sshguard-iptables.start
install_boot_hook sshguard-iptables "$HOOK"
rm -f "$HOOK"
rc-update add sshguard default
rc-service sshguard restart || rc-service sshguard start
svc_enable_start sshguard || warn "Could not start sshguard; check sshguard.conf on this distro."
# ----------------------------------------------------------------------------
# 9b. Optional: SSH login notifier (pam_exec -> ntfy)
# ----------------------------------------------------------------------------
# Enabled when NTFY_URL is provided. On a bastion we default to notifying for
# the two SSH groups and tag the alert with this host's region.
if [[ -n "${NTFY_URL:-}" ]]; then
: "${NOTIFY_GROUPS:=ssh-admins ssh-jumpers}"
: "${NTFY_REGION:=$(host_region)}"
log "Installing SSH login notifier (ntfy)..."
install_login_notifier "$SCRIPT_DIR/ntfy-ssh-login.sh" || warn "Notifier install had issues."
fi
# ----------------------------------------------------------------------------
# 9c. Daily unattended updates (default ON -- recommended for an SSH-only
# bastion; set AUTO_UPDATE=0 to skip). New Alpine *branches* are reported, not
# auto-applied.
# ----------------------------------------------------------------------------
if [[ "${AUTO_UPDATE:-1}" == "1" && -f "$SCRIPT_DIR/auto-update.sh" ]]; then
log "Scheduling daily auto-update (reboot only when idle)..."
AUTO_REBOOT="${AUTO_REBOOT:-idle}" \
ALLOW_RELEASE_UPGRADE="${ALLOW_RELEASE_UPGRADE:-0}" \
NOTIFY="${NOTIFY:-1}" \
bash "$SCRIPT_DIR/auto-update.sh" install || warn "Could not schedule auto-update."
fi
# ----------------------------------------------------------------------------
# 10. Enable sshd
# ----------------------------------------------------------------------------
log "Enabling sshd at boot..."
rc-update add sshd default
SSHD_SVC="$(sshd_service)"
log "Enabling ${SSHD_SVC} at boot..."
svc_enable "$SSHD_SVC"
cat <<EOF
================================================================
JUMP HOST SETUP COMPLETE
JUMP HOST SETUP COMPLETE (${OS_ID})
Groups created:
ssh-admins -- full shell on the jump host (root added)
ssh-jumpers -- ProxyJump only, no shell
Add a jumper user:
adduser -D -s /sbin/nologin alice
addgroup alice ssh-jumpers
mkdir -p /home/alice/.ssh && chmod 700 /home/alice/.ssh
echo 'ssh-ed25519 AAA...' > /home/alice/.ssh/authorized_keys
chmod 600 /home/alice/.ssh/authorized_keys
chown -R alice:alice /home/alice/.ssh
Add a jumper user (using the installed tool):
sshuser add -u alice -r jumper -k "ssh-ed25519 AAA..."
Note the user's shell can be /sbin/nologin -- ProxyJump still works
because it never opens a session channel.
Or an admin:
sshuser add -u bob -r admin -k "ssh-ed25519 AAA..."
Allowed jump targets (PermitOpen):
${PERMIT_OPEN_LINE}
To change targets: edit JUMP_TARGETS and re-run, or edit the Match
To change targets: re-run with JUMP_TARGETS set, or edit the Match
block in /etc/ssh/sshd_config directly.
COPY THIS PRIVATE KEY TO YOUR CLIENT *NOW* (admin/root key):
@@ -366,10 +382,8 @@ Host fingerprint:
$(ssh-keygen -l -f /etc/ssh/ssh_host_ed25519_key.pub)
Client usage examples:
# Admin shell on the jump host:
ssh -i ~/.ssh/id_ed25519_jump -p ${SSH_PORT} root@<jumphost>
# ProxyJump through to an internal target:
ssh -J root@<jumphost>:${SSH_PORT} -i ~/.ssh/id_ed25519_target user@<target>
@@ -390,11 +404,11 @@ Reload sshd now? [y/N]
EOF
read -r ans
if [[ "${ans,,}" != "y" && "${ans,,}" != "yes" ]]; then
warn "Skipping reload. Run 'rc-service sshd reload' manually when ready."
warn "Skipping reload. Reload ${SSHD_SVC} manually when ready."
exit 0
fi
fi
log "Reloading sshd..."
rc-service sshd reload || rc-service sshd restart
log "Reloading ${SSHD_SVC}..."
svc_reload "$SSHD_SVC"
log "Done."
scripts/harden-ssh.sh
+144 -146
View File
@@ -2,129 +2,108 @@
#
# harden-ssh.sh
#
# SSH hardening for Alpine Linux. Run BEFORE deploy-simplex.sh on a fresh box.
# SSH hardening for Alpine, Debian, and Alma Linux. Run on a fresh box (and,
# for the simplex relay, BEFORE deploy-simplex.sh).
#
# All distro-specific operations go through scripts/oslib.sh. The OS-specific
# surface for this script is: package install, the sshd service name, the
# external sftp-server path, host-key keygen suppression (Alpine only), the
# sshguard log source + firewall backend, and the boot hook that installs the
# iptables jump. Each is clearly marked.
#
# What this does:
# 1. Generates fresh Ed25519 host keys; removes RSA/ECDSA/DSA host keys
# 2. Generates an Ed25519 root keypair on the host, installs the public key
# into /root/.ssh/authorized_keys, and PRINTS the private key to stdout
# so you can copy it to your client. THIS IS YOUR ONLY CHANCE TO COPY IT.
# 3. Forces post-quantum hybrid KEX only:
# mlkem768x25519-sha256 (the future default, NIST ML-KEM hybrid)
# sntrup761x25519-sha512 (older PQ KEX, kept as fallback)
# Drops every classical-only KEX. Connections that don't speak PQ KEX
# will be rejected.
# 4. Modern ciphers and MACs only (chacha20-poly1305, aes256-gcm,
# hmac-sha2-512-etm)
# 5. Disables everything not needed for an interactive terminal:
# - password auth, root password login (key-only)
# - challenge-response, GSSAPI, PAM, host-based auth
# - X11 forwarding, agent forwarding
# - TCP forwarding, stream-local forwarding (UNIX sockets)
# - tunneling (PermitTunnel), gateway ports
# - SFTP subsystem (kept ON — needed for backup retrieval)
# - empty passwords, .ssh/rc execution, compression
# Result: a session can run a shell. That's it. No -L, no -R, no -D, no
# jump hosting, no sftp, no scp.
# 6. Optional non-default port (-p PORT)
# 7. Installs sshguard with iptables backend for brute-force protection
# 8. Validates config with `sshd -t` and prompts for confirmation before
# reloading sshd (so a config error or a typo doesn't lock you out)
# 2. Generates an Ed25519 root keypair, installs the public key into
# /root/.ssh/authorized_keys, and PRINTS the private key to stdout once.
# 3. Forces post-quantum hybrid KEX only (mlkem768x25519, sntrup761x25519).
# 4. Modern ciphers and MACs only.
# 5. Disables everything but an interactive terminal + SFTP (no forwarding,
# tunneling, X11, agent, password auth).
# 6. Optional non-default port (SSH_PORT).
# 7. Installs sshguard for brute-force protection.
# 8. Validates with `sshd -t` and prompts before reloading (so you don't
# lock yourself out).
#
# A note on "quantum-safe":
# Stock OpenSSH provides PQ KEY EXCHANGE (the session key, the thing that
# matters for "store now, decrypt later"). It does NOT yet provide PQ
# AUTHENTICATION KEYS -- there is no standardized PQ host or user key
# algorithm in mainline OpenSSH yet. So:
# - Your session is PQ-protected against SNDL: yes
# - Your auth keypair (Ed25519) is classical: yes, and that's the best
# practical choice today. PQ signature support exists only in the
# open-quantum-safe/openssh fork, which breaks compatibility with
# every standard SSH client.
# This script gives you the strongest stock-OpenSSH posture available.
# A note on "quantum-safe": stock OpenSSH gives PQ KEY EXCHANGE (protects the
# session key against store-now-decrypt-later) but classical Ed25519 AUTH
# keys -- the strongest practical posture available without breaking client
# compatibility.
#
# Usage:
# bash harden-ssh.sh # port stays 22, default
# bash harden-ssh.sh # port stays 22
# SSH_PORT=2222 bash harden-ssh.sh # change port
# ALLOWED_IP=1.2.3.4 bash harden-ssh.sh # whitelist your client IP in sshguard
# FORCE=1 bash harden-ssh.sh # skip the "are you sure" prompt
# ALLOWED_IP=1.2.3.4 bash harden-ssh.sh # whitelist your client IP
# FORCE=1 bash harden-ssh.sh # skip the confirm prompt
set -euo pipefail
# Load the OS abstraction layer (sits next to this script).
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=scripts/oslib.sh
. "$SCRIPT_DIR/oslib.sh"
# ============================================================================
# CONFIG
# ============================================================================
: "${SSH_PORT:=22}"
: "${ALLOWED_IP:=}" # optional: your client IP, will be sshguard-whitelisted
: "${ALLOWED_IP:=}" # optional: your client IP, sshguard-whitelisted
: "${KEY_COMMENT:=root@$(hostname)-$(date +%Y%m%d)}"
: "${FORCE:=0}"
log() { printf '\033[1;32m[+]\033[0m %s\n' "$*"; }
warn() { printf '\033[1;33m[!]\033[0m %s\n' "$*" >&2; }
die() { printf '\033[1;31m[x]\033[0m %s\n' "$*" >&2; exit 1; }
# log()/warn()/die() come from oslib (_log/_warn/_die); alias for readability.
log() { _log "$@"; }
warn() { _warn "$@"; }
die() { _die "$@"; }
[[ $EUID -eq 0 ]] || die "Run as root."
[[ -f /etc/alpine-release ]] || die "This script targets Alpine Linux."
os_detect
log "Detected OS: ${OS_ID} (family ${OS_FAMILY}, init ${INIT_SYSTEM})"
# ----------------------------------------------------------------------------
# 1. Pre-flight checks
# 1. Pre-flight: ensure an ssh client exists before probing its version
# ----------------------------------------------------------------------------
# Ensure ssh client is installed before probing its version. On a fresh
# Alpine box there's no `ssh` binary at all, and the version-detect
# pipeline below would die silently under `set -euo pipefail`.
if ! command -v ssh >/dev/null 2>&1; then
log "ssh not found; installing openssh + openssh-server..."
apk add -q openssh openssh-server
log "ssh not found; installing openssh..."
install_openssh
fi
log "Checking OpenSSH version supports PQ KEX..."
SSH_VER=$(ssh -V 2>&1 | grep -oE 'OpenSSH_[0-9]+\.[0-9]+' | head -1 \
| sed 's/OpenSSH_//')
SSH_VER=$(ssh -V 2>&1 | grep -oE 'OpenSSH_[0-9]+\.[0-9]+' | head -1 | sed 's/OpenSSH_//')
SSH_MAJOR=${SSH_VER%%.*}
SSH_MINOR=${SSH_VER##*.}
# OpenSSH 9.0+ has sntrup761x25519-sha512.
# OpenSSH 9.9+ also has mlkem768x25519-sha256.
# OpenSSH 9.0+ has sntrup761x25519-sha512; 9.9+ adds mlkem768x25519-sha256.
HAS_MLKEM=0
HAS_SNTRUP=0
if [[ $SSH_MAJOR -gt 9 || ( $SSH_MAJOR -eq 9 && $SSH_MINOR -ge 0 ) ]]; then
HAS_SNTRUP=1
fi
if [[ $SSH_MAJOR -gt 9 || ( $SSH_MAJOR -eq 9 && $SSH_MINOR -ge 9 ) ]]; then
HAS_MLKEM=1
fi
[[ $SSH_MAJOR -gt 9 || ( $SSH_MAJOR -eq 9 && $SSH_MINOR -ge 0 ) ]] && HAS_SNTRUP=1
[[ $SSH_MAJOR -gt 9 || ( $SSH_MAJOR -eq 9 && $SSH_MINOR -ge 9 ) ]] && HAS_MLKEM=1
[[ $HAS_SNTRUP -eq 1 || $HAS_MLKEM -eq 1 ]] \
|| die "OpenSSH ${SSH_VER} has no PQ KEX. Need >= 9.0. Upgrade Alpine first."
|| die "OpenSSH ${SSH_VER} has no PQ KEX. Need >= 9.0. Upgrade the base OS first."
log "OpenSSH ${SSH_VER}: ML-KEM=${HAS_MLKEM} sntrup761=${HAS_SNTRUP}"
# Build the KEX list from what's actually available.
KEX_LIST=""
[[ $HAS_MLKEM -eq 1 ]] && KEX_LIST="mlkem768x25519-sha256"
[[ $HAS_SNTRUP -eq 1 ]] && KEX_LIST="${KEX_LIST:+$KEX_LIST,}sntrup761x25519-sha512"
# ----------------------------------------------------------------------------
# 2. Install packages
# 2. Install packages (OS-gated inside oslib)
# ----------------------------------------------------------------------------
log "Installing openssh-server-pam, sshguard, iptables..."
# openssh-server-pam replaces openssh-server (PAM-enabled sshd). If the
# non-pam version was installed earlier, swap it out cleanly.
if apk info -e openssh-server >/dev/null 2>&1 && \
! apk info -e openssh-server-pam >/dev/null 2>&1; then
apk del -q openssh-server || true
fi
apk add -q openssh openssh-server-pam linux-pam sshguard iptables ip6tables openrc
log "Installing OpenSSH server + sshguard + iptables..."
install_openssh
install_bruteforce_protection
# The external SFTP subsystem binary path differs per distro.
SFTP_PATH="$(sftp_server_path)"
[[ -x "$SFTP_PATH" ]] || warn "sftp-server not found at expected path ($SFTP_PATH); SFTP may not work until installed."
# ----------------------------------------------------------------------------
# 3. Host keys -- regenerate with Ed25519 only
# ----------------------------------------------------------------------------
log "Regenerating host keys (Ed25519 only)..."
rm -f /etc/ssh/ssh_host_rsa_key* \
/etc/ssh/ssh_host_ecdsa_key* \
rm -f /etc/ssh/ssh_host_rsa_key* \
/etc/ssh/ssh_host_ecdsa_key* \
/etc/ssh/ssh_host_dsa_key*
# Keep existing ed25519 key if there is one (so the host fingerprint doesn't
# change unnecessarily on re-runs). Generate one if not.
if [[ ! -f /etc/ssh/ssh_host_ed25519_key ]]; then
ssh-keygen -q -t ed25519 -f /etc/ssh/ssh_host_ed25519_key -N "" \
-C "host@$(hostname)-$(date +%Y%m%d)"
@@ -135,16 +114,9 @@ chmod 644 /etc/ssh/ssh_host_ed25519_key.pub
log "Host key fingerprint (verify on first connect):"
ssh-keygen -l -f /etc/ssh/ssh_host_ed25519_key.pub | sed 's/^/ /'
# Stop Alpine's sshd init from regenerating RSA/ECDSA keys on every start.
# /etc/conf.d/sshd: pin sshd_disable_keygen=no but only generate ed25519 by
# overriding the keygen line in the init script via a drop-in.
if [[ -f /etc/conf.d/sshd ]]; then
if ! grep -q '^sshd_disable_keygen=' /etc/conf.d/sshd; then
echo 'sshd_disable_keygen="yes"' >> /etc/conf.d/sshd
else
sed -i 's/^sshd_disable_keygen=.*/sshd_disable_keygen="yes"/' /etc/conf.d/sshd
fi
fi
# Alpine's OpenRC sshd init regenerates RSA/ECDSA keys on each start; pin off.
# No-op on systemd distros.
sshd_disable_keygen
# ----------------------------------------------------------------------------
# 4. Root user keypair
@@ -155,54 +127,64 @@ chmod 700 /root/.ssh
touch /root/.ssh/authorized_keys
chmod 600 /root/.ssh/authorized_keys
# Always create a brand-new key pair in a temp location so we can show the
# private key to the user and then add the public key to authorized_keys.
TMP_KEY=$(mktemp -u /tmp/root_ed25519.XXXXXX)
ssh-keygen -q -t ed25519 -f "$TMP_KEY" -N "" -C "$KEY_COMMENT"
ROOT_PUB=$(cat "${TMP_KEY}.pub")
ROOT_PRIV=$(cat "$TMP_KEY")
# Idempotency: don't add the same pubkey twice.
if ! grep -qxF "$ROOT_PUB" /root/.ssh/authorized_keys; then
echo "$ROOT_PUB" >> /root/.ssh/authorized_keys
grep -qxF "$ROOT_PUB" /root/.ssh/authorized_keys || echo "$ROOT_PUB" >> /root/.ssh/authorized_keys
# Seed root's authorized_keys with the shared admin keys from globals/ so the
# box has a known, secure default login (SSH_KEYS_URL first, else
# globals/authorized_keys). Best-effort: needs the repo's lib.sh present.
if [[ "${SEED_KEYS:-1}" == "1" && -f "$SCRIPT_DIR/lib.sh" ]]; then
# shellcheck source=scripts/lib.sh
. "$SCRIPT_DIR/lib.sh"
load_globals
if declare -f resolve_ssh_keys >/dev/null 2>&1; then
SEEDED=0
while IFS= read -r k; do
[[ -n "$k" ]] || continue
grep -qxF "$k" /root/.ssh/authorized_keys || { echo "$k" >> /root/.ssh/authorized_keys; SEEDED=$((SEEDED+1)); }
done <<< "$(resolve_ssh_keys 2>/dev/null || true)"
[[ "$SEEDED" -gt 0 ]] && log "Seeded ${SEEDED} admin key(s) into /root/.ssh/authorized_keys from globals."
fi
fi
# ----------------------------------------------------------------------------
# 5. sshd_config
# ----------------------------------------------------------------------------
log "Writing /etc/ssh/sshd_config..."
# Back up whatever was there before, once.
[[ -f /etc/ssh/sshd_config.orig ]] || cp /etc/ssh/sshd_config /etc/ssh/sshd_config.orig
cat > /etc/ssh/sshd_config <<EOF
# Generated by harden-ssh.sh -- $(date -u +%FT%TZ)
# Generated by harden-ssh.sh on ${OS_ID} -- $(date -u +%FT%TZ)
# Original config preserved at /etc/ssh/sshd_config.orig
Port ${SSH_PORT}
AddressFamily any
ListenAddress 0.0.0.0
ListenAddress ::
PidFile /run/sshd.pid
# --- Host key: Ed25519 only ---
HostKey /etc/ssh/ssh_host_ed25519_key
# --- Post-quantum hybrid KEX only ---
# Anything not in this list (every classical-only KEX) is rejected. This
# protects against "store now, decrypt later" because the session key is
# derived from a PQ KEM hybrid.
# Anything not in this list (every classical-only KEX) is rejected, which is
# what protects the session key against "store now, decrypt later".
KexAlgorithms ${KEX_LIST}
# --- Modern ciphers and MACs ---
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,umac-128-etm@openssh.com
# --- Host key signature algorithms (for the host proving itself) ---
# --- Host key signature algorithms ---
HostKeyAlgorithms ssh-ed25519,ssh-ed25519-cert-v01@openssh.com
# --- Public key algorithms accepted from clients ---
# Ed25519 + RSA-4096 (RSA kept for older YubiKey firmware that lacks Ed25519
# in the PIV applet -- pre-5.7). Plain ssh-rsa (SHA-1) is NOT included.
# Ed25519 + RSA-4096 (RSA kept for older YubiKey PIV firmware pre-5.7).
PubkeyAcceptedAlgorithms ssh-ed25519,ssh-ed25519-cert-v01@openssh.com,sk-ssh-ed25519@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256,rsa-sha2-256-cert-v01@openssh.com
RequiredRSASize 4096
@@ -213,21 +195,16 @@ PasswordAuthentication no
PermitEmptyPasswords no
ChallengeResponseAuthentication no
KbdInteractiveAuthentication no
# PAM is enabled so the session stack runs on every login (used by
# pam_exec.so for post-login notification hooks). Auth still requires a
# pubkey -- PAM only handles account/session, not auth, because
# PasswordAuthentication and KbdInteractiveAuthentication are off.
UsePAM yes
AuthenticationMethods publickey
MaxAuthTries 3
MaxSessions 4
LoginGraceTime 30s
# Expose the authenticated key to the session (file at \$SSH_USER_AUTH) so the
# pam_exec login notifier can report which key was used.
ExposeAuthInfo yes
# --- Session restrictions: terminal access only, no forwarding/tunneling ---
# This server runs SimpleX relays. The only legitimate SSH use is an admin
# terminal session. Every forwarding feature below is therefore disabled --
# they're just attack surface and lateral-movement aids if the host is ever
# compromised.
# --- Session restrictions: terminal + SFTP only, no forwarding/tunneling ---
X11Forwarding no
X11UseLocalhost yes
AllowAgentForwarding no
@@ -236,7 +213,7 @@ AllowStreamLocalForwarding no
DisableForwarding yes # belt-and-braces: kills *all* forwarding types
GatewayPorts no
PermitTunnel no
PermitUserRC no # don't run ~/.ssh/rc on login
PermitUserRC no
PermitListen none
PermitOpen none
PrintMotd no
@@ -251,10 +228,10 @@ HostbasedAuthentication no
Compression no
Banner none
# SFTP subsystem: enabled for backup retrieval only.
# Note: this allows sftp and scp to work. Only the forwarding/tunneling
# features above are disabled -- file transfer via SFTP is legitimate.
Subsystem sftp internal-sftp
# SFTP subsystem: external sftp-server binary (path is per-distro, resolved
# by oslib's sftp_server_path). Enabled for backup retrieval, with logging to
# AUTHPRIV at INFO so file transfers are auditable.
Subsystem sftp ${SFTP_PATH} -f AUTHPRIV -l INFO
# Restrict who can SSH in. Add other users here if you create them.
AllowUsers root
@@ -274,10 +251,9 @@ rm -f /tmp/sshd-test.err
# ----------------------------------------------------------------------------
# 7. sshguard (brute-force protection)
# ----------------------------------------------------------------------------
log "Configuring sshguard with iptables backend..."
log "Configuring sshguard (backend + log source are OS-gated in oslib)..."
mkdir -p /etc/sshguard
# Whitelist: localhost always, plus optional caller-supplied IP.
WHITELIST=/etc/sshguard/whitelist
{
echo "127.0.0.1"
@@ -285,10 +261,13 @@ WHITELIST=/etc/sshguard/whitelist
[[ -n "$ALLOWED_IP" ]] && echo "$ALLOWED_IP"
} > "$WHITELIST"
# sshguard.conf: explicitly point it at iptables backend.
SSHGUARD_BACKEND="$(sshguard_backend)"
SSHGUARD_LOGREADER="$(sshguard_logreader)"
[[ -x "${SSHGUARD_BACKEND}" ]] || warn "sshguard backend not found at ${SSHGUARD_BACKEND}; brute-force blocking may be inactive."
cat > /etc/sshguard/sshguard.conf <<EOF
BACKEND="/usr/libexec/sshg-fw-iptables"
LOGREADER="LANG=C journalctl -afb -p info -n1 -u sshd -o cat"
BACKEND="${SSHGUARD_BACKEND}"
${SSHGUARD_LOGREADER:+LOGREADER="${SSHGUARD_LOGREADER}"}
THRESHOLD=30
BLOCK_TIME=300
DETECTION_TIME=1800
@@ -296,44 +275,68 @@ PID_FILE=/run/sshguard.pid
WHITELIST_FILE=${WHITELIST}
EOF
# sshguard inserts rules into chain "sshguard"; we need a jump from INPUT.
# awall doesn't manage this chain, so we add it once and make it persist via
# a small startup hook.
cat > /etc/local.d/sshguard-iptables.start <<'EOF'
# Boot hook: ensure the sshguard chain exists and INPUT jumps to it for the
# SSH port. oslib installs this as an OpenRC local.d script or a systemd
# oneshot unit, depending on the init system.
HOOK=$(mktemp)
cat > "$HOOK" <<'EOF'
#!/bin/sh
# Ensure sshguard chain exists and INPUT jumps to it for tcp/22 (and PORT).
# Ensure sshguard chain exists and INPUT jumps to it for the SSH port.
SSH_PORT=$(awk '/^Port / {print $2; exit}' /etc/ssh/sshd_config)
SSH_PORT=${SSH_PORT:-22}
for ipt in iptables ip6tables; do
command -v "$ipt" >/dev/null 2>&1 || continue
$ipt -N sshguard 2>/dev/null || true
$ipt -C INPUT -p tcp --dport "$SSH_PORT" -j sshguard 2>/dev/null \
|| $ipt -I INPUT -p tcp --dport "$SSH_PORT" -j sshguard
done
EOF
chmod +x /etc/local.d/sshguard-iptables.start
rc-update add local default 2>/dev/null || true
/etc/local.d/sshguard-iptables.start
install_boot_hook sshguard-iptables "$HOOK"
rm -f "$HOOK"
rc-update add sshguard default
rc-service sshguard restart || rc-service sshguard start
svc_enable_start sshguard || warn "Could not start sshguard; check 'sshguard.conf' on this distro."
# ----------------------------------------------------------------------------
# 8. SSHD enable & reload (with safety prompt)
# 7b. Optional: SSH login notifier (pam_exec -> ntfy)
# ----------------------------------------------------------------------------
log "Enabling sshd at boot..."
rc-update add sshd default
# Enabled when NTFY_URL is provided. Reports user + source IP + the key used,
# filtered by NOTIFY_GROUPS (empty here = every login on this host).
if [[ -n "${NTFY_URL:-}" ]]; then
: "${NTFY_REGION:=$(host_region)}"
log "Installing SSH login notifier (ntfy)..."
install_login_notifier "$SCRIPT_DIR/ntfy-ssh-login.sh" || warn "Notifier install had issues."
fi
# Print the private key BEFORE reloading sshd so even if reload locks the
# user out, they have what they need to come back in via console.
# ----------------------------------------------------------------------------
# 7c. Optional: daily unattended updates (set AUTO_UPDATE=1). New OS branches
# are reported, not auto-applied.
# ----------------------------------------------------------------------------
if [[ "${AUTO_UPDATE:-0}" == "1" && -f "$SCRIPT_DIR/auto-update.sh" ]]; then
log "Scheduling daily auto-update..."
AUTO_REBOOT="${AUTO_REBOOT:-0}" \
ALLOW_RELEASE_UPGRADE="${ALLOW_RELEASE_UPGRADE:-0}" \
NOTIFY="${NOTIFY:-1}" \
bash "$SCRIPT_DIR/auto-update.sh" install || warn "Could not schedule auto-update."
fi
# ----------------------------------------------------------------------------
# 8. Enable sshd & reload (with safety prompt)
# ----------------------------------------------------------------------------
SSHD_SVC="$(sshd_service)"
log "Enabling ${SSHD_SVC} at boot..."
svc_enable "$SSHD_SVC"
# Print the private key BEFORE reloading sshd so a bad reload still leaves you
# with what you need to get back in via console.
cat <<EOF
================================================================
COPY THIS PRIVATE KEY TO YOUR CLIENT *NOW*
Save it as e.g. ~/.ssh/id_ed25519_simplex on your local machine,
Save it as e.g. ~/.ssh/id_ed25519_host on your local machine,
chmod 600, and connect with:
ssh -i ~/.ssh/id_ed25519_simplex -p ${SSH_PORT} root@<host>
ssh -i ~/.ssh/id_ed25519_host -p ${SSH_PORT} root@<host>
----- BEGIN ROOT PRIVATE KEY (Ed25519) -----
${ROOT_PRIV}
@@ -348,19 +351,15 @@ $(ssh-keygen -l -f /etc/ssh/ssh_host_ed25519_key.pub)
================================================================
EOF
# Wipe the temp files holding the private key.
shred -u "$TMP_KEY" "${TMP_KEY}.pub" 2>/dev/null || rm -f "$TMP_KEY" "${TMP_KEY}.pub"
# Final guard: confirm before reloading sshd. A bad reload is recoverable from
# console; a bad reload while you assumed everything was fine is not.
if [[ "$FORCE" != "1" ]]; then
cat <<EOF
sshd config has passed validation. Ready to reload sshd.
If you are connected via SSH RIGHT NOW, opening a SECOND ssh session in
another terminal -- before answering yes -- to verify the new keys, port,
and PQ KEX work is the safest path. If your new key/port/KEX is wrong,
this reload will end your current session.
If you are connected via SSH RIGHT NOW, open a SECOND session in another
terminal -- before answering yes -- to verify the new keys, port, and PQ
KEX work. If something is wrong, this reload will end your current session.
Test in another terminal first:
ssh -i ~/.ssh/<your saved key> -p ${SSH_PORT} \\
@@ -370,13 +369,12 @@ Reload sshd now? [y/N]
EOF
read -r ans
if [[ "${ans,,}" != "y" && "${ans,,}" != "yes" ]]; then
warn "Skipping reload. Run 'rc-service sshd reload' manually when ready."
warn "Skipping reload. Run 'svc reload of ${SSHD_SVC}' manually when ready."
exit 0
fi
fi
log "Reloading sshd..."
rc-service sshd reload || rc-service sshd restart
log "Reloading ${SSHD_SVC}..."
svc_reload "$SSHD_SVC"
log "Done. Your session, if any, should remain alive (reload preserves connections)."
log "Test from another machine before closing this session."
scripts/lib.sh
+78
View File
@@ -0,0 +1,78 @@
#!/usr/bin/env bash
#
# lib.sh -- shared helpers for the launcher and the automation scripts.
#
# Source it, don't execute it:
# . "$(dirname "$0")/lib.sh" # from within scripts/
# . "$REPO_ROOT/scripts/lib.sh"
#
# Sourcing has no side effects beyond defining functions and REPO_ROOT, and
# pulling in oslib.sh. Targets Alpine/Debian/Alma + bash. Safe to source under
# `set -euo pipefail`.
# Resolve the repository root from this file's location (scripts/lib.sh ->
# repo root is one level up). Works whether sourced by an absolute or
# relative path.
_lib_self="${BASH_SOURCE[0]}"
REPO_ROOT="$(cd "$(dirname "$_lib_self")/.." && pwd)"
export REPO_ROOT
_log() { printf '\033[1;32m[+]\033[0m %s\n' "$*"; }
_warn() { printf '\033[1;33m[!]\033[0m %s\n' "$*" >&2; }
_die() { printf '\033[1;31m[x]\033[0m %s\n' "$*" >&2; exit 1; }
# Pull in the OS abstraction layer (multi-OS ensure_gum, pkg/service helpers,
# os_detect). It reuses the _log/_warn/_die defined above.
# shellcheck source=scripts/oslib.sh
. "$(cd "$(dirname "$_lib_self")" && pwd)/oslib.sh"
# ----------------------------------------------------------------------------
# load_globals -- export shared defaults from globals/globals.env, falling
# back to globals.env.example. Existing environment values win (a var already
# set in the environment is not overwritten), so callers and cloud-init can
# override anything.
# ----------------------------------------------------------------------------
load_globals() {
local f
for f in "$REPO_ROOT/globals/globals.env" "$REPO_ROOT/globals/globals.env.example"; do
[[ -f "$f" ]] || continue
local line key val
while IFS= read -r line || [[ -n "$line" ]]; do
# skip comments and blanks
[[ "$line" =~ ^[[:space:]]*# ]] && continue
[[ "$line" =~ ^[[:space:]]*$ ]] && continue
key="${line%%=*}"
val="${line#*=}"
key="${key//[[:space:]]/}"
[[ -n "$key" ]] || continue
# don't clobber values already provided in the environment
[[ -n "${!key:-}" ]] && continue
printf -v "$key" '%s' "$val"
export "${key?}"
done < "$f"
return 0
done
return 0
}
# ensure_gum is provided by oslib.sh (multi-OS: Alpine community repo, or the
# Charm apt/yum repos on Debian/Alma).
# ----------------------------------------------------------------------------
# resolve_ssh_keys -- print the set of admin SSH public keys, one per line.
# URL-preferred: fetch SSH_KEYS_URL when set; otherwise read
# globals/authorized_keys. Comment (#...) and blank lines are stripped from
# both sources.
# ----------------------------------------------------------------------------
resolve_ssh_keys() {
local raw
if [[ -n "${SSH_KEYS_URL:-}" ]]; then
command -v curl >/dev/null 2>&1 || _die "SSH_KEYS_URL is set but curl is unavailable."
raw="$(curl -fsSL "$SSH_KEYS_URL")" || _die "Failed to fetch SSH_KEYS_URL: $SSH_KEYS_URL"
else
local f="$REPO_ROOT/globals/authorized_keys"
[[ -f "$f" ]] || _die "No SSH_KEYS_URL set and $f is missing."
raw="$(cat "$f")"
fi
printf '%s\n' "$raw" | grep -vE '^[[:space:]]*(#|$)'
}
scripts/ntfy-ssh-login.sh
+127
View File
@@ -0,0 +1,127 @@
#!/bin/sh
#
# ntfy-ssh-login.sh -- pam_exec session hook that posts an SSH login event to
# an ntfy topic. POSIX sh (runs under busybox ash on Alpine too).
#
# Installed at /opt/scripts/ntfy-ssh-login.sh and wired into /etc/pam.d/sshd:
# session optional pam_exec.so /opt/scripts/ntfy-ssh-login.sh
#
# Reads /etc/ssh-notify.conf (see ssh-notify.conf.example). It reports:
# - the user and the source IP (PAM_USER / PAM_RHOST)
# - the SSH public key the user authenticated with (fingerprint)
# - the next hop in a ProxyJump path, when discoverable (best-effort)
# - the bastion's region tag, so you know which location it is
# and only fires for users in NOTIFY_GROUPS (if set).
#
# Notes:
# - Key capture needs `ExposeAuthInfo yes` in sshd_config (the harden
# scripts set it); it falls back to parsing the auth log.
# - Jump-target capture is best-effort: a ProxyJump opens a direct-tcpip
# channel (no session), so the target only appears in sshd logs at
# LogLevel VERBOSE/DEBUG. Absent that, it is omitted.
set -eu
CONF="${SSH_NOTIFY_CONF:-/etc/ssh-notify.conf}"
[ -r "$CONF" ] || exit 0
# shellcheck disable=SC1090
. "$CONF"
# Only act on session open, and only if a destination URL is configured.
[ "${PAM_TYPE:-}" = "open_session" ] || exit 0
[ -n "${NTFY_URL:-}" ] || exit 0
user="${PAM_USER:-unknown}"
rhost="${PAM_RHOST:-unknown}"
# ---------------------------------------------------------------------------
# Read the most recent auth-log lines, wherever this distro keeps them.
# ---------------------------------------------------------------------------
read_authlog() {
if command -v journalctl >/dev/null 2>&1; then
journalctl -n 300 --no-pager 2>/dev/null
elif [ -r /var/log/auth.log ]; then tail -n 300 /var/log/auth.log
elif [ -r /var/log/secure ]; then tail -n 300 /var/log/secure
elif [ -r /var/log/messages ]; then tail -n 300 /var/log/messages
fi
}
# ---------------------------------------------------------------------------
# Group / security-level filter. NOTIFY_GROUPS empty => notify for everyone.
# ---------------------------------------------------------------------------
ugroups="$(id -nG "$user" 2>/dev/null || echo '')"
if [ -n "${NOTIFY_GROUPS:-}" ]; then
match=0
for g in $NOTIFY_GROUPS; do
for ug in $ugroups; do [ "$g" = "$ug" ] && match=1 && break; done
[ "$match" = 1 ] && break
done
[ "$match" = 1 ] || exit 0
fi
# Per-group priority override: NOTIFY_PRIORITY_MAP="ssh-admins:high ssh-jumpers:min"
prio="${NTFY_PRIORITY:-min}"
if [ -n "${NOTIFY_PRIORITY_MAP:-}" ]; then
for entry in $NOTIFY_PRIORITY_MAP; do
g="${entry%%:*}"; p="${entry#*:}"
for ug in $ugroups; do [ "$g" = "$ug" ] && prio="$p"; done
done
fi
# ---------------------------------------------------------------------------
# Which SSH key did the user authenticate with?
# ---------------------------------------------------------------------------
keyinfo=""
if [ -n "${SSH_USER_AUTH:-}" ] && [ -r "${SSH_USER_AUTH:-}" ]; then
# Lines look like: publickey ssh-ed25519 AAAA... [comment]
pk="$(awk '$1=="publickey"{print $2" "$3; exit}' "$SSH_USER_AUTH" 2>/dev/null || true)"
if [ -n "$pk" ]; then
# ssh-keygen -l prints: "<bits> SHA256:<fp> <comment...> (<ALGO>)".
# $2 is the fingerprint; $NF is the "(ALGO)" field regardless of comment.
keyinfo="$(printf '%s\n' "$pk" | ssh-keygen -lf - 2>/dev/null | awk '{print $NF" "$2}')"
[ -n "$keyinfo" ] || keyinfo="$(printf '%s' "$pk" | awk '{print $1}')"
fi
fi
if [ -z "$keyinfo" ]; then
# Fallback: the "Accepted publickey for USER ..." auth-log line carries
# the algorithm + SHA256 fingerprint.
line="$(read_authlog | grep "Accepted publickey for $user " | tail -n1 || true)"
keyinfo="$(printf '%s' "$line" | sed -n 's/.*: \([A-Za-z0-9-]*\) \(SHA256:[A-Za-z0-9+/=]*\).*/\1 \2/p')"
fi
[ -n "$keyinfo" ] || keyinfo="(key unknown)"
# ---------------------------------------------------------------------------
# Best-effort: the next hop in a ProxyJump path (direct-tcpip target).
# ---------------------------------------------------------------------------
jump=""
jline="$(read_authlog | grep -i 'direct-tcpip' | grep -F "$rhost" | tail -n1 || true)"
[ -z "$jline" ] && jline="$(read_authlog | grep -i 'direct-tcpip' | tail -n1 || true)"
# Match "... to HOST port PORT" or "... HOST:PORT ...".
jump="$(printf '%s' "$jline" | sed -n 's/.* to \([^ ]*\) port \([0-9]*\).*/\1:\2/p')"
[ -n "$jump" ] || jump="$(printf '%s' "$jline" | grep -oE '[A-Za-z0-9._-]+:[0-9]+' | tail -n1 || true)"
# ---------------------------------------------------------------------------
# Compose and send.
# ---------------------------------------------------------------------------
ts="$(date --utc +%FT%T.%3N%Z 2>/dev/null || date -u +%FT%TZ)"
selfhost="$(hostname -f 2>/dev/null || hostname 2>/dev/null || echo unknown)"
body="SSH login: ${user} from ${rhost}
key: ${keyinfo}"
[ -n "$jump" ] && body="${body}
jump-target: ${jump}"
body="${body}
host: ${selfhost} at ${ts}"
# Build curl args.
set -- -fsS -m 5 \
-H "X-Title: ${NTFY_TITLE:-Bastion Notification}" \
-H "X-Priority: ${prio}"
[ -n "${NTFY_TOKEN:-}" ] && set -- "$@" -H "Authorization: Bearer ${NTFY_TOKEN}"
[ -n "${NTFY_EMAIL:-}" ] && set -- "$@" -H "X-Email: ${NTFY_EMAIL}"
tags="warning"
[ -n "${NTFY_REGION:-}" ] && tags="${tags},${NTFY_REGION}"
set -- "$@" -H "X-Tags: ${tags}"
curl "$@" -d "$body" "$NTFY_URL" >/dev/null 2>&1 || true
exit 0
scripts/oslib.sh
+520
View File
@@ -0,0 +1,520 @@
#!/usr/bin/env bash
#
# oslib.sh -- OS abstraction layer for Alpine, Debian, and Alma Linux.
#
# Source it; it has no side effects beyond defining functions and, after you
# call os_detect, exporting OS_ID / OS_FAMILY / INIT_SYSTEM.
#
# . "$(dirname "$0")/oslib.sh"
# os_detect
# pkg_install curl jq
#
# Every distro-specific decision lives here, behind a function or a per-OS
# `case "$OS_FAMILY"`. Consumers (harden-ssh.sh, harden-jumphost.sh,
# sshuser.sh, setup-host.sh) should never call apk/apt/dnf, rc-service, or
# systemctl directly -- they call these helpers instead. That keeps the
# OS-specific surface in ONE auditable file.
#
# Supported targets:
# OS_ID OS_FAMILY INIT_SYSTEM package mgr
# ------ --------- ----------- -----------
# alpine alpine openrc apk
# debian debian systemd apt-get (also ubuntu)
# alma rhel systemd dnf (also rocky/rhel/centos)
# Reuse the log helpers if a consumer already defined them; otherwise define.
if ! declare -f _log >/dev/null 2>&1; then
_log() { printf '\033[1;32m[+]\033[0m %s\n' "$*"; }
_warn() { printf '\033[1;33m[!]\033[0m %s\n' "$*" >&2; }
_die() { printf '\033[1;31m[x]\033[0m %s\n' "$*" >&2; exit 1; }
fi
# ============================================================================
# Detection
# ============================================================================
os_detect() {
local id="" like="" osr="${OS_RELEASE_FILE:-/etc/os-release}"
if [[ -r "$osr" ]]; then
# shellcheck disable=SC1091
id="$(. "$osr" 2>/dev/null && echo "${ID:-}")"
like="$(. "$osr" 2>/dev/null && echo "${ID_LIKE:-}")"
elif [[ -f /etc/alpine-release ]]; then
id=alpine
fi
case "$id" in
alpine) OS_ID=alpine; OS_FAMILY=alpine ;;
debian|ubuntu|raspbian) OS_ID=debian; OS_FAMILY=debian ;;
almalinux|alma|rocky|rhel|centos|fedora) OS_ID=alma; OS_FAMILY=rhel ;;
*)
# Fall back to ID_LIKE for derivatives we didn't name explicitly.
case " $like " in
*" alpine "*) OS_ID=alpine; OS_FAMILY=alpine ;;
*" debian "*|*" ubuntu "*) OS_ID=debian; OS_FAMILY=debian ;;
*" rhel "*|*" fedora "*|*" centos "*) OS_ID=alma; OS_FAMILY=rhel ;;
*) _die "Unsupported OS (ID='$id', ID_LIKE='$like'). Supported: Alpine, Debian, Alma." ;;
esac ;;
esac
case "$OS_FAMILY" in
alpine) INIT_SYSTEM=openrc ;;
*) INIT_SYSTEM=systemd ;;
esac
export OS_ID OS_FAMILY INIT_SYSTEM
}
_require_detected() { [[ -n "${OS_FAMILY:-}" ]] || os_detect; }
# ============================================================================
# Packages
# ============================================================================
pkg_update() {
_require_detected
case "$OS_FAMILY" in
alpine) apk update -q ;;
debian) apt-get update -qq ;;
rhel) dnf -q makecache || true ;;
esac
}
pkg_install() { # pkg_install <name>...
_require_detected
case "$OS_FAMILY" in
alpine) apk add -q "$@" ;;
debian) DEBIAN_FRONTEND=noninteractive apt-get install -y -qq "$@" ;;
rhel) dnf install -y -q "$@" ;;
esac
}
pkg_remove() { # pkg_remove <name>...
_require_detected
case "$OS_FAMILY" in
alpine) apk del -q "$@" 2>/dev/null || true ;;
debian) DEBIAN_FRONTEND=noninteractive apt-get remove -y -qq "$@" 2>/dev/null || true ;;
rhel) dnf remove -y -q "$@" 2>/dev/null || true ;;
esac
}
pkg_installed() { # pkg_installed <name> -> 0 if installed
_require_detected
case "$OS_FAMILY" in
alpine) apk info -e "$1" >/dev/null 2>&1 ;;
debian) dpkg -s "$1" >/dev/null 2>&1 ;;
rhel) rpm -q "$1" >/dev/null 2>&1 ;;
esac
}
# Logical package-name differences across families. Echo the right name(s).
pkg_name() { # pkg_name <logical>
_require_detected
case "$1" in
openssh-server)
# Alpine: the PAM-enabled variant so the session stack runs.
case "$OS_FAMILY" in alpine) echo openssh-server-pam ;; *) echo openssh-server ;; esac ;;
openssh-client)
case "$OS_FAMILY" in
alpine) echo openssh-client ;;
debian) echo openssh-client ;; # Debian/Ubuntu: singular
rhel) echo openssh-clients ;; # RHEL/Alma: plural
esac ;;
sftp-server)
# The external SFTP subsystem binary's package. Alpine ships it
# separately; Debian/Alma bundle it inside openssh-server.
case "$OS_FAMILY" in alpine) echo openssh-sftp-server ;; *) echo "" ;; esac ;;
*) echo "$1" ;;
esac
}
# ============================================================================
# Services (OpenRC vs systemd)
# ============================================================================
svc_enable() { # enable at boot
_require_detected
case "$INIT_SYSTEM" in
openrc) rc-update add "$1" default >/dev/null 2>&1 || true ;;
systemd) systemctl enable "$1" >/dev/null 2>&1 || true ;;
esac
}
svc_start() { _require_detected; [[ "$INIT_SYSTEM" == openrc ]] && rc-service "$1" start || systemctl start "$1"; }
svc_restart() { _require_detected; [[ "$INIT_SYSTEM" == openrc ]] && { rc-service "$1" restart || rc-service "$1" start; } || systemctl restart "$1"; }
svc_reload() { _require_detected; [[ "$INIT_SYSTEM" == openrc ]] && { rc-service "$1" reload || rc-service "$1" restart; } || { systemctl reload "$1" || systemctl restart "$1"; }; }
svc_enable_start() { svc_enable "$1"; svc_start "$1"; }
# The sshd service is named differently per distro.
sshd_service() {
_require_detected
[[ "$OS_FAMILY" == debian ]] && echo ssh || echo sshd
}
# ============================================================================
# SSH-specific paths
# ============================================================================
# External sftp-server binary path. The user asked for the external subsystem
# (not internal-sftp), and the path differs by distro.
sftp_server_path() {
_require_detected
local p
case "$OS_FAMILY" in
alpine) p=/usr/lib/ssh/sftp-server ;;
debian) p=/usr/lib/openssh/sftp-server ;;
rhel) p=/usr/libexec/openssh/sftp-server ;;
esac
# Verify; fall back to a search so an unusual layout still works.
if [[ ! -x "$p" ]]; then
local found
found="$(command -v sftp-server 2>/dev/null)" || true
[[ -z "$found" ]] && for c in /usr/lib/ssh/sftp-server /usr/lib/openssh/sftp-server \
/usr/libexec/openssh/sftp-server /usr/libexec/sftp-server; do
[[ -x "$c" ]] && { found="$c"; break; }
done
[[ -n "$found" ]] && p="$found"
fi
echo "$p"
}
# After writing sshd_config, Alpine's OpenRC init can regenerate RSA/ECDSA
# host keys on every start. Pin it off. No-op on systemd distros (they only
# generate host keys at install time, via ssh-keygen -A).
sshd_disable_keygen() {
_require_detected
[[ "$OS_FAMILY" == alpine ]] || return 0
[[ -f /etc/conf.d/sshd ]] || return 0
if grep -q '^sshd_disable_keygen=' /etc/conf.d/sshd; then
sed -i 's/^sshd_disable_keygen=.*/sshd_disable_keygen="yes"/' /etc/conf.d/sshd
else
echo 'sshd_disable_keygen="yes"' >> /etc/conf.d/sshd
fi
}
# ============================================================================
# Users & groups (busybox adduser/addgroup vs shadow useradd/groupadd)
# ============================================================================
group_add_system() { # group_add_system <group>
_require_detected
getent group "$1" >/dev/null && return 0
case "$OS_FAMILY" in
alpine) addgroup -S "$1" ;;
*) groupadd -r "$1" ;;
esac
}
user_add_to_group() { # user_add_to_group <user> <group>
_require_detected
case "$OS_FAMILY" in
alpine) addgroup "$1" "$2" 2>/dev/null || adduser "$1" "$2" 2>/dev/null || true ;;
*) usermod -aG "$2" "$1" ;;
esac
}
# Create a no-login user (for ssh jumpers) with the right nologin shell.
user_add_nologin() { # user_add_nologin <name>
_require_detected
getent passwd "$1" >/dev/null && return 0
case "$OS_FAMILY" in
alpine) adduser -D -s /sbin/nologin "$1" ;;
*) useradd -m -s /usr/sbin/nologin "$1" 2>/dev/null \
|| useradd -m -s /sbin/nologin "$1" ;;
esac
}
# Default interactive shell present on the distro (for admin users).
default_shell() {
_require_detected
if [[ -x /bin/bash ]]; then echo /bin/bash
elif [[ "$OS_FAMILY" == alpine ]]; then echo /bin/ash
else echo /bin/sh; fi
}
# Path to nologin (consistent across distros, but verify).
nologin_path() {
for p in /sbin/nologin /usr/sbin/nologin; do [[ -x "$p" ]] && { echo "$p"; return; }; done
echo /sbin/nologin
}
# ============================================================================
# Hostname
# ============================================================================
set_hostname() { # set_hostname <fqdn>
_require_detected
local fqdn="$1" short="${1%%.*}"
if command -v hostnamectl >/dev/null 2>&1; then
hostnamectl set-hostname "$fqdn"
else
# Alpine / no-systemd path.
echo "$short" > /etc/hostname # Alpine stores the short name
hostname "$short" 2>/dev/null || true
command -v rc-service >/dev/null 2>&1 && rc-service hostname restart >/dev/null 2>&1 || true
fi
# Maintain /etc/hosts so `hostname -f` resolves.
_update_etc_hosts "$fqdn" "$short"
}
# Echo the region segment of this host's FQDN (e.g. "us-evi-1" from
# ssh-1.us-evi-1.srvno.de), or nothing when the name carries no region
# (e.g. sto-1.srvno.de). Used to tag login notifications by location.
host_region() {
local fqdn seg
fqdn="$(hostname -f 2>/dev/null || hostname 2>/dev/null || echo)"
seg="$(printf '%s' "$fqdn" | cut -d. -f2)"
printf '%s' "$seg" | grep -qE '^[a-z]{2}-[a-z]{2,4}-[0-9]+$' && printf '%s' "$seg" || printf ''
}
_update_etc_hosts() { # <fqdn> <short>
local fqdn="$1" short="$2"
touch /etc/hosts
# Debian convention: 127.0.1.1 for the box's own FQDN.
local line="127.0.1.1 ${fqdn} ${short}"
if grep -qE '^127\.0\.1\.1[[:space:]]' /etc/hosts; then
sed -i "s|^127\.0\.1\.1[[:space:]].*|${line}|" /etc/hosts
else
printf '%s\n' "$line" >> /etc/hosts
fi
}
# ============================================================================
# Boot hooks -- run a script once at every boot, init-system agnostic.
# Used to (re)install the iptables INPUT->sshguard jump.
# ============================================================================
install_boot_hook() { # install_boot_hook <name> <path-to-script>
_require_detected
local name="$1" src="$2"
case "$INIT_SYSTEM" in
openrc)
install -m 0755 "$src" "/etc/local.d/${name}.start"
rc-update add local default >/dev/null 2>&1 || true
"/etc/local.d/${name}.start" || true ;;
systemd)
install -m 0755 "$src" "/usr/local/sbin/${name}"
cat > "/etc/systemd/system/${name}.service" <<UNIT
[Unit]
Description=${name} (oslib boot hook)
After=network.target
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/${name}
RemainAfterExit=yes
[Install]
WantedBy=multi-user.target
UNIT
systemctl daemon-reload
systemctl enable "${name}.service" >/dev/null 2>&1 || true
systemctl start "${name}.service" || true ;;
esac
}
# ============================================================================
# Daily scheduled jobs -- busybox crond (/etc/periodic/daily) on Alpine, a
# systemd timer elsewhere.
# ============================================================================
install_daily_job() { # install_daily_job <name> <script-src> [run-args...]
_require_detected
local name="$1" src="$2"; shift 2
local args="$*"
# The job scripts use bash; ensure it's present (Alpine images often lack it).
command -v bash >/dev/null 2>&1 || pkg_install bash || true
install -m 0755 "$src" "/usr/local/sbin/$name"
# Co-install oslib.sh so a script that sources it still works standalone.
local srcdir; srcdir="$(dirname "$src")"
[[ -f "$srcdir/oslib.sh" ]] && install -m 0644 "$srcdir/oslib.sh" /usr/local/sbin/oslib.sh
case "$INIT_SYSTEM" in
openrc)
command -v crond >/dev/null 2>&1 || pkg_install busybox-suid 2>/dev/null || true
cat > "/etc/periodic/daily/$name" <<HOOK
#!/bin/sh
exec /usr/local/sbin/$name $args
HOOK
chmod +x "/etc/periodic/daily/$name"
rc-update add crond default >/dev/null 2>&1 || true
rc-service crond status >/dev/null 2>&1 || rc-service crond start >/dev/null 2>&1 || true ;;
systemd)
cat > "/etc/systemd/system/$name.service" <<UNIT
[Unit]
Description=$name (daily job)
After=network-online.target
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/$name $args
UNIT
cat > "/etc/systemd/system/$name.timer" <<UNIT
[Unit]
Description=Run $name daily
[Timer]
OnCalendar=daily
Persistent=true
RandomizedDelaySec=1h
[Install]
WantedBy=timers.target
UNIT
systemctl daemon-reload
systemctl enable --now "$name.timer" >/dev/null 2>&1 || true ;;
esac
}
remove_daily_job() { # remove_daily_job <name>
_require_detected
case "$INIT_SYSTEM" in
openrc) rm -f "/etc/periodic/daily/$1" ;;
systemd) systemctl disable --now "$1.timer" >/dev/null 2>&1 || true
rm -f "/etc/systemd/system/$1.timer" "/etc/systemd/system/$1.service"
systemctl daemon-reload ;;
esac
rm -f "/usr/local/sbin/$1"
}
# ============================================================================
# Brute-force protection (sshguard) -- log source & firewall backend differ.
# ============================================================================
# A sshguard LOGREADER line appropriate for the distro's logging.
sshguard_logreader() {
_require_detected
local svc; svc="$(sshd_service)"
case "$OS_FAMILY" in
# Alpine uses busybox syslogd -> /var/log/messages (no journald).
alpine) echo "LANG=C tail -F -n0 /var/log/messages" ;;
# Debian & Alma run systemd-journald.
*) echo "LANG=C journalctl -afb -p info -n1 -u ${svc} -o cat" ;;
esac
}
# ============================================================================
# High-level installers (compose the primitives above; OS knowledge lives here)
# ============================================================================
# Install OpenSSH server + client + the external SFTP subsystem. On Alpine we
# swap the non-PAM server for the PAM-enabled one so the session stack runs.
install_openssh() {
_require_detected
if [[ "$OS_FAMILY" == alpine ]]; then
if pkg_installed openssh-server && ! pkg_installed openssh-server-pam; then
pkg_remove openssh-server
fi
fi
local sftp_pkg; sftp_pkg="$(pkg_name sftp-server)"
# shellcheck disable=SC2046
pkg_install $(pkg_name openssh-server) $(pkg_name openssh-client) ${sftp_pkg:+$sftp_pkg}
# Alpine needs linux-pam present for the PAM server build.
[[ "$OS_FAMILY" == alpine ]] && pkg_install linux-pam openrc
}
# Install sshguard + an iptables firewall backend. On RHEL/Alma sshguard lives
# in EPEL, so enable that first.
install_bruteforce_protection() {
_require_detected
case "$OS_FAMILY" in
alpine) pkg_install sshguard iptables ip6tables ;;
debian) pkg_install sshguard iptables ;;
rhel) pkg_install epel-release || true
pkg_install sshguard iptables ;;
esac
}
# ============================================================================
# gum (Charm TUI) -- multi-OS installer. Best-effort; callers that can fall
# back to CLI prompts should not treat failure as fatal.
# ============================================================================
ensure_gum() {
command -v gum >/dev/null 2>&1 && return 0
_require_detected
case "$OS_FAMILY" in
alpine)
apk add -q gum 2>/dev/null && return 0
# community repo may be disabled; enable it for this release, retry.
local rel mirror
rel="$(cut -d. -f1,2 < /etc/alpine-release 2>/dev/null || echo edge)"
mirror="https://dl-cdn.alpinelinux.org/alpine/v${rel}/community"
grep -qF "$mirror" /etc/apk/repositories 2>/dev/null || echo "$mirror" >> /etc/apk/repositories
apk update -q && apk add -q gum ;;
debian)
# Charm's apt repo.
pkg_install curl gnupg ca-certificates || true
mkdir -p /etc/apt/keyrings
curl -fsSL https://repo.charm.sh/apt/gpg.key | gpg --dearmor -o /etc/apt/keyrings/charm.gpg
echo "deb [signed-by=/etc/apt/keyrings/charm.gpg] https://repo.charm.sh/apt/ * *" \
> /etc/apt/sources.list.d/charm.list
apt-get update -qq && pkg_install gum ;;
rhel)
cat > /etc/yum.repos.d/charm.repo <<'REPO'
[charm]
name=Charm
baseurl=https://repo.charm.sh/yum/
enabled=1
gpgcheck=1
gpgkey=https://repo.charm.sh/yum/gpg.key
REPO
pkg_install gum ;;
esac
command -v gum >/dev/null 2>&1
}
# ============================================================================
# SSH login notifier (pam_exec -> ntfy)
# ============================================================================
# Install the ntfy login hook: drop the script in /opt/scripts, write
# /etc/ssh-notify.conf from the NTFY_* / NOTIFY_* environment, and add the
# pam_exec line to /etc/pam.d/sshd (idempotent). Requires curl.
#
# install_login_notifier <path-to-ntfy-ssh-login.sh>
#
# Honored env: NTFY_URL (required to be useful), NTFY_TOKEN, NTFY_EMAIL,
# NTFY_TITLE, NTFY_PRIORITY, NTFY_REGION, NOTIFY_GROUPS, NOTIFY_PRIORITY_MAP.
# Set NTFY_FORCE_CONF=1 to overwrite an existing /etc/ssh-notify.conf.
install_login_notifier() {
_require_detected
local src="$1"
[[ -f "$src" ]] || { _warn "notifier script not found: $src"; return 1; }
command -v curl >/dev/null 2>&1 || pkg_install curl || _warn "curl not installed; notifier needs it."
install -d -m 0755 /opt/scripts
install -m 0755 "$src" /opt/scripts/ntfy-ssh-login.sh
if [[ ! -f /etc/ssh-notify.conf || "${NTFY_FORCE_CONF:-0}" == "1" ]]; then
( umask 077
cat > /etc/ssh-notify.conf <<CONF
# Generated by oslib install_login_notifier -- $(date -u +%FT%TZ)
NTFY_URL="${NTFY_URL:-}"
NTFY_TOKEN="${NTFY_TOKEN:-}"
NTFY_EMAIL="${NTFY_EMAIL:-}"
NTFY_TITLE="${NTFY_TITLE:-Bastion Notification}"
NTFY_PRIORITY="${NTFY_PRIORITY:-min}"
NTFY_REGION="${NTFY_REGION:-}"
NOTIFY_GROUPS="${NOTIFY_GROUPS:-}"
NOTIFY_PRIORITY_MAP="${NOTIFY_PRIORITY_MAP:-}"
CONF
)
chmod 600 /etc/ssh-notify.conf
_log "Wrote /etc/ssh-notify.conf (mode 0600)."
else
_log "/etc/ssh-notify.conf exists; left untouched (set NTFY_FORCE_CONF=1 to replace)."
fi
# Wire pam_exec into the sshd PAM stack (idempotent).
local pam=/etc/pam.d/sshd
local line='session optional pam_exec.so /opt/scripts/ntfy-ssh-login.sh'
if [[ -f "$pam" ]]; then
grep -qF '/opt/scripts/ntfy-ssh-login.sh' "$pam" || echo "$line" >> "$pam"
_log "Enabled pam_exec login notifier in $pam."
else
_warn "$pam not found; add this line to your sshd PAM stack manually:"
_warn " $line"
fi
}
# Locate the sshguard iptables backend binary (path varies by packaging).
sshguard_backend() {
local c
for c in /usr/libexec/sshguard/sshg-fw-iptables \
/usr/libexec/sshg-fw-iptables \
/usr/lib/sshguard/sshg-fw-iptables \
/usr/libexec/sshguard/sshg-fw-nft \
/usr/sbin/sshg-fw-iptables; do
[[ -x "$c" ]] && { echo "$c"; return; }
done
# Sensible default if nothing matched; caller may warn.
echo /usr/libexec/sshg-fw-iptables
}
scripts/setup-host.sh
+166
View File
@@ -0,0 +1,166 @@
#!/usr/bin/env bash
#
# setup-host.sh -- set a host's name (per the Network Domain Name Schema) and
# install the shared SSH MOTD banner. Works on Alpine, Debian, and Alma via
# oslib.sh.
#
# Hostnames follow globals/Network Domain Name Schema.md. Our VMs skip the
# region code and use srvno.de as the base, so the form is:
# <service>-<n>.srvno.de e.g. sto-1.srvno.de
# from which we derive:
# Node ID = <SERVICE>-<N> e.g. STO-1 (short name, uppercased)
#
# The MOTD is rendered from globals/motd.txt. You edit the *content* there;
# this script draws the borders and computes every bit of padding, so the box
# stays aligned regardless of value length (the spacing problem, solved).
#
# Usage:
# bash setup-host.sh sto-1 # -> sto-1.srvno.de, Node ID STO-1
# HOST=dns-1 bash setup-host.sh
# HOST=web-2.srvno.de bash setup-host.sh # full FQDN accepted too
# BASE_DOMAIN=example.net HOST=app-1 bash setup-host.sh
# DATACENTER="Stockholm SE" HOST=sto-1 bash setup-host.sh
#
# Env:
# HOST required short name (sto-1) or FQDN (sto-1.srvno.de)
# BASE_DOMAIN srvno.de appended when HOST has no dot
# DATACENTER "Globally Everywhere" shown in the MOTD
# MOTD_TEMPLATE <repo>/globals/motd.txt
# MOTD_WIDTH 60 inner width of the MOTD box
# SET_HOSTNAME 1 set to 0 to render MOTD only (skip hostname)
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
# shellcheck source=scripts/oslib.sh
. "$REPO_ROOT/scripts/oslib.sh"
: "${HOST:=${1:-}}"
: "${BASE_DOMAIN:=srvno.de}"
: "${DATACENTER:=Globally Everywhere}"
: "${MOTD_TEMPLATE:=$REPO_ROOT/globals/motd.txt}"
: "${MOTD_WIDTH:=60}"
: "${MOTD_OUT:=/etc/motd}"
: "${SET_HOSTNAME:=1}"
[[ $EUID -eq 0 ]] || _die "Run as root."
[[ -n "$HOST" ]] || _die "Set HOST=<service>-<n> (e.g. sto-1) or a full FQDN."
os_detect
# ----------------------------------------------------------------------------
# Derive FQDN / short name / Node ID from HOST
# ----------------------------------------------------------------------------
if [[ "$HOST" == *.* ]]; then
FQDN="$HOST"; SHORT="${HOST%%.*}"
else
SHORT="$HOST"; FQDN="$HOST.$BASE_DOMAIN"
fi
NODE_ID="$(printf '%s' "$SHORT" | tr '[:lower:]' '[:upper:]')"
# Light sanity check against the schema (svc code + instance number). Warn
# only -- we don't want to block an intentional exception.
[[ "$SHORT" =~ ^[a-z]{2,4}-[0-9]+$ ]] \
|| _warn "Short name '$SHORT' doesn't match <svc>-<n> (see globals/Network Domain Name Schema.md)."
_log "Host: $FQDN"
_log "Node ID: $NODE_ID"
_log "Data Cntr: $DATACENTER"
# ----------------------------------------------------------------------------
# Set hostname (OS-gated inside oslib)
# ----------------------------------------------------------------------------
if [[ "$SET_HOSTNAME" == "1" ]]; then
_log "Setting hostname to $FQDN ..."
set_hostname "$FQDN"
fi
# ============================================================================
# MOTD renderer
# ============================================================================
# All padding is computed in ASCII (spaces, labels, values are ASCII so
# ${#str} is the column count). Box-drawing glyphs are only ever emitted by
# repetition, never measured -- so this is correct under any shell/locale.
W="$MOTD_WIDTH"
_spaces() { printf '%*s' "$1" ''; } # N ASCII spaces
_repeat() { local i; for ((i=0;i<$2;i++)); do printf '%s' "$1"; done; } # glyph xN
# Bordered content line: ┃ + <inner W cols> + ┃
_line() { printf '┃%s┃\n' "$1"; }
# Center an ASCII string within W.
_center() {
local s="$1" len=${#1} pad left
(( len > W )) && { s="${s:0:W}"; len=$W; }
pad=$(( W - len )); left=$(( pad / 2 ))
_line "$(_spaces "$left")$s$(_spaces $(( pad - left )))"
}
# Center a pre-built string of KNOWN visible width vis (may contain glyphs).
_center_known() {
local s="$1" vis="$2" pad left
pad=$(( W - vis )); left=$(( pad / 2 ))
_line "$(_spaces "$left")$s$(_spaces $(( pad - left )))"
}
# Field line: left margin, right-aligned label, ": ", value, pad to W.
LM=8 # left margin before the label column
_field() {
local label="$1" value="$2" content
content="$(printf '%*s%*s: %s' "$LM" '' "$LABELW" "$label" "$value")"
local len=${#content}
(( len > W )) && { content="${content:0:W}"; len=$W; }
_line "$content$(_spaces $(( W - len )))"
}
# Mini double-line box around centered ASCII text, itself centered in W.
_inner_box() {
local text="$1" tw=$(( ${#1} + 2 )) vis
vis=$(( tw + 2 )) # ║ + text(+2 spaces) + ║
_center_known "╔$(_repeat '═' "$tw")╗" "$vis"
_center_known "║ $text ║" "$vis"
_center_known "╚$(_repeat '═' "$tw")╝" "$vis"
}
render_motd() {
# First pass: compute LABELW (max @F@ label width) so colons align.
LABELW=0
local line tok rest label
while IFS= read -r line || [[ -n "$line" ]]; do
[[ "$line" =~ ^[[:space:]]*# ]] && continue
[[ "$line" == "@F@ "* ]] || continue
rest="${line#@F@ }"; label="${rest%%|*}"
(( ${#label} > LABELW )) && LABELW=${#label}
done < "$MOTD_TEMPLATE"
# Second pass: emit. Borders come from the @TOP@/@HR@/@BOT@ tokens.
while IFS= read -r line || [[ -n "$line" ]]; do
[[ "$line" =~ ^[[:space:]]*# ]] && continue
[[ -z "$line" ]] && continue
tok="${line%% *}"; rest="${line#"$tok"}"; rest="${rest# }"
# substitute placeholders
rest="${rest//\{\{HOSTNAME\}\}/$FQDN}"
rest="${rest//\{\{NODE_ID\}\}/$NODE_ID}"
rest="${rest//\{\{DATACENTER\}\}/$DATACENTER}"
case "$tok" in
@TOP@) printf '┏%s┓\n' "$(_repeat '━' "$W")" ;;
@BOT@) printf '┗%s┛\n' "$(_repeat '━' "$W")" ;;
@HR@) printf '┣%s┫\n' "$(_repeat '━' "$W")" ;;
@BLANK@) _line "$(_spaces "$W")" ;;
@C@) _center "$rest" ;;
@BOX@) _inner_box "$rest" ;;
@F@) _field "${rest%%|*}" "${rest#*|}" ;;
*) _warn "Unknown MOTD token: $tok (skipped)" ;;
esac
done < "$MOTD_TEMPLATE"
}
[[ -f "$MOTD_TEMPLATE" ]] || _die "MOTD template not found: $MOTD_TEMPLATE"
_log "Rendering MOTD -> $MOTD_OUT"
render_motd > "$MOTD_OUT"
chmod 644 "$MOTD_OUT" 2>/dev/null || true
_log "Done."
echo
cat "$MOTD_OUT"
scripts/ssh-notify.conf.example
+31
View File
@@ -0,0 +1,31 @@
# ssh-notify.conf -- config for /opt/scripts/ntfy-ssh-login.sh (the pam_exec
# SSH-login notifier). Installed at /etc/ssh-notify.conf (mode 0600 -- it may
# hold a token). The harden scripts generate it from NTFY_* env vars; this is
# the reference of every key.
# Destination ntfy topic URL. REQUIRED -- the notifier is a no-op if empty.
NTFY_URL="https://msg-1.srvno.de/canary"
# Bearer token for publishing. Optional: leave empty if the topic allows
# unauthenticated publish (e.g. read-gated). Kept here so it stays out of the
# script and the file can be 0600.
NTFY_TOKEN=""
# Optional ntfy headers.
NTFY_EMAIL="sysadmin@example.com"
NTFY_TITLE="Bastion Notification"
NTFY_PRIORITY="min"
# Region tag added to X-Tags so you can tell which bastion/location fired the
# alert (e.g. us-evi-1). The harden scripts default this to the region segment
# of the host's own FQDN when present.
NTFY_REGION="us-evi-1"
# --- Who to notify for (security-level filter) ---
# Space-separated group list: only notify when the logging-in user belongs to
# one of these groups. Empty => notify for every login.
NOTIFY_GROUPS="ssh-admins ssh-jumpers"
# Optional per-group priority overrides: "group:priority" entries. Lets you,
# e.g., page loudly for admins but whisper for routine jumpers.
NOTIFY_PRIORITY_MAP="ssh-admins:high ssh-jumpers:min"
scripts/sshuser.sh
+50 -16
View File
@@ -1,10 +1,14 @@
#!/usr/bin/env bash
#
# sshuser -- manage SSH users on a hardened Alpine box.
# sshuser -- manage SSH users on a hardened box (Alpine, Debian, or Alma).
#
# Two roles, matching harden-jumphost.sh:
# admin -> group ssh-admins, shell /bin/ash (full shell)
# jumper -> group ssh-jumpers, shell /sbin/nologin (ProxyJump only)
# admin -> group ssh-admins, interactive shell (full shell)
# jumper -> group ssh-jumpers, nologin shell (ProxyJump only)
#
# The interactive shell and nologin paths, and the user-management commands,
# differ per distro -- this script detects the OS and adapts (it's installed
# standalone, so it can't share scripts/oslib.sh).
#
# Two modes:
# - TUI (gum) : run with no command, or any command with missing args
@@ -12,7 +16,7 @@
#
# Install:
# install -m 0755 sshuser.sh /usr/local/bin/sshuser
# apk add gum # only needed for TUI mode
# gum is only needed for TUI mode (apk add gum / apt install gum / dnf install gum)
#
# Usage:
# sshuser # interactive TUI
@@ -30,8 +34,40 @@ set -euo pipefail
ADMIN_GROUP="ssh-admins"
JUMPER_GROUP="ssh-jumpers"
ADMIN_SHELL="/bin/ash"
# ---------------------------------------------------------------------------
# OS detection + per-distro user-management primitives. (Standalone install,
# so we can't source oslib.sh -- this is the minimal slice we need.)
# ---------------------------------------------------------------------------
_OS_FAMILY=alpine
if [[ -r /etc/os-release ]]; then
_osid="$(. /etc/os-release 2>/dev/null && echo "${ID:-}")"
_oslike="$(. /etc/os-release 2>/dev/null && echo "${ID_LIKE:-}")"
case " ${_osid} ${_oslike} " in
*" debian "*|*" ubuntu "*) _OS_FAMILY=debian ;;
*" rhel "*|*" fedora "*|*" centos "*|*" almalinux "*|*" rocky "*) _OS_FAMILY=rhel ;;
*" alpine "*) _OS_FAMILY=alpine ;;
esac
fi
# Admin (interactive) shell: ash on Alpine, bash elsewhere.
case "$_OS_FAMILY" in
alpine) ADMIN_SHELL="/bin/ash" ;;
*) ADMIN_SHELL="/bin/bash" ;;
esac
[[ -x "$ADMIN_SHELL" ]] || ADMIN_SHELL="/bin/sh"
# Nologin path: /usr/sbin/nologin on Debian, /sbin/nologin on Alpine/Alma.
JUMPER_SHELL="/sbin/nologin"
[[ -x "$JUMPER_SHELL" ]] || for _p in /usr/sbin/nologin /sbin/nologin; do
[[ -x "$_p" ]] && { JUMPER_SHELL="$_p"; break; }
done
user_create() { case "$_OS_FAMILY" in alpine) adduser -D -s "$2" -g "" "$1";; *) useradd -m -s "$2" "$1";; esac; }
user_join_group() { case "$_OS_FAMILY" in alpine) adduser "$1" "$2";; *) usermod -aG "$2" "$1";; esac; }
user_leave_group() { case "$_OS_FAMILY" in alpine) deluser "$1" "$2" 2>/dev/null || true;; *) gpasswd -d "$1" "$2" 2>/dev/null || true;; esac; }
user_delete() { case "$_OS_FAMILY" in alpine) deluser --remove-home "$1" 2>/dev/null || deluser "$1";; *) userdel -r "$1" 2>/dev/null || userdel "$1";; esac; }
set_user_shell() { usermod -s "$2" "$1" 2>/dev/null || chsh -s "$2" "$1" 2>/dev/null \
|| sed -i "s|^\($1:.*:\)[^:]*$|\1$2|" /etc/passwd; }
# ---------------------------------------------------------------------------
# Helpers
@@ -147,8 +183,8 @@ cmd_add() {
confirm "Create user $user as $role (group $group, shell $shell)?" || { warn "Aborted."; exit 1; }
log "Creating $user..."
adduser -D -s "$shell" -g "" "$user"
adduser "$user" "$group"
user_create "$user" "$shell"
user_join_group "$user" "$group"
if [[ -n "$key" ]]; then
local ak; ak=$(ssh_dir_setup "$user")
@@ -184,7 +220,7 @@ cmd_edit() {
"add ssh key") ADD_KEY_ARG=$(ask "Paste SSH public key") ;;
"remove ssh key") REMOVE_KEY_ARG=$(ask "Substring of key to remove (comment is fine)") ;;
"change role") ROLE_ARG=$(choose "New role" admin jumper) ;;
"change shell") SHELL_ARG=$(ask "New shell" "/bin/ash") ;;
"change shell") SHELL_ARG=$(ask "New shell" "$ADMIN_SHELL") ;;
*) warn "Cancelled."; return 0 ;;
esac
fi
@@ -195,16 +231,14 @@ cmd_edit() {
# Remove from the other ssh group, add to the target.
local other
[[ "$new_group" == "$ADMIN_GROUP" ]] && other="$JUMPER_GROUP" || other="$ADMIN_GROUP"
deluser "$user" "$other" 2>/dev/null || true
adduser "$user" "$new_group" 2>/dev/null || true
usermod -s "$new_shell" "$user" 2>/dev/null || \
sed -i "s|^\($user:.*:\)[^:]*$|\1$new_shell|" /etc/passwd
user_leave_group "$user" "$other"
user_join_group "$user" "$new_group"
set_user_shell "$user" "$new_shell"
fi
if [[ -n "${SHELL_ARG:-}" ]]; then
log "Setting $user shell to $SHELL_ARG"
usermod -s "$SHELL_ARG" "$user" 2>/dev/null || \
sed -i "s|^\($user:.*:\)[^:]*$|\1$SHELL_ARG|" /etc/passwd
set_user_shell "$user" "$SHELL_ARG"
fi
if [[ -n "${ADD_KEY_ARG:-}" ]]; then
@@ -238,7 +272,7 @@ cmd_remove() {
confirm "DELETE user $user and their home directory?" || { warn "Aborted."; exit 1; }
log "Deleting $user..."
deluser --remove-home "$user" 2>/dev/null || deluser "$user"
user_delete "$user"
log "Done."
}
@@ -294,7 +328,7 @@ cmd_show() {
}
cmd_tui() {
have_gum || err "TUI mode requires gum: 'apk add gum'. Or use CLI flags (sshuser --help)."
have_gum || err "TUI mode requires gum (apk/apt/dnf install gum). Or use CLI flags (sshuser --help)."
local action
action=$(choose "What do you want to do?" \
"add user" "edit user" "remove user" "list users" "show user" "quit")