Files
automations/deployments/openbao/README.md
T
57_WolveandClaude Opus 5 920edc50b3 fix(openbao): validate the publish bind before writing any state
deploy.sh narrows the Docker publish bind to OPENBAO_ADDR when that is an
IP, but never checked that the host actually holds that address. A typo'd
IP therefore failed late, inside `docker compose up`:

  failed to bind host port 192.160.100.50:8200/tcp:
  cannot assign requested address

...by which point .env had been seeded and the TLS cert generated with the
bad address in its SAN. Neither is rewritten on a re-run (.env is never
overwritten, gen-tls.sh never regenerates over an existing pair), so
re-running with a corrected OPENBAO_ADDR silently changed nothing.

Add host_addrs() + check_bind_addr(), run before anything is written:
- lists the host's addresses from plain `ip addr show` -- no -o/scope
  filters, since busybox ip supports neither -- falling back to ifconfig,
  and skipping the check when neither exists rather than blocking;
- skips 0.0.0.0 / :: / *, and unwraps an [IPv6] publish literal;
- SKIP_BIND_CHECK=1 overrides for an address that only comes up later.

Resolve the bind compose will really interpolate, which follows compose's
own precedence -- shell environment before .env:
- exported (automations.sh passes answers via `env VAR=...`, or a
  standalone OPENBAO_BIND=... run): the environment wins, so warn when
  .env disagrees, because a later bare `docker compose up` would not;
- derived here: that assignment is not exported, so .env wins;
- in neither: compose falls back to 0.0.0.0 and publishes the API on
  every interface -- warn, since that is a silent exposure.

Also warn when .env's OPENBAO_ADDR differs from this run's, naming the
cert that has to be deleted for the SAN to be regenerated.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 13:09:23 -05:00

8.8 KiB

openbao

Hardened OpenBao — the tape-encryption key store for Kanrisha (the LTO tape-archive system; separate repo, separate host). Deliberately separate from the tape host: a compromise of the tape node must not reach the vault, and OpenBao's mlock/TLS/unseal lifecycle is cleaner on its own box.

Unlike the other stacks here there is no Caddy / Let's Encrypt — a secrets store terminates TLS itself and is reached over the LAN, not the public internet.

  • Native TLS on the listener — self-signed by default, or a CA-signed cert from your Smallstep CA over ACME.
  • Integrated raft storage — snapshot-based DR.
  • mlock on — key material never hits swap.
  • Manual unseal by default, or PKCS#11 HSM auto-unseal.

⚠️ This vault is the sole recovery path for encrypted tapes. Losing the OpenBao data and the unseal keys/root token loses every encrypted tape. Take raft snapshots and store the unseal material out of band (below).

Required .env values

Variable Notes
OPENBAO_ADDR IP/DNS the Kanrisha tape host uses to reach this vault (goes in the cert SAN; you point the daemon at https://$OPENBAO_ADDR:8200).
OPENBAO_BIND Host interface the API publishes on. Left at the default, deploy.sh narrows it to OPENBAO_ADDR when that is an IP (a published port bypasses the INPUT firewall, so this bind is the real restriction — see Exposure). It must be an address this host actually holds — deploy.sh checks that before writing anything and refuses otherwise (SKIP_BIND_CHECK=1 to override).
OPENBAO_TLS_SANS Extra SANs beyond OPENBAO_ADDR + loopback (which deploy.sh always adds). Read from the environment at deploy time — export it before running deploy.sh.
OPENBAO_TAG OpenBao image tag (pin it).
OPENBAO_HSM_PIN Only for PKCS#11 auto-unseal. Leave blank for manual unseal.

See .env.example for the full list.

Deploy

./automations.sh        # Deploy on this host → deploy: openbao
# or, non-interactive:
OPENBAO_ADDR=10.0.0.10 SKIP_PROMPTS=1 bash deployments/openbao/deploy.sh

deploy.sh installs Docker, generates a self-signed TLS cert (if none present), seeds .env, disables swap (for mlock), narrows the API bind to the LAN IP, opens 8200/tcp, aligns file/volume ownership to the container's UID, and brings the stack up. OpenBao starts sealed — initialise + unseal once (do this immediately; an uninitialised vault reachable on the LAN can be init'd by anyone who connects):

docker compose exec -e BAO_ADDR=https://127.0.0.1:8200 openbao \
  bao operator init -tls-skip-verify           # prints 5 unseal keys + root token
docker compose exec -e BAO_ADDR=https://127.0.0.1:8200 openbao \
  bao operator unseal -tls-skip-verify <key>   # x3, three different keys

Store the unseal keys + root token out of band — ideally age-encrypted with your backup recipient (globals/age-pubkey.txt), never on this host.

Re-running after a wrong address

deploy.sh is idempotent, but two things it writes are deliberately sticky: .env (never overwritten) and tls/tls.{crt,key} (never regenerated over an existing pair). So passing a corrected OPENBAO_ADDR to a re-run does not move the vault — the cert keeps the old SAN, and the old .env usually still decides the bind. Which OPENBAO_BIND wins follows Compose's own precedence:

How OPENBAO_BIND is set What Compose uses
Exported into deploy.sh's environment (OPENBAO_BIND=… bash deploy.sh, or answering the bind prompt — automations.sh passes answers via env VAR=…) the environment value; .env is ignored for this run, so a later hand-run docker compose up -d can bind somewhere else
Derived by deploy.sh (bind prompt left blank → narrowed to OPENBAO_ADDR) the .env value, since that assignment is never exported
Absent from both 0.0.0.0all interfaces

deploy.sh warns on each of those mismatches. To actually change the address:

cd /srv/openbao
docker compose down                                   # keeps the raft volume
sed -i 's/OLD_IP/NEW_IP/g' .env                       # OPENBAO_ADDR + OPENBAO_BIND
rm -f tls/tls.crt tls/tls.key                         # force a new SAN
# then re-run deploy.sh with the corrected OPENBAO_ADDR

A bind address the host does not hold is caught up front; without that check Docker fails the up with cannot assign requested address only after the bad value is already in .env and the cert.

TLS

  • Self-signed (default): deploy.sh runs gen-tls.sh to create ./tls/tls.{crt,key} with OPENBAO_ADDR in the SAN. Hand tls.crt to the Kanrisha daemon as [encryption.openbao].ca_cert.

  • Smallstep CA over ACME (option): issue a cert from your step-ca and drop it in ./tls instead — gen-tls.sh then no-ops. e.g. with the step client:

    step ca certificate "$OPENBAO_ADDR" ./tls/tls.crt ./tls/tls.key \
      --provisioner acme --acme https://ca.lan/acme/acme/directory
    # renew on a timer:  step ca renew --daemon ./tls/tls.crt ./tls/tls.key
    

    Give the Kanrisha daemon your Smallstep root as ca_cert (then it trusts the vault without -tls-skip-verify).

Auto-unseal (optional)

Default is manual unseal after each restart. For hands-off restarts, enable the seal "pkcs11" stanza in config.hcl, mount the PKCS#11 module + device into the openbao service, and set OPENBAO_HSM_PIN in .env.

Bootstrap for Kanrisha

Once unsealed, enable KV v2 + AppRole and seed the domain key with the Kanrisha bootstrap script (from the Kanrisha repo, deploy/openbao/bootstrap.sh). It calls the bao CLI directly, so run it from a host that has bao (the Kanrisha host or your workstation) pointed at this vault — this vault host only ships Docker. Copy tls.crt to that host first and pass it as BAO_CACERT:

BAO_ADDR=https://$OPENBAO_ADDR:8200 BAO_CACERT=/path/to/openbao-ca.crt \
  BAO_TOKEN=<root> bash bootstrap.sh

It enables the kanrisha-tape KV-v2 mount (with effectively-unlimited max_versions so a key rotation never orphans old tapes), creates the AppRole + policy, seeds the domain key, and prints the [encryption.openbao] block for the Kanrisha config. Point the daemon at address = "https://$OPENBAO_ADDR:8200".

Exposure

This is a secrets store on the LAN, not a public service. Two things to know:

  • A Docker-published port is DNAT'd and bypasses the host INPUT firewall. A source-restricting rule on INPUT does not gate :8200. The real interface restriction is the publish bind: deploy.sh defaults OPENBAO_BIND to OPENBAO_ADDR when that is an IP, so the API listens only on that LAN IP. To restrict by source host, use a FORWARD/DOCKER-USER rule or network segmentation, not INPUT.
  • mTLS (optional): to require the tape host to present a client cert, enable the tls_require_and_verify_client_cert stanza in config.hcl and issue the tape host a client cert from the same CA.

And initialise the vault immediately after deploy.sh — an uninitialised vault reachable on the LAN can be bao operator init'd by anyone who connects, handing them the root token and unseal keys.

Backup / DR

The vault is the sole recovery path for encrypted tapes — back it up:

Snapshot save/restore are token-gated (sys/storage/raft/snapshot is sudo-capable) — pass a token that has that path (the root token works, or mint a dedicated snapshot-policy token). The container has no ambient token, so supply it via -e BAO_TOKEN.

# Consistent raft snapshot (safe while running); write it to a scratch path, NOT
# into the live raft dir:
docker compose exec -T -e BAO_TOKEN=<token> openbao \
  bao operator raft snapshot save -address=https://127.0.0.1:8200 -tls-skip-verify /tmp/openbao.snap
# then stream the RAW file off-box, age-encrypted. Use `exec -T ... cat`, not
# `compose cp openbao:… -` (which emits a TAR wrapper that won't restore):
docker compose exec -T openbao cat /tmp/openbao.snap | \
  age -r "$(cat /path/to/globals/age-pubkey.txt)" > "openbao-$(date +%F).snap.age"

Restore (into a fresh, unsealed vault): decrypt, copy the raw .snap in, and apply it (also token-gated):

age -d -i ~/.age/key.txt openbao-DATE.snap.age > openbao.snap
docker compose cp openbao.snap openbao:/tmp/openbao.snap
docker compose exec -T -e BAO_TOKEN=<token> openbao \
  bao operator raft snapshot restore -address=https://127.0.0.1:8200 -tls-skip-verify /tmp/openbao.snap

Snapshots do not contain the unseal keys or root token — you still need those to unseal a restored vault, which is why they are stored separately.