57_WolveandClaude Opus 5
4643b77083
.env was seeded only when absent, so a re-run with a corrected value logged
".env exists; leaving it alone" and dropped it. That looked harmless because
compose reads the shell environment before .env: an exported BIND_ADDR did
narrow the bind for that run, `ss -ltn` confirmed it, and the deploy reported
success -- while .env still said 0.0.0.0.
The drift surfaces later. The documented update path is a plain
`docker compose up -d`, which has no such environment, falls back to .env, and
republishes an SSL-bumping intercepting proxy on every interface. Nothing
warned. TRUSTED_CIDR has the same shape: a tightened allow-list silently
reverts to whatever .env kept.
Record which runtime keys actually arrived in the environment BEFORE the ":="
defaults run -- PROXY_PORT especially, whose default is a non-empty 3128, so
afterwards an unset variable is indistinguishable from a supplied one. On a
re-run, write just those keys through with the existing set_env() (a targeted
per-key rewrite, not a file overwrite) and log each change. Keys not passed
that run are untouched, so hand-edits to .env survive.
Verified: narrowing BIND_ADDR updates .env and logs it; a re-run with nothing
exported leaves .env alone; re-passing identical values is a silent no-op; and
a hand-edited PROXY_PORT=8080 survives all three, which is what the
capture-before-defaults ordering exists for.
Found by an adversarial sweep for the openbao bug class (920edc5), then
confirmed by hand.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
144 lines
6.7 KiB
Markdown
144 lines
6.7 KiB
Markdown
# squid — SSL-bump caching forward proxy
|
|
|
|
A [Squid](https://www.squid-cache.org/) **forward proxy** that caches static
|
|
content to cut bandwidth across a fleet and **intercepts TLS** (SSL-bump) with a
|
|
locally-generated CA so HTTPS can be cached and inspected.
|
|
|
|
This is the one deployment that **breaks the repo's Caddy/Let's-Encrypt
|
|
convention on purpose**: it is not an inbound web service, has no public
|
|
hostname, and uses no ACME cert. Clients point `HTTP(S)_PROXY` at it and trust
|
|
its CA.
|
|
|
|
> ⚠️ **TLS interception — read first.** SSL-bump decrypts your clients' HTTPS.
|
|
> The CA private key it generates can impersonate **any** site to **any** client
|
|
> that trusts it. Only run this on networks and devices **you own and are
|
|
> authorized to inspect**. Keep `ssl/squid-ca-key.pem` secret. Cert-pinned and
|
|
> HSTS sites (banking, app stores) will break unless you **splice** them
|
|
> (passthrough, see below).
|
|
|
|
## What it does
|
|
|
|
- **Explicit forward proxy** on port `3128`. Clients set `http_proxy` /
|
|
`https_proxy` (transparent/intercepting mode is future work — see end).
|
|
- **SSL-bump**: peek at the TLS SNI → **splice** (passthrough, no decryption)
|
|
the domains in `splice-domains.txt` → **bump** (decrypt) everything else,
|
|
minting per-host leaf certs on the fly from the local CA.
|
|
- **Hostname-targeted caching** with wildcards, and a storage gate that **never
|
|
caches HTML or dynamic content**.
|
|
|
|
## Deploy
|
|
|
|
**Via the launcher** (from a clone or the one-liner): pick `deploy: squid`, then
|
|
answer the prompts (trusted CIDR, etc.).
|
|
|
|
**Standalone** (self-contained `deploy.sh`, scp'd to a host):
|
|
|
|
```bash
|
|
TRUSTED_CIDR=100.64.0.0/10 BIND_ADDR=100.64.0.1 SKIP_PROMPTS=1 bash deploy.sh
|
|
```
|
|
|
|
**Fresh VM**: paste [`cloud-init.yml`](cloud-init.yml) as user-data (it hardens
|
|
SSH first, then deploys).
|
|
|
|
The deploy is idempotent. On first run it builds the local image, generates the
|
|
CA into `ssl/` (never overwritten), seeds `.env`, registers the port with the
|
|
host firewall if present, and brings the stack up.
|
|
|
|
On a **re-run**, an existing `.env` is kept — except for values you pass
|
|
explicitly that run (`TRUSTED_CIDR`, `BIND_ADDR`, `PROXY_PORT`, `CACHE_SIZE_MB`,
|
|
`CACHE_ONLY_LISTED`), which are written through and logged. That matters because
|
|
Compose reads the shell environment *before* `.env`: without the write-through, a
|
|
re-run that narrowed `BIND_ADDR` would apply only to that run, and the next plain
|
|
`docker compose up -d` — which has no such environment — would fall back to the
|
|
old `.env` and republish the intercepting proxy on `0.0.0.0`. Values you do not
|
|
pass are left untouched, so hand-edits to `.env` survive.
|
|
|
|
## Point clients at it
|
|
|
|
```bash
|
|
export http_proxy=http://<host>:3128
|
|
export https_proxy=http://<host>:3128
|
|
```
|
|
|
|
Per-tool: apt → `Acquire::http(s)::Proxy "http://<host>:3128";`; dnf →
|
|
`proxy=http://<host>:3128` in `/etc/dnf/dnf.conf`; apk → `http_proxy` env or
|
|
`--proxy`.
|
|
|
|
**Trust the CA** (so bumped HTTPS validates) — distribute `ssl/squid-ca-cert.pem`:
|
|
|
|
| Client | Install |
|
|
|---|---|
|
|
| Debian/Ubuntu | `cp squid-ca-cert.pem /usr/local/share/ca-certificates/squid-ca.crt && update-ca-certificates` |
|
|
| Alpine | `cp squid-ca-cert.pem /usr/local/share/ca-certificates/squid-ca.crt && update-ca-certificates` |
|
|
| Alma/RHEL | `cp squid-ca-cert.pem /etc/pki/ca-trust/source/anchors/squid-ca.pem && update-ca-trust` |
|
|
|
|
Browsers, Java, and some language runtimes keep their own trust stores — import
|
|
there too if needed.
|
|
|
|
## Caching model
|
|
|
|
Two knobs and three lists, all in the stack dir; they are bind-mounted, so edit
|
|
then `docker compose restart`.
|
|
|
|
- **`cache-domains.txt`** — hostnames to cache hard (long TTL, force-cache past
|
|
`Cache-Control: private/no-store`). A **leading dot** is a subdomain wildcard:
|
|
`.ubuntu.com` matches `ubuntu.com` and every subdomain.
|
|
- **`cache-domains.regex`** — optional `dstdom_regex` patterns for wildcards
|
|
*inside* a label (e.g. `^mirror[0-9]+\.example\.com$`). Comments-only = disabled.
|
|
- **`CACHE_ONLY_LISTED`** (`.env`):
|
|
- `0` (default, *boost*): cache everything per normal HTTP rules, and
|
|
force-cache the listed domains aggressively.
|
|
- `1` (*strict allowlist*): store **only** the listed domains; pass the rest
|
|
through uncached.
|
|
|
|
**Never cached** (storage gate, applies even to boosted domains): HTML (by
|
|
`.html` extension and `text/html` content-type) and dynamic content (script
|
|
endpoints + query strings). Query strings are **exempt on boosted domains**, so
|
|
versioned static assets like `app.js?v=123` still cache there.
|
|
|
|
### splice ⇄ cache are mutually exclusive
|
|
|
|
A spliced domain is passed through encrypted — there is nothing to cache or
|
|
inspect. **Do not list the same domain in both** `splice-domains.txt` and
|
|
`cache-domains.txt`; splice wins.
|
|
|
|
## Security posture
|
|
|
|
- **Access control is Squid's `http_access`** (`TRUSTED_CIDR`), deny-by-default.
|
|
This matters because a **published Docker port bypasses the host `INPUT`
|
|
firewall** — so also pin **`BIND_ADDR`** to a trusted interface (e.g. your
|
|
Tailscale IP). The `/etc/firewall/ports.d/squid.rule` entry is belt-and-braces.
|
|
- **CA key** lives at `ssl/squid-ca-key.pem`, mode `0600` root, mounted
|
|
read-only; the container stages a squid-readable copy into `/run` (tmpfs) at
|
|
start. The key never enters the embedded archive and is git-ignored.
|
|
- **Upstream certs are validated** (`sslproxy_cert_error deny all`) — the proxy
|
|
won't silently launder a broken origin certificate to clients.
|
|
|
|
## Caching caveats (be realistic)
|
|
|
|
Even with bump, much of the web is `Cache-Control: private/no-store`, dynamic,
|
|
or personalized. The real wins are **distro packages** (apk/apt/dnf),
|
|
**container layers**, **OS updates**, and large static assets across many hosts
|
|
— which is what the default `cache-domains.txt` targets. It is not a blanket
|
|
"cache the whole internet."
|
|
|
|
## Files
|
|
|
|
| File | Purpose |
|
|
|---|---|
|
|
| `Dockerfile` | Minimal Alpine image (`apk add squid` — ssl-bump compiled in). |
|
|
| `entrypoint.sh` | Renders `squid.conf`, generates the cache policy from the lists, stages the CA, inits the cert DB/cache, starts squid. |
|
|
| `squid.conf.tmpl` | Static config (ports, bump policy, access control, cache sizing). |
|
|
| `cache-domains.txt` / `.regex` | Wildcard hostnames to cache hard. |
|
|
| `splice-domains.txt` | Domains to pass through without decrypting. |
|
|
| `docker-compose.yml` / `.env.example` | Stack definition + tunables. |
|
|
| `deploy.sh` / `build.sh` | Self-contained installer + archive embedder. |
|
|
| `cloud-init.yml` | Fresh-VM bootstrap (harden SSH, then deploy). |
|
|
|
|
## Future work
|
|
|
|
- **Transparent / intercepting mode** — add `intercept`/`tproxy` ports and an
|
|
iptables `REDIRECT` recipe for when the box is the gateway (clients need no
|
|
proxy config). Out of scope for v1.
|
|
- Upstream `cache_peer` chaining; access-log shipping / dashboard.
|