Files
comfyui-nvidia/deployments/ai-stack/README.md
T
57_WolveandClaude Opus 4.7 21e976e275
release / Build & Push Docker Image (push) Successful in 31m43s
Externalise WEBUI_URL / LLM_URL to .env
So changing the deployment's hostnames is a one-file edit (.env) instead
of touching docker-compose.yml. WEBUI_URL is the full URL with scheme
(Open WebUI uses it for auth redirects); LLM_URL is the bare hostname
(Anubis wants it for COOKIE_DOMAIN).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-19 10:49:29 -05:00

184 lines
7.3 KiB
Markdown

# ai-stack — deployment
The full multi-service stack: Caddy (TLS + reverse proxy) in front of Open
WebUI (chat + image generation panel), Ollama (LLMs), and ComfyUI (image
generation), with an optional Anubis PoW anti-bot sidecar. One GPU host,
one bridge network, one TLS entry point.
This is the only supported deployment shape — sanitized snapshot of the
production `srvno.de` deployment.
## Files
| File | Purpose |
| ------------------- | -------------------------------------------------------- |
| `docker-compose.yml`| Service definitions, volumes, GPU reservations |
| `Caddyfile` | TLS + reverse proxy config (one site block per hostname) |
| `init-models.sh` | Models to preseed into Ollama on first boot |
| `.env.example` | Secrets and image-tag pins. Copy to `.env` |
## 1. Host prerequisites
- Linux (or WSL2) with an NVIDIA GPU and a recent driver.
- cu126 wheels (default Dockerfile): driver >= 545
- cu130 wheels (swap in Dockerfile): driver >= 580
- Docker Engine + Compose v2.
- [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
installed and the Docker runtime configured (`nvidia-ctk runtime configure
--runtime=docker && systemctl restart docker`).
- DNS for the chat / ComfyUI hostnames already pointing at this host
(Caddy needs working DNS to provision Let's Encrypt certs on first boot).
Confirm GPU passthrough works before bringing the stack up:
```sh
docker run --rm --gpus all nvidia/cuda:12.6.3-base-ubuntu24.04 nvidia-smi
```
## 2. Configure
```sh
cp .env.example .env
# generate the two keys with: openssl rand -hex 32
```
Then edit:
- **`.env`** — fill in:
- `WEBUI_URL` (full URL with scheme) and `LLM_URL` (bare hostname). Both
point at the same Open WebUI host; Open WebUI wants the URL form for
auth redirects, Anubis wants the bare hostname for its cookie domain.
- `WEBUI_SECRET_KEY` and (if using Anubis) `ANUBIS_OWUI_KEY` —
`openssl rand -hex 32` for each.
- Optionally pin `COMFYUI_IMAGE_TAG` to a specific `v*` release.
- **`Caddyfile`** — replace the `chat.example.com` and `comfyui.example.com`
hostnames with yours; replace `REPLACE_WITH_BCRYPT_HASH` with a real
bcrypt hash:
```sh
docker run --rm caddy:latest caddy hash-password --plaintext 'your-password'
```
- **`init-models.sh`** — keep the LLMs you want preseeded, drop the rest.
Check sizes at <https://ollama.com/library> first; the host needs disk
for everything listed.
## 3. Bring it up
```sh
docker compose up -d
docker compose logs -f
```
First boot: Caddy provisions Let's Encrypt certs, the `model-init`
container pulls the LLMs in `init-models.sh` (slow — `mistral-nemo:12b`
alone is ~7 GB), and ComfyUI initialises empty volumes.
Health-check:
```sh
docker compose exec comfyui curl -sf http://127.0.0.1:8188/system_stats | head -c 200
docker compose exec open-webui curl -sf http://127.0.0.1:8080/health
```
## 4. Drop in at least one ComfyUI checkpoint
ComfyUI ships no models. The shipped workflow templates reference
`v1-5-pruned-emaonly.safetensors` as a placeholder; drop any
SD/SDXL/Flux checkpoint into the `comfyui-models` volume under
`checkpoints/`:
```sh
docker run --rm -v ai-stack_comfyui-models:/models -w /models/checkpoints \
curlimages/curl:latest -L -O \
https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.safetensors
```
Or open the ComfyUI native UI at `https://comfyui.example.com` (after
basic-auth login), use the **Manager** button (added by ComfyUI-Manager),
and install one through **Model Manager**.
## 5. First-user signup in Open WebUI
Open `https://chat.example.com`. The first account created becomes the
admin. Subsequent signups land in `pending` and need admin approval (set
by `DEFAULT_USER_ROLE: pending` in compose).
## 6. Wire Open WebUI to ComfyUI
Open WebUI ships the ComfyUI integration but won't know which workflow to
submit until you paste one in. Do this once for txt2img, once for img2img.
In Open WebUI: **Admin Panel -> Settings -> Images**.
1. **Image Generation Engine** -> `ComfyUI` (preselected via env var).
2. **ComfyUI Base URL** -> `http://comfyui:8188` (preselected).
3. **ComfyUI Workflow** -> paste the entire contents of
[`../../workflows/txt2img.json`](../../workflows/txt2img.json).
4. **ComfyUI Workflow Nodes** -> paste the contents of
[`../../workflows/txt2img.nodes.json`](../../workflows/txt2img.nodes.json).
5. **Default Model** -> the filename of the checkpoint you dropped in
step 4 (e.g. `v1-5-pruned-emaonly.safetensors`).
6. Save.
For image editing (img2img), scroll to the **Image Editing** section in
the same panel and repeat with
[`../../workflows/img2img.json`](../../workflows/img2img.json) and
[`../../workflows/img2img.nodes.json`](../../workflows/img2img.nodes.json).
## 7. Test it
In any chat, click the image-generation button and prompt for an image.
Open WebUI submits the workflow to ComfyUI; the result drops back into
the chat when KSampler finishes. To test img2img, attach an image and
use the edit action.
## Enabling Anubis (later)
The `anubis-owui` service is defined in compose but no Caddy site block
points at it yet. To activate:
1. Generate a key: `openssl rand -hex 32` and set `ANUBIS_OWUI_KEY` in `.env`.
2. In `Caddyfile`, change `reverse_proxy open-webui:8080` to
`reverse_proxy anubis-owui:8923` for the chat hostname.
3. `docker compose up -d`.
## How the workflow node mappings work
Open WebUI doesn't introspect the workflow graph. The `*.nodes.json`
files tell it which node IDs and input fields to overwrite when the user
provides a prompt, image, seed, etc. Each entry:
```json
{ "type": "<placeholder>", "node_ids": ["<id>"], "key": "<input field>" }
```
Recognised `type` strings (per Open WebUI source): `model`, `prompt`,
`negative_prompt`, `width`, `height`, `n` (batch size), `steps`, `seed`,
and `image` (img2img / edit only).
If you swap in a fancier workflow (SDXL, Flux, ControlNet, custom
samplers, NL masking via SAM nodes, etc.), update the matching
`*.nodes.json` so the node IDs and input keys still line up.
## Common gotchas
- **"Model not found" in Open WebUI's image panel.** ComfyUI lists
models from `/opt/comfyui/models/checkpoints/`. Confirm the file is
there and that **Default Model** matches the filename exactly
(including extension).
- **Out-of-memory on first generate.** Lower `IMAGE_SIZE` in compose
(e.g. `768x768`) or pass `--lowvram` / `--medvram` in the Dockerfile
CMD and rebuild.
- **Custom nodes need extra pip packages.** Install via ComfyUI-Manager
(it pip-installs into the container's venv). The `custom_nodes`
volume persists, but `/opt/venv` does not — so packages installed by
the manager survive container restarts only because the manager
re-installs them on boot. For permanent custom-node deps, add a
`RUN pip install …` to the Dockerfile and rebuild.
- **GPU not visible inside container.** Re-run the `nvidia-smi` test in
step 1. If it fails, the toolkit is misconfigured.
- **Caddy can't get a cert.** First-boot ACME requires DNS A/AAAA
records pointing at this host's public IP and ports 80+443 reachable
from the internet. Check `docker compose logs caddy` for the specific
challenge failure.