Initial commit — ComfyUI on NVIDIA fronted by Open WebUI

Replaces the figment/segment/Forge stack with a single ComfyUI backend
fronted by Open WebUI's native ComfyUI integration. ComfyUI is built from
the official manual install for NVIDIA. Ships txt2img and img2img workflow
templates plus matching node-mapping JSONs that paste into Open WebUI's
admin panel.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
57_WolveandClaude Opus 4.7 committed 2026-04-19 10:21:43 -05:00
commit a73aa4a38a
10 files changed
+491

No files matched your search

+135
View File
@@ -0,0 +1,135 @@
# Setup
End-to-end walkthrough: clean host -> running stack -> first generated image
in Open WebUI.
## 1. Host prerequisites
- Linux (or WSL2) with an NVIDIA GPU and a recent driver.
- cu126 wheels (default Dockerfile): driver >= 545
- cu130 wheels (swap in Dockerfile): driver >= 580
- Docker Engine + Compose v2.
- [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
installed and the Docker runtime configured (`nvidia-ctk runtime configure
--runtime=docker && systemctl restart docker`).
Confirm GPU passthrough works before bringing the stack up:
```sh
docker run --rm --gpus all nvidia/cuda:12.6.3-base-ubuntu24.04 nvidia-smi
```
## 2. Configure environment
```sh
cp .env.example .env
```
Edit `.env`:
- `OPEN_WEBUI_URL` — public URL Open WebUI is reached at (used for auth
redirects). For local-only, leave the default.
- `OPEN_WEBUI_SECRET_KEY` — generate with `openssl rand -hex 32`. Keep stable;
rotating it logs every user out.
## 3. Build and start
```sh
docker compose up -d --build
```
First build pulls the CUDA base, PyTorch wheels, and ComfyUI source — expect
several minutes. `docker compose logs -f comfyui` should end with a line like
`To see the GUI go to: http://0.0.0.0:8188`.
Health-check both services:
```sh
curl -sf http://localhost:8188/system_stats | head -c 200
curl -sf http://localhost:3000/health
```
## 4. Add at least one checkpoint
ComfyUI ships no models. The shipped workflow templates reference
`v1-5-pruned-emaonly.safetensors` as a placeholder; drop any SD/SDXL/Flux
checkpoint into the `comfyui-models` volume under `checkpoints/`:
```sh
docker run --rm -v comfyui-nvidia_comfyui-models:/models -w /models/checkpoints \
curlimages/curl:latest -L -O \
https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.safetensors
```
Or open the ComfyUI web UI at <http://localhost:8188>, click the **Manager**
button (added by ComfyUI-Manager), and use **Model Manager** to install one
through the UI.
## 5. First-user signup in Open WebUI
Open <http://localhost:3000>. The first account created becomes the admin.
Subsequent signups land in `pending` and need admin approval (set by
`DEFAULT_USER_ROLE: pending` in compose).
## 6. Wire Open WebUI to ComfyUI
Open WebUI ships the ComfyUI integration but won't know which workflow to
submit until you paste one in. Do this once per workflow (txt2img and
img2img).
In Open WebUI: **Admin Panel -> Settings -> Images**.
1. **Image Generation Engine** -> `ComfyUI` (already preselected via env var).
2. **ComfyUI Base URL** -> `http://comfyui:8188` (already preselected).
3. **ComfyUI Workflow** -> paste the entire contents of
[`workflows/txt2img.json`](workflows/txt2img.json).
4. **ComfyUI Workflow Nodes** -> paste the contents of
[`workflows/txt2img.nodes.json`](workflows/txt2img.nodes.json).
5. **Default Model** -> the filename of the checkpoint you dropped in step 4
(e.g. `v1-5-pruned-emaonly.safetensors`).
6. Save.
For image editing (img2img), scroll to the **Image Editing** section in the
same panel and repeat with [`workflows/img2img.json`](workflows/img2img.json)
and [`workflows/img2img.nodes.json`](workflows/img2img.nodes.json).
## 7. Test it
In any chat, click the image-generation button and prompt for an image. Open
WebUI submits the workflow to ComfyUI; the result drops back into the chat
when KSampler finishes. To test img2img, attach an image and use the edit
action.
## How the workflow node mappings work
Open WebUI doesn't introspect the workflow graph. The `*.nodes.json` files
tell it which node IDs and input fields to overwrite when the user provides
a prompt, image, seed, etc. Each entry:
```json
{ "type": "<placeholder>", "node_ids": ["<id>"], "key": "<input field>" }
```
Recognised `type` strings (per Open WebUI source): `model`, `prompt`,
`negative_prompt`, `width`, `height`, `n` (batch size), `steps`, `seed`, and
`image` (img2img / edit only).
If you swap in a fancier workflow (SDXL, Flux, ControlNet, custom samplers,
NL masking via SAM nodes, etc.), update the matching `*.nodes.json` so the
node IDs and input keys still line up.
## Common gotchas
- **"Model not found" in Open WebUI's image panel.** ComfyUI lists models from
`/opt/comfyui/models/checkpoints/`. Confirm the file is there and matches
the **Default Model** field exactly (filename including extension).
- **Out-of-memory on first generate.** Lower `IMAGE_SIZE` in compose
(e.g. `768x768`) or pass `--lowvram` / `--medvram` in the Dockerfile CMD.
- **Custom nodes need extra pip packages.** Install via ComfyUI-Manager (it
pip-installs into the container's venv). Persisted because
`/opt/comfyui/custom_nodes` is a named volume — but the venv at `/opt/venv`
is not, so packages added by the manager survive container restarts only if
the manager re-installs them on boot. For permanent custom-node deps, add a
`RUN pip install ...` to the Dockerfile and rebuild.
- **GPU not visible inside container.** Re-run the `nvidia-smi` test in step
1. If it fails, the toolkit is misconfigured.