comfyui-nvidia/SETUP.md

Setup

End-to-end walkthrough: clean host -> running stack -> first generated image in Open WebUI.

1. Host prerequisites

• Linux (or WSL2) with an NVIDIA GPU and a recent driver.
  • cu126 wheels (default Dockerfile): driver >= 545
  • cu130 wheels (swap in Dockerfile): driver >= 580
• Docker Engine + Compose v2.
• NVIDIA Container Toolkit installed and the Docker runtime configured (nvidia-ctk runtime configure --runtime=docker && systemctl restart docker).

Confirm GPU passthrough works before bringing the stack up:

docker run --rm --gpus all nvidia/cuda:12.6.3-base-ubuntu24.04 nvidia-smi

2. Configure environment

cp .env.example .env

Edit .env:

• OPEN_WEBUI_URL — public URL Open WebUI is reached at (used for auth redirects). For local-only, leave the default.
• OPEN_WEBUI_SECRET_KEY — generate with openssl rand -hex 32. Keep stable; rotating it logs every user out.

3. Start the stack

docker compose up -d

This pulls the prebuilt comfyui-nvidia image from the Gitea registry (CI publishes it on every v* tag) plus the upstream Open WebUI image, then brings both up. docker compose logs -f comfyui should end with a line like To see the GUI go to: http://0.0.0.0:8188.

To build the image locally instead (e.g. while iterating on the Dockerfile):

docker compose build comfyui
docker compose up -d

First local build pulls the CUDA base, PyTorch wheels, and ComfyUI source — expect several minutes.

Health-check both services:

curl -sf http://localhost:8188/system_stats | head -c 200
curl -sf http://localhost:3000/health

4. Add at least one checkpoint

ComfyUI ships no models. The shipped workflow templates reference v1-5-pruned-emaonly.safetensors as a placeholder; drop any SD/SDXL/Flux checkpoint into the comfyui-models volume under checkpoints/:

docker run --rm -v comfyui-nvidia_comfyui-models:/models -w /models/checkpoints \
    curlimages/curl:latest -L -O \
    https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.safetensors

Or open the ComfyUI web UI at http://localhost:8188, click the Manager button (added by ComfyUI-Manager), and use Model Manager to install one through the UI.

5. First-user signup in Open WebUI

Open http://localhost:3000. The first account created becomes the admin. Subsequent signups land in pending and need admin approval (set by DEFAULT_USER_ROLE: pending in compose).

6. Wire Open WebUI to ComfyUI

Open WebUI ships the ComfyUI integration but won't know which workflow to submit until you paste one in. Do this once per workflow (txt2img and img2img).

In Open WebUI: Admin Panel -> Settings -> Images.

1. Image Generation Engine -> ComfyUI (already preselected via env var).
2. ComfyUI Base URL -> http://comfyui:8188 (already preselected).
3. ComfyUI Workflow -> paste the entire contents of workflows/txt2img.json.
4. ComfyUI Workflow Nodes -> paste the contents of workflows/txt2img.nodes.json.
5. Default Model -> the filename of the checkpoint you dropped in step 4 (e.g. v1-5-pruned-emaonly.safetensors).
6. Save.

For image editing (img2img), scroll to the Image Editing section in the same panel and repeat with workflows/img2img.json and workflows/img2img.nodes.json.

7. Test it

In any chat, click the image-generation button and prompt for an image. Open WebUI submits the workflow to ComfyUI; the result drops back into the chat when KSampler finishes. To test img2img, attach an image and use the edit action.

How the workflow node mappings work

Open WebUI doesn't introspect the workflow graph. The *.nodes.json files tell it which node IDs and input fields to overwrite when the user provides a prompt, image, seed, etc. Each entry:

{ "type": "<placeholder>", "node_ids": ["<id>"], "key": "<input field>" }

Recognised type strings (per Open WebUI source): model, prompt, negative_prompt, width, height, n (batch size), steps, seed, and image (img2img / edit only).

If you swap in a fancier workflow (SDXL, Flux, ControlNet, custom samplers, NL masking via SAM nodes, etc.), update the matching *.nodes.json so the node IDs and input keys still line up.

Common gotchas

• "Model not found" in Open WebUI's image panel. ComfyUI lists models from /opt/comfyui/models/checkpoints/. Confirm the file is there and matches the Default Model field exactly (filename including extension).
• Out-of-memory on first generate. Lower IMAGE_SIZE in compose (e.g. 768x768) or pass --lowvram / --medvram in the Dockerfile CMD.
• Custom nodes need extra pip packages. Install via ComfyUI-Manager (it pip-installs into the container's venv). Persisted because /opt/comfyui/custom_nodes is a named volume — but the venv at /opt/venv is not, so packages added by the manager survive container restarts only if the manager re-installs them on boot. For permanent custom-node deps, add a RUN pip install ... to the Dockerfile and rebuild.
• GPU not visible inside container. Re-run the nvidia-smi test in step
  1. If it fails, the toolkit is misconfigured.