Add Image Studio model preset — forces smart_image_gen tool use

A documented Open WebUI custom-model preset wrapping mistral-nemo:12b
with: aggressive system prompt that mandates calling generate_image,
only the smart_image_gen tool attached, native function calling,
lower temperature for tool-call reliability. Users pick "Image Studio"
from the chat-model dropdown when they want images.

Solves the common case where general-purpose chat models describe an
image in text instead of firing the tool — usually on conversational
phrasings like "can you draw me…". The preset removes the ambiguity
by giving the LLM exactly one job and one tool.

Setup walkthrough in openwebui-models/image_studio.md; deployment
README §9 points users at it as the recommended path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
57_WolveandClaude Opus 4.7 committed 2026-04-19 12:54:13 -05:00
1 parent 9e22de0328
commit 41d571d8d1
2 files changed
+114

No files matched your search

+18
View File
@@ -17,6 +17,7 @@ production `srvno.de` deployment.
| `init-models.sh` | LLMs to preseed into Ollama on first boot |
| `comfyui-init-models.sh` | Checkpoints/VAEs/LoRAs to preseed into ComfyUI on first boot |
| `openwebui-tools/smart_image_gen.py` | Tool that auto-routes image generation to the right SDXL checkpoint |
| `openwebui-models/image_studio.md` | Dedicated chat-model preset — system prompt that forces tool use |
| `.env.example` | Secrets and image-tag pins. Copy to `.env` |
## 1. Host prerequisites
@@ -171,6 +172,23 @@ portrait of a cyberpunk samurai" — the LLM should call
`generate_image(prompt=..., style="photo")`. The status bar shows
"Routing to photo (CyberRealisticXLPlay…)" while it generates.
If the LLM responds in text instead of calling the tool, install the
**Image Studio** chat-model preset (next section) — a dedicated model
with a system prompt that removes the ambiguity.
## 9. (Recommended) Install the Image Studio model preset
General-purpose chat models often "describe" an image in text instead
of firing the `generate_image` tool, especially on conversational
phrasing ("can you draw me…", "I'd love a picture of…"). The
**Image Studio** preset wraps `mistral-nemo:12b` in a system prompt
that mandates tool use — every message is treated as an image request.
Setup (under 5 minutes): see
[`openwebui-models/image_studio.md`](openwebui-models/image_studio.md).
Users then pick **Image Studio** from the chat-model dropdown when
they want to generate.
To extend (new checkpoint, new style):
- Add the filename to `comfyui-init-models.sh` so it gets pulled.