Add Image Studio model preset — forces smart_image_gen tool use
A documented Open WebUI custom-model preset wrapping mistral-nemo:12b with: aggressive system prompt that mandates calling generate_image, only the smart_image_gen tool attached, native function calling, lower temperature for tool-call reliability. Users pick "Image Studio" from the chat-model dropdown when they want images. Solves the common case where general-purpose chat models describe an image in text instead of firing the tool — usually on conversational phrasings like "can you draw me…". The preset removes the ambiguity by giving the LLM exactly one job and one tool. Setup walkthrough in openwebui-models/image_studio.md; deployment README §9 points users at it as the recommended path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
9e22de0328
commit
41d571d8d1
2 files changed
+114
No files matched your search
@@ -17,6 +17,7 @@ production `srvno.de` deployment.
|
||||
| `init-models.sh` | LLMs to preseed into Ollama on first boot |
|
||||
| `comfyui-init-models.sh` | Checkpoints/VAEs/LoRAs to preseed into ComfyUI on first boot |
|
||||
| `openwebui-tools/smart_image_gen.py` | Tool that auto-routes image generation to the right SDXL checkpoint |
|
||||
| `openwebui-models/image_studio.md` | Dedicated chat-model preset — system prompt that forces tool use |
|
||||
| `.env.example` | Secrets and image-tag pins. Copy to `.env` |
|
||||
|
||||
## 1. Host prerequisites
|
||||
@@ -171,6 +172,23 @@ portrait of a cyberpunk samurai" — the LLM should call
|
||||
`generate_image(prompt=..., style="photo")`. The status bar shows
|
||||
"Routing to photo (CyberRealisticXLPlay…)" while it generates.
|
||||
|
||||
If the LLM responds in text instead of calling the tool, install the
|
||||
**Image Studio** chat-model preset (next section) — a dedicated model
|
||||
with a system prompt that removes the ambiguity.
|
||||
|
||||
## 9. (Recommended) Install the Image Studio model preset
|
||||
|
||||
General-purpose chat models often "describe" an image in text instead
|
||||
of firing the `generate_image` tool, especially on conversational
|
||||
phrasing ("can you draw me…", "I'd love a picture of…"). The
|
||||
**Image Studio** preset wraps `mistral-nemo:12b` in a system prompt
|
||||
that mandates tool use — every message is treated as an image request.
|
||||
|
||||
Setup (under 5 minutes): see
|
||||
[`openwebui-models/image_studio.md`](openwebui-models/image_studio.md).
|
||||
Users then pick **Image Studio** from the chat-model dropdown when
|
||||
they want to generate.
|
||||
|
||||
To extend (new checkpoint, new style):
|
||||
|
||||
- Add the filename to `comfyui-init-models.sh` so it gets pulled.
|
||||
|
||||
Reference in new issue
Block a user