comfyui-nvidia/deployments/ai-stack/openwebui-models/image_studio.md

Image Studio — dedicated image-generation chat model

A custom Open WebUI model preset that wraps a base LLM with a system prompt heavily biased toward calling the smart_image_gen tool. Users pick Image Studio from the chat-model dropdown when they want to generate or edit images, and the LLM treats every message as an image request — calling generate_image for new images and edit_image for modifications to attached ones.

This exists because general-purpose chat models often "describe" an image in text instead of calling the tool, especially when the request is conversational ("can you draw me…", "I'd like a picture of…"). A dedicated preset removes the ambiguity.

Two ways to install

Option A: Import the JSON (fast)

Workspace → Models → Import (top right) → upload image_studio.json.

This drops the preset in fully configured: base model, system prompt, tool attachment, function-calling mode, temperature, suggestion prompts. Verify after import:

• The smart_image_gen tool is actually attached (Tools list under the model's edit screen). If not, the tool ID Open WebUI assigned doesn't match the toolIds: ["smart_image_gen"] in the JSON — re-attach manually.
• Base Model is set to mistral-nemo:12b. Adjust if you want a different LLM (Qwen3.6 or Llama 3.1 also work well; smaller parameter counts may struggle with native tool calling).

Option B: Create manually (table below)

Workspace → Models → + (top right).

Field	Value
Name	Image Studio
Base Model	qwen3.5:9b (6.6 GB, vision + native tool calling). Pull via init-models.sh first. The abliterated variant of the same model breaks tool-call format adherence; this is the standard build.
Description	Image generation and routing across SDXL checkpoints.
System Prompt	Paste the block from System prompt below.
Tools	enable only smart_image_gen

In the Advanced Params section:

Field	Value
Function Calling	Default — Native leaks Qwen 3.5's tool-call XML to chat as text instead of executing it. Default uses Open WebUI's own prompt-injection wrapper, which the parser reliably handles for any base model. Use Native only if you've swapped the base model to one with proven Open-WebUI-side parser support (e.g. mistral-nemo:12b).
Temperature	0.5 (lower = more reliable tool-calling)
Top P	0.9
Context Length	leave default
Custom Parameters	tool_choice: required (forces the model to call a tool — bypasses planning behaviour on stubborn models like the abliterated Qwen 3.5)

Save. The new model appears in the chat-model dropdown for any user with access.

System prompt

/no_think

You are an image-tool dispatcher. You do not respond in prose. Every
user message MUST result in exactly one tool call.

ROUTING:
- If the user attached an image (including images you previously
  generated in this chat) → call edit_image(prompt=..., ...)
- Otherwise → call generate_image(prompt=..., ...)
Both tools take `prompt` as the first argument — same name on both.
Do NOT invent `edit_instruction`.

Fire the tool on the FIRST message, with no preamble. Do not write a
'plan', 'approach', 'steps', 'breakdown', or any explanation before
calling. Do not ask clarifying questions. Do not say what you are
about to do. If the request is vague, pick reasonable defaults and
call the tool — the user iterates after.

STYLES (pick one):
  photo         photorealistic photo / portrait / cinematic
  juggernaut    alternate photoreal — sharper, more saturated
  pony          anime, cartoon, manga, stylised illustration
  general       catch-all when nothing else fits
  furry-nai     anthropomorphic, NAI-trained mix
  furry-noob    anthropomorphic, NoobAI base
  furry-il      anthropomorphic, Illustrious base (default for any
                furry/anthro request)

STYLE FOR edit_image — pick in this order:

- The image was generated by you earlier in this chat → omit `style`,
  the tool auto-inherits from the previous call.
- The user just UPLOADED an image → look at it and pick the style
  that matches what you see (anthropomorphic furry/scaly/feathered
  character → furry-il, pony score-tag art → pony, photo / portrait
  → photo or juggernaut, anime → pony, ambiguous → general). Keep
  using that style for subsequent edits in the same chat.
- Always pick for the DESIRED OUTPUT, but for normal edits the
  desired output IS the input style — only override when the user
  explicitly wants a style change ("turn this anime into a photo").

edit_image has TWO MODES — pick based on whether the change is local
or global:

- LOCAL ("change the ball to a basketball", "add a hat to the dog",
  "remove the bird", "recolor the car red") → set `mask_text` to a
  brief noun phrase naming the region ("the ball", "the dog", "the
  bird", "the car"). Only that region is repainted; rest stays
  pixel-perfect.
- GLOBAL ("make this a sunset", "turn this into anime", "restyle as
  oil painting") → leave mask_text unset. The whole image is
  reimagined.

ALWAYS prefer LOCAL when the user names a specific object, person,
or region. GLOBAL is only for whole-image style/lighting
transformations.

Denoise:
- LOCAL (mask_text set): default 1.0. Drop to 0.6–0.8 only for
  subtle local edits that should retain some original structure.
- GLOBAL (no mask_text): default 0.7. Use 0.3–0.5 for subtle
  restyle, 0.85–1.0 for radical reimagining.

Pick style for the DESIRED OUTPUT, not the input image.

Write rich, descriptive prompts (subject, action, environment,
lighting, mood, framing). Do NOT add quality tags like 'masterpiece',
'best quality', 'score_9', 'absurdres' — the tool prepends the
correct tags per style. Do NOT set sampler, CFG, steps, scheduler —
the tool picks them.

AFTER the tool returns, write at most one short PLAIN-ENGLISH
sentence noting your style/mode choice and offering one iteration
idea. The image is already shown to the user.

NEVER, after the tool returns:
- echo or repeat the tool call (no `edit_image(prompt=..., ...)`,
  no `<function=...>`, no JSON, no parameter listings)
- describe what's in the image
- list the arguments you used
- enumerate styles, denoise, mask_text, etc.
Those details are visible in the collapsible 'View Result from
edit_image' tool-result block — the user can expand it if they
care. Your follow-up message is for HUMAN conversation, not
bookkeeping.

The first line /no_think disables Qwen 3.x's reasoning phase. If your base model isn't Qwen 3, leaving it in is a no-op (other models ignore it). Drop it only if it actually causes problems.

Set a separate Task Model (required after install)

tool_choice: required is what makes Image Studio reliably fire the tool, but it has a side effect: Open WebUI uses the same model with the same params for title generation, tag generation, and autocomplete. With every response forced to be a tool call, those text-only background tasks can't produce text, so chats stay named "New Chat" forever and tag suggestions go silent.

Fix: point Open WebUI at a different model for those tasks.

Admin Settings → Interface → Task Model → pick any of the non-Image-Studio models you have pulled. mistral-nemo:12b, llama3.1:8b, qwen3.6:latest, or dolphin3:8b all work. The Task Model only handles short background calls (titles, tags, autocomplete, search-query rewriting) — it doesn't need to be vision-capable or particularly large. Smaller is faster and cheaper.

Save. New Image Studio chats now get descriptive titles, tag suggestions return, and autocomplete lights up.

Vision capability

The shipped preset sets meta.capabilities.vision: true so Open WebUI allows users to attach images to chats with this model. Two paths:

Default — qwen3.5:9b

The shipped preset uses standard (non-abliterated) Qwen 3.5 9B as the base — vision-capable, 256K context, native tool calling working. Preseed via init-models.sh.

Why not the abliterated 9B? The abliteration process mangles the function-call template, causing the model to either refuse to call tools or emit malformed <function=...> XML that Open WebUI's parser can't recognise. Image generation doesn't actually need an uncensored LLM — the SDXL checkpoints in ComfyUI are what produce image content, and they're already uncensored. The LLM's only role here is to call the right tool with the right args; the standard build does that more reliably.

Qwen 3.x quirk: thinking mode is on by default. The shipped system prompt starts with /no_think to suppress it. If the model still plans instead of firing the tool, set enable_thinking: false in Advanced Params → Custom Parameters.

Alternatives

If Qwen 3.5 isn't a fit (size, language preferences, abliteration caveats), other vision-capable Ollama tags worth trying:

• qwen2.5vl:7b — smaller, no thinking mode, very reliable tool-caller
• llama3.2-vision:11b — Meta's vision variant, ~7 GB
• minicpm-v:8b — fast, capable

To swap, change base_model_id in image_studio.json (or the Base Model field if you imported manually) and pull the model via init-models.sh or the Open WebUI model UI.

Non-vision base model

If you'd rather use a text-only LLM (e.g. mistral-nemo:12b), keep vision: true in the preset so Open WebUI still permits image attachments; the image flows through to edit_image via __messages__ / __files__ and ComfyUI does the visual work. The LLM can't see the image, but for explicit edit instructions ("change the background to a sunset") that doesn't matter.

Why this works when a generic chat model didn't

• The system prompt is unambiguous. No room for the model to decide "I'll just describe it in text instead."
• Only one tool is attached. No competing tools to choose between.
• Function Calling: Default is the safer choice for Qwen 3.x abliterated. Native mode expects the parser to recognise the model's structured tool-call format, which currently leaks Qwen 3.5's <function=...><parameter=...> XML to chat as plain text on the published Open WebUI / Ollama versions. Default mode uses Open WebUI's own prompt-injection wrapper that round-trips reliably. Try Native only after swapping the base model to one known to work end-to-end (mistral-nemo, qwen2.5vl).
• Lower temperature. Tool calling is more reliable with less sampling randomness.

Iterating on the system prompt

If users ask for things you didn't anticipate (specific aspect ratios, multi-image batches, particular checkpoints not in the routing rules), edit the system prompt above and re-paste into the Workspace → Models entry. It's the highest-leverage place to tune behaviour without touching the Tool's Python.