smart_image_gen v0.4: emit image to chat, return only confirmation

The data URI returned from the tool was being given to the LLM as the
tool result — the LLM then either echoed the base64 to the user as
plain text (screenshot 1) or hallucinated a description of what it
thought the image looked like (screenshot 2 — "an image of a cat
sitting on a windowsill" for a fox-warrior prompt).

Fix: push the markdown image into the chat directly via
__event_emitter__ as a "message" event, and return a short text
confirmation as the function value. The confirmation is worded to
prevent the LLM from describing the image or repeating the markdown
(both common failure modes for tool-using LLMs).

Both generate_image and edit_image fixed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-19 13:14:59 -05:00
co-authored by Claude Opus 4.7
parent 6adf133558
commit 4d996e1205
@@ -1,14 +1,17 @@
"""
title: Smart Image Generator & Editor (ComfyUI)
author: ai-stack
version: 0.3.0
version: 0.4.0
description: Generate or edit images via ComfyUI with automatic SDXL
checkpoint routing. Two methods — generate_image (txt2img) and
edit_image (img2img on the user's most recently attached image). The
LLM picks (or auto-detects) the right model — photoreal, Pony
score-tag, NoobAI/Illustrious furry, etc. — and each style ships
with the creator-recommended sampler, scheduler, CFG, steps, CLIP
skip, prompt-prefix dialect, and negatives.
skip, prompt-prefix dialect, and negatives. The actual image is
pushed into the chat via the event emitter; the function return is
just a short confirmation so the LLM doesn't try to describe or
re-emit the data URI.
required_open_webui_version: 0.5.0
"""
@@ -483,8 +486,23 @@ class Tools:
return err
b64 = base64.b64encode(raw).decode("ascii")
# Push the image straight into the chat. The function return is just
# a confirmation for the LLM — if we returned the markdown, the LLM
# would either echo the base64 to the user as text, or hallucinate
# a description of what it thinks the image looks like.
if __event_emitter__:
await __event_emitter__({
"type": "message",
"data": {"content": f"![{chosen}](data:image/png;base64,{b64})"},
})
await emit(f"Done — {chosen}", done=True)
return f"![{chosen}](data:image/png;base64,{b64})"
return (
f"Image generated and shown to the user above (style: {chosen}, "
f"checkpoint: {settings['ckpt']}). Do NOT describe the image, "
f"do NOT repeat any base64 or markdown — the user can see it. "
f"You may briefly note your style choice and offer one or two "
f"iteration ideas (different style, tighter framing, etc)."
)
async def edit_image(
self,
@@ -582,5 +600,16 @@ class Tools:
return err
b64 = base64.b64encode(raw_out).decode("ascii")
if __event_emitter__:
await __event_emitter__({
"type": "message",
"data": {"content": f"![edit:{chosen}](data:image/png;base64,{b64})"},
})
await emit(f"Done — {chosen} (denoise {denoise:.2f})", done=True)
return f"![edit:{chosen}](data:image/png;base64,{b64})"
return (
f"Edited image shown to the user above (style: {chosen}, "
f"checkpoint: {settings['ckpt']}, denoise: {denoise:.2f}). Do NOT "
f"describe the image, do NOT repeat any base64 or markdown — the "
f"user can see it. You may briefly note your choice and offer "
f"iterations (different denoise, alternate style, etc)."
)