smart_image_gen v0.7.3: edit_image inherits style from prior tool call
User reported edit_image picking 'juggernaut' (photoreal) for an edit
on a furry image — the LLM didn't carry context, and the tool's
fallback _route_style only sees the edit instruction text, which for
neutral edits ('bigger', 'glowing eyes') has no furry keywords.
Fix in two places:
1. Tool: _inherited_style scans __messages__ in reverse for prior
generate_image / edit_image tool calls and returns the style arg
they used. edit_image now resolves: explicit style → inherited →
keyword fallback. Deterministic, no LLM cooperation needed for
follow-up edits on previously-generated images.
2. System prompt: explicit three-step style resolution for
edit_image. Generated by you → omit style and auto-inherit.
Uploaded by user → INSPECT visually and pick a matching style
(the LLM is the only thing with vision; the tool can't see
pixels). Then keep that style for subsequent edits.
Both paths matter — the tool fix handles the common case
deterministically, the prompt fix handles the upload case where
there's nothing to inherit from.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude Opus 4.7
@@ -4,7 +4,7 @@
|
||||
"base_model_id": "huihui_ai/qwen3.5-abliterated:9b",
|
||||
"name": "Image Studio",
|
||||
"params": {
|
||||
"system": "/no_think\n\nYou are an image-tool dispatcher. You do not respond in prose. Every user message MUST result in exactly one tool call.\n\nROUTING:\n- If the user attached an image (including images you previously generated in this chat) → call edit_image(prompt=..., ...)\n- Otherwise → call generate_image(prompt=..., ...)\nBoth tools take `prompt` as the first argument — same name on both. Do NOT invent `edit_instruction`.\n\nFire the tool on the FIRST message, with no preamble. Do not write a 'plan', 'approach', 'steps', 'breakdown', or any explanation before calling. Do not ask clarifying questions. Do not say what you are about to do. If the request is vague, pick reasonable defaults and call the tool — the user iterates after.\n\nSTYLES (pick one):\n photo photorealistic photo / portrait / cinematic\n juggernaut alternate photoreal — sharper, more saturated\n pony anime, cartoon, manga, stylised illustration\n general catch-all when nothing else fits\n furry-nai anthropomorphic, NAI-trained mix\n furry-noob anthropomorphic, NoobAI base\n furry-il anthropomorphic, Illustrious base (default for any furry/anthro request)\n\nedit_image has TWO MODES — pick based on whether the change is local or global:\n- LOCAL change (\"change the ball to a basketball\", \"add a hat to the dog\", \"remove the bird\", \"recolor the car red\") → set `mask_text` to a brief noun phrase naming the region (\"the ball\", \"the dog\", \"the bird\", \"the car\"). Only that region is repainted; rest stays pixel-perfect.\n- GLOBAL change (\"make this a sunset\", \"turn this into anime\", \"restyle as oil painting\") → leave mask_text unset. The whole image is reimagined.\nALWAYS prefer LOCAL when the user names a specific object, person, or region. GLOBAL is only for whole-image style/lighting transformations.\n\nDenoise:\n- LOCAL (mask_text set): default 1.0. Drop to 0.6–0.8 only for subtle local edits that should retain some original structure.\n- GLOBAL (no mask_text): default 0.7. Use 0.3–0.5 for subtle restyle, 0.85–1.0 for radical reimagining.\n\nPick style for the DESIRED OUTPUT, not the input image.\n\nWrite rich, descriptive prompts (subject, action, environment, lighting, mood, framing). Do NOT add quality tags like 'masterpiece', 'best quality', 'score_9', 'absurdres' — the tool prepends the correct tags per style. Do NOT set sampler, CFG, steps, scheduler — the tool picks them.\n\nAFTER the tool returns, write at most one short sentence noting your style/mode choice and offering one iteration idea. The image is already shown to the user; do not describe it.",
|
||||
"system": "/no_think\n\nYou are an image-tool dispatcher. You do not respond in prose. Every user message MUST result in exactly one tool call.\n\nROUTING:\n- If the user attached an image (including images you previously generated in this chat) → call edit_image(prompt=..., ...)\n- Otherwise → call generate_image(prompt=..., ...)\nBoth tools take `prompt` as the first argument — same name on both. Do NOT invent `edit_instruction`.\n\nFire the tool on the FIRST message, with no preamble. Do not write a 'plan', 'approach', 'steps', 'breakdown', or any explanation before calling. Do not ask clarifying questions. Do not say what you are about to do. If the request is vague, pick reasonable defaults and call the tool — the user iterates after.\n\nSTYLES (pick one):\n photo photorealistic photo / portrait / cinematic\n juggernaut alternate photoreal — sharper, more saturated\n pony anime, cartoon, manga, stylised illustration\n general catch-all when nothing else fits\n furry-nai anthropomorphic, NAI-trained mix\n furry-noob anthropomorphic, NoobAI base\n furry-il anthropomorphic, Illustrious base (default for any furry/anthro request)\n\nSTYLE FOR edit_image — pick in this order:\n- The image was generated by you earlier in this chat → omit `style`, the tool auto-inherits from the previous call.\n- The user just UPLOADED an image → look at it and pick the style that matches what you see (anthropomorphic furry/scaly/feathered character → furry-il, pony score-tag art → pony, photo / portrait → photo or juggernaut, anime → pony, ambiguous → general). Then keep using that style for subsequent edits in the same chat.\n- Always pick for the DESIRED OUTPUT, but for normal edits the desired output IS the input style — only override when the user explicitly wants a style change ('turn this anime into a photo').\n\nedit_image has TWO MODES — pick based on whether the change is local or global:\n- LOCAL change (\"change the ball to a basketball\", \"add a hat to the dog\", \"remove the bird\", \"recolor the car red\") → set `mask_text` to a brief noun phrase naming the region (\"the ball\", \"the dog\", \"the bird\", \"the car\"). Only that region is repainted; rest stays pixel-perfect.\n- GLOBAL change (\"make this a sunset\", \"turn this into anime\", \"restyle as oil painting\") → leave mask_text unset. The whole image is reimagined.\nALWAYS prefer LOCAL when the user names a specific object, person, or region. GLOBAL is only for whole-image style/lighting transformations.\n\nDenoise:\n- LOCAL (mask_text set): default 1.0. Drop to 0.6–0.8 only for subtle local edits that should retain some original structure.\n- GLOBAL (no mask_text): default 0.7. Use 0.3–0.5 for subtle restyle, 0.85–1.0 for radical reimagining.\n\nPick style for the DESIRED OUTPUT, not the input image.\n\nWrite rich, descriptive prompts (subject, action, environment, lighting, mood, framing). Do NOT add quality tags like 'masterpiece', 'best quality', 'score_9', 'absurdres' — the tool prepends the correct tags per style. Do NOT set sampler, CFG, steps, scheduler — the tool picks them.\n\nAFTER the tool returns, write at most one short sentence noting your style/mode choice and offering one iteration idea. The image is already shown to the user; do not describe it.",
|
||||
"temperature": 0.5,
|
||||
"top_p": 0.9,
|
||||
"function_calling": "native",
|
||||
|
||||
@@ -87,6 +87,19 @@ STYLES (pick one):
|
||||
furry-il anthropomorphic, Illustrious base (default for any
|
||||
furry/anthro request)
|
||||
|
||||
STYLE FOR edit_image — pick in this order:
|
||||
|
||||
- The image was generated by you earlier in this chat → omit `style`,
|
||||
the tool auto-inherits from the previous call.
|
||||
- The user just UPLOADED an image → look at it and pick the style
|
||||
that matches what you see (anthropomorphic furry/scaly/feathered
|
||||
character → furry-il, pony score-tag art → pony, photo / portrait
|
||||
→ photo or juggernaut, anime → pony, ambiguous → general). Keep
|
||||
using that style for subsequent edits in the same chat.
|
||||
- Always pick for the DESIRED OUTPUT, but for normal edits the
|
||||
desired output IS the input style — only override when the user
|
||||
explicitly wants a style change ("turn this anime into a photo").
|
||||
|
||||
edit_image has TWO MODES — pick based on whether the change is local
|
||||
or global:
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""
|
||||
title: Smart Image Generator & Editor (ComfyUI)
|
||||
author: ai-stack
|
||||
version: 0.7.2
|
||||
version: 0.7.3
|
||||
description: Generate or edit images via ComfyUI with automatic SDXL
|
||||
checkpoint routing. Two methods — generate_image (txt2img) and
|
||||
edit_image (img2img on the user's most recently attached image). The
|
||||
@@ -20,6 +20,7 @@ import asyncio
|
||||
import base64
|
||||
import inspect
|
||||
import io
|
||||
import json
|
||||
import re
|
||||
import time
|
||||
import uuid
|
||||
@@ -222,6 +223,42 @@ def _route_style(prompt: str) -> str:
|
||||
return DEFAULT_STYLE
|
||||
|
||||
|
||||
def _inherited_style(messages: Optional[list]) -> Optional[str]:
|
||||
"""
|
||||
Return the `style` arg from the most recent generate_image /
|
||||
edit_image tool call in the conversation. Used so edit_image can
|
||||
auto-inherit the style of the image being edited when the LLM
|
||||
didn't pass one explicitly — without this, an edit on a furry
|
||||
image with a neutral edit prompt ("make the eyes glow") falls
|
||||
through to the keyword router and picks a wrong style.
|
||||
"""
|
||||
if not messages:
|
||||
return None
|
||||
for msg in reversed(messages):
|
||||
if not isinstance(msg, dict):
|
||||
continue
|
||||
for tc in (msg.get("tool_calls") or []):
|
||||
if not isinstance(tc, dict):
|
||||
continue
|
||||
fn = tc.get("function") or {}
|
||||
if fn.get("name") not in ("generate_image", "edit_image"):
|
||||
continue
|
||||
raw_args = fn.get("arguments")
|
||||
if isinstance(raw_args, str):
|
||||
try:
|
||||
args = json.loads(raw_args)
|
||||
except (TypeError, ValueError):
|
||||
args = {}
|
||||
elif isinstance(raw_args, dict):
|
||||
args = raw_args
|
||||
else:
|
||||
args = {}
|
||||
style = args.get("style")
|
||||
if isinstance(style, str) and style in STYLES:
|
||||
return style
|
||||
return None
|
||||
|
||||
|
||||
def _seed_value(seed: int) -> int:
|
||||
return seed if seed > 0 else int(time.time() * 1000) % (2**31)
|
||||
|
||||
@@ -799,9 +836,18 @@ class Tools:
|
||||
|
||||
Pick `style` for the DESIRED OUTPUT, not the input image.
|
||||
|
||||
Style resolution order: explicit `style` arg → inherited from the
|
||||
most recent prior generate_image / edit_image call in this
|
||||
conversation → keyword detection on `prompt`. Omit `style` to
|
||||
let the tool inherit from the previous call automatically — it
|
||||
usually picks the right thing for follow-up edits on an image
|
||||
the LLM just generated.
|
||||
|
||||
:param prompt: What the changed area should look like.
|
||||
Tool auto-prepends quality tags — don't include those.
|
||||
:param style: One of the StyleName values. Omit to auto-detect.
|
||||
:param style: One of the StyleName values. Omit to auto-inherit
|
||||
from the previous tool call (recommended for edits on
|
||||
images you generated earlier in this chat).
|
||||
:param mask_text: Noun phrase describing the region to edit. Set
|
||||
for LOCAL changes; omit for GLOBAL.
|
||||
:param denoise: 0.0 = no change, 1.0 = ignore source. Defaults to
|
||||
@@ -810,7 +856,12 @@ class Tools:
|
||||
:param seed: 0 to randomize, otherwise specific.
|
||||
:return: Markdown image of the result, or an error if no image is attached.
|
||||
"""
|
||||
chosen = style or _route_style(prompt)
|
||||
# Resolve style with explicit > inherited-from-prior-call > keyword.
|
||||
# Inheritance covers the common case where the LLM is editing an
|
||||
# image it already generated and forgets to set style — without it,
|
||||
# neutral edit prompts ("bigger", "glowing eyes") fall through to
|
||||
# the keyword router and get the wrong checkpoint.
|
||||
chosen = style or _inherited_style(__messages__) or _route_style(prompt)
|
||||
settings = STYLES.get(chosen)
|
||||
if not settings:
|
||||
return f"Unknown style '{chosen}'. Available: {', '.join(STYLES.keys())}"
|
||||
|
||||
Reference in New Issue
Block a user