smart_image_gen v0.3: add edit_image (img2img) method
The Tool now exposes two methods the LLM picks between based on whether the user attached an image: generate_image — txt2img (existing, unchanged behavior) edit_image — img2img on the most recently attached image edit_image extracts the source image from __messages__ (base64 data URIs in image_url content blocks) or __files__ (local path or URL), uploads to ComfyUI's /upload/image, runs an img2img workflow at the caller-specified denoise (default 0.7), and returns the edited result. Same per-style routing / sampler / CFG / prefix logic as generation. Refactored the submit-and-poll loop into _submit_and_fetch shared by both methods. Image extraction is defensive — tries messages first, then files (path then URL), returns a clear "no image attached" message rather than silently generating from scratch. Image Studio system prompt rewritten to teach the LLM when to call edit_image vs generate_image and how to pick denoise. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
41d571d8d1
commit
d4e2058859
3 files changed
+342
-117
No files matched your search
@@ -16,7 +16,7 @@ production `srvno.de` deployment.
|
||||
| `Caddyfile` | TLS + reverse proxy config (one site block per hostname) |
|
||||
| `init-models.sh` | LLMs to preseed into Ollama on first boot |
|
||||
| `comfyui-init-models.sh` | Checkpoints/VAEs/LoRAs to preseed into ComfyUI on first boot |
|
||||
| `openwebui-tools/smart_image_gen.py` | Tool that auto-routes image generation to the right SDXL checkpoint |
|
||||
| `openwebui-tools/smart_image_gen.py` | Tool that auto-routes image generation AND editing to the right SDXL checkpoint |
|
||||
| `openwebui-models/image_studio.md` | Dedicated chat-model preset — system prompt that forces tool use |
|
||||
| `.env.example` | Secrets and image-tag pins. Copy to `.env` |
|
||||
|
||||
@@ -153,8 +153,14 @@ use the edit action.
|
||||
The image-button path always uses the admin's **Default Model**. To get
|
||||
per-prompt checkpoint routing — e.g. "draw me a cyberpunk city" picks
|
||||
CyberRealistic, "anthro fox warrior" picks one of the furry checkpoints —
|
||||
install the `smart_image_gen.py` Tool. The LLM calls it instead of the
|
||||
built-in image action and chooses the right SDXL checkpoint per request.
|
||||
install the `smart_image_gen.py` Tool. It exposes two methods the LLM
|
||||
calls:
|
||||
|
||||
- **`generate_image`** for new images from scratch (txt2img).
|
||||
- **`edit_image`** for modifying an image the user attached to the
|
||||
chat (img2img).
|
||||
|
||||
Both auto-route to the right SDXL checkpoint per request.
|
||||
|
||||
1. **Workspace -> Tools -> +** (top-right).
|
||||
2. Paste the contents of
|
||||
|
||||
Reference in new issue
Block a user