Add S3 mirror path for Ollama models + mirror-ollama-model.sh helper

Three pieces:

1. mirror-ollama-model.sh — run on any machine that has the model
   pulled. Parses the manifest at
   ~/.ollama/models/manifests/registry.ollama.ai/<ns>/<name>/<tag>,
   greps every sha256:* digest, tars manifest + referenced blobs into
   one .tgz. Output is portable — extract over any other Ollama
   data dir and the model is immediately visible.

2. init-models.sh gains an s3_pull function that curls a tarball from
   $S3_OLLAMA_BASE and extracts into /root/.ollama/models/. Falls back
   to ollama pull when S3_OLLAMA_BASE is unset, so s3_pull lines are
   safe to commit before the bucket is ready. huihui_ai/qwen3.5-
   abliterated:9b promoted to s3_pull as the example.

3. docker-compose.yml model-init service propagates S3_OLLAMA_BASE
   from .env. Curl auto-installs at script start because ollama/ollama
   doesn't always ship it.

README documents the mirror workflow under "Mirroring models to S3".

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
57_WolveandClaude Opus 4.7 committed 2026-04-19 13:43:26 -05:00
1 parent f77f5993fb
commit 5a34ced8f1
5 files changed
+187 -11

No files matched your search

+57 -1
View File
@@ -15,6 +15,7 @@ production `srvno.de` deployment.
| `docker-compose.yml` | Service definitions, volumes, GPU reservations |
| `Caddyfile` | TLS + reverse proxy config (one site block per hostname) |
| `init-models.sh` | LLMs to preseed into Ollama on first boot |
| `mirror-ollama-model.sh` | Helper — mirror an Ollama model into a tarball you can host on S3 |
| `comfyui-init-models.sh` | Checkpoints/VAEs/LoRAs to preseed into ComfyUI on first boot |
| `openwebui-tools/smart_image_gen.py` | Tool that auto-routes image generation AND editing to the right SDXL checkpoint |
| `openwebui-models/image_studio.md` | Dedicated chat-model preset — manual setup walkthrough |
@@ -64,7 +65,14 @@ Then edit:
```
- **`init-models.sh`** — keep the LLMs you want preseeded, drop the rest.
Check sizes at <https://ollama.com/library> first; the host needs disk
for everything listed.
for everything listed. Two pull paths are available:
- `pull "<model:tag>"` — standard registry pull from
`registry.ollama.ai`.
- `s3_pull "<model:tag>" "<archive.tgz>"` — fetches from your own
mirror set via `S3_OLLAMA_BASE` in `.env`. Falls back to
`ollama pull` if the env var isn't set, so this is safe to enable
incrementally. Create the tarballs once with
`mirror-ollama-model.sh` (see [Mirroring models to S3](#mirroring-models-to-s3)).
- **`comfyui-init-models.sh`** — checkpoints/VAEs/LoRAs to preseed into
ComfyUI. Ships empty (no active fetches) — uncomment the SDXL/Flux/
upscaler examples or add your own. Whatever filename you pick should
@@ -217,6 +225,54 @@ To extend (new checkpoint, new style):
auto-detect path.
- Re-paste the Tool source in Workspace -> Tools.
## Mirroring models to S3
For models you want to pin against upstream changes (or pull faster
from your own infra), mirror them to S3 once and have the
deployment fetch from there.
### Create the mirror tarball
Run [`mirror-ollama-model.sh`](mirror-ollama-model.sh) on any machine
that has the model pulled locally. It reads `~/.ollama/models/`,
pulls the manifest's referenced blobs, and tars everything together:
```sh
./mirror-ollama-model.sh huihui_ai/qwen3.5-abliterated:9b qwen3.5-abliterated-9b.tgz
```
### Upload to S3
Whatever fits — `aws s3 cp`, `mc`, `rclone`, etc. The bucket needs
to expose the file over HTTPS (public-read ACL on the object, a
CloudFront distribution, R2 with public URLs, etc.):
```sh
aws s3 cp qwen3.5-abliterated-9b.tgz s3://your-bucket/ollama-models/ --acl public-read
```
### Wire the deployment to fetch from there
In `.env`:
```
S3_OLLAMA_BASE=https://your-bucket.s3.amazonaws.com/ollama-models
```
In `init-models.sh`, switch the affected models from `pull` to
`s3_pull`:
```sh
s3_pull "huihui_ai/qwen3.5-abliterated:9b" "qwen3.5-abliterated-9b.tgz"
```
`docker compose up -d model-init` re-runs the init container; the
script downloads the tarball, extracts into the `ollama-data` volume,
and the running Ollama daemon picks it up on its next manifest scan.
If `S3_OLLAMA_BASE` isn't set, `s3_pull` transparently falls back to
`ollama pull` — safe to commit `s3_pull` lines without S3 ready yet.
## Enabling Anubis (later)
The `anubis-owui` service is defined in compose but no Caddy site block