Add S3 mirror path for Ollama models + mirror-ollama-model.sh helper
Three pieces: 1. mirror-ollama-model.sh — run on any machine that has the model pulled. Parses the manifest at ~/.ollama/models/manifests/registry.ollama.ai/<ns>/<name>/<tag>, greps every sha256:* digest, tars manifest + referenced blobs into one .tgz. Output is portable — extract over any other Ollama data dir and the model is immediately visible. 2. init-models.sh gains an s3_pull function that curls a tarball from $S3_OLLAMA_BASE and extracts into /root/.ollama/models/. Falls back to ollama pull when S3_OLLAMA_BASE is unset, so s3_pull lines are safe to commit before the bucket is ready. huihui_ai/qwen3.5- abliterated:9b promoted to s3_pull as the example. 3. docker-compose.yml model-init service propagates S3_OLLAMA_BASE from .env. Curl auto-installs at script start because ollama/ollama doesn't always ship it. README documents the mirror workflow under "Mirroring models to S3". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
f77f5993fb
commit
5a34ced8f1
5 files changed
+187
-11
No files matched your search
@@ -15,6 +15,7 @@ production `srvno.de` deployment.
|
||||
| `docker-compose.yml` | Service definitions, volumes, GPU reservations |
|
||||
| `Caddyfile` | TLS + reverse proxy config (one site block per hostname) |
|
||||
| `init-models.sh` | LLMs to preseed into Ollama on first boot |
|
||||
| `mirror-ollama-model.sh` | Helper — mirror an Ollama model into a tarball you can host on S3 |
|
||||
| `comfyui-init-models.sh` | Checkpoints/VAEs/LoRAs to preseed into ComfyUI on first boot |
|
||||
| `openwebui-tools/smart_image_gen.py` | Tool that auto-routes image generation AND editing to the right SDXL checkpoint |
|
||||
| `openwebui-models/image_studio.md` | Dedicated chat-model preset — manual setup walkthrough |
|
||||
@@ -64,7 +65,14 @@ Then edit:
|
||||
```
|
||||
- **`init-models.sh`** — keep the LLMs you want preseeded, drop the rest.
|
||||
Check sizes at <https://ollama.com/library> first; the host needs disk
|
||||
for everything listed.
|
||||
for everything listed. Two pull paths are available:
|
||||
- `pull "<model:tag>"` — standard registry pull from
|
||||
`registry.ollama.ai`.
|
||||
- `s3_pull "<model:tag>" "<archive.tgz>"` — fetches from your own
|
||||
mirror set via `S3_OLLAMA_BASE` in `.env`. Falls back to
|
||||
`ollama pull` if the env var isn't set, so this is safe to enable
|
||||
incrementally. Create the tarballs once with
|
||||
`mirror-ollama-model.sh` (see [Mirroring models to S3](#mirroring-models-to-s3)).
|
||||
- **`comfyui-init-models.sh`** — checkpoints/VAEs/LoRAs to preseed into
|
||||
ComfyUI. Ships empty (no active fetches) — uncomment the SDXL/Flux/
|
||||
upscaler examples or add your own. Whatever filename you pick should
|
||||
@@ -217,6 +225,54 @@ To extend (new checkpoint, new style):
|
||||
auto-detect path.
|
||||
- Re-paste the Tool source in Workspace -> Tools.
|
||||
|
||||
## Mirroring models to S3
|
||||
|
||||
For models you want to pin against upstream changes (or pull faster
|
||||
from your own infra), mirror them to S3 once and have the
|
||||
deployment fetch from there.
|
||||
|
||||
### Create the mirror tarball
|
||||
|
||||
Run [`mirror-ollama-model.sh`](mirror-ollama-model.sh) on any machine
|
||||
that has the model pulled locally. It reads `~/.ollama/models/`,
|
||||
pulls the manifest's referenced blobs, and tars everything together:
|
||||
|
||||
```sh
|
||||
./mirror-ollama-model.sh huihui_ai/qwen3.5-abliterated:9b qwen3.5-abliterated-9b.tgz
|
||||
```
|
||||
|
||||
### Upload to S3
|
||||
|
||||
Whatever fits — `aws s3 cp`, `mc`, `rclone`, etc. The bucket needs
|
||||
to expose the file over HTTPS (public-read ACL on the object, a
|
||||
CloudFront distribution, R2 with public URLs, etc.):
|
||||
|
||||
```sh
|
||||
aws s3 cp qwen3.5-abliterated-9b.tgz s3://your-bucket/ollama-models/ --acl public-read
|
||||
```
|
||||
|
||||
### Wire the deployment to fetch from there
|
||||
|
||||
In `.env`:
|
||||
|
||||
```
|
||||
S3_OLLAMA_BASE=https://your-bucket.s3.amazonaws.com/ollama-models
|
||||
```
|
||||
|
||||
In `init-models.sh`, switch the affected models from `pull` to
|
||||
`s3_pull`:
|
||||
|
||||
```sh
|
||||
s3_pull "huihui_ai/qwen3.5-abliterated:9b" "qwen3.5-abliterated-9b.tgz"
|
||||
```
|
||||
|
||||
`docker compose up -d model-init` re-runs the init container; the
|
||||
script downloads the tarball, extracts into the `ollama-data` volume,
|
||||
and the running Ollama daemon picks it up on its next manifest scan.
|
||||
|
||||
If `S3_OLLAMA_BASE` isn't set, `s3_pull` transparently falls back to
|
||||
`ollama pull` — safe to commit `s3_pull` lines without S3 ready yet.
|
||||
|
||||
## Enabling Anubis (later)
|
||||
|
||||
The `anubis-owui` service is defined in compose but no Caddy site block
|
||||
|
||||
Reference in new issue
Block a user