Ollama: keep loaded models resident until evicted (KEEP_ALIVE=-1)

Was 30m, which evicts after 30 minutes of inactivity and forces a
reload penalty on the next request. Setting -1 holds models in VRAM
indefinitely; MAX_LOADED_MODELS=3 caps how many can stay resident
simultaneously (vs the previous 2). Tune MAX higher if you're
rotating between more than three models AND your GPU has the VRAM
for it — comment in the compose explains the trade-off.

For the live srvno.de stack: OLLAMA_KEEP_ALIVE=-1 takes effect on
the next `docker compose up -d ollama`. Loaded models survive the
restart only if they're re-requested before swap-out anyway.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-19 19:25:42 -05:00
co-authored by Claude Opus 4.7
parent a1af88a632
commit 02a4bece5d
+6 -2
View File
@@ -60,8 +60,12 @@ services:
- ollama-data:/root/.ollama
environment:
- OLLAMA_HOST=0.0.0.0:11434
- OLLAMA_KEEP_ALIVE=30m
- OLLAMA_MAX_LOADED_MODELS=2
# KEEP_ALIVE=-1 holds loaded models in VRAM until evicted by another
# load (vs the default 5m / our previous 30m which forces a reload
# penalty on every cold use). Pair with MAX_LOADED_MODELS sized to
# whatever fits in your GPU's VRAM — see README "VRAM sizing".
- OLLAMA_KEEP_ALIVE=-1
- OLLAMA_MAX_LOADED_MODELS=3
- OLLAMA_FLASH_ATTENTION=1
deploy:
resources: