helm: reject emptyDir for volume idx, and rebuild a missing idx on restart (#10005)

* helm: reject emptyDir for volume idx

An ephemeral index on a separate volume is wiped on every pod restart
while the .dat/.vif persist on the data PVCs. The volume server then
finds data with no matching .idx and exits via glog.Fatalf, putting the
pod into CrashLoopBackOff with no automatic recovery.

emptyDir is never the right choice for idx: if the data is persistent it
is a durability mismatch, and if the data is also ephemeral the default
(idx co-located with the data) already covers it. Fail the render with a
clear message pointing at the default or a persistent volume instead, and
drop emptyDir from the documented idx options.

* helm: rebuild a missing volume idx on restart

With emptyDir rejected, a separate idx volume is always persistent
(hostPath/PVC/existingClaim) -- but it can still lose its .idx out of
band (e.g. a node-local PVC reprovisioned on reschedule, or a pre-9944
compaction crash that left a .dat without its matching .idx). The
seaweedfs-vol-move-idx init container already moves idx files next to the
data into the index dir; have it first regenerate, via weed fix, any .idx
absent from both the data dir and the index dir, then move it into place.
The rebuild only runs when an idx is genuinely missing, so a healthy
index adds no startup cost.
This commit is contained in:
Chris Lu
2026-06-18 01:30:34 -07:00
committed by GitHub
parent 443b5b1184
commit 40615e3d9d
2 changed files with 29 additions and 8 deletions
@@ -4,6 +4,10 @@
{{- $volumeName := trimSuffix "-" (printf "volume-%s" $vname) }}
{{- $volume := mergeOverwrite (deepCopy $.Values.volume) (dict "enabled" true) $volume }}
{{- if and $volume.idx (eq ($volume.idx.type | toString) "emptyDir") }}
{{- fail (printf "%s: idx.type \"emptyDir\" is not supported. An ephemeral index is wiped on every pod restart while the data persists, forcing a full index rebuild from the .dat on each start (a fatal crash on older versions). Use the default (idx: {}, kept next to the data) or a persistentVolumeClaim/hostPath/existingClaim for a separate persistent index." $volumeName) }}
{{- end }}
{{- if $volume.enabled }}
---
apiVersion: apps/v1
@@ -79,7 +83,23 @@ spec:
image: {{ template "seaweedfs.volume.image" $ }}
imagePullPolicy: {{ $.Values.global.seaweedfs.imagePullPolicy | default "IfNotPresent" }}
command: [ '/bin/sh', '-c' ]
args: [ '{{range $dir := $volume.dataDirs }}if ls /{{$dir.name}}/*.idx >/dev/null 2>&1; then mv /{{$dir.name}}/*.idx /idx/ ; fi; {{end}}' ]
args:
- |
for dir in {{ range $index, $dir := $volume.dataDirs }}{{ if ne $index 0 }} {{ end }}/{{ $dir.name }}{{ end }}; do
# Rebuild a .idx missing from both dirs (e.g. an index volume that lost files) from the .dat, else the server fatally exits on its idx check.
for dat in "$dir"/*.dat; do
[ -e "$dat" ] || continue
base=${dat##*/}; base=${base%.dat}
if [ ! -e "$dir/$base.idx" ] && [ ! -e "/idx/$base.idx" ]; then
echo "rebuilding missing idx in $dir (trigger: volume $base)"
weed fix "$dir"
break
fi
done
if ls "$dir"/*.idx >/dev/null 2>&1; then
mv "$dir"/*.idx /idx/
fi
done
volumeMounts:
- name: idx
mountPath: /idx
+8 -7
View File
@@ -385,7 +385,7 @@ volume:
enabled: true
image: alpine/k8s:1.28.4
# idx can be defined by:
# idx (the volume index) may use a separate PERSISTENT volume:
#
# idx:
# type: "hostPath"
@@ -404,13 +404,14 @@ volume:
# type: "existingClaim"
# claimName: "myClaim"
#
# or
# "emptyDir" is rejected for idx (the chart fails to render): an ephemeral
# index is wiped on every restart while the data persists, forcing a full
# rebuild each start (a fatal crash on older versions). "logs" may still use
# any of the above or emptyDir.
#
# idx:
# type: "emptyDir"
# same applies to "logs"
# Default {} keeps the index next to the data, so it persists with the data and
# needs no separate volume. Recommended unless you must split the index onto
# its own persistent storage.
idx: {}
# Resource requests, limits, etc. for the vol-move-idx initContainer. This