helm: decouple JWT signing from cert-manager mTLS (fixes #9506) (#9508)

* helm(security): decouple JWT signing from cert-manager mTLS

The filer needs jwt.filer_signing.key to register the IAM gRPC service the
Admin UI Users tab calls (PR #9442). The chart only rendered security.toml
under enableSecurity, which also pulls in cert-manager for mTLS — much heavier
than the Admin UI needs. Operators on Helm without cert-manager have no way
to flip the JWT key on, so the Users tab fails with Unimplemented after
upgrading past 4.24.

Introduce seaweedfs.securityConfigEnabled, true when enableSecurity OR any
explicit jwtSigning toggle (volumeRead/filerWrite/filerRead) is set. The
configmap renders under that helper; the [grpc.*]/[https.*] sections inside
stay gated on enableSecurity. Each pod template splits the security-config
mount onto the helper and keeps the cert volume mounts on enableSecurity.

volumeWrite is intentionally excluded from the helper trigger because it
defaults to true; including it would silently start mounting security.toml on
every fresh install. With this change, enableSecurity=false + defaults
renders nothing (unchanged), enableSecurity=true renders the full toml
(unchanged), and enableSecurity=false + filerWrite=true renders just the
[jwt.*] sections so the Admin UI works without mTLS.

Fixes #9506.

* helm(security): trim verbose comments

* helm(security): handle null securityConfig in helper

Address review feedback: (.Values.global.seaweedfs.securityConfig).jwtSigning
errored if a user explicitly set securityConfig: null in their values. Drop
into intermediate $sec/$jwt with default dict at each step so a missing or
nulled-out parent is tolerated.

* helm(ci): cover IAM gRPC decoupling (issue #9506)

Five regression assertions exercised against the rendered chart so a
future change cannot silently re-couple jwt.filer_signing to mTLS:

1. defaults render no security-config ConfigMap (preserves baseline)
2. filerWrite=true alone renders [jwt.filer_signing] with no [grpc.*]
3. filerWrite=true mounts security-config on filer + admin without
   pulling in cert volumes — the actual fix for the Admin UI Users tab
4. enableSecurity=true still produces the full toml with [grpc.master]
5. securityConfig=null and securityConfig.jwtSigning=null both render
   cleanly (gemini-code-assist review nit, applied chart-wide)

Patch a pre-existing direct-access in filer-statefulset.yaml that
crashed on securityConfig=null, surfaced by the new null assertion.

* helm(ci): drop issue numbers from comments

* helm(ci): install pyyaml; assert [jwt.signing] in mTLS path

Address coderabbit review:

- The new IAM gRPC test block uses `import yaml` but ran before the
  later `pip install pyyaml -q` step that the security+S3 block
  performs. CI happens to pass because the runner image carries
  PyYAML, but make the dependency explicit so a future runner change
  cannot silently break the regression test.

- The enableSecurity=true assertion only checked for [grpc.master].
  Also assert [jwt.signing] so a refactor that drops the volume-side
  JWT stanza from the mTLS path fails the test instead of slipping
  through.
This commit is contained in:
Chris Lu
2026-05-14 23:43:24 -07:00
committed by GitHub
parent bfb2661fec
commit 2ed95d7ea9
14 changed files with 201 additions and 21 deletions

View File

@@ -68,6 +68,135 @@ jobs:
grep -q "security-config" /tmp/security.yaml
echo "✓ Security configuration renders correctly"
echo ""
echo "=== Testing IAM gRPC opt-in path ==="
# Regression test: the filer registers the IAM gRPC service the
# Admin UI Users tab calls only when jwt.filer_signing.key is in
# security.toml. Operators must be able to enable that without
# the cert-manager mTLS bundle.
# Install PyYAML explicitly: this block runs before the later
# security+S3 block that does the same install, and we don't
# want to rely on the runner image shipping it.
pip install pyyaml -q
python3 - "$CHART_DIR" <<'PYEOF'
import subprocess, sys, yaml
chart = sys.argv[1]
def render(values):
args = ["helm", "template", "test", chart]
for k, v in values.items():
args += ["--set", f"{k}={v}"]
return subprocess.check_output(args, text=True)
def docs(manifest):
return [d for d in yaml.safe_load_all(manifest) if d]
def configmap(manifest, name):
for d in docs(manifest):
if d.get("kind") == "ConfigMap" and d["metadata"]["name"] == name:
return d
return None
def workload_mounts(manifest, name):
for d in docs(manifest):
if d.get("kind") not in ("Deployment", "StatefulSet"):
continue
if d["metadata"]["name"] != name:
continue
pod = d["spec"]["template"]["spec"]
vols = {v["name"] for v in pod.get("volumes", [])}
mounts = set()
for c in pod.get("containers", []):
for vm in c.get("volumeMounts", []):
mounts.add(vm["name"])
return vols, mounts
return None, None
failed = []
# Case 1: defaults. The chart historically rendered nothing
# security-related; preserve that so this PR is non-breaking on
# existing installs.
out = render({})
if configmap(out, "test-seaweedfs-security-config") is not None:
failed.append("defaults: security ConfigMap should not render")
else:
print("✓ defaults: no security-config ConfigMap (unchanged)")
# Case 2: filerWrite=true alone is the documented opt-in for
# the Admin UI Users tab. Configmap must render with
# [jwt.filer_signing] and NO [grpc.*] sections (cert paths
# only exist with mTLS).
out = render({
"global.seaweedfs.securityConfig.jwtSigning.filerWrite": "true",
"admin.enabled": "true",
})
cm = configmap(out, "test-seaweedfs-security-config")
if cm is None:
failed.append("filerWrite=true: security ConfigMap missing")
else:
toml = cm["data"]["security.toml"]
if "[jwt.filer_signing]" not in toml:
failed.append("filerWrite=true: security.toml missing [jwt.filer_signing]")
if "[grpc" in toml:
failed.append("filerWrite=true: security.toml unexpectedly has [grpc.*] (would need cert mounts)")
if "[jwt.filer_signing]" in toml and "[grpc" not in toml:
print("✓ filerWrite=true: security.toml has [jwt.filer_signing], no [grpc.*]")
# Case 3: filer + admin pods must MOUNT the security ConfigMap
# under filerWrite=true so the JWT key reaches both processes.
# Cert volumes must NOT be present (no mTLS).
for wl in ("test-seaweedfs-filer", "test-seaweedfs-admin"):
vols, mounts = workload_mounts(out, wl)
if vols is None:
failed.append(f"filerWrite=true: workload {wl} not found")
continue
if "security-config" not in vols or "security-config" not in mounts:
failed.append(f"filerWrite=true: {wl} does not mount security-config (IAM gRPC would still fail)")
else:
print(f"✓ filerWrite=true: {wl} mounts security-config")
cert_vols = {v for v in vols if v.endswith("-cert")}
if cert_vols:
failed.append(f"filerWrite=true: {wl} unexpectedly has cert volumes {sorted(cert_vols)}")
# Case 4: enableSecurity=true must still render the full toml
# with both [jwt.signing] and [grpc.*]. Guards against the
# decoupling change accidentally regressing the mTLS path.
out = render({"global.seaweedfs.enableSecurity": "true"})
cm = configmap(out, "test-seaweedfs-security-config")
if cm is None:
failed.append("enableSecurity=true: security ConfigMap missing")
else:
toml = cm["data"]["security.toml"]
missing = [s for s in ("[jwt.signing]", "[grpc.master]") if s not in toml]
if missing:
failed.append(f"enableSecurity=true: security.toml missing {missing}")
else:
print("✓ enableSecurity=true: security.toml has [jwt.signing] + [grpc.*] preserved")
# Case 5: helper must tolerate explicit nulls (gemini-code-assist
# PR review). securityConfig=null was the parens-pattern crash
# the helper review caught.
for null_path in ("global.seaweedfs.securityConfig",
"global.seaweedfs.securityConfig.jwtSigning"):
try:
out = render({null_path: "null"})
except subprocess.CalledProcessError as e:
failed.append(f"{null_path}=null: render failed: {e.output[:200] if e.output else e}")
continue
if configmap(out, "test-seaweedfs-security-config") is not None:
failed.append(f"{null_path}=null: should not render configmap")
else:
print(f"✓ {null_path}=null: render tolerates explicit null")
if failed:
print("\nFAIL:", file=sys.stderr)
for f in failed:
print(f" - {f}", file=sys.stderr)
sys.exit(1)
PYEOF
echo "✓ IAM gRPC decoupling tests passed"
echo "=== Testing with monitoring enabled ==="
helm template test $CHART_DIR \
--set global.seaweedfs.monitoring.enabled=true \

View File

@@ -3,4 +3,4 @@ description: SeaweedFS
name: seaweedfs
appVersion: "4.25"
# Dev note: Trigger a helm chart release by `git tag -a helm-<version>`
version: 4.25.0
version: 4.25.1

View File

@@ -180,11 +180,13 @@ spec:
- name: admin-logs
mountPath: /logs
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
{{- if include "seaweedfs.securityConfigEnabled" . }}
- name: security-config
readOnly: true
mountPath: /etc/seaweedfs/security.toml
subPath: security.toml
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
- name: ca-cert
readOnly: true
mountPath: /usr/local/share/ca-certificates/ca/
@@ -275,10 +277,12 @@ spec:
persistentVolumeClaim:
claimName: {{ .Values.admin.logs.claimName }}
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
{{- if include "seaweedfs.securityConfigEnabled" . }}
- name: security-config
configMap:
name: {{ include "seaweedfs.fullname" . }}-security-config
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
- name: ca-cert
secret:
secretName: {{ include "seaweedfs.fullname" . }}-ca-cert

View File

@@ -330,11 +330,13 @@ spec:
mountPath: /etc/seaweedfs/master.toml
subPath: master.toml
readOnly: true
{{- if .Values.global.seaweedfs.enableSecurity }}
{{- if include "seaweedfs.securityConfigEnabled" . }}
- name: security-config
mountPath: /etc/seaweedfs/security.toml
subPath: security.toml
readOnly: true
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
- name: ca-cert
mountPath: /usr/local/share/ca-certificates/ca/
readOnly: true
@@ -461,10 +463,12 @@ spec:
- name: master-config
configMap:
name: {{ include "seaweedfs.fullname" . }}-master-config
{{- if .Values.global.seaweedfs.enableSecurity }}
{{- if include "seaweedfs.securityConfigEnabled" . }}
- name: security-config
configMap:
name: {{ include "seaweedfs.fullname" . }}-security-config
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
- name: ca-cert
secret:
secretName: {{ include "seaweedfs.fullname" . }}-ca-cert

View File

@@ -117,11 +117,13 @@ spec:
name: config-users
readOnly: true
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
{{- if include "seaweedfs.securityConfigEnabled" . }}
- name: security-config
readOnly: true
mountPath: /etc/seaweedfs/security.toml
subPath: security.toml
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
- name: ca-cert
readOnly: true
mountPath: /usr/local/share/ca-certificates/ca/
@@ -179,10 +181,12 @@ spec:
secretName: {{ include "seaweedfs.fullname" . }}-s3-secret
{{- end }}
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
{{- if include "seaweedfs.securityConfigEnabled" . }}
- name: security-config
configMap:
name: {{ include "seaweedfs.fullname" . }}-security-config
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
- name: ca-cert
secret:
secretName: {{ include "seaweedfs.fullname" . }}-ca-cert

View File

@@ -234,11 +234,13 @@ spec:
mountPath: /etc/seaweedfs/notification.toml
subPath: notification.toml
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
{{- if include "seaweedfs.securityConfigEnabled" . }}
- name: security-config
readOnly: true
mountPath: /etc/seaweedfs/security.toml
subPath: security.toml
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
- name: ca-cert
readOnly: true
mountPath: /usr/local/share/ca-certificates/ca/
@@ -274,7 +276,8 @@ spec:
name: swfs-s3-tls
{{- end }}
{{- end }}
{{- $isJwtEnabled := or .Values.global.seaweedfs.securityConfig.jwtSigning.filerWrite .Values.global.seaweedfs.securityConfig.jwtSigning.filerRead }}
{{- $jwt := (.Values.global.seaweedfs.securityConfig).jwtSigning | default dict }}
{{- $isJwtEnabled := or $jwt.filerWrite $jwt.filerRead }}
{{- if .Values.filer.readinessProbe.enabled }}
readinessProbe:
{{- if or $isJwtEnabled .Values.filer.readinessProbe.tcpSocket }}
@@ -368,10 +371,12 @@ spec:
configMap:
name: {{ include "seaweedfs.fullname" . }}-notification-config
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
{{- if include "seaweedfs.securityConfigEnabled" . }}
- name: security-config
configMap:
name: {{ include "seaweedfs.fullname" . }}-security-config
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
- name: ca-cert
secret:
secretName: {{ include "seaweedfs.fullname" . }}-ca-cert

View File

@@ -188,11 +188,13 @@ spec:
readOnly: true
mountPath: /etc/seaweedfs/master.toml
subPath: master.toml
{{- if .Values.global.seaweedfs.enableSecurity }}
{{- if include "seaweedfs.securityConfigEnabled" . }}
- name: security-config
readOnly: true
mountPath: /etc/seaweedfs/security.toml
subPath: security.toml
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
- name: ca-cert
readOnly: true
mountPath: /usr/local/share/ca-certificates/ca/
@@ -287,10 +289,12 @@ spec:
- name: master-config
configMap:
name: {{ include "seaweedfs.fullname" . }}-master-config
{{- if .Values.global.seaweedfs.enableSecurity }}
{{- if include "seaweedfs.securityConfigEnabled" . }}
- name: security-config
configMap:
name: {{ include "seaweedfs.fullname" . }}-security-config
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
- name: ca-cert
secret:
secretName: {{ include "seaweedfs.fullname" . }}-ca-cert

View File

@@ -156,11 +156,13 @@ spec:
name: config-users
readOnly: true
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
{{- if include "seaweedfs.securityConfigEnabled" . }}
- name: security-config
readOnly: true
mountPath: /etc/seaweedfs/security.toml
subPath: security.toml
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
- name: ca-cert
readOnly: true
mountPath: /usr/local/share/ca-certificates/ca/
@@ -249,10 +251,12 @@ spec:
- name: logs
emptyDir: {}
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
{{- if include "seaweedfs.securityConfigEnabled" . }}
- name: security-config
configMap:
name: {{ include "seaweedfs.fullname" . }}-security-config
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
- name: ca-cert
secret:
secretName: {{ include "seaweedfs.fullname" . }}-ca-cert

View File

@@ -176,11 +176,13 @@ spec:
- mountPath: /etc/sw/ssh
name: config-ssh
readOnly: true
{{- if .Values.global.seaweedfs.enableSecurity }}
{{- if include "seaweedfs.securityConfigEnabled" . }}
- name: security-config
readOnly: true
mountPath: /etc/seaweedfs/security.toml
subPath: security.toml
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
- name: ca-cert
readOnly: true
mountPath: /usr/local/share/ca-certificates/ca/
@@ -264,10 +266,12 @@ spec:
- name: logs
emptyDir: {}
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
{{- if include "seaweedfs.securityConfigEnabled" . }}
- name: security-config
configMap:
name: {{ include "seaweedfs.fullname" . }}-security-config
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
- name: ca-cert
secret:
secretName: {{ include "seaweedfs.fullname" . }}-ca-cert

View File

@@ -332,6 +332,16 @@ Create the name of the service account to use
{{- .Values.global.seaweedfs.serviceAccountName | default "seaweedfs" -}}
{{- end -}}
{{/* True when security.toml should be rendered and mounted. volumeWrite is
excluded since it defaults to true. */}}
{{- define "seaweedfs.securityConfigEnabled" -}}
{{- $sec := (.Values.global.seaweedfs).securityConfig | default dict -}}
{{- $jwt := $sec.jwtSigning | default dict -}}
{{- if or .Values.global.seaweedfs.enableSecurity $jwt.volumeRead $jwt.filerWrite $jwt.filerRead -}}
true
{{- end -}}
{{- end -}}
{{/* S3 TLS cert/key arguments, using custom secret if s3.tlsSecret is set */}}
{{- define "seaweedfs.s3.tlsArgs" -}}
{{- $prefix := .prefix -}}

View File

@@ -1,5 +1,5 @@
{{- include "seaweedfs.compat" . -}}
{{- if .Values.global.seaweedfs.enableSecurity }}
{{- if include "seaweedfs.securityConfigEnabled" . }}
apiVersion: v1
kind: ConfigMap
metadata:
@@ -55,6 +55,7 @@ data:
key = "{{ dig "jwt" "filer_signing" "read" "key" (randAlphaNum 10 | b64enc) $securityConfig }}"
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
# all grpc tls authentications are mutual
# the values for the following ca, cert, and key are paths to the PERM files.
[grpc]
@@ -94,4 +95,5 @@ data:
[https.volume]
cert = ""
key = ""
{{- end }}
{{- end }}

View File

@@ -211,11 +211,13 @@ spec:
- name: idx
mountPath: "/idx/"
{{- end }}
{{- if $.Values.global.seaweedfs.enableSecurity }}
{{- if include "seaweedfs.securityConfigEnabled" $ }}
- name: security-config
readOnly: true
mountPath: /etc/seaweedfs/security.toml
subPath: security.toml
{{- end }}
{{- if $.Values.global.seaweedfs.enableSecurity }}
- name: ca-cert
readOnly: true
mountPath: /usr/local/share/ca-certificates/ca/
@@ -333,10 +335,12 @@ spec:
emptyDir: {}
{{- end }}
{{- end }}
{{- if $.Values.global.seaweedfs.enableSecurity }}
{{- if include "seaweedfs.securityConfigEnabled" $ }}
- name: security-config
configMap:
name: {{ include "seaweedfs.fullname" $ }}-security-config
{{- end }}
{{- if $.Values.global.seaweedfs.enableSecurity }}
- name: ca-cert
secret:
secretName: {{ include "seaweedfs.fullname" $ }}-ca-cert

View File

@@ -149,11 +149,13 @@ spec:
- name: worker-logs
mountPath: /logs
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
{{- if include "seaweedfs.securityConfigEnabled" . }}
- name: security-config
readOnly: true
mountPath: /etc/seaweedfs/security.toml
subPath: security.toml
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
- name: ca-cert
readOnly: true
mountPath: /usr/local/share/ca-certificates/ca/
@@ -252,10 +254,12 @@ spec:
persistentVolumeClaim:
claimName: {{ .Values.worker.logs.claimName }}
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
{{- if include "seaweedfs.securityConfigEnabled" . }}
- name: security-config
configMap:
name: {{ include "seaweedfs.fullname" . }}-security-config
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
- name: ca-cert
secret:
secretName: {{ include "seaweedfs.fullname" . }}-ca-cert

View File

@@ -18,6 +18,8 @@ global:
loggingLevel: 1
enableSecurity: false
masterServer: null
# filerWrite: true mounts security.toml on filer + admin without needing
# enableSecurity (mTLS); required for the Admin UI Users tab.
securityConfig:
jwtSigning:
volumeWrite: true