Switch to a slimmer distroless base image.

At a high level, it switches us to a distroless base container image, but that also includes several related bits:

- Add a writable /tmp but make the rest of our filesystems read-only at runtime.

- Condense our main server binaries into a single pinniped-server binary. This saves a bunch of space in
  the image due to duplicated library code. The correct behavior is dispatched based on `os.Args[0]`, and
  the `pinniped-server` binary is symlinked to `pinniped-concierge` and `pinniped-supervisor`.

- Strip debug symbols from our binaries. These aren't really useful in a distroless image anyway and all the
  normal stuff you'd expect to work, such as stack traces, still does.

- Add a separate `pinniped-concierge-kube-cert-agent` binary with "sleep" and "print" functionality instead of
  using builtin /bin/sleep and /bin/cat for the kube-cert-agent. This is split from the main server binary
  because the loading/init time of the main server binary was too large for the tiny resource footprint we
  established in our kube-cert-agent PodSpec. Using a separate binary eliminates this issue and the extra
  binary adds only around 1.5MiB of image size.

- Switch the kube-cert-agent code to use a JSON `{"tls.crt": "<b64 cert>", "tls.key": "<b64 key>"}` format.
  This is more robust to unexpected input formatting than the old code, which simply concatenated the files
  with some extra newlines and split on whitespace.

- Update integration tests that made now-invalid assumptions about the `pinniped-server` image.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
This commit is contained in:
Matt Moyer
2021-08-09 15:05:13 -04:00
committed by Monis Khan
parent a464c81711
commit 58bbffded4
19 changed files with 538 additions and 105 deletions
+11 -1
View File
@@ -116,6 +116,7 @@ spec:
scheduler.alpha.kubernetes.io/critical-pod: ""
spec:
securityContext:
readOnlyRootFilesystem: true
runAsUser: #@ data.values.run_as_user
runAsGroup: #@ data.values.run_as_group
serviceAccountName: #@ defaultResourceName()
@@ -138,10 +139,13 @@ spec:
limits:
cpu: "100m"
memory: "128Mi"
args:
command:
- pinniped-concierge
- --config=/etc/config/pinniped.yaml
- --downward-api-path=/etc/podinfo
volumeMounts:
- name: tmp
mountPath: /tmp
- name: config-volume
mountPath: /etc/config
- name: podinfo
@@ -167,7 +171,12 @@ spec:
periodSeconds: 10
failureThreshold: 3
volumes:
- name: tmp
emptyDir:
medium: Memory
sizeLimit: 100Mi
- name: config-volume
readOnly: true
configMap:
name: #@ defaultResourceNameWithSuffix("config")
- name: impersonation-proxy
@@ -177,6 +186,7 @@ spec:
- key: token
path: token
- name: podinfo
readOnly: true
downwardAPI:
items:
- path: "labels"
@@ -63,8 +63,8 @@ spec:
image: #@ data.values.image_repo + ":" + data.values.image_tag
#@ end
imagePullPolicy: IfNotPresent
command: #! override the default entrypoint
- /usr/local/bin/local-user-authenticator
command:
- local-user-authenticator
---
apiVersion: v1
kind: Service
+5 -3
View File
@@ -65,6 +65,7 @@ spec:
labels: #@ defaultLabel()
spec:
securityContext:
readOnlyRootFilesystem: true
runAsUser: #@ data.values.run_as_user
runAsGroup: #@ data.values.run_as_group
serviceAccountName: #@ defaultResourceName()
@@ -80,9 +81,8 @@ spec:
image: #@ data.values.image_repo + ":" + data.values.image_tag
#@ end
imagePullPolicy: IfNotPresent
command: #! override the default entrypoint
- /usr/local/bin/pinniped-supervisor
args:
command:
- pinniped-supervisor
- /etc/podinfo
- /etc/config/pinniped.yaml
resources:
@@ -131,9 +131,11 @@ spec:
failureThreshold: 3
volumes:
- name: config-volume
readOnly: true
configMap:
name: #@ defaultResourceNameWithSuffix("static-config")
- name: podinfo
readOnly: true
downwardAPI:
items:
- path: "labels"