The code in the branch previously lived in a private repository. It was made public by moving the code into the ci branch of the Pinniped repository in late 2024. The previous git history for these files was not copied from the private repository at the time of this migration.

Reporting an issue in this branch

Found a bug or would like to make an enhancement request? Please report issues in this repo.

Reporting security vulnerabilities

Please follow the procedure described in SECURITY.md.

Creating a release

When the team is preparing to ship a release, a maintainer will create a new GitHub Issue in this repo to collaboratively track progress on the release checklist. As tasks are completed, the team will check them off. When all the tasks are completed, the issue is closed.

The release checklist is committed to this repo as an issue template.

Pipelines

Pinniped uses Concourse for CI/CD. Our Concourse can be found at ci.pinniped.dev.

The following pipelines are implemented in this branch. Not all pipelines are necessarily publicly visible, although our goal is to make them all visible.

main

This is the main pipeline that runs on merges to main. It builds, tests, and (when manually triggered) releases from main.
pull-requests

This is a pipeline that triggers for each open pull request. It runs a smaller subset of the integration tests and validations as pinniped.
dockerfile-builders

This pipeline builds a bunch of custom utility container images that are used in our CI and testing.
- build-gi-cli (a container image that includes the GitHub CLI)
- build-github-pr-resource (a fork of the github-pr-resource with support for gating PRs for untrusted users)
- build-code-coverage-uploader (uploading code coverage during unit tests)
- build-eks-deployer-dockerfile (deploying our app to EKS clusters)
- build-k8s-app-deployer-dockerfile (deploying our app to clusters)
- build-pool-trigger-resource-dockerfile (an updated implementation of the pool-trigger-resource for use in our CI)
- build-integration-test-runner-dockerfile (running our integration tests)
- build-integration-test-runner-beta-dockerfile (running our integration tests with the latest Chrome beta version)
- build-deployment-yaml-formatter-dockerfile (templating our deployment YAML during a release)
- build-crane (copy and tag container images during release)
- build-k8s-code-generator-* (running our Kubernetes code generation under different Kubernetes dependency versions)
- build-test-dex (a Dex used during tests)
- build-test-cfssl (a cfssl used during tests)
- build-test-kubectl (a kubectl used during tests)
- build-test-forward-proxy (a Squid forward proxy used during tests)
- build-test-bitnami-ldap (an OpenLDAP used during tests)
cleanup-aws

This runs a script that runs aws-nuke against our test AWS account. This was occasionally needed because eksctl sometimes fails and leaks AWS resources. These resources cost money and use up our AWS quota. However, we seem to have worked around these issues and this pipeline has not been used for some time.

These jobs are only triggered manually. This is dangerous and should be used with care.
concourse-workers

Deploys worker replicas on a long-lived GKE cluster that runs the Concourse workers, and can scale them up or down.
go-compatibility

This pipeline runs nightly jobs that validate the compatibility of our code as a Go module in various contexts. We have jobs that test that our code compiles under older Go versions and that our CLI can be installed using go install.
security-scan

This pipeline has nightly jobs that run security scans on our current main branch and most recently released artifacts.

The tools we use are:
- sonatype-nexus-community/nancy, which scans Go module versions.
- aquasecurity/trivy, which scans container images and Go binaries.
- govulncheck, which scans Go code to find calls to known-vulnerable dependencies.
This pipeline also has a job called all-golang-deps-updated which automatically submits PRs to update all direct dependencies in Pinniped's go.mod file, and update the Golang and distroless container images used in Pinniped's Dockerfiles.
kind-node-builder

A nightly build job which uses the latest version of kind to build the HEAD of master of Kubernetes as a container image that can be used to deploy kind clusters. Other pipelines use this container image to install Pinniped and run integration tests. This gives us insight in any compatibility problems with the upcoming next release of Kubernetes.

Deploying pipeline changes

After any shared tasks (./pipelines/shared-tasks) or helpers (./pipelines/shared-helpers) are edited, the commits must be pushed to the ci branch of this repository to take effect.

After editing any CI secrets or pipeline definitions, a maintainer must run the corresponding ./pipelines/$PIPELINE_NAME/update-pipeline.sh script to apply the changes to Concourse. To deploy all pipelines, a maintainer can run ./pipelines/update-all-pipelines.sh. Don't forget to commit and push your changes after applying them!

Github webhooks for pipelines

Some pipelines use github webhooks to trigger resource checks, rather than the default of polling every minute, to make these pipelines more responsive and use fewer compute resources for running checks. Refer to places where webhook_token is configured in various pipeline.yml files.

To make these webhooks work, they must be defined on the GitHub repo's settings.

Installing and operating Concourse

See infra/README.md for details about how Concourse was installed and how it can be operated.

Acceptance environments

In addition to the many ephemeral Kubernetes clusters we use for testing, we also deploy a long-running acceptance environment.

Google Kubernetes Engine (GKE) in the gke-acceptance-cluster cluster in our GCP project in the us-central1-c availability zone.

To access this cluster, download the kubeconfig to gke-acceptance.yaml by running:

KUBECONFIG=gke-acceptance.yaml gcloud container clusters get-credentials gke-acceptance-cluster --project "$PINNIPED_GCP_PROJECT" --zone us-central1-c

The above command assumes that you have already set PINNIPED_GCP_PROJECT to be the name of the GCP project.

CI secrets

We use Google Secret Manager on GCP to store build/test/release secrets. These secrets are only available to the maintainers.

Using the gcloud secrets list command or the web console, you can list the available secrets. The content of each secret is a YAML file with secret key/value pairs. You can also use the ./hack/edit-gcloud-secret.sh <secretName> script to edit or inspect each secret.

Setting Up Active Directory Test Environment

To test the ActiveDirectoryIdentityProvider functionality, we have a long-running Active Directory Domain Controller server instance in our GCP account. See AD-SETUP.md for details.

Running integration tests on your laptop using AD

The relevant environment variables can be pulled from the secret manager via the hack/get-active-directory-env-vars.sh script. This can be used by maintainers with Pinniped's /hack/prepare-for-integration-tests.sh script in the following way:

# Must authenticate to glcoud to access the secret manager.
gcloud auth login
# In the pinniped repo's main branch or in your PR branch:
hack/prepare-for-integration-tests.sh --get-active-directory-vars "$HOME/path/to/pinniped-ci-branch/hack/get-active-directory-env-vars.sh"

Running integration tests on your laptop using GitHub

The relevant environment variables can be pulled from the secret manager via the hack/get-github-env-vars.sh script. This can be used by maintainers with Pinniped's /hack/prepare-for-integration-tests.sh script in the following way:

# Must authenticate to glcoud to access the secret manager.
gcloud auth login
# In the pinniped repo's main branch or in your PR branch:
hack/prepare-for-integration-tests.sh --get-github-vars "$HOME/path/to/pinniped-ci-branch/hack/get-github-env-vars.sh"

License

Pinniped is open source and licensed under Apache License Version 2.0. See LICENSE.

Languages

Go 97.3%

Shell 1.6%

HTML 0.4%

SCSS 0.4%

CSS 0.1%

Other 0.1%

README.md

Pinniped's ci branch