Merge pull request #5644 from Lyndon-Li/release-1.10

Rollback goreleaser upgrade

.github/ISSUE_TEMPLATE/bug_report.md

@@ -10,10 +10,13 @@ about: Tell us about a problem you are experiencing
 
 **What did you expect to happen:**
 
+**The following information will help us better understand what's going on**:
 
-**The output of the following commands will help us better understand what's going on**:
-(Pasting long output into a [GitHub gist](https://gist.github.com) or other pastebin is fine.)
+_If you are using velero v1.7.0+:_  
+Please use `velero debug  --backup <backupname> --restore <restorename>` to generate the support bundle, and attach to this issue, more options please refer to `velero debug --help` 
 
+_If you are using earlier versions:_  
+Please provide the output of the following commands (Pasting long output into a [GitHub gist](https://gist.github.com) or other pastebin is fine.)
 - `kubectl logs deployment/velero -n velero`
 - `velero backup describe <backupname>` or `kubectl get backup/<backupname> -n velero -o yaml`
 - `velero backup logs <backupname>`

.github/auto-assignees.yml

@@ -0,0 +1,41 @@
+---
+# This assigns a PR to its author
+addAssignees: author
+
+reviewers:
+  # The default reviewers
+  defaults:
+    - maintainers
+
+  groups:
+    maintainers:
+      - dsu-igeek
+      - sseago
+      - reasonerjt
+      - ywk253100
+      - blackpiglet
+      - qiuming-best
+      - shubham-pampattiwar
+      - Lyndon-Li
+
+    tech-writer:
+      - a-mccarthy
+
+files:
+  'site/**':
+    - tech-writer
+  '**/*.md':
+    - tech-writer
+  # Technical design requests are ".md" files but should
+  # be reviewed by maintainers
+  '/design/**':
+    - maintainers
+
+options:
+  ignore_draft: true
+  ignored_keywords:
+    - WIP
+    - wip
+    - DO NOT MERGE
+  enable_group_assignment: true
+  number_of_reviewers: 2

.github/auto_assign.yml

@@ -1,14 +0,0 @@
-addReviewers: true
-addAssignees: author
-
-# Only require 2, random reviewers.
-# TODO expand this to support using reviewGroups
-numberOfReviewers: 2
-
-reviewers:
-  - nrb
-  - ashish-amarnath
-  - carlisia
-  - zubron
-  - dsu-igeek
-  - jenting

.github/dependabot.yml

@@ -0,0 +1,12 @@
+version: 2
+updates:
+  # Dependencies listed in go.mod
+  - package-ecosystem: "gomod"
+    directory: "/" # Location of package manifests
+    schedule:
+      interval: "weekly"
+    labels:
+      - "kind/changelog-not-required"
+    ignore:
+      - dependency-name: "*"
+        update-types: ["version-update:semver-major", "version-update:semver-minor", "version-update:semver-patch"]

.github/pull_request_template.md

@@ -9,5 +9,5 @@ Fixes #(issue)
 # Please indicate you've done the following:
 
 - [ ] [Accepted the DCO](https://velero.io/docs/v1.5/code-standards/#dco-sign-off). Commits without the DCO will delay acceptance.
-- [ ] [Created a changelog file](https://velero.io/docs/v1.5/code-standards/#adding-a-changelog) or added `/kind changelog-not-required`.
+- [ ] [Created a changelog file](https://velero.io/docs/v1.5/code-standards/#adding-a-changelog) or added `/kind changelog-not-required` as a comment on this pull request.
 - [ ] Updated the corresponding documentation in `site/content/docs/main`.

.github/stale.yml

@@ -0,0 +1,44 @@
+# Number of days of inactivity before an issue becomes stale
+daysUntilStale: 60
+# Number of days of inactivity before a stale issue is closed
+daysUntilClose: 14
+# Issues with these labels will never be considered stale
+exemptLabels:
+  - Epic
+  - Area/CLI
+  - Area/Cloud/AWS
+  - Area/Cloud/Azure
+  - Area/Cloud/GCP
+  - Area/Cloud/vSphere
+  - Area/CSI
+  - Area/Design
+  - Area/Documentation
+  - Area/Plugins
+  - Bug
+  - Enhancement/User
+  - kind/requirement
+  - kind/refactor
+  - kind/tech-debt
+  - limitation
+  - Needs investigation
+  - Needs triage
+  - Needs Product
+  - P0 - Hair on fire
+  - P1 - Important
+  - P2 - Long-term important
+  - P3 - Wouldn't it be nice if...
+  - Product Requirements
+  - Restic - GA
+  - Restic
+  - release-blocker
+  - Security
+# Label to use when marking an issue as stale
+staleLabel: staled
+# Comment to post when marking an issue as stale. Set to `false` to disable
+markComment: >
+  This issue has been automatically marked as stale because it has not had
+  recent activity. It will be closed if no further activity occurs. Thank you
+  for your contributions.
+# Comment to post when closing a stale issue. Set to `false` to disable
+closeComment: >
+  Closing the stale issue.

.github/workflows/auto_assign_prs.yml

@@ -1,6 +1,8 @@
-name: "Auto Assign PR Reviewers"
-# pull_request_target means that this will run on pull requests, but in the context of the base repo.
-# This should mean PRs from forks are supported.
+---
+name: "Auto Assign Author"
+
+# pull_request_target means that this will run on pull requests, but in
+# the context of the base repo. This should mean PRs from forks are supported.
 on:
   pull_request_target:
     types: [opened, reopened, ready_for_review]
@@ -10,7 +12,8 @@ jobs:
   add-reviews:
     runs-on: ubuntu-latest
     steps:
-      - uses: kentaro-m/auto-assign-action@v1.1.1
+      - name: Set the author of a PR as the assignee
+        uses: kentaro-m/auto-assign-action@v1.1.1
         with:
-          configuration-path: ".github/auto_assign.yml"
-          repo-token: "${{ secrets.GITHUB_TOKEN }}"
+          configuration-path: ".github/auto-assignees.yml"
+          repo-token: "${{ secrets.GITHUB_TOKEN }}"

.github/workflows/auto_request_review.yml

@@ -0,0 +1,17 @@
+---
+name: "Auto Request Review"
+
+on:
+  pull_request_target:
+    types: [opened, ready_for_review, reopened]
+
+jobs:
+  auto-request-review:
+    name: Auto Request Review
+    runs-on: ubuntu-latest
+    steps:
+      - name: Request a PR review based on files types/paths, and/or groups the author belongs to
+        uses: necojackarc/auto-request-review@v0.7.0
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+          config: .github/auto-assignees.yml

.github/workflows/crds-verify-kind.yaml

@@ -11,6 +11,11 @@ jobs:
   build-cli:
     runs-on: ubuntu-latest
     steps:
+      - name: Set up Go
+        uses: actions/setup-go@v2
+        with:
+          go-version: 1.18.8
+        id: go
       # Look for a CLI that's made for this PR
       - name: Fetch built CLI
         id: cache
@@ -52,14 +57,13 @@ jobs:
       matrix:
         # Latest k8s versions. There's no series-based tag, nor is there a latest tag.
         k8s:
-          - 1.15.12
-          - 1.16.15
-          - 1.17.17
-          - 1.18.15
           - 1.19.7
           - 1.20.2
           - 1.21.1
           - 1.22.0
+          - 1.23.6
+          - 1.24.2
+          - 1.25.3
     # All steps run in parallel unless otherwise specified.
     # See https://docs.github.com/en/actions/learn-github-actions/managing-complex-workflows#creating-dependent-jobs
     steps:
@@ -77,7 +81,7 @@ jobs:
             velero-${{ github.event.pull_request.number }}-
       - uses: engineerd/setup-kind@v0.5.0
         with:
-          version: "v0.11.1"
+          version: "v0.17.0"
           image: "kindest/node:v${{ matrix.k8s }}"
       - name: Install CRDs
         run: |

.github/workflows/e2e-test-kind.yaml

@@ -11,6 +11,11 @@ jobs:
   build:
     runs-on: ubuntu-latest
     steps:
+      - name: Set up Go
+        uses: actions/setup-go@v2
+        with:
+          go-version: 1.18.8
+        id: go
       # Look for a CLI that's made for this PR
       - name: Fetch built CLI
         id: cli-cache
@@ -55,17 +60,20 @@ jobs:
     strategy:
       matrix:
         k8s:
-          # doesn't cover 1.15 as 1.15 doesn't support "apiextensions.k8s.io/v1" that is needed for the case
-          #- 1.15.12
-          - 1.16.15
-          - 1.17.17
-          - 1.18.15
-          - 1.19.7
-          - 1.20.2
-          - 1.21.1
-          - 1.22.0
+          - 1.19.16
+          - 1.20.15
+          - 1.21.12
+          - 1.22.9
+          - 1.23.6
+          - 1.24.0
+          - 1.25.3
       fail-fast: false
     steps:
+      - name: Set up Go
+        uses: actions/setup-go@v2
+        with:
+          go-version: 1.18.8
+        id: go
       - name: Check out the code
         uses: actions/checkout@v2
       - name: Install MinIO
@@ -73,7 +81,7 @@ jobs:
           docker run -d --rm -p 9000:9000 -e "MINIO_ACCESS_KEY=minio" -e "MINIO_SECRET_KEY=minio123" -e "MINIO_DEFAULT_BUCKETS=bucket,additional-bucket" bitnami/minio:2021.6.17-debian-10-r7
       - uses: engineerd/setup-kind@v0.5.0
         with:
-          version: "v0.11.1"
+          version: "v0.17.0"
           image: "kindest/node:v${{ matrix.k8s }}"
       - name: Fetch built CLI
         id: cli-cache
@@ -105,10 +113,22 @@ jobs:
           aws_access_key_id=minio
           aws_secret_access_key=minio123
           EOF
+
+          # Match kubectl version to k8s server version
+          curl -LO https://dl.k8s.io/release/v${{ matrix.k8s }}/bin/linux/amd64/kubectl
+          sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
+
           GOPATH=~/go CLOUD_PROVIDER=kind \
               OBJECT_STORE_PROVIDER=aws BSL_CONFIG=region=minio,s3ForcePathStyle="true",s3Url=http://$(hostname -i):9000 \
               CREDS_FILE=/tmp/credential BSL_BUCKET=bucket \
               ADDITIONAL_OBJECT_STORE_PROVIDER=aws ADDITIONAL_BSL_CONFIG=region=minio,s3ForcePathStyle="true",s3Url=http://$(hostname -i):9000 \
               ADDITIONAL_CREDS_FILE=/tmp/credential ADDITIONAL_BSL_BUCKET=additional-bucket \
-              GINKGO_FOCUS=Basic VELERO_IMAGE=velero:pr-test \
-              make -C test/e2e run
+              GINKGO_FOCUS='Basic\]\[ClusterResource' VELERO_IMAGE=velero:pr-test \
+              make -C test/e2e run
+        timeout-minutes: 30
+      - name: Upload debug bundle
+        if: ${{ failure() }}
+        uses: actions/upload-artifact@v2
+        with:
+          name: DebugBundle
+          path: /home/runner/work/velero/velero/test/e2e/debug-bundle*

.github/workflows/opened-issues-triage.yml

@@ -1,15 +0,0 @@
-name: Move new issues into Triage
-
-on:
-  issues:
-    types: [opened]
-
-jobs:
-  automate-project-columns:
-    runs-on: ubuntu-latest
-    steps:
-      - uses: alex-page/github-project-automation-plus@v0.3.0
-        with:
-          project: "Velero Support Board"
-          column: "New"
-          repo-token: ${{ secrets.GH_TOKEN }}

.github/workflows/pr-changelog-check.yml

@@ -1,5 +1,9 @@
 name: Pull Request Changelog Check
-on: [pull_request]
+# by setting `on: [pull_request]`, that means action will be trigger when PR is opened, synchronize, reopened.
+# Add labeled and unlabeled events too.
+on:
+  pull_request:
+    types: [opened, synchronize, reopened, labeled, unlabeled]
 jobs:
 
   build:

.github/workflows/pr-ci-check.yml

@@ -5,9 +5,13 @@ jobs:
     name: Run CI
     runs-on: ubuntu-latest
     steps:
+      - name: Set up Go
+        uses: actions/setup-go@v2
+        with:
+          go-version: 1.18.8
+        id: go
       - name: Check out the code
         uses: actions/checkout@v2
-
       - name: Fetch cached go modules
         uses: actions/cache@v2
         with:
@@ -15,6 +19,11 @@ jobs:
           key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
           restore-keys: |
             ${{ runner.os }}-go-
-
       - name: Make ci
         run: make ci
+      - name: Upload test coverage
+        uses: codecov/codecov-action@v2
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: coverage.out
+          verbose: true

.github/workflows/pr-codespell.yml

@@ -14,7 +14,7 @@ jobs:
       uses: codespell-project/actions-codespell@master
       with:
         # ignore the config/.../crd.go file as it's generated binary data that is edited elswhere.
-        skip: .git,*.png,*.jpg,*.woff,*.ttf,*.gif,*.ico,./config/crd/v1beta1/crds/crds.go,./config/crd/v1/crds/crds.go
-        ignore_words_list: iam,aks,ist,bridget,ue
+        skip: .git,*.png,*.jpg,*.woff,*.ttf,*.gif,*.ico,./config/crd/v1beta1/crds/crds.go,./config/crd/v1/crds/crds.go,./go.sum,./LICENSE
+        ignore_words_list: iam,aks,ist,bridget,ue,shouldnot
         check_filenames: true
         check_hidden: true

.github/workflows/push-builder.yml

@@ -2,7 +2,9 @@ name: build-image
 
 on:
   push:
-    branches: [ main ]
+    branches:
+      - 'main'
+      - 'release-**'
     paths:
     - 'hack/build-image/Dockerfile'
 
@@ -12,7 +14,16 @@ jobs:
     runs-on: ubuntu-latest
     steps:
 
-    - uses: actions/checkout@master
+    - uses: actions/checkout@v2
+      with:
+        # The default value is "1" which fetches only a single commit. If we merge PR without squash or rebase,
+        # there are at least two commits: the first one is the merge commit and the second one is the real commit
+        # contains the changes.
+        # As we use the Dockerfile's commit ID as the tag of the build-image, fetching only 1 commit causes the merge
+        # commit ID to be the tag.
+        # While when running make commands locally, as the local git repository usually contains all commits, the Dockerfile's
+        # commit ID is the second one. This is mismatch with the images in Dockerhub
+        fetch-depth: 2
 
     - name: Build
       run: make build-image

.github/workflows/push.yml

@@ -2,7 +2,9 @@ name: Main CI
 
 on:
   push:
-    branches: [ main ]
+    branches:
+      - 'main'
+      - 'release-**'
     tags:
       - '*'
 
@@ -16,11 +18,22 @@ jobs:
     - name: Set up Go
       uses: actions/setup-go@v2
       with:
-        go-version: 1.15
+        go-version: 1.18.8
       id: go
 
-    - name: Check out code into the Go module directory
-      uses: actions/checkout@v2
+    - uses: actions/checkout@v3
+
+    # Fix issue of setup-gcloud
+    - run: |
+        sudo apt-get install python2.7
+        export CLOUDSDK_PYTHON="/usr/bin/python2"
+
+    - uses: google-github-actions/setup-gcloud@v0
+      with:
+        version: '285.0.0'
+        service_account_key: ${{ secrets.GCS_SA_KEY }}
+        export_default_credentials: true
+    - run: gcloud info
 
     - name: Set up QEMU
       id: qemu
@@ -40,9 +53,50 @@ jobs:
     - name: Test
       run: make test
 
+    - name: Upload test coverage
+      uses: codecov/codecov-action@v2
+      with:
+        token: ${{ secrets.CODECOV_TOKEN }}
+        files: coverage.out
+        verbose: true
+
     # Only try to publish the container image from the root repo; forks don't have permission to do so and will always get failures.
     - name: Publish container image
       if: github.repository == 'vmware-tanzu/velero'
       run: |
+        # Build and push Velero image to docker registry
         docker login -u ${{ secrets.DOCKER_USER }} -p ${{ secrets.DOCKER_PASSWORD }}
-        ./hack/docker-push.sh
+        VERSION=$(./hack/docker-push.sh | grep 'VERSION:' | awk -F: '{print $2}' | xargs)
+
+        # Upload Velero image package to GCS
+        source hack/ci/build_util.sh
+        BIN=velero
+        RESTORE_HELPER_BIN=velero-restore-helper
+        GCS_BUCKET=velero-builds
+        VELERO_IMAGE=${BIN}-${VERSION}
+        VELERO_RESTORE_HELPER_IMAGE=${RESTORE_HELPER_BIN}-${VERSION}
+        VELERO_IMAGE_FILE=${VELERO_IMAGE}.tar.gz
+        VELERO_RESTORE_HELPER_IMAGE_FILE=${VELERO_RESTORE_HELPER_IMAGE}.tar.gz
+        VELERO_IMAGE_BACKUP_FILE=${VELERO_IMAGE}-'build.'${GITHUB_RUN_NUMBER}.tar.gz
+        VELERO_RESTORE_HELPER_IMAGE_BACKUP_FILE=${VELERO_RESTORE_HELPER_IMAGE}-'build.'${GITHUB_RUN_NUMBER}.tar.gz
+
+        cp ${VELERO_IMAGE_FILE} ${VELERO_IMAGE_BACKUP_FILE}
+        cp ${VELERO_RESTORE_HELPER_IMAGE_FILE} ${VELERO_RESTORE_HELPER_IMAGE_BACKUP_FILE}
+
+        uploader ${VELERO_IMAGE_FILE} ${GCS_BUCKET}
+        uploader ${VELERO_RESTORE_HELPER_IMAGE_FILE} ${GCS_BUCKET}
+        uploader ${VELERO_IMAGE_BACKUP_FILE} ${GCS_BUCKET}
+        uploader ${VELERO_RESTORE_HELPER_IMAGE_BACKUP_FILE} ${GCS_BUCKET}
+
+    # Use the JSON key in secret to login gcr.io
+    - uses: 'docker/login-action@v1'
+      with:
+        registry: 'gcr.io' # or REGION.docker.pkg.dev
+        username: '_json_key'
+        password: '${{ secrets.GCR_SA_KEY }}'
+
+    # Push image to GCR to facilitate some environments that have rate limitation to docker hub, e.g. vSphere.
+    - name: Publish container image to GCR
+      if: github.repository == 'vmware-tanzu/velero'
+      run: |
+        REGISTRY=gcr.io/velero-gcp ./hack/docker-push.sh

.gitignore

@@ -24,8 +24,6 @@ _testmain.go
 *.test
 *.prof
 
-debug
-
 /velero
 .idea/
 
@@ -48,5 +46,7 @@ _tiltbuild
 tilt-resources/tilt-settings.json
 tilt-resources/velero_v1_backupstoragelocation.yaml
 tilt-resources/deployment.yaml
-tilt-resources/restic.yaml
-tilt-resources/cloud
+tilt-resources/node-agent.yaml
+tilt-resources/cloud
+
+test/e2e/report.xml

.goreleaser.yml

@@ -14,7 +14,7 @@
 
 dist: _output
 builds:
-  - main: ./cmd/velero/main.go
+  - main: ./cmd/velero/velero.go
     env:
       - CGO_ENABLED=0
     goos:
@@ -27,11 +27,9 @@ builds:
       - arm64
       - ppc64le
     ignore:
-      # don't build arm/arm64 for darwin or windows
+      # don't build arm for darwin and arm/arm64 for windows
       - goos: darwin
         goarch: arm
-      - goos: darwin
-        goarch: arm64
       - goos: darwin
         goarch: ppc64le
       - goos: windows

ADOPTERS.md

@@ -13,7 +13,8 @@ If you're using Velero and want to add your organization to this list,
 <a href="https://banzaicloud.com/" border="0" target="_blank"><img alt="banzaicloud.com" src="site/static/img/adopters/banzaicloud.svg" height="50"></a>&nbsp; &nbsp; &nbsp;
 <a href="https://sighup.io/" border="0" target="_blank"><img alt="sighup.io" src="site/static/img/adopters/sighup.svg" height="50"></a>&nbsp; &nbsp; &nbsp;
 <a href="https://mayadata.io/" border="0" target="_blank"><img alt="mayadata.io" src="site/static/img/adopters/mayadata.svg" height="50"></a>&nbsp; &nbsp; &nbsp;
-
+<a href="https://www.replicated.com/" border="0" target="_blank"><img alt="replicated.com" src="site/static/img/adopters/replicated-logo-red.svg" height="50"></a>
+<a href="https://cloudcasa.io/" border="0" target="_blank"><img alt="cloudcasa.io" src="site/static/img/adopters/cloudcasa.svg" height="50"></a>
 ## Success Stories
 
 Below is a list of adopters of Velero in **production environments** that have
@@ -39,7 +40,9 @@ We have integrated our [solution with Velero][11] to provide our customers with
 Kyma [integrates with Velero][41] to effortlessly back up and restore Kyma clusters with all its resources. Velero capabilities allow Kyma users to define and run manual and scheduled backups in order to successfully handle a disaster-recovery scenario.
 
 **[Red Hat][50]**  
-Red Hat has developed the [Cluster Application Migration Tool][51] which uses [Velero and Restic][52] to drive the migration of applications between OpenShift clusters.
+Red Hat has developed 2 operators for the OpenShift platform:
+- [Migration Toolkit for Containers][51] (Crane): This operator uses [Velero and Restic][52] to drive the migration of applications between OpenShift clusters.
+- [OADP (OpenShift API for Data Protection) Operator][53]: This operator sets up and installs Velero on the OpenShift platform, allowing users to backup and restore applications.
 
 **[Dell EMC][70]**  
 For Kubernetes environments, [PowerProtect Data Manager][71] leverages the Container Storage Interface (CSI) framework to take snapshots to back up the persistent data or the data that the application creates e.g. databases. [Dell EMC leverages Velero][72] to backup the namespace configuration files (also known as Namespace meta data) for enterprise grade data protection.
@@ -53,7 +56,13 @@ MayaData is a large user of Velero as well as a contributor. MayaData offers a D
 
 **[Okteto][93]**  
 Okteto integrates Velero in [Okteto Cloud][94] and [Okteto Enterprise][95] to periodically backup and restore our clusters for disaster recovery. Velero is also a core software building block to provide namespace cloning capabilities, a feature that allows our users cloning staging environments into their personal development namespace for providing production-like development environments.
-​
+
+**[Replicated][100]**<br>
+Replicated uses the Velero open source project to enable snapshots in [KOTS][101] to backup Kubernetes manifests & persistent volumes. In addition to the default functionality that Velero provides, [KOTS][101] provides a detailed interface in the [Admin Console][102] that can be used to manage the storage destination and schedule, and to perform and monitor the backup and restore process.<br>
+
+**[CloudCasa][103]**<br>
+[Catalogic Software][104] integrates Velero with [CloudCasa][103] - A Smart Home in the Cloud for Backups. CloudCasa is a simple, scalable, cloud-native solution providing data protection and disaster recovery as a service. This solution is built using Kubernetes for protecting Kubernetes clusters.<br>
+
 ## Adding your organization to the list of Velero Adopters
 
 If you are using Velero and would like to be included in the list of `Velero Adopters`, add an SVG version of your logo to the `site/static/img/adopters` directory in this repo and submit a [pull request][3] with your change. Name the image file something that reflects your company (e.g., if your company is called Acme, name the image acme.png). See this for an example [PR][4].
@@ -82,6 +91,7 @@ If you would like to add your logo to a future `Adopters of Velero` section on [
 [50]: https://redhat.com
 [51]: https://github.com/fusor/mig-operator
 [52]: https://github.com/fusor/mig-operator/blob/master/docs/usage/2.md
+[53]: https://github.com/openshift/oadp-operator
 
 [60]: https://banzaicloud.com
 [61]: https://banzaicloud.com/products/pipeline/
@@ -102,3 +112,10 @@ If you would like to add your logo to a future `Adopters of Velero` section on [
 [93]: https://okteto.com
 [94]: https://cloud.okteto.com
 [95]: https://okteto.com/enterprise/
+
+[100]: https://www.replicated.com
+[101]: https://kots.io
+[102]: https://kots.io/kotsadm/snapshots/overview/
+
+[103]: https://cloudcasa.io/
+[104]: https://www.catalogicsoftware.com/

CHANGELOG.md

@@ -1,7 +1,10 @@
 ## Current release:
-  * [CHANGELOG-1.6.md][16]
+  * [CHANGELOG-1.9.md][19]
 
 ## Older releases:
+  * [CHANGELOG-1.8.md][18]
+  * [CHANGELOG-1.7.md][17]
+  * [CHANGELOG-1.6.md][16]
   * [CHANGELOG-1.5.md][15]
   * [CHANGELOG-1.4.md][14]
   * [CHANGELOG-1.3.md][13]
@@ -19,6 +22,9 @@
   * [CHANGELOG-0.3.md][1]
 
 
+[19]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.9.md
+[18]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.8.md
+[17]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.7.md
 [16]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.6.md
 [15]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.5.md
 [14]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.4.md

Dockerfile

@@ -11,7 +11,7 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-FROM --platform=$BUILDPLATFORM golang:1.15 as builder-env
+FROM --platform=$BUILDPLATFORM golang:1.18.8-bullseye as builder-env
 
 ARG GOPROXY
 ARG PKG
@@ -29,8 +29,6 @@ WORKDIR /go/src/github.com/vmware-tanzu/velero
 
 COPY . /go/src/github.com/vmware-tanzu/velero
 
-RUN apt-get update && apt-get install -y bzip2
-
 FROM --platform=$BUILDPLATFORM builder-env as builder
 
 ARG TARGETOS
@@ -45,18 +43,16 @@ ENV GOOS=${TARGETOS} \
     GOARM=${TARGETVARIANT}
 
 RUN mkdir -p /output/usr/bin && \
-    bash ./hack/download-restic.sh && \
     export GOARM=$( echo "${GOARM}" | cut -c2-) && \
+    bash ./hack/build-restic.sh && \
     go build -o /output/${BIN} \
     -ldflags "${LDFLAGS}" ${PKG}/cmd/${BIN}
 
-FROM ubuntu:focal
+FROM gcr.io/distroless/base-debian11@sha256:4b22ca3c68018333c56f8dddcf1f8b55f32889f2dd12d28ab60856eba1130d04
 
 LABEL maintainer="Nolan Brubaker <brubakern@vmware.com>"
 
-RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -qq -y ca-certificates tzdata && rm -rf /var/lib/apt/lists/*
-
 COPY --from=builder /output /
 
-USER nobody:nogroup
+USER nonroot:nonroot
 

MAINTAINERS.md

@@ -6,24 +6,34 @@
 
 | Maintainer | GitHub ID | Affiliation |
 | --------------- | --------- | ----------- |
-| Carlisia Thompson | [carlisia](https://github.com/carlisia) | [VMware](https://www.github.com/vmware/) |
-| Nolan Brubaker | [nrb](https://github.com/nrb) | [VMware](https://www.github.com/vmware/) |
-| Ashish Amarnath | [ashish-amarnath](https://github.com/ashish-amarnath) | [VMware](https://www.github.com/vmware/) |
-| Bridget McErlean | [zubron](https://github.com/zubron) | [VMware](https://www.github.com/vmware/) |
-| Dave Smith-Uchida | [dsu-igeek](https://github.com/dsu-igeek) | [VMware](https://www.github.com/vmware/) |
-| JenTing Hsiao | [jenting](https://github.com/jenting) | [SUSE](https://github.com/SUSE/) 
+| Dave Smith-Uchida | [dsu-igeek](https://github.com/dsu-igeek) | [Kasten](https://github.com/kastenhq/) |
+| Scott Seago | [sseago](https://github.com/sseago) | [OpenShift](https://github.com/openshift)
+| Daniel Jiang | [reasonerjt](https://github.com/reasonerjt) | [VMware](https://www.github.com/vmware/)
+| Wenkai Yin | [ywk253100](https://github.com/ywk253100) | [VMware](https://www.github.com/vmware/) |
+| Xun Jiang | [blackpiglet](https://github.com/blackpiglet) | [VMware](https://www.github.com/vmware/) |
+| Ming Qiu | [qiuming-best](https://github.com/qiuming-best) | [VMware](https://www.github.com/vmware/) |
+| Shubham Pampattiwar | [shubham-pampattiwar](https://github.com/shubham-pampattiwar) | [OpenShift](https://github.com/openshift)
+| Yonghui Li | [Lyndon-Li](https://github.com/Lyndon-Li) | [VMware](https://www.github.com/vmware/) |
 
 ## Emeritus Maintainers
 * Adnan Abdulhussein ([prydonius](https://github.com/prydonius))
 * Andy Goldstein ([ncdc](https://github.com/ncdc))
 * Steve Kriss ([skriss](https://github.com/skriss))
+* Carlos Panato ([cpanato](https://github.com/cpanato))
+* Nolan Brubaker ([nrb](https://github.com/nrb))
+* Ashish Amarnath ([ashish-amarnath](https://github.com/ashish-amarnath))
+* Carlisia Thompson ([carlisia](https://github.com/carlisia))
+* Bridget McErlean ([zubron](https://github.com/zubron))
+* JenTing Hsiao ([jenting](https://github.com/jenting))
 
 ## Velero Contributors & Stakeholders
 
 | Feature Area | Lead |
 | ----------------------------- | :---------------------: |
-| Technical Lead | Nolan Brubaker (nrb) |
-| Kubernetes CSI Liaison | Nolan Brubaker (nrb), Ashish Amarnath (ashish-amarnath) |
-| Deployment | Carlisia Thompson (carlisia), Carlos Tadeu Panato Junior (cpanato), JenTing Hsiao (jenting) | 
-| Community Management | Jonas Rosland (jonasrosland) |
-| Product Management | Michael Michael (michmike) |
+| Architect | Dave Smith-Uchida [dsu-igeek](https://github.com/dsu-igeek) |
+| Technical Lead | Daniel Jiang [reasonerjt](https://github.com/reasonerjt) |
+| Kubernetes CSI Liaison |  |
+| Deployment |  |
+| Community Management | Orlin Vasilev [OrlinVasilev](https://github.com/OrlinVasilev) |
+| Product Management | Pradeep Kumar Chaturvedi [pradeepkchaturvedi](https://github.com/pradeepkchaturvedi) |
+

Makefile

@@ -81,10 +81,10 @@ buildx not enabled, refusing to run this recipe
 see: https://velero.io/docs/main/build-from-source/#making-images-and-updating-velero for more info
 endef
 
-# The version of restic binary to be downloaded for power architecture
-RESTIC_VERSION ?= 0.12.0
+# The version of restic binary to be downloaded
+RESTIC_VERSION ?= 0.13.1
 
-CLI_PLATFORMS ?= linux-amd64 linux-arm linux-arm64 darwin-amd64 windows-amd64 linux-ppc64le
+CLI_PLATFORMS ?= linux-amd64 linux-arm linux-arm64 darwin-amd64 darwin-arm64 windows-amd64 linux-ppc64le
 BUILDX_PLATFORMS ?= $(subst -,/,$(ARCH))
 BUILDX_OUTPUT_TYPE ?= docker
 
@@ -112,19 +112,20 @@ GOPROXY ?= https://proxy.golang.org
 # If you want to build all containers, see the 'all-containers' rule.
 all:
 	@$(MAKE) build
-	@$(MAKE) build BIN=velero-restic-restore-helper
+	@$(MAKE) build BIN=velero-restore-helper
 
 build-%:
 	@$(MAKE) --no-print-directory ARCH=$* build
-	@$(MAKE) --no-print-directory ARCH=$* build BIN=velero-restic-restore-helper
+	@$(MAKE) --no-print-directory ARCH=$* build BIN=velero-restore-helper
 
 all-build: $(addprefix build-, $(CLI_PLATFORMS))
 
 all-containers: container-builder-env
 	@$(MAKE) --no-print-directory container
-	@$(MAKE) --no-print-directory container BIN=velero-restic-restore-helper
+	@$(MAKE) --no-print-directory container BIN=velero-restore-helper
 
 local: build-dirs
+# Add DEBUG=1 to enable debug locally
 	GOOS=$(GOOS) \
 	GOARCH=$(GOARCH) \
 	VERSION=$(VERSION) \
@@ -162,6 +163,7 @@ shell: build-dirs build-env
 	@# under $GOPATH).
 	@docker run \
 		-e GOFLAGS \
+		-e GOPROXY \
 		-i $(TTY) \
 		--rm \
 		-u $$(id -u):$$(id -g) \
@@ -207,6 +209,12 @@ endif
 	--build-arg=RESTIC_VERSION=$(RESTIC_VERSION) \
 	-f $(VELERO_DOCKERFILE) .
 	@echo "container: $(IMAGE):$(VERSION)"
+ifeq ($(BUILDX_OUTPUT_TYPE)_$(REGISTRY), registry_velero)
+	docker pull $(IMAGE):$(VERSION)
+	rm -f $(BIN)-$(VERSION).tar
+	docker save $(IMAGE):$(VERSION) -o $(BIN)-$(VERSION).tar
+	gzip -f $(BIN)-$(VERSION).tar
+endif
 
 SKIP_TESTS ?=
 test: build-dirs
@@ -290,12 +298,12 @@ push-build-image:
 	@# this target will push the build-image it assumes you already have docker
 	@# credentials needed to accomplish this.
 	@# Pushing will be skipped if a custom Dockerfile was used to build the image.
-	ifneq "$(origin BUILDER_IMAGE_DOCKERFILE)" "file"
-		@echo "Dockerfile for builder image has been overridden"
-		@echo "Skipping push of custom image"
-	else
-		docker push $(BUILDER_IMAGE)
-	endif
+ifneq "$(origin BUILDER_IMAGE_DOCKERFILE)" "file"
+	@echo "Dockerfile for builder image has been overridden"
+	@echo "Skipping push of custom image"
+else
+	docker push $(BUILDER_IMAGE)
+endif
 
 build-image-hugo:
 	cd site && docker build --pull -t $(HUGO_IMAGE) .
@@ -338,9 +346,9 @@ changelog:
 #		PUBLISH=false \
 #		make release
 #
-# To run the release, which will publish a *DRAFT* GitHub release in github.com/vmware-tanzu/velero 
+# To run the release, which will publish a *DRAFT* GitHub release in github.com/vmware-tanzu/velero
 # (you still need to review/publish the GitHub release manually):
-#		GITHUB_TOKEN=your-github-token \ 
+#		GITHUB_TOKEN=your-github-token \
 #		RELEASE_NOTES_FILE=changelogs/CHANGELOG-1.2.md \
 #		PUBLISH=true \
 #		make release
@@ -359,11 +367,11 @@ serve-docs: build-image-hugo
 	-it -p 1313:1313 \
 	$(HUGO_IMAGE) \
 	hugo server --bind=0.0.0.0 --enableGitInfo=false
-# gen-docs generates a new versioned docs directory under site/content/docs. 
+# gen-docs generates a new versioned docs directory under site/content/docs.
 # Please read the documentation in the script for instructions on how to use it.
 gen-docs:
 	@hack/release-tools/gen-docs.sh
 
 .PHONY: test-e2e
 test-e2e: local
-	$(MAKE) -C test/e2e run
+	$(MAKE) -e VERSION=$(VERSION) -C test/e2e run

PROJECT

@@ -1,7 +0,0 @@
-domain: io
-repo: github.com/vmware-tanzu/velero
-resources:
-- group: velero
-  kind: BackupStorageLocation
-  version: v1
-version: "2"

README.md

@@ -34,6 +34,26 @@ If you are ready to jump in and test, add code, or help with documentation, foll
 
 See [the list of releases][6] to find out about feature changes.
 
+### Velero compatibility matrix
+
+The following is a list of the supported Kubernetes versions for each Velero version.
+
+| Velero version | Expected Kubernetes version compatibility| Tested on Kubernetes version|
+|----------------|--------------------|--------------------|
+| 1.10           | 1.16-latest        |  1.22.5, 1.23.8, 1.24.6 and 1.25.1 |
+| 1.9            | 1.16-latest        |  1.20.5, 1.21.2, 1.22.5, 1.23, and 1.24 |
+| 1.8            | 1.16-latest        |  |
+| 1.6.3-1.7.1    | 1.12-latest        ||
+| 1.60-1.6.2     | 1.12-1.21          ||
+| 1.5            | 1.12-1.21          ||
+| 1.4            | 1.10-1.21          | |
+
+Velero supports IPv4, IPv6, and dual stack environments. Support for this was tested against Velero v1.8.
+
+The Velero maintainers are continuously working to expand testing coverage, but are not able to test every combination of Velero and supported Kubernetes versions for each Velero release. The table above is meant to track the current testing coverage and the expected supported Kubernetes versions for each Velero version. If you have a question about test coverage before v1.9, please reach out in the [#velero-users](https://kubernetes.slack.com/archives/C6VCGP4MT) Slack channel.
+
+If you are interested in using a different version of Kubernetes with a given Velero version, we'd recommend that you perform testing before installing or upgrading your environment. For full information around capabilities within a release, also see the Velero [release notes](https://github.com/vmware-tanzu/velero/releases) or Kubernetes [release notes](https://github.com/kubernetes/kubernetes/tree/master/CHANGELOG). See the Velero [support page](https://velero.io/docs/latest/support-process/) for information about supported versions of Velero.
+
 [1]: https://github.com/vmware-tanzu/velero/workflows/Main%20CI/badge.svg
 [2]: https://github.com/vmware-tanzu/velero/actions?query=workflow%3A"Main+CI"
 [4]: https://github.com/vmware-tanzu/velero/issues

ROADMAP.md

@@ -1,69 +1 @@
-## Velero Roadmap
-
-### About this document
-This document provides a link to the [Velero Project boards](https://github.com/vmware-tanzu/velero/projects) that serves as the up to date description of items that are in the release pipeline. The release boards have separate swim lanes based on prioritization. Most items are gathered from the community or include a feedback loop with the community. This should serve as a reference point for Velero users and contributors to understand where the project is heading, and help determine if a contribution could be conflicting with a longer term plan. 
-
-### How to help?
-Discussion on the roadmap can take place in threads under [Issues](https://github.com/vmware-tanzu/velero/issues) or in [community meetings](https://velero.io/community/). Please open and comment on an issue if you want to provide suggestions, use cases, and feedback to an item in the roadmap. Please review the roadmap to avoid potential duplicated effort.
-
-### How to add an item to the roadmap?
-One of the most important aspects in any open source community is the concept of proposals. Large changes to the codebase and / or new features should be preceded by a [proposal](https://github.com/vmware-tanzu/velero/blob/main/GOVERNANCE.md#proposal-process) in our repo.
-For smaller enhancements, you can open an issue to track that initiative or feature request.
-We work with and rely on community feedback to focus our efforts to improve Velero and maintain a healthy roadmap.
-
-### Current Roadmap
-The following table includes the current roadmap for Velero. If you have any questions or would like to contribute to Velero, please attend a [community meeting](https://velero.io/community/) to discuss with our team. If you don't know where to start, we are always looking for contributors that will help us reduce technical, automation, and documentation debt.
-Please take the timelines & dates as proposals and goals. Priorities and requirements change based on community feedback, roadblocks encountered, community contributions, etc. If you depend on a specific item, we encourage you to attend community meetings to get updated status information, or help us deliver that feature by contributing to Velero.
-
-`Last Updated: March 2021`
-
-#### 1.7.0 Roadmap
-The release roadmap is split into Core items that are required for the release, desired items that may be removed from the
-release and opportunistic items that will be added to the release if possible.
-
-##### Core items
-
-|Issue|Description|
-|---|---|
-|[3493](https://github.com/vmware-tanzu/velero/issues/3493)|[Carvel](https://github.com/vmware-tanzu/velero/issues/3493) based installation (in addition to the existing *velero install* CLI).|
-|[3531](https://github.com/vmware-tanzu/velero/issues/3531)|Test plan for Velero|
-|[675](https://github.com/vmware-tanzu/velero/issues/675)|Velero command to generate debugging information.  Will integrate with [Crashd - Crash Diagnostics](https://github.com/vmware-tanzu/velero/issues/675)|
-|[2066](https://github.com/vmware-tanzu/velero/issues/2066)|CSI Snapshots GA|
-|[3285](https://github.com/vmware-tanzu/velero/issues/3285)|Support Velero plugin versioning|
-|[1975](https://github.com/vmware-tanzu/velero/issues/1975)|IPV6 support|
-
-
-
-##### Desired items
-|Issue|Description|
-|---|---|
-|[3533](https://github.com/vmware-tanzu/velero/issues/3533)|Upload Progress Monitoring|
-|[2922](https://github.com/vmware-tanzu/velero/issues/2922)|Plugin timeouts|
-|[3500](https://github.com/vmware-tanzu/velero/issues/3500)|Use distroless containers as a base|
-|[3535](https://github.com/vmware-tanzu/velero/issues/3535)|Design doc for multiple cluster support|
-|[3536](https://github.com/vmware-tanzu/velero/issues/3536)|Manifest for backup/restore|
-
-##### Opportunistic items
-|Issue|Description|
-|---|---|
-|Issues TBD|Controller migrations|
-
-#### Long term roadmap items
-|Theme|Description|Timeline|
-|---|---|---|
-|Restic Improvements|Introduce improvements in annotating resources for Restic backup|TBD|
-|Extensibility|Add restore hooks for enhanced recovery scenarios|TBD|
-|CSI|Continue improving the CSI snapshot capabilities and participate in the upstream K8s CSI community|1.7.0 + Long running (dependent on CSI working group)|
-|Backup/Restore|Improvements to long-running copy operations from a performance and reliability standpoint|1.7.0|
-|Quality/Reliability| Enable automated end-to-end testing |1.6.0|
-|UX|Improvements to install and configuration user experience|Dec 2020|
-|Restic Improvements|Improve the use of Restic in Velero and offer stable support|TBD|
-|Perf & Scale|Introduce a scalable model by using a worker pod for each backup/restore operation and improve operations|1.8.0|
-|Backup/Restore|Better backup and restore semantics for certain Kubernetes resources like stateful sets, operators|2.0|
-|Security|Enable the use of custom credential providers|1.6.0|
-|Self-Service & Multitenancy|Reduce friction by enabling developers to backup their namespaces via self-service. Introduce a Velero multi-tenancy model, enabling owners of namespaces to backup and restore within their access scope|TBD|
-|Backup/Restore|Cross availability zone or region backup and restore|TBD|
-|Application Consistency|Offer blueprints for backing up and restoring popular applications|TBD|
-|Backup/Restore|Data only backup and restore|TBD|
-|Backup/Restore|Introduce the ability to overwrite existing objects during a restore|TBD|
-|Backup/Restore|What-if dry run for backup and restore|1.7.0|
+# Please go to the [Velero Wiki](https://github.com/vmware-tanzu/velero/wiki/) to see our latest roadmap, archived roadmaps and roadmap guidance.

Tiltfile

@@ -7,7 +7,7 @@ k8s_yaml([
     'config/crd/v1/bases/velero.io_downloadrequests.yaml',
     'config/crd/v1/bases/velero.io_podvolumebackups.yaml',
     'config/crd/v1/bases/velero.io_podvolumerestores.yaml',
-    'config/crd/v1/bases/velero.io_resticrepositories.yaml',
+    'config/crd/v1/bases/velero.io_backuprepositories.yaml',
     'config/crd/v1/bases/velero.io_restores.yaml',
     'config/crd/v1/bases/velero.io_schedules.yaml',
     'config/crd/v1/bases/velero.io_serverstatusrequests.yaml',
@@ -16,8 +16,8 @@ k8s_yaml([
 
 # default values
 settings = {
-    "default_registry": "",
-    "enable_restic": False,
+    "default_registry": "docker.io/velero",
+    "use_node_agent": False,
     "enable_debug": False,
     "debug_continue_on_start": True,  # Continue the velero process by default when in debug mode
     "create_backup_locations": False,
@@ -34,9 +34,9 @@ k8s_yaml(kustomize('tilt-resources'))
 k8s_yaml('tilt-resources/deployment.yaml')
 if settings.get("enable_debug"):
     k8s_resource('velero', port_forwards = '2345')
-    # TODO: Need to figure out how to apply port forwards for all restic pods
-if settings.get("enable_restic"):
-    k8s_yaml('tilt-resources/restic.yaml')
+    # TODO: Need to figure out how to apply port forwards for all node-agent pods
+if settings.get("use_node_agent"):
+    k8s_yaml('tilt-resources/node-agent.yaml')
 if settings.get("create_backup_locations"):
     k8s_yaml('tilt-resources/velero_v1_backupstoragelocation.yaml')
 if settings.get("setup-minio"):
@@ -50,7 +50,7 @@ git_sha = str(local("git rev-parse HEAD", quiet = True, echo_off = True)).strip(
 
 tilt_helper_dockerfile_header = """
 # Tilt image
-FROM golang:1.15.3 as tilt-helper
+FROM golang:1.18.8 as tilt-helper
 
 # Support live reloading with Tilt
 RUN wget --output-document /restart.sh --quiet https://raw.githubusercontent.com/windmilleng/rerun-process-wrapper/master/restart.sh  && \
@@ -90,20 +90,20 @@ def get_debug_flag():
 # Set up a local_resource build of the Velero binary. The binary is written to _tiltbuild/velero.
 local_resource(
     "velero_server_binary",
-    cmd = 'cd ' + '.' + ';mkdir -p _tiltbuild;PKG=. BIN=velero GOOS=linux GOARCH=amd64 GIT_SHA=' + git_sha + ' VERSION=main GIT_TREE_STATE=dirty OUTPUT_DIR=_tiltbuild ' + get_debug_flag() + ' ./hack/build.sh',
+    cmd = 'cd ' + '.' + ';mkdir -p _tiltbuild;PKG=. BIN=velero GOOS=linux GOARCH=amd64 GIT_SHA=' + git_sha + ' VERSION=main GIT_TREE_STATE=dirty OUTPUT_DIR=_tiltbuild ' + get_debug_flag() + ' REGISTRY=' + settings.get("default_registry") + ' ./hack/build.sh',
     deps = ["cmd", "internal", "pkg"],
     ignore = ["pkg/cmd"],
 )
 
 local_resource(
     "velero_local_binary",
-    cmd = 'cd ' + '.' + ';mkdir -p _tiltbuild/local;PKG=. BIN=velero GOOS=' + local_goos + ' GOARCH=amd64 GIT_SHA=' + git_sha + ' VERSION=main GIT_TREE_STATE=dirty OUTPUT_DIR=_tiltbuild/local ' + get_debug_flag() + ' ./hack/build.sh',
+    cmd = 'cd ' + '.' + ';mkdir -p _tiltbuild/local;PKG=. BIN=velero GOOS=' + local_goos + ' GOARCH=amd64 GIT_SHA=' + git_sha + ' VERSION=main GIT_TREE_STATE=dirty OUTPUT_DIR=_tiltbuild/local ' + get_debug_flag() + ' REGISTRY=' + settings.get("default_registry") + ' ./hack/build.sh',
     deps = ["internal", "pkg/cmd"],
 )
 
 local_resource(
     "restic_binary",
-    cmd = 'cd ' + '.' + ';mkdir -p _tiltbuild/restic; BIN=velero GOOS=' + local_goos + ' GOARCH=amd64 RESTIC_VERSION=0.12.0 OUTPUT_DIR=_tiltbuild/restic ./hack/download-restic.sh',
+    cmd = 'cd ' + '.' + ';mkdir -p _tiltbuild/restic; BIN=velero GOOS=linux GOARCH=amd64 RESTIC_VERSION=0.13.1 OUTPUT_DIR=_tiltbuild/restic ./hack/download-restic.sh',
 )
 
 # Note: we need a distro with a bash shell to exec into the Velero container

assets/README.md

@@ -0,0 +1,11 @@
+# Velero Assets
+
+This folder contains logo images for Velero in gray (for light backgrounds) and white (for dark backgrounds like black tshirts or dark mode!) – horizontal and stacked… in .eps and .svg.
+
+## Some general guidelines for usage
+
+• Don’t alter the logos/graphics: resize, reformat, recolor. Keep them intact.
+
+• Don’t separate the word mark (Velero) from the icon) – we are still building a strong name and identity – and the logo by itself doesn’t have any strong recognition or association with as yet: so best practice keep the two together. Nike kept its name with the swoosh for quite some time before the swoosh became iconic.
+
+• Don’t append the name to another brand – let it stand alone!

assets/one-line/199150-vmw-os-lgo-velero-final_gry.svg

@@ -0,0 +1,105 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!-- Generator: Adobe Illustrator 24.0.3, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
+<svg version="1.1" id="Layer_5" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
+	 viewBox="0 0 431.3 150" style="enable-background:new 0 0 431.3 150;" xml:space="preserve">
+<style type="text/css">
+	.st0{fill:#FFFFFF;}
+	.st1{fill:none;}
+	.st2{fill:#009BDB;}
+	.st3{fill:#717074;}
+</style>
+<g>
+	<g>
+		<g>
+			<path class="st3" d="M196.6,55.8l-18.2,41.2h-5.1l-18.2-41.2h5.1l15.7,35.5l15.6-35.5H196.6z"/>
+			<path class="st3" d="M206.6,60.3v13.6h22.4v4.4h-22.4v14.3h24.8v4.4h-29.6V55.8h29.6v4.4H206.6z"/>
+			<path class="st3" d="M265.7,92.6v4.4h-27.2V55.8h4.7v36.8H265.7z"/>
+			<path class="st3" d="M275.7,60.3v13.6h22.4v4.4h-22.4v14.3h24.8v4.4H271V55.8h29.6v4.4H275.7z"/>
+			<path class="st3" d="M338,75.3c-1.1,1.9-2.6,3.4-4.6,4.6c-2,1.2-4.2,1.9-6.6,2.2l10.5,14.9H332l-10.4-14.8h-9.1v14.8h-4.7V55.8
+				h16.7c2.8,0,5.4,0.6,7.7,1.7c2.3,1.1,4.1,2.7,5.5,4.7c1.3,2,2,4.3,2,6.8C339.6,71.3,339.1,73.4,338,75.3z M312.4,77.8h11.2
+				c3.4,0,6.1-0.8,8.2-2.3c2.1-1.6,3.1-3.7,3.1-6.4c0-2.7-1-4.9-3.1-6.4c-2.1-1.6-4.8-2.3-8.2-2.3h-11.2V77.8z"/>
+			<path class="st3" d="M354.4,94.9c-3.3-1.9-5.8-4.5-7.8-7.8c-1.9-3.3-2.9-6.8-2.9-10.6c0-3.8,1-7.3,2.9-10.6
+				c1.9-3.3,4.5-5.9,7.8-7.8c3.3-1.9,6.8-2.9,10.5-2.9c3.8,0,7.2,1,10.5,2.9c3.2,1.9,5.8,4.5,7.7,7.8c1.9,3.3,2.9,6.8,2.9,10.6
+				c0,3.8-1,7.3-2.9,10.6c-1.9,3.3-4.5,5.9-7.7,7.8c-3.2,1.9-6.7,2.9-10.5,2.9C361.2,97.8,357.7,96.8,354.4,94.9z M373,91.1
+				c2.5-1.6,4.5-3.6,6-6.2c1.5-2.6,2.2-5.4,2.2-8.5c0-3-0.7-5.8-2.2-8.4c-1.5-2.6-3.5-4.7-6-6.2c-2.5-1.5-5.2-2.3-8.1-2.3
+				c-2.9,0-5.6,0.8-8.1,2.3c-2.5,1.5-4.5,3.6-6,6.2c-1.5,2.6-2.2,5.4-2.2,8.4c0,3,0.8,5.9,2.2,8.5c1.5,2.6,3.5,4.7,6,6.2
+				s5.2,2.3,8.1,2.3C367.8,93.5,370.5,92.7,373,91.1z"/>
+		</g>
+	</g>
+	<g>
+		<g>
+			<path class="st1" d="M132.2,77.1c0-0.7,0.1-1.4,0.1-2s0-1.4-0.1-2V77.1z"/>
+			<path class="st2" d="M117,109.3c-3.3,0-4.9,1.2-6.7,2.7c-2,1.6-4.2,3.4-8.6,3.4c-4.3,0-6.6-1.8-8.6-3.4c-1.8-1.4-3.4-2.7-6.7-2.7
+				c-3.3,0-4.9,1.2-6.7,2.7c-2,1.6-4.2,3.4-8.6,3.4c-4.3,0-6.6-1.8-8.6-3.4c-1.8-1.4-3.4-2.7-6.7-2.7c-3.3,0-4.9,1.2-6.7,2.7
+				c-1.3,1-2.7,2.2-4.8,2.8c8.3,7.3,18.9,12,30.5,13c0.3,0,0.6,0.1,1,0.1c1.1,0.1,2.3,0.1,3.4,0.1c1.2,0,2.3-0.1,3.4-0.1
+				c0.3,0,0.6,0,1-0.1c14.2-1.2,26.8-8,35.6-18.2C118.7,109.4,117.9,109.3,117,109.3z"/>
+			<path class="st2" d="M40.8,69.8c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2s6.1,1.6,8,3.2c1.9,1.5,3.6,2.9,7.2,2.9
+				c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2c4,0,6.1,1.6,8,3.2c1.9,1.5,3.6,2.9,7.2,2.9c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2
+				c4,0,6.1,1.6,8,3.2c1.8,1.5,3.6,2.8,7,2.9c-0.5-4.8-1.6-9.5-3.3-13.8c-1.8-0.6-3.1-1.6-4.4-2.6c-1.9-1.5-3.7-2.9-7.4-2.9
+				c-3.7,0-5.5,1.4-7.4,2.9c-1.9,1.5-3.9,3.1-7.9,3.1c-4,0-5.9-1.6-7.9-3.1c-1.9-1.5-3.7-2.9-7.4-2.9c-3.7,0-5.5,1.4-7.4,2.9
+				c-1.9,1.5-3.9,3.1-7.9,3.1c-4,0-5.9-1.6-7.9-3.1c-1.9-1.5-3.7-2.9-7.4-2.9c-3.7,0-5.5,1.4-7.4,2.9c-1.9,1.5-3.9,3.1-7.9,3.1
+				c-4,0-5.9-1.6-7.9-3.1c-0.4-0.3-0.9-0.7-1.3-1c-1.7,3.6-3.1,7.5-3.9,11.6c2.7,0.5,4.3,1.7,5.9,3C35.5,68.4,37.2,69.8,40.8,69.8z"
+				/>
+			<path class="st2" d="M40.8,55.7c3.7,0,5.5-1.4,7.4-2.9c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.9,1.5,3.7,2.9,7.4,2.9
+				c3.7,0,5.5-1.4,7.4-2.9c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.9,1.5,3.7,2.9,7.4,2.9c3.7,0,5.5-1.4,7.4-2.9
+				c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.1,0.8,2.1,1.7,3.4,2.2c-7.9-19.2-26.9-32.8-48.9-32.8c-20.8,0-38.7,12-47.4,29.5
+				c0.5,0.4,1,0.7,1.4,1.1C35.3,54.2,37.1,55.7,40.8,55.7z"/>
+			<path class="st2" d="M117,94.6c-3.4,0-5,1.3-6.8,2.8c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3c-1.9-1.5-3.5-2.8-6.8-2.8
+				c-3.4,0-5,1.3-6.8,2.8c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3c-1.9-1.5-3.5-2.8-6.8-2.8c-3.4,0-5,1.3-6.8,2.8
+				c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3c-0.5-0.4-1-0.8-1.5-1.1c2.6,6,6.3,11.4,10.9,16c2.6-0.2,4-1.4,5.6-2.6
+				c2-1.6,4.2-3.4,8.6-3.4s6.6,1.8,8.6,3.4c1.8,1.4,3.4,2.7,6.7,2.7c3.3,0,4.9-1.2,6.7-2.7c2-1.6,4.2-3.4,8.6-3.4
+				c4.3,0,6.6,1.8,8.6,3.4c1.8,1.4,3.4,2.7,6.7,2.7c3.3,0,4.9-1.2,6.7-2.7c2-1.6,4.2-3.4,8.6-3.4c1.8,0,3.3,0.3,4.5,0.8
+				c1.9-2.5,3.5-5.1,5-7.9c-1-0.6-1.8-1.2-2.6-1.8C122,95.8,120.3,94.6,117,94.6z"/>
+			<path class="st2" d="M132.1,71.2c-4,0-6-1.6-8-3.2c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2
+				c-4,0-6.1-1.6-8-3.2c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2c-4,0-6.1-1.6-8-3.2
+				c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2c-4,0-6.1-1.6-8-3.2c-1.5-1.2-3-2.3-5.3-2.7
+				c-0.5,2.5-0.8,5.1-0.9,7.7c0,0.7-0.1,1.4-0.1,2c0,0.7,0,1.4,0.1,2c0,0.3,0,0.6,0,0.9c3.5,0.3,5.4,1.8,7.2,3.2
+				c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8c1.9-1.5,4.1-3.2,8.2-3.2s6.3,1.7,8.2,3.2c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8
+				c1.9-1.5,4.1-3.2,8.2-3.2c4.1,0,6.3,1.7,8.2,3.2c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8c1.9-1.5,4.1-3.2,8.2-3.2
+				c4.1,0,6.3,1.7,8.2,3.2c1.7,1.3,3.3,2.6,6.3,2.8c0.3-1.6,0.5-3.2,0.6-4.9c0-0.6,0.1-1.3,0.1-1.9V73
+				C132.2,72.4,132.1,71.2,132.1,71.2z"/>
+			<path class="st2" d="M117,79.9c-3.5,0-5.2,1.4-7,2.8c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.8-1.4-3.5-2.8-7-2.8
+				c-3.5,0-5.2,1.4-7,2.8c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.8-1.4-3.5-2.8-7-2.8c-3.5,0-5.2,1.4-7,2.8
+				c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.6-1.3-3.1-2.5-5.8-2.8c0.4,4.5,1.4,8.7,2.8,12.8c1.9,0.6,3.2,1.7,4.4,2.7
+				c1.9,1.5,3.5,2.8,6.8,2.8c3.4,0,5-1.3,6.8-2.8c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c1.9,1.5,3.5,2.8,6.8,2.8
+				c3.4,0,5-1.3,6.8-2.8c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c1.9,1.5,3.5,2.8,6.8,2.8c3.4,0,5-1.3,6.8-2.8
+				c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c0.7,0.6,1.4,1.1,2.2,1.6c1.6-3.5,2.8-7.2,3.6-11.1c-3.5-0.3-5.4-1.8-7.2-3.2
+				C122.2,81.2,120.4,79.9,117,79.9z"/>
+			<path class="st0" d="M108.4,109.6c-1.8,1.4-3.4,2.7-6.7,2.7c-3.3,0-4.9-1.2-6.7-2.7c-2-1.6-4.2-3.4-8.6-3.4
+				c-4.3,0-6.6,1.8-8.6,3.4c-1.8,1.4-3.4,2.7-6.7,2.7c-3.3,0-4.9-1.2-6.7-2.7c-2-1.6-4.2-3.4-8.6-3.4s-6.6,1.8-8.6,3.4
+				c-1.6,1.3-3,2.4-5.6,2.6c0.9,0.9,1.8,1.7,2.7,2.5c2.1-0.6,3.5-1.8,4.8-2.8c1.8-1.4,3.4-2.7,6.7-2.7c3.3,0,4.9,1.2,6.7,2.7
+				c2,1.6,4.2,3.4,8.6,3.4c4.3,0,6.6-1.8,8.6-3.4c1.8-1.4,3.4-2.7,6.7-2.7c3.3,0,4.9,1.2,6.7,2.7c2,1.6,4.2,3.4,8.6,3.4
+				c4.3,0,6.6-1.8,8.6-3.4c1.8-1.4,3.4-2.7,6.7-2.7c0.9,0,1.7,0.1,2.4,0.3c0.7-0.8,1.4-1.7,2-2.5c-1.2-0.5-2.7-0.8-4.5-0.8
+				C112.6,106.2,110.4,108,108.4,109.6z"/>
+			<path class="st0" d="M117,92.1c-4.2,0-6.4,1.7-8.4,3.3c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.9-1.5-4.1-3.3-8.4-3.3
+				c-4.2,0-6.4,1.7-8.4,3.3c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.9-1.5-4.1-3.3-8.4-3.3c-4.2,0-6.4,1.7-8.4,3.3
+				c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.2-1-2.5-2-4.4-2.7c0.4,1.2,0.9,2.3,1.4,3.5c0.5,0.3,1,0.7,1.5,1.1
+				c1.9,1.5,4.1,3.3,8.4,3.3c4.2,0,6.4-1.7,8.4-3.3c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c1.9,1.5,4.1,3.3,8.4,3.3
+				c4.2,0,6.4-1.7,8.4-3.3c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c1.9,1.5,4.1,3.3,8.4,3.3c4.2,0,6.4-1.7,8.4-3.3
+				c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c0.8,0.6,1.6,1.3,2.6,1.8c0.4-0.7,0.7-1.5,1.1-2.2c-0.8-0.4-1.4-1-2.2-1.6
+				C123.4,93.8,121.2,92.1,117,92.1z"/>
+			<path class="st0" d="M117,77.9c-4.1,0-6.3,1.7-8.2,3.2c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.9-1.5-4.1-3.2-8.2-3.2
+				c-4.1,0-6.3,1.7-8.2,3.2c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.9-1.5-4.1-3.2-8.2-3.2c-4.1,0-6.3,1.7-8.2,3.2
+				c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.7-1.4-3.7-2.9-7.2-3.2c0,0.6,0.1,1.3,0.1,1.9c2.7,0.3,4.2,1.5,5.8,2.8
+				c1.9,1.5,4.1,3.2,8.2,3.2c4.1,0,6.3-1.7,8.2-3.2c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.9,1.5,4.1,3.2,8.2,3.2
+				c4.1,0,6.3-1.7,8.2-3.2c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.9,1.5,4.1,3.2,8.2,3.2c4.1,0,6.3-1.7,8.2-3.2
+				c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.7,1.4,3.7,2.9,7.2,3.2c0.1-0.6,0.2-1.3,0.3-1.9c-3-0.2-4.6-1.4-6.3-2.8
+				C123.3,79.6,121.1,77.9,117,77.9z"/>
+			<path class="st0" d="M40.8,71.2c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9c3.6,0,5.3,1.4,7.2,2.9c2,1.6,4,3.2,8,3.2
+				c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9c3.6,0,5.3,1.4,7.2,2.9c2,1.6,4,3.2,8,3.2c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9
+				c3.6,0,5.3,1.4,7.2,2.9c1.9,1.5,4,3.1,8,3.2l0-0.1c0-0.4-0.1-0.8-0.1-1.3c-3.4-0.1-5.2-1.4-7-2.9c-2-1.6-4-3.2-8-3.2
+				c-4,0-6.1,1.6-8,3.2c-1.9,1.5-3.6,2.9-7.2,2.9c-3.6,0-5.3-1.4-7.2-2.9c-2-1.6-4-3.2-8-3.2c-4,0-6.1,1.6-8,3.2
+				c-1.9,1.5-3.6,2.9-7.2,2.9c-3.6,0-5.3-1.4-7.2-2.9c-2-1.6-4-3.2-8-3.2s-6.1,1.6-8,3.2c-1.9,1.5-3.6,2.9-7.2,2.9
+				c-3.6,0-5.3-1.4-7.2-2.9c-1.6-1.3-3.2-2.5-5.9-3c-0.1,0.4-0.2,0.9-0.3,1.3c2.4,0.4,3.8,1.5,5.3,2.7
+				C34.7,69.6,36.7,71.2,40.8,71.2z"/>
+			<path class="st0" d="M40.8,56.5c4,0,5.9-1.6,7.9-3.1c1.9-1.5,3.7-2.9,7.4-2.9c3.7,0,5.5,1.4,7.4,2.9c1.9,1.5,3.9,3.1,7.9,3.1
+				c4,0,5.9-1.6,7.9-3.1c1.9-1.5,3.7-2.9,7.4-2.9c3.7,0,5.5,1.4,7.4,2.9c1.9,1.5,3.9,3.1,7.9,3.1c4,0,5.9-1.6,7.9-3.1
+				c1.9-1.5,3.7-2.9,7.4-2.9c3.7,0,5.5,1.4,7.4,2.9c1.3,1,2.5,2,4.4,2.6c-0.1-0.3-0.3-0.7-0.4-1c-1.3-0.6-2.4-1.4-3.4-2.2
+				c-1.9-1.5-3.9-3.1-7.9-3.1c-4,0-5.9,1.6-7.9,3.1c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9c-1.9-1.5-3.9-3.1-7.9-3.1
+				c-4,0-5.9,1.6-7.9,3.1c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9C62,51.2,60,49.6,56,49.6c-4,0-5.9,1.6-7.9,3.1
+				c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9C33,52.4,32.5,52,32,51.6c-0.1,0.2-0.3,0.5-0.4,0.8c0.4,0.3,0.9,0.6,1.3,1
+				C34.8,54.9,36.8,56.5,40.8,56.5z"/>
+		</g>
+	</g>
+</g>
+</svg>

assets/one-line/199150-vmw-os-lgo-velero-final_wht.svg

@@ -0,0 +1,105 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!-- Generator: Adobe Illustrator 24.0.3, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
+<svg version="1.1" id="Layer_5" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
+	 viewBox="0 0 431.3 150" style="enable-background:new 0 0 431.3 150;" xml:space="preserve">
+<style type="text/css">
+	.st0{fill:#FFFFFF;}
+	.st1{fill:none;}
+	.st2{fill:#009BDB;}
+	.st3{fill:#717074;}
+</style>
+<g>
+	<g>
+		<g>
+			<path class="st0" d="M196.6,55.8l-18.2,41.2h-5.1l-18.2-41.2h5.1l15.7,35.5l15.6-35.5H196.6z"/>
+			<path class="st0" d="M206.6,60.3v13.6h22.4v4.4h-22.4v14.3h24.8v4.4h-29.6V55.8h29.6v4.4H206.6z"/>
+			<path class="st0" d="M265.7,92.6v4.4h-27.2V55.8h4.7v36.8H265.7z"/>
+			<path class="st0" d="M275.7,60.3v13.6h22.4v4.4h-22.4v14.3h24.8v4.4H271V55.8h29.6v4.4H275.7z"/>
+			<path class="st0" d="M338,75.3c-1.1,1.9-2.6,3.4-4.6,4.6c-2,1.2-4.2,1.9-6.6,2.2l10.5,14.9H332l-10.4-14.8h-9.1v14.8h-4.7V55.8
+				h16.7c2.8,0,5.4,0.6,7.7,1.7c2.3,1.1,4.1,2.7,5.5,4.7c1.3,2,2,4.3,2,6.8C339.6,71.3,339.1,73.4,338,75.3z M312.4,77.8h11.2
+				c3.4,0,6.1-0.8,8.2-2.3c2.1-1.6,3.1-3.7,3.1-6.4c0-2.7-1-4.9-3.1-6.4c-2.1-1.6-4.8-2.3-8.2-2.3h-11.2V77.8z"/>
+			<path class="st0" d="M354.4,94.9c-3.3-1.9-5.8-4.5-7.8-7.8c-1.9-3.3-2.9-6.8-2.9-10.6c0-3.8,1-7.3,2.9-10.6
+				c1.9-3.3,4.5-5.9,7.8-7.8c3.3-1.9,6.8-2.9,10.5-2.9c3.8,0,7.2,1,10.5,2.9c3.2,1.9,5.8,4.5,7.7,7.8c1.9,3.3,2.9,6.8,2.9,10.6
+				c0,3.8-1,7.3-2.9,10.6c-1.9,3.3-4.5,5.9-7.7,7.8c-3.2,1.9-6.7,2.9-10.5,2.9C361.2,97.8,357.7,96.8,354.4,94.9z M373,91.1
+				c2.5-1.6,4.5-3.6,6-6.2c1.5-2.6,2.2-5.4,2.2-8.5c0-3-0.7-5.8-2.2-8.4c-1.5-2.6-3.5-4.7-6-6.2c-2.5-1.5-5.2-2.3-8.1-2.3
+				c-2.9,0-5.6,0.8-8.1,2.3c-2.5,1.5-4.5,3.6-6,6.2c-1.5,2.6-2.2,5.4-2.2,8.4c0,3,0.8,5.9,2.2,8.5c1.5,2.6,3.5,4.7,6,6.2
+				s5.2,2.3,8.1,2.3C367.8,93.5,370.5,92.7,373,91.1z"/>
+		</g>
+	</g>
+	<g>
+		<g>
+			<path class="st1" d="M132.2,77.1c0-0.7,0.1-1.4,0.1-2s0-1.4-0.1-2V77.1z"/>
+			<path class="st2" d="M117,109.3c-3.3,0-4.9,1.2-6.7,2.7c-2,1.6-4.2,3.4-8.6,3.4c-4.3,0-6.6-1.8-8.6-3.4c-1.8-1.4-3.4-2.7-6.7-2.7
+				c-3.3,0-4.9,1.2-6.7,2.7c-2,1.6-4.2,3.4-8.6,3.4c-4.3,0-6.6-1.8-8.6-3.4c-1.8-1.4-3.4-2.7-6.7-2.7c-3.3,0-4.9,1.2-6.7,2.7
+				c-1.3,1-2.7,2.2-4.8,2.8c8.3,7.3,18.9,12,30.5,13c0.3,0,0.6,0.1,1,0.1c1.1,0.1,2.3,0.1,3.4,0.1c1.2,0,2.3-0.1,3.4-0.1
+				c0.3,0,0.6,0,1-0.1c14.2-1.2,26.8-8,35.6-18.2C118.7,109.4,117.9,109.3,117,109.3z"/>
+			<path class="st2" d="M40.8,69.8c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2s6.1,1.6,8,3.2c1.9,1.5,3.6,2.9,7.2,2.9
+				c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2c4,0,6.1,1.6,8,3.2c1.9,1.5,3.6,2.9,7.2,2.9c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2
+				c4,0,6.1,1.6,8,3.2c1.8,1.5,3.6,2.8,7,2.9c-0.5-4.8-1.6-9.5-3.3-13.8c-1.8-0.6-3.1-1.6-4.4-2.6c-1.9-1.5-3.7-2.9-7.4-2.9
+				c-3.7,0-5.5,1.4-7.4,2.9c-1.9,1.5-3.9,3.1-7.9,3.1c-4,0-5.9-1.6-7.9-3.1c-1.9-1.5-3.7-2.9-7.4-2.9c-3.7,0-5.5,1.4-7.4,2.9
+				c-1.9,1.5-3.9,3.1-7.9,3.1c-4,0-5.9-1.6-7.9-3.1c-1.9-1.5-3.7-2.9-7.4-2.9c-3.7,0-5.5,1.4-7.4,2.9c-1.9,1.5-3.9,3.1-7.9,3.1
+				c-4,0-5.9-1.6-7.9-3.1c-0.4-0.3-0.9-0.7-1.3-1c-1.7,3.6-3.1,7.5-3.9,11.6c2.7,0.5,4.3,1.7,5.9,3C35.5,68.4,37.2,69.8,40.8,69.8z"
+				/>
+			<path class="st2" d="M40.8,55.7c3.7,0,5.5-1.4,7.4-2.9c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.9,1.5,3.7,2.9,7.4,2.9
+				c3.7,0,5.5-1.4,7.4-2.9c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.9,1.5,3.7,2.9,7.4,2.9c3.7,0,5.5-1.4,7.4-2.9
+				c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.1,0.8,2.1,1.7,3.4,2.2c-7.9-19.2-26.9-32.8-48.9-32.8c-20.8,0-38.7,12-47.4,29.5
+				c0.5,0.4,1,0.7,1.4,1.1C35.3,54.2,37.1,55.7,40.8,55.7z"/>
+			<path class="st2" d="M117,94.6c-3.4,0-5,1.3-6.8,2.8c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3c-1.9-1.5-3.5-2.8-6.8-2.8
+				c-3.4,0-5,1.3-6.8,2.8c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3c-1.9-1.5-3.5-2.8-6.8-2.8c-3.4,0-5,1.3-6.8,2.8
+				c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3c-0.5-0.4-1-0.8-1.5-1.1c2.6,6,6.3,11.4,10.9,16c2.6-0.2,4-1.4,5.6-2.6
+				c2-1.6,4.2-3.4,8.6-3.4s6.6,1.8,8.6,3.4c1.8,1.4,3.4,2.7,6.7,2.7c3.3,0,4.9-1.2,6.7-2.7c2-1.6,4.2-3.4,8.6-3.4
+				c4.3,0,6.6,1.8,8.6,3.4c1.8,1.4,3.4,2.7,6.7,2.7c3.3,0,4.9-1.2,6.7-2.7c2-1.6,4.2-3.4,8.6-3.4c1.8,0,3.3,0.3,4.5,0.8
+				c1.9-2.5,3.5-5.1,5-7.9c-1-0.6-1.8-1.2-2.6-1.8C122,95.8,120.3,94.6,117,94.6z"/>
+			<path class="st2" d="M132.1,71.2c-4,0-6-1.6-8-3.2c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2
+				c-4,0-6.1-1.6-8-3.2c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2c-4,0-6.1-1.6-8-3.2
+				c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2c-4,0-6.1-1.6-8-3.2c-1.5-1.2-3-2.3-5.3-2.7
+				c-0.5,2.5-0.8,5.1-0.9,7.7c0,0.7-0.1,1.4-0.1,2c0,0.7,0,1.4,0.1,2c0,0.3,0,0.6,0,0.9c3.5,0.3,5.4,1.8,7.2,3.2
+				c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8c1.9-1.5,4.1-3.2,8.2-3.2s6.3,1.7,8.2,3.2c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8
+				c1.9-1.5,4.1-3.2,8.2-3.2c4.1,0,6.3,1.7,8.2,3.2c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8c1.9-1.5,4.1-3.2,8.2-3.2
+				c4.1,0,6.3,1.7,8.2,3.2c1.7,1.3,3.3,2.6,6.3,2.8c0.3-1.6,0.5-3.2,0.6-4.9c0-0.6,0.1-1.3,0.1-1.9V73
+				C132.2,72.4,132.1,71.2,132.1,71.2z"/>
+			<path class="st2" d="M117,79.9c-3.5,0-5.2,1.4-7,2.8c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.8-1.4-3.5-2.8-7-2.8
+				c-3.5,0-5.2,1.4-7,2.8c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.8-1.4-3.5-2.8-7-2.8c-3.5,0-5.2,1.4-7,2.8
+				c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.6-1.3-3.1-2.5-5.8-2.8c0.4,4.5,1.4,8.7,2.8,12.8c1.9,0.6,3.2,1.7,4.4,2.7
+				c1.9,1.5,3.5,2.8,6.8,2.8c3.4,0,5-1.3,6.8-2.8c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c1.9,1.5,3.5,2.8,6.8,2.8
+				c3.4,0,5-1.3,6.8-2.8c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c1.9,1.5,3.5,2.8,6.8,2.8c3.4,0,5-1.3,6.8-2.8
+				c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c0.7,0.6,1.4,1.1,2.2,1.6c1.6-3.5,2.8-7.2,3.6-11.1c-3.5-0.3-5.4-1.8-7.2-3.2
+				C122.2,81.2,120.4,79.9,117,79.9z"/>
+			<path class="st0" d="M108.4,109.6c-1.8,1.4-3.4,2.7-6.7,2.7c-3.3,0-4.9-1.2-6.7-2.7c-2-1.6-4.2-3.4-8.6-3.4
+				c-4.3,0-6.6,1.8-8.6,3.4c-1.8,1.4-3.4,2.7-6.7,2.7c-3.3,0-4.9-1.2-6.7-2.7c-2-1.6-4.2-3.4-8.6-3.4s-6.6,1.8-8.6,3.4
+				c-1.6,1.3-3,2.4-5.6,2.6c0.9,0.9,1.8,1.7,2.7,2.5c2.1-0.6,3.5-1.8,4.8-2.8c1.8-1.4,3.4-2.7,6.7-2.7c3.3,0,4.9,1.2,6.7,2.7
+				c2,1.6,4.2,3.4,8.6,3.4c4.3,0,6.6-1.8,8.6-3.4c1.8-1.4,3.4-2.7,6.7-2.7c3.3,0,4.9,1.2,6.7,2.7c2,1.6,4.2,3.4,8.6,3.4
+				c4.3,0,6.6-1.8,8.6-3.4c1.8-1.4,3.4-2.7,6.7-2.7c0.9,0,1.7,0.1,2.4,0.3c0.7-0.8,1.4-1.7,2-2.5c-1.2-0.5-2.7-0.8-4.5-0.8
+				C112.6,106.2,110.4,108,108.4,109.6z"/>
+			<path class="st0" d="M117,92.1c-4.2,0-6.4,1.7-8.4,3.3c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.9-1.5-4.1-3.3-8.4-3.3
+				c-4.2,0-6.4,1.7-8.4,3.3c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.9-1.5-4.1-3.3-8.4-3.3c-4.2,0-6.4,1.7-8.4,3.3
+				c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.2-1-2.5-2-4.4-2.7c0.4,1.2,0.9,2.3,1.4,3.5c0.5,0.3,1,0.7,1.5,1.1
+				c1.9,1.5,4.1,3.3,8.4,3.3c4.2,0,6.4-1.7,8.4-3.3c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c1.9,1.5,4.1,3.3,8.4,3.3
+				c4.2,0,6.4-1.7,8.4-3.3c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c1.9,1.5,4.1,3.3,8.4,3.3c4.2,0,6.4-1.7,8.4-3.3
+				c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c0.8,0.6,1.6,1.3,2.6,1.8c0.4-0.7,0.7-1.5,1.1-2.2c-0.8-0.4-1.4-1-2.2-1.6
+				C123.4,93.8,121.2,92.1,117,92.1z"/>
+			<path class="st0" d="M117,77.9c-4.1,0-6.3,1.7-8.2,3.2c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.9-1.5-4.1-3.2-8.2-3.2
+				c-4.1,0-6.3,1.7-8.2,3.2c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.9-1.5-4.1-3.2-8.2-3.2c-4.1,0-6.3,1.7-8.2,3.2
+				c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.7-1.4-3.7-2.9-7.2-3.2c0,0.6,0.1,1.3,0.1,1.9c2.7,0.3,4.2,1.5,5.8,2.8
+				c1.9,1.5,4.1,3.2,8.2,3.2c4.1,0,6.3-1.7,8.2-3.2c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.9,1.5,4.1,3.2,8.2,3.2
+				c4.1,0,6.3-1.7,8.2-3.2c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.9,1.5,4.1,3.2,8.2,3.2c4.1,0,6.3-1.7,8.2-3.2
+				c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.7,1.4,3.7,2.9,7.2,3.2c0.1-0.6,0.2-1.3,0.3-1.9c-3-0.2-4.6-1.4-6.3-2.8
+				C123.3,79.6,121.1,77.9,117,77.9z"/>
+			<path class="st0" d="M40.8,71.2c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9c3.6,0,5.3,1.4,7.2,2.9c2,1.6,4,3.2,8,3.2
+				c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9c3.6,0,5.3,1.4,7.2,2.9c2,1.6,4,3.2,8,3.2c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9
+				c3.6,0,5.3,1.4,7.2,2.9c1.9,1.5,4,3.1,8,3.2l0-0.1c0-0.4-0.1-0.8-0.1-1.3c-3.4-0.1-5.2-1.4-7-2.9c-2-1.6-4-3.2-8-3.2
+				c-4,0-6.1,1.6-8,3.2c-1.9,1.5-3.6,2.9-7.2,2.9c-3.6,0-5.3-1.4-7.2-2.9c-2-1.6-4-3.2-8-3.2c-4,0-6.1,1.6-8,3.2
+				c-1.9,1.5-3.6,2.9-7.2,2.9c-3.6,0-5.3-1.4-7.2-2.9c-2-1.6-4-3.2-8-3.2s-6.1,1.6-8,3.2c-1.9,1.5-3.6,2.9-7.2,2.9
+				c-3.6,0-5.3-1.4-7.2-2.9c-1.6-1.3-3.2-2.5-5.9-3c-0.1,0.4-0.2,0.9-0.3,1.3c2.4,0.4,3.8,1.5,5.3,2.7
+				C34.7,69.6,36.7,71.2,40.8,71.2z"/>
+			<path class="st0" d="M40.8,56.5c4,0,5.9-1.6,7.9-3.1c1.9-1.5,3.7-2.9,7.4-2.9c3.7,0,5.5,1.4,7.4,2.9c1.9,1.5,3.9,3.1,7.9,3.1
+				c4,0,5.9-1.6,7.9-3.1c1.9-1.5,3.7-2.9,7.4-2.9c3.7,0,5.5,1.4,7.4,2.9c1.9,1.5,3.9,3.1,7.9,3.1c4,0,5.9-1.6,7.9-3.1
+				c1.9-1.5,3.7-2.9,7.4-2.9c3.7,0,5.5,1.4,7.4,2.9c1.3,1,2.5,2,4.4,2.6c-0.1-0.3-0.3-0.7-0.4-1c-1.3-0.6-2.4-1.4-3.4-2.2
+				c-1.9-1.5-3.9-3.1-7.9-3.1c-4,0-5.9,1.6-7.9,3.1c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9c-1.9-1.5-3.9-3.1-7.9-3.1
+				c-4,0-5.9,1.6-7.9,3.1c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9C62,51.2,60,49.6,56,49.6c-4,0-5.9,1.6-7.9,3.1
+				c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9C33,52.4,32.5,52,32,51.6c-0.1,0.2-0.3,0.5-0.4,0.8c0.4,0.3,0.9,0.6,1.3,1
+				C34.8,54.9,36.8,56.5,40.8,56.5z"/>
+		</g>
+	</g>
+</g>
+</svg>

assets/stacked/199150-vmw-os-lgo-velero-final_stacked-gry.svg

@@ -0,0 +1,103 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!-- Generator: Adobe Illustrator 24.0.3, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
+<svg version="1.1" id="Layer_5" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
+	 viewBox="0 0 311.5 245.2" style="enable-background:new 0 0 311.5 245.2;" xml:space="preserve">
+<style type="text/css">
+	.st0{fill:#FFFFFF;}
+	.st1{fill:none;}
+	.st2{fill:#009BDB;}
+	.st3{fill:#717074;}
+</style>
+<g>
+	<g>
+		<path class="st1" d="M211.5,83.4c0-0.7,0.1-1.4,0.1-2c0-0.7,0-1.4-0.1-2V83.4z"/>
+		<path class="st2" d="M196.3,115.5c-3.3,0-4.9,1.2-6.7,2.7c-2,1.6-4.2,3.4-8.6,3.4c-4.3,0-6.6-1.8-8.6-3.4
+			c-1.8-1.4-3.4-2.7-6.7-2.7c-3.3,0-4.9,1.2-6.7,2.7c-2,1.6-4.2,3.4-8.6,3.4c-4.3,0-6.6-1.8-8.6-3.4c-1.8-1.4-3.4-2.7-6.7-2.7
+			s-4.9,1.2-6.7,2.7c-1.3,1-2.7,2.2-4.8,2.8c8.3,7.3,18.9,12,30.5,13c0.3,0,0.6,0.1,1,0.1c1.1,0.1,2.3,0.1,3.4,0.1
+			c1.2,0,2.3-0.1,3.4-0.1c0.3,0,0.6,0,1-0.1c14.2-1.2,26.8-8,35.6-18.2C198,115.7,197.2,115.5,196.3,115.5z"/>
+		<path class="st2" d="M120.1,76.1c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2c4,0,6.1,1.6,8,3.2c1.9,1.5,3.6,2.9,7.2,2.9
+			c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2c4,0,6.1,1.6,8,3.2c1.9,1.5,3.6,2.9,7.2,2.9c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2
+			c4,0,6.1,1.6,8,3.2c1.8,1.5,3.6,2.8,7,2.9c-0.5-4.8-1.6-9.5-3.3-13.8c-1.8-0.6-3.1-1.6-4.4-2.6c-1.9-1.5-3.7-2.9-7.4-2.9
+			c-3.7,0-5.5,1.4-7.4,2.9c-1.9,1.5-3.9,3.1-7.9,3.1c-4,0-5.9-1.6-7.9-3.1c-1.9-1.5-3.7-2.9-7.4-2.9c-3.7,0-5.5,1.4-7.4,2.9
+			c-1.9,1.5-3.9,3.1-7.9,3.1c-4,0-5.9-1.6-7.9-3.1c-1.9-1.5-3.7-2.9-7.4-2.9s-5.5,1.4-7.4,2.9c-1.9,1.5-3.9,3.1-7.9,3.1
+			c-4,0-5.9-1.6-7.9-3.1c-0.4-0.3-0.9-0.7-1.3-1c-1.7,3.6-3.1,7.5-3.9,11.6c2.7,0.5,4.3,1.7,5.9,3C114.8,74.7,116.5,76.1,120.1,76.1
+			z"/>
+		<path class="st2" d="M120.1,61.9c3.7,0,5.5-1.4,7.4-2.9c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.9,1.5,3.7,2.9,7.4,2.9
+			c3.7,0,5.5-1.4,7.4-2.9c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.9,1.5,3.7,2.9,7.4,2.9c3.7,0,5.5-1.4,7.4-2.9
+			c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.1,0.8,2.1,1.7,3.4,2.2c-7.9-19.2-26.9-32.8-48.9-32.8c-20.8,0-38.7,12-47.4,29.5
+			c0.5,0.4,1,0.7,1.4,1.1C114.6,60.5,116.4,61.9,120.1,61.9z"/>
+		<path class="st2" d="M196.3,100.8c-3.4,0-5,1.3-6.8,2.8c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3
+			c-1.9-1.5-3.5-2.8-6.8-2.8c-3.4,0-5,1.3-6.8,2.8c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3c-1.9-1.5-3.5-2.8-6.8-2.8
+			c-3.4,0-5,1.3-6.8,2.8c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3c-0.5-0.4-1-0.8-1.5-1.1c2.6,6,6.3,11.4,10.9,16
+			c2.6-0.2,4-1.4,5.6-2.6c2-1.6,4.2-3.4,8.6-3.4c4.3,0,6.6,1.8,8.6,3.4c1.8,1.4,3.4,2.7,6.7,2.7c3.3,0,4.9-1.2,6.7-2.7
+			c2-1.6,4.2-3.4,8.6-3.4c4.3,0,6.6,1.8,8.6,3.4c1.8,1.4,3.4,2.7,6.7,2.7c3.3,0,4.9-1.2,6.7-2.7c2-1.6,4.2-3.4,8.6-3.4
+			c1.8,0,3.3,0.3,4.5,0.8c1.9-2.5,3.5-5.1,5-7.9c-1-0.6-1.8-1.2-2.6-1.8C201.3,102.1,199.6,100.8,196.3,100.8z"/>
+		<path class="st2" d="M211.4,77.5c-4,0-6-1.6-8-3.2c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2
+			c-4,0-6.1-1.6-8-3.2c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2c-4,0-6.1-1.6-8-3.2
+			c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2c-4,0-6.1-1.6-8-3.2c-1.5-1.2-3-2.3-5.3-2.7
+			c-0.5,2.5-0.8,5.1-0.9,7.7c0,0.7-0.1,1.4-0.1,2c0,0.7,0,1.4,0.1,2c0,0.3,0,0.6,0,0.9c3.5,0.3,5.4,1.8,7.2,3.2
+			c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8c1.9-1.5,4.1-3.2,8.2-3.2s6.3,1.7,8.2,3.2c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8
+			c1.9-1.5,4.1-3.2,8.2-3.2c4.1,0,6.3,1.7,8.2,3.2c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8c1.9-1.5,4.1-3.2,8.2-3.2
+			c4.1,0,6.3,1.7,8.2,3.2c1.7,1.3,3.3,2.6,6.3,2.8c0.3-1.6,0.5-3.2,0.6-4.9c0-0.6,0.1-1.3,0.1-1.9v-4.1
+			C211.5,78.6,211.4,77.5,211.4,77.5z"/>
+		<path class="st2" d="M196.3,86.1c-3.5,0-5.2,1.4-7,2.8c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.8-1.4-3.5-2.8-7-2.8
+			c-3.5,0-5.2,1.4-7,2.8c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.8-1.4-3.5-2.8-7-2.8c-3.5,0-5.2,1.4-7,2.8
+			c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.6-1.3-3.1-2.5-5.8-2.8c0.4,4.5,1.4,8.7,2.8,12.8c1.9,0.6,3.2,1.7,4.4,2.7
+			c1.9,1.5,3.5,2.8,6.8,2.8c3.4,0,5-1.3,6.8-2.8c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c1.9,1.5,3.5,2.8,6.8,2.8
+			c3.4,0,5-1.3,6.8-2.8c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c1.9,1.5,3.5,2.8,6.8,2.8c3.4,0,5-1.3,6.8-2.8
+			c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c0.7,0.6,1.4,1.1,2.2,1.6c1.6-3.5,2.8-7.2,3.6-11.1c-3.5-0.3-5.4-1.8-7.2-3.2
+			C201.5,87.5,199.7,86.1,196.3,86.1z"/>
+		<path class="st0" d="M187.7,115.9c-1.8,1.4-3.4,2.7-6.7,2.7c-3.3,0-4.9-1.2-6.7-2.7c-2-1.6-4.2-3.4-8.6-3.4
+			c-4.3,0-6.6,1.8-8.6,3.4c-1.8,1.4-3.4,2.7-6.7,2.7c-3.3,0-4.9-1.2-6.7-2.7c-2-1.6-4.2-3.4-8.6-3.4c-4.3,0-6.6,1.8-8.6,3.4
+			c-1.6,1.3-3,2.4-5.6,2.6c0.9,0.9,1.8,1.7,2.7,2.5c2.1-0.6,3.5-1.8,4.8-2.8c1.8-1.4,3.4-2.7,6.7-2.7c3.3,0,4.9,1.2,6.7,2.7
+			c2,1.6,4.2,3.4,8.6,3.4c4.3,0,6.6-1.8,8.6-3.4c1.8-1.4,3.4-2.7,6.7-2.7c3.3,0,4.9,1.2,6.7,2.7c2,1.6,4.2,3.4,8.6,3.4
+			c4.3,0,6.6-1.8,8.6-3.4c1.8-1.4,3.4-2.7,6.7-2.7c0.9,0,1.7,0.1,2.4,0.3c0.7-0.8,1.4-1.7,2-2.5c-1.2-0.5-2.7-0.8-4.5-0.8
+			C191.9,112.5,189.7,114.3,187.7,115.9z"/>
+		<path class="st0" d="M196.3,98.4c-4.2,0-6.4,1.7-8.4,3.3c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.9-1.5-4.1-3.3-8.4-3.3
+			c-4.2,0-6.4,1.7-8.4,3.3c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.9-1.5-4.1-3.3-8.4-3.3c-4.2,0-6.4,1.7-8.4,3.3
+			c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.2-1-2.5-2-4.4-2.7c0.4,1.2,0.9,2.3,1.4,3.5c0.5,0.3,1,0.7,1.5,1.1
+			c1.9,1.5,4.1,3.3,8.4,3.3c4.2,0,6.4-1.7,8.4-3.3c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c1.9,1.5,4.1,3.3,8.4,3.3
+			c4.2,0,6.4-1.7,8.4-3.3c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c1.9,1.5,4.1,3.3,8.4,3.3c4.2,0,6.4-1.7,8.4-3.3
+			c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c0.8,0.6,1.6,1.3,2.6,1.8c0.4-0.7,0.7-1.5,1.1-2.2c-0.8-0.4-1.4-1-2.2-1.6
+			C202.7,100.1,200.5,98.4,196.3,98.4z"/>
+		<path class="st0" d="M196.3,84.2c-4.1,0-6.3,1.7-8.2,3.2c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.9-1.5-4.1-3.2-8.2-3.2
+			c-4.1,0-6.3,1.7-8.2,3.2c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.9-1.5-4.1-3.2-8.2-3.2c-4.1,0-6.3,1.7-8.2,3.2
+			c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.7-1.4-3.7-2.9-7.2-3.2c0,0.6,0.1,1.3,0.1,1.9c2.7,0.3,4.2,1.5,5.8,2.8
+			c1.9,1.5,4.1,3.2,8.2,3.2c4.1,0,6.3-1.7,8.2-3.2c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.9,1.5,4.1,3.2,8.2,3.2
+			c4.1,0,6.3-1.7,8.2-3.2c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.9,1.5,4.1,3.2,8.2,3.2c4.1,0,6.3-1.7,8.2-3.2
+			c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.7,1.4,3.7,2.9,7.2,3.2c0.1-0.6,0.2-1.3,0.3-1.9c-3-0.2-4.6-1.4-6.3-2.8
+			C202.6,85.9,200.4,84.2,196.3,84.2z"/>
+		<path class="st0" d="M120.1,77.5c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9c3.6,0,5.3,1.4,7.2,2.9c2,1.6,4,3.2,8,3.2
+			c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9c3.6,0,5.3,1.4,7.2,2.9c2,1.6,4,3.2,8,3.2c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9
+			c3.6,0,5.3,1.4,7.2,2.9c1.9,1.5,4,3.1,8,3.2l0-0.1c0-0.4-0.1-0.8-0.1-1.3c-3.4-0.1-5.2-1.4-7-2.9c-2-1.6-4-3.2-8-3.2
+			c-4,0-6.1,1.6-8,3.2c-1.9,1.5-3.6,2.9-7.2,2.9c-3.6,0-5.3-1.4-7.2-2.9c-2-1.6-4-3.2-8-3.2c-4,0-6.1,1.6-8,3.2
+			c-1.9,1.5-3.6,2.9-7.2,2.9c-3.6,0-5.3-1.4-7.2-2.9c-2-1.6-4-3.2-8-3.2c-4,0-6.1,1.6-8,3.2c-1.9,1.5-3.6,2.9-7.2,2.9
+			c-3.6,0-5.3-1.4-7.2-2.9c-1.6-1.3-3.2-2.5-5.9-3c-0.1,0.4-0.2,0.9-0.3,1.3c2.4,0.4,3.8,1.5,5.3,2.7C114,75.9,116,77.5,120.1,77.5z
+			"/>
+		<path class="st0" d="M120.1,62.8c4,0,5.9-1.6,7.9-3.1c1.9-1.5,3.7-2.9,7.4-2.9s5.5,1.4,7.4,2.9c1.9,1.5,3.9,3.1,7.9,3.1
+			c4,0,5.9-1.6,7.9-3.1c1.9-1.5,3.7-2.9,7.4-2.9c3.7,0,5.5,1.4,7.4,2.9c1.9,1.5,3.9,3.1,7.9,3.1c4,0,5.9-1.6,7.9-3.1
+			c1.9-1.5,3.7-2.9,7.4-2.9c3.7,0,5.5,1.4,7.4,2.9c1.3,1,2.5,2,4.4,2.6c-0.1-0.3-0.3-0.7-0.4-1c-1.3-0.6-2.4-1.4-3.4-2.2
+			c-1.9-1.5-3.9-3.1-7.9-3.1c-4,0-5.9,1.6-7.9,3.1c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9c-1.9-1.5-3.9-3.1-7.9-3.1
+			c-4,0-5.9,1.6-7.9,3.1c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9c-1.9-1.5-3.9-3.1-7.9-3.1c-4,0-5.9,1.6-7.9,3.1
+			c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9c-0.5-0.4-0.9-0.7-1.4-1.1c-0.1,0.2-0.3,0.5-0.4,0.8c0.4,0.3,0.9,0.6,1.3,1
+			C114.1,61.2,116.1,62.8,120.1,62.8z"/>
+	</g>
+</g>
+<g>
+	<g>
+		<path class="st3" d="M81.7,161.9l-18.2,41.2h-5.1l-18.2-41.2h5.1L61,197.4l15.6-35.5H81.7z"/>
+		<path class="st3" d="M91.7,166.3v13.6h22.4v4.4H91.7v14.3h24.8v4.4H87v-41.2h29.6v4.4H91.7z"/>
+		<path class="st3" d="M150.9,198.7v4.4h-27.2v-41.2h4.7v36.8H150.9z"/>
+		<path class="st3" d="M160.9,166.3v13.6h22.4v4.4h-22.4v14.3h24.8v4.4h-29.6v-41.2h29.6v4.4H160.9z"/>
+		<path class="st3" d="M223.1,181.3c-1.1,1.9-2.6,3.4-4.6,4.6c-2,1.2-4.2,1.9-6.6,2.2l10.5,14.9h-5.3l-10.4-14.8h-9.1v14.8h-4.7
+			v-41.2h16.7c2.8,0,5.4,0.6,7.7,1.7c2.3,1.1,4.1,2.7,5.5,4.7c1.3,2,2,4.3,2,6.8C224.8,177.4,224.2,179.5,223.1,181.3z M197.5,183.9
+			h11.2c3.4,0,6.1-0.8,8.2-2.3c2.1-1.6,3.1-3.7,3.1-6.4c0-2.7-1-4.9-3.1-6.4c-2.1-1.6-4.8-2.3-8.2-2.3h-11.2V183.9z"/>
+		<path class="st3" d="M239.6,200.9c-3.3-1.9-5.8-4.5-7.8-7.8c-1.9-3.3-2.9-6.8-2.9-10.6c0-3.8,1-7.3,2.9-10.6
+			c1.9-3.3,4.5-5.9,7.8-7.8c3.3-1.9,6.8-2.9,10.5-2.9c3.8,0,7.2,1,10.5,2.9c3.2,1.9,5.8,4.5,7.7,7.8c1.9,3.3,2.9,6.8,2.9,10.6
+			c0,3.8-1,7.3-2.9,10.6c-1.9,3.3-4.5,5.9-7.7,7.8c-3.2,1.9-6.7,2.9-10.5,2.9C246.3,203.8,242.8,202.9,239.6,200.9z M258.2,197.2
+			c2.5-1.6,4.5-3.6,6-6.2c1.5-2.6,2.2-5.4,2.2-8.5c0-3-0.7-5.8-2.2-8.4c-1.5-2.6-3.5-4.7-6-6.2c-2.5-1.5-5.2-2.3-8.1-2.3
+			c-2.9,0-5.6,0.8-8.1,2.3c-2.5,1.5-4.5,3.6-6,6.2c-1.5,2.6-2.2,5.4-2.2,8.4c0,3,0.8,5.9,2.2,8.5c1.5,2.6,3.5,4.7,6,6.2
+			s5.2,2.3,8.1,2.3C253,199.5,255.7,198.7,258.2,197.2z"/>
+	</g>
+</g>
+</svg>

assets/stacked/199150-vmw-os-lgo-velero-final_stacked-wht.svg

@@ -0,0 +1,103 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!-- Generator: Adobe Illustrator 24.0.3, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
+<svg version="1.1" id="Layer_5" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
+	 viewBox="0 0 311.5 245.2" style="enable-background:new 0 0 311.5 245.2;" xml:space="preserve">
+<style type="text/css">
+	.st0{fill:#FFFFFF;}
+	.st1{fill:none;}
+	.st2{fill:#009BDB;}
+	.st3{fill:#717074;}
+</style>
+<g>
+	<g>
+		<path class="st1" d="M211.5,83.4c0-0.7,0.1-1.4,0.1-2c0-0.7,0-1.4-0.1-2V83.4z"/>
+		<path class="st2" d="M196.3,115.5c-3.3,0-4.9,1.2-6.7,2.7c-2,1.6-4.2,3.4-8.6,3.4c-4.3,0-6.6-1.8-8.6-3.4
+			c-1.8-1.4-3.4-2.7-6.7-2.7c-3.3,0-4.9,1.2-6.7,2.7c-2,1.6-4.2,3.4-8.6,3.4c-4.3,0-6.6-1.8-8.6-3.4c-1.8-1.4-3.4-2.7-6.7-2.7
+			s-4.9,1.2-6.7,2.7c-1.3,1-2.7,2.2-4.8,2.8c8.3,7.3,18.9,12,30.5,13c0.3,0,0.6,0.1,1,0.1c1.1,0.1,2.3,0.1,3.4,0.1
+			c1.2,0,2.3-0.1,3.4-0.1c0.3,0,0.6,0,1-0.1c14.2-1.2,26.8-8,35.6-18.2C198,115.7,197.2,115.5,196.3,115.5z"/>
+		<path class="st2" d="M120.1,76.1c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2c4,0,6.1,1.6,8,3.2c1.9,1.5,3.6,2.9,7.2,2.9
+			c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2c4,0,6.1,1.6,8,3.2c1.9,1.5,3.6,2.9,7.2,2.9c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2
+			c4,0,6.1,1.6,8,3.2c1.8,1.5,3.6,2.8,7,2.9c-0.5-4.8-1.6-9.5-3.3-13.8c-1.8-0.6-3.1-1.6-4.4-2.6c-1.9-1.5-3.7-2.9-7.4-2.9
+			c-3.7,0-5.5,1.4-7.4,2.9c-1.9,1.5-3.9,3.1-7.9,3.1c-4,0-5.9-1.6-7.9-3.1c-1.9-1.5-3.7-2.9-7.4-2.9c-3.7,0-5.5,1.4-7.4,2.9
+			c-1.9,1.5-3.9,3.1-7.9,3.1c-4,0-5.9-1.6-7.9-3.1c-1.9-1.5-3.7-2.9-7.4-2.9s-5.5,1.4-7.4,2.9c-1.9,1.5-3.9,3.1-7.9,3.1
+			c-4,0-5.9-1.6-7.9-3.1c-0.4-0.3-0.9-0.7-1.3-1c-1.7,3.6-3.1,7.5-3.9,11.6c2.7,0.5,4.3,1.7,5.9,3C114.8,74.7,116.5,76.1,120.1,76.1
+			z"/>
+		<path class="st2" d="M120.1,61.9c3.7,0,5.5-1.4,7.4-2.9c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.9,1.5,3.7,2.9,7.4,2.9
+			c3.7,0,5.5-1.4,7.4-2.9c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.9,1.5,3.7,2.9,7.4,2.9c3.7,0,5.5-1.4,7.4-2.9
+			c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.1,0.8,2.1,1.7,3.4,2.2c-7.9-19.2-26.9-32.8-48.9-32.8c-20.8,0-38.7,12-47.4,29.5
+			c0.5,0.4,1,0.7,1.4,1.1C114.6,60.5,116.4,61.9,120.1,61.9z"/>
+		<path class="st2" d="M196.3,100.8c-3.4,0-5,1.3-6.8,2.8c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3
+			c-1.9-1.5-3.5-2.8-6.8-2.8c-3.4,0-5,1.3-6.8,2.8c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3c-1.9-1.5-3.5-2.8-6.8-2.8
+			c-3.4,0-5,1.3-6.8,2.8c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3c-0.5-0.4-1-0.8-1.5-1.1c2.6,6,6.3,11.4,10.9,16
+			c2.6-0.2,4-1.4,5.6-2.6c2-1.6,4.2-3.4,8.6-3.4c4.3,0,6.6,1.8,8.6,3.4c1.8,1.4,3.4,2.7,6.7,2.7c3.3,0,4.9-1.2,6.7-2.7
+			c2-1.6,4.2-3.4,8.6-3.4c4.3,0,6.6,1.8,8.6,3.4c1.8,1.4,3.4,2.7,6.7,2.7c3.3,0,4.9-1.2,6.7-2.7c2-1.6,4.2-3.4,8.6-3.4
+			c1.8,0,3.3,0.3,4.5,0.8c1.9-2.5,3.5-5.1,5-7.9c-1-0.6-1.8-1.2-2.6-1.8C201.3,102.1,199.6,100.8,196.3,100.8z"/>
+		<path class="st2" d="M211.4,77.5c-4,0-6-1.6-8-3.2c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2
+			c-4,0-6.1-1.6-8-3.2c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2c-4,0-6.1-1.6-8-3.2
+			c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2c-4,0-6.1-1.6-8-3.2c-1.5-1.2-3-2.3-5.3-2.7
+			c-0.5,2.5-0.8,5.1-0.9,7.7c0,0.7-0.1,1.4-0.1,2c0,0.7,0,1.4,0.1,2c0,0.3,0,0.6,0,0.9c3.5,0.3,5.4,1.8,7.2,3.2
+			c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8c1.9-1.5,4.1-3.2,8.2-3.2s6.3,1.7,8.2,3.2c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8
+			c1.9-1.5,4.1-3.2,8.2-3.2c4.1,0,6.3,1.7,8.2,3.2c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8c1.9-1.5,4.1-3.2,8.2-3.2
+			c4.1,0,6.3,1.7,8.2,3.2c1.7,1.3,3.3,2.6,6.3,2.8c0.3-1.6,0.5-3.2,0.6-4.9c0-0.6,0.1-1.3,0.1-1.9v-4.1
+			C211.5,78.6,211.4,77.5,211.4,77.5z"/>
+		<path class="st2" d="M196.3,86.1c-3.5,0-5.2,1.4-7,2.8c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.8-1.4-3.5-2.8-7-2.8
+			c-3.5,0-5.2,1.4-7,2.8c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.8-1.4-3.5-2.8-7-2.8c-3.5,0-5.2,1.4-7,2.8
+			c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.6-1.3-3.1-2.5-5.8-2.8c0.4,4.5,1.4,8.7,2.8,12.8c1.9,0.6,3.2,1.7,4.4,2.7
+			c1.9,1.5,3.5,2.8,6.8,2.8c3.4,0,5-1.3,6.8-2.8c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c1.9,1.5,3.5,2.8,6.8,2.8
+			c3.4,0,5-1.3,6.8-2.8c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c1.9,1.5,3.5,2.8,6.8,2.8c3.4,0,5-1.3,6.8-2.8
+			c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c0.7,0.6,1.4,1.1,2.2,1.6c1.6-3.5,2.8-7.2,3.6-11.1c-3.5-0.3-5.4-1.8-7.2-3.2
+			C201.5,87.5,199.7,86.1,196.3,86.1z"/>
+		<path class="st0" d="M187.7,115.9c-1.8,1.4-3.4,2.7-6.7,2.7c-3.3,0-4.9-1.2-6.7-2.7c-2-1.6-4.2-3.4-8.6-3.4
+			c-4.3,0-6.6,1.8-8.6,3.4c-1.8,1.4-3.4,2.7-6.7,2.7c-3.3,0-4.9-1.2-6.7-2.7c-2-1.6-4.2-3.4-8.6-3.4c-4.3,0-6.6,1.8-8.6,3.4
+			c-1.6,1.3-3,2.4-5.6,2.6c0.9,0.9,1.8,1.7,2.7,2.5c2.1-0.6,3.5-1.8,4.8-2.8c1.8-1.4,3.4-2.7,6.7-2.7c3.3,0,4.9,1.2,6.7,2.7
+			c2,1.6,4.2,3.4,8.6,3.4c4.3,0,6.6-1.8,8.6-3.4c1.8-1.4,3.4-2.7,6.7-2.7c3.3,0,4.9,1.2,6.7,2.7c2,1.6,4.2,3.4,8.6,3.4
+			c4.3,0,6.6-1.8,8.6-3.4c1.8-1.4,3.4-2.7,6.7-2.7c0.9,0,1.7,0.1,2.4,0.3c0.7-0.8,1.4-1.7,2-2.5c-1.2-0.5-2.7-0.8-4.5-0.8
+			C191.9,112.5,189.7,114.3,187.7,115.9z"/>
+		<path class="st0" d="M196.3,98.4c-4.2,0-6.4,1.7-8.4,3.3c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.9-1.5-4.1-3.3-8.4-3.3
+			c-4.2,0-6.4,1.7-8.4,3.3c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.9-1.5-4.1-3.3-8.4-3.3c-4.2,0-6.4,1.7-8.4,3.3
+			c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.2-1-2.5-2-4.4-2.7c0.4,1.2,0.9,2.3,1.4,3.5c0.5,0.3,1,0.7,1.5,1.1
+			c1.9,1.5,4.1,3.3,8.4,3.3c4.2,0,6.4-1.7,8.4-3.3c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c1.9,1.5,4.1,3.3,8.4,3.3
+			c4.2,0,6.4-1.7,8.4-3.3c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c1.9,1.5,4.1,3.3,8.4,3.3c4.2,0,6.4-1.7,8.4-3.3
+			c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c0.8,0.6,1.6,1.3,2.6,1.8c0.4-0.7,0.7-1.5,1.1-2.2c-0.8-0.4-1.4-1-2.2-1.6
+			C202.7,100.1,200.5,98.4,196.3,98.4z"/>
+		<path class="st0" d="M196.3,84.2c-4.1,0-6.3,1.7-8.2,3.2c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.9-1.5-4.1-3.2-8.2-3.2
+			c-4.1,0-6.3,1.7-8.2,3.2c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.9-1.5-4.1-3.2-8.2-3.2c-4.1,0-6.3,1.7-8.2,3.2
+			c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.7-1.4-3.7-2.9-7.2-3.2c0,0.6,0.1,1.3,0.1,1.9c2.7,0.3,4.2,1.5,5.8,2.8
+			c1.9,1.5,4.1,3.2,8.2,3.2c4.1,0,6.3-1.7,8.2-3.2c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.9,1.5,4.1,3.2,8.2,3.2
+			c4.1,0,6.3-1.7,8.2-3.2c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.9,1.5,4.1,3.2,8.2,3.2c4.1,0,6.3-1.7,8.2-3.2
+			c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.7,1.4,3.7,2.9,7.2,3.2c0.1-0.6,0.2-1.3,0.3-1.9c-3-0.2-4.6-1.4-6.3-2.8
+			C202.6,85.9,200.4,84.2,196.3,84.2z"/>
+		<path class="st0" d="M120.1,77.5c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9c3.6,0,5.3,1.4,7.2,2.9c2,1.6,4,3.2,8,3.2
+			c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9c3.6,0,5.3,1.4,7.2,2.9c2,1.6,4,3.2,8,3.2c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9
+			c3.6,0,5.3,1.4,7.2,2.9c1.9,1.5,4,3.1,8,3.2l0-0.1c0-0.4-0.1-0.8-0.1-1.3c-3.4-0.1-5.2-1.4-7-2.9c-2-1.6-4-3.2-8-3.2
+			c-4,0-6.1,1.6-8,3.2c-1.9,1.5-3.6,2.9-7.2,2.9c-3.6,0-5.3-1.4-7.2-2.9c-2-1.6-4-3.2-8-3.2c-4,0-6.1,1.6-8,3.2
+			c-1.9,1.5-3.6,2.9-7.2,2.9c-3.6,0-5.3-1.4-7.2-2.9c-2-1.6-4-3.2-8-3.2c-4,0-6.1,1.6-8,3.2c-1.9,1.5-3.6,2.9-7.2,2.9
+			c-3.6,0-5.3-1.4-7.2-2.9c-1.6-1.3-3.2-2.5-5.9-3c-0.1,0.4-0.2,0.9-0.3,1.3c2.4,0.4,3.8,1.5,5.3,2.7C114,75.9,116,77.5,120.1,77.5z
+			"/>
+		<path class="st0" d="M120.1,62.8c4,0,5.9-1.6,7.9-3.1c1.9-1.5,3.7-2.9,7.4-2.9s5.5,1.4,7.4,2.9c1.9,1.5,3.9,3.1,7.9,3.1
+			c4,0,5.9-1.6,7.9-3.1c1.9-1.5,3.7-2.9,7.4-2.9c3.7,0,5.5,1.4,7.4,2.9c1.9,1.5,3.9,3.1,7.9,3.1c4,0,5.9-1.6,7.9-3.1
+			c1.9-1.5,3.7-2.9,7.4-2.9c3.7,0,5.5,1.4,7.4,2.9c1.3,1,2.5,2,4.4,2.6c-0.1-0.3-0.3-0.7-0.4-1c-1.3-0.6-2.4-1.4-3.4-2.2
+			c-1.9-1.5-3.9-3.1-7.9-3.1c-4,0-5.9,1.6-7.9,3.1c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9c-1.9-1.5-3.9-3.1-7.9-3.1
+			c-4,0-5.9,1.6-7.9,3.1c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9c-1.9-1.5-3.9-3.1-7.9-3.1c-4,0-5.9,1.6-7.9,3.1
+			c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9c-0.5-0.4-0.9-0.7-1.4-1.1c-0.1,0.2-0.3,0.5-0.4,0.8c0.4,0.3,0.9,0.6,1.3,1
+			C114.1,61.2,116.1,62.8,120.1,62.8z"/>
+	</g>
+</g>
+<g>
+	<g>
+		<path class="st0" d="M81.7,161.9l-18.2,41.2h-5.1l-18.2-41.2h5.1L61,197.4l15.6-35.5H81.7z"/>
+		<path class="st0" d="M91.7,166.3v13.6h22.4v4.4H91.7v14.3h24.8v4.4H87v-41.2h29.6v4.4H91.7z"/>
+		<path class="st0" d="M150.9,198.7v4.4h-27.2v-41.2h4.7v36.8H150.9z"/>
+		<path class="st0" d="M160.9,166.3v13.6h22.4v4.4h-22.4v14.3h24.8v4.4h-29.6v-41.2h29.6v4.4H160.9z"/>
+		<path class="st0" d="M223.1,181.3c-1.1,1.9-2.6,3.4-4.6,4.6c-2,1.2-4.2,1.9-6.6,2.2l10.5,14.9h-5.3l-10.4-14.8h-9.1v14.8h-4.7
+			v-41.2h16.7c2.8,0,5.4,0.6,7.7,1.7c2.3,1.1,4.1,2.7,5.5,4.7c1.3,2,2,4.3,2,6.8C224.8,177.4,224.2,179.5,223.1,181.3z M197.5,183.9
+			h11.2c3.4,0,6.1-0.8,8.2-2.3c2.1-1.6,3.1-3.7,3.1-6.4c0-2.7-1-4.9-3.1-6.4c-2.1-1.6-4.8-2.3-8.2-2.3h-11.2V183.9z"/>
+		<path class="st0" d="M239.6,200.9c-3.3-1.9-5.8-4.5-7.8-7.8c-1.9-3.3-2.9-6.8-2.9-10.6c0-3.8,1-7.3,2.9-10.6
+			c1.9-3.3,4.5-5.9,7.8-7.8c3.3-1.9,6.8-2.9,10.5-2.9c3.8,0,7.2,1,10.5,2.9c3.2,1.9,5.8,4.5,7.7,7.8c1.9,3.3,2.9,6.8,2.9,10.6
+			c0,3.8-1,7.3-2.9,10.6c-1.9,3.3-4.5,5.9-7.7,7.8c-3.2,1.9-6.7,2.9-10.5,2.9C246.3,203.8,242.8,202.9,239.6,200.9z M258.2,197.2
+			c2.5-1.6,4.5-3.6,6-6.2c1.5-2.6,2.2-5.4,2.2-8.5c0-3-0.7-5.8-2.2-8.4c-1.5-2.6-3.5-4.7-6-6.2c-2.5-1.5-5.2-2.3-8.1-2.3
+			c-2.9,0-5.6,0.8-8.1,2.3s-4.5,3.6-6,6.2s-2.2,5.4-2.2,8.4c0,3,0.8,5.9,2.2,8.5c1.5,2.6,3.5,4.7,6,6.2c2.5,1.6,5.2,2.3,8.1,2.3
+			C253,199.5,255.7,198.7,258.2,197.2z"/>
+	</g>
+</g>
+</svg>

changelogs/CHANGELOG-1.10.md

@@ -0,0 +1,190 @@
+  ## v1.10.0
+### 2022-11-23
+
+### Download
+https://github.com/vmware-tanzu/velero/releases/tag/v1.10.0
+
+### Container Image
+`velero/velero:v1.10.0`
+
+### Documentation
+https://velero.io/docs/v1.10/
+
+### Upgrading
+https://velero.io/docs/v1.10/upgrade-to-1.10/
+
+### Highlights
+
+#### Unified Repository and Kopia integration
+In this release, we introduced the Unified Repository architecture to build a data path where data movers and the backup repository are decoupled and a unified backup repository could serve various data movement activities.
+
+In this release, we also deeply integrate Velero with Kopia, specifically, Kopia's uploader modules are isolated as a generic file system uploader; Kopia's repository modules are encapsulated as the unified backup repository.
+
+For more information, refer to the [design document](https://github.com/vmware-tanzu/velero/blob/v1.10.0/design/unified-repo-and-kopia-integration/unified-repo-and-kopia-integration.md).
+
+#### File system backup refactor
+Velero's file system backup (a.k.s. pod volume backup or formerly restic backup) is refactored as the first user of the Unified Repository architecture. Specifically, we added a new path, the Kopia path, besides the existing Restic path. While Restic path is still available and set as default, you can opt in Kopia path by specifying the `uploader-type` parameter at installation time. Meanwhile, you are free to restore from existing backups under either path, Velero dynamically switches to the correct path to process the restore.
+
+Because of the new path, we renamed some modules and parameters, refer to the Break Changes section for more details.
+
+For more information, visit the [file system backup document](https://velero.io/docs/v1.10/file-system-backup/) and [v1.10 upgrade guide document](https://velero.io/docs/v1.10/upgrade-to-1.10/).
+
+Meanwhile, we've created a performance guide for both Restic path and Kopia path, which helps you to choose between the two paths and provides you the best practice to configure them under different scenarios. Please note that the results in the guide are based on our testing environments, you may get different results when testing in your own ones. For more information, visit the [performance guide document](https://velero.io/docs/v1.10/performance-guidance/).
+
+#### Plugin versioning V1 refactor
+In this release, Velero moves plugins BackupItemAction, RestoreItemAction and VolumeSnapshotterAction to version v1, this allows future plugin changes that do not support backward compatibility, so is a preparation for various complex tasks, for example, data movement tasks.
+For more information, refer to the [plugin versioning design document](https://github.com/vmware-tanzu/velero/blob/v1.10.0/design/plugin-versioning.md).
+
+#### Refactor the controllers using Kubebuilder v3
+In this release we continued our code modernization work, rewriting some controllers using Kubebuilder v3. This work is ongoing and we will continue to make progress in future releases.
+
+#### Add credentials to volume snapshot locations
+In this release, we enabled dedicate credentials options to volume snapshot locations so that you can specify credentials per volume snapshot location as same as backup storage location.
+
+For more information, please visit the [locations document](https://velero.io/docs/v1.10/locations/).
+
+#### CSI snapshot enhancements
+In this release we added several changes to enhance the robustness of CSI snapshot procedures, for example, some protection code for error handling, and a mechanism to skip exclusion checks so that CSI snapshot works with various backup resource filters.
+
+#### Backup schedule pause/unpause
+In this release, Velero supports to pause/unpause a backup schedule during or after its creation. Specifically:
+
+At creation time, you can specify `–paused` flag to `velero schedule create` command, if so, you will create a paused schedule that will not run until it is unpaused
+After creation, you can run `velero schedule pause` or `velero schedule unpause` command to pause/unpause a schedule
+
+#### Runtime and dependencies
+In order to fix CVEs, we changed Velero's runtime and dependencies as follows:
+
+Bump go runtime to v1.18.8
+Bump some core dependent libraries to newer versions
+Compile Restic (v0.13.1) with go 1.18.8 instead of packaging the official binary
+
+
+#### Breaking changes
+Due to file system backup refactor, below modules and parameters name have been changed in this release:
+
+`restic` daemonset is renamed to `node-agent`
+`resticRepository` CR is renamed to `backupRepository`
+`velero restic repo` command is renamed to `velero repo`
+`velero-restic-credentials` secret is renamed to `velero-repo-credentials`
+`default-volumes-to-restic` parameter is renamed to `default-volumes-to-fs-backup`
+`restic-timeout` parameter is renamed to `fs-backup-timeout`
+`default-restic-prune-frequency` parameter is renamed to `default-repo-maintain-frequency`
+
+#### Upgrade
+Due to the major changes of file system backup, the old upgrade steps are not suitable any more. For the new upgrade steps, visit [v1.10 upgrade guide document](https://velero.io/docs/v1.10/upgrade-to-1.10/).
+
+#### Limitations/Known issues
+In this release, Kopia backup repository (so the Kopia path of file system backup) doesn't support self signed certificate for S3 compatible storage. To track this problem, refer to this [Velero issue](https://github.com/vmware-tanzu/velero/issues/5123) or [Kopia issue](https://github.com/kopia/kopia/issues/1443). 
+
+Due to the code change in Velero, there will be some code change required in vSphere plugin, without which the functionality may be impacted.  Therefore, if you are using vSphere plugin in your workflow, please hold the upgrade until the issue [#485](https://github.com/vmware-tanzu/velero-plugin-for-vsphere/issues/485) is fixed in vSphere plugin.
+
+### All changes
+  
+  * Restore ClusterBootstrap before Cluster otherwise a new default ClusterBootstrap object is create for the cluster  (#5616, @ywk253100)
+  * Add compile restic binary for CVE fix  (#5574, @qiuming-best)
+  * Fix controller problematic log output  (#5572, @qiuming-best)
+  * Enhance the restore priorities list to support specifying the low prioritized resources that need to be restored in the last (#5535, @ywk253100)
+  * fix restic backup progress error (#5534, @qiuming-best)
+  * fix restic backup failure with self-signed certification backend storage (#5526, @qiuming-best)
+  * Add credential store in backup deletion controller to support VSL credential. (#5521, @blackpiglet)
+  * Fix issue 5505: the pod volume backups/restores except the first one fail under the kopia path if "AZURE_CLOUD_NAME" is specified (#5512, @Lyndon-Li)
+  * After Pod Volume Backup/Restore refactor, remove all the unreasonable appearance of "restic" word from documents (#5499, @Lyndon-Li)
+  * Refactor Pod Volume Backup/Restore doc to match the new behavior (#5484, @Lyndon-Li)
+  * Remove redundancy code block left by #5388. (#5483, @blackpiglet)
+  * Issue fix 5477: create the common way to support S3 compatible object storages that work for both Restic and Kopia; Keep the resticRepoPrefix parameter for compatibility (#5478, @Lyndon-Li)
+  * Update the k8s.io dependencies to 0.24.0.
+This also required an update to github.com/bombsimon/logrusr/v3.
+Removed the `WithClusterName` method
+as it is a "legacy field that was
+always cleared by the system and never used" as per upstream k8s
+https://github.com/kubernetes/apimachinery/blob/release-1.24/pkg/apis/meta/v1/types.go#L257-L259 (#5471, @kcboyle)
+  * Add v1.10 velero upgrade doc (#5468, @qiuming-best)
+  * Upgrade velero docker image to use go 1.18 and upgrade golangci-lint to 1.45.0 (#5459, @Lyndon-Li)
+  * Add VolumeSnapshot client back. (#5449, @blackpiglet)
+  * Change subcommand `velero restic repo` to `velero repo` (#5446, @allenxu404)
+  * Remove irrational "Restic" names in Velero code after the PVBR refactor (#5444, @Lyndon-Li)
+  * moved RIA execute input/output structs back to velero package (#5441, @sseago)
+  * Rename Velero pod volume restore init helper from "velero-restic-restore-helper" to "velero-restore-helper" (#5432, @Lyndon-Li)
+  * Skip the exclusion check for additional resources returned by BIA (#5429, @reasonerjt)
+  * Change B/R describe CLI to support Kopia (#5412, @allenxu404)
+  * Add nil check before execution of csi snapshot delete (#5401, @shubham-pampattiwar)
+  * update velero using klog to version v2.9.0 (#5396, @blackpiglet)
+  * Fix Test_prepareBackupRequest_BackupStorageLocation UT failure. (#5394, @blackpiglet)
+  * Rename Velero daemonset from "restic" to "node-agent" (#5390, @Lyndon-Li)
+  * Add some corner cases checking for CSI snapshot in backup controller. (#5388, @blackpiglet)
+  * Fix issue 5386: Velero providers a full URL as the S3Url while the underlying minio client only accept the host part of the URL as the endpoint and the schema should be specified separately. (#5387, @Lyndon-Li)
+  * Fix restore error with flag namespace-mappings (#5377, @qiuming-best)
+  * Pod Volume Backup/Restore Refactor: Rename parameters in CRDs and commands to remove "Restic" word (#5370, @Lyndon-Li)
+  * Added backupController's UT to test the prepareBackupRequest() method BackupStorageLocation processing logic (#5362, @niulechuan)
+  * Fix a repoEnsurer problem introduced by the refactor - The repoEnsurer didn't check "" state of BackupRepository, as a result, the function GetBackupRepository always returns without an error even though the ensreReady is specified. (#5359, @Lyndon-Li)
+  * Add E2E test for schedule backup (#5355, @danfengliu)
+  * Add useOwnerReferencesInBackup field doc for schedule. (#5353, @cleverhu)
+  * Clarify the help message for the default value of parameter --snapshot-volumes, when it's not set. (#5350, @blackpiglet)
+  * Fix restore cmd extraflag overwrite bug (#5347, @qiuming-best)
+  * Resolve gopkg.in/yaml.v3 vulnerabilities by upgrading gopkg.in/yaml.v3 to v3.0.1 (#5344, @kaovilai)
+  * Increase ensure restic repository timeout to 5m (#5335, @shubham-pampattiwar)
+  * Add opt-in and opt-out PersistentVolume backup to E2E tests (#5331, @danfengliu)
+  * Cancel downloadRequest when timeout without downloadURL (#5329, @kaovilai)
+  * Fix PVB finds wrong parent snapshot (#5322, @qiuming-best)
+  * Fix issue 4874 and 4752: check the daemonset pod is running in the node where the workload pod resides before running the PVB for the pod (#5319, @Lyndon-Li)
+  * plugin versioning v1 refactor for VolumeSnapshotter (#5318, @sseago)
+  * Change the status of restore to completed from partially failed when restore empty backup (#5314, @allenxu404)
+  * RestoreItemAction v1 refactoring for plugin api versioning (#5312, @sseago)
+  * Refactor the repoEnsurer code to use controller runtime client and wrap some common BackupRepository operations to share with other modules (#5308, @Lyndon-Li)
+  * Remove snapshot related lister, informer and client from backup controller. (#5299, @jxun)
+  * Remove github.com/apex/log logger. (#5297, @blackpiglet)
+  * change CSISnapshotTimeout from pointer to normal variables. (#5294, @cleverhu)
+  * Optimize code for restore exists resources. (#5293, @cleverhu)
+  * Add more detailed comments for labels columns. (#5291, @cleverhu)
+  * Add backup status checking in schedule controller. (#5283, @blackpiglet)
+  * Add changes for problems/enhancements found during smoking test for Kopia pod volume backup/restore (#5282, @Lyndon-Li)
+  * Support pause/unpause schedules (#5279, @ywk253100)
+  *  plugin/clientmgmt refactoring for BackupItemAction v1 (#5271, @sseago)
+  * Don't move velero v1 plugins to new proto dir (#5263, @sseago)
+  * Fill gaps for Kopia path of PVBR: integrate Repo Manager with Unified Repo; pass UploaderType to PVBR backupper and restorer; pass RepositoryType to BackupRepository controller and Repo Ensurer (#5259, @Lyndon-Li)
+  * Add csiSnapshotTimeout for describe backup (#5252, @cleverhu)
+  * equip gc controller with configurable frequency (#5248, @allenxu404)
+  * Fix nil pointer panic when restoring StatefulSets (#5247, @divolgin)
+  * Controller refactor code modifications. (#5241, @jxun)
+  * Fix edge cases for already exists resources (#5239, @shubham-pampattiwar)
+  * Check for empty ns list before checking nslist[0] (#5236, @sseago)
+  * Remove reference to non-existent doc (#5234, @reasonerjt)
+  * Add changes for Kopia Integration: Kopia Lib - method implementation. Add changes to write Kopia Repository logs to Velero log (#5233, @Lyndon-Li)
+  * Add changes for Kopia Integration: Kopia Lib - initialize Kopia repo (#5231, @Lyndon-Li)
+  * Uploader Implementation: Kopia backup and restore (#5221, @qiuming-best)
+  * Migrate backup sync controller from code-generator to kubebuilder. (#5218, @jxun)
+  * check vsc null pointer (#5217, @lilongfeng0902)
+  * Refactor GCController with kubebuilder (#5215, @allenxu404)
+  * Uploader Implementation: Restic backup and restore (#5214, @qiuming-best)
+  * Add parameter "uploader-type" to velero server (#5212, @reasonerjt)
+  * Add annotation "pv.kubernetes.io/migrated-to" for CSI checking. (#5181, @jxun)
+  * Add changes for Kopia Integration: Unified Repository Provider - method implementation (#5179, @Lyndon-Li)
+  * Treat namespaces with exclude label as excludedNamespaces
+Related issue: #2413 (#5178, @allenxu404)
+  * Reduce CRD size. (#5174, @jxun)
+  * Fix restic backups to multiple backup storage locations bug (#5172, @qiuming-best)
+  * Add changes for Kopia Integration: Unified Repository Provider - Repo Password (#5167, @Lyndon-Li)
+  * Skip registering "crd-remap-version" plugin when feature flag "EnableAPIGroupVersions" is set (#5165, @reasonerjt)
+  * Kopia uploader integration on shim progress uploader module (#5163, @qiuming-best)
+  * Add labeled and unlabeled events for PR changelog check action. (#5157, @jxun)
+  * VolumeSnapshotLocation refactor with kubebuilder. (#5148, @jxun)
+  * Delay CA file deletion in PVB controller. (#5145, @jxun)
+  * This commit splits the pkg/restic package into several packages to support Kopia integration works (#5143, @ywk253100)
+  * Kopia Integration: Add the Unified Repository Interface definition. Kopia Integration: Add the changes for Unified Repository storage config. Related Issues; #5076, #5080 (#5142, @Lyndon-Li)
+  * Update the CRD for kopia integration (#5135, @reasonerjt)
+  * Let "make shell xxx" respect GOPROXY (#5128, @reasonerjt)
+  * Modify BackupStoreGetter to avoid BSL spec changes (#5122, @sseago)
+  * Dump stack trace when the plugin server handles panic (#5110, @reasonerjt)
+  * Make CSI snapshot creation timeout configurable. (#5104, @jxun)
+  *  Fix bsl validation bug: the BSL is validated continually and doesn't respect the validation period configured (#5101, @ywk253100)
+  * Exclude "csinodes.storage.k8s.io" and "volumeattachments.storage.k8s.io" from restore by default. (#5064, @jxun)
+  * Move 'velero.io/exclude-from-backup' label string to const (#5053, @niulechuan)
+  * Modify Github actions. (#5052, @jxun)
+  * Fix typo in doc, in https://velero.io/docs/main/restore-reference/ "Restore order" section, "Mamespace" should be "Namespace". (#5051, @niulechuan)
+  * Delete opened issues triage action. (#5041, @jxun)
+  * When spec.RestoreStatus is empty, don't restore status (#5008, @sseago)
+  * Added DownloadTargetKindCSIBackupVolumeSnapshots for retrieving the signed URL to download only the `<backup name>`-csi-volumesnapshots.json.gz  and DownloadTargetKindCSIBackupVolumeSnapshotContents to download only `<backup name>`-csi-volumesnapshotcontents.json.gz in the DownloadRequest CR structure. These files are already present in the backup layout.  (#4980, @anshulahuja98)
+  * Refactor BackupItemAction proto and related code to backupitemaction/v1 package.  This is part of implementation of the plugin version design https://github.com/vmware-tanzu/velero/blob/main/design/plugin-versioning.md (#4943, @phuongatemc)
+  * Unified Repository Design (#4926, @Lyndon-Li)
+  * Add credentials to volume snapshot locations (#4864, @sseago)

changelogs/CHANGELOG-1.6.md

@@ -1,80 +1,3 @@
-## v1.6.3
-### 2021-07-30
-
-### Download
-https://github.com/vmware-tanzu/velero/releases/tag/v1.6.3
-
-### Container Image
-`velero/velero:v1.6.3`
-
-### Documentation
-https://velero.io/docs/v1.6/
-
-### Upgrading
-https://velero.io/docs/v1.6/upgrade-to-1.6/
-
-### Highlights
-
-This release introduces changes to provide compatibility with Kubernetes v1.22.
-
-The `apiextensions.k8s.io/v1beta1` API version of `CustomResourceDefinition` will no longer be served in Kubernetes v1.22.
-Velero will now use the cluster preferred API version for the `CustomResourceDefinition`s that it creates.
-
-If you are using Kubernetes v1.15 or earlier, the `apiextensions.k8s.io/v1beta1` API version will be used.
-If you are using Kubernetes v1.22 or later, the `apiextensions.k8s.io/v1` API version will be used.
-For clusters between these versions, the cluster preferred API version will be used.
-
-The `rbac.authorization.k8s.io/v1beta1` API version of `ClusterRoleBinding` will no longer be served in Kubernetes v1.22.
-Velero will now use the `rbac.authorization.k8s.io/v1` API version for the `ClusterRoleBinding`s that it creates.
-This API version was introduced in Kubernetes v1.8.
-
-### All Changes
-
-  * enable e2e tests to choose crd apiVersion (#3941, @sseago)
-  * Upgrade Velero ClusterRoleBinding to use v1 API (#3995, @jenting)
-  * Install Kubernetes preferred CRDs API version (v1beta1/v1). (#3999, @jenting)
-  * Use the cluster preferred CRD API version when polling for Velero CRD readiness. (#4015, @zubron)
-  * Add a RestoreItemAction plugin (`velero.io/apiservice`) which skips the restore of any `APIService` which is managed by Kubernetes. These are identified using the `kube-aggregator.kubernetes.io/automanaged` label. (#4028, @zubron)
-
-## v1.6.2
-### 2021-07-16
-
-### Download
-https://github.com/vmware-tanzu/velero/releases/tag/v1.6.2
-
-### Container Image
-`velero/velero:v1.6.2`
-
-### Documentation
-https://velero.io/docs/v1.6/
-
-### Upgrading
-https://velero.io/docs/v1.6/upgrade-to-1.6/
-
-This release contains no user facing changes but includes fixes for CVE-2021-3121 and CVE-2021-3580.
-
-## v1.6.1
-### 2021-06-21
-
-### Download
-https://github.com/vmware-tanzu/velero/releases/tag/v1.6.1
-
-### Container Image
-`velero/velero:v1.6.1`
-
-### Documentation
-https://velero.io/docs/v1.6/
-
-### Upgrading
-https://velero.io/docs/v1.6/upgrade-to-1.6/
-
-### All Changes
-
-  * Fix CR restore regression introduced in 1.6 restore progress. (#3845, @sseago)
-  * Skip the restore of volumes that originally came from a projected volume when using restic. (#3877, @zubron)
-  * skip backuping projected volume when using restic (#3866, @alaypatel07)
-  * 🐛 Fix plugin name derivation from image name (#3711, @ashish-amarnath)
-
 ## v1.6.0
 ### 2021-04-12
 

changelogs/CHANGELOG-1.7.md

@@ -0,0 +1,80 @@
+## v1.7.0
+### 2021-09-07
+
+### Download
+https://github.com/vmware-tanzu/velero/releases/tag/v1.7.0
+
+### Container Image
+`velero/velero:v1.7.0`
+
+### Documentation
+https://velero.io/docs/v1.7/
+
+### Upgrading
+https://velero.io/docs/v1.7/upgrade-to-1.7/
+
+### Highlights
+
+#### Distroless images
+
+The Velero container images now use [distroless base images](https://github.com/GoogleContainerTools/distroless).
+Using distroless images as the base ensures that only the packages and programs necessary for running Velero are included.
+Unrelated libraries and OS packages, that often contain security vulnerabilities, are now excluded.
+This change reduces the size of both the server and restic restore helper image by approximately 62MB.
+
+As the [distroless](https://github.com/GoogleContainerTools/distroless) images do not contain a shell, it will no longer be possible to exec into Velero containers using these images.
+
+#### New "debug" command
+
+This release introduces the new `velero debug` command.
+This command collects information about a Velero installation, such as pod logs and resources managed by Velero, in a tarball which can be provided to the Velero maintainer team to help diagnose issues.
+
+### All changes
+
+  * Distinguish between different unnamed node ports when preserving (#4026, @sseago)
+  * Validate namespace in Velero backup create command (#4057, @codegold79)
+  * Empty the "ClusterIPs" along with "ClusterIP" when "ClusterIP" isn't "None" (#4101, @ywk253100)
+  * Add a RestoreItemAction plugin (`velero.io/apiservice`) which skips the restore of any `APIService` which is managed by Kubernetes. These are identified using the `kube-aggregator.kubernetes.io/automanaged` label. (#4028, @zubron)
+  * Change the base image to distroless (#4055, @ywk253100)
+  * Updated the version of velero/velero-plugin-for-aws version from v1.2.0 to v1.2.1 (#4064, @kahirokunn)
+  * Skip the backup and restore of DownwardAPI volumes when using restic. (#4076, @zubron)
+  * Bump up Go to 1.16 (#3990, @reasonerjt)
+  * Fix restic error when volume is emptyDir and Pod not running (#3993, @mahaupt)
+  * Select the velero deployment with both label and container name (#3996, @ywk253100)
+  * Wait for the namespace to be deleted before removing the CRDs during uninstall. This deprecates the `--wait` flag of the `uninstall` command (#4007, @ywk253100)
+  * Use the cluster preferred CRD API version when polling for Velero CRD readiness. (#4015, @zubron)
+  * Implement velero debug (#4022, @reasonerjt)
+  * Skip the restore of volumes that originally came from a projected volume when using restic. (#3877, @zubron)
+  * Run the E2E test with kind(provision various versions of k8s cluster) and MinIO on Github Action (#3912, @ywk253100)
+  * Fix -install-velero flag for e2e tests (#3919, @jaidevmane)
+  * Upgrade Velero ClusterRoleBinding to use v1 API (#3926, @jenting)
+  * enable e2e tests to choose crd apiVersion (#3941, @sseago)
+  * Fixing multipleNamespaceTest bug - Missing expect statement in test (#3983, @jaidevmane)
+  * Add --client-page-size flag to server to allow chunking Kubernetes API LIST calls across multiple requests on large clusters (#3823, @dharmab)
+  * Fix CR restore regression introduced in 1.6 restore progress. (#3845, @sseago)
+  * Use region specified in the BackupStorageLocation spec when getting restic repo identifier. Originally fixed by @jala-dx in #3617. (#3857, @zubron)
+  * skip backuping projected volume when using restic (#3866, @alaypatel07)
+  * Install Kubernetes preferred CRDs API version (v1beta1/v1). (#3614, @jenting)
+  * Add Label to BackupSpec so that labels can explicitly be provided to Schedule.Spec.Template.Metadata.Labels which will be reflected on the backups created. (#3641, @arush-sal)
+  * Add PVC UID label to PodVolumeRestore (#3792, @sseago)
+  * Support pulling plugin images by digest (#3803, @2uasimojo)
+  * Added BackupPhaseUploading and BackupPhaseUploadingPartialFailure backup phases as part of Upload Progress Monitoring. (#3805, @dsmithuchida)
+
+    Uploading (new)
+    The "Uploading" phase signifies that the main part of the backup, including 
+    snapshotting has completed successfully and uploading is continuing. In 
+    the event of an error during uploading, the phase will change to 
+    UploadingPartialFailure. On success, the phase changes to Completed. The 
+    backup cannot be restored from when it is in the Uploading state.
+
+    UploadingPartialFailure (new)
+    The "UploadingPartialFailure" phase signifies that the main part of the backup,
+    including snapshotting has completed, but there were partial failures either 
+    during the main part or during the uploading. The backup cannot be restored 
+    from when it is in the UploadingPartialFailure state.
+  * 🐛 Fix plugin name derivation from image name (#3711, @ashish-amarnath)
+  * ✨ ⚠️ Remove CSI volumesnapshot artifact deletion
+
+This change requires https://github.com/vmware-tanzu/velero-plugin-for-csi/pull/86 for Velero to continue
+deleting of CSI volumesnapshots when the corresponding backups are deleted. (#3734, @ashish-amarnath)
+  * use unstructured to marshal selective fields for service restore action (#3789, @alaypatel07)

changelogs/CHANGELOG-1.8.md

@@ -0,0 +1,110 @@
+## v1.8.0
+### 2022-01-14
+
+### Download
+https://github.com/vmware-tanzu/velero/releases/tag/v1.8.0
+
+### Container Image
+`velero/velero:v1.8.0`
+
+### Documentation
+https://velero.io/docs/v1.8
+
+### Upgrading
+https://velero.io/docs/v1.8/upgrade-to-1.8/
+
+### Highlights
+
+#### Velero plugins now support handling volumes created by the CSI drivers of cloud providers
+Versions 1.4 of the Velero plugins for AWS, Azure and GCP now support snapshotting and restoring the persistent volumes provisioned by CSI driver via the APIs of the cloud providers. With this enhancement, users can backup and restore the persistent volumes on these cloud providers without using the Velero CSI plugin. The CSI plugin will remain beta and the feature flag `EnableCSI` will be disabled by default.
+
+For the version of the plugins and the CSI drivers they support respectively please see the table:
+
+| Plugin | Version | CSI Driver |
+| --- | ----------- | ---------- |
+| velero-plugin-for-aws | v1.4.0 | ebs.csi.aws.com |
+| velero-plugin-for-microsoft-azure | v1.4.0 | disk.csi.azure.com |
+| velero-plugin-for-gcp | v1.4.0 | pd.csi.storage.gke.io |
+
+#### IPv6 dual stack support
+We've verified the functionality of Velero on IPv6 dual stack by successfully running the E2E test on IPv6 dual stack environment.
+#### Refactor the controllers using Kubebuilder v3
+In this release we continued our code modernization work, rewriting some controllers using Kubebuilder v3. This work is ongoing and we will continue to make progress in future releases.
+#### Enhancements to E2E test cases
+More test cases have been added to the E2E test suite to improve the release health.
+#### Respect the cron setting of scheduled backup
+The creation time is now taken into account to calculate the next run for scheduled backup.
+
+#### Deleting BSLs also cleans up related resources
+
+When a Backup Storage Location (BSL) is deleted, backup and Restic repository resources will also be deleted.
+
+#### Breaking changes
+
+Starting in v1.8, Velero will only support Kubernetes v1 CRD meaning that Velero v1.8+ will only run on Kubernetes v1.16+. Before upgrading, make sure you are running a supported Kubernetes version. For more information, see our [compatibility matrix](https://github.com/vmware-tanzu/velero#velero-compatibility-matrix).
+
+#### Upload Progress Monitoring and Item Snapshotter
+Item Snapshotter plugin API was merged.  This will support both Upload Progress
+monitoring and the planned Data Mover.  Upload Progress monitoring PRs are
+in progress for 1.9.
+
+### All changes
+
+* E2E test on ssr object with controller namespace mix-ups (#4521, @mqiu)
+* Check whether the volume is provisioned by CSI driver or not by the annotation as well (#4513, @ywk253100)
+* Initialize the labels field of `velero backup-location create` option to avoid #4484 (#4491, @ywk253100)
+* Fix e2e 2500 namespaces scale test timeout problem (#4480, @mqiu)
+* Add backup deletion e2e test  (#4401, @danfengliu)
+* Return the error when getting backup store in backup deletion controller (#4465, @reasonerjt)
+* Ignore the provided port is already allocated error when restoring the LoadBalancer service (#4462, @ywk253100)
+* Revert #4423 migrate backup sync controller to kubebuilder. (#4457, @jxun)
+* Add rbac and annotation test cases (#4455, @mqiu)
+* remove --crds-version in velero install command. (#4446, @jxun)
+* Upgrade e2e test vsphere plugin (#4440, @mqiu)
+* Fix e2e test failures for the inappropriate optimaze of velero install (#4438, @mqiu)
+* Limit backup namespaces on test resource filtering cases (#4437, @mqiu)
+* Bump up Go to 1.17 (#4431, @reasonerjt)
+* Added `<backup name>`-itemsnapshots.json.gz to the backup format.  This file exists
+  when item snapshots are taken and contains an array of volume.Itemsnapshots
+  containing the information about the snapshots.  This will not be used unless
+  upload progress monitoring and item snapshots are enabled and an ItemSnapshot
+  plugin is used to take snapshots.
+
+Also added DownloadTargetKindBackupItemSnapshots for retrieving the signed URL to download only the `<backup name>`-itemsnapshots.json.gz part of a backup for use by
+`velero backup describe`. (#4429, @dsmithuchida)
+* Migrate backup sync controller from code-generator to kubebuilder. (#4423, @jxun)
+* Added UploadProgressFeature flag to enable Upload Progress Monitoring and Item
+  Snapshotters. (#4416, @dsmithuchida)
+* Added BackupWithResolvers and RestoreWithResolvers calls.  Will eventually replace Backup and Restore methods.
+  Adds ItemSnapshotters to Backup and Restore workflows. (#4410, @dsu)
+* Build for darwin-arm64 (#4409, @epk)
+* Add resource filtering test cases (#4404, @mqiu)
+* Fix the issue that the backup cannot be deleted after the application uninstalled (#4398, @ywk253100)
+* Add restoreactionitem plugin to handle admission webhook configurations (#4397, @reasonerjt)
+* Keep the annotation "pv.kubernetes.io/provisioned-by" when restoring PVs (#4391, @ywk253100)
+* Adjust structure of e2e test codes (#4386, @mqiu)
+* feat: migrate velero controller from kubebuilder v2 to v3
+  From Velero v1.8, apiextesions.k8s.io/v1beta1 is no longer supported,
+  which means only CRD of apiextensions.k8s.io/v1 is supported,
+  and the supported Kubernetes version is updated to v1.16 and later. (#4382, @jxun)
+* Delete backups and Restic repos associated with deleted BSL(s) (#4377, @codegold79)
+* Add the key for GKE zone for AZ collection (#4376, @reasonerjt)
+* Fix statefulsets volumeClaimTemplates storageClassName when use Changing PV/PVC Storage Classes (#4375, @Box-Cube)
+* Fix snapshot e2e test issue of jsonpath (#4372, @danfengliu)
+* Modify the timestamp in the name of a backup generated from schedule to use UTC. (#4353, @jxun)
+* Read Availability zone from nodeAffinity requirements  (#4350, @reasonerjt)
+* Use factory.Namespace() to replace hardcoded velero namespace (#4346, @half-life666)
+* Return the error if velero failed to detect S3 region for restic repo (#4343, @reasonerjt)
+* Add init log option for velero controller-runtime manager. (#4341, @jxun)
+* Ignore the `provided port is already allocated` error when restoring the `NodePort` service (#4336, @ywk253100)
+* Fixed an issue with the `backup-location create` command where the BSL Credential field would be set to an invalid empty SecretKeySelector when no credential details were provided. (#4322, @zubron)
+* fix buggy pager func (#4306, @alaypatel07)
+* Don't create a backup immediately after creating a schedule (#4281, @ywk253100)
+* Fix CVE-2020-29652 and CVE-2020-26160 (#4274, @ywk253100)
+* Refine tag-release.sh to align with change in release process (#4185, @reasonerjt)
+* Fix plugins incompatible issue in upgrade test (#4141, @danfengliu)
+* Verify group before treating resource as cohabitating (#4126, @sseago)
+* Added ItemSnapshotter plugin definition and plugin framework - addresses #3533.
+  Part of the Upload Progress enhancement (#3533) (#4077, @dsmithuchida)
+* Add upgrade test in E2E test (#4058, @danfengliu)
+* Handle namespace mapping for PVs without snapshots on restore (#3708, @sseago)

changelogs/CHANGELOG-1.9.md

@@ -0,0 +1,104 @@
+## v1.9.0
+### 2022-06-13
+
+### Download
+https://github.com/vmware-tanzu/velero/releases/tag/v1.9.0
+
+### Container Image
+`velero/velero:v1.9.0`
+
+### Documentation
+https://velero.io/docs/v1.9/
+
+### Upgrading
+https://velero.io/docs/v1.9/upgrade-to-1.9/
+
+### Highlights
+
+#### Improvement to the CSI plugin
+- Bump up to the CSI volume snapshot v1 API
+- No VolumeSnapshot will be left in the source namespace of the workload
+- Report metrics for CSI snapshots
+
+More improvements please refer to [CSI plugin improvement](https://github.com/vmware-tanzu/velero/issues?q=is%3Aissue+label%3A%22CSI+plugin+-+GA+-+phase1%22+is%3Aclosed)
+
+With these improvements we'll provide official support for CSI snapshots on AKS/EKS clusters. (with CSI plugin v0.3.0)
+
+#### Refactor the controllers using Kubebuilder v3
+In this release we continued our code modernization work, rewriting some controllers using Kubebuilder v3. This work is ongoing and we will continue to make progress in future releases.
+
+#### Optionally restore status on selected resources
+Options are added to the CLI and Restore spec to control the group of resources whose status will be restored.
+
+#### ExistingResourcePolicy in the restore API
+Users can choose to overwrite or patch the existing resources during restore by setting this policy.
+
+#### Upgrade integrated Restic version and add skip TLS validation in Restic command
+Upgrade integrated Restic version, which will resolve some of the CVEs, and support skip TLS validation in Restic backup/restore.
+
+#### Breaking changes
+With bumping up the API to v1 in CSI plugin, the v0.3.0 CSI plugin will only work for Kubernetes v1.20+
+
+### All changes
+
+  * restic: add full support for setting SecurityContext for restore init container from configMap. (#4084, @MatthieuFin)
+  * Add metrics backup_items_total and backup_items_errors (#4296, @tobiasgiese)
+  * Convert PodVolumebackup controller to the Kubebuilder framework (#4436, @fgold)
+  * Skip not mounted volumes when backing up (#4497, @dkeven)
+  * Update doc for v1.8 (#4517, @reasonerjt)
+  * Fix bug to make the restic prune frequency configurable (#4518, @ywk253100)
+  * Add E2E test of backups sync from BSL (#4545, @mqiu)
+  * Fix: OrderedResources in Schedules (#4550, @dbrekau)
+  * Skip volumes of non-running pods when backing up (#4584, @bynare)
+  * E2E SSR test add retry mechanism and logs  (#4591, @mqiu)
+  * Add pushing image to GCR in github workflow to facilitate some environments that have rate limitation to docker hub, e.g. vSphere. (#4623, @jxun)
+  * Add existingResourcePolicy to Restore API (#4628, @shubham-pampattiwar)
+  * Fix E2E backup namespaces test (#4634, @qiuming-best)
+  * Update image used by E2E test to gcr.io (#4639, @jxun)
+  * Add multiple label selector support to Velero Backup and Restore APIs (#4650, @shubham-pampattiwar)
+  * Convert Pod Volume Restore resource/controller to the Kubebuilder framework (#4655, @ywk253100)
+  * Update --use-owner-references-in-backup description in velero command line. (#4660, @jxun)
+  * Avoid overwritten hook's exec.container parameter when running pod command executor. (#4661, @jxun)
+  * Support regional pv for GKE (#4680, @jxun)
+  * Bypass the remap CRD version plugin when v1beta1 CRD is not supported (#4686, @reasonerjt)
+  * Add GINKGO_SKIP to support skip specific case in e2e test. (#4692, @jxun)
+  * Add --pod-labels flag to velero install (#4694, @j4m3s-s)
+  * Enable coverage in test.sh and upload to codecov (#4704, @reasonerjt)
+  * Mark the BSL as "Unavailable" when gets any error and add a new field "Message" to the status to record the error message (#4719, @ywk253100)
+  * Support multiple skip option for E2E test (#4725, @jxun)
+  * Add PriorityClass to the AdditionalItems of Backup's PodAction and Restore's PodAction plugin to backup and restore PriorityClass if it is used by a Pod. (#4740, @phuongatemc)
+  * Insert all restore errors and warnings into restore log. (#4743, @sseago)
+  * Refactor schedule controller with kubebuilder (#4748, @ywk253100)
+  * Garbage collector now adds labels to backups that failed to delete for BSLNotFound, BSLCannotGet, BSLReadOnly reasons. (#4757, @kaovilai)
+  * Skip podvolumerestore creation when restore excludes pv/pvc (#4769, @half-life666)
+  * Add parameter for e2e test to support modify kibishii install path. (#4778, @jxun)
+  * Ensure the restore hook applied to new namespace based on the mapping (#4779, @reasonerjt)
+  * Add ability to restore status on selected resources (#4785, @RafaeLeal)
+  * Do not take snapshot for PV to avoid duplicated snapshotting, when CSI feature is enabled. (#4797, @jxun)
+  * Bump up to v1 API for CSI snapshot (#4800, @reasonerjt)
+  * fix: delete empty backups (#4817, @yuvalman)
+  * Add CSI VolumeSnapshot related metrics. (#4818, @jxun)
+  * Fix default-backup-ttl not work (#4831, @qiuming-best)
+  * Make the vsc created by backup sync controller deletable (#4832, @reasonerjt)
+  * Make in-progress backup/restore as failed when doing the reconcile to avoid hanging in in-progress status (#4833, @ywk253100)
+  * Use controller-gen to generate the deep copy methods for objects (#4838, @ywk253100)
+  * Update integrated Restic version and add insecureSkipTLSVerify for Restic CLI. (#4839, @jxun)
+  * Modify CSI VolumeSnapshot metric related code. (#4854, @jxun)
+  * Refactor backup deletion controller based on kubebuilder (#4855, @reasonerjt)
+  * Remove VolumeSnapshots created during backup when CSI feature is enabled. (#4858, @jxun)
+  * Convert Restic Repository resource/controller to the Kubebuilder framework (#4859, @qiuming-best)
+  * Add ClusterClasses to the restore priority list (#4866, @reasonerjt)
+  * Cleanup the .velero folder after restic done (#4872, @big-appled)
+  * Delete orphan CSI snapshots in backup sync controller (#4887, @reasonerjt)
+  * Make waiting VolumeSnapshot to ready process parallel. (#4889, @jxun)
+  * continue rather than return for non-matching restore action label (#4890, @sseago)
+  * Make in-progress PVB/PVR as failed when restic controller restarts to avoid hanging backup/restore (#4893, @ywk253100)
+  * Refactor BSL controller with periodical enqueue source (#4894, @jxun)
+  * Make garbage collection for expired backups configurable (#4897, @ywk253100)
+  * Bump up the version of distroless to base-debian11 (#4898, @ywk253100)
+  * Add schedule ordered resources E2E test (#4913, @qiuming-best)
+  * Make velero completion zsh command output can be used by `source` command. (#4914, @jxun)
+  * Enhance the map flag to support parsing input value contains entry delimiters (#4920, @ywk253100)
+  * Fix E2E test [Backups][Deletion][Restic] on GCP. (#4968, @jxun)
+  * Disable status as sub resource in CRDs (#4972, @ywk253100)
+  * Add more information for failing to get path or snapshot in restic backup and restore. (#4988, @jxun)

cmd/velero-restore-helper/velero-restore-helper.go

@@ -38,6 +38,12 @@ func main() {
 		case <-ticker.C:
 			if done() {
 				fmt.Println("All restic restores are done")
+				err := removeFolder()
+				if err != nil {
+					fmt.Println(err)
+				} else {
+					fmt.Println("Done cleanup .velero folder")
+				}
 				return
 			}
 		}
@@ -75,3 +81,28 @@ func done() bool {
 
 	return true
 }
+
+// remove .velero folder
+func removeFolder() error {
+	children, err := ioutil.ReadDir("/restores")
+	if err != nil {
+		return err
+	}
+
+	for _, child := range children {
+		if !child.IsDir() {
+			fmt.Printf("%s is not a directory, skipping.\n", child.Name())
+			continue
+		}
+
+		donePath := filepath.Join("/restores", child.Name(), ".velero")
+
+		err = os.RemoveAll(donePath)
+		if err != nil {
+			return err
+		}
+		fmt.Printf("Deleted %s", donePath)
+	}
+
+	return nil
+}

cmd/velero/velero.go

@@ -20,7 +20,7 @@ import (
 	"os"
 	"path/filepath"
 
-	"k8s.io/klog"
+	"k8s.io/klog/v2"
 
 	"github.com/vmware-tanzu/velero/pkg/cmd"
 	"github.com/vmware-tanzu/velero/pkg/cmd/velero"

config/crd/v1/bases/velero.io_backuprepositories.yaml

@@ -4,19 +4,26 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    controller-gen.kubebuilder.io/version: v0.3.0
+    controller-gen.kubebuilder.io/version: v0.7.0
   creationTimestamp: null
-  name: resticrepositories.velero.io
+  name: backuprepositories.velero.io
 spec:
   group: velero.io
   names:
-    kind: ResticRepository
-    listKind: ResticRepositoryList
-    plural: resticrepositories
-    singular: resticrepository
+    kind: BackupRepository
+    listKind: BackupRepositoryList
+    plural: backuprepositories
+    singular: backuprepository
   scope: Namespaced
   versions:
-  - name: v1
+  - additionalPrinterColumns:
+    - jsonPath: .metadata.creationTimestamp
+      name: Age
+      type: date
+    - jsonPath: .spec.repositoryType
+      name: Repository Type
+      type: string
+    name: v1
     schema:
       openAPIV3Schema:
         properties:
@@ -33,7 +40,7 @@ spec:
           metadata:
             type: object
           spec:
-            description: ResticRepositorySpec is the specification for a ResticRepository.
+            description: BackupRepositorySpec is the specification for a BackupRepository.
             properties:
               backupStorageLocation:
                 description: BackupStorageLocation is the name of the BackupStorageLocation
@@ -43,12 +50,19 @@ spec:
                 description: MaintenanceFrequency is how often maintenance should
                   be run.
                 type: string
+              repositoryType:
+                description: RepositoryType indicates the type of the backend repository
+                enum:
+                - kopia
+                - restic
+                - ""
+                type: string
               resticIdentifier:
                 description: ResticIdentifier is the full restic-compatible string
                   for identifying this repository.
                 type: string
               volumeNamespace:
-                description: VolumeNamespace is the namespace this restic repository
+                description: VolumeNamespace is the namespace this backup repository
                   contains pod volume backups for.
                 type: string
             required:
@@ -58,7 +72,7 @@ spec:
             - volumeNamespace
             type: object
           status:
-            description: ResticRepositoryStatus is the current status of a ResticRepository.
+            description: BackupRepositoryStatus is the current status of a BackupRepository.
             properties:
               lastMaintenanceTime:
                 description: LastMaintenanceTime is the last time maintenance was
@@ -68,10 +82,10 @@ spec:
                 type: string
               message:
                 description: Message is a message about the current status of the
-                  ResticRepository.
+                  BackupRepository.
                 type: string
               phase:
-                description: Phase is the current state of the ResticRepository.
+                description: Phase is the current state of the BackupRepository.
                 enum:
                 - New
                 - Ready
@@ -81,6 +95,7 @@ spec:
         type: object
     served: true
     storage: true
+    subresources: {}
 status:
   acceptedNames:
     kind: ""

config/crd/v1/bases/velero.io_backups.yaml

@@ -4,7 +4,7 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    controller-gen.kubebuilder.io/version: v0.3.0
+    controller-gen.kubebuilder.io/version: v0.7.0
   creationTimestamp: null
   name: backups.velero.io
 spec:
@@ -37,9 +37,22 @@ spec:
           spec:
             description: BackupSpec defines the specification for a Velero backup.
             properties:
+              csiSnapshotTimeout:
+                description: CSISnapshotTimeout specifies the time used to wait for
+                  CSI VolumeSnapshot status turns to ReadyToUse during creation, before
+                  returning error as timeout. The default value is 10 minute.
+                type: string
+              defaultVolumesToFsBackup:
+                description: DefaultVolumesToFsBackup specifies whether pod volume
+                  file system backup should be used for all volumes by default.
+                nullable: true
+                type: boolean
               defaultVolumesToRestic:
-                description: DefaultVolumesToRestic specifies whether restic should
-                  be used to take a backup of all pod volumes by default.
+                description: "DefaultVolumesToRestic specifies whether restic should
+                  be used to take a backup of all pod volumes by default. \n Deprecated:
+                  this field is no longer used and will be removed entirely in future.
+                  Use DefaultVolumesToFsBackup instead."
+                nullable: true
                 type: boolean
               excludedNamespaces:
                 description: ExcludedNamespaces contains a list of namespaces that
@@ -307,13 +320,76 @@ spec:
                       are ANDed.
                     type: object
                 type: object
+              metadata:
+                properties:
+                  labels:
+                    additionalProperties:
+                      type: string
+                    type: object
+                type: object
+              orLabelSelectors:
+                description: OrLabelSelectors is list of metav1.LabelSelector to filter
+                  with when adding individual objects to the backup. If multiple provided
+                  they will be joined by the OR operator. LabelSelector as well as
+                  OrLabelSelectors cannot co-exist in backup request, only one of
+                  them can be used.
+                items:
+                  description: A label selector is a label query over a set of resources.
+                    The result of matchLabels and matchExpressions are ANDed. An empty
+                    label selector matches all objects. A null label selector matches
+                    no objects.
+                  properties:
+                    matchExpressions:
+                      description: matchExpressions is a list of label selector requirements.
+                        The requirements are ANDed.
+                      items:
+                        description: A label selector requirement is a selector that
+                          contains values, a key, and an operator that relates the
+                          key and values.
+                        properties:
+                          key:
+                            description: key is the label key that the selector applies
+                              to.
+                            type: string
+                          operator:
+                            description: operator represents a key's relationship
+                              to a set of values. Valid operators are In, NotIn, Exists
+                              and DoesNotExist.
+                            type: string
+                          values:
+                            description: values is an array of string values. If the
+                              operator is In or NotIn, the values array must be non-empty.
+                              If the operator is Exists or DoesNotExist, the values
+                              array must be empty. This array is replaced during a
+                              strategic merge patch.
+                            items:
+                              type: string
+                            type: array
+                        required:
+                        - key
+                        - operator
+                        type: object
+                      type: array
+                    matchLabels:
+                      additionalProperties:
+                        type: string
+                      description: matchLabels is a map of {key,value} pairs. A single
+                        {key,value} in the matchLabels map is equivalent to an element
+                        of matchExpressions, whose key field is "key", the operator
+                        is "In", and the values array contains only "value". The requirements
+                        are ANDed.
+                      type: object
+                  type: object
+                nullable: true
+                type: array
               orderedResources:
                 additionalProperties:
                   type: string
                 description: OrderedResources specifies the backup order of resources
-                  of specific Kind. The map key is the Kind name and value is a list
-                  of resource names separated by commas. Each resource name has format
-                  "namespace/resourcename".  For cluster resources, simply use "resourcename".
+                  of specific Kind. The map key is the resource name and value is
+                  a list of object names separated by commas. Each resource name has
+                  format "namespace/objectname".  For cluster resources, simply use
+                  "objectname".
                 nullable: true
                 type: object
               snapshotVolumes:
@@ -347,6 +423,14 @@ spec:
                 format: date-time
                 nullable: true
                 type: string
+              csiVolumeSnapshotsAttempted:
+                description: CSIVolumeSnapshotsAttempted is the total number of attempted
+                  CSI VolumeSnapshots for this backup.
+                type: integer
+              csiVolumeSnapshotsCompleted:
+                description: CSIVolumeSnapshotsCompleted is the total number of successfully
+                  completed CSI VolumeSnapshots for this backup.
+                type: integer
               errors:
                 description: Errors is a count of all error messages that were generated
                   during execution of the backup.  The actual errors are in the backup's
@@ -357,6 +441,10 @@ spec:
                 format: date-time
                 nullable: true
                 type: string
+              failureReason:
+                description: FailureReason is an error that caused the entire backup
+                  to fail.
+                type: string
               formatVersion:
                 description: FormatVersion is the backup format version, including
                   major, minor, and patch version.

config/crd/v1/bases/velero.io_backupstoragelocations.yaml

@@ -4,7 +4,7 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    controller-gen.kubebuilder.io/version: v0.3.0
+    controller-gen.kubebuilder.io/version: v0.7.0
   creationTimestamp: null
   name: backupstoragelocations.velero.io
 spec:
@@ -158,6 +158,10 @@ spec:
                 format: date-time
                 nullable: true
                 type: string
+              message:
+                description: Message is a message about the backup storage location's
+                  status.
+                type: string
               phase:
                 description: Phase is the current state of the BackupStorageLocation.
                 enum:
@@ -168,8 +172,7 @@ spec:
         type: object
     served: true
     storage: true
-    subresources:
-      status: {}
+    subresources: {}
 status:
   acceptedNames:
     kind: ""

config/crd/v1/bases/velero.io_deletebackuprequests.yaml

@@ -4,7 +4,7 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    controller-gen.kubebuilder.io/version: v0.3.0
+    controller-gen.kubebuilder.io/version: v0.7.0
   creationTimestamp: null
   name: deletebackuprequests.velero.io
 spec:
@@ -16,7 +16,16 @@ spec:
     singular: deletebackuprequest
   scope: Namespaced
   versions:
-  - name: v1
+  - additionalPrinterColumns:
+    - description: The name of the backup to be deleted
+      jsonPath: .spec.backupName
+      name: BackupName
+      type: string
+    - description: The status of the deletion request
+      jsonPath: .status.phase
+      name: Status
+      type: string
+    name: v1
     schema:
       openAPIV3Schema:
         description: DeleteBackupRequest is a request to delete one or more backups.
@@ -63,6 +72,7 @@ spec:
         type: object
     served: true
     storage: true
+    subresources: {}
 status:
   acceptedNames:
     kind: ""

config/crd/v1/bases/velero.io_downloadrequests.yaml

@@ -4,7 +4,7 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    controller-gen.kubebuilder.io/version: v0.3.0
+    controller-gen.kubebuilder.io/version: v0.7.0
   creationTimestamp: null
   name: downloadrequests.velero.io
 spec:
@@ -46,9 +46,12 @@ spec:
                     - BackupLog
                     - BackupContents
                     - BackupVolumeSnapshots
+                    - BackupItemSnapshots
                     - BackupResourceList
                     - RestoreLog
                     - RestoreResults
+                    - CSIBackupVolumeSnapshots
+                    - CSIBackupVolumeSnapshotContents
                     type: string
                   name:
                     description: Name is the name of the kubernetes resource with
@@ -84,8 +87,6 @@ spec:
         type: object
     served: true
     storage: true
-    subresources:
-      status: {}
 status:
   acceptedNames:
     kind: ""

config/crd/v1/bases/velero.io_podvolumebackups.yaml

@@ -4,7 +4,7 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    controller-gen.kubebuilder.io/version: v0.3.0
+    controller-gen.kubebuilder.io/version: v0.7.0
   creationTimestamp: null
   name: podvolumebackups.velero.io
 spec:
@@ -16,7 +16,44 @@ spec:
     singular: podvolumebackup
   scope: Namespaced
   versions:
-  - name: v1
+  - additionalPrinterColumns:
+    - description: Pod Volume Backup status such as New/InProgress
+      jsonPath: .status.phase
+      name: Status
+      type: string
+    - description: Time when this backup was started
+      jsonPath: .status.startTimestamp
+      name: Created
+      type: date
+    - description: Namespace of the pod containing the volume to be backed up
+      jsonPath: .spec.pod.namespace
+      name: Namespace
+      type: string
+    - description: Name of the pod containing the volume to be backed up
+      jsonPath: .spec.pod.name
+      name: Pod
+      type: string
+    - description: Name of the volume to be backed up
+      jsonPath: .spec.volume
+      name: Volume
+      type: string
+    - description: Backup repository identifier for this backup
+      jsonPath: .spec.repoIdentifier
+      name: Repository ID
+      type: string
+    - description: The type of the uploader to handle data transfer
+      jsonPath: .spec.uploaderType
+      name: Uploader Type
+      type: string
+    - description: Name of the Backup Storage Location where this backup should be
+        stored
+      jsonPath: .spec.backupStorageLocation
+      name: Storage Location
+      type: string
+    - jsonPath: .metadata.creationTimestamp
+      name: Age
+      type: date
+    name: v1
     schema:
       openAPIV3Schema:
         properties:
@@ -37,7 +74,7 @@ spec:
             properties:
               backupStorageLocation:
                 description: BackupStorageLocation is the name of the backup storage
-                  location where the restic repository is stored.
+                  location where the backup repository is stored.
                 type: string
               node:
                 description: Node is the name of the node that the Pod is running
@@ -81,7 +118,7 @@ spec:
                     type: string
                 type: object
               repoIdentifier:
-                description: RepoIdentifier is the restic repository identifier.
+                description: RepoIdentifier is the backup repository identifier.
                 type: string
               tags:
                 additionalProperties:
@@ -89,6 +126,14 @@ spec:
                 description: Tags are a map of key-value pairs that should be applied
                   to the volume backup as tags.
                 type: object
+              uploaderType:
+                description: UploaderType is the type of the uploader to handle the
+                  data transfer.
+                enum:
+                - kopia
+                - restic
+                - ""
+                type: string
               volume:
                 description: Volume is the name of the volume within the Pod to be
                   backed up.
@@ -153,6 +198,7 @@ spec:
         type: object
     served: true
     storage: true
+    subresources: {}
 status:
   acceptedNames:
     kind: ""

config/crd/v1/bases/velero.io_podvolumerestores.yaml

@@ -4,7 +4,7 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    controller-gen.kubebuilder.io/version: v0.3.0
+    controller-gen.kubebuilder.io/version: v0.7.0
   creationTimestamp: null
   name: podvolumerestores.velero.io
 spec:
@@ -16,7 +16,41 @@ spec:
     singular: podvolumerestore
   scope: Namespaced
   versions:
-  - name: v1
+  - additionalPrinterColumns:
+    - description: Namespace of the pod containing the volume to be restored
+      jsonPath: .spec.pod.namespace
+      name: Namespace
+      type: string
+    - description: Name of the pod containing the volume to be restored
+      jsonPath: .spec.pod.name
+      name: Pod
+      type: string
+    - description: The type of the uploader to handle data transfer
+      jsonPath: .spec.uploaderType
+      name: Uploader Type
+      type: string
+    - description: Name of the volume to be restored
+      jsonPath: .spec.volume
+      name: Volume
+      type: string
+    - description: Pod Volume Restore status such as New/InProgress
+      jsonPath: .status.phase
+      name: Status
+      type: string
+    - description: Pod Volume Restore status such as New/InProgress
+      format: int64
+      jsonPath: .status.progress.totalBytes
+      name: TotalBytes
+      type: integer
+    - description: Pod Volume Restore status such as New/InProgress
+      format: int64
+      jsonPath: .status.progress.bytesDone
+      name: BytesDone
+      type: integer
+    - jsonPath: .metadata.creationTimestamp
+      name: Age
+      type: date
+    name: v1
     schema:
       openAPIV3Schema:
         properties:
@@ -37,7 +71,7 @@ spec:
             properties:
               backupStorageLocation:
                 description: BackupStorageLocation is the name of the backup storage
-                  location where the restic repository is stored.
+                  location where the backup repository is stored.
                 type: string
               pod:
                 description: Pod is a reference to the pod containing the volume to
@@ -77,11 +111,23 @@ spec:
                     type: string
                 type: object
               repoIdentifier:
-                description: RepoIdentifier is the restic repository identifier.
+                description: RepoIdentifier is the backup repository identifier.
                 type: string
               snapshotID:
                 description: SnapshotID is the ID of the volume snapshot to be restored.
                 type: string
+              sourceNamespace:
+                description: SourceNamespace is the original namespace for namaspace
+                  mapping.
+                type: string
+              uploaderType:
+                description: UploaderType is the type of the uploader to handle the
+                  data transfer.
+                enum:
+                - kopia
+                - restic
+                - ""
+                type: string
               volume:
                 description: Volume is the name of the volume within the Pod to be
                   restored.
@@ -91,6 +137,7 @@ spec:
             - pod
             - repoIdentifier
             - snapshotID
+            - sourceNamespace
             - volume
             type: object
           status:
@@ -136,6 +183,7 @@ spec:
         type: object
     served: true
     storage: true
+    subresources: {}
 status:
   acceptedNames:
     kind: ""

config/crd/v1/bases/velero.io_schedules.yaml

@@ -4,7 +4,7 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    controller-gen.kubebuilder.io/version: v0.3.0
+    controller-gen.kubebuilder.io/version: v0.7.0
   creationTimestamp: null
   name: schedules.velero.io
 spec:
@@ -16,7 +16,26 @@ spec:
     singular: schedule
   scope: Namespaced
   versions:
-  - name: v1
+  - additionalPrinterColumns:
+    - description: Status of the schedule
+      jsonPath: .status.phase
+      name: Status
+      type: string
+    - description: A Cron expression defining when to run the Backup
+      jsonPath: .spec.schedule
+      name: Schedule
+      type: string
+    - description: The last time a Backup was run for this schedule
+      jsonPath: .status.lastBackup
+      name: LastBackup
+      type: date
+    - jsonPath: .metadata.creationTimestamp
+      name: Age
+      type: date
+    - jsonPath: .spec.paused
+      name: Paused
+      type: boolean
+    name: v1
     schema:
       openAPIV3Schema:
         description: Schedule is a Velero resource that represents a pre-scheduled
@@ -37,6 +56,9 @@ spec:
           spec:
             description: ScheduleSpec defines the specification for a Velero schedule
             properties:
+              paused:
+                description: Paused specifies whether the schedule is paused or not
+                type: boolean
               schedule:
                 description: Schedule is a Cron expression defining when to run the
                   Backup.
@@ -45,9 +67,22 @@ spec:
                 description: Template is the definition of the Backup to be run on
                   the provided schedule
                 properties:
+                  csiSnapshotTimeout:
+                    description: CSISnapshotTimeout specifies the time used to wait
+                      for CSI VolumeSnapshot status turns to ReadyToUse during creation,
+                      before returning error as timeout. The default value is 10 minute.
+                    type: string
+                  defaultVolumesToFsBackup:
+                    description: DefaultVolumesToFsBackup specifies whether pod volume
+                      file system backup should be used for all volumes by default.
+                    nullable: true
+                    type: boolean
                   defaultVolumesToRestic:
-                    description: DefaultVolumesToRestic specifies whether restic should
-                      be used to take a backup of all pod volumes by default.
+                    description: "DefaultVolumesToRestic specifies whether restic
+                      should be used to take a backup of all pod volumes by default.
+                      \n Deprecated: this field is no longer used and will be removed
+                      entirely in future. Use DefaultVolumesToFsBackup instead."
+                    nullable: true
                     type: boolean
                   excludedNamespaces:
                     description: ExcludedNamespaces contains a list of namespaces
@@ -320,14 +355,76 @@ spec:
                           "value". The requirements are ANDed.
                         type: object
                     type: object
+                  metadata:
+                    properties:
+                      labels:
+                        additionalProperties:
+                          type: string
+                        type: object
+                    type: object
+                  orLabelSelectors:
+                    description: OrLabelSelectors is list of metav1.LabelSelector
+                      to filter with when adding individual objects to the backup.
+                      If multiple provided they will be joined by the OR operator.
+                      LabelSelector as well as OrLabelSelectors cannot co-exist in
+                      backup request, only one of them can be used.
+                    items:
+                      description: A label selector is a label query over a set of
+                        resources. The result of matchLabels and matchExpressions
+                        are ANDed. An empty label selector matches all objects. A
+                        null label selector matches no objects.
+                      properties:
+                        matchExpressions:
+                          description: matchExpressions is a list of label selector
+                            requirements. The requirements are ANDed.
+                          items:
+                            description: A label selector requirement is a selector
+                              that contains values, a key, and an operator that relates
+                              the key and values.
+                            properties:
+                              key:
+                                description: key is the label key that the selector
+                                  applies to.
+                                type: string
+                              operator:
+                                description: operator represents a key's relationship
+                                  to a set of values. Valid operators are In, NotIn,
+                                  Exists and DoesNotExist.
+                                type: string
+                              values:
+                                description: values is an array of string values.
+                                  If the operator is In or NotIn, the values array
+                                  must be non-empty. If the operator is Exists or
+                                  DoesNotExist, the values array must be empty. This
+                                  array is replaced during a strategic merge patch.
+                                items:
+                                  type: string
+                                type: array
+                            required:
+                            - key
+                            - operator
+                            type: object
+                          type: array
+                        matchLabels:
+                          additionalProperties:
+                            type: string
+                          description: matchLabels is a map of {key,value} pairs.
+                            A single {key,value} in the matchLabels map is equivalent
+                            to an element of matchExpressions, whose key field is
+                            "key", the operator is "In", and the values array contains
+                            only "value". The requirements are ANDed.
+                          type: object
+                      type: object
+                    nullable: true
+                    type: array
                   orderedResources:
                     additionalProperties:
                       type: string
                     description: OrderedResources specifies the backup order of resources
-                      of specific Kind. The map key is the Kind name and value is
-                      a list of resource names separated by commas. Each resource
-                      name has format "namespace/resourcename".  For cluster resources,
-                      simply use "resourcename".
+                      of specific Kind. The map key is the resource name and value
+                      is a list of object names separated by commas. Each resource
+                      name has format "namespace/objectname".  For cluster resources,
+                      simply use "objectname".
                     nullable: true
                     type: object
                   snapshotVolumes:
@@ -386,6 +483,7 @@ spec:
         type: object
     served: true
     storage: true
+    subresources: {}
 status:
   acceptedNames:
     kind: ""

config/crd/v1/bases/velero.io_serverstatusrequests.yaml

@@ -4,7 +4,7 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    controller-gen.kubebuilder.io/version: v0.3.0
+    controller-gen.kubebuilder.io/version: v0.7.0
   creationTimestamp: null
   name: serverstatusrequests.velero.io
 spec:
@@ -77,8 +77,6 @@ spec:
         type: object
     served: true
     storage: true
-    subresources:
-      status: {}
 status:
   acceptedNames:
     kind: ""

config/crd/v1/bases/velero.io_volumesnapshotlocations.yaml

@@ -4,7 +4,7 @@ apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
   annotations:
-    controller-gen.kubebuilder.io/version: v0.3.0
+    controller-gen.kubebuilder.io/version: v0.7.0
   creationTimestamp: null
   name: volumesnapshotlocations.velero.io
 spec:
@@ -13,6 +13,8 @@ spec:
     kind: VolumeSnapshotLocation
     listKind: VolumeSnapshotLocationList
     plural: volumesnapshotlocations
+    shortNames:
+    - vsl
     singular: volumesnapshotlocation
   scope: Namespaced
   versions:
@@ -43,6 +45,24 @@ spec:
                   type: string
                 description: Config is for provider-specific configuration fields.
                 type: object
+              credential:
+                description: Credential contains the credential information intended
+                  to be used with this location
+                properties:
+                  key:
+                    description: The key of the secret to select from.  Must be a
+                      valid secret key.
+                    type: string
+                  name:
+                    description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+                      TODO: Add other useful fields. apiVersion, kind, uid?'
+                    type: string
+                  optional:
+                    description: Specify whether the Secret or its key must be defined
+                    type: boolean
+                required:
+                - key
+                type: object
               provider:
                 description: Provider is the provider of the volume storage.
                 type: string

config/crd/v1beta1/bases/velero.io_backups.yaml

@@ -1,432 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: v0.3.0
-  creationTimestamp: null
-  name: backups.velero.io
-spec:
-  group: velero.io
-  names:
-    kind: Backup
-    listKind: BackupList
-    plural: backups
-    singular: backup
-  preserveUnknownFields: false
-  scope: Namespaced
-  validation:
-    openAPIV3Schema:
-      description: Backup is a Velero resource that represents the capture of Kubernetes
-        cluster state at a point in time (API objects and associated volume state).
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        spec:
-          description: BackupSpec defines the specification for a Velero backup.
-          properties:
-            defaultVolumesToRestic:
-              description: DefaultVolumesToRestic specifies whether restic should
-                be used to take a backup of all pod volumes by default.
-              type: boolean
-            excludedNamespaces:
-              description: ExcludedNamespaces contains a list of namespaces that are
-                not included in the backup.
-              items:
-                type: string
-              nullable: true
-              type: array
-            excludedResources:
-              description: ExcludedResources is a slice of resource names that are
-                not included in the backup.
-              items:
-                type: string
-              nullable: true
-              type: array
-            hooks:
-              description: Hooks represent custom behaviors that should be executed
-                at different phases of the backup.
-              properties:
-                resources:
-                  description: Resources are hooks that should be executed when backing
-                    up individual instances of a resource.
-                  items:
-                    description: BackupResourceHookSpec defines one or more BackupResourceHooks
-                      that should be executed based on the rules defined for namespaces,
-                      resources, and label selector.
-                    properties:
-                      excludedNamespaces:
-                        description: ExcludedNamespaces specifies the namespaces to
-                          which this hook spec does not apply.
-                        items:
-                          type: string
-                        nullable: true
-                        type: array
-                      excludedResources:
-                        description: ExcludedResources specifies the resources to
-                          which this hook spec does not apply.
-                        items:
-                          type: string
-                        nullable: true
-                        type: array
-                      includedNamespaces:
-                        description: IncludedNamespaces specifies the namespaces to
-                          which this hook spec applies. If empty, it applies to all
-                          namespaces.
-                        items:
-                          type: string
-                        nullable: true
-                        type: array
-                      includedResources:
-                        description: IncludedResources specifies the resources to
-                          which this hook spec applies. If empty, it applies to all
-                          resources.
-                        items:
-                          type: string
-                        nullable: true
-                        type: array
-                      labelSelector:
-                        description: LabelSelector, if specified, filters the resources
-                          to which this hook spec applies.
-                        nullable: true
-                        properties:
-                          matchExpressions:
-                            description: matchExpressions is a list of label selector
-                              requirements. The requirements are ANDed.
-                            items:
-                              description: A label selector requirement is a selector
-                                that contains values, a key, and an operator that
-                                relates the key and values.
-                              properties:
-                                key:
-                                  description: key is the label key that the selector
-                                    applies to.
-                                  type: string
-                                operator:
-                                  description: operator represents a key's relationship
-                                    to a set of values. Valid operators are In, NotIn,
-                                    Exists and DoesNotExist.
-                                  type: string
-                                values:
-                                  description: values is an array of string values.
-                                    If the operator is In or NotIn, the values array
-                                    must be non-empty. If the operator is Exists or
-                                    DoesNotExist, the values array must be empty.
-                                    This array is replaced during a strategic merge
-                                    patch.
-                                  items:
-                                    type: string
-                                  type: array
-                              required:
-                              - key
-                              - operator
-                              type: object
-                            type: array
-                          matchLabels:
-                            additionalProperties:
-                              type: string
-                            description: matchLabels is a map of {key,value} pairs.
-                              A single {key,value} in the matchLabels map is equivalent
-                              to an element of matchExpressions, whose key field is
-                              "key", the operator is "In", and the values array contains
-                              only "value". The requirements are ANDed.
-                            type: object
-                        type: object
-                      name:
-                        description: Name is the name of this hook.
-                        type: string
-                      post:
-                        description: PostHooks is a list of BackupResourceHooks to
-                          execute after storing the item in the backup. These are
-                          executed after all "additional items" from item actions
-                          are processed.
-                        items:
-                          description: BackupResourceHook defines a hook for a resource.
-                          properties:
-                            exec:
-                              description: Exec defines an exec hook.
-                              properties:
-                                command:
-                                  description: Command is the command and arguments
-                                    to execute.
-                                  items:
-                                    type: string
-                                  minItems: 1
-                                  type: array
-                                container:
-                                  description: Container is the container in the pod
-                                    where the command should be executed. If not specified,
-                                    the pod's first container is used.
-                                  type: string
-                                onError:
-                                  description: OnError specifies how Velero should
-                                    behave if it encounters an error executing this
-                                    hook.
-                                  enum:
-                                  - Continue
-                                  - Fail
-                                  type: string
-                                timeout:
-                                  description: Timeout defines the maximum amount
-                                    of time Velero should wait for the hook to complete
-                                    before considering the execution a failure.
-                                  type: string
-                              required:
-                              - command
-                              type: object
-                          required:
-                          - exec
-                          type: object
-                        type: array
-                      pre:
-                        description: PreHooks is a list of BackupResourceHooks to
-                          execute prior to storing the item in the backup. These are
-                          executed before any "additional items" from item actions
-                          are processed.
-                        items:
-                          description: BackupResourceHook defines a hook for a resource.
-                          properties:
-                            exec:
-                              description: Exec defines an exec hook.
-                              properties:
-                                command:
-                                  description: Command is the command and arguments
-                                    to execute.
-                                  items:
-                                    type: string
-                                  minItems: 1
-                                  type: array
-                                container:
-                                  description: Container is the container in the pod
-                                    where the command should be executed. If not specified,
-                                    the pod's first container is used.
-                                  type: string
-                                onError:
-                                  description: OnError specifies how Velero should
-                                    behave if it encounters an error executing this
-                                    hook.
-                                  enum:
-                                  - Continue
-                                  - Fail
-                                  type: string
-                                timeout:
-                                  description: Timeout defines the maximum amount
-                                    of time Velero should wait for the hook to complete
-                                    before considering the execution a failure.
-                                  type: string
-                              required:
-                              - command
-                              type: object
-                          required:
-                          - exec
-                          type: object
-                        type: array
-                    required:
-                    - name
-                    type: object
-                  nullable: true
-                  type: array
-              type: object
-            includeClusterResources:
-              description: IncludeClusterResources specifies whether cluster-scoped
-                resources should be included for consideration in the backup.
-              nullable: true
-              type: boolean
-            includedNamespaces:
-              description: IncludedNamespaces is a slice of namespace names to include
-                objects from. If empty, all namespaces are included.
-              items:
-                type: string
-              nullable: true
-              type: array
-            includedResources:
-              description: IncludedResources is a slice of resource names to include
-                in the backup. If empty, all resources are included.
-              items:
-                type: string
-              nullable: true
-              type: array
-            labelSelector:
-              description: LabelSelector is a metav1.LabelSelector to filter with
-                when adding individual objects to the backup. If empty or nil, all
-                objects are included. Optional.
-              nullable: true
-              properties:
-                matchExpressions:
-                  description: matchExpressions is a list of label selector requirements.
-                    The requirements are ANDed.
-                  items:
-                    description: A label selector requirement is a selector that contains
-                      values, a key, and an operator that relates the key and values.
-                    properties:
-                      key:
-                        description: key is the label key that the selector applies
-                          to.
-                        type: string
-                      operator:
-                        description: operator represents a key's relationship to a
-                          set of values. Valid operators are In, NotIn, Exists and
-                          DoesNotExist.
-                        type: string
-                      values:
-                        description: values is an array of string values. If the operator
-                          is In or NotIn, the values array must be non-empty. If the
-                          operator is Exists or DoesNotExist, the values array must
-                          be empty. This array is replaced during a strategic merge
-                          patch.
-                        items:
-                          type: string
-                        type: array
-                    required:
-                    - key
-                    - operator
-                    type: object
-                  type: array
-                matchLabels:
-                  additionalProperties:
-                    type: string
-                  description: matchLabels is a map of {key,value} pairs. A single
-                    {key,value} in the matchLabels map is equivalent to an element
-                    of matchExpressions, whose key field is "key", the operator is
-                    "In", and the values array contains only "value". The requirements
-                    are ANDed.
-                  type: object
-              type: object
-            orderedResources:
-              additionalProperties:
-                type: string
-              description: OrderedResources specifies the backup order of resources
-                of specific Kind. The map key is the Kind name and value is a list
-                of resource names separated by commas. Each resource name has format
-                "namespace/resourcename".  For cluster resources, simply use "resourcename".
-              nullable: true
-              type: object
-            snapshotVolumes:
-              description: SnapshotVolumes specifies whether to take cloud snapshots
-                of any PV's referenced in the set of objects included in the Backup.
-              nullable: true
-              type: boolean
-            storageLocation:
-              description: StorageLocation is a string containing the name of a BackupStorageLocation
-                where the backup should be stored.
-              type: string
-            ttl:
-              description: TTL is a time.Duration-parseable string describing how
-                long the Backup should be retained for.
-              type: string
-            volumeSnapshotLocations:
-              description: VolumeSnapshotLocations is a list containing names of VolumeSnapshotLocations
-                associated with this backup.
-              items:
-                type: string
-              type: array
-          type: object
-        status:
-          description: BackupStatus captures the current status of a Velero backup.
-          properties:
-            completionTimestamp:
-              description: CompletionTimestamp records the time a backup was completed.
-                Completion time is recorded even on failed backups. Completion time
-                is recorded before uploading the backup object. The server's time
-                is used for CompletionTimestamps
-              format: date-time
-              nullable: true
-              type: string
-            errors:
-              description: Errors is a count of all error messages that were generated
-                during execution of the backup.  The actual errors are in the backup's
-                log file in object storage.
-              type: integer
-            expiration:
-              description: Expiration is when this Backup is eligible for garbage-collection.
-              format: date-time
-              nullable: true
-              type: string
-            formatVersion:
-              description: FormatVersion is the backup format version, including major,
-                minor, and patch version.
-              type: string
-            phase:
-              description: Phase is the current state of the Backup.
-              enum:
-              - New
-              - FailedValidation
-              - InProgress
-              - Completed
-              - PartiallyFailed
-              - Failed
-              - Deleting
-              type: string
-            progress:
-              description: Progress contains information about the backup's execution
-                progress. Note that this information is best-effort only -- if Velero
-                fails to update it during a backup for any reason, it may be inaccurate/stale.
-              nullable: true
-              properties:
-                itemsBackedUp:
-                  description: ItemsBackedUp is the number of items that have actually
-                    been written to the backup tarball so far.
-                  type: integer
-                totalItems:
-                  description: TotalItems is the total number of items to be backed
-                    up. This number may change throughout the execution of the backup
-                    due to plugins that return additional related items to back up,
-                    the velero.io/exclude-from-backup label, and various other filters
-                    that happen as items are processed.
-                  type: integer
-              type: object
-            startTimestamp:
-              description: StartTimestamp records the time a backup was started. Separate
-                from CreationTimestamp, since that value changes on restores. The
-                server's time is used for StartTimestamps
-              format: date-time
-              nullable: true
-              type: string
-            validationErrors:
-              description: ValidationErrors is a slice of all validation errors (if
-                applicable).
-              items:
-                type: string
-              nullable: true
-              type: array
-            version:
-              description: 'Version is the backup format major version. Deprecated:
-                Please see FormatVersion'
-              type: integer
-            volumeSnapshotsAttempted:
-              description: VolumeSnapshotsAttempted is the total number of attempted
-                volume snapshots for this backup.
-              type: integer
-            volumeSnapshotsCompleted:
-              description: VolumeSnapshotsCompleted is the total number of successfully
-                completed volume snapshots for this backup.
-              type: integer
-            warnings:
-              description: Warnings is a count of all warning messages that were generated
-                during execution of the backup. The actual warnings are in the backup's
-                log file in object storage.
-              type: integer
-          type: object
-      type: object
-  version: v1
-  versions:
-  - name: v1
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crd/v1beta1/bases/velero.io_backupstoragelocations.yaml

@@ -1,179 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: v0.3.0
-  creationTimestamp: null
-  name: backupstoragelocations.velero.io
-spec:
-  additionalPrinterColumns:
-  - JSONPath: .status.phase
-    description: Backup Storage Location status such as Available/Unavailable
-    name: Phase
-    type: string
-  - JSONPath: .status.lastValidationTime
-    description: LastValidationTime is the last time the backup store location was
-      validated
-    name: Last Validated
-    type: date
-  - JSONPath: .metadata.creationTimestamp
-    name: Age
-    type: date
-  - JSONPath: .spec.default
-    description: Default backup storage location
-    name: Default
-    type: boolean
-  group: velero.io
-  names:
-    kind: BackupStorageLocation
-    listKind: BackupStorageLocationList
-    plural: backupstoragelocations
-    shortNames:
-    - bsl
-    singular: backupstoragelocation
-  preserveUnknownFields: false
-  scope: Namespaced
-  subresources:
-    status: {}
-  validation:
-    openAPIV3Schema:
-      description: BackupStorageLocation is a location where Velero stores backup
-        objects
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        spec:
-          description: BackupStorageLocationSpec defines the desired state of a Velero
-            BackupStorageLocation
-          properties:
-            accessMode:
-              description: AccessMode defines the permissions for the backup storage
-                location.
-              enum:
-              - ReadOnly
-              - ReadWrite
-              type: string
-            backupSyncPeriod:
-              description: BackupSyncPeriod defines how frequently to sync backup
-                API objects from object storage. A value of 0 disables sync.
-              nullable: true
-              type: string
-            config:
-              additionalProperties:
-                type: string
-              description: Config is for provider-specific configuration fields.
-              type: object
-            credential:
-              description: Credential contains the credential information intended
-                to be used with this location
-              properties:
-                key:
-                  description: The key of the secret to select from.  Must be a valid
-                    secret key.
-                  type: string
-                name:
-                  description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
-                    TODO: Add other useful fields. apiVersion, kind, uid?'
-                  type: string
-                optional:
-                  description: Specify whether the Secret or its key must be defined
-                  type: boolean
-              required:
-              - key
-              type: object
-            default:
-              description: Default indicates this location is the default backup storage
-                location.
-              type: boolean
-            objectStorage:
-              description: ObjectStorageLocation specifies the settings necessary
-                to connect to a provider's object storage.
-              properties:
-                bucket:
-                  description: Bucket is the bucket to use for object storage.
-                  type: string
-                caCert:
-                  description: CACert defines a CA bundle to use when verifying TLS
-                    connections to the provider.
-                  format: byte
-                  type: string
-                prefix:
-                  description: Prefix is the path inside a bucket to use for Velero
-                    storage. Optional.
-                  type: string
-              required:
-              - bucket
-              type: object
-            provider:
-              description: Provider is the provider of the backup storage.
-              type: string
-            validationFrequency:
-              description: ValidationFrequency defines how frequently to validate
-                the corresponding object storage. A value of 0 disables validation.
-              nullable: true
-              type: string
-          required:
-          - objectStorage
-          - provider
-          type: object
-        status:
-          description: BackupStorageLocationStatus defines the observed state of BackupStorageLocation
-          properties:
-            accessMode:
-              description: "AccessMode is an unused field. \n Deprecated: there is
-                now an AccessMode field on the Spec and this field will be removed
-                entirely as of v2.0."
-              enum:
-              - ReadOnly
-              - ReadWrite
-              type: string
-            lastSyncedRevision:
-              description: "LastSyncedRevision is the value of the `metadata/revision`
-                file in the backup storage location the last time the BSL's contents
-                were synced into the cluster. \n Deprecated: this field is no longer
-                updated or used for detecting changes to the location's contents and
-                will be removed entirely in v2.0."
-              type: string
-            lastSyncedTime:
-              description: LastSyncedTime is the last time the contents of the location
-                were synced into the cluster.
-              format: date-time
-              nullable: true
-              type: string
-            lastValidationTime:
-              description: LastValidationTime is the last time the backup store location
-                was validated the cluster.
-              format: date-time
-              nullable: true
-              type: string
-            phase:
-              description: Phase is the current state of the BackupStorageLocation.
-              enum:
-              - Available
-              - Unavailable
-              type: string
-          type: object
-      type: object
-  version: v1
-  versions:
-  - name: v1
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crd/v1beta1/bases/velero.io_deletebackuprequests.yaml

@@ -1,73 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: v0.3.0
-  creationTimestamp: null
-  name: deletebackuprequests.velero.io
-spec:
-  group: velero.io
-  names:
-    kind: DeleteBackupRequest
-    listKind: DeleteBackupRequestList
-    plural: deletebackuprequests
-    singular: deletebackuprequest
-  preserveUnknownFields: false
-  scope: Namespaced
-  validation:
-    openAPIV3Schema:
-      description: DeleteBackupRequest is a request to delete one or more backups.
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        spec:
-          description: DeleteBackupRequestSpec is the specification for which backups
-            to delete.
-          properties:
-            backupName:
-              type: string
-          required:
-          - backupName
-          type: object
-        status:
-          description: DeleteBackupRequestStatus is the current status of a DeleteBackupRequest.
-          properties:
-            errors:
-              description: Errors contains any errors that were encountered during
-                the deletion process.
-              items:
-                type: string
-              nullable: true
-              type: array
-            phase:
-              description: Phase is the current state of the DeleteBackupRequest.
-              enum:
-              - New
-              - InProgress
-              - Processed
-              type: string
-          type: object
-      type: object
-  version: v1
-  versions:
-  - name: v1
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crd/v1beta1/bases/velero.io_downloadrequests.yaml

@@ -1,96 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: v0.3.0
-  creationTimestamp: null
-  name: downloadrequests.velero.io
-spec:
-  group: velero.io
-  names:
-    kind: DownloadRequest
-    listKind: DownloadRequestList
-    plural: downloadrequests
-    singular: downloadrequest
-  preserveUnknownFields: false
-  scope: Namespaced
-  subresources:
-    status: {}
-  validation:
-    openAPIV3Schema:
-      description: DownloadRequest is a request to download an artifact from backup
-        object storage, such as a backup log file.
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        spec:
-          description: DownloadRequestSpec is the specification for a download request.
-          properties:
-            target:
-              description: Target is what to download (e.g. logs for a backup).
-              properties:
-                kind:
-                  description: Kind is the type of file to download.
-                  enum:
-                  - BackupLog
-                  - BackupContents
-                  - BackupVolumeSnapshots
-                  - BackupResourceList
-                  - RestoreLog
-                  - RestoreResults
-                  type: string
-                name:
-                  description: Name is the name of the kubernetes resource with which
-                    the file is associated.
-                  type: string
-              required:
-              - kind
-              - name
-              type: object
-          required:
-          - target
-          type: object
-        status:
-          description: DownloadRequestStatus is the current status of a DownloadRequest.
-          properties:
-            downloadURL:
-              description: DownloadURL contains the pre-signed URL for the target
-                file.
-              type: string
-            expiration:
-              description: Expiration is when this DownloadRequest expires and can
-                be deleted by the system.
-              format: date-time
-              nullable: true
-              type: string
-            phase:
-              description: Phase is the current state of the DownloadRequest.
-              enum:
-              - New
-              - Processed
-              type: string
-          type: object
-      type: object
-  version: v1
-  versions:
-  - name: v1
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crd/v1beta1/bases/velero.io_podvolumebackups.yaml

@@ -1,162 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: v0.3.0
-  creationTimestamp: null
-  name: podvolumebackups.velero.io
-spec:
-  group: velero.io
-  names:
-    kind: PodVolumeBackup
-    listKind: PodVolumeBackupList
-    plural: podvolumebackups
-    singular: podvolumebackup
-  preserveUnknownFields: false
-  scope: Namespaced
-  validation:
-    openAPIV3Schema:
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        spec:
-          description: PodVolumeBackupSpec is the specification for a PodVolumeBackup.
-          properties:
-            backupStorageLocation:
-              description: BackupStorageLocation is the name of the backup storage
-                location where the restic repository is stored.
-              type: string
-            node:
-              description: Node is the name of the node that the Pod is running on.
-              type: string
-            pod:
-              description: Pod is a reference to the pod containing the volume to
-                be backed up.
-              properties:
-                apiVersion:
-                  description: API version of the referent.
-                  type: string
-                fieldPath:
-                  description: 'If referring to a piece of an object instead of an
-                    entire object, this string should contain a valid JSON/Go field
-                    access statement, such as desiredState.manifest.containers[2].
-                    For example, if the object reference is to a container within
-                    a pod, this would take on a value like: "spec.containers{name}"
-                    (where "name" refers to the name of the container that triggered
-                    the event) or if no container name is specified "spec.containers[2]"
-                    (container with index 2 in this pod). This syntax is chosen only
-                    to have some well-defined way of referencing a part of an object.
-                    TODO: this design is not final and this field is subject to change
-                    in the future.'
-                  type: string
-                kind:
-                  description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-                  type: string
-                name:
-                  description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
-                  type: string
-                namespace:
-                  description: 'Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
-                  type: string
-                resourceVersion:
-                  description: 'Specific resourceVersion to which this reference is
-                    made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
-                  type: string
-                uid:
-                  description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
-                  type: string
-              type: object
-            repoIdentifier:
-              description: RepoIdentifier is the restic repository identifier.
-              type: string
-            tags:
-              additionalProperties:
-                type: string
-              description: Tags are a map of key-value pairs that should be applied
-                to the volume backup as tags.
-              type: object
-            volume:
-              description: Volume is the name of the volume within the Pod to be backed
-                up.
-              type: string
-          required:
-          - backupStorageLocation
-          - node
-          - pod
-          - repoIdentifier
-          - volume
-          type: object
-        status:
-          description: PodVolumeBackupStatus is the current status of a PodVolumeBackup.
-          properties:
-            completionTimestamp:
-              description: CompletionTimestamp records the time a backup was completed.
-                Completion time is recorded even on failed backups. Completion time
-                is recorded before uploading the backup object. The server's time
-                is used for CompletionTimestamps
-              format: date-time
-              nullable: true
-              type: string
-            message:
-              description: Message is a message about the pod volume backup's status.
-              type: string
-            path:
-              description: Path is the full path within the controller pod being backed
-                up.
-              type: string
-            phase:
-              description: Phase is the current state of the PodVolumeBackup.
-              enum:
-              - New
-              - InProgress
-              - Completed
-              - Failed
-              type: string
-            progress:
-              description: Progress holds the total number of bytes of the volume
-                and the current number of backed up bytes. This can be used to display
-                progress information about the backup operation.
-              properties:
-                bytesDone:
-                  format: int64
-                  type: integer
-                totalBytes:
-                  format: int64
-                  type: integer
-              type: object
-            snapshotID:
-              description: SnapshotID is the identifier for the snapshot of the pod
-                volume.
-              type: string
-            startTimestamp:
-              description: StartTimestamp records the time a backup was started. Separate
-                from CreationTimestamp, since that value changes on restores. The
-                server's time is used for StartTimestamps
-              format: date-time
-              nullable: true
-              type: string
-          type: object
-      type: object
-  version: v1
-  versions:
-  - name: v1
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crd/v1beta1/bases/velero.io_podvolumerestores.yaml

@@ -1,145 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: v0.3.0
-  creationTimestamp: null
-  name: podvolumerestores.velero.io
-spec:
-  group: velero.io
-  names:
-    kind: PodVolumeRestore
-    listKind: PodVolumeRestoreList
-    plural: podvolumerestores
-    singular: podvolumerestore
-  preserveUnknownFields: false
-  scope: Namespaced
-  validation:
-    openAPIV3Schema:
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        spec:
-          description: PodVolumeRestoreSpec is the specification for a PodVolumeRestore.
-          properties:
-            backupStorageLocation:
-              description: BackupStorageLocation is the name of the backup storage
-                location where the restic repository is stored.
-              type: string
-            pod:
-              description: Pod is a reference to the pod containing the volume to
-                be restored.
-              properties:
-                apiVersion:
-                  description: API version of the referent.
-                  type: string
-                fieldPath:
-                  description: 'If referring to a piece of an object instead of an
-                    entire object, this string should contain a valid JSON/Go field
-                    access statement, such as desiredState.manifest.containers[2].
-                    For example, if the object reference is to a container within
-                    a pod, this would take on a value like: "spec.containers{name}"
-                    (where "name" refers to the name of the container that triggered
-                    the event) or if no container name is specified "spec.containers[2]"
-                    (container with index 2 in this pod). This syntax is chosen only
-                    to have some well-defined way of referencing a part of an object.
-                    TODO: this design is not final and this field is subject to change
-                    in the future.'
-                  type: string
-                kind:
-                  description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-                  type: string
-                name:
-                  description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
-                  type: string
-                namespace:
-                  description: 'Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
-                  type: string
-                resourceVersion:
-                  description: 'Specific resourceVersion to which this reference is
-                    made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
-                  type: string
-                uid:
-                  description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
-                  type: string
-              type: object
-            repoIdentifier:
-              description: RepoIdentifier is the restic repository identifier.
-              type: string
-            snapshotID:
-              description: SnapshotID is the ID of the volume snapshot to be restored.
-              type: string
-            volume:
-              description: Volume is the name of the volume within the Pod to be restored.
-              type: string
-          required:
-          - backupStorageLocation
-          - pod
-          - repoIdentifier
-          - snapshotID
-          - volume
-          type: object
-        status:
-          description: PodVolumeRestoreStatus is the current status of a PodVolumeRestore.
-          properties:
-            completionTimestamp:
-              description: CompletionTimestamp records the time a restore was completed.
-                Completion time is recorded even on failed restores. The server's
-                time is used for CompletionTimestamps
-              format: date-time
-              nullable: true
-              type: string
-            message:
-              description: Message is a message about the pod volume restore's status.
-              type: string
-            phase:
-              description: Phase is the current state of the PodVolumeRestore.
-              enum:
-              - New
-              - InProgress
-              - Completed
-              - Failed
-              type: string
-            progress:
-              description: Progress holds the total number of bytes of the snapshot
-                and the current number of restored bytes. This can be used to display
-                progress information about the restore operation.
-              properties:
-                bytesDone:
-                  format: int64
-                  type: integer
-                totalBytes:
-                  format: int64
-                  type: integer
-              type: object
-            startTimestamp:
-              description: StartTimestamp records the time a restore was started.
-                The server's time is used for StartTimestamps
-              format: date-time
-              nullable: true
-              type: string
-          type: object
-      type: object
-  version: v1
-  versions:
-  - name: v1
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crd/v1beta1/bases/velero.io_resticrepositories.yaml

@@ -1,89 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: v0.3.0
-  creationTimestamp: null
-  name: resticrepositories.velero.io
-spec:
-  group: velero.io
-  names:
-    kind: ResticRepository
-    listKind: ResticRepositoryList
-    plural: resticrepositories
-    singular: resticrepository
-  preserveUnknownFields: false
-  scope: Namespaced
-  validation:
-    openAPIV3Schema:
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        spec:
-          description: ResticRepositorySpec is the specification for a ResticRepository.
-          properties:
-            backupStorageLocation:
-              description: BackupStorageLocation is the name of the BackupStorageLocation
-                that should contain this repository.
-              type: string
-            maintenanceFrequency:
-              description: MaintenanceFrequency is how often maintenance should be
-                run.
-              type: string
-            resticIdentifier:
-              description: ResticIdentifier is the full restic-compatible string for
-                identifying this repository.
-              type: string
-            volumeNamespace:
-              description: VolumeNamespace is the namespace this restic repository
-                contains pod volume backups for.
-              type: string
-          required:
-          - backupStorageLocation
-          - maintenanceFrequency
-          - resticIdentifier
-          - volumeNamespace
-          type: object
-        status:
-          description: ResticRepositoryStatus is the current status of a ResticRepository.
-          properties:
-            lastMaintenanceTime:
-              description: LastMaintenanceTime is the last time maintenance was run.
-              format: date-time
-              nullable: true
-              type: string
-            message:
-              description: Message is a message about the current status of the ResticRepository.
-              type: string
-            phase:
-              description: Phase is the current state of the ResticRepository.
-              enum:
-              - New
-              - Ready
-              - NotReady
-              type: string
-          type: object
-      type: object
-  version: v1
-  versions:
-  - name: v1
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crd/v1beta1/bases/velero.io_schedules.yaml

@@ -1,394 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: v0.3.0
-  creationTimestamp: null
-  name: schedules.velero.io
-spec:
-  group: velero.io
-  names:
-    kind: Schedule
-    listKind: ScheduleList
-    plural: schedules
-    singular: schedule
-  preserveUnknownFields: false
-  scope: Namespaced
-  validation:
-    openAPIV3Schema:
-      description: Schedule is a Velero resource that represents a pre-scheduled or
-        periodic Backup that should be run.
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        spec:
-          description: ScheduleSpec defines the specification for a Velero schedule
-          properties:
-            schedule:
-              description: Schedule is a Cron expression defining when to run the
-                Backup.
-              type: string
-            template:
-              description: Template is the definition of the Backup to be run on the
-                provided schedule
-              properties:
-                defaultVolumesToRestic:
-                  description: DefaultVolumesToRestic specifies whether restic should
-                    be used to take a backup of all pod volumes by default.
-                  type: boolean
-                excludedNamespaces:
-                  description: ExcludedNamespaces contains a list of namespaces that
-                    are not included in the backup.
-                  items:
-                    type: string
-                  nullable: true
-                  type: array
-                excludedResources:
-                  description: ExcludedResources is a slice of resource names that
-                    are not included in the backup.
-                  items:
-                    type: string
-                  nullable: true
-                  type: array
-                hooks:
-                  description: Hooks represent custom behaviors that should be executed
-                    at different phases of the backup.
-                  properties:
-                    resources:
-                      description: Resources are hooks that should be executed when
-                        backing up individual instances of a resource.
-                      items:
-                        description: BackupResourceHookSpec defines one or more BackupResourceHooks
-                          that should be executed based on the rules defined for namespaces,
-                          resources, and label selector.
-                        properties:
-                          excludedNamespaces:
-                            description: ExcludedNamespaces specifies the namespaces
-                              to which this hook spec does not apply.
-                            items:
-                              type: string
-                            nullable: true
-                            type: array
-                          excludedResources:
-                            description: ExcludedResources specifies the resources
-                              to which this hook spec does not apply.
-                            items:
-                              type: string
-                            nullable: true
-                            type: array
-                          includedNamespaces:
-                            description: IncludedNamespaces specifies the namespaces
-                              to which this hook spec applies. If empty, it applies
-                              to all namespaces.
-                            items:
-                              type: string
-                            nullable: true
-                            type: array
-                          includedResources:
-                            description: IncludedResources specifies the resources
-                              to which this hook spec applies. If empty, it applies
-                              to all resources.
-                            items:
-                              type: string
-                            nullable: true
-                            type: array
-                          labelSelector:
-                            description: LabelSelector, if specified, filters the
-                              resources to which this hook spec applies.
-                            nullable: true
-                            properties:
-                              matchExpressions:
-                                description: matchExpressions is a list of label selector
-                                  requirements. The requirements are ANDed.
-                                items:
-                                  description: A label selector requirement is a selector
-                                    that contains values, a key, and an operator that
-                                    relates the key and values.
-                                  properties:
-                                    key:
-                                      description: key is the label key that the selector
-                                        applies to.
-                                      type: string
-                                    operator:
-                                      description: operator represents a key's relationship
-                                        to a set of values. Valid operators are In,
-                                        NotIn, Exists and DoesNotExist.
-                                      type: string
-                                    values:
-                                      description: values is an array of string values.
-                                        If the operator is In or NotIn, the values
-                                        array must be non-empty. If the operator is
-                                        Exists or DoesNotExist, the values array must
-                                        be empty. This array is replaced during a
-                                        strategic merge patch.
-                                      items:
-                                        type: string
-                                      type: array
-                                  required:
-                                  - key
-                                  - operator
-                                  type: object
-                                type: array
-                              matchLabels:
-                                additionalProperties:
-                                  type: string
-                                description: matchLabels is a map of {key,value} pairs.
-                                  A single {key,value} in the matchLabels map is equivalent
-                                  to an element of matchExpressions, whose key field
-                                  is "key", the operator is "In", and the values array
-                                  contains only "value". The requirements are ANDed.
-                                type: object
-                            type: object
-                          name:
-                            description: Name is the name of this hook.
-                            type: string
-                          post:
-                            description: PostHooks is a list of BackupResourceHooks
-                              to execute after storing the item in the backup. These
-                              are executed after all "additional items" from item
-                              actions are processed.
-                            items:
-                              description: BackupResourceHook defines a hook for a
-                                resource.
-                              properties:
-                                exec:
-                                  description: Exec defines an exec hook.
-                                  properties:
-                                    command:
-                                      description: Command is the command and arguments
-                                        to execute.
-                                      items:
-                                        type: string
-                                      minItems: 1
-                                      type: array
-                                    container:
-                                      description: Container is the container in the
-                                        pod where the command should be executed.
-                                        If not specified, the pod's first container
-                                        is used.
-                                      type: string
-                                    onError:
-                                      description: OnError specifies how Velero should
-                                        behave if it encounters an error executing
-                                        this hook.
-                                      enum:
-                                      - Continue
-                                      - Fail
-                                      type: string
-                                    timeout:
-                                      description: Timeout defines the maximum amount
-                                        of time Velero should wait for the hook to
-                                        complete before considering the execution
-                                        a failure.
-                                      type: string
-                                  required:
-                                  - command
-                                  type: object
-                              required:
-                              - exec
-                              type: object
-                            type: array
-                          pre:
-                            description: PreHooks is a list of BackupResourceHooks
-                              to execute prior to storing the item in the backup.
-                              These are executed before any "additional items" from
-                              item actions are processed.
-                            items:
-                              description: BackupResourceHook defines a hook for a
-                                resource.
-                              properties:
-                                exec:
-                                  description: Exec defines an exec hook.
-                                  properties:
-                                    command:
-                                      description: Command is the command and arguments
-                                        to execute.
-                                      items:
-                                        type: string
-                                      minItems: 1
-                                      type: array
-                                    container:
-                                      description: Container is the container in the
-                                        pod where the command should be executed.
-                                        If not specified, the pod's first container
-                                        is used.
-                                      type: string
-                                    onError:
-                                      description: OnError specifies how Velero should
-                                        behave if it encounters an error executing
-                                        this hook.
-                                      enum:
-                                      - Continue
-                                      - Fail
-                                      type: string
-                                    timeout:
-                                      description: Timeout defines the maximum amount
-                                        of time Velero should wait for the hook to
-                                        complete before considering the execution
-                                        a failure.
-                                      type: string
-                                  required:
-                                  - command
-                                  type: object
-                              required:
-                              - exec
-                              type: object
-                            type: array
-                        required:
-                        - name
-                        type: object
-                      nullable: true
-                      type: array
-                  type: object
-                includeClusterResources:
-                  description: IncludeClusterResources specifies whether cluster-scoped
-                    resources should be included for consideration in the backup.
-                  nullable: true
-                  type: boolean
-                includedNamespaces:
-                  description: IncludedNamespaces is a slice of namespace names to
-                    include objects from. If empty, all namespaces are included.
-                  items:
-                    type: string
-                  nullable: true
-                  type: array
-                includedResources:
-                  description: IncludedResources is a slice of resource names to include
-                    in the backup. If empty, all resources are included.
-                  items:
-                    type: string
-                  nullable: true
-                  type: array
-                labelSelector:
-                  description: LabelSelector is a metav1.LabelSelector to filter with
-                    when adding individual objects to the backup. If empty or nil,
-                    all objects are included. Optional.
-                  nullable: true
-                  properties:
-                    matchExpressions:
-                      description: matchExpressions is a list of label selector requirements.
-                        The requirements are ANDed.
-                      items:
-                        description: A label selector requirement is a selector that
-                          contains values, a key, and an operator that relates the
-                          key and values.
-                        properties:
-                          key:
-                            description: key is the label key that the selector applies
-                              to.
-                            type: string
-                          operator:
-                            description: operator represents a key's relationship
-                              to a set of values. Valid operators are In, NotIn, Exists
-                              and DoesNotExist.
-                            type: string
-                          values:
-                            description: values is an array of string values. If the
-                              operator is In or NotIn, the values array must be non-empty.
-                              If the operator is Exists or DoesNotExist, the values
-                              array must be empty. This array is replaced during a
-                              strategic merge patch.
-                            items:
-                              type: string
-                            type: array
-                        required:
-                        - key
-                        - operator
-                        type: object
-                      type: array
-                    matchLabels:
-                      additionalProperties:
-                        type: string
-                      description: matchLabels is a map of {key,value} pairs. A single
-                        {key,value} in the matchLabels map is equivalent to an element
-                        of matchExpressions, whose key field is "key", the operator
-                        is "In", and the values array contains only "value". The requirements
-                        are ANDed.
-                      type: object
-                  type: object
-                orderedResources:
-                  additionalProperties:
-                    type: string
-                  description: OrderedResources specifies the backup order of resources
-                    of specific Kind. The map key is the Kind name and value is a
-                    list of resource names separated by commas. Each resource name
-                    has format "namespace/resourcename".  For cluster resources, simply
-                    use "resourcename".
-                  nullable: true
-                  type: object
-                snapshotVolumes:
-                  description: SnapshotVolumes specifies whether to take cloud snapshots
-                    of any PV's referenced in the set of objects included in the Backup.
-                  nullable: true
-                  type: boolean
-                storageLocation:
-                  description: StorageLocation is a string containing the name of
-                    a BackupStorageLocation where the backup should be stored.
-                  type: string
-                ttl:
-                  description: TTL is a time.Duration-parseable string describing
-                    how long the Backup should be retained for.
-                  type: string
-                volumeSnapshotLocations:
-                  description: VolumeSnapshotLocations is a list containing names
-                    of VolumeSnapshotLocations associated with this backup.
-                  items:
-                    type: string
-                  type: array
-              type: object
-            useOwnerReferencesInBackup:
-              description: UseOwnerReferencesBackup specifies whether to use OwnerReferences
-                on backups created by this Schedule.
-              nullable: true
-              type: boolean
-          required:
-          - schedule
-          - template
-          type: object
-        status:
-          description: ScheduleStatus captures the current state of a Velero schedule
-          properties:
-            lastBackup:
-              description: LastBackup is the last time a Backup was run for this Schedule
-                schedule
-              format: date-time
-              nullable: true
-              type: string
-            phase:
-              description: Phase is the current phase of the Schedule
-              enum:
-              - New
-              - Enabled
-              - FailedValidation
-              type: string
-            validationErrors:
-              description: ValidationErrors is a slice of all validation errors (if
-                applicable)
-              items:
-                type: string
-              type: array
-          type: object
-      type: object
-  version: v1
-  versions:
-  - name: v1
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crd/v1beta1/bases/velero.io_serverstatusrequests.yaml

@@ -1,89 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: v0.3.0
-  creationTimestamp: null
-  name: serverstatusrequests.velero.io
-spec:
-  group: velero.io
-  names:
-    kind: ServerStatusRequest
-    listKind: ServerStatusRequestList
-    plural: serverstatusrequests
-    shortNames:
-    - ssr
-    singular: serverstatusrequest
-  preserveUnknownFields: false
-  scope: Namespaced
-  subresources:
-    status: {}
-  validation:
-    openAPIV3Schema:
-      description: ServerStatusRequest is a request to access current status information
-        about the Velero server.
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        spec:
-          description: ServerStatusRequestSpec is the specification for a ServerStatusRequest.
-          type: object
-        status:
-          description: ServerStatusRequestStatus is the current status of a ServerStatusRequest.
-          properties:
-            phase:
-              description: Phase is the current lifecycle phase of the ServerStatusRequest.
-              enum:
-              - New
-              - Processed
-              type: string
-            plugins:
-              description: Plugins list information about the plugins running on the
-                Velero server
-              items:
-                description: PluginInfo contains attributes of a Velero plugin
-                properties:
-                  kind:
-                    type: string
-                  name:
-                    type: string
-                required:
-                - kind
-                - name
-                type: object
-              nullable: true
-              type: array
-            processedTimestamp:
-              description: ProcessedTimestamp is when the ServerStatusRequest was
-                processed by the ServerStatusRequestController.
-              format: date-time
-              nullable: true
-              type: string
-            serverVersion:
-              description: ServerVersion is the Velero server version.
-              type: string
-          type: object
-      type: object
-  version: v1
-  versions:
-  - name: v1
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crd/v1beta1/bases/velero.io_volumesnapshotlocations.yaml

@@ -1,74 +0,0 @@
-
----
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
-  annotations:
-    controller-gen.kubebuilder.io/version: v0.3.0
-  creationTimestamp: null
-  name: volumesnapshotlocations.velero.io
-spec:
-  group: velero.io
-  names:
-    kind: VolumeSnapshotLocation
-    listKind: VolumeSnapshotLocationList
-    plural: volumesnapshotlocations
-    singular: volumesnapshotlocation
-  preserveUnknownFields: false
-  scope: Namespaced
-  validation:
-    openAPIV3Schema:
-      description: VolumeSnapshotLocation is a location where Velero stores volume
-        snapshots.
-      properties:
-        apiVersion:
-          description: 'APIVersion defines the versioned schema of this representation
-            of an object. Servers should convert recognized schemas to the latest
-            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
-          type: string
-        kind:
-          description: 'Kind is a string value representing the REST resource this
-            object represents. Servers may infer this from the endpoint the client
-            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
-          type: string
-        metadata:
-          type: object
-        spec:
-          description: VolumeSnapshotLocationSpec defines the specification for a
-            Velero VolumeSnapshotLocation.
-          properties:
-            config:
-              additionalProperties:
-                type: string
-              description: Config is for provider-specific configuration fields.
-              type: object
-            provider:
-              description: Provider is the provider of the volume storage.
-              type: string
-          required:
-          - provider
-          type: object
-        status:
-          description: VolumeSnapshotLocationStatus describes the current status of
-            a Velero VolumeSnapshotLocation.
-          properties:
-            phase:
-              description: VolumeSnapshotLocationPhase is the lifecycle phase of a
-                Velero VolumeSnapshotLocation.
-              enum:
-              - Available
-              - Unavailable
-              type: string
-          type: object
-      type: object
-  version: v1
-  versions:
-  - name: v1
-    served: true
-    storage: true
-status:
-  acceptedNames:
-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []

config/crd/v1beta1/crds/doc.go

@@ -1,4 +0,0 @@
-// Package crds embeds the controller-tools generated CRD manifests
-package crds
-
-//go:generate go run ../../../../hack/crd-gen/v1beta1/main.go

config/rbac/role.yaml

@@ -4,8 +4,53 @@ apiVersion: rbac.authorization.k8s.io/v1
 kind: ClusterRole
 metadata:
   creationTimestamp: null
-  name: manager-role
+  name: velero-perms
 rules:
+- apiGroups:
+  - ""
+  resources:
+  - persistentvolumerclaims
+  verbs:
+  - get
+- apiGroups:
+  - ""
+  resources:
+  - persistentvolumes
+  verbs:
+  - get
+- apiGroups:
+  - ""
+  resources:
+  - pods
+  verbs:
+  - get
+- apiGroups:
+  - velero.io
+  resources:
+  - backuprepositories
+  verbs:
+  - create
+  - delete
+  - get
+  - list
+  - patch
+  - update
+  - watch
+- apiGroups:
+  - velero.io
+  resources:
+  - backuprepositories/status
+  verbs:
+  - get
+  - patch
+  - update
+- apiGroups:
+  - velero.io
+  resources:
+  - backups
+  verbs:
+  - create
+  - delete
 - apiGroups:
   - velero.io
   resources:
@@ -26,6 +71,26 @@ rules:
   - get
   - patch
   - update
+- apiGroups:
+  - velero.io
+  resources:
+  - deletebackuprequests
+  verbs:
+  - create
+  - delete
+  - get
+  - list
+  - patch
+  - update
+  - watch
+- apiGroups:
+  - velero.io
+  resources:
+  - deletebackuprequests/status
+  verbs:
+  - get
+  - patch
+  - update
 - apiGroups:
   - velero.io
   resources:
@@ -46,6 +111,66 @@ rules:
   - get
   - patch
   - update
+- apiGroups:
+  - velero.io
+  resources:
+  - podvolumebackups
+  verbs:
+  - create
+  - delete
+  - get
+  - list
+  - patch
+  - update
+  - watch
+- apiGroups:
+  - velero.io
+  resources:
+  - podvolumebackups/status
+  verbs:
+  - get
+  - patch
+  - update
+- apiGroups:
+  - velero.io
+  resources:
+  - podvolumerestores
+  verbs:
+  - create
+  - delete
+  - get
+  - list
+  - patch
+  - update
+  - watch
+- apiGroups:
+  - velero.io
+  resources:
+  - podvolumerestores/status
+  verbs:
+  - get
+  - patch
+  - update
+- apiGroups:
+  - velero.io
+  resources:
+  - schedules
+  verbs:
+  - create
+  - delete
+  - get
+  - list
+  - patch
+  - update
+  - watch
+- apiGroups:
+  - velero.io
+  resources:
+  - schedules/status
+  verbs:
+  - get
+  - patch
+  - update
 - apiGroups:
   - velero.io
   resources:
@@ -66,3 +191,15 @@ rules:
   - get
   - patch
   - update
+- apiGroups:
+  - velero.io
+  resources:
+  - volumesnapshotlocations
+  verbs:
+  - create
+  - delete
+  - get
+  - list
+  - patch
+  - update
+  - watch

design/2082-bsl-delete-associated-resources_design.md

@@ -0,0 +1,40 @@
+# Delete Backup and Restic Repo Resources when BSL is Deleted
+
+## Abstract
+
+Issue #2082 requested that with the command `velero backup-location delete <bsl name>` (implemented in Velero 1.6 with #3073), the following will be deleted:
+
+- associated Velero backups (to be clear, these are custom Kubernetes resources called "backups" that are stored in the API server)
+- associated Restic repositories (custom Kubernetes resources called "resticrepositories")
+
+This design doc explains how the request will be implemented.
+
+## Background
+
+When a BSL resource is deleted from its Velero namespace, the associated custom Kubernetes resources, backups and Restic repositories, can no longer be used.
+It makes sense to clean those resources up when a BSL is deleted.
+
+## Goals
+
+Update the `velero backup-location delete <bsl name>` command to delete associated backup and Restic repository resources in the same Velero namespace.
+
+## Non Goals
+
+[It was suggested](https://github.com/vmware-tanzu/velero/issues/2082#issuecomment-827951311) to fix bug #2697 alongside this issue.
+However, I think that should be fixed separately because although it is similar (restore objects are not being deleted), it is also quite different.
+One is adding a command feature update (this issue) and the other is a bug fix and each affect different parts of the code base.
+
+## High-Level Design
+
+Update the `velero backup-location delete <bsl name>` command to do the following:
+
+- find in the same Velero namespace from which the BSL was deleted the associated backup resources and Restic repositories, called "backups.velero.io" and "resticrepositories.velero.io" respectively
+- delete the resources found
+
+The above logic will be added to [where BSLs are deleted](https://github.com/vmware-tanzu/velero/blob/main/pkg/cmd/cli/backuplocation/delete.go).
+
+## Alternative Considered
+
+I had considered deleting the backup files (the ones in json format and tarballs) in the BSL itself.
+However, a standard use case is to back up a cluster and then restore into a new cluster.
+Deleting the backup storage location in either location is not expected to remove all of the backups in the backup storage location and should not be done.

design/CLI/PoC/base/CRDs.yaml

@@ -505,6 +505,8 @@ spec:
                       - BackupResourceList
                       - RestoreLog
                       - RestoreResults
+                      - CSIBackupVolumeSnapshots
+                      - CSIBackupVolumeSnapshotContents
                   type: string
                 name:
                   description: Name is the name of the kubernetes resource with

design/CLI/PoC/overlays/plugins/node-agent.yaml

@@ -5,22 +5,22 @@ metadata:
   creationTimestamp: null
   labels:
     component: velero
-  name: restic
+  name: node-agent
   namespace: velero
 spec:
   selector:
     matchLabels:
-      name: restic
+      name: node-agent
   template:
     metadata:
       creationTimestamp: null
       labels:
         component: velero
-        name: restic
+        name: node-agent
     spec:
       containers:
         - args:
-            - restic
+            - node-agent
             - server
           command:
             - /velero
@@ -43,7 +43,7 @@ spec:
               value: /credentials/cloud
           image: velero/velero:latest
           imagePullPolicy: Always
-          name: restic
+          name: node-agent
           resources: {}
           volumeMounts:
             - mountPath: /host_pods

design/Implemented/backup-resources-order.md

@@ -2,7 +2,7 @@
 This document proposes a solution that allows user to specify a backup order for resources of specific resource type.
 
 ## Background
-During backup process, user may need to back up resources of specific type in some specific order to ensure the resources were backup properly because these resources are related and ordering might be required to preserve the consistency for the apps to recover itself <EFBFBD>from the backup image 
+During backup process, user may need to back up resources of specific type in some specific order to ensure the resources were backup properly because these resources are related and ordering might be required to preserve the consistency for the apps to recover itself from the backup image 
 (Ex: primary-secondary database pods in a cluster).
 
 ## Goals
@@ -12,7 +12,7 @@ During backup process, user may need to back up resources of specific type in so
 - Use a plugin to backup an resources and all the sub resources.  For example use a plugin for StatefulSet and backup pods belong to the StatefulSet in specific order.  This plugin solution is not generic and requires plugin for each resource type.
 
 ## High-Level Design
-User will specify a map of resource type to list resource names (separate by semicolons).  Each name will be in the format "namespaceName/resourceName" to enable ordering accross namespaces.  Based on this map, the resources of each resource type will be sorted by the order specified in the list of resources.  If a resource instance belong to that specific type but its name is not in the order list, then it will be put behind other resources that are in the list.
+User will specify a map of resource type to list resource names (separate by semicolons).  Each name will be in the format "namespaceName/resourceName" to enable ordering across namespaces.  Based on this map, the resources of each resource type will be sorted by the order specified in the list of resources.  If a resource instance belong to that specific type but its name is not in the order list, then it will be put behind other resources that are in the list.
 
 ### Changes to BackupSpec
 Add new field to BackupSpec
@@ -36,5 +36,5 @@ Example:
 >velero backup create mybackup --ordered-resources "pod=ns1/pod1,ns1/pod2;persistentvolumeclaim=n2/slavepod,ns2/primarypod"
 
 ## Open Issues
-- In the CLI, the design proposes to use commas to separate items of a resource type and semicolon to separate key-value pairs.  This follows the convention of using commas to separate items in a list (For example: --include-namespaces ns1,ns2).  However, the syntax for map in labels and annotations use commas to seperate key-value pairs.  So it introduces some inconsistency.
+- In the CLI, the design proposes to use commas to separate items of a resource type and semicolon to separate key-value pairs.  This follows the convention of using commas to separate items in a list (For example: --include-namespaces ns1,ns2).  However, the syntax for map in labels and annotations use commas to separate key-value pairs.  So it introduces some inconsistency.
 - For pods that managed by Deployment or DaemonSet, this design may not work because the pods' name is randomly generated and if pods are restarted, they would have different names so the Backup operation may not consider the restarted pods in the sorting algorithm.  This problem will be addressed when we enhance the design to use regular expression to specify the OrderResources instead of exact match. 

design/Implemented/generating-velero-crds-with-structural-schema.md

@@ -28,7 +28,7 @@ This document proposes adding _controller-tools_ to the project to automatically
 _controller-tools_ works by reading the Go files that contain the API type definitions.
 It uses a combination of the struct fields, types, tags and comments to build the OpenAPIv3 schema for the CRDs. The tooling makes some assumptions based on conventions followed in upstream Kubernetes and the ecosystem, which involves some changes to the Velero API type definitions, especially around optional fields.
 
-In order for _controller-tools_ to read the Go files containing Velero API type defintiions, the CRDs need to be generated at build time, as these files are not available at runtime (i.e. the Go files are not accessible by the compiled binary).
+In order for _controller-tools_ to read the Go files containing Velero API type definitions, the CRDs need to be generated at build time, as these files are not available at runtime (i.e. the Go files are not accessible by the compiled binary).
 These generated CRD manifests (YAML) will then need to be available to the `pkg/install` package for it to include when installing Velero resources.
 
 ## Detailed Design

design/Implemented/plugin-backup-and-restore-progress-design.md

@@ -429,7 +429,7 @@ Instead, a new method for 'Progress' will be added to interface. Velero server r
 
 But, this involves good amount of changes and needs a way for backward compatibility.
 
-As volume plugins are mostly K8s native, its fine to go ahead with current limiation.
+As volume plugins are mostly K8s native, its fine to go ahead with current limitation.
 
 ### Update Backup CR
 Instead of creating new CRs, plugins can directly update the status of Backup CR. But, this deviates from current approach of having separate CRs like PodVolumeBackup/PodVolumeRestore to know operations progress.

design/Implemented/restore-with-EnableAPIGroupVersions-feature.md

@@ -1,6 +1,6 @@
 # Restore API Group Version by Priority Level When EnableAPIGroupVersions Feature is Set
 
-Status: Draft
+Status: Accepted
 
 ## Abstract
 

design/Implemented/velero-debug.md

@@ -0,0 +1,122 @@
+# `velero debug` command for gathering troubleshooting information
+
+## Abstract
+To simplify the communication between velero users and developers, this document proposes the `velero debug` command to generate a tarball including the logs needed for debugging.
+
+Github issue: https://github.com/vmware-tanzu/velero/issues/675
+
+## Background
+Gathering information to troubleshoot a Velero deployment is currently spread across multiple commands, and is not very efficient. Logs for the Velero server itself are accessed via a kubectl logs command, while information on specific backups or restores are accessed via a Velero subcommand. Restic logs are even more complicated to retrieve, since one must gather logs for every instance of the daemonset, and there’s currently no good mechanism to locate which node a particular restic backup ran against.  
+A dedicated subcommand can lower this effort and reduce back-and-forth between user and developer for collecting the logs.
+
+
+## Goals
+- Enable efficient log collection for Velero and associated components, like plugins and restic.
+
+## Non Goals
+- Collecting logs for components that do not belong to velero such as storage service.
+- Automated log analysis.
+
+## High-Level Design
+With the introduction of the new command `velero debug`, the command would download all of the following information:
+- velero deployment logs
+- restic DaemonSet logs
+- plugin logs 
+- All the resources in the group `velero.io` that are created such as:
+    - Backup
+    - Restore
+    - BackupStorageLocation
+    - PodVolumeBackup
+    - PodVolumeRestore
+    - *etc ...*
+- Log of the backup and restore, if specified in the param
+
+A project called `crash-diagnostics` (or `crashd`) (https://github.com/vmware-tanzu/crash-diagnostics)  implements the Kubernetes API queries and provides Starlark scripting language to abstract details, and collect the information into a local copy.   It can be used as a standalone CLI executing a Starlark script file.
+With the capabilities of embedding files in Go 1.16, we can define a Starlark script gathering the necessary information, embed the script at build time, then the velero debug command will invoke `crashd`, passing in the script’s text contents.
+
+## Detailed Design
+### Triggering the script
+The Starlark script to be called by crashd:
+
+```python
+def capture_backup_logs(cmd, namespace):
+    if args.backup:
+        log("Collecting log and information for backup: {}".format(args.backup))
+        backupDescCmd = "{} --namespace={} backup describe {} --details".format(cmd, namespace, args.backup)
+        capture_local(cmd=backupDescCmd, file_name="backup_describe_{}.txt".format(args.backup))
+        backupLogsCmd = "{} --namespace={} backup logs {}".format(cmd, namespace, args.backup)
+        capture_local(cmd=backupLogsCmd, file_name="backup_{}.log".format(args.backup))
+def capture_restore_logs(cmd, namespace):
+    if args.restore:
+        log("Collecting log and information for restore: {}".format(args.restore))
+        restoreDescCmd = "{} --namespace={} restore describe {} --details".format(cmd, namespace, args.restore)
+        capture_local(cmd=restoreDescCmd, file_name="restore_describe_{}.txt".format(args.restore))
+        restoreLogsCmd = "{} --namespace={} restore logs {}".format(cmd, namespace, args.restore)
+        capture_local(cmd=restoreLogsCmd, file_name="restore_{}.log".format(args.restore))
+
+ns = args.namespace if args.namespace else "velero"
+output = args.output if args.output else "bundle.tar.gz"
+cmd = args.cmd if args.cmd else "velero"
+# Working dir for writing during script execution
+crshd = crashd_config(workdir="./velero-bundle")
+set_defaults(kube_config(path=args.kubeconfig, cluster_context=args.kubecontext))
+log("Collecting velero resources in namespace: {}". format(ns))
+kube_capture(what="objects", namespaces=[ns], groups=['velero.io'])
+capture_local(cmd="{} version -n {}".format(cmd, ns), file_name="version.txt")
+log("Collecting velero deployment logs in namespace: {}". format(ns))
+kube_capture(what="logs", namespaces=[ns])
+capture_backup_logs(cmd, ns)
+capture_restore_logs(cmd, ns)
+archive(output_file=output, source_paths=[crshd.workdir])
+log("Generated debug information bundle: {}".format(output))
+```
+The sample command to trigger the script via crashd:
+```shell
+./crashd run ./velero.cshd --args 
+'backup=harbor-backup-2nd,namespace=velero,basedir=,restore=,kubeconfig=/home/.kube/minikube-250-224/config,output='
+```
+To trigger the script in `velero debug`, in the package `pkg/cmd/cli/debug` a struct `option` will be introduced
+```go
+type option struct {
+	// currCmd the velero command
+	currCmd string
+	// workdir for crashd will be $baseDir/velero-debug
+	baseDir string
+	// the namespace where velero server is installed
+	namespace string
+	// the absolute path for the log bundle to be generated
+	outputPath string
+	// the absolute path for the kubeconfig file that will be read by crashd for calling K8S API
+	kubeconfigPath string
+	// the kubecontext to be used for calling K8S API
+	kubeContext string
+	// optional, the name of the backup resource whose log will be packaged into the debug bundle
+	backup string
+	// optional, the name of the restore resource whose log will be packaged into the debug bundle
+	restore string
+	// optional, it controls whether to print the debug log messages when calling crashd
+	verbose bool
+}
+```
+The code will consolidate the input parameters and execution context of the `velero` CLI to form the option struct, which can be transformed into the `argsMap` that can be used when calling the func `exec.Execute` in `crashd`:
+https://github.com/vmware-tanzu/crash-diagnostics/blob/v0.3.4/exec/executor.go#L17
+
+## Alternatives Considered
+The collection could be done via the kubernetes client-go API, but such integration is not necessarily trivial to implement, therefore, `crashd` is preferred approach
+
+## Security Considerations
+- The starlark script will be embedded into the velero binary, and the byte slice will be passed to the `exec.Execute` func directly, so there’s little risk that the script will be modified before being executed.
+
+## Compatibility
+As the `crashd` project evolves the behavior of the internal functions used in the Starlark script may change.  We’ll ensure the correctness of the script via regular E2E tests.
+
+
+## Implementation
+1. Bump up to use Go v1.16 to compile velero
+2. Embed the starlark script
+3. Implement the `velero debug` sub-command to call the script
+4. Add E2E test case
+
+## Open Questions
+- **Command dependencies:** In the Starlark script, for collecting version info and backup logs, it calls the `velero backup logs` and `velero version`, which makes the call stack like velero debug -> crashd -> velero xxx.  We need to make sure this works under different PATH settings.
+- **Progress and error handling:** The log collection may take a relatively long time, log messages should be printed to indicate the progress when different items are being downloaded and packaged.  Additionally, when an error happens, `crashd` may omit some errors, so before the script is executed we'll do some validation and make sure the `debug` command fail early if some parameters are incorrect.

design/existing-resource-policy_design.md

@@ -0,0 +1,262 @@
+# Add support for `ExistingResourcePolicy` to restore API
+## Abstract
+Velero currently does not support any restore policy on kubernetes resources that are already present in-cluster. Velero skips over the restore of the resource if it already exists in the namespace/cluster irrespective of whether the resource present in the restore is the same or different from the one present on the cluster. It is desired that Velero gives the option to the user to decide whether or not the resource in backup should overwrite the one present in the cluster.
+
+## Background
+As of Today, Velero will skip over the restoration of resources that already exist in the cluster. The current workflow followed by Velero is (Using a `service` that is backed up for example):
+- Velero tries to attempt restore of the `service` 
+- Fetches the `service` from the cluster 
+- If the `service` exists then:
+    - Checks whether the `service` instance in the cluster is equal to the `service`  instance present in backup
+        - If not equal then skips the restore of the `service` and adds a restore warning (except for [ServiceAccount objects](https://github.com/vmware-tanzu/velero/blob/574baeb3c920f97b47985ec3957debdc70bcd5f8/pkg/restore/restore.go#L1246))
+        - If equal then skips the restore of the `service` and mentions that the restore of resource `service` is skipped in logs
+
+It is desired to add the functionality to specify whether or not to overwrite the instance of resource `service` in cluster with the one present in backup during the restore process.
+
+Related issue: https://github.com/vmware-tanzu/velero/issues/4066
+
+## Goals
+- Add support for `ExistingResourcePolicy` to restore API for Kubernetes resources.
+
+## Non Goals
+- Change existing restore workflow for `ServiceAccount` objects
+- Add support for `ExistingResourcePolicy` as `recreate` for Kubernetes resources. (Future scope feature)
+
+## Unrelated Proposals (Completely different functionalities than the one proposed in the design)
+- Add support for `ExistingResourcePolicy` to restore API for Non-Kubernetes resources.
+- Add support for `ExistingResourcePolicy` to restore API for `PersistentVolume` data.
+
+### Use-cases/Scenarios
+
+### A. Production Cluster - Backup Cluster:
+Let's say you have a Backup Cluster which is identical to the Production Cluster. After some operations/usage/time the Production Cluster had changed itself, there might be new deployments, some secrets might have been updated. Now, this means that the Backup cluster will no longer be identical to the Production Cluster. In order to keep the Backup Cluster up to date/identical to the Production Cluster with respect to Kubernetes resources except PV data we would like to use Velero for scheduling new backups which would in turn help us update the Backup Cluster via Velero restore.
+
+Reference: https://github.com/vmware-tanzu/velero/issues/4066#issuecomment-954320686
+
+### B. Help identify resource delta:
+Here delta resources mean the resources restored by a previous backup, but they are no longer in the latest backup. Let's follow a sequence of steps to understand this scenario:
+- Consider there are 2 clusters, Cluster A, which has 3 resources - P1, P2 and P3.
+- Create a Backup1 from Cluster A which has P1, P2 and P3.
+- Perform restore on a new Cluster B using Backup1.
+- Now, Lets say in Cluster A resource P1 gets deleted and resource P2 gets updated.
+- Create a new Backup2 with the new state of Cluster A, keep in mind Backup1 has P1, P2 and P3 while Backup2 has P2' and P3.
+- So the Delta here is (|Cluster B - Backup2|), Delete P1 and Update P2.
+- During Restore time we would want the Restore to help us identify this resource delta.
+
+Reference: https://github.com/vmware-tanzu/velero/pull/4613#issuecomment-1027260446
+
+## High-Level Design
+### Approach 1: Add a new spec field `existingResourcePolicy` to the Restore API
+In this approach we do *not* change existing velero behavior. If the resource to restore in cluster is equal to the one backed up then do nothing following current Velero behavior. For resources that already exist in the cluster that are not equal to the resource in the backup (other than Service Accounts). We add a new optional spec field `existingResourcePolicy` which can have the following values:
+1. `none`: This is the existing behavior, if Velero encounters a resource that already exists in the cluster, we simply
+skip restoration.
+2. `update`: This option would provide the following behavior.
+   - Unchanged resources: Velero would update the backup/restore labels on the unchanged resources, if labels patch fails Velero adds a restore error.
+   - Changed resources: Velero will first try to patch the changed resource, Now if the patch:
+       - succeeds: Then the in-cluster resource gets updated with the labels as well as the resource diff
+       - fails: Velero adds a restore warning and tries to just update the backup/restore labels on the resource, if the labels patch also fails then we add restore error.
+3. `recreate`:  If resource already exists, then Velero will delete it and recreate the resource.
+
+*Note:* The `recreate` option is a non-goal for this enhancement proposal, but it is considered as a future scope.
+Another thing to highlight is that Velero will not be deleting any resources in any of the policy options proposed in 
+this design but Velero will patch the resources in `update` policy option.
+
+Example:
+A. The following Restore will execute the `existingResourcePolicy` restore type `none` for the `services` and `deployments` present in the `velero-protection` namespace.
+
+```
+Kind: Restore
+
+…
+
+includeNamespaces: velero-protection
+includeResources:
+  - services
+  - deployments
+existingResourcePolicy: none
+
+```
+
+B. The following Restore will execute the `existingResourcePolicy` restore type `update` for the `secrets` and `daemonsets` present in the `gdpr-application` namespace.
+```
+Kind: Restore
+
+…
+includeNamespaces: gdpr-application
+includeResources:
+  - secrets
+  - daemonsets
+existingResourcePolicy: update
+```
+
+### Approach 2: Add a new spec field `existingResourcePolicyConfig` to the Restore API
+In this approach we give user the ability to specify which resources are to be included for a particular kind of force update behaviour, essentially a more granular approach where in the user is able to specify a resource:behaviour mapping. It would look like:
+`existingResourcePolicyConfig`:
+- `patch:` 
+    - `includedResources:` [ ]string
+- `recreate:`
+    - `includedResources:` [ ]string
+
+*Note:* 
+- There is no `none` behaviour in this approach as that would conform to the current/default Velero restore behaviour.
+- The `recreate` option is a non-goal for this enhancement proposal, but it is considered as a future scope.
+
+
+Example:
+A. The following Restore will execute the restore type `patch` and apply the `existingResourcePolicyConfig` for `secrets` and `daemonsets` present in the `inventory-app` namespace.
+```
+Kind: Restore
+…
+includeNamespaces: inventory-app
+existingResourcePolicyConfig:
+  patch:
+    includedResources
+     - secrets
+     - daemonsets
+
+```
+
+
+### Approach 3: Combination of Approach 1 and Approach 2
+
+Now, this approach is somewhat a combination of the aforementioned approaches. Here we propose addition of two spec fields to the Restore API - `existingResourceDefaultPolicy` and `existingResourcePolicyOverrides`. As the names suggest ,the idea being that `existingResourceDefaultPolicy` would describe the default velero behaviour for this restore and `existingResourcePolicyOverrides` would override the default policy explicitly for some resources.
+
+Example:
+A. The following Restore will execute the restore type `patch` as the `existingResourceDefaultPolicy` but will override the default policy for `secrets` using the `existingResourcePolicyOverrides` spec as `none`.
+```
+Kind: Restore
+…
+includeNamespaces: inventory-app
+existingResourceDefaultPolicy: patch
+existingResourcePolicyOverrides:
+  none:
+    includedResources
+     - secrets
+
+```
+
+## Detailed Design
+### Approach 1: Add a new spec field `existingResourcePolicy` to the Restore API
+The `existingResourcePolicy` spec field will be an `PolicyType` type field. 
+
+Restore API:
+```
+type RestoreSpec struct {
+.
+.
+.
+// ExistingResourcePolicy specifies the restore behaviour for the kubernetes resource to be restored
+// +optional
+ExistingResourcePolicy PolicyType
+
+}
+```
+PolicyType:
+```
+type PolicyType string
+const PolicyTypeNone PolicyType  = "none"
+const PolicyTypePatch PolicyType = "update"
+```
+
+### Approach 2: Add a new spec field `existingResourcePolicyConfig` to the Restore API
+The `existingResourcePolicyConfig` will be a spec of type `PolicyConfiguration` which gets added to the Restore API.
+
+Restore API:
+```
+type RestoreSpec struct {
+.
+.
+.
+// ExistingResourcePolicyConfig specifies the restore behaviour for a particular/list of kubernetes resource(s) to be restored
+// +optional
+ExistingResourcePolicyConfig []PolicyConfiguration
+
+}
+```
+
+PolicyConfiguration:
+```
+type PolicyConfiguration struct {
+
+PolicyTypeMapping map[PolicyType]ResourceList
+
+}
+```
+
+PolicyType:
+```
+type PolicyType string
+const PolicyTypePatch PolicyType = "patch"
+const PolicyTypeRecreate PolicyType = "recreate"
+```
+
+ResourceList:
+```
+type ResourceList struct {
+IncludedResources []string 
+}
+```
+
+### Approach 3: Combination of Approach 1 and Approach 2
+
+Restore API:
+```
+type RestoreSpec struct {
+.
+.
+.
+// ExistingResourceDefaultPolicy specifies the default restore behaviour for the kubernetes resource to be restored
+// +optional
+existingResourceDefaultPolicy PolicyType
+
+// ExistingResourcePolicyOverrides specifies the restore behaviour for a particular/list of kubernetes resource(s) to be restored
+// +optional
+existingResourcePolicyOverrides []PolicyConfiguration
+
+}
+```
+
+PolicyType:
+```
+type PolicyType string
+const PolicyTypeNone PolicyType  = "none"
+const PolicyTypePatch PolicyType = "patch"
+const PolicyTypeRecreate PolicyType = "recreate"
+```
+PolicyConfiguration:
+```
+type PolicyConfiguration struct {
+
+PolicyTypeMapping map[PolicyType]ResourceList
+
+}
+```
+ResourceList:
+```
+type ResourceList struct {
+IncludedResources []string 
+}
+```
+
+The restore workflow changes will be done [here](https://github.com/vmware-tanzu/velero/blob/b40bbda2d62af2f35d1406b9af4d387d4b396839/pkg/restore/restore.go#L1245)
+
+### CLI changes for Approach 1
+We would introduce a new CLI flag called `existing-resource-policy` of string type. This flag would be used to accept the 
+policy from the user. The velero restore command would look somewhat like this:
+```
+velero create restore <restore_name> --existing-resource-policy=update 
+```
+
+Help message `Restore Policy to be used during the restore workflow, can be - none, update`
+
+The CLI changes will go at `pkg/cmd/cli/restore/create.go`
+
+We would also add a validation which checks for invalid policy values provided to this flag.
+
+Restore describer will also be updated to reflect the policy `pkg/cmd/util/output/restore_describer.go`
+
+### Implementation Decision
+We have decided to go ahead with the implementation of Approach 1 as:
+- It is easier to implement 
+- It is also easier to scale and leaves room for improvement and the door open to expanding to approach 3
+- It also provides an option to preserve the existing velero restore workflow

design/graph-manifest.md

@@ -0,0 +1,219 @@
+# Object Graph Manifest for Velero
+
+## Abstract
+
+One to two sentences that describes the goal of this proposal and the problem being solved by the proposed change.
+The reader should be able to tell by the title, and the opening paragraph, if this document is relevant to them.
+
+Currently, Velero does not have a complete manifest of everything in the backup, aside from the backup tarball itself.
+This change introduces a new data structure to be stored with a backup in object storage which will allow for more efficient operations in reporting of what a backup contains.
+Additionally, this manifest should enable advancements in Velero's features and architecture, enabling dry-run support, concurrent backup and restore operations, and reliable restoration of complex applications.
+
+## Background
+
+Right now, Velero backs up items one at a time, sorted by API Group and namespace.
+It also restores items one at a time, using the restoreResourcePriorities flag to indicate which order API Groups should have their objects restored first.
+While this does work currently, it presents challenges for more complex applications that have their dependencies in the form of a graph rather than strictly linear.
+
+For example, Cluster API clusters are a set of complex Kubernetes objects that require that the "root" objects are restored first, before their "leaf" objects.
+If a Cluster that a ClusterResourceSetBinding refers to does not exist, then a restore of the CAPI cluster will fail.
+
+Additionally, Velero does not have a reliable way to communicate what objects will be affected in a backup or restore operation without actually performing the operation.
+This complicates dry-run tasks, because a user must simply perform the action without knowing what will be touched.
+It also complicates allowing backups and restores to run in parallel, because there is currently no way to know if a single Kubernetes object is included in multiple backups or restores, which can lead to unreliability, deadlocking, and race conditions were Velero made to be more concurrent today.
+
+## Goals
+
+- Introduce a manifest data structure that defines the contents of a backup.
+- Store the manifest data into object storage alongside existing backup data.
+
+## Non Goals
+
+This proposal seeks to enable, but not define, the following.
+
+- Implementing concurrency beyond what already exists in Velero.
+- Implementing a dry-run feature.
+- Implementing a new restore ordering procedure.
+
+While the data structure should take these scenarios into account, they will not be implemented alongside it.
+
+## High-Level Design
+
+To uniquely identify a Kubernetes object within a cluster or backup, the following fields are sufficient:
+
+- API Group and Version (example: backup.velero.io/v1)
+- Namespace 
+- Name
+- Labels
+
+This criteria covers the majority of Velero's inclusion or exclusion logic.
+However, some additional fields enable further use cases.
+
+- Owners, which are other Kubernetes objects that have some relationship to this object. They may be strict or soft dependencies.
+- Annotations, which provide extra metadata about the object that might be useful for other programs to consume.
+- UUID generated by Kubernetes. This is useful in defining Owner relationships, providing a single, immutable key to find an object. This is _not_ considered at restore time, only internally for defining links.
+
+All of this information already exists within a Velero backup's tarball of resources, but extracting such data is inefficient.
+The entire tarball must be downloaded and extracted, and then JSON within parsed to read labels, owners, annotations, and a UUID.
+The rest of the information is encoded in the file system structure within the Velero backup tarball.
+While doable, this is heavyweight in terms of time and potentially memory.
+
+Instead, this proposal suggests adding a new manifest structure that is kept alongside the backup tarball.
+This structure would contain the above fields only, and could be used to perform inclusion/exclusion logic on a backup, select a resource from within a backup, and do set operations over backup or restore contents to identify overlapping resources.
+
+Here are some use cases that this data structure should enable, that have been difficult to implement prior to its existence:
+
+- A dry-run operation on backup, informing the user what would be selected if they were to perform the operation.
+ A manifest could be created and saved, allowing for a user to do a dry-run, then accept it to perform the backup.
+ Restore operations can be treated similarly.
+- Efficient, non-overlapping parallelization of backup and restore operations.
+ By building or reading a manifest before performing a backup or restore, Velero can determine if there are overlapping resources.
+ If there are no overlaps, the operations can proceed in parallel.
+ If there are overlaps, the operations can proveed serially.
+- Graph-based restores for non-linear dependencies.
+ Not all resources in a Kubernetes cluster can be defined in a strict, linear way.
+ They may have multiple owners, and writing BackupItemActions or RestoreItemActions to simply return a chain of owners is not an efficient way to support the many Kubernetes operators/controllers being written.
+ Instead, by having a manifest with enough information, Velero can build a discrete list that ensures dependencies are restored before their dependents, with less input from plugin authors.
+
+## Detailed Design
+
+The Manifest data structure would look like this, in Go type structure:
+
+```golang
+// NamespacedItems maps a given namespace to all of its contained items.
+type NamespacedItems map[string]*Item
+
+// APIGroupNamespaces maps an API group/version to a map of namespaces and their items.
+type KindNamespaces map[string]NamespacedItems
+
+type Manifest struct {
+    // Kinds holds the top level map of all resources in a manifest.
+    Kinds KindNamespaces
+
+    // Index is used to look up an individual item quickly based on UUID.
+    // This enables fetching owners out of the maps more efficiently at the cost of memory space.
+    Index map[string]*Item
+}
+
+
+// Item represents a Kubernetes resource within a backup based on it's selectable criteria.
+// It is not the whole Kubernetes resource as retrieved from the API server, but rather a collection of important fields needed for filtering.
+type Item struct {
+    // Kubernetes API group which this Item belongs to.
+    // Could be a core resource, or a CustomResourceDefinition.
+    APIGroup string
+
+    // Version of the APIGroup that the Item belongs to.
+    APIVersion string
+
+    // Kubernetes namespace which contains this item.
+    // Empty string for cluster-level resource.
+    Namespace string
+
+    // Item's given name.
+    Name string
+
+    // Map of labels that the Item had at backup time.
+    Labels map[string]string
+
+    // Map of annotations that the Item had at Backup time.
+    // Useful for plugins that may decide to process only Items with specific annotations.
+    Annotations map[string]string
+
+    // Owners is a list of UUIDs to other items that own or refer to this item.
+    Owners []string
+
+    // Manifest is a pointer to the Manifest in which this object is contained.
+    // Useful for getting access to things like the Manifest.Index map.
+    Manifest *Manifest
+}
+```
+
+In addition to the new types, the following Go interfaces would be provided for convenience.
+
+```golang
+type Itermer interface {
+    // Returns the Item as a string, following the current Velero backup version 1.1.0 tarball structure format.
+    // <APIGroup>/<Namespace>/<APIVersion>/<name>.json
+    String() string
+
+    // Owners returns a slice of realized Items that own or refer to the current Item.
+    // Useful for building out a full graph of Items to restore.
+    // Will use the UUIDs in Item.Owners to look up the owner Items in the Manifest.
+    Owners() []*Item
+
+    // Kind returns the Kind of an object, which is a combination of the APIGroup and APIVersion.
+    // Useful for verifying the needed CustomResourceDefinition exists before actually restoring this Item.
+    Kind() *Item
+
+    // Children returns a slice of all Items that refer to this item as an Owner.
+    Children() []*Items
+}
+
+// This error type is being created in order to make reliable sentinel errors.
+// See https://dave.cheney.net/2019/06/10/constant-time for more details.
+type ManifestError string
+
+func (e ManifestError) Error() string {
+    return string(e)
+}
+
+const ItemAlreadyExists = ManifestError("item already exists in manifest")
+
+type Manifester interface {
+    // Set returns the entire list of resources as a set of strings (using Itemer.String).
+    // This is useful for comparing two manifests and determining if they have any overlapping resources.
+    // In the future, when implementing concurrent operations, this can be used as a sanity check to ensure resources aren't being backed up or restored by two operations at once.
+    Set() sets.String
+
+    // Adds an item to the appropriate APIGroup and Namespace within a Manifest
+    // Returns (true, nil) if the Item is successfully added to the Manifest,
+    // Returns (false, ItemAlreadyExists) if the Item is already in the Manifest.
+    Add(*Item) (bool, error)
+}
+```
+
+### Serialization
+
+The entire `Manifest` should be serialized into the `manifest.json` file within the object storage for a single backup.
+It is possible that this file could also be compressed for space efficiency.
+
+### Memory Concerns
+
+Because the `Manifest` is holding a minimal amount of data, memory sizes should not be a concern for most clusters.
+TODO: Document known limits on API group name, resource name, and kind name character limits.
+
+## Security Considerations
+
+Introducing this manifest does not increase the attack surface of Velero, as this data is already present in the existing backups.
+Storing the manifest.json file next to the existing backup data in the object storage does not change access patterns.
+
+## Compatibility
+
+The introduction of this file should trigger Velero backup version 1.2.0, but it will not interfere with Velero versions that do not support the `Manifest` as the file will be additive.
+In time, this file will replace the `<backupname>-resource-list.json.gz` file, but for compatibility the two will appear side by side.
+
+When first implemented, Velero should simply build the `Manifest` as it backs up items, and serialize it at the end.
+Any logic changes that rely on the `Manifest` file must be introduced with their own design document, with their own compatibility concerns.
+
+## Implementation
+
+The `Manifest` object will _not_ be implemented as a Kubernetes CustomResourceDefinition, but rather one of Velero's own internal constructs.
+
+Implementation for the data structure alone should be minimal - the types will need to be defined in a `manifest` package.
+Then, the backup process should create a `Manifest`, passing it to the various `*Backuppers` in the `backup` package.
+These methods will insert individual `Items` into the `Manifest`.
+Finally, logic should be added to the `persistence` package to ensure that the new `manifest.json` file is uploadable and allowed.
+
+## Alternatives Considered
+
+None so far.
+
+## Open Issues
+
+- When should compatibility with the `<backupname>-resource-list.json.gz` file be dropped?
+- What are some good test case Kubernetes resources and controllers to try this out with?
+Cluster API seems like an obvious choice, but are there others?
+- Since it is not implemented as a CustomResourceDefinition, how can a `Manifest` be retained so that users could issue a dry-run command, then perform their actual desire operation?
+Could it be stored in Velero's temp directories?
+Note that this is making Velero itself more stateful.

design/multiple-label-selectors_design.md

@@ -0,0 +1,138 @@
+# Ensure support for backing up resources based on multiple labels
+## Abstract
+As of today Velero supports filtering of resources based on single label selector per backup. It is desired that Velero
+support backing up of resources based on multiple labels (OR logic).
+
+**Note:** This solution is required because kubernetes label selectors only allow AND logic of labels.
+
+## Background
+Currently, Velero's Backup/Restore API has a spec field `LabelSelector` which helps in filtering of resources based on
+a **single** label value per backup/restore request. For instance, if the user specifies the `Backup.Spec.LabelSelector` as
+`data-protection-app: true`, Velero will grab all the resources that possess this label and perform the backup
+operation on them. The `LabelSelector` field does not accept more than one labels, and thus if the user want to take
+backup for resources consisting of a label from a set of labels (label1 OR label2 OR label3) then the user needs to
+create multiple backups per label rule. It would be really useful if Velero Backup API could respect a set of
+labels (OR Rule) for a single backup request.
+
+Related Issue: https://github.com/vmware-tanzu/velero/issues/1508
+
+## Goals
+- Enable support for backing up resources based on multiple labels (OR Logic) in a single backup config.
+- Enable support for restoring resources based on multiple labels (OR Logic) in a single restore config.
+
+## Use Case/Scenario
+Let's say as a Velero user you want to take a backup of secrets, but all these secrets do not have one single consistent
+label on them. We want to take backup of secrets having any one label in `app=gdpr`, `app=wpa` and `app=ccpa`. Here
+we would have to create 3 instances of backup for each label rule. This can become cumbersome at scale.
+
+## High-Level Design
+### Addition of `OrLabelSelectors` spec to Velero Backup/Restore API
+For Velero to back up resources if they consist of any one label from a set of labels, we would like to add a new spec
+field `OrLabelSelectors` which would enable user to specify them. The Velero backup would somewhat look like:
+
+```
+apiVersion: velero.io/v1
+kind: Backup
+metadata:
+  name: backup-101
+  namespace: openshift-adp
+spec:
+  includedNamespaces:
+  - test
+  storageLocation: velero-sample-1
+  ttl: 720h0m0s
+  orLabelSelectors:
+  - matchLabels:
+      app=gdpr
+  - matchLabels:
+      app=wpa
+  - matchLabels:
+      app=ccpa
+```
+
+**Note:** This approach will **not** be changing any current behavior related to Backup API spec `LabelSelector`. Rather we
+propose that the label in `LabelSelector` spec and labels in `OrLabelSelectors` should be treated as different Velero functionalities.
+Both these fields will be treated as separate Velero Backup API specs. If `LabelSelector` (singular) is present then just match that label.
+And if `OrLabelSelectors` is present then match to any label in the set specified by the user. For backup case, if both the `LabelSelector` and `OrLabelSelectors` 
+are specified (we do not anticipate this as a real world use-case) then the `OrLabelSelectors` will take precedence, `LabelSelector` will
+only be used to filter only when `OrLabelSelectors` is not specified by the user. This helps to keep both spec behaviour independent and not confuse the users. 
+This way we preserve the existing Velero behaviour and implement the new functionality in a much cleaner way.
+For instance, let's take a look the following cases:
+
+1. Only `LabelSelector` specified: Velero will create a backup with resources matching label `app=protect-db`
+```
+apiVersion: velero.io/v1
+kind: Backup
+metadata:
+  name: backup-101
+  namespace: openshift-adp
+spec:
+  includedNamespaces:
+  - test
+  storageLocation: velero-sample-1
+  ttl: 720h0m0s
+  labelSelector:
+  - matchLabels:
+      app=gdpr
+```
+2. Only `OrLabelSelectors` specified: Velero will create a backup with resources matching any label from set `{app=gdpr, app=wpa, app=ccpa}`
+```
+apiVersion: velero.io/v1
+kind: Backup
+metadata:
+  name: backup-101
+  namespace: openshift-adp
+spec:
+  includedNamespaces:
+  - test
+  storageLocation: velero-sample-1
+  ttl: 720h0m0s
+  orLabelSelectors:
+  - matchLabels:
+      app=gdpr
+  - matchLabels:
+      app=wpa
+  - matchLabels:
+      app=ccpa
+```
+
+Similar implementation will be done for the Restore API as well.
+
+## Detailed Design
+With the Introduction of `OrLabelSelectors` the BackupSpec and RestoreSpec will look like:
+
+BackupSpec:
+```
+type BackupSpec struct {
+[...]
+// OrLabelSelectors is a set of []metav1.LabelSelector to filter with
+// when adding individual objects to the backup. Resources matching any one
+// label from the set of labels will be added to the backup. If empty
+// or nil, all objects are included. Optional.
+// +optional
+OrLabelSelectors []\*metav1.LabelSelector
+[...]
+}
+```
+
+RestoreSpec:
+```
+type RestoreSpec struct {
+[...]
+// OrLabelSelectors is a set of []metav1.LabelSelector to filter with
+// when restoring objects from the backup. Resources matching any one
+// label from the set of labels will be restored from the backup. If empty
+// or nil, all objects are included from the backup. Optional.
+// +optional
+OrLabelSelectors []\*metav1.LabelSelector
+[...]
+}
+```
+
+The logic to collect resources to be backed up for a particular backup will be updated in the `backup/item_collector.go`
+around [here](https://github.com/vmware-tanzu/velero/blob/574baeb3c920f97b47985ec3957debdc70bcd5f8/pkg/backup/item_collector.go#L294).
+
+And for filtering the resources to be restored, the changes will go [here](https://github.com/vmware-tanzu/velero/blob/d1063bda7e513150fd9ae09c3c3c8b1115cb1965/pkg/restore/restore.go#L1769)
+
+**Note:**
+- This feature will not be exposed via Velero CLI.

design/new-prepost-backuprestore-plugin-hooks.md

@@ -0,0 +1,735 @@
+# Pre-Backup, Post-Backup, Pre-Restore, and Post-Restore Action Plugin Hooks
+
+## Abstract
+
+Velero should provide a way to trigger actions before and after each backup and restore.
+**Important**: These proposed plugin hooks are fundamentally different from the existing plugin hooks, BackupItemAction and RestoreItemAction, which are triggered per resource item during backup and restore, respectively.
+The proposed plugin hooks are to be executed only once: pre-backup (before backup starts), post-backup (after the backup is completed and uploaded to object storage, including volumes snapshots), pre-restore (before restore starts) and post-restore (after the restore is completed, including volumes are restored).
+
+### PreBackup and PostBackup Actions
+
+For the backup,  the sequence of events of Velero backup are the following (these sequence depicted is prior upcoming changes for [upload progress #3533](https://github.com/vmware-tanzu/velero/issues/3533) ):
+
+```
+New Backup Request
+ |--> Validation of the request
+       |--> Set Backup Phase "In Progress"
+            | --> Start Backup
+                  | --> Discover all Plugins
+                        |--> Check if Backup Exists
+                             |--> Backup all K8s Resource Items
+                                  |--> Perform all Volumes Snapshots
+                                       |--> Final Backup Phase is determined
+                                            |--> Persist Backup and Logs on Object Storage
+```
+We propose the pre-backup and post-backup plugin hooks to be executed in this sequence:
+
+```
+New Backup Request
+ |--> Validation of the request
+       |--> Set Backup Phase "In Progress"
+            | --> Start Backup
+                  | --> Discover all Plugins
+                        |--> Check if Backup Exists
+                             |--> **PreBackupActions** are executed, logging actions on existent backup log file
+                                   |--> Backup all K8s Resource Items
+                                        |--> Perform all Volumes Snapshots
+                                            |--> Final Backup Phase is determined
+                                                 |--> Persist Backup and logs on Object Storage
+                                                    |--> **PostBackupActions** are executed, logging to its own file
+```
+These plugin hooks will be invoked:
+
+- PreBackupAction: plugin actions are executed after the backup object is created and validated but before the backup is being processed, more precisely _before_ function [c.backupper.Backup](https://github.com/vmware-tanzu/velero/blob/74476db9d791fa91bba0147eac8ec189820adb3d/pkg/controller/backup_controller.go#L590). If the PreBackupActions return an err, the backup object is not processed and the Backup phase will be set as `FailedPreBackupActions`.
+
+- PostBackupAction: plugin actions are executed after the backup is finished and persisted, more precisely _after_ function [c.runBackup](https://github.com/vmware-tanzu/velero/blob/74476db9d791fa91bba0147eac8ec189820adb3d/pkg/controller/backup_controller.go#L274).
+
+The proposed plugin hooks will execute actions that will have statuses on their own:
+`Backup.Status.PreBackupActionsStatuses` and `Backup.Status.PostBackupActionsStatuses` which will be an array of a proposed struct `ActionStatus` with PluginName, StartTimestamp, CompletionTimestamp and Phase.
+
+### PreRestore and PostRestore Actions
+
+For the restore,  the sequence of events of Velero restore are the following (these sequence depicted is prior upcoming changes for [upload progress #3533](https://github.com/vmware-tanzu/velero/issues/3533) ):
+```
+New Restore Request
+ |--> Validation of the request
+      |--> Checks if restore is from a backup or a schedule
+           |--> Fetches backup
+                |--> Set Restore Phase "In Progress"
+                     |--> Start Restore
+                          |--> Discover all Plugins
+                               |--> Download backup file to temp
+                                    |--> Fetch list of volumes snapshots
+                                         |--> Restore K8s items, including PVs
+                                              |--> Final Restore Phase is determined
+                                                   |--> Persist Restore logs on Object Storage
+```
+We propose the pre-restore and post-restore plugin hooks to be executed in this sequence:
+```
+New Restore Request
+ |--> Validation of the request
+      |--> Checks if restore is from a backup or a schedule
+           |--> Fetches backup
+                |--> Set Restore Phase "In Progress"
+                     |--> Start Restore
+                          |--> Discover all Plugins
+                                |--> Download backup file to temp
+                                         |--> Fetch list of volumes snapshots
+                                            |--> **PreRestoreActions** are executed, logging actions on existent backup log file
+                                              |--> Restore K8s items, including PVs
+                                                   |--> Final Restore Phase is determined
+                                                        |--> Persist Restore logs on Object Storage
+                                                            |--> **PostRestoreActions** are executed, logging to its own file
+```
+
+These plugin hooks will be invoked:
+
+- PreRestoreAction: plugin actions are executed after the restore object is created and validated and before the backup object is fetched, more precisely in function `runValidatedRestore` _after_ function [info.backupStore.GetBackupVolumeSnapshots](https://github.com/vmware-tanzu/velero/blob/7c75cd6cf854064c9a454e53ba22cc5881d3f1f0/pkg/controller/restore_controller.go#L460). If the PreRestoreActions return an err, the restore object is not processed and the Restore phase will be set a `FailedPreRestoreActions`.
+  
+- PostRestoreAction: plugin actions are executed after the restore finishes processing all items and volumes snapshots are restored and logs persisted, more precisely in function `processRestore` _after_ setting [`restore.Status.CompletionTimestamp`](https://github.com/vmware-tanzu/velero/blob/7c75cd6cf854064c9a454e53ba22cc5881d3f1f0/pkg/controller/restore_controller.go#L273).
+
+The proposed plugin hooks will execute actions that will have statuses on their own:
+`Restore.Status.PreRestoreActionsStatuses` and `Restore.Status.PostRestoreActionsStatuses` which will be an array of a proposed struct `ActionStatus` with PluginName, StartTimestamp, CompletionTimestamp and Phase.
+
+## Background
+
+Increasingly, Velero is employed for workload migrations across different Kubernetes clusters.
+Using Velero for migrations requires an atomic operation involving a Velero backup on a source cluster followed by a Velero restore on a destination cluster.
+
+It is common during these migrations to perform many actions inside and outside Kubernetes clusters.
+**Attention**: these actions are not per resource item, but they are actions to be executed _once_ before and/or after the migration itself (remember, migration in this context is Velero Backup + Velero Restore).
+
+One important use case driving this proposal is migrating stateful workloads at scale across different clusters/storage backends.
+Today, Velero's Restic integration is the response for such use cases, but there are some limitations:
+
+- Quiesce/unquiesce workloads: Pod hooks are useful for quiescing/unquiescing workloads, but platform engineers often do not have the luxury/visibility/time/knowledge to go through each pod in order to add specific commands to quiesce/unquiesce workloads.
+- Orphan PVC/PV pairs: PVCs/PVs that do not have associated running pods are not backed up and consequently, are not migrated.
+
+Aiming to address these two limitations, and separate from this proposal, we would like to write a Velero plugin that takes advantage of the proposed Pre-Backup plugin hook. This plugin will be executed _once_ (not per resource item) prior backup. It will scale down the applications setting `.spec.replicas=0` to all deployments, statefulsets, daemonsets, replicasets, etc. and will start a small-footprint staging pod that will mount all PVC/PV pairs. Similarly, we would like to write another plugin that will utilize the proposed Post-Restore plugin hook. This plugin will unquiesce migrated applications by killing the staging pod and reinstating original `.spec.replicas` values after the Velero restore is completed.
+
+Other examples of plugins that can use the proposed plugin hooks are:
+
+- PostBackupAction: trigger a Velero Restore after a successful Velero backup (and complete the migration operation).
+- PreRestoreAction: pre-expand the cluster's capacity via Cluster API to avoid starvation of cluster resources before the restore.
+- PostRestoreAction: call actions to be performed outside Kubernetes clusters, such as configure a global load balancer (GLB) that enables the new cluster.
+
+The post backup actions will be executed after the backup is uploaded (persisted) on the disk. The logs of post-backup actions will be uploaded on the disk once the actions are completed.
+
+The post restore actions will be executed after the restore is uploaded (persisted) on the disk. The logs of post-restore actions will be uploaded on the disk once the actions are completed.
+
+This design seeks to provide missing extension points. This proposal's scope is to only add the new plugin hooks, not the plugins themselves.
+
+## Goals
+
+- Provide PreBackupAction, PostBackupAction, PreRestoreAction, and PostRestoreAction APIs for plugins to implement.
+- Update Velero backup and restore creation logic to invoke registered PreBackupAction and PreRestoreAction plugins before processing the backup and restore respectively.
+- Update Velero backup and restore complete logic to invoke registered PostBackupAction and PostRestoreAction plugins the objects are uploaded on disk.
+- Create one `ActionStatus` struct to keep track of execution of the plugin hooks. This struct has PluginName, StartTimestamp, CompletionTimestamp and Phase.
+- Add sub statuses for the plugins on Backup object: `Backup.Status.PreBackupActionsStatuses` and `Backup.Status.PostBackupActionsStatuses`. They will be flagged as optional and nullable. They will be populated only each plugin registered for the PreBackup and PostBackup hooks, respectively.
+- Add sub statuses for the plugins on Restore object: `Backup.Status.PreRestoreActionsStatuses` and `Backup.Status.PostRestoreActionsStatuses`. They will be flagged as optional and nullable. They will be populated only each plugin registered for the PreRestore and PostRestore hooks, respectively.
+- that will be populated optionally if Pre/Post Backup/Restore.
+
+## Non-Goals
+
+- Specific implementations of the PreBackupAction, PostBackupAction, PreRestoreAction and PostRestoreAction API beyond test cases.
+- For migration specific actions (Velero Backup + Velero Restore), add disk synchronization during the validation of the Restore (making sure the newly created backup will show during restore)
+
+## High-Level Design
+
+The Velero backup controller package will be modified for `PreBackupAction` and `PostBackupAction`.
+
+The PreBackupAction plugin API will resemble the BackupItemAction plugin hook design, but with the fundamental difference that it will receive only as input the Velero `Backup` object created.
+It will not receive any resource list items because the backup is not yet running at that stage.
+In addition, the `PreBackupAction` interface will only have an `Execute()` method since the plugin will be executed once per Backup creation, not per item.
+
+The Velero backup controller will be modified so that if there are any PreBackupAction plugins registered, they will be
+
+The PostBackupAction plugin API will resemble the BackupItemAction plugin design, but with the fundamental difference that it will receive only as input the Velero `Backup` object without any resource list items.
+By this stage, the backup has already been executed, with items backed up and volumes snapshots processed and persisted.
+The `PostBackupAction` interface will only have an `Execute()` method since the plugin will be executed only once per Backup, not per item.
+
+If there are any PostBackupAction plugins registered, they will be executed after the backup is finished and persisted, more precisely _after_ function [c.runBackup](https://github.com/vmware-tanzu/velero/blob/74476db9d791fa91bba0147eac8ec189820adb3d/pkg/controller/backup_controller.go#L274).
+
+The Velero restore controller package will be modified for `PreRestoreAction` and `PostRestoreAction`.
+
+The PreRestoreAction plugin API will resemble the RestoreItemAction plugin design, but with the fundamental difference that it will receive only as input the Velero `Restore` object created.
+It will not receive any resource list items because the restore has not yet been running at that stage.
+In addition, the `PreRestoreAction` interface will only have an `Execute()` method since the plugin will be executed only once per Restore creation, not per item.
+
+The Velero restore controller will be modified so that if there are any PreRestoreAction plugins registered, they will be executed after the restore object is created and validated and before the backup object is fetched, more precisely in function `runValidatedRestore` _after_ function [info.backupStore.GetBackupVolumeSnapshots](https://github.com/vmware-tanzu/velero/blob/7c75cd6cf854064c9a454e53ba22cc5881d3f1f0/pkg/controller/restore_controller.go#L460). If the PreRestoreActions return an err, the restore object is not processed and the Restore phase will be set a `FailedPreRestoreActions`.
+
+The PostRestoreAction plugin API will resemble the RestoreItemAction plugin design, but with the fundamental difference that it will receive only as input the Velero `Restore` object without any resource list items.
+At this stage, the restore has already been executed.
+The `PostRestoreAction` interface will only have an `Execute()` method since the plugin will be executed only once per Restore, not per item.
+
+If any PostRestoreAction plugins are registered, they will be executed after the restore finishes processing all items and volumes snapshots are restored and logs persisted, more precisely in function `processRestore` _after_ setting [`restore.Status.CompletionTimestamp`](https://github.com/vmware-tanzu/velero/blob/7c75cd6cf854064c9a454e53ba22cc5881d3f1f0/pkg/controller/restore_controller.go#L273).
+
+## Detailed Design
+
+### New Status struct
+
+To keep the status of the plugins, we propose the following struct:
+
+```go
+type ActionStatus struct {
+    // PluginName is the name of the registered plugin
+    // retrieved by the PluginManager as id.Name
+    // +optional
+    // +nullable
+    PluginName string `json:"pluginName,omitempty"`
+
+    // StartTimestamp records the time the plugin started.
+    // +optional
+    // +nullable
+    StartTimestamp *metav1.Time `json:"startTimestamp,omitempty"`
+
+    // CompletionTimestamp records the time the plugin was completed.
+    // +optional
+    // +nullable
+    CompletionTimestamp *metav1.Time `json:"completionTimestamp,omitempty"`
+
+    // Phase is the current state of the Action.
+    // +optional
+    // +nullable
+    Phase ActionPhase `json:"phase,omitempty"`
+}
+
+// ActionPhase is a string representation of the lifecycle phase of an action being executed by a plugin
+// of a Velero backup.
+// +kubebuilder:validation:Enum=InProgress;Completed;Failed
+type ActionPhase string
+
+const (
+    // ActionPhaseInProgress means the action has being executed
+    ActionPhaseInProgress ActionPhase = "InProgress"
+
+    // ActionPhaseCompleted means the action finished successfully
+    ActionPhaseCompleted ActionPhase = "Completed"
+
+    // ActionPhaseFailed means the action failed
+    ActionPhaseFailed ActionPhase = "Failed"
+)
+
+```
+
+### Backup Status of the Plugins
+
+The `Backup` Status section will have the follow:
+
+```go
+type BackupStatus struct {
+    (...)
+    // PreBackupActionsStatuses contains information about the pre backup plugins's execution.
+    // Note that this information is will be only populated if there are prebackup plugins actions
+    // registered
+    // +optional
+    // +nullable
+    PreBackupActionsStatuses *[]ActionStatus `json:"preBackupActionsStatuses,omitempty"`
+
+    // PostBackupActionsStatuses contains information about the post backup plugins's execution.
+    // Note that this information is will be only populated if there are postbackup plugins actions
+    // registered
+    // +optional
+    // +nullable
+    PostBackupActionsStatuses *[]ActionStatus `json:"postBackupActionsStatuses,omitempty"`
+
+}
+```
+
+### Restore Status of the Plugins
+
+The `Restore` Status section will have the follow:
+
+```go
+type RestoreStatus struct {
+    (...)
+    // PreRestoreActionsStatuses contains information about the pre Restore plugins's execution.
+    // Note that this information is will be only populated if there are preRestore plugins actions
+    // registered
+    // +optional
+    // +nullable
+    PreRestoreActionsStatuses *[]ActionStatus `json:"preRestoreActionsStatuses,omitempty"`
+
+    // PostRestoreActionsStatuses contains information about the post restore plugins's execution.
+    // Note that this information is will be only populated if there are postrestore plugins actions
+    // registered
+    // +optional
+    // +nullable
+    PostRestoreActionsStatuses *[]ActionStatus `json:"postRestoreActionsStatuses,omitempty"`
+
+}
+```
+
+### New Backup and Restore Phases
+
+#### New Backup Phase: FailedPreBackupActions
+
+In case the PreBackupActionsStatuses has at least one `ActionPhase` = `Failed`, it means al least one of the plugins returned an error and consequently, the backup will not move forward. The final status of the Backup object will be set as `FailedPreBackupActions`:
+
+```go
+
+// BackupPhase is a string representation of the lifecycle phase
+// of a Velero backup.
+// +kubebuilder:validation:Enum=New;FailedValidation;FailedPreBackupActions;InProgress;Uploading;UploadingPartialFailure;Completed;PartiallyFailed;Failed;Deleting
+type BackupPhase string
+
+const (
+
+    (...)
+
+    // BackupPhaseFailedPreBackupActions means one or more the Pre Backup Actions has failed
+    // and therefore backup will not run.
+    BackupPhaseFailedPreBackupActions BackupPhase = "FailedPreBackupActions"
+
+    (...)
+)
+
+```
+
+#### New Restore Phase FailedPreRestoreActions
+
+In case the PreRestoreActionsStatuses has at least one `ActionPhase` = `Failed`, it means al least one of the plugins returned an error and consequently, the restore will not move forward. The final status of the Restore object will be set as `FailedPreRestoreActions`:
+
+```go
+
+// RestorePhase is a string representation of the lifecycle phase
+// of a Velero restore
+// +kubebuilder:validation:Enum=New;FailedValidation;FailedPreRestoreActions;InProgress;Completed;PartiallyFailed;Failed
+type RestorePhase string
+
+const (
+
+    (...)
+
+    // RestorePhaseFailedPreRestoreActions means one or more the Pre Restore Actions has failed
+    // and therefore restore will not run.
+    RestorePhaseFailedPreRestoreActions BackupPhase = "FailedPreRestoreActions"
+
+    (...)
+)
+
+```
+
+### New Interface types
+
+#### PreBackupAction
+
+The `PreBackupAction` interface is as follows:
+
+```go
+// PreBackupAction provides a hook into the backup process before it begins.
+type PreBackupAction interface {
+    // Execute the PreBackupAction plugin providing it access to the Backup that
+    // is being executed
+    Execute(backup *api.Backup) error
+}
+```
+
+`PreBackupAction` will be defined in `pkg/plugin/velero/pre_backup_action.go`.
+
+#### PostBackupAction
+
+The `PostBackupAction` interface is as follows:
+
+```go
+// PostBackupAction provides a hook into the backup process after it completes.
+type PostBackupAction interface {
+    // Execute the PostBackupAction plugin providing it access to the Backup that
+    // has been completed
+    Execute(backup *api.Backup) error
+}
+```
+
+`PostBackupAction` will be defined in `pkg/plugin/velero/post_backup_action.go`.
+
+#### PreRestoreAction
+
+The `PreRestoreAction` interface is as follows:
+
+```go
+// PreRestoreAction provides a hook into the restore process before it begins.
+type PreRestoreAction interface {
+    // Execute the PreRestoreAction plugin providing it access to the Restore that
+    // is being executed
+    Execute(restore *api.Restore) error
+}
+```
+
+`PreRestoreAction` will be defined in `pkg/plugin/velero/pre_restore_action.go`.
+
+#### PostRestoreAction
+
+The `PostRestoreAction` interface is as follows:
+
+```go
+// PostRestoreAction provides a hook into the restore process after it completes.
+type PostRestoreAction interface {
+    // Execute the PostRestoreAction plugin providing it access to the Restore that
+    // has been completed
+    Execute(restore *api.Restore) error
+}
+```
+
+`PostRestoreAction` will be defined in `pkg/plugin/velero/post_restore_action.go`.
+
+### New BackupStore Interface Methods
+
+For the persistence of the logs originated from the PostBackup and PostRestore plugins, create two additional methods on `BackupStore` interface:
+
+```go
+type BackupStore interface {
+    (...)
+    PutPostBackuplog(backup string, log io.Reader) error
+    PutPostRestoreLog(backup, restore string, log io.Reader) error
+    (...)
+```
+
+The implementation of these new two methods will go hand-in-hand with the changes of uploading phases rebase.
+
+
+### Generate Protobuf Definitions and Client/Servers
+
+In `pkg/plugin/proto`, add the following:
+
+1. Protobuf definitions will be necessary for PreBackupAction in `pkg/plugin/proto/PreBackupAction.proto`.
+
+```protobuf
+message PreBackupActionExecuteRequest {
+    ...
+}
+
+service PreBackupAction {
+    rpc Execute(PreBackupActionExecuteRequest) returns (Empty)
+}
+```
+
+Once these are written, then a client and server implementation can be written in `pkg/plugin/framework/pre_backup_action_client.go` and `pkg/plugin/framework/pre_backup_action_server.go`, respectively.
+
+2. Protobuf definitions will be necessary for PostBackupAction in `pkg/plugin/proto/PostBackupAction.proto`.
+
+```protobuf
+message PostBackupActionExecuteRequest {
+    ...
+}
+
+service PostBackupAction {
+    rpc Execute(PostBackupActionExecuteRequest) returns (Empty)
+}
+```
+
+Once these are written, then a client and server implementation can be written in `pkg/plugin/framework/post_backup_action_client.go` and `pkg/plugin/framework/post_backup_action_server.go`, respectively.
+
+3. Protobuf definitions will be necessary for PreRestoreAction in `pkg/plugin/proto/PreRestoreAction.proto`.
+
+```protobuf
+message PreRestoreActionExecuteRequest {
+    ...
+}
+
+service PreRestoreAction {
+    rpc Execute(PreRestoreActionExecuteRequest) returns (Empty)
+}
+```
+
+Once these are written, then a client and server implementation can be written in `pkg/plugin/framework/pre_restore_action_client.go` and `pkg/plugin/framework/pre_restore_action_server.go`, respectively.
+
+4. Protobuf definitions will be necessary for PostRestoreAction in `pkg/plugin/proto/PostRestoreAction.proto`.
+
+```protobuf
+message PostRestoreActionExecuteRequest {
+    ...
+}
+
+service PostRestoreAction {
+    rpc Execute(PostRestoreActionExecuteRequest) returns (Empty)
+}
+```
+
+Once these are written, then a client and server implementation can be written in `pkg/plugin/framework/post_restore_action_client.go` and `pkg/plugin/framework/post_restore_action_server.go`, respectively.
+
+### Restartable Delete Plugins
+
+Similar to the `RestoreItemAction` and `BackupItemAction` plugins, restartable processes will need to be implemented (with the difference that there is no `AppliedTo()` method).
+
+In `pkg/plugin/clientmgmt/`, add
+
+1. `restartable_pre_backup_action.go`, creating the following unexported type:
+
+```go
+type restartablePreBackupAction struct {
+    key                 kindAndName
+    sharedPluginProcess RestartableProcess
+}
+
+func newRestartablePreBackupAction(name string, sharedPluginProcess RestartableProcess) *restartablePreBackupAction {
+    // ...
+}
+
+func (r *restartablePreBackupAction) getPreBackupAction() (velero.PreBackupAction, error) {
+    // ...
+}
+
+func (r *restartablePreBackupAction) getDelegate() (velero.PreBackupAction, error) {
+    // ...
+}
+
+// Execute restarts the plugin's process if needed, then delegates the call.
+func (r *restartablePreBackupAction) Execute(input *velero.PreBackupActionInput) (error) {
+    // ...
+}
+```
+
+2. `restartable_post_backup_action.go`, creating the following unexported type:
+
+```go
+type restartablePostBackupAction struct {
+    key                 kindAndName
+    sharedPluginProcess RestartableProcess
+}
+
+func newRestartablePostBackupAction(name string, sharedPluginProcess RestartableProcess) *restartablePostBackupAction {
+    // ...
+}
+
+func (r *restartablePostBackupAction) getPostBackupAction() (velero.PostBackupAction, error) {
+    // ...
+}
+
+func (r *restartablePostBackupAction) getDelegate() (velero.PostBackupAction, error) {
+    // ...
+}
+
+// Execute restarts the plugin's process if needed, then delegates the call.
+func (r *restartablePostBackupAction) Execute(input *velero.PostBackupActionInput) (error) {
+    // ...
+}
+```
+
+3. `restartable_pre_restore_action.go`, creating the following unexported type:
+
+```go
+type restartablePreRestoreAction struct {
+    key                 kindAndName
+    sharedPluginProcess RestartableProcess
+}
+
+func newRestartablePreRestoreAction(name string, sharedPluginProcess RestartableProcess) *restartablePreRestoreAction {
+    // ...
+}
+
+func (r *restartablePreRestoreAction) getPreRestoreAction() (velero.PreRestoreAction, error) {
+    // ...
+}
+
+func (r *restartablePreRestoreAction) getDelegate() (velero.PreRestoreAction, error) {
+    // ...
+}
+
+// Execute restarts the plugin's process if needed, then delegates the call.
+func (r *restartablePreRestoreAction) Execute(input *velero.PreRestoreActionInput) (error) {
+    // ...
+}
+```
+
+4. `restartable_post_restore_action.go`, creating the following unexported type:
+
+```go
+type restartablePostRestoreAction struct {
+    key                 kindAndName
+    sharedPluginProcess RestartableProcess
+}
+
+func newRestartablePostRestoreAction(name string, sharedPluginProcess RestartableProcess) *restartablePostRestoreAction {
+    // ...
+}
+
+func (r *restartablePostRestoreAction) getPostRestoreAction() (velero.PostRestoreAction, error) {
+    // ...
+}
+
+func (r *restartablePostRestoreAction) getDelegate() (velero.PostRestoreAction, error) {
+    // ...
+}
+
+// Execute restarts the plugin's process if needed, then delegates the call.
+func (r *restartablePostRestoreAction) Execute(input *velero.PostRestoreActionInput) (error) {
+    // ...
+}
+```
+
+### Plugin Manager Changes
+
+Add the following methods to the `Manager` interface in `pkg/plugin/clientmgmt/manager.go`:
+
+```go
+type Manager interface {
+    ...
+    // Get PreBackupAction returns a PreBackupAction plugin for name.
+    GetPreBackupAction(name string) (PreBackupAction, error)
+
+    // Get PreBackupActions returns the all PreBackupAction plugins.
+    GetPreBackupActions() ([]PreBackupAction, error)
+
+    // Get PostBackupAction returns a PostBackupAction plugin for name.
+    GetPostBackupAction(name string) (PostBackupAction, error)
+
+    // GetPostBackupActions returns the all PostBackupAction plugins.
+    GetPostBackupActions() ([]PostBackupAction, error)
+
+    // Get PreRestoreAction returns a PreRestoreAction plugin for name.
+    GetPreRestoreAction(name string) (PreRestoreAction, error)
+
+    // Get PreRestoreActions returns the all PreRestoreAction plugins.
+    GetPreRestoreActions() ([]PreRestoreAction, error)
+
+    // Get PostRestoreAction returns a PostRestoreAction plugin for name.
+    GetPostRestoreAction(name string) (PostRestoreAction, error)
+
+    // GetPostRestoreActions returns the all PostRestoreAction plugins.
+    GetPostRestoreActions() ([]PostRestoreAction, error)
+
+}
+```
+
+`GetPreBackupAction` and `GetPreBackupActions` will invoke the `restartablePreBackupAction` implementations.
+`GetPostBackupAction` and `GetPostBackupActions` will invoke the `restartablePostBackupAction` implementations.
+`GetPreRestoreAction` and `GetPreRestoreActions` will invoke the `restartablePreRestoreAction` implementations.
+`GetPostRestoreAction` and `GetPostRestoreActions` will invoke the `restartablePostRestoreAction` implementations.
+
+### How to invoke the Plugins
+
+#### Getting Pre/Post Backup Actions
+
+Getting Actions on `backup_controller.go` in `runBackup`:
+
+```go
+
+    backupLog.Info("Getting PreBackup actions")
+    preBackupActions, err := pluginManager.GetPreBackupActions()
+    if err != nil {
+        return err
+    }
+
+    backupLog.Info("Getting PostBackup actions")
+    postBackupActions, err := pluginManager.GetPostBackupActions()
+    if err != nil {
+        return err
+    }
+```
+
+#### Pre Backup Actions Plugins
+
+Calling the Pre Backup actions:
+
+```go
+    for _, preBackupAction := range preBackupActions {
+        err := preBackupAction.Execute(backup.Backup)
+        if err != nil {
+            backup.Backup.Status.Phase = velerov1api.BackupPhaseFailedPreBackupActions
+            return err
+        }
+    }
+```
+
+#### Post Backup Actions Plugins
+
+Calling the Post Backup actions:
+
+```go
+    for _, postBackupAction := range postBackupActions {
+        err := postBackupAction.Execute(backup.Backup)
+        if err != nil {
+            postBackupLog.Error(err)
+        }
+    }
+```
+
+#### Getting Pre/Post Restore Actions
+
+Getting Actions on `restore_controller.go` in `runValidatedRestore`:
+
+```go
+
+    restoreLog.Info("Getting PreRestore actions")
+    preRestoreActions, err := pluginManager.GetPreRestoreActions()
+    if err != nil {
+        return errors.Wrap(err, "error getting pre-restore actions")
+    }
+
+    restoreLog.Info("Getting PostRestore actions")
+    postRestoreActions, err := pluginManager.GetPostRestoreActions()
+    if err != nil {
+        return errors.Wrap(err, "error getting post-restore actions")
+    }
+```
+
+#### Pre Restore Actions Plugins
+
+Calling the Pre Restore actions:
+
+```go
+    for _, preRestoreAction := range preRestoreActions {
+        err := preRestoreAction.Execute(restoreReq.Restore)
+        if err != nil {
+            restoreReq.Restore.Status.Phase = velerov1api.RestorePhaseFailedPreRestoreActions
+            return errors.Wrap(err, "error executing pre-restore action")
+        }
+    }
+```
+
+#### Post Restore Actions Plugins
+
+Calling the Post Restore actions:
+
+```go
+    for _, postRestoreAction := range postRestoreActions {
+        err := postRestoreAction.Execute(restoreReq.Restore)
+        if err != nil {
+            postRestoreLog.Error(err.Error())
+        }
+    }
+```
+
+### Giving the User the Option to Skip the Execution of the Plugins
+
+Velero plugins are loaded as init containers. If plugins are unloaded, they trigger a restart of the Velero controller.
+Not mentioning if one plugin does get loaded for any reason (i.e., docker hub image pace limit), Velero does not start.
+In other words, the constant load/unload of plugins can disrupt the Velero controller, and they cannot be the only method to run the actions from these plugins selectively.
+As part of this proposal, we want to give the velero user the ability to skip the execution of the plugins via annotations on the Velero CR backup and restore objects.
+If one of these exists, the given plugin, referenced below as `plugin-name`, will be skipped.
+
+Backup Object Annotations:
+
+```
+   <plugin-name>/prebackup=skip
+   <plugin-name>/postbackup=skip
+```
+
+Restore Object Annotations:
+
+```
+   <plugin-name>/prerestore=skip
+   <plugin-name>/postrestore=skip
+```
+
+## Alternatives Considered
+
+An alternative to these plugin hooks is to implement all the pre/post backup/restore logic _outside_ Velero.
+In this case, one would need to write an external controller that works similar to what [Konveyor Crane](https://github.com/konveyor/mig-controller/blob/master/pkg/controller/migmigration/quiesce.go) does today when quiescing applications.
+We find this a viable way, but we think that Velero users can benefit from Velero having greater embedded capabilities, which will allow users to write or load plugins extensions without relying on an external components.
+
+## Security Considerations
+
+The plugins will only be invoked if loaded per a user's discretion.
+It is recommended to check security vulnerabilities before execution.
+
+## Compatibility
+
+In terms of backward compatibility, this design should stay compatible with most Velero installations that are upgrading.
+If plugins are not present, then the backup/restore process should proceed the same way it worked before their inclusion.
+
+## Implementation
+
+The implementation dependencies are roughly in the order as they are described in the [Detailed Design](#detailed-design) section.
+
+## Open Issues

design/plugin-versioning.md

@@ -0,0 +1,292 @@
+# Plugin Versioning
+
+## Abstract
+This proposal outlines an approach to support versioning of Velero's plugin APIs to enable changes to those APIs.
+It will allow for backwards compatible changes to be made, such as the addition of new plugin methods, but also backwards incompatible changes such as method removal or method signature changes.
+
+
+## Background
+When changes are made to Velero’s plugin APIs, there is no mechanism for the Velero server to communicate the version of the API that is supported, or for plugins to communicate what version they implement.
+This means that any modification to a plugin API is a backwards incompatible change as it requires all plugins which implement the API to update and implement the new method.
+
+There are several components involved to use plugins within Velero.
+From the perspective of the core Velero codebase, all plugin kinds (e.g. `ObjectStore`, `BackupItemAction`) are defined by a single API interface and all interactions with plugins are managed by a plugin manager which provides an implementation of the plugin API interface for Velero to use.
+
+Velero communicates with plugins via gRPC.
+The core Velero project provides a framework (using the [go-plugin project](https://github.com/hashicorp/go-plugin)) for plugin authors to use to implement their plugins which manages the creation of gRPC servers and clients.
+Velero plugins import the Velero plugin library in order to use this framework.
+When a change is made to a plugin API, it needs to be made to the Go interface used by the Velero codebase, and also to the rpc service definition which is compiled to form part of the framework.
+As each plugin kind is defined by a single interface, when a plugin imports the latest version of the Velero framework, it will need to implement the new APIs in order to build and run successfully.
+If a plugin does not use the latest version of the framework, and is used with a newer version of Velero that expects the plugin to implement those methods, this will result in a runtime error as the plugin is incompatible.
+
+With this proposal, we aim to break this coupling and introduce plugin API versions.
+
+## Scenarios to Support
+The following describes interactions between Velero and its plugins that will be supported with the implementation of this proposal.
+For the purposes of this list, we will refer to existing Velero and plugin versions as `v1` and all following versions as version `n`.
+
+Velero client communicating with plugins or plugin client calling other plugins:
+
+- Version `n` client will be able to communicate with Version `n` plugin
+- Version `n` client will be able to communicate with all previous versions of the plugin (Version `n-1` back to `v1`)
+
+Velero plugins importing Velero framework:
+- `v1` plugin built against Version `n` Velero framework
+    - A plugin may choose to only implement a `v1` API, but it must be able to be built using Version `n` of the Velero framework
+
+
+## Goals
+
+- Allow plugin APIs to change without requiring all plugins to implement the latest changes (even if they upgrade the version of Velero that is imported)
+- Allow plugins to choose which plugin versions they support and enable them to support multiple versions
+- Support breaking changes in the plugin APIs such as method removal or method signature changes
+- Establish a design process for modifying plugin APIs such as method addition and removal and signature changes
+- Establish a process for newer Velero clients to use older versions of a plugin API through adaptation
+
+## Non Goals
+
+- Change how plugins are managed or added
+- Allow older plugin clients to communicate with new versions of plugins
+
+## High-Level Design
+
+With each change to a plugin API, a new version of the plugin interface and the proto service definition will be created which describes the new plugin API.
+The plugin framework will be adapted to allow these new plugin versions to be registered.
+Plugins can opt to implement any or all versions of an API, however Velero will always attempt to use the latest version, and the plugin management will be modified to adapt earlier versions of a plugin to be compatible with the latest API where possible.
+Under the existing plugin framework, any new plugin version will be treated as a new plugin with a new kind.
+The plugin manager (which provides implementations of a plugin to Velero) will include an adapter layer which will manage the different versions and provide the adaptation for versions which do not implement the latest version of the plugin API.
+Providing an adaptation layer enables Velero and other plugin clients to use an older version of a plugin if it can be safely adapted.
+As the plugins will be able to introduce backwards incompatible changes, it will _not_ be possible for older version of Velero to use plugins which only support the latest versions of the plugin APIs.
+
+Although adding new rpc methods to a service is considered a backwards compatible change within gRPC, due to the way the proto definitions are compiled and included in the framework used by plugins, this will require every plugin to implement the new methods.
+Instead, we are opting to treat the addition of a method to an API as one requiring versioning.
+
+The addition of optional fields to existing structs which are used as parameters to or return values of API methods will not be considered as a change requiring versioning.
+These kinds of changes do not modify method signatures and have been safely made in the past with no impact on existing plugins.
+
+## Detailed Design
+
+The following areas will need to be adapted to support plugin versioning.
+
+### Plugin Interface Definitions
+
+To provide versioned plugins, any change to a plugin interface (method addition, removal, or signature change) will require a new versioned interface to be created.
+Currently, all plugin interface definitions reside in `pkg/plugin/velero` in a file corresponding to their plugin kind.
+These files will be rearranged to be grouped by kind and then versioned: `pkg/plugin/velero/<plugin_kind>/<version>/`.
+
+The following are examples of how each change may be treated:
+
+#### Complete Interface Change
+If the entire `ObjectStore` interface is being changed such that no previous methods are being included, a file would be added to `pkg/plugin/velero/objectstore/v2/` and would contain the new interface definition:
+
+```
+type ObjectStore interface {
+	// Only include new methods that the new API version will support
+	
+	NewMethod()
+	// ...
+}
+```
+
+#### Method Addition
+If a method is being added to the `ObjectStore` API, a file would be added to `pkg/plugin/velero/objectstore/v2/` and may contain a new API definition as follows:
+
+```
+import "github.com/vmware-tanzu/velero/pkg/plugin/velero/objectstore/v1"
+
+type ObjectStore interface {
+    // Import all the methods from the previous version of the API if they are to be included as is
+    v1.ObjectStore
+
+    // Provide definitions of any new methods
+    NewMethod()
+```
+
+#### Method Removal
+If a method is being removed from the `ObjectStore` API, a file would be added to `pkg/plugin/velero/objectstore/v2/` and may contain a new API definition as follows:
+
+```
+type ObjectStore interface {
+	// Methods which are required from the previous API version must be included, for example
+	Init(config)
+	PutObject(bucket, key, body)
+    // ...
+
+    // Methods which are to be removed are not included
+```
+
+#### Method Signature modification
+If a method signature in the `ObjectStore` API is being modified, a file would be added to `pkg/plugin/velero/objectstore/v2/` and may contain a new API definition as follows:
+
+```
+type ObjectStore interface {
+	// Methods which are required from the previous API version must be included, for example
+	Init(config)
+	PutObject(bucket, key, body)
+    // ...	
+
+    // Provide new definitions for methods which are being modified
+	List(bucket, prefix, newParameter)
+	
+}
+```
+
+### Proto Service Definitions
+
+The proto service definitions of the plugins will also be versioned and arranged by their plugin kind.
+Currently, all the proto definitions reside under `pkg/plugin/proto` in a file corresponding to their plugin kind.
+These files will be rearranged to be grouped by kind and then versioned: `pkg/plugin/proto/<plugin_kind>/<version>`,
+except for the current v1 plugins. Those will remain in their current package/location for backwards compatibility.
+This will allow plugin images built with earlier versions of velero to work with the latest velero (for v1 plugins
+only). The go_package option will be added to all proto service definitions to allow the proto compilation script
+to place the generated go code for each plugin api version in the proper go package directory.
+
+It is not possible to import an existing proto service into a new one, so any methods will need to be duplicated across versions if they are required by the new version.
+The message definitions can be shared however, so these could be extracted from the service definition files and placed in a file that can be shared across all versions of the service.
+
+### Plugin Framework
+
+To allow plugins to register which versions of the API they implement, the plugin framework will need to be adapted to accept new versions.
+Currently, the plugin manager stores a [`map[string]RestartableProcess`](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/clientmgmt/manager.go#L69), where the string key is the binary name for the plugin process (e.g. "velero-plugin-for-aws").
+Each `RestartableProcess` contains a [`map[kindAndName]interface{}`](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/clientmgmt/restartable_process.go#L60) which represents each of the unique plugin implementations provided by that binary.
+[`kindAndName`](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/clientmgmt/registry.go#L42) is a struct which combines the plugin kind (`ObjectStore`, `VolumeSnapshotter`) and the plugin name ("velero.io/aws", "velero.io/azure").
+
+Each plugin version registration must be unique (to allow for multiple versions to be implemented within the same plugin binary).
+This will be achieved by adding a specific registration method for each version to the Server interface in the plugin framework.
+For example, if adding a V2 `RestoreItemAction` plugin, the Server interface would be modified to add the `RegisterRestoreItemActionV2` method.
+This would require [adding a new plugin Kind const](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/framework/plugin_kinds.go#L28-L46) to represent the new plugin version, e.g. `PluginKindRestoreItemActionV2`.
+It also requires the creation of a new implementation of the go-plugin interface ([example](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/framework/object_store.go)) to support that version and use the generated gRPC code from the proto definition (including a client and server implementation).
+The Server will also need to be adapted to recognize this new plugin Kind and to serve the new implementation.
+
+Existing plugin Kind consts and registration methods will be left unchanged and will correspond to the current version of the plugin APIs (assumed to be v1).
+
+### Plugin Manager
+
+The plugin manager is responsible for managing the lifecycle of plugins.
+It provides an interface which is used by Velero to retrieve an instance of a plugin kind with a specific name (e.g. `ObjectStore` with the name "velero.io/aws").
+The manager contains a registry of all available plugins which is populated during the main Velero server startup.
+When the plugin manager is requested to provide a particular plugin, it checks the registry for that plugin kind and name.
+If it is available in the registry, the manager retrieves a `RestartableProcess` for the plugin binary, creating it if it does not already exist.
+That `RestartableProcess` is then used by individual restartable implementations of a plugin kind (e.g. `restartableObjectStore`, `restartableVolumeSnapshotter`).
+
+As new plugin versions are added, the plugin manager will be modified to always retrieve the latest version of a plugin kind.
+This is to allow the remainder of the Velero codebase to assume that it will always interact with the latest version of a plugin.
+If the latest version of a plugin is not available, it will attempt to fall back to previous versions and use an implementation adapted to the latest version if available.
+It will be up to the author of new plugin versions to determine whether a previous version of a plugin can be adapted to work with the interface of the new version.
+
+For each plugin kind, a new `Restartable<PluginKind>` struct will be introduced which will contain the plugin Kind and a function, `Get`, which will instantiate a restartable instance of that plugin kind and perform any adaptation required to make it compatible with the latest version.
+For example, `RestartableObjectStore` or `RestartableVolumeSnapshotter`.
+For each restartable plugin kind, a new function will be introduced which will return a slice of `Restartable<PluginKind>` objects, sorted by version in descending order.
+
+The manager will iterate through the list of `Restartable<PluginKind>`s and will check the registry for the given plugin kind and name.
+If the requested version is not found, it will skip and continue to iterate, attempting to fetch previous versions of the plugin kind.
+Once the requested version is found, the `Get` function will be called, returning the restartable implementation of the latest version of that plugin Kind.
+
+```
+type RestartableObjectStore struct {
+	kind framework.PluginKind
+	
+	// Get returns a restartable ObjectStore for the given name and process, wrapping if necessary
+	Get func(name string, restartableProcess RestartableProcess) v2.ObjectStore
+}
+
+func (m *manager) restartableObjectStores() []RestartableObjectStore {
+	return []RestartableObjectStore{
+		{
+			kind: framework.PluginKindObjectStoreV2,
+			Get: newRestartableObjectStoreV2,
+		},
+		{
+			kind: framework.PluginKindObjectStore,
+			Get: func(name string, restartableProcess RestartableProcess) v2.ObjectStore {
+				// Adapt the existing restartable v1 plugin to be compatible with the v2 interface
+				return newAdaptedV1ObjectStore(newRestartableObjectStore(name, restartableProcess))
+			},
+		},
+	}
+}
+
+// GetObjectStore returns a restartableObjectStore for name.
+func (m *manager) GetObjectStore(name string) (v2.ObjectStore, error) {
+	name = sanitizeName(name)
+
+	for _, restartableObjStore := range m.restartableObjectStores() {
+		restartableProcess, err := m.getRestartableProcess(restartableObjStore.kind, name)
+		if err != nil {
+			// Check if plugin was not found
+			if errors.Is(err, &pluginNotFoundError{}) {
+				continue
+			}
+			return nil, err
+		}
+		return restartableObjStore.Get(name, restartableProcess), nil
+	}
+
+	return nil, fmt.Errorf("unable to get valid ObjectStore for %q", name)
+}
+```
+
+If the previous version is not available, or can not be adapted to the latest version, it should not be included in the `restartableObjectStores` slice.
+This will result in an error being returned as is currently the case when a plugin implementation for a particular kind and provider can not be found.
+
+There are situations where it may be beneficial to check at the point where a plugin API call is made whether it implements a specific version of the API.
+This is something that can be addressed with future amendments to this design, however it does not seem to be necessary at this time.
+
+#### Plugin Adaptation
+
+When a new plugin API version is being proposed, it will be up to the author and the maintainer team to determine whether older versions of an API can be safely adapted to the latest version.
+An adaptation will implement the latest version of the plugin API interface but will use the methods from the version that is being adapted.
+In cases where the methods signatures remain the same, the adaptation layer will call through to the same method in the version being adapted.
+
+Examples where an adaptation may be safe:
+- A method signature is being changed to add a new parameter but the parameter could be optional (for example, adding a context parameter). The adaptation could call through to the method provided in the previous version but omit the parameter.
+- A method signature is being changed to remove a parameter, but it is safe to pass a default value to the previous version. The adaptation could call through to the method provided in the previous version but use a default value for the parameter.
+- A new method is being added but does not impact any existing behaviour of Velero (for example, a new method which will allow Velero to [wait for additional items to be ready](https://github.com/vmware-tanzu/velero/blob/main/design/wait-for-additional-items.md)). The adaptation would return a value which allows the existing behaviour to be performed.
+- A method is being deleted as it is no longer used. The adaptation would call through to any methods which are still included but would omit the deleted method in the adaptation.
+
+Examples where an adaptation may not be safe:
+- A new method is added which is used to provide new critical functionality in Velero. If this functionality can not be replicated using existing plugin methods in previous API versions, this should not be adapted and instead the plugin manager should return an error indicating that the plugin implementation can not be found.
+
+### Restartable Plugin Process
+
+As new versions of plugins are added, new restartable implementations of plugins will also need to be created.
+These are currently located within "pkg/plugin/clientmgmt" but will be rearranged to be grouped by kind and version like other plugin files.
+
+## Versioning Considerations
+
+It should be noted that if changes are being made to a plugin's API, it will only be necessary to bump the API version once within a release cycle, regardless of how many changes are made within that cycle.
+This is because the changes will only be available to consumers when they upgrade to the next minor version of the Velero library.
+New plugin API versions will not be introduced or backported to patch releases.
+
+Once a new minor or major version of Velero has been released however, any further changes will need to follow the process above and use a new API version.
+
+## Alternatives Considered
+
+### Relying on gRPC’s backwards compatibility when adding new methods
+
+One approach for adapting the plugin APIs would have been to rely on the fact that adding methods to gRPC services is a backwards compatible change.
+This approach would allow older clients to communicate with newer plugins as the existing interface would still be provided.
+This was considered but ruled out as our current framework would require any plugin that recompiles using the latest version of the framework to adapt to the new version.
+Also, without specific versioned interfaces, it would require checking plugin implementations at runtime for the specific methods that are supported.
+
+## Compatibility
+
+This design doc aims to allow plugin API changes to be made in a manner that may provide some backwards compatibility.
+Older versions of Velero will not be able to make use of new plugin versions however may continue to use previous versions of a plugin API if supported by the plugin.
+
+All compatibility concerns are addressed earlier in the document.
+
+## Implementation
+
+This design document primarily outlines an approach to allow future plugin API changes to be made.
+However, there are changes to the existing code base that will be made to allow plugin authors to more easily propose and introduce changes to these APIs.
+
+* Plugin interface definitions (currently in `pkg/plugin/velero`) will be rearranged to be grouped by kind and then versioned: `pkg/plugin/velero/<plugin_kind>/<version>/`.
+* Proto definitions (currently in `pkg/plugin/proto`) will be rearranged to be grouped by kind and then versioned: `pkg/plugin/proto/<plugin_kind>/<version>`.
+  * This will also require changes to the `make update` build task to correctly find the new proto location and output to the versioned directories.
+
+It is anticipated that changes to the plugin APIs will be made as part of the 1.9 release cycle.
+To assist with this work, an additional follow-up task to the ones listed above would be to prepare a V2 version of each of the existing plugins.
+These new versions will not yet provide any new API methods but will provide a layout for new additions to be made
+
+## Open Issues

design/secrets.md

@@ -8,15 +8,19 @@ This makes it so switching from one plugin to another necessitates overriding th
 
 - To allow Velero to create and store multiple secrets for provider credentials, even multiple credentials for the same provider
 - To improve the UX for configuring the velero deployment with multiple plugins/providers.
+- Enable use cases such as AWS volume snapshots w/Minio as the object storage
+- Continue to support use cases where multiple Backup Storage Locations are in use simultaneously 
+    - `velero backup logs` while backup/restore is running
+- Handle changes in configuration while operations are happening as well as they currently are
 
 ## Non Goals
 
 - To make any change except what's necessary to handle multiple credentials
-- To allow multiple credentials for or change the UX for node-based authentication (e.g. AWS IAM, GCP Workload Identity, Azure AAD Pod Identity).
+- To allow multiple credentials for or change the UX for node-based authentication (e.g. AWS IAM, GCP Workload Identity, Azure AAD Pod Identity).  Node-based authentication will not allow cases such as a mix of AWS snapshots with Minio object storage.
 
 ## Design overview
 
-Instead of one credential per Velero deployment, multiple credentials can be added and used with different BSLs VSLs.
+Instead of one credential per Velero deployment, multiple credentials can be added and used with different BSLs.
   
 There are two aspects to handling multiple credentials:
 
@@ -30,36 +34,98 @@ Each of these aspects will be discussed in turn.
 Currently, Velero creates a secret (`cloud-credentials`) during install with a single entry that contains the contents of the credentials file passed by the user.
 
 Instead of adding new CLI options to Velero to create and manage credentials, users will create their own Kubernetes secrets within the Velero namespace and reference these.
-This approach is being chosen as it allows users to directly manage Kubernetes secrets objects as they wish and it removes the need for wrapper functions to be created within Velero to manage the creation of secrets.
-An initial approach to this problem included modifying the existing `cloud-credentials` secret to add a new entry with each new set of credentials.
-It is likely that this approach would encounter problems as users added more credentials as the maximum size of Secret in Kubernetes is 1MB.
-By allowing users to create Secrets as they need to, we remove these potential limitations.
+This approach is being chosen as it allows users to directly manage Kubernetes secrets objects as they wish and it removes the need for wrapper functions to be created within Velero to manage the creation of secrets.  Separate credentials rather than combining credentials in a single secret also avoids issues with maximum size of credentials as well as update in place issues. 
 
-To enable the use of existing Kubernetes secrets, BSLs and VSLs will be modified to have a new field `Credential`.
-This field will be a [`SecretKeySelector`](https://godoc.org/k8s.io/api/core/v1#SecretKeySelector) which will enable the user to specify which key within a particular secret the BSL/VSL should use.
+To enable the use of existing Kubernetes secrets, BSLs will be modified to have a new field `Credential`.
+This field will be a [`SecretKeySelector`](https://godoc.org/k8s.io/api/core/v1#SecretKeySelector) which will enable the user to specify which key within a particular secret the BSL should use.
 
-The CLI for managing BSLs and VSLs will be modified to allow the user to set these credentials.
-Both `velero backup-location (create|set)` and `velero snapshot-location (create|set)` will have a new flag (`--credential`) to specify the secret and key within the secret to use.
+Existing BackupStorageLocationSpec definition:
+
+	// BackupStorageLocationSpec defines the desired state of a Velero BackupStorageLocation
+	type BackupStorageLocationSpec struct {
+		// Provider is the provider of the backup storage.
+		Provider string `json:"provider"`
+	
+		// Config is for provider-specific configuration fields.
+		// +optional
+		Config map[string]string `json:"config,omitempty"`
+	
+		StorageType `json:",inline"`
+	
+		// Default indicates this location is the default backup storage location.
+		// +optional
+		Default bool `json:"default,omitempty"`
+	
+		// AccessMode defines the permissions for the backup storage location.
+		// +optional
+		AccessMode BackupStorageLocationAccessMode `json:"accessMode,omitempty"`
+	
+		// BackupSyncPeriod defines how frequently to sync backup API objects from object storage. A value of 0 disables sync.
+		// +optional
+		// +nullable
+		BackupSyncPeriod *metav1.Duration `json:"backupSyncPeriod,omitempty"`
+	
+		// ValidationFrequency defines how frequently to validate the corresponding object storage. A value of 0 disables validation.
+		// +optional
+		// +nullable
+		ValidationFrequency *metav1.Duration `json:"validationFrequency,omitempty"`
+	}
+
+The following field will be added:
+
+	Credential *corev1api.SecretKeySelector `json:"credential,omitempty"`
+
+The resulting BackupStorageLocationSpec will be this:
+
+	// BackupStorageLocationSpec defines the desired state of a Velero BackupStorageLocation
+	type BackupStorageLocationSpec struct {
+		// Provider is the provider of the backup storage.
+		Provider string `json:"provider"`
+	
+		// Config is for provider-specific configuration fields.
+		// +optional
+		Config map[string]string `json:"config,omitempty"`
+	
+		// Credential contains the credential information intended to be used with this location
+		// +optional
+		Credential *corev1api.SecretKeySelector `json:"credential,omitempty"`
+	
+		StorageType `json:",inline"`
+	
+		// Default indicates this location is the default backup storage location.
+		// +optional
+		Default bool `json:"default,omitempty"`
+	
+		// AccessMode defines the permissions for the backup storage location.
+		// +optional
+		AccessMode BackupStorageLocationAccessMode `json:"accessMode,omitempty"`
+	
+		// BackupSyncPeriod defines how frequently to sync backup API objects from object storage. A value of 0 disables sync.
+		// +optional
+		// +nullable
+		BackupSyncPeriod *metav1.Duration `json:"backupSyncPeriod,omitempty"`
+	
+		// ValidationFrequency defines how frequently to validate the corresponding object storage. A value of 0 disables validation.
+		// +optional
+		// +nullable
+		ValidationFrequency *metav1.Duration `json:"validationFrequency,omitempty"`
+	}
+	
+The CLI for managing Backup Storage Locations (BSLs) will be modified to allow the user to set these credentials.
+Both `velero backup-location (create|set)` will have a new flag (`--credential`) to specify the secret and key within the secret to use.
 This flag will take a key-value pair in the format `<secret-name>=<key-in-secret>`.
 The arguments will be validated to ensure that the secret exists in the Velero namespace.
+If the Credential field is empty in a BSL, the default credentials from `cloud-credentials` will be used as they
+are currently.
 
 ### Making credentials available to plugins
 
-There are three different approaches that can be taken to provide credentials to plugin processes:
-
-1. Providing the path to the credentials file as an environment variable per plugin. This is how credentials are currently passed.
-1. Include the path to the credentials file in the `config` map passed to a plugin.
-1. Include the details of the secret in the `config` map passed to a plugin.
-
-The last two options require changes to the plugin as the plugin will need to instantiate a client using the provided credentials.
-The client libraries used by the plugins will not be able to rely on the credentials details being available in the environment as they currently do.
-
-We have selected option 2 as the approach to take which will be described below.
+The approach we have chosen is to include the path to the credentials file in the `config` map passed to a plugin.
 
 ### Including the credentials file path in the `config` map
 
-Prior to using any secret for a BSL or VSL, it will need to be serialized to disk.
-Using the details in the `Credential` field in the BSL/VSL, the contents of the Secret will be read and serialized.
+Prior to using any secret for a BSL, it will need to be serialized to disk.
+Using the details in the `Credential` field in the BSL, the contents of the Secret will be read and serialized.
 To achieve this, we will create a new package, `credentials`, which will introduce new types and functions to manage the fetching of credentials based on a `SecretKeySelector`.
 This will also be responsible for serializing the fetched credentials to a temporary directory on the Velero pod filesystem.
 
@@ -71,8 +137,8 @@ This means that any time a `BackupStore`, or other type which requires credentia
 If we instead wanted to use an unique file each time, we could work around the of multiple files being written by cleaning up the temporary files upon completion of the plugin operations, if this information is known.
 
 Once the credentials have been serialized, this path will be made available to the plugins.
-Instead of setting the necessary environment variable for the plugin process, the `config` map for the BSL/VSL will be modified to include an addiitional entry with the path to the credentials file: `credentialsFile`.
-This will be passed through when [initializing the BSL/VSL](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/velero/object_store.go#L27-L30) and it will be the responsibility of the plugin to use the passed credentials when starting a session.
+Instead of setting the necessary environment variable for the plugin process, the `config` map for the BSL will be modified to include an addiitional entry with the path to the credentials file: `credentialsFile`.
+This will be passed through when [initializing the BSL](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/velero/object_store.go#L27-L30) and it will be the responsibility of the plugin to use the passed credentials when starting a session.
 For an example of how this would affect the AWS plugin, see [this PR](https://github.com/vmware-tanzu/velero-plugin-for-aws/pull/69).
 
 The restic controllers will also need to be updated to use the correct credentials.
@@ -84,6 +150,22 @@ Currently, GCP is the only provider that relies on the existing environment vari
 For GCP, the environment variable will be overwritten with the path of the serialized secret.
 
 
+## Split credentials between VolumeSnapshotter and ObjectStore plugins
+
+One of the use cases we wish to satisfy is the ability to specify a different object store than the cloud provider offers,
+for example, using a Minio S3 object store from within AWS.  Currently the VolumeSnapshotter and the ObjectStore plugin
+share the cloud credentials.  Each backup/restore has a BackupStorageLocation associated
+with it.  The BackupStorageLocation can optionally specify the credential used by the ObjectStorePlugin and Restic daemons 
+while the cloud credential will always be used for the VolumeSnapshotter.
+
+## Velero Plugin for vSphere compatibility
+
+The vSphere plugin is implemented as a BackupItemAction and shares the credentials of the AWS plug-in for S3 access.
+The backup storage location is passed in _Backup.Spec.StorageLocation_.  Currently the plugin retrieves the S3 bucket and
+server from the BSL and creates a BackupRespositoryClaim with that and the credentials retrieved from the cloud credential.
+The plug-in will need to be modified to retrieve the credentials field from the BSL and use that credential in the
+BackupRepositoryClaim.
+
 ## Backwards compatibility
 
 For now, regardless of the approaches used above, we will still support the existing workflow.
@@ -92,15 +174,33 @@ Users will be able to set credentials during install and a secret will be create
 This secret will still be mounted into the Velero pods and the appropriate environment variables set.
 This will allow users to use versions of plugins which haven't yet been updated to use credentials directly, such as with many community created plugins.
 
-Multiple credential handling will only be used in the case where a particular BSL/VSL has been modified to use an existing secret.
+Multiple credential handling will only be used in the case where a particular BSL has been modified to use an existing secret.
 
 ## Security Considerations
 
 Although the handling of secrets will be similar to how credentials are currently managed within Velero, care must be taken to ensure that any new code does not leak the contents of secrets, for example, including them within logs.
 
+## Parallelism
+In order to support parallelism, Velero will need to be able to use multiple credentials simultaneously with the
+ObjectStore.  Currently backups are single threaded and a single BSL will be used throughout the entire backup.  The only
+existing points of parallelism are when a user downloads logs for a backup or the BackupStorageLocationReconciler 
+reconciles while a backup or restore is running.  In the current code, `download_request_controller.go` and 
+`backup_storage_location_controller.go` create a new plug-in manager and hence another ObjectStore plugin in 
+parallel with the ObjectStore plugin servicing a backup or restore (if one is running).
+
 ## Alternatives Considered
 
-As mentioned above, there were three potential approaches for providing this support.
+Three different approaches can be taken to provide credentials to plugin processes:
+
+1. Providing the path to the credentials file as an environment variable per plugin. This is how credentials are currently passed.
+1. Include the path to the credentials file in the `config` map passed to a plugin.
+1. Include the details of the secret in the `config` map passed to a plugin.
+
+The last two options require changes to the plugin as the plugin will need to instantiate a client using the provided credentials.
+The client libraries used by the plugins will not be able to rely on the credentials details being available in the environment as they currently do.
+
+We have selected option 2 as the approach to take.
+
 The approaches that were not selected are detailed below for reference.
 
 #### Providing the credentials via environment variables
@@ -110,11 +210,11 @@ Currently, there is a single secret, which is mounted into every pod deployed by
 This path is made known to all plugins through provider specific environment variables and all possible provider environment variables are set to this path.
 
 Instead of setting the environment variables for all the pods, we can modify plugin processes are created so that the environment variables are set on a per plugin process basis.
-Prior to using any secret for a BSL or VSL, it will need to be serialized to disk.
-Using the details in the `Credential` field in the BSL/VSL, the contents of the Secret will be read and serialized to a file.
+Prior to using any secret for a BSL, it will need to be serialized to disk.
+Using the details in the `Credential` field in the BSL, the contents of the Secret will be read and serialized to a file.
 Each plugin process would still have the same set of environment variables set, however the value used for each of these variables would instead be the path to the serialized secret.
 
-To set the environment variables for a plugin process, the plugin manager must be modified so that when creating an ObjectStore or VolumeSnapshotter, we pass in the entire BSL/VSL object, rather than [just the provider](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/clientmgmt/manager.go#L132-L158).
+To set the environment variables for a plugin process, the plugin manager must be modified so that when creating an ObjectStore, we pass in the entire BSL object, rather than [just the provider](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/clientmgmt/manager.go#L132-L158).
 The plugin manager currently stores a map of [plugin executables to an associated `RestartableProcess`](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/clientmgmt/manager.go#L59-L70).
 New restartable processes are created only [with the executable that the process would run](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/clientmgmt/manager.go#L122).
 This could be modified to also take the necessary environment variables so that when [underlying go-plugin process is created](https://github.com/vmware-tanzu/velero/blob/main/pkg/plugin/clientmgmt/client_builder.go#L78), these environment variables could be provided and would be set on the plugin process.

design/unified-repo-and-kopia-integration/unified-repo-and-kopia-integration.md

@@ -0,0 +1,483 @@
+# Unified Repository & Kopia Integration Design
+
+## Glossary & Abbreviation
+
+**BR**: Backup & Restore  
+**Backup Storage**: The storage that meets BR requirements, for example, scalable, durable, cost-effective, etc., therefore, Backup Storage is usually implemented as Object storage or File System storage, it may be on-premise or in cloud. Backup Storage is not BR specific necessarily, so it usually doesn’t provide most of the BR related features. On the other hand, storage vendors may provide BR specific storages that include some BR features like deduplication, compression, encryption, etc. For a standalone BR solution (i.e. Velero), the Backup Storage is not part of the solution, it is provided by users, so the BR solution should not assume the BR related features are always available from the Backup Storage.  
+**Backup Repository**: Backup repository is layered between BR data movers and Backup Storage to provide BR related features. Backup Repository is a part of BR solution, so generally, BR solution by default leverages the Backup Repository to provide the features because Backup Repository is always available; when Backup Storage provides duplicated features, and the latter is more beneficial (i.e., performance is better), BR solution should have the ability to opt to use the Backup Storage’s implementation.  
+**Data Mover**: The BR module to read/write data from/to workloads, the aim is to eliminate the differences of workloads.  
+**TCO**: Total Cost of Ownership. This is a general criteria for products/solutions, but also means a lot for BR solutions. For example, this means what kind of backup storage (and its cost) it requires, the retention policy of backup copies, the ways to remove backup data redundancy, etc.   
+**RTO**: Recovery Time Objective. This is the duration of time that users’ business can recover after a disaster.  
+
+## Background
+
+As a Kubernetes BR solution, Velero is pursuing the capability to back up data from the volatile and limited production environment into the durable, heterogeneous and scalable backup storage. This relies on two parts:  
+
+- Move data from various production workloads. The data mover has this role. Depending on the type of workload, Velero needs different data movers. For example, file system data mover, block data mover, and data movers for specific applications. At present, Velero supports moving file system data from PVs through Restic, which plays the role of the File System Data Mover.  
+- Persist data in backup storage. For a BR solution, this is the responsibility of the backup repository. Specifically, the backup repository is required to:  
+  - Efficiently save data so as to reduce TCO. For example, deduplicate and compress the data before saving it
+  - Securely save data so as to meet security criteria. For example, encrypt the data on rest, make the data immutable after backup, and detect/protect from ransomware
+  - Efficiently retrieve data during restore so as to meet RTO. For example, restore a small unit of data or data associated with a small span of time 
+  - Effectively manage data from all kinds of data movers in all kinds of backup storage. This means 2 things: first, apparently, backup storages are different from each other; second, some data movers may save quite different data from others, for example, some data movers save a portion of the logical object for each backup and need to visit and manage the portions as an entire logic object, aka. incremental backup. The backup repository needs to provide unified functionalities to eliminate the differences from the both ends
+  - Provide scalabilities so that users could assign resources (CPU, memory, network, etc.) in a flexible way to the backup repository since backup repository contains resource consuming modules
+
+At present, Velero provides some of these capabilities by leveraging Restic (e.g., deduplication and  encryption on rest). This means that in addition to being a data mover for file system level data, Restic also plays the role of a backup repository, albeit one that is incomplete and limited:  
+
+- Restic is an inseparable unit made up of a file system data mover and a repository. This means that the repository capabilities are only available for Restic file system backup. We cannot provide the same capabilities to other data movers using Restic.
+- The backup storage Velero supports through our Restic backup path depends on the storage Restic supports. As a result, if there is a requirement to introduce backup storage that Restic doesn’t support, we have no way to make it.
+- There is no way to enhance or extend the repository capabilities, because of the same reason – Restic is an inseparable unit, we cannot insert one or more customized layers to make the enhancements and extensions.
+
+Moreover, as reflected by user-reported issues, Restic seems to have many performance issues on both the file system data mover side and the repository side.  
+
+On the other hand, based on a previous analysis and testing, we found that Kopia has better performance, with more features and more suitable to fulfill Velero’s repository targets (Kopia’s architecture divides modules more clearly according to their responsibilities, every module plays a complete role with clear interfaces. This makes it easier to take individual modules to Velero without losing critical functionalities).  
+
+## Goals
+
+- Define a Unified Repository Interface that various data movers could interact with. This is for below purposes:
+  - All kinds of data movers acquire the same set of backup repository capabilities very easily
+  - Provide the possibility to plugin in different backup repositories/backup storages without affecting the upper layers
+  - Provide the possibility to plugin in modules between data mover and backup repository, so as to extend the repository capabilities
+  - Provide the possibility to scale the backup repository without affecting the upper layers
+- Use Kopia repository to implement the Unified Repository
+- Use Kopia uploader as the file system data mover for Pod Volume Backup
+- Have Kopia uploader calling the Unified Repository Interface and save/retrieve data to/from the Unified Repository
+- Make Kopia uploader generic enough to move any file system data so that other data movement cases could use it
+- Use the existing logic or add new logic to manage the unified repository and Kopia uploader
+- Preserve the legacy Restic path, this is for the consideration of backward compatibility
+
+## Non-Goals
+
+- The Unified Repository supports all kinds of data movers to save logic objects into it. How these logic objects are organized for a specific data mover (for example, how a volume’s block data is organized and represented by a unified repository object) should be included in the related data mover design.
+- At present, Velero saves Kubernetes resources, backup metedata, debug logs separately. Eventually, we want to save them in the Unified Repository. How to organize these data into the Unified Repository should be included in a separate design.
+- For PodVolume BR, this design focuses on the data path only, other parts beyond the data read/write and data persistency are irrelevant and kept unchanged. 
+- Kopia uploader is made generic enough to move any file system data. How it is integrated in other cases, is irrelevant to this design. Take CSI snapshot backup for example, how the snapshot is taken and exposed to Kopia uploader should be included in the related data mover design.
+- The adanced modes of the Unified Repository, for example, backup repository/storage plugin, backup repository extension, etc. are not included in this design. We will have separate designs to cover them whenever necessary.
+
+## Architecture of Unified Repository
+
+Below shows the primary modules and their responsibilities:
+
+- Kopia uploader, as been well isolated, could move all file system data either from the production PV (as Velero’s PodVolume BR does), or from any kind of snapshot (i.e., CSI snapshot).
+- Unified Repository Interface, data movers call the Unified Repository Interface to write/read data to/from the Unified Repository.
+- Kopia repository layers, CAOS and CABS, work as the backup repository and expose the Kopia Repository interface.
+- A Kopia Repository Library works as an adapter between Unified Repository Interface and Kopia Repository interface. Specifically, it implements Unified Repository Interface and calls Kopia Repository interface.
+- At present, there is only one kind of backup repository -- Kopia Repository. If a new backup repository/storage is required, we need to create a new Library as an adapter to the Unified Repository Interface
+- At present, the Kopia Repository works as a single piece in the same process of the caller, in future, we may run its CABS into a dedicated process or node.
+- At present, we don’t have a requirement to extend the backup repository, if needed, an extra module could be added as an upper layer into the Unified Repository without changing the data movers.  
+
+Neither Kopia uploader nor Kopia Repository is invoked through CLI, instead, they are invoked through code interfaces, because we need to do lots of customizations.  
+
+The Unified Repository takes two kinds of data:
+- Unified Repository Object: This is the user's logical data, for example, files/directories, blocks of a volume, data of a database, etc.
+- Unified Repository Manifest: This could include all other data to maintain the object data, for example, snapshot information, etc.  
+
+For Unified Repository Object/Manifest, a brief guidance to data movers are as below:
+- Data movers treat the simple unit of data they recognize as an Object. For example, file system data movers treat a file or a directory as an Object; block data movers treat a volume as an Object. However, it is unnecessary that every data mover has a unique data format in the Unified Repository, to the opposite, it is recommended that data movers could share the data formats unless there is any reason not to, in this way, the data generated by one data mover could be used by other data movers.
+- Data movers don't need to care about the differences between full and incremental backups regarding the data organization. Data movers always have full views of their objects, if an object is partially written, they use the object writer's Seek function to skip the unchanged parts
+- Unified Repository may divide the data movers' logical Object into sub-objects or slices, or append internal metadata, but they are transparent to data movers 
+- Every Object has an unified identifier, in order to retrieve the Object later, data movers need to save the identifiers into the snapshot information. The snapshot information is saved as a Manifest.
+- Manifests could hold any kind of small piece data in a K-V manner. Inside the backup repository, these kinds of data may be processed differently from Object data, but it is transparent to data movers.
+- A Manifest also has an unified identifier, the Unified Repository provides the capabilities to list all the Manifests or a specified Manifest by its identifier, or a specified Manifest by its name, or a set of Manifests by their labels.
+ 
+![A Unified Repository Architecture](unified-repo.png)
+
+Velero by default uses the Unified Repository for all kinds of data movement, it is also able to integrate with other data movement paths from any party, for any purpose. Details are concluded as below:
+
+- Built-in Data Path: this is the default data movement path, which uses Velero built-in data movers to backup/restore workloads, the data is written to/read from the Unified Repository.
+- Data Mover Replacement: Any party could write its own data movers and plug them into Velero. Meanwhile, these plugin data movers could also write/read data to/from Velero’s Unified Repository so that these data movers could expose the same capabilities that provided by the Unified Repository. In order to do this, the data mover providers need to call the Unified Repository Interface from inside their plugin data movers.
+- Data Path Replacement: Some vendors may already have their own data movers and backup repository and they want to replace Velero’s entire data path (including data movers and backup repository). In this case, the providers only need to implement their plugin data movers, all the things downwards are a black box to Velero and managed by providers themselves (including API call, data transport, installation, life cycle management, etc.). Therefore, this case is out of the scope of Unified Repository.  
+![A Scope](scope.png)  
+
+# Detailed Design
+
+## The Unified Repository Interface  
+Below are the definitions of the Unified Repository Interface. All the functions are synchronization functions.   
+```
+// BackupRepoService is used to initialize, open or maintain a backup repository
+type BackupRepoService interface {
+	// Init creates a backup repository or connect to an existing backup repository.
+	// repoOption: option to the backup repository and the underlying backup storage.
+	// createNew: indicates whether to create a new or connect to an existing backup repository.
+	Init(ctx context.Context, repoOption RepoOptions, createNew bool) error
+
+	// Open opens an backup repository that has been created/connected.
+	// repoOption: options to open the backup repository and the underlying storage.
+	Open(ctx context.Context, repoOption RepoOptions) (BackupRepo, error)
+
+	// Maintain is periodically called to maintain the backup repository to eliminate redundant data.
+	// repoOption: options to maintain the backup repository.
+	Maintain(ctx context.Context, repoOption RepoOptions) error
+
+	// DefaultMaintenanceFrequency returns the defgault frequency of maintenance, callers refer this
+	// frequency to maintain the backup repository to get the best maintenance performance
+	DefaultMaintenanceFrequency() time.Duration
+}
+
+// BackupRepo provides the access to the backup repository
+type BackupRepo interface {
+	// OpenObject opens an existing object for read.
+	// id: the object's unified identifier.
+	OpenObject(ctx context.Context, id ID) (ObjectReader, error)
+
+	// GetManifest gets a manifest data from the backup repository.
+	GetManifest(ctx context.Context, id ID, mani *RepoManifest) error
+
+	// FindManifests gets one or more manifest data that match the given labels
+	FindManifests(ctx context.Context, filter ManifestFilter) ([]*ManifestEntryMetadata, error)
+
+	// NewObjectWriter creates a new object and return the object's writer interface.
+	// return: A unified identifier of the object on success.
+	NewObjectWriter(ctx context.Context, opt ObjectWriteOptions) ObjectWriter
+
+	// PutManifest saves a manifest object into the backup repository.
+	PutManifest(ctx context.Context, mani RepoManifest) (ID, error)
+
+	// DeleteManifest deletes a manifest object from the backup repository.
+	DeleteManifest(ctx context.Context, id ID) error
+
+	// Flush flushes all the backup repository data
+	Flush(ctx context.Context) error
+
+	// Time returns the local time of the backup repository. It may be different from the time of the caller
+	Time() time.Time
+
+	// Close closes the backup repository
+	Close(ctx context.Context) error
+
+type ObjectReader interface {
+	io.ReadCloser
+	io.Seeker
+
+	// Length returns the logical size of the object
+	Length() int64
+}
+ 
+type ObjectWriter interface {
+	io.WriteCloser
+
+	// Seeker is used in the cases that the object is not written sequentially
+	io.Seeker
+
+	// Checkpoint is periodically called to preserve the state of data written to the repo so far.
+	// Checkpoint returns a unified identifier that represent the current state.
+	// An empty ID could be returned on success if the backup repository doesn't support this.
+	Checkpoint() (ID, error)
+
+	// Result waits for the completion of the object write.
+	// Result returns the object's unified identifier after the write completes.
+	Result() (ID, error)
+}
+```
+ 
+Some data structure & constants used by the interfaces:  
+```
+type RepoOptions struct {
+	// StorageType is a repository specific string to identify a backup storage, i.e., "s3", "filesystem"
+	StorageType string
+	// RepoPassword is the backup repository's password, if any
+	RepoPassword string
+	// ConfigFilePath is a custom path to save the repository's configuration, if any
+	ConfigFilePath string
+	// GeneralOptions takes other repository specific options
+	GeneralOptions map[string]string
+	// StorageOptions takes storage specific options
+	StorageOptions map[string]string
+	// Description is a description of the backup repository/backup repository operation.
+	// It is for logging/debugging purpose only and doesn't control any behavior of the backup repository.
+	Description string
+}
+
+// ObjectWriteOptions defines the options when creating an object for write
+type ObjectWriteOptions struct {
+	FullPath    string // Full logical path of the object
+	DataType    int    // OBJECT_DATA_TYPE_*
+	Description string // A description of the object, could be empty
+	Prefix      ID     // A prefix of the name used to save the object
+	AccessMode  int    // OBJECT_DATA_ACCESS_*
+	BackupMode  int    // OBJECT_DATA_BACKUP_*
+}
+
+const (
+	// Below consts descrbe the data type of one object.
+	// Metadata: This type describes how the data is organized.
+	// For a file system backup, the Metadata describes a Dir or File.
+	// For a block backup, the Metadata describes a Disk and its incremental link.
+	ObjectDataTypeUnknown  int = 0
+	ObjectDataTypeMetadata int = 1
+	ObjectDataTypeData     int = 2
+
+	// Below consts defines the access mode when creating an object for write
+	ObjectDataAccessModeUnknown int = 0
+	ObjectDataAccessModeFile    int = 1
+	ObjectDataAccessModeBlock   int = 2
+
+	ObjectDataBackupModeUnknown int = 0
+	ObjectDataBackupModeFull    int = 1
+	ObjectDataBackupModeInc     int = 2
+)
+
+// ManifestEntryMetadata is the metadata describing one manifest data
+type ManifestEntryMetadata struct {
+	ID      ID                // The ID of the manifest data
+	Length  int32             // The data size of the manifest data
+	Labels  map[string]string // Labels saved together with the manifest data
+	ModTime time.Time         // Modified time of the manifest data
+}
+ 
+type RepoManifest struct {
+	Payload  interface{}            // The user data of manifest
+	Metadata *ManifestEntryMetadata // The metadata data of manifest
+}
+ 
+type ManifestFilter struct {
+    Labels map[string]string
+}
+```  
+
+## Workflow
+
+### Backup & Restore Workflow
+
+We preserve the bone of the existing BR workflow, that is:
+
+- Still use the Velero Server pod and VeleroNodeAgent daemonSet (originally called Restic daemonset) pods to hold the corresponding controllers and modules
+- Still use the Backup/Restore CR and BackupRepository CR (originally called ResticRepository CR) to drive the BR workflow
+
+The modules in gray color in below diagram are the existing modules and with no significant changes.  
+In the new design, we will have separate and independent modules/logics for backup repository and uploader (data mover), specifically:
+
+- Repository Provider provides functionalities to manage the backup repository. For example, initialize a repository, connect to a repository, manage the snapshots in the repository, maintain a repository, etc.
+- Uploader Provider provides functionalities to run a backup or restore.
+
+The Repository Provider and Uploader Provider use options to choose the path --- legacy path vs. new path (Kopia uploader + Unified Repository). Specifically, for legacy path, Repository Provider will manage Restic Repository only, otherwise, it manages Unified Repository only; for legacy path, Uploader Provider calls Restic to do the BR, otherwise, it calls Kopia uploader to do the BR.  
+
+In order to manage Restic Repository, the Repository Provider calls Restic Repository Provider, the latter invokes the existing Restic CLIs.  
+In order to manage Unified Repository, the Repository Provider calls Unified Repository Provider, the latter calls the Unified Repository module through the udmrepo.BackupRepoService interface. It doesn’t know how the Unified Repository is implemented necessarily.  
+In order to use Restic to do BR, the Uploader Provider calls Restic Uploader Provider, the latter invokes the existing Restic CLIs.  
+In order to use Kopia to do BR, the Uploader Provider calls Kopia Uploader Provider, the latter do the following things:
+
+- Call Unified Repository through the udmrepo.BackupRepoService interface to open the unified repository for read/write. Again, it doesn’t know how the Unified Repository is implemented necessarily. It gets a BackupRepo’s read/write handle after the call succeeds
+- Wrap the BackupRepo handle into a Kopia Shim which implements Kopia Repository interface
+- Call the Kopia Uploader. Kopia Uploader is a Kopia module without any change, so it only understands Kopia Repository interface
+- Kopia Uploader starts to backup/restore the corresponding PV’s file system data and write/read data to/from the provided Kopia Repository implementation, that is, Kopia Shim here
+- When read/write calls go into Kopia Shim, it in turn calls the BackupRepo handle for read/write
+- Finally, the read/write calls flow to Unified Repository module
+
+The Unified Repository provides all-in-one functionalities of a Backup Repository and exposes the Unified Repository Interface. Inside, Kopia Library is an adapter for Kopia Repository to translate the Unified Repository Interface calls to Kopia Repository interface calls.  
+Both Kopia Shim and Kopia Library rely on Kopia Repository interface, so we need to have some Kopia version control. We may need to change Kopia Shim and Kopia Library when upgrading Kopia to a new version and the Kopia Repository interface has some changes in the new version.
+![A BR Workflow](br-workflow.png)  
+The modules in blue color in below diagram represent the newly added modules/logics or reorganized logics.  
+The modules in yellow color in below diagram represent the called Kopia modules without changes.  
+
+### Delete Snapshot Workflow
+The Delete Snapshot workflow follows the similar manner with BR workflow, that is, we preserve the upper-level workflows until the calls reach to BackupDeletionController, then:
+- Leverage Repository Provider to switch between Restic implementation and Unified Repository implementation in the same mechanism as BR
+- For Restic implementation, the Restic Repository Provider invokes the existing “Forget” Restic CLI
+- For Unified Repository implementation, the Unified Repository Provider calls udmrepo.BackupRepo’s DeleteManifest to delete a snapshot
+![A Snapshot Deletion Workflow](snapshot-deletion-workflow.png) 
+
+### Maintenance Workflow
+Backup Repository/Backup Storage may need to periodically reorganize its data so that it could guarantee its QOS during the long-time service. Some Backup Repository/Backup Storage does this in background automatically, so the user doesn’t need to interfere; some others need the caller to explicitly call their maintenance interface periodically. Restic and Kopia both go with the second way, that is, Velero needs to periodically call their maintenance interface.  
+Velero already has an existing workflow to call Restic maintenance (it is called “Prune” in Restic, so Velero uses the same word). The existing workflow is as follows:
+- The Prune is triggered at the time of the backup
+- When a BackupRepository CR (originally called ResticRepository CR) is created by PodVolumeBackup/Restore Controller, the BackupRepository controller checks if it reaches to the Prune Due Time, if so, it calls PruneRepo  
+- In the new design, the Repository Provider implements PruneRepo call, it uses the same way to switch between Restic Repository Provider and Unified Repository Provider, then:
+  - For Restic Repository, Restic Repository Provider invokes the existing “Prune” CLI of Restic
+  - For Unified Repository, Unified Repository Provider calls udmrepo.BackupRepoService’s Maintain function
+
+Kopia has two maintenance modes – the full maintenance and quick maintenance. There are many differences between full and quick mode, but briefly speaking, quick mode only processes the hottest data (primarily, it is the metadata and index data), so quick maintenance is much faster than full maintenance. On the other hand, quick maintenance also scatters the burden of full maintenance so that the full maintenance could finish fastly and make less impact. We will also take this quick maintenance into Velero.  
+We will add a new Due Time to Velero, finally, we have two Prune Due Time:
+- Normal Due Time: For Restic, this will invoke Restic Prune; for Unified Repository, this will invoke udmrepo.BackupRepoService’s Maintain(full) call and finally call Kopia’s full maintenance
+- Quick Due Time: For Restic, this does nothing; for Unified Repository, this will invoke udmrepo.BackupRepoService’s Maintain(quick) call and finally call Kopia’s quick maintenance  
+
+We assign different values to Normal Due Time and Quick Due Time, as a result of which, the quick maintenance happens more frequently than full maintenance.  
+![A Maintenance Workflow](maintenance-workflow.png) 
+
+### Progress Update
+Because Kopia Uploader is an unchanged Kopia module, we need to find a way to get its progress during the BR.  
+Kopia Uploader accepts a Progress interface to update rich information during the BR, so the Kopia Uploader Provider will implement a Kopia’s Progress interface and then pass it to Kopia Uploader during its initialization.  
+In this way, Velero will be able to get the progress as shown in the diagram below.  
+![A Progress Update](progress-update.png) 
+
+### Logs
+In the current design, Velero is using two unchanged Kopia modules --- the Kopia Uploader and the Kopia Repository. Both will generate debug logs during their run. Velero will collect these logs in order to aid the debug.  
+Kopia’s Uploader and Repository both get the Logger information from the current GO Context, therefore, the Kopia Uploader Provider/Kopia Library could set the Logger interface into the current context and pass the context to Kopia Uploader/Kopia Repository.  
+Velero will set Logger interfaces separately for Kopia Uploader and Kopia Repository. In this way, the Unified Repository could serve other data movers without losing the debug log capability; and the Kopia Uploader could write to any repository without losing the debug log capability.  
+Kopia’s debug logs will be written to the same log file as Velero server or VeleroNodeAgent daemonset, so Velero doesn’t need to upload/download these debug logs separately.  
+![A Debug Log for Uploader](debug-log-uploader.png) 
+![A Debug Log for Repository](debug-log-repository.png) 
+
+## Path Switch & Coexist
+As mentioned above, There will be two paths. The related controllers need to identify the path during runtime and adjust its working mode.  
+According to the requirements, path changing is fulfilled at the backup/restore level. In order to let the controllers know the path, we need to add some option values. Specifically, there will be option/mode values for path selection in two places:
+- Add the “uploader-type” option as a parameter of the Velero server. The parameters will be set by the installation. Currently the option has two values, either "restic" or "kopia" (in future, we may add other file system uploaders, then we will have more values).
+- Add a "uploaderType" value in the PodVolume Backup/Restore CR and a "repositoryType" value in the BackupRepository CR. "uploaderType" currently has two values , either "restic" or "kopia";  "repositoryType" currently has two values, either "restic" or "kopia" (in future, the Unified Repository could opt among multiple backup repository/backup storage, so there may be more values. This is a good reason that repositoryType is a multivariate flag, however, in which way to opt among the backup repository/backup storage is not covered in this PR). If the values are missing in the CRs, it by default means "uploaderType=restic" and "repositoryType=restic", so the legacy CRs are handled correctly by Restic.  
+
+The corresponding controllers handle the CRs by checking the CRs' path value. Some examples are as below:
+- The PodVolume BR controller checks the "uploaderType" value from PodVolume CRs and decide its working path
+- The BackupRepository controller checks the "repositoryType" value from BackupRepository CRs and decide its working path
+- The Backup controller that runs in Velero server checks its “uploader-type” parameter to decide the path for the Backup it is going to create and then create the PodVolume Backup CR and BackupRepository CR
+- The Restore controller checks the Backup, from which it is going to restore, for the path and then create the PodVolume Restore CR and BackupRepository CR
+
+As described above, the “uploader-type” parameter of the Velero server is only used to decide the path when creating a new Backup, for other cases, the path selection is driven by the related CRs. Therefore, we only need to add this parameter to the Velero server.  
+
+## Velero CR Name Changes
+We will change below CRs' name to make them more generic:
+- "ResticRepository" CR to "BackupRepository" CR  
+
+This means, we add a new CR type and deprecate the old one. As a result, if users upgrade from the old release, the old CRs will be orphaned, Velero will neither refer to it nor manage it, users need to delete these CRs manually.  
+As a side effect, when upgrading from an old release, even though the path is not changed, the BackupRepository gets created all the time, because Velero will not refer to the old CR's status. This seems to cause the repository to initialize more than once, however, it won't happen. In the BackupRepository controller, before initializing a repository, it always tries to connect to the repository first, if it is connectable, it won't do the initialization.  
+When backing up with the new release, Velero always creates BackupRepository CRs instead of ResticRepository CRs.  
+When restoring from an old backup, Velero always creates BackupRepository CRs instead of ResticRepository CRs.  
+When there are already backups or restores running during the upgrade, since after upgrade, the Velero server pods and VeleroNodeAgent daemonset pods are restarted, the existing backups/restores will fail immediately. 
+
+## Storage Configuration
+The backup repository needs some parameters to connect to various backup storage. For example, for a S3 compatible storage, the parameters may include bucket name, region, endpoint, etc. Different backup storage have totally different parameters. BackupRepository CRs, PodVolume Backup CRs and PodVolume Restore CRs save these parameters in their spec, as a string called repoIdentififer. The format of the string is for S3 storage only, it meets Restic CLI's requirements but is not enough for other backup repository. On the other hand, the parameters that are used to generate the repoIdentififer all come from the BackupStorageLocation. The latter has a map structure that could take parameters from any storage kind.  
+Therefore, for the new path, Velero uses the information in the BackupStorageLocation directly. That is, whenever Velero needs to initialize/connect to the Unified Repository, it acquires the storage configuration from the corresponding BackupStorageLocation. Then no more elements will be added in BackupRepository CRs, PodVolume Backup CRs or PodVolume Restore CRs.  
+The legacy path will be kept as is. That is, Velero still sets/gets the repoIdentififer in BackupRepository CRs, PodVolume Backup CRs and PodVolume Restore CRs and then passes to Restic CLI.  
+
+## Installation
+ We will add a new flag "--pod-volume-backup-uploader" during installation. The flag has 3 meanings:
+ - It indicates PodVolume BR as the default method to protect PV data over other methods, i.e., durable snapshot. Therefore, the existing --use-restic option will be replaced
+ - It indicates the file system uploader to be used by PodVolume BR
+ - It implies the backup repository type manner, Restic if pod-volume-backup-uploader=restic, Unified Repository in all other cases
+
+ The flag has below two values:  
+ **"Restic"**: it means Velero will use Restic to do the pod volume backup. Therefore, the Velero server deployment will be created as below:
+ ```
+    spec:
+      containers:
+      - args:
+        - server
+        - --features=
+        - --uploader-type=restic
+        command:
+        - /velero
+```
+The BackupRepository CRs and PodVolume Backup/Restore CRs created in this case are as below:
+```
+spec:
+  backupStorageLocation: default
+  maintenanceFrequency: 168h0m0s
+  repositoryType: restic
+  volumeNamespace: nginx-example
+```
+```
+spec:
+  backupStorageLocation: default
+  node: aks-agentpool-27359964-vmss000000
+  pod:
+    kind: Pod
+    name: nginx-stateful-0
+    namespace: nginx-example
+    uid: 86aaec56-2b21-4736-9964-621047717133   
+  tags:
+    ...
+  uploaderType: restic
+  volume: nginx-log
+```
+```
+spec:
+  backupStorageLocation: default
+  pod:
+    kind: Pod
+    name: nginx-stateful-0
+    namespace: nginx-example
+    uid: e56d5872-3d94-4125-bfe8-8a222bf0fcf1
+  snapshotID: 1741e5f1
+  uploaderType: restic
+  volume: nginx-log
+```
+ **"Kopia"**: it means Velero will use Kopia uploader to do the pod volume backup (so it will use Unified Repository as the backup target). Therefore, the Velero server deployment will be created as below:
+  ```
+    spec:
+      containers:
+      - args:
+        - server
+        - --features=
+        - --uploader-type=kopia
+        command:
+        - /velero
+```
+The BackupRepository CRs created in this case are hard set with "kopia" at present, sice Kopia is the only option as a backup repository. The PodVolume Backup/Restore CRs are created with "kopia" as well:
+```
+spec:
+  backupStorageLocation: default
+  maintenanceFrequency: 168h0m0s
+  repositoryType: kopia
+  volumeNamespace: nginx-example
+```
+```
+spec:
+  backupStorageLocation: default
+  node: aks-agentpool-27359964-vmss000000
+  pod:
+    kind: Pod
+    name: nginx-stateful-0
+    namespace: nginx-example
+    uid: 86aaec56-2b21-4736-9964-621047717133   
+  tags:
+    ...
+  uploaderType: kopia
+  volume: nginx-log
+```
+```
+spec:
+  backupStorageLocation: default
+  pod:
+    kind: Pod
+    name: nginx-stateful-0
+    namespace: nginx-example
+    uid: e56d5872-3d94-4125-bfe8-8a222bf0fcf1
+  snapshotID: 1741e5f1
+  uploaderType: kopia
+  volume: nginx-log
+```
+We will add the flag for both CLI installation and Helm Chart Installation. Specifically:
+- Helm Chart Installation: add the "--pod-volume-backup-uploader" flag into its value.yaml and then generate the deployments according to the value. Value.yaml is the user-provided configuration file, therefore, users could set this value at the time of installation. The changes in Value.yaml are as below:
+```
+          command:
+            - /velero
+          args:
+            - server
+          {{- with .Values.configuration }}
+            {{- if .pod-volume-backup-uploader "restic" }}
+            - --legacy
+            {{- end }} 
+```     
+- CLI Installation: add the "--pod-volume-backup-uploader" flag into the installation command line, and then create the two deployments accordingly. Users could change the option at the time of installation. The CLI is as below:  
+```velero install --pod-volume-backup-uploader=restic```  
+```velero install --pod-volume-backup-uploader=kopia``` 
+
+## Upgrade
+For upgrade, we allow users to change the path by specifying "--pod-volume-backup-uploader" flag in the same way as the fresh installation. Therefore, the flag change should be applied to the Velero server after upgrade. Additionally, We need to add a label to Velero server to indicate the current path, so as to provide an easy for querying it.  
+Moreover, if users upgrade from the old release, we need to change the existing Restic Daemonset name to VeleroNodeAgent daemonSet. The name change should be applied after upgrade.  
+The recommended way for upgrade is to modify the related Velero resource directly through kubectl, the above changes will be applied in the same way. We need to modify the Velero doc for all these changes.  
+
+## CLI
+Below Velero CLI or its output needs some changes:  
+- ```Velero backup describe```: the output should indicate the path  
+- ```Velero restore describe```: the output should indicate the path  
+- ```Velero restic repo get```: the name of this CLI should be changed to a generic one, for example, "Velero repo get"; the output of this CLI should print all the backup repository if Restic repository and Unified Repository exist at the same time  
+
+At present, we don't have a requirement for selecting the path during backup, so we don't change the ```Velero backup create``` CLI for now. If there is a requirement in future, we could simply add a flag similar to "--pod-volume-backup-uploader" to select the path.  
+
+## CR Example
+Below sample files demonstrate complete CRs with all the changes mentioned above:
+- BackupRepository CR: https://gist.github.com/Lyndon-Li/f38ad69dd8c4785c046cd7ed0ef2b6ed#file-backup-repository-sample-yaml
+- PodVolumeBackup CR: https://gist.github.com/Lyndon-Li/f38ad69dd8c4785c046cd7ed0ef2b6ed#file-pvb-sample-yaml
+- PodVolumeRestore CR: https://gist.github.com/Lyndon-Li/f38ad69dd8c4785c046cd7ed0ef2b6ed#file-pvr-sample-yaml
+
+## User Perspective
+This design aims to provide a flexible backup repository layer and a generic file system uploader, which are fundermental for PodVolume and other data movements. Although this will make Velero more capable, at present, we don't pursue to expose differentiated features end to end. Specifically:  
+- By default, Velero still uses Restic for PodVolume BR
+- Even when changing to the new path, Velero still allows users to restore from the data backed up by Restic
+- The capability of PodVolume BR under the new path is kept the same as it under Restic path and the same as the existing PodVolume BR
+- The operational experiences are kept the same as much as possible, the known changes are listed below
+
+Below user experiences are changed for this design:
+- Installation CLI change: a new option is added to the installation CLI, see the Installation section for details
+- CR change: One or more existing CRs have been renamed, see the Velero CR Changes section for details
+- Velero CLI name and output change, see the CLI section for details
+- Velero daemonset name change
+- Wording Alignment: as the existing situation, many places are using the word of "Restic", for example, "default-volume-to-restic" option, most of them are not accurate anymore, we will change these words and give a detailed list of the changes