Compare commits

..

22 Commits

Author SHA1 Message Date
Scott Seago
5fe3a50bfd Merge pull request #4030 from zubron/cherry-pick-apiservice-action-1-6-3
Cherry-pick and update changelog for v1.6.3
2021-08-11 21:43:26 -04:00
Bridget McErlean
3743ca4d53 Update changelog for v1.6.3
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-08-11 21:04:59 -04:00
Scott Seago
6feb84a1cc Merge pull request #4028 from zubron/add-restore-item-action-to-skip-automanaged-apiservices
Skip restore of APIServices managed by Kubernetes
2021-08-11 21:03:27 -04:00
Bridget McErlean
b090b27275 Cherry-pick and update changelog for v1.6.3 (#4018)
* Use appropriate CRD API during readiness check (#4015)

* Use appropriate CRD API during readiness check

The readiness check for the Velero CRDs was still using the v1beta1 API.
This would cause the readiness check to fail on 1.22 clusters as the
v1beta1 API is no longer available. Previously, this error would be
ignored and the installation would proceed, however with #4002, we are
no longer ignoring errors from this check.

This change modifies the CRD readiness check to check the CRDs using the
same API version that was used when submitting the CRDs to the cluster.
It also introduces a new CRD builder using the V1 API for testing.

This change also fixes a bug that was identified in the polling code
where if the CRDs were not ready on the first polling iteration, they
would be added again to the list of CRDs to check resulting in
duplicates. This would cause the length check to fail on all subsequent
polls and the timeout would always be reached.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Remove duplicate V1 CRD builder and update comment

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Merge pull request #4012 from jenting/add-k8s-1.22-ci-test

Add Kubernetes v1.22 CI test

* Update changelog for v1.6.3

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

Co-authored-by: Scott Seago <sseago@redhat.com>
2021-08-06 22:49:34 +08:00
Bridget McErlean
499631ba8e Cherry pick changes for 1.6.3 and add changelog (#4006)
* Merge pull request #3941 from sseago/e2e-crdversion

enable e2e tests to choose crd apiVersion

* Updated uninstall to remove both v1beta1 and v1 CRDs if present (#3997)

* Add changelog for v1.6.3

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

Co-authored-by: Wenkai Yin(尹文开) <yinw@vmware.com>
Co-authored-by: David L. Smith-Uchida <dsmithuchida@vmware.com>
2021-07-30 07:31:48 +08:00
Bridget McErlean
aa274e8c65 Merge pull request #3999 from ywk253100/210728_1.6_cherry_pick
[cherry-pick]Support both v1beta1 and v1 CRDs for velero
2021-07-28 20:52:15 -04:00
Wenkai Yin(尹文开)
9e7daf7e37 Fix bugs of E2E test cases
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2021-07-29 07:37:07 +08:00
JenTing Hsiao
e2581866bc Support both v1beta1 and v1 CRDs for velero
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
2021-07-29 07:37:07 +08:00
Wenkai Yin(尹文开)
037e475227 Upgrade Velero ClusterRoleBinding to use v1 API (#3995)
* Upgrade Velero ClusterRoleBinding to use v1 API

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Add the change log

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>

Co-authored-by: JenTing Hsiao <jenting.hsiao@suse.com>
2021-07-27 22:38:10 -07:00
Daniel Jiang
8c9cdb9603 Merge pull request #3971 from zubron/release-1.6.2
Add cherry-pick commits and changelog for v1.6.2
2021-07-20 12:29:28 +08:00
Bridget McErlean
a77702c885 Add changelog for v1.6.2
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-07-16 15:24:05 -04:00
Bridget McErlean
1d882c6509 Merge pull request #3928 from zubron/customize-velero-image-at-build-time
Allow image registry to be configured at build time

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-07-16 15:23:53 -04:00
Bridget McErlean
07060d7fea Update k8s libraries to latest patch version (#3953)
Also enforce the use of the latest version of github.com/gogo/protobuf.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-07-16 15:10:16 -04:00
Bridget McErlean
ef34b9b654 Merge pull request #3889 from zubron/release-1.6.1
Add cherry-pick commits and changelog for v1.6.1
2021-06-22 09:15:31 -04:00
Bridget McErlean
e22d6591e4 Add changelog for v1.6.1
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-06-21 12:01:27 -04:00
Scott Seago
38493995ad regression introduced in 1.6 restore progress: fix CR restore (#3845)
Signed-off-by: Scott Seago <sseago@redhat.com>
2021-06-21 12:01:21 -04:00
Bridget McErlean
74a0b39e3e Skip volume restores from projected sources (#3877)
In #3863, it was discovered that volumes from projected sources were
being backed up by restic when they should have been skipped. Restoring
these volumes triggers a known bug in restic.

In #3866, we started skipping volumes from a projected source, however
there will exist backups that were taken before this fix was introduced.
This change modifies the restore logic to skip the restore of any volume
that came from a projected source, allowing backups taken before #3866
to be restored successfully.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-06-21 12:01:21 -04:00
codegold79
a378d3a9d4 API groups e2e tests remove controllers (#3564)
* Remove controllers and sleeps in API groups e2e tests

Signed-off-by: F. Gold <fgold@vmware.com>

* Print command in AfterEach(...) and check error

Signed-off-by: F. Gold <fgold@vmware.com>

* Make change ahead of PR3764 changes in main

Signed-off-by: F. Gold <fgold@vmware.com>

* Update go.{mod,sum} files

Signed-off-by: F. Gold <fgold@vmware.com>

* Run make update

Signed-off-by: F. Gold <fgold@vmware.com>
2021-06-21 12:01:21 -04:00
Scott Seago
0f576fb748 Merge pull request #3866 from alaypatel07/fix-projected-volume-for-restic
skip backuping projected volume when using restic

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-06-21 12:00:58 -04:00
Carlisia Thompson
119529c9a2 Consolidate api clients for e2e tests (#3764)
* Consolidate api clients
* Adress Nolan reviews
* Adding back output warning for consistency
* Remove unnecessary documentation
* Address Bridget's reviews
* Update go.sum files

Signed-off-by: Carlisia <carlisia@grokkingtech.io>
Co-authored-by: Bridget McErlean <bmcerlean@vmware.com>
2021-06-21 11:17:35 -04:00
Carlisia Thompson
2c26119b10 A small refactor of the e2e tests (#3726)
* A small refactor of the e2e tests

Signed-off-by: Carlisia <carlisia@grokkingtech.io>

* Add copyright header

Signed-off-by: Carlisia <carlisia@grokkingtech.io>

* Fix CI

Signed-off-by: Carlisia <carlisia@grokkingtech.io>

* Revert unneeded changes

Signed-off-by: Carlisia <carlisia@grokkingtech.io>

* Remove file that doesnt belong here

Signed-off-by: Carlisia <carlisia@grokkingtech.io>
2021-06-21 11:17:08 -04:00
Ashish Amarnath
cbccdbd05a 🐛 Fix plugin name derivation from image name (#3711)
* 🐛 Fix plugin name derivation from image name

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* changelog

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2021-06-21 09:43:33 -04:00
1488 changed files with 22716 additions and 148113 deletions

View File

@@ -5,18 +5,15 @@ about: Tell us about a problem you are experiencing
---
**What steps did you take and what happened:**
<!--A clear and concise description of what the bug is, and what commands you ran.-->
[A clear and concise description of what the bug is, and what commands you ran.)
**What did you expect to happen:**
**The following information will help us better understand what's going on**:
_If you are using velero v1.7.0+:_
Please use `velero debug --backup <backupname> --restore <restorename>` to generate the support bundle, and attach to this issue, more options please refer to `velero debug --help`
**The output of the following commands will help us better understand what's going on**:
(Pasting long output into a [GitHub gist](https://gist.github.com) or other pastebin is fine.)
_If you are using earlier versions:_
Please provide the output of the following commands (Pasting long output into a [GitHub gist](https://gist.github.com) or other pastebin is fine.)
- `kubectl logs deployment/velero -n velero`
- `velero backup describe <backupname>` or `kubectl get backup/<backupname> -n velero -o yaml`
- `velero backup logs <backupname>`
@@ -25,7 +22,7 @@ Please provide the output of the following commands (Pasting long output into a
**Anything else you would like to add:**
<!--Miscellaneous information that will assist in solving the issue.-->
[Miscellaneous information that will assist in solving the issue.]
**Environment:**

View File

@@ -5,15 +5,15 @@ about: Suggest an idea for this project
---
**Describe the problem/challenge you have**
<!--A description of the current limitation/problem/challenge that you are experiencing.-->
[A description of the current limitation/problem/challenge that you are experiencing.]
**Describe the solution you'd like**
<!--A clear and concise description of what you want to happen.-->
[A clear and concise description of what you want to happen.]
**Anything else you would like to add:**
<!--Miscellaneous information that will assist in solving the issue.-->
[Miscellaneous information that will assist in solving the issue.]
**Environment:**

View File

@@ -1,43 +0,0 @@
---
# This assigns a PR to its author
addAssignees: author
reviewers:
# The default reviewers
defaults:
- maintainers
groups:
maintainers:
- sseago
- reasonerjt
- ywk253100
- blackpiglet
- qiuming-best
- shubham-pampattiwar
- Lyndon-Li
tech-writer:
- sseago
- reasonerjt
- ywk253100
- Lyndon-Li
files:
'site/**':
- tech-writer
'**/*.md':
- tech-writer
# Technical design requests are ".md" files but should
# be reviewed by maintainers
'/design/**':
- maintainers
options:
ignore_draft: true
ignored_keywords:
- WIP
- wip
- DO NOT MERGE
enable_group_assignment: true
number_of_reviewers: 2

14
.github/auto_assign.yml vendored Normal file
View File

@@ -0,0 +1,14 @@
addReviewers: true
addAssignees: author
# Only require 2, random reviewers.
# TODO expand this to support using reviewGroups
numberOfReviewers: 2
reviewers:
- nrb
- ashish-amarnath
- carlisia
- zubron
- dsu-igeek
- jenting

View File

@@ -1,12 +0,0 @@
version: 2
updates:
# Dependencies listed in go.mod
- package-ecosystem: "gomod"
directory: "/" # Location of package manifests
schedule:
interval: "weekly"
labels:
- "kind/changelog-not-required"
ignore:
- dependency-name: "*"
update-types: ["version-update:semver-major", "version-update:semver-minor", "version-update:semver-patch"]

View File

@@ -9,5 +9,5 @@ Fixes #(issue)
# Please indicate you've done the following:
- [ ] [Accepted the DCO](https://velero.io/docs/v1.5/code-standards/#dco-sign-off). Commits without the DCO will delay acceptance.
- [ ] [Created a changelog file](https://velero.io/docs/v1.5/code-standards/#adding-a-changelog) or added `/kind changelog-not-required` as a comment on this pull request.
- [ ] [Created a changelog file](https://velero.io/docs/v1.5/code-standards/#adding-a-changelog) or added `/kind changelog-not-required`.
- [ ] Updated the corresponding documentation in `site/content/docs/main`.

View File

@@ -1,8 +1,6 @@
---
name: "Auto Assign Author"
# pull_request_target means that this will run on pull requests, but in
# the context of the base repo. This should mean PRs from forks are supported.
name: "Auto Assign PR Reviewers"
# pull_request_target means that this will run on pull requests, but in the context of the base repo.
# This should mean PRs from forks are supported.
on:
pull_request_target:
types: [opened, reopened, ready_for_review]
@@ -12,8 +10,7 @@ jobs:
add-reviews:
runs-on: ubuntu-latest
steps:
- name: Set the author of a PR as the assignee
uses: kentaro-m/auto-assign-action@v1.1.1
- uses: kentaro-m/auto-assign-action@v1.1.1
with:
configuration-path: ".github/auto-assignees.yml"
repo-token: "${{ secrets.GITHUB_TOKEN }}"
configuration-path: ".github/auto_assign.yml"
repo-token: "${{ secrets.GITHUB_TOKEN }}"

View File

@@ -1,17 +0,0 @@
---
name: "Auto Request Review"
on:
pull_request_target:
types: [opened, ready_for_review, reopened]
jobs:
auto-request-review:
name: Auto Request Review
runs-on: ubuntu-latest
steps:
- name: Request a PR review based on files types/paths, and/or groups the author belongs to
uses: necojackarc/auto-request-review@v0.7.0
with:
token: ${{ secrets.GITHUB_TOKEN }}
config: .github/auto-assignees.yml

View File

@@ -11,11 +11,6 @@ jobs:
build-cli:
runs-on: ubuntu-latest
steps:
- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: '1.20.7'
id: go
# Look for a CLI that's made for this PR
- name: Fetch built CLI
id: cache
@@ -49,7 +44,7 @@ jobs:
run: |
make local
# Check the common CLI against all Kubernetes versions
# Check the common CLI against all kubernetes versions
crd-check:
needs: build-cli
runs-on: ubuntu-latest
@@ -57,13 +52,14 @@ jobs:
matrix:
# Latest k8s versions. There's no series-based tag, nor is there a latest tag.
k8s:
- 1.15.12
- 1.16.15
- 1.17.17
- 1.18.15
- 1.19.7
- 1.20.2
- 1.21.1
- 1.22.0
- 1.23.6
- 1.24.2
- 1.25.3
# All steps run in parallel unless otherwise specified.
# See https://docs.github.com/en/actions/learn-github-actions/managing-complex-workflows#creating-dependent-jobs
steps:
@@ -81,7 +77,7 @@ jobs:
velero-${{ github.event.pull_request.number }}-
- uses: engineerd/setup-kind@v0.5.0
with:
version: "v0.17.0"
version: "v0.11.1"
image: "kindest/node:v${{ matrix.k8s }}"
- name: Install CRDs
run: |

View File

@@ -11,11 +11,6 @@ jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: '1.20.7'
id: go
# Look for a CLI that's made for this PR
- name: Fetch built CLI
id: cli-cache
@@ -53,27 +48,24 @@ jobs:
run: |
IMAGE=velero VERSION=pr-test make container
docker save velero:pr-test -o ./velero.tar
# Run E2E test against all Kubernetes versions on kind
# Run E2E test against all kubernetes versions on kind
run-e2e-test:
needs: build
runs-on: ubuntu-latest
strategy:
matrix:
k8s:
- 1.19.16
- 1.20.15
- 1.21.12
- 1.22.9
- 1.23.6
- 1.24.0
- 1.25.3
# doesn't cover 1.15 as 1.15 doesn't support "apiextensions.k8s.io/v1" that is needed for the case
#- 1.15.12
- 1.16.15
- 1.17.17
- 1.18.15
- 1.19.7
- 1.20.2
- 1.21.1
- 1.22.0
fail-fast: false
steps:
- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: '1.20.7'
id: go
- name: Check out the code
uses: actions/checkout@v2
- name: Install MinIO
@@ -81,7 +73,7 @@ jobs:
docker run -d --rm -p 9000:9000 -e "MINIO_ACCESS_KEY=minio" -e "MINIO_SECRET_KEY=minio123" -e "MINIO_DEFAULT_BUCKETS=bucket,additional-bucket" bitnami/minio:2021.6.17-debian-10-r7
- uses: engineerd/setup-kind@v0.5.0
with:
version: "v0.17.0"
version: "v0.11.1"
image: "kindest/node:v${{ matrix.k8s }}"
- name: Fetch built CLI
id: cli-cache
@@ -113,22 +105,10 @@ jobs:
aws_access_key_id=minio
aws_secret_access_key=minio123
EOF
# Match kubectl version to k8s server version
curl -LO https://dl.k8s.io/release/v${{ matrix.k8s }}/bin/linux/amd64/kubectl
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
GOPATH=~/go CLOUD_PROVIDER=kind \
OBJECT_STORE_PROVIDER=aws BSL_CONFIG=region=minio,s3ForcePathStyle="true",s3Url=http://$(hostname -i):9000 \
CREDS_FILE=/tmp/credential BSL_BUCKET=bucket \
ADDITIONAL_OBJECT_STORE_PROVIDER=aws ADDITIONAL_BSL_CONFIG=region=minio,s3ForcePathStyle="true",s3Url=http://$(hostname -i):9000 \
ADDITIONAL_CREDS_FILE=/tmp/credential ADDITIONAL_BSL_BUCKET=additional-bucket \
GINKGO_FOCUS='Basic\]\[ClusterResource' VELERO_IMAGE=velero:pr-test \
make -C test/e2e run
timeout-minutes: 30
- name: Upload debug bundle
if: ${{ failure() }}
uses: actions/upload-artifact@v2
with:
name: DebugBundle
path: /home/runner/work/velero/velero/test/e2e/debug-bundle*
GINKGO_FOCUS=Basic VELERO_IMAGE=velero:pr-test \
make -C test/e2e run

18
.github/workflows/milestoned-issues.yml vendored Normal file
View File

@@ -0,0 +1,18 @@
name: Add issues with a milestone to the milestone's board
on:
issues:
types: [milestoned]
jobs:
automate-project-columns:
runs-on: ubuntu-latest
steps:
- uses: alex-page/github-project-automation-plus@v0.3.0
with:
# Do NOT add PRs to the board, as that's duplication. Their corresponding issue should be on the board.
if: ${{ !github.event.issue.pull_request }}
project: "${{ github.event.issue.milestone.title }}"
column: "To Do"
repo-token: ${{ secrets.GH_TOKEN }}

View File

@@ -1,36 +0,0 @@
name: Trivy Nightly Scan
on:
schedule:
- cron: '0 2 * * *' # run at 2 AM UTC
jobs:
nightly-scan:
name: Trivy nightly scan
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
# maintain the versions of Velero those need security scan
versions: [main]
# list of images that need scan
images: [velero, velero-restore-helper]
permissions:
security-events: write # for github/codeql-action/upload-sarif to upload SARIF results
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@master
with:
image-ref: 'docker.io/velero/${{ matrix.images }}:${{ matrix.versions }}'
severity: 'CRITICAL,HIGH,MEDIUM'
format: 'template'
template: '@/contrib/sarif.tpl'
output: 'trivy-results.sarif'
- name: Upload Trivy scan results to GitHub Security tab
uses: github/codeql-action/upload-sarif@v2
with:
sarif_file: 'trivy-results.sarif'

View File

@@ -0,0 +1,15 @@
name: Move new issues into Triage
on:
issues:
types: [opened]
jobs:
automate-project-columns:
runs-on: ubuntu-latest
steps:
- uses: alex-page/github-project-automation-plus@v0.3.0
with:
project: "Velero Support Board"
column: "New"
repo-token: ${{ secrets.GH_TOKEN }}

View File

@@ -1,9 +1,5 @@
name: Pull Request Changelog Check
# by setting `on: [pull_request]`, that means action will be trigger when PR is opened, synchronize, reopened.
# Add labeled and unlabeled events too.
on:
pull_request:
types: [opened, synchronize, reopened, labeled, unlabeled]
on: [pull_request]
jobs:
build:

View File

@@ -4,16 +4,10 @@ jobs:
build:
name: Run CI
runs-on: ubuntu-latest
strategy:
fail-fast: false
steps:
- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: '1.20.7'
id: go
- name: Check out the code
uses: actions/checkout@v2
- name: Fetch cached go modules
uses: actions/cache@v2
with:
@@ -21,12 +15,6 @@ jobs:
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Make ci
run: make ci
- name: Upload test coverage
uses: codecov/codecov-action@v3
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: coverage.out
verbose: true
fail_ci_if_error: true

View File

@@ -14,44 +14,7 @@ jobs:
uses: codespell-project/actions-codespell@master
with:
# ignore the config/.../crd.go file as it's generated binary data that is edited elswhere.
skip: .git,*.png,*.jpg,*.woff,*.ttf,*.gif,*.ico,./config/crd/v1beta1/crds/crds.go,./config/crd/v1/crds/crds.go,./config/crd/v2alpha1/crds/crds.go,./go.sum,./LICENSE
ignore_words_list: iam,aks,ist,bridget,ue,shouldnot,atleast
skip: .git,*.png,*.jpg,*.woff,*.ttf,*.gif,*.ico,./config/crd/v1beta1/crds/crds.go,./config/crd/v1/crds/crds.go
ignore_words_list: iam,aks,ist,bridget,ue
check_filenames: true
check_hidden: true
- name: Velero.io word list check
shell: bash {0}
run: |
IGNORE_COMMENT="Velero.io word list : ignore"
FILES_TO_CHECK=$(find . -type f \
! -path "./.git/*" \
! -path "./site/content/docs/v*" \
! -path "./changelogs/CHANGELOG-*" \
! -path "./.github/workflows/pr-codespell.yml" \
! -path "./site/static/fonts/Metropolis/Open Font License.md" \
! -regex '.*\.\(png\|jpg\|woff\|ttf\|gif\|ico\|svg\)'
)
function check_word_in_files() {
local word=$1
xargs grep -Iinr "$word" <<< "$FILES_TO_CHECK" | \
grep -v "$IGNORE_COMMENT" | \
grep -i --color=always "$word" && \
EXIT_STATUS=1
}
function check_word_case_sensitive_in_files() {
local word=$1
xargs grep -Inr "$word" <<< "$FILES_TO_CHECK" | \
grep -v "$IGNORE_COMMENT" | \
grep --color=always "$word" && \
EXIT_STATUS=1
}
EXIT_STATUS=0
check_word_case_sensitive_in_files ' kubernetes '
check_word_in_files 'on-premise\b'
check_word_in_files 'back-up'
check_word_in_files 'plug-in'
check_word_in_files 'whitelist'
check_word_in_files 'blacklist'
exit $EXIT_STATUS

View File

@@ -1,37 +0,0 @@
name: build Velero containers on Dockerfile change
on:
pull_request:
branches:
- 'main'
- 'release-**'
paths:
- 'Dockerfile'
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
name: Checkout
- name: Set up QEMU
id: qemu
uses: docker/setup-qemu-action@v1
with:
platforms: all
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v1
with:
version: latest
# Although this action also calls docker-push.sh, it is not triggered
# by push, so BRANCH and TAG are empty by default. docker-push.sh will
# only build Velero image without pushing.
- name: Make Velero container without pushing to registry.
if: github.repository == 'vmware-tanzu/velero'
run: |
./hack/docker-push.sh

View File

@@ -1,29 +0,0 @@
name: Verify goreleaser change
on:
pull_request:
branches:
- 'main'
- 'release-**'
paths:
- '.goreleaser.yml'
- 'hack/release-tools/goreleaser.sh'
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
name: Checkout
- name: Verify .goreleaser.yml and try a dryrun release.
if: github.repository == 'vmware-tanzu/velero'
run: |
CHANGELOG=$(ls changelogs | sort -V -r | head -n 1)
GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }} \
REGISTRY=velero \
RELEASE_NOTES_FILE=changelogs/$CHANGELOG \
PUBLISH=false \
make release

View File

@@ -12,16 +12,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
# The default value is "1" which fetches only a single commit. If we merge PR without squash or rebase,
# there are at least two commits: the first one is the merge commit and the second one is the real commit
# contains the changes.
# As we use the Dockerfile's commit ID as the tag of the build-image, fetching only 1 commit causes the merge
# commit ID to be the tag.
# While when running make commands locally, as the local git repository usually contains all commits, the Dockerfile's
# commit ID is the second one. This is mismatch with the images in Dockerhub
fetch-depth: 2
- uses: actions/checkout@master
- name: Build
run: make build-image

View File

@@ -2,9 +2,7 @@ name: Main CI
on:
push:
branches:
- 'main'
- 'release-**'
branches: [ main ]
tags:
- '*'
@@ -16,24 +14,13 @@ jobs:
steps:
- name: Set up Go
uses: actions/setup-go@v4
uses: actions/setup-go@v2
with:
go-version: '1.20.7'
go-version: 1.15
id: go
- uses: actions/checkout@v3
# Fix issue of setup-gcloud
- run: |
sudo apt-get install python2.7
export CLOUDSDK_PYTHON="/usr/bin/python2"
- uses: google-github-actions/setup-gcloud@v0
with:
version: '285.0.0'
service_account_key: ${{ secrets.GCS_SA_KEY }}
export_default_credentials: true
- run: gcloud info
- name: Check out code into the Go module directory
uses: actions/checkout@v2
- name: Set up QEMU
id: qemu
@@ -53,48 +40,9 @@ jobs:
- name: Test
run: make test
- name: Upload test coverage
uses: codecov/codecov-action@v2
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: coverage.out
verbose: true
# Use the JSON key in secret to login gcr.io
- uses: 'docker/login-action@v2'
with:
registry: 'gcr.io' # or REGION.docker.pkg.dev
username: '_json_key'
password: '${{ secrets.GCR_SA_KEY }}'
# Only try to publish the container image from the root repo; forks don't have permission to do so and will always get failures.
- name: Publish container image
if: github.repository == 'vmware-tanzu/velero'
run: |
sudo swapoff -a
sudo rm -f /mnt/swapfile
docker image prune -a --force
# Build and push Velero image to docker registry
docker login -u ${{ secrets.DOCKER_USER }} -p ${{ secrets.DOCKER_PASSWORD }}
VERSION=$(./hack/docker-push.sh | grep 'VERSION:' | awk -F: '{print $2}' | xargs)
# Upload Velero image package to GCS
source hack/ci/build_util.sh
BIN=velero
RESTORE_HELPER_BIN=velero-restore-helper
GCS_BUCKET=velero-builds
VELERO_IMAGE=${BIN}-${VERSION}
VELERO_RESTORE_HELPER_IMAGE=${RESTORE_HELPER_BIN}-${VERSION}
VELERO_IMAGE_FILE=${VELERO_IMAGE}.tar.gz
VELERO_RESTORE_HELPER_IMAGE_FILE=${VELERO_RESTORE_HELPER_IMAGE}.tar.gz
VELERO_IMAGE_BACKUP_FILE=${VELERO_IMAGE}-'build.'${GITHUB_RUN_NUMBER}.tar.gz
VELERO_RESTORE_HELPER_IMAGE_BACKUP_FILE=${VELERO_RESTORE_HELPER_IMAGE}-'build.'${GITHUB_RUN_NUMBER}.tar.gz
cp ${VELERO_IMAGE_FILE} ${VELERO_IMAGE_BACKUP_FILE}
cp ${VELERO_RESTORE_HELPER_IMAGE_FILE} ${VELERO_RESTORE_HELPER_IMAGE_BACKUP_FILE}
uploader ${VELERO_IMAGE_FILE} ${GCS_BUCKET}
uploader ${VELERO_RESTORE_HELPER_IMAGE_FILE} ${GCS_BUCKET}
uploader ${VELERO_IMAGE_BACKUP_FILE} ${GCS_BUCKET}
uploader ${VELERO_RESTORE_HELPER_IMAGE_BACKUP_FILE} ${GCS_BUCKET}
./hack/docker-push.sh

View File

@@ -1,7 +1,8 @@
name: "Close stale issues and PRs"
on:
schedule:
- cron: "30 1 * * *" # Every day at 1:30 UTC
# First of every month
- cron: "30 1 * * *"
jobs:
stale:
@@ -10,14 +11,14 @@ jobs:
- uses: actions/stale@v3
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
stale-issue-message: "This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 14 days. If a Velero team member has requested log or more information, please provide the output of the shared commands."
close-issue-message: "This issue was closed because it has been stalled for 14 days with no activity."
days-before-issue-stale: 60
days-before-issue-close: 14
stale-issue-label: staled
stale-issue-message: "This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days. If a Velero team member has requested log or more information, please provide the output of the shared commands."
close-issue-message: "This issue was closed because it has been stalled for 5 days with no activity."
days-before-issue-stale: 30
days-before-issue-close: 5
# Disable stale PRs for now; they can remain open.
days-before-pr-stale: -1
days-before-pr-close: -1
# Only issues made after Feb 09 2021.
start-date: "2021-09-02T00:00:00"
exempt-issue-labels: "Epic,Area/CLI,Area/Cloud/AWS,Area/Cloud/Azure,Area/Cloud/GCP,Area/Cloud/vSphere,Area/CSI,Area/Design,Area/Documentation,Area/Plugins,Bug,Enhancement/User,kind/requirement,kind/refactor,kind/tech-debt,limitation,Needs investigation,Needs triage,Needs Product,P0 - Hair on fire,P1 - Important,P2 - Long-term important,P3 - Wouldn't it be nice if...,Product Requirements,Restic - GA,Restic,release-blocker,Security"
# Only make issues stale if they have these labels. Comma separated.
only-labels: "Needs info,Duplicate"

12
.gitignore vendored
View File

@@ -24,6 +24,8 @@ _testmain.go
*.test
*.prof
debug
/velero
.idea/
@@ -38,7 +40,6 @@ _testmain.go
# Hugo compiled data
site/public
site/resources
site/.hugo_build.lock
.vs
@@ -47,10 +48,5 @@ _tiltbuild
tilt-resources/tilt-settings.json
tilt-resources/velero_v1_backupstoragelocation.yaml
tilt-resources/deployment.yaml
tilt-resources/node-agent.yaml
tilt-resources/cloud
# test generated files
test/e2e/report.xml
coverage.out
__debug_bin*
tilt-resources/restic.yaml
tilt-resources/cloud

View File

@@ -14,7 +14,7 @@
dist: _output
builds:
- main: ./cmd/velero/velero.go
- main: ./cmd/velero/main.go
env:
- CGO_ENABLED=0
goos:
@@ -27,9 +27,11 @@ builds:
- arm64
- ppc64le
ignore:
# don't build arm for darwin and arm/arm64 for windows
# don't build arm/arm64 for darwin or windows
- goos: darwin
goarch: arm
- goos: darwin
goarch: arm64
- goos: darwin
goarch: ppc64le
- goos: windows
@@ -46,9 +48,6 @@ archives:
files:
- LICENSE
- examples/**/*
# Add the setting to resolve the DEPRECATED warning. Actually, Velero's case is not affected by the rlcp behavior change.
# https://github.com/orgs/goreleaser/discussions/3659#discussioncomment-4587257
rlcp: true
checksum:
name_template: 'CHECKSUM'
release:
@@ -57,10 +56,3 @@ release:
name: velero
draft: true
prerelease: auto
git:
# What should be used to sort tags when gathering the current and previous
# tags if there are more than one tag in the same commit.
#
# Default: `-version:refname`
tag_sort: -version:creatordate

View File

@@ -3,7 +3,6 @@
If you're using Velero and want to add your organization to this list,
[follow these directions][1]!
<a href="https://www.pitsdatarecovery.net/" border="0" target="_blank"><img alt="pitsdatarecovery.net" src="site/static/img/adopters/PITSGlobalDataRecoveryServices.svg" height="50"></a>
<a href="https://www.bitgo.com" border="0" target="_blank"><img alt="bitgo.com" src="site/static/img/adopters/BitGo.svg" height="50"></a>&nbsp; &nbsp; &nbsp;
<a href="https://www.nirmata.com" border="0" target="_blank"><img alt="nirmata.com" src="site/static/img/adopters/nirmata.svg" height="50"></a>&nbsp; &nbsp; &nbsp;
<a href="https://kyma-project.io/" border="0" target="_blank"><img alt="kyma-project.io" src="site/static/img/adopters/kyma.svg" height="50"></a>&nbsp; &nbsp; &nbsp;
@@ -14,18 +13,17 @@ If you're using Velero and want to add your organization to this list,
<a href="https://banzaicloud.com/" border="0" target="_blank"><img alt="banzaicloud.com" src="site/static/img/adopters/banzaicloud.svg" height="50"></a>&nbsp; &nbsp; &nbsp;
<a href="https://sighup.io/" border="0" target="_blank"><img alt="sighup.io" src="site/static/img/adopters/sighup.svg" height="50"></a>&nbsp; &nbsp; &nbsp;
<a href="https://mayadata.io/" border="0" target="_blank"><img alt="mayadata.io" src="site/static/img/adopters/mayadata.svg" height="50"></a>&nbsp; &nbsp; &nbsp;
<a href="https://www.replicated.com/" border="0" target="_blank"><img alt="replicated.com" src="site/static/img/adopters/replicated-logo-red.svg" height="50"></a>
<a href="https://cloudcasa.io/" border="0" target="_blank"><img alt="cloudcasa.io" src="site/static/img/adopters/cloudcasa.svg" height="50"></a>
## Success Stories
Below is a list of adopters of Velero in **production environments** that have
publicly shared the details of how they use it.
**[BitGo][20]**
BitGo uses Velero backup and restore capabilities to seamlessly provision and scale fullnode statefulsets on the fly as well as having it serve an integral piece for our Kubernetes disaster-recovery story.
BitGo uses Velero backup and restore capabilities to seamlessly provision and scale fullnode statefulsets on the fly as well as having it serve an integral piece for our kubernetes disaster-recovery story.
**[Bugsnag][30]**
We use Velero for managing backups of an internal instance of our on-premise clustered solution. We also recommend our users of [on-premise Bugsnag installations](https://www.bugsnag.com/on-premise) use Velero for [managing their own backups](https://docs.bugsnag.com/on-premise/clustered/backup-restore/). <!-- Velero.io word list : ignore -->
We use Velero for managing backups of an internal instance of our on-premise clustered solution. We also recommend our users of [on-premise Bugsnag installations][31] use Velero for [managing their own backups][32].
**[Banzai Cloud][60]**
[Banzai Cloud Pipeline][61] is a Kubernetes-based microservices platform that integrates services needed for Day-1 and Day-2 operations along with first-class support both for on-prem and hybrid multi-cloud deployments. We use Velero to periodically [backup and restore these clusters in case of disasters][62].
@@ -41,9 +39,7 @@ We have integrated our [solution with Velero][11] to provide our customers with
Kyma [integrates with Velero][41] to effortlessly back up and restore Kyma clusters with all its resources. Velero capabilities allow Kyma users to define and run manual and scheduled backups in order to successfully handle a disaster-recovery scenario.
**[Red Hat][50]**
Red Hat has developed 2 operators for the OpenShift platform:
- [Migration Toolkit for Containers][51] (Crane): This operator uses [Velero and Restic][52] to drive the migration of applications between OpenShift clusters.
- [OADP (OpenShift API for Data Protection) Operator][53]: This operator sets up and installs Velero on the OpenShift platform, allowing users to backup and restore applications.
Red Hat has developed the [Cluster Application Migration Tool][51] which uses [Velero and Restic][52] to drive the migration of applications between OpenShift clusters.
**[Dell EMC][70]**
For Kubernetes environments, [PowerProtect Data Manager][71] leverages the Container Storage Interface (CSI) framework to take snapshots to back up the persistent data or the data that the application creates e.g. databases. [Dell EMC leverages Velero][72] to backup the namespace configuration files (also known as Namespace meta data) for enterprise grade data protection.
@@ -57,13 +53,7 @@ MayaData is a large user of Velero as well as a contributor. MayaData offers a D
**[Okteto][93]**
Okteto integrates Velero in [Okteto Cloud][94] and [Okteto Enterprise][95] to periodically backup and restore our clusters for disaster recovery. Velero is also a core software building block to provide namespace cloning capabilities, a feature that allows our users cloning staging environments into their personal development namespace for providing production-like development environments.
**[Replicated][100]**<br>
Replicated uses the Velero open source project to enable snapshots in [KOTS][101] to backup Kubernetes manifests & persistent volumes. In addition to the default functionality that Velero provides, [KOTS][101] provides a detailed interface in the [Admin Console][102] that can be used to manage the storage destination and schedule, and to perform and monitor the backup and restore process.<br>
**[CloudCasa][103]**<br>
[Catalogic Software][104] integrates Velero with [CloudCasa][103] - A Smart Home in the Cloud for Backups. CloudCasa is a simple, scalable, cloud-native solution providing data protection and disaster recovery as a service. This solution is built using Kubernetes for protecting Kubernetes clusters.<br>
## Adding your organization to the list of Velero Adopters
If you are using Velero and would like to be included in the list of `Velero Adopters`, add an SVG version of your logo to the `site/static/img/adopters` directory in this repo and submit a [pull request][3] with your change. Name the image file something that reflects your company (e.g., if your company is called Acme, name the image acme.png). See this for an example [PR][4].
@@ -83,6 +73,8 @@ If you would like to add your logo to a future `Adopters of Velero` section on [
[20]: https://bitgo.com
[30]: https://bugsnag.com
[31]: https://www.bugsnag.com/on-premise
[32]: https://docs.bugsnag.com/on-premise/clustered/backup-restore/
[40]: https://kyma-project.io
[41]: https://kyma-project.io/docs/components/backup/#overview-overview
@@ -90,7 +82,6 @@ If you would like to add your logo to a future `Adopters of Velero` section on [
[50]: https://redhat.com
[51]: https://github.com/fusor/mig-operator
[52]: https://github.com/fusor/mig-operator/blob/master/docs/usage/2.md
[53]: https://github.com/openshift/oadp-operator
[60]: https://banzaicloud.com
[61]: https://banzaicloud.com/products/pipeline/
@@ -111,10 +102,3 @@ If you would like to add your logo to a future `Adopters of Velero` section on [
[93]: https://okteto.com
[94]: https://cloud.okteto.com
[95]: https://okteto.com/enterprise/
[100]: https://www.replicated.com
[101]: https://kots.io
[102]: https://kots.io/kotsadm/snapshots/overview/
[103]: https://cloudcasa.io/
[104]: https://www.catalogicsoftware.com/

View File

@@ -1,12 +1,7 @@
## Current release:
* [CHANGELOG-1.11.md][21]
* [CHANGELOG-1.6.md][16]
## Older releases:
* [CHANGELOG-1.10.md][20]
* [CHANGELOG-1.9.md][19]
* [CHANGELOG-1.8.md][18]
* [CHANGELOG-1.7.md][17]
* [CHANGELOG-1.6.md][16]
* [CHANGELOG-1.5.md][15]
* [CHANGELOG-1.4.md][14]
* [CHANGELOG-1.3.md][13]
@@ -24,11 +19,6 @@
* [CHANGELOG-0.3.md][1]
[21]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.11.md
[20]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.10.md
[19]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.9.md
[18]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.8.md
[17]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.7.md
[16]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.6.md
[15]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.5.md
[14]: https://github.com/vmware-tanzu/velero/blob/main/changelogs/CHANGELOG-1.4.md

View File

@@ -11,70 +11,52 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Velero binary build section
FROM --platform=$BUILDPLATFORM golang:1.20.7-bullseye as velero-builder
FROM --platform=$BUILDPLATFORM golang:1.15 as builder-env
ARG GOPROXY
ARG BIN
ARG PKG
ARG VERSION
ARG REGISTRY
ARG GIT_SHA
ARG GIT_TREE_STATE
ARG TARGETOS
ARG TARGETARCH
ARG TARGETVARIANT
ARG REGISTRY
ENV CGO_ENABLED=0 \
GO111MODULE=on \
GOPROXY=${GOPROXY} \
GOOS=${TARGETOS} \
GOARCH=${TARGETARCH} \
GOARM=${TARGETVARIANT} \
LDFLAGS="-X ${PKG}/pkg/buildinfo.Version=${VERSION} -X ${PKG}/pkg/buildinfo.GitSHA=${GIT_SHA} -X ${PKG}/pkg/buildinfo.GitTreeState=${GIT_TREE_STATE} -X ${PKG}/pkg/buildinfo.ImageRegistry=${REGISTRY}"
WORKDIR /go/src/github.com/vmware-tanzu/velero
COPY . /go/src/github.com/vmware-tanzu/velero
RUN mkdir -p /output/usr/bin && \
export GOARM=$( echo "${GOARM}" | cut -c2-) && \
go build -o /output/${BIN} \
-ldflags "${LDFLAGS}" ${PKG}/cmd/${BIN} && \
go build -o /output/velero-helper \
-ldflags "${LDFLAGS}" ${PKG}/cmd/velero-helper
RUN apt-get update && apt-get install -y bzip2
# Restic binary build section
FROM --platform=$BUILDPLATFORM golang:1.20.7-bullseye as restic-builder
FROM --platform=$BUILDPLATFORM builder-env as builder
ARG BIN
ARG TARGETOS
ARG TARGETARCH
ARG TARGETVARIANT
ARG PKG
ARG BIN
ARG RESTIC_VERSION
ENV CGO_ENABLED=0 \
GO111MODULE=on \
GOPROXY=${GOPROXY} \
GOOS=${TARGETOS} \
ENV GOOS=${TARGETOS} \
GOARCH=${TARGETARCH} \
GOARM=${TARGETVARIANT}
COPY . /go/src/github.com/vmware-tanzu/velero
RUN mkdir -p /output/usr/bin && \
export GOARM=$(echo "${GOARM}" | cut -c2-) && \
/go/src/github.com/vmware-tanzu/velero/hack/build-restic.sh
bash ./hack/download-restic.sh && \
export GOARM=$( echo "${GOARM}" | cut -c2-) && \
go build -o /output/${BIN} \
-ldflags "${LDFLAGS}" ${PKG}/cmd/${BIN}
# Velero image packing section
FROM gcr.io/distroless/base-nossl-debian11@sha256:f10e1fbf558c630a4b74a987e6c754d45bf59f9ddcefce090f6b111925996767
FROM ubuntu:focal
LABEL maintainer="Xun Jiang <jxun@vmware.com>"
LABEL maintainer="Nolan Brubaker <brubakern@vmware.com>"
COPY --from=velero-builder /output /
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -qq -y ca-certificates tzdata && rm -rf /var/lib/apt/lists/*
COPY --from=restic-builder /output /
COPY --from=builder /output /
USER nonroot:nonroot
USER nobody:nogroup

View File

@@ -4,36 +4,26 @@
## Maintainers
| Maintainer | GitHub ID | Affiliation |
|---------------------|---------------------------------------------------------------|-------------------------------------------|
| Dave Smith-Uchida | [dsu-igeek](https://github.com/dsu-igeek) | [Kasten](https://github.com/kastenhq/) |
| Scott Seago | [sseago](https://github.com/sseago) | [OpenShift](https://github.com/openshift) |
| Daniel Jiang | [reasonerjt](https://github.com/reasonerjt) | [VMware](https://www.github.com/vmware/) |
| Wenkai Yin | [ywk253100](https://github.com/ywk253100) | [VMware](https://www.github.com/vmware/) |
| Xun Jiang | [blackpiglet](https://github.com/blackpiglet) | [VMware](https://www.github.com/vmware/) |
| Ming Qiu | [qiuming-best](https://github.com/qiuming-best) | [VMware](https://www.github.com/vmware/) |
| Shubham Pampattiwar | [shubham-pampattiwar](https://github.com/shubham-pampattiwar) | [OpenShift](https://github.com/openshift) |
| Yonghui Li | [Lyndon-Li](https://github.com/Lyndon-Li) | [VMware](https://www.github.com/vmware/) |
| Maintainer | GitHub ID | Affiliation |
| --------------- | --------- | ----------- |
| Carlisia Thompson | [carlisia](https://github.com/carlisia) | [VMware](https://www.github.com/vmware/) |
| Nolan Brubaker | [nrb](https://github.com/nrb) | [VMware](https://www.github.com/vmware/) |
| Ashish Amarnath | [ashish-amarnath](https://github.com/ashish-amarnath) | [VMware](https://www.github.com/vmware/) |
| Bridget McErlean | [zubron](https://github.com/zubron) | [VMware](https://www.github.com/vmware/) |
| Dave Smith-Uchida | [dsu-igeek](https://github.com/dsu-igeek) | [VMware](https://www.github.com/vmware/) |
| JenTing Hsiao | [jenting](https://github.com/jenting) | [SUSE](https://github.com/SUSE/)
## Emeritus Maintainers
* Adnan Abdulhussein ([prydonius](https://github.com/prydonius))
* Andy Goldstein ([ncdc](https://github.com/ncdc))
* Steve Kriss ([skriss](https://github.com/skriss))
* Carlos Panato ([cpanato](https://github.com/cpanato))
* Nolan Brubaker ([nrb](https://github.com/nrb))
* Ashish Amarnath ([ashish-amarnath](https://github.com/ashish-amarnath))
* Carlisia Thompson ([carlisia](https://github.com/carlisia))
* Bridget McErlean ([zubron](https://github.com/zubron))
* JenTing Hsiao ([jenting](https://github.com/jenting))
## Velero Contributors & Stakeholders
| Feature Area | Lead |
|------------------------|:------------------------------------------------------------------------------------:|
| Architect | Dave Smith-Uchida [dsu-igeek](https://github.com/dsu-igeek) |
| Technical Lead | Daniel Jiang [reasonerjt](https://github.com/reasonerjt) |
| Kubernetes CSI Liaison | |
| Deployment | |
| Community Management | Orlin Vasilev [OrlinVasilev](https://github.com/OrlinVasilev) |
| Product Management | Pradeep Kumar Chaturvedi [pradeepkchaturvedi](https://github.com/pradeepkchaturvedi) |
| Feature Area | Lead |
| ----------------------------- | :---------------------: |
| Technical Lead | Nolan Brubaker (nrb) |
| Kubernetes CSI Liaison | Nolan Brubaker (nrb), Ashish Amarnath (ashish-amarnath) |
| Deployment | Carlisia Thompson (carlisia), Carlos Tadeu Panato Junior (cpanato), JenTing Hsiao (jenting) |
| Community Management | Jonas Rosland (jonasrosland) |
| Product Management | Michael Michael (michmike) |

View File

@@ -22,11 +22,9 @@ PKG := github.com/vmware-tanzu/velero
# Where to push the docker image.
REGISTRY ?= velero
GCR_REGISTRY ?= gcr.io/velero-gcp
# Image name
IMAGE ?= $(REGISTRY)/$(BIN)
GCR_IMAGE ?= $(GCR_REGISTRY)/$(BIN)
# We allow the Dockerfile to be configurable to enable the use of custom Dockerfiles
# that pull base images from different registries.
@@ -68,10 +66,8 @@ TAG_LATEST ?= false
ifeq ($(TAG_LATEST), true)
IMAGE_TAGS ?= $(IMAGE):$(VERSION) $(IMAGE):latest
GCR_IMAGE_TAGS ?= $(GCR_IMAGE):$(VERSION) $(GCR_IMAGE):latest
else
IMAGE_TAGS ?= $(IMAGE):$(VERSION)
GCR_IMAGE_TAGS ?= $(GCR_IMAGE):$(VERSION)
endif
ifeq ($(shell docker buildx inspect 2>/dev/null | awk '/Status/ { print $$2 }'), running)
@@ -85,10 +81,10 @@ buildx not enabled, refusing to run this recipe
see: https://velero.io/docs/main/build-from-source/#making-images-and-updating-velero for more info
endef
# The version of restic binary to be downloaded
RESTIC_VERSION ?= 0.15.0
# The version of restic binary to be downloaded for power architecture
RESTIC_VERSION ?= 0.12.0
CLI_PLATFORMS ?= linux-amd64 linux-arm linux-arm64 darwin-amd64 darwin-arm64 windows-amd64 linux-ppc64le
CLI_PLATFORMS ?= linux-amd64 linux-arm linux-arm64 darwin-amd64 windows-amd64 linux-ppc64le
BUILDX_PLATFORMS ?= $(subst -,/,$(ARCH))
BUILDX_OUTPUT_TYPE ?= docker
@@ -100,6 +96,9 @@ else
GIT_TREE_STATE ?= clean
endif
# The default linters used by lint and local-lint
LINTERS ?= "gosec,goconst,gofmt,goimports,unparam"
###
### These variables should not need tweaking.
###
@@ -113,20 +112,19 @@ GOPROXY ?= https://proxy.golang.org
# If you want to build all containers, see the 'all-containers' rule.
all:
@$(MAKE) build
@$(MAKE) build BIN=velero-restore-helper
@$(MAKE) build BIN=velero-restic-restore-helper
build-%:
@$(MAKE) --no-print-directory ARCH=$* build
@$(MAKE) --no-print-directory ARCH=$* build BIN=velero-restore-helper
@$(MAKE) --no-print-directory ARCH=$* build BIN=velero-restic-restore-helper
all-build: $(addprefix build-, $(CLI_PLATFORMS))
all-containers:
all-containers: container-builder-env
@$(MAKE) --no-print-directory container
@$(MAKE) --no-print-directory container BIN=velero-restore-helper
@$(MAKE) --no-print-directory container BIN=velero-restic-restore-helper
local: build-dirs
# Add DEBUG=1 to enable debug locally
GOOS=$(GOOS) \
GOARCH=$(GOARCH) \
VERSION=$(VERSION) \
@@ -164,7 +162,6 @@ shell: build-dirs build-env
@# under $GOPATH).
@docker run \
-e GOFLAGS \
-e GOPROXY \
-i $(TTY) \
--rm \
-u $$(id -u):$$(id -g) \
@@ -179,6 +176,20 @@ shell: build-dirs build-env
$(BUILDER_IMAGE) \
/bin/sh $(CMD)
container-builder-env:
ifneq ($(BUILDX_ENABLED), true)
$(error $(BUILDX_ERROR))
endif
@docker buildx build \
--target=builder-env \
--build-arg=GOPROXY=$(GOPROXY) \
--build-arg=PKG=$(PKG) \
--build-arg=VERSION=$(VERSION) \
--build-arg=GIT_SHA=$(GIT_SHA) \
--build-arg=GIT_TREE_STATE=$(GIT_TREE_STATE) \
--build-arg=REGISTRY=$(REGISTRY) \
-f $(VELERO_DOCKERFILE) .
container:
ifneq ($(BUILDX_ENABLED), true)
$(error $(BUILDX_ERROR))
@@ -187,8 +198,6 @@ endif
--output=type=$(BUILDX_OUTPUT_TYPE) \
--platform $(BUILDX_PLATFORMS) \
$(addprefix -t , $(IMAGE_TAGS)) \
$(addprefix -t , $(GCR_IMAGE_TAGS)) \
--build-arg=GOPROXY=$(GOPROXY) \
--build-arg=PKG=$(PKG) \
--build-arg=BIN=$(BIN) \
--build-arg=VERSION=$(VERSION) \
@@ -198,12 +207,6 @@ endif
--build-arg=RESTIC_VERSION=$(RESTIC_VERSION) \
-f $(VELERO_DOCKERFILE) .
@echo "container: $(IMAGE):$(VERSION)"
ifeq ($(BUILDX_OUTPUT_TYPE)_$(REGISTRY), registry_velero)
docker pull $(IMAGE):$(VERSION)
rm -f $(BIN)-$(VERSION).tar
docker save $(IMAGE):$(VERSION) -o $(BIN)-$(VERSION).tar
gzip -f $(BIN)-$(VERSION).tar
endif
SKIP_TESTS ?=
test: build-dirs
@@ -223,21 +226,27 @@ endif
lint:
ifneq ($(SKIP_TESTS), 1)
@$(MAKE) shell CMD="-c 'hack/lint.sh'"
@$(MAKE) shell CMD="-c 'hack/lint.sh $(LINTERS)'"
endif
local-lint:
ifneq ($(SKIP_TESTS), 1)
@hack/lint.sh
@hack/lint.sh $(LINTERS)
endif
lint-all:
ifneq ($(SKIP_TESTS), 1)
@$(MAKE) shell CMD="-c 'hack/lint.sh $(LINTERS) true'"
endif
local-lint-all:
ifneq ($(SKIP_TESTS), 1)
@hack/lint.sh $(LINTERS) true
endif
update:
@$(MAKE) shell CMD="-c 'hack/update-all.sh'"
# update-crd is for development purpose only, it is faster than update, so is a shortcut when you want to generate CRD changes only
update-crd:
@$(MAKE) shell CMD="-c 'hack/update-3generated-crd-code.sh'"
build-dirs:
@mkdir -p _output/bin/$(GOOS)/$(GOARCH)
@mkdir -p .go/src/$(PKG) .go/pkg .go/bin .go/std/$(GOOS)/$(GOARCH) .go/go-build .go/golangci-lint
@@ -281,12 +290,12 @@ push-build-image:
@# this target will push the build-image it assumes you already have docker
@# credentials needed to accomplish this.
@# Pushing will be skipped if a custom Dockerfile was used to build the image.
ifneq "$(origin BUILDER_IMAGE_DOCKERFILE)" "file"
@echo "Dockerfile for builder image has been overridden"
@echo "Skipping push of custom image"
else
docker push $(BUILDER_IMAGE)
endif
ifneq "$(origin BUILDER_IMAGE_DOCKERFILE)" "file"
@echo "Dockerfile for builder image has been overridden"
@echo "Skipping push of custom image"
else
docker push $(BUILDER_IMAGE)
endif
build-image-hugo:
cd site && docker build --pull -t $(HUGO_IMAGE) .
@@ -329,9 +338,9 @@ changelog:
# PUBLISH=false \
# make release
#
# To run the release, which will publish a *DRAFT* GitHub release in github.com/vmware-tanzu/velero
# To run the release, which will publish a *DRAFT* GitHub release in github.com/vmware-tanzu/velero
# (you still need to review/publish the GitHub release manually):
# GITHUB_TOKEN=your-github-token \
# GITHUB_TOKEN=your-github-token \
# RELEASE_NOTES_FILE=changelogs/CHANGELOG-1.2.md \
# PUBLISH=true \
# make release
@@ -349,19 +358,12 @@ serve-docs: build-image-hugo
-v "$$(pwd)/site:/srv/hugo" \
-it -p 1313:1313 \
$(HUGO_IMAGE) \
server --bind=0.0.0.0 --enableGitInfo=false
# gen-docs generates a new versioned docs directory under site/content/docs.
hugo server --bind=0.0.0.0 --enableGitInfo=false
# gen-docs generates a new versioned docs directory under site/content/docs.
# Please read the documentation in the script for instructions on how to use it.
gen-docs:
@hack/release-tools/gen-docs.sh
.PHONY: test-e2e
test-e2e: local
$(MAKE) -e VERSION=$(VERSION) -C test/e2e run
.PHONY: test-perf
test-perf: local
$(MAKE) -e VERSION=$(VERSION) -C test/perf run
go-generate:
go generate ./pkg/...
$(MAKE) -C test/e2e run

7
PROJECT Normal file
View File

@@ -0,0 +1,7 @@
domain: io
repo: github.com/vmware-tanzu/velero
resources:
- group: velero
kind: BackupStorageLocation
version: v1
version: "2"

View File

@@ -1,13 +1,11 @@
![100]
[![Build Status][1]][2] [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/3811/badge)](https://bestpractices.coreinfrastructure.org/projects/3811)
![GitHub release (latest SemVer)](https://img.shields.io/github/v/release/vmware-tanzu/velero)
## Overview
Velero (formerly Heptio Ark) gives you tools to back up and restore your Kubernetes cluster resources and persistent volumes. You can run Velero with a public cloud platform or on-premises.
Velero lets you:
Velero (formerly Heptio Ark) gives you tools to back up and restore your Kubernetes cluster resources and persistent volumes. You can run Velero with a public cloud platform or on-premises. Velero lets you:
* Take backups of your cluster and restore in case of loss.
* Migrate cluster resources to other clusters.
@@ -20,7 +18,7 @@ Velero consists of:
## Documentation
[The documentation][29] provides a getting started guide and information about building from source, architecture, extending Velero and more.
[The documentation][29] provides a getting started guide and information about building from source, architecture, extending Velero, and more.
Please use the version selector at the top of the site to ensure you are using the appropriate documentation for your version of Velero.
@@ -36,26 +34,6 @@ If you are ready to jump in and test, add code, or help with documentation, foll
See [the list of releases][6] to find out about feature changes.
### Velero compatibility matrix
The following is a list of the supported Kubernetes versions for each Velero version.
| Velero version | Expected Kubernetes version compatibility | Tested on Kubernetes version |
|----------------|-------------------------------------------|----------------------------------------|
| 1.12 | 1.18-latest | 1.25.7, 1.26.5, 1.26.7, and 1.27.3 |
| 1.11 | 1.18-latest | 1.23.10, 1.24.9, 1.25.5, and 1.26.1 |
| 1.10 | 1.18-latest | 1.22.5, 1.23.8, 1.24.6 and 1.25.1 |
| 1.9 | 1.18-latest | 1.20.5, 1.21.2, 1.22.5, 1.23, and 1.24 |
| 1.8 | 1.18-latest | |
Velero supports IPv4, IPv6, and dual stack environments. Support for this was tested against Velero v1.8.
The Velero maintainers are continuously working to expand testing coverage, but are not able to test every combination of Velero and supported Kubernetes versions for each Velero release. The table above is meant to track the current testing coverage and the expected supported Kubernetes versions for each Velero version. If you have a question about test coverage before v1.9, please reach out in the [#velero-users](https://kubernetes.slack.com/archives/C6VCGP4MT) Slack channel.
If you are interested in using a different version of Kubernetes with a given Velero version, we'd recommend that you perform testing before installing or upgrading your environment. For full information around capabilities within a release, also see the Velero [release notes](https://github.com/vmware-tanzu/velero/releases) or Kubernetes [release notes](https://github.com/kubernetes/kubernetes/tree/master/CHANGELOG). See the Velero [support page](https://velero.io/docs/latest/support-process/) for information about supported versions of Velero.
For each release, Velero maintainers run the test to ensure the upgrade path from n-2 minor release. For example, before the release of v1.10.x, the test will verify that the backup created by v1.9.x and v1.8.x can be restored using the build to be tagged as v1.10.x.
[1]: https://github.com/vmware-tanzu/velero/workflows/Main%20CI/badge.svg
[2]: https://github.com/vmware-tanzu/velero/actions?query=workflow%3A"Main+CI"
[4]: https://github.com/vmware-tanzu/velero/issues

View File

@@ -1 +1,69 @@
# Please go to the [Velero Wiki](https://github.com/vmware-tanzu/velero/wiki/) to see our latest roadmap, archived roadmaps and roadmap guidance.
## Velero Roadmap
### About this document
This document provides a link to the [Velero Project boards](https://github.com/vmware-tanzu/velero/projects) that serves as the up to date description of items that are in the release pipeline. The release boards have separate swim lanes based on prioritization. Most items are gathered from the community or include a feedback loop with the community. This should serve as a reference point for Velero users and contributors to understand where the project is heading, and help determine if a contribution could be conflicting with a longer term plan.
### How to help?
Discussion on the roadmap can take place in threads under [Issues](https://github.com/vmware-tanzu/velero/issues) or in [community meetings](https://velero.io/community/). Please open and comment on an issue if you want to provide suggestions, use cases, and feedback to an item in the roadmap. Please review the roadmap to avoid potential duplicated effort.
### How to add an item to the roadmap?
One of the most important aspects in any open source community is the concept of proposals. Large changes to the codebase and / or new features should be preceded by a [proposal](https://github.com/vmware-tanzu/velero/blob/main/GOVERNANCE.md#proposal-process) in our repo.
For smaller enhancements, you can open an issue to track that initiative or feature request.
We work with and rely on community feedback to focus our efforts to improve Velero and maintain a healthy roadmap.
### Current Roadmap
The following table includes the current roadmap for Velero. If you have any questions or would like to contribute to Velero, please attend a [community meeting](https://velero.io/community/) to discuss with our team. If you don't know where to start, we are always looking for contributors that will help us reduce technical, automation, and documentation debt.
Please take the timelines & dates as proposals and goals. Priorities and requirements change based on community feedback, roadblocks encountered, community contributions, etc. If you depend on a specific item, we encourage you to attend community meetings to get updated status information, or help us deliver that feature by contributing to Velero.
`Last Updated: March 2021`
#### 1.7.0 Roadmap
The release roadmap is split into Core items that are required for the release, desired items that may be removed from the
release and opportunistic items that will be added to the release if possible.
##### Core items
|Issue|Description|
|---|---|
|[3493](https://github.com/vmware-tanzu/velero/issues/3493)|[Carvel](https://github.com/vmware-tanzu/velero/issues/3493) based installation (in addition to the existing *velero install* CLI).|
|[3531](https://github.com/vmware-tanzu/velero/issues/3531)|Test plan for Velero|
|[675](https://github.com/vmware-tanzu/velero/issues/675)|Velero command to generate debugging information. Will integrate with [Crashd - Crash Diagnostics](https://github.com/vmware-tanzu/velero/issues/675)|
|[2066](https://github.com/vmware-tanzu/velero/issues/2066)|CSI Snapshots GA|
|[3285](https://github.com/vmware-tanzu/velero/issues/3285)|Support Velero plugin versioning|
|[1975](https://github.com/vmware-tanzu/velero/issues/1975)|IPV6 support|
##### Desired items
|Issue|Description|
|---|---|
|[3533](https://github.com/vmware-tanzu/velero/issues/3533)|Upload Progress Monitoring|
|[2922](https://github.com/vmware-tanzu/velero/issues/2922)|Plugin timeouts|
|[3500](https://github.com/vmware-tanzu/velero/issues/3500)|Use distroless containers as a base|
|[3535](https://github.com/vmware-tanzu/velero/issues/3535)|Design doc for multiple cluster support|
|[3536](https://github.com/vmware-tanzu/velero/issues/3536)|Manifest for backup/restore|
##### Opportunistic items
|Issue|Description|
|---|---|
|Issues TBD|Controller migrations|
#### Long term roadmap items
|Theme|Description|Timeline|
|---|---|---|
|Restic Improvements|Introduce improvements in annotating resources for Restic backup|TBD|
|Extensibility|Add restore hooks for enhanced recovery scenarios|TBD|
|CSI|Continue improving the CSI snapshot capabilities and participate in the upstream K8s CSI community|1.7.0 + Long running (dependent on CSI working group)|
|Backup/Restore|Improvements to long-running copy operations from a performance and reliability standpoint|1.7.0|
|Quality/Reliability| Enable automated end-to-end testing |1.6.0|
|UX|Improvements to install and configuration user experience|Dec 2020|
|Restic Improvements|Improve the use of Restic in Velero and offer stable support|TBD|
|Perf & Scale|Introduce a scalable model by using a worker pod for each backup/restore operation and improve operations|1.8.0|
|Backup/Restore|Better backup and restore semantics for certain Kubernetes resources like stateful sets, operators|2.0|
|Security|Enable the use of custom credential providers|1.6.0|
|Self-Service & Multitenancy|Reduce friction by enabling developers to backup their namespaces via self-service. Introduce a Velero multi-tenancy model, enabling owners of namespaces to backup and restore within their access scope|TBD|
|Backup/Restore|Cross availability zone or region backup and restore|TBD|
|Application Consistency|Offer blueprints for backing up and restoring popular applications|TBD|
|Backup/Restore|Data only backup and restore|TBD|
|Backup/Restore|Introduce the ability to overwrite existing objects during a restore|TBD|
|Backup/Restore|What-if dry run for backup and restore|1.7.0|

View File

@@ -7,19 +7,17 @@ k8s_yaml([
'config/crd/v1/bases/velero.io_downloadrequests.yaml',
'config/crd/v1/bases/velero.io_podvolumebackups.yaml',
'config/crd/v1/bases/velero.io_podvolumerestores.yaml',
'config/crd/v1/bases/velero.io_backuprepositories.yaml',
'config/crd/v1/bases/velero.io_resticrepositories.yaml',
'config/crd/v1/bases/velero.io_restores.yaml',
'config/crd/v1/bases/velero.io_schedules.yaml',
'config/crd/v1/bases/velero.io_serverstatusrequests.yaml',
'config/crd/v1/bases/velero.io_volumesnapshotlocations.yaml',
'config/crd/v2alpha1/bases/velero.io_datauploads.yaml',
'config/crd/v2alpha1/bases/velero.io_datadownloads.yaml',
])
# default values
settings = {
"default_registry": "docker.io/velero",
"use_node_agent": False,
"default_registry": "",
"enable_restic": False,
"enable_debug": False,
"debug_continue_on_start": True, # Continue the velero process by default when in debug mode
"create_backup_locations": False,
@@ -36,9 +34,9 @@ k8s_yaml(kustomize('tilt-resources'))
k8s_yaml('tilt-resources/deployment.yaml')
if settings.get("enable_debug"):
k8s_resource('velero', port_forwards = '2345')
# TODO: Need to figure out how to apply port forwards for all node-agent pods
if settings.get("use_node_agent"):
k8s_yaml('tilt-resources/node-agent.yaml')
# TODO: Need to figure out how to apply port forwards for all restic pods
if settings.get("enable_restic"):
k8s_yaml('tilt-resources/restic.yaml')
if settings.get("create_backup_locations"):
k8s_yaml('tilt-resources/velero_v1_backupstoragelocation.yaml')
if settings.get("setup-minio"):
@@ -52,7 +50,7 @@ git_sha = str(local("git rev-parse HEAD", quiet = True, echo_off = True)).strip(
tilt_helper_dockerfile_header = """
# Tilt image
FROM golang:1.20.7 as tilt-helper
FROM golang:1.15.3 as tilt-helper
# Support live reloading with Tilt
RUN wget --output-document /restart.sh --quiet https://raw.githubusercontent.com/windmilleng/rerun-process-wrapper/master/restart.sh && \
@@ -62,9 +60,9 @@ RUN wget --output-document /restart.sh --quiet https://raw.githubusercontent.com
additional_docker_helper_commands = """
# Install delve to allow debugging
RUN go install github.com/go-delve/delve/cmd/dlv@latest
RUN go get github.com/go-delve/delve/cmd/dlv
RUN wget -qO- https://dl.k8s.io/v1.25.2/kubernetes-client-linux-amd64.tar.gz | tar xvz
RUN wget -qO- https://dl.k8s.io/v1.19.2/kubernetes-client-linux-amd64.tar.gz | tar xvz
RUN wget -qO- https://get.docker.com | sh
"""
@@ -92,25 +90,25 @@ def get_debug_flag():
# Set up a local_resource build of the Velero binary. The binary is written to _tiltbuild/velero.
local_resource(
"velero_server_binary",
cmd = 'cd ' + '.' + ';mkdir -p _tiltbuild;PKG=. BIN=velero GOOS=linux GOARCH=amd64 GIT_SHA=' + git_sha + ' VERSION=main GIT_TREE_STATE=dirty OUTPUT_DIR=_tiltbuild ' + get_debug_flag() + ' REGISTRY=' + settings.get("default_registry") + ' ./hack/build.sh',
cmd = 'cd ' + '.' + ';mkdir -p _tiltbuild;PKG=. BIN=velero GOOS=linux GOARCH=amd64 GIT_SHA=' + git_sha + ' VERSION=main GIT_TREE_STATE=dirty OUTPUT_DIR=_tiltbuild ' + get_debug_flag() + ' ./hack/build.sh',
deps = ["cmd", "internal", "pkg"],
ignore = ["pkg/cmd"],
)
local_resource(
"velero_local_binary",
cmd = 'cd ' + '.' + ';mkdir -p _tiltbuild/local;PKG=. BIN=velero GOOS=' + local_goos + ' GOARCH=amd64 GIT_SHA=' + git_sha + ' VERSION=main GIT_TREE_STATE=dirty OUTPUT_DIR=_tiltbuild/local ' + get_debug_flag() + ' REGISTRY=' + settings.get("default_registry") + ' ./hack/build.sh',
cmd = 'cd ' + '.' + ';mkdir -p _tiltbuild/local;PKG=. BIN=velero GOOS=' + local_goos + ' GOARCH=amd64 GIT_SHA=' + git_sha + ' VERSION=main GIT_TREE_STATE=dirty OUTPUT_DIR=_tiltbuild/local ' + get_debug_flag() + ' ./hack/build.sh',
deps = ["internal", "pkg/cmd"],
)
local_resource(
"restic_binary",
cmd = 'cd ' + '.' + ';mkdir -p _tiltbuild/restic; BIN=velero GOOS=linux GOARCH=amd64 GOARM="" RESTIC_VERSION=0.13.1 OUTPUT_DIR=_tiltbuild/restic ./hack/build-restic.sh',
cmd = 'cd ' + '.' + ';mkdir -p _tiltbuild/restic; BIN=velero GOOS=' + local_goos + ' GOARCH=amd64 RESTIC_VERSION=0.12.0 OUTPUT_DIR=_tiltbuild/restic ./hack/download-restic.sh',
)
# Note: we need a distro with a bash shell to exec into the Velero container
tilt_dockerfile_header = """
FROM ubuntu:22.04 as tilt
FROM ubuntu:focal as tilt
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -qq -y ca-certificates tzdata && rm -rf /var/lib/apt/lists/*
@@ -218,7 +216,7 @@ def enable_provider(provider):
# Note: we need a distro with a shell to do a copy of the plugin binary
tilt_dockerfile_header = """
FROM ubuntu:22.04 as tilt
FROM ubuntu:focal as tilt
WORKDIR /
COPY --from=tilt-helper /start.sh .
COPY --from=tilt-helper /restart.sh .

View File

@@ -1,11 +0,0 @@
# Velero Assets
This folder contains logo images for Velero in gray (for light backgrounds) and white (for dark backgrounds like black t-shirts or dark mode!) horizontal and stacked… in .eps and .svg.
## Some general guidelines for usage
• Dont alter the logos/graphics: resize, reformat, recolor. Keep them intact.
• Dont separate the word mark (Velero) from the icon) we are still building a strong name and identity and the logo by itself doesnt have any strong recognition or association with as yet: so best practice keep the two together. Nike kept its name with the swoosh for quite some time before the swoosh became iconic.
• Dont append the name to another brand let it stand alone!

Binary file not shown.

Before

Width:  |  Height:  |  Size: 129 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 36 KiB

View File

@@ -1,105 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Generator: Adobe Illustrator 24.0.3, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
<svg version="1.1" id="Layer_5" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
viewBox="0 0 431.3 150" style="enable-background:new 0 0 431.3 150;" xml:space="preserve">
<style type="text/css">
.st0{fill:#FFFFFF;}
.st1{fill:none;}
.st2{fill:#009BDB;}
.st3{fill:#717074;}
</style>
<g>
<g>
<g>
<path class="st3" d="M196.6,55.8l-18.2,41.2h-5.1l-18.2-41.2h5.1l15.7,35.5l15.6-35.5H196.6z"/>
<path class="st3" d="M206.6,60.3v13.6h22.4v4.4h-22.4v14.3h24.8v4.4h-29.6V55.8h29.6v4.4H206.6z"/>
<path class="st3" d="M265.7,92.6v4.4h-27.2V55.8h4.7v36.8H265.7z"/>
<path class="st3" d="M275.7,60.3v13.6h22.4v4.4h-22.4v14.3h24.8v4.4H271V55.8h29.6v4.4H275.7z"/>
<path class="st3" d="M338,75.3c-1.1,1.9-2.6,3.4-4.6,4.6c-2,1.2-4.2,1.9-6.6,2.2l10.5,14.9H332l-10.4-14.8h-9.1v14.8h-4.7V55.8
h16.7c2.8,0,5.4,0.6,7.7,1.7c2.3,1.1,4.1,2.7,5.5,4.7c1.3,2,2,4.3,2,6.8C339.6,71.3,339.1,73.4,338,75.3z M312.4,77.8h11.2
c3.4,0,6.1-0.8,8.2-2.3c2.1-1.6,3.1-3.7,3.1-6.4c0-2.7-1-4.9-3.1-6.4c-2.1-1.6-4.8-2.3-8.2-2.3h-11.2V77.8z"/>
<path class="st3" d="M354.4,94.9c-3.3-1.9-5.8-4.5-7.8-7.8c-1.9-3.3-2.9-6.8-2.9-10.6c0-3.8,1-7.3,2.9-10.6
c1.9-3.3,4.5-5.9,7.8-7.8c3.3-1.9,6.8-2.9,10.5-2.9c3.8,0,7.2,1,10.5,2.9c3.2,1.9,5.8,4.5,7.7,7.8c1.9,3.3,2.9,6.8,2.9,10.6
c0,3.8-1,7.3-2.9,10.6c-1.9,3.3-4.5,5.9-7.7,7.8c-3.2,1.9-6.7,2.9-10.5,2.9C361.2,97.8,357.7,96.8,354.4,94.9z M373,91.1
c2.5-1.6,4.5-3.6,6-6.2c1.5-2.6,2.2-5.4,2.2-8.5c0-3-0.7-5.8-2.2-8.4c-1.5-2.6-3.5-4.7-6-6.2c-2.5-1.5-5.2-2.3-8.1-2.3
c-2.9,0-5.6,0.8-8.1,2.3c-2.5,1.5-4.5,3.6-6,6.2c-1.5,2.6-2.2,5.4-2.2,8.4c0,3,0.8,5.9,2.2,8.5c1.5,2.6,3.5,4.7,6,6.2
s5.2,2.3,8.1,2.3C367.8,93.5,370.5,92.7,373,91.1z"/>
</g>
</g>
<g>
<g>
<path class="st1" d="M132.2,77.1c0-0.7,0.1-1.4,0.1-2s0-1.4-0.1-2V77.1z"/>
<path class="st2" d="M117,109.3c-3.3,0-4.9,1.2-6.7,2.7c-2,1.6-4.2,3.4-8.6,3.4c-4.3,0-6.6-1.8-8.6-3.4c-1.8-1.4-3.4-2.7-6.7-2.7
c-3.3,0-4.9,1.2-6.7,2.7c-2,1.6-4.2,3.4-8.6,3.4c-4.3,0-6.6-1.8-8.6-3.4c-1.8-1.4-3.4-2.7-6.7-2.7c-3.3,0-4.9,1.2-6.7,2.7
c-1.3,1-2.7,2.2-4.8,2.8c8.3,7.3,18.9,12,30.5,13c0.3,0,0.6,0.1,1,0.1c1.1,0.1,2.3,0.1,3.4,0.1c1.2,0,2.3-0.1,3.4-0.1
c0.3,0,0.6,0,1-0.1c14.2-1.2,26.8-8,35.6-18.2C118.7,109.4,117.9,109.3,117,109.3z"/>
<path class="st2" d="M40.8,69.8c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2s6.1,1.6,8,3.2c1.9,1.5,3.6,2.9,7.2,2.9
c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2c4,0,6.1,1.6,8,3.2c1.9,1.5,3.6,2.9,7.2,2.9c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2
c4,0,6.1,1.6,8,3.2c1.8,1.5,3.6,2.8,7,2.9c-0.5-4.8-1.6-9.5-3.3-13.8c-1.8-0.6-3.1-1.6-4.4-2.6c-1.9-1.5-3.7-2.9-7.4-2.9
c-3.7,0-5.5,1.4-7.4,2.9c-1.9,1.5-3.9,3.1-7.9,3.1c-4,0-5.9-1.6-7.9-3.1c-1.9-1.5-3.7-2.9-7.4-2.9c-3.7,0-5.5,1.4-7.4,2.9
c-1.9,1.5-3.9,3.1-7.9,3.1c-4,0-5.9-1.6-7.9-3.1c-1.9-1.5-3.7-2.9-7.4-2.9c-3.7,0-5.5,1.4-7.4,2.9c-1.9,1.5-3.9,3.1-7.9,3.1
c-4,0-5.9-1.6-7.9-3.1c-0.4-0.3-0.9-0.7-1.3-1c-1.7,3.6-3.1,7.5-3.9,11.6c2.7,0.5,4.3,1.7,5.9,3C35.5,68.4,37.2,69.8,40.8,69.8z"
/>
<path class="st2" d="M40.8,55.7c3.7,0,5.5-1.4,7.4-2.9c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.9,1.5,3.7,2.9,7.4,2.9
c3.7,0,5.5-1.4,7.4-2.9c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.9,1.5,3.7,2.9,7.4,2.9c3.7,0,5.5-1.4,7.4-2.9
c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.1,0.8,2.1,1.7,3.4,2.2c-7.9-19.2-26.9-32.8-48.9-32.8c-20.8,0-38.7,12-47.4,29.5
c0.5,0.4,1,0.7,1.4,1.1C35.3,54.2,37.1,55.7,40.8,55.7z"/>
<path class="st2" d="M117,94.6c-3.4,0-5,1.3-6.8,2.8c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3c-1.9-1.5-3.5-2.8-6.8-2.8
c-3.4,0-5,1.3-6.8,2.8c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3c-1.9-1.5-3.5-2.8-6.8-2.8c-3.4,0-5,1.3-6.8,2.8
c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3c-0.5-0.4-1-0.8-1.5-1.1c2.6,6,6.3,11.4,10.9,16c2.6-0.2,4-1.4,5.6-2.6
c2-1.6,4.2-3.4,8.6-3.4s6.6,1.8,8.6,3.4c1.8,1.4,3.4,2.7,6.7,2.7c3.3,0,4.9-1.2,6.7-2.7c2-1.6,4.2-3.4,8.6-3.4
c4.3,0,6.6,1.8,8.6,3.4c1.8,1.4,3.4,2.7,6.7,2.7c3.3,0,4.9-1.2,6.7-2.7c2-1.6,4.2-3.4,8.6-3.4c1.8,0,3.3,0.3,4.5,0.8
c1.9-2.5,3.5-5.1,5-7.9c-1-0.6-1.8-1.2-2.6-1.8C122,95.8,120.3,94.6,117,94.6z"/>
<path class="st2" d="M132.1,71.2c-4,0-6-1.6-8-3.2c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2
c-4,0-6.1-1.6-8-3.2c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2c-4,0-6.1-1.6-8-3.2
c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2c-4,0-6.1-1.6-8-3.2c-1.5-1.2-3-2.3-5.3-2.7
c-0.5,2.5-0.8,5.1-0.9,7.7c0,0.7-0.1,1.4-0.1,2c0,0.7,0,1.4,0.1,2c0,0.3,0,0.6,0,0.9c3.5,0.3,5.4,1.8,7.2,3.2
c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8c1.9-1.5,4.1-3.2,8.2-3.2s6.3,1.7,8.2,3.2c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8
c1.9-1.5,4.1-3.2,8.2-3.2c4.1,0,6.3,1.7,8.2,3.2c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8c1.9-1.5,4.1-3.2,8.2-3.2
c4.1,0,6.3,1.7,8.2,3.2c1.7,1.3,3.3,2.6,6.3,2.8c0.3-1.6,0.5-3.2,0.6-4.9c0-0.6,0.1-1.3,0.1-1.9V73
C132.2,72.4,132.1,71.2,132.1,71.2z"/>
<path class="st2" d="M117,79.9c-3.5,0-5.2,1.4-7,2.8c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.8-1.4-3.5-2.8-7-2.8
c-3.5,0-5.2,1.4-7,2.8c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.8-1.4-3.5-2.8-7-2.8c-3.5,0-5.2,1.4-7,2.8
c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.6-1.3-3.1-2.5-5.8-2.8c0.4,4.5,1.4,8.7,2.8,12.8c1.9,0.6,3.2,1.7,4.4,2.7
c1.9,1.5,3.5,2.8,6.8,2.8c3.4,0,5-1.3,6.8-2.8c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c1.9,1.5,3.5,2.8,6.8,2.8
c3.4,0,5-1.3,6.8-2.8c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c1.9,1.5,3.5,2.8,6.8,2.8c3.4,0,5-1.3,6.8-2.8
c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c0.7,0.6,1.4,1.1,2.2,1.6c1.6-3.5,2.8-7.2,3.6-11.1c-3.5-0.3-5.4-1.8-7.2-3.2
C122.2,81.2,120.4,79.9,117,79.9z"/>
<path class="st0" d="M108.4,109.6c-1.8,1.4-3.4,2.7-6.7,2.7c-3.3,0-4.9-1.2-6.7-2.7c-2-1.6-4.2-3.4-8.6-3.4
c-4.3,0-6.6,1.8-8.6,3.4c-1.8,1.4-3.4,2.7-6.7,2.7c-3.3,0-4.9-1.2-6.7-2.7c-2-1.6-4.2-3.4-8.6-3.4s-6.6,1.8-8.6,3.4
c-1.6,1.3-3,2.4-5.6,2.6c0.9,0.9,1.8,1.7,2.7,2.5c2.1-0.6,3.5-1.8,4.8-2.8c1.8-1.4,3.4-2.7,6.7-2.7c3.3,0,4.9,1.2,6.7,2.7
c2,1.6,4.2,3.4,8.6,3.4c4.3,0,6.6-1.8,8.6-3.4c1.8-1.4,3.4-2.7,6.7-2.7c3.3,0,4.9,1.2,6.7,2.7c2,1.6,4.2,3.4,8.6,3.4
c4.3,0,6.6-1.8,8.6-3.4c1.8-1.4,3.4-2.7,6.7-2.7c0.9,0,1.7,0.1,2.4,0.3c0.7-0.8,1.4-1.7,2-2.5c-1.2-0.5-2.7-0.8-4.5-0.8
C112.6,106.2,110.4,108,108.4,109.6z"/>
<path class="st0" d="M117,92.1c-4.2,0-6.4,1.7-8.4,3.3c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.9-1.5-4.1-3.3-8.4-3.3
c-4.2,0-6.4,1.7-8.4,3.3c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.9-1.5-4.1-3.3-8.4-3.3c-4.2,0-6.4,1.7-8.4,3.3
c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.2-1-2.5-2-4.4-2.7c0.4,1.2,0.9,2.3,1.4,3.5c0.5,0.3,1,0.7,1.5,1.1
c1.9,1.5,4.1,3.3,8.4,3.3c4.2,0,6.4-1.7,8.4-3.3c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c1.9,1.5,4.1,3.3,8.4,3.3
c4.2,0,6.4-1.7,8.4-3.3c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c1.9,1.5,4.1,3.3,8.4,3.3c4.2,0,6.4-1.7,8.4-3.3
c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c0.8,0.6,1.6,1.3,2.6,1.8c0.4-0.7,0.7-1.5,1.1-2.2c-0.8-0.4-1.4-1-2.2-1.6
C123.4,93.8,121.2,92.1,117,92.1z"/>
<path class="st0" d="M117,77.9c-4.1,0-6.3,1.7-8.2,3.2c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.9-1.5-4.1-3.2-8.2-3.2
c-4.1,0-6.3,1.7-8.2,3.2c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.9-1.5-4.1-3.2-8.2-3.2c-4.1,0-6.3,1.7-8.2,3.2
c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.7-1.4-3.7-2.9-7.2-3.2c0,0.6,0.1,1.3,0.1,1.9c2.7,0.3,4.2,1.5,5.8,2.8
c1.9,1.5,4.1,3.2,8.2,3.2c4.1,0,6.3-1.7,8.2-3.2c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.9,1.5,4.1,3.2,8.2,3.2
c4.1,0,6.3-1.7,8.2-3.2c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.9,1.5,4.1,3.2,8.2,3.2c4.1,0,6.3-1.7,8.2-3.2
c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.7,1.4,3.7,2.9,7.2,3.2c0.1-0.6,0.2-1.3,0.3-1.9c-3-0.2-4.6-1.4-6.3-2.8
C123.3,79.6,121.1,77.9,117,77.9z"/>
<path class="st0" d="M40.8,71.2c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9c3.6,0,5.3,1.4,7.2,2.9c2,1.6,4,3.2,8,3.2
c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9c3.6,0,5.3,1.4,7.2,2.9c2,1.6,4,3.2,8,3.2c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9
c3.6,0,5.3,1.4,7.2,2.9c1.9,1.5,4,3.1,8,3.2l0-0.1c0-0.4-0.1-0.8-0.1-1.3c-3.4-0.1-5.2-1.4-7-2.9c-2-1.6-4-3.2-8-3.2
c-4,0-6.1,1.6-8,3.2c-1.9,1.5-3.6,2.9-7.2,2.9c-3.6,0-5.3-1.4-7.2-2.9c-2-1.6-4-3.2-8-3.2c-4,0-6.1,1.6-8,3.2
c-1.9,1.5-3.6,2.9-7.2,2.9c-3.6,0-5.3-1.4-7.2-2.9c-2-1.6-4-3.2-8-3.2s-6.1,1.6-8,3.2c-1.9,1.5-3.6,2.9-7.2,2.9
c-3.6,0-5.3-1.4-7.2-2.9c-1.6-1.3-3.2-2.5-5.9-3c-0.1,0.4-0.2,0.9-0.3,1.3c2.4,0.4,3.8,1.5,5.3,2.7
C34.7,69.6,36.7,71.2,40.8,71.2z"/>
<path class="st0" d="M40.8,56.5c4,0,5.9-1.6,7.9-3.1c1.9-1.5,3.7-2.9,7.4-2.9c3.7,0,5.5,1.4,7.4,2.9c1.9,1.5,3.9,3.1,7.9,3.1
c4,0,5.9-1.6,7.9-3.1c1.9-1.5,3.7-2.9,7.4-2.9c3.7,0,5.5,1.4,7.4,2.9c1.9,1.5,3.9,3.1,7.9,3.1c4,0,5.9-1.6,7.9-3.1
c1.9-1.5,3.7-2.9,7.4-2.9c3.7,0,5.5,1.4,7.4,2.9c1.3,1,2.5,2,4.4,2.6c-0.1-0.3-0.3-0.7-0.4-1c-1.3-0.6-2.4-1.4-3.4-2.2
c-1.9-1.5-3.9-3.1-7.9-3.1c-4,0-5.9,1.6-7.9,3.1c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9c-1.9-1.5-3.9-3.1-7.9-3.1
c-4,0-5.9,1.6-7.9,3.1c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9C62,51.2,60,49.6,56,49.6c-4,0-5.9,1.6-7.9,3.1
c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9C33,52.4,32.5,52,32,51.6c-0.1,0.2-0.3,0.5-0.4,0.8c0.4,0.3,0.9,0.6,1.3,1
C34.8,54.9,36.8,56.5,40.8,56.5z"/>
</g>
</g>
</g>
</svg>

Before

Width:  |  Height:  |  Size: 9.3 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 34 KiB

View File

@@ -1,105 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Generator: Adobe Illustrator 24.0.3, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
<svg version="1.1" id="Layer_5" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
viewBox="0 0 431.3 150" style="enable-background:new 0 0 431.3 150;" xml:space="preserve">
<style type="text/css">
.st0{fill:#FFFFFF;}
.st1{fill:none;}
.st2{fill:#009BDB;}
.st3{fill:#717074;}
</style>
<g>
<g>
<g>
<path class="st0" d="M196.6,55.8l-18.2,41.2h-5.1l-18.2-41.2h5.1l15.7,35.5l15.6-35.5H196.6z"/>
<path class="st0" d="M206.6,60.3v13.6h22.4v4.4h-22.4v14.3h24.8v4.4h-29.6V55.8h29.6v4.4H206.6z"/>
<path class="st0" d="M265.7,92.6v4.4h-27.2V55.8h4.7v36.8H265.7z"/>
<path class="st0" d="M275.7,60.3v13.6h22.4v4.4h-22.4v14.3h24.8v4.4H271V55.8h29.6v4.4H275.7z"/>
<path class="st0" d="M338,75.3c-1.1,1.9-2.6,3.4-4.6,4.6c-2,1.2-4.2,1.9-6.6,2.2l10.5,14.9H332l-10.4-14.8h-9.1v14.8h-4.7V55.8
h16.7c2.8,0,5.4,0.6,7.7,1.7c2.3,1.1,4.1,2.7,5.5,4.7c1.3,2,2,4.3,2,6.8C339.6,71.3,339.1,73.4,338,75.3z M312.4,77.8h11.2
c3.4,0,6.1-0.8,8.2-2.3c2.1-1.6,3.1-3.7,3.1-6.4c0-2.7-1-4.9-3.1-6.4c-2.1-1.6-4.8-2.3-8.2-2.3h-11.2V77.8z"/>
<path class="st0" d="M354.4,94.9c-3.3-1.9-5.8-4.5-7.8-7.8c-1.9-3.3-2.9-6.8-2.9-10.6c0-3.8,1-7.3,2.9-10.6
c1.9-3.3,4.5-5.9,7.8-7.8c3.3-1.9,6.8-2.9,10.5-2.9c3.8,0,7.2,1,10.5,2.9c3.2,1.9,5.8,4.5,7.7,7.8c1.9,3.3,2.9,6.8,2.9,10.6
c0,3.8-1,7.3-2.9,10.6c-1.9,3.3-4.5,5.9-7.7,7.8c-3.2,1.9-6.7,2.9-10.5,2.9C361.2,97.8,357.7,96.8,354.4,94.9z M373,91.1
c2.5-1.6,4.5-3.6,6-6.2c1.5-2.6,2.2-5.4,2.2-8.5c0-3-0.7-5.8-2.2-8.4c-1.5-2.6-3.5-4.7-6-6.2c-2.5-1.5-5.2-2.3-8.1-2.3
c-2.9,0-5.6,0.8-8.1,2.3c-2.5,1.5-4.5,3.6-6,6.2c-1.5,2.6-2.2,5.4-2.2,8.4c0,3,0.8,5.9,2.2,8.5c1.5,2.6,3.5,4.7,6,6.2
s5.2,2.3,8.1,2.3C367.8,93.5,370.5,92.7,373,91.1z"/>
</g>
</g>
<g>
<g>
<path class="st1" d="M132.2,77.1c0-0.7,0.1-1.4,0.1-2s0-1.4-0.1-2V77.1z"/>
<path class="st2" d="M117,109.3c-3.3,0-4.9,1.2-6.7,2.7c-2,1.6-4.2,3.4-8.6,3.4c-4.3,0-6.6-1.8-8.6-3.4c-1.8-1.4-3.4-2.7-6.7-2.7
c-3.3,0-4.9,1.2-6.7,2.7c-2,1.6-4.2,3.4-8.6,3.4c-4.3,0-6.6-1.8-8.6-3.4c-1.8-1.4-3.4-2.7-6.7-2.7c-3.3,0-4.9,1.2-6.7,2.7
c-1.3,1-2.7,2.2-4.8,2.8c8.3,7.3,18.9,12,30.5,13c0.3,0,0.6,0.1,1,0.1c1.1,0.1,2.3,0.1,3.4,0.1c1.2,0,2.3-0.1,3.4-0.1
c0.3,0,0.6,0,1-0.1c14.2-1.2,26.8-8,35.6-18.2C118.7,109.4,117.9,109.3,117,109.3z"/>
<path class="st2" d="M40.8,69.8c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2s6.1,1.6,8,3.2c1.9,1.5,3.6,2.9,7.2,2.9
c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2c4,0,6.1,1.6,8,3.2c1.9,1.5,3.6,2.9,7.2,2.9c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2
c4,0,6.1,1.6,8,3.2c1.8,1.5,3.6,2.8,7,2.9c-0.5-4.8-1.6-9.5-3.3-13.8c-1.8-0.6-3.1-1.6-4.4-2.6c-1.9-1.5-3.7-2.9-7.4-2.9
c-3.7,0-5.5,1.4-7.4,2.9c-1.9,1.5-3.9,3.1-7.9,3.1c-4,0-5.9-1.6-7.9-3.1c-1.9-1.5-3.7-2.9-7.4-2.9c-3.7,0-5.5,1.4-7.4,2.9
c-1.9,1.5-3.9,3.1-7.9,3.1c-4,0-5.9-1.6-7.9-3.1c-1.9-1.5-3.7-2.9-7.4-2.9c-3.7,0-5.5,1.4-7.4,2.9c-1.9,1.5-3.9,3.1-7.9,3.1
c-4,0-5.9-1.6-7.9-3.1c-0.4-0.3-0.9-0.7-1.3-1c-1.7,3.6-3.1,7.5-3.9,11.6c2.7,0.5,4.3,1.7,5.9,3C35.5,68.4,37.2,69.8,40.8,69.8z"
/>
<path class="st2" d="M40.8,55.7c3.7,0,5.5-1.4,7.4-2.9c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.9,1.5,3.7,2.9,7.4,2.9
c3.7,0,5.5-1.4,7.4-2.9c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.9,1.5,3.7,2.9,7.4,2.9c3.7,0,5.5-1.4,7.4-2.9
c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.1,0.8,2.1,1.7,3.4,2.2c-7.9-19.2-26.9-32.8-48.9-32.8c-20.8,0-38.7,12-47.4,29.5
c0.5,0.4,1,0.7,1.4,1.1C35.3,54.2,37.1,55.7,40.8,55.7z"/>
<path class="st2" d="M117,94.6c-3.4,0-5,1.3-6.8,2.8c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3c-1.9-1.5-3.5-2.8-6.8-2.8
c-3.4,0-5,1.3-6.8,2.8c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3c-1.9-1.5-3.5-2.8-6.8-2.8c-3.4,0-5,1.3-6.8,2.8
c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3c-0.5-0.4-1-0.8-1.5-1.1c2.6,6,6.3,11.4,10.9,16c2.6-0.2,4-1.4,5.6-2.6
c2-1.6,4.2-3.4,8.6-3.4s6.6,1.8,8.6,3.4c1.8,1.4,3.4,2.7,6.7,2.7c3.3,0,4.9-1.2,6.7-2.7c2-1.6,4.2-3.4,8.6-3.4
c4.3,0,6.6,1.8,8.6,3.4c1.8,1.4,3.4,2.7,6.7,2.7c3.3,0,4.9-1.2,6.7-2.7c2-1.6,4.2-3.4,8.6-3.4c1.8,0,3.3,0.3,4.5,0.8
c1.9-2.5,3.5-5.1,5-7.9c-1-0.6-1.8-1.2-2.6-1.8C122,95.8,120.3,94.6,117,94.6z"/>
<path class="st2" d="M132.1,71.2c-4,0-6-1.6-8-3.2c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2
c-4,0-6.1-1.6-8-3.2c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2c-4,0-6.1-1.6-8-3.2
c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2c-4,0-6.1-1.6-8-3.2c-1.5-1.2-3-2.3-5.3-2.7
c-0.5,2.5-0.8,5.1-0.9,7.7c0,0.7-0.1,1.4-0.1,2c0,0.7,0,1.4,0.1,2c0,0.3,0,0.6,0,0.9c3.5,0.3,5.4,1.8,7.2,3.2
c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8c1.9-1.5,4.1-3.2,8.2-3.2s6.3,1.7,8.2,3.2c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8
c1.9-1.5,4.1-3.2,8.2-3.2c4.1,0,6.3,1.7,8.2,3.2c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8c1.9-1.5,4.1-3.2,8.2-3.2
c4.1,0,6.3,1.7,8.2,3.2c1.7,1.3,3.3,2.6,6.3,2.8c0.3-1.6,0.5-3.2,0.6-4.9c0-0.6,0.1-1.3,0.1-1.9V73
C132.2,72.4,132.1,71.2,132.1,71.2z"/>
<path class="st2" d="M117,79.9c-3.5,0-5.2,1.4-7,2.8c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.8-1.4-3.5-2.8-7-2.8
c-3.5,0-5.2,1.4-7,2.8c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.8-1.4-3.5-2.8-7-2.8c-3.5,0-5.2,1.4-7,2.8
c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.6-1.3-3.1-2.5-5.8-2.8c0.4,4.5,1.4,8.7,2.8,12.8c1.9,0.6,3.2,1.7,4.4,2.7
c1.9,1.5,3.5,2.8,6.8,2.8c3.4,0,5-1.3,6.8-2.8c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c1.9,1.5,3.5,2.8,6.8,2.8
c3.4,0,5-1.3,6.8-2.8c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c1.9,1.5,3.5,2.8,6.8,2.8c3.4,0,5-1.3,6.8-2.8
c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c0.7,0.6,1.4,1.1,2.2,1.6c1.6-3.5,2.8-7.2,3.6-11.1c-3.5-0.3-5.4-1.8-7.2-3.2
C122.2,81.2,120.4,79.9,117,79.9z"/>
<path class="st0" d="M108.4,109.6c-1.8,1.4-3.4,2.7-6.7,2.7c-3.3,0-4.9-1.2-6.7-2.7c-2-1.6-4.2-3.4-8.6-3.4
c-4.3,0-6.6,1.8-8.6,3.4c-1.8,1.4-3.4,2.7-6.7,2.7c-3.3,0-4.9-1.2-6.7-2.7c-2-1.6-4.2-3.4-8.6-3.4s-6.6,1.8-8.6,3.4
c-1.6,1.3-3,2.4-5.6,2.6c0.9,0.9,1.8,1.7,2.7,2.5c2.1-0.6,3.5-1.8,4.8-2.8c1.8-1.4,3.4-2.7,6.7-2.7c3.3,0,4.9,1.2,6.7,2.7
c2,1.6,4.2,3.4,8.6,3.4c4.3,0,6.6-1.8,8.6-3.4c1.8-1.4,3.4-2.7,6.7-2.7c3.3,0,4.9,1.2,6.7,2.7c2,1.6,4.2,3.4,8.6,3.4
c4.3,0,6.6-1.8,8.6-3.4c1.8-1.4,3.4-2.7,6.7-2.7c0.9,0,1.7,0.1,2.4,0.3c0.7-0.8,1.4-1.7,2-2.5c-1.2-0.5-2.7-0.8-4.5-0.8
C112.6,106.2,110.4,108,108.4,109.6z"/>
<path class="st0" d="M117,92.1c-4.2,0-6.4,1.7-8.4,3.3c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.9-1.5-4.1-3.3-8.4-3.3
c-4.2,0-6.4,1.7-8.4,3.3c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.9-1.5-4.1-3.3-8.4-3.3c-4.2,0-6.4,1.7-8.4,3.3
c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.2-1-2.5-2-4.4-2.7c0.4,1.2,0.9,2.3,1.4,3.5c0.5,0.3,1,0.7,1.5,1.1
c1.9,1.5,4.1,3.3,8.4,3.3c4.2,0,6.4-1.7,8.4-3.3c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c1.9,1.5,4.1,3.3,8.4,3.3
c4.2,0,6.4-1.7,8.4-3.3c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c1.9,1.5,4.1,3.3,8.4,3.3c4.2,0,6.4-1.7,8.4-3.3
c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c0.8,0.6,1.6,1.3,2.6,1.8c0.4-0.7,0.7-1.5,1.1-2.2c-0.8-0.4-1.4-1-2.2-1.6
C123.4,93.8,121.2,92.1,117,92.1z"/>
<path class="st0" d="M117,77.9c-4.1,0-6.3,1.7-8.2,3.2c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.9-1.5-4.1-3.2-8.2-3.2
c-4.1,0-6.3,1.7-8.2,3.2c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.9-1.5-4.1-3.2-8.2-3.2c-4.1,0-6.3,1.7-8.2,3.2
c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.7-1.4-3.7-2.9-7.2-3.2c0,0.6,0.1,1.3,0.1,1.9c2.7,0.3,4.2,1.5,5.8,2.8
c1.9,1.5,4.1,3.2,8.2,3.2c4.1,0,6.3-1.7,8.2-3.2c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.9,1.5,4.1,3.2,8.2,3.2
c4.1,0,6.3-1.7,8.2-3.2c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.9,1.5,4.1,3.2,8.2,3.2c4.1,0,6.3-1.7,8.2-3.2
c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.7,1.4,3.7,2.9,7.2,3.2c0.1-0.6,0.2-1.3,0.3-1.9c-3-0.2-4.6-1.4-6.3-2.8
C123.3,79.6,121.1,77.9,117,77.9z"/>
<path class="st0" d="M40.8,71.2c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9c3.6,0,5.3,1.4,7.2,2.9c2,1.6,4,3.2,8,3.2
c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9c3.6,0,5.3,1.4,7.2,2.9c2,1.6,4,3.2,8,3.2c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9
c3.6,0,5.3,1.4,7.2,2.9c1.9,1.5,4,3.1,8,3.2l0-0.1c0-0.4-0.1-0.8-0.1-1.3c-3.4-0.1-5.2-1.4-7-2.9c-2-1.6-4-3.2-8-3.2
c-4,0-6.1,1.6-8,3.2c-1.9,1.5-3.6,2.9-7.2,2.9c-3.6,0-5.3-1.4-7.2-2.9c-2-1.6-4-3.2-8-3.2c-4,0-6.1,1.6-8,3.2
c-1.9,1.5-3.6,2.9-7.2,2.9c-3.6,0-5.3-1.4-7.2-2.9c-2-1.6-4-3.2-8-3.2s-6.1,1.6-8,3.2c-1.9,1.5-3.6,2.9-7.2,2.9
c-3.6,0-5.3-1.4-7.2-2.9c-1.6-1.3-3.2-2.5-5.9-3c-0.1,0.4-0.2,0.9-0.3,1.3c2.4,0.4,3.8,1.5,5.3,2.7
C34.7,69.6,36.7,71.2,40.8,71.2z"/>
<path class="st0" d="M40.8,56.5c4,0,5.9-1.6,7.9-3.1c1.9-1.5,3.7-2.9,7.4-2.9c3.7,0,5.5,1.4,7.4,2.9c1.9,1.5,3.9,3.1,7.9,3.1
c4,0,5.9-1.6,7.9-3.1c1.9-1.5,3.7-2.9,7.4-2.9c3.7,0,5.5,1.4,7.4,2.9c1.9,1.5,3.9,3.1,7.9,3.1c4,0,5.9-1.6,7.9-3.1
c1.9-1.5,3.7-2.9,7.4-2.9c3.7,0,5.5,1.4,7.4,2.9c1.3,1,2.5,2,4.4,2.6c-0.1-0.3-0.3-0.7-0.4-1c-1.3-0.6-2.4-1.4-3.4-2.2
c-1.9-1.5-3.9-3.1-7.9-3.1c-4,0-5.9,1.6-7.9,3.1c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9c-1.9-1.5-3.9-3.1-7.9-3.1
c-4,0-5.9,1.6-7.9,3.1c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9C62,51.2,60,49.6,56,49.6c-4,0-5.9,1.6-7.9,3.1
c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9C33,52.4,32.5,52,32,51.6c-0.1,0.2-0.3,0.5-0.4,0.8c0.4,0.3,0.9,0.6,1.3,1
C34.8,54.9,36.8,56.5,40.8,56.5z"/>
</g>
</g>
</g>
</svg>

Before

Width:  |  Height:  |  Size: 9.3 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 130 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 34 KiB

View File

@@ -1,103 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Generator: Adobe Illustrator 24.0.3, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
<svg version="1.1" id="Layer_5" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
viewBox="0 0 311.5 245.2" style="enable-background:new 0 0 311.5 245.2;" xml:space="preserve">
<style type="text/css">
.st0{fill:#FFFFFF;}
.st1{fill:none;}
.st2{fill:#009BDB;}
.st3{fill:#717074;}
</style>
<g>
<g>
<path class="st1" d="M211.5,83.4c0-0.7,0.1-1.4,0.1-2c0-0.7,0-1.4-0.1-2V83.4z"/>
<path class="st2" d="M196.3,115.5c-3.3,0-4.9,1.2-6.7,2.7c-2,1.6-4.2,3.4-8.6,3.4c-4.3,0-6.6-1.8-8.6-3.4
c-1.8-1.4-3.4-2.7-6.7-2.7c-3.3,0-4.9,1.2-6.7,2.7c-2,1.6-4.2,3.4-8.6,3.4c-4.3,0-6.6-1.8-8.6-3.4c-1.8-1.4-3.4-2.7-6.7-2.7
s-4.9,1.2-6.7,2.7c-1.3,1-2.7,2.2-4.8,2.8c8.3,7.3,18.9,12,30.5,13c0.3,0,0.6,0.1,1,0.1c1.1,0.1,2.3,0.1,3.4,0.1
c1.2,0,2.3-0.1,3.4-0.1c0.3,0,0.6,0,1-0.1c14.2-1.2,26.8-8,35.6-18.2C198,115.7,197.2,115.5,196.3,115.5z"/>
<path class="st2" d="M120.1,76.1c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2c4,0,6.1,1.6,8,3.2c1.9,1.5,3.6,2.9,7.2,2.9
c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2c4,0,6.1,1.6,8,3.2c1.9,1.5,3.6,2.9,7.2,2.9c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2
c4,0,6.1,1.6,8,3.2c1.8,1.5,3.6,2.8,7,2.9c-0.5-4.8-1.6-9.5-3.3-13.8c-1.8-0.6-3.1-1.6-4.4-2.6c-1.9-1.5-3.7-2.9-7.4-2.9
c-3.7,0-5.5,1.4-7.4,2.9c-1.9,1.5-3.9,3.1-7.9,3.1c-4,0-5.9-1.6-7.9-3.1c-1.9-1.5-3.7-2.9-7.4-2.9c-3.7,0-5.5,1.4-7.4,2.9
c-1.9,1.5-3.9,3.1-7.9,3.1c-4,0-5.9-1.6-7.9-3.1c-1.9-1.5-3.7-2.9-7.4-2.9s-5.5,1.4-7.4,2.9c-1.9,1.5-3.9,3.1-7.9,3.1
c-4,0-5.9-1.6-7.9-3.1c-0.4-0.3-0.9-0.7-1.3-1c-1.7,3.6-3.1,7.5-3.9,11.6c2.7,0.5,4.3,1.7,5.9,3C114.8,74.7,116.5,76.1,120.1,76.1
z"/>
<path class="st2" d="M120.1,61.9c3.7,0,5.5-1.4,7.4-2.9c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.9,1.5,3.7,2.9,7.4,2.9
c3.7,0,5.5-1.4,7.4-2.9c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.9,1.5,3.7,2.9,7.4,2.9c3.7,0,5.5-1.4,7.4-2.9
c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.1,0.8,2.1,1.7,3.4,2.2c-7.9-19.2-26.9-32.8-48.9-32.8c-20.8,0-38.7,12-47.4,29.5
c0.5,0.4,1,0.7,1.4,1.1C114.6,60.5,116.4,61.9,120.1,61.9z"/>
<path class="st2" d="M196.3,100.8c-3.4,0-5,1.3-6.8,2.8c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3
c-1.9-1.5-3.5-2.8-6.8-2.8c-3.4,0-5,1.3-6.8,2.8c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3c-1.9-1.5-3.5-2.8-6.8-2.8
c-3.4,0-5,1.3-6.8,2.8c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3c-0.5-0.4-1-0.8-1.5-1.1c2.6,6,6.3,11.4,10.9,16
c2.6-0.2,4-1.4,5.6-2.6c2-1.6,4.2-3.4,8.6-3.4c4.3,0,6.6,1.8,8.6,3.4c1.8,1.4,3.4,2.7,6.7,2.7c3.3,0,4.9-1.2,6.7-2.7
c2-1.6,4.2-3.4,8.6-3.4c4.3,0,6.6,1.8,8.6,3.4c1.8,1.4,3.4,2.7,6.7,2.7c3.3,0,4.9-1.2,6.7-2.7c2-1.6,4.2-3.4,8.6-3.4
c1.8,0,3.3,0.3,4.5,0.8c1.9-2.5,3.5-5.1,5-7.9c-1-0.6-1.8-1.2-2.6-1.8C201.3,102.1,199.6,100.8,196.3,100.8z"/>
<path class="st2" d="M211.4,77.5c-4,0-6-1.6-8-3.2c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2
c-4,0-6.1-1.6-8-3.2c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2c-4,0-6.1-1.6-8-3.2
c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2c-4,0-6.1-1.6-8-3.2c-1.5-1.2-3-2.3-5.3-2.7
c-0.5,2.5-0.8,5.1-0.9,7.7c0,0.7-0.1,1.4-0.1,2c0,0.7,0,1.4,0.1,2c0,0.3,0,0.6,0,0.9c3.5,0.3,5.4,1.8,7.2,3.2
c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8c1.9-1.5,4.1-3.2,8.2-3.2s6.3,1.7,8.2,3.2c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8
c1.9-1.5,4.1-3.2,8.2-3.2c4.1,0,6.3,1.7,8.2,3.2c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8c1.9-1.5,4.1-3.2,8.2-3.2
c4.1,0,6.3,1.7,8.2,3.2c1.7,1.3,3.3,2.6,6.3,2.8c0.3-1.6,0.5-3.2,0.6-4.9c0-0.6,0.1-1.3,0.1-1.9v-4.1
C211.5,78.6,211.4,77.5,211.4,77.5z"/>
<path class="st2" d="M196.3,86.1c-3.5,0-5.2,1.4-7,2.8c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.8-1.4-3.5-2.8-7-2.8
c-3.5,0-5.2,1.4-7,2.8c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.8-1.4-3.5-2.8-7-2.8c-3.5,0-5.2,1.4-7,2.8
c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.6-1.3-3.1-2.5-5.8-2.8c0.4,4.5,1.4,8.7,2.8,12.8c1.9,0.6,3.2,1.7,4.4,2.7
c1.9,1.5,3.5,2.8,6.8,2.8c3.4,0,5-1.3,6.8-2.8c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c1.9,1.5,3.5,2.8,6.8,2.8
c3.4,0,5-1.3,6.8-2.8c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c1.9,1.5,3.5,2.8,6.8,2.8c3.4,0,5-1.3,6.8-2.8
c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c0.7,0.6,1.4,1.1,2.2,1.6c1.6-3.5,2.8-7.2,3.6-11.1c-3.5-0.3-5.4-1.8-7.2-3.2
C201.5,87.5,199.7,86.1,196.3,86.1z"/>
<path class="st0" d="M187.7,115.9c-1.8,1.4-3.4,2.7-6.7,2.7c-3.3,0-4.9-1.2-6.7-2.7c-2-1.6-4.2-3.4-8.6-3.4
c-4.3,0-6.6,1.8-8.6,3.4c-1.8,1.4-3.4,2.7-6.7,2.7c-3.3,0-4.9-1.2-6.7-2.7c-2-1.6-4.2-3.4-8.6-3.4c-4.3,0-6.6,1.8-8.6,3.4
c-1.6,1.3-3,2.4-5.6,2.6c0.9,0.9,1.8,1.7,2.7,2.5c2.1-0.6,3.5-1.8,4.8-2.8c1.8-1.4,3.4-2.7,6.7-2.7c3.3,0,4.9,1.2,6.7,2.7
c2,1.6,4.2,3.4,8.6,3.4c4.3,0,6.6-1.8,8.6-3.4c1.8-1.4,3.4-2.7,6.7-2.7c3.3,0,4.9,1.2,6.7,2.7c2,1.6,4.2,3.4,8.6,3.4
c4.3,0,6.6-1.8,8.6-3.4c1.8-1.4,3.4-2.7,6.7-2.7c0.9,0,1.7,0.1,2.4,0.3c0.7-0.8,1.4-1.7,2-2.5c-1.2-0.5-2.7-0.8-4.5-0.8
C191.9,112.5,189.7,114.3,187.7,115.9z"/>
<path class="st0" d="M196.3,98.4c-4.2,0-6.4,1.7-8.4,3.3c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.9-1.5-4.1-3.3-8.4-3.3
c-4.2,0-6.4,1.7-8.4,3.3c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.9-1.5-4.1-3.3-8.4-3.3c-4.2,0-6.4,1.7-8.4,3.3
c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.2-1-2.5-2-4.4-2.7c0.4,1.2,0.9,2.3,1.4,3.5c0.5,0.3,1,0.7,1.5,1.1
c1.9,1.5,4.1,3.3,8.4,3.3c4.2,0,6.4-1.7,8.4-3.3c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c1.9,1.5,4.1,3.3,8.4,3.3
c4.2,0,6.4-1.7,8.4-3.3c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c1.9,1.5,4.1,3.3,8.4,3.3c4.2,0,6.4-1.7,8.4-3.3
c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c0.8,0.6,1.6,1.3,2.6,1.8c0.4-0.7,0.7-1.5,1.1-2.2c-0.8-0.4-1.4-1-2.2-1.6
C202.7,100.1,200.5,98.4,196.3,98.4z"/>
<path class="st0" d="M196.3,84.2c-4.1,0-6.3,1.7-8.2,3.2c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.9-1.5-4.1-3.2-8.2-3.2
c-4.1,0-6.3,1.7-8.2,3.2c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.9-1.5-4.1-3.2-8.2-3.2c-4.1,0-6.3,1.7-8.2,3.2
c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.7-1.4-3.7-2.9-7.2-3.2c0,0.6,0.1,1.3,0.1,1.9c2.7,0.3,4.2,1.5,5.8,2.8
c1.9,1.5,4.1,3.2,8.2,3.2c4.1,0,6.3-1.7,8.2-3.2c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.9,1.5,4.1,3.2,8.2,3.2
c4.1,0,6.3-1.7,8.2-3.2c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.9,1.5,4.1,3.2,8.2,3.2c4.1,0,6.3-1.7,8.2-3.2
c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.7,1.4,3.7,2.9,7.2,3.2c0.1-0.6,0.2-1.3,0.3-1.9c-3-0.2-4.6-1.4-6.3-2.8
C202.6,85.9,200.4,84.2,196.3,84.2z"/>
<path class="st0" d="M120.1,77.5c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9c3.6,0,5.3,1.4,7.2,2.9c2,1.6,4,3.2,8,3.2
c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9c3.6,0,5.3,1.4,7.2,2.9c2,1.6,4,3.2,8,3.2c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9
c3.6,0,5.3,1.4,7.2,2.9c1.9,1.5,4,3.1,8,3.2l0-0.1c0-0.4-0.1-0.8-0.1-1.3c-3.4-0.1-5.2-1.4-7-2.9c-2-1.6-4-3.2-8-3.2
c-4,0-6.1,1.6-8,3.2c-1.9,1.5-3.6,2.9-7.2,2.9c-3.6,0-5.3-1.4-7.2-2.9c-2-1.6-4-3.2-8-3.2c-4,0-6.1,1.6-8,3.2
c-1.9,1.5-3.6,2.9-7.2,2.9c-3.6,0-5.3-1.4-7.2-2.9c-2-1.6-4-3.2-8-3.2c-4,0-6.1,1.6-8,3.2c-1.9,1.5-3.6,2.9-7.2,2.9
c-3.6,0-5.3-1.4-7.2-2.9c-1.6-1.3-3.2-2.5-5.9-3c-0.1,0.4-0.2,0.9-0.3,1.3c2.4,0.4,3.8,1.5,5.3,2.7C114,75.9,116,77.5,120.1,77.5z
"/>
<path class="st0" d="M120.1,62.8c4,0,5.9-1.6,7.9-3.1c1.9-1.5,3.7-2.9,7.4-2.9s5.5,1.4,7.4,2.9c1.9,1.5,3.9,3.1,7.9,3.1
c4,0,5.9-1.6,7.9-3.1c1.9-1.5,3.7-2.9,7.4-2.9c3.7,0,5.5,1.4,7.4,2.9c1.9,1.5,3.9,3.1,7.9,3.1c4,0,5.9-1.6,7.9-3.1
c1.9-1.5,3.7-2.9,7.4-2.9c3.7,0,5.5,1.4,7.4,2.9c1.3,1,2.5,2,4.4,2.6c-0.1-0.3-0.3-0.7-0.4-1c-1.3-0.6-2.4-1.4-3.4-2.2
c-1.9-1.5-3.9-3.1-7.9-3.1c-4,0-5.9,1.6-7.9,3.1c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9c-1.9-1.5-3.9-3.1-7.9-3.1
c-4,0-5.9,1.6-7.9,3.1c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9c-1.9-1.5-3.9-3.1-7.9-3.1c-4,0-5.9,1.6-7.9,3.1
c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9c-0.5-0.4-0.9-0.7-1.4-1.1c-0.1,0.2-0.3,0.5-0.4,0.8c0.4,0.3,0.9,0.6,1.3,1
C114.1,61.2,116.1,62.8,120.1,62.8z"/>
</g>
</g>
<g>
<g>
<path class="st3" d="M81.7,161.9l-18.2,41.2h-5.1l-18.2-41.2h5.1L61,197.4l15.6-35.5H81.7z"/>
<path class="st3" d="M91.7,166.3v13.6h22.4v4.4H91.7v14.3h24.8v4.4H87v-41.2h29.6v4.4H91.7z"/>
<path class="st3" d="M150.9,198.7v4.4h-27.2v-41.2h4.7v36.8H150.9z"/>
<path class="st3" d="M160.9,166.3v13.6h22.4v4.4h-22.4v14.3h24.8v4.4h-29.6v-41.2h29.6v4.4H160.9z"/>
<path class="st3" d="M223.1,181.3c-1.1,1.9-2.6,3.4-4.6,4.6c-2,1.2-4.2,1.9-6.6,2.2l10.5,14.9h-5.3l-10.4-14.8h-9.1v14.8h-4.7
v-41.2h16.7c2.8,0,5.4,0.6,7.7,1.7c2.3,1.1,4.1,2.7,5.5,4.7c1.3,2,2,4.3,2,6.8C224.8,177.4,224.2,179.5,223.1,181.3z M197.5,183.9
h11.2c3.4,0,6.1-0.8,8.2-2.3c2.1-1.6,3.1-3.7,3.1-6.4c0-2.7-1-4.9-3.1-6.4c-2.1-1.6-4.8-2.3-8.2-2.3h-11.2V183.9z"/>
<path class="st3" d="M239.6,200.9c-3.3-1.9-5.8-4.5-7.8-7.8c-1.9-3.3-2.9-6.8-2.9-10.6c0-3.8,1-7.3,2.9-10.6
c1.9-3.3,4.5-5.9,7.8-7.8c3.3-1.9,6.8-2.9,10.5-2.9c3.8,0,7.2,1,10.5,2.9c3.2,1.9,5.8,4.5,7.7,7.8c1.9,3.3,2.9,6.8,2.9,10.6
c0,3.8-1,7.3-2.9,10.6c-1.9,3.3-4.5,5.9-7.7,7.8c-3.2,1.9-6.7,2.9-10.5,2.9C246.3,203.8,242.8,202.9,239.6,200.9z M258.2,197.2
c2.5-1.6,4.5-3.6,6-6.2c1.5-2.6,2.2-5.4,2.2-8.5c0-3-0.7-5.8-2.2-8.4c-1.5-2.6-3.5-4.7-6-6.2c-2.5-1.5-5.2-2.3-8.1-2.3
c-2.9,0-5.6,0.8-8.1,2.3c-2.5,1.5-4.5,3.6-6,6.2c-1.5,2.6-2.2,5.4-2.2,8.4c0,3,0.8,5.9,2.2,8.5c1.5,2.6,3.5,4.7,6,6.2
s5.2,2.3,8.1,2.3C253,199.5,255.7,198.7,258.2,197.2z"/>
</g>
</g>
</svg>

Before

Width:  |  Height:  |  Size: 9.3 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 34 KiB

View File

@@ -1,103 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Generator: Adobe Illustrator 24.0.3, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
<svg version="1.1" id="Layer_5" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
viewBox="0 0 311.5 245.2" style="enable-background:new 0 0 311.5 245.2;" xml:space="preserve">
<style type="text/css">
.st0{fill:#FFFFFF;}
.st1{fill:none;}
.st2{fill:#009BDB;}
.st3{fill:#717074;}
</style>
<g>
<g>
<path class="st1" d="M211.5,83.4c0-0.7,0.1-1.4,0.1-2c0-0.7,0-1.4-0.1-2V83.4z"/>
<path class="st2" d="M196.3,115.5c-3.3,0-4.9,1.2-6.7,2.7c-2,1.6-4.2,3.4-8.6,3.4c-4.3,0-6.6-1.8-8.6-3.4
c-1.8-1.4-3.4-2.7-6.7-2.7c-3.3,0-4.9,1.2-6.7,2.7c-2,1.6-4.2,3.4-8.6,3.4c-4.3,0-6.6-1.8-8.6-3.4c-1.8-1.4-3.4-2.7-6.7-2.7
s-4.9,1.2-6.7,2.7c-1.3,1-2.7,2.2-4.8,2.8c8.3,7.3,18.9,12,30.5,13c0.3,0,0.6,0.1,1,0.1c1.1,0.1,2.3,0.1,3.4,0.1
c1.2,0,2.3-0.1,3.4-0.1c0.3,0,0.6,0,1-0.1c14.2-1.2,26.8-8,35.6-18.2C198,115.7,197.2,115.5,196.3,115.5z"/>
<path class="st2" d="M120.1,76.1c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2c4,0,6.1,1.6,8,3.2c1.9,1.5,3.6,2.9,7.2,2.9
c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2c4,0,6.1,1.6,8,3.2c1.9,1.5,3.6,2.9,7.2,2.9c3.6,0,5.3-1.4,7.2-2.9c2-1.6,4-3.2,8-3.2
c4,0,6.1,1.6,8,3.2c1.8,1.5,3.6,2.8,7,2.9c-0.5-4.8-1.6-9.5-3.3-13.8c-1.8-0.6-3.1-1.6-4.4-2.6c-1.9-1.5-3.7-2.9-7.4-2.9
c-3.7,0-5.5,1.4-7.4,2.9c-1.9,1.5-3.9,3.1-7.9,3.1c-4,0-5.9-1.6-7.9-3.1c-1.9-1.5-3.7-2.9-7.4-2.9c-3.7,0-5.5,1.4-7.4,2.9
c-1.9,1.5-3.9,3.1-7.9,3.1c-4,0-5.9-1.6-7.9-3.1c-1.9-1.5-3.7-2.9-7.4-2.9s-5.5,1.4-7.4,2.9c-1.9,1.5-3.9,3.1-7.9,3.1
c-4,0-5.9-1.6-7.9-3.1c-0.4-0.3-0.9-0.7-1.3-1c-1.7,3.6-3.1,7.5-3.9,11.6c2.7,0.5,4.3,1.7,5.9,3C114.8,74.7,116.5,76.1,120.1,76.1
z"/>
<path class="st2" d="M120.1,61.9c3.7,0,5.5-1.4,7.4-2.9c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.9,1.5,3.7,2.9,7.4,2.9
c3.7,0,5.5-1.4,7.4-2.9c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.9,1.5,3.7,2.9,7.4,2.9c3.7,0,5.5-1.4,7.4-2.9
c1.9-1.5,3.9-3.1,7.9-3.1c4,0,5.9,1.6,7.9,3.1c1.1,0.8,2.1,1.7,3.4,2.2c-7.9-19.2-26.9-32.8-48.9-32.8c-20.8,0-38.7,12-47.4,29.5
c0.5,0.4,1,0.7,1.4,1.1C114.6,60.5,116.4,61.9,120.1,61.9z"/>
<path class="st2" d="M196.3,100.8c-3.4,0-5,1.3-6.8,2.8c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3
c-1.9-1.5-3.5-2.8-6.8-2.8c-3.4,0-5,1.3-6.8,2.8c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3c-1.9-1.5-3.5-2.8-6.8-2.8
c-3.4,0-5,1.3-6.8,2.8c-1.9,1.5-4.1,3.3-8.4,3.3c-4.2,0-6.4-1.7-8.4-3.3c-0.5-0.4-1-0.8-1.5-1.1c2.6,6,6.3,11.4,10.9,16
c2.6-0.2,4-1.4,5.6-2.6c2-1.6,4.2-3.4,8.6-3.4c4.3,0,6.6,1.8,8.6,3.4c1.8,1.4,3.4,2.7,6.7,2.7c3.3,0,4.9-1.2,6.7-2.7
c2-1.6,4.2-3.4,8.6-3.4c4.3,0,6.6,1.8,8.6,3.4c1.8,1.4,3.4,2.7,6.7,2.7c3.3,0,4.9-1.2,6.7-2.7c2-1.6,4.2-3.4,8.6-3.4
c1.8,0,3.3,0.3,4.5,0.8c1.9-2.5,3.5-5.1,5-7.9c-1-0.6-1.8-1.2-2.6-1.8C201.3,102.1,199.6,100.8,196.3,100.8z"/>
<path class="st2" d="M211.4,77.5c-4,0-6-1.6-8-3.2c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2
c-4,0-6.1-1.6-8-3.2c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2c-4,0-6.1-1.6-8-3.2
c-1.9-1.5-3.6-2.9-7.2-2.9c-3.6,0-5.3,1.4-7.2,2.9c-2,1.6-4,3.2-8,3.2c-4,0-6.1-1.6-8-3.2c-1.5-1.2-3-2.3-5.3-2.7
c-0.5,2.5-0.8,5.1-0.9,7.7c0,0.7-0.1,1.4-0.1,2c0,0.7,0,1.4,0.1,2c0,0.3,0,0.6,0,0.9c3.5,0.3,5.4,1.8,7.2,3.2
c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8c1.9-1.5,4.1-3.2,8.2-3.2s6.3,1.7,8.2,3.2c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8
c1.9-1.5,4.1-3.2,8.2-3.2c4.1,0,6.3,1.7,8.2,3.2c1.8,1.4,3.5,2.8,7,2.8c3.5,0,5.2-1.4,7-2.8c1.9-1.5,4.1-3.2,8.2-3.2
c4.1,0,6.3,1.7,8.2,3.2c1.7,1.3,3.3,2.6,6.3,2.8c0.3-1.6,0.5-3.2,0.6-4.9c0-0.6,0.1-1.3,0.1-1.9v-4.1
C211.5,78.6,211.4,77.5,211.4,77.5z"/>
<path class="st2" d="M196.3,86.1c-3.5,0-5.2,1.4-7,2.8c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.8-1.4-3.5-2.8-7-2.8
c-3.5,0-5.2,1.4-7,2.8c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.8-1.4-3.5-2.8-7-2.8c-3.5,0-5.2,1.4-7,2.8
c-1.9,1.5-4.1,3.2-8.2,3.2c-4.1,0-6.3-1.7-8.2-3.2c-1.6-1.3-3.1-2.5-5.8-2.8c0.4,4.5,1.4,8.7,2.8,12.8c1.9,0.6,3.2,1.7,4.4,2.7
c1.9,1.5,3.5,2.8,6.8,2.8c3.4,0,5-1.3,6.8-2.8c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c1.9,1.5,3.5,2.8,6.8,2.8
c3.4,0,5-1.3,6.8-2.8c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c1.9,1.5,3.5,2.8,6.8,2.8c3.4,0,5-1.3,6.8-2.8
c1.9-1.5,4.1-3.3,8.4-3.3c4.2,0,6.4,1.7,8.4,3.3c0.7,0.6,1.4,1.1,2.2,1.6c1.6-3.5,2.8-7.2,3.6-11.1c-3.5-0.3-5.4-1.8-7.2-3.2
C201.5,87.5,199.7,86.1,196.3,86.1z"/>
<path class="st0" d="M187.7,115.9c-1.8,1.4-3.4,2.7-6.7,2.7c-3.3,0-4.9-1.2-6.7-2.7c-2-1.6-4.2-3.4-8.6-3.4
c-4.3,0-6.6,1.8-8.6,3.4c-1.8,1.4-3.4,2.7-6.7,2.7c-3.3,0-4.9-1.2-6.7-2.7c-2-1.6-4.2-3.4-8.6-3.4c-4.3,0-6.6,1.8-8.6,3.4
c-1.6,1.3-3,2.4-5.6,2.6c0.9,0.9,1.8,1.7,2.7,2.5c2.1-0.6,3.5-1.8,4.8-2.8c1.8-1.4,3.4-2.7,6.7-2.7c3.3,0,4.9,1.2,6.7,2.7
c2,1.6,4.2,3.4,8.6,3.4c4.3,0,6.6-1.8,8.6-3.4c1.8-1.4,3.4-2.7,6.7-2.7c3.3,0,4.9,1.2,6.7,2.7c2,1.6,4.2,3.4,8.6,3.4
c4.3,0,6.6-1.8,8.6-3.4c1.8-1.4,3.4-2.7,6.7-2.7c0.9,0,1.7,0.1,2.4,0.3c0.7-0.8,1.4-1.7,2-2.5c-1.2-0.5-2.7-0.8-4.5-0.8
C191.9,112.5,189.7,114.3,187.7,115.9z"/>
<path class="st0" d="M196.3,98.4c-4.2,0-6.4,1.7-8.4,3.3c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.9-1.5-4.1-3.3-8.4-3.3
c-4.2,0-6.4,1.7-8.4,3.3c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.9-1.5-4.1-3.3-8.4-3.3c-4.2,0-6.4,1.7-8.4,3.3
c-1.9,1.5-3.5,2.8-6.8,2.8c-3.4,0-5-1.3-6.8-2.8c-1.2-1-2.5-2-4.4-2.7c0.4,1.2,0.9,2.3,1.4,3.5c0.5,0.3,1,0.7,1.5,1.1
c1.9,1.5,4.1,3.3,8.4,3.3c4.2,0,6.4-1.7,8.4-3.3c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c1.9,1.5,4.1,3.3,8.4,3.3
c4.2,0,6.4-1.7,8.4-3.3c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c1.9,1.5,4.1,3.3,8.4,3.3c4.2,0,6.4-1.7,8.4-3.3
c1.9-1.5,3.5-2.8,6.8-2.8c3.4,0,5,1.3,6.8,2.8c0.8,0.6,1.6,1.3,2.6,1.8c0.4-0.7,0.7-1.5,1.1-2.2c-0.8-0.4-1.4-1-2.2-1.6
C202.7,100.1,200.5,98.4,196.3,98.4z"/>
<path class="st0" d="M196.3,84.2c-4.1,0-6.3,1.7-8.2,3.2c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.9-1.5-4.1-3.2-8.2-3.2
c-4.1,0-6.3,1.7-8.2,3.2c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.9-1.5-4.1-3.2-8.2-3.2c-4.1,0-6.3,1.7-8.2,3.2
c-1.8,1.4-3.5,2.8-7,2.8c-3.5,0-5.2-1.4-7-2.8c-1.7-1.4-3.7-2.9-7.2-3.2c0,0.6,0.1,1.3,0.1,1.9c2.7,0.3,4.2,1.5,5.8,2.8
c1.9,1.5,4.1,3.2,8.2,3.2c4.1,0,6.3-1.7,8.2-3.2c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.9,1.5,4.1,3.2,8.2,3.2
c4.1,0,6.3-1.7,8.2-3.2c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.9,1.5,4.1,3.2,8.2,3.2c4.1,0,6.3-1.7,8.2-3.2
c1.8-1.4,3.5-2.8,7-2.8c3.5,0,5.2,1.4,7,2.8c1.7,1.4,3.7,2.9,7.2,3.2c0.1-0.6,0.2-1.3,0.3-1.9c-3-0.2-4.6-1.4-6.3-2.8
C202.6,85.9,200.4,84.2,196.3,84.2z"/>
<path class="st0" d="M120.1,77.5c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9c3.6,0,5.3,1.4,7.2,2.9c2,1.6,4,3.2,8,3.2
c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9c3.6,0,5.3,1.4,7.2,2.9c2,1.6,4,3.2,8,3.2c4,0,6.1-1.6,8-3.2c1.9-1.5,3.6-2.9,7.2-2.9
c3.6,0,5.3,1.4,7.2,2.9c1.9,1.5,4,3.1,8,3.2l0-0.1c0-0.4-0.1-0.8-0.1-1.3c-3.4-0.1-5.2-1.4-7-2.9c-2-1.6-4-3.2-8-3.2
c-4,0-6.1,1.6-8,3.2c-1.9,1.5-3.6,2.9-7.2,2.9c-3.6,0-5.3-1.4-7.2-2.9c-2-1.6-4-3.2-8-3.2c-4,0-6.1,1.6-8,3.2
c-1.9,1.5-3.6,2.9-7.2,2.9c-3.6,0-5.3-1.4-7.2-2.9c-2-1.6-4-3.2-8-3.2c-4,0-6.1,1.6-8,3.2c-1.9,1.5-3.6,2.9-7.2,2.9
c-3.6,0-5.3-1.4-7.2-2.9c-1.6-1.3-3.2-2.5-5.9-3c-0.1,0.4-0.2,0.9-0.3,1.3c2.4,0.4,3.8,1.5,5.3,2.7C114,75.9,116,77.5,120.1,77.5z
"/>
<path class="st0" d="M120.1,62.8c4,0,5.9-1.6,7.9-3.1c1.9-1.5,3.7-2.9,7.4-2.9s5.5,1.4,7.4,2.9c1.9,1.5,3.9,3.1,7.9,3.1
c4,0,5.9-1.6,7.9-3.1c1.9-1.5,3.7-2.9,7.4-2.9c3.7,0,5.5,1.4,7.4,2.9c1.9,1.5,3.9,3.1,7.9,3.1c4,0,5.9-1.6,7.9-3.1
c1.9-1.5,3.7-2.9,7.4-2.9c3.7,0,5.5,1.4,7.4,2.9c1.3,1,2.5,2,4.4,2.6c-0.1-0.3-0.3-0.7-0.4-1c-1.3-0.6-2.4-1.4-3.4-2.2
c-1.9-1.5-3.9-3.1-7.9-3.1c-4,0-5.9,1.6-7.9,3.1c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9c-1.9-1.5-3.9-3.1-7.9-3.1
c-4,0-5.9,1.6-7.9,3.1c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9c-1.9-1.5-3.9-3.1-7.9-3.1c-4,0-5.9,1.6-7.9,3.1
c-1.9,1.5-3.7,2.9-7.4,2.9c-3.7,0-5.5-1.4-7.4-2.9c-0.5-0.4-0.9-0.7-1.4-1.1c-0.1,0.2-0.3,0.5-0.4,0.8c0.4,0.3,0.9,0.6,1.3,1
C114.1,61.2,116.1,62.8,120.1,62.8z"/>
</g>
</g>
<g>
<g>
<path class="st0" d="M81.7,161.9l-18.2,41.2h-5.1l-18.2-41.2h5.1L61,197.4l15.6-35.5H81.7z"/>
<path class="st0" d="M91.7,166.3v13.6h22.4v4.4H91.7v14.3h24.8v4.4H87v-41.2h29.6v4.4H91.7z"/>
<path class="st0" d="M150.9,198.7v4.4h-27.2v-41.2h4.7v36.8H150.9z"/>
<path class="st0" d="M160.9,166.3v13.6h22.4v4.4h-22.4v14.3h24.8v4.4h-29.6v-41.2h29.6v4.4H160.9z"/>
<path class="st0" d="M223.1,181.3c-1.1,1.9-2.6,3.4-4.6,4.6c-2,1.2-4.2,1.9-6.6,2.2l10.5,14.9h-5.3l-10.4-14.8h-9.1v14.8h-4.7
v-41.2h16.7c2.8,0,5.4,0.6,7.7,1.7c2.3,1.1,4.1,2.7,5.5,4.7c1.3,2,2,4.3,2,6.8C224.8,177.4,224.2,179.5,223.1,181.3z M197.5,183.9
h11.2c3.4,0,6.1-0.8,8.2-2.3c2.1-1.6,3.1-3.7,3.1-6.4c0-2.7-1-4.9-3.1-6.4c-2.1-1.6-4.8-2.3-8.2-2.3h-11.2V183.9z"/>
<path class="st0" d="M239.6,200.9c-3.3-1.9-5.8-4.5-7.8-7.8c-1.9-3.3-2.9-6.8-2.9-10.6c0-3.8,1-7.3,2.9-10.6
c1.9-3.3,4.5-5.9,7.8-7.8c3.3-1.9,6.8-2.9,10.5-2.9c3.8,0,7.2,1,10.5,2.9c3.2,1.9,5.8,4.5,7.7,7.8c1.9,3.3,2.9,6.8,2.9,10.6
c0,3.8-1,7.3-2.9,10.6c-1.9,3.3-4.5,5.9-7.7,7.8c-3.2,1.9-6.7,2.9-10.5,2.9C246.3,203.8,242.8,202.9,239.6,200.9z M258.2,197.2
c2.5-1.6,4.5-3.6,6-6.2c1.5-2.6,2.2-5.4,2.2-8.5c0-3-0.7-5.8-2.2-8.4c-1.5-2.6-3.5-4.7-6-6.2c-2.5-1.5-5.2-2.3-8.1-2.3
c-2.9,0-5.6,0.8-8.1,2.3s-4.5,3.6-6,6.2s-2.2,5.4-2.2,8.4c0,3,0.8,5.9,2.2,8.5c1.5,2.6,3.5,4.7,6,6.2c2.5,1.6,5.2,2.3,8.1,2.3
C253,199.5,255.7,198.7,258.2,197.2z"/>
</g>
</g>
</svg>

Before

Width:  |  Height:  |  Size: 9.3 KiB

View File

@@ -154,7 +154,7 @@
* Skip completed jobs and pods when restoring (#463, @nrb)
* Set namespace correctly when syncing backups from object storage (#472, @skriss)
* When building on macOS, bind-mount volumes with delegated config (#478, @skriss)
* Add replica sets and daemonsets to cohabiting resources so they're not backed up twice (#482 #485, @skriss)
* Add replica sets and daemonsets to cohabitating resources so they're not backed up twice (#482 #485, @skriss)
* Shut down the Ark server gracefully on SIGINT/SIGTERM (#483, @skriss)
* Only back up resources that support GET and DELETE in addition to LIST and CREATE (#486, @nrb)
* Show a better error message when trying to get an incomplete restore's logs (#496, @nrb)

View File

@@ -1,190 +0,0 @@
## v1.10.0
### 2022-11-23
### Download
https://github.com/vmware-tanzu/velero/releases/tag/v1.10.0
### Container Image
`velero/velero:v1.10.0`
### Documentation
https://velero.io/docs/v1.10/
### Upgrading
https://velero.io/docs/v1.10/upgrade-to-1.10/
### Highlights
#### Unified Repository and Kopia integration
In this release, we introduced the Unified Repository architecture to build a data path where data movers and the backup repository are decoupled and a unified backup repository could serve various data movement activities.
In this release, we also deeply integrate Velero with Kopia, specifically, Kopia's uploader modules are isolated as a generic file system uploader; Kopia's repository modules are encapsulated as the unified backup repository.
For more information, refer to the [design document](https://github.com/vmware-tanzu/velero/blob/v1.10.0/design/unified-repo-and-kopia-integration/unified-repo-and-kopia-integration.md).
#### File system backup refactor
Velero's file system backup (a.k.s. pod volume backup or formerly restic backup) is refactored as the first user of the Unified Repository architecture. Specifically, we added a new path, the Kopia path, besides the existing Restic path. While Restic path is still available and set as default, you can opt in Kopia path by specifying the `uploader-type` parameter at installation time. Meanwhile, you are free to restore from existing backups under either path, Velero dynamically switches to the correct path to process the restore.
Because of the new path, we renamed some modules and parameters, refer to the Break Changes section for more details.
For more information, visit the [file system backup document](https://velero.io/docs/v1.10/file-system-backup/) and [v1.10 upgrade guide document](https://velero.io/docs/v1.10/upgrade-to-1.10/).
Meanwhile, we've created a performance guide for both Restic path and Kopia path, which helps you to choose between the two paths and provides you the best practice to configure them under different scenarios. Please note that the results in the guide are based on our testing environments, you may get different results when testing in your own ones. For more information, visit the [performance guide document](https://velero.io/docs/v1.10/performance-guidance/).
#### Plugin versioning V1 refactor
In this release, Velero moves plugins BackupItemAction, RestoreItemAction and VolumeSnapshotterAction to version v1, this allows future plugin changes that do not support backward compatibility, so is a preparation for various complex tasks, for example, data movement tasks.
For more information, refer to the [plugin versioning design document](https://github.com/vmware-tanzu/velero/blob/v1.10.0/design/plugin-versioning.md).
#### Refactor the controllers using Kubebuilder v3
In this release we continued our code modernization work, rewriting some controllers using Kubebuilder v3. This work is ongoing and we will continue to make progress in future releases.
#### Add credentials to volume snapshot locations
In this release, we enabled dedicate credentials options to volume snapshot locations so that you can specify credentials per volume snapshot location as same as backup storage location.
For more information, please visit the [locations document](https://velero.io/docs/v1.10/locations/).
#### CSI snapshot enhancements
In this release we added several changes to enhance the robustness of CSI snapshot procedures, for example, some protection code for error handling, and a mechanism to skip exclusion checks so that CSI snapshot works with various backup resource filters.
#### Backup schedule pause/unpause
In this release, Velero supports to pause/unpause a backup schedule during or after its creation. Specifically:
At creation time, you can specify `paused` flag to `velero schedule create` command, if so, you will create a paused schedule that will not run until it is unpaused
After creation, you can run `velero schedule pause` or `velero schedule unpause` command to pause/unpause a schedule
#### Runtime and dependencies
In order to fix CVEs, we changed Velero's runtime and dependencies as follows:
Bump go runtime to v1.18.8
Bump some core dependent libraries to newer versions
Compile Restic (v0.13.1) with go 1.18.8 instead of packaging the official binary
#### Breaking changes
Due to file system backup refactor, below modules and parameters name have been changed in this release:
`restic` daemonset is renamed to `node-agent`
`resticRepository` CR is renamed to `backupRepository`
`velero restic repo` command is renamed to `velero repo`
`velero-restic-credentials` secret is renamed to `velero-repo-credentials`
`default-volumes-to-restic` parameter is renamed to `default-volumes-to-fs-backup`
`restic-timeout` parameter is renamed to `fs-backup-timeout`
`default-restic-prune-frequency` parameter is renamed to `default-repo-maintain-frequency`
#### Upgrade
Due to the major changes of file system backup, the old upgrade steps are not suitable any more. For the new upgrade steps, visit [v1.10 upgrade guide document](https://velero.io/docs/v1.10/upgrade-to-1.10/).
#### Limitations/Known issues
In this release, Kopia backup repository (so the Kopia path of file system backup) doesn't support self signed certificate for S3 compatible storage. To track this problem, refer to this [Velero issue](https://github.com/vmware-tanzu/velero/issues/5123) or [Kopia issue](https://github.com/kopia/kopia/issues/1443).
Due to the code change in Velero, there will be some code change required in vSphere plugin, without which the functionality may be impacted. Therefore, if you are using vSphere plugin in your workflow, please hold the upgrade until the issue [#485](https://github.com/vmware-tanzu/velero-plugin-for-vsphere/issues/485) is fixed in vSphere plugin.
### All changes
* Restore ClusterBootstrap before Cluster otherwise a new default ClusterBootstrap object is create for the cluster (#5616, @ywk253100)
* Add compile restic binary for CVE fix (#5574, @qiuming-best)
* Fix controller problematic log output (#5572, @qiuming-best)
* Enhance the restore priorities list to support specifying the low prioritized resources that need to be restored in the last (#5535, @ywk253100)
* fix restic backup progress error (#5534, @qiuming-best)
* fix restic backup failure with self-signed certification backend storage (#5526, @qiuming-best)
* Add credential store in backup deletion controller to support VSL credential. (#5521, @blackpiglet)
* Fix issue 5505: the pod volume backups/restores except the first one fail under the kopia path if "AZURE_CLOUD_NAME" is specified (#5512, @Lyndon-Li)
* After Pod Volume Backup/Restore refactor, remove all the unreasonable appearance of "restic" word from documents (#5499, @Lyndon-Li)
* Refactor Pod Volume Backup/Restore doc to match the new behavior (#5484, @Lyndon-Li)
* Remove redundancy code block left by #5388. (#5483, @blackpiglet)
* Issue fix 5477: create the common way to support S3 compatible object storages that work for both Restic and Kopia; Keep the resticRepoPrefix parameter for compatibility (#5478, @Lyndon-Li)
* Update the k8s.io dependencies to 0.24.0.
This also required an update to github.com/bombsimon/logrusr/v3.
Removed the `WithClusterName` method
as it is a "legacy field that was
always cleared by the system and never used" as per upstream k8s
https://github.com/kubernetes/apimachinery/blob/release-1.24/pkg/apis/meta/v1/types.go#L257-L259 (#5471, @kcboyle)
* Add v1.10 velero upgrade doc (#5468, @qiuming-best)
* Upgrade velero docker image to use go 1.18 and upgrade golangci-lint to 1.45.0 (#5459, @Lyndon-Li)
* Add VolumeSnapshot client back. (#5449, @blackpiglet)
* Change subcommand `velero restic repo` to `velero repo` (#5446, @allenxu404)
* Remove irrational "Restic" names in Velero code after the PVBR refactor (#5444, @Lyndon-Li)
* moved RIA execute input/output structs back to velero package (#5441, @sseago)
* Rename Velero pod volume restore init helper from "velero-restic-restore-helper" to "velero-restore-helper" (#5432, @Lyndon-Li)
* Skip the exclusion check for additional resources returned by BIA (#5429, @reasonerjt)
* Change B/R describe CLI to support Kopia (#5412, @allenxu404)
* Add nil check before execution of csi snapshot delete (#5401, @shubham-pampattiwar)
* update velero using klog to version v2.9.0 (#5396, @blackpiglet)
* Fix Test_prepareBackupRequest_BackupStorageLocation UT failure. (#5394, @blackpiglet)
* Rename Velero daemonset from "restic" to "node-agent" (#5390, @Lyndon-Li)
* Add some corner cases checking for CSI snapshot in backup controller. (#5388, @blackpiglet)
* Fix issue 5386: Velero providers a full URL as the S3Url while the underlying minio client only accept the host part of the URL as the endpoint and the schema should be specified separately. (#5387, @Lyndon-Li)
* Fix restore error with flag namespace-mappings (#5377, @qiuming-best)
* Pod Volume Backup/Restore Refactor: Rename parameters in CRDs and commands to remove "Restic" word (#5370, @Lyndon-Li)
* Added backupController's UT to test the prepareBackupRequest() method BackupStorageLocation processing logic (#5362, @niulechuan)
* Fix a repoEnsurer problem introduced by the refactor - The repoEnsurer didn't check "" state of BackupRepository, as a result, the function GetBackupRepository always returns without an error even though the ensreReady is specified. (#5359, @Lyndon-Li)
* Add E2E test for schedule backup (#5355, @danfengliu)
* Add useOwnerReferencesInBackup field doc for schedule. (#5353, @cleverhu)
* Clarify the help message for the default value of parameter --snapshot-volumes, when it's not set. (#5350, @blackpiglet)
* Fix restore cmd extraflag overwrite bug (#5347, @qiuming-best)
* Resolve gopkg.in/yaml.v3 vulnerabilities by upgrading gopkg.in/yaml.v3 to v3.0.1 (#5344, @kaovilai)
* Increase ensure restic repository timeout to 5m (#5335, @shubham-pampattiwar)
* Add opt-in and opt-out PersistentVolume backup to E2E tests (#5331, @danfengliu)
* Cancel downloadRequest when timeout without downloadURL (#5329, @kaovilai)
* Fix PVB finds wrong parent snapshot (#5322, @qiuming-best)
* Fix issue 4874 and 4752: check the daemonset pod is running in the node where the workload pod resides before running the PVB for the pod (#5319, @Lyndon-Li)
* plugin versioning v1 refactor for VolumeSnapshotter (#5318, @sseago)
* Change the status of restore to completed from partially failed when restore empty backup (#5314, @allenxu404)
* RestoreItemAction v1 refactoring for plugin api versioning (#5312, @sseago)
* Refactor the repoEnsurer code to use controller runtime client and wrap some common BackupRepository operations to share with other modules (#5308, @Lyndon-Li)
* Remove snapshot related lister, informer and client from backup controller. (#5299, @jxun)
* Remove github.com/apex/log logger. (#5297, @blackpiglet)
* change CSISnapshotTimeout from pointer to normal variables. (#5294, @cleverhu)
* Optimize code for restore exists resources. (#5293, @cleverhu)
* Add more detailed comments for labels columns. (#5291, @cleverhu)
* Add backup status checking in schedule controller. (#5283, @blackpiglet)
* Add changes for problems/enhancements found during smoking test for Kopia pod volume backup/restore (#5282, @Lyndon-Li)
* Support pause/unpause schedules (#5279, @ywk253100)
* plugin/clientmgmt refactoring for BackupItemAction v1 (#5271, @sseago)
* Don't move velero v1 plugins to new proto dir (#5263, @sseago)
* Fill gaps for Kopia path of PVBR: integrate Repo Manager with Unified Repo; pass UploaderType to PVBR backupper and restorer; pass RepositoryType to BackupRepository controller and Repo Ensurer (#5259, @Lyndon-Li)
* Add csiSnapshotTimeout for describe backup (#5252, @cleverhu)
* equip gc controller with configurable frequency (#5248, @allenxu404)
* Fix nil pointer panic when restoring StatefulSets (#5247, @divolgin)
* Controller refactor code modifications. (#5241, @jxun)
* Fix edge cases for already exists resources (#5239, @shubham-pampattiwar)
* Check for empty ns list before checking nslist[0] (#5236, @sseago)
* Remove reference to non-existent doc (#5234, @reasonerjt)
* Add changes for Kopia Integration: Kopia Lib - method implementation. Add changes to write Kopia Repository logs to Velero log (#5233, @Lyndon-Li)
* Add changes for Kopia Integration: Kopia Lib - initialize Kopia repo (#5231, @Lyndon-Li)
* Uploader Implementation: Kopia backup and restore (#5221, @qiuming-best)
* Migrate backup sync controller from code-generator to kubebuilder. (#5218, @jxun)
* check vsc null pointer (#5217, @lilongfeng0902)
* Refactor GCController with kubebuilder (#5215, @allenxu404)
* Uploader Implementation: Restic backup and restore (#5214, @qiuming-best)
* Add parameter "uploader-type" to velero server (#5212, @reasonerjt)
* Add annotation "pv.kubernetes.io/migrated-to" for CSI checking. (#5181, @jxun)
* Add changes for Kopia Integration: Unified Repository Provider - method implementation (#5179, @Lyndon-Li)
* Treat namespaces with exclude label as excludedNamespaces
Related issue: #2413 (#5178, @allenxu404)
* Reduce CRD size. (#5174, @jxun)
* Fix restic backups to multiple backup storage locations bug (#5172, @qiuming-best)
* Add changes for Kopia Integration: Unified Repository Provider - Repo Password (#5167, @Lyndon-Li)
* Skip registering "crd-remap-version" plugin when feature flag "EnableAPIGroupVersions" is set (#5165, @reasonerjt)
* Kopia uploader integration on shim progress uploader module (#5163, @qiuming-best)
* Add labeled and unlabeled events for PR changelog check action. (#5157, @jxun)
* VolumeSnapshotLocation refactor with kubebuilder. (#5148, @jxun)
* Delay CA file deletion in PVB controller. (#5145, @jxun)
* This commit splits the pkg/restic package into several packages to support Kopia integration works (#5143, @ywk253100)
* Kopia Integration: Add the Unified Repository Interface definition. Kopia Integration: Add the changes for Unified Repository storage config. Related Issues; #5076, #5080 (#5142, @Lyndon-Li)
* Update the CRD for kopia integration (#5135, @reasonerjt)
* Let "make shell xxx" respect GOPROXY (#5128, @reasonerjt)
* Modify BackupStoreGetter to avoid BSL spec changes (#5122, @sseago)
* Dump stack trace when the plugin server handles panic (#5110, @reasonerjt)
* Make CSI snapshot creation timeout configurable. (#5104, @jxun)
* Fix bsl validation bug: the BSL is validated continually and doesn't respect the validation period configured (#5101, @ywk253100)
* Exclude "csinodes.storage.k8s.io" and "volumeattachments.storage.k8s.io" from restore by default. (#5064, @jxun)
* Move 'velero.io/exclude-from-backup' label string to const (#5053, @niulechuan)
* Modify Github actions. (#5052, @jxun)
* Fix typo in doc, in https://velero.io/docs/main/restore-reference/ "Restore order" section, "Mamespace" should be "Namespace". (#5051, @niulechuan)
* Delete opened issues triage action. (#5041, @jxun)
* When spec.RestoreStatus is empty, don't restore status (#5008, @sseago)
* Added DownloadTargetKindCSIBackupVolumeSnapshots for retrieving the signed URL to download only the `<backup name>`-csi-volumesnapshots.json.gz and DownloadTargetKindCSIBackupVolumeSnapshotContents to download only `<backup name>`-csi-volumesnapshotcontents.json.gz in the DownloadRequest CR structure. These files are already present in the backup layout. (#4980, @anshulahuja98)
* Refactor BackupItemAction proto and related code to backupitemaction/v1 package. This is part of implementation of the plugin version design https://github.com/vmware-tanzu/velero/blob/main/design/plugin-versioning.md (#4943, @phuongatemc)
* Unified Repository Design (#4926, @Lyndon-Li)
* Add credentials to volume snapshot locations (#4864, @sseago)

View File

@@ -1,126 +0,0 @@
## v1.11
### 2023-04-07
### Download
https://github.com/vmware-tanzu/velero/releases/tag/v1.11.0
### Container Image
`velero/velero:v1.11.0`
### Documentation
https://velero.io/docs/v1.11/
### Upgrading
https://velero.io/docs/v1.11/upgrade-to-1.11/
### Highlights
#### BackupItemAction v2
This feature implements the BackupItemAction v2. BIA v2 has two new methods: Progress() and Cancel() and modifies the Execute() return value.
The API change is needed to facilitate long-running BackupItemAction plugin actions that may not be complete when the Execute() method returns. This will allow long-running BackupItemAction plugin actions to continue in the background while the Velero moves to the following plugin or the next item.
#### RestoreItemAction v2
This feature implemented the RestoreItemAction v2. RIA v2 has three new methods: Progress(), Cancel(), and AreAdditionalItemsReady(), and it modifies RestoreItemActionExecuteOutput() structure in the RIA return value.
The Progress() and Cancel() methods are needed to facilitate long-running RestoreItemAction plugin actions that may not be complete when the Execute() method returns. This will allow long-running RestoreItemAction plugin actions to continue in the background while the Velero moves to the following plugin or the next item. The AreAdditionalItemsReady() method is needed to allow plugins to tell Velero to wait until the returned additional items have been restored and are ready for use in the cluster before restoring the current item.
#### Plugin Progress Monitoring
This is intended as a replacement for the previously-approved Upload Progress Monitoring design ([Upload Progress Monitoring](https://github.com/vmware-tanzu/velero/blob/main/design/upload-progress.md)) to expand the supported use cases beyond snapshot upload to include what was previously called Async Backup/Restore Item Actions.
#### Flexible resource policy that can filter volumes to skip in the backup
This feature provides a flexible policy to filter volumes in the backup without requiring patching any labels or annotations to the pods or volumes. This policy is configured as k8s ConfigMap and maintained by the users themselves, and it can be extended to more scenarios in the future. By now, the policy rules out volumes from backup depending on the CSI driver, NFS setting, volume size, and StorageClass setting. Please refer to [policy API design](https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/handle-backup-of-volumes-by-resources-filters.md#api-design) for the policy's ConifgMap format. It is not guaranteed to work on unofficial third-party plugins as it may not follow the existing backup workflow code logic of Velero.
#### Resource Filters that can distinguish cluster scope and namespace scope resources
This feature adds four new resource filters for backup. The new filters are separated into cluster scope and namespace scope. Before this feature, Velero could not filter cluster scope resources precisely. This feature provides the ability and refactors existing resource filter parameters.
#### Add a parameter for setting the Velero server connection with the k8s API server's timeout
In Velero, some code pieces need to communicate with the k8s API server. Before v1.11, these code pieces used hard-code timeout settings. This feature adds a resource-timeout parameter in the velero server binary to make it configurable.
#### Add resource list in the output of the restore describe command
Before this feature, Velero restore didn't have a restored resources list as the Velero backup. It's not convenient for users to learn what is restored. This feature adds the resources list and the handling result of the resources (including created, updated, failed, and skipped).
#### Refactor controllers with controller-runtime
In v1.11, Backup Controller and Restore controller are refactored with controller-runtime. Till v1.11, all Velero controllers use the controller-runtime framework.
#### Runtime and dependencies
To fix CVEs and keep pace with Golang, Velero made changes as follows:
* Bump Golang runtime to v1.19.8.
* Bump several dependent libraries to new versions.
* Compile Restic (v0.15.0) with Golang v1.19.8 instead of packaging the official binary.
### Breaking changes
* The Velero CSI plugin now determines whether to restore Volume's data from snapshots on the restore's restorePVs setting. Before v1.11, the CSI plugin doesn't check the restorePVs parameter setting.
### Limitations/Known issues
* The Flexible resource policy that can filter volumes to skip in the backup is not guaranteed to work on unofficial third-party plugins because the plugins may not follow the existing backup workflow code logic of Velero. The ConfigMap used as the policy is supposed to be maintained by users.
### All Changes
* Modify new scope resource filters name. (#6089, @blackpiglet)
* Make Velero not exits when EnableCSI is on and CSI snapshot not installed (#6062, @blackpiglet)
* Restore Services before Clusters (#6057, @ywk253100)
* Fixed backup deletion bug related to async operations (#6041, @sseago)
* Update Golang version to v1.19 for branch main. (#6039, @blackpiglet)
* Fix issue #5972, don't assume errorField as error type when dealing with logger.WithError (#6028, @Lyndon-Li)
* distinguish between New and InProgress operations (#6012, @sseago)
* Modify golangci.yaml file. Resolve found lint issues. (#6008, @blackpiglet)
* Remove Reference of itemsnapshotter (#5997, @reasonerjt)
* minor fixes for backup_operations_controller (#5996, @sseago)
* RIAv2 async operations controller work (#5993, @sseago)
* Follow-on fixes for BIAv2 controller work (#5971, @sseago)
* Refactor backup controller based on the controller-runtime framework. (#5969, @qiuming-best)
* Fix client wait problem after async operation change, velero backup/restore --wait should check a full list of the terminal status (#5964, @Lyndon-Li)
* Fix issue #5935, refactor the logics for backup/restore persistent log, so as to remove the contest to gzip writer (#5956, @Lyndon-Li)
* Switch the base image to distroless/base-nossl-debian11 to reduce the CVE triage efforts (#5939, @ywk253100)
* Wait for additional items to be ready before restoring current item (#5933, @sseago)
* Add configurable server setting for default timeouts (#5926, @eemcmullan)
* Add warning/error result to cmd `velero backup describe` (#5916, @allenxu404)
* Fix Dependabot alerts. Use 1.18 and 1.19 golang instead of patch image in dockerfile. Add release-1.10 and release-1.9 in Trivy daily scan. (#5911, @blackpiglet)
* Update client-go to v0.25.6 (#5907, @kaovilai)
* Limit the concurrent number for backup's VolumeSnapshot operation. (#5900, @blackpiglet)
* Fix goreleaser issue for resolving tags and updated it's version. (#5899, @anshulahuja98)
* This is to fix issue 5881, enhance the PVB tracker in two modes, Track and Taken (#5894, @Lyndon-Li)
* Add labels for velero installed namespace to support PSA. (#5873, @blackpiglet)
* Add restored resource list in the restore describe command (#5867, @ywk253100)
* Add a json output to cmd velero backup describe (#5865, @allenxu404)
* Make restore controller adopting the controller-runtime framework. (#5864, @blackpiglet)
* Replace k8s.io/apimachinery/pkg/util/clock with k8s.io/utils/clock (#5859, @hezhizhen)
* Restore finalizer and managedFields of metadata during the restoration (#5853, @ywk253100)
* BIAv2 async operations controller work (#5849, @sseago)
* Add secret restore item action to handle service account token secret (#5843, @ywk253100)
* Add new resource filters can separate cluster and namespace scope resources. (#5838, @blackpiglet)
* Correct PVB/PVR Failed Phase patching during startup (#5828, @kaovilai)
* bump up golang net to fix CVE-2022-41721 (#5812, @Lyndon-Li)
* Update CRD descriptions for SnapshotVolumes and restorePVs (#5807, @shubham-pampattiwar)
* Add mapped selected-node existence check (#5806, @blackpiglet)
* Add option "--service-account-name" to install cmd (#5802, @reasonerjt)
* Enable staticcheck linter. (#5788, @blackpiglet)
* Set Kopia IgnoreUnknownTypes in ErrorHandlingPolicy to True for ignoring backup unknown file type (#5786, @qiuming-best)
* Bump up Restic version to 0.15.0 (#5784, @qiuming-best)
* Add File system backup related matrics to Grafana dashboard
- Add metrics backup_warning_total for record of total warnings
- Add metrics backup_last_status for record of last status of the backup (#5779, @allenxu404)
* Design for Handling backup of volumes by resources filters (#5773, @qiuming-best)
* Add PR container build action, which will not push image. Add GOARM parameter. (#5771, @blackpiglet)
* Fix issue 5458, track pod volume backup until the CR is submitted in case it is skipped half way (#5769, @Lyndon-Li)
* Fix issue 5226, invalidate the related backup repositories whenever the backup storage info change in BSL (#5768, @Lyndon-Li)
* Add Restic builder in Dockerfile, and keep the used built Golang image version in accordance with upstream Restic. (#5764, @blackpiglet)
* Fix issue 5043, after the restore pod is scheduled, check if the node-agent pod is running in the same node. (#5760, @Lyndon-Li)
* Remove restore controller's redundant client. (#5759, @blackpiglet)
* Define itemoperations.json format and update DownloadRequest API (#5752, @sseago)
* Add Trivy nightly scan. (#5740, @jxun)
* Fix issue 5696, check if the repo is still openable before running the prune and forget operation, if not, try to reconnect the repo (#5715, @Lyndon-Li)
* Fix error with Restic backup empty volumes (#5713, @qiuming-best)
* new backup and restore phases to support async plugin operations:
- WaitingForPluginOperations
- WaitingForPluginOperationsPartiallyFailed (#5710, @sseago)
* Prevent nil panic on exec restore hooks (#5675, @dymurray)
* Fix CVEs scanned by trivy (#5653, @qiuming-best)
* Publish backupresults json to enhance error info during backups. (#5576, @anshulahuja98)
* RestoreItemAction v2 API implementation (#5569, @sseago)
* add new RestoreItemAction of "velero.io/change-image-name" to handle the issue mentioned at #5519 (#5540, @wenterjoy)
* BackupItemAction v2 API implementation (#5442, @sseago)
* Proposal to separate resource filter into cluster scope and namespace scope (#5333, @blackpiglet)

View File

@@ -1,142 +0,0 @@
## v1.12
### 2023-08-18
### Download
https://github.com/vmware-tanzu/velero/releases/tag/v1.12.0
### Container Image
`velero/velero:v1.12.0`
### Documentation
https://velero.io/docs/v1.12/
### Upgrading
https://velero.io/docs/v1.12/upgrade-to-1.12/
### Highlights
#### CSI Snapshot Data Movement
CSI Snapshot Data Movement refers to back up CSI snapshot data from the volatile and limited production environment into durable, heterogeneous, and scalable backup storage in a consistent manner; and restore the data to volumes in the original or alternative environment.
CSI Snapshot Data Movement is useful in below scenarios:
* For on-premises users, the storage usually doesn't support durable snapshots, so it is impossible/less efficient/cost ineffective to keep volume snapshots by the storage This feature helps to move the snapshot data to a storage with lower cost and larger scale for long time preservation.
* For public cloud users, this feature helps users to fulfill the multiple cloud strategy. It allows users to back up volume snapshots from one cloud provider and preserve or restore the data to another cloud provider. Then users will be free to flow their business data across cloud providers based on Velero backup and restore
CSI Snapshot Data Movement is built according to the Volume Snapshot Data Movement design ([Volume Snapshot Data Movement](https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/unified-repo-and-kopia-integration/unified-repo-and-kopia-integration.md)). More details can be found in the design.
#### Resource Modifiers
In many use cases, customers often need to substitute specific values in Kubernetes resources during the restoration process like changing the namespace, changing the storage class, etc.
To address this need, Resource Modifiers (also known as JSON Substitutions) offer a generic solution in the restore workflow. It allows the user to define filters for specific resources and then specify a JSON patch (operator, path, value) to apply to the resource. This feature simplifies the process of making substitutions without requiring the implementation of a new RestoreItemAction plugin. More details can be found in Volume Snapshot Resource Modifiers design ([Resource Modifiers](https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/json-substitution-action-design.md)).
#### Multiple VolumeSnapshotClasses
Prior to version 1.12, the Velero CSI plugin would choose the VolumeSnapshotClass in the cluster based on matching driver names and the presence of the "velero.io/csi-volumesnapshot-class" label. However, this approach proved inadequate for many user scenarios.
With the introduction of version 1.12, Velero now offers support for multiple VolumeSnapshotClasses in the CSI Plugin, enabling users to select a specific class for a particular backup. More details can be found in Multiple VolumeSnapshotClasses design ([Multiple VolumeSnapshotClasses](https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/multiple-csi-volumesnapshotclass-support.md)).
#### Restore Finalizer
Before v1.12, the restore controller would only delete restore resources but wouldnt delete restore data from the backup storage location when the command `velero restore delete` was executed. The only chance Velero deletes restores data from the backup storage location is when the associated backup is deleted.
In this version, Velero introduces a finalizer that ensures the cleanup of all associated data for restores when running the command `velero restore delete`.
#### Runtime and dependencies
To fix CVEs and keep pace with Golang, Velero made changes as follows:
* Bump Golang runtime to v1.20.7.
* Bump several dependent libraries to new versions.
* Bump Kopia to v0.13.
### Breaking changes
* Prior to v1.12, the parameter `uploader-type` for Velero installation had a default value of "restic". However, starting from this version, the default value has been changed to "kopia". This means that Velero will now use Kopia as the default path for file system backup.
* The ways of setting CSI snapshot time have changed in v1.12. First, the sync waiting time for creating a snapshot handle in the CSI plugin is changed from the fixed 10 minutes into backup.Spec.CSISnapshotTimeout. The second, the async waiting time for VolumeSnapshot and VolumeSnapshotContent's status turning into `ReadyToUse` in operation uses the operation's timeout. The default value is 4 hours.
* As from [Velero helm chart v4.0.0](https://github.com/vmware-tanzu/helm-charts/releases/tag/velero-4.0.0), it supports multiple BSL and VSL, and the BSL and VSL have changed from the map into a slice, and[ this breaking change](https://github.com/vmware-tanzu/helm-charts/pull/413) is not backward compatible. So it would be best to change the BSL and VSL configuration into slices before the Upgrade.
* Prior to v1.12, deleting the Velero namespace would easily remove all the resources within it. However, with the introduction of finalizers attached to the Velero CR including `restore`, `dataupload`, and `datadownload` in this version, directly deleting Velero namespace may get stuck indefinitely because the pods responsible for handling the finalizers might be deleted before the resources attached to the finalizers. To avoid this issue, please use the command `velero uninstall` to delete all the Velero resources or ensure that you handle the finalizer appropriately before deleting the Velero namespace.
### Limitations/Known issues
* The Azure plugin supports Azure AD Workload identity way, but it only works for Velero native snapshots. It cannot support filesystem backup and snapshot data mover scenarios.
### All Changes
* Fixes #6498. Get resource client again after restore actions in case resource's gv is changed. This is an improvement of pr #6499, to support group changes. A group change usually happens in a restore plugin which is used for resource conversion: convert a resource from a not supported gv to a supported gv (#6634, @27149chen)
* Add API support for volMode block, only error for now. (#6608, @shawn-hurley)
* Fix how the AWS credentials are obtained from configuration (#6598, @aws_creds)
* Add performance E2E test (#6569, @qiuming-best)
* Non default s3 credential profiles work on Unified Repository Provider (kopia) (#6558, @kaovilai)
* Fix issue #6571, fix the problem for restore item operation to set the errors correctly so that they can be recorded by Velero restore and then reflect the correct status for Velero restore. (#6594, @Lyndon-Li)
* Fix issue 6575, flush the repo after delete the snapshot, otherwise, the changes(deleting repo snapshot) cannot be committed to the repo. (#6587, @Lyndon-Li)
* Delete moved snapshots when the backup is deleted (#6547, @reasonerjt)
* check if restore crd exist before operating restores (#6544, @allenxu404)
* Remove PVC's selector in backup's PVC action. (#6481, @blackpiglet)
* Delete the expired deletebackuprequests that are stuck in "InProgress" (#6476, @reasonerjt)
* Fix issue #6534, reset PVB CR's StorageLocation to the latest one during backup sync as same as the backup CR. Also fix similar problem with DataUploadResult for data mover restore. (#6533, @Lyndon-Li)
* Fix issue #6519. Restrict the client manager of node-agent server to include only Velero resources from the server's namespace, otherwise, the controllers will try to reconcile CRs from all the installed Velero namespaces. (#6523, @Lyndon-Li)
* Track the skipped PVC and print the summary in backup log (#6496, @reasonerjt)
* Add restore finalizer to clean up external resources (#6479, @allenxu404)
* fix: Typos and add more spell checking rules to CI (#6415, @mateusoliveira43)
* Add missing CompletionTimestamp and metrics when restore moved into terminal phase in restoreOperationsReconciler (#6397, @Nutrymaco)
* Add support for resource Modifications in the restore flow. Also known as JSON Substitutions. (#6452, @anshulahuja98)
* Remove dependency of the legacy client code from pkg/cmd directory part 2 (#6497, @blackpiglet)
* Add data upload and download metrics (#6493, @allenxu404)
* Fix issue 6490, If a backup/restore has multiple async operations and one operation fails while others are still in-progress, when all the operations finish, the backup/restore will be set as Completed falsely (#6491, @Lyndon-Li)
* Velero Plugins no longer need kopia indirect dependency in their go.mod (#6484, @kaovilai)
* Remove dependency of the legacy client code from pkg/cmd directory (#6469, @blackpiglet)
* Add support for OpenStack CSI drivers topology keys (#6464, @openstack-csi-topology-keys)
* Add exit code log and possible memory shortage warning log for Restic command failure. (#6459, @blackpiglet)
* Modify DownloadRequest controller logic (#6433, @blackpiglet)
* Add data download controller for data mover (#6436, @qiuming-best)
* Fix hook filter display issue for backup describer (#6434, @allenxu404)
* Retrieve DataUpload into backup result ConfigMap during volume snapshot restore. (#6410, @blackpiglet)
* Design to add support for Multiple VolumeSnapshotClasses in CSI Plugin. (#5774, @anshulahuja98)
* Clarify the deletion frequency for gc controller (#6414, @allenxu404)
* Add unit tests for pkg/archive (#6396, @allenxu404)
* Add UT for pkg/discovery (#6394, @qiuming-best)
* Add UT for pkg/util (#6368, @Lyndon-Li)
* Add the code for data mover restore expose (#6357, @Lyndon-Li)
* Restore Endpoints before Services (#6315, @ywk253100)
* Add warning message for volume snapshotter in data mover case. (#6377, @blackpiglet)
* Add unit test for pkg/uploader (#6374, @qiuming-best)
* Change kopia as the default path of PVB (#6370, @Lyndon-Li)
* Do not persist VolumeSnapshot and VolumeSnapshotContent for snapshot DataMover case. (#6366, @blackpiglet)
* Add data mover related options in CLI (#6365, @ywk253100)
* Add dataupload controller (#6337, @qiuming-best)
* Add UT cases for pkg/podvolume (#6336, @Lyndon-Li)
* Remove Wait VolumeSnapshot to ReadyToUse logic. (#6327, @blackpiglet)
* Enhance the code because of #6297, the return value of GetBucketRegion is not recorded, as a result, when it fails, we have no way to get the cause (#6326, @Lyndon-Li)
* Skip updating status when CRDs are restored (#6325, @reasonerjt)
* Include namespaces needed by namespaced-scope resources in backup. (#6320, @blackpiglet)
* Update metrics when backup failed with validation error (#6318, @ywk253100)
* Add the code for data mover backup expose (#6308, @Lyndon-Li)
* Fix a PVR issue for generic data path -- the namespace remap was not honored, and enhance the code for better error handling (#6303, @Lyndon-Li)
* Add default values for defaultItemOperationTimeout and itemOperationSyncFrequency in velero CLI (#6298, @shubham-pampattiwar)
* Add UT cases for pkg/repository (#6296, @Lyndon-Li)
* Fix issue #5875. Since Kopia has supported IAM, Velero should not require static credentials all the time (#6283, @Lyndon-Li)
* Fixed a bug where status.progress is not getting updated for backups. (#6276, @kkothule)
* Add code change for async generic data path that is used by both PVB/PVR and data mover (#6226, @Lyndon-Li)
* Add data mover CRD under v2alpha1, include DataUpload CRD and DataDownload CRD (#6176, @Lyndon-Li)
* Remove any dataSource or dataSourceRef fields from PVCs in PVC BIA for cases of
prior PVC restores with CSI (#6111, @eemcmullan)
* Add the design for Volume Snapshot Data Movement (#5968, @Lyndon-Li)
* Fix issue #5123, Kopia repository supports self-cert CA for S3 compatible storage. (#6268, @Lyndon-Li)
* Bump up Kopia to v0.13 (#6248, @Lyndon-Li)
* log volumes to backup to help debug why `IsPodRunning` is called. (#6232, @kaovilai)
* Enable errcheck linter and resolve found issues (#6208, @blackpiglet)
* Enable more linters, and remove mal-functioned milestoned issue action. (#6194, @blackpiglet)
* Enable stylecheck linter and resolve found issues. (#6185, @blackpiglet)
* Fix issue #6182. If pod is not running, don't treat it as an error, let it go and leave a warning. (#6184, @Lyndon-Li)
* Enable staticcheck and resolve found issues (#6183, @blackpiglet)
* Enable linter revive and resolve found errors: part 2 (#6177, @blackpiglet)
* Enable linter revive and resolve found errors: part 1 (#6173, @blackpiglet)
* Fix usestdlibvars and whitespace linters issues. (#6162, @blackpiglet)
* Update Golang to v1.20 for main. (#6158, @blackpiglet)
* Make GetPluginConfig accessible from other packages. (#6151, @tkaovila)
* Ignore not found error during patching managedFields (#6136, @ywk253100)
* Fix the goreleaser issues and add a new goreleaser action (#6109, @blackpiglet)
* Add CSI snapshot data movement doc (#6793, @Lyndon-Li)
* Use old(origin) namespace in resource modifier conditions in case namespace may change during restore (#6724, @27149chen)
* Fix #6752: add namespace exclude check. (#6762, @blackpiglet)
* Update restore controller logic for restore deletion (#6761, @ywk253100)
* Fix issue #6753, remove the check for read-only BSL in restore async operation controller since Velero cannot fully support read-only mode BSL in restore at present (#6758, @Lyndon-Li)
* Fixes #6636, skip subresource in resource discovery (#6688, @27149chen)
* This pr made some improvements in Resource Modifiers:1. add label selector 2. change the field name from groupKind to groupResource (#6704, @27149chen)

View File

@@ -1,3 +1,80 @@
## v1.6.3
### 2021-07-30
### Download
https://github.com/vmware-tanzu/velero/releases/tag/v1.6.3
### Container Image
`velero/velero:v1.6.3`
### Documentation
https://velero.io/docs/v1.6/
### Upgrading
https://velero.io/docs/v1.6/upgrade-to-1.6/
### Highlights
This release introduces changes to provide compatibility with Kubernetes v1.22.
The `apiextensions.k8s.io/v1beta1` API version of `CustomResourceDefinition` will no longer be served in Kubernetes v1.22.
Velero will now use the cluster preferred API version for the `CustomResourceDefinition`s that it creates.
If you are using Kubernetes v1.15 or earlier, the `apiextensions.k8s.io/v1beta1` API version will be used.
If you are using Kubernetes v1.22 or later, the `apiextensions.k8s.io/v1` API version will be used.
For clusters between these versions, the cluster preferred API version will be used.
The `rbac.authorization.k8s.io/v1beta1` API version of `ClusterRoleBinding` will no longer be served in Kubernetes v1.22.
Velero will now use the `rbac.authorization.k8s.io/v1` API version for the `ClusterRoleBinding`s that it creates.
This API version was introduced in Kubernetes v1.8.
### All Changes
* enable e2e tests to choose crd apiVersion (#3941, @sseago)
* Upgrade Velero ClusterRoleBinding to use v1 API (#3995, @jenting)
* Install Kubernetes preferred CRDs API version (v1beta1/v1). (#3999, @jenting)
* Use the cluster preferred CRD API version when polling for Velero CRD readiness. (#4015, @zubron)
* Add a RestoreItemAction plugin (`velero.io/apiservice`) which skips the restore of any `APIService` which is managed by Kubernetes. These are identified using the `kube-aggregator.kubernetes.io/automanaged` label. (#4028, @zubron)
## v1.6.2
### 2021-07-16
### Download
https://github.com/vmware-tanzu/velero/releases/tag/v1.6.2
### Container Image
`velero/velero:v1.6.2`
### Documentation
https://velero.io/docs/v1.6/
### Upgrading
https://velero.io/docs/v1.6/upgrade-to-1.6/
This release contains no user facing changes but includes fixes for CVE-2021-3121 and CVE-2021-3580.
## v1.6.1
### 2021-06-21
### Download
https://github.com/vmware-tanzu/velero/releases/tag/v1.6.1
### Container Image
`velero/velero:v1.6.1`
### Documentation
https://velero.io/docs/v1.6/
### Upgrading
https://velero.io/docs/v1.6/upgrade-to-1.6/
### All Changes
* Fix CR restore regression introduced in 1.6 restore progress. (#3845, @sseago)
* Skip the restore of volumes that originally came from a projected volume when using restic. (#3877, @zubron)
* skip backuping projected volume when using restic (#3866, @alaypatel07)
* 🐛 Fix plugin name derivation from image name (#3711, @ashish-amarnath)
## v1.6.0
### 2021-04-12

View File

@@ -1,80 +0,0 @@
## v1.7.0
### 2021-09-07
### Download
https://github.com/vmware-tanzu/velero/releases/tag/v1.7.0
### Container Image
`velero/velero:v1.7.0`
### Documentation
https://velero.io/docs/v1.7/
### Upgrading
https://velero.io/docs/v1.7/upgrade-to-1.7/
### Highlights
#### Distroless images
The Velero container images now use [distroless base images](https://github.com/GoogleContainerTools/distroless).
Using distroless images as the base ensures that only the packages and programs necessary for running Velero are included.
Unrelated libraries and OS packages, that often contain security vulnerabilities, are now excluded.
This change reduces the size of both the server and restic restore helper image by approximately 62MB.
As the [distroless](https://github.com/GoogleContainerTools/distroless) images do not contain a shell, it will no longer be possible to exec into Velero containers using these images.
#### New "debug" command
This release introduces the new `velero debug` command.
This command collects information about a Velero installation, such as pod logs and resources managed by Velero, in a tarball which can be provided to the Velero maintainer team to help diagnose issues.
### All changes
* Distinguish between different unnamed node ports when preserving (#4026, @sseago)
* Validate namespace in Velero backup create command (#4057, @codegold79)
* Empty the "ClusterIPs" along with "ClusterIP" when "ClusterIP" isn't "None" (#4101, @ywk253100)
* Add a RestoreItemAction plugin (`velero.io/apiservice`) which skips the restore of any `APIService` which is managed by Kubernetes. These are identified using the `kube-aggregator.kubernetes.io/automanaged` label. (#4028, @zubron)
* Change the base image to distroless (#4055, @ywk253100)
* Updated the version of velero/velero-plugin-for-aws version from v1.2.0 to v1.2.1 (#4064, @kahirokunn)
* Skip the backup and restore of DownwardAPI volumes when using restic. (#4076, @zubron)
* Bump up Go to 1.16 (#3990, @reasonerjt)
* Fix restic error when volume is emptyDir and Pod not running (#3993, @mahaupt)
* Select the velero deployment with both label and container name (#3996, @ywk253100)
* Wait for the namespace to be deleted before removing the CRDs during uninstall. This deprecates the `--wait` flag of the `uninstall` command (#4007, @ywk253100)
* Use the cluster preferred CRD API version when polling for Velero CRD readiness. (#4015, @zubron)
* Implement velero debug (#4022, @reasonerjt)
* Skip the restore of volumes that originally came from a projected volume when using restic. (#3877, @zubron)
* Run the E2E test with kind(provision various versions of k8s cluster) and MinIO on Github Action (#3912, @ywk253100)
* Fix -install-velero flag for e2e tests (#3919, @jaidevmane)
* Upgrade Velero ClusterRoleBinding to use v1 API (#3926, @jenting)
* enable e2e tests to choose crd apiVersion (#3941, @sseago)
* Fixing multipleNamespaceTest bug - Missing expect statement in test (#3983, @jaidevmane)
* Add --client-page-size flag to server to allow chunking Kubernetes API LIST calls across multiple requests on large clusters (#3823, @dharmab)
* Fix CR restore regression introduced in 1.6 restore progress. (#3845, @sseago)
* Use region specified in the BackupStorageLocation spec when getting restic repo identifier. Originally fixed by @jala-dx in #3617. (#3857, @zubron)
* skip backuping projected volume when using restic (#3866, @alaypatel07)
* Install Kubernetes preferred CRDs API version (v1beta1/v1). (#3614, @jenting)
* Add Label to BackupSpec so that labels can explicitly be provided to Schedule.Spec.Template.Metadata.Labels which will be reflected on the backups created. (#3641, @arush-sal)
* Add PVC UID label to PodVolumeRestore (#3792, @sseago)
* Support pulling plugin images by digest (#3803, @2uasimojo)
* Added BackupPhaseUploading and BackupPhaseUploadingPartialFailure backup phases as part of Upload Progress Monitoring. (#3805, @dsmithuchida)
Uploading (new)
The "Uploading" phase signifies that the main part of the backup, including
snapshotting has completed successfully and uploading is continuing. In
the event of an error during uploading, the phase will change to
UploadingPartialFailure. On success, the phase changes to Completed. The
backup cannot be restored from when it is in the Uploading state.
UploadingPartialFailure (new)
The "UploadingPartialFailure" phase signifies that the main part of the backup,
including snapshotting has completed, but there were partial failures either
during the main part or during the uploading. The backup cannot be restored
from when it is in the UploadingPartialFailure state.
* 🐛 Fix plugin name derivation from image name (#3711, @ashish-amarnath)
* ✨ ⚠️ Remove CSI volumesnapshot artifact deletion
This change requires https://github.com/vmware-tanzu/velero-plugin-for-csi/pull/86 for Velero to continue
deleting of CSI volumesnapshots when the corresponding backups are deleted. (#3734, @ashish-amarnath)
* use unstructured to marshal selective fields for service restore action (#3789, @alaypatel07)

View File

@@ -1,110 +0,0 @@
## v1.8.0
### 2022-01-14
### Download
https://github.com/vmware-tanzu/velero/releases/tag/v1.8.0
### Container Image
`velero/velero:v1.8.0`
### Documentation
https://velero.io/docs/v1.8
### Upgrading
https://velero.io/docs/v1.8/upgrade-to-1.8/
### Highlights
#### Velero plugins now support handling volumes created by the CSI drivers of cloud providers
Versions 1.4 of the Velero plugins for AWS, Azure and GCP now support snapshotting and restoring the persistent volumes provisioned by CSI driver via the APIs of the cloud providers. With this enhancement, users can backup and restore the persistent volumes on these cloud providers without using the Velero CSI plugin. The CSI plugin will remain beta and the feature flag `EnableCSI` will be disabled by default.
For the version of the plugins and the CSI drivers they support respectively please see the table:
| Plugin | Version | CSI Driver |
| --- | ----------- | ---------- |
| velero-plugin-for-aws | v1.4.0 | ebs.csi.aws.com |
| velero-plugin-for-microsoft-azure | v1.4.0 | disk.csi.azure.com |
| velero-plugin-for-gcp | v1.4.0 | pd.csi.storage.gke.io |
#### IPv6 dual stack support
We've verified the functionality of Velero on IPv6 dual stack by successfully running the E2E test on IPv6 dual stack environment.
#### Refactor the controllers using Kubebuilder v3
In this release we continued our code modernization work, rewriting some controllers using Kubebuilder v3. This work is ongoing and we will continue to make progress in future releases.
#### Enhancements to E2E test cases
More test cases have been added to the E2E test suite to improve the release health.
#### Respect the cron setting of scheduled backup
The creation time is now taken into account to calculate the next run for scheduled backup.
#### Deleting BSLs also cleans up related resources
When a Backup Storage Location (BSL) is deleted, backup and Restic repository resources will also be deleted.
#### Breaking changes
Starting in v1.8, Velero will only support Kubernetes v1 CRD meaning that Velero v1.8+ will only run on Kubernetes v1.16+. Before upgrading, make sure you are running a supported Kubernetes version. For more information, see our [compatibility matrix](https://github.com/vmware-tanzu/velero#velero-compatibility-matrix).
#### Upload Progress Monitoring and Item Snapshotter
Item Snapshotter plugin API was merged. This will support both Upload Progress
monitoring and the planned Data Mover. Upload Progress monitoring PRs are
in progress for 1.9.
### All changes
* E2E test on ssr object with controller namespace mix-ups (#4521, @mqiu)
* Check whether the volume is provisioned by CSI driver or not by the annotation as well (#4513, @ywk253100)
* Initialize the labels field of `velero backup-location create` option to avoid #4484 (#4491, @ywk253100)
* Fix e2e 2500 namespaces scale test timeout problem (#4480, @mqiu)
* Add backup deletion e2e test (#4401, @danfengliu)
* Return the error when getting backup store in backup deletion controller (#4465, @reasonerjt)
* Ignore the provided port is already allocated error when restoring the LoadBalancer service (#4462, @ywk253100)
* Revert #4423 migrate backup sync controller to kubebuilder. (#4457, @jxun)
* Add rbac and annotation test cases (#4455, @mqiu)
* remove --crds-version in velero install command. (#4446, @jxun)
* Upgrade e2e test vsphere plugin (#4440, @mqiu)
* Fix e2e test failures for the inappropriate optimaze of velero install (#4438, @mqiu)
* Limit backup namespaces on test resource filtering cases (#4437, @mqiu)
* Bump up Go to 1.17 (#4431, @reasonerjt)
* Added `<backup name>`-itemsnapshots.json.gz to the backup format. This file exists
when item snapshots are taken and contains an array of volume.Itemsnapshots
containing the information about the snapshots. This will not be used unless
upload progress monitoring and item snapshots are enabled and an ItemSnapshot
plugin is used to take snapshots.
Also added DownloadTargetKindBackupItemSnapshots for retrieving the signed URL to download only the `<backup name>`-itemsnapshots.json.gz part of a backup for use by
`velero backup describe`. (#4429, @dsmithuchida)
* Migrate backup sync controller from code-generator to kubebuilder. (#4423, @jxun)
* Added UploadProgressFeature flag to enable Upload Progress Monitoring and Item
Snapshotters. (#4416, @dsmithuchida)
* Added BackupWithResolvers and RestoreWithResolvers calls. Will eventually replace Backup and Restore methods.
Adds ItemSnapshotters to Backup and Restore workflows. (#4410, @dsu)
* Build for darwin-arm64 (#4409, @epk)
* Add resource filtering test cases (#4404, @mqiu)
* Fix the issue that the backup cannot be deleted after the application uninstalled (#4398, @ywk253100)
* Add restoreactionitem plugin to handle admission webhook configurations (#4397, @reasonerjt)
* Keep the annotation "pv.kubernetes.io/provisioned-by" when restoring PVs (#4391, @ywk253100)
* Adjust structure of e2e test codes (#4386, @mqiu)
* feat: migrate velero controller from kubebuilder v2 to v3
From Velero v1.8, apiextesions.k8s.io/v1beta1 is no longer supported,
which means only CRD of apiextensions.k8s.io/v1 is supported,
and the supported Kubernetes version is updated to v1.16 and later. (#4382, @jxun)
* Delete backups and Restic repos associated with deleted BSL(s) (#4377, @codegold79)
* Add the key for GKE zone for AZ collection (#4376, @reasonerjt)
* Fix statefulsets volumeClaimTemplates storageClassName when use Changing PV/PVC Storage Classes (#4375, @Box-Cube)
* Fix snapshot e2e test issue of jsonpath (#4372, @danfengliu)
* Modify the timestamp in the name of a backup generated from schedule to use UTC. (#4353, @jxun)
* Read Availability zone from nodeAffinity requirements (#4350, @reasonerjt)
* Use factory.Namespace() to replace hardcoded velero namespace (#4346, @half-life666)
* Return the error if velero failed to detect S3 region for restic repo (#4343, @reasonerjt)
* Add init log option for velero controller-runtime manager. (#4341, @jxun)
* Ignore the `provided port is already allocated` error when restoring the `NodePort` service (#4336, @ywk253100)
* Fixed an issue with the `backup-location create` command where the BSL Credential field would be set to an invalid empty SecretKeySelector when no credential details were provided. (#4322, @zubron)
* fix buggy pager func (#4306, @alaypatel07)
* Don't create a backup immediately after creating a schedule (#4281, @ywk253100)
* Fix CVE-2020-29652 and CVE-2020-26160 (#4274, @ywk253100)
* Refine tag-release.sh to align with change in release process (#4185, @reasonerjt)
* Fix plugins incompatible issue in upgrade test (#4141, @danfengliu)
* Verify group before treating resource as cohabiting (#4126, @sseago)
* Added ItemSnapshotter plugin definition and plugin framework - addresses #3533.
Part of the Upload Progress enhancement (#3533) (#4077, @dsmithuchida)
* Add upgrade test in E2E test (#4058, @danfengliu)
* Handle namespace mapping for PVs without snapshots on restore (#3708, @sseago)

View File

@@ -1,104 +0,0 @@
## v1.9.0
### 2022-06-13
### Download
https://github.com/vmware-tanzu/velero/releases/tag/v1.9.0
### Container Image
`velero/velero:v1.9.0`
### Documentation
https://velero.io/docs/v1.9/
### Upgrading
https://velero.io/docs/v1.9/upgrade-to-1.9/
### Highlights
#### Improvement to the CSI plugin
- Bump up to the CSI volume snapshot v1 API
- No VolumeSnapshot will be left in the source namespace of the workload
- Report metrics for CSI snapshots
More improvements please refer to [CSI plugin improvement](https://github.com/vmware-tanzu/velero/issues?q=is%3Aissue+label%3A%22CSI+plugin+-+GA+-+phase1%22+is%3Aclosed)
With these improvements we'll provide official support for CSI snapshots on AKS/EKS clusters. (with CSI plugin v0.3.0)
#### Refactor the controllers using Kubebuilder v3
In this release we continued our code modernization work, rewriting some controllers using Kubebuilder v3. This work is ongoing and we will continue to make progress in future releases.
#### Optionally restore status on selected resources
Options are added to the CLI and Restore spec to control the group of resources whose status will be restored.
#### ExistingResourcePolicy in the restore API
Users can choose to overwrite or patch the existing resources during restore by setting this policy.
#### Upgrade integrated Restic version and add skip TLS validation in Restic command
Upgrade integrated Restic version, which will resolve some of the CVEs, and support skip TLS validation in Restic backup/restore.
#### Breaking changes
With bumping up the API to v1 in CSI plugin, the v0.3.0 CSI plugin will only work for Kubernetes v1.20+
### All changes
* restic: add full support for setting SecurityContext for restore init container from configMap. (#4084, @MatthieuFin)
* Add metrics backup_items_total and backup_items_errors (#4296, @tobiasgiese)
* Convert PodVolumebackup controller to the Kubebuilder framework (#4436, @fgold)
* Skip not mounted volumes when backing up (#4497, @dkeven)
* Update doc for v1.8 (#4517, @reasonerjt)
* Fix bug to make the restic prune frequency configurable (#4518, @ywk253100)
* Add E2E test of backups sync from BSL (#4545, @mqiu)
* Fix: OrderedResources in Schedules (#4550, @dbrekau)
* Skip volumes of non-running pods when backing up (#4584, @bynare)
* E2E SSR test add retry mechanism and logs (#4591, @mqiu)
* Add pushing image to GCR in github workflow to facilitate some environments that have rate limitation to docker hub, e.g. vSphere. (#4623, @jxun)
* Add existingResourcePolicy to Restore API (#4628, @shubham-pampattiwar)
* Fix E2E backup namespaces test (#4634, @qiuming-best)
* Update image used by E2E test to gcr.io (#4639, @jxun)
* Add multiple label selector support to Velero Backup and Restore APIs (#4650, @shubham-pampattiwar)
* Convert Pod Volume Restore resource/controller to the Kubebuilder framework (#4655, @ywk253100)
* Update --use-owner-references-in-backup description in velero command line. (#4660, @jxun)
* Avoid overwritten hook's exec.container parameter when running pod command executor. (#4661, @jxun)
* Support regional pv for GKE (#4680, @jxun)
* Bypass the remap CRD version plugin when v1beta1 CRD is not supported (#4686, @reasonerjt)
* Add GINKGO_SKIP to support skip specific case in e2e test. (#4692, @jxun)
* Add --pod-labels flag to velero install (#4694, @j4m3s-s)
* Enable coverage in test.sh and upload to codecov (#4704, @reasonerjt)
* Mark the BSL as "Unavailable" when gets any error and add a new field "Message" to the status to record the error message (#4719, @ywk253100)
* Support multiple skip option for E2E test (#4725, @jxun)
* Add PriorityClass to the AdditionalItems of Backup's PodAction and Restore's PodAction plugin to backup and restore PriorityClass if it is used by a Pod. (#4740, @phuongatemc)
* Insert all restore errors and warnings into restore log. (#4743, @sseago)
* Refactor schedule controller with kubebuilder (#4748, @ywk253100)
* Garbage collector now adds labels to backups that failed to delete for BSLNotFound, BSLCannotGet, BSLReadOnly reasons. (#4757, @kaovilai)
* Skip podvolumerestore creation when restore excludes pv/pvc (#4769, @half-life666)
* Add parameter for e2e test to support modify kibishii install path. (#4778, @jxun)
* Ensure the restore hook applied to new namespace based on the mapping (#4779, @reasonerjt)
* Add ability to restore status on selected resources (#4785, @RafaeLeal)
* Do not take snapshot for PV to avoid duplicated snapshotting, when CSI feature is enabled. (#4797, @jxun)
* Bump up to v1 API for CSI snapshot (#4800, @reasonerjt)
* fix: delete empty backups (#4817, @yuvalman)
* Add CSI VolumeSnapshot related metrics. (#4818, @jxun)
* Fix default-backup-ttl not work (#4831, @qiuming-best)
* Make the vsc created by backup sync controller deletable (#4832, @reasonerjt)
* Make in-progress backup/restore as failed when doing the reconcile to avoid hanging in in-progress status (#4833, @ywk253100)
* Use controller-gen to generate the deep copy methods for objects (#4838, @ywk253100)
* Update integrated Restic version and add insecureSkipTLSVerify for Restic CLI. (#4839, @jxun)
* Modify CSI VolumeSnapshot metric related code. (#4854, @jxun)
* Refactor backup deletion controller based on kubebuilder (#4855, @reasonerjt)
* Remove VolumeSnapshots created during backup when CSI feature is enabled. (#4858, @jxun)
* Convert Restic Repository resource/controller to the Kubebuilder framework (#4859, @qiuming-best)
* Add ClusterClasses to the restore priority list (#4866, @reasonerjt)
* Cleanup the .velero folder after restic done (#4872, @big-appled)
* Delete orphan CSI snapshots in backup sync controller (#4887, @reasonerjt)
* Make waiting VolumeSnapshot to ready process parallel. (#4889, @jxun)
* continue rather than return for non-matching restore action label (#4890, @sseago)
* Make in-progress PVB/PVR as failed when restic controller restarts to avoid hanging backup/restore (#4893, @ywk253100)
* Refactor BSL controller with periodical enqueue source (#4894, @jxun)
* Make garbage collection for expired backups configurable (#4897, @ywk253100)
* Bump up the version of distroless to base-debian11 (#4898, @ywk253100)
* Add schedule ordered resources E2E test (#4913, @qiuming-best)
* Make velero completion zsh command output can be used by `source` command. (#4914, @jxun)
* Enhance the map flag to support parsing input value contains entry delimiters (#4920, @ywk253100)
* Fix E2E test [Backups][Deletion][Restic] on GCP. (#4968, @jxun)
* Disable status as sub resource in CRDs (#4972, @ywk253100)
* Add more information for failing to get path or snapshot in restic backup and restore. (#4988, @jxun)

View File

@@ -1,27 +0,0 @@
package main
import (
"fmt"
"os"
"time"
)
const (
// workingModePause indicates it is for general purpose to hold the pod under running state
workingModePause = "pause"
)
func main() {
if len(os.Args) < 2 {
fmt.Fprintln(os.Stderr, "ERROR: at least one argument must be provided, the working mode")
os.Exit(1)
}
switch os.Args[1] {
case workingModePause:
time.Sleep(time.Duration(1<<63 - 1))
default:
fmt.Fprintln(os.Stderr, "ERROR: wrong working mode provided")
os.Exit(1)
}
}

View File

@@ -18,6 +18,7 @@ package main
import (
"fmt"
"io/ioutil"
"os"
"path/filepath"
"time"
@@ -33,16 +34,12 @@ func main() {
defer ticker.Stop()
for {
<-ticker.C
if done() {
fmt.Println("All restic restores are done")
err := removeFolder()
if err != nil {
fmt.Println(err)
} else {
fmt.Println("Done cleanup .velero folder")
select {
case <-ticker.C:
if done() {
fmt.Println("All restic restores are done")
return
}
return
}
}
}
@@ -51,7 +48,7 @@ func main() {
// within the .velero/ subdirectory whose name is equal to os.Args[1], or
// false otherwise
func done() bool {
children, err := os.ReadDir("/restores")
children, err := ioutil.ReadDir("/restores")
if err != nil {
fmt.Fprintf(os.Stderr, "ERROR reading /restores directory: %s\n", err)
return false
@@ -78,28 +75,3 @@ func done() bool {
return true
}
// remove .velero folder
func removeFolder() error {
children, err := os.ReadDir("/restores")
if err != nil {
return err
}
for _, child := range children {
if !child.IsDir() {
fmt.Printf("%s is not a directory, skipping.\n", child.Name())
continue
}
donePath := filepath.Join("/restores", child.Name(), ".velero")
err = os.RemoveAll(donePath)
if err != nil {
return err
}
fmt.Printf("Deleted %s", donePath)
}
return nil
}

View File

@@ -20,7 +20,7 @@ import (
"os"
"path/filepath"
"k8s.io/klog/v2"
"k8s.io/klog"
"github.com/vmware-tanzu/velero/pkg/cmd"
"github.com/vmware-tanzu/velero/pkg/cmd/velero"

View File

@@ -1,9 +1,11 @@
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.12.0
controller-gen.kubebuilder.io/version: v0.3.0
creationTimestamp: null
name: backups.velero.io
spec:
group: velero.io
@@ -35,46 +37,10 @@ spec:
spec:
description: BackupSpec defines the specification for a Velero backup.
properties:
csiSnapshotTimeout:
description: CSISnapshotTimeout specifies the time used to wait for
CSI VolumeSnapshot status turns to ReadyToUse during creation, before
returning error as timeout. The default value is 10 minute.
type: string
datamover:
description: DataMover specifies the data mover to be used by the
backup. If DataMover is "" or "velero", the built-in data mover
will be used.
type: string
defaultVolumesToFsBackup:
description: DefaultVolumesToFsBackup specifies whether pod volume
file system backup should be used for all volumes by default.
nullable: true
type: boolean
defaultVolumesToRestic:
description: "DefaultVolumesToRestic specifies whether restic should
be used to take a backup of all pod volumes by default. \n Deprecated:
this field is no longer used and will be removed entirely in future.
Use DefaultVolumesToFsBackup instead."
nullable: true
description: DefaultVolumesToRestic specifies whether restic should
be used to take a backup of all pod volumes by default.
type: boolean
excludedClusterScopedResources:
description: ExcludedClusterScopedResources is a slice of cluster-scoped
resource type names to exclude from the backup. If set to "*", all
cluster-scoped resource types are excluded. The default value is
empty.
items:
type: string
nullable: true
type: array
excludedNamespaceScopedResources:
description: ExcludedNamespaceScopedResources is a slice of namespace-scoped
resource type names to exclude from the backup. If set to "*", all
namespace-scoped resource types are excluded. The default value
is empty.
items:
type: string
nullable: true
type: array
excludedNamespaces:
description: ExcludedNamespaces contains a list of namespaces that
are not included in the backup.
@@ -178,7 +144,6 @@ spec:
contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
name:
description: Name is the name of this hook.
type: string
@@ -281,23 +246,6 @@ spec:
resources should be included for consideration in the backup.
nullable: true
type: boolean
includedClusterScopedResources:
description: IncludedClusterScopedResources is a slice of cluster-scoped
resource type names to include in the backup. If set to "*", all
cluster-scoped resource types are included. The default value is
empty, which means only related cluster-scoped resources are included.
items:
type: string
nullable: true
type: array
includedNamespaceScopedResources:
description: IncludedNamespaceScopedResources is a slice of namespace-scoped
resource type names to include in the backup. The default value
is "*".
items:
type: string
nullable: true
type: array
includedNamespaces:
description: IncludedNamespaces is a slice of namespace names to include
objects from. If empty, all namespaces are included.
@@ -312,11 +260,6 @@ spec:
type: string
nullable: true
type: array
itemOperationTimeout:
description: ItemOperationTimeout specifies the time used to wait
for asynchronous BackupItemAction operations The default value is
1 hour.
type: string
labelSelector:
description: LabelSelector is a metav1.LabelSelector to filter with
when adding individual objects to the backup. If empty or nil, all
@@ -364,109 +307,18 @@ spec:
are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
metadata:
properties:
labels:
additionalProperties:
type: string
type: object
type: object
orLabelSelectors:
description: OrLabelSelectors is list of metav1.LabelSelector to filter
with when adding individual objects to the backup. If multiple provided
they will be joined by the OR operator. LabelSelector as well as
OrLabelSelectors cannot co-exist in backup request, only one of
them can be used.
items:
description: A label selector is a label query over a set of resources.
The result of matchLabels and matchExpressions are ANDed. An empty
label selector matches all objects. A null label selector matches
no objects.
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements.
The requirements are ANDed.
items:
description: A label selector requirement is a selector that
contains values, a key, and an operator that relates the
key and values.
properties:
key:
description: key is the label key that the selector applies
to.
type: string
operator:
description: operator represents a key's relationship
to a set of values. Valid operators are In, NotIn, Exists
and DoesNotExist.
type: string
values:
description: values is an array of string values. If the
operator is In or NotIn, the values array must be non-empty.
If the operator is Exists or DoesNotExist, the values
array must be empty. This array is replaced during a
strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single
{key,value} in the matchLabels map is equivalent to an element
of matchExpressions, whose key field is "key", the operator
is "In", and the values array contains only "value". The requirements
are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
nullable: true
type: array
orderedResources:
additionalProperties:
type: string
description: OrderedResources specifies the backup order of resources
of specific Kind. The map key is the resource name and value is
a list of object names separated by commas. Each resource name has
format "namespace/objectname". For cluster resources, simply use
"objectname".
of specific Kind. The map key is the Kind name and value is a list
of resource names separated by commas. Each resource name has format
"namespace/resourcename". For cluster resources, simply use "resourcename".
nullable: true
type: object
resourcePolicy:
description: ResourcePolicy specifies the referenced resource policies
that backup should follow
properties:
apiGroup:
description: APIGroup is the group for the resource being referenced.
If APIGroup is not specified, the specified Kind must be in
the core API group. For any other third-party types, APIGroup
is required.
type: string
kind:
description: Kind is the type of resource being referenced
type: string
name:
description: Name is the name of resource being referenced
type: string
required:
- kind
- name
type: object
x-kubernetes-map-type: atomic
snapshotMoveData:
description: SnapshotMoveData specifies whether snapshot data should
be moved
nullable: true
type: boolean
snapshotVolumes:
description: SnapshotVolumes specifies whether to take snapshots of
any PV's referenced in the set of objects included in the Backup.
description: SnapshotVolumes specifies whether to take cloud snapshots
of any PV's referenced in the set of objects included in the Backup.
nullable: true
type: boolean
storageLocation:
@@ -487,20 +339,6 @@ spec:
status:
description: BackupStatus captures the current status of a Velero backup.
properties:
backupItemOperationsAttempted:
description: BackupItemOperationsAttempted is the total number of
attempted async BackupItemAction operations for this backup.
type: integer
backupItemOperationsCompleted:
description: BackupItemOperationsCompleted is the total number of
successfully completed async BackupItemAction operations for this
backup.
type: integer
backupItemOperationsFailed:
description: BackupItemOperationsFailed is the total number of async
BackupItemAction operations for this backup which ended with an
error.
type: integer
completionTimestamp:
description: CompletionTimestamp records the time a backup was completed.
Completion time is recorded even on failed backups. Completion time
@@ -509,14 +347,6 @@ spec:
format: date-time
nullable: true
type: string
csiVolumeSnapshotsAttempted:
description: CSIVolumeSnapshotsAttempted is the total number of attempted
CSI VolumeSnapshots for this backup.
type: integer
csiVolumeSnapshotsCompleted:
description: CSIVolumeSnapshotsCompleted is the total number of successfully
completed CSI VolumeSnapshots for this backup.
type: integer
errors:
description: Errors is a count of all error messages that were generated
during execution of the backup. The actual errors are in the backup's
@@ -527,10 +357,6 @@ spec:
format: date-time
nullable: true
type: string
failureReason:
description: FailureReason is an error that caused the entire backup
to fail.
type: string
formatVersion:
description: FormatVersion is the backup format version, including
major, minor, and patch version.
@@ -541,10 +367,6 @@ spec:
- New
- FailedValidation
- InProgress
- WaitingForPluginOperations
- WaitingForPluginOperationsPartiallyFailed
- Finalizing
- FinalizingPartiallyFailed
- Completed
- PartiallyFailed
- Failed
@@ -603,3 +425,9 @@ spec:
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

View File

@@ -1,9 +1,11 @@
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.12.0
controller-gen.kubebuilder.io/version: v0.3.0
creationTimestamp: null
name: backupstoragelocations.velero.io
spec:
group: velero.io
@@ -90,7 +92,6 @@ spec:
required:
- key
type: object
x-kubernetes-map-type: atomic
default:
description: Default indicates this location is the default backup
storage location.
@@ -157,10 +158,6 @@ spec:
format: date-time
nullable: true
type: string
message:
description: Message is a message about the backup storage location's
status.
type: string
phase:
description: Phase is the current state of the BackupStorageLocation.
enum:
@@ -171,4 +168,11 @@ spec:
type: object
served: true
storage: true
subresources: {}
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

View File

@@ -1,9 +1,11 @@
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.12.0
controller-gen.kubebuilder.io/version: v0.3.0
creationTimestamp: null
name: deletebackuprequests.velero.io
spec:
group: velero.io
@@ -14,16 +16,7 @@ spec:
singular: deletebackuprequest
scope: Namespaced
versions:
- additionalPrinterColumns:
- description: The name of the backup to be deleted
jsonPath: .spec.backupName
name: BackupName
type: string
- description: The status of the deletion request
jsonPath: .status.phase
name: Status
type: string
name: v1
- name: v1
schema:
openAPIV3Schema:
description: DeleteBackupRequest is a request to delete one or more backups.
@@ -70,4 +63,9 @@ spec:
type: object
served: true
storage: true
subresources: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

View File

@@ -1,9 +1,11 @@
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.12.0
controller-gen.kubebuilder.io/version: v0.3.0
creationTimestamp: null
name: downloadrequests.velero.io
spec:
group: velero.io
@@ -44,18 +46,12 @@ spec:
- BackupLog
- BackupContents
- BackupVolumeSnapshots
- BackupItemOperations
- BackupResourceList
- BackupResults
- RestoreLog
- RestoreResults
- RestoreResourceList
- RestoreItemOperations
- CSIBackupVolumeSnapshots
- CSIBackupVolumeSnapshotContents
type: string
name:
description: Name is the name of the Kubernetes resource with
description: Name is the name of the kubernetes resource with
which the file is associated.
type: string
required:
@@ -88,3 +84,11 @@ spec:
type: object
served: true
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

View File

@@ -1,9 +1,11 @@
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.12.0
controller-gen.kubebuilder.io/version: v0.3.0
creationTimestamp: null
name: podvolumebackups.velero.io
spec:
group: velero.io
@@ -14,44 +16,7 @@ spec:
singular: podvolumebackup
scope: Namespaced
versions:
- additionalPrinterColumns:
- description: Pod Volume Backup status such as New/InProgress
jsonPath: .status.phase
name: Status
type: string
- description: Time when this backup was started
jsonPath: .status.startTimestamp
name: Created
type: date
- description: Namespace of the pod containing the volume to be backed up
jsonPath: .spec.pod.namespace
name: Namespace
type: string
- description: Name of the pod containing the volume to be backed up
jsonPath: .spec.pod.name
name: Pod
type: string
- description: Name of the volume to be backed up
jsonPath: .spec.volume
name: Volume
type: string
- description: Backup repository identifier for this backup
jsonPath: .spec.repoIdentifier
name: Repository ID
type: string
- description: The type of the uploader to handle data transfer
jsonPath: .spec.uploaderType
name: Uploader Type
type: string
- description: Name of the Backup Storage Location where this backup should be
stored
jsonPath: .spec.backupStorageLocation
name: Storage Location
type: string
- jsonPath: .metadata.creationTimestamp
name: Age
type: date
name: v1
- name: v1
schema:
openAPIV3Schema:
properties:
@@ -72,7 +37,7 @@ spec:
properties:
backupStorageLocation:
description: BackupStorageLocation is the name of the backup storage
location where the backup repository is stored.
location where the restic repository is stored.
type: string
node:
description: Node is the name of the node that the Pod is running
@@ -115,9 +80,8 @@ spec:
description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
type: string
type: object
x-kubernetes-map-type: atomic
repoIdentifier:
description: RepoIdentifier is the backup repository identifier.
description: RepoIdentifier is the restic repository identifier.
type: string
tags:
additionalProperties:
@@ -125,14 +89,6 @@ spec:
description: Tags are a map of key-value pairs that should be applied
to the volume backup as tags.
type: object
uploaderType:
description: UploaderType is the type of the uploader to handle the
data transfer.
enum:
- kopia
- restic
- ""
type: string
volume:
description: Volume is the name of the volume within the Pod to be
backed up.
@@ -197,4 +153,9 @@ spec:
type: object
served: true
storage: true
subresources: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

View File

@@ -1,9 +1,11 @@
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.12.0
controller-gen.kubebuilder.io/version: v0.3.0
creationTimestamp: null
name: podvolumerestores.velero.io
spec:
group: velero.io
@@ -14,41 +16,7 @@ spec:
singular: podvolumerestore
scope: Namespaced
versions:
- additionalPrinterColumns:
- description: Namespace of the pod containing the volume to be restored
jsonPath: .spec.pod.namespace
name: Namespace
type: string
- description: Name of the pod containing the volume to be restored
jsonPath: .spec.pod.name
name: Pod
type: string
- description: The type of the uploader to handle data transfer
jsonPath: .spec.uploaderType
name: Uploader Type
type: string
- description: Name of the volume to be restored
jsonPath: .spec.volume
name: Volume
type: string
- description: Pod Volume Restore status such as New/InProgress
jsonPath: .status.phase
name: Status
type: string
- description: Pod Volume Restore status such as New/InProgress
format: int64
jsonPath: .status.progress.totalBytes
name: TotalBytes
type: integer
- description: Pod Volume Restore status such as New/InProgress
format: int64
jsonPath: .status.progress.bytesDone
name: BytesDone
type: integer
- jsonPath: .metadata.creationTimestamp
name: Age
type: date
name: v1
- name: v1
schema:
openAPIV3Schema:
properties:
@@ -69,7 +37,7 @@ spec:
properties:
backupStorageLocation:
description: BackupStorageLocation is the name of the backup storage
location where the backup repository is stored.
location where the restic repository is stored.
type: string
pod:
description: Pod is a reference to the pod containing the volume to
@@ -108,25 +76,12 @@ spec:
description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
type: string
type: object
x-kubernetes-map-type: atomic
repoIdentifier:
description: RepoIdentifier is the backup repository identifier.
description: RepoIdentifier is the restic repository identifier.
type: string
snapshotID:
description: SnapshotID is the ID of the volume snapshot to be restored.
type: string
sourceNamespace:
description: SourceNamespace is the original namespace for namaspace
mapping.
type: string
uploaderType:
description: UploaderType is the type of the uploader to handle the
data transfer.
enum:
- kopia
- restic
- ""
type: string
volume:
description: Volume is the name of the volume within the Pod to be
restored.
@@ -136,7 +91,6 @@ spec:
- pod
- repoIdentifier
- snapshotID
- sourceNamespace
- volume
type: object
status:
@@ -182,4 +136,9 @@ spec:
type: object
served: true
storage: true
subresources: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

View File

@@ -1,27 +1,22 @@
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.12.0
name: backuprepositories.velero.io
controller-gen.kubebuilder.io/version: v0.3.0
creationTimestamp: null
name: resticrepositories.velero.io
spec:
group: velero.io
names:
kind: BackupRepository
listKind: BackupRepositoryList
plural: backuprepositories
singular: backuprepository
kind: ResticRepository
listKind: ResticRepositoryList
plural: resticrepositories
singular: resticrepository
scope: Namespaced
versions:
- additionalPrinterColumns:
- jsonPath: .metadata.creationTimestamp
name: Age
type: date
- jsonPath: .spec.repositoryType
name: Repository Type
type: string
name: v1
- name: v1
schema:
openAPIV3Schema:
properties:
@@ -38,7 +33,7 @@ spec:
metadata:
type: object
spec:
description: BackupRepositorySpec is the specification for a BackupRepository.
description: ResticRepositorySpec is the specification for a ResticRepository.
properties:
backupStorageLocation:
description: BackupStorageLocation is the name of the BackupStorageLocation
@@ -48,19 +43,12 @@ spec:
description: MaintenanceFrequency is how often maintenance should
be run.
type: string
repositoryType:
description: RepositoryType indicates the type of the backend repository
enum:
- kopia
- restic
- ""
type: string
resticIdentifier:
description: ResticIdentifier is the full restic-compatible string
for identifying this repository.
type: string
volumeNamespace:
description: VolumeNamespace is the namespace this backup repository
description: VolumeNamespace is the namespace this restic repository
contains pod volume backups for.
type: string
required:
@@ -70,7 +58,7 @@ spec:
- volumeNamespace
type: object
status:
description: BackupRepositoryStatus is the current status of a BackupRepository.
description: ResticRepositoryStatus is the current status of a ResticRepository.
properties:
lastMaintenanceTime:
description: LastMaintenanceTime is the last time maintenance was
@@ -80,10 +68,10 @@ spec:
type: string
message:
description: Message is a message about the current status of the
BackupRepository.
ResticRepository.
type: string
phase:
description: Phase is the current state of the BackupRepository.
description: Phase is the current state of the ResticRepository.
enum:
- New
- Ready
@@ -93,4 +81,9 @@ spec:
type: object
served: true
storage: true
subresources: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

File diff suppressed because it is too large Load Diff

View File

@@ -1,9 +1,11 @@
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.12.0
controller-gen.kubebuilder.io/version: v0.3.0
creationTimestamp: null
name: schedules.velero.io
spec:
group: velero.io
@@ -14,26 +16,7 @@ spec:
singular: schedule
scope: Namespaced
versions:
- additionalPrinterColumns:
- description: Status of the schedule
jsonPath: .status.phase
name: Status
type: string
- description: A Cron expression defining when to run the Backup
jsonPath: .spec.schedule
name: Schedule
type: string
- description: The last time a Backup was run for this schedule
jsonPath: .status.lastBackup
name: LastBackup
type: date
- jsonPath: .metadata.creationTimestamp
name: Age
type: date
- jsonPath: .spec.paused
name: Paused
type: boolean
name: v1
- name: v1
schema:
openAPIV3Schema:
description: Schedule is a Velero resource that represents a pre-scheduled
@@ -54,9 +37,6 @@ spec:
spec:
description: ScheduleSpec defines the specification for a Velero schedule
properties:
paused:
description: Paused specifies whether the schedule is paused or not
type: boolean
schedule:
description: Schedule is a Cron expression defining when to run the
Backup.
@@ -65,46 +45,10 @@ spec:
description: Template is the definition of the Backup to be run on
the provided schedule
properties:
csiSnapshotTimeout:
description: CSISnapshotTimeout specifies the time used to wait
for CSI VolumeSnapshot status turns to ReadyToUse during creation,
before returning error as timeout. The default value is 10 minute.
type: string
datamover:
description: DataMover specifies the data mover to be used by
the backup. If DataMover is "" or "velero", the built-in data
mover will be used.
type: string
defaultVolumesToFsBackup:
description: DefaultVolumesToFsBackup specifies whether pod volume
file system backup should be used for all volumes by default.
nullable: true
type: boolean
defaultVolumesToRestic:
description: "DefaultVolumesToRestic specifies whether restic
should be used to take a backup of all pod volumes by default.
\n Deprecated: this field is no longer used and will be removed
entirely in future. Use DefaultVolumesToFsBackup instead."
nullable: true
description: DefaultVolumesToRestic specifies whether restic should
be used to take a backup of all pod volumes by default.
type: boolean
excludedClusterScopedResources:
description: ExcludedClusterScopedResources is a slice of cluster-scoped
resource type names to exclude from the backup. If set to "*",
all cluster-scoped resource types are excluded. The default
value is empty.
items:
type: string
nullable: true
type: array
excludedNamespaceScopedResources:
description: ExcludedNamespaceScopedResources is a slice of namespace-scoped
resource type names to exclude from the backup. If set to "*",
all namespace-scoped resource types are excluded. The default
value is empty.
items:
type: string
nullable: true
type: array
excludedNamespaces:
description: ExcludedNamespaces contains a list of namespaces
that are not included in the backup.
@@ -209,7 +153,6 @@ spec:
requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
name:
description: Name is the name of this hook.
type: string
@@ -316,24 +259,6 @@ spec:
resources should be included for consideration in the backup.
nullable: true
type: boolean
includedClusterScopedResources:
description: IncludedClusterScopedResources is a slice of cluster-scoped
resource type names to include in the backup. If set to "*",
all cluster-scoped resource types are included. The default
value is empty, which means only related cluster-scoped resources
are included.
items:
type: string
nullable: true
type: array
includedNamespaceScopedResources:
description: IncludedNamespaceScopedResources is a slice of namespace-scoped
resource type names to include in the backup. The default value
is "*".
items:
type: string
nullable: true
type: array
includedNamespaces:
description: IncludedNamespaces is a slice of namespace names
to include objects from. If empty, all namespaces are included.
@@ -348,11 +273,6 @@ spec:
type: string
nullable: true
type: array
itemOperationTimeout:
description: ItemOperationTimeout specifies the time used to wait
for asynchronous BackupItemAction operations The default value
is 1 hour.
type: string
labelSelector:
description: LabelSelector is a metav1.LabelSelector to filter
with when adding individual objects to the backup. If empty
@@ -400,108 +320,18 @@ spec:
"value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
metadata:
properties:
labels:
additionalProperties:
type: string
type: object
type: object
orLabelSelectors:
description: OrLabelSelectors is list of metav1.LabelSelector
to filter with when adding individual objects to the backup.
If multiple provided they will be joined by the OR operator.
LabelSelector as well as OrLabelSelectors cannot co-exist in
backup request, only one of them can be used.
items:
description: A label selector is a label query over a set of
resources. The result of matchLabels and matchExpressions
are ANDed. An empty label selector matches all objects. A
null label selector matches no objects.
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector
that contains values, a key, and an operator that relates
the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: operator represents a key's relationship
to a set of values. Valid operators are In, NotIn,
Exists and DoesNotExist.
type: string
values:
description: values is an array of string values.
If the operator is In or NotIn, the values array
must be non-empty. If the operator is Exists or
DoesNotExist, the values array must be empty. This
array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs.
A single {key,value} in the matchLabels map is equivalent
to an element of matchExpressions, whose key field is
"key", the operator is "In", and the values array contains
only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
nullable: true
type: array
orderedResources:
additionalProperties:
type: string
description: OrderedResources specifies the backup order of resources
of specific Kind. The map key is the resource name and value
is a list of object names separated by commas. Each resource
name has format "namespace/objectname". For cluster resources,
simply use "objectname".
of specific Kind. The map key is the Kind name and value is
a list of resource names separated by commas. Each resource
name has format "namespace/resourcename". For cluster resources,
simply use "resourcename".
nullable: true
type: object
resourcePolicy:
description: ResourcePolicy specifies the referenced resource
policies that backup should follow
properties:
apiGroup:
description: APIGroup is the group for the resource being
referenced. If APIGroup is not specified, the specified
Kind must be in the core API group. For any other third-party
types, APIGroup is required.
type: string
kind:
description: Kind is the type of resource being referenced
type: string
name:
description: Name is the name of resource being referenced
type: string
required:
- kind
- name
type: object
x-kubernetes-map-type: atomic
snapshotMoveData:
description: SnapshotMoveData specifies whether snapshot data
should be moved
nullable: true
type: boolean
snapshotVolumes:
description: SnapshotVolumes specifies whether to take snapshots
description: SnapshotVolumes specifies whether to take cloud snapshots
of any PV's referenced in the set of objects included in the
Backup.
nullable: true
@@ -556,4 +386,9 @@ spec:
type: object
served: true
storage: true
subresources: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

View File

@@ -1,9 +1,11 @@
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.12.0
controller-gen.kubebuilder.io/version: v0.3.0
creationTimestamp: null
name: serverstatusrequests.velero.io
spec:
group: velero.io
@@ -75,3 +77,11 @@ spec:
type: object
served: true
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

View File

@@ -1,9 +1,11 @@
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.12.0
controller-gen.kubebuilder.io/version: v0.3.0
creationTimestamp: null
name: volumesnapshotlocations.velero.io
spec:
group: velero.io
@@ -11,8 +13,6 @@ spec:
kind: VolumeSnapshotLocation
listKind: VolumeSnapshotLocationList
plural: volumesnapshotlocations
shortNames:
- vsl
singular: volumesnapshotlocation
scope: Namespaced
versions:
@@ -43,25 +43,6 @@ spec:
type: string
description: Config is for provider-specific configuration fields.
type: object
credential:
description: Credential contains the credential information intended
to be used with this location
properties:
key:
description: The key of the secret to select from. Must be a
valid secret key.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind, uid?'
type: string
optional:
description: Specify whether the Secret or its key must be defined
type: boolean
required:
- key
type: object
x-kubernetes-map-type: atomic
provider:
description: Provider is the provider of the volume storage.
type: string
@@ -83,3 +64,9 @@ spec:
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,432 @@
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.3.0
creationTimestamp: null
name: backups.velero.io
spec:
group: velero.io
names:
kind: Backup
listKind: BackupList
plural: backups
singular: backup
preserveUnknownFields: false
scope: Namespaced
validation:
openAPIV3Schema:
description: Backup is a Velero resource that represents the capture of Kubernetes
cluster state at a point in time (API objects and associated volume state).
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: BackupSpec defines the specification for a Velero backup.
properties:
defaultVolumesToRestic:
description: DefaultVolumesToRestic specifies whether restic should
be used to take a backup of all pod volumes by default.
type: boolean
excludedNamespaces:
description: ExcludedNamespaces contains a list of namespaces that are
not included in the backup.
items:
type: string
nullable: true
type: array
excludedResources:
description: ExcludedResources is a slice of resource names that are
not included in the backup.
items:
type: string
nullable: true
type: array
hooks:
description: Hooks represent custom behaviors that should be executed
at different phases of the backup.
properties:
resources:
description: Resources are hooks that should be executed when backing
up individual instances of a resource.
items:
description: BackupResourceHookSpec defines one or more BackupResourceHooks
that should be executed based on the rules defined for namespaces,
resources, and label selector.
properties:
excludedNamespaces:
description: ExcludedNamespaces specifies the namespaces to
which this hook spec does not apply.
items:
type: string
nullable: true
type: array
excludedResources:
description: ExcludedResources specifies the resources to
which this hook spec does not apply.
items:
type: string
nullable: true
type: array
includedNamespaces:
description: IncludedNamespaces specifies the namespaces to
which this hook spec applies. If empty, it applies to all
namespaces.
items:
type: string
nullable: true
type: array
includedResources:
description: IncludedResources specifies the resources to
which this hook spec applies. If empty, it applies to all
resources.
items:
type: string
nullable: true
type: array
labelSelector:
description: LabelSelector, if specified, filters the resources
to which this hook spec applies.
nullable: true
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector
that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: operator represents a key's relationship
to a set of values. Valid operators are In, NotIn,
Exists and DoesNotExist.
type: string
values:
description: values is an array of string values.
If the operator is In or NotIn, the values array
must be non-empty. If the operator is Exists or
DoesNotExist, the values array must be empty.
This array is replaced during a strategic merge
patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs.
A single {key,value} in the matchLabels map is equivalent
to an element of matchExpressions, whose key field is
"key", the operator is "In", and the values array contains
only "value". The requirements are ANDed.
type: object
type: object
name:
description: Name is the name of this hook.
type: string
post:
description: PostHooks is a list of BackupResourceHooks to
execute after storing the item in the backup. These are
executed after all "additional items" from item actions
are processed.
items:
description: BackupResourceHook defines a hook for a resource.
properties:
exec:
description: Exec defines an exec hook.
properties:
command:
description: Command is the command and arguments
to execute.
items:
type: string
minItems: 1
type: array
container:
description: Container is the container in the pod
where the command should be executed. If not specified,
the pod's first container is used.
type: string
onError:
description: OnError specifies how Velero should
behave if it encounters an error executing this
hook.
enum:
- Continue
- Fail
type: string
timeout:
description: Timeout defines the maximum amount
of time Velero should wait for the hook to complete
before considering the execution a failure.
type: string
required:
- command
type: object
required:
- exec
type: object
type: array
pre:
description: PreHooks is a list of BackupResourceHooks to
execute prior to storing the item in the backup. These are
executed before any "additional items" from item actions
are processed.
items:
description: BackupResourceHook defines a hook for a resource.
properties:
exec:
description: Exec defines an exec hook.
properties:
command:
description: Command is the command and arguments
to execute.
items:
type: string
minItems: 1
type: array
container:
description: Container is the container in the pod
where the command should be executed. If not specified,
the pod's first container is used.
type: string
onError:
description: OnError specifies how Velero should
behave if it encounters an error executing this
hook.
enum:
- Continue
- Fail
type: string
timeout:
description: Timeout defines the maximum amount
of time Velero should wait for the hook to complete
before considering the execution a failure.
type: string
required:
- command
type: object
required:
- exec
type: object
type: array
required:
- name
type: object
nullable: true
type: array
type: object
includeClusterResources:
description: IncludeClusterResources specifies whether cluster-scoped
resources should be included for consideration in the backup.
nullable: true
type: boolean
includedNamespaces:
description: IncludedNamespaces is a slice of namespace names to include
objects from. If empty, all namespaces are included.
items:
type: string
nullable: true
type: array
includedResources:
description: IncludedResources is a slice of resource names to include
in the backup. If empty, all resources are included.
items:
type: string
nullable: true
type: array
labelSelector:
description: LabelSelector is a metav1.LabelSelector to filter with
when adding individual objects to the backup. If empty or nil, all
objects are included. Optional.
nullable: true
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements.
The requirements are ANDed.
items:
description: A label selector requirement is a selector that contains
values, a key, and an operator that relates the key and values.
properties:
key:
description: key is the label key that the selector applies
to.
type: string
operator:
description: operator represents a key's relationship to a
set of values. Valid operators are In, NotIn, Exists and
DoesNotExist.
type: string
values:
description: values is an array of string values. If the operator
is In or NotIn, the values array must be non-empty. If the
operator is Exists or DoesNotExist, the values array must
be empty. This array is replaced during a strategic merge
patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single
{key,value} in the matchLabels map is equivalent to an element
of matchExpressions, whose key field is "key", the operator is
"In", and the values array contains only "value". The requirements
are ANDed.
type: object
type: object
orderedResources:
additionalProperties:
type: string
description: OrderedResources specifies the backup order of resources
of specific Kind. The map key is the Kind name and value is a list
of resource names separated by commas. Each resource name has format
"namespace/resourcename". For cluster resources, simply use "resourcename".
nullable: true
type: object
snapshotVolumes:
description: SnapshotVolumes specifies whether to take cloud snapshots
of any PV's referenced in the set of objects included in the Backup.
nullable: true
type: boolean
storageLocation:
description: StorageLocation is a string containing the name of a BackupStorageLocation
where the backup should be stored.
type: string
ttl:
description: TTL is a time.Duration-parseable string describing how
long the Backup should be retained for.
type: string
volumeSnapshotLocations:
description: VolumeSnapshotLocations is a list containing names of VolumeSnapshotLocations
associated with this backup.
items:
type: string
type: array
type: object
status:
description: BackupStatus captures the current status of a Velero backup.
properties:
completionTimestamp:
description: CompletionTimestamp records the time a backup was completed.
Completion time is recorded even on failed backups. Completion time
is recorded before uploading the backup object. The server's time
is used for CompletionTimestamps
format: date-time
nullable: true
type: string
errors:
description: Errors is a count of all error messages that were generated
during execution of the backup. The actual errors are in the backup's
log file in object storage.
type: integer
expiration:
description: Expiration is when this Backup is eligible for garbage-collection.
format: date-time
nullable: true
type: string
formatVersion:
description: FormatVersion is the backup format version, including major,
minor, and patch version.
type: string
phase:
description: Phase is the current state of the Backup.
enum:
- New
- FailedValidation
- InProgress
- Completed
- PartiallyFailed
- Failed
- Deleting
type: string
progress:
description: Progress contains information about the backup's execution
progress. Note that this information is best-effort only -- if Velero
fails to update it during a backup for any reason, it may be inaccurate/stale.
nullable: true
properties:
itemsBackedUp:
description: ItemsBackedUp is the number of items that have actually
been written to the backup tarball so far.
type: integer
totalItems:
description: TotalItems is the total number of items to be backed
up. This number may change throughout the execution of the backup
due to plugins that return additional related items to back up,
the velero.io/exclude-from-backup label, and various other filters
that happen as items are processed.
type: integer
type: object
startTimestamp:
description: StartTimestamp records the time a backup was started. Separate
from CreationTimestamp, since that value changes on restores. The
server's time is used for StartTimestamps
format: date-time
nullable: true
type: string
validationErrors:
description: ValidationErrors is a slice of all validation errors (if
applicable).
items:
type: string
nullable: true
type: array
version:
description: 'Version is the backup format major version. Deprecated:
Please see FormatVersion'
type: integer
volumeSnapshotsAttempted:
description: VolumeSnapshotsAttempted is the total number of attempted
volume snapshots for this backup.
type: integer
volumeSnapshotsCompleted:
description: VolumeSnapshotsCompleted is the total number of successfully
completed volume snapshots for this backup.
type: integer
warnings:
description: Warnings is a count of all warning messages that were generated
during execution of the backup. The actual warnings are in the backup's
log file in object storage.
type: integer
type: object
type: object
version: v1
versions:
- name: v1
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

View File

@@ -0,0 +1,179 @@
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.3.0
creationTimestamp: null
name: backupstoragelocations.velero.io
spec:
additionalPrinterColumns:
- JSONPath: .status.phase
description: Backup Storage Location status such as Available/Unavailable
name: Phase
type: string
- JSONPath: .status.lastValidationTime
description: LastValidationTime is the last time the backup store location was
validated
name: Last Validated
type: date
- JSONPath: .metadata.creationTimestamp
name: Age
type: date
- JSONPath: .spec.default
description: Default backup storage location
name: Default
type: boolean
group: velero.io
names:
kind: BackupStorageLocation
listKind: BackupStorageLocationList
plural: backupstoragelocations
shortNames:
- bsl
singular: backupstoragelocation
preserveUnknownFields: false
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
description: BackupStorageLocation is a location where Velero stores backup
objects
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: BackupStorageLocationSpec defines the desired state of a Velero
BackupStorageLocation
properties:
accessMode:
description: AccessMode defines the permissions for the backup storage
location.
enum:
- ReadOnly
- ReadWrite
type: string
backupSyncPeriod:
description: BackupSyncPeriod defines how frequently to sync backup
API objects from object storage. A value of 0 disables sync.
nullable: true
type: string
config:
additionalProperties:
type: string
description: Config is for provider-specific configuration fields.
type: object
credential:
description: Credential contains the credential information intended
to be used with this location
properties:
key:
description: The key of the secret to select from. Must be a valid
secret key.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind, uid?'
type: string
optional:
description: Specify whether the Secret or its key must be defined
type: boolean
required:
- key
type: object
default:
description: Default indicates this location is the default backup storage
location.
type: boolean
objectStorage:
description: ObjectStorageLocation specifies the settings necessary
to connect to a provider's object storage.
properties:
bucket:
description: Bucket is the bucket to use for object storage.
type: string
caCert:
description: CACert defines a CA bundle to use when verifying TLS
connections to the provider.
format: byte
type: string
prefix:
description: Prefix is the path inside a bucket to use for Velero
storage. Optional.
type: string
required:
- bucket
type: object
provider:
description: Provider is the provider of the backup storage.
type: string
validationFrequency:
description: ValidationFrequency defines how frequently to validate
the corresponding object storage. A value of 0 disables validation.
nullable: true
type: string
required:
- objectStorage
- provider
type: object
status:
description: BackupStorageLocationStatus defines the observed state of BackupStorageLocation
properties:
accessMode:
description: "AccessMode is an unused field. \n Deprecated: there is
now an AccessMode field on the Spec and this field will be removed
entirely as of v2.0."
enum:
- ReadOnly
- ReadWrite
type: string
lastSyncedRevision:
description: "LastSyncedRevision is the value of the `metadata/revision`
file in the backup storage location the last time the BSL's contents
were synced into the cluster. \n Deprecated: this field is no longer
updated or used for detecting changes to the location's contents and
will be removed entirely in v2.0."
type: string
lastSyncedTime:
description: LastSyncedTime is the last time the contents of the location
were synced into the cluster.
format: date-time
nullable: true
type: string
lastValidationTime:
description: LastValidationTime is the last time the backup store location
was validated the cluster.
format: date-time
nullable: true
type: string
phase:
description: Phase is the current state of the BackupStorageLocation.
enum:
- Available
- Unavailable
type: string
type: object
type: object
version: v1
versions:
- name: v1
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

View File

@@ -0,0 +1,73 @@
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.3.0
creationTimestamp: null
name: deletebackuprequests.velero.io
spec:
group: velero.io
names:
kind: DeleteBackupRequest
listKind: DeleteBackupRequestList
plural: deletebackuprequests
singular: deletebackuprequest
preserveUnknownFields: false
scope: Namespaced
validation:
openAPIV3Schema:
description: DeleteBackupRequest is a request to delete one or more backups.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: DeleteBackupRequestSpec is the specification for which backups
to delete.
properties:
backupName:
type: string
required:
- backupName
type: object
status:
description: DeleteBackupRequestStatus is the current status of a DeleteBackupRequest.
properties:
errors:
description: Errors contains any errors that were encountered during
the deletion process.
items:
type: string
nullable: true
type: array
phase:
description: Phase is the current state of the DeleteBackupRequest.
enum:
- New
- InProgress
- Processed
type: string
type: object
type: object
version: v1
versions:
- name: v1
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

View File

@@ -0,0 +1,96 @@
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.3.0
creationTimestamp: null
name: downloadrequests.velero.io
spec:
group: velero.io
names:
kind: DownloadRequest
listKind: DownloadRequestList
plural: downloadrequests
singular: downloadrequest
preserveUnknownFields: false
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
description: DownloadRequest is a request to download an artifact from backup
object storage, such as a backup log file.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: DownloadRequestSpec is the specification for a download request.
properties:
target:
description: Target is what to download (e.g. logs for a backup).
properties:
kind:
description: Kind is the type of file to download.
enum:
- BackupLog
- BackupContents
- BackupVolumeSnapshots
- BackupResourceList
- RestoreLog
- RestoreResults
type: string
name:
description: Name is the name of the kubernetes resource with which
the file is associated.
type: string
required:
- kind
- name
type: object
required:
- target
type: object
status:
description: DownloadRequestStatus is the current status of a DownloadRequest.
properties:
downloadURL:
description: DownloadURL contains the pre-signed URL for the target
file.
type: string
expiration:
description: Expiration is when this DownloadRequest expires and can
be deleted by the system.
format: date-time
nullable: true
type: string
phase:
description: Phase is the current state of the DownloadRequest.
enum:
- New
- Processed
type: string
type: object
type: object
version: v1
versions:
- name: v1
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

View File

@@ -0,0 +1,162 @@
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.3.0
creationTimestamp: null
name: podvolumebackups.velero.io
spec:
group: velero.io
names:
kind: PodVolumeBackup
listKind: PodVolumeBackupList
plural: podvolumebackups
singular: podvolumebackup
preserveUnknownFields: false
scope: Namespaced
validation:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: PodVolumeBackupSpec is the specification for a PodVolumeBackup.
properties:
backupStorageLocation:
description: BackupStorageLocation is the name of the backup storage
location where the restic repository is stored.
type: string
node:
description: Node is the name of the node that the Pod is running on.
type: string
pod:
description: Pod is a reference to the pod containing the volume to
be backed up.
properties:
apiVersion:
description: API version of the referent.
type: string
fieldPath:
description: 'If referring to a piece of an object instead of an
entire object, this string should contain a valid JSON/Go field
access statement, such as desiredState.manifest.containers[2].
For example, if the object reference is to a container within
a pod, this would take on a value like: "spec.containers{name}"
(where "name" refers to the name of the container that triggered
the event) or if no container name is specified "spec.containers[2]"
(container with index 2 in this pod). This syntax is chosen only
to have some well-defined way of referencing a part of an object.
TODO: this design is not final and this field is subject to change
in the future.'
type: string
kind:
description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
type: string
namespace:
description: 'Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
type: string
resourceVersion:
description: 'Specific resourceVersion to which this reference is
made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
type: string
uid:
description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
type: string
type: object
repoIdentifier:
description: RepoIdentifier is the restic repository identifier.
type: string
tags:
additionalProperties:
type: string
description: Tags are a map of key-value pairs that should be applied
to the volume backup as tags.
type: object
volume:
description: Volume is the name of the volume within the Pod to be backed
up.
type: string
required:
- backupStorageLocation
- node
- pod
- repoIdentifier
- volume
type: object
status:
description: PodVolumeBackupStatus is the current status of a PodVolumeBackup.
properties:
completionTimestamp:
description: CompletionTimestamp records the time a backup was completed.
Completion time is recorded even on failed backups. Completion time
is recorded before uploading the backup object. The server's time
is used for CompletionTimestamps
format: date-time
nullable: true
type: string
message:
description: Message is a message about the pod volume backup's status.
type: string
path:
description: Path is the full path within the controller pod being backed
up.
type: string
phase:
description: Phase is the current state of the PodVolumeBackup.
enum:
- New
- InProgress
- Completed
- Failed
type: string
progress:
description: Progress holds the total number of bytes of the volume
and the current number of backed up bytes. This can be used to display
progress information about the backup operation.
properties:
bytesDone:
format: int64
type: integer
totalBytes:
format: int64
type: integer
type: object
snapshotID:
description: SnapshotID is the identifier for the snapshot of the pod
volume.
type: string
startTimestamp:
description: StartTimestamp records the time a backup was started. Separate
from CreationTimestamp, since that value changes on restores. The
server's time is used for StartTimestamps
format: date-time
nullable: true
type: string
type: object
type: object
version: v1
versions:
- name: v1
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

View File

@@ -0,0 +1,145 @@
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.3.0
creationTimestamp: null
name: podvolumerestores.velero.io
spec:
group: velero.io
names:
kind: PodVolumeRestore
listKind: PodVolumeRestoreList
plural: podvolumerestores
singular: podvolumerestore
preserveUnknownFields: false
scope: Namespaced
validation:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: PodVolumeRestoreSpec is the specification for a PodVolumeRestore.
properties:
backupStorageLocation:
description: BackupStorageLocation is the name of the backup storage
location where the restic repository is stored.
type: string
pod:
description: Pod is a reference to the pod containing the volume to
be restored.
properties:
apiVersion:
description: API version of the referent.
type: string
fieldPath:
description: 'If referring to a piece of an object instead of an
entire object, this string should contain a valid JSON/Go field
access statement, such as desiredState.manifest.containers[2].
For example, if the object reference is to a container within
a pod, this would take on a value like: "spec.containers{name}"
(where "name" refers to the name of the container that triggered
the event) or if no container name is specified "spec.containers[2]"
(container with index 2 in this pod). This syntax is chosen only
to have some well-defined way of referencing a part of an object.
TODO: this design is not final and this field is subject to change
in the future.'
type: string
kind:
description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
type: string
namespace:
description: 'Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
type: string
resourceVersion:
description: 'Specific resourceVersion to which this reference is
made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
type: string
uid:
description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
type: string
type: object
repoIdentifier:
description: RepoIdentifier is the restic repository identifier.
type: string
snapshotID:
description: SnapshotID is the ID of the volume snapshot to be restored.
type: string
volume:
description: Volume is the name of the volume within the Pod to be restored.
type: string
required:
- backupStorageLocation
- pod
- repoIdentifier
- snapshotID
- volume
type: object
status:
description: PodVolumeRestoreStatus is the current status of a PodVolumeRestore.
properties:
completionTimestamp:
description: CompletionTimestamp records the time a restore was completed.
Completion time is recorded even on failed restores. The server's
time is used for CompletionTimestamps
format: date-time
nullable: true
type: string
message:
description: Message is a message about the pod volume restore's status.
type: string
phase:
description: Phase is the current state of the PodVolumeRestore.
enum:
- New
- InProgress
- Completed
- Failed
type: string
progress:
description: Progress holds the total number of bytes of the snapshot
and the current number of restored bytes. This can be used to display
progress information about the restore operation.
properties:
bytesDone:
format: int64
type: integer
totalBytes:
format: int64
type: integer
type: object
startTimestamp:
description: StartTimestamp records the time a restore was started.
The server's time is used for StartTimestamps
format: date-time
nullable: true
type: string
type: object
type: object
version: v1
versions:
- name: v1
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

View File

@@ -0,0 +1,89 @@
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.3.0
creationTimestamp: null
name: resticrepositories.velero.io
spec:
group: velero.io
names:
kind: ResticRepository
listKind: ResticRepositoryList
plural: resticrepositories
singular: resticrepository
preserveUnknownFields: false
scope: Namespaced
validation:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: ResticRepositorySpec is the specification for a ResticRepository.
properties:
backupStorageLocation:
description: BackupStorageLocation is the name of the BackupStorageLocation
that should contain this repository.
type: string
maintenanceFrequency:
description: MaintenanceFrequency is how often maintenance should be
run.
type: string
resticIdentifier:
description: ResticIdentifier is the full restic-compatible string for
identifying this repository.
type: string
volumeNamespace:
description: VolumeNamespace is the namespace this restic repository
contains pod volume backups for.
type: string
required:
- backupStorageLocation
- maintenanceFrequency
- resticIdentifier
- volumeNamespace
type: object
status:
description: ResticRepositoryStatus is the current status of a ResticRepository.
properties:
lastMaintenanceTime:
description: LastMaintenanceTime is the last time maintenance was run.
format: date-time
nullable: true
type: string
message:
description: Message is a message about the current status of the ResticRepository.
type: string
phase:
description: Phase is the current state of the ResticRepository.
enum:
- New
- Ready
- NotReady
type: string
type: object
type: object
version: v1
versions:
- name: v1
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,394 @@
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.3.0
creationTimestamp: null
name: schedules.velero.io
spec:
group: velero.io
names:
kind: Schedule
listKind: ScheduleList
plural: schedules
singular: schedule
preserveUnknownFields: false
scope: Namespaced
validation:
openAPIV3Schema:
description: Schedule is a Velero resource that represents a pre-scheduled or
periodic Backup that should be run.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: ScheduleSpec defines the specification for a Velero schedule
properties:
schedule:
description: Schedule is a Cron expression defining when to run the
Backup.
type: string
template:
description: Template is the definition of the Backup to be run on the
provided schedule
properties:
defaultVolumesToRestic:
description: DefaultVolumesToRestic specifies whether restic should
be used to take a backup of all pod volumes by default.
type: boolean
excludedNamespaces:
description: ExcludedNamespaces contains a list of namespaces that
are not included in the backup.
items:
type: string
nullable: true
type: array
excludedResources:
description: ExcludedResources is a slice of resource names that
are not included in the backup.
items:
type: string
nullable: true
type: array
hooks:
description: Hooks represent custom behaviors that should be executed
at different phases of the backup.
properties:
resources:
description: Resources are hooks that should be executed when
backing up individual instances of a resource.
items:
description: BackupResourceHookSpec defines one or more BackupResourceHooks
that should be executed based on the rules defined for namespaces,
resources, and label selector.
properties:
excludedNamespaces:
description: ExcludedNamespaces specifies the namespaces
to which this hook spec does not apply.
items:
type: string
nullable: true
type: array
excludedResources:
description: ExcludedResources specifies the resources
to which this hook spec does not apply.
items:
type: string
nullable: true
type: array
includedNamespaces:
description: IncludedNamespaces specifies the namespaces
to which this hook spec applies. If empty, it applies
to all namespaces.
items:
type: string
nullable: true
type: array
includedResources:
description: IncludedResources specifies the resources
to which this hook spec applies. If empty, it applies
to all resources.
items:
type: string
nullable: true
type: array
labelSelector:
description: LabelSelector, if specified, filters the
resources to which this hook spec applies.
nullable: true
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: A label selector requirement is a selector
that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: operator represents a key's relationship
to a set of values. Valid operators are In,
NotIn, Exists and DoesNotExist.
type: string
values:
description: values is an array of string values.
If the operator is In or NotIn, the values
array must be non-empty. If the operator is
Exists or DoesNotExist, the values array must
be empty. This array is replaced during a
strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs.
A single {key,value} in the matchLabels map is equivalent
to an element of matchExpressions, whose key field
is "key", the operator is "In", and the values array
contains only "value". The requirements are ANDed.
type: object
type: object
name:
description: Name is the name of this hook.
type: string
post:
description: PostHooks is a list of BackupResourceHooks
to execute after storing the item in the backup. These
are executed after all "additional items" from item
actions are processed.
items:
description: BackupResourceHook defines a hook for a
resource.
properties:
exec:
description: Exec defines an exec hook.
properties:
command:
description: Command is the command and arguments
to execute.
items:
type: string
minItems: 1
type: array
container:
description: Container is the container in the
pod where the command should be executed.
If not specified, the pod's first container
is used.
type: string
onError:
description: OnError specifies how Velero should
behave if it encounters an error executing
this hook.
enum:
- Continue
- Fail
type: string
timeout:
description: Timeout defines the maximum amount
of time Velero should wait for the hook to
complete before considering the execution
a failure.
type: string
required:
- command
type: object
required:
- exec
type: object
type: array
pre:
description: PreHooks is a list of BackupResourceHooks
to execute prior to storing the item in the backup.
These are executed before any "additional items" from
item actions are processed.
items:
description: BackupResourceHook defines a hook for a
resource.
properties:
exec:
description: Exec defines an exec hook.
properties:
command:
description: Command is the command and arguments
to execute.
items:
type: string
minItems: 1
type: array
container:
description: Container is the container in the
pod where the command should be executed.
If not specified, the pod's first container
is used.
type: string
onError:
description: OnError specifies how Velero should
behave if it encounters an error executing
this hook.
enum:
- Continue
- Fail
type: string
timeout:
description: Timeout defines the maximum amount
of time Velero should wait for the hook to
complete before considering the execution
a failure.
type: string
required:
- command
type: object
required:
- exec
type: object
type: array
required:
- name
type: object
nullable: true
type: array
type: object
includeClusterResources:
description: IncludeClusterResources specifies whether cluster-scoped
resources should be included for consideration in the backup.
nullable: true
type: boolean
includedNamespaces:
description: IncludedNamespaces is a slice of namespace names to
include objects from. If empty, all namespaces are included.
items:
type: string
nullable: true
type: array
includedResources:
description: IncludedResources is a slice of resource names to include
in the backup. If empty, all resources are included.
items:
type: string
nullable: true
type: array
labelSelector:
description: LabelSelector is a metav1.LabelSelector to filter with
when adding individual objects to the backup. If empty or nil,
all objects are included. Optional.
nullable: true
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements.
The requirements are ANDed.
items:
description: A label selector requirement is a selector that
contains values, a key, and an operator that relates the
key and values.
properties:
key:
description: key is the label key that the selector applies
to.
type: string
operator:
description: operator represents a key's relationship
to a set of values. Valid operators are In, NotIn, Exists
and DoesNotExist.
type: string
values:
description: values is an array of string values. If the
operator is In or NotIn, the values array must be non-empty.
If the operator is Exists or DoesNotExist, the values
array must be empty. This array is replaced during a
strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: matchLabels is a map of {key,value} pairs. A single
{key,value} in the matchLabels map is equivalent to an element
of matchExpressions, whose key field is "key", the operator
is "In", and the values array contains only "value". The requirements
are ANDed.
type: object
type: object
orderedResources:
additionalProperties:
type: string
description: OrderedResources specifies the backup order of resources
of specific Kind. The map key is the Kind name and value is a
list of resource names separated by commas. Each resource name
has format "namespace/resourcename". For cluster resources, simply
use "resourcename".
nullable: true
type: object
snapshotVolumes:
description: SnapshotVolumes specifies whether to take cloud snapshots
of any PV's referenced in the set of objects included in the Backup.
nullable: true
type: boolean
storageLocation:
description: StorageLocation is a string containing the name of
a BackupStorageLocation where the backup should be stored.
type: string
ttl:
description: TTL is a time.Duration-parseable string describing
how long the Backup should be retained for.
type: string
volumeSnapshotLocations:
description: VolumeSnapshotLocations is a list containing names
of VolumeSnapshotLocations associated with this backup.
items:
type: string
type: array
type: object
useOwnerReferencesInBackup:
description: UseOwnerReferencesBackup specifies whether to use OwnerReferences
on backups created by this Schedule.
nullable: true
type: boolean
required:
- schedule
- template
type: object
status:
description: ScheduleStatus captures the current state of a Velero schedule
properties:
lastBackup:
description: LastBackup is the last time a Backup was run for this Schedule
schedule
format: date-time
nullable: true
type: string
phase:
description: Phase is the current phase of the Schedule
enum:
- New
- Enabled
- FailedValidation
type: string
validationErrors:
description: ValidationErrors is a slice of all validation errors (if
applicable)
items:
type: string
type: array
type: object
type: object
version: v1
versions:
- name: v1
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

View File

@@ -0,0 +1,89 @@
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.3.0
creationTimestamp: null
name: serverstatusrequests.velero.io
spec:
group: velero.io
names:
kind: ServerStatusRequest
listKind: ServerStatusRequestList
plural: serverstatusrequests
shortNames:
- ssr
singular: serverstatusrequest
preserveUnknownFields: false
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
description: ServerStatusRequest is a request to access current status information
about the Velero server.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: ServerStatusRequestSpec is the specification for a ServerStatusRequest.
type: object
status:
description: ServerStatusRequestStatus is the current status of a ServerStatusRequest.
properties:
phase:
description: Phase is the current lifecycle phase of the ServerStatusRequest.
enum:
- New
- Processed
type: string
plugins:
description: Plugins list information about the plugins running on the
Velero server
items:
description: PluginInfo contains attributes of a Velero plugin
properties:
kind:
type: string
name:
type: string
required:
- kind
- name
type: object
nullable: true
type: array
processedTimestamp:
description: ProcessedTimestamp is when the ServerStatusRequest was
processed by the ServerStatusRequestController.
format: date-time
nullable: true
type: string
serverVersion:
description: ServerVersion is the Velero server version.
type: string
type: object
type: object
version: v1
versions:
- name: v1
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

View File

@@ -0,0 +1,74 @@
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.3.0
creationTimestamp: null
name: volumesnapshotlocations.velero.io
spec:
group: velero.io
names:
kind: VolumeSnapshotLocation
listKind: VolumeSnapshotLocationList
plural: volumesnapshotlocations
singular: volumesnapshotlocation
preserveUnknownFields: false
scope: Namespaced
validation:
openAPIV3Schema:
description: VolumeSnapshotLocation is a location where Velero stores volume
snapshots.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: VolumeSnapshotLocationSpec defines the specification for a
Velero VolumeSnapshotLocation.
properties:
config:
additionalProperties:
type: string
description: Config is for provider-specific configuration fields.
type: object
provider:
description: Provider is the provider of the volume storage.
type: string
required:
- provider
type: object
status:
description: VolumeSnapshotLocationStatus describes the current status of
a Velero VolumeSnapshotLocation.
properties:
phase:
description: VolumeSnapshotLocationPhase is the lifecycle phase of a
Velero VolumeSnapshotLocation.
enum:
- Available
- Unavailable
type: string
type: object
type: object
version: v1
versions:
- name: v1
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []

File diff suppressed because one or more lines are too long

View File

@@ -1,4 +1,4 @@
// Package crds embeds the controller-tools generated CRD manifests
package crds
//go:generate go run ../../../../hack/crd-gen/v1/main.go
//go:generate go run ../../../../hack/crd-gen/v1beta1/main.go

View File

@@ -1,176 +0,0 @@
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.12.0
name: datadownloads.velero.io
spec:
group: velero.io
names:
kind: DataDownload
listKind: DataDownloadList
plural: datadownloads
singular: datadownload
scope: Namespaced
versions:
- additionalPrinterColumns:
- description: DataDownload status such as New/InProgress
jsonPath: .status.phase
name: Status
type: string
- description: Time duration since this DataDownload was started
jsonPath: .status.startTimestamp
name: Started
type: date
- description: Completed bytes
format: int64
jsonPath: .status.progress.bytesDone
name: Bytes Done
type: integer
- description: Total bytes
format: int64
jsonPath: .status.progress.totalBytes
name: Total Bytes
type: integer
- description: Name of the Backup Storage Location where the backup data is stored
jsonPath: .spec.backupStorageLocation
name: Storage Location
type: string
- description: Time duration since this DataDownload was created
jsonPath: .metadata.creationTimestamp
name: Age
type: date
- description: Name of the node where the DataDownload is processed
jsonPath: .status.node
name: Node
type: string
name: v2alpha1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: DataDownloadSpec is the specification for a DataDownload.
properties:
backupStorageLocation:
description: BackupStorageLocation is the name of the backup storage
location where the backup repository is stored.
type: string
cancel:
description: Cancel indicates request to cancel the ongoing DataDownload.
It can be set when the DataDownload is in InProgress phase
type: boolean
dataMoverConfig:
additionalProperties:
type: string
description: DataMoverConfig is for data-mover-specific configuration
fields.
type: object
datamover:
description: DataMover specifies the data mover to be used by the
backup. If DataMover is "" or "velero", the built-in data mover
will be used.
type: string
operationTimeout:
description: OperationTimeout specifies the time used to wait internal
operations, before returning error as timeout.
type: string
snapshotID:
description: SnapshotID is the ID of the Velero backup snapshot to
be restored from.
type: string
sourceNamespace:
description: SourceNamespace is the original namespace where the volume
is backed up from. It may be different from SourcePVC's namespace
if namespace is remapped during restore.
type: string
targetVolume:
description: TargetVolume is the information of the target PVC and
PV.
properties:
namespace:
description: Namespace is the target namespace
type: string
pv:
description: PV is the name of the target PV that is created by
Velero restore
type: string
pvc:
description: PVC is the name of the target PVC that is created
by Velero restore
type: string
required:
- namespace
- pv
- pvc
type: object
required:
- backupStorageLocation
- operationTimeout
- snapshotID
- sourceNamespace
- targetVolume
type: object
status:
description: DataDownloadStatus is the current status of a DataDownload.
properties:
completionTimestamp:
description: CompletionTimestamp records the time a restore was completed.
Completion time is recorded even on failed restores. The server's
time is used for CompletionTimestamps
format: date-time
nullable: true
type: string
message:
description: Message is a message about the DataDownload's status.
type: string
node:
description: Node is name of the node where the DataDownload is processed.
type: string
phase:
description: Phase is the current state of the DataDownload.
enum:
- New
- Accepted
- Prepared
- InProgress
- Canceling
- Canceled
- Completed
- Failed
type: string
progress:
description: Progress holds the total number of bytes of the snapshot
and the current number of restored bytes. This can be used to display
progress information about the restore operation.
properties:
bytesDone:
format: int64
type: integer
totalBytes:
format: int64
type: integer
type: object
startTimestamp:
description: StartTimestamp records the time a restore was started.
The server's time is used for StartTimestamps
format: date-time
nullable: true
type: string
type: object
type: object
served: true
storage: true
subresources: {}

View File

@@ -1,200 +0,0 @@
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.12.0
name: datauploads.velero.io
spec:
group: velero.io
names:
kind: DataUpload
listKind: DataUploadList
plural: datauploads
singular: dataupload
scope: Namespaced
versions:
- additionalPrinterColumns:
- description: DataUpload status such as New/InProgress
jsonPath: .status.phase
name: Status
type: string
- description: Time duration since this DataUpload was started
jsonPath: .status.startTimestamp
name: Started
type: date
- description: Completed bytes
format: int64
jsonPath: .status.progress.bytesDone
name: Bytes Done
type: integer
- description: Total bytes
format: int64
jsonPath: .status.progress.totalBytes
name: Total Bytes
type: integer
- description: Name of the Backup Storage Location where this backup should be
stored
jsonPath: .spec.backupStorageLocation
name: Storage Location
type: string
- description: Time duration since this DataUpload was created
jsonPath: .metadata.creationTimestamp
name: Age
type: date
- description: Name of the node where the DataUpload is processed
jsonPath: .status.node
name: Node
type: string
name: v2alpha1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: DataUploadSpec is the specification for a DataUpload.
properties:
backupStorageLocation:
description: BackupStorageLocation is the name of the backup storage
location where the backup repository is stored.
type: string
cancel:
description: Cancel indicates request to cancel the ongoing DataUpload.
It can be set when the DataUpload is in InProgress phase
type: boolean
csiSnapshot:
description: If SnapshotType is CSI, CSISnapshot provides the information
of the CSI snapshot.
nullable: true
properties:
snapshotClass:
description: SnapshotClass is the name of the snapshot class that
the volume snapshot is created with
type: string
storageClass:
description: StorageClass is the name of the storage class of
the PVC that the volume snapshot is created from
type: string
volumeSnapshot:
description: VolumeSnapshot is the name of the volume snapshot
to be backed up
type: string
required:
- storageClass
- volumeSnapshot
type: object
dataMoverConfig:
additionalProperties:
type: string
description: DataMoverConfig is for data-mover-specific configuration
fields.
nullable: true
type: object
datamover:
description: DataMover specifies the data mover to be used by the
backup. If DataMover is "" or "velero", the built-in data mover
will be used.
type: string
operationTimeout:
description: OperationTimeout specifies the time used to wait internal
operations, before returning error as timeout.
type: string
snapshotType:
description: SnapshotType is the type of the snapshot to be backed
up.
type: string
sourceNamespace:
description: SourceNamespace is the original namespace where the volume
is backed up from. It is the same namespace for SourcePVC and CSI
namespaced objects.
type: string
sourcePVC:
description: SourcePVC is the name of the PVC which the snapshot is
taken for.
type: string
required:
- backupStorageLocation
- operationTimeout
- snapshotType
- sourceNamespace
- sourcePVC
type: object
status:
description: DataUploadStatus is the current status of a DataUpload.
properties:
completionTimestamp:
description: CompletionTimestamp records the time a backup was completed.
Completion time is recorded even on failed backups. Completion time
is recorded before uploading the backup object. The server's time
is used for CompletionTimestamps
format: date-time
nullable: true
type: string
dataMoverResult:
additionalProperties:
type: string
description: DataMoverResult stores data-mover-specific information
as a result of the DataUpload.
nullable: true
type: object
message:
description: Message is a message about the DataUpload's status.
type: string
node:
description: Node is name of the node where the DataUpload is processed.
type: string
path:
description: Path is the full path of the snapshot volume being backed
up.
type: string
phase:
description: Phase is the current state of the DataUpload.
enum:
- New
- Accepted
- Prepared
- InProgress
- Canceling
- Canceled
- Completed
- Failed
type: string
progress:
description: Progress holds the total number of bytes of the volume
and the current number of backed up bytes. This can be used to display
progress information about the backup operation.
properties:
bytesDone:
format: int64
type: integer
totalBytes:
format: int64
type: integer
type: object
snapshotID:
description: SnapshotID is the identifier for the snapshot in the
backup repository.
type: string
startTimestamp:
description: StartTimestamp records the time a backup was started.
Separate from CreationTimestamp, since that value changes on restores.
The server's time is used for StartTimestamps
format: date-time
nullable: true
type: string
type: object
type: object
served: true
storage: true
subresources: {}

File diff suppressed because one or more lines are too long

View File

@@ -1,67 +1,11 @@
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: velero-perms
creationTimestamp: null
name: manager-role
rules:
- apiGroups:
- ""
resources:
- persistentvolumerclaims
verbs:
- get
- apiGroups:
- ""
resources:
- persistentvolumes
verbs:
- get
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- apiGroups:
- velero.io
resources:
- backuprepositories
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- velero.io
resources:
- backuprepositories/status
verbs:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
- backups
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- velero.io
resources:
- backups/status
verbs:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
@@ -82,66 +26,6 @@ rules:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
- datadownloads
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- velero.io
resources:
- datadownloads/status
verbs:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
- datauploads
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- velero.io
resources:
- datauploads/status
verbs:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
- deletebackuprequests
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- velero.io
resources:
- deletebackuprequests/status
verbs:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
@@ -162,86 +46,6 @@ rules:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
- podvolumebackups
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- velero.io
resources:
- podvolumebackups/status
verbs:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
- podvolumerestores
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- velero.io
resources:
- podvolumerestores/status
verbs:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
- restores
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- velero.io
resources:
- restores/status
verbs:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
- schedules
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- velero.io
resources:
- schedules/status
verbs:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
@@ -262,15 +66,3 @@ rules:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
- volumesnapshotlocations
verbs:
- create
- delete
- get
- list
- patch
- update
- watch

View File

@@ -1,40 +0,0 @@
# Delete Backup and Restic Repo Resources when BSL is Deleted
## Abstract
Issue #2082 requested that with the command `velero backup-location delete <bsl name>` (implemented in Velero 1.6 with #3073), the following will be deleted:
- associated Velero backups (to be clear, these are custom Kubernetes resources called "backups" that are stored in the API server)
- associated Restic repositories (custom Kubernetes resources called "resticrepositories")
This design doc explains how the request will be implemented.
## Background
When a BSL resource is deleted from its Velero namespace, the associated custom Kubernetes resources, backups and Restic repositories, can no longer be used.
It makes sense to clean those resources up when a BSL is deleted.
## Goals
Update the `velero backup-location delete <bsl name>` command to delete associated backup and Restic repository resources in the same Velero namespace.
## Non Goals
[It was suggested](https://github.com/vmware-tanzu/velero/issues/2082#issuecomment-827951311) to fix bug #2697 alongside this issue.
However, I think that should be fixed separately because although it is similar (restore objects are not being deleted), it is also quite different.
One is adding a command feature update (this issue) and the other is a bug fix and each affect different parts of the code base.
## High-Level Design
Update the `velero backup-location delete <bsl name>` command to do the following:
- find in the same Velero namespace from which the BSL was deleted the associated backup resources and Restic repositories, called "backups.velero.io" and "resticrepositories.velero.io" respectively
- delete the resources found
The above logic will be added to [where BSLs are deleted](https://github.com/vmware-tanzu/velero/blob/main/pkg/cmd/cli/backuplocation/delete.go).
## Alternative Considered
I had considered deleting the backup files (the ones in json format and tarballs) in the BSL itself.
However, a standard use case is to back up a cluster and then restore into a new cluster.
Deleting the backup storage location in either location is not expected to remove all of the backups in the backup storage location and should not be done.

View File

@@ -505,11 +505,9 @@ spec:
- BackupResourceList
- RestoreLog
- RestoreResults
- CSIBackupVolumeSnapshots
- CSIBackupVolumeSnapshotContents
type: string
name:
description: Name is the name of the Kubernetes resource with
description: Name is the name of the kubernetes resource with
which the file is associated.
type: string
required:

View File

@@ -57,7 +57,7 @@ spec:
- emptyDir: {}
name: scratch
---
apiVersion: rbac.authorization.k8s.io/v1
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
labels:

View File

@@ -5,22 +5,22 @@ metadata:
creationTimestamp: null
labels:
component: velero
name: node-agent
name: restic
namespace: velero
spec:
selector:
matchLabels:
name: node-agent
name: restic
template:
metadata:
creationTimestamp: null
labels:
component: velero
name: node-agent
name: restic
spec:
containers:
- args:
- node-agent
- restic
- server
command:
- /velero
@@ -43,7 +43,7 @@ spec:
value: /credentials/cloud
image: velero/velero:latest
imagePullPolicy: Always
name: node-agent
name: restic
resources: {}
volumeMounts:
- mountPath: /host_pods

Binary file not shown.

Before

Width:  |  Height:  |  Size: 75 KiB

View File

@@ -2,17 +2,17 @@
This document proposes a solution that allows user to specify a backup order for resources of specific resource type.
## Background
During backup process, user may need to back up resources of specific type in some specific order to ensure the resources were backup properly because these resources are related and ordering might be required to preserve the consistency for the apps to recover itself from the backup image
During backup process, user may need to back up resources of specific type in some specific order to ensure the resources were backup properly because these resources are related and ordering might be required to preserve the consistency for the apps to recover itself <EFBFBD>from the backup image
(Ex: primary-secondary database pods in a cluster).
## Goals
- Enable user to specify an order of backup resources belong to specific resource type
- Enable user to specify an order of back up resources belong to specific resource type
## Alternatives Considered
- Use a plugin to backup an resources and all the sub resources. For example use a plugin for StatefulSet and backup pods belong to the StatefulSet in specific order. This plugin solution is not generic and requires plugin for each resource type.
## High-Level Design
User will specify a map of resource type to list resource names (separate by semicolons). Each name will be in the format "namespaceName/resourceName" to enable ordering across namespaces. Based on this map, the resources of each resource type will be sorted by the order specified in the list of resources. If a resource instance belong to that specific type but its name is not in the order list, then it will be put behind other resources that are in the list.
User will specify a map of resource type to list resource names (separate by semicolons). Each name will be in the format "namespaceName/resourceName" to enable ordering accross namespaces. Based on this map, the resources of each resource type will be sorted by the order specified in the list of resources. If a resource instance belong to that specific type but its name is not in the order list, then it will be put behind other resources that are in the list.
### Changes to BackupSpec
Add new field to BackupSpec
@@ -36,5 +36,5 @@ Example:
>velero backup create mybackup --ordered-resources "pod=ns1/pod1,ns1/pod2;persistentvolumeclaim=n2/slavepod,ns2/primarypod"
## Open Issues
- In the CLI, the design proposes to use commas to separate items of a resource type and semicolon to separate key-value pairs. This follows the convention of using commas to separate items in a list (For example: --include-namespaces ns1,ns2). However, the syntax for map in labels and annotations use commas to separate key-value pairs. So it introduces some inconsistency.
- In the CLI, the design proposes to use commas to separate items of a resource type and semicolon to separate key-value pairs. This follows the convention of using commas to separate items in a list (For example: --include-namespaces ns1,ns2). However, the syntax for map in labels and annotations use commas to seperate key-value pairs. So it introduces some inconsistency.
- For pods that managed by Deployment or DaemonSet, this design may not work because the pods' name is randomly generated and if pods are restarted, they would have different names so the Backup operation may not consider the restarted pods in the sorting algorithm. This problem will be addressed when we enhance the design to use regular expression to specify the OrderResources instead of exact match.

View File

@@ -1,103 +0,0 @@
# Design for BackupItemAction v2 API
## Abstract
This design includes the changes to the BackupItemAction (BIA) api design as required by the [Item Action Progress Monitoring](general-progress-monitoring.md) feature.
The BIA v2 interface will have two new methods, and the Execute() return signature will be modified.
If there are any additional BIA API changes that are needed in the same Velero release cycle as this change, those can be added here as well.
## Background
This API change is needed to facilitate long-running plugin actions that may not be complete when the Execute() method returns.
It is an optional feature, so plugins which don't need this feature can simply return an empty operation ID and the new methods can be no-ops.
This will allow long-running plugin actions to continue in the background while Velero moves on to the next plugin, the next item, etc.
## Goals
- Allow for BIA Execute() to optionally initiate a long-running operation and report on operation status.
## Non Goals
- Allowing velero control over when the long-running operation begins.
## High-Level Design
As per the [Plugin Versioning](plugin-versioning.md) design, a new BIAv2 plugin `.proto` file will be created to define the GRPC interface.
v2 go files will also be created in `plugin/clientmgmt/backupitemaction` and `plugin/framework/backupitemaction`, and a new PluginKind will be created.
The velero Backup process will be modified to reference v2 plugins instead of v1 plugins.
An adapter will be created so that any existing BIA v1 plugin can be executed as a v2 plugin when executing a backup.
## Detailed Design
### proto changes (compiled into golang by protoc)
The v2 BackupItemAction.proto will be like the current v1 version with the following changes:
ExecuteResponse gets a new field:
```
message ExecuteResponse {
bytes item = 1;
repeated generated.ResourceIdentifier additionalItems = 2;
string operationID = 3;
repeated generated.ResourceIdentifier itemsToUpdate = 4;
}
```
The BackupItemAction service gets two new rpc methods:
```
service BackupItemAction {
rpc AppliesTo(BackupItemActionAppliesToRequest) returns (BackupItemActionAppliesToResponse);
rpc Execute(ExecuteRequest) returns (ExecuteResponse);
rpc Progress(BackupItemActionProgressRequest) returns (BackupItemActionProgressResponse);
rpc Cancel(BackupItemActionCancelRequest) returns (google.protobuf.Empty);
}
```
To support these new rpc methods, we define new request/response message types:
```
message BackupItemActionProgressRequest {
string plugin = 1;
string operationID = 2;
bytes backup = 3;
}
message BackupItemActionProgressResponse {
generated.OperationProgress progress = 1;
}
message BackupItemActionCancelRequest {
string plugin = 1;
string operationID = 2;
bytes backup = 3;
}
```
One new shared message type will be added, as this will also be needed for v2 RestoreItemAction and VolmeSnapshotter:
```
message OperationProgress {
bool completed = 1;
string err = 2;
int64 nCompleted = 3;
int64 nTotal = 4;
string operationUnits = 5;
string description = 6;
google.protobuf.Timestamp started = 7;
google.protobuf.Timestamp updated = 8;
}
```
In addition to the two new rpc methods added to the BackupItemAction interface, there is also a new `Name()` method. This one is only actually used internally by Velero to get the name that the plugin was registered with, but it still must be defined in a plugin which implements BackupItemActionV2 in order to implement the interface. It doesn't really matter what it returns, though, as this particular method is not delegated to the plugin via RPC calls. The new (and modified) interface methods for `BackupItemAction` are as follows:
```
type BackupItemAction interface {
...
Name() string
...
Execute(item runtime.Unstructured, backup *api.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error)
Progress(operationID string, backup *api.Backup) (velero.OperationProgress, error)
Cancel(operationID string, backup *api.Backup) error
...
}
```
A new PluginKind, `BackupItemActionV2`, will be created, and the backup process will be modified to use this plugin kind.
See [Plugin Versioning](plugin-versioning.md) for more details on implementation plans, including v1 adapters, etc.
## Compatibility
The included v1 adapter will allow any existing BackupItemAction plugin to work as expected, with an empty operation ID returned from Execute() and no-op Progress() and Cancel() methods.
## Implementation
This will be implemented during the Velero 1.11 development cycle.

View File

@@ -1,402 +0,0 @@
# Proposal to add resource filters for backup can distinguish whether resource is cluster-scoped or namespace-scoped.
- [Proposal to add resource filters for backup can distinguish whether resource is cluster-scoped or namespace-scoped.](#proposal-to-add-resource-filters-for-backup-can-distinguish-whether-resource-is-cluster-scoped-or-namespace-scoped)
- [Abstract](#abstract)
- [Background](#background)
- [Goals](#goals)
- [Non Goals](#non-goals)
- [High-Level Design](#high-level-design)
- [Parameters Rules](#parameters-rules)
- [Using scenarios:](#using-scenarios)
- [no namespace-scoped resources + some cluster-scoped resources](#no-namespace-scoped-resources--some-cluster-scoped-resources)
- [no namespace-scoped resources + all cluster-scoped resources](#no-namespace-scoped-resources--all-cluster-scoped-resources)
- [some namespace-scoped resources + no cluster-scoped resources](#some-namespace-scoped-resources--no-cluster-scoped-resources)
- [scenario 1](#scenario-1)
- [scenario 2](#scenario-2)
- [scenario 3](#scenario-3)
- [scenario 4](#scenario-4)
- [some namespace-scoped resources + only related cluster-scoped resources](#some-namespace-scoped-resources--only-related-cluster-scoped-resources)
- [scenario 1](#scenario-1-1)
- [scenario 2](#scenario-2-1)
- [scenario 3](#scenario-3-1)
- [some namespace-scoped resources + some additional cluster-scoped resources](#some-namespace-scoped-resources--some-additional-cluster-scoped-resources)
- [scenario 1](#scenario-1-2)
- [scenario 2](#scenario-2-2)
- [scenario 3](#scenario-3-2)
- [scenario 4](#scenario-4-1)
- [some namespace-scoped resources + all cluster-scoped resources](#some-namespace-scoped-resources--all-cluster-scoped-resources)
- [scenario 1](#scenario-1-3)
- [scenario 2](#scenario-2-3)
- [scenario 3](#scenario-3-3)
- [all namespace-scoped resources + no cluster-scoped resources](#all-namespace-scoped-resources--no-cluster-scoped-resources)
- [all namespace-scoped resources + some additional cluster-scoped resources](#all-namespace-scoped-resources--some-additional-cluster-scoped-resources)
- [all namespace-scoped resources + all cluster-scoped resources](#all-namespace-scoped-resources--all-cluster-scoped-resources)
- [describe command change](#describe-command-change)
- [Detailed Design](#detailed-design)
- [Alternatives Considered](#alternatives-considered)
- [Security Considerations](#security-considerations)
- [Compatibility](#compatibility)
- [Implementation](#implementation)
- [Open Issues](#open-issues)
## Abstract
The current filter (IncludedResources/ExcludedResources + IncludeClusterResources flag) is not enough for some special cases, e.g. all namespace-scoped resources + some kind of cluster-scoped resource and all namespace-scoped resources + cluster-scoped resource excludes.
Propose to add a new group of resource filtering parameters, which can distinguish cluster-scoped and namespace-scoped resources.
## Background
There are two sets of resource filters for Velero: `IncludedNamespaces/ExcludedNamespaces` and `IncludedResources/ExcludedResources`.
`IncludedResources` means only including the resource types specified in the parameter. Both cluster-scoped and namespace-scoped resources are handled in this parameter by now.
The k8s resources are separated into cluster-scoped and namespace-scoped.
As a result, it's hard to include all resources in one group and only including specified resource in the other group.
## Goals
- Make Velero can support more complicated namespace-scoped and cluster-scoped resources filtering scenarios in backup.
## Non Goals
- Enrich the resource filtering rules, for example, advanced PV filtering and filtering by resource names.
## High-Level Design
Four new parameters are added into command `velero backup create`: `--include-cluster-scoped-resources`, `--exclude-cluster-scoped-resources`, `--include-namespace-scoped-resources` and `--exclude-namespace-scoped-resources`.
`--include-cluster-scoped-resources` and `--exclude-cluster-scoped-resources` are used to filter cluster-scoped resources included or excluded in backup per resource type.
`--include-namespace-scoped-resources` and `--exclude-namespace-scoped-resources` are used to filter namespace-scoped resources included or excluded in backup per resource type.
Restore and other code pieces also use resource filtering will be handled in future releases.
### Parameters Rules
* `--include-cluster-scoped-resources`, `--include-namespace-scoped-resources`, `--exclude-cluster-scoped-resources` and `--exclude-namespace-scoped-resources` valid value include `*` and comma separated string. Each element of the CSV string should a k8s resource name. The format should be `resource.group`, such as `storageclasses.storage.k8s.io.`.
* `--include-cluster-scoped-resources`, `--include-namespace-scoped-resources`, `--exclude-cluster-scoped-resources` and `--exclude-namespace-scoped-resources` parameters are mutual exclusive with `--include-cluster-resources`, `--include-resources` and `--exclude-resources` parameters. If both sets of parameters are provisioned, validation failure should be returned.
* `--include-cluster-scoped-resources` and `--exclude-cluster-scoped-resources` should only contain cluster-scoped resource type names. If namespace-scoped resource type names are included, they are ignored.
* If there are conflicts between `--include-cluster-scoped-resources` and `--exclude-cluster-scoped-resources` specified resources type lists, `--exclude-cluster-scoped-resources` parameter has higher priority.
* `--include-namespace-scoped-resources` and `--exclude-namespace-scoped-resources` should only contain namespace-scoped resource type names. If cluster-scoped resource type names are included, they are ignored.
* If there are conflicts between `--include-namespace-scoped-resources` and `--exclude-namespace-scoped-resources` specified resources type lists, `--exclude-namespace-scoped-resources` parameter has higher priority.
* If `--include-namespace-scoped-resources` is not present, it means all namespace-scoped resources are included per resource type.
* If both `--include-cluster-scoped-resources` and `--exclude-cluster-scoped-resources` are not present, it means no additional cluster-scoped resource is included per resource type, just as the existing `--include-cluster-resources` parameter not setting value. Cluster-scoped resources are related to the namespace-scoped resources, which means those are returned in the namespace-scoped resources' BackupItemAction's result AdditionalItems array, are still included in backup by default. Taking backing up PVC scenario as an example, PVC is namespace-scoped, PV is cluster-scoped. PVC's BIA will include PVC related PV into backup too.
### Using scenarios:
Please notice, if the scenario give the example of using old filtering parameters (`--include-cluster-resources`, `--include-resources` and `--exclude-resources`), that means the old parameters also work for this case. If old parameters example is not given, that means they don't work for this scenario, only new parameters (`--include-cluster-scoped-resources`, `--include-namespace-scoped-resources`, `--exclude-cluster-scoped-resources` and `--exclude-namespace-scoped-resources`) work.
#### no namespace-scoped resources + some cluster-scoped resources
The following command means backup no namespace-scoped resources and some cluster-scoped resources.
``` bash
velero backup create <backup-name>
--exclude-namespace-scoped-resources=*
--include-cluster-scoped-resources=storageclass
```
#### no namespace-scoped resources + all cluster-scoped resources
The following command means backup no namespace-scoped resources and all cluster-scoped resources.
``` bash
velero backup create <backup-name>
--exclude-namespace-scoped-resources=*
--include-cluster-scoped-resources=*
```
#### some namespace-scoped resources + no cluster-scoped resources
##### scenario 1
The following commands mean backup all resources in namespaces default and kube-system, and no cluster-scoped resources.
Example of new parameters:
``` bash
velero backup create <backup-name>
--include-namespaces=default,kube-system
--exclude-cluster-scoped-resources=*
```
Example of old parameters:
``` bash
velero backup create <backup-name>
--include-namespaces=default,kube-system
--include-cluster-resources=false
```
##### scenario 2
The following commands mean backup PVC, Deployment, Service, Endpoint, Pod and ReplicaSet resources in all namespaces, and no cluster-scoped resources. Although PVC's related PV should be included, due to no cluster-scoped resources are included, so they are ruled out too.
Example of new parameters:
``` bash
velero backup create <backup-name>
--include-namespace-scoped-resources=persistentvolumeclaim,deployment,service,endpoint,pod,replicaset
--exclude-cluster-scope-resources=*
```
Example of old parameters:
``` bash
velero backup create <backup-name>
--include-resources=persistentvolumeclaim,deployment,service,endpoint,pod,replicaset
--include-cluster-resources=false
```
##### scenario 3
The following commands mean backup PVC, Deployment, Service, Endpoint, Pod and ReplicaSet resources in namespace default and kube-system, and no cluster-scoped resources. Although PVC's related PV should be included, due to no cluster-scoped resources are included, so they are ruled out too.
Example of new parameters:
``` bash
velero backup create <backup-name>
--include-namespaces=default,kube-system
--include-namespace-scoped-resources=persistentvolumeclaim,deployment,service,endpoint,pod,replicaset
--exclude-cluster-scope-resources=*
```
Example of old parameters:
``` bash
velero backup create <backup-name>
--include-namespaces=default,kube-system
--include-resources=persistentvolumeclaim,deployment,service,endpoint,pod,replicaset
--include-cluster-resources=false
```
##### scenario 4
The following commands mean backup all resources except Ingress type resources in all namespaces, and no cluster-scoped resources.
Example of new parameters:
``` bash
velero backup create <backup-name>
--exclude-namespace-scoped-resources=ingress
--exclude-cluster-scoped-resources=*
```
Example of old parameters:
``` bash
velero backup create <backup-name>
--exclude-resources=ingress
--include-cluster-resources=false
```
#### some namespace-scoped resources + only related cluster-scoped resources
##### scenario 1
This means backup all resources in namespaces default and kube-system, and related cluster-scoped resources.
``` bash
velero backup create <backup-name>
--include-namespaces=default,kube-system
```
##### scenario 2
This means backup pods and configmaps in namespaces default and kube-system, and related cluster-scoped resources.
``` bash
velero backup create <backup-name>
--include-namespaces=default,kube-system
--include-namespace-scoped-resources=pods,configmaps
```
##### scenario 3
This means backup all resources except Ingress type resources in all namespaces, and related cluster-scoped resources.
Example of new parameters:
``` bash
velero backup create <backup-name>
--exclude-namespace-scoped-resources=ingress
```
Example of old parameters:
``` bash
velero backup create <backup-name>
--exclude-resources=ingress
```
#### some namespace-scoped resources + some additional cluster-scoped resources
##### scenario 1
This means backup all resources in namespace in default, kube-system, and related cluster-scoped resources, plus all StorageClass resources.
``` bash
velero backup create <backup-name>
--include-namespaces=default,kube-system
--include-cluster-scoped-resources=storageclass
```
##### scenario 2
This means backup PVC, Deployment, Service, Endpoint, Pod and ReplicaSet resources in all namespaces, and related cluster-scoped resources, plus all StorageClass resources, and PVC related PV.
``` bash
velero backup create <backup-name>
--include-namespace-scoped-resources=persistentvolumeclaim,deployment,service,endpoint,pod,replicaset
--include-cluster-scoped-resources=storageclass
```
##### scenario 3
This means backup PVC, Deployment, Service, Endpoint, Pod and ReplicaSet resources in default and kube-system namespaces, and related cluster-scoped resources, plus all StorageClass resources, and PVC related PV.
``` bash
velero backup create <backup-name>
--include-namespace-scoped-resources=persistentvolumeclaim,deployment,service,endpoint,pod,replicaset
--include-namespaces=default,kube-system
--include-cluster-scoped-resources=storageclass
```
##### scenario 4
This means backup PVC, Deployment, Service, Endpoint, Pod and ReplicaSet resources in default and kube-system namespaces, and related cluster-scoped resources, plus all cluster-scoped resources except StorageClass type resources.
``` bash
velero backup create <backup-name>
--include-namespace-scoped-resources=persistentvolumeclaim,deployment,service,endpoint,pod,replicaset
--include-namespaces=default,kube-system
--exclude-cluster-scoped-resources=storageclass
```
#### some namespace-scoped resources + all cluster-scoped resources
##### scenario 1
The following commands mean backup all resources in namespace in default, kube-system, and all cluster-scoped resources.
Example of new parameters:
``` bash
velero backup create <backup-name>
--include-namespaces=default,kube-system
--include-cluster-scoped-resources=*
```
Example of old parameters:
``` bash
velero backup create <backup-name>
--include-namespaces=default,kube-system
--include-cluster-resources=true
```
##### scenario 2
This means backup Deployment, Service, Endpoint, Pod and ReplicaSet resources in all namespaces, and all cluster-scoped resources.
``` bash
velero backup create <backup-name>
--include-namespace-scoped-resources=deployment,service,endpoint,pod,replicaset
--include-cluster-scoped-resources=*
```
##### scenario 3
This means backup Deployment, Service, Endpoint, Pod and ReplicaSet resources in default and kube-system namespaces, and all cluster-scoped resources.
``` bash
velero backup create <backup-name>
--include-namespaces=default,kube-system
--include-namespace-scoped-resources=deployment,service,endpoint,pod,replicaset
--include-cluster-scoped-resources=*
```
#### all namespace-scoped resources + no cluster-scoped resources
The following commands all mean backup all namespace-scoped resources and no cluster-scoped resources.
Example of new parameters:
``` bash
velero backup create <backup-name>
--exclude-cluster-scoped-resources=*
```
Example of old parameters:
``` bash
velero backup create <backup-name>
--include-cluster-resources=false
```
#### all namespace-scoped resources + some additional cluster-scoped resources
This command means backup all namespace-scoped resources, and related cluster-scoped resources, plus all PersistentVolume resources.
``` bash
velero backup create <backup-name>
--include-namespaces=*
--include-cluster-scoped-resources=persistentvolume
```
#### all namespace-scoped resources + all cluster-scoped resources
The following commands have the same meaning: backup all namespace-scoped resources, and all cluster-scoped resources.
``` bash
velero backup create <backup-name>
--include-cluster-scoped-resources=*
```
``` bash
velero backup create <backup-name>
--include-cluster-resources=true
```
#### describe command change
In `velero backup describe` command, the four new parameters should be outputted too.
``` bash
velero backup describe <backup-name>
......
Namespaces:
Included: ns2
Excluded: <none>
Resources:
Included cluster-scoped: StorageClass,PersistentVolume
Excluded cluster-scoped: <none>
Included namespace-scoped: default
Excluded namespace-scoped: <none>
......
```
**Note:** `velero restore` command doesn't support those four new parameter in Velero v1.11, but `velero schedule` supports the four new parameters through backup specification.
## Detailed Design
With adding `IncludedNamespaceScopedResources`, `ExcludedNamespaceScopedResources`, `IncludedClusterScopedResources` and `ExcludedClusterScopedResources`, the `BackupSpec` looks like:
``` go
type BackupSpec struct {
......
// IncludedResources is a slice of resource names to include
// in the backup. If empty, all resources are included.
// +optional
// +nullable
IncludedResources []string `json:"includedResources,omitempty"`
// ExcludedResources is a slice of resource names that are not
// included in the backup.
// +optional
// +nullable
ExcludedResources []string `json:"excludedResources,omitempty"`
// IncludeClusterResources specifies whether cluster-scoped resources
// should be included for consideration in the backup.
// +optional
// +nullable
IncludeClusterResources *bool `json:"includeClusterResources,omitempty"`
// IncludedClusterScopedResources is a slice of cluster-scoped
// resource type names to include in the backup.
// If set to "*", all cluster scope resource types are included.
// The default value is empty, which means only related cluster
// scope resources are included.
// +optional
// +nullable
IncludedClusterScopedResources []string `json:"includedClusterScopedResources,omitempty"`
// ExcludedClusterScopedResources is a slice of cluster-scoped
// resource type names to exclude from the backup.
// If set to "*", all cluster scope resource types are excluded.
// +optional
// +nullable
ExcludedClusterScopedResources []string `json:"excludedClusterScopedResources,omitempty"`
// IncludedNamespaceScopedResources is a slice of namespace-scoped
// resource type names to include in the backup.
// The default value is "*".
// +optional
// +nullable
IncludedNamespaceScopedResources []string `json:"includedNamespaceScopedResources,omitempty"`
// ExcludedNamespaceScopedResources is a slice of namespace-scoped
// resource type names to exclude from the backup.
// If set to "*", all namespace scope resource types are excluded.
// +optional
// +nullable
ExcludedNamespaceScopedResources []string `json:"excludedNamespaceScopedResources,omitempty"`
......
}
```
## Alternatives Considered
Proposal from Jibu Data [Issue 5120](https://github.com/vmware-tanzu/velero/issues/5120#issue-1304534563)
## Security Considerations
No security impact.
## Compatibility
The four new parameters cannot be mixed with existing resource filter parameters: `IncludedResources`, `ExcludedResources` and `IncludeClusterResources`.
If the new parameters and old parameters both appears in command line, or are specified in backup spec, the command line and the backup should fail.
## Implementation
This change should be included into Velero v1.11.
New parameters will coexist with `IncludedResources`, `ExcludedResources` and `IncludeClusterResources`.
Plan to deprecate `IncludedResources`, `ExcludedResources` and `IncludeClusterResources` in future releases, but also open to the community's feedback.
## Open Issues
`LabelSelector/OrLabelSelectors` apply to namespace-scoped resources.
It may be reasonable to make them also working on cluster-scoped resources.
An issue is created to trace this topic [resource label selector not work for cluster-scoped resources](https://github.com/vmware-tanzu/velero/issues/5787)

View File

@@ -304,8 +304,8 @@ Without these objects, the provider-level snapshots cannot be located in order t
[1]: https://kubernetes.io/blog/2018/10/09/introducing-volume-snapshot-alpha-for-kubernetes/
[2]: https://github.com/kubernetes-csi/external-snapshotter/blob/master/client/apis/volumesnapshot/v1/types.go#L42
[3]: https://github.com/kubernetes-csi/external-snapshotter/blob/master/client/apis/volumesnapshot/v1/types.go#L262
[2]: https://github.com/kubernetes-csi/external-snapshotter/blob/master/pkg/apis/volumesnapshot/v1alpha1/types.go#L41
[3]: https://github.com/kubernetes-csi/external-snapshotter/blob/master/pkg/apis/volumesnapshot/v1alpha1/types.go#L161
[4]: https://github.com/heptio/velero/blob/main/pkg/volume/snapshot.go#L21
[5]: https://github.com/heptio/velero/blob/main/pkg/apis/velero/v1/pod_volume_backup.go#L88
[6]: https://github.com/heptio/velero-csi-plugin/

View File

@@ -1,262 +0,0 @@
# Add support for `ExistingResourcePolicy` to restore API
## Abstract
Velero currently does not support any restore policy on Kubernetes resources that are already present in-cluster. Velero skips over the restore of the resource if it already exists in the namespace/cluster irrespective of whether the resource present in the restore is the same or different from the one present on the cluster. It is desired that Velero gives the option to the user to decide whether or not the resource in backup should overwrite the one present in the cluster.
## Background
As of Today, Velero will skip over the restoration of resources that already exist in the cluster. The current workflow followed by Velero is (Using a `service` that is backed up for example):
- Velero tries to attempt restore of the `service`
- Fetches the `service` from the cluster
- If the `service` exists then:
- Checks whether the `service` instance in the cluster is equal to the `service` instance present in backup
- If not equal then skips the restore of the `service` and adds a restore warning (except for [ServiceAccount objects](https://github.com/vmware-tanzu/velero/blob/574baeb3c920f97b47985ec3957debdc70bcd5f8/pkg/restore/restore.go#L1246))
- If equal then skips the restore of the `service` and mentions that the restore of resource `service` is skipped in logs
It is desired to add the functionality to specify whether or not to overwrite the instance of resource `service` in cluster with the one present in backup during the restore process.
Related issue: https://github.com/vmware-tanzu/velero/issues/4066
## Goals
- Add support for `ExistingResourcePolicy` to restore API for Kubernetes resources.
## Non Goals
- Change existing restore workflow for `ServiceAccount` objects
- Add support for `ExistingResourcePolicy` as `recreate` for Kubernetes resources. (Future scope feature)
## Unrelated Proposals (Completely different functionalities than the one proposed in the design)
- Add support for `ExistingResourcePolicy` to restore API for Non-Kubernetes resources.
- Add support for `ExistingResourcePolicy` to restore API for `PersistentVolume` data.
### Use-cases/Scenarios
### A. Production Cluster - Backup Cluster:
Let's say you have a Backup Cluster which is identical to the Production Cluster. After some operations/usage/time the Production Cluster had changed itself, there might be new deployments, some secrets might have been updated. Now, this means that the Backup cluster will no longer be identical to the Production Cluster. In order to keep the Backup Cluster up to date/identical to the Production Cluster with respect to Kubernetes resources except PV data we would like to use Velero for scheduling new backups which would in turn help us update the Backup Cluster via Velero restore.
Reference: https://github.com/vmware-tanzu/velero/issues/4066#issuecomment-954320686
### B. Help identify resource delta:
Here delta resources mean the resources restored by a previous backup, but they are no longer in the latest backup. Let's follow a sequence of steps to understand this scenario:
- Consider there are 2 clusters, Cluster A, which has 3 resources - P1, P2 and P3.
- Create a Backup1 from Cluster A which has P1, P2 and P3.
- Perform restore on a new Cluster B using Backup1.
- Now, Lets say in Cluster A resource P1 gets deleted and resource P2 gets updated.
- Create a new Backup2 with the new state of Cluster A, keep in mind Backup1 has P1, P2 and P3 while Backup2 has P2' and P3.
- So the Delta here is (|Cluster B - Backup2|), Delete P1 and Update P2.
- During Restore time we would want the Restore to help us identify this resource delta.
Reference: https://github.com/vmware-tanzu/velero/pull/4613#issuecomment-1027260446
## High-Level Design
### Approach 1: Add a new spec field `existingResourcePolicy` to the Restore API
In this approach we do *not* change existing velero behavior. If the resource to restore in cluster is equal to the one backed up then do nothing following current Velero behavior. For resources that already exist in the cluster that are not equal to the resource in the backup (other than Service Accounts). We add a new optional spec field `existingResourcePolicy` which can have the following values:
1. `none`: This is the existing behavior, if Velero encounters a resource that already exists in the cluster, we simply
skip restoration.
2. `update`: This option would provide the following behavior.
- Unchanged resources: Velero would update the backup/restore labels on the unchanged resources, if labels patch fails Velero adds a restore error.
- Changed resources: Velero will first try to patch the changed resource, Now if the patch:
- succeeds: Then the in-cluster resource gets updated with the labels as well as the resource diff
- fails: Velero adds a restore warning and tries to just update the backup/restore labels on the resource, if the labels patch also fails then we add restore error.
3. `recreate`: If resource already exists, then Velero will delete it and recreate the resource.
*Note:* The `recreate` option is a non-goal for this enhancement proposal, but it is considered as a future scope.
Another thing to highlight is that Velero will not be deleting any resources in any of the policy options proposed in
this design but Velero will patch the resources in `update` policy option.
Example:
A. The following Restore will execute the `existingResourcePolicy` restore type `none` for the `services` and `deployments` present in the `velero-protection` namespace.
```
Kind: Restore
includeNamespaces: velero-protection
includeResources:
- services
- deployments
existingResourcePolicy: none
```
B. The following Restore will execute the `existingResourcePolicy` restore type `update` for the `secrets` and `daemonsets` present in the `gdpr-application` namespace.
```
Kind: Restore
includeNamespaces: gdpr-application
includeResources:
- secrets
- daemonsets
existingResourcePolicy: update
```
### Approach 2: Add a new spec field `existingResourcePolicyConfig` to the Restore API
In this approach we give user the ability to specify which resources are to be included for a particular kind of force update behaviour, essentially a more granular approach where in the user is able to specify a resource:behaviour mapping. It would look like:
`existingResourcePolicyConfig`:
- `patch:`
- `includedResources:` [ ]string
- `recreate:`
- `includedResources:` [ ]string
*Note:*
- There is no `none` behaviour in this approach as that would conform to the current/default Velero restore behaviour.
- The `recreate` option is a non-goal for this enhancement proposal, but it is considered as a future scope.
Example:
A. The following Restore will execute the restore type `patch` and apply the `existingResourcePolicyConfig` for `secrets` and `daemonsets` present in the `inventory-app` namespace.
```
Kind: Restore
includeNamespaces: inventory-app
existingResourcePolicyConfig:
patch:
includedResources
- secrets
- daemonsets
```
### Approach 3: Combination of Approach 1 and Approach 2
Now, this approach is somewhat a combination of the aforementioned approaches. Here we propose addition of two spec fields to the Restore API - `existingResourceDefaultPolicy` and `existingResourcePolicyOverrides`. As the names suggest ,the idea being that `existingResourceDefaultPolicy` would describe the default velero behaviour for this restore and `existingResourcePolicyOverrides` would override the default policy explicitly for some resources.
Example:
A. The following Restore will execute the restore type `patch` as the `existingResourceDefaultPolicy` but will override the default policy for `secrets` using the `existingResourcePolicyOverrides` spec as `none`.
```
Kind: Restore
includeNamespaces: inventory-app
existingResourceDefaultPolicy: patch
existingResourcePolicyOverrides:
none:
includedResources
- secrets
```
## Detailed Design
### Approach 1: Add a new spec field `existingResourcePolicy` to the Restore API
The `existingResourcePolicy` spec field will be an `PolicyType` type field.
Restore API:
```
type RestoreSpec struct {
.
.
.
// ExistingResourcePolicy specifies the restore behaviour for the Kubernetes resource to be restored
// +optional
ExistingResourcePolicy PolicyType
}
```
PolicyType:
```
type PolicyType string
const PolicyTypeNone PolicyType = "none"
const PolicyTypePatch PolicyType = "update"
```
### Approach 2: Add a new spec field `existingResourcePolicyConfig` to the Restore API
The `existingResourcePolicyConfig` will be a spec of type `PolicyConfiguration` which gets added to the Restore API.
Restore API:
```
type RestoreSpec struct {
.
.
.
// ExistingResourcePolicyConfig specifies the restore behaviour for a particular/list of Kubernetes resource(s) to be restored
// +optional
ExistingResourcePolicyConfig []PolicyConfiguration
}
```
PolicyConfiguration:
```
type PolicyConfiguration struct {
PolicyTypeMapping map[PolicyType]ResourceList
}
```
PolicyType:
```
type PolicyType string
const PolicyTypePatch PolicyType = "patch"
const PolicyTypeRecreate PolicyType = "recreate"
```
ResourceList:
```
type ResourceList struct {
IncludedResources []string
}
```
### Approach 3: Combination of Approach 1 and Approach 2
Restore API:
```
type RestoreSpec struct {
.
.
.
// ExistingResourceDefaultPolicy specifies the default restore behaviour for the Kubernetes resource to be restored
// +optional
existingResourceDefaultPolicy PolicyType
// ExistingResourcePolicyOverrides specifies the restore behaviour for a particular/list of Kubernetes resource(s) to be restored
// +optional
existingResourcePolicyOverrides []PolicyConfiguration
}
```
PolicyType:
```
type PolicyType string
const PolicyTypeNone PolicyType = "none"
const PolicyTypePatch PolicyType = "patch"
const PolicyTypeRecreate PolicyType = "recreate"
```
PolicyConfiguration:
```
type PolicyConfiguration struct {
PolicyTypeMapping map[PolicyType]ResourceList
}
```
ResourceList:
```
type ResourceList struct {
IncludedResources []string
}
```
The restore workflow changes will be done [here](https://github.com/vmware-tanzu/velero/blob/b40bbda2d62af2f35d1406b9af4d387d4b396839/pkg/restore/restore.go#L1245)
### CLI changes for Approach 1
We would introduce a new CLI flag called `existing-resource-policy` of string type. This flag would be used to accept the
policy from the user. The velero restore command would look somewhat like this:
```
velero create restore <restore_name> --existing-resource-policy=update
```
Help message `Restore Policy to be used during the restore workflow, can be - none, update`
The CLI changes will go at `pkg/cmd/cli/restore/create.go`
We would also add a validation which checks for invalid policy values provided to this flag.
Restore describer will also be updated to reflect the policy `pkg/cmd/util/output/restore_describer.go`
### Implementation Decision
We have decided to go ahead with the implementation of Approach 1 as:
- It is easier to implement
- It is also easier to scale and leaves room for improvement and the door open to expanding to approach 3
- It also provides an option to preserve the existing velero restore workflow

View File

@@ -1,579 +0,0 @@
# Plugin Progress Monitoring
This is intended as a replacement for the previously-approved Upload Progress Monitoring design
([Upload Progress Monitoring](upload-progress.md)) in order to expand the supported use cases beyond
snapshot uploads to include what was previously called Async Backup/Restore Item Actions. This
updated design should handle the combined set of use cases for those previously separate designs.
Volume snapshotter plugin are used by Velero to take snapshots of persistent volume contents.
Depending on the underlying storage system, those snapshots may be available to use immediately,
they may be uploaded to stable storage internally by the plugin or they may need to be uploaded after
the snapshot has been taken. We would like for Velero to continue on to the next part of the backup as quickly
as possible but we would also like the backup to not be marked as complete until it is a usable backup. We'd also
eventually like to bring the control of upload under the control of Velero and allow the user to make decisions
about the ultimate destination of backup data independent of the storage system they're using.
We would also like any internal or third party Backup or Restore Item Action to have the option of
making use of this same ability to run some external process without blocking the current backup or
restore operation. Beyond Volume Snapshotters, this is also needed for data mover operations on both
backup and restore, and potentially useful for other third party operations -- for example
in-cluster registry image backup or restore could make use of this feature in a third party plugin).
### Glossary
- <b>BIA</b>: BackupItemAction
- <b>RIA</b>: RestoreItemAction
## Examples
- AWS - AWS snapshots return quickly, but are then uploaded in the background and cannot be used until EBS moves
the data into S3 internally.
- vSphere - The vSphere plugin takes a local snapshot and then the vSphere plugin uploads the data to S3. The local
snapshot is usable before the upload completes.
- Restic - Does not go through the volume snapshot path. Restic backups will block Velero progress
until completed. However, with the more generalized approach in the revised design, restic/kopia
backup and restore *could* make use of this framework if their actions are refactored as
Backup/RestoreItemActions.
- Data Movers
- Data movers are asynchronous processes executed inside backup/restore item actions that applies to a specific Kubernetes resources. A common use case for data mover is to backup/restore PVCs whose data we want to move to some form of backup storage outside of using velero kopia/restic implementations.
- Workflow
- User takes velero backup of PVC A
- BIA plugin applies to PVCs with compatible storage driver
- BIA plugin triggers data mover
- Most common use case would be for the plugin action to create a new CR which would
trigger an external controller action
- Another possible use case would be for the plugin to run its own async action in a
goroutine, although this would be less resilient to plugin container restarts.
- BIA plugin returns
- Velero backup process continues
- Main velero backup process monitors running BIA threads via gRPC to determine if process is done and healthy
## Primary changes from the original Upload Progress Monitoring design
The most fundamental change here is that rather than proposing a new special-purpose
SnapshotItemAction, the existing BackupItemAction plugin will be modified to accommodate an optional
snapshot (or other item operation) ID return. The primary reasons for this change are as follows:
1. The intended scope has moved beyond snapshot processing, so it makes sense to support
asynchronous operations in other backup or restore item actions.
2. We expect to have plugin API versioning implemented in Velero 1.10, making it feasible to
implement changes in the existing plugin APIs now.
3. We will need this feature on both backup and restore, meaning that if we took the "new plugin
type" approach, we'd need two new plugin types.
4. Other than the snapshot/operation ID return, the rest of the plugin processing is identical to
Backup/RestoreItemActions. With separate plugin types, we'd have to repeat all of that logic
(including dealing with additional items, etc.) twice.
The other major change is that we will be applying this to both backups and restores, although the
Volume Snapshotter use case only needs this on backup. This means that everything we're doing around
backup phase and workflow will also need to be done for restore.
Then there are various minor changes around terminology to make things more generic. Instead of
"snapshotID", we'll have "operationID" (which for volume snapshotters will be a snapshot
ID).
## Goals
- Enable monitoring of backup/restore item action operations that continue after snapshotting and other operations have completed
- Keep non-usable backups and restores (upload/persistence has not finished, etc.) from appearing as completed
- Make use of plugin API versioning functionality to manage changes to Backup/RestoreItemAction interfaces
- Enable vendors to plug their own data movers into velero using BIA/RIA plugins
## Non-goals
- Today, Velero is unable to recover from an in progress backup when the velero server crashes (pod is deleted). This has an impact on running asynchronous processes, but its not something we intend to solve in this design.
## Models
### Internal configuration and management
In this model, movement of the snapshot to stable storage is under the control of the snapshot
plugin. Decisions about where and when the snapshot gets moved to stable storage are not
directly controlled by Velero. This is the model for the current VolumeSnapshot plugins.
### Velero controlled management
In this model, the snapshot is moved to external storage under the control of Velero. This
enables Velero to move data between storage systems. This also allows backup partners to use
Velero to snapshot data and then move the data into their backup repository.
## Backup and Restore phases
Velero currently has backup/restore phases "InProgress" and "Completed". A backup moves to the
Completed phase when all of the volume snapshots have completed and the Kubernetes metadata has been
written into the object store. However, the actual data movement may be happening in the background
after the backup has been marked "Completed". The backup is not actually a stable backup until the
data has been persisted properly. In some cases (e.g. AWS) the backup cannot be restored from until
the snapshots have been persisted.
Once the snapshots have been taken, however, it is possible for additional backups or restores (as
long as they don't use not-yet-completed backups) to be made without interference. Waiting until
all data has been moved before starting the next backup will slow the progress of the system without
adding any actual benefit to the user.
New backup/restore phases, "WaitingForPluginOperations" and
"WaitingForPluginOperationsPartiallyFailed" will be introduced. When a backup or restore has
entered one of these phases, Velero is free to start another backup/restore. The backup/restore
will remain in the "WaitingForPluginOperations" phase until all BIA/RIA operations have completed
(for example, for a volume snapshotter, until all data has been successfully moved to persistent
storage). The backup/restore will not fail once it reaches this phase, although an error return
from a plugin could cause a backup or restore to move to "PartiallyFailed". If the backup is
deleted (cancelled), the plugins will attempt to delete the snapshots and stop the data movement -
this may not be possible with all storage systems.
In addition, for backups (but not restores), there will also be two additional phases, "Finalizing"
and "FinalizingPartiallyFailed", which will handle any steps required after plugin operations have
all completed. Initially, this will just include adding any required resources to the backup that
might have changed during asynchronous operation execution, although eventually other cleanup
actions could be added to this phase.
### State progression
![image](AsyncActionFSM.png)
### New
When a backup/restore request is initially created, it is in the "New" phase.
The next state is either "InProgress" or "FailedValidation"
### FailedValidation
If the backup/restore request is incorrectly formed, it goes to the "FailedValidation" phase and
terminates
### InProgress
When work on the backup/restore begins, it moves to the "InProgress" phase. It remains in the
"InProgress" phase until all pre/post execution hooks have been executed, all snapshots have been
taken and the Kubernetes metadata and backup/restore info is safely written to the object store
plugin.
In the current implementation, Restic backups will move data during the "InProgress" phase. In the
future, it may be possible to combine a snapshot with a Restic (or equivalent) backup which would
allow for data movement to be handled in the "WaitingForPluginOperations" phase,
The next phase would be "WaitingForPluginOperations" for backups or restores which have unfinished
asynchronous plugin operations and no errors so far, "WaitingForPluginOperationsPartiallyFailed" for
backups or restores which have unfinished asynchronous plugin operations at least one error,
"Completed" for restores with no unfinished asynchronous plugin operations and no errors,
"PartiallyFailed" for restores with no unfinished asynchronous plugin operations and at least one
error, "Finalizing" for backups with no unfinished asynchronous plugin operations and no errors,
"FinalizingPartiallyFailed" for backups with no unfinished asynchronous plugin operations and at
least one error, or "PartiallyFailed". Backups/restores which would have a final phase of
"Completed" or "PartiallyFailed" may move to the "WaitingForPluginOperations" or
"WaitingForPluginOperationsPartiallyFailed" state. A backup/restore which will be marked "Failed"
will go directly to the "Failed" phase. Uploads may continue in the background for snapshots that
were taken by a "Failed" backup/restore, but no progress will not be monitored or updated. If there
are any operations in progress when a backup is moved to the "Failed" phase (although with the
current workflow, that shouldn't happen), Cancel() should be called on these operations. When a
"Failed" backup is deleted, all snapshots will be deleted and at that point any uploads still in
progress should be aborted.
### WaitingForPluginOperations (new)
The "WaitingForPluginOperations" phase signifies that the main part of the backup/restore, including
snapshotting has completed successfully and uploading and any other asynchronous BIA/RIA plugin
operations are continuing. In the event of an error during this phase, the phase will change to
WaitingForPluginOperationsPartiallyFailed. On success, the phase changes to
"Finalizing" for backups and "Completed" for restores. Backups cannot be
restored from when they are in the WaitingForPluginOperations state.
### WaitingForPluginOperationsPartiallyFailed (new)
The "WaitingForPluginOperationsPartiallyFailed" phase signifies that the main part of the
backup/restore, including snapshotting has completed, but there were partial failures either during
the main part or during any async operations, including snapshot uploads. Backups cannot be
restored from when they are in the WaitingForPluginOperationsPartiallyFailed state.
### Finalizing (new)
The "Finalizing" phase signifies that asynchronous backup operations have all completed successfully
and Velero is currently backing up any resources indicated by asynchronous plugins as items to back
up after operations complete. Once this is done, the phase changes to Completed. Backups cannot be
restored from when they are in the Finalizing state.
### FinalizingPartiallyFailed (new)
The "FinalizingPartiallyFailed" phase signifies that, for a backup which had errors during initial
processing or asynchronous plugin operation, asynchronous backup operations have all completed and
Velero is currently backing up any resources indicated by asynchronous plugins as items to back up
after operations complete. Once this is done, the phase changes to PartiallyFailed. Backups cannot
be restored from when they are in the FinalizingPartiallyFailed state.
### Failed
When a backup/restore has had fatal errors it is marked as "Failed" Backups in this state cannot be
restored from.
### Completed
The "Completed" phase signifies that the backup/restore has completed, all data has been transferred
to stable storage (or restored to the cluster) and any backup in this state is ready to be used in a
restore. When the Completed phase has been reached for a backup it is safe to remove any of the
items that were backed up.
### PartiallyFailed
The "PartiallyFailed" phase signifies that the backup/restore has completed and at least part of the
backup/restore is usable. Restoration from a PartiallyFailed backup will not result in a complete
restoration but pieces may be available.
## Workflow
When a Backup or Restore Action is executed, any BackupItemAction, RestoreItemAction, or
VolumeSnapshot plugins will return operation IDs (snapshot IDs or other plugin-specific
identifiers). The plugin should be able to provide status on the progress for the snapshot and
handle cancellation of the operation/upload if the snapshot is deleted. If the plugin is restarted,
the operation ID should remain valid.
When all snapshots have been taken and Kubernetes resources have been persisted to the ObjectStorePlugin
the backup will either have fatal errors or will be at least partially usable.
If the backup/restore has fatal errors it will move to the "Failed" state and finish. If a
backup/restore fails, the upload or other operation will not be cancelled but it will not be
monitored either. For backups in any phase, all snapshots will be deleted when the backup is
deleted. Plugins will cancel any data movement or other operations and remove snapshots and other
associated resources when the VolumeSnapshotter DeleteSnapshot method or DeleteItemAction Execute
method is called.
Velero will poll the plugins for status on the operations when the backup/restore exits the
"InProgress" phase and has no fatal errors.
If any operations are not complete, the backup/restore will move to either WaitingForPluginOperations
or WaitingForPluginOperationsPartiallyFailed or Failed.
Post-snapshot and other operations may take a long time and Velero and its plugins may be restarted
during this time. Once a backup/restore has moved into the WaitingForPluginOperations or
WaitingForPluginOperationsPartiallyFailed phase, another backup/restore may be started.
While in the WaitingForPluginOperations or WaitingForPluginOperationsPartiallyFailed phase, the
snapshots and item actions will be periodically polled. When all of the snapshots and item actions
have reported success, restores will move directly to the Completed or PartiallyFailed phase, and
backups will move to the Finalizing or FinalizingPartiallyFailed phase, depending on whether the
backup/restore was in the WaitingForPluginOperations or WaitingForPluginOperationsPartiallyFailed
phase.
While in the Finalizing or FinalizingPartiallyFailed phase, Velero will update the backup with any
resources indicated by plugins that they must be added to the backup after operations are completed,
and then the backup will move to the Completed or PartiallyFailed phase, depending on whether there
are any backup errors.
The Backup resources will be written to object storage at the time the backup leaves the InProgress
phase, but it will not be synced to other clusters (or usable for restores in the current cluster)
until the backup has entered a final phase: Completed, Failed or PartiallyFailed. During the
Finalizing phases, a the backup resources will be updated with any required resources related to
asynchronous plugins.
## Reconciliation of InProgress backups
InProgress backups will not have a `velero-backup.json` present in the object store. During
reconciliation, backups which do not have a `velero-backup.json` object in the object store will be
ignored.
## Plugin API changes
### OperationProgress struct
type OperationProgress struct {
Completed bool // True when the operation has completed, either successfully or with a failure
Err string // Set when the operation has failed
NCompleted, NTotal int64 // Quantity completed so far and the total quantity associated with the operation in operationUnits
// For data mover and volume snapshotter use cases, this would be in bytes
// On successful completion, completed and total should be the same.
OperationUnits string // Units represented by completed and total -- for data mover and item
// snapshotters, this will usually be bytes.
Description string // Optional description of operation progress
Started, Updated time.Time // When the upload was started and when the last update was seen. Not all
// systems retain when the upload was begun, return Time 0 (time.Unix(0, 0))
// if unknown.
}
### VolumeSnapshotter changes
Two new methods will be added to the VolumeSnapshotter interface:
Progress(snapshotID string) (OperationProgress, error)
Cancel(snapshotID string) (error)
Progress will report the current status of a snapshot upload. This should be callable at
any time after the snapshot has been taken. In the event a plugin is restarted, if the operationID
(snapshot ID) continues to be valid it should be possible to retrieve the progress.
`error` is set if there is an issue retrieving progress. If the snapshot is has encountered an
error during the upload, the error should be returned in OperationProgress and error should be nil.
### BackupItemAction and RestoreItemAction plugin changes
Currently CSI snapshots and the Velero Plugin for vSphere are implemented as BackupItemAction
plugins. While the majority of BackupItemAction plugins do not take snapshots or upload data, this
functionality is useful for any longstanding plugin operation managed by an external
process/controller so we will modify BackupItemAction and RestoreItemAction to optionally return an
operationID in addition to the modified item.
Velero can attempt to cancel an operation by calling the Cancel API call on the BIA/RIA. The plugin
can then take any appropriate action as needed. Cancel will be called for unfinished operations on
backup deletion, and possibly reaching timeout. Cancel is not intended to be used to delete/remove
the results of completed actions and will have no effect on a completed action. Cancel has no return
value apart from the standard Error return, but this should only be used for unexpected
failures. Under normal operations, Cancel will simply return a nil error (and nothing else), whether
or not the plugin is able to cancel the operation.
_AsyncOperationsNotSupportedError_ should only be returned (by Progress) if the
Backup/RestoreItemAction plugin should not be handling the item. If the Backup/RestoreItemAction
plugin should handle the item but, for example, the item/snapshot ID cannot be found to report
progress, Progress will return an InvalidOperationIDError error rather than a populated
OperationProgress struct. If the item action does not start an asynchronous operation, then
operationID will be empty.
Three new methods will be added to the BackupItemAction interface, and the Execute() return signature
will be modified:
// Name returns the name of this BIA. Plugins which implement this interface must defined Name,
// but its content is unimportant, as it won't actually be called via RPC. Velero's plugin infrastructure
// will implement this directly rather than delegating to the RPC plugin in order to return the name
// that the plugin was registered under. The plugins must implement the method to complete the interface.
Name() string
// Execute allows the BackupItemAction to perform arbitrary logic with the item being backed up,
// including mutating the item itself prior to backup. The item (unmodified or modified)
// should be returned, along with an optional slice of ResourceIdentifiers specifying
// additional related items that should be backed up now, an optional operationID for actions which
// initiate asynchronous actions, and a second slice of ResourceIdentifiers specifying related items
// which should be backed up after all asynchronous operations have completed. This last field will be
// ignored if operationID is empty, and should not be filled in unless the resource must be updated in the
// backup after async operations complete (i.e. some of the item's Kubernetes metadata will be updated
// during the asynch operation which will be required during restore)
Execute(item runtime.Unstructured, backup *api.Backup) (runtime.Unstructured, []velero.ResourceIdentifier, string, []velero.ResourceIdentifier, error)
// Progress
Progress(operationID string, backup *api.Backup) (velero.OperationProgress, error)
// Cancel
Cancel(operationID string, backup *api.Backup) error
Three new methods will be added to the RestoreItemAction interface, and the
RestoreItemActionExecuteOutput struct will be modified:
// Name returns the name of this RIA. Plugins which implement this interface must defined Name,
// but its content is unimportant, as it won't actually be called via RPC. Velero's plugin infrastructure
// will implement this directly rather than delegating to the RPC plugin in order to return the name
// that the plugin was registered under. The plugins must implement the method to complete the interface.
Name() string
// Execute allows the ItemAction to perform arbitrary logic with the item being restored,
// including mutating the item itself prior to restore. The item (unmodified or modified)
// should be returned, an optional OperationID, along with an optional slice of ResourceIdentifiers
// specifying additional related items that should be restored, a warning (which will be
// logged but will not prevent the item from being restored) or error (which will be logged
// and will prevent the item from being restored) if applicable. If OperationID is specified
// then velero will wait for this operation to complete before the restore is marked Completed.
Execute(input *RestoreItemActionExecuteInput) (*RestoreItemActionExecuteOutput, error)
// Progress
Progress(operationID string, restore *api.Restore) (velero.OperationProgress, error)
// Cancel
Cancel(operationID string, restore *api.Restore) error
// RestoreItemActionExecuteOutput contains the output variables for the ItemAction's Execution function.
type RestoreItemActionExecuteOutput struct {
// UpdatedItem is the item being restored mutated by ItemAction.
UpdatedItem runtime.Unstructured
// AdditionalItems is a list of additional related items that should
// be restored.
AdditionalItems []ResourceIdentifier
// SkipRestore tells velero to stop executing further actions
// on this item, and skip the restore step. When this field's
// value is true, AdditionalItems will be ignored.
SkipRestore bool
// OperationID is an identifier which indicates an ongoing asynchronous action which Velero will
// continue to monitor after restoring this item. If left blank, then there is no ongoing operation
OperationID string
}
## Changes in Velero backup format
No changes to the existing format are introduced by this change. As part of the backup workflow changes, a
`<backup-name>-itemoperations.json.gz` file will be added that contains the items and operation IDs
(snapshotIDs) returned by VolumeSnapshotter and BackupItemAction plugins. Also, the creation of the
`velero-backup.json` object will not occur until the backup moves to one of the terminal phases
(_Completed_, _PartiallyFailed_, or _Failed_). Reconciliation should ignore backups that do not
have a `velero-backup.json` object.
The Backup/RestoreItemAction plugin identifier as well as the ItemID and OperationID will be stored
in the `<backup-name>-itemoperations.json.gz`. When checking for progress, this info will be used
to select the appropriate Backup/RestoreItemAction plugin to query for progress. Here's an example
of what a record for a datamover plugin might look like:
```
{
"spec": {
"backupName": "backup-1",
"backupUID": "f8c72709-0f73-46e1-a071-116bc4a76b07",
"backupItemAction": "velero.io/volumesnapshotcontent-backup",
"resourceIdentifier": {
"Group": "snapshot.storage.k8s.io",
"Resource": "VolumeSnapshotContent",
"Namespace": "my-app",
"Name": "my-volume-vsc"
},
"operationID": "<DataMoverBackup objectReference>",
"itemsToUpdate": [
{
"Group": "velero.io",
"Resource": "VolumeSnapshotBackup",
"Namespace": "my-app",
"Name": "vsb-1"
}
]
},
"status": {
"operationPhase": "Completed",
"error": "",
"nCompleted": 12345,
"nTotal": 12345,
"operationUnits": "byte",
"description": "",
"Created": "2022-12-14T12:00:00Z",
"Started": "2022-12-14T12:01:00Z",
"Updated": "2022-12-14T12:11:02Z"
},
}
```
The cluster that is creating the backup will have the Backup resource present and will be able to
manage the backup before the backup completes.
If the Backup resource is removed (e.g. Velero is uninstalled) before a backup completes and writes
its `velero-backup.json` object, the other objects in the object store for the backup will be
effectively orphaned. This can currently happen but the current window is much smaller.
### `<backup-name>-itemoperations.json.gz`
The itemoperations file is similar to the existing `<backup-name>-itemsnapshots.json.gz` Each snapshot taken via
BackupItemAction will have a JSON record in the file. Exact format TBD.
This file will be uploaded to object storage at the end of processing all of the items in the
backup, before the phase moves away from `InProgress`.
## Changes to Velero restores
A `<restore-name>-itemoperations.json.gz` file will be added that contains the items and operation
IDs returned by RestoreItemActions. The format will be the same as the
`<backup-name>-itemoperations.json.gz` generated for backups.
This file will be uploaded to object storage at the end of processing all of the items in the
restore, before the phase moves away from `InProgress`.
## CSI snapshots
For systems such as EBS, a snapshot is not available until the storage system has transferred the
snapshot to stable storage. CSI snapshots expose the _readyToUse_ state that, in the case of EBS,
indicates that the snapshot has been transferred to durable storage and is ready to be used. The
CSI BackupItemAction.Progress method will poll that field and when completed, return completion.
## vSphere plugin
The vSphere Plugin for Velero uploads snapshots to S3 in the background. This is also a
BackupItemAction plugin, it will check the status of the Upload records for the snapshot and return
progress.
## Backup workflow changes
The backup workflow remains the same until we get to the point where the `velero-backup.json` object
is written. At this point, Velero will
run across all of the VolumeSnapshotter/BackupItemAction operations and call the _Progress_ method
on each of them.
If all backup item operations have finished (either successfully or failed), the backup will move to
one of the finalize phases.
If any of the snapshots or backup items are still being processed, the phase of the backup will be
set to the appropriate phase (_WaitingForPluginOperations_ or
_WaitingForPluginOperationsPartiallyFailed_), and the async backup operations controller will
reconcile periodically and call Progress on any unfinished operations. In the event of any of the
progress checks return an error, the phase will move to _WaitingForPluginOperationsPartiallyFailed_.
Once all operations have completed, the backup will be moved to one of the finalize phases, and the
backup finalizer controller will update the the `velero-backup.json`in the object store with any
resources necessary after asynchronous operations are complete and the backup will move to the
appropriate terminal phase.
## Restore workflow changes
The restore workflow remains the same until velero would currently move the backup into one of the
terminal states. At this point, Velero will run across all of the RestoreItemAction operations and
call the _Progress_ method on each of them.
If all restore item operations have finished (either successfully or failed), the restore will be
completed and the restore will move to the appropriate terminal phase and the restore will be
complete.
If any of the restore items are still being processed, the phase of the restore will be set to the
appropriate phase (_WaitingForPluginOperations_ or _WaitingForPluginOperationsPartiallyFailed_), and
the async restore operations controller will reconcile periodically and call Progress on any
unfinished operations. In the event of any of the progress checks return an error, the phase will
move to _WaitingForPluginOperationsPartiallyFailed_. Once all of the operations have completed, the
restore will be moved to the appropriate terminal phase.
## Restart workflow
On restart, the Velero server will scan all Backup/Restore resources. Any Backup/Restore resources
which are in the _InProgress_ phase will be moved to the _Failed_ phase. Any Backup/Restore
resources in the _WaitingForPluginOperations_ or _WaitingForPluginOperationsPartiallyFailed_ phase
will be treated as if they have been requeued and progress checked and the backup/restore will be
requeued or moved to a terminal phase as appropriate.
## Notes on already-merged code which may need updating
Since this design is modifying a previously-approved design, there is some preparation work based on
the earlier upload progress monitoring design that may need modification as a result of these
updates. Here is a list of some of these items:
1. Consts for the "Uploading" and "UploadingPartiallyFailed" phases have already been defined. These
will need to be removed when the "WaitingForPluginOperations" and
"WaitingForPluginOperationsPartiallyFailed" phases are defined.
- https://github.com/vmware-tanzu/velero/pull/3805
1. Remove the ItemSnapshotter plugin APIs (and related code) since the revised design will reuse
VolumeSnapshotter and BackupItemAction plugins.
- https://github.com/vmware-tanzu/velero/pull/4077
- https://github.com/vmware-tanzu/velero/pull/4417
1. UploadProgressFeatureFlag shouldn't be needed anymore. The current design won't really need a
feature flag here -- the new features will be added to V2 of the VolumeSnapshotter,
BackupItemAction, and RestoreItemAction plugins, and it will only be used if there are plugins which
return operation IDs.
- https://github.com/vmware-tanzu/velero/pull/4416
1. Adds <backup-name>-itemsnapshots.gz file to backup (when provided) -- this is still part of the
revised design, so it should stay.
- https://github.com/vmware-tanzu/velero/pull/4429
1. Upload Progress Monitoring and Item Snapshotter basic support: This PR is not yet merged, so
nothing will need to be reverted. While the implementation here will be useful in informing the
implementation of the new design, several things have changed in the design proposal since the PR
was written.
- https://github.com/vmware-tanzu/velero/pull/4467
# Implementation tasks
VolumeSnapshotter new plugin APIs
BackupItemAction new plugin APIs
RestoreItemAction new plugin APIs
New backup phases
New restore phases
Defer uploading `velero-backup.json`
AWS EBS plugin Progress implementation
Operation monitoring
Implementation of `<backup-name>-itemoperations.json.gz` file
Implementation of `<restore-name>-itemoperations.json.gz` file
Restart logic
Change in reconciliation logic to ignore backups/restores that have not completed
CSI plugin BackupItemAction Progress implementation
vSphere plugin BackupItemAction Progress implementation (vSphere plugin team)
# Open Questions
1. Do we need a Cancel operation for VolumeSnapshotter?
- From feedback, I'm thinking we probably don't need it. The only real purpose of Cancel
here is to tell the plugin that Velero won't be waiting anymore, so if there are any
required custom cancellation actions, now would be a good time to perform them. For snapshot
uploads that are already in proress, there's not really anything else to cancel.
2. Should we actually write the backup *before* moving to the WaitingForPluginOperations or
WaitingForPluginOperationsPartiallyFailed phase rather than waiting until all operations
have completed? The operations in question won't affect what gets written to object storage
for the backup, and since we've already written the list of operations we're waiting for to
object storage, writing the backup now would make the process resilient to Velero restart if
it happens during WaitingForPluginOperations or WaitingForPluginOperationsPartiallyFailed

View File

@@ -28,7 +28,7 @@ This document proposes adding _controller-tools_ to the project to automatically
_controller-tools_ works by reading the Go files that contain the API type definitions.
It uses a combination of the struct fields, types, tags and comments to build the OpenAPIv3 schema for the CRDs. The tooling makes some assumptions based on conventions followed in upstream Kubernetes and the ecosystem, which involves some changes to the Velero API type definitions, especially around optional fields.
In order for _controller-tools_ to read the Go files containing Velero API type definitions, the CRDs need to be generated at build time, as these files are not available at runtime (i.e. the Go files are not accessible by the compiled binary).
In order for _controller-tools_ to read the Go files containing Velero API type defintiions, the CRDs need to be generated at build time, as these files are not available at runtime (i.e. the Go files are not accessible by the compiled binary).
These generated CRD manifests (YAML) will then need to be available to the `pkg/install` package for it to include when installing Velero resources.
## Detailed Design

Some files were not shown because too many files have changed in this diff Show More