Use the same pvb/pvr update functions across pkg/controller and pkg/cli/nodeagent for consistency of behavior
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
Restore ClusterBootstrap before Cluster otherwise a new default ClusterBootstrap object is create for the cluster
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
Enhance the restore priorities list to support specifying the low prioritized resources that need to be r
estored in the last
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
This commit provides a simple contract that if the BackupItemAction
plugin sets an annotation in a resource it has handled, the additional
items will considered "must include" i.e. each of them will skip the
"include-exclude" filter, such that the plugin developer can make sure
they are included in the backup disregarding the filter setting in the
bakcup CR.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
In determining whether a backup includes all namespaces, item_collector
checks for an empty string in the first element of the ns list. If processing
includes+excludes results in an empty list, treat this as another case
of a not-all-namespaces backup rather than crashing velero.
Signed-off-by: Scott Seago <sseago@redhat.com>
1. Also checking annotation "pv.kubernetes.io/migrated-to" to find out whether volume is provisioned by CSI.
2. Add UT cases.
Signed-off-by: Xun Jiang <jxun@vmware.com>
"EnableAPIGroupVersions" is set
The crd-remap-version plugin will always backup v1b1 resource for some
CRD. It impacts the feature flag `EnableAPIGroupVersions` which means to
backup all versions, and make migration fail.
In this commit the featureSet was removed from plugin server struct b/c
it blocks the parm `--features` to be populated correctly. This change
should not have negative impact b/c the attribute in server struct is never used.
Fixes#5146
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Pass in a new copy of the map of config values rather than
modifying the BSL Spec.Config and then pass in that field.
Signed-off-by: Scott Seago <sseago@redhat.com>
Fix bsl validation bug: the BSL is validated continually and doesn't respect the validation period configured
Fixes#5056
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
Update the release steps to reflect the change in the `tag-release.sh`,
that the release branch must be created manually before RC is tagged.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. Use patch rather status patch in backup sync controller as we have disable status as sub resource
2. Set the GC period with default value if it isn't set
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
It's not necessary to set the deletion policy as the delete item action
plugin in CSI plugin will set it to Delete when the backup is deleted.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
When enabling the status as sub resource in CRD, the status will be ignored when creating the CR with status, this will cause issues when syncing backups/pvbs
Fixes#4950
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
We have made a few changes to the CSI plugin to provide official support
for AWS/Azure. This commit makes change to the docs to reflect those
changes.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Add filter functions for PeriodicalEnqueueSource.
Move BSL's valication frequency check test case to PeriodicalEnqueueSource's test.
Signed-off-by: Xun Jiang <jxun@vmware.com>
Make in-progress PVB/PVR as failed when restic controller restarts to avoid hanging backup/restore
Fixes#4772
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
1. Add checkpoint in snapshot E2E test to verify snapshot CR should be created and snapshot should be created in cloud side after backup completion;
2. Fix snapshot name issue that CSI snapshot name in cloud side is not the same with other non-CSI cloud snapshots;
Signed-off-by: danfengl <danfengl@vmware.com>
When iterating over applicable restore actions, if a non-matching label
selector is found, velero should continue to the next action rather than
returning from the restoreItem func, which ends up preventing the item's
restore entirely.
Signed-off-by: Scott Seago <sseago@redhat.com>
1. Delete VolumeSnapshot directly when DeletionPolicy set to Retain.
2. Change VolumeSnapshotContent's DeletionPolicy to Retain, then delete VolumeSnapshot. After that delete VolumeSnapshotContent and change VSC DeletionPolicy to Delete back, then re-create the VolumeSnapshotContent.
Signed-off-by: Xun Jiang <jxun@vmware.com>
This commit makes backup sync controller delete the volumesnapshot and
volumesnapshotcontent created by the backup which is cleaned up as orphan
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Make in-progress backup/restore as failed when doing the reconcile to avoid hanging in in-progress status
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
Fixes#4760
This commit make changes in 2 parts:
1) When a volumesnapshotcontent is persisted during backup, velero will reset its
`Source` field to remove the VolumeHandle, so that the
csi-snapshotter will not try to call `CreateSnapshot` when its synced
to another cluster with a backup.
2) Make sure the referenced volumesnapshotclasses are persisted and
synced with the backup, so that when the volumesnapshotcontent is
deleted the storage snapshot is also removed.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. Add --insecure-tls for ResticManager's commands.
2. Add --insecure-tls in PodVolumeBackup and PodVolumeRestore controller.
3. Upgrade integrated Restic version to v0.13.1
4. Change --last flag in Restic command to --latest=1 due to Restic version update.
Signed-off-by: Xun Jiang <jxun@vmware.com>
As we are refactoring controllers with kubebuilder, use the controller-gen rather than code-generator to generate the deep copy methods for objects
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
- go install cmd/velero/velero.go
- go install cmd/velero-restic-restore-helper/velero-restic-restore-helper.go
Will generate binary in `$(go env GOPATH)/bin/` with the correct name.
build.sh still works the same.
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
* Add bsl related TTL gc errors to labelSelectors
* if backup label map is nil, make map
* clear label if not BSL error
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
This allows a user inspecting the restore logs to see any
errors or warnings generated by the restore so that they
will be seen even without having to use the describe cli.
Signed-off-by: Scott Seago <sseago@redhat.com>
* Add plugin versioning design doc
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Use more generic versions in scenarios section
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Address code review
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Address code review
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Modify design to allow other interface changes
The previous design assumed that only method addition would be
supported. It now includes guidance for making changes such as method
removal or signature changes.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
Co-authored-by: Bridget McErlean <bmcerlean@vmware.com>
The GINKGO_SKIP option is updated to string that can be separated by "." for "make test-e2e".
Signed-off-by: Xun Jiang <jxun@vmware.com>
Signed-off-by: Hoang, Phuong <phuong.n.hoang@dell.com>
1. Mark the BSL as "Unavailable" when gets any error
2. Add a new field "Message" to the BSL status to record the error message
Fixes#4485Fixes#4405
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
When velero is running on clusters that don't support v1beta1 CRD, the
plugin will not try to backup v1beta1 CRD.
The plugin should be kept for backward compatibility. It will be
removed when velero drop the support for k8s v1.21
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. rename zoneSeparator to gkeZoneSeparator
2. add example of regional PV's node affinity. modify test case description.
Signed-off-by: Xun Jiang <jxun@vmware.com>
Specify the risk of this parameter set to true. Add the issue first reported about this topic which includeds the google document illustrates about it.
Signed-off-by: Xun Jiang <jxun@vmware.com>
Test case description is "Deleted backups are deleted from object storage and backups deleted from object storage can be deleted locally",
in this test, only resource backup objects are target for verifition, restic repo verification is not included in this PR, and snapshot verification will be in later PR
Signed-off-by: danfengl <danfengl@vmware.com>
Fix#4499
When hook influnce multiple pods, current logic's first pod's container will overwrite the hook's exec.container parameter. That will cause the other pod fail on the hook executing.
Signed-off-by: Xun Jiang <jxun@vmware.com>
By now, only busybox:latest is used by e2e. It is already upload to gcr.io/velero-gcp/busybox:latest
Change the image to gcr.io to avoid pulling rate limitation from docker hub.
Signed-off-by: Xun Jiang <jxun@vmware.com>
Push to GCR in github workflow to faciliate some environments that have rate limitation to docker hub, e.g. vSphere.
<root@jxun-jumpserver.c.velero-gcp.internal>
Signed-off-by: Xun Jiang <jxun@vmware.com>
Since Itemsnapshotter plugin is still WIP,
this commit removes the reference and the deprecation of volumeSnapshotter plugin
from the doc to avoid confusion.
We'll update the doc when it's ready and we have a reference
implementation.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
* Use OrderedResources in schedules
Make ParseOrderedResources public for use in schedules
Add changelog
Signed-off-by: Dominic <dominic@xdnx.org>
* Rename function in comment section
Signed-off-by: Dominic <dominic@xdnx.org>
* #4067 Initial design of the new plugins - pre-post backup and restore
Signed-off-by: Rafael Brito <rbrito@vmware.com>
* Update new-prepost-backuprestore-plugin-hooks.md
* Updated design doc as per feedback
Signed-off-by: Rafael Brito <rbrito@vmware.com>
* Adding design changes as per feedback
* Update design on prepost-backup-restore plugins
* More color on how to call plugins
Signed-off-by: Rafael Brito <rbrito@vmware.com>
* Proposing annotations to skip plugin execution
Signed-off-by: Rafael Brito <rbrito@vmware.com>
We introduces the installation option "--default-restic-prune-frequency" to make restic prune frequency configuration in the previous release, but there is a bug that make the option don't take effect. This commit fixes the bug by removing the evaluation part. The restic repository controller will take care the prune frequency for the repository
Fixes#3062
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
Check the existence of the expected service when ignoring the NodePort already allocated error
Fixes 2308
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
Test case description is "Deleted backups are deleted from object storage and backups deleted from object storage can be deleted locally",
in this test, only resource backup objects are target for verifition, restic repo verification is not included in this PR, and snapshot verification will be in later PR
Signed-off-by: danfengl <danfengl@vmware.com>
* Migrate backup sync controller from code-generator to kubebuilder
1. use kubebuilder's reconcile logic to replace controller's old logic.
2. use ginkgo and gomega to replace testing.
Signed-off-by: Xun Jiang <jxun@vmware.com>
* Fix: modify code according to comments
1. Remove DefaultBackupLocation
2. Remove unneccessary comment line
3. Add syncPeriod default value setting logic
4. Modify ListBackupStorageLocations function's context parameter
5. Add RequeueAfter parameter in Reconcile function return value
Signed-off-by: Xun Jiang <jxun@vmware.com>
* Reconcile function use context passed from parameter
1. Use context passed from parameter, instead of using Reconciler struct's context.
2. Delete Reconciler struct's context member.
3. Modify test case accordingly.
Signed-off-by: Xun Jiang <jxun@vmware.com>
* Remove backups and restic repos associated with deleted BSL(s)
Signed-off-by: F. Gold <fgold@vmware.com>
* add changelog
Signed-off-by: F. Gold <fgold@vmware.com>
* Add PR number to changelog
Signed-off-by: F. Gold <fgold@vmware.com>
* Fix typo
Signed-off-by: F. Gold <fgold@vmware.com>
* Only delete backups and restic repos and report success when without errors
Signed-off-by: F. Gold <fgold@vmware.com>
* Adds <backup-name>-itemsnapshots.gz file to backup (when provided). Also
adds DownloadTargetKindBackupItemSnapshots type to allow downloading.
Updated object store unit test
Fixes#3758
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Removed redundant checks
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Consolidated code for resolving actions and plugins into ActionResolver. Added BackupWithResolvers and
RestoreWithResolvers. Introduces ItemSnapshooterResolver to bring ItemSnapshotter plugins into backup and
restore. ItemSnapshotters are not used yet.
Added action_resolver_test
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Addressed review comments
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
This commit adds a restore action item plugin to reset invalid value
of "sideEffects" in resource of mutatingwebhookconfiguration and
validating webhookconfiguration.
To fix the problem the "sideEffects" is illegal for resource migrated
from v1beta1.
fixes#3516
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. remove config/crd/v1beta1
2. remove PROJECT file
3. update controller-gen and kubebuilder version
4. generate client and CRD file
5. add changelog and remove v1beta1 CRD generated code.
6. add kubebuilder test bundle setup command.
7. due to apiextensions.k8s.io/v1beta1 is not supported, only k8s after v1.16 is supported, so remove v1.15 check.
8. add CRD and k8s suppored version update in changelog.
Signed-off-by: Xun Jiang <jxun@vmware.com>
* fix: modify generated from schedule's backup name timestamp to UTC timezone
fix#4279
When backup is created from schedule, and the backup name is not specified, a containing-timestamp generated name will be used. Due to velero client not set timezone to UTC, a local timezone will be used for the generated name.
Signed-off-by: Xun Jiang <jxun@vmware.com>
* fix: modify generated from schedule's backup name timestamp to UTC timezone
fix#4279
When backup is created from schedule, and the backup name is not specified, a containing-timestamp generated name will be used. Due to velero client not set timezone to UTC, a local timezone will be used for the generated name.
Signed-off-by: Xun Jiang <jxun@vmware.com>
* fix: modify generated from schedule's backup name timestamp to UTC timezone
fix#4279
When backup is created from schedule, and the backup name is not specified, a containing-timestamp generated name will be used. Due to velero client not set timezone to UTC, a local timezone will be used for the generated name.
Signed-off-by: Xun Jiang <jxun@vmware.com>
* modify changelog description
Reword the changelog description according to comments.
Signed-off-by: Xun Jiang <jxun@vmware.com>
Co-authored-by: jxun <jxun@jxun-a01.vmware.com>
Co-authored-by: Xun Jiang <jxun@vmware.com>
logrusr is a open source convertor, which can convert logrus logger into logr.
By using logrusr, velero can use exsiting formatted logrus logger, other than introducing zap as a new logger.
Signed-off-by: Xun Jiang <jxun@vmware.com>
Added ItemSnapshotter.proto
Added item_snapshotter Go interface
Added framework components for item_snapshotter
Updated plugins doc with ItemSnapshotter info
Added SnapshotPhase to item_snapshotter.go
ProgressOutputOutput now includes a phase as well as an error string for problems that occured
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Update EnableAPIGroupVersion feature design doc as implemented
Signed-off-by: F. Gold <fgold@vmware.com>
* Design doc for issue 2082 to delete associated resources when deleting BSLs
Signed-off-by: F. Gold <fgold@vmware.com>
* Changes per @dsu-igeek review comments
Signed-off-by: F. Gold <fgold@vmware.com>
The error should be returned explicitly, because when the default URL is
used S3 will return a 301 and the response can't be handled by restic.
Fixes#4178
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
When the snapshot uploading is failed, it should not be treat as completed and continue.
This commit covers both the phases of in progress and failed when uploading snapshot with vSphere plugin
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
Previously, the BSL credential field would always be set when using the
`create` command, even if no credential details were provided. This
would result in an empty `SecretKeySelector` in the BSL which would
cause operations using this BSL to fail as Velero would attempt to fetch
a `Secret` with an empty name from the K8s API server.
With this change, the `Credential` field is only set if credential
details have been specified. This change also includes some refactoring
to allow the change to be tested.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
fix paging items in to use list options passed by the paging function
The client-go pager sets the Limit options for the list call
to paginate the request[1]. This PR fixes the paging function
to use the options passed by the pager instead of shadowed options
This is required for the pagination to work correctly.
- simplify the pager list implementation by using pager.List()
The List() function already implements a lot of the logic that was
needed for paging here, using it simplifies the code.
1. 3f40906dd8/staging/src/k8s.io/client-go/tools/pager/pager.go (L219)
Signed-off-by: Alay Patel <alay1431@gmail.com>
Bump up restic to v0.12.1 to fix CVE-2020-26160.
Bump up module "github.com/vmware-tanzu/crash-diagnostics" to v0.3.7 to fix CVE-2020-29652.
The "github.com/vmware-tanzu/crash-diagnostics" updates client-go to v0.22.2 which introduces several break changes, this commit updates the related codes as well
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
After the PR to implement `velero debug` - #4022 is reviewed, there are some
suggestion to let the command collect more resources, this commit make
the change to the design doc to reflect those changes.
It also remove some sections that are no longer relevant after `crashd`
has made enhancement in the v0.3.4 release.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
* #4040 - documentation - adding more troubleshooting information during Restic restore
Signed-off-by: Rafael Brito <rbrito@vmware.com>
* #4040 - documentation - adding more troubleshooting information during Restic restore and minor changes
Signed-off-by: Rafael Brito <rbrito@vmware.com>
* #4040 - documentation - tweaks on restic page
Signed-off-by: Rafael Brito <rbrito@vmware.com>
If the "--snapshot-volumes=false" isn't specified explicitly, the vSphere plugin will always take snapshots for the volumes even though the "--default-volumes-to-restic" is specified
This can be removed if the logic of vSphere plugin changes
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
This commit makes several changes to `tag-release.sh` according to the
change in release process:
1. It will support a "ON_RELEASE_BRANCH" param passed via env variable.
When it's set to "TRUE". The release will be created on the commit of
branch like `release-xxx`. This enables us to create release branch
before GA and tag RC release.
2. It removes the code to push a new branch to upstream. This is
because we decided to create branch manually. For patch releases, we
will not push the change to release branch, instead, we will make
sure the release branch has all commits cherrypicked BEFORE we run
this script to tag the release.
After the change the script will focus on only tag the release, not
making other code change to release branches.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
In upgrade test, both original and to-be-upgrading velero installation should use the compatible plugins, but currently, plugin value is determined by provider.
Signed-off-by: danfengl <danfengl@vmware.com>
The errors of restore/backup may be stored in object storage
The well formatted output of describe is also helpful for debugging.
This commit add the command to the crashd script so the output of
"velero backup/restore describe xxx" can be collected
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commits updates the `troubleshooting` section in the doc to ask
users to collect log via `velero debug`.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
* Add namespace validation in the client
Signed-off-by: F. Gold <fgold@vmware.com>
* Add namespace validation in the backup controller
Signed-off-by: F. Gold <fgold@vmware.com>
* Add changelog for PR 4057
Signed-off-by: F. Gold <fgold@vmware.com>
* Update Copyright notice
Signed-off-by: F. Gold <fgold@vmware.com>
* Update include_excludes_test.go to follow Go standards and be easier to read
Signed-off-by: F. Gold <fgold@vmware.com>
* Add unit tests for namespace validation functions
Signed-off-by: F. Gold <fgold@vmware.com>
* Make changes per review comments
- use one set of namespace validation logic instead of writing two
- remove duplicate namespace validation functions and tests
- add namespace validation tests in includes_excludes_test.go
Signed-off-by: F. Gold <fgold@vmware.com>
* Return all ns validation err msgs as error list
Signed-off-by: F. Gold <fgold@vmware.com>
* Make error message more clear
Signed-off-by: F. Gold <fgold@vmware.com>
Velero was including DownwardAPI volumes when backing up with restic.
When restoring these volumes, it triggered a known issue with restic (as
seen in #3863). Like projected volumes, these volumes should be skipped
as their contents are populated by the Kubernetes API server.
With this change, we are now skipping the restic backup of volumes with
a DownwardAPI source. We are also skipping the restore of any volume
that had a DownwardAPI source as there will exist backups that were
taken prior to this fix being introduced. This will allow these backups
to be restored succesfully.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
This commit removes `IsUnstructuredCRDReady` since
kubernetes/kubernetes#87675 is fixed.
Is uses `Is1CRDReady` to check the readiness of CRD.
After v1.7 we may consider merge the funcx `IsV1Beta1CRDReady` and
`IsV1CRDReady`
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. Support to customize the restic restore helper image
2. Use a seperated context when doing the clean up works
3. Wait a while before doing the the restore for aws to avoid #1799
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
1. Check the error when waiting for restice daemonset to be ready, so
the timeout will be reported
2. Add support for gcp provider and fail early if the provider is
unknown
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This PR added a subcommand `velero debug`, which leverages `crashd` to
collect logs and specs of velero server components and bundle them in a
tarball.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Do this for two reasons:
1. Verify the functionalities for installation and uninstllation of CLI
2. We want to add upgrade test case which needs to install different versions of velero, calling libraries is impossible for this
fixes#4062
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
* Add document for TLS error 116
When using a custom S3 compatible server, backups/restore may fail with
TLS error 116. This happens because the S3 server expects Velero to
send client certificate during SSL TLS v1.3 handshake.
You will need to modify your S3 server settings to turn off client
certificate authentication.
Signed-off-by: Himanshu Mehra <himanshu.mehra91@gmail.com>
* Add document for TLS error 116
When using a custom S3 compatible server, backups/restore may fail with
TLS error 116. This happens because the S3 server expects Velero to
send client certificate during SSL TLS v1.3 handshake.
You will need to modify your S3 server settings to turn off client
certificate authentication.
Signed-off-by: Himanshu Mehra <himanshu.mehra91@gmail.com>
* Address comments from reviewers
Signed-off-by: Himanshu Mehra <himanshu.mehra91@gmail.com>
Wait the namespace deletion completed before removing the CRDs when uninstalling the velero
Fixes#3974
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
a timestamp. If two requests were happening very close together for the
same backup, the second would fail randomly.
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
Fix the random failure by increasing the timeout and introducing few minor refactor/bug fixes
Fixes#3970
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
Instead of converting the unstructured item to check for the presence of
the `kube-aggregator.kubernetes.io/automanaged` label, use this label in
the `AppliesTo` to enable the restore logic to select the item. This
means that any item that matches the selector will have restore skipped.
Also add a new test case to the restore action test to check that label
selectors are applied correctly.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
It was discovered during Velero 1.6.3 upgrade testing that Velero was
restoring `APIService` objects for APIs that are no longer being served
by Kubernetes 1.22. If these items were restored, it would break the
behaviour of discovery within the cluster.
This change introduces a new RestoreItemAction plugin that skips the
restore of any `APIService` object which is managed by Kubernetes such
as those for built-in APIs or CRDs. The `APIService`s for these will be
created when the Kubernetes API server starts or when new CRDs are
registered. These objects are identified by looking for the
`kube-aggregator.kubernetes.io/automanaged` label.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Use appropriate CRD API during readiness check
The readiness check for the Velero CRDs was still using the v1beta1 API.
This would cause the readiness check to fail on 1.22 clusters as the
v1beta1 API is no longer available. Previously, this error would be
ignored and the installation would proceed, however with #4002, we are
no longer ignoring errors from this check.
This change modifies the CRD readiness check to check the CRDs using the
same API version that was used when submitting the CRDs to the cluster.
It also introduces a new CRD builder using the V1 API for testing.
This change also fixes a bug that was identified in the polling code
where if the CRDs were not ready on the first polling iteration, they
would be added again to the list of CRDs to check resulting in
duplicates. This would cause the length check to fail on all subsequent
polls and the timeout would always be reached.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Remove duplicate V1 CRD builder and update comment
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
Add the image pull secret to the service account when deploying velero and kibishii to avoid the image pull limit issue of Docker Hub
Fixes#3966
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
The backup name must be no more than 63 characters otherwise we'll get error on vSphere platform:
Failed to create snapshot record: Snapshot.backupdriver.cnsdp.vmware.com \"snap-8945e7df-069e-4f56-aeb5-75b1dd87547f\" is invalid: metadata.labels: Invalid value: \"backup-bsl-e7a1d0f3-2f29-4d80-9184-6214dac91d96-e7a1d0f3-2f29-4d80-9184-6214dac91d96\": must be no more than 63 characters"
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
If the Velero CLI can't discover the Kubernetes preferred CRD API
version, use the flag --crds-version to determine the CRDs version.
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
As we add more E2E test cases, this'll cause the job takes a lot of time before checking pass for the pull requests, this commit changes the test cases(only basic cases) runs for PR
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
Generate test report for the E2E testing so that we can check the test result in the automation pipelines easily
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
This commit add `Enhancement/User` as an exempt label such that issues
like #3772 won't be closed by the stale bot.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
* Add the design for `velero debug`
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
* Add namespace for capturing `velero version`
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
The `push-build-image` target was broken in #3634. The `ifneq`
conditional block had tabs for indentation which results in incorrect
behaviour. Instead, remove whitespace before the conditional block like
we do for other similar blocks.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
This adds a new `buildinfo` variable `ImageRegistry` that can set at
build time like the `Version` variable. This allows us to customise the
Velero binary to use different registries.
If the variable is set, this variable wille be used when creating the
URIs for both the main `velero` and `velero-restic-restore-helper`
images. If it is not set, default to using Dockerhub (`velero/velero`,
`velero/velero-restic-restore-helper`).
There are numerous ways in which the Velero binary can be built so all
of them have been updated to add the new link time flag to set the
variable:
* `make local` (used for local developer builds to build for the local
OS and ARCH)
* `make build` (used by developers and also VMware internal builds to
build a specific OS and ARCH)
* Goreleaser config (used when creating OSS release binaries)
* Dockerfile (used to build the Velero binary used within the image)
All of these workflows are currently triggered from our Makefile where
the variable `REGISTRY` is already available with the default value of
`velero` and used to build the image tag. Where the new `ImageRegistry`
build variable is needed, we pass through this Makefile variable to
those tasks so it can be used accordingly.
The GitHub action and the `./hack/docker-push.sh` script used to push
container images has not been modified. This will continue to use the
default registry specified in the Makefile and will not explicitly pass
it in.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
Wenkai Yin recently joined the Velero team within VMware. He has been
contributing to the technical health of Velero, introducing important
changes such as running our E2E tests as part of our PR checks and will
continue to focus in this area.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
Daniel Jiang recently joined the Velero team within VMware and will be
taking on a technical leadership role. He has been contributing to the
project through community engagement including issue triage and
community support, and is taking on more significant feature development
within Velero such as the design and development of the `velero debug`
feature.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
1. Run the E2E test with kind(provision various versions of k8s cluster) and MinIO on Github Action
2. Bug fix: the variable "stdoutBuf" is assigned to both "installPluginCmd.Stdout" and "installPluginCmd.Stderr", this causes 'if !strings.Contains(stderrBuf.String(), "Duplicate value")' takes no effect as the "stderrBuf.String()" is always empty
3. Print the stdout and stderr for easy debugging
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
In #3863, it was discovered that volumes from projected sources were
being backed up by restic when they should have been skipped. Restoring
these volumes triggers a known bug in restic.
In #3866, we started skipping volumes from a projected source, however
there will exist backups that were taken before this fix was introduced.
This change modifies the restore logic to skip the restore of any volume
that came from a projected source, allowing backups taken before #3866
to be restored successfully.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Remove controllers and sleeps in API groups e2e tests
Signed-off-by: F. Gold <fgold@vmware.com>
* Print command in AfterEach(...) and check error
Signed-off-by: F. Gold <fgold@vmware.com>
* Make change ahead of PR3764 changes in main
Signed-off-by: F. Gold <fgold@vmware.com>
* Update go.{mod,sum} files
Signed-off-by: F. Gold <fgold@vmware.com>
* Run make update
Signed-off-by: F. Gold <fgold@vmware.com>
* Add document describing manual test cases
This introduces a new document, `TESTING.md`, which describes manual
tests that are currently run as part of a Velero release and test cases
that we will want to introduce for future releases.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Move testing requirements doc to website
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
phases as part of Upload Progress Monitoring, fixes#3755 Add backup phases
needed for Upload Progress Monitoring
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
Previously `WithPlugins` only supported passing image URIs "by tag" --
e.g. `gcr.io/my-repo/my-image:v0.1.2`. With this commit, we add support
for pulling "by digest" -- e.g.
`gcr.io/my-repo/my-image@sha256:a75f9e8c3ced3943515f249597be389f8233e1258d289b11184796edceaa7dab`
Signed-off-by: Eric Fried <efried@redhat.com>
* use unstructured to marshal selective fields
Signed-off-by: Alay Patel <alay1431@gmail.com>
* add a sample test for string port in applied config
Signed-off-by: Alay Patel <alay1431@gmail.com>
* update changelog
Signed-off-by: Alay Patel <alay1431@gmail.com>
* Fix gh action
* Fix it maybe
* Update GH action version
* Set write permission for the job
* Use target
* Remove config that is already default
Signed-off-by: Carlisia <carlisia@grokkingtech.io>
This change is incompatible with velero-plugin-for-csi
releases <= v0.1.2
Remove special casing of CSI volumesnapshot artifacts
from backup deletion logic as this has been moved to
a DeleteItemAction plugin in the velero-plugin-for-csi repo
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
Change to add new plugin SnapshotItemAction, added started/updated fields to UploadProgress
Updated SnapshotItemAction, added additional tasks
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Changes to secrets design
Removed references to Volume Storage Locations/VSLs
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Description of current parallelism points
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
For internal builds of Velero, we need to be able to specify an
alternative Dockerfile which uses an alternative image registry to pull
the base images from. This change adapts our Makefile such that both the
main Dockerfile and build image Dockerfile can be overridden.
We have some special handling for the build image to only build when the
Dockerfile has changed. In this case, we check whether a custom
Dockerfile has been provided, and always rebuild in that case. For
custom build image Dockerfiles, use a fixed tag rather than the one
based on commit SHA of the original file.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
Snapshot tests can be run with Ginkgo focus "Snapshot" and restic tests with Ginkgo focus "Restic".
Restic and volume snapshot tests can now be run simultaneously.
Added check for kibishii app start after restore.
Consolidated kibishii pod checks into waitForKibishiiPods.
Added WaitForPods function to e2e/tests/common.goSnapshot tests are skipped automatically on kind clusters.
Fixed issue where velero_utils InstallVeleroServer was looking for the Restic daemon set in the "velero" namespace only (was ignoring io.Namespace)
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Improve readbility and formatting of pkg/restore/restore.go
Signed-off-by: F. Gold <fgold@vmware.com>
* Update paths to include API group versions
Signed-off-by: F. Gold <fgold@vmware.com>
* Use full word, 'resource' instead of 'resrc'
Signed-off-by: F. Gold <fgold@vmware.com>
The test for multiple credentials assumed that the plugin for the
additional BSL provider was already installed. This will not be the case
when performing a clean install of Velero between tests.
This adds a new utility function to add the plugins that are necessary
for the additional BSL provider. It doesn't check which plugins are
already installed, it will just attempt to install and if the stderr
contains the message that it is a duplicate plugin, we ignore the error
and continue. This could be improved by instpecting the output from
`velero plugin get` but I opted for a quicker solution given the
upcoming release.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
Our previous render hook to create links would drop the fragment when
linking to headings within the current page or within other markdown
pages on the site.
This change parses the URL and formats the link correctly if it includes
a fragment. If the link is a header on the current page, it is rendered
as `http://<current-url>/#header`. If the link is a header on a
different page (e.g. page.md#header), it is rendered as
`http://<page-url>/#header`.
This change is taken from the following Hugo community support post:
https://discourse.gohugo.io/t/markdown-render-hooks-github-and-hugo-compatible-links/22543/14
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Use Credential from BSL for restic commands
This change introduces support for restic to make use of per-BSL
credentials. It makes use of the `credentials.FileStore` introduced in
PR #3442 to write the BSL credentials to disk. To support per-BSL
credentials for restic, the environment for the restic commands needs to
be modified for each provider to ensure that the credentials are
provided via the correct provider specific environment variables.
This change introduces a new function `restic.CmdEnv` to check the BSL
provider and create the correct mapping of environment variables for
each provider.
Previously, AWS and GCP could rely on the environment variables in the
Velero deployments to obtain the credentials file, but now these
environment variables need to be set with the path to the serialized
credentials file if a credential is set on the BSL.
For Azure, the credentials file in the environment was loaded and parsed
to set the environment variables for restic. Now, we check if the BSL
has a credential, and if it does, load and parse that file instead.
This change also introduces a few other small improvements. Now that we
are fetching the BSL to check for the `Credential` field, we can use the
BSL directly to get the `CACert` which means that we can remove the
`GetCACert` function. Also, now that we have a way to serialize secrets
to disk, we can use the `credentials.FileStore` to get a temp file for
the restic repo password and remove the `restic.TempCredentialsFile`
function.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Add documentation for per-BSL credentials
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Address review feedback
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Address review comments
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
We are no longer adding the Credentials field to the VSL so this reverts
part the change that added it (#3409).
The original PR also added the `snapshot-location set` command. This
command only included options for setting the credential but is part of
the work for #2426. Due to this, the command has been left in place
(with the credentials option removed) but has been hidden.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
This change adds an E2E test which exercises the mulitple credentials
feature added in #3489. The test creates a secret from the given
credentials and creates a BackupStorageLocation which uses those
credentials. A backup and restore is then performed to the default
BSL and to the newly created BSL.
This change adds new flags to the E2E test suite to configure the BSL
created and used in the test.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Load credentials and pass to ObjectStorage plugins
Update NewObjectBackupStore to take a CredentialsGetter which can be
used to get the credentials for a BackupStorageLocation if it has been
configured with a Credential. If the BSL has a credential, use that
SecretKeySelector to fetch the secret, write the contents to a temp file
and then pass that file through to the plugin via the config map using
the key `credentialsFile`. This relies on the plugin being able to use
this new config field.
This does not yet handle VolumeSnapshotLocations or ResticRepositories.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Address code reviews
Add godocs and comments.
Improve formatting and test names.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Address code reviews
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Add uninstall cmd
- init fn to uninstall velero
- abstract dynamic client creation to a separate fn
- creates a separate client per unstructured resource
- add delete client for CRDs
- export appendUnstructured
- add uninstall command to main cmd
- export `podTemplateOption`
- uninstall resources in the reverse order of installation
- fallback to `velero` if no ns is provided during uninstall
- skip deletion if the resource doesn't exist
- handle resource not found error
- match log formatting with cli install logs
- add Delete fn to fake client
- fix import order
- add changelog
- add comment doc for CreateClient fn
Signed-off-by: Suraj Banakar <suraj@infracloud.io>
* Re-use uninstall code from test suite
- move helper functions out of test suite
- this is to prevent cyclic imports
- move uninstall helpers to uninstall cmd
- call them from test suite
- revert export of variables/fns from install code
- because not required anymore
Signed-off-by: Suraj Banakar <suraj@infracloud.io>
* Revert `PodTemplateOption` -> `podTemplateOption`
Signed-off-by: Suraj Banakar <suraj@infracloud.io>
* Use uninstall helper under VeleroUninstall
- as a wrapper
- fix import related errors in test suite
Signed-off-by: Suraj Banakar <suraj@infracloud.io>
* Validate CRDs against latest Kubernetes versions
Add Kubernetes v1.19 and v1.20 series images, and consolidate the job
into a single file to reduce repetition.
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Ignore job if the changes are only site/design
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Fix codespell error
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Cache Velero binary for reuse on workers
This will cache the Velero binary based on the PR number and a SHA256 of
the generated binary.
This way, the runners testing each version of Kubernetes do not need to
build it independently.
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Fix GitHub event access
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Wrap output path in quotes
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Move code checkout to build step
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Also cache go modules
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Fix syntax issues
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Download cached binary on each node
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Use cached go modules on main CI
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
HTTPS requests were failing due to the ca-certificates package not being
installed in the Tilt image.
This change takes the command to install this package from our main
Dockerfile (which also includes installing tzdata).
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Use pod namespace from backup when matching PVBs
In #3051, we introduced an additional check to ensure that a PVB matched
a particular pod by checking both the name and the namespace of the pod.
This caused an issue when using a namespace mapping on restore. In the
case where a namespace mapping is being used, the check for whether a
PVB matches a particular pod will fail as the PVB was created for the
original pod namespace and is not aware of the new namespace mapping
being used. This resulted in PVRs not being created for pods that were
being restored into new namespaces. The restic init containers were
being created to wait on the volume restore, however this would cause
the restored pods to block indefinitely as they would be waiting for a
volume restore that was not scheduled.
To fix this, we use the original namespace of the pod from the backup to
match the PVB to the pod being restored, not the new namespace where
the pod is being restored into.
Fixes#3467.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Explain why the namespace mapping can't be used
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
Split plug-in provider into cloud provider/object provider
Moved velero install/uninstall for tests into velero_utils
Added remove of CRDs to test v elero uninstall
Added remove of cluster role binding to test velero uninstall
Added dump of velero describe and logs on error
Added velero namespace argument to velero_utils functions
Modified api group versions e2e tests to use VeleroInstall
Added velero logs dumps for api group versions e2e testing
Added DeleteNamespace to test/e2e/common.go
Fixed VeleroInstall to use the image specified
Changed enable_api_group_versions_test to use veleroNamespace instead of hardcoded "velero"
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Use kubebuilder client for fetching restic secrets
Instead of using a SecretInformer for fetching secrets for restic, use
the cached client provided by the controller-runtime manager.
In order to use this client, the scheme for Secrets must be added to the
scheme used by the manager so this is added when creating the manager in
both the velero and restic servers.
This change also refactors some of the tests to add a shared utility for
creating a fake controller-runtime client which is now used among all
tests which use that client. This has been added to ensure that all
tests use the same client with the same scheme.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Add builder for SecretKeySelector
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Restore API group version by priority
Signed-off-by: F. Gold <fgold@vmware.com>
* Add changelog
Signed-off-by: F. Gold <fgold@vmware.com>
* Correct spelling
Signed-off-by: F. Gold <fgold@vmware.com>
* Refactor userResourceGroupVersionPriorities(...) to accept config map, adjust unit test
Signed-off-by: F. Gold <fgold@vmware.com>
* Move some unit tests into e2e
Signed-off-by: F. Gold <fgold@vmware.com>
* Add three e2e tests using Testify Suites
Summary of changes
Makefile - add testify e2e test target
go.sum - changed with go mod tidy
pkg/install/install.go - increased polling timeout
test/e2e/restore_priority_group_test.go - deleted
test/e2e/restore_test.go - deleted
test/e2e/velero_utils.go - made restic optional in velero install
test/e2e_testify/Makefile - makefile for testify e2e tests
test/e2e_testify/README.md - example command for running tests
test/e2e_testify/common_test.go - helper functions
test/e2e_testify/e2e_suite_test.go - prepare for tests and run
test/e2e_testify/restore_priority_apigv_test.go - test cases
Signed-off-by: F. Gold <fgold@vmware.com>
* Make changes per @nrb code review
Signed-off-by: F. Gold <fgold@vmware.com>
* Wait for pods in e2e tests
Signed-off-by: F. Gold <fgold@vmware.com>
* Remove testify suites e2e scaffolding moved to PR #3354
Signed-off-by: F. Gold <fgold@vmware.com>
* Make changes per @brito-rafa and Velero maintainers code reviews
- Made changes suggested by @brito-rafa in GitHub.
- We had a code review meeting with @carlisia, @dsu-igeek, @zubron, and @nrb
- and changes were made based on their suggetions:
- pull in logic from 'meetsAPIGVResotreReqs()' to restore.go.
- add TODO to remove APIGroupVersionFeatureFlag check
- have feature flag and backup version format checks in separate `if` statements.
- rename variables to be sourceGVs, targetGVs, and userGVs.
Signed-off-by: F. Gold <fgold@vmware.com>
* Convert Testify Suites e2e tests to existing Ginkgo framework
Signed-off-by: F. Gold <fgold@vmware.com>
* Made changes per @zubron PR review
Signed-off-by: F. Gold <fgold@vmware.com>
* Run go mod tidy after resolving go.sum merge conflict
Signed-off-by: F. Gold <fgold@vmware.com>
* Add feature documentation to velero.io site
Signed-off-by: F. Gold <fgold@vmware.com>
* Add config map e2e test; rename e2e test file and name
Signed-off-by: F. Gold <fgold@vmware.com>
* Update go.{mod,sum} files
Signed-off-by: F. Gold <fgold@vmware.com>
* Move CRDs and CRs to testdata folder
Signed-off-by: F. Gold <fgold@vmware.com>
* Fix typos in cert-manager to pass codespell CICD check
Signed-off-by: F. Gold <fgold@vmware.com>
* Make changes per @nrb code review round 2
- make checkAndReadDir function private
- add info level messages when priorties 1-3 API group versions can not be used
Signed-off-by: F. Gold <fgold@vmware.com>
* Make user config map rules less strict
Signed-off-by: F. Gold <fgold@vmware.com>
* Update e2e test image version in example
Signed-off-by: F. Gold <fgold@vmware.com>
* Update case A music-system controller code
Signed-off-by: F. Gold <fgold@vmware.com>
* Documentation updates
Signed-off-by: F. Gold <fgold@vmware.com>
* Update migration case documentation
Signed-off-by: F. Gold <fgold@vmware.com>
* Use label to select Velero deployment in plugin cmd
Signed-off-by: F. Gold <fgold@vmware.com>
* Move veleroLabel constant closer to usage
Signed-off-by: F. Gold <fgold@vmware.com>
* Add changelog
Signed-off-by: F. Gold <fgold@vmware.com>
* Remove year from copyright in new file
Signed-off-by: F. Gold <fgold@vmware.com>
* Export and use install.Labels() function
Signed-off-by: F. Gold <fgold@vmware.com>
Restoring CAPI workload clusters without this ordering caused the
capi-controller-manager code to panic, resulting in an unhealthy cluster
state.
This can be worked around
(https://community.pivotal.io/s/article/5000e00001pJyN41611954332537?language=en_US),
but we provide the inclusion of these resources as a default in order to
provide a better out-of-the-box experience.
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
The labeler action was failing as it was looking for
`.github/labels.yaml` but the file has the suffix `.yml`. This change
fixes the path used by the labeler action.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
The prow-action plugin will pre-pend `area` or `kind` to labels, so
unify them into a common format.
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Add colors to describe command
* Add colors to describe backups/restore/schedules commands
* Make name in the output bold
* Disable colors via `--colorized` flag or if velero isn't in TTY
Co-authored-by: Clay Kauzlaric <ckauzlaric@vmware.com>
Signed-off-by: Clay Kauzlaric <ckauzlaric@vmware.com>
Signed-off-by: Mikael Manukyan <mmanukyan@vmware.com>
* Add changelog
* and run make update
Co-authored-by: Mikael Manukyan <mmanukyan@vmware.com>
Signed-off-by: Mikael Manukyan <mmanukyan@vmware.com>
Signed-off-by: Clay Kauzlaric <ckauzlaric@vmware.com>
* Add colorized to the client config file
Co-authored-by: Mikael Manukyan <mmanukyan@vmware.com>
Signed-off-by: Clay Kauzlaric <ckauzlaric@vmware.com>
Co-authored-by: Mikael Manukyan <mmanukyan@vmware.com>
* allow client config to use string values
* the command `velero client config set colorized=false` writes a string
value of "false" into the config. This change allows that string to be
accepted and converted into a boolean when used in program.
Signed-off-by: Clay Kauzlaric <ckauzlaric@vmware.com>
* Add docs about colored CLI output
Co-authored-by: Mikael Manukyan <mmanukyan@vmware.com>
Signed-off-by: Clay Kauzlaric <ckauzlaric@vmware.com>
* Update site/content/docs/main/customize-installation.md
Co-authored-by: JenTing Hsiao <jenting.hsiao@suse.com>
Signed-off-by: Clay Kauzlaric <ckauzlaric@vmware.com>
* docs: remove comma
* as per @carlisia 's suggestion
Signed-off-by: Clay Kauzlaric <ckauzlaric@vmware.com>
Co-authored-by: Clay Kauzlaric <ckauzlaric@vmware.com>
Co-authored-by: Clay Kauzlaric <clay.kauzlaric@gmail.com>
Co-authored-by: JenTing Hsiao <jenting.hsiao@suse.com>
With #3327, the restic binary for the Tilt Velero image is downloaded on
the local machine using the `./hack/download-restic.sh` script. This
script relies on `wget` being availabe on the local machine. `wget` is
not commonly available on macOS but `curl` is. This change modifies the
`./hack/download-restic.sh` script to use `curl` instead as it is
available on both Linux and macOS and is available in our `golang`
docker build image.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
In preparation for modifying the instantiation of `BackupStores` to be
able to load credentials, change the function `NewObjectBackupStore` to
be an interface that is passed in to all controllers.
Previously, the function to get a new backup store was configurable but
for many controllers was fixed to use `NewObjectBackupStore`. This
change introduces an interface for getting the backup store and wraps
the functionality from `NewObjectBackupStore` in a type which implements
this interface. This will allow more flexibility when introducing
credentials for a specific backup store as it will allow us to create a
new `ObjectBackupStoreGetter` type which can be configured to add
credentials config when creating the ObjectBackupStore without needing
to change the API used by the controllers.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
In #3310, the Dockerfile for the Tilt Velero container was modified to
call the `./hack/download-restic.sh` script. A side effect of this
change was that the context for the docker build was much larger as it
was the root of the Velero repo, rather than just the `_tiltbuild`
directory. With the frequent rebuilds of the image that happen when
using Tilt, a large amounts of disk space was being consumed by the
different layers of images builds in the Docker overlay filesystem (as
diffs could include the `.go` directory which can be several GBs).
This change modifies the `download-restic.sh` script to allow the output
directory for the restic binary to be configured. This means that the
script can be called directly from the Tiltfile and can be managed
outside the container build. This allows us to restore the previous
`_tiltbuild` context. It also speeds up image builds as we can download
restic once and use it for all builds rather than redownloading
frequently.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
now versions are working and there are code changes that need to happen
- release candidate versions are aligned and working
- replaces fields are removed and not required anymore
controller runtime has been changed during the 'make' command
Signed-off-by: Ron Green <11993626+georgettica@users.noreply.github.com>
This change customises the issue template chooser to include a link to
the Community Support Q&A discussion board. This lets users know that
there is another place to ask questions related to using Velero.
This change also disables the creation of blank issues to prevent issues
that don't follow either the bug or feature request templates from being
opened.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Add Tilt configuration to debug using Delve
This change adds support to run the Velero process in Tilt using
[Delve](https://github.com/go-delve/delve).
This does not include support for debugging the Velero process in the
restic pods, just in the Velero deployment.
For an optimal debugging experience, this change also introduces a new
flag (`DEBUG`) to the `hack/build.sh` script to enable a "debug" build
of the Velero binary. This flag, if enabled, will build the binary
without optimisations and inlining. Disabling optimisations and inlining
is recommended by Delve.
Two configuration options have been added to the Tilt settings. The
first, `enable_debug`, is to control whether debugging should be
enabled. If enabled, the process will be started by Delve, and the Delve
server port (2345) will be forwarded to the local machine.
The second option, `debug_continue_on_start`, is to control whether the
process should "continue" when started by Delve or should be paused.
By default, debugging is disabled, and if in debug mode, the process
will continue.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Add spaces around keyword args
Starlark prefers spaces around `=` in keyword arguments:
https://docs.bazel.build/versions/master/skylark/bzl-style.html#keyword-arguments
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Remove unnecessary command from Dockerfile
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Add note to connect after Tilt is running
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
This change adds an additional set of commands to Dockerfile for the
Velero image which adds the `hack/download-restic.sh` script, installs
the necessary dependencies, and then runs that script.
In order to copy the script from the `hack` directory, the context for
building the image has been changed to the root of the velero
repository.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
This changes the codespell action config to use a relative path for the
generated crds.go file as the current pattern used fails the check used
by codespell (which uses the `fnmatch` module).
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* added useOwnerReferencesInBackup to crd velerio.io_schedules
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* added UseOwnerReferencesInBackup property to schedule.go
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* deepcopy schedule configured for reference the property UseOwnerReferencesInBackup
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* added UseOwnerReferencesInBackup property verification to modify OwnerReferences from backup
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* created changelog
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* removed deepcopy schedule configured for reference the property UseOwnerReferencesInBackup
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* running make update
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* running make update
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* updated the year at the top of the schedule.go file for 2020
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* -> Preserve nodePort support when restoring via "--preserve-nodeports" flag
Signed-off-by: Yusuf Güngör <yusuf.gungor@hepsiburada.com>
* -> Added changelog.
Signed-off-by: Yusuf Güngör <yusuf.gungor@hepsiburada.com>
* -> Unit test added.
-> Using boolptr.IsSetToTrue for bool ptr check.
Signed-off-by: Yusuf Güngör <yusuf.gungor@hepsiburada.com>
* -> Unit test added.
-> Using boolptr.IsSetToTrue for bool ptr check.
Signed-off-by: Yusuf Güngör <yusuf.gungor@hepsiburada.com>
* -> Other restore errors log level changed from info to error.
-> Documentation updated about Velero nodePort restore logic and preservation of them.
Signed-off-by: Yusuf Güngör <yusuf.gungor@hepsiburada.com>
Co-authored-by: Yusuf Güngör <yusuf.gungor@hepsiburada.com>
As long as a milestone and the board have the same title, then this
workflow should take care of adding an issue into the GitHub Project
board when an existing issue is given a milestone.
It does NOT support checking for a milestone when an issue is edited or
created though, due to limitations on GitHub Actions syntax right now -
there's not a great way to validate against an empty `milestone` object
at the moment, per https://docs.github.com/en/free-pro-team@latest/actions/reference/context-and-expression-syntax-for-github-actions
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Draft design doc for restoring API group version by priority level
Signed-off-by: F. Gold <fgold@vmware.com>
* Make changes per @jenting review and use filepath to join paths
Signed-off-by: F. Gold <fgold@vmware.com>
* Update design doc with config map and k8s version priorities
Signed-off-by: F. Gold <fgold@vmware.com>
* Edit k8s doc URL per @jenting's review comment
Signed-off-by: F. Gold <fgold@vmware.com>
* Editorial changes
Signed-off-by: F. Gold <fgold@vmware.com>
* Changes per @nrb PR review and other edits
Signed-off-by: F. Gold <fgold@vmware.com>
* Update Status.FormatVersion check sections and minor edits
Signed-off-by: F. Gold <fgold@vmware.com>
* Add default field to BSL CRD
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Add a new flag `--default` under `velero backup-location create`
add a new flag `--default` under `velero backup-location create`
to specify this new location to be the new default BSL.
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Add a new default field under `velero backup-location get`
add a new default field under `velero backup-location get` to indicate
which BSL is the default one.
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Add a new sub-command and flag under `velero backup-location`
Add a new sub-command called `velero backup-location set` sub-command
and a new flag `velero backup-cation set --default` to configure which
BSL is the default one.
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Add new flag to get the default backup-location
Add a new flag `--default` under `velero backup-location get`
to displays the current default BSL.
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Configures default BSL in BSL controller
When upgrade the BSL CRDs, none of the BSL has been labeled as default.
Sets the BSL default field to true if the BSL name matches to the default BSL setting.
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Configures the default BSL in BSL controller for velero upgrade
When upgrade the BSL CRDs, none of the BSL be marked as the default.
Sets the BSL `.spec.default: true` if the BSL name matches against the
`velero server --default-backup-storage-location`.
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Add unit test to test default BSL behavior
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Update check which one is the default BSL in backup/backup_sync/restore controller
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Add changelog
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Update docs locations.md and upgrade-to-1.6.md
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* 🐛 BSLs with validation disabled should be validated at least once
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
* review comments
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
PR #3110 introduced a new action for performing the login to Dockerhub
as part of image building and pushing however there is an error with the
configuration and the credentials are not being passed through
correctly. This change reverts to the previous log in approach.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
The previous buildx action that we were using has been archived and
users are recommended to switch to the new action provided by Docker.
The previous action also included setting up QEMU. This is now provided
as a separate action which needs to be run separately.
This change also replaces the direct use of `docker login` with the new
`login-action`. This new action also handles logging out once the build
is complete.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* feat: add delete sub-command for backup-location
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Change to use kubebuilder/runtimecontroller API
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* fix get BSL by label doesn't work
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Update changelog
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Ordering by alphabet
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Better example format for help message
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Capital the comments
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Don't fail backup if downloading tarball fails
Previously, we would always attempt to download the tarball for a backup
for processing DeleteItemAction plugins, even if there weren't any.
This caused an issue for some users in the case where the backup tarball
had been deleted from object storage as the backup deletion would fail.
Now, we only attempt to download the tarball in the case where there are
DeleteItemAction plugins. If downloading that tarball fails, we log
the error, skip the processing of the DeleteItemAction plugins and
proceed with the rest of the deletion.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Skip file removal in closeAndRemoveFile if nil
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Basic end-to-end tests, generate data/backup/remove/restore/verify
Uses distributed data generator
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Moved backup/restore into velero_utils, started using a name for the restore
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* remove checked in binary and update test/e2e Makefile
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
* Ran make update
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Save
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
* Ran make update
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Basic end-to-end test, generate data/backup/remove/restore/verify
Uses distributed data generator
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Changed tests/e2e Makefile to just use go get to install ginkgo in the GOPATH/bin
Updated to ginkgo 1.14.2
Put cobra back to v0.0.7
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Added CLOUD_PLATFORM env variable to Makefile, updated README, removed ginkgo from .gitignore
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* choose velero CLI binary based on local env
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
Co-authored-by: Ashish Amarnath <ashisham@vmware.com>
By running the following command:
codespell -S .git,*.png,*.jpg,*.woff,*.ttf,*.gif,*.ico -L \
iam,aks,ist,bridget,ue
Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
* fixing label for 'velero.io/change-pvc-node-selector' plugin in site document
Signed-off-by: mayank <mayank.patel@mayadata.io>
* Fixing "velero.io/change-pvc-node-selector" to fetch config using plugin name
Signed-off-by: mayank <mayank.patel@mayadata.io>
* adding changelog
Signed-off-by: mayank <mayank.patel@mayadata.io>
This change modifies the kubebuilder annotations for the Velero CRDs to
include `additionalPrinterColumns` so that more information is exposed
when using `kubectl get`.
For each of the CRDs, annotations have been added to make the output
for `kubectl get` match the output from the equivalent `velero get`
command as closely as possible. There are some cases where this output
could not be replicated, such as the `EXPIRES` column for Backups, due
to the limitations of JSONPath expressions within the resulting CRD
defition. Some columns undergo processing and formatting before being
printed by the Velero CLI which cannot be replicated using JSONPath. In
these cases, these printer columns have been omitted.
For other CRDs where there is no `velero get` equivalent, such as
`PodVolumeBackup` and `PodVolumeRestore`, a best effort has been made to
expose information that provides value.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Adding handling of restic-wait init container at any order with warning.
Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>
* Adding newline at end of files to match convention.
Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>
* Formatting.
Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>
* Update copyright year on modified files.
Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>
* Only remove the UID from a PV's claimRef
The UID is the only part of a claimRef that might prevent it from being
rebound correctly on a restore. The namespace and name within the
claimRef should be preserved in order to ensure that the PV is claimed
by the correct PVC on restore.
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Remap PVs claimRef.namespace on relevant restores
When remapping namespaces, any included PVs need to have their claimRef
updated to point remapped namespaces to the new namespace name in order
to be bound to the correct PVC.
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Update tests and ensure claimRef namespace remaps
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Remove lowercased uid field from unstructured PV
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Fix issues that prevented PVs from being restored
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Add changelog
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Dynamically reprovision volumes without snapshots
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Update test for lower case uid field
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Remove stray debugging print statement
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Fix typo, remove extra code, add tests.
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* create CRB with velero-<namespace>
This will allow creating multiple instances of velero,
across two different namespaces
Signed-off-by: Alay Patel <alay1431@gmail.com>
* add changelog
Signed-off-by: Alay Patel <alay1431@gmail.com>
* add package var DefaultVeleroNamespace and use it wherever needed
Signed-off-by: Alay Patel <alay1431@gmail.com>
The command to check for an existing release branch only checked for
local branches. We should be considering both local and remote branches
before cherry-picking commits for the new release.
This change checks for existing local and remote release branches and
creates or updates them accordingly.
* If a remote branch exists, but a local branch does not, checkout the
remote branch and track it.
* If the remote branch and local branch exists, checkout the local
branch and ensure that the latest commits from the remote are pulled.
* Otherwise, if the remote branch does not exist, create it locally if
needed.
This also handles the case where an existing release branch may be
tracked in multiple remotes as the remote to use is explicitly stated.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
We instruct users to update the CRDs when upgrading to 1.4 and 1.5 which
involves using `kubectl apply` to apply the CRD configuration. The CRD
configuration generated by `velero install` includes fields which are
not valid when running Kubernetes v1.14 or earlier. We instruct users to
work around this when doing a customised velero install, but not when
upgrading to newer versions. This change updates the upgrade
instructions for v1.4 and v1.5 to include the use of `--validate=false`
flag when running `kubectl apply`.
See #2077 and #2311 for more context.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* restore proper lowercase/plural CRD resource
This commit restores the proper resource string
"customresourcedefinitions" for CRD. The prior change to
"CustomResourceDefinition" was made because this was being used
in another place to populate the CRD "Kind" field in
remap_crd_version_action.go -- there, just use the correct Kind
string instead of pulling from Resource.
Signed-off-by: Scott Seago <sseago@redhat.com>
* add changelog
Signed-off-by: Scott Seago <sseago@redhat.com>
The release script assumes that the remote for the vmware-tanzu/velero
repository is called `upstream`. It may be the case that this remote is
configured to use a different name. This change updates the script to
allow the remote name being used to be configured by setting the
environment variable `REMOTE` before running the script. If the variable
is not set, the remote defaults to `upstream`.
The release instructions have also been updated to reflect this change.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
This change addresses some issues in the documentation and scripts that
were found during the v1.5.1 release:
* Fix the path to the changelog script in the Makefile
* Fix the path to the pre-release TOC in the docs
* Improve the instructions for creating/updating the upgrade
instructions page.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Don't attempt to publish docker images on forks
When the Main CI workflow runs on a fork, the docker push step will
always fail because the appropriate secrets are missing. This is
annoying at best and causes CI on forks to be untrustworthy at worst.
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Use single quotes for string, as github expects
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Blop post announcing Velero 1.5
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
* Remove hardcoded deploy preview URL
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Remove base URL entirely
Since there's not really an easy way to use the preview URL environment
variables in the netlify.toml, remove the baseURL argument entirely
from the build command.
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
* Update blog post date and expected tag link
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
Co-authored-by: Ashish Amarnath <ashisham@vmware.com>
2020-09-16 17:56:47 -04:00
1000 changed files with 67332 additions and 12451 deletions
@@ -10,10 +10,13 @@ about: Tell us about a problem you are experiencing
**What did you expect to happen:**
**The following information will help us better understand what's going on**:
**The output of the following commands will help us better understand what's going on**:
(Pasting long output into a [GitHub gist](https://gist.github.com) or other pastebin is fine.)
_If you are using velero v1.7.0+:_
Please use `velero debug --backup <backupname> --restore <restorename>` to generate the support bundle, and attach to this issue, more options please refer to `velero debug --help`
_If you are using earlier versions:_
Please provide the output of the following commands (Pasting long output into a [GitHub gist](https://gist.github.com) or other pastebin is fine.)
-`kubectl logs deployment/velero -n velero`
-`velero backup describe <backupname>` or `kubectl get backup/<backupname> -n velero -o yaml`
- [ ] [Accepted the DCO](https://velero.io/docs/v1.5/code-standards/#dco-sign-off). Commits without the DCO will delay acceptance.
- [ ] [Created a changelog file](https://velero.io/docs/v1.5/code-standards/#adding-a-changelog) or added `/kind changelog-not-required` as a comment on this pull request.
- [ ] Updated the corresponding documentation in `site/content/docs/main`.
stale-issue-message:"This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days. If a Velero team member has requested log or more information, please provide the output of the shared commands."
close-issue-message:"This issue was closed because it has been stalled for 5 days with no activity."
days-before-issue-stale:30
days-before-issue-close:5
# Disable stale PRs for now; they can remain open.
days-before-pr-stale:-1
days-before-pr-close:-1
# Only issues made after Feb 09 2021.
start-date:"2021-09-02T00:00:00"
# Only make issues stale if they have these labels. Comma separated.
Below is a list of adopters of Velero in **production environments** that have
@@ -53,10 +54,16 @@ MayaData is a large user of Velero as well as a contributor. MayaData offers a D
**[Okteto][93]**
Okteto integrates Velero in [Okteto Cloud][94] and [Okteto Enterprise][95] to periodically backup and restore our clusters for disaster recovery. Velero is also a core software building block to provide namespace cloning capabilities, a feature that allows our users cloning staging environments into their personal development namespace for providing production-like development environments.
**[Replicated][100]**<br>
Replicated uses the Velero open source project to enable snapshots in [KOTS][101] to backup Kubernetes manifests & persistent volumes. In addition to the default functionality that Velero provides, [KOTS][101] provides a detailed interface in the [Admin Console][102] that can be used to manage the storage destination and schedule, and to perform and monitor the backup and restore process.<br>
**[CloudCasa][103]**<br>
[Catalogic Software][104] integrates Velero with [CloudCasa][103] - A Smart Home in the Cloud for Backups. CloudCasa is a simple, scalable, cloud-native solution providing data protection and disaster recovery as a service. This solution is built using Kubernetes for protecting Kubernetes clusters.<br>
## Adding your organization to the list of Velero Adopters
If you are using Velero and would like to be included in the list of `Velero Adopters`, add an SVG version of your logo to the `site/img/adopters` directory in this repo and submit a [pull request][3] with your change. Name the image file something that reflects your company (e.g., if your company is called Acme, name the image acme.png). See this for an example [PR][4].
If you are using Velero and would like to be included in the list of `Velero Adopters`, add an SVG version of your logo to the `site/static/img/adopters` directory in this repo and submit a [pull request][3] with your change. Name the image file something that reflects your company (e.g., if your company is called Acme, name the image acme.png). See this for an example [PR][4].
### Adding a logo to velero.io
@@ -102,3 +109,10 @@ If you would like to add your logo to a future `Adopters of Velero` section on [
We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation.
We as members, contributors, and leaders pledge to make participation in the Velero project and our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity
and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.
We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our community include:
Examples of behavior that contributes to a positive environment for our
community include:
* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience
* Focusing on what is best not just for us as individuals, but for the overall community
* Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the
overall community
Examples of unacceptable behavior include:
@@ -29,56 +38,90 @@ Examples of unacceptable behavior include:
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful.
Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.
Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate.
Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.
## Scope
This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event.
This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official e-mail address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at [oss-coc@vmware.com](mailto:oss-coc@vmware.com). All complaints will be reviewed and investigated promptly and fairly.
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at oss-coc@vmware.com.
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the reporter of any incident.
All community leaders are obligated to respect the privacy and security of the
reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct:
Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community.
**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested.
**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series of actions.
**Community Impact**: A violation through a single incident or series
of actions.
**Consequence**: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban.
**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or
permanent ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including sustained inappropriate behavior.
**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban.
**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.
**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within the community.
**Consequence**: A permanent ban from any sort of public interaction within
the community.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 2.0,
available at https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
Velero supports IPv4, IPv6, and dual stack environments. Support for this was tested against Velero v1.8.
The Velero maintainers are continuously working to expand testing coverage, but are not able to test every combination of Velero and supported Kubernetes versions for each Velero release. The table above is meant to track the current testing coverage and the expected supported Kubernetes versions for each Velero version. If you have a question about test coverage before v1.9, please reach out in the [#velero-users](https://kubernetes.slack.com/archives/C6VCGP4MT) Slack channel.
If you are interested in using a different version of Kubernetes with a given Velero version, we'd recommend that you perform testing before installing or upgrading your environment. For full information around capabilities within a release, also see the Velero [release notes](https://github.com/vmware-tanzu/velero/releases) or Kubernetes [release notes](https://github.com/kubernetes/kubernetes/tree/master/CHANGELOG). See the Velero [support page](https://velero.io/docs/latest/support-process/) for information about supported versions of Velero.
This document provides a link to the [Velero Project board](https://app.zenhub.com/workspaces/velero-5c59c15e39d47b774b5864e3/board?repos=99143276,112385197,190224441,214524700,214524630,213946861) that serves as the up to date description of items that are in the release pipeline. The board has separate swim lanes based on prioritization. Most items are gathered from the community or include a feedback loop with the community. This should serve as a reference point for Velero users and contributors to understand where the project is heading, and help determine if a contribution could be conflicting with a longer term plan. You will need the ZenHub plugin to view the board.
### How to help?
Discussion on the roadmap can take place in threads under [Issues](https://github.com/vmware-tanzu/velero/issues) or in [community meetings](https://velero.io/community/). Please open and comment on an issue if you want to provide suggestions, use cases, and feedback to an item in the roadmap. Please review the roadmap to avoid potential duplicated effort.
### How to add an item to the roadmap?
One of the most important aspects in any open source community is the concept of proposals. Large changes to the codebase and / or new features should be preceded by a [proposal](https://github.com/vmware-tanzu/velero/blob/main/GOVERNANCE.md#proposal-process) in our repo.
For smaller enhancements, you can open an issue to track that initiative or feature request.
We work with and rely on community feedback to focus our efforts to improve Velero and maintain a healthy roadmap.
### Current Roadmap
The following table includes the current roadmap for Velero. If you have any questions or would like to contribute to Velero, please attend a [community meeting](https://velero.io/community/) to discuss with our team. If you don't know where to start, we are always looking for contributors that will help us reduce technical, automation, and documentation debt.
Please take the timelines & dates as proposals and goals. Priorities and requirements change based on community feedback, roadblocks encountered, community contributions, etc. If you depend on a specific item, we encourage you to attend community meetings to get updated status information, or help us deliver that feature by contributing to Velero.
`Last Updated: May 2020`
|Theme|Description|Timeline|
|--|--|--|
|Restic Improvements|Introduce improvements in annotating resources for Restic backup|August 2020|
|Extensibility|Add restore hooks for enhanced recovery scenarios|August 2020|
|CSI|Continue improving the CSI snapshot capabilities and participate in the upstream K8s CSI community|Long running (dependent on CSI working group)|
|Backup/Restore|Improvements to long-running copy operations from a performance and reliability standpoint|August 2020|
|UX|Improvements to install and configuration user experience|August 2020|
|Restic Improvements|Improve the use of Restic in Velero and offer stable support|Dec 2020|
|Perf & Scale|Introduce a scalable model by using a worker pod for each backup/restore operation and improve operations|Dec 2020|
|Backup/Restore|Better backup and restore semantics for certain Kubernetes resources like stateful sets, operators|Dec 2020|
|Security|Enable the use of custom credential providers|Dec 2020|
|Self-Service & Multitenancy|Reduce friction by enabling developers to backup their namespaces via self-service. Introduce a Velero multi-tenancy model, enabling owners of namespaces to backup and restore within their access scope|Mar 2021|
|Backup/Restore|Cross availability zone or region backup and restore|Mar 2021|
|Application Consistency|Offer blueprints for backing up and restoring popular applications|May 2021|
|Backup/Restore|Data only backup and restore|May 2021|
|Backup/Restore|Introduce the ability to overwrite existing objects during a restore|May 2021|
|Backup/Restore|What-if dry run for backup and restore|May 2021|
# Please go to the [Velero Wiki](https://github.com/vmware-tanzu/velero/wiki/) to see our latest roadmap, archived roadmaps and roadmap guidance.
# Set up a local_resource build of the plugin binary. The main.go path must be provided via go_main option. The binary is written to _tiltbuild/<NAME>.
This folder contains logo images for Velero in gray (for light backgrounds) and white (for dark backgrounds like black tshirts or dark mode!) – horizontal and stacked… in .eps and .svg.
## Some general guidelines for usage
• Don’t alter the logos/graphics: resize, reformat, recolor. Keep them intact.
• Don’t separate the word mark (Velero) from the icon) – we are still building a strong name and identity – and the logo by itself doesn’t have any strong recognition or association with as yet: so best practice keep the two together. Nike kept its name with the swoosh for quite some time before the swoosh became iconic.
• Don’t append the name to another brand – let it stand alone!
@@ -75,7 +75,7 @@ Finally, thanks to testing by [Dylan Murray](https://github.com/dymurray) and [S
* Adds configurable CPU/memory requests and limits to the Velero Deployment generated by velero install. (#1678, @prydonius)
* Store restic PodVolumeBackups in obj storage & use that as source of truth like regular backups. (#1577, @carlisia)
* Update Velero Deployment to use apps/v1 API group. `velero install` and `velero plugin add/remove` commands will now require Kubernetes 1.9+ (#1673, @nrb)
* Respect the --kubecontext and --kubeconfig arugments for `velero install`. (#1656, @nrb)
* Respect the --kubecontext and --kubeconfig arguments for `velero install`. (#1656, @nrb)
* add plugin for updating PV & PVC storage classes on restore based on a config map (#1621, @skriss)
* Add restic support for CSI volumes (#1615, @nrb)
* Increased default Velero pod memory limit to 512Mi (#3234, @dsmithuchida)
* 🐛 BSLs with validation disabled should be validated at least once (#3084, @ashish-amarnath)
* Fixed an issue where the deletion of a backup would fail if the backup tarball couldn't be downloaded from object storage. Now the tarball is only downloaded if there are associated DeleteItemAction plugins and if downloading the tarball fails, the plugins are skipped. (#2993, @zubron)
* 🐛 ItemAction plugins for unresolvable types should not be run for all types (#3059, @ashish-amarnath)
* 🐛 Use namespace and name to match PVB to Pod restore (#3051, @ashish-amarnath)
* Allows the restic-wait container to exist in any order in the pod being restored. Prints a warning message in the case where the restic-wait container isn't the first container in the list of initialization containers. (#3011, @doughepi)
* Add support for restic to use per-BSL credentials. Velero will now serialize the secret referenced by the `Credential` field in the BSL and use this path when setting provider specific environment variables for restic commands. (#3489, @zubron)
* Upgrade restic from v0.9.6 to v0.12.0. (#3528, @ashish-amarnath)
* Progress reporting added for Velero Restores (#3125, @pranavgaikwad)
* Add uninstall option for velero cli (#3399, @vadasambar)
* Add support for per-BSL credentials. Velero will now serialize the secret referenced by the `Credential` field in the BSL and pass this path through to Object Storage plugins via the `config` map using the `credentialsFile` key. (#3442, @zubron)
* Fixed a bug where restic volumes would not be restored when using a namespace mapping. (#3475, @zubron)
* Restore API group version by priority. Increase timeout to 3 minutes in DeploymentIsReady(...) function in the install package (#3133, @codegold79)
* Add field and cli flag to associate a credential with a BSL on BSL create|set. (#3190, @carlisia)
* Add colored output to `describe schedule/backup/restore` commands (#3275, @mike1808)
* Add CAPI Cluster and ClusterResourceSets to default restore priorities so that the capi-controller-manager does not panic on restores. (#3446, @nrb)
* Use label to select Velero deployment in plugin cmd (#3447, @codegold79)
* feat: support setting BackupStorageLocation CA certificate via `velero backup-location set --cacert` (#3167, @jenting)
* Add restic initContainer length check in pod volume restore to prevent restic plugin container disappear in runtime (#3198, @shellwedance)
* Bump versions of external snapshotter and others in order to make `go get` to succeed (#3202, @georgettica)
* Support fish shell completion (#3231, @jenting)
* Change the logging level of PV deletion timeout from Debug to Warn (#3316, @MadhavJivrajani)
* Set the BSL created at install time as the "default" (#3172, @carlisia)
* Capitalize all help messages (#3209, @jenting)
* Increased default Velero pod memory limit to 512Mi (#3234, @dsmithuchida)
* Fixed an issue where the deletion of a backup would fail if the backup tarball couldn't be downloaded from object storage. Now the tarball is only downloaded if there are associated DeleteItemAction plugins and if downloading the tarball fails, the plugins are skipped. (#2993, @zubron)
* feat: add delete sub-command for BSL (#3073, @jenting)
* 🐛 BSLs with validation disabled should be validated at least once (#3084, @ashish-amarnath)
* feat: support configures BackupStorageLocation custom resources to indicate which one is the default (#3092, @jenting)
* Added "--preserve-nodeports" flag to preserve original nodePorts when restoring. (#3095, @yusufgungor)
* Owner reference in backup when created from schedule (#3127, @matheusjuvelino)
* issue: add flag to the schedule cmd to configure the `useOwnerReferencesInBackup` option #3176 (#3182, @matheusjuvelino)
* cli: allow creating multiple instances of Velero across two different namespaces (#2886, @alaypatel07)
* Feature: It is possible to change the timezone of the container by specifying in the manifest.. env: [TZ: Zone/Country], or in the Helm Chart.. configuration: {extraEnvVars: [TZ: 'Zone/Country']} (#2944, @mickkael)
* Fix issue where bare `velero` command returned an error code. (#2947, @nrb)
* Restore CRD Resource name to fix CRD wait functionality. (#2949, @sseago)
* Fixed 'velero.io/change-pvc-node-selector' plugin to fetch configmap using label key "velero.io/change-pvc-node-selector" (#2970, @mynktl)
* Compile with Go 1.15 (#2974, @gliptak)
* Fix BSL controller to avoid invoking init() on all BSLs regardless of ValidationFrequency (#2992, @betta1)
* Ensure that bound PVCs and PVs remain bound on restore. (#3007, @nrb)
* Allows the restic-wait container to exist in any order in the pod being restored. Prints a warning message in the case where the restic-wait container isn't the first container in the list of initialization containers. (#3011, @doughepi)
* Add warning to velero version cmd if the client and server versions mismatch. (#3024, @cvhariharan)
* 🐛 Use namespace and name to match PVB to Pod restore (#3051, @ashish-amarnath)
* Fixed various typos across codebase (#3057, @invidian)
* 🐛 ItemAction plugins for unresolvable types should not be run for all types (#3059, @ashish-amarnath)
* Pass annotations from schedule to backup it creates the same way it is done for labels. Add WithannotationsMap function to builder to be able to pass map instead of key/val list (#3067, @funkycode)
* Add instructions to clone repository for examples in docs (#3074, @MadhavJivrajani)
* 🏃♂️ update setup-kind github actions CI (#3085, @ashish-amarnath)
* Modify wrong function name to correct one. (#3106, @shellwedance)
The Velero container images now use [distroless base images](https://github.com/GoogleContainerTools/distroless).
Using distroless images as the base ensures that only the packages and programs necessary for running Velero are included.
Unrelated libraries and OS packages, that often contain security vulnerabilities, are now excluded.
This change reduces the size of both the server and restic restore helper image by approximately 62MB.
As the [distroless](https://github.com/GoogleContainerTools/distroless) images do not contain a shell, it will no longer be possible to exec into Velero containers using these images.
#### New "debug" command
This release introduces the new `velero debug` command.
This command collects information about a Velero installation, such as pod logs and resources managed by Velero, in a tarball which can be provided to the Velero maintainer team to help diagnose issues.
### All changes
* Distinguish between different unnamed node ports when preserving (#4026, @sseago)
* Validate namespace in Velero backup create command (#4057, @codegold79)
* Empty the "ClusterIPs" along with "ClusterIP" when "ClusterIP" isn't "None" (#4101, @ywk253100)
* Add a RestoreItemAction plugin (`velero.io/apiservice`) which skips the restore of any `APIService` which is managed by Kubernetes. These are identified using the `kube-aggregator.kubernetes.io/automanaged` label. (#4028, @zubron)
* Change the base image to distroless (#4055, @ywk253100)
* Updated the version of velero/velero-plugin-for-aws version from v1.2.0 to v1.2.1 (#4064, @kahirokunn)
* Skip the backup and restore of DownwardAPI volumes when using restic. (#4076, @zubron)
* Bump up Go to 1.16 (#3990, @reasonerjt)
* Fix restic error when volume is emptyDir and Pod not running (#3993, @mahaupt)
* Select the velero deployment with both label and container name (#3996, @ywk253100)
* Wait for the namespace to be deleted before removing the CRDs during uninstall. This deprecates the `--wait` flag of the `uninstall` command (#4007, @ywk253100)
* Use the cluster preferred CRD API version when polling for Velero CRD readiness. (#4015, @zubron)
* Implement velero debug (#4022, @reasonerjt)
* Skip the restore of volumes that originally came from a projected volume when using restic. (#3877, @zubron)
* Run the E2E test with kind(provision various versions of k8s cluster) and MinIO on Github Action (#3912, @ywk253100)
* Fix -install-velero flag for e2e tests (#3919, @jaidevmane)
* Upgrade Velero ClusterRoleBinding to use v1 API (#3926, @jenting)
* enable e2e tests to choose crd apiVersion (#3941, @sseago)
* Fixing multipleNamespaceTest bug - Missing expect statement in test (#3983, @jaidevmane)
* Add --client-page-size flag to server to allow chunking Kubernetes API LIST calls across multiple requests on large clusters (#3823, @dharmab)
* Use region specified in the BackupStorageLocation spec when getting restic repo identifier. Originally fixed by @jala-dx in #3617. (#3857, @zubron)
* skip backuping projected volume when using restic (#3866, @alaypatel07)
* Install Kubernetes preferred CRDs API version (v1beta1/v1). (#3614, @jenting)
* Add Label to BackupSpec so that labels can explicitly be provided to Schedule.Spec.Template.Metadata.Labels which will be reflected on the backups created. (#3641, @arush-sal)
* Add PVC UID label to PodVolumeRestore (#3792, @sseago)
* Support pulling plugin images by digest (#3803, @2uasimojo)
* Added BackupPhaseUploading and BackupPhaseUploadingPartialFailure backup phases as part of Upload Progress Monitoring. (#3805, @dsmithuchida)
Uploading (new)
The "Uploading" phase signifies that the main part of the backup, including
snapshotting has completed successfully and uploading is continuing. In
the event of an error during uploading, the phase will change to
UploadingPartialFailure. On success, the phase changes to Completed. The
backup cannot be restored from when it is in the Uploading state.
UploadingPartialFailure (new)
The "UploadingPartialFailure" phase signifies that the main part of the backup,
including snapshotting has completed, but there were partial failures either
during the main part or during the uploading. The backup cannot be restored
from when it is in the UploadingPartialFailure state.
* 🐛 Fix plugin name derivation from image name (#3711, @ashish-amarnath)
#### Velero plugins now support handling volumes created by the CSI drivers of cloud providers
Versions 1.4 of the Velero plugins for AWS, Azure and GCP now support snapshotting and restoring the persistent volumes provisioned by CSI driver via the APIs of the cloud providers. With this enhancement, users can backup and restore the persistent volumes on these cloud providers without using the Velero CSI plugin. The CSI plugin will remain beta and the feature flag `EnableCSI` will be disabled by default.
For the version of the plugins and the CSI drivers they support respectively please see the table:
We've verified the functionality of Velero on IPv6 dual stack by successfully running the E2E test on IPv6 dual stack environment.
#### Refactor the controllers using Kubebuilder v3
In this release we continued our code modernization work, rewriting some controllers using Kubebuilder v3. This work is ongoing and we will continue to make progress in future releases.
#### Enhancements to E2E test cases
More test cases have been added to the E2E test suite to improve the release health.
#### Respect the cron setting of scheduled backup
The creation time is now taken into account to calculate the next run for scheduled backup.
#### Deleting BSLs also cleans up related resources
When a Backup Storage Location (BSL) is deleted, backup and Restic repository resources will also be deleted.
#### Breaking changes
Starting in v1.8, Velero will only support Kubernetes v1 CRD meaning that Velero v1.8+ will only run on Kubernetes v1.16+. Before upgrading, make sure you are running a supported Kubernetes version. For more information, see our [compatibility matrix](https://github.com/vmware-tanzu/velero#velero-compatibility-matrix).
#### Upload Progress Monitoring and Item Snapshotter
Item Snapshotter plugin API was merged. This will support both Upload Progress
monitoring and the planned Data Mover. Upload Progress monitoring PRs are
in progress for 1.9.
### All changes
* E2E test on ssr object with controller namespace mix-ups (#4521, @mqiu)
* Check whether the volume is provisioned by CSI driver or not by the annotation as well (#4513, @ywk253100)
* Initialize the labels field of `velero backup-location create` option to avoid #4484 (#4491, @ywk253100)
* Fix e2e 2500 namespaces scale test timeout problem (#4480, @mqiu)
* Add backup deletion e2e test (#4401, @danfengliu)
* Return the error when getting backup store in backup deletion controller (#4465, @reasonerjt)
* Ignore the provided port is already allocated error when restoring the LoadBalancer service (#4462, @ywk253100)
* Add rbac and annotation test cases (#4455, @mqiu)
* remove --crds-version in velero install command. (#4446, @jxun)
* Upgrade e2e test vsphere plugin (#4440, @mqiu)
* Fix e2e test failures for the inappropriate optimaze of velero install (#4438, @mqiu)
* Limit backup namespaces on test resource filtering cases (#4437, @mqiu)
* Bump up Go to 1.17 (#4431, @reasonerjt)
* Added `<backup name>`-itemsnapshots.json.gz to the backup format. This file exists
when item snapshots are taken and contains an array of volume.Itemsnapshots
containing the information about the snapshots. This will not be used unless
upload progress monitoring and item snapshots are enabled and an ItemSnapshot
plugin is used to take snapshots.
Also added DownloadTargetKindBackupItemSnapshots for retrieving the signed URL to download only the `<backup name>`-itemsnapshots.json.gz part of a backup for use by
`velero backup describe`. (#4429, @dsmithuchida)
* Migrate backup sync controller from code-generator to kubebuilder. (#4423, @jxun)
* Added UploadProgressFeature flag to enable Upload Progress Monitoring and Item
Snapshotters. (#4416, @dsmithuchida)
* Added BackupWithResolvers and RestoreWithResolvers calls. Will eventually replace Backup and Restore methods.
Adds ItemSnapshotters to Backup and Restore workflows. (#4410, @dsu)
* Build for darwin-arm64 (#4409, @epk)
* Add resource filtering test cases (#4404, @mqiu)
* Fix the issue that the backup cannot be deleted after the application uninstalled (#4398, @ywk253100)
* Ignore the `provided port is already allocated` error when restoring the `NodePort` service (#4336, @ywk253100)
* Fixed an issue with the `backup-location create` command where the BSL Credential field would be set to an invalid empty SecretKeySelector when no credential details were provided. (#4322, @zubron)
* fix buggy pager func (#4306, @alaypatel07)
* Don't create a backup immediately after creating a schedule (#4281, @ywk253100)
* Fix CVE-2020-29652 and CVE-2020-26160 (#4274, @ywk253100)
* Refine tag-release.sh to align with change in release process (#4185, @reasonerjt)
* Fix plugins incompatible issue in upgrade test (#4141, @danfengliu)
* Verify group before treating resource as cohabiting (#4126, @sseago)
* Add compile restic binary for CVE fix (#5564, @qiuming-best)
* Bump up golang version to 1.18.8 (#5558, @qiuming-best)
* Enhance the restore priorities list to support specifying the low prioritized resources that need to be restored in the last (#5529, @ywk253100)
* Fix v1.9.3 CSI VolumeSnapshot status duplicate issue. (#5518, @blackpiglet)
* Bump up the distroless image to the latest version (#5500, @ywk253100)
* Add some corner cases checking for CSI snapshot in backup controller. (#5482, @blackpiglet)
* Skip the exclusion check for additional resources returned by BIA (#5406, @reasonerjt)
* Exclude "csinodes.storage.k8s.io" and "volumeattachments.storage.k8s.io" from restore by default. (#5448, @jxun)
* Update the k8s.io dependencies to 0.24.0 and Removed the `WithClusterName` method as it is a "legacy field that was always cleared by the system and never used" as per upstream k8s. (#5472, @kcboyle)
* Fix nil pointer panic when restoring StatefulSets (#5301, @divolgin)
* Check for empty ns list before checking nslist[0] (#5302, @sseago)
* check vsc null pointer (#5303, @lilongfeng0902)
* Fix edge cases for already exists resources (#5304, @shubham-pampattiwar)
* Increase ensure restic repository timeout to 5m (#5336, @shubham-pampattiwar)
* Added DownloadTargetKindCSIBackupVolumeSnapshots for retrieving the signed URL to download only the `<backup name>`-csi-volumesnapshots.json.gz and DownloadTargetKindCSIBackupVolumeSnapshotContents to download only `<backup name>`-csi-volumesnapshotcontents.json.gz in the DownloadRequest CR structure. These files are already present in the backup layout. (#5307, @anshulahuja98)
- No VolumeSnapshot will be left in the source namespace of the workload
- Report metrics for CSI snapshots
More improvements please refer to [CSI plugin improvement](https://github.com/vmware-tanzu/velero/issues?q=is%3Aissue+label%3A%22CSI+plugin+-+GA+-+phase1%22+is%3Aclosed)
With these improvements we'll provide official support for CSI snapshots on AKS/EKS clusters. (with CSI plugin v0.3.0)
#### Refactor the controllers using Kubebuilder v3
In this release we continued our code modernization work, rewriting some controllers using Kubebuilder v3. This work is ongoing and we will continue to make progress in future releases.
#### Optionally restore status on selected resources
Options are added to the CLI and Restore spec to control the group of resources whose status will be restored.
#### ExistingResourcePolicy in the restore API
Users can choose to overwrite or patch the existing resources during restore by setting this policy.
#### Upgrade integrated Restic version and add skip TLS validation in Restic command
Upgrade integrated Restic version, which will resolve some of the CVEs, and support skip TLS validation in Restic backup/restore.
#### Breaking changes
With bumping up the API to v1 in CSI plugin, the v0.3.0 CSI plugin will only work for Kubernetes v1.20+
### All changes
* restic: add full support for setting SecurityContext for restore init container from configMap. (#4084, @MatthieuFin)
* Add metrics backup_items_total and backup_items_errors (#4296, @tobiasgiese)
* Convert PodVolumebackup controller to the Kubebuilder framework (#4436, @fgold)
* Skip not mounted volumes when backing up (#4497, @dkeven)
* Update doc for v1.8 (#4517, @reasonerjt)
* Fix bug to make the restic prune frequency configurable (#4518, @ywk253100)
* Add E2E test of backups sync from BSL (#4545, @mqiu)
* Fix: OrderedResources in Schedules (#4550, @dbrekau)
* Skip volumes of non-running pods when backing up (#4584, @bynare)
* E2E SSR test add retry mechanism and logs (#4591, @mqiu)
* Add pushing image to GCR in github workflow to facilitate some environments that have rate limitation to docker hub, e.g. vSphere. (#4623, @jxun)
* Add existingResourcePolicy to Restore API (#4628, @shubham-pampattiwar)
* Fix E2E backup namespaces test (#4634, @qiuming-best)
* Update image used by E2E test to gcr.io (#4639, @jxun)
* Add multiple label selector support to Velero Backup and Restore APIs (#4650, @shubham-pampattiwar)
* Convert Pod Volume Restore resource/controller to the Kubebuilder framework (#4655, @ywk253100)
* Update --use-owner-references-in-backup description in velero command line. (#4660, @jxun)
* Avoid overwritten hook's exec.container parameter when running pod command executor. (#4661, @jxun)
* Support regional pv for GKE (#4680, @jxun)
* Bypass the remap CRD version plugin when v1beta1 CRD is not supported (#4686, @reasonerjt)
* Add GINKGO_SKIP to support skip specific case in e2e test. (#4692, @jxun)
* Add --pod-labels flag to velero install (#4694, @j4m3s-s)
* Enable coverage in test.sh and upload to codecov (#4704, @reasonerjt)
* Mark the BSL as "Unavailable" when gets any error and add a new field "Message" to the status to record the error message (#4719, @ywk253100)
* Support multiple skip option for E2E test (#4725, @jxun)
* Add PriorityClass to the AdditionalItems of Backup's PodAction and Restore's PodAction plugin to backup and restore PriorityClass if it is used by a Pod. (#4740, @phuongatemc)
* Insert all restore errors and warnings into restore log. (#4743, @sseago)
* Refactor schedule controller with kubebuilder (#4748, @ywk253100)
* Garbage collector now adds labels to backups that failed to delete for BSLNotFound, BSLCannotGet, BSLReadOnly reasons. (#4757, @kaovilai)
* Skip podvolumerestore creation when restore excludes pv/pvc (#4769, @half-life666)
* Add parameter for e2e test to support modify kibishii install path. (#4778, @jxun)
* Ensure the restore hook applied to new namespace based on the mapping (#4779, @reasonerjt)
* Add ability to restore status on selected resources (#4785, @RafaeLeal)
* Do not take snapshot for PV to avoid duplicated snapshotting, when CSI feature is enabled. (#4797, @jxun)
* Bump up to v1 API for CSI snapshot (#4800, @reasonerjt)
* fix: delete empty backups (#4817, @yuvalman)
* Add CSI VolumeSnapshot related metrics. (#4818, @jxun)
* Fix default-backup-ttl not work (#4831, @qiuming-best)
* Make the vsc created by backup sync controller deletable (#4832, @reasonerjt)
* Make in-progress backup/restore as failed when doing the reconcile to avoid hanging in in-progress status (#4833, @ywk253100)
* Use controller-gen to generate the deep copy methods for objects (#4838, @ywk253100)
* Update integrated Restic version and add insecureSkipTLSVerify for Restic CLI. (#4839, @jxun)
* Modify CSI VolumeSnapshot metric related code. (#4854, @jxun)
* Refactor backup deletion controller based on kubebuilder (#4855, @reasonerjt)
* Remove VolumeSnapshots created during backup when CSI feature is enabled. (#4858, @jxun)
* Convert Restic Repository resource/controller to the Kubebuilder framework (#4859, @qiuming-best)
* Add ClusterClasses to the restore priority list (#4866, @reasonerjt)
* Cleanup the .velero folder after restic done (#4872, @big-appled)
description:Backup is a Velero resource that respresents the capture of Kubernetes
cluster state at a point in time (API objects and associated volume state).
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:BackupSpec defines the specification for a Velero backup.
properties:
defaultVolumesToRestic:
description:DefaultVolumesToRestic specifies whether restic should
be used to take a backup of all pod volumes by default.
type:boolean
excludedNamespaces:
description:ExcludedNamespaces contains a list of namespaces that are
not included in the backup.
items:
type:string
nullable:true
type:array
excludedResources:
description:ExcludedResources is a slice of resource names that are
not included in the backup.
items:
type:string
nullable:true
type:array
hooks:
description:Hooks represent custom behaviors that should be executed
at different phases of the backup.
properties:
resources:
description:Resources are hooks that should be executed when backing
up individual instances of a resource.
items:
description:BackupResourceHookSpec defines one or more BackupResourceHooks
that should be executed based on the rules defined for namespaces,
resources, and label selector.
properties:
excludedNamespaces:
description:ExcludedNamespaces specifies the namespaces to
which this hook spec does not apply.
items:
type:string
nullable:true
type:array
excludedResources:
description:ExcludedResources specifies the resources to
which this hook spec does not apply.
items:
type:string
nullable:true
type:array
includedNamespaces:
description:IncludedNamespaces specifies the namespaces to
which this hook spec applies. If empty, it applies to all
namespaces.
items:
type:string
nullable:true
type:array
includedResources:
description:IncludedResources specifies the resources to
which this hook spec applies. If empty, it applies to all
resources.
items:
type:string
nullable:true
type:array
labelSelector:
description:LabelSelector, if specified, filters the resources
to which this hook spec applies.
nullable:true
properties:
matchExpressions:
description:matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description:A label selector requirement is a selector
that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description:key is the label key that the selector
applies to.
type:string
operator:
description:operator represents a key's relationship
to a set of values. Valid operators are In, NotIn,
Exists and DoesNotExist.
type:string
values:
description:values is an array of string values.
If the operator is In or NotIn, the values array
must be non-empty. If the operator is Exists or
DoesNotExist, the values array must be empty.
This array is replaced during a strategic merge
patch.
items:
type:string
type:array
required:
- key
- operator
type:object
type:array
matchLabels:
additionalProperties:
type:string
description:matchLabels is a map of {key,value} pairs.
A single {key,value} in the matchLabels map is equivalent
to an element of matchExpressions, whose key field is
"key",the operator is "In", and the values array contains
only "value". The requirements are ANDed.
type:object
type:object
name:
description:Name is the name of this hook.
type:string
post:
description:PostHooks is a list of BackupResourceHooks to
execute after storing the item in the backup. These are
executed after all "additional items" from item actions
are processed.
items:
description:BackupResourceHook defines a hook for a resource.
properties:
exec:
description:Exec defines an exec hook.
properties:
command:
description:Command is the command and arguments
to execute.
items:
type:string
minItems:1
type:array
container:
description:Container is the container in the pod
where the command should be executed. If not specified,
the pod's first container is used.
type:string
onError:
description:OnError specifies how Velero should
behave if it encounters an error executing this
hook.
enum:
- Continue
- Fail
type:string
timeout:
description:Timeout defines the maximum amount
of time Velero should wait for the hook to complete
before considering the execution a failure.
type:string
required:
- command
type:object
required:
- exec
type:object
type:array
pre:
description:PreHooks is a list of BackupResourceHooks to
execute prior to storing the item in the backup. These are
executed before any "additional items" from item actions
are processed.
items:
description:BackupResourceHook defines a hook for a resource.
properties:
exec:
description:Exec defines an exec hook.
properties:
command:
description:Command is the command and arguments
to execute.
items:
type:string
minItems:1
type:array
container:
description:Container is the container in the pod
where the command should be executed. If not specified,
the pod's first container is used.
type:string
onError:
description:OnError specifies how Velero should
behave if it encounters an error executing this
hook.
enum:
- Continue
- Fail
type:string
timeout:
description:Timeout defines the maximum amount
of time Velero should wait for the hook to complete
description:Backup Storage Location status such as Available/Unavailable
name:Phase
type:string
- JSONPath:.status.lastValidationTime
description:LastValidationTime is the last time the backup store location was
validated
name:Last Validated
type:date
- JSONPath:.metadata.creationTimestamp
name:Age
type:date
group:velero.io
names:
kind:BackupStorageLocation
listKind:BackupStorageLocationList
plural:backupstoragelocations
shortNames:
- bsl
singular:backupstoragelocation
preserveUnknownFields:false
scope:Namespaced
subresources:
status:{}
validation:
openAPIV3Schema:
description:BackupStorageLocation is a location where Velero stores backup
objects
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:BackupStorageLocationSpec defines the desired state of a Velero
BackupStorageLocation
properties:
accessMode:
description:AccessMode defines the permissions for the backup storage
location.
enum:
- ReadOnly
- ReadWrite
type:string
backupSyncPeriod:
description:BackupSyncPeriod defines how frequently to sync backup
API objects from object storage. A value of 0 disables sync.
nullable:true
type:string
config:
additionalProperties:
type:string
description:Config is for provider-specific configuration fields.
type:object
objectStorage:
description:ObjectStorageLocation specifies the settings necessary
to connect to a provider's object storage.
properties:
bucket:
description:Bucket is the bucket to use for object storage.
type:string
caCert:
description:CACert defines a CA bundle to use when verifying TLS
connections to the provider.
format:byte
type:string
prefix:
description:Prefix is the path inside a bucket to use for Velero
storage. Optional.
type:string
required:
- bucket
type:object
provider:
description:Provider is the provider of the backup storage.
type:string
validationFrequency:
description:ValidationFrequency defines how frequently to validate
the corresponding object storage. A value of 0 disables validation.
nullable:true
type:string
required:
- objectStorage
- provider
type:object
status:
description:BackupStorageLocationStatus defines the observed state of BackupStorageLocation
properties:
accessMode:
description:"AccessMode is an unused field. \n Deprecated: there is
now an AccessMode field on the Spec and this field will be removed
entirely as of v2.0."
enum:
- ReadOnly
- ReadWrite
type:string
lastSyncedRevision:
description:"LastSyncedRevision is the value of the `metadata/revision`
file in the backup storage location the last time the BSL's contents
were synced into the cluster. \n Deprecated: this field is no longer
updated or used for detecting changes to the location's contents and
will be removed entirely in v2.0."
type:string
lastSyncedTime:
description:LastSyncedTime is the last time the contents of the location
were synced into the cluster.
format:date-time
nullable:true
type:string
lastValidationTime:
description:LastValidationTime is the last time the backup store location
was validated the cluster.
format:date-time
nullable:true
type:string
phase:
description:Phase is the current state of the BackupStorageLocation.
description:DeleteBackupRequest is a request to delete one or more backups.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:DeleteBackupRequestSpec is the specification for which backups
to delete.
properties:
backupName:
type:string
required:
- backupName
type:object
status:
description:DeleteBackupRequestStatus is the current status of a DeleteBackupRequest.
properties:
errors:
description:Errors contains any errors that were encountered during
the deletion process.
items:
type:string
nullable:true
type:array
phase:
description:Phase is the current state of the DeleteBackupRequest.
description:DownloadRequest is a request to download an artifact from backup
object storage, such as a backup log file.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:DownloadRequestSpec is the specification for a download request.
properties:
target:
description:Target is what to download (e.g. logs for a backup).
properties:
kind:
description:Kind is the type of file to download.
enum:
- BackupLog
- BackupContents
- BackupVolumeSnapshots
- BackupResourceList
- RestoreLog
- RestoreResults
type:string
name:
description:Name is the name of the kubernetes resource with which
the file is associated.
type:string
required:
- kind
- name
type:object
required:
- target
type:object
status:
description:DownloadRequestStatus is the current status of a DownloadRequest.
properties:
downloadURL:
description:DownloadURL contains the pre-signed URL for the target
file.
type:string
expiration:
description:Expiration is when this DownloadRequest expires and can
be deleted by the system.
format:date-time
nullable:true
type:string
phase:
description:Phase is the current state of the DownloadRequest.
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:PodVolumeBackupSpec is the specification for a PodVolumeBackup.
properties:
backupStorageLocation:
description:BackupStorageLocation is the name of the backup storage
location where the restic repository is stored.
type:string
node:
description:Node is the name of the node that the Pod is running on.
type:string
pod:
description:Pod is a reference to the pod containing the volume to
be backed up.
properties:
apiVersion:
description:API version of the referent.
type:string
fieldPath:
description:'If referring to a piece of an object instead of an
entire object, this string should contain a valid JSON/Go field
access statement, such as desiredState.manifest.containers[2].
For example, if the object reference is to a container within
a pod, this would take on a value like: "spec.containers{name}"
(where "name" refers to the name of the container that triggered
the event) or if no container name is specified "spec.containers[2]"
(container with index 2 in this pod). This syntax is chosen only
to have some well-defined way of referencing a part of an object.
TODO: this design is not final and this field is subject to change
in the future.'
type:string
kind:
description: 'Kind of the referent. More info:https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
name:
description: 'Name of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
type:string
namespace:
description: 'Namespace of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
type:string
resourceVersion:
description:'Specific resourceVersion to which this reference is
made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
type:string
uid:
description: 'UID of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
type:string
type:object
repoIdentifier:
description:RepoIdentifier is the restic repository identifier.
type:string
tags:
additionalProperties:
type:string
description:Tags are a map of key-value pairs that should be applied
to the volume backup as tags.
type:object
volume:
description:Volume is the name of the volume within the Pod to be backed
up.
type:string
required:
- backupStorageLocation
- node
- pod
- repoIdentifier
- volume
type:object
status:
description:PodVolumeBackupStatus is the current status of a PodVolumeBackup.
properties:
completionTimestamp:
description:CompletionTimestamp records the time a backup was completed.
Completion time is recorded even on failed backups. Completion time
is recorded before uploading the backup object. The server's time
is used for CompletionTimestamps
format:date-time
nullable:true
type:string
message:
description:Message is a message about the pod volume backup's status.
type:string
path:
description:Path is the full path within the controller pod being backed
up.
type:string
phase:
description:Phase is the current state of the PodVolumeBackup.
enum:
- New
- InProgress
- Completed
- Failed
type:string
progress:
description:Progress holds the total number of bytes of the volume
and the current number of backed up bytes. This can be used to display
progress information about the backup operation.
properties:
bytesDone:
format:int64
type:integer
totalBytes:
format:int64
type:integer
type:object
snapshotID:
description:SnapshotID is the identifier for the snapshot of the pod
volume.
type:string
startTimestamp:
description:StartTimestamp records the time a backup was started. Separate
from CreationTimestamp, since that value changes on restores. The
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:PodVolumeRestoreSpec is the specification for a PodVolumeRestore.
properties:
backupStorageLocation:
description:BackupStorageLocation is the name of the backup storage
location where the restic repository is stored.
type:string
pod:
description:Pod is a reference to the pod containing the volume to
be restored.
properties:
apiVersion:
description:API version of the referent.
type:string
fieldPath:
description:'If referring to a piece of an object instead of an
entire object, this string should contain a valid JSON/Go field
access statement, such as desiredState.manifest.containers[2].
For example, if the object reference is to a container within
a pod, this would take on a value like: "spec.containers{name}"
(where "name" refers to the name of the container that triggered
the event) or if no container name is specified "spec.containers[2]"
(container with index 2 in this pod). This syntax is chosen only
to have some well-defined way of referencing a part of an object.
TODO: this design is not final and this field is subject to change
in the future.'
type:string
kind:
description: 'Kind of the referent. More info:https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
name:
description: 'Name of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
type:string
namespace:
description: 'Namespace of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
type:string
resourceVersion:
description:'Specific resourceVersion to which this reference is
made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
type:string
uid:
description: 'UID of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
type:string
type:object
repoIdentifier:
description:RepoIdentifier is the restic repository identifier.
type:string
snapshotID:
description:SnapshotID is the ID of the volume snapshot to be restored.
type:string
volume:
description:Volume is the name of the volume within the Pod to be restored.
type:string
required:
- backupStorageLocation
- pod
- repoIdentifier
- snapshotID
- volume
type:object
status:
description:PodVolumeRestoreStatus is the current status of a PodVolumeRestore.
properties:
completionTimestamp:
description:CompletionTimestamp records the time a restore was completed.
Completion time is recorded even on failed restores. The server's
time is used for CompletionTimestamps
format:date-time
nullable:true
type:string
message:
description:Message is a message about the pod volume restore's status.
type:string
phase:
description:Phase is the current state of the PodVolumeRestore.
enum:
- New
- InProgress
- Completed
- Failed
type:string
progress:
description:Progress holds the total number of bytes of the snapshot
and the current number of restored bytes. This can be used to display
progress information about the restore operation.
properties:
bytesDone:
format:int64
type:integer
totalBytes:
format:int64
type:integer
type:object
startTimestamp:
description:StartTimestamp records the time a restore was started.
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:ResticRepositorySpec is the specification for a ResticRepository.
properties:
backupStorageLocation:
description:BackupStorageLocation is the name of the BackupStorageLocation
that should contain this repository.
type:string
maintenanceFrequency:
description:MaintenanceFrequency is how often maintenance should be
run.
type:string
resticIdentifier:
description:ResticIdentifier is the full restic-compatible string for
identifying this repository.
type:string
volumeNamespace:
description:VolumeNamespace is the namespace this restic repository
contains pod volume backups for.
type:string
required:
- backupStorageLocation
- maintenanceFrequency
- resticIdentifier
- volumeNamespace
type:object
status:
description:ResticRepositoryStatus is the current status of a ResticRepository.
properties:
lastMaintenanceTime:
description:LastMaintenanceTime is the last time maintenance was run.
format:date-time
nullable:true
type:string
message:
description:Message is a message about the current status of the ResticRepository.
type:string
phase:
description:Phase is the current state of the ResticRepository.
description:Schedule is a Velero resource that represents a pre-scheduled or
periodic Backup that should be run.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:ScheduleSpec defines the specification for a Velero schedule
properties:
schedule:
description:Schedule is a Cron expression defining when to run the
Backup.
type:string
template:
description:Template is the definition of the Backup to be run on the
provided schedule
properties:
defaultVolumesToRestic:
description:DefaultVolumesToRestic specifies whether restic should
be used to take a backup of all pod volumes by default.
type:boolean
excludedNamespaces:
description:ExcludedNamespaces contains a list of namespaces that
are not included in the backup.
items:
type:string
nullable:true
type:array
excludedResources:
description:ExcludedResources is a slice of resource names that
are not included in the backup.
items:
type:string
nullable:true
type:array
hooks:
description:Hooks represent custom behaviors that should be executed
at different phases of the backup.
properties:
resources:
description:Resources are hooks that should be executed when
backing up individual instances of a resource.
items:
description:BackupResourceHookSpec defines one or more BackupResourceHooks
that should be executed based on the rules defined for namespaces,
resources, and label selector.
properties:
excludedNamespaces:
description:ExcludedNamespaces specifies the namespaces
to which this hook spec does not apply.
items:
type:string
nullable:true
type:array
excludedResources:
description:ExcludedResources specifies the resources
to which this hook spec does not apply.
items:
type:string
nullable:true
type:array
includedNamespaces:
description:IncludedNamespaces specifies the namespaces
to which this hook spec applies. If empty, it applies
to all namespaces.
items:
type:string
nullable:true
type:array
includedResources:
description:IncludedResources specifies the resources
to which this hook spec applies. If empty, it applies
to all resources.
items:
type:string
nullable:true
type:array
labelSelector:
description:LabelSelector, if specified, filters the
resources to which this hook spec applies.
nullable:true
properties:
matchExpressions:
description:matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description:A label selector requirement is a selector
that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description:key is the label key that the selector
applies to.
type:string
operator:
description:operator represents a key's relationship
to a set of values. Valid operators are In,
NotIn, Exists and DoesNotExist.
type:string
values:
description:values is an array of string values.
If the operator is In or NotIn, the values
array must be non-empty. If the operator is
Exists or DoesNotExist, the values array must
be empty. This array is replaced during a
strategic merge patch.
items:
type:string
type:array
required:
- key
- operator
type:object
type:array
matchLabels:
additionalProperties:
type:string
description:matchLabels is a map of {key,value} pairs.
A single {key,value} in the matchLabels map is equivalent
to an element of matchExpressions, whose key field
is "key", the operator is "In", and the values array
contains only "value". The requirements are ANDed.
type:object
type:object
name:
description:Name is the name of this hook.
type:string
post:
description:PostHooks is a list of BackupResourceHooks
to execute after storing the item in the backup. These
are executed after all "additional items" from item
actions are processed.
items:
description:BackupResourceHook defines a hook for a
resource.
properties:
exec:
description:Exec defines an exec hook.
properties:
command:
description:Command is the command and arguments
to execute.
items:
type:string
minItems:1
type:array
container:
description:Container is the container in the
pod where the command should be executed.
If not specified, the pod's first container
is used.
type:string
onError:
description:OnError specifies how Velero should
behave if it encounters an error executing
this hook.
enum:
- Continue
- Fail
type:string
timeout:
description:Timeout defines the maximum amount
of time Velero should wait for the hook to
complete before considering the execution
a failure.
type:string
required:
- command
type:object
required:
- exec
type:object
type:array
pre:
description:PreHooks is a list of BackupResourceHooks
to execute prior to storing the item in the backup.
These are executed before any "additional items" from
item actions are processed.
items:
description:BackupResourceHook defines a hook for a
description:ServerStatusRequest is a request to access current status information
about the Velero server.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:ServerStatusRequestSpec is the specification for a ServerStatusRequest.
type:object
status:
description:ServerStatusRequestStatus is the current status of a ServerStatusRequest.
properties:
phase:
description:Phase is the current lifecycle phase of the ServerStatusRequest.
enum:
- New
- Processed
type:string
plugins:
description:Plugins list information about the plugins running on the
Velero server
items:
description:PluginInfo contains attributes of a Velero plugin
properties:
kind:
type:string
name:
type:string
required:
- kind
- name
type:object
nullable:true
type:array
processedTimestamp:
description:ProcessedTimestamp is when the ServerStatusRequest was
processed by the ServerStatusRequestController.
format:date-time
nullable:true
type:string
serverVersion:
description:ServerVersion is the Velero server version.
description:VolumeSnapshotLocation is a location where Velero stores volume
snapshots.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:VolumeSnapshotLocationSpec defines the specification for a
Velero VolumeSnapshotLocation.
properties:
config:
additionalProperties:
type:string
description:Config is for provider-specific configuration fields.
type:object
provider:
description:Provider is the provider of the volume storage.
type:string
required:
- provider
type:object
status:
description:VolumeSnapshotLocationStatus describes the current status of
a Velero VolumeSnapshotLocation.
properties:
phase:
description:VolumeSnapshotLocationPhase is the lifecyle phase of a
description:Backup is a Velero resource that represents the capture of Kubernetes
cluster state at a point in time (API objects and associated volume state).
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:BackupSpec defines the specification for a Velero backup.
properties:
csiSnapshotTimeout:
description:CSISnapshotTimeout specifies the time used to wait for
CSI VolumeSnapshot status turns to ReadyToUse during creation, before
returning error as timeout. The default value is 10 minute.
type:string
defaultVolumesToRestic:
description:DefaultVolumesToRestic specifies whether restic should
be used to take a backup of all pod volumes by default.
type:boolean
excludedNamespaces:
description:ExcludedNamespaces contains a list of namespaces that
are not included in the backup.
items:
type:string
nullable:true
type:array
excludedResources:
description:ExcludedResources is a slice of resource names that are
not included in the backup.
items:
type:string
nullable:true
type:array
hooks:
description:Hooks represent custom behaviors that should be executed
at different phases of the backup.
properties:
resources:
description:Resources are hooks that should be executed when
backing up individual instances of a resource.
items:
description:BackupResourceHookSpec defines one or more BackupResourceHooks
that should be executed based on the rules defined for namespaces,
resources, and label selector.
properties:
excludedNamespaces:
description:ExcludedNamespaces specifies the namespaces
to which this hook spec does not apply.
items:
type:string
nullable:true
type:array
excludedResources:
description:ExcludedResources specifies the resources to
which this hook spec does not apply.
items:
type:string
nullable:true
type:array
includedNamespaces:
description:IncludedNamespaces specifies the namespaces
to which this hook spec applies. If empty, it applies
to all namespaces.
items:
type:string
nullable:true
type:array
includedResources:
description:IncludedResources specifies the resources to
which this hook spec applies. If empty, it applies to
all resources.
items:
type:string
nullable:true
type:array
labelSelector:
description:LabelSelector, if specified, filters the resources
to which this hook spec applies.
nullable:true
properties:
matchExpressions:
description:matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description:A label selector requirement is a selector
that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description:key is the label key that the selector
applies to.
type:string
operator:
description:operator represents a key's relationship
to a set of values. Valid operators are In,
NotIn, Exists and DoesNotExist.
type:string
values:
description:values is an array of string values.
If the operator is In or NotIn, the values array
must be non-empty. If the operator is Exists
or DoesNotExist, the values array must be empty.
This array is replaced during a strategic merge
patch.
items:
type:string
type:array
required:
- key
- operator
type:object
type:array
matchLabels:
additionalProperties:
type:string
description:matchLabels is a map of {key,value} pairs.
A single {key,value} in the matchLabels map is equivalent
to an element of matchExpressions, whose key field
is "key", the operator is "In", and the values array
contains only "value". The requirements are ANDed.
type:object
type:object
name:
description:Name is the name of this hook.
type:string
post:
description:PostHooks is a list of BackupResourceHooks
to execute after storing the item in the backup. These
are executed after all "additional items" from item actions
are processed.
items:
description:BackupResourceHook defines a hook for a resource.
properties:
exec:
description:Exec defines an exec hook.
properties:
command:
description:Command is the command and arguments
to execute.
items:
type:string
minItems:1
type:array
container:
description:Container is the container in the
pod where the command should be executed. If
not specified, the pod's first container is
used.
type:string
onError:
description:OnError specifies how Velero should
behave if it encounters an error executing this
hook.
enum:
- Continue
- Fail
type:string
timeout:
description:Timeout defines the maximum amount
of time Velero should wait for the hook to complete
before considering the execution a failure.
type:string
required:
- command
type:object
required:
- exec
type:object
type:array
pre:
description:PreHooks is a list of BackupResourceHooks to
execute prior to storing the item in the backup. These
are executed before any "additional items" from item actions
are processed.
items:
description:BackupResourceHook defines a hook for a resource.
properties:
exec:
description:Exec defines an exec hook.
properties:
command:
description:Command is the command and arguments
to execute.
items:
type:string
minItems:1
type:array
container:
description:Container is the container in the
pod where the command should be executed. If
not specified, the pod's first container is
used.
type:string
onError:
description:OnError specifies how Velero should
behave if it encounters an error executing this
hook.
enum:
- Continue
- Fail
type:string
timeout:
description:Timeout defines the maximum amount
of time Velero should wait for the hook to complete
- description:Backup Storage Location status such as Available/Unavailable
jsonPath:.status.phase
name:Phase
type:string
- description:LastValidationTime is the last time the backup store location was
validated
jsonPath:.status.lastValidationTime
name:Last Validated
type:date
- jsonPath:.metadata.creationTimestamp
name:Age
type:date
- description:Default backup storage location
jsonPath:.spec.default
name:Default
type:boolean
name:v1
schema:
openAPIV3Schema:
description:BackupStorageLocation is a location where Velero stores backup
objects
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:BackupStorageLocationSpec defines the desired state of a
Velero BackupStorageLocation
properties:
accessMode:
description:AccessMode defines the permissions for the backup storage
location.
enum:
- ReadOnly
- ReadWrite
type:string
backupSyncPeriod:
description:BackupSyncPeriod defines how frequently to sync backup
API objects from object storage. A value of 0 disables sync.
nullable:true
type:string
config:
additionalProperties:
type:string
description:Config is for provider-specific configuration fields.
type:object
credential:
description:Credential contains the credential information intended
to be used with this location
properties:
key:
description:The key of the secret to select from. Must be a
valid secret key.
type:string
name:
description: 'Name of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO:Add other useful fields. apiVersion, kind, uid?'
type:string
optional:
description:Specify whether the Secret or its key must be defined
type:boolean
required:
- key
type:object
default:
description:Default indicates this location is the default backup
storage location.
type:boolean
objectStorage:
description:ObjectStorageLocation specifies the settings necessary
to connect to a provider's object storage.
properties:
bucket:
description:Bucket is the bucket to use for object storage.
type:string
caCert:
description:CACert defines a CA bundle to use when verifying
TLS connections to the provider.
format:byte
type:string
prefix:
description:Prefix is the path inside a bucket to use for Velero
storage. Optional.
type:string
required:
- bucket
type:object
provider:
description:Provider is the provider of the backup storage.
type:string
validationFrequency:
description:ValidationFrequency defines how frequently to validate
the corresponding object storage. A value of 0 disables validation.
nullable:true
type:string
required:
- objectStorage
- provider
type:object
status:
description:BackupStorageLocationStatus defines the observed state of
BackupStorageLocation
properties:
accessMode:
description:"AccessMode is an unused field. \n Deprecated: there
is now an AccessMode field on the Spec and this field will be removed
entirely as of v2.0."
enum:
- ReadOnly
- ReadWrite
type:string
lastSyncedRevision:
description:"LastSyncedRevision is the value of the `metadata/revision`
file in the backup storage location the last time the BSL's contents
were synced into the cluster. \n Deprecated: this field is no longer
updated or used for detecting changes to the location's contents
and will be removed entirely in v2.0."
type:string
lastSyncedTime:
description:LastSyncedTime is the last time the contents of the location
were synced into the cluster.
format:date-time
nullable:true
type:string
lastValidationTime:
description:LastValidationTime is the last time the backup store
location was validated the cluster.
format:date-time
nullable:true
type:string
message:
description:Message is a message about the backup storage location's
status.
type:string
phase:
description:Phase is the current state of the BackupStorageLocation.
- description:The name of the backup to be deleted
jsonPath:.spec.backupName
name:BackupName
type:string
- description:The status of the deletion request
jsonPath:.status.phase
name:Status
type:string
name:v1
schema:
openAPIV3Schema:
description:DeleteBackupRequest is a request to delete one or more backups.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:DeleteBackupRequestSpec is the specification for which backups
to delete.
properties:
backupName:
type:string
required:
- backupName
type:object
status:
description:DeleteBackupRequestStatus is the current status of a DeleteBackupRequest.
properties:
errors:
description:Errors contains any errors that were encountered during
the deletion process.
items:
type:string
nullable:true
type:array
phase:
description:Phase is the current state of the DeleteBackupRequest.
description:DownloadRequest is a request to download an artifact from backup
object storage, such as a backup log file.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:DownloadRequestSpec is the specification for a download request.
properties:
target:
description:Target is what to download (e.g. logs for a backup).
properties:
kind:
description:Kind is the type of file to download.
enum:
- BackupLog
- BackupContents
- BackupVolumeSnapshots
- BackupItemSnapshots
- BackupResourceList
- RestoreLog
- RestoreResults
- CSIBackupVolumeSnapshots
- CSIBackupVolumeSnapshotContents
type:string
name:
description:Name is the name of the kubernetes resource with
which the file is associated.
type:string
required:
- kind
- name
type:object
required:
- target
type:object
status:
description:DownloadRequestStatus is the current status of a DownloadRequest.
properties:
downloadURL:
description:DownloadURL contains the pre-signed URL for the target
file.
type:string
expiration:
description:Expiration is when this DownloadRequest expires and can
be deleted by the system.
format:date-time
nullable:true
type:string
phase:
description:Phase is the current state of the DownloadRequest.
- description:Pod Volume Backup status such as New/InProgress
jsonPath:.status.phase
name:Status
type:string
- description:Time when this backup was started
jsonPath:.status.startTimestamp
name:Created
type:date
- description:Namespace of the pod containing the volume to be backed up
jsonPath:.spec.pod.namespace
name:Namespace
type:string
- description:Name of the pod containing the volume to be backed up
jsonPath:.spec.pod.name
name:Pod
type:string
- description:Name of the volume to be backed up
jsonPath:.spec.volume
name:Volume
type:string
- description:Restic repository identifier for this backup
jsonPath:.spec.repoIdentifier
name:Restic Repo
type:string
- description:Name of the Backup Storage Location where this backup should be
stored
jsonPath:.spec.backupStorageLocation
name:Storage Location
type:string
- jsonPath:.metadata.creationTimestamp
name:Age
type:date
name:v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:PodVolumeBackupSpec is the specification for a PodVolumeBackup.
properties:
backupStorageLocation:
description:BackupStorageLocation is the name of the backup storage
location where the restic repository is stored.
type:string
node:
description:Node is the name of the node that the Pod is running
on.
type:string
pod:
description:Pod is a reference to the pod containing the volume to
be backed up.
properties:
apiVersion:
description:API version of the referent.
type:string
fieldPath:
description:'If referring to a piece of an object instead of
an entire object, this string should contain a valid JSON/Go
field access statement, such as desiredState.manifest.containers[2].
For example, if the object reference is to a container within
a pod, this would take on a value like: "spec.containers{name}"
(where "name" refers to the name of the container that triggered
the event) or if no container name is specified "spec.containers[2]"
(container with index 2 in this pod). This syntax is chosen
only to have some well-defined way of referencing a part of
an object. TODO: this design is not final and this field is
subject to change in the future.'
type:string
kind:
description: 'Kind of the referent. More info:https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
name:
description: 'Name of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
type:string
namespace:
description: 'Namespace of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
type:string
resourceVersion:
description:'Specific resourceVersion to which this reference
is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
type:string
uid:
description: 'UID of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
type:string
type:object
repoIdentifier:
description:RepoIdentifier is the restic repository identifier.
type:string
tags:
additionalProperties:
type:string
description:Tags are a map of key-value pairs that should be applied
to the volume backup as tags.
type:object
volume:
description:Volume is the name of the volume within the Pod to be
backed up.
type:string
required:
- backupStorageLocation
- node
- pod
- repoIdentifier
- volume
type:object
status:
description:PodVolumeBackupStatus is the current status of a PodVolumeBackup.
properties:
completionTimestamp:
description:CompletionTimestamp records the time a backup was completed.
Completion time is recorded even on failed backups. Completion time
is recorded before uploading the backup object. The server's time
is used for CompletionTimestamps
format:date-time
nullable:true
type:string
message:
description:Message is a message about the pod volume backup's status.
type:string
path:
description:Path is the full path within the controller pod being
backed up.
type:string
phase:
description:Phase is the current state of the PodVolumeBackup.
enum:
- New
- InProgress
- Completed
- Failed
type:string
progress:
description:Progress holds the total number of bytes of the volume
and the current number of backed up bytes. This can be used to display
progress information about the backup operation.
properties:
bytesDone:
format:int64
type:integer
totalBytes:
format:int64
type:integer
type:object
snapshotID:
description:SnapshotID is the identifier for the snapshot of the
pod volume.
type:string
startTimestamp:
description:StartTimestamp records the time a backup was started.
Separate from CreationTimestamp, since that value changes on restores.
- description:Namespace of the pod containing the volume to be restored
jsonPath:.spec.pod.namespace
name:Namespace
type:string
- description:Name of the pod containing the volume to be restored
jsonPath:.spec.pod.name
name:Pod
type:string
- description:Name of the volume to be restored
jsonPath:.spec.volume
name:Volume
type:string
- description:Pod Volume Restore status such as New/InProgress
jsonPath:.status.phase
name:Status
type:string
- description:Pod Volume Restore status such as New/InProgress
format:int64
jsonPath:.status.progress.totalBytes
name:TotalBytes
type:integer
- description:Pod Volume Restore status such as New/InProgress
format:int64
jsonPath:.status.progress.bytesDone
name:BytesDone
type:integer
- jsonPath:.metadata.creationTimestamp
name:Age
type:date
name:v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:PodVolumeRestoreSpec is the specification for a PodVolumeRestore.
properties:
backupStorageLocation:
description:BackupStorageLocation is the name of the backup storage
location where the restic repository is stored.
type:string
pod:
description:Pod is a reference to the pod containing the volume to
be restored.
properties:
apiVersion:
description:API version of the referent.
type:string
fieldPath:
description:'If referring to a piece of an object instead of
an entire object, this string should contain a valid JSON/Go
field access statement, such as desiredState.manifest.containers[2].
For example, if the object reference is to a container within
a pod, this would take on a value like: "spec.containers{name}"
(where "name" refers to the name of the container that triggered
the event) or if no container name is specified "spec.containers[2]"
(container with index 2 in this pod). This syntax is chosen
only to have some well-defined way of referencing a part of
an object. TODO: this design is not final and this field is
subject to change in the future.'
type:string
kind:
description: 'Kind of the referent. More info:https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
name:
description: 'Name of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
type:string
namespace:
description: 'Namespace of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
type:string
resourceVersion:
description:'Specific resourceVersion to which this reference
is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
type:string
uid:
description: 'UID of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
type:string
type:object
repoIdentifier:
description:RepoIdentifier is the restic repository identifier.
type:string
snapshotID:
description:SnapshotID is the ID of the volume snapshot to be restored.
type:string
volume:
description:Volume is the name of the volume within the Pod to be
restored.
type:string
required:
- backupStorageLocation
- pod
- repoIdentifier
- snapshotID
- volume
type:object
status:
description:PodVolumeRestoreStatus is the current status of a PodVolumeRestore.
properties:
completionTimestamp:
description:CompletionTimestamp records the time a restore was completed.
Completion time is recorded even on failed restores. The server's
time is used for CompletionTimestamps
format:date-time
nullable:true
type:string
message:
description:Message is a message about the pod volume restore's status.
type:string
phase:
description:Phase is the current state of the PodVolumeRestore.
enum:
- New
- InProgress
- Completed
- Failed
type:string
progress:
description:Progress holds the total number of bytes of the snapshot
and the current number of restored bytes. This can be used to display
progress information about the restore operation.
properties:
bytesDone:
format:int64
type:integer
totalBytes:
format:int64
type:integer
type:object
startTimestamp:
description:StartTimestamp records the time a restore was started.
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:ResticRepositorySpec is the specification for a ResticRepository.
properties:
backupStorageLocation:
description:BackupStorageLocation is the name of the BackupStorageLocation
that should contain this repository.
type:string
maintenanceFrequency:
description:MaintenanceFrequency is how often maintenance should
be run.
type:string
resticIdentifier:
description:ResticIdentifier is the full restic-compatible string
for identifying this repository.
type:string
volumeNamespace:
description:VolumeNamespace is the namespace this restic repository
contains pod volume backups for.
type:string
required:
- backupStorageLocation
- maintenanceFrequency
- resticIdentifier
- volumeNamespace
type:object
status:
description:ResticRepositoryStatus is the current status of a ResticRepository.
properties:
lastMaintenanceTime:
description:LastMaintenanceTime is the last time maintenance was
run.
format:date-time
nullable:true
type:string
message:
description:Message is a message about the current status of the
ResticRepository.
type:string
phase:
description:Phase is the current state of the ResticRepository.
- description:A Cron expression defining when to run the Backup
jsonPath:.spec.schedule
name:Schedule
type:string
- description:The last time a Backup was run for this schedule
jsonPath:.status.lastBackup
name:LastBackup
type:date
- jsonPath:.metadata.creationTimestamp
name:Age
type:date
name:v1
schema:
openAPIV3Schema:
description:Schedule is a Velero resource that represents a pre-scheduled
or periodic Backup that should be run.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:ScheduleSpec defines the specification for a Velero schedule
properties:
schedule:
description:Schedule is a Cron expression defining when to run the
Backup.
type:string
template:
description:Template is the definition of the Backup to be run on
the provided schedule
properties:
csiSnapshotTimeout:
description:CSISnapshotTimeout specifies the time used to wait
for CSI VolumeSnapshot status turns to ReadyToUse during creation,
before returning error as timeout. The default value is 10 minute.
type:string
defaultVolumesToRestic:
description:DefaultVolumesToRestic specifies whether restic should
be used to take a backup of all pod volumes by default.
type:boolean
excludedNamespaces:
description:ExcludedNamespaces contains a list of namespaces
that are not included in the backup.
items:
type:string
nullable:true
type:array
excludedResources:
description:ExcludedResources is a slice of resource names that
are not included in the backup.
items:
type:string
nullable:true
type:array
hooks:
description:Hooks represent custom behaviors that should be executed
at different phases of the backup.
properties:
resources:
description:Resources are hooks that should be executed when
backing up individual instances of a resource.
items:
description:BackupResourceHookSpec defines one or more
BackupResourceHooks that should be executed based on the
rules defined for namespaces, resources, and label selector.
properties:
excludedNamespaces:
description:ExcludedNamespaces specifies the namespaces
to which this hook spec does not apply.
items:
type:string
nullable:true
type:array
excludedResources:
description:ExcludedResources specifies the resources
to which this hook spec does not apply.
items:
type:string
nullable:true
type:array
includedNamespaces:
description:IncludedNamespaces specifies the namespaces
to which this hook spec applies. If empty, it applies
to all namespaces.
items:
type:string
nullable:true
type:array
includedResources:
description:IncludedResources specifies the resources
to which this hook spec applies. If empty, it applies
to all resources.
items:
type:string
nullable:true
type:array
labelSelector:
description:LabelSelector, if specified, filters the
resources to which this hook spec applies.
nullable:true
properties:
matchExpressions:
description:matchExpressions is a list of label
selector requirements. The requirements are ANDed.
items:
description:A label selector requirement is a
selector that contains values, a key, and an
operator that relates the key and values.
properties:
key:
description:key is the label key that the
selector applies to.
type:string
operator:
description:operator represents a key's relationship
to a set of values. Valid operators are
In, NotIn, Exists and DoesNotExist.
type:string
values:
description:values is an array of string
values. If the operator is In or NotIn,
the values array must be non-empty. If the
operator is Exists or DoesNotExist, the
values array must be empty. This array is
replaced during a strategic merge patch.
items:
type:string
type:array
required:
- key
- operator
type:object
type:array
matchLabels:
additionalProperties:
type:string
description:matchLabels is a map of {key,value}
pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions,
whose key field is "key", the operator is "In",
and the values array contains only "value". The
requirements are ANDed.
type:object
type:object
name:
description:Name is the name of this hook.
type:string
post:
description:PostHooks is a list of BackupResourceHooks
to execute after storing the item in the backup. These
are executed after all "additional items" from item
actions are processed.
items:
description:BackupResourceHook defines a hook for
a resource.
properties:
exec:
description:Exec defines an exec hook.
properties:
command:
description:Command is the command and arguments
to execute.
items:
type:string
minItems:1
type:array
container:
description:Container is the container in
the pod where the command should be executed.
If not specified, the pod's first container
is used.
type:string
onError:
description:OnError specifies how Velero
should behave if it encounters an error
executing this hook.
enum:
- Continue
- Fail
type:string
timeout:
description:Timeout defines the maximum amount
of time Velero should wait for the hook
to complete before considering the execution
a failure.
type:string
required:
- command
type:object
required:
- exec
type:object
type:array
pre:
description:PreHooks is a list of BackupResourceHooks
to execute prior to storing the item in the backup.
These are executed before any "additional items" from
description:ServerStatusRequest is a request to access current status information
about the Velero server.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:ServerStatusRequestSpec is the specification for a ServerStatusRequest.
type:object
status:
description:ServerStatusRequestStatus is the current status of a ServerStatusRequest.
properties:
phase:
description:Phase is the current lifecycle phase of the ServerStatusRequest.
enum:
- New
- Processed
type:string
plugins:
description:Plugins list information about the plugins running on
the Velero server
items:
description:PluginInfo contains attributes of a Velero plugin
properties:
kind:
type:string
name:
type:string
required:
- kind
- name
type:object
nullable:true
type:array
processedTimestamp:
description:ProcessedTimestamp is when the ServerStatusRequest was
processed by the ServerStatusRequestController.
format:date-time
nullable:true
type:string
serverVersion:
description:ServerVersion is the Velero server version.
description:VolumeSnapshotLocation is a location where Velero stores volume
snapshots.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:VolumeSnapshotLocationSpec defines the specification for
a Velero VolumeSnapshotLocation.
properties:
config:
additionalProperties:
type:string
description:Config is for provider-specific configuration fields.
type:object
provider:
description:Provider is the provider of the volume storage.
type:string
required:
- provider
type:object
status:
description:VolumeSnapshotLocationStatus describes the current status
of a Velero VolumeSnapshotLocation.
properties:
phase:
description:VolumeSnapshotLocationPhase is the lifecycle phase of
# Delete Backup and Restic Repo Resources when BSL is Deleted
## Abstract
Issue #2082 requested that with the command `velero backup-location delete <bsl name>` (implemented in Velero 1.6 with #3073), the following will be deleted:
- associated Velero backups (to be clear, these are custom Kubernetes resources called "backups" that are stored in the API server)
- associated Restic repositories (custom Kubernetes resources called "resticrepositories")
This design doc explains how the request will be implemented.
## Background
When a BSL resource is deleted from its Velero namespace, the associated custom Kubernetes resources, backups and Restic repositories, can no longer be used.
It makes sense to clean those resources up when a BSL is deleted.
## Goals
Update the `velero backup-location delete <bsl name>` command to delete associated backup and Restic repository resources in the same Velero namespace.
## Non Goals
[It was suggested](https://github.com/vmware-tanzu/velero/issues/2082#issuecomment-827951311) to fix bug #2697 alongside this issue.
However, I think that should be fixed separately because although it is similar (restore objects are not being deleted), it is also quite different.
One is adding a command feature update (this issue) and the other is a bug fix and each affect different parts of the code base.
## High-Level Design
Update the `velero backup-location delete <bsl name>` command to do the following:
- find in the same Velero namespace from which the BSL was deleted the associated backup resources and Restic repositories, called "backups.velero.io" and "resticrepositories.velero.io" respectively
- delete the resources found
The above logic will be added to [where BSLs are deleted](https://github.com/vmware-tanzu/velero/blob/main/pkg/cmd/cli/backuplocation/delete.go).
## Alternative Considered
I had considered deleting the backup files (the ones in json format and tarballs) in the BSL itself.
However, a standard use case is to back up a cluster and then restore into a new cluster.
Deleting the backup storage location in either location is not expected to remove all of the backups in the backup storage location and should not be done.
@@ -84,7 +84,7 @@ If the metadata file does not exist, this is an older backup and we cannot displ
### Fetch backup contents archive and walkthrough to list contents
Instead of recording new metadata about what resources have been backed up, we could simply download the backup contents archive and walkthrough it to list the contents everytime `velero backup describe <name> --details` is run.
Instead of recording new metadata about what resources have been backed up, we could simply download the backup contents archive and walkthrough it to list the contents everytime `velero backup describe <name> --details` is run.
The advantage of this approach is that we don't need to change any backup procedures as we already have this content, and we will also be able to list resources for older backups.
Additionally, if we wanted to expose more information about the backed up resources, we can do so without having to update what we store in the metadata.
This document proposes a solution that allows user to specify a backup order for resources of specific resource type.
## Background
During backup process, user may need to back up resources of specific type in some specific order to ensure the resources were backup properly because these resources are related and ordering might be required to preserve the consistency for the apps to recover itself from the backup image
During backup process, user may need to back up resources of specific type in some specific order to ensure the resources were backup properly because these resources are related and ordering might be required to preserve the consistency for the apps to recover itself from the backup image
(Ex: primary-secondary database pods in a cluster).
## Goals
@@ -12,7 +12,7 @@ During backup process, user may need to back up resources of specific type in so
- Use a plugin to backup an resources and all the sub resources. For example use a plugin for StatefulSet and backup pods belong to the StatefulSet in specific order. This plugin solution is not generic and requires plugin for each resource type.
## High-Level Design
User will specify a map of resource type to list resource names (separate by semicolons). Each name will be in the format "namespaceName/resourceName" to enable ordering accross namespaces. Based on this map, the resources of each resource type will be sorted by the order specified in the list of resources. If a resource instance belong to that specific type but its name is not in the order list, then it will be put behind other resources that are in the list.
User will specify a map of resource type to list resource names (separate by semicolons). Each name will be in the format "namespaceName/resourceName" to enable ordering across namespaces. Based on this map, the resources of each resource type will be sorted by the order specified in the list of resources. If a resource instance belong to that specific type but its name is not in the order list, then it will be put behind other resources that are in the list.
- In the CLI, the design proposes to use commas to separate items of a resource type and semicolon to separate key-value pairs. This follows the convention of using commas to separate items in a list (For example: --include-namespaces ns1,ns2). However, the syntax for map in labels and annotations use commas to seperate key-value pairs. So it introduces some inconsistency.
- In the CLI, the design proposes to use commas to separate items of a resource type and semicolon to separate key-value pairs. This follows the convention of using commas to separate items in a list (For example: --include-namespaces ns1,ns2). However, the syntax for map in labels and annotations use commas to separate key-value pairs. So it introduces some inconsistency.
- For pods that managed by Deployment or DaemonSet, this design may not work because the pods' name is randomly generated and if pods are restarted, they would have different names so the Backup operation may not consider the restarted pods in the sorting algorithm. This problem will be addressed when we enhance the design to use regular expression to specify the OrderResources instead of exact match.
@@ -176,7 +176,7 @@ This will allow the development to continue on the feature while it's in pre-pro
[`BackupStore.PutBackup`][9] will receive an additional argument, `volumeSnapshots io.Reader`, that contains the JSON representation of `VolumeSnapshots`.
This will be written to a file named `csi-snapshots.json.gz`.
[`defaultRestorePriorities`][11] should be rewritten to the following to accomodate proper association between the CSI objects and PVCs. `CustomResourceDefinition`s are moved up because they're necessary for creating the CSI CRDs. The CSI CRDs are created before `PersistentVolume`s and `PersistentVolumeClaim`s so that they may be used as data sources.
[`defaultRestorePriorities`][11] should be rewritten to the following to accommodate proper association between the CSI objects and PVCs. `CustomResourceDefinition`s are moved up because they're necessary for creating the CSI CRDs. The CSI CRDs are created before `PersistentVolume`s and `PersistentVolumeClaim`s so that they may be used as data sources.
GitHub issue [1565][17] represents this work.
```go
@@ -248,7 +248,7 @@ Volumes with any other `PersistentVolumeSource` set will use Velero's current Vo
### VolumeSnapshotLocations and VolumeSnapshotClasses
Velero uses its own `VolumeSnapshotLocation` CRDs to specify configuration options for a given storage system.
In Velero, this often includes topology information such as regions or availibility zones, as well as credential information.
In Velero, this often includes topology information such as regions or availability zones, as well as credential information.
CSI volume snapshotting has a `VolumeSnapshotClass` CRD which also contains configuration options for a given storage system, but these options are not the same as those that Velero would use.
Since CSI volume snapshotting is operating within the same storage system that manages the volumes already, it does not need the same topology or credential information that Velero does.
@@ -269,7 +269,7 @@ Additionally, the VolumeSnapshotter plugins and CSI volume snapshot drivers over
Thus, there's not a logical place to fit the creation of VolumeSnapshot creation in the VolumeSnapshotter interface.
* Implement CSI logic directly in Velero core code.
The plugins could be packaged separately, but that doesn't necessarily make sense with server and client changes being made to accomodate CSI snapshot lookup.
The plugins could be packaged separately, but that doesn't necessarily make sense with server and client changes being made to accommodate CSI snapshot lookup.
* Implementing the CSI logic entirely in external plugins.
As mentioned above, the necessary plugins for `PersistentVolumeClaim`, `VolumeSnapshot`, and `VolumeSnapshotContent` could be hosted out-out-of-tree from Velero.
@@ -28,7 +28,7 @@ This document proposes adding _controller-tools_ to the project to automatically
_controller-tools_ works by reading the Go files that contain the API type definitions.
It uses a combination of the struct fields, types, tags and comments to build the OpenAPIv3 schema for the CRDs. The tooling makes some assumptions based on conventions followed in upstream Kubernetes and the ecosystem, which involves some changes to the Velero API type definitions, especially around optional fields.
In order for _controller-tools_ to read the Go files containing Velero API type defintiions, the CRDs need to be generated at build time, as these files are not available at runtime (i.e. the Go files are not accessible by the compiled binary).
In order for _controller-tools_ to read the Go files containing Velero API type definitions, the CRDs need to be generated at build time, as these files are not available at runtime (i.e. the Go files are not accessible by the compiled binary).
These generated CRD manifests (YAML) will then need to be available to the `pkg/install` package for it to include when installing Velero resources.
@@ -45,7 +45,7 @@ Currently, the Velero repository sits under the Heptio GitHub organization. With
### Notes/How-Tos
#### Transfering the GH repository
#### Transferring the GH repository
All action items needed for the repo transfer are listed in the Todo list above. For details about what gets moved and other info, this is the GH documentation: https://help.github.com/en/articles/transferring-a-repository
@@ -57,7 +57,7 @@ Someone with owner permission on the new repository needs to go to their Travis
After this, webhook notifications can be added following these instructions: https://docs.travis-ci.com/user/notifications/#configuring-webhook-notifications.
#### Transfering ZenHub
#### Transferring ZenHub
Pre-requisite: A new Zenhub account must exist for a vmware or vmware-tanzu organization.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.