Check whether the namespaces specified in the
backup.Spec.IncludeNamespaces exist during backup resource collcetion
If not, log error to mark the backup as PartiallyFailed.
Signed-off-by: Xun Jiang <blackpigletbruce@gmail.com>
This commit fixes#7849.
It will use PVC instead of PV to track CSI snapshots to generate restore
volume info metadata. So that in the case the PVC is not bound to PV
the metadata can be populated correctly.
Signed-off-by: Daniel Jiang <daniel.jiang@broadcom.com>
When dry-run the tag-release.sh, there's an error
"fatal: detected dubious ownership in repository at
'/github.com/vmware-tanzu/velero'"
This commit works around this issue to make sure "tag-release.sh"
can finish successful
Signed-off-by: Daniel Jiang <daniel.jiang@broadcom.com>
Tweak the command and remove the sections which include upgrading from
older versions, given v1.13.x is a prerequisite.
Signed-off-by: Daniel Jiang <daniel.jiang@broadcom.com>
As per PR #7281, if repository count is more than 1, then snapshots deletion is achieved with a fast way, then we should have more than 1 FS backup repository per backup.
Signed-off-by: danfengl <danfengl@vmware.com>
1. In data movement scenario, volumesnapshotcontent by Velero backup will be deleted instead of retained in CSI scenaito, so add
a checkpoint for data movement scenario to verify no volumesnapshotcontent left after Velero backup;
2. Fix global context varaible issue, context varaible is not effective due to it's initialized right after the very beginning of
all tests instead of beginning of each test, so if someone script a new E2E test and did not overwrite it in the test body, then it
will fail the test if it was triggerd one hour later;
3. Due to CSI plugin is deprecated, it breaked down migration tests, because v1.13 still needs to install CSI plugin for the test.
Signed-off-by: danfengl <danfengl@vmware.com>
1. Add sleep for native snapshot tests when using test.go interface;
2. Add --confirm for velero plugin add CLI as new feature introduced.
Signed-off-by: danfengl <danfengl@vmware.com>
This commit makes change to CLI so `velero restore describe` will
download restore volume info and render the CSI snapshot restores based
on its content.
Signed-off-by: Daniel Jiang <daniel.jiang@broadcom.com>
This commit bumps up the golang for building and testing velero to v1.22
It also updates controller-gen to v0.14.0 to fix an issue under new
versino of go.
More details see https://github.com/golang/go/issues/65637
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
The wait error changed from `timed out waiting for the condition`
to `context deadline exceeded`.
Signed-off-by: Xun Jiang <blackpigletbruce@gmail.com>
handleSkippedPVHasRetainPolicy
According to comment, calling executePVAction aims to reset PV's
claimRef, but the reset logic was moved into resetVolumeBindingInfo
since release-1.4.
Signed-off-by: Xun Jiang <blackpigletbruce@gmail.com>
Check the existence of the namespaces provided in the "--include-namespaces" opt
ion and reports validation error if not found
Fixes#7431
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
This commit makes sure when kopia connects to the repository the
crendentials file specified in BSL.spec.config has the higher priority over
Pod Environment credentials when IRSA is configured.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
When debugging this error it is currently hard to identify what
CRD is causing the issue. This is particularly difficult when
dealing with over a hundred CRDs.
Signed-off-by: Jose Arevalo <jose.matias.arevalo@gmail.com>
Make "disable-informer-cache" option false(enabled) by default to keep it consi
stent with the help message
Fixes#7264
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
fixes#7263
This commit makes the data structures more consistent, that namespaces,
as cluster scoped resource will not have "targetNamespace" in the
"restoreableItem" instance.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. Add sleep to avoid snapshot limitation issue https://docs.aws.amazon.com/AWSEC2/latest/APIReference/errors-overview.html#:~:text=SnapshotCreationPerVolumeRateExceeded;
2. Move InstallVelero variable out of struct of Veleroconfig as a global one since it's not for controlling any individual case;
3. Unskip migration test case on AWS pipeline, because we added a new EKS pipeline and deleted TKG AWS pipline in internal E2E test, so this restriction for TKG AWS pipline is no long existed;
4. Skip retainPV test on vSphere pipeline due to PV longtime bounding issue;
5. Fix failing get snapshot by CSI from EC2 issue, snapshot by CSI has no label of backup name.
Signed-off-by: danfengl <danfengl@vmware.com>
VolumeInfo contains several sub-structures. They are filled for
different scenarios. Do not generate empty structure for the
not filled sub-structures.
Signed-off-by: Xun Jiang <jxun@vmware.com>
Update CSIVolumeSnapshotsCompleted in backup's status and the metric
during backup finalize stage according to async operations content.
Signed-off-by: Xun Jiang <jxun@vmware.com>
Fixes#1970
Namespaces will be handled as cluster-scope resource, but for
consistency they will still created via "Ensure namespace" flow for
consistency.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commit makes sure if a PV is not taken snapshot b/c the flag
SnapshotVolumes is set to false in a backup CR, the PV is also also
tracked as skipped in the tracker.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Modify design according to comments.
Add PVInfo structure.
Add backup VolumeInfo's object storage's put and get methods.
Signed-off-by: Xun Jiang <jxun@vmware.com>
Remove dependecy of generate client from pkg/cmd/cli/snapshotLocation.
Remove the Velero generated informer from PVB and PVR.
Remove dependency of generated client from pkg/podvolume directory.
Replace generated codec with runtime codec.
Signed-off-by: Xun Jiang <jxun@vmware.com>
enabled, before executing the action.
The DeleteItemAction is not checked, because the DIA doesn't have a
method to get the action's plugin name.
This should be OK, because the CSI will check whether the VS and VSC
have a backup name annotation. If the VS and VSC is not handled by
the CSI plugin, then they don't have the annotation.
Signed-off-by: Xun Jiang <jxun@vmware.com>
PVC block mode backup and restore introduced some OS specific
system calls. Those calls are not available for Windows, so
add both non Windows version and Windows version code, and
return error for block mode on the Windows platform.
Signed-off-by: Xun Jiang <jxun@vmware.com>
Use informer cache with dynamic client for Get calls on restore
When enabled, also make the Get call before create.
Add server and install parameter to allow disabling this feature,
but enable by default
Signed-off-by: Scott Seago <sseago@redhat.com>
When creating resources with generateName, apimachinery
does not guarantee uniqueness when it appends the random
suffix to the generateName stub, so if it fails with
already exists error, we need to retry.
Signed-off-by: Scott Seago <sseago@redhat.com>
* doc: Alert that plugins run as binaries when turning on debug logs
Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>
* fixup! doc: Alert that plugins run as binaries when turning on debug logs
Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>
* fixup! doc: Alert that plugins run as binaries when turning on debug logs
Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>
* fixup! doc: Alert that plugins run as binaries when turning on debug logs
Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>
---------
Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>
When preparing a backup repository, Velero tries to connect to it, if fails then create it. The repository status always records the error reported by creation but the real reason maybe caused by the connect operation. This is confuseing and hard to debug
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
This commit introduces our own Azure storage provider by wrapping Kopia's implementation rather than contributing to upstream based on the following considerations:
1. Velero needs the capability to interact with the repository concurrently while Kopia doesn't, this will increase the complexity of Kopia if we contribute to upstream
2. The configuration items provided by Velero and Kopia are conflict, e.g. Velero supports customizing storage account URI which is a full path while Kopia supports customizing storage account domain which is part of the URI. We need to consider the backward compatibility and upgrade case if we contribute to upstream which needs extra efforts
3. Contribute to upstream is a longer cycle when we need to introduce new changes. With this commit, we no longer depends on upstream for the Azure storage provider part and is easy for us to maintain
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
1. Skip deleting the restore files from storage if the backup/BSL is not found
2. Allow deleting the restore files from storage even though the BSL is readonly
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
1. Capture Velero pod log and K8S cluster event;
2. Fix wrong path of storageclass yaml file issue caused by pert test;
3. Fix change storageclass test issue that no sc named 'default' in EKS cluster;
4. Support AWS credential as config format;
5. Support more E2E script input parameters like standy cluster plugins and provider.
Signed-off-by: danfengl <danfengl@vmware.com>
Enlarge throttle of UT case TestThrottle_ShouldOutput to avoid occasional CI
failure due to timeout caused by test environment's CPU speed
Signed-off-by: Xun Jiang <jxun@vmware.com>
This commit introduces a deleteItemAction which writes a temporary configmap to
record the snapshot info so that the controller can trigger repo manager
to remove the snapshot
This process is a bit chatty and we should consider to refactor the code
so it's easier to connect to the repo directly in the DIA
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
* fix: Typos and add more spell checking rules to CI
Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>
* fixup! fix: Typos and add more spell checking rules to CI
Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>
* fixup! fix: Typos and add more spell checking rules to CI
Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>
* fixup! fix: Typos and add more spell checking rules to CI
Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>
* fixup! fix: Typos and add more spell checking rules to CI
Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>
* fixup! fix: Typos and add more spell checking rules to CI
Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>
* fixup! fix: Typos and add more spell checking rules to CI
Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>
* fixup! fix: Typos and add more spell checking rules to CI
Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>
---------
Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>
1. In K8S v1.27 API Version v1beta1 for CR volumesnapshotclass is deprcated, so E2E test should adapt both API versions to cover all K8S versio;
2. Support getting additional plugin from input;
3. Velero version and plugin map should not deprated version older than v1.10, because upgrade test will use them.
Signed-off-by: danfengl <danfengl@vmware.com>
when running `go mod why -m github.com/kopia/kopia` in velero-plugins prior to this change you will see following
```
❯ go mod why -m github.com/kopia/kopia
github.com/konveyor/openshift-velero-plugin/velero-plugins
github.com/vmware-tanzu/velero/pkg/plugin/framework
github.com/vmware-tanzu/velero/pkg/util/logging
github.com/kopia/kopia/repo/logging
```
after
```
❯ go mod why -m github.com/kopia/kopia
(main module does not need module github.com/kopia/kopia)
```
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
Following the examples instructions[1], the nginx-deployment is not
backed up or restored. Add a label to the deployment so it will be
backed up and restored.
Similar change is needed for `examples/nginx-app/with-pv.yaml` but I did
not try that example.
[1] https://velero.io/docs/v1.11/contributions/minio/Fixes#6347
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
1. Because VolumeSnapshot and VolumeSnapshotContent CRs are not kept after backup completed,
don't persist them in the backup metadata.
2. Add some builder methods needed by CSI plugin.
Signed-off-by: Xun Jiang <jxun@vmware.com>
1. Bumpup velero version to the latest 2 versions in upgrade script;
2. Bumpup velero verioin to the latest 1 vesion in migration script;
3. Bring B/R with restic test back in vSphere pipeline since vSphere plugin issue fix was included
in v1.5;
4. Disable nodeport test in AWS pipeline since AWS k8s version bumpup;
5. Prepare for data mover test, allow object store provider diffrent from cloud provider.
Signed-off-by: danfengl <danfengl@vmware.com>
Due to the logic moving to plugin, and the plugin cannot read the
Velero server's resourceTimeout setting, add the resourceTimeout
in the backup annotation to pass to plugin.
Remove VolumeSnapshotContent reset code from Velero server.
Signed-off-by: Xun Jiang <jxun@vmware.com>
For some use cases, namespaced-scope resources are inluded into backup,
but the namespaces are not included due to filters setting.
To do this, removing label selector filter from namespace resource.
Namespace resource only honor namespace exclude/include filters.
Signed-off-by: Xun Jiang <jxun@vmware.com>
This commit skips updating the restore progress, in the first loop for
restoration when CRDs are handled, so that the misleading "totalItem"
will not appear in the CR.
Fixes#5990
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. Fix context issues produced by previous PR, increase timeout or add case scpoed global timeout param to make backup/restore command timeout configurable.
2. Add global param for storage class name using by test cases;
3. Fix param DefaultVolumesToFsBackup usage issue: set DefaultVolumesToFsBackup to false in backup CLI in case it was set to true in install CLI.
4. Make namespace names of each namespace mapping test unique from being interfered by each other.
Signed-off-by: danfengl <danfengl@vmware.com>
The log message should be clarified, otherwise when a user chooses to do
the backup via podvolme there will be confusing logs, but actually it's
just skipping the BIA for CSI plugin.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Restore Services before Clusters so they can be adopted by AKO-operator and no new Services will be created for the same clusters
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
Update README to clarify the backward compatibility.
Trivial update to the support process to reflect how issues are labeled
as for now.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1) default frequency 10s
2) per-reconcile log is now Debug not info
3) added predicate to reduce reconcile events
Signed-off-by: Scott Seago <sseago@redhat.com>
Add secret restore item action to handle service account token secret:
1. Skip the restoration for the auto-created service account token secret
2. Remove several fields for non-auto-created service account token secret to make sure the secret can be restored
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
Use the same pvb/pvr update functions across pkg/controller and pkg/cli/nodeagent for consistency of behavior
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
The option "--service-account-name" is to be added to that user can use
an existing service account for velero and node-agent pods. This is
helpful for users who wanna use IRSA.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Update the group members of "maintainers" and "tech-writer" to reflect
the change in the team.
As for the group "tech-writer" I just selected a few members from
maintains team who has been working on velero for a relatively longer
time.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1) clarification around Cancel() return values
2) updates to itemoperation json upload to account for progress
3) update to OperationProgress struct to avoid duplicate parameter
4) update new B/R phase name to WaitingForPluginOperationsPartiallyFailed for consistency
Signed-off-by: Scott Seago <sseago@redhat.com>
Due to CSIDriver is checked for Restic volume mounting path, and CSIDriver is GA and moved to storage v1 group in k8s v1.18, so update Velero v1.8, v1.9 and v1.10 compatible k8s version to 1.18-latest.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
This commit makes update to the update api-types docs to add missing
fields.
It also includes misc changes to the inline comment, and a change to
Dockerfile to make sure the build-image works on mac
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Restore ClusterBootstrap before Cluster otherwise a new default ClusterBootstrap object is create for the cluster
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
This design combines the requirements for the previously-merged
Upload Progress Monitoring design with the requirements for the
(not submitted but discussed in meetings and slack) proposed asynchronous
item action plugins into one integrated proposal.
Signed-off-by: Scott Seago <sseago@redhat.com>
Enhance the restore priorities list to support specifying the low prioritized resources that need to be r
estored in the last
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
The container name for the aws plugin is `velero-plugin-for-aws`. There was an extra `velero-` prefix in the doc.
Signed-off-by: Dave Pedu <dave@davepedu.com>
1. Fix issue of kubectl client and server mismatch version in GitAction E2E job, refer to https://github.com/elastic/cloud-on-k8s/issues/4737;
2. Adapt to the changing of keyword for involing Kpoia as fs backupper, new installtion breaked upgrade and migration tests;
3. Accept multi-labels of Ginkgo focus as input of E2E make command;
4. Distinguish workload namespace from each tests;
5. Fix issues of not using Velero util to perform Velero commands;
6. Add snapshot test case for NamespaceMapping E2E test;
7. Collect debug bundle after catching error of Velero backup or restore command;
Signed-off-by: danfengl <danfengl@vmware.com>
The RIA refactoring moved velero.RestoreItemAction into a separate
(restoreitemaction) v1 package. Unfortunately, this change would require
plugins to make code changes to locate the RestoreItemActionExecuteInput
and RestoreItemActionExecuteOutput structs.
This commit restores those structs to the original velero package, leaving
just the RestoreItemAction interface in the new v1 package.
Signed-off-by: Scott Seago <sseago@redhat.com>
This commit provides a simple contract that if the BackupItemAction
plugin sets an annotation in a resource it has handled, the additional
items will considered "must include" i.e. each of them will skip the
"include-exclude" filter, such that the plugin developer can make sure
they are included in the backup disregarding the filter setting in the
bakcup CR.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. One of API group test failed due to other PR with fix for treat PartiallyFailed as failure to collect debugbundle without wrap the origin error;
2. Fix migration test issue of wrong velero cli for backup commmand;
3. Fix wrong pararmeter name issue for pv opt-out backup test.
Signed-off-by: danfengl <danfengl@vmware.com>
When running velero backup/restore command, if the command result is "PartiallyFailed", it won't reture error as design, but we do need to know the debug information to figure out the reason, so the command output is needed to get the command result, then further action will be taken.
Signed-off-by: danfengl <danfengl@vmware.com>
Refactors the framework package to implement the plugin versioning changes
needed for BIA v1 and overall package refactoring to support plugin versions
in different packages. This should be all that's needed to move on to
v2 for BackupItemAction. The remaining plugin types still need similar
refactoring to what's being done here for BIA before attempting a
v2 implementation.
Signed-off-by: Scott Seago <sseago@redhat.com>
Refactors the clientmgmt package to implement the plugin versioning changes
needed for BIA v1 and overall package refactoring to support plugin versions
in different packages. This should be all that's needed to move on to
v2 for BackupItemAction. The remaining plugin types still need similar
refactoring to what's being done here for BIA before attempting a
v2 implementation.
Signed-off-by: Scott Seago <sseago@redhat.com>
I think is necessary this little comment about TTL expiration, because it can be confusing when the expiration time has passed and the data allocated and the snapshots are not erased at that time.
Signed-off-by: Aaron Arias <33655005+aaronariasperez@users.noreply.github.com>
If generating protoc go files from scratch, `make update` fails if
CRD generation happens first, since the protoc-generated
files are imported by the api go files.
protoc generation needs to happen earlier.
Signed-off-by: Scott Seago <sseago@redhat.com>
In determining whether a backup includes all namespaces, item_collector
checks for an empty string in the first element of the ns list. If processing
includes+excludes results in an empty list, treat this as another case
of a not-all-namespaces backup rather than crashing velero.
Signed-off-by: Scott Seago <sseago@redhat.com>
1. Add some refactored controllers initiation code into enabledRuntimeControllers.
2. Add reconciler struct initiation function for DownloadRequest and ServerStatusRequest controllers.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
This stops subheading description from showing in posted issues by default.
Signed-off-by: Tiger Kaovilai <passawit.kaovilai@gmail.com>
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
1. Clean backups after each test to avoid exceeding limitation of storage capability during E2E test;
2. Fix exlude label test issue that namespace should not be included and excluded at the same time no matter by which way to config.
Signed-off-by: danfengl <danfengl@vmware.com>
This commit adds the parameter "uploader-type" to velero server, add exposes the
setting via "velero install" in CLI.
fixes#5062
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. Also checking annotation "pv.kubernetes.io/migrated-to" to find out whether volume is provisioned by CSI.
2. Add UT cases.
Signed-off-by: Xun Jiang <jxun@vmware.com>
This commit splits the pkg/restic package into several packages to support Kopia integration works
Fixes#5055
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
1. Make the Restore hook.InitConatianer server side field pruing disable.
2. Remove restore patch in update-generate-crd-code.sh.
3. Modify related testcases.
4. Add Container fields validation in Restore Init hook.
Signed-off-by: Xun Jiang <jxun@vmware.com>
"EnableAPIGroupVersions" is set
The crd-remap-version plugin will always backup v1b1 resource for some
CRD. It impacts the feature flag `EnableAPIGroupVersions` which means to
backup all versions, and make migration fail.
In this commit the featureSet was removed from plugin server struct b/c
it blocks the parm `--features` to be populated correctly. This change
should not have negative impact b/c the attribute in server struct is never used.
Fixes#5146
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
This commit adds additional fields to podvolumebackup
and podvolumerestore. The resticrepository will be renamed to
backuprepository
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Pass in a new copy of the map of config values rather than
modifying the BSL Spec.Config and then pass in that field.
Signed-off-by: Scott Seago <sseago@redhat.com>
This commit mitigates the issue for running "make update" locally when
the network is not friendly for accessing the default "proxy.golang.org"
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Mitigate the issue mentioned in #4782
When there's a bug or misconfiguration that causes nil pointer there
will be more stack trace information to help us debug.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Fix bsl validation bug: the BSL is validated continually and doesn't respect the validation period configured
Fixes#5056
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
* move 'velero.io/exclude-from-backup' label name to const
Signed-off-by: Niu Lechuan <lechuan.niu@daocloud.io>
* add changelog file (in changelogs/unreleased) of this PR
Signed-off-by: Niu Lechuan <lechuan.niu@daocloud.io>
1. remove go.sum file from code spell check action.
2. change go version to 1.17 in CRD verify action, and add k8s 1.23 and 1.24 in verification list.
Signed-off-by: Xun Jiang <jxun@vmware.com>
Because the column and project specified by this action do not exist anymore, and Velero team doesn't use this action to assign issue and triage anymore, remove this action.
Signed-off-by: Xun Jiang <jxun@vmware.com>
Update the release steps to reflect the change in the `tag-release.sh`,
that the release branch must be created manually before RC is tagged.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. Use patch rather status patch in backup sync controller as we have disable status as sub resource
2. Set the GC period with default value if it isn't set
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
It's not necessary to set the deletion policy as the delete item action
plugin in CSI plugin will set it to Delete when the backup is deleted.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
When enabling the status as sub resource in CRD, the status will be ignored when creating the CR with status, this will cause issues when syncing backups/pvbs
Fixes#4950
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
We have made a few changes to the CSI plugin to provide official support
for AWS/Azure. This commit makes change to the docs to reflect those
changes.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Add filter functions for PeriodicalEnqueueSource.
Move BSL's valication frequency check test case to PeriodicalEnqueueSource's test.
Signed-off-by: Xun Jiang <jxun@vmware.com>
Make in-progress PVB/PVR as failed when restic controller restarts to avoid hanging backup/restore
Fixes#4772
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
1. Add checkpoint in snapshot E2E test to verify snapshot CR should be created and snapshot should be created in cloud side after backup completion;
2. Fix snapshot name issue that CSI snapshot name in cloud side is not the same with other non-CSI cloud snapshots;
Signed-off-by: danfengl <danfengl@vmware.com>
When iterating over applicable restore actions, if a non-matching label
selector is found, velero should continue to the next action rather than
returning from the restoreItem func, which ends up preventing the item's
restore entirely.
Signed-off-by: Scott Seago <sseago@redhat.com>
1. Delete VolumeSnapshot directly when DeletionPolicy set to Retain.
2. Change VolumeSnapshotContent's DeletionPolicy to Retain, then delete VolumeSnapshot. After that delete VolumeSnapshotContent and change VSC DeletionPolicy to Delete back, then re-create the VolumeSnapshotContent.
Signed-off-by: Xun Jiang <jxun@vmware.com>
This commit makes backup sync controller delete the volumesnapshot and
volumesnapshotcontent created by the backup which is cleaned up as orphan
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
Make in-progress backup/restore as failed when doing the reconcile to avoid hanging in in-progress status
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
Fixes#4760
This commit make changes in 2 parts:
1) When a volumesnapshotcontent is persisted during backup, velero will reset its
`Source` field to remove the VolumeHandle, so that the
csi-snapshotter will not try to call `CreateSnapshot` when its synced
to another cluster with a backup.
2) Make sure the referenced volumesnapshotclasses are persisted and
synced with the backup, so that when the volumesnapshotcontent is
deleted the storage snapshot is also removed.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. Add --insecure-tls for ResticManager's commands.
2. Add --insecure-tls in PodVolumeBackup and PodVolumeRestore controller.
3. Upgrade integrated Restic version to v0.13.1
4. Change --last flag in Restic command to --latest=1 due to Restic version update.
Signed-off-by: Xun Jiang <jxun@vmware.com>
As we are refactoring controllers with kubebuilder, use the controller-gen rather than code-generator to generate the deep copy methods for objects
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
- go install cmd/velero/velero.go
- go install cmd/velero-restic-restore-helper/velero-restic-restore-helper.go
Will generate binary in `$(go env GOPATH)/bin/` with the correct name.
build.sh still works the same.
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
* Add bsl related TTL gc errors to labelSelectors
* if backup label map is nil, make map
* clear label if not BSL error
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
This allows a user inspecting the restore logs to see any
errors or warnings generated by the restore so that they
will be seen even without having to use the describe cli.
Signed-off-by: Scott Seago <sseago@redhat.com>
* Add plugin versioning design doc
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Use more generic versions in scenarios section
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Address code review
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Address code review
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Modify design to allow other interface changes
The previous design assumed that only method addition would be
supported. It now includes guidance for making changes such as method
removal or signature changes.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
Co-authored-by: Bridget McErlean <bmcerlean@vmware.com>
The GINKGO_SKIP option is updated to string that can be separated by "." for "make test-e2e".
Signed-off-by: Xun Jiang <jxun@vmware.com>
Signed-off-by: Hoang, Phuong <phuong.n.hoang@dell.com>
1. Mark the BSL as "Unavailable" when gets any error
2. Add a new field "Message" to the BSL status to record the error message
Fixes#4485Fixes#4405
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
When velero is running on clusters that don't support v1beta1 CRD, the
plugin will not try to backup v1beta1 CRD.
The plugin should be kept for backward compatibility. It will be
removed when velero drop the support for k8s v1.21
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. rename zoneSeparator to gkeZoneSeparator
2. add example of regional PV's node affinity. modify test case description.
Signed-off-by: Xun Jiang <jxun@vmware.com>
Specify the risk of this parameter set to true. Add the issue first reported about this topic which includeds the google document illustrates about it.
Signed-off-by: Xun Jiang <jxun@vmware.com>
Test case description is "Deleted backups are deleted from object storage and backups deleted from object storage can be deleted locally",
in this test, only resource backup objects are target for verifition, restic repo verification is not included in this PR, and snapshot verification will be in later PR
Signed-off-by: danfengl <danfengl@vmware.com>
Fix#4499
When hook influnce multiple pods, current logic's first pod's container will overwrite the hook's exec.container parameter. That will cause the other pod fail on the hook executing.
Signed-off-by: Xun Jiang <jxun@vmware.com>
By now, only busybox:latest is used by e2e. It is already upload to gcr.io/velero-gcp/busybox:latest
Change the image to gcr.io to avoid pulling rate limitation from docker hub.
Signed-off-by: Xun Jiang <jxun@vmware.com>
Push to GCR in github workflow to faciliate some environments that have rate limitation to docker hub, e.g. vSphere.
<root@jxun-jumpserver.c.velero-gcp.internal>
Signed-off-by: Xun Jiang <jxun@vmware.com>
Since Itemsnapshotter plugin is still WIP,
this commit removes the reference and the deprecation of volumeSnapshotter plugin
from the doc to avoid confusion.
We'll update the doc when it's ready and we have a reference
implementation.
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
* Use OrderedResources in schedules
Make ParseOrderedResources public for use in schedules
Add changelog
Signed-off-by: Dominic <dominic@xdnx.org>
* Rename function in comment section
Signed-off-by: Dominic <dominic@xdnx.org>
* #4067 Initial design of the new plugins - pre-post backup and restore
Signed-off-by: Rafael Brito <rbrito@vmware.com>
* Update new-prepost-backuprestore-plugin-hooks.md
* Updated design doc as per feedback
Signed-off-by: Rafael Brito <rbrito@vmware.com>
* Adding design changes as per feedback
* Update design on prepost-backup-restore plugins
* More color on how to call plugins
Signed-off-by: Rafael Brito <rbrito@vmware.com>
* Proposing annotations to skip plugin execution
Signed-off-by: Rafael Brito <rbrito@vmware.com>
We introduces the installation option "--default-restic-prune-frequency" to make restic prune frequency configuration in the previous release, but there is a bug that make the option don't take effect. This commit fixes the bug by removing the evaluation part. The restic repository controller will take care the prune frequency for the repository
Fixes#3062
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
Check the existence of the expected service when ignoring the NodePort already allocated error
Fixes 2308
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
Test case description is "Deleted backups are deleted from object storage and backups deleted from object storage can be deleted locally",
in this test, only resource backup objects are target for verifition, restic repo verification is not included in this PR, and snapshot verification will be in later PR
Signed-off-by: danfengl <danfengl@vmware.com>
* Migrate backup sync controller from code-generator to kubebuilder
1. use kubebuilder's reconcile logic to replace controller's old logic.
2. use ginkgo and gomega to replace testing.
Signed-off-by: Xun Jiang <jxun@vmware.com>
* Fix: modify code according to comments
1. Remove DefaultBackupLocation
2. Remove unneccessary comment line
3. Add syncPeriod default value setting logic
4. Modify ListBackupStorageLocations function's context parameter
5. Add RequeueAfter parameter in Reconcile function return value
Signed-off-by: Xun Jiang <jxun@vmware.com>
* Reconcile function use context passed from parameter
1. Use context passed from parameter, instead of using Reconciler struct's context.
2. Delete Reconciler struct's context member.
3. Modify test case accordingly.
Signed-off-by: Xun Jiang <jxun@vmware.com>
* Remove backups and restic repos associated with deleted BSL(s)
Signed-off-by: F. Gold <fgold@vmware.com>
* add changelog
Signed-off-by: F. Gold <fgold@vmware.com>
* Add PR number to changelog
Signed-off-by: F. Gold <fgold@vmware.com>
* Fix typo
Signed-off-by: F. Gold <fgold@vmware.com>
* Only delete backups and restic repos and report success when without errors
Signed-off-by: F. Gold <fgold@vmware.com>
* Adds <backup-name>-itemsnapshots.gz file to backup (when provided). Also
adds DownloadTargetKindBackupItemSnapshots type to allow downloading.
Updated object store unit test
Fixes#3758
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Removed redundant checks
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Consolidated code for resolving actions and plugins into ActionResolver. Added BackupWithResolvers and
RestoreWithResolvers. Introduces ItemSnapshooterResolver to bring ItemSnapshotter plugins into backup and
restore. ItemSnapshotters are not used yet.
Added action_resolver_test
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Addressed review comments
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
This commit adds a restore action item plugin to reset invalid value
of "sideEffects" in resource of mutatingwebhookconfiguration and
validating webhookconfiguration.
To fix the problem the "sideEffects" is illegal for resource migrated
from v1beta1.
fixes#3516
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
1. remove config/crd/v1beta1
2. remove PROJECT file
3. update controller-gen and kubebuilder version
4. generate client and CRD file
5. add changelog and remove v1beta1 CRD generated code.
6. add kubebuilder test bundle setup command.
7. due to apiextensions.k8s.io/v1beta1 is not supported, only k8s after v1.16 is supported, so remove v1.15 check.
8. add CRD and k8s suppored version update in changelog.
Signed-off-by: Xun Jiang <jxun@vmware.com>
* fix: modify generated from schedule's backup name timestamp to UTC timezone
fix#4279
When backup is created from schedule, and the backup name is not specified, a containing-timestamp generated name will be used. Due to velero client not set timezone to UTC, a local timezone will be used for the generated name.
Signed-off-by: Xun Jiang <jxun@vmware.com>
* fix: modify generated from schedule's backup name timestamp to UTC timezone
fix#4279
When backup is created from schedule, and the backup name is not specified, a containing-timestamp generated name will be used. Due to velero client not set timezone to UTC, a local timezone will be used for the generated name.
Signed-off-by: Xun Jiang <jxun@vmware.com>
* fix: modify generated from schedule's backup name timestamp to UTC timezone
fix#4279
When backup is created from schedule, and the backup name is not specified, a containing-timestamp generated name will be used. Due to velero client not set timezone to UTC, a local timezone will be used for the generated name.
Signed-off-by: Xun Jiang <jxun@vmware.com>
* modify changelog description
Reword the changelog description according to comments.
Signed-off-by: Xun Jiang <jxun@vmware.com>
Co-authored-by: jxun <jxun@jxun-a01.vmware.com>
Co-authored-by: Xun Jiang <jxun@vmware.com>
logrusr is a open source convertor, which can convert logrus logger into logr.
By using logrusr, velero can use exsiting formatted logrus logger, other than introducing zap as a new logger.
Signed-off-by: Xun Jiang <jxun@vmware.com>
Added ItemSnapshotter.proto
Added item_snapshotter Go interface
Added framework components for item_snapshotter
Updated plugins doc with ItemSnapshotter info
Added SnapshotPhase to item_snapshotter.go
ProgressOutputOutput now includes a phase as well as an error string for problems that occured
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Update EnableAPIGroupVersion feature design doc as implemented
Signed-off-by: F. Gold <fgold@vmware.com>
* Design doc for issue 2082 to delete associated resources when deleting BSLs
Signed-off-by: F. Gold <fgold@vmware.com>
* Changes per @dsu-igeek review comments
Signed-off-by: F. Gold <fgold@vmware.com>
The error should be returned explicitly, because when the default URL is
used S3 will return a 301 and the response can't be handled by restic.
Fixes#4178
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
When the snapshot uploading is failed, it should not be treat as completed and continue.
This commit covers both the phases of in progress and failed when uploading snapshot with vSphere plugin
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
Previously, the BSL credential field would always be set when using the
`create` command, even if no credential details were provided. This
would result in an empty `SecretKeySelector` in the BSL which would
cause operations using this BSL to fail as Velero would attempt to fetch
a `Secret` with an empty name from the K8s API server.
With this change, the `Credential` field is only set if credential
details have been specified. This change also includes some refactoring
to allow the change to be tested.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
fix paging items in to use list options passed by the paging function
The client-go pager sets the Limit options for the list call
to paginate the request[1]. This PR fixes the paging function
to use the options passed by the pager instead of shadowed options
This is required for the pagination to work correctly.
- simplify the pager list implementation by using pager.List()
The List() function already implements a lot of the logic that was
needed for paging here, using it simplifies the code.
1. 3f40906dd8/staging/src/k8s.io/client-go/tools/pager/pager.go (L219)
Signed-off-by: Alay Patel <alay1431@gmail.com>
@@ -5,15 +5,18 @@ about: Tell us about a problem you are experiencing
---
**What steps did you take and what happened:**
[A clear and concise description of what the bug is, and what commands you ran.)
<!--A clear and concise description of what the bug is, and what commands you ran.-->
**What did you expect to happen:**
**The following information will help us better understand what's going on**:
**The output of the following commands will help us better understand what's going on**:
(Pasting long output into a [GitHub gist](https://gist.github.com) or other pastebin is fine.)
_If you are using velero v1.7.0+:_
Please use `velero debug --backup <backupname> --restore <restorename>` to generate the support bundle, and attach to this issue, more options please refer to `velero debug --help`
_If you are using earlier versions:_
Please provide the output of the following commands (Pasting long output into a [GitHub gist](https://gist.github.com) or other pastebin is fine.)
-`kubectl logs deployment/velero -n velero`
-`velero backup describe <backupname>` or `kubectl get backup/<backupname> -n velero -o yaml`
-`velero backup logs <backupname>`
@@ -22,7 +25,7 @@ about: Tell us about a problem you are experiencing
**Anything else you would like to add:**
[Miscellaneous information that will assist in solving the issue.]
<!--Miscellaneous information that will assist in solving the issue.-->
- [ ] [Accepted the DCO](https://velero.io/docs/v1.5/code-standards/#dco-sign-off). Commits without the DCO will delay acceptance.
- [ ] [Created a changelog file](https://velero.io/docs/v1.5/code-standards/#adding-a-changelog) or added `/kind changelog-not-required`.
- [ ] [Created a changelog file](https://velero.io/docs/v1.5/code-standards/#adding-a-changelog) or added `/kind changelog-not-required` as a comment on this pull request.
- [ ] Updated the corresponding documentation in `site/content/docs/main`.
stale-issue-message:"This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days. If a Velero team member has requested log or more information, please provide the output of the shared commands."
close-issue-message:"This issue was closed because it has been stalled for 5 days with no activity."
days-before-issue-stale:30
days-before-issue-close:5
stale-issue-message:"This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 14 days. If a Velero team member has requested log or more information, please provide the output of the shared commands."
close-issue-message:"This issue was closed because it has been stalled for 14 days with no activity."
days-before-issue-stale:60
days-before-issue-close:14
stale-issue-label:staled
# Disable stale PRs for now; they can remain open.
days-before-pr-stale:-1
days-before-pr-close:-1
# Only issues made after Feb 09 2021.
start-date:"2021-09-02T00:00:00"
# Only make issues stale if they have these labels. Comma separated.
only-labels:"Needs info,Duplicate"
exempt-issue-labels:"Epic,Area/CLI,Area/Cloud/AWS,Area/Cloud/Azure,Area/Cloud/GCP,Area/Cloud/vSphere,Area/CSI,Area/Design,Area/Documentation,Area/Plugins,Bug,Enhancement/User,kind/requirement,kind/refactor,kind/tech-debt,limitation,Needs investigation,Needs triage,Needs Product,P0 - Hair on fire,P1 - Important,P2 - Long-term important,P3 - Wouldn't it be nice if...,Product Requirements,Restic - GA,Restic,release-blocker,Security"
Below is a list of adopters of Velero in **production environments** that have
publicly shared the details of how they use it.
**[BitGo][20]**
BitGo uses Velero backup and restore capabilities to seamlessly provision and scale fullnode statefulsets on the fly as well as having it serve an integral piece for our kubernetes disaster-recovery story.
BitGo uses Velero backup and restore capabilities to seamlessly provision and scale fullnode statefulsets on the fly as well as having it serve an integral piece for our Kubernetes disaster-recovery story.
**[Bugsnag][30]**
We use Velero for managing backups of an internal instance of our on-premise clustered solution. We also recommend our users of [on-premise Bugsnag installations][31] use Velero for [managing their own backups][32].
We use Velero for managing backups of an internal instance of our on-premise clustered solution. We also recommend our users of [on-premise Bugsnag installations](https://www.bugsnag.com/on-premise) use Velero for [managing their own backups](https://docs.bugsnag.com/on-premise/clustered/backup-restore/). <!-- Velero.io word list : ignore -->
**[Banzai Cloud][60]**
[Banzai Cloud Pipeline][61] is a Kubernetes-based microservices platform that integrates services needed for Day-1 and Day-2 operations along with first-class support both for on-prem and hybrid multi-cloud deployments. We use Velero to periodically [backup and restore these clusters in case of disasters][62].
@@ -40,7 +42,9 @@ We have integrated our [solution with Velero][11] to provide our customers with
Kyma [integrates with Velero][41] to effortlessly back up and restore Kyma clusters with all its resources. Velero capabilities allow Kyma users to define and run manual and scheduled backups in order to successfully handle a disaster-recovery scenario.
**[Red Hat][50]**
Red Hat has developed the [Cluster Application Migration Tool][51] which uses [Velero and Restic][52] to drive the migration of applications between OpenShift clusters.
Red Hat has developed 2 operators for the OpenShift platform:
- [Migration Toolkit for Containers][51] (Crane): This operator uses [Velero and Restic][52] to drive the migration of applications between OpenShift clusters.
- [OADP (OpenShift API for Data Protection) Operator][53]: This operator sets up and installs Velero on the OpenShift platform, allowing users to backup and restore applications.
**[Dell EMC][70]**
For Kubernetes environments, [PowerProtect Data Manager][71] leverages the Container Storage Interface (CSI) framework to take snapshots to back up the persistent data or the data that the application creates e.g. databases. [Dell EMC leverages Velero][72] to backup the namespace configuration files (also known as Namespace meta data) for enterprise grade data protection.
@@ -56,8 +60,14 @@ MayaData is a large user of Velero as well as a contributor. MayaData offers a D
Okteto integrates Velero in [Okteto Cloud][94] and [Okteto Enterprise][95] to periodically backup and restore our clusters for disaster recovery. Velero is also a core software building block to provide namespace cloning capabilities, a feature that allows our users cloning staging environments into their personal development namespace for providing production-like development environments.
**[Replicated][100]**<br>
Replicated uses the Velero open source project to enable snapshots in [KOTS][101] to backup Kubernetes manifests & persistent volumes. In addition to the default functionality that Velero provides, [KOTS][101] provides a detailed interface in the [Admin Console][102] that can be used to manage the storage destination and schedule, and to perform and monitor the backup and restore process.
Replicated uses the Velero open source project to enable snapshots in [KOTS][101] to backup Kubernetes manifests & persistent volumes. In addition to the default functionality that Velero provides, [KOTS][101] provides a detailed interface in the [Admin Console][102] that can be used to manage the storage destination and schedule, and to perform and monitor the backup and restore process.<br>
**[CloudCasa][103]**<br>
[Catalogic Software][104] integrates Velero with [CloudCasa][103] - A Smart Home in the Cloud for Backups. CloudCasa is a full-featured, scalable, cloud-native solution providing Kubernetes data protection, disaster recovery, and migration as a service. An option to manage existing Velero instances and an enterprise self-hosted option are also available.<br>
**[Microsoft Azure][105]**<br>
[Azure Backup for AKS][106] is an Azure native, Kubernetes aware, Enterprise ready backup for containerized applications deployed on Azure Kubernetes Service (AKS). AKS Backup utilizes Velero to perform backup and restore operations to protect stateful applications in AKS clusters.<br>
## Adding your organization to the list of Velero Adopters
If you are using Velero and would like to be included in the list of `Velero Adopters`, add an SVG version of your logo to the `site/static/img/adopters` directory in this repo and submit a [pull request][3] with your change. Name the image file something that reflects your company (e.g., if your company is called Acme, name the image acme.png). See this for an example [PR][4].
@@ -77,8 +87,6 @@ If you would like to add your logo to a future `Adopters of Velero` section on [
[![Build Status][1]][2] [](https://bestpractices.coreinfrastructure.org/projects/3811)
Velero (formerly Heptio Ark) gives you tools to back up and restore your Kubernetes cluster resources and persistent volumes. You can run Velero with a public cloud platform or on-premises. Velero lets you:
Velero (formerly Heptio Ark) gives you tools to back up and restore your Kubernetes cluster resources and persistent volumes. You can run Velero with a public cloud platform or on-premises.
Velero lets you:
* Take backups of your cluster and restore in case of loss.
* Migrate cluster resources to other clusters.
@@ -18,7 +20,7 @@ Velero consists of:
## Documentation
[The documentation][29] provides a getting started guide and information about building from source, architecture, extending Velero, and more.
[The documentation][29] provides a getting started guide and information about building from source, architecture, extending Velero and more.
Please use the version selector at the top of the site to ensure you are using the appropriate documentation for your version of Velero.
@@ -34,6 +36,26 @@ If you are ready to jump in and test, add code, or help with documentation, foll
See [the list of releases][6] to find out about feature changes.
### Velero compatibility matrix
The following is a list of the supported Kubernetes versions for each Velero version.
| Velero version | Expected Kubernetes version compatibility | Tested on Kubernetes version |
Velero supports IPv4, IPv6, and dual stack environments. Support for this was tested against Velero v1.8.
The Velero maintainers are continuously working to expand testing coverage, but are not able to test every combination of Velero and supported Kubernetes versions for each Velero release. The table above is meant to track the current testing coverage and the expected supported Kubernetes versions for each Velero version.
If you are interested in using a different version of Kubernetes with a given Velero version, we'd recommend that you perform testing before installing or upgrading your environment. For full information around capabilities within a release, also see the Velero [release notes](https://github.com/vmware-tanzu/velero/releases) or Kubernetes [release notes](https://github.com/kubernetes/kubernetes/tree/master/CHANGELOG). See the Velero [support page](https://velero.io/docs/latest/support-process/) for information about supported versions of Velero.
For each release, Velero maintainers run the test to ensure the upgrade path from n-2 minor release. For example, before the release of v1.10.x, the test will verify that the backup created by v1.9.x and v1.8.x can be restored using the build to be tagged as v1.10.x.
This document provides a link to the [Velero Project boards](https://github.com/vmware-tanzu/velero/projects) that serves as the up to date description of items that are in the release pipeline. The release boards have separate swim lanes based on prioritization. Most items are gathered from the community or include a feedback loop with the community. This should serve as a reference point for Velero users and contributors to understand where the project is heading, and help determine if a contribution could be conflicting with a longer term plan.
### How to help?
Discussion on the roadmap can take place in threads under [Issues](https://github.com/vmware-tanzu/velero/issues) or in [community meetings](https://velero.io/community/). Please open and comment on an issue if you want to provide suggestions, use cases, and feedback to an item in the roadmap. Please review the roadmap to avoid potential duplicated effort.
### How to add an item to the roadmap?
One of the most important aspects in any open source community is the concept of proposals. Large changes to the codebase and / or new features should be preceded by a [proposal](https://github.com/vmware-tanzu/velero/blob/main/GOVERNANCE.md#proposal-process) in our repo.
For smaller enhancements, you can open an issue to track that initiative or feature request.
We work with and rely on community feedback to focus our efforts to improve Velero and maintain a healthy roadmap.
### Current Roadmap
The following table includes the current roadmap for Velero. If you have any questions or would like to contribute to Velero, please attend a [community meeting](https://velero.io/community/) to discuss with our team. If you don't know where to start, we are always looking for contributors that will help us reduce technical, automation, and documentation debt.
Please take the timelines & dates as proposals and goals. Priorities and requirements change based on community feedback, roadblocks encountered, community contributions, etc. If you depend on a specific item, we encourage you to attend community meetings to get updated status information, or help us deliver that feature by contributing to Velero.
`Last Updated: October 2021`
#### 1.8.0 Roadmap (to be delivered January/February 2021)
|Issue|Description|Timeline|Notes|
|---|---|---|---|
|[4108](https://github.com/vmware-tanzu/velero/issues/4108), [4109](https://github.com/vmware-tanzu/velero/issues/4109)|Solution for CSI - Azure and AWS|2022 H1|Currently, Velero plugins for AWS and Azure cannot back up persistent volumes that were provisioned using the CSI driver. This will fix that.|
|[3229](https://github.com/vmware-tanzu/velero/issues/3229),[4112](https://github.com/vmware-tanzu/velero/issues/4112)|Moving data mover functionality from the Velero Plugin for vSphere into Velero proper|2022 H1|This work is a precursor to decoupling the Astrolabe snapshotting infrastructure.|
|[3533](https://github.com/vmware-tanzu/velero/issues/3533)|Upload Progress Monitoring|2022 H1|Finishing up the work done in the 1.7 timeframe. The data mover work depends on this.|
|[1975](https://github.com/vmware-tanzu/velero/issues/1975)|Test dual stack mode|2022 H1|We already tested IPv6, but we want to confirm that dual stack mode works as well.|
|[2082](https://github.com/vmware-tanzu/velero/issues/2082)|Delete Backup CRs on removing target location. |2022 H1||
|[3516](https://github.com/vmware-tanzu/velero/issues/3516)|Restore issue with MutatingWebhookConfiguration v1beta1 API version|2022 H1||
|[2308](https://github.com/vmware-tanzu/velero/issues/2308)|Restoring nodePort service that has nodePort preservation always fails if service already exists in the namespace|2022 H1||
|[4115](https://github.com/vmware-tanzu/velero/issues/4115)|Support for multiple set of credentials for VolumeSnapshotLocations|2022 H1||
|[1980](https://github.com/vmware-tanzu/velero/issues/1980)|Velero triggers backup immediately for scheduled backups|2022 H1||
|[4067](https://github.com/vmware-tanzu/velero/issues/4067)|Pre and post backup and restore hooks|2022 H1||
|[3742](https://github.com/vmware-tanzu/velero/issues/3742)|Carvel packaging for Velero for vSphere|2022 H1|AWS and Azure have been completed already.|
|[3285](https://github.com/vmware-tanzu/velero/issues/3285)|Design doc for Velero plugin versioning|2022 H1||
|[4231](https://github.com/vmware-tanzu/velero/issues/4231)|Technical health (prioritizing giving developers confidence and saving developers time)|2022 H1|More automated tests (especially the pre-release manual tests) and more automation of the running of tests.|
|[4110](https://github.com/vmware-tanzu/velero/issues/4110)|Solution for CSI - GCP|2022 H1|Currently, the Velero plugin for GCP cannot back up persistent volumes that were provisioned using the CSI driver. This will fix that.|
|[3742](https://github.com/vmware-tanzu/velero/issues/3742)|Carvel packaging for Velero for restic|2022 H1|AWS and Azure have been completed already.|
|[4111](https://github.com/vmware-tanzu/velero/issues/4111)|Ignore items returned by ItemSnapshotter.AlsoHandles during backup|2022 H1|This will enable backup of complex objects, because we can then tell Velero to ignore things that were already backed up when Velero was previously called recursively.|
Other work may make it into the 1.8 release, but this is the work that will be prioritized first.
# Please go to the [Velero Wiki](https://github.com/vmware-tanzu/velero/wiki/) to see our latest roadmap, archived roadmaps and roadmap guidance.
This folder contains logo images for Velero in gray (for light backgrounds) and white (for dark backgrounds like black tshirts or dark mode!) – horizontal and stacked… in .eps and .svg.
This folder contains logo images for Velero in gray (for light backgrounds) and white (for dark backgrounds like black t-shirts or dark mode!) – horizontal and stacked… in .eps and .svg.
In this release, we introduced the Unified Repository architecture to build a data path where data movers and the backup repository are decoupled and a unified backup repository could serve various data movement activities.
In this release, we also deeply integrate Velero with Kopia, specifically, Kopia's uploader modules are isolated as a generic file system uploader; Kopia's repository modules are encapsulated as the unified backup repository.
For more information, refer to the [design document](https://github.com/vmware-tanzu/velero/blob/v1.10.0/design/unified-repo-and-kopia-integration/unified-repo-and-kopia-integration.md).
#### File system backup refactor
Velero's file system backup (a.k.s. pod volume backup or formerly restic backup) is refactored as the first user of the Unified Repository architecture. Specifically, we added a new path, the Kopia path, besides the existing Restic path. While Restic path is still available and set as default, you can opt in Kopia path by specifying the `uploader-type` parameter at installation time. Meanwhile, you are free to restore from existing backups under either path, Velero dynamically switches to the correct path to process the restore.
Because of the new path, we renamed some modules and parameters, refer to the Break Changes section for more details.
For more information, visit the [file system backup document](https://velero.io/docs/v1.10/file-system-backup/) and [v1.10 upgrade guide document](https://velero.io/docs/v1.10/upgrade-to-1.10/).
Meanwhile, we've created a performance guide for both Restic path and Kopia path, which helps you to choose between the two paths and provides you the best practice to configure them under different scenarios. Please note that the results in the guide are based on our testing environments, you may get different results when testing in your own ones. For more information, visit the [performance guide document](https://velero.io/docs/v1.10/performance-guidance/).
#### Plugin versioning V1 refactor
In this release, Velero moves plugins BackupItemAction, RestoreItemAction and VolumeSnapshotterAction to version v1, this allows future plugin changes that do not support backward compatibility, so is a preparation for various complex tasks, for example, data movement tasks.
For more information, refer to the [plugin versioning design document](https://github.com/vmware-tanzu/velero/blob/v1.10.0/design/plugin-versioning.md).
#### Refactor the controllers using Kubebuilder v3
In this release we continued our code modernization work, rewriting some controllers using Kubebuilder v3. This work is ongoing and we will continue to make progress in future releases.
#### Add credentials to volume snapshot locations
In this release, we enabled dedicate credentials options to volume snapshot locations so that you can specify credentials per volume snapshot location as same as backup storage location.
For more information, please visit the [locations document](https://velero.io/docs/v1.10/locations/).
#### CSI snapshot enhancements
In this release we added several changes to enhance the robustness of CSI snapshot procedures, for example, some protection code for error handling, and a mechanism to skip exclusion checks so that CSI snapshot works with various backup resource filters.
#### Backup schedule pause/unpause
In this release, Velero supports to pause/unpause a backup schedule during or after its creation. Specifically:
At creation time, you can specify `–paused` flag to `velero schedule create` command, if so, you will create a paused schedule that will not run until it is unpaused
After creation, you can run `velero schedule pause` or `velero schedule unpause` command to pause/unpause a schedule
#### Runtime and dependencies
In order to fix CVEs, we changed Velero's runtime and dependencies as follows:
Bump go runtime to v1.18.8
Bump some core dependent libraries to newer versions
Compile Restic (v0.13.1) with go 1.18.8 instead of packaging the official binary
#### Breaking changes
Due to file system backup refactor, below modules and parameters name have been changed in this release:
`restic` daemonset is renamed to `node-agent`
`resticRepository` CR is renamed to `backupRepository`
`velero restic repo` command is renamed to `velero repo`
`velero-restic-credentials` secret is renamed to `velero-repo-credentials`
`default-volumes-to-restic` parameter is renamed to `default-volumes-to-fs-backup`
`restic-timeout` parameter is renamed to `fs-backup-timeout`
`default-restic-prune-frequency` parameter is renamed to `default-repo-maintain-frequency`
#### Upgrade
Due to the major changes of file system backup, the old upgrade steps are not suitable any more. For the new upgrade steps, visit [v1.10 upgrade guide document](https://velero.io/docs/v1.10/upgrade-to-1.10/).
#### Limitations/Known issues
In this release, Kopia backup repository (so the Kopia path of file system backup) doesn't support self signed certificate for S3 compatible storage. To track this problem, refer to this [Velero issue](https://github.com/vmware-tanzu/velero/issues/5123) or [Kopia issue](https://github.com/kopia/kopia/issues/1443).
Due to the code change in Velero, there will be some code change required in vSphere plugin, without which the functionality may be impacted. Therefore, if you are using vSphere plugin in your workflow, please hold the upgrade until the issue [#485](https://github.com/vmware-tanzu/velero-plugin-for-vsphere/issues/485) is fixed in vSphere plugin.
### All changes
* Restore ClusterBootstrap before Cluster otherwise a new default ClusterBootstrap object is create for the cluster (#5616, @ywk253100)
* Add compile restic binary for CVE fix (#5574, @qiuming-best)
* Add credential store in backup deletion controller to support VSL credential. (#5521, @blackpiglet)
* Fix issue 5505: the pod volume backups/restores except the first one fail under the kopia path if "AZURE_CLOUD_NAME" is specified (#5512, @Lyndon-Li)
* After Pod Volume Backup/Restore refactor, remove all the unreasonable appearance of "restic" word from documents (#5499, @Lyndon-Li)
* Refactor Pod Volume Backup/Restore doc to match the new behavior (#5484, @Lyndon-Li)
* Remove redundancy code block left by #5388. (#5483, @blackpiglet)
* Issue fix 5477: create the common way to support S3 compatible object storages that work for both Restic and Kopia; Keep the resticRepoPrefix parameter for compatibility (#5478, @Lyndon-Li)
* Update the k8s.io dependencies to 0.24.0.
This also required an update to github.com/bombsimon/logrusr/v3.
Removed the `WithClusterName` method
as it is a "legacy field that was
always cleared by the system and never used" as per upstream k8s
* Remove irrational "Restic" names in Velero code after the PVBR refactor (#5444, @Lyndon-Li)
* moved RIA execute input/output structs back to velero package (#5441, @sseago)
* Rename Velero pod volume restore init helper from "velero-restic-restore-helper" to "velero-restore-helper" (#5432, @Lyndon-Li)
* Skip the exclusion check for additional resources returned by BIA (#5429, @reasonerjt)
* Change B/R describe CLI to support Kopia (#5412, @allenxu404)
* Add nil check before execution of csi snapshot delete (#5401, @shubham-pampattiwar)
* update velero using klog to version v2.9.0 (#5396, @blackpiglet)
* Fix Test_prepareBackupRequest_BackupStorageLocation UT failure. (#5394, @blackpiglet)
* Rename Velero daemonset from "restic" to "node-agent" (#5390, @Lyndon-Li)
* Add some corner cases checking for CSI snapshot in backup controller. (#5388, @blackpiglet)
* Fix issue 5386: Velero providers a full URL as the S3Url while the underlying minio client only accept the host part of the URL as the endpoint and the schema should be specified separately. (#5387, @Lyndon-Li)
* Fix restore error with flag namespace-mappings (#5377, @qiuming-best)
* Pod Volume Backup/Restore Refactor: Rename parameters in CRDs and commands to remove "Restic" word (#5370, @Lyndon-Li)
* Added backupController's UT to test the prepareBackupRequest() method BackupStorageLocation processing logic (#5362, @niulechuan)
* Fix a repoEnsurer problem introduced by the refactor - The repoEnsurer didn't check "" state of BackupRepository, as a result, the function GetBackupRepository always returns without an error even though the ensreReady is specified. (#5359, @Lyndon-Li)
* Add E2E test for schedule backup (#5355, @danfengliu)
* Add useOwnerReferencesInBackup field doc for schedule. (#5353, @cleverhu)
* Clarify the help message for the default value of parameter --snapshot-volumes, when it's not set. (#5350, @blackpiglet)
* Fix issue 4874 and 4752: check the daemonset pod is running in the node where the workload pod resides before running the PVB for the pod (#5319, @Lyndon-Li)
* plugin versioning v1 refactor for VolumeSnapshotter (#5318, @sseago)
* Change the status of restore to completed from partially failed when restore empty backup (#5314, @allenxu404)
* RestoreItemAction v1 refactoring for plugin api versioning (#5312, @sseago)
* Refactor the repoEnsurer code to use controller runtime client and wrap some common BackupRepository operations to share with other modules (#5308, @Lyndon-Li)
* Remove snapshot related lister, informer and client from backup controller. (#5299, @jxun)
* change CSISnapshotTimeout from pointer to normal variables. (#5294, @cleverhu)
* Optimize code for restore exists resources. (#5293, @cleverhu)
* Add more detailed comments for labels columns. (#5291, @cleverhu)
* Add backup status checking in schedule controller. (#5283, @blackpiglet)
* Add changes for problems/enhancements found during smoking test for Kopia pod volume backup/restore (#5282, @Lyndon-Li)
* Support pause/unpause schedules (#5279, @ywk253100)
* plugin/clientmgmt refactoring for BackupItemAction v1 (#5271, @sseago)
* Don't move velero v1 plugins to new proto dir (#5263, @sseago)
* Fill gaps for Kopia path of PVBR: integrate Repo Manager with Unified Repo; pass UploaderType to PVBR backupper and restorer; pass RepositoryType to BackupRepository controller and Repo Ensurer (#5259, @Lyndon-Li)
* Add csiSnapshotTimeout for describe backup (#5252, @cleverhu)
* equip gc controller with configurable frequency (#5248, @allenxu404)
* Fix nil pointer panic when restoring StatefulSets (#5247, @divolgin)
* Add labeled and unlabeled events for PR changelog check action. (#5157, @jxun)
* VolumeSnapshotLocation refactor with kubebuilder. (#5148, @jxun)
* Delay CA file deletion in PVB controller. (#5145, @jxun)
* This commit splits the pkg/restic package into several packages to support Kopia integration works (#5143, @ywk253100)
* Kopia Integration: Add the Unified Repository Interface definition. Kopia Integration: Add the changes for Unified Repository storage config. Related Issues; #5076, #5080 (#5142, @Lyndon-Li)
* Update the CRD for kopia integration (#5135, @reasonerjt)
* Let "make shell xxx" respect GOPROXY (#5128, @reasonerjt)
* Modify BackupStoreGetter to avoid BSL spec changes (#5122, @sseago)
* Dump stack trace when the plugin server handles panic (#5110, @reasonerjt)
* Make CSI snapshot creation timeout configurable. (#5104, @jxun)
* Fix bsl validation bug: the BSL is validated continually and doesn't respect the validation period configured (#5101, @ywk253100)
* Exclude "csinodes.storage.k8s.io" and "volumeattachments.storage.k8s.io" from restore by default. (#5064, @jxun)
* Move 'velero.io/exclude-from-backup' label string to const (#5053, @niulechuan)
* Modify Github actions. (#5052, @jxun)
* Fix typo in doc, in https://velero.io/docs/main/restore-reference/ "Restore order" section, "Mamespace" should be "Namespace". (#5051, @niulechuan)
* Delete opened issues triage action. (#5041, @jxun)
* When spec.RestoreStatus is empty, don't restore status (#5008, @sseago)
* Added DownloadTargetKindCSIBackupVolumeSnapshots for retrieving the signed URL to download only the `<backup name>`-csi-volumesnapshots.json.gz and DownloadTargetKindCSIBackupVolumeSnapshotContents to download only `<backup name>`-csi-volumesnapshotcontents.json.gz in the DownloadRequest CR structure. These files are already present in the backup layout. (#4980, @anshulahuja98)
* Refactor BackupItemAction proto and related code to backupitemaction/v1 package. This is part of implementation of the plugin version design https://github.com/vmware-tanzu/velero/blob/main/design/plugin-versioning.md (#4943, @phuongatemc)
* Unified Repository Design (#4926, @Lyndon-Li)
* Add credentials to volume snapshot locations (#4864, @sseago)
This feature implements the BackupItemAction v2. BIA v2 has two new methods: Progress() and Cancel() and modifies the Execute() return value.
The API change is needed to facilitate long-running BackupItemAction plugin actions that may not be complete when the Execute() method returns. This will allow long-running BackupItemAction plugin actions to continue in the background while the Velero moves to the following plugin or the next item.
#### RestoreItemAction v2
This feature implemented the RestoreItemAction v2. RIA v2 has three new methods: Progress(), Cancel(), and AreAdditionalItemsReady(), and it modifies RestoreItemActionExecuteOutput() structure in the RIA return value.
The Progress() and Cancel() methods are needed to facilitate long-running RestoreItemAction plugin actions that may not be complete when the Execute() method returns. This will allow long-running RestoreItemAction plugin actions to continue in the background while the Velero moves to the following plugin or the next item. The AreAdditionalItemsReady() method is needed to allow plugins to tell Velero to wait until the returned additional items have been restored and are ready for use in the cluster before restoring the current item.
#### Plugin Progress Monitoring
This is intended as a replacement for the previously-approved Upload Progress Monitoring design ([Upload Progress Monitoring](https://github.com/vmware-tanzu/velero/blob/main/design/upload-progress.md)) to expand the supported use cases beyond snapshot upload to include what was previously called Async Backup/Restore Item Actions.
#### Flexible resource policy that can filter volumes to skip in the backup
This feature provides a flexible policy to filter volumes in the backup without requiring patching any labels or annotations to the pods or volumes. This policy is configured as k8s ConfigMap and maintained by the users themselves, and it can be extended to more scenarios in the future. By now, the policy rules out volumes from backup depending on the CSI driver, NFS setting, volume size, and StorageClass setting. Please refer to [policy API design](https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/handle-backup-of-volumes-by-resources-filters.md#api-design)for the policy's ConifgMap format. It is not guaranteed to work on unofficial third-party plugins as it may not follow the existing backup workflow code logic of Velero.
#### Resource Filters that can distinguish cluster scope and namespace scope resources
This feature adds four new resource filters for backup. The new filters are separated into cluster scope and namespace scope. Before this feature, Velero could not filter cluster scope resources precisely. This feature provides the ability and refactors existing resource filter parameters.
#### Add a parameter for setting the Velero server connection with the k8s API server's timeout
In Velero, some code pieces need to communicate with the k8s API server. Before v1.11, these code pieces used hard-code timeout settings. This feature adds a resource-timeout parameter in the velero server binary to make it configurable.
#### Add resource list in the output of the restore describe command
Before this feature, Velero restore didn't have a restored resources list as the Velero backup. It's not convenient for users to learn what is restored. This feature adds the resources list and the handling result of the resources (including created, updated, failed, and skipped).
#### Refactor controllers with controller-runtime
In v1.11, Backup Controller and Restore controller are refactored with controller-runtime. Till v1.11, all Velero controllers use the controller-runtime framework.
#### Runtime and dependencies
To fix CVEs and keep pace with Golang, Velero made changes as follows:
* Bump Golang runtime to v1.19.8.
* Bump several dependent libraries to new versions.
* Compile Restic (v0.15.0) with Golang v1.19.8 instead of packaging the official binary.
### Breaking changes
* The Velero CSI plugin now determines whether to restore Volume's data from snapshots on the restore's restorePVs setting. Before v1.11, the CSI plugin doesn't check the restorePVs parameter setting.
### Limitations/Known issues
* The Flexible resource policy that can filter volumes to skip in the backup is not guaranteed to work on unofficial third-party plugins because the plugins may not follow the existing backup workflow code logic of Velero. The ConfigMap used as the policy is supposed to be maintained by users.
### All Changes
* Modify new scope resource filters name. (#6089, @blackpiglet)
* Make Velero not exits when EnableCSI is on and CSI snapshot not installed (#6062, @blackpiglet)
* Restore Services before Clusters (#6057, @ywk253100)
* Fixed backup deletion bug related to async operations (#6041, @sseago)
* Update Golang version to v1.19 for branch main. (#6039, @blackpiglet)
* Fix issue #5972, don't assume errorField as error type when dealing with logger.WithError (#6028, @Lyndon-Li)
* distinguish between New and InProgress operations (#6012, @sseago)
* Modify golangci.yaml file. Resolve found lint issues. (#6008, @blackpiglet)
* Remove Reference of itemsnapshotter (#5997, @reasonerjt)
* minor fixes for backup_operations_controller (#5996, @sseago)
* RIAv2 async operations controller work (#5993, @sseago)
* Follow-on fixes for BIAv2 controller work (#5971, @sseago)
* Refactor backup controller based on the controller-runtime framework. (#5969, @qiuming-best)
* Fix client wait problem after async operation change, velero backup/restore --wait should check a full list of the terminal status (#5964, @Lyndon-Li)
* Fix issue #5935, refactor the logics for backup/restore persistent log, so as to remove the contest to gzip writer (#5956, @Lyndon-Li)
* Switch the base image to distroless/base-nossl-debian11 to reduce the CVE triage efforts (#5939, @ywk253100)
* Wait for additional items to be ready before restoring current item (#5933, @sseago)
* Add configurable server setting for default timeouts (#5926, @eemcmullan)
* Add warning/error result to cmd `velero backup describe` (#5916, @allenxu404)
* Fix Dependabot alerts. Use 1.18 and 1.19 golang instead of patch image in dockerfile. Add release-1.10 and release-1.9 in Trivy daily scan. (#5911, @blackpiglet)
* Update client-go to v0.25.6 (#5907, @kaovilai)
* Limit the concurrent number for backup's VolumeSnapshot operation. (#5900, @blackpiglet)
* Fix goreleaser issue for resolving tags and updated it's version. (#5899, @anshulahuja98)
* This is to fix issue 5881, enhance the PVB tracker in two modes, Track and Taken (#5894, @Lyndon-Li)
* Add labels for velero installed namespace to support PSA. (#5873, @blackpiglet)
* Add restored resource list in the restore describe command (#5867, @ywk253100)
* Add a json output to cmd velero backup describe (#5865, @allenxu404)
* Make restore controller adopting the controller-runtime framework. (#5864, @blackpiglet)
* Replace k8s.io/apimachinery/pkg/util/clock with k8s.io/utils/clock (#5859, @hezhizhen)
* Restore finalizer and managedFields of metadata during the restoration (#5853, @ywk253100)
* BIAv2 async operations controller work (#5849, @sseago)
* Add secret restore item action to handle service account token secret (#5843, @ywk253100)
* Add new resource filters can separate cluster and namespace scope resources. (#5838, @blackpiglet)
* Correct PVB/PVR Failed Phase patching during startup (#5828, @kaovilai)
* bump up golang net to fix CVE-2022-41721 (#5812, @Lyndon-Li)
* Update CRD descriptions for SnapshotVolumes and restorePVs (#5807, @shubham-pampattiwar)
* Add mapped selected-node existence check (#5806, @blackpiglet)
* Add option "--service-account-name" to install cmd (#5802, @reasonerjt)
* Define itemoperations.json format and update DownloadRequest API (#5752, @sseago)
* Add Trivy nightly scan. (#5740, @jxun)
* Fix issue 5696, check if the repo is still openable before running the prune and forget operation, if not, try to reconnect the repo (#5715, @Lyndon-Li)
* Fix error with Restic backup empty volumes (#5713, @qiuming-best)
* new backup and restore phases to support async plugin operations:
CSI Snapshot Data Movement refers to back up CSI snapshot data from the volatile and limited production environment into durable, heterogeneous, and scalable backup storage in a consistent manner; and restore the data to volumes in the original or alternative environment.
CSI Snapshot Data Movement is useful in below scenarios:
* For on-premises users, the storage usually doesn't support durable snapshots, so it is impossible/less efficient/cost ineffective to keep volume snapshots by the storage This feature helps to move the snapshot data to a storage with lower cost and larger scale for long time preservation.
* For public cloud users, this feature helps users to fulfill the multiple cloud strategy. It allows users to back up volume snapshots from one cloud provider and preserve or restore the data to another cloud provider. Then users will be free to flow their business data across cloud providers based on Velero backup and restore
CSI Snapshot Data Movement is built according to the Volume Snapshot Data Movement design ([Volume Snapshot Data Movement](https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/unified-repo-and-kopia-integration/unified-repo-and-kopia-integration.md)). More details can be found in the design.
#### Resource Modifiers
In many use cases, customers often need to substitute specific values in Kubernetes resources during the restoration process like changing the namespace, changing the storage class, etc.
To address this need, Resource Modifiers (also known as JSON Substitutions) offer a generic solution in the restore workflow. It allows the user to define filters for specific resources and then specify a JSON patch (operator, path, value) to apply to the resource. This feature simplifies the process of making substitutions without requiring the implementation of a new RestoreItemAction plugin. More details can be found in Volume Snapshot Resource Modifiers design ([Resource Modifiers](https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/json-substitution-action-design.md)).
#### Multiple VolumeSnapshotClasses
Prior to version 1.12, the Velero CSI plugin would choose the VolumeSnapshotClass in the cluster based on matching driver names and the presence of the "velero.io/csi-volumesnapshot-class" label. However, this approach proved inadequate for many user scenarios.
With the introduction of version 1.12, Velero now offers support for multiple VolumeSnapshotClasses in the CSI Plugin, enabling users to select a specific class for a particular backup. More details can be found in Multiple VolumeSnapshotClasses design ([Multiple VolumeSnapshotClasses](https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/multiple-csi-volumesnapshotclass-support.md)).
#### Restore Finalizer
Before v1.12, the restore controller would only delete restore resources but wouldn’t delete restore data from the backup storage location when the command `velero restore delete` was executed. The only chance Velero deletes restores data from the backup storage location is when the associated backup is deleted.
In this version, Velero introduces a finalizer that ensures the cleanup of all associated data for restores when running the command `velero restore delete`.
#### Runtime and dependencies
To fix CVEs and keep pace with Golang, Velero made changes as follows:
* Bump Golang runtime to v1.20.7.
* Bump several dependent libraries to new versions.
* Bump Kopia to v0.13.
### Breaking changes
* Prior to v1.12, the parameter `uploader-type` for Velero installation had a default value of "restic". However, starting from this version, the default value has been changed to "kopia". This means that Velero will now use Kopia as the default path for file system backup.
* The ways of setting CSI snapshot time have changed in v1.12. First, the sync waiting time for creating a snapshot handle in the CSI plugin is changed from the fixed 10 minutes into backup.Spec.CSISnapshotTimeout. The second, the async waiting time for VolumeSnapshot and VolumeSnapshotContent's status turning into `ReadyToUse` in operation uses the operation's timeout. The default value is 4 hours.
* As from [Velero helm chart v4.0.0](https://github.com/vmware-tanzu/helm-charts/releases/tag/velero-4.0.0), it supports multiple BSL and VSL, and the BSL and VSL have changed from the map into a slice, and[ this breaking change](https://github.com/vmware-tanzu/helm-charts/pull/413) is not backward compatible. So it would be best to change the BSL and VSL configuration into slices before the Upgrade.
### Limitations/Known issues
* The Azure plugin supports Azure AD Workload identity way, but it only works for Velero native snapshots. It cannot support filesystem backup and snapshot data mover scenarios.
### All Changes
* Fixes #6498. Get resource client again after restore actions in case resource's gv is changed. This is an improvement of pr #6499, to support group changes. A group change usually happens in a restore plugin which is used for resource conversion: convert a resource from a not supported gv to a supported gv (#6634, @27149chen)
* Add API support for volMode block, only error for now. (#6608, @shawn-hurley)
* Fix how the AWS credentials are obtained from configuration (#6598, @aws_creds)
* Add performance E2E test (#6569, @qiuming-best)
* Non default s3 credential profiles work on Unified Repository Provider (kopia) (#6558, @kaovilai)
* Fix issue #6571, fix the problem for restore item operation to set the errors correctly so that they can be recorded by Velero restore and then reflect the correct status for Velero restore. (#6594, @Lyndon-Li)
* Fix issue 6575, flush the repo after delete the snapshot, otherwise, the changes(deleting repo snapshot) cannot be committed to the repo. (#6587, @Lyndon-Li)
* Delete moved snapshots when the backup is deleted (#6547, @reasonerjt)
* check if restore crd exist before operating restores (#6544, @allenxu404)
* Remove PVC's selector in backup's PVC action. (#6481, @blackpiglet)
* Delete the expired deletebackuprequests that are stuck in "InProgress" (#6476, @reasonerjt)
* Fix issue #6534, reset PVB CR's StorageLocation to the latest one during backup sync as same as the backup CR. Also fix similar problem with DataUploadResult for data mover restore. (#6533, @Lyndon-Li)
* Fix issue #6519. Restrict the client manager of node-agent server to include only Velero resources from the server's namespace, otherwise, the controllers will try to reconcile CRs from all the installed Velero namespaces. (#6523, @Lyndon-Li)
* Track the skipped PVC and print the summary in backup log (#6496, @reasonerjt)
* Add restore finalizer to clean up external resources (#6479, @allenxu404)
* fix: Typos and add more spell checking rules to CI (#6415, @mateusoliveira43)
* Add missing CompletionTimestamp and metrics when restore moved into terminal phase in restoreOperationsReconciler (#6397, @Nutrymaco)
* Add support for resource Modifications in the restore flow. Also known as JSON Substitutions. (#6452, @anshulahuja98)
* Remove dependency of the legacy client code from pkg/cmd directory part 2 (#6497, @blackpiglet)
* Add data upload and download metrics (#6493, @allenxu404)
* Fix issue 6490, If a backup/restore has multiple async operations and one operation fails while others are still in-progress, when all the operations finish, the backup/restore will be set as Completed falsely (#6491, @Lyndon-Li)
* Velero Plugins no longer need kopia indirect dependency in their go.mod (#6484, @kaovilai)
* Remove dependency of the legacy client code from pkg/cmd directory (#6469, @blackpiglet)
* Add support for OpenStack CSI drivers topology keys (#6464, @openstack-csi-topology-keys)
* Add exit code log and possible memory shortage warning log for Restic command failure. (#6459, @blackpiglet)
* Add UT cases for pkg/podvolume (#6336, @Lyndon-Li)
* Remove Wait VolumeSnapshot to ReadyToUse logic. (#6327, @blackpiglet)
* Enhance the code because of #6297, the return value of GetBucketRegion is not recorded, as a result, when it fails, we have no way to get the cause (#6326, @Lyndon-Li)
* Skip updating status when CRDs are restored (#6325, @reasonerjt)
* Include namespaces needed by namespaced-scope resources in backup. (#6320, @blackpiglet)
* Update metrics when backup failed with validation error (#6318, @ywk253100)
* Add the code for data mover backup expose (#6308, @Lyndon-Li)
* Fix a PVR issue for generic data path -- the namespace remap was not honored, and enhance the code for better error handling (#6303, @Lyndon-Li)
* Add default values for defaultItemOperationTimeout and itemOperationSyncFrequency in velero CLI (#6298, @shubham-pampattiwar)
* Add UT cases for pkg/repository (#6296, @Lyndon-Li)
* Fix issue #5875. Since Kopia has supported IAM, Velero should not require static credentials all the time (#6283, @Lyndon-Li)
* Fixed a bug where status.progress is not getting updated for backups. (#6276, @kkothule)
* Add code change for async generic data path that is used by both PVB/PVR and data mover (#6226, @Lyndon-Li)
* Add data mover CRD under v2alpha1, include DataUpload CRD and DataDownload CRD (#6176, @Lyndon-Li)
* Remove any dataSource or dataSourceRef fields from PVCs in PVC BIA for cases of
prior PVC restores with CSI (#6111, @eemcmullan)
* Add the design for Volume Snapshot Data Movement (#5968, @Lyndon-Li)
* Fix issue #5123, Kopia repository supports self-cert CA for S3 compatible storage. (#6268, @Lyndon-Li)
* Bump up Kopia to v0.13 (#6248, @Lyndon-Li)
* log volumes to backup to help debug why `IsPodRunning` is called. (#6232, @kaovilai)
* Enable errcheck linter and resolve found issues (#6208, @blackpiglet)
* Enable more linters, and remove mal-functioned milestoned issue action. (#6194, @blackpiglet)
* Enable stylecheck linter and resolve found issues. (#6185, @blackpiglet)
* Fix issue #6182. If pod is not running, don't treat it as an error, let it go and leave a warning. (#6184, @Lyndon-Li)
* Enable staticcheck and resolve found issues (#6183, @blackpiglet)
* Enable linter revive and resolve found errors: part 2 (#6177, @blackpiglet)
* Enable linter revive and resolve found errors: part 1 (#6173, @blackpiglet)
* Fix usestdlibvars and whitespace linters issues. (#6162, @blackpiglet)
* Update Golang to v1.20 for main. (#6158, @blackpiglet)
* Make GetPluginConfig accessible from other packages. (#6151, @tkaovila)
* Ignore not found error during patching managedFields (#6136, @ywk253100)
* Fix the goreleaser issues and add a new goreleaser action (#6109, @blackpiglet)
Velero introduced the Resource Modifiers in v1.12.0. This feature allows users to specify a ConfigMap with a set of rules to modify the resources during restoration. However, only the JSON Patch is supported when creating the rules, and JSON Patch has some limitations, which cannot cover all use cases. In v1.13.0, Velero adds new support for JSON Merge Patch and Strategic Merge Patch, which provide more power and flexibility and allow users to use the same ConfigMap to apply patches on the resources. More design details can be found in [Support JSON Merge Patch and Strategic Merge Patch in Resource Modifiers](https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/merge-patch-and-strategic-in-resource-modifier.md) design. For instructions on how to use the feature, please refer to the [Resource Modifiers](https://velero.io/docs/v1.13/restore-resource-modifiers/) doc.
#### Node-Agent Concurrency
Velero data movement activities from fs-backups and CSI snapshot data movements run in Velero node-agent, so may be hosted by every node in the cluster and consume resources (i.e. CPU, memory, network bandwidth) from there. With v1.13, users are allowed to configure how many data movement activities (a.k.a, loads) run in each node globally or by node, so that users can better leverage the performance of Velero data movement activities and the resource consumption in the cluster. For more information, check the [Node-Agent Concurrency](https://velero.io/docs/v1.13/node-agent-concurrency/) document.
#### Parallel Files Upload Options
Velero now supports configurable options for parallel files upload when using Kopia uploader to do fs-backups or CSI snapshot data movements which makes speed up backup possible.
For more information, please check [Here](https://velero.io/docs/v1.13/backup-reference/#parallel-files-upload).
#### Write Sparse Files Options
If using fs-restore or CSI snapshot data movements, it’s supported to write sparse files during restore. For more information, please check [Here](https://velero.io/docs/v1.13/restore-reference/#write-sparse-files).
#### Backup Describe
In v1.13, the Backup Volume section is added to the velero backup describe command output. The backup Volumes section describes information for all the volumes included in the backup of various backup types, i.e. native snapshot, fs-backup, CSI snapshot, and CSI snapshot data movement. Particularly, the velero backup description now supports showing the information of CSI snapshot data movements, which is not supported in v1.12.
Additionally, backup describe command will not check EnableCSI feature gate from client side, so if a backup has volumes with CSI snapshot or CSI snapshot data movement, backup describe command always shows the corresponding information in its output.
#### Backup's new VolumeInfo metadata
Create a new metadata file in the backup repository's backup name sub-directory to store the backup-including PVC and PV information. The information includes the backing-up method of the PVC and PV data, snapshot information, and status. The VolumeInfo metadata file determines how the PV resource should be restored. The Velero downstream software can also use this metadata file to get a summary of the backup's volume data information.
#### Enhancement for CSI Snapshot Data Movements when Velero Pod Restart
When performing backup and restore operations, enhancements have been implemented for Velero server pods or node agents to ensure that the current backup or restore process is not stuck or interrupted after restart due to certain exceptional circumstances.
#### New status fields added to show hook execution details
Hook execution status is now included in the backup/restore CR status and displayed in the backup/restore describe command output. Specifically, it will show the number of hooks which attempted to execute under the HooksAttempted field and the number of hooks which failed to execute under the HooksFailed field.
#### AWS SDK Bump Up
Bump up AWS SDK for Go to version 2, which offers significant performance improvements in CPU and memory utilization over version 1.
#### Azure AD/Workload Identity Support
Azure AD/Workload Identity is the recommended approach to do the authentication with Azure services/AKS, Velero has introduced support for Azure AD/Workload Identity on the Velero Azure plugin side in previous releases, and in v1.13.0 Velero adds new support for Kopia operations(file system backup/data mover/etc.) with Azure AD/Workload Identity.
#### Runtime and dependencies
To fix CVEs and keep pace with Golang, Velero made changes as follows:
* Bump Golang runtime to v1.21.6.
* Bump several dependent libraries to new versions.
* Bump Kopia to v0.15.0.
### Breaking changes
* Backup describe command: due to the backup describe output enhancement, some existing information (i.e. the output for native snapshot, CSI snapshot, and fs-backup) has been moved to the Backup Volumes section with some format changes.
* API type changes: changes the field [DataMoverConfig](https://github.com/vmware-tanzu/velero/blob/v1.13.0/pkg/apis/velero/v2alpha1/data_upload_types.go#L54) in DataUploadSpec from `*map[string][string]`` to `map[string]string`
* Velero install command: due to the issue [#7264](https://github.com/vmware-tanzu/velero/issues/7264), v1.13.0 introduces a break change that make the informer cache enabled by default to keep the actual behavior consistent with the helper message(the informer cache is disabled by default before the change).
### Limitations/Known issues
* The backup's VolumeInfo metadata doesn't have the information updated in the async operations. This function could be supported in v1.14 release.
### Note
* Velero introduces the informer cache which is enabled by default. The informer cache improves the restore performance but may cause higher memory consumption. Increase the memory limit of the Velero pod or disable the informer cache by specifying the `--disable-informer-cache` option when installing Velero if you get the OOM error.
### Deprecation announcement
* The generated k8s clients, informers, and listers are deprecated in the Velero v1.13 release. They are put in the Velero repository's pkg/generated directory. According to the n+2 supporting policy, the deprecated are kept for two more releases. The pkg/generated directory should be deleted in the v1.15 release.
* After the backup VolumeInfo metadata file is added to the backup, Velero decides how to restore the PV resource according to the VolumeInfo content. To support the backup generated by the older version of Velero, the old logic is also kept. The support for the backup without the VolumeInfo metadata file will be kept for two releases. The support logic will be deleted in the v1.15 release.
### All Changes
* Make "disable-informer-cache" option false(enabled) by default to keep it consistent with the help message (#7294, @ywk253100)
* Do not set "targetNamespace" to namespace items (#7274, @reasonerjt)
* Fix issue #7244. By the end of the upload, check the outstanding incomplete snapshots and delete them by calling ApplyRetentionPolicy (#7245, @Lyndon-Li)
* Adjust the newline output of resource list in restore describer (#7238, @allenxu404)
* Remove the redundant newline in backup describe output (#7229, @allenxu404)
* Fix issue #7189, data mover generic restore - don't assume the first volume as the restore volume (#7201, @Lyndon-Li)
* Update CSIVolumeSnapshotsCompleted in backup's status and the metric
during backup finalize stage according to async operations content. (#7184, @blackpiglet)
* Refactor DownloadRequest Stream function (#7175, @blackpiglet)
* Add `--skip-immediately` flag to schedule commands; `--schedule-skip-immediately` server and install (#7169, @kaovilai)
* Add node-agent concurrency doc and change the config name from dataPathConcurrency to loadCocurrency (#7161, @Lyndon-Li)
* Enhance hooks tracker by adding a returned error to record function (#7153, @allenxu404)
* Track the skipped PV when SnapshotVolumes set as false (#7152, @reasonerjt)
* Add more linters part 2. (#7151, @blackpiglet)
* Fix issue #7135, check pod status before checking node-agent pod status (#7150, @Lyndon-Li)
* Treat namespace as a regular restorable item (#7143, @reasonerjt)
* Fix issue #6695, add describe for data mover backups (#7125, @Lyndon-Li)
* Add hooks status to backup/restore CR (#7117, @allenxu404)
* Include plugin name in the error message by operations (#7115, @reasonerjt)
* Fix issue #7068, due to a behavior of CSI external snapshotter, manipulations of VS and VSC may not be handled in the same order inside external snapshotter as the API is called. So add a protection finalizer to ensure the order (#7102, @Lyndon-Li)
* Generate VolumeInfo for backup. (#7100, @blackpiglet)
* Fix issue #7094, fallback to full backup if previous snapshot is not found (#7096, @Lyndon-Li)
* Fix issue #7068, due to an behavior of CSI external snapshotter, manipulations of VS and VSC may not be handled in the same order inside external snapshotter as the API is called. So add a protection finalizer to ensure the order (#7095, @Lyndon-Li)
* Skip syncing the backup which doesn't contain backup metadata (#7081, @ywk253100)
* Fix issue #6693, partially fail restore if CSI snapshot is involved but CSI feature is not ready, i.e., CSI feature gate is not enabled or CSI plugin is not installed. (#7077, @Lyndon-Li)
* Truncate the credential file to avoid the change of secret content messing it up (#7072, @ywk253100)
* improve discoveryHelper.Refresh() in restore (#7069, @27149chen)
* Add DataUpload Result and CSI VolumeSnapshot check for restore PV. (#7061, @blackpiglet)
* Add the implementation for design #6950, configurable data path concurrency (#7059, @Lyndon-Li)
* Make data mover fail early (#7052, @qiuming-best)
* Remove dependency of generated client part 3. (#7051, @blackpiglet)
* Update Backup.Status.CSIVolumeSnapshotsCompleted during finalize (#7046, @kaovilai)
* Remove the Velero generated client. (#7041, @blackpiglet)
* Fix issue #7027, data mover backup exposer should not assume the first volume as the backup volume in backup pod (#7038, @Lyndon-Li)
* Read information from the credential specified by BSL (#7034, @ywk253100)
* Fix #6857. Added check for matching Owner References when synchronizing backups, removing references that are not found/have mismatched uid. (#7032, @deefdragon)
* Add description markers for dataupload and datadownload CRDs (#7028, @shubham-pampattiwar)
* Add HealthCheckNodePort deletion logic for Service restore. (#7026, @blackpiglet)
* Fix inconsistent behavior of Backup and Restore hook execution (#7022, @allenxu404)
* Fix #6964. Don't use csiSnapshotTimeout (10 min) for waiting snapshot to readyToUse for data mover, so as to make the behavior complied with CSI snapshot backup (#7011, @Lyndon-Li)
* restore: Use warning when Create IsAlreadyExist and Get error (#7004, @kaovilai)
* Bump kopia to 0.15.0 (#7001, @Lyndon-Li)
* Make Kopia file parallelism configurable (#7000, @qiuming-best)
* It is a valid case that the Status.RestoreSize field in VolumeSnapshot is not set, if so, get the volume size from the source PVC to create the backup PVC (#6976, @Lyndon-Li)
* Check whether the action is a CSI action and whether CSI feature is enabled, before executing the action. (#6968, @blackpiglet)
* Add the PV backup information design document. (#6962, @blackpiglet)
* Change controller-runtime List option from MatchingFields to ListOptions (#6958, @blackpiglet)
* Add the design for node-agent concurrency (#6950, @Lyndon-Li)
* Import auth provider plugins (#6947, @0x113)
* Fix #6668, add a limitation for file system restore parallelism with other types of restores (CSI snapshot restore, CSI snapshot movement restore) (#6946, @Lyndon-Li)
* Add MSI Support for Azure plugin. (#6938, @yanggangtony)
* Partially fix #6734, guide Kubernetes' scheduler to spread backup pods evenly across nodes as much as possible, so that data mover backup could achieve better parallelism (#6926, @Lyndon-Li)
* Bump up aws sdk to aws-sdk-go-v2 (#6923, @reasonerjt)
* Optional check if targeted container is ready before executing a hook (#6918, @Ripolin)
* Support JSON Merge Patch and Strategic Merge Patch in Resource Modifiers (#6917, @27149chen)
* Fix issue 6913: Velero Built-in Datamover: Backup stucks in phase WaitingForPluginOperations when Node Agent pod gets restarted (#6914, @shubham-pampattiwar)
* Set ParallelUploadAboveSize as MaxInt64 and flush repo after setting up policy so that policy is retrieved correctly by TreeForSource (#6885, @Lyndon-Li)
* Replace the base image with paketobuildpacks image (#6883, @ywk253100)
* Fix issue #6859, move plugin depending podvolume functions to util pkg, so as to remove the dependencies to unnecessary repository packages like kopia, azure, etc. (#6875, @Lyndon-Li)
* Fix #6861. Only Restic path requires repoIdentifier, so for non-restic path, set the repoIdentifier fields as empty in PVB and PVR and also remove the RepoIdentifier column in the get output of PVBs and PVRs (#6872, @Lyndon-Li)
* Add volume types filter in resource policies (#6863, @qiuming-best)
* change the metrics backup_attempt_total default value to 1. (#6838, @yanggangtony)
* Bump kopia to v0.14 (#6833, @Lyndon-Li)
* Retry failed create when using generateName (#6830, @sseago)
* Fix issue #6786, always delete VSC regardless of the deletion policy (#6827, @Lyndon-Li)
* Proposal to support JSON Merge Patch and Strategic Merge Patch in Resource Modifiers (#6797, @27149chen)
* Fix the node-agent missing metrics-address defines. (#6784, @yanggangtony)
* Fix default BSL setting not work (#6771, @qiuming-best)
* Update restore controller logic for restore deletion (#6770, @ywk253100)
* Fix issue #6753, remove the check for read-only BSL in restore async operation controller since Velero cannot fully support read-only mode BSL in restore at present (#6757, @Lyndon-Li)
* Fix issue #6647, add the --default-snapshot-move-data parameter to Velero install, so that users don't need to specify --snapshot-move-data per backup when they want to move snapshot data for all backups (#6751, @Lyndon-Li)
* Use old(origin) namespace in resource modifier conditions in case namespace may change during restore (#6724, @27149chen)
* Perf improvements for existing resource restore (#6723, @sseago)
* Remove schedule-related metrics on schedule delete (#6715, @nilesh-akhade)
* Kubernetes 1.27 new job label batch.kubernetes.io/controller-uid are deleted during restore per https://github.com/kubernetes/kubernetes/pull/114930 (#6712, @kaovilai)
* This pr made some improvements in Resource Modifiers: 1. add label selector 2. change the field name from groupKind to groupResource (#6704, @27149chen)
* Make Kopia support Azure AD (#6686, @ywk253100)
* Add support for block volumes with Kopia (#6680, @dzaninovic)
* Delete PartiallyFailed orphaned backups as well as Completed ones (#6649, @sseago)
* Add CSI snapshot data movement doc (#6637, @Lyndon-Li)
* Fixes #6636, skip subresource in resource discovery (#6635, @27149chen)
* Add `orLabelSelectors` for backup, restore commands (#6475, @nilesh-akhade)
* fix run preHook and postHook on completed pods (#5211, @cleverhu)
#### The maintenance work for kopia/restic backup repositories is run in jobs
Since velero started using kopia as the approach for filesystem-level backup/restore, we've noticed an issue when velero connects to the kopia backup repositories and performs maintenance, it sometimes consumes excessive memory that can cause the velero pod to get OOM Killed. To mitigate this issue, the maintenance work will be moved out of velero pod to a separate kubernetes job, and the user will be able to specify the resource request in "velero install".
#### Volume Policies are extended to support more actions to handle volumes
In an earlier release, a flexible volume policy was introduced to skip certain volumes from a backup. In v1.14 we've made enhancement to this policy to allow the user to set how the volumes should be backed up. The user will be able to set "fs-backup" or "snapshot" as value of “action" in the policy and velero will backup the volumes accordingly. This enhancement allows the user to achieve a fine-grained control like "opt-in/out" without having to update the target workload. For more details please refer to https://velero.io/docs/v1.14/resource-filtering/#supported-volumepolicy-actions
#### Node Selection for Data Movement Backup
In velero the data movement flow relies on datamover pods, and these pods may take substantial resources and keep running for a long time. In v1.14, the user will be able to create a configmap to define the eligible nodes on which the datamover pods are launched. For more details refer to https://velero.io/docs/v1.14/data-movement-backup-node-selection/
#### VolumeInfo metadata for restored volumes
In v1.13, we introduced volumeinfo metadata for backup to help velero CLI and downstream adopter understand how velero handles each volume during backup. In v1.14, similar metadata will be persisted for each restore. velero CLI is also updated to bring more info in the output of "velero restore describe".
#### "Finalizing" phase is introduced to restores
The "Finalizing" phase is added to the state transition flow to restore, which helps us fix several issues: The labels added to PVs will be restored after the data in the PV is restored via volumesnapshotter. The post restore hook will be executed after datamovement is finished.
#### Certificate-based authentication support for Azure
Besides the service principal with secret(password)-based authentication, Velero introduces the new support for service principal with certificate-based authentication in v1.14.0. This approach enables you to adopt a phishing resistant authentication by using conditional access policies, which better protects Azure resources and is the recommended way by Azure.
### Runtime and dependencies
* Golang runtime: v1.22.2
* kopia: v0.17.0
### Limitations/Known issues
* For the external BackupItemAction plugins that take snapshots for PVs, such as vsphere plugin. If the plugin checks the value of the field "snapshotVolumes" in the backup spec as a criteria for snapshot, the settings in the volume policy will not take effect. For example, if the "snapshotVolumes" is set to False in the backup spec, but a volume meets the condition in the volume policy for "snapshot" action, because the plugin will not check the settings in the volume policy, the plugin will not take snapshot for the volume. For more details please refer to #7818
### Breaking changes
* CSI plugin has been merged into velero repo in v1.14 release. It will be installed by default as an internal plugin, and should not be installed via "–plugins " parameter in "velero install" command.
* The default resource requests and limitations for node agent are removed in v1.14, to make the node agent pods have the QoS class of "BestEffort", more details please refer to #7391
* There's a change in namespace filtering behavior during backup: In v1.14, when the includedNamespaces/excludedNamespaces fields are not set and the labelSelector/OrLabelSelectors are set in the backup spec, the backup will only include the namespaces which contain the resources that match the label selectors, while in previous releases all namespaces will be included in the backup with such settings. More details refer to #7105
* Patching the PV in the "Finalizing" state may cause the restore to be in "PartiallyFailed" state when the PV is blocked in "Pending" state, while in the previous release the restore may end up being in "Complete" state. For more details refer to #7866
### All Changes
* Fix backup log to show error string, not index (#7805, @piny940)
* Modify the volume helper logic. (#7794, @blackpiglet)
* Add documentation for extension of volume policy feature (#7779, @shubham-pampattiwar)
* Surface errors when waiting for backupRepository and timeout occurs (#7762, @kaovilai)
* Add existingResourcePolicy restore CR validation to controller (#7757, @kaovilai)
* Fix condition matching in resource modifier when there are multiple rules (#7715, @27149chen)
* Bump up the version of KinD and k8s in github actions (#7702, @reasonerjt)
* Implementation for Extending VolumePolicies to support more actions (#7664, @shubham-pampattiwar)
* Migrate from `github.com/Azure/azure-storage-blob-go` to `github.com/Azure/azure-sdk-for-go/sdk/storage/azblob` (#7598, @mmorel-35)
* When Included/ExcludedNamespaces are omitted, and LabelSelector or OrLabelSelector is used, namespaces without selected items are excluded from backup. (#7697, @blackpiglet)
* Display CSI snapshot restores in restore describe (#7687, @reasonerjt)
* Use specific credential rather than the credential chain for Azure (#7680, @ywk253100)
* Modify hook docs for clarity on displaying hook execution results (#7679, @allenxu404)
* Wait for results of restore exec hook executions in Finalizing phase instead of InProgress phase (#7619, @allenxu404)
* migrating to `sdk/resourcemanager/**/arm**` from `services/**/mgmt/**` (#7596, @mmorel-35)
* Bump up to go1.22 (#7666, @reasonerjt)
* Fix issue #7648. Adjust the exposing logic to avoid exposing failure and snapshot leak when expose fails (#7662, @Lyndon-Li)
* Track and persist restore volume info (#7630, @reasonerjt)
* Check the existence of the namespaces provided in the "--include-namespaces" option (#7569, @ywk253100)
* Add the finalization phase to the restore workflow (#7377, @allenxu404)
* Upgrade the version of go plugin related libs/tools (#7373, @ywk253100)
* Check resource Group Version and Kind is available in cluster before attempting restore to prevent being stuck. (#7322, @kaovilai)
* Merge CSI plugin code into Velero. (#7609, @blackpiglet)
* Fix issue #7391, remove the default constraint for node-agent pods (#7488, @Lyndon-Li)
* Fix DataDownload fails during restore for empty PVC workload (#7521, @qiuming-best)
#### Velero plugins now support handling volumes created by the CSI drivers of cloud providers
Versions 1.4 of the Velero plugins for AWS, Azure and GCP now support snapshotting and restoring the persistent volumes provisioned by CSI driver via the APIs of the cloud providers. With this enhancement, users can backup and restore the persistent volumes on these cloud providers without using the Velero CSI plugin. The CSI plugin will remain beta and the feature flag `EnableCSI` will be disabled by default.
For the version of the plugins and the CSI drivers they support respectively please see the table:
We've verified the functionality of Velero on IPv6 dual stack by successfully running the E2E test on IPv6 dual stack environment.
#### Refactor the controllers using Kubebuilder v3
In this release we continued our code modernization work, rewriting some controllers using Kubebuilder v3. This work is ongoing and we will continue to make progress in future releases.
#### Enhancements to E2E test cases
More test cases have been added to the E2E test suite to improve the release health.
#### Respect the cron setting of scheduled backup
The creation time is now taken into account to calculate the next run for scheduled backup.
#### Deleting BSLs also cleans up related resources
When a Backup Storage Location (BSL) is deleted, backup and Restic repository resources will also be deleted.
#### Breaking changes
Starting in v1.8, Velero will only support Kubernetes v1 CRD meaning that Velero v1.8+ will only run on Kubernetes v1.16+. Before upgrading, make sure you are running a supported Kubernetes version. For more information, see our [compatibility matrix](https://github.com/vmware-tanzu/velero#velero-compatibility-matrix).
#### Upload Progress Monitoring and Item Snapshotter
Item Snapshotter plugin API was merged. This will support both Upload Progress
monitoring and the planned Data Mover. Upload Progress monitoring PRs are
in progress for 1.9.
### All changes
* E2E test on ssr object with controller namespace mix-ups (#4521, @mqiu)
* Check whether the volume is provisioned by CSI driver or not by the annotation as well (#4513, @ywk253100)
* Initialize the labels field of `velero backup-location create` option to avoid #4484 (#4491, @ywk253100)
* Fix e2e 2500 namespaces scale test timeout problem (#4480, @mqiu)
* Add backup deletion e2e test (#4401, @danfengliu)
* Return the error when getting backup store in backup deletion controller (#4465, @reasonerjt)
* Ignore the provided port is already allocated error when restoring the LoadBalancer service (#4462, @ywk253100)
* Add rbac and annotation test cases (#4455, @mqiu)
* remove --crds-version in velero install command. (#4446, @jxun)
* Upgrade e2e test vsphere plugin (#4440, @mqiu)
* Fix e2e test failures for the inappropriate optimize of velero install (#4438, @mqiu)
* Limit backup namespaces on test resource filtering cases (#4437, @mqiu)
* Bump up Go to 1.17 (#4431, @reasonerjt)
* Added `<backup name>`-itemsnapshots.json.gz to the backup format. This file exists
when item snapshots are taken and contains an array of volume.Itemsnapshots
containing the information about the snapshots. This will not be used unless
upload progress monitoring and item snapshots are enabled and an ItemSnapshot
plugin is used to take snapshots.
Also added DownloadTargetKindBackupItemSnapshots for retrieving the signed URL to download only the `<backup name>`-itemsnapshots.json.gz part of a backup for use by
`velero backup describe`. (#4429, @dsmithuchida)
* Migrate backup sync controller from code-generator to kubebuilder. (#4423, @jxun)
* Added UploadProgressFeature flag to enable Upload Progress Monitoring and Item
Snapshotters. (#4416, @dsmithuchida)
* Added BackupWithResolvers and RestoreWithResolvers calls. Will eventually replace Backup and Restore methods.
Adds ItemSnapshotters to Backup and Restore workflows. (#4410, @dsu)
* Build for darwin-arm64 (#4409, @epk)
* Add resource filtering test cases (#4404, @mqiu)
* Fix the issue that the backup cannot be deleted after the application uninstalled (#4398, @ywk253100)
* Ignore the `provided port is already allocated` error when restoring the `NodePort` service (#4336, @ywk253100)
* Fixed an issue with the `backup-location create` command where the BSL Credential field would be set to an invalid empty SecretKeySelector when no credential details were provided. (#4322, @zubron)
* fix buggy pager func (#4306, @alaypatel07)
* Don't create a backup immediately after creating a schedule (#4281, @ywk253100)
* Fix CVE-2020-29652 and CVE-2020-26160 (#4274, @ywk253100)
* Refine tag-release.sh to align with change in release process (#4185, @reasonerjt)
* Fix plugins incompatible issue in upgrade test (#4141, @danfengliu)
* Verify group before treating resource as cohabiting (#4126, @sseago)
- No VolumeSnapshot will be left in the source namespace of the workload
- Report metrics for CSI snapshots
More improvements please refer to [CSI plugin improvement](https://github.com/vmware-tanzu/velero/issues?q=is%3Aissue+label%3A%22CSI+plugin+-+GA+-+phase1%22+is%3Aclosed)
With these improvements we'll provide official support for CSI snapshots on AKS/EKS clusters. (with CSI plugin v0.3.0)
#### Refactor the controllers using Kubebuilder v3
In this release we continued our code modernization work, rewriting some controllers using Kubebuilder v3. This work is ongoing and we will continue to make progress in future releases.
#### Optionally restore status on selected resources
Options are added to the CLI and Restore spec to control the group of resources whose status will be restored.
#### ExistingResourcePolicy in the restore API
Users can choose to overwrite or patch the existing resources during restore by setting this policy.
#### Upgrade integrated Restic version and add skip TLS validation in Restic command
Upgrade integrated Restic version, which will resolve some of the CVEs, and support skip TLS validation in Restic backup/restore.
#### Breaking changes
With bumping up the API to v1 in CSI plugin, the v0.3.0 CSI plugin will only work for Kubernetes v1.20+
### All changes
* restic: add full support for setting SecurityContext for restore init container from configMap. (#4084, @MatthieuFin)
* Add metrics backup_items_total and backup_items_errors (#4296, @tobiasgiese)
* Convert PodVolumebackup controller to the Kubebuilder framework (#4436, @fgold)
* Skip not mounted volumes when backing up (#4497, @dkeven)
* Update doc for v1.8 (#4517, @reasonerjt)
* Fix bug to make the restic prune frequency configurable (#4518, @ywk253100)
* Add E2E test of backups sync from BSL (#4545, @mqiu)
* Fix: OrderedResources in Schedules (#4550, @dbrekau)
* Skip volumes of non-running pods when backing up (#4584, @bynare)
* E2E SSR test add retry mechanism and logs (#4591, @mqiu)
* Add pushing image to GCR in github workflow to facilitate some environments that have rate limitation to docker hub, e.g. vSphere. (#4623, @jxun)
* Add existingResourcePolicy to Restore API (#4628, @shubham-pampattiwar)
* Fix E2E backup namespaces test (#4634, @qiuming-best)
* Update image used by E2E test to gcr.io (#4639, @jxun)
* Add multiple label selector support to Velero Backup and Restore APIs (#4650, @shubham-pampattiwar)
* Convert Pod Volume Restore resource/controller to the Kubebuilder framework (#4655, @ywk253100)
* Update --use-owner-references-in-backup description in velero command line. (#4660, @jxun)
* Avoid overwritten hook's exec.container parameter when running pod command executor. (#4661, @jxun)
* Support regional pv for GKE (#4680, @jxun)
* Bypass the remap CRD version plugin when v1beta1 CRD is not supported (#4686, @reasonerjt)
* Add GINKGO_SKIP to support skip specific case in e2e test. (#4692, @jxun)
* Add --pod-labels flag to velero install (#4694, @j4m3s-s)
* Enable coverage in test.sh and upload to codecov (#4704, @reasonerjt)
* Mark the BSL as "Unavailable" when gets any error and add a new field "Message" to the status to record the error message (#4719, @ywk253100)
* Support multiple skip option for E2E test (#4725, @jxun)
* Add PriorityClass to the AdditionalItems of Backup's PodAction and Restore's PodAction plugin to backup and restore PriorityClass if it is used by a Pod. (#4740, @phuongatemc)
* Insert all restore errors and warnings into restore log. (#4743, @sseago)
* Refactor schedule controller with kubebuilder (#4748, @ywk253100)
* Garbage collector now adds labels to backups that failed to delete for BSLNotFound, BSLCannotGet, BSLReadOnly reasons. (#4757, @kaovilai)
* Skip podvolumerestore creation when restore excludes pv/pvc (#4769, @half-life666)
* Add parameter for e2e test to support modify kibishii install path. (#4778, @jxun)
* Ensure the restore hook applied to new namespace based on the mapping (#4779, @reasonerjt)
* Add ability to restore status on selected resources (#4785, @RafaeLeal)
* Do not take snapshot for PV to avoid duplicated snapshotting, when CSI feature is enabled. (#4797, @jxun)
* Bump up to v1 API for CSI snapshot (#4800, @reasonerjt)
* fix: delete empty backups (#4817, @yuvalman)
* Add CSI VolumeSnapshot related metrics. (#4818, @jxun)
* Fix default-backup-ttl not work (#4831, @qiuming-best)
* Make the vsc created by backup sync controller deletable (#4832, @reasonerjt)
* Make in-progress backup/restore as failed when doing the reconcile to avoid hanging in in-progress status (#4833, @ywk253100)
* Use controller-gen to generate the deep copy methods for objects (#4838, @ywk253100)
* Update integrated Restic version and add insecureSkipTLSVerify for Restic CLI. (#4839, @jxun)
* Modify CSI VolumeSnapshot metric related code. (#4854, @jxun)
* Refactor backup deletion controller based on kubebuilder (#4855, @reasonerjt)
* Remove VolumeSnapshots created during backup when CSI feature is enabled. (#4858, @jxun)
* Convert Restic Repository resource/controller to the Kubebuilder framework (#4859, @qiuming-best)
* Add ClusterClasses to the restore priority list (#4866, @reasonerjt)
* Cleanup the .velero folder after restic done (#4872, @big-appled)
description:Backup is a Velero resource that represents the capture of Kubernetes
description:|-
Backup is a Velero resource that represents the capture of Kubernetes
cluster state at a point in time (API objects and associated volume state).
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
description:|-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
description:|-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type:string
metadata:
type:object
spec:
description:BackupSpec defines the specification for a Velero backup.
properties:
defaultVolumesToRestic:
description:DefaultVolumesToRestic specifies whether restic should
be used to take a backup of all pod volumes by default.
csiSnapshotTimeout:
description:|-
CSISnapshotTimeout specifies the time used to wait for CSI VolumeSnapshot status turns to
ReadyToUse during creation, before returning error as timeout.
The default value is 10 minute.
type:string
datamover:
description:|-
DataMover specifies the data mover to be used by the backup.
If DataMover is "" or "velero", the built-in data mover will be used.
type:string
defaultVolumesToFsBackup:
description:|-
DefaultVolumesToFsBackup specifies whether pod volume file system backup should be used
for all volumes by default.
nullable:true
type:boolean
defaultVolumesToRestic:
description:|-
DefaultVolumesToRestic specifies whether restic should be used to take a
backup of all pod volumes by default.
Deprecated: this field is no longer used and will be removed entirely in future. Use DefaultVolumesToFsBackup instead.
nullable:true
type:boolean
excludedClusterScopedResources:
description:|-
ExcludedClusterScopedResources is a slice of cluster-scoped
resource type names to exclude from the backup.
If set to "*", all cluster-scoped resource types are excluded.
The default value is empty.
items:
type:string
nullable:true
type:array
excludedNamespaceScopedResources:
description:|-
ExcludedNamespaceScopedResources is a slice of namespace-scoped
resource type names to exclude from the backup.
If set to "*", all namespace-scoped resource types are excluded.
The default value is empty.
items:
type:string
nullable:true
type:array
excludedNamespaces:
description:ExcludedNamespaces contains a list of namespaces that
are not included in the backup.
description:|-
ExcludedNamespaces contains a list of namespaces that are not
included in the backup.
items:
type:string
nullable:true
type:array
excludedResources:
description:ExcludedResources is a slice of resource names that are
not included in the backup.
description:|-
ExcludedResources is a slice of resource names that are not
included in the backup.
items:
type:string
nullable:true
@@ -63,9 +111,9 @@ spec:
description:Resources are hooks that should be executed when
backing up individual instances of a resource.
items:
description:BackupResourceHookSpec defines one or more BackupResourceHooks
that should be executed based on the rules defined for namespaces,
resources, and label selector.
description:|-
BackupResourceHookSpec defines one or more BackupResourceHooks that should be executed based on
the rules defined for namespaces, resources, and label selector.
properties:
excludedNamespaces:
description:ExcludedNamespaces specifies the namespaces
@@ -82,17 +130,17 @@ spec:
nullable:true
type:array
includedNamespaces:
description:IncludedNamespaces specifies the namespaces
to which this hook spec applies. If empty, it applies
description:|-
IncludedNamespaces specifies the namespaces to which this hook spec applies. If empty, it applies
to all namespaces.
items:
type:string
nullable:true
type:array
includedResources:
description:IncludedResources specifies the resources to
which this hook spec applies. If empty, it applies to
all resources.
description:|-
IncludedResources specifies the resources to which this hook spec applies. If empty, it applies
to all resources.
items:
type:string
nullable:true
@@ -106,8 +154,8 @@ spec:
description:matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description:A label selector requirement is a selector
that contains values, a key, and an operator that
description:|-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
@@ -115,17 +163,16 @@ spec:
applies to.
type:string
operator:
description:operator represents a key's relationship
to a set of values. Valid operators are In,
NotIn, Exists and DoesNotExist.
description:|-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type:string
values:
description:values is an array of string values.
If the operator is In or NotIn, the values array
must be non-empty. If the operator is Exists
or DoesNotExist, the values array must be empty.
This array is replaced during a strategic merge
patch.
description:|-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type:string
type:array
@@ -137,21 +184,20 @@ spec:
matchLabels:
additionalProperties:
type:string
description:matchLabels is a map of {key,value} pairs.
A single {key,value} in the matchLabels map is equivalent
to an element of matchExpressions, whose key field
is "key", the operator is "In", and the values array
contains only "value". The requirements are ANDed.
description:|-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type:object
type:object
x-kubernetes-map-type:atomic
name:
description:Name is the name of this hook.
type:string
post:
description:PostHooks is a list of BackupResourceHooks
to execute after storing the item in the backup. These
are executed after all "additional items" from item actions
are processed.
description:|-
PostHooks is a list of BackupResourceHooks to execute after storing the item in the backup.
These are executed after all "additional items" from item actions are processed.
items:
description:BackupResourceHook defines a hook for a resource.
properties:
@@ -166,10 +212,9 @@ spec:
minItems:1
type:array
container:
description:Container is the container in the
pod where the command should be executed. If
not specified, the pod's first container is
used.
description:|-
Container is the container in the pod where the command should be executed. If not specified,
the pod's first container is used.
type:string
onError:
description:OnError specifies how Velero should
@@ -180,9 +225,9 @@ spec:
- Fail
type:string
timeout:
description:Timeout defines the maximum amount
of time Velero should wait for the hook to complete
before considering the execution a failure.
description:|-
Timeout defines the maximum amount of time Velero should wait for the hook to complete before
considering the execution a failure.
type:string
required:
- command
@@ -192,10 +237,9 @@ spec:
type:object
type:array
pre:
description:PreHooks is a list of BackupResourceHooks to
execute prior to storing the item in the backup. These
are executed before any "additional items" from item actions
are processed.
description:|-
PreHooks is a list of BackupResourceHooks to execute prior to storing the item in the backup.
These are executed before any "additional items" from item actions are processed.
items:
description:BackupResourceHook defines a hook for a resource.
properties:
@@ -210,10 +254,9 @@ spec:
minItems:1
type:array
container:
description:Container is the container in the
pod where the command should be executed. If
not specified, the pod's first container is
used.
description:|-
Container is the container in the pod where the command should be executed. If not specified,
the pod's first container is used.
type:string
onError:
description:OnError specifies how Velero should
@@ -224,9 +267,9 @@ spec:
- Fail
type:string
timeout:
description:Timeout defines the maximum amount
of time Velero should wait for the hook to complete
before considering the execution a failure.
description:|-
Timeout defines the maximum amount of time Velero should wait for the hook to complete before
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
description:|-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
description:|-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type:string
metadata:
type:object
@@ -83,8 +86,10 @@ spec:
valid secret key.
type:string
name:
description: 'Name of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO:Add other useful fields. apiVersion, kind, uid?'
description:|-
Name of the referent.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind, uid?
type:string
optional:
description:Specify whether the Secret or its key must be defined
@@ -92,6 +97,7 @@ spec:
required:
- key
type:object
x-kubernetes-map-type:atomic
default:
description:Default indicates this location is the default backup
storage location.
@@ -132,32 +138,43 @@ spec:
BackupStorageLocation
properties:
accessMode:
description:"AccessMode is an unused field. \n Deprecated: there
is now an AccessMode field on the Spec and this field will be removed
entirely as of v2.0."
description:|-
AccessMode is an unused field.
Deprecated: there is now an AccessMode field on the Spec and this field
will be removed entirely as of v2.0.
enum:
- ReadOnly
- ReadWrite
type:string
lastSyncedRevision:
description:"LastSyncedRevision is the value of the `metadata/revision`
file in the backup storage location the last time the BSL's contents
were synced into the cluster. \n Deprecated: this field is no longer
updated or used for detecting changes to the location's contents
and will be removed entirely in v2.0."
description:|-
LastSyncedRevision is the value of the `metadata/revision` file in the backup
storage location the last time the BSL's contents were synced into the cluster.
Deprecated: this field is no longer updated or used for detecting changes to
the location's contents and will be removed entirely in v2.0.
type:string
lastSyncedTime:
description:LastSyncedTime is the last time the contents of the location
were synced into the cluster.
description:|-
LastSyncedTime is the last time the contents of the location were synced into
the cluster.
format:date-time
nullable:true
type:string
lastValidationTime:
description:LastValidationTime is the last time the backup store
location was validated the cluster.
description:|-
LastValidationTime is the last time the backup store location was validated
the cluster.
format:date-time
nullable:true
type:string
message:
description:Message is a message about the backup storage location's
status.
type:string
phase:
description:Phase is the current state of the BackupStorageLocation.
- description:The name of the backup to be deleted
jsonPath:.spec.backupName
name:BackupName
type:string
- description:The status of the deletion request
jsonPath:.status.phase
name:Status
type:string
name:v1
schema:
openAPIV3Schema:
description:DeleteBackupRequest is a request to delete one or more backups.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
description:|-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
description:|-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
description:DownloadRequest is a request to download an artifact from backup
object storage, such as a backup log file.
description:|-
DownloadRequest is a request to download an artifact from backup object storage, such as a backup
log file.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
description:|-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
description:|-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type:string
metadata:
type:object
@@ -46,12 +50,20 @@ spec:
- BackupLog
- BackupContents
- BackupVolumeSnapshots
- BackupItemOperations
- BackupResourceList
- BackupResults
- RestoreLog
- RestoreResults
- RestoreResourceList
- RestoreItemOperations
- CSIBackupVolumeSnapshots
- CSIBackupVolumeSnapshotContents
- BackupVolumeInfos
- RestoreVolumeInfo
type:string
name:
description:Name is the name of the kubernetes resource with
description:Name is the name of the Kubernetes resource with
- description:Pod Volume Backup status such as New/InProgress
jsonPath:.status.phase
name:Status
type:string
- description:Time when this backup was started
jsonPath:.status.startTimestamp
name:Created
type:date
- description:Namespace of the pod containing the volume to be backed up
jsonPath:.spec.pod.namespace
name:Namespace
type:string
- description:Name of the pod containing the volume to be backed up
jsonPath:.spec.pod.name
name:Pod
type:string
- description:Name of the volume to be backed up
jsonPath:.spec.volume
name:Volume
type:string
- description:The type of the uploader to handle data transfer
jsonPath:.spec.uploaderType
name:Uploader Type
type:string
- description:Name of the Backup Storage Location where this backup should be
stored
jsonPath:.spec.backupStorageLocation
name:Storage Location
type:string
- jsonPath:.metadata.creationTimestamp
name:Age
type:date
name:v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
description:|-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
description:|-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type:string
metadata:
type:object
@@ -36,8 +72,9 @@ spec:
description:PodVolumeBackupSpec is the specification for a PodVolumeBackup.
properties:
backupStorageLocation:
description:BackupStorageLocation is the name of the backup storage
location where the restic repository is stored.
description:|-
BackupStorageLocation is the name of the backup storage location
where the backup repository is stored.
type:string
node:
description:Node is the name of the node that the Pod is running
@@ -51,47 +88,73 @@ spec:
description:API version of the referent.
type:string
fieldPath:
description:'If referring to a piece of an object instead of
an entire object, this string should contain a valid JSON/Go
field access statement, such as desiredState.manifest.containers[2].
For example, if the object reference is to a container within
a pod, this would take on a value like: "spec.containers{name}"
(where "name" refers to the name of the container that triggered
the event) or if no container name is specified "spec.containers[2]"
(container with index 2 in this pod). This syntax is chosen
only to have some well-defined way of referencing a part of
an object. TODO: this design is not final and this field is
subject to change in the future.'
description:|-
If referring to a piece of an object instead of an entire object, this string
should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2].
For example, if the object reference is to a container within a pod, this would take on a value like:
"spec.containers{name}" (where "name" refers to the name of the container that triggered
the event) or if no container name is specified "spec.containers[2]" (container with
index 2 in this pod). This syntax is chosen only to have some well-defined way of
referencing a part of an object.
TODO: this design is not final and this field is subject to change in the future.
type:string
kind:
description: 'Kind of the referent. More info:https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
description:|-
Kind of the referent.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type:string
name:
description: 'Name of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
description:|-
Name of the referent.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
type:string
namespace:
description: 'Namespace of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
description:|-
Namespace of the referent.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/
type:string
resourceVersion:
description:'Specific resourceVersion to which this reference
is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
description:|-
Specific resourceVersion to which this reference is made, if any.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency
type:string
uid:
description: 'UID of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
description:|-
UID of the referent.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids
type:string
type:object
x-kubernetes-map-type:atomic
repoIdentifier:
description:RepoIdentifier is the restic repository identifier.
description:RepoIdentifier is the backup repository identifier.
type:string
tags:
additionalProperties:
type:string
description:Tags are a map of key-value pairs that should be applied
to the volume backup as tags.
description:|-
Tags are a map of key-value pairs that should be applied to the
volume backup as tags.
type:object
uploaderSettings:
additionalProperties:
type:string
description:|-
UploaderSettings are a map of key-value pairs that should be applied to the
uploader configuration.
nullable:true
type:object
uploaderType:
description:UploaderType is the type of the uploader to handle the
data transfer.
enum:
- kopia
- restic
- ""
type:string
volume:
description:Volume is the name of the volume within the Pod to be
backed up.
description:|-
Volume is the name of the volume within the Pod to be backed
up.
type:string
required:
- backupStorageLocation
@@ -104,10 +167,11 @@ spec:
description:PodVolumeBackupStatus is the current status of a PodVolumeBackup.
properties:
completionTimestamp:
description:CompletionTimestamp records the time a backup was completed.
Completion time is recorded even on failed backups. Completion time
is recorded before uploading the backup object. The server's time
is used for CompletionTimestamps
description:|-
CompletionTimestamp records the time a backup was completed.
Completion time is recorded even on failed backups.
Completion time is recorded before uploading the backup object.
The server's time is used for CompletionTimestamps
format:date-time
nullable:true
type:string
@@ -127,9 +191,10 @@ spec:
- Failed
type:string
progress:
description:Progress holds the total number of bytes of the volume
and the current number of backed up bytes. This can be used to display
progress information about the backup operation.
description:|-
Progress holds the total number of bytes of the volume and the current
number of backed up bytes. This can be used to display progress information
about the backup operation.
properties:
bytesDone:
format:int64
@@ -143,8 +208,10 @@ spec:
pod volume.
type:string
startTimestamp:
description:StartTimestamp records the time a backup was started.
Separate from CreationTimestamp, since that value changes on restores.
description:|-
StartTimestamp records the time a backup was started.
Separate from CreationTimestamp, since that value changes
- description:Namespace of the pod containing the volume to be restored
jsonPath:.spec.pod.namespace
name:Namespace
type:string
- description:Name of the pod containing the volume to be restored
jsonPath:.spec.pod.name
name:Pod
type:string
- description:The type of the uploader to handle data transfer
jsonPath:.spec.uploaderType
name:Uploader Type
type:string
- description:Name of the volume to be restored
jsonPath:.spec.volume
name:Volume
type:string
- description:Pod Volume Restore status such as New/InProgress
jsonPath:.status.phase
name:Status
type:string
- description:Pod Volume Restore status such as New/InProgress
format:int64
jsonPath:.status.progress.totalBytes
name:TotalBytes
type:integer
- description:Pod Volume Restore status such as New/InProgress
format:int64
jsonPath:.status.progress.bytesDone
name:BytesDone
type:integer
- jsonPath:.metadata.creationTimestamp
name:Age
type:date
name:v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
description:|-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
description:|-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type:string
metadata:
type:object
@@ -36,8 +73,9 @@ spec:
description:PodVolumeRestoreSpec is the specification for a PodVolumeRestore.
properties:
backupStorageLocation:
description:BackupStorageLocation is the name of the backup storage
location where the restic repository is stored.
description:|-
BackupStorageLocation is the name of the backup storage location
where the backup repository is stored.
type:string
pod:
description:Pod is a reference to the pod containing the volume to
@@ -47,41 +85,69 @@ spec:
description:API version of the referent.
type:string
fieldPath:
description:'If referring to a piece of an object instead of
an entire object, this string should contain a valid JSON/Go
field access statement, such as desiredState.manifest.containers[2].
For example, if the object reference is to a container within
a pod, this would take on a value like: "spec.containers{name}"
(where "name" refers to the name of the container that triggered
the event) or if no container name is specified "spec.containers[2]"
(container with index 2 in this pod). This syntax is chosen
only to have some well-defined way of referencing a part of
an object. TODO: this design is not final and this field is
subject to change in the future.'
description:|-
If referring to a piece of an object instead of an entire object, this string
should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2].
For example, if the object reference is to a container within a pod, this would take on a value like:
"spec.containers{name}" (where "name" refers to the name of the container that triggered
the event) or if no container name is specified "spec.containers[2]" (container with
index 2 in this pod). This syntax is chosen only to have some well-defined way of
referencing a part of an object.
TODO: this design is not final and this field is subject to change in the future.
type:string
kind:
description: 'Kind of the referent. More info:https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
description:|-
Kind of the referent.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type:string
name:
description: 'Name of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
description:|-
Name of the referent.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
type:string
namespace:
description: 'Namespace of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
description:|-
Namespace of the referent.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/
type:string
resourceVersion:
description:'Specific resourceVersion to which this reference
is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
description:|-
Specific resourceVersion to which this reference is made, if any.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency
type:string
uid:
description: 'UID of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
description:|-
UID of the referent.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids
type:string
type:object
x-kubernetes-map-type:atomic
repoIdentifier:
description:RepoIdentifier is the restic repository identifier.
description:RepoIdentifier is the backup repository identifier.
type:string
snapshotID:
description:SnapshotID is the ID of the volume snapshot to be restored.
type:string
sourceNamespace:
description:SourceNamespace is the original namespace for namaspace
mapping.
type:string
uploaderSettings:
additionalProperties:
type:string
description:|-
UploaderSettings are a map of key-value pairs that should be applied to the
uploader configuration.
nullable:true
type:object
uploaderType:
description:UploaderType is the type of the uploader to handle the
data transfer.
enum:
- kopia
- restic
- ""
type:string
volume:
description:Volume is the name of the volume within the Pod to be
restored.
@@ -91,15 +157,17 @@ spec:
- pod
- repoIdentifier
- snapshotID
- sourceNamespace
- volume
type:object
status:
description:PodVolumeRestoreStatus is the current status of a PodVolumeRestore.
properties:
completionTimestamp:
description:CompletionTimestamp records the time a restore was completed.
Completion time is recorded even on failed restores. The server's
time is used for CompletionTimestamps
description:|-
CompletionTimestamp records the time a restore was completed.
Completion time is recorded even on failed restores.
The server's time is used for CompletionTimestamps
format:date-time
nullable:true
type:string
@@ -115,9 +183,10 @@ spec:
- Failed
type:string
progress:
description:Progress holds the total number of bytes of the snapshot
and the current number of restored bytes. This can be used to display
progress information about the restore operation.
description:|-
Progress holds the total number of bytes of the snapshot and the current
number of restored bytes. This can be used to display progress information
about the restore operation.
properties:
bytesDone:
format:int64
@@ -127,7 +196,8 @@ spec:
type:integer
type:object
startTimestamp:
description:StartTimestamp records the time a restore was started.
description:|-
StartTimestamp records the time a restore was started.
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:ResticRepositorySpec is the specification for a ResticRepository.
properties:
backupStorageLocation:
description:BackupStorageLocation is the name of the BackupStorageLocation
that should contain this repository.
type:string
maintenanceFrequency:
description:MaintenanceFrequency is how often maintenance should
be run.
type:string
resticIdentifier:
description:ResticIdentifier is the full restic-compatible string
for identifying this repository.
type:string
volumeNamespace:
description:VolumeNamespace is the namespace this restic repository
contains pod volume backups for.
type:string
required:
- backupStorageLocation
- maintenanceFrequency
- resticIdentifier
- volumeNamespace
type:object
status:
description:ResticRepositoryStatus is the current status of a ResticRepository.
properties:
lastMaintenanceTime:
description:LastMaintenanceTime is the last time maintenance was
run.
format:date-time
nullable:true
type:string
message:
description:Message is a message about the current status of the
ResticRepository.
type:string
phase:
description:Phase is the current state of the ResticRepository.
- description:A Cron expression defining when to run the Backup
jsonPath:.spec.schedule
name:Schedule
type:string
- description:The last time a Backup was run for this schedule
jsonPath:.status.lastBackup
name:LastBackup
type:date
- jsonPath:.metadata.creationTimestamp
name:Age
type:date
- jsonPath:.spec.paused
name:Paused
type:boolean
name:v1
schema:
openAPIV3Schema:
description:Schedule is a Velero resource that represents a pre-scheduled
or periodic Backup that should be run.
description:|-
Schedule is a Velero resource that represents a pre-scheduled or
periodic Backup that should be run.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
description:|-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
description:|-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type:string
metadata:
type:object
spec:
description:ScheduleSpec defines the specification for a Velero schedule
properties:
paused:
description:Paused specifies whether the schedule is paused or not
type:boolean
schedule:
description:Schedule is a Cron expression defining when to run the
Backup.
description:|-
Schedule is a Cron expression defining when to run
the Backup.
type:string
skipImmediately:
description:|-
SkipImmediately specifies whether to skip backup if schedule is due immediately from `schedule.status.lastBackup` timestamp when schedule is unpaused or if schedule is new.
If true, backup will be skipped immediately when schedule is unpaused if it is due based on .Status.LastBackupTimestamp or schedule is new, and will run at next schedule time.
If false, backup will not be skipped immediately when schedule is unpaused, but will run at next schedule time.
If empty, will follow server configuration (default: false).
type:boolean
template:
description:Template is the definition of the Backup to be run on
the provided schedule
description:|-
Template is the definition of the Backup to be run
on the provided schedule
properties:
defaultVolumesToRestic:
description:DefaultVolumesToRestic specifies whether restic should
be used to take a backup of all pod volumes by default.
csiSnapshotTimeout:
description:|-
CSISnapshotTimeout specifies the time used to wait for CSI VolumeSnapshot status turns to
ReadyToUse during creation, before returning error as timeout.
The default value is 10 minute.
type:string
datamover:
description:|-
DataMover specifies the data mover to be used by the backup.
If DataMover is "" or "velero", the built-in data mover will be used.
type:string
defaultVolumesToFsBackup:
description:|-
DefaultVolumesToFsBackup specifies whether pod volume file system backup should be used
for all volumes by default.
nullable:true
type:boolean
defaultVolumesToRestic:
description:|-
DefaultVolumesToRestic specifies whether restic should be used to take a
backup of all pod volumes by default.
Deprecated: this field is no longer used and will be removed entirely in future. Use DefaultVolumesToFsBackup instead.
nullable:true
type:boolean
excludedClusterScopedResources:
description:|-
ExcludedClusterScopedResources is a slice of cluster-scoped
resource type names to exclude from the backup.
If set to "*", all cluster-scoped resource types are excluded.
The default value is empty.
items:
type:string
nullable:true
type:array
excludedNamespaceScopedResources:
description:|-
ExcludedNamespaceScopedResources is a slice of namespace-scoped
resource type names to exclude from the backup.
If set to "*", all namespace-scoped resource types are excluded.
The default value is empty.
items:
type:string
nullable:true
type:array
excludedNamespaces:
description:ExcludedNamespaces contains a list of namespaces
that are not included in the backup.
description:|-
ExcludedNamespaces contains a list of namespaces that are not
included in the backup.
items:
type:string
nullable:true
type:array
excludedResources:
description:ExcludedResources is a slice of resource names that
are not included in the backup.
description:|-
ExcludedResources is a slice of resource names that are not
included in the backup.
items:
type:string
nullable:true
@@ -71,9 +150,9 @@ spec:
description:Resources are hooks that should be executed when
backing up individual instances of a resource.
items:
description:BackupResourceHookSpec defines one or more
BackupResourceHooks that should be executed based on the
rules defined for namespaces, resources, and label selector.
description:|-
BackupResourceHookSpec defines one or more BackupResourceHooks that should be executed based on
the rules defined for namespaces, resources, and label selector.
properties:
excludedNamespaces:
description:ExcludedNamespaces specifies the namespaces
@@ -90,16 +169,16 @@ spec:
nullable:true
type:array
includedNamespaces:
description:IncludedNamespaces specifies the namespaces
to which this hook spec applies. If empty, it applies
description:|-
IncludedNamespaces specifies the namespaces to which this hook spec applies. If empty, it applies
to all namespaces.
items:
type:string
nullable:true
type:array
includedResources:
description:IncludedResources specifies the resources
to which this hook spec applies. If empty, it applies
description:|-
IncludedResources specifies the resources to which this hook spec applies. If empty, it applies
to all resources.
items:
type:string
@@ -114,26 +193,25 @@ spec:
description:matchExpressions is a list of label
selector requirements. The requirements are ANDed.
items:
description:A label selector requirement is a
selector that contains values, a key, and an
operator that relates the key and values.
description:|-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description:key is the label key that the
selector applies to.
type:string
operator:
description:operator represents a key's relationship
to a set of values. Valid operators are
In, NotIn, Exists and DoesNotExist.
description:|-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type:string
values:
description:values is an array of string
values. If the operator is In or NotIn,
the values array must be non-empty. If the
operator is Exists or DoesNotExist, the
values array must be empty. This array is
replaced during a strategic merge patch.
description:|-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type:string
type:array
@@ -145,22 +223,20 @@ spec:
matchLabels:
additionalProperties:
type:string
description:matchLabels is a map of {key,value}
pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions,
whose key field is "key", the operator is "In",
and the values array contains only "value". The
requirements are ANDed.
description:|-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type:object
type:object
x-kubernetes-map-type:atomic
name:
description:Name is the name of this hook.
type:string
post:
description:PostHooks is a list of BackupResourceHooks
to execute after storing the item in the backup. These
are executed after all "additional items" from item
actions are processed.
description:|-
PostHooks is a list of BackupResourceHooks to execute after storing the item in the backup.
These are executed after all "additional items" from item actions are processed.
items:
description:BackupResourceHook defines a hook for
a resource.
@@ -176,10 +252,9 @@ spec:
minItems:1
type:array
container:
description:Container is the container in
the pod where the command should be executed.
If not specified, the pod's first container
is used.
description:|-
Container is the container in the pod where the command should be executed. If not specified,
the pod's first container is used.
type:string
onError:
description:OnError specifies how Velero
@@ -190,10 +265,9 @@ spec:
- Fail
type:string
timeout:
description:Timeout defines the maximum amount
of time Velero should wait for the hook
to complete before considering the execution
a failure.
description:|-
Timeout defines the maximum amount of time Velero should wait for the hook to complete before
considering the execution a failure.
type:string
required:
- command
@@ -203,10 +277,9 @@ spec:
type:object
type:array
pre:
description:PreHooks is a list of BackupResourceHooks
to execute prior to storing the item in the backup.
These are executed before any "additional items" from
item actions are processed.
description:|-
PreHooks is a list of BackupResourceHooks to execute prior to storing the item in the backup.
These are executed before any "additional items" from item actions are processed.
items:
description:BackupResourceHook defines a hook for
a resource.
@@ -222,10 +295,9 @@ spec:
minItems:1
type:array
container:
description:Container is the container in
the pod where the command should be executed.
If not specified, the pod's first container
is used.
description:|-
Container is the container in the pod where the command should be executed. If not specified,
the pod's first container is used.
type:string
onError:
description:OnError specifies how Velero
@@ -236,10 +308,9 @@ spec:
- Fail
type:string
timeout:
description:Timeout defines the maximum amount
of time Velero should wait for the hook
to complete before considering the execution
a failure.
description:|-
Timeout defines the maximum amount of time Velero should wait for the hook to complete before
description:ServerStatusRequest is a request to access current status information
about the Velero server.
description:|-
ServerStatusRequest is a request to access current status information about
the Velero server.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
description:|-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
description:|-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type:string
metadata:
type:object
@@ -65,8 +69,9 @@ spec:
nullable:true
type:array
processedTimestamp:
description:ProcessedTimestamp is when the ServerStatusRequest was
processed by the ServerStatusRequestController.
description:|-
ProcessedTimestamp is when the ServerStatusRequest was processed
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
description:|-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
description:|-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type:string
metadata:
type:object
@@ -43,6 +48,27 @@ spec:
type:string
description:Config is for provider-specific configuration fields.
type:object
credential:
description:Credential contains the credential information intended
to be used with this location
properties:
key:
description:The key of the secret to select from. Must be a
valid secret key.
type:string
name:
description:|-
Name of the referent.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind, uid?
type:string
optional:
description:Specify whether the Secret or its key must be defined
type:boolean
required:
- key
type:object
x-kubernetes-map-type:atomic
provider:
description:Provider is the provider of the volume storage.
description:Backup is a Velero resource that represents the capture of Kubernetes
cluster state at a point in time (API objects and associated volume state).
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:BackupSpec defines the specification for a Velero backup.
properties:
defaultVolumesToRestic:
description:DefaultVolumesToRestic specifies whether restic should
be used to take a backup of all pod volumes by default.
type:boolean
excludedNamespaces:
description:ExcludedNamespaces contains a list of namespaces that are
not included in the backup.
items:
type:string
nullable:true
type:array
excludedResources:
description:ExcludedResources is a slice of resource names that are
not included in the backup.
items:
type:string
nullable:true
type:array
hooks:
description:Hooks represent custom behaviors that should be executed
at different phases of the backup.
properties:
resources:
description:Resources are hooks that should be executed when backing
up individual instances of a resource.
items:
description:BackupResourceHookSpec defines one or more BackupResourceHooks
that should be executed based on the rules defined for namespaces,
resources, and label selector.
properties:
excludedNamespaces:
description:ExcludedNamespaces specifies the namespaces to
which this hook spec does not apply.
items:
type:string
nullable:true
type:array
excludedResources:
description:ExcludedResources specifies the resources to
which this hook spec does not apply.
items:
type:string
nullable:true
type:array
includedNamespaces:
description:IncludedNamespaces specifies the namespaces to
which this hook spec applies. If empty, it applies to all
namespaces.
items:
type:string
nullable:true
type:array
includedResources:
description:IncludedResources specifies the resources to
which this hook spec applies. If empty, it applies to all
resources.
items:
type:string
nullable:true
type:array
labelSelector:
description:LabelSelector, if specified, filters the resources
to which this hook spec applies.
nullable:true
properties:
matchExpressions:
description:matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description:A label selector requirement is a selector
that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description:key is the label key that the selector
applies to.
type:string
operator:
description:operator represents a key's relationship
to a set of values. Valid operators are In, NotIn,
Exists and DoesNotExist.
type:string
values:
description:values is an array of string values.
If the operator is In or NotIn, the values array
must be non-empty. If the operator is Exists or
DoesNotExist, the values array must be empty.
This array is replaced during a strategic merge
patch.
items:
type:string
type:array
required:
- key
- operator
type:object
type:array
matchLabels:
additionalProperties:
type:string
description:matchLabels is a map of {key,value} pairs.
A single {key,value} in the matchLabels map is equivalent
to an element of matchExpressions, whose key field is
"key",the operator is "In", and the values array contains
only "value". The requirements are ANDed.
type:object
type:object
name:
description:Name is the name of this hook.
type:string
post:
description:PostHooks is a list of BackupResourceHooks to
execute after storing the item in the backup. These are
executed after all "additional items" from item actions
are processed.
items:
description:BackupResourceHook defines a hook for a resource.
properties:
exec:
description:Exec defines an exec hook.
properties:
command:
description:Command is the command and arguments
to execute.
items:
type:string
minItems:1
type:array
container:
description:Container is the container in the pod
where the command should be executed. If not specified,
the pod's first container is used.
type:string
onError:
description:OnError specifies how Velero should
behave if it encounters an error executing this
hook.
enum:
- Continue
- Fail
type:string
timeout:
description:Timeout defines the maximum amount
of time Velero should wait for the hook to complete
before considering the execution a failure.
type:string
required:
- command
type:object
required:
- exec
type:object
type:array
pre:
description:PreHooks is a list of BackupResourceHooks to
execute prior to storing the item in the backup. These are
executed before any "additional items" from item actions
are processed.
items:
description:BackupResourceHook defines a hook for a resource.
properties:
exec:
description:Exec defines an exec hook.
properties:
command:
description:Command is the command and arguments
to execute.
items:
type:string
minItems:1
type:array
container:
description:Container is the container in the pod
where the command should be executed. If not specified,
the pod's first container is used.
type:string
onError:
description:OnError specifies how Velero should
behave if it encounters an error executing this
hook.
enum:
- Continue
- Fail
type:string
timeout:
description:Timeout defines the maximum amount
of time Velero should wait for the hook to complete
description:Backup Storage Location status such as Available/Unavailable
name:Phase
type:string
- JSONPath:.status.lastValidationTime
description:LastValidationTime is the last time the backup store location was
validated
name:Last Validated
type:date
- JSONPath:.metadata.creationTimestamp
name:Age
type:date
- JSONPath:.spec.default
description:Default backup storage location
name:Default
type:boolean
group:velero.io
names:
kind:BackupStorageLocation
listKind:BackupStorageLocationList
plural:backupstoragelocations
shortNames:
- bsl
singular:backupstoragelocation
preserveUnknownFields:false
scope:Namespaced
subresources:
status:{}
validation:
openAPIV3Schema:
description:BackupStorageLocation is a location where Velero stores backup
objects
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:BackupStorageLocationSpec defines the desired state of a Velero
BackupStorageLocation
properties:
accessMode:
description:AccessMode defines the permissions for the backup storage
location.
enum:
- ReadOnly
- ReadWrite
type:string
backupSyncPeriod:
description:BackupSyncPeriod defines how frequently to sync backup
API objects from object storage. A value of 0 disables sync.
nullable:true
type:string
config:
additionalProperties:
type:string
description:Config is for provider-specific configuration fields.
type:object
credential:
description:Credential contains the credential information intended
to be used with this location
properties:
key:
description:The key of the secret to select from. Must be a valid
secret key.
type:string
name:
description: 'Name of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO:Add other useful fields. apiVersion, kind, uid?'
type:string
optional:
description:Specify whether the Secret or its key must be defined
type:boolean
required:
- key
type:object
default:
description:Default indicates this location is the default backup storage
location.
type:boolean
objectStorage:
description:ObjectStorageLocation specifies the settings necessary
to connect to a provider's object storage.
properties:
bucket:
description:Bucket is the bucket to use for object storage.
type:string
caCert:
description:CACert defines a CA bundle to use when verifying TLS
connections to the provider.
format:byte
type:string
prefix:
description:Prefix is the path inside a bucket to use for Velero
storage. Optional.
type:string
required:
- bucket
type:object
provider:
description:Provider is the provider of the backup storage.
type:string
validationFrequency:
description:ValidationFrequency defines how frequently to validate
the corresponding object storage. A value of 0 disables validation.
nullable:true
type:string
required:
- objectStorage
- provider
type:object
status:
description:BackupStorageLocationStatus defines the observed state of BackupStorageLocation
properties:
accessMode:
description:"AccessMode is an unused field. \n Deprecated: there is
now an AccessMode field on the Spec and this field will be removed
entirely as of v2.0."
enum:
- ReadOnly
- ReadWrite
type:string
lastSyncedRevision:
description:"LastSyncedRevision is the value of the `metadata/revision`
file in the backup storage location the last time the BSL's contents
were synced into the cluster. \n Deprecated: this field is no longer
updated or used for detecting changes to the location's contents and
will be removed entirely in v2.0."
type:string
lastSyncedTime:
description:LastSyncedTime is the last time the contents of the location
were synced into the cluster.
format:date-time
nullable:true
type:string
lastValidationTime:
description:LastValidationTime is the last time the backup store location
was validated the cluster.
format:date-time
nullable:true
type:string
phase:
description:Phase is the current state of the BackupStorageLocation.
description:DeleteBackupRequest is a request to delete one or more backups.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:DeleteBackupRequestSpec is the specification for which backups
to delete.
properties:
backupName:
type:string
required:
- backupName
type:object
status:
description:DeleteBackupRequestStatus is the current status of a DeleteBackupRequest.
properties:
errors:
description:Errors contains any errors that were encountered during
the deletion process.
items:
type:string
nullable:true
type:array
phase:
description:Phase is the current state of the DeleteBackupRequest.
description:DownloadRequest is a request to download an artifact from backup
object storage, such as a backup log file.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:DownloadRequestSpec is the specification for a download request.
properties:
target:
description:Target is what to download (e.g. logs for a backup).
properties:
kind:
description:Kind is the type of file to download.
enum:
- BackupLog
- BackupContents
- BackupVolumeSnapshots
- BackupResourceList
- RestoreLog
- RestoreResults
type:string
name:
description:Name is the name of the kubernetes resource with which
the file is associated.
type:string
required:
- kind
- name
type:object
required:
- target
type:object
status:
description:DownloadRequestStatus is the current status of a DownloadRequest.
properties:
downloadURL:
description:DownloadURL contains the pre-signed URL for the target
file.
type:string
expiration:
description:Expiration is when this DownloadRequest expires and can
be deleted by the system.
format:date-time
nullable:true
type:string
phase:
description:Phase is the current state of the DownloadRequest.
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:PodVolumeBackupSpec is the specification for a PodVolumeBackup.
properties:
backupStorageLocation:
description:BackupStorageLocation is the name of the backup storage
location where the restic repository is stored.
type:string
node:
description:Node is the name of the node that the Pod is running on.
type:string
pod:
description:Pod is a reference to the pod containing the volume to
be backed up.
properties:
apiVersion:
description:API version of the referent.
type:string
fieldPath:
description:'If referring to a piece of an object instead of an
entire object, this string should contain a valid JSON/Go field
access statement, such as desiredState.manifest.containers[2].
For example, if the object reference is to a container within
a pod, this would take on a value like: "spec.containers{name}"
(where "name" refers to the name of the container that triggered
the event) or if no container name is specified "spec.containers[2]"
(container with index 2 in this pod). This syntax is chosen only
to have some well-defined way of referencing a part of an object.
TODO: this design is not final and this field is subject to change
in the future.'
type:string
kind:
description: 'Kind of the referent. More info:https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
name:
description: 'Name of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
type:string
namespace:
description: 'Namespace of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
type:string
resourceVersion:
description:'Specific resourceVersion to which this reference is
made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
type:string
uid:
description: 'UID of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
type:string
type:object
repoIdentifier:
description:RepoIdentifier is the restic repository identifier.
type:string
tags:
additionalProperties:
type:string
description:Tags are a map of key-value pairs that should be applied
to the volume backup as tags.
type:object
volume:
description:Volume is the name of the volume within the Pod to be backed
up.
type:string
required:
- backupStorageLocation
- node
- pod
- repoIdentifier
- volume
type:object
status:
description:PodVolumeBackupStatus is the current status of a PodVolumeBackup.
properties:
completionTimestamp:
description:CompletionTimestamp records the time a backup was completed.
Completion time is recorded even on failed backups. Completion time
is recorded before uploading the backup object. The server's time
is used for CompletionTimestamps
format:date-time
nullable:true
type:string
message:
description:Message is a message about the pod volume backup's status.
type:string
path:
description:Path is the full path within the controller pod being backed
up.
type:string
phase:
description:Phase is the current state of the PodVolumeBackup.
enum:
- New
- InProgress
- Completed
- Failed
type:string
progress:
description:Progress holds the total number of bytes of the volume
and the current number of backed up bytes. This can be used to display
progress information about the backup operation.
properties:
bytesDone:
format:int64
type:integer
totalBytes:
format:int64
type:integer
type:object
snapshotID:
description:SnapshotID is the identifier for the snapshot of the pod
volume.
type:string
startTimestamp:
description:StartTimestamp records the time a backup was started. Separate
from CreationTimestamp, since that value changes on restores. The
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:PodVolumeRestoreSpec is the specification for a PodVolumeRestore.
properties:
backupStorageLocation:
description:BackupStorageLocation is the name of the backup storage
location where the restic repository is stored.
type:string
pod:
description:Pod is a reference to the pod containing the volume to
be restored.
properties:
apiVersion:
description:API version of the referent.
type:string
fieldPath:
description:'If referring to a piece of an object instead of an
entire object, this string should contain a valid JSON/Go field
access statement, such as desiredState.manifest.containers[2].
For example, if the object reference is to a container within
a pod, this would take on a value like: "spec.containers{name}"
(where "name" refers to the name of the container that triggered
the event) or if no container name is specified "spec.containers[2]"
(container with index 2 in this pod). This syntax is chosen only
to have some well-defined way of referencing a part of an object.
TODO: this design is not final and this field is subject to change
in the future.'
type:string
kind:
description: 'Kind of the referent. More info:https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
name:
description: 'Name of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
type:string
namespace:
description: 'Namespace of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
type:string
resourceVersion:
description:'Specific resourceVersion to which this reference is
made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
type:string
uid:
description: 'UID of the referent. More info:https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
type:string
type:object
repoIdentifier:
description:RepoIdentifier is the restic repository identifier.
type:string
snapshotID:
description:SnapshotID is the ID of the volume snapshot to be restored.
type:string
volume:
description:Volume is the name of the volume within the Pod to be restored.
type:string
required:
- backupStorageLocation
- pod
- repoIdentifier
- snapshotID
- volume
type:object
status:
description:PodVolumeRestoreStatus is the current status of a PodVolumeRestore.
properties:
completionTimestamp:
description:CompletionTimestamp records the time a restore was completed.
Completion time is recorded even on failed restores. The server's
time is used for CompletionTimestamps
format:date-time
nullable:true
type:string
message:
description:Message is a message about the pod volume restore's status.
type:string
phase:
description:Phase is the current state of the PodVolumeRestore.
enum:
- New
- InProgress
- Completed
- Failed
type:string
progress:
description:Progress holds the total number of bytes of the snapshot
and the current number of restored bytes. This can be used to display
progress information about the restore operation.
properties:
bytesDone:
format:int64
type:integer
totalBytes:
format:int64
type:integer
type:object
startTimestamp:
description:StartTimestamp records the time a restore was started.
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:ResticRepositorySpec is the specification for a ResticRepository.
properties:
backupStorageLocation:
description:BackupStorageLocation is the name of the BackupStorageLocation
that should contain this repository.
type:string
maintenanceFrequency:
description:MaintenanceFrequency is how often maintenance should be
run.
type:string
resticIdentifier:
description:ResticIdentifier is the full restic-compatible string for
identifying this repository.
type:string
volumeNamespace:
description:VolumeNamespace is the namespace this restic repository
contains pod volume backups for.
type:string
required:
- backupStorageLocation
- maintenanceFrequency
- resticIdentifier
- volumeNamespace
type:object
status:
description:ResticRepositoryStatus is the current status of a ResticRepository.
properties:
lastMaintenanceTime:
description:LastMaintenanceTime is the last time maintenance was run.
format:date-time
nullable:true
type:string
message:
description:Message is a message about the current status of the ResticRepository.
type:string
phase:
description:Phase is the current state of the ResticRepository.
description:Schedule is a Velero resource that represents a pre-scheduled or
periodic Backup that should be run.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:ScheduleSpec defines the specification for a Velero schedule
properties:
schedule:
description:Schedule is a Cron expression defining when to run the
Backup.
type:string
template:
description:Template is the definition of the Backup to be run on the
provided schedule
properties:
defaultVolumesToRestic:
description:DefaultVolumesToRestic specifies whether restic should
be used to take a backup of all pod volumes by default.
type:boolean
excludedNamespaces:
description:ExcludedNamespaces contains a list of namespaces that
are not included in the backup.
items:
type:string
nullable:true
type:array
excludedResources:
description:ExcludedResources is a slice of resource names that
are not included in the backup.
items:
type:string
nullable:true
type:array
hooks:
description:Hooks represent custom behaviors that should be executed
at different phases of the backup.
properties:
resources:
description:Resources are hooks that should be executed when
backing up individual instances of a resource.
items:
description:BackupResourceHookSpec defines one or more BackupResourceHooks
that should be executed based on the rules defined for namespaces,
resources, and label selector.
properties:
excludedNamespaces:
description:ExcludedNamespaces specifies the namespaces
to which this hook spec does not apply.
items:
type:string
nullable:true
type:array
excludedResources:
description:ExcludedResources specifies the resources
to which this hook spec does not apply.
items:
type:string
nullable:true
type:array
includedNamespaces:
description:IncludedNamespaces specifies the namespaces
to which this hook spec applies. If empty, it applies
to all namespaces.
items:
type:string
nullable:true
type:array
includedResources:
description:IncludedResources specifies the resources
to which this hook spec applies. If empty, it applies
to all resources.
items:
type:string
nullable:true
type:array
labelSelector:
description:LabelSelector, if specified, filters the
resources to which this hook spec applies.
nullable:true
properties:
matchExpressions:
description:matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description:A label selector requirement is a selector
that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description:key is the label key that the selector
applies to.
type:string
operator:
description:operator represents a key's relationship
to a set of values. Valid operators are In,
NotIn, Exists and DoesNotExist.
type:string
values:
description:values is an array of string values.
If the operator is In or NotIn, the values
array must be non-empty. If the operator is
Exists or DoesNotExist, the values array must
be empty. This array is replaced during a
strategic merge patch.
items:
type:string
type:array
required:
- key
- operator
type:object
type:array
matchLabels:
additionalProperties:
type:string
description:matchLabels is a map of {key,value} pairs.
A single {key,value} in the matchLabels map is equivalent
to an element of matchExpressions, whose key field
is "key", the operator is "In", and the values array
contains only "value". The requirements are ANDed.
type:object
type:object
name:
description:Name is the name of this hook.
type:string
post:
description:PostHooks is a list of BackupResourceHooks
to execute after storing the item in the backup. These
are executed after all "additional items" from item
actions are processed.
items:
description:BackupResourceHook defines a hook for a
resource.
properties:
exec:
description:Exec defines an exec hook.
properties:
command:
description:Command is the command and arguments
to execute.
items:
type:string
minItems:1
type:array
container:
description:Container is the container in the
pod where the command should be executed.
If not specified, the pod's first container
is used.
type:string
onError:
description:OnError specifies how Velero should
behave if it encounters an error executing
this hook.
enum:
- Continue
- Fail
type:string
timeout:
description:Timeout defines the maximum amount
of time Velero should wait for the hook to
complete before considering the execution
a failure.
type:string
required:
- command
type:object
required:
- exec
type:object
type:array
pre:
description:PreHooks is a list of BackupResourceHooks
to execute prior to storing the item in the backup.
These are executed before any "additional items" from
item actions are processed.
items:
description:BackupResourceHook defines a hook for a
description:ServerStatusRequest is a request to access current status information
about the Velero server.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:ServerStatusRequestSpec is the specification for a ServerStatusRequest.
type:object
status:
description:ServerStatusRequestStatus is the current status of a ServerStatusRequest.
properties:
phase:
description:Phase is the current lifecycle phase of the ServerStatusRequest.
enum:
- New
- Processed
type:string
plugins:
description:Plugins list information about the plugins running on the
Velero server
items:
description:PluginInfo contains attributes of a Velero plugin
properties:
kind:
type:string
name:
type:string
required:
- kind
- name
type:object
nullable:true
type:array
processedTimestamp:
description:ProcessedTimestamp is when the ServerStatusRequest was
processed by the ServerStatusRequestController.
format:date-time
nullable:true
type:string
serverVersion:
description:ServerVersion is the Velero server version.
description:VolumeSnapshotLocation is a location where Velero stores volume
snapshots.
properties:
apiVersion:
description:'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type:string
kind:
description:'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type:string
metadata:
type:object
spec:
description:VolumeSnapshotLocationSpec defines the specification for a
Velero VolumeSnapshotLocation.
properties:
config:
additionalProperties:
type:string
description:Config is for provider-specific configuration fields.
type:object
provider:
description:Provider is the provider of the volume storage.
type:string
required:
- provider
type:object
status:
description:VolumeSnapshotLocationStatus describes the current status of
a Velero VolumeSnapshotLocation.
properties:
phase:
description:VolumeSnapshotLocationPhase is the lifecycle phase of a
# Delete Backup and Restic Repo Resources when BSL is Deleted
## Abstract
Issue #2082 requested that with the command `velero backup-location delete <bsl name>` (implemented in Velero 1.6 with #3073), the following will be deleted:
- associated Velero backups (to be clear, these are custom Kubernetes resources called "backups" that are stored in the API server)
- associated Restic repositories (custom Kubernetes resources called "resticrepositories")
This design doc explains how the request will be implemented.
## Background
When a BSL resource is deleted from its Velero namespace, the associated custom Kubernetes resources, backups and Restic repositories, can no longer be used.
It makes sense to clean those resources up when a BSL is deleted.
## Goals
Update the `velero backup-location delete <bsl name>` command to delete associated backup and Restic repository resources in the same Velero namespace.
## Non Goals
[It was suggested](https://github.com/vmware-tanzu/velero/issues/2082#issuecomment-827951311) to fix bug #2697 alongside this issue.
However, I think that should be fixed separately because although it is similar (restore objects are not being deleted), it is also quite different.
One is adding a command feature update (this issue) and the other is a bug fix and each affect different parts of the code base.
## High-Level Design
Update the `velero backup-location delete <bsl name>` command to do the following:
- find in the same Velero namespace from which the BSL was deleted the associated backup resources and Restic repositories, called "backups.velero.io" and "resticrepositories.velero.io" respectively
- delete the resources found
The above logic will be added to [where BSLs are deleted](https://github.com/vmware-tanzu/velero/blob/main/pkg/cmd/cli/backuplocation/delete.go).
## Alternative Considered
I had considered deleting the backup files (the ones in json format and tarballs) in the BSL itself.
However, a standard use case is to back up a cluster and then restore into a new cluster.
Deleting the backup storage location in either location is not expected to remove all of the backups in the backup storage location and should not be done.
Currently, the [VolumePolicies feature](https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/handle-backup-of-volumes-by-resources-filters.md) which can be used to filter/handle volumes during backup only supports the skip action on matching conditions. Users need more actions to be supported.
## Background
The `VolumePolicies` feature was introduced in Velero 1.11 as a flexible way to handle volumes. The main agenda of
introducing the VolumePolicies feature was to improve the overall user experience when performing backup operations
for volume resources, the feature enables users to group volumes according the `conditions` (criteria) specified and
also lets you specify the `action` that velero needs to take for these grouped volumes during the backup operation.
The limitation being that currently `VolumePolicies` only supports `skip` as an action, We want to extend the `action`
functionality to support more usable options like `fs-backup` (File system backup) and `snapshot` (VolumeSnapshots).
## Goals
- Extending the VolumePolicies to support more actions like `fs-backup` (File system backup) and `snapshot` (VolumeSnapshots).
- Improve user experience when backing up Volumes via Velero
## Non-Goals
- No changes to existing approaches to opt-in/opt-out annotations for volumes
- No changes to existing `VolumePolicies` functionalities
- No additions or implementations to support more granular actions like `snapshot-csi` and `snapshot-datamover`. These actions can be implemented as a future enhancement
## Use-cases/Scenarios
**Use-case 1:**
- A user wants to use `snapshot` (volumesnapshots) backup option for all the csi supported volumes and `fs-backup` for the rest of the volumes.
- Currently, velero supports this use-case but the user experience is not that great.
- The user will have to individually annotate the volume mounting pod with the annotation "backup.velero.io/backup-volumes" for `fs-backup`
- This becomes cumbersome at scale.
- Using `VolumePolicies`, the user can just specify 2 simple `VolumePolicies` like for csi supported volumes as `snapshot` action and rest can be backed up`fs-backup` action:
```yaml
version:v1
volumePolicies:
- conditions:
storageClass:
- gp2
action:
type:snapshot
- conditions:{}
action:
type:fs-backup
```
**Use-case 2:**
- A user wants to use `fs-backup` for nfs volumes pertaining to a particular server
- In such a scenario the user can just specify a `VolumePolicy` like:
```yaml
version:v1
volumePolicies:
- conditions:
nfs:
server:192.168.200.90
action:
type:fs-backup
```
## High-Level Design
- When the VolumePolicy action is set as `fs-backup` the backup workflow modifications would be:
- We call [backupItem() -> backupItemInternal()](https://github.com/vmware-tanzu/velero/blob/main/pkg/backup/item_backupper.go#L95) on all the items that are to be backed up
- Here when we encounter [Pod as an item ](https://github.com/vmware-tanzu/velero/blob/main/pkg/backup/item_backupper.go#L195)
- We will have to modify the backup workflow to account for the `fs-backup` VolumePolicy action
- When the VolumePolicy action is set as `snapshot` the backup workflow modifications would be:
- Once again, We call [backupItem() -> backupItemInternal()](https://github.com/vmware-tanzu/velero/blob/main/pkg/backup/item_backupper.go#L95) on all the items that are to be backed up
- Here when we encounter [Persistent Volume as an item](https://github.com/vmware-tanzu/velero/blob/d4128542590470b204a642ee43311921c11db880/pkg/backup/item_backupper.go#L253)
- And we call the [takePVSnapshot func](https://github.com/vmware-tanzu/velero/blob/d4128542590470b204a642ee43311921c11db880/pkg/backup/item_backupper.go#L508)
- We need to modify the takePVSnapshot function to account for the `snapshot` VolumePolicy action.
- In case of csi snapshots for PVC objects, these snapshot actions are taken by the velero-plugin-for-csi, we need to modify the [executeActions()](https://github.com/vmware-tanzu/velero/blob/512fe0dabdcb3bbf1ca68a9089056ae549663bcf/pkg/backup/item_backupper.go#L232) function to account for the `snapshot` VolumePolicy action.
**Note:**`Snapshot` action can either be a native snapshot or a csi snapshot, as is the case with the current flow where velero itself makes the decision based on the backup CR.
## Detailed Design
- Update VolumePolicy action type validation to account for `fs-backup` and `snapshot` as valid VolumePolicy actions.
- Modifications needed for `fs-backup` action:
- Now based on the specification of volume policy on backup request we will decide whether to go via legacy pod annotations approach or the newer volume policy based fs-backup action approach.
- If there is a presence of volume policy(fs-backup/snapshot) on the backup request that matches as an action for a volume we use the newer volume policy approach to get the list of the volumes for `fs-backup` action
- Else continue with the annotation based legacy approach workflow.
- Modifications needed for `snapshot` action:
- In the [takePVSnapshot function](https://github.com/vmware-tanzu/velero/blob/d4128542590470b204a642ee43311921c11db880/pkg/backup/item_backupper.go#L508) we will check the PV fits the volume policy criteria and see if the associated action is `snapshot`
- If it is not snapshot then we skip the further workflow and avoid taking the snapshot of the PV
- Similarly, For csi snapshot of PVC object, we need to do similar changes in [executeAction() function](https://github.com/vmware-tanzu/velero/blob/512fe0dabdcb3bbf1ca68a9089056ae549663bcf/pkg/backup/item_backupper.go#L348). we will check the PVC fits the volume policy criteria and see if the associated action is `snapshot` via csi plugin
- If it is not snapshot then we skip the csi BIA execute action and avoid taking the snapshot of the PVC by not invoking the csi plugin action for the PVC
**Note:**
- When we are using the `VolumePolicy` approach for backing up the volumes then the volume policy criteria and action need to be specific and explicit, there is no default behavior, if a volume matches `fs-backup` action then `fs-backup` method will be used for that volume and similarly if the volume matches the criteria for `snapshot` action then the snapshot workflow will be used for the volume backup.
- Another thing to note is the workflow proposed in this design uses the legacy `opt-in/opt-out` approach as a fallback option. For instance, the user specifies a VolumePolicy but for a particular volume included in the backup there are no actions(fs-backup/snapshot) matching in the volume policy for that volume, in such a scenario the legacy approach will be used for backing up the particular volume.
- The relation between the `VolumePolicy` and the backup's legacy parameter `SnapshotVolumes`:
- The `VolumePolicy`'s `snapshot` action matching for volume has higher priority. When there is a `snapshot` action matching for the selected volume, it will be backed by the snapshot way, no matter of the `backup.Spec.SnapshotVolumes` setting.
- If there is no `snapshot` action matching the selected volume in the `VolumePolicy`, then the volume will be backed up by `snapshot` way, if the `backup.Spec.SnapshotVolumes` is not set to false.
- The relation between the `VolumePolicy` and the backup's legacy filesystem `opt-in/opt-out` approach:
- The `VolumePolicy`'s `fs-backup` action matching for volume has higher priority. When there is a `fs-backup` action matching for the selected volume, it will be backed by the fs-backup way, no matter of the `backup.Spec.DefaultVolumesToFsBackup` setting and the pod's `opt-in/opt-out` annotation setting.
- If there is no `fs-backup` action matching the selected volume in the `VolumePolicy`, then the volume will be backed up by the legacy `opt-in/opt-out` way.
## Implementation
- The implementation should be included in velero 1.14
- We will introduce a `VolumeHelper` interface. It will consist of two methods:
- The `VolumeHelperImpl` struct will implement the `VolumeHelper` interface and will consist of the functions that we will use through the backup workflow to accommodate volume policies for PVs and PVCs.
```go
typevolumeHelperImplstruct{
volumePolicy*resourcepolicies.Policies
snapshotVolumes*bool
loggerlogrus.FieldLogger
clientcrclient.Client
defaultVolumesToFSBackupbool
backupExcludePVCbool
}
```
- We will create an instance of the structure `volumeHelperImpl` in `item_backupper.go`
- Making sure that `snapshot` action is skipped for PVs that do not fit the volume policy criteria, for this we will use the `vh.ShouldPerformSnapshot` from the `VolumeHelperImpl(vh)` receiver.
log.Info(fmt.Sprintf("skipping volume snapshot for PV %s as it does not fit the volume policy criteria specified by the user for snapshot action",pv.Name))
ib.trackSkippedPV(obj,kuberesource.PersistentVolumes,volumeSnapshotApproach,"does not satisfy the criteria for volume policy based snapshot action",log)
returnnil
}
```
#### Snapshot (PVC)
- Making sure that `snapshot` action is skipped for PVCs that do not fit the volume policy criteria, for this we will again use the `vh.ShouldPerformSnapshot` from the `VolumeHelperImpl(vh)` receiver.
- We will pass the `VolumeHelperImpl(vh)` instance in `executeActions` method so that it is available to use.
```go
```
- The above function will be used as follows in the `executeActions` function of backup workflow.
- Considering the vSphere plugin doesn't support the VolumePolicy yet, don't use the VolumePolicy for vSphere plugin by now.
log.Info(fmt.Sprintf("skipping csi volume snapshot for PVC %s as it does not fit the volume policy criteria specified by the user for snapshot action",namespace+"/"+name))
ib.trackSkippedPV(obj,kuberesource.PersistentVolumeClaims,volumeSnapshotApproach,"does not satisfy the criteria for volume policy based snapshot action",log)
continue
}
}
}
```
## Future Implementation
It makes sense to add more specific actions in the future, once we deprecate the legacy opt-in/opt-out approach to keep things simple. Another point of note is, csi related action can be
easier to implement once we decide to merge the csi plugin into main velero code flow.
In the future, we envision the following actions that can be implemented:
-`snapshot-native`: only use volume snapshotter (native cloud provider snapshots), do nothing if not present/not compatible
-`snapshot-csi`: only use csi-plugin, don't use volume snapshotter(native cloud provider snapshots), don't use datamover even if snapshotMoveData is true
-`snapshot-datamover`: only use csi with datamover, don't use volume snapshotter (native cloud provider snapshots), use datamover even if snapshotMoveData is false
**Note:** The above actions are just suggestions for future scope, we may not use/implement them as is. We could definitely merge these suggested actions as `Snapshot` actions and use volume policy parameters and criteria to segregate them instead of making the user explicitly supply the action names to such granular levels.
## Related to Design
[Handle backup of volumes by resources filters](https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/handle-backup-of-volumes-by-resources-filters.md)
## Alternatives Considered
Same as the earlier design as this is an extension of the original VolumePolicies design
This document proposes a solution that allows user to specify a backup order for resources of specific resource type.
## Background
During backup process, user may need to back up resources of specific type in some specific order to ensure the resources were backup properly because these resources are related and ordering might be required to preserve the consistency for the apps to recover itself <EFBFBD>from the backup image
During backup process, user may need to back up resources of specific type in some specific order to ensure the resources were backup properly because these resources are related and ordering might be required to preserve the consistency for the apps to recover itself from the backup image
(Ex: primary-secondary database pods in a cluster).
## Goals
- Enable user to specify an order of backup resources belong to specific resource type
- Enable user to specify an order of backup resources belong to specific resource type
## Alternatives Considered
- Use a plugin to backup an resources and all the sub resources. For example use a plugin for StatefulSet and backup pods belong to the StatefulSet in specific order. This plugin solution is not generic and requires plugin for each resource type.
## High-Level Design
User will specify a map of resource type to list resource names (separate by semicolons). Each name will be in the format "namespaceName/resourceName" to enable ordering accross namespaces. Based on this map, the resources of each resource type will be sorted by the order specified in the list of resources. If a resource instance belong to that specific type but its name is not in the order list, then it will be put behind other resources that are in the list.
User will specify a map of resource type to list resource names (separate by semicolons). Each name will be in the format "namespaceName/resourceName" to enable ordering across namespaces. Based on this map, the resources of each resource type will be sorted by the order specified in the list of resources. If a resource instance belong to that specific type but its name is not in the order list, then it will be put behind other resources that are in the list.
- In the CLI, the design proposes to use commas to separate items of a resource type and semicolon to separate key-value pairs. This follows the convention of using commas to separate items in a list (For example: --include-namespaces ns1,ns2). However, the syntax for map in labels and annotations use commas to seperate key-value pairs. So it introduces some inconsistency.
- In the CLI, the design proposes to use commas to separate items of a resource type and semicolon to separate key-value pairs. This follows the convention of using commas to separate items in a list (For example: --include-namespaces ns1,ns2). However, the syntax for map in labels and annotations use commas to separate key-value pairs. So it introduces some inconsistency.
- For pods that managed by Deployment or DaemonSet, this design may not work because the pods' name is randomly generated and if pods are restarted, they would have different names so the Backup operation may not consider the restarted pods in the sorting algorithm. This problem will be addressed when we enhance the design to use regular expression to specify the OrderResources instead of exact match.
This design includes the changes to the BackupItemAction (BIA) api design as required by the [Item Action Progress Monitoring](general-progress-monitoring.md) feature.
The BIA v2 interface will have two new methods, and the Execute() return signature will be modified.
If there are any additional BIA API changes that are needed in the same Velero release cycle as this change, those can be added here as well.
## Background
This API change is needed to facilitate long-running plugin actions that may not be complete when the Execute() method returns.
It is an optional feature, so plugins which don't need this feature can simply return an empty operation ID and the new methods can be no-ops.
This will allow long-running plugin actions to continue in the background while Velero moves on to the next plugin, the next item, etc.
## Goals
- Allow for BIA Execute() to optionally initiate a long-running operation and report on operation status.
## Non Goals
- Allowing velero control over when the long-running operation begins.
## High-Level Design
As per the [Plugin Versioning](plugin-versioning.md) design, a new BIAv2 plugin `.proto` file will be created to define the GRPC interface.
v2 go files will also be created in `plugin/clientmgmt/backupitemaction` and `plugin/framework/backupitemaction`, and a new PluginKind will be created.
The velero Backup process will be modified to reference v2 plugins instead of v1 plugins.
An adapter will be created so that any existing BIA v1 plugin can be executed as a v2 plugin when executing a backup.
## Detailed Design
### proto changes (compiled into golang by protoc)
The v2 BackupItemAction.proto will be like the current v1 version with the following changes:
To support these new rpc methods, we define new request/response message types:
```
message BackupItemActionProgressRequest {
string plugin = 1;
string operationID = 2;
bytes backup = 3;
}
message BackupItemActionProgressResponse {
generated.OperationProgress progress = 1;
}
message BackupItemActionCancelRequest {
string plugin = 1;
string operationID = 2;
bytes backup = 3;
}
```
One new shared message type will be added, as this will also be needed for v2 RestoreItemAction and VolmeSnapshotter:
```
message OperationProgress {
bool completed = 1;
string err = 2;
int64 nCompleted = 3;
int64 nTotal = 4;
string operationUnits = 5;
string description = 6;
google.protobuf.Timestamp started = 7;
google.protobuf.Timestamp updated = 8;
}
```
In addition to the two new rpc methods added to the BackupItemAction interface, there is also a new `Name()` method. This one is only actually used internally by Velero to get the name that the plugin was registered with, but it still must be defined in a plugin which implements BackupItemActionV2 in order to implement the interface. It doesn't really matter what it returns, though, as this particular method is not delegated to the plugin via RPC calls. The new (and modified) interface methods for `BackupItemAction` are as follows:
A new PluginKind, `BackupItemActionV2`, will be created, and the backup process will be modified to use this plugin kind.
See [Plugin Versioning](plugin-versioning.md) for more details on implementation plans, including v1 adapters, etc.
## Compatibility
The included v1 adapter will allow any existing BackupItemAction plugin to work as expected, with an empty operation ID returned from Execute() and no-op Progress() and Cancel() methods.
## Implementation
This will be implemented during the Velero 1.11 development cycle.
# Proposal to add resource filters for backup can distinguish whether resource is cluster-scoped or namespace-scoped.
- [Proposal to add resource filters for backup can distinguish whether resource is cluster-scoped or namespace-scoped.](#proposal-to-add-resource-filters-for-backup-can-distinguish-whether-resource-is-cluster-scoped-or-namespace-scoped)
- [Abstract](#abstract)
- [Background](#background)
- [Goals](#goals)
- [Non Goals](#non-goals)
- [High-Level Design](#high-level-design)
- [Parameters Rules](#parameters-rules)
- [Using scenarios:](#using-scenarios)
- [no namespace-scoped resources + some cluster-scoped resources](#no-namespace-scoped-resources--some-cluster-scoped-resources)
- [no namespace-scoped resources + all cluster-scoped resources](#no-namespace-scoped-resources--all-cluster-scoped-resources)
- [some namespace-scoped resources + no cluster-scoped resources](#some-namespace-scoped-resources--no-cluster-scoped-resources)
- [scenario 1](#scenario-1)
- [scenario 2](#scenario-2)
- [scenario 3](#scenario-3)
- [scenario 4](#scenario-4)
- [some namespace-scoped resources + only related cluster-scoped resources](#some-namespace-scoped-resources--only-related-cluster-scoped-resources)
- [scenario 1](#scenario-1-1)
- [scenario 2](#scenario-2-1)
- [scenario 3](#scenario-3-1)
- [some namespace-scoped resources + some additional cluster-scoped resources](#some-namespace-scoped-resources--some-additional-cluster-scoped-resources)
- [scenario 1](#scenario-1-2)
- [scenario 2](#scenario-2-2)
- [scenario 3](#scenario-3-2)
- [scenario 4](#scenario-4-1)
- [some namespace-scoped resources + all cluster-scoped resources](#some-namespace-scoped-resources--all-cluster-scoped-resources)
- [scenario 1](#scenario-1-3)
- [scenario 2](#scenario-2-3)
- [scenario 3](#scenario-3-3)
- [all namespace-scoped resources + no cluster-scoped resources](#all-namespace-scoped-resources--no-cluster-scoped-resources)
- [all namespace-scoped resources + some additional cluster-scoped resources](#all-namespace-scoped-resources--some-additional-cluster-scoped-resources)
- [all namespace-scoped resources + all cluster-scoped resources](#all-namespace-scoped-resources--all-cluster-scoped-resources)
The current filter (IncludedResources/ExcludedResources + IncludeClusterResources flag) is not enough for some special cases, e.g. all namespace-scoped resources + some kind of cluster-scoped resource and all namespace-scoped resources + cluster-scoped resource excludes.
Propose to add a new group of resource filtering parameters, which can distinguish cluster-scoped and namespace-scoped resources.
## Background
There are two sets of resource filters for Velero: `IncludedNamespaces/ExcludedNamespaces` and `IncludedResources/ExcludedResources`.
`IncludedResources` means only including the resource types specified in the parameter. Both cluster-scoped and namespace-scoped resources are handled in this parameter by now.
The k8s resources are separated into cluster-scoped and namespace-scoped.
As a result, it's hard to include all resources in one group and only including specified resource in the other group.
## Goals
- Make Velero can support more complicated namespace-scoped and cluster-scoped resources filtering scenarios in backup.
## Non Goals
- Enrich the resource filtering rules, for example, advanced PV filtering and filtering by resource names.
## High-Level Design
Four new parameters are added into command `velero backup create`: `--include-cluster-scoped-resources`, `--exclude-cluster-scoped-resources`, `--include-namespace-scoped-resources` and `--exclude-namespace-scoped-resources`.
`--include-cluster-scoped-resources` and `--exclude-cluster-scoped-resources` are used to filter cluster-scoped resources included or excluded in backup per resource type.
`--include-namespace-scoped-resources` and `--exclude-namespace-scoped-resources` are used to filter namespace-scoped resources included or excluded in backup per resource type.
Restore and other code pieces also use resource filtering will be handled in future releases.
### Parameters Rules
*`--include-cluster-scoped-resources`, `--include-namespace-scoped-resources`, `--exclude-cluster-scoped-resources` and `--exclude-namespace-scoped-resources` valid value include `*` and comma separated string. Each element of the CSV string should a k8s resource name. The format should be `resource.group`, such as `storageclasses.storage.k8s.io.`.
*`--include-cluster-scoped-resources`, `--include-namespace-scoped-resources`, `--exclude-cluster-scoped-resources` and `--exclude-namespace-scoped-resources` parameters are mutual exclusive with `--include-cluster-resources`, `--include-resources` and `--exclude-resources` parameters. If both sets of parameters are provisioned, validation failure should be returned.
*`--include-cluster-scoped-resources` and `--exclude-cluster-scoped-resources` should only contain cluster-scoped resource type names. If namespace-scoped resource type names are included, they are ignored.
* If there are conflicts between `--include-cluster-scoped-resources` and `--exclude-cluster-scoped-resources` specified resources type lists, `--exclude-cluster-scoped-resources` parameter has higher priority.
*`--include-namespace-scoped-resources` and `--exclude-namespace-scoped-resources` should only contain namespace-scoped resource type names. If cluster-scoped resource type names are included, they are ignored.
* If there are conflicts between `--include-namespace-scoped-resources` and `--exclude-namespace-scoped-resources` specified resources type lists, `--exclude-namespace-scoped-resources` parameter has higher priority.
* If `--include-namespace-scoped-resources` is not present, it means all namespace-scoped resources are included per resource type.
* If both `--include-cluster-scoped-resources` and `--exclude-cluster-scoped-resources` are not present, it means no additional cluster-scoped resource is included per resource type, just as the existing `--include-cluster-resources` parameter not setting value. Cluster-scoped resources are related to the namespace-scoped resources, which means those are returned in the namespace-scoped resources' BackupItemAction's result AdditionalItems array, are still included in backup by default. Taking backing up PVC scenario as an example, PVC is namespace-scoped, PV is cluster-scoped. PVC's BIA will include PVC related PV into backup too.
### Using scenarios:
Please notice, if the scenario give the example of using old filtering parameters (`--include-cluster-resources`, `--include-resources` and `--exclude-resources`), that means the old parameters also work for this case. If old parameters example is not given, that means they don't work for this scenario, only new parameters (`--include-cluster-scoped-resources`, `--include-namespace-scoped-resources`, `--exclude-cluster-scoped-resources` and `--exclude-namespace-scoped-resources`) work.
#### no namespace-scoped resources + some cluster-scoped resources
The following command means backup no namespace-scoped resources and some cluster-scoped resources.
``` bash
velero backup create <backup-name>
--exclude-namespace-scoped-resources=*
--include-cluster-scoped-resources=storageclass
```
#### no namespace-scoped resources + all cluster-scoped resources
The following command means backup no namespace-scoped resources and all cluster-scoped resources.
``` bash
velero backup create <backup-name>
--exclude-namespace-scoped-resources=*
--include-cluster-scoped-resources=*
```
#### some namespace-scoped resources + no cluster-scoped resources
##### scenario 1
The following commands mean backup all resources in namespaces default and kube-system, and no cluster-scoped resources.
Example of new parameters:
``` bash
velero backup create <backup-name>
--include-namespaces=default,kube-system
--exclude-cluster-scoped-resources=*
```
Example of old parameters:
``` bash
velero backup create <backup-name>
--include-namespaces=default,kube-system
--include-cluster-resources=false
```
##### scenario 2
The following commands mean backup PVC, Deployment, Service, Endpoint, Pod and ReplicaSet resources in all namespaces, and no cluster-scoped resources. Although PVC's related PV should be included, due to no cluster-scoped resources are included, so they are ruled out too.
The following commands mean backup PVC, Deployment, Service, Endpoint, Pod and ReplicaSet resources in namespace default and kube-system, and no cluster-scoped resources. Although PVC's related PV should be included, due to no cluster-scoped resources are included, so they are ruled out too.
This means backup all resources except Ingress type resources in all namespaces, and related cluster-scoped resources.
Example of new parameters:
``` bash
velero backup create <backup-name>
--exclude-namespace-scoped-resources=ingress
```
Example of old parameters:
``` bash
velero backup create <backup-name>
--exclude-resources=ingress
```
#### some namespace-scoped resources + some additional cluster-scoped resources
##### scenario 1
This means backup all resources in namespace in default, kube-system, and related cluster-scoped resources, plus all StorageClass resources.
``` bash
velero backup create <backup-name>
--include-namespaces=default,kube-system
--include-cluster-scoped-resources=storageclass
```
##### scenario 2
This means backup PVC, Deployment, Service, Endpoint, Pod and ReplicaSet resources in all namespaces, and related cluster-scoped resources, plus all StorageClass resources, and PVC related PV.
This means backup PVC, Deployment, Service, Endpoint, Pod and ReplicaSet resources in default and kube-system namespaces, and related cluster-scoped resources, plus all StorageClass resources, and PVC related PV.
This means backup PVC, Deployment, Service, Endpoint, Pod and ReplicaSet resources in default and kube-system namespaces, and related cluster-scoped resources, plus all cluster-scoped resources except StorageClass type resources.
#### all namespace-scoped resources + all cluster-scoped resources
The following commands have the same meaning: backup all namespace-scoped resources, and all cluster-scoped resources.
``` bash
velero backup create <backup-name>
--include-cluster-scoped-resources=*
```
``` bash
velero backup create <backup-name>
--include-cluster-resources=true
```
#### describe command change
In `velero backup describe` command, the four new parameters should be outputted too.
``` bash
velero backup describe <backup-name>
......
Namespaces:
Included: ns2
Excluded: <none>
Resources:
Included cluster-scoped: StorageClass,PersistentVolume
Excluded cluster-scoped: <none>
Included namespace-scoped: default
Excluded namespace-scoped: <none>
......
```
**Note:** `velero restore` command doesn't support those four new parameter in Velero v1.11, but `velero schedule` supports the four new parameters through backup specification.
## Detailed Design
With adding `IncludedNamespaceScopedResources`, `ExcludedNamespaceScopedResources`, `IncludedClusterScopedResources` and `ExcludedClusterScopedResources`, the `BackupSpec` looks like:
``` go
type BackupSpec struct {
......
// IncludedResources is a slice of resource names to include
// in the backup. If empty, all resources are included.
Proposal from Jibu Data [Issue 5120](https://github.com/vmware-tanzu/velero/issues/5120#issue-1304534563)
## Security Considerations
No security impact.
## Compatibility
The four new parameters cannot be mixed with existing resource filter parameters: `IncludedResources`, `ExcludedResources` and `IncludeClusterResources`.
If the new parameters and old parameters both appears in command line, or are specified in backup spec, the command line and the backup should fail.
## Implementation
This change should be included into Velero v1.11.
New parameters will coexist with `IncludedResources`, `ExcludedResources` and `IncludeClusterResources`.
Plan to deprecate `IncludedResources`, `ExcludedResources` and `IncludeClusterResources` in future releases, but also open to the community's feedback.
## Open Issues
`LabelSelector/OrLabelSelectors` apply to namespace-scoped resources.
It may be reasonable to make them also working on cluster-scoped resources.
An issue is created to trace this topic [resource label selector not work for cluster-scoped resources](https://github.com/vmware-tanzu/velero/issues/5787)
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
