Compare commits

..

57 Commits

Author SHA1 Message Date
Xun Jiang/Bruce Jiang
32499fc287 Merge pull request #8546 from blackpiglet/skip_deprecation_message
Some checks failed
Run the E2E test on kind / build (push) Failing after 4m1s
Run the E2E test on kind / run-e2e-test (1.23.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Main CI / Build (push) Failing after 20s
Skip the deprecation message for the dry-run install CLI JSON output.
2024-12-23 18:21:51 +08:00
Xun Jiang
b281ddbb9b Skip the deprecation message for the dry-run install CLI JSON output.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-12-23 17:26:36 +08:00
Wenkai Yin(尹文开)
be14d4f25c Merge pull request #8547 from ywk253100/241223_ut
Bug fix: increase the WaitGroup counter before start the goroutine
2024-12-23 17:25:41 +08:00
Wenkai Yin(尹文开)
0e54406352 Bug fix: increase the WaitGroup counter before start the goroutine
Bug fix: increase the WaitGroup counter before start the goroutine

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2024-12-23 17:04:12 +08:00
Tiger Kaovilai
845f4ee4f7 Merge pull request #8545 from Lyndon-Li/release-1.15
Pin the version of Golang and base image for v1.15.1
2024-12-23 14:31:19 +07:00
Lyndon-Li
6e9397d785 Pin the version of Golang and base image for v1.15.1
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-23 14:49:26 +08:00
Daniel Jiang
5ca1d184e1 Merge pull request #8544 from blackpiglet/migration_init_release_1.15
Some checks failed
Run the E2E test on kind / build (push) Failing after 4m22s
Run the E2E test on kind / run-e2e-test (1.23.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Main CI / Build (push) Failing after 33s
Modify the Init logic to fix the migration case error.
2024-12-23 12:52:50 +08:00
Xun Jiang
68e381d288 Modify the Init logic to fix the migration case error.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-12-23 10:54:47 +08:00
Tiger Kaovilai
b661ed7430 Merge pull request #8540 from Lyndon-Li/release-1.15
Some checks failed
Run the E2E test on kind / build (push) Failing after 4m12s
Run the E2E test on kind / run-e2e-test (1.23.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Main CI / Build (push) Failing after 24s
[1.15] hide restic deprecation warning for install with crd-only
2024-12-20 15:25:05 +07:00
Lyndon-Li
11a2be69e1 hide restic deprecation warning for install with crd-only
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-20 16:10:59 +08:00
Wenkai Yin(尹文开)
4376a89bc9 Merge pull request #8533 from blackpiglet/refactor_migration_e2e_release_1.15
Some checks failed
Run the E2E test on kind / build (push) Failing after 5m5s
Run the E2E test on kind / run-e2e-test (1.23.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Main CI / Build (push) Failing after 42s
[cherry-pick] Refactor the migration cases.
2024-12-19 17:04:15 +08:00
Xun Jiang
6f7704a090 Refactor the migration cases.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-12-19 16:40:15 +08:00
Wenkai Yin(尹文开)
2b8b6dcd4b Merge pull request #8531 from blackpiglet/8323_fix_release_1.15
Modify the schedule cases.
2024-12-19 16:38:45 +08:00
Xun Jiang
57f873b415 Modify the schedule cases.
* Modify the OrderResource case's verification code.
* Simplify the Periodical case.
* Simplify the InProgess case.
* Prettify the code.
* Replace math/rand with crypto/rand
* Replace PollUnitl with PollUntilContextTimeout

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-12-19 15:59:25 +08:00
Daniel Jiang
49a8e2413c Merge pull request #8530 from blackpiglet/fix_storageclass_release_1.15
[cherry-pick] Modify the StorageClass install and delete code.
2024-12-19 15:41:33 +08:00
Xun Jiang/Bruce Jiang
e78d082ede Merge branch 'release-1.15' into fix_storageclass_release_1.15 2024-12-19 15:00:31 +08:00
lyndon-li
a01e82df53 Merge pull request #8529 from Lyndon-Li/release-1.15
1.15.1 change-log
2024-12-19 14:53:23 +08:00
Xun Jiang
5ff921f1c4 Modify the StorageClass install and delete code.
* Only install and uninstall SC and VSC once for default cluster.
* Install and uninstall SC and VSC for standby cluster on migration case.
* Refactor the StorageClass and VolumeSnapshotClass YAMLs.
* Prettify the e2e_suite_test.go

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-12-19 14:39:53 +08:00
lyndon-li
e7e66f7d7c Merge branch 'release-1.15' into release-1.15 2024-12-19 14:31:10 +08:00
Daniel Jiang
005a7eeeb8 Merge pull request #8528 from blackpiglet/vks_support_release_1.15
[cherry-pick] VKS support release 1.15
2024-12-19 14:28:17 +08:00
Lyndon-Li
659197bf10 1.15.1 change-log
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-19 14:06:25 +08:00
Xun Jiang
1a15c72262 Modify upgrade and migration cases.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-12-19 13:54:49 +08:00
Xun Jiang
53a34757ac Modify other cases to support VKS environment.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-12-19 13:53:07 +08:00
Xun Jiang
cd61732f04 E2E supports VKS data mover environment.
* Add new flag HAS_VSPHERE_PLUGIN for E2E test.
* Modify the E2E README for the new parameter.
* Add the VolumeSnapshotClass for VKS.
* Modify the plugin install logic.
* Modify the cases to support data mover case in VKS.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-12-19 13:52:57 +08:00
Xun Jiang
3a627f0c81 Make change to support VKS environment.
FYI, the TKGm envrionment support is deprecated.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-12-19 13:52:22 +08:00
Tiger Kaovilai
579b1d18b7 Merge pull request #8526 from blackpiglet/bump_crypto_in_restic_release_1.15
Bump the golang.org/x/crypto for Restic in release-1.15.
2024-12-19 11:46:35 +07:00
Xun Jiang
f6c2aca65b Bump the golang.org/x/crypto for Restic to fix CVE-2024-45337.
Bump the golang.org/x/net for Velero and Restic to fix CVE-2024-45338.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-12-19 10:55:14 +08:00
Tiger Kaovilai
8e3389d274 Merge pull request #8519 from blackpiglet/bump_crypto_release_1.15
Bump golang.org/x/crypto to v0.31.0 to fix CVEs.
2024-12-17 20:02:15 +07:00
Xun Jiang
a10d78c948 Bump golang.org/x/crypto to v0.31.0 to fix CVEs.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-12-17 15:06:44 +08:00
Wenkai Yin(尹文开)
f96b1c240b Merge pull request #8517 from ywk253100/241217_hook
[cherry-pick]Fix backup post hook issue
2024-12-17 14:26:15 +08:00
Wenkai Yin(尹文开)
fe8c086299 Fix backup post hook issue
Fix backup post hook issue

Fixes #8159

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2024-12-17 13:05:46 +08:00
Daniel Jiang
f0001a5b9c Merge pull request #8511 from Lyndon-Li/release-1.15
Some checks failed
Run the E2E test on kind / build (push) Failing after 4m35s
Run the E2E test on kind / run-e2e-test (1.23.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Main CI / Build (push) Failing after 52s
[1.15] Data mover exposer diagnostic
2024-12-13 15:35:03 +08:00
Lyndon-Li
a217e33221 add diagnostic for data mover exposer
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-13 15:12:07 +08:00
Lyndon-Li
1e2ef374d6 add diagnostic for data mover exposer
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-13 15:09:17 +08:00
Lyndon-Li
8087c7f13a add diagnostic for data mover exposer
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-13 15:09:01 +08:00
Daniel Jiang
bda3ec1bc4 Merge pull request #8505 from reasonerjt/repo-aws-region-hint-1.15
Some checks failed
Run the E2E test on kind / build (push) Failing after 4m39s
Run the E2E test on kind / run-e2e-test (1.23.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Main CI / Build (push) Failing after 45s
[Cherry-pick-1.15] Set hinting region to use for GetBucketRegion() in pkg/repository/config/aws.go
2024-12-11 19:34:57 +08:00
Tiger Kaovilai
5ddb319073 Set hinting region to use for GetBucketRegion() in pkg/repository/config/aws.go
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2024-12-11 15:36:51 +08:00
lyndon-li
f499025d80 Merge pull request #8501 from Lyndon-Li/release-1.15
Some checks failed
Run the E2E test on kind / build (push) Failing after 4m32s
Run the E2E test on kind / run-e2e-test (1.23.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Main CI / Build (push) Failing after 31s
[1.15] Issue 8433: add third party labels to data mover pods when the same labels exist in node-agent pods
2024-12-10 14:26:21 +08:00
Lyndon-Li
3f31730003 check existence of the same label from node-agent
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-10 13:44:54 +08:00
Lyndon-Li
cce4d4815a issue 8433: add ask label to data mover pods
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-10 13:42:10 +08:00
lyndon-li
a72aff2a3b Merge pull request #8496 from Lyndon-Li/release-1.15
Some checks failed
Run the E2E test on kind / build (push) Failing after 5m13s
Run the E2E test on kind / run-e2e-test (1.23.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Main CI / Build (push) Failing after 1m5s
[1.15] Fix prepare timeout issue
2024-12-09 15:25:53 +08:00
Lyndon-Li
d78947a755 fix prepare timeout issue
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-09 15:01:33 +08:00
Daniel Jiang
7d2110ecd1 Merge pull request #8495 from reasonerjt/restore-help-1.15
[Cherry-pick-1.15] Add SecurityContext to restore-helper
2024-12-09 14:36:39 +08:00
Daniel Jiang
666a0ee233 Add SecurityContext to restore-helper
This commit adds SecurityContext that complies with "restricted" level
per Pod Security Standards to "restore-helper" initContainer.
It ensures the restore won't fail when the cluster enforces PSA.

Signed-off-by: Daniel Jiang <daniel.jiang@broadcom.com>
2024-12-09 13:50:49 +08:00
Tiger Kaovilai
347653061d Merge pull request #8399 from kaovilai/removeMinimumGoToolchainGo.mod
Some checks failed
Run the E2E test on kind / build (push) Failing after 12m43s
Run the E2E test on kind / run-e2e-test (1.23.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Main CI / Build (push) Failing after 12m46s
release-1.15: Reduce minimum go toolchain in go.mod.
2024-12-03 20:08:29 -05:00
Tiger Kaovilai
c3967c3365 Reduces go.mod minimum go to patch 1.22.0
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2024-11-28 03:42:48 -05:00
Tiger Kaovilai
dd1c6a6aab Merge pull request #8465 from blackpiglet/8440_fix
Some checks failed
Run the E2E test on kind / build (push) Failing after 13m1s
Run the E2E test on kind / run-e2e-test (1.23.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Main CI / Build (push) Failing after 11m13s
[release-1.15]Add nil check for updating DataUpload VolumeInfo in finalizing phase.
2024-11-27 12:21:53 -05:00
Xun Jiang
ecd206acce Add nil check for updating DataUpload VolumeInfo in finalizing phase.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-11-27 18:59:38 +08:00
lyndon-li
40b8551a18 Merge pull request #8404 from Lyndon-Li/release-1.15
Some checks failed
Run the E2E test on kind / build (push) Failing after 10m0s
Run the E2E test on kind / run-e2e-test (1.23.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.23.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.24.17, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.25.16, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.26.13, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.27.10, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.28.6, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, Basic && (ClusterResource || NodePort || StorageClass)) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceFiltering && !Restic) (push) Has been skipped
Run the E2E test on kind / run-e2e-test (1.29.1, ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources) (push) Has been skipped
Main CI / Build (push) Failing after 14m17s
[1.15] Issue 8391: check ErrCancelled from suffix
2024-11-13 13:42:03 +08:00
Lyndon-Li
0fc822b790 issue 8391: check ErrCancelled from suffix
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-11-13 13:18:30 +08:00
lyndon-li
13d8e7de47 Merge pull request #8402 from Lyndon-Li/release-1.15
[1.15] Issue 8394: move closeDataPath outside callbacks
2024-11-13 13:14:39 +08:00
Lyndon-Li
56cab0608f issue 8394: move closeDataPath outside callbacks
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-11-13 11:16:03 +08:00
Xun Jiang/Bruce Jiang
1d4f147597 Merge pull request #8349 from blackpiglet/fix_v1.15.0_migration_case_error
Fix v1.15.0 migration case error
2024-10-28 11:33:28 +08:00
Xun Jiang
6f79c54dfa Fix the KIBISHII_DIRECTORY parameter not working issue.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-10-25 22:25:22 +08:00
Xun Jiang
15ee6a4a58 Refactor the code to get the plugin images for migration cases.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-10-25 15:46:37 +08:00
Wenkai Yin(尹文开)
d2dec9de8b Merge pull request #8336 from Lyndon-Li/release-1.15
Pin the version of Golang and base image for v1.15.0
2024-10-23 14:06:55 +08:00
Lyndon-Li
7cd92a5aac Pin the version of Golang and base image for v1.15.0
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-10-23 13:36:33 +08:00
1103 changed files with 17766 additions and 100759 deletions

View File

@@ -16,7 +16,6 @@ reviewers:
- shubham-pampattiwar
- Lyndon-Li
- anshulahuja98
- kaovilai
tech-writer:
- sseago

View File

@@ -7,10 +7,6 @@ on:
pull_request_target:
types: [opened, reopened, ready_for_review]
permissions:
contents: read
pull-requests: write
jobs:
# Automatically assigns reviewers and owner
add-reviews:
@@ -20,3 +16,4 @@ jobs:
uses: kentaro-m/auto-assign-action@v2.0.0
with:
configuration-path: ".github/auto-assignees.yml"
repo-token: "${{ secrets.GITHUB_TOKEN }}"

View File

@@ -8,10 +8,6 @@ on:
pull_request_target:
types: [opened, reopened, synchronize, ready_for_review]
permissions:
contents: read
pull-requests: write
jobs:
# Automatically labels PRs based on file globs in the change.
triage:
@@ -19,4 +15,5 @@ jobs:
steps:
- uses: actions/labeler@v5
with:
repo-token: "${{ secrets.GITHUB_TOKEN }}"
configuration-path: .github/labeler.yml

View File

@@ -5,10 +5,6 @@ on:
pull_request_target:
types: [opened, ready_for_review, reopened]
permissions:
contents: read
pull-requests: write
jobs:
auto-request-review:
name: Auto Request Review
@@ -17,5 +13,5 @@ jobs:
- name: Request a PR review based on files types/paths, and/or groups the author belongs to
uses: necojackarc/auto-request-review@v0.13.0
with:
config: .github/auto-assignees.yml
token: ${{ secrets.GITHUB_TOKEN }}
config: .github/auto-assignees.yml

79
.github/workflows/crds-verify-kind.yaml vendored Normal file
View File

@@ -0,0 +1,79 @@
name: "Verify Velero CRDs across k8s versions"
on:
pull_request:
# Do not run when the change only includes these directories.
paths-ignore:
- "site/**"
- "design/**"
jobs:
# Build the Velero CLI once for all Kubernetes versions, and cache it so the fan-out workers can get it.
build-cli:
runs-on: ubuntu-latest
steps:
- name: Check out the code
uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version-file: 'go.mod'
# Look for a CLI that's made for this PR
- name: Fetch built CLI
id: cache
uses: actions/cache@v4
env:
cache-name: cache-velero-cli
with:
path: ./_output/bin/linux/amd64/velero
# The cache key a combination of the current PR number, and a SHA256 hash of the Velero binary
key: velero-${{ github.event.pull_request.number }}-${{ hashFiles('./_output/bin/linux/amd64/velero') }}
# This key controls the prefixes that we'll look at in the cache to restore from
restore-keys: |
velero-${{ github.event.pull_request.number }}-
# If no binaries were built for this PR, build it now.
- name: Build Velero CLI
if: steps.cache.outputs.cache-hit != 'true'
run: |
make local
# Check the common CLI against all Kubernetes versions
crd-check:
needs: build-cli
runs-on: ubuntu-latest
strategy:
matrix:
# Latest k8s versions. There's no series-based tag, nor is there a latest tag.
k8s:
- 1.23.17
- 1.24.17
- 1.25.16
- 1.26.13
- 1.27.10
- 1.28.6
- 1.29.1
# All steps run in parallel unless otherwise specified.
# See https://docs.github.com/en/actions/learn-github-actions/managing-complex-workflows#creating-dependent-jobs
steps:
- name: Fetch built CLI
id: cache
uses: actions/cache@v4
env:
cache-name: cache-velero-cli
with:
path: ./_output/bin/linux/amd64/velero
# The cache key a combination of the current PR number, and a SHA256 hash of the Velero binary
key: velero-${{ github.event.pull_request.number }}-${{ hashFiles('./_output/bin/linux/amd64/velero') }}
# This key controls the prefixes that we'll look at in the cache to restore from
restore-keys: |
velero-${{ github.event.pull_request.number }}-
- uses: engineerd/setup-kind@v0.5.0
with:
version: "v0.21.0"
image: "kindest/node:v${{ matrix.k8s }}"
- name: Install CRDs
run: |
kubectl cluster-info
kubectl get pods -n kube-system
kubectl version
echo "current-context:" $(kubectl config current-context)
echo "environment-kubeconfig:" ${KUBECONFIG}
./_output/bin/linux/amd64/velero install --crds-only --dry-run -oyaml | kubectl apply -f -

View File

@@ -6,28 +6,17 @@ on:
paths-ignore:
- "site/**"
- "design/**"
- "**/*.md"
jobs:
get-go-version:
uses: ./.github/workflows/get-go-version.yaml
with:
ref: ${{ github.event.pull_request.base.ref }}
# Build the Velero CLI and image once for all Kubernetes versions, and cache it so the fan-out workers can get it.
build:
runs-on: ubuntu-latest
needs: get-go-version
outputs:
minio-dockerfile-sha: ${{ steps.minio-version.outputs.dockerfile_sha }}
steps:
- name: Check out the code
uses: actions/checkout@v6
- name: Set up Go version
uses: actions/setup-go@v6
uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ needs.get-go-version.outputs.version }}
go-version-file: 'go.mod'
# Look for a CLI that's made for this PR
- name: Fetch built CLI
id: cli-cache
@@ -52,91 +41,42 @@ jobs:
- name: Build Velero Image
if: steps.image-cache.outputs.cache-hit != 'true'
run: |
IMAGE=velero VERSION=pr-test BUILD_OUTPUT_TYPE=docker make container
docker save velero:pr-test-linux-amd64 -o ./velero.tar
# Check and build MinIO image once for all e2e tests
- name: Check Bitnami MinIO Dockerfile version
id: minio-version
run: |
DOCKERFILE_SHA=$(curl -s https://api.github.com/repos/bitnami/containers/commits?path=bitnami/minio/2025/debian-12/Dockerfile\&per_page=1 | jq -r '.[0].sha')
echo "dockerfile_sha=${DOCKERFILE_SHA}" >> $GITHUB_OUTPUT
- name: Cache MinIO Image
uses: actions/cache@v4
id: minio-cache
with:
path: ./minio-image.tar
key: minio-bitnami-${{ steps.minio-version.outputs.dockerfile_sha }}
- name: Build MinIO Image from Bitnami Dockerfile
if: steps.minio-cache.outputs.cache-hit != 'true'
run: |
echo "Building MinIO image from Bitnami Dockerfile..."
git clone --depth 1 https://github.com/bitnami/containers.git /tmp/bitnami-containers
cd /tmp/bitnami-containers/bitnami/minio/2026/debian-12
docker build -t bitnami/minio:local .
docker save bitnami/minio:local > ${{ github.workspace }}/minio-image.tar
# Create json of k8s versions to test
# from guide: https://stackoverflow.com/a/65094398/4590470
setup-test-matrix:
runs-on: ubuntu-latest
env:
GH_TOKEN: ${{ github.token }}
outputs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
steps:
- name: Set k8s versions
id: set-matrix
# everything excluding older tags. limits needs to be high enough to cover all latest versions
# and test labels
# grep -E "v[1-9]\.(2[5-9]|[3-9][0-9])" filters for v1.25 to v9.99
# and removes older patches of the same minor version
# awk -F. '{if(!a[$1"."$2]++)print $1"."$2"."$NF}'
run: |
echo "matrix={\
\"k8s\":$(wget -q -O - "https://hub.docker.com/v2/namespaces/kindest/repositories/node/tags?page_size=50" | grep -o '"name": *"[^"]*' | grep -o '[^"]*$' | grep -v -E "alpha|beta" | grep -E "v[1-9]\.(2[5-9]|[3-9][0-9])" | awk -F. '{if(!a[$1"."$2]++)print $1"."$2"."$NF}' | sort -r | sed s/v//g | jq -R -c -s 'split("\n")[:-1]'),\
\"labels\":[\
\"Basic && (ClusterResource || NodePort || StorageClass)\", \
\"ResourceFiltering && !Restic\", \
\"ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources\", \
\"(NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)\"\
]}" >> $GITHUB_OUTPUT
IMAGE=velero VERSION=pr-test make container
docker save velero:pr-test -o ./velero.tar
# Run E2E test against all Kubernetes versions on kind
run-e2e-test:
needs:
- build
- setup-test-matrix
- get-go-version
needs: build
runs-on: ubuntu-latest
strategy:
matrix: ${{fromJson(needs.setup-test-matrix.outputs.matrix)}}
matrix:
k8s:
- 1.23.17
- 1.24.17
- 1.25.16
- 1.26.13
- 1.27.10
- 1.28.6
- 1.29.1
labels:
# labels are used to filter running E2E cases
- Basic && (ClusterResource || NodePort || StorageClass)
- ResourceFiltering && !Restic
- ResourceModifier || (Backups && BackupsSync) || PrivilegesMgmt || OrderedResources
- (NamespaceMapping && Single && Restic) || (NamespaceMapping && Multiple && Restic)
fail-fast: false
steps:
- name: Check out the code
uses: actions/checkout@v6
- name: Set up Go version
uses: actions/setup-go@v6
uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ needs.get-go-version.outputs.version }}
# Fetch the pre-built MinIO image from the build job
- name: Fetch built MinIO Image
uses: actions/cache@v4
id: minio-cache
with:
path: ./minio-image.tar
key: minio-bitnami-${{ needs.build.outputs.minio-dockerfile-sha }}
- name: Load MinIO Image
run: |
echo "Loading MinIO image..."
docker load < ./minio-image.tar
go-version-file: 'go.mod'
- name: Install MinIO
run: |
docker run -d --rm -p 9000:9000 -e "MINIO_ROOT_USER=minio" -e "MINIO_ROOT_PASSWORD=minio123" -e "MINIO_DEFAULT_BUCKETS=bucket,additional-bucket" bitnami/minio:local
- uses: engineerd/setup-kind@v0.6.2
run:
docker run -d --rm -p 9000:9000 -e "MINIO_ACCESS_KEY=minio" -e "MINIO_SECRET_KEY=minio123" -e "MINIO_DEFAULT_BUCKETS=bucket,additional-bucket" bitnami/minio:2021.6.17-debian-10-r7
- uses: engineerd/setup-kind@v0.5.0
with:
skipClusterLogsExport: true
version: "v0.27.0"
version: "v0.21.0"
image: "kindest/node:v${{ matrix.k8s }}"
- name: Fetch built CLI
id: cli-cache
@@ -165,8 +105,6 @@ jobs:
curl -LO https://dl.k8s.io/release/v${{ matrix.k8s }}/bin/linux/amd64/kubectl
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
git clone https://github.com/vmware-tanzu-experiments/distributed-data-generator.git -b main /tmp/kibishii
GOPATH=~/go \
CLOUD_PROVIDER=kind \
OBJECT_STORE_PROVIDER=aws \
@@ -177,15 +115,13 @@ jobs:
ADDITIONAL_BSL_CONFIG=region=minio,s3ForcePathStyle="true",s3Url=http://$(hostname -i):9000 \
ADDITIONAL_CREDS_FILE=/tmp/credential \
ADDITIONAL_BSL_BUCKET=additional-bucket \
VELERO_IMAGE=velero:pr-test-linux-amd64 \
PLUGINS=velero/velero-plugin-for-aws:latest \
VELERO_IMAGE=velero:pr-test \
GINKGO_LABELS="${{ matrix.labels }}" \
KIBISHII_DIRECTORY=/tmp/kibishii/kubernetes/yaml/ \
make -C test/ run-e2e
timeout-minutes: 30
- name: Upload debug bundle
if: ${{ failure() }}
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@v4
with:
name: DebugBundle-k8s-${{ matrix.k8s }}-job-${{ strategy.job-index }}
name: DebugBundle
path: /home/runner/work/velero/velero/test/e2e/debug-bundle*

View File

@@ -1,33 +0,0 @@
on:
workflow_call:
inputs:
ref:
description: "The target branch's ref"
required: true
type: string
outputs:
version:
description: "The expected Go version"
value: ${{ jobs.extract.outputs.version }}
jobs:
extract:
runs-on: ubuntu-latest
outputs:
version: ${{ steps.pick-version.outputs.version }}
steps:
- name: Check out the code
uses: actions/checkout@v6
- id: pick-version
run: |
if [ "${{ inputs.ref }}" == "main" ]; then
version=$(grep '^go ' go.mod | awk '{print $2}' | cut -d. -f1-2)
else
goDirectiveVersion=$(grep '^go ' go.mod | awk '{print $2}')
toolChainVersion=$(grep '^toolchain ' go.mod | awk '{print $2}' | sed 's/^go//')
version=$(printf "%s\n%s\n" "$goDirectiveVersion" "$toolChainVersion" | sort -V | tail -n1)
fi
echo "version=$version"
echo "version=$version" >> $GITHUB_OUTPUT

View File

@@ -13,16 +13,16 @@ jobs:
# maintain the versions of Velero those need security scan
versions: [main]
# list of images that need scan
images: [velero, velero-plugin-for-aws, velero-plugin-for-gcp, velero-plugin-for-microsoft-azure]
images: [velero, velero-restore-helper]
permissions:
security-events: write # for github/codeql-action/upload-sarif to upload SARIF results
steps:
- name: Checkout code
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@57a97c7e7821a5776cebc9bb87c984fa69cba8f1
uses: aquasecurity/trivy-action@master
with:
image-ref: 'docker.io/velero/${{ matrix.images }}:${{ matrix.versions }}'
severity: 'CRITICAL,HIGH,MEDIUM'

View File

@@ -12,7 +12,7 @@ jobs:
steps:
- name: Check out the code
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Changelog check
if: ${{ !(contains(github.event.pull_request.labels.*.name, 'kind/changelog-not-required') || contains(github.event.pull_request.labels.*.name, 'Design') || contains(github.event.pull_request.labels.*.name, 'Website') || contains(github.event.pull_request.labels.*.name, 'Documentation'))}}

View File

@@ -1,30 +1,22 @@
name: Pull Request CI Check
on: [pull_request]
jobs:
get-go-version:
uses: ./.github/workflows/get-go-version.yaml
with:
ref: ${{ github.event.pull_request.base.ref }}
build:
name: Run CI
needs: get-go-version
runs-on: ubuntu-latest
strategy:
fail-fast: false
steps:
- name: Check out the code
uses: actions/checkout@v6
- name: Set up Go version
uses: actions/setup-go@v6
uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ needs.get-go-version.outputs.version }}
go-version-file: 'go.mod'
- name: Make ci
run: make ci
- name: Upload test coverage
uses: codecov/codecov-action@v5
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: coverage.out

View File

@@ -8,14 +8,14 @@ jobs:
steps:
- name: Check out the code
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Codespell
uses: codespell-project/actions-codespell@master
with:
# ignore the config/.../crd.go file as it's generated binary data that is edited elsewhere.
# ignore the config/.../crd.go file as it's generated binary data that is edited elswhere.
skip: .git,*.png,*.jpg,*.woff,*.ttf,*.gif,*.ico,./config/crd/v1beta1/crds/crds.go,./config/crd/v1/crds/crds.go,./config/crd/v2alpha1/crds/crds.go,./go.sum,./LICENSE
ignore_words_list: iam,aks,ist,bridget,ue,shouldnot,atleast,notin,sme,optin,sie
ignore_words_list: iam,aks,ist,bridget,ue,shouldnot,atleast,notin,sme,optin
check_filenames: true
check_hidden: true

View File

@@ -7,14 +7,13 @@ on:
- 'release-**'
paths:
- 'Dockerfile'
- 'Dockerfile-Windows'
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
name: Checkout
- name: Set up QEMU
@@ -33,6 +32,6 @@ jobs:
# by push, so BRANCH and TAG are empty by default. docker-push.sh will
# only build Velero image without pushing.
- name: Make Velero container without pushing to registry.
if: github.repository == 'velero-io/velero'
if: github.repository == 'vmware-tanzu/velero'
run: |
./hack/docker-push.sh

View File

@@ -1,93 +0,0 @@
name: Pull Request File Path Check
on: [pull_request]
jobs:
filepath-check:
name: Check for invalid characters in file paths
runs-on: ubuntu-latest
steps:
- name: Check out the code
uses: actions/checkout@v6
- name: Validate file paths for Go module compatibility
run: |
# Go's module zip rejects filenames containing certain characters.
# See golang.org/x/mod/module fileNameOK() for the full specification.
#
# Allowed ASCII: letters, digits, and: !#$%&()+,-.=@[]^_{}~ and space
# Allowed non-ASCII: unicode letters only
# Rejected: " ' * < > ? ` | / \ : and any non-letter unicode (control
# chars, format chars like U+200E LEFT-TO-RIGHT MARK, etc.)
#
# This check catches issues like the U+200E incident in PR #9552.
EXIT_STATUS=0
git ls-files -z | python3 -c "
import sys, unicodedata
data = sys.stdin.buffer.read()
files = data.split(b'\x00')
# Characters explicitly rejected by Go's fileNameOK
# (path separators / and \ are inherent to paths so we check per-element)
bad_ascii = set('\"' + \"'\" + '*<>?\`|:')
allowed_ascii = set('!#$%&()+,-.=@[]^_{}~ ')
def is_ok(ch):
if ch.isascii():
return ch.isalnum() or ch in allowed_ascii
return ch.isalpha()
bad_files = [] # list of (original_path, clean_path, char_desc)
for f in files:
if not f:
continue
try:
name = f.decode('utf-8')
except UnicodeDecodeError:
print(f'::error::Non-UTF-8 bytes in filename: {f!r}')
bad_files.append((repr(f), None, 'non-UTF-8 bytes'))
continue
# Check each path element (split on /)
for element in name.split('/'):
for ch in element:
if not is_ok(ch):
cp = ord(ch)
char_name = unicodedata.name(ch, f'U+{cp:04X}')
char_desc = f'U+{cp:04X} ({char_name})'
# Build cleaned path by stripping invalid chars
clean = '/'.join(
''.join(c for c in elem if is_ok(c))
for elem in name.split('/')
)
print(f'::error file={name}::File \"{name}\" contains invalid char {char_desc}')
bad_files.append((name, clean, char_desc))
break
if bad_files:
print()
print('The following files have characters that are invalid in Go module zip archives:')
print()
for original, clean, desc in bad_files:
print(f' {original} — {desc}')
print()
print('To fix, rename the files to remove the problematic characters:')
print()
for original, clean, desc in bad_files:
if clean:
print(f' mv \"{original}\" \"{clean}\" && git add \"{clean}\"')
print(f' # or: git mv \"{original}\" \"{clean}\"')
else:
print(f' # {original} — cannot auto-suggest rename (non-UTF-8)')
print()
print('See https://github.com/velero-io/velero/pull/9552 for context.')
sys.exit(1)
else:
print('All file paths are valid for Go module zip.')
" || EXIT_STATUS=1
exit $EXIT_STATUS

View File

@@ -14,11 +14,11 @@ jobs:
name: Build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
name: Checkout
- name: Verify .goreleaser.yml and try a dryrun release.
if: github.repository == 'velero-io/velero'
if: github.repository == 'vmware-tanzu/velero'
run: |
CHANGELOG=$(ls changelogs | sort -V -r | head -n 1)
GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }} \

View File

@@ -1,32 +1,19 @@
name: Pull Request Linter Check
on:
pull_request:
# Do not run when the change only includes these directories.
paths-ignore:
- "site/**"
- "design/**"
- "**/*.md"
on: [pull_request]
jobs:
get-go-version:
uses: ./.github/workflows/get-go-version.yaml
with:
ref: ${{ github.event.pull_request.base.ref }}
build:
name: Run Linter Check
runs-on: ubuntu-latest
needs: get-go-version
steps:
- name: Check out the code
uses: actions/checkout@v6
- name: Set up Go version
uses: actions/setup-go@v6
uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ needs.get-go-version.outputs.version }}
go-version-file: 'go.mod'
- name: Linter check
uses: golangci/golangci-lint-action@v9
uses: golangci/golangci-lint-action@v6
with:
version: v2.12.0
version: v1.57.2
args: --verbose

View File

@@ -5,10 +5,6 @@ on:
issue_comment:
types: [created]
permissions:
issues: write
pull-requests: write
jobs:
execute:
runs-on: ubuntu-latest

View File

@@ -12,7 +12,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
with:
# The default value is "1" which fetches only a single commit. If we merge PR without squash or rebase,
# there are at least two commits: the first one is the merge commit and the second one is the real commit
@@ -28,7 +28,7 @@ jobs:
# Only try to publish the container image from the root repo; forks don't have permission to do so and will always get failures.
- name: Publish container image
if: github.repository == 'velero-io/velero'
if: github.repository == 'vmware-tanzu/velero'
run: |
docker login -u ${{ secrets.DOCKER_USER }} -p ${{ secrets.DOCKER_PASSWORD }}

View File

@@ -9,24 +9,26 @@ on:
- '*'
jobs:
get-go-version:
uses: ./.github/workflows/get-go-version.yaml
with:
ref: ${{ github.ref_name }}
build:
name: Build
runs-on: ubuntu-latest
needs: get-go-version
steps:
- name: Check out the code
uses: actions/checkout@v6
- name: Set up Go version
uses: actions/setup-go@v6
uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ needs.get-go-version.outputs.version }}
go-version-file: 'go.mod'
- id: 'auth'
uses: google-github-actions/auth@v2
with:
credentials_json: '${{ secrets.GCS_SA_KEY }}'
- name: 'set up GCloud SDK'
uses: google-github-actions/setup-gcloud@v2
- name: 'use gcloud CLI'
run: |
gcloud info
- name: Set up QEMU
id: qemu
uses: docker/setup-qemu-action@v3
@@ -45,14 +47,20 @@ jobs:
- name: Test
run: make test
- name: Upload test coverage
uses: codecov/codecov-action@v5
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: coverage.out
verbose: true
# Use the JSON key in secret to login gcr.io
- uses: 'docker/login-action@v3'
with:
registry: 'gcr.io' # or REGION.docker.pkg.dev
username: '_json_key'
password: '${{ secrets.GCR_SA_KEY }}'
# Only try to publish the container image from the root repo; forks don't have permission to do so and will always get failures.
- name: Publish container image
if: github.repository == 'velero-io/velero'
if: github.repository == 'vmware-tanzu/velero'
run: |
sudo swapoff -a
sudo rm -f /mnt/swapfile
@@ -60,4 +68,24 @@ jobs:
# Build and push Velero image to docker registry
docker login -u ${{ secrets.DOCKER_USER }} -p ${{ secrets.DOCKER_PASSWORD }}
./hack/docker-push.sh
VERSION=$(./hack/docker-push.sh | grep 'VERSION:' | awk -F: '{print $2}' | xargs)
# Upload Velero image package to GCS
source hack/ci/build_util.sh
BIN=velero
RESTORE_HELPER_BIN=velero-restore-helper
GCS_BUCKET=velero-builds
VELERO_IMAGE=${BIN}-${VERSION}
VELERO_RESTORE_HELPER_IMAGE=${RESTORE_HELPER_BIN}-${VERSION}
VELERO_IMAGE_FILE=${VELERO_IMAGE}.tar.gz
VELERO_RESTORE_HELPER_IMAGE_FILE=${VELERO_RESTORE_HELPER_IMAGE}.tar.gz
VELERO_IMAGE_BACKUP_FILE=${VELERO_IMAGE}-'build.'${GITHUB_RUN_NUMBER}.tar.gz
VELERO_RESTORE_HELPER_IMAGE_BACKUP_FILE=${VELERO_RESTORE_HELPER_IMAGE}-'build.'${GITHUB_RUN_NUMBER}.tar.gz
cp ${VELERO_IMAGE_FILE} ${VELERO_IMAGE_BACKUP_FILE}
cp ${VELERO_RESTORE_HELPER_IMAGE_FILE} ${VELERO_RESTORE_HELPER_IMAGE_BACKUP_FILE}
uploader ${VELERO_IMAGE_FILE} ${GCS_BUCKET}
uploader ${VELERO_RESTORE_HELPER_IMAGE_FILE} ${GCS_BUCKET}
uploader ${VELERO_IMAGE_BACKUP_FILE} ${GCS_BUCKET}
uploader ${VELERO_RESTORE_HELPER_IMAGE_BACKUP_FILE} ${GCS_BUCKET}

View File

@@ -9,7 +9,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout the latest code
uses: actions/checkout@v6
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Automatic Rebase

View File

@@ -7,7 +7,7 @@ jobs:
stale:
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v10.1.1
- uses: actions/stale@v9.0.0
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
stale-issue-message: "This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 14 days. If a Velero team member has requested log or more information, please provide the output of the shared commands."

11
.gitignore vendored
View File

@@ -53,13 +53,4 @@ tilt-resources/cloud
# test generated files
test/e2e/report.xml
coverage.out
__debug_bin*
debug.test*
# make lint cache
.cache/
# Go telemetry directory created when container sets HOME to working directory
# This happens because Makefile uses 'docker run -w /github.com/vmware-tanzu/velero'
# and Go's os.UserConfigDir() falls back to $HOME/.config when XDG_CONFIG_HOME is unset
.config/
__debug_bin*

View File

@@ -6,12 +6,17 @@ run:
# default concurrency is a available CPU number
concurrency: 4
# timeout for analysis, e.g. 30s, 5m, default is 0
# timeout for analysis, e.g. 30s, 5m, default is 1m
timeout: 20m
# exit code when at least one issue was found, default is 1
issues-exit-code: 1
# default is true. Enables skipping of directories:
# vendor$, third_party$, testdata$, examples$, Godeps$, builtin$
skip-dirs-use-default: true
# by default isn't set. If set we pass it to "go list -mod={option}". From "go help modules":
# If invoked with -mod=readonly, the go command is disallowed from the implicit
# automatic updating of go.mod described above. Instead, it fails when any changes
@@ -27,406 +32,362 @@ run:
# If false (default) - golangci-lint acquires file lock on start.
allow-parallel-runners: false
# output configuration options
output:
# colored-line-number|line-number|json|tab|checkstyle|code-climate, default is "colored-line-number"
formats:
text:
- format: colored-line-number
path: stdout
# print lines of code with issue, default is true
print-issued-lines: true
# print lines of code with issue, default is true
print-issued-lines: true
# print linter name in the end of issue text, default is true
print-linter-name: true
# print linter name in the end of issue text, default is true
print-linter-name: true
# Show statistics per linter.
show-stats: false
linters:
# all available settings of specific linters
settings:
depguard:
rules:
main:
deny:
# specify an error message to output when a denylisted package is used
- pkg: github.com/sirupsen/logrus
desc: "logging is allowed only by logutils.Log"
dogsled:
# checks assignments with too many blank identifiers; default is 2
max-blank-identifiers: 2
dupl:
# tokens count to trigger issue, 150 by default
threshold: 100
errcheck:
# report about not checking of errors in type assertions: `a := b.(MyStruct)`;
# default is false: such cases aren't reported by default.
check-type-assertions: false
# report about assignment of errors to blank identifier: `num, _ := strconv.Atoi(numStr)`;
# default is false: such cases aren't reported by default.
check-blank: false
# make issues output unique by line, default is true
uniq-by-line: true
exhaustive:
# indicates that switch statements are to be considered exhaustive if a
# 'default' case is present, even if all enum members aren't listed in the
# switch
default-signifies-exhaustive: false
# all available settings of specific linters
linters-settings:
dogsled:
# checks assignments with too many blank identifiers; default is 2
max-blank-identifiers: 2
dupl:
# tokens count to trigger issue, 150 by default
threshold: 100
errcheck:
# report about not checking of errors in type assertions: `a := b.(MyStruct)`;
# default is false: such cases aren't reported by default.
check-type-assertions: false
funlen:
lines: 60
statements: 40
# report about assignment of errors to blank identifier: `num, _ := strconv.Atoi(numStr)`;
# default is false: such cases aren't reported by default.
check-blank: false
gocognit:
# minimal code complexity to report, 30 by default (but we recommend 10-20)
min-complexity: 10
# [deprecated] comma-separated list of pairs of the form pkg:regex
# the regex is used to ignore names within pkg. (default "fmt:.*").
# see https://github.com/kisielk/errcheck#the-deprecated-method for details
# ignore: fmt:.*,io/ioutil:^Read.*
nestif:
# minimal complexity of if statements to report, 5 by default
min-complexity: 4
# path to a file containing a list of functions to exclude from checking
# see https://github.com/kisielk/errcheck#excluding-functions for details
# exclude: /path/to/file.txt
exhaustive:
# indicates that switch statements are to be considered exhaustive if a
# 'default' case is present, even if all enum members aren't listed in the
# switch
default-signifies-exhaustive: false
funlen:
lines: 60
statements: 40
gocognit:
# minimal code complexity to report, 30 by default (but we recommend 10-20)
min-complexity: 10
nestif:
# minimal complexity of if statements to report, 5 by default
min-complexity: 4
goconst:
# minimal length of string constant, 3 by default
min-len: 3
# minimal occurrences count to trigger, 3 by default
min-occurrences: 5
gocritic:
# Which checks should be enabled; can't be combined with 'disabled-checks';
# See https://go-critic.github.io/overview#checks-overview
# To check which checks are enabled run `GL_DEBUG=gocritic golangci-lint run`
# By default list of stable checks is used.
# enabled-checks:
# - rangeValCopy
goconst:
# minimal length of string constant, 3 by default
min-len: 3
# minimal occurrences count to trigger, 3 by default
min-occurrences: 5
# Which checks should be disabled; can't be combined with 'enabled-checks'; default is empty
# disabled-checks:
# - regexpMust
gocritic:
# Which checks should be enabled; can't be combined with 'disabled-checks';
# See https://go-critic.github.io/overview#checks-overview
# To check which checks are enabled run `GL_DEBUG=gocritic golangci-lint run`
# By default list of stable checks is used.
settings: # settings passed to gocritic
captLocal: # must be valid enabled check name
paramsOnly: true
# Enable multiple checks by tags, run `GL_DEBUG=gocritic golangci-lint run` to see all tags and checks.
# Empty list by default. See https://github.com/go-critic/go-critic#usage -> section "Tags".
# enabled-tags:
# - performance
# disabled-tags:
# - experimental
gocyclo:
# minimal code complexity to report, 30 by default (but we recommend 10-20)
min-complexity: 10
settings: # settings passed to gocritic
captLocal: # must be valid enabled check name
paramsOnly: true
# rangeValCopy:
# sizeThreshold: 32
gocyclo:
# minimal code complexity to report, 30 by default (but we recommend 10-20)
min-complexity: 10
godot:
# check all top-level comments, not only declarations
check-all: false
godox:
# report any comments starting with keywords, this is useful for TODO or FIXME comments that
# might be left in the code accidentally and should be resolved before merging
keywords: # default keywords are TODO, BUG, and FIXME, these can be overwritten by this setting
- NOTE
- OPTIMIZE # marks code that should be optimized before merging
- HACK # marks hack-arounds that should be removed before merging
gofmt:
# simplify code: gofmt with `-s` option, true by default
simplify: true
goimports:
# put imports beginning with prefix after 3rd-party packages;
# it's a comma-separated list of prefixes
local-prefixes: github.com/org/project
golint:
# minimal confidence for issues, default is 0.8
min-confidence: 0.8
gomnd:
# the list of enabled checks, see https://github.com/tommy-muehle/go-mnd/#checks for description.
checks: argument,case,condition,operation,return,assign
gomodguard:
allowed:
modules: # List of allowed modules
# - gopkg.in/yaml.v2
domains: # List of allowed module domains
# - golang.org
blocked:
modules: # List of blocked modules
# - github.com/uudashr/go-module: # Blocked module
# recommendations: # Recommended modules that should be used instead (Optional)
# - golang.org/x/mod
# reason: "`mod` is the official go.mod parser library." # Reason why the recommended module should be used (Optional)
versions: # List of blocked module version constraints
# - github.com/mitchellh/go-homedir: # Blocked module with version constraint
# version: "< 1.1.0" # Version constraint, see https://github.com/Masterminds/semver#basic-comparisons
# reason: "testing if blocked version constraint works." # Reason why the version constraint exists. (Optional)
govet:
# report about shadowed variables
# check-shadowing: true
godot:
# check all top-level comments, not only declarations
check-all: false
# settings per analyzer
settings:
printf: # analyzer name, run `go tool vet help` to see all analyzers
funcs: # run `go tool vet help printf` to see available settings for `printf` analyzer
- (github.com/golangci/golangci-lint/pkg/logutils.Log).Infof
- (github.com/golangci/golangci-lint/pkg/logutils.Log).Warnf
- (github.com/golangci/golangci-lint/pkg/logutils.Log).Errorf
- (github.com/golangci/golangci-lint/pkg/logutils.Log).Fatalf
godox:
# report any comments starting with keywords, this is useful for TODO or FIXME comments that
# might be left in the code accidentally and should be resolved before merging
keywords: # default keywords are TODO, BUG, and FIXME, these can be overwritten by this setting
- NOTE
- OPTIMIZE # marks code that should be optimized before merging
- HACK # marks hack-arounds that should be removed before merging
gosec:
excludes:
- G115
govet:
# enable or disable analyzers by name
enable:
- atomicalign
enable-all: false
disable:
- shadow
disable-all: false
importas:
alias:
- alias: appsv1api
pkg: k8s.io/api/apps/v1
- alias: corev1api
pkg: k8s.io/api/core/v1
- alias: rbacv1
pkg: k8s.io/api/rbac/v1
- alias: apierrors
pkg: k8s.io/apimachinery/pkg/api/errors
- alias: apiextv1
pkg: k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1
- alias: metav1
pkg: k8s.io/apimachinery/pkg/apis/meta/v1
- alias: storagev1api
pkg: k8s.io/api/storage/v1
- alias: batchv1api
pkg: k8s.io/api/batch/v1
lll:
# enable or disable analyzers by name
enable:
- atomicalign
enable-all: false
disable:
- shadow
disable-all: false
depguard:
list-type: blacklist # Velero.io word list : ignore
include-go-root: false
packages:
- github.com/sirupsen/logrus
packages-with-error-message:
# specify an error message to output when a denylisted package is used
- github.com/sirupsen/logrus: "logging is allowed only by logutils.Log"
lll:
# max line length, lines longer will be reported. Default is 120.
# '\t' is counted as 1 character by default, and can be changed with the tab-width option
line-length: 120
# tab width in spaces. Default to 1.
tab-width: 1
line-length: 120
# tab width in spaces. Default to 1.
tab-width: 1
maligned:
# print struct with more effective memory layout or not, false by default
suggest-new: true
misspell:
# Correct spellings using locale preferences for US or UK.
# Default is to use a neutral variety of English.
# Setting locale to US will correct the British spelling of 'colour' to 'color'.
locale: US
ignore-words:
- someword
nakedret:
# make an issue if func has more lines of code than this setting and it has naked returns; default is 30
max-func-lines: 30
prealloc:
# XXX: we don't recommend using this linter before doing performance profiling.
# For most programs usage of prealloc will be a premature optimization.
misspell:
# Correct spellings using locale preferences for US or UK.
# Default is to use a neutral variety of English.
# Setting locale to US will correct the British spelling of 'colour' to 'color'.
locale: US
ignore-rules:
- someword
nakedret:
# make an issue if func has more lines of code than this setting and it has naked returns; default is 30
max-func-lines: 30
prealloc:
# XXX: we don't recommend using this linter before doing performance profiling.
# For most programs usage of prealloc will be a premature optimization.
# Report preallocation suggestions only on simple loops that have no returns/breaks/continues/gotos in them.
# True by default.
simple: true
range-loops: true # Report preallocation suggestions on range loops, true by default
for-loops: false # Report preallocation suggestions on for loops, false by default
nolintlint:
# Enable to ensure that nolint directives are all used. Default is true.
allow-unused: false
# Exclude following linters from requiring an explanation. Default is [].
allow-no-explanation: []
# Enable to require an explanation of nonzero length after each nolint directive. Default is false.
require-explanation: true
# Enable to require nolint directives to mention the specific linter being suppressed. Default is false.
require-specific: true
perfsprint:
strconcat: false
sprintf1: false
errorf: false
int-conversion: true
revive:
rules:
- name: blank-imports
disabled: true
- name: context-as-argument
disabled: true
- name: context-keys-type
- name: dot-imports
disabled: true
- name: early-return
disabled: true
arguments:
- "preserveScope"
- name: empty-block
disabled: true
- name: error-naming
disabled: true
- name: error-return
disabled: true
- name: error-strings
disabled: true
- name: errorf
disabled: true
- name: increment-decrement
- name: indent-error-flow
disabled: true
- name: range
- name: receiver-naming
disabled: true
- name: redefines-builtin-id
disabled: true
- name: superfluous-else
disabled: true
arguments:
- "preserveScope"
- name: time-naming
- name: unexported-return
disabled: true
- name: unnecessary-stmt
- name: unreachable-code
- name: unused-parameter
disabled: true
- name: use-any
- name: var-declaration
- name: var-naming
disabled: true
rowserrcheck:
packages:
- github.com/jmoiron/sqlx
staticcheck:
checks:
- all
- -QF1001 # FIXME
- -QF1003 # FIXME
- -QF1004 # FIXME
- -QF1007 # FIXME
- -QF1008 # FIXME
- -QF1009 # FIXME
- -QF1012 # FIXME
testifylint:
# Report preallocation suggestions only on simple loops that have no returns/breaks/continues/gotos in them.
# True by default.
simple: true
range-loops: true # Report preallocation suggestions on range loops, true by default
for-loops: false # Report preallocation suggestions on for loops, false by default
nolintlint:
# Enable to ensure that nolint directives are all used. Default is true.
allow-unused: false
# Disable to ensure that nolint directives don't have a leading space. Default is true.
allow-leading-space: true
# Exclude following linters from requiring an explanation. Default is [].
allow-no-explanation: []
# Enable to require an explanation of nonzero length after each nolint directive. Default is false.
require-explanation: true
# Enable to require nolint directives to mention the specific linter being suppressed. Default is false.
require-specific: true
revive:
rules:
- name: unexported-return
disabled: true
rowserrcheck:
packages:
- github.com/jmoiron/sqlx
testifylint:
# TODO: enable them all
disable:
- float-compare
- go-require
- float-compare
- require-error
enable-all: true
testpackage:
# regexp pattern to skip files
skip-regexp: (export|internal)_test\.go
unparam:
# Inspect exported functions, default is false. Set to true if no external program/library imports your code.
# XXX: if you enable this setting, unparam will report a lot of false-positives in text editors:
# if it's called for subdir of a project it can't find external interfaces. All text editor integrations
# with golangci-lint call it on a directory with the changed file.
check-exported: false
unused:
# treat code as a program (not a library) and report unused exported identifiers; default is false.
# XXX: if you enable this setting, unused will report a lot of false-positives in text editors:
# if it's called for subdir of a project it can't find funcs usages. All text editor integrations
# with golangci-lint call it on a directory with the changed file.
check-exported: false
whitespace:
multi-if: false # Enforces newlines (or comments) after every multi-line if statement
multi-func: false # Enforces newlines (or comments) after every multi-line function signature
wsl:
# If true append is only allowed to be cuddled if appending value is
# matching variables, fields or types on line above. Default is true.
strict-append: true
# Allow calls and assignments to be cuddled as long as the lines have any
# matching variables, fields or types. Default is true.
allow-assign-and-call: true
# Allow multiline assignments to be cuddled. Default is true.
allow-multiline-assign: true
# Allow declarations (var) to be cuddled.
allow-cuddle-declarations: false
# Allow trailing comments in ending of blocks
allow-trailing-comment: false
# Force newlines in end of case at this limit (0 = never).
force-case-trailing-whitespace: 0
# Force cuddling of err checks with err var assignment
force-err-cuddling: false
# Allow leading comments to be separated with empty liens
allow-separated-leading-comment: false
testpackage:
# regexp pattern to skip files
skip-regexp: (export|internal)_test\.go
unparam:
# Inspect exported functions, default is false. Set to true if no external program/library imports your code.
# XXX: if you enable this setting, unparam will report a lot of false-positives in text editors:
# if it's called for subdir of a project it can't find external interfaces. All text editor integrations
# with golangci-lint call it on a directory with the changed file.
check-exported: false
usetesting:
os-setenv: false
whitespace:
multi-if: false # Enforces newlines (or comments) after every multi-line if statement
multi-func: false # Enforces newlines (or comments) after every multi-line function signature
wsl:
# If true append is only allowed to be cuddled if appending value is
# matching variables, fields or types on line above. Default is true.
strict-append: true
# Allow calls and assignments to be cuddled as long as the lines have any
# matching variables, fields or types. Default is true.
allow-assign-and-call: true
# Allow multiline assignments to be cuddled. Default is true.
allow-multiline-assign: true
# Allow declarations (var) to be cuddled.
allow-cuddle-declarations: false
# Allow trailing comments in ending of blocks
allow-trailing-comment: false
# Force newlines in end of case at this limit (0 = never).
force-case-trailing-whitespace: 0
# Force cuddling of err checks with err var assignment
force-err-cuddling: false
# Allow leading comments to be separated with empty lines
allow-separated-leading-comment: false
default: none
linters:
disable-all: true
enable:
- asasalint
- asciicheck
- bidichk
- bodyclose
- copyloopvar
- dogsled
- dupword
- durationcheck
- dupword
- errcheck
- exportloopref
- errchkjson
- exptostd
- ginkgolinter
#- goconst # Disable goconst for now, as it reports a lot of false positives. We can enable it later after refactoring the codebase to reduce the number of string literals.
- goconst
- gofmt
- goheader
- goimports
- goprintffuncname
- gosec
- gosimple
- govet
- ginkgolinter
- importas
- ineffassign
- misspell
- nakedret
- nosprintfhostport
- nilerr
- noctx
- nolintlint
- nosprintfhostport
- perfsprint
- revive
- staticcheck
- stylecheck
- testifylint
- thelper
- typecheck
- unconvert
- unparam
- unused
- usestdlibvars
- usetesting
- whitespace
fast: false
exclusions:
# which dirs to skip: issues from them won't be reported;
# can use regexp here: generated.*, regexp is applied on full path;
# default value is empty list, but default dirs are skipped independently
# from this option's value (see skip-dirs-use-default).
# "/" will be replaced by current OS file path separator to properly work
# on Windows.
paths:
- pkg/plugin/generated/*
- third_party
rules:
- linters:
- staticcheck
text: "DefaultVolumesToRestic" # No need to report deprecate for DefaultVolumesToRestic.
- path: ".*_test.go$"
linters:
- errcheck
- goconst
- gosec
- govet
- staticcheck
- unparam
- unused
- path: test/
linters:
- errcheck
- goconst
- gosec
- nilerr
- staticcheck
- unparam
- unused
- path: ".*data_upload_controller_test.go$"
linters:
- dupword
text: "type"
- path: ".*config_test.go$"
linters:
- dupword
text: "bucket"
- text: "non-constant format string"
linters:
- govet
generated: lax
presets:
- comments
- common-false-positives
- legacy
- std-error-handling
issues:
exclude-rules:
- linters:
- staticcheck
text: "DefaultVolumesToRestic" # No need to report deprecate for DefaultVolumesToRestic.
- path: ".*_test.go$"
linters:
- dupword
- errcheck
- goconst
- gosec
- govet
- staticcheck
- stylecheck
- unparam
- unused
- path: test/
linters:
- dupword
- errcheck
- goconst
- gosec
- nilerr
- staticcheck
- stylecheck
- unparam
- unused
# The list of ids of default excludes to include or disable. By default it's empty.
include:
- EXC0002 # disable excluding of issues about comments from golint
# Maximum issues count per one linter. Set to 0 to disable. Default is 50.
max-issues-per-linter: 0
# Maximum count of issues with the same text. Set to 0 to disable. Default is 3.
max-same-issues: 0
# make issues output unique by line, default is true
uniq-by-line: true
# Show only new issues created after git revision `REV`
# new-from-rev: origin/main
# This file contains all available configuration options
# with their default values.
formatters:
enable:
- gofmt
- goimports
exclusions:
generated: lax
paths:
- pkg/plugin/generated/*
- third_party
settings:
gofmt:
# simplify code: gofmt with `-s` option, true by default
simplify: true
goimports:
local-prefixes:
- github.com/vmware-tanzu/velero
# which dirs to skip: issues from them won't be reported;
# can use regexp here: generated.*, regexp is applied on full path;
# default value is empty list, but default dirs are skipped independently
# from this option's value (see skip-dirs-use-default).
# "/" will be replaced by current OS file path separator to properly work
# on Windows.
exclude-dirs:
- pkg/plugin/generated/*
severity:
default: error
# Default value is empty string.
# Set the default severity for issues. If severity rules are defined and the issues
# do not match or no severity is provided to the rule this will be the default
# severity applied. Severities should match the supported severity names of the
# selected out format.
# - Code climate: https://docs.codeclimate.com/docs/issues#issue-severity
# - Checkstyle: https://checkstyle.sourceforge.io/property_types.html#severity
# - Github: https://help.github.com/en/actions/reference/workflow-commands-for-github-actions#setting-an-error-message
default-severity: error
# The default value is false.
# If set to true severity-rules regular expressions become case sensitive.
case-sensitive: false
# Default value is empty list.
# When a list of severity rules are provided, severity information will be added to lint
@@ -435,7 +396,5 @@ severity:
# Only affects out formats that support setting severity information.
rules:
- linters:
- dupl
- dupl
severity: info
version: "2"

View File

@@ -26,23 +26,18 @@ builds:
- arm
- arm64
- ppc64le
- s390x
ignore:
# don't build arm for darwin and arm/arm64 for windows
- goos: darwin
goarch: arm
- goos: darwin
goarch: ppc64le
- goos: darwin
goarch: s390x
- goos: windows
goarch: arm
- goos: windows
goarch: arm64
- goos: windows
goarch: ppc64le
- goos: windows
goarch: s390x
ldflags:
- -X "github.com/vmware-tanzu/velero/pkg/buildinfo.Version={{ .Tag }}" -X "github.com/vmware-tanzu/velero/pkg/buildinfo.GitSHA={{ .FullCommit }}" -X "github.com/vmware-tanzu/velero/pkg/buildinfo.GitTreeState={{ .Env.GIT_TREE_STATE }}" -X "github.com/vmware-tanzu/velero/pkg/buildinfo.ImageRegistry={{ .Env.REGISTRY }}"
archives:
@@ -55,7 +50,7 @@ checksum:
name_template: 'CHECKSUM'
release:
github:
owner: velero-io
owner: vmware-tanzu
name: velero
draft: true
prerelease: auto
@@ -65,4 +60,4 @@ git:
# tags if there are more than one tag in the same commit.
#
# Default: `-version:refname`
tag_sort: -version:creatordate
tag_sort: -version:creatordate

View File

@@ -17,7 +17,6 @@ If you're using Velero and want to add your organization to this list,
<a href="https://www.replicated.com/" border="0" target="_blank"><img alt="replicated.com" src="site/static/img/adopters/replicated-logo-red.svg" height="50"></a>
<a href="https://cloudcasa.io/" border="0" target="_blank"><img alt="cloudcasa.io" src="site/static/img/adopters/cloudcasa.svg" height="50"></a>
<a href="https://azure.microsoft.com/" border="0" target="_blank"><img alt="azure.com" src="site/static/img/adopters/azure.svg" height="50"></a>
<a href="https://www.broadcom.com/" border="0" target="_blank"><img alt="broadcom.com" src="site/static/img/adopters/broadcom.svg" height="50"></a>
## Success Stories
Below is a list of adopters of Velero in **production environments** that have
@@ -69,9 +68,6 @@ Replicated uses the Velero open source project to enable snapshots in [KOTS][101
**[Microsoft Azure][105]**<br>
[Azure Backup for AKS][106] is an Azure native, Kubernetes aware, Enterprise ready backup for containerized applications deployed on Azure Kubernetes Service (AKS). AKS Backup utilizes Velero to perform backup and restore operations to protect stateful applications in AKS clusters.<br>
**[Broadcom][107]**<br>
[VMware Cloud Foundation][108] (VCF) offers built-in [vSphere Kubernetes Service][109] (VKS), a Kubernetes runtime that includes a CNCF certified Kubernetes distribution, to deploy and manage containerized workloads. VCF empowers platform engineers with native [Kubernetes multi-cluster management][110] capability for managing Kubernetes (K8s) infrastructure at scale. VCF utilizes Velero for Kubernetes data protection enabling platform engineers to back up and restore containerized workloads manifests & persistent volumes, helping to increase the resiliency of stateful applications in VKS cluster.
## Adding your organization to the list of Velero Adopters
If you are using Velero and would like to be included in the list of `Velero Adopters`, add an SVG version of your logo to the `site/static/img/adopters` directory in this repo and submit a [pull request][3] with your change. Name the image file something that reflects your company (e.g., if your company is called Acme, name the image acme.png). See this for an example [PR][4].
@@ -129,8 +125,3 @@ If you would like to add your logo to a future `Adopters of Velero` section on [
[105]: https://azure.microsoft.com/
[106]: https://learn.microsoft.com/azure/backup/backup-overview
[107]: https://www.broadcom.com/
[108]: https://www.vmware.com/products/cloud-infrastructure/vmware-cloud-foundation
[109]: https://www.vmware.com/products/cloud-infrastructure/vsphere-kubernetes-service
[110]: https://blogs.vmware.com/cloud-foundation/2025/09/29/empowering-platform-engineers-with-native-kubernetes-multi-cluster-management-in-vmware-cloud-foundation/

View File

@@ -1,23 +1,3 @@
# Velero Code of Conduct
Velero is a [Cloud Native Computing Foundation](https://www.cncf.io/) sandbox
project. As a CNCF project, the Velero community follows the
[**CNCF Code of Conduct**](https://github.com/cncf/foundation/blob/main/code-of-conduct.md).
The text below is the project's adopted Code of Conduct, based on the
[Contributor Covenant](https://www.contributor-covenant.org/), and is
substantively aligned with the CNCF Code of Conduct. Where any conflict exists,
the CNCF Code of Conduct prevails.
Instances of unacceptable behavior may be reported to the CNCF Code of
Conduct Committee at [conduct@cncf.io](mailto:conduct@cncf.io). For more
detailed instructions on how to submit a report, including how to submit a
report anonymously, please see the CNCF
[Incident Resolution Procedures](https://github.com/cncf/foundation/blob/main/code-of-conduct/coc-incident-resolution-procedures.md).
You can expect a response within three business days.
---
# Contributor Covenant Code of Conduct
## Our Pledge
@@ -79,8 +59,7 @@ representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the CNCF Code of Conduct Committee at
[conduct@cncf.io](mailto:conduct@cncf.io).
reported to the community leaders responsible for enforcement at oss-coc@vmware.com.
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the

View File

@@ -13,7 +13,7 @@
# limitations under the License.
# Velero binary build section
FROM --platform=$BUILDPLATFORM golang:1.26-trixie AS velero-builder
FROM --platform=$BUILDPLATFORM golang:1.22.10-bookworm AS velero-builder
ARG GOPROXY
ARG BIN
@@ -42,17 +42,41 @@ RUN mkdir -p /output/usr/bin && \
export GOARM=$( echo "${GOARM}" | cut -c2-) && \
go build -o /output/${BIN} \
-ldflags "${LDFLAGS}" ${PKG}/cmd/${BIN} && \
go build -o /output/velero-restore-helper \
-ldflags "${LDFLAGS}" ${PKG}/cmd/velero-restore-helper && \
go build -o /output/velero-helper \
-ldflags "${LDFLAGS}" ${PKG}/cmd/velero-helper && \
go clean -modcache -cache
# Restic binary build section
FROM --platform=$BUILDPLATFORM golang:1.22.10-bookworm AS restic-builder
ARG BIN
ARG TARGETOS
ARG TARGETARCH
ARG TARGETVARIANT
ARG RESTIC_VERSION
ENV CGO_ENABLED=0 \
GO111MODULE=on \
GOPROXY=${GOPROXY} \
GOOS=${TARGETOS} \
GOARCH=${TARGETARCH} \
GOARM=${TARGETVARIANT}
COPY . /go/src/github.com/vmware-tanzu/velero
RUN mkdir -p /output/usr/bin && \
export GOARM=$(echo "${GOARM}" | cut -c2-) && \
/go/src/github.com/vmware-tanzu/velero/hack/build-restic.sh && \
go clean -modcache -cache
# Velero image packing section
FROM paketobuildpacks/run-jammy-tiny:latest
FROM paketobuildpacks/run-jammy-tiny:0.2.56
LABEL maintainer="Xun Jiang <jxun@vmware.com>"
COPY --from=velero-builder /output /
COPY --from=restic-builder /output /
USER cnb:cnb

View File

@@ -1,57 +0,0 @@
# Copyright the Velero contributors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
ARG OS_VERSION=1809
# Velero binary build section
FROM --platform=$BUILDPLATFORM golang:1.26-trixie AS velero-builder
ARG GOPROXY
ARG BIN
ARG PKG
ARG VERSION
ARG REGISTRY
ARG GIT_SHA
ARG GIT_TREE_STATE
ARG TARGETOS
ARG TARGETARCH
ARG TARGETVARIANT
ENV CGO_ENABLED=0 \
GO111MODULE=on \
GOPROXY=${GOPROXY} \
GOOS=${TARGETOS} \
GOARCH=${TARGETARCH} \
GOARM=${TARGETVARIANT} \
LDFLAGS="-X ${PKG}/pkg/buildinfo.Version=${VERSION} -X ${PKG}/pkg/buildinfo.GitSHA=${GIT_SHA} -X ${PKG}/pkg/buildinfo.GitTreeState=${GIT_TREE_STATE} -X ${PKG}/pkg/buildinfo.ImageRegistry=${REGISTRY}"
WORKDIR /go/src/github.com/vmware-tanzu/velero
COPY . /go/src/github.com/vmware-tanzu/velero
RUN mkdir -p /output/usr/bin && \
export GOARM=$( echo "${GOARM}" | cut -c2-) && \
go build -o /output/${BIN}.exe \
-ldflags "${LDFLAGS}" ${PKG}/cmd/${BIN} && \
go build -o /output/velero-restore-helper.exe \
-ldflags "${LDFLAGS}" ${PKG}/cmd/velero-restore-helper && \
go build -o /output/velero-helper.exe \
-ldflags "${LDFLAGS}" ${PKG}/cmd/velero-helper && \
go clean -modcache -cache
# Velero image packing section
FROM mcr.microsoft.com/windows/nanoserver:${OS_VERSION}
COPY --from=velero-builder /output /
USER ContainerUser

View File

@@ -7,13 +7,13 @@
| Maintainer | GitHub ID | Affiliation |
|---------------------|---------------------------------------------------------------|--------------------------------------------------|
| Scott Seago | [sseago](https://github.com/sseago) | [OpenShift](https://github.com/openshift) |
| Daniel Jiang | [reasonerjt](https://github.com/reasonerjt) | Broadcom |
| Wenkai Yin | [ywk253100](https://github.com/ywk253100) | Broadcom |
| Xun Jiang | [blackpiglet](https://github.com/blackpiglet) | Broadcom |
| Daniel Jiang | [reasonerjt](https://github.com/reasonerjt) | [VMware](https://www.github.com/vmware/) |
| Wenkai Yin | [ywk253100](https://github.com/ywk253100) | [VMware](https://www.github.com/vmware/) |
| Xun Jiang | [blackpiglet](https://github.com/blackpiglet) | [VMware](https://www.github.com/vmware/) |
| Ming Qiu | [qiuming-best](https://github.com/qiuming-best) | [VMware](https://www.github.com/vmware/) |
| Shubham Pampattiwar | [shubham-pampattiwar](https://github.com/shubham-pampattiwar) | [OpenShift](https://github.com/openshift) |
| Yonghui Li | [Lyndon-Li](https://github.com/Lyndon-Li) | Broadcom |
| Yonghui Li | [Lyndon-Li](https://github.com/Lyndon-Li) | [VMware](https://www.github.com/vmware/) |
| Anshul Ahuja | [anshulahuja98](https://github.com/anshulahuja98) | [Microsoft Azure](https://www.github.com/azure/) |
| Tiger Kaovilai | [kaovilai](https://github.com/kaovilai) | [OpenShift](https://github.com/openshift) |
## Emeritus Maintainers
* Adnan Abdulhussein ([prydonius](https://github.com/prydonius))
@@ -26,4 +26,14 @@
* Bridget McErlean ([zubron](https://github.com/zubron))
* JenTing Hsiao ([jenting](https://github.com/jenting))
* Dave Smith-Uchida ([dsu-igeek](https://github.com/dsu-igeek))
* Ming Qiu ([qiuming-best](https://github.com/qiuming-best))
## Velero Contributors & Stakeholders
| Feature Area | Lead |
|------------------------|:------------------------------------------------------------------------------------:|
| Technical Lead | Daniel Jiang [reasonerjt](https://github.com/reasonerjt) |
| Kubernetes CSI Liaison | |
| Deployment | |
| Community Management | Orlin Vasilev [OrlinVasilev](https://github.com/OrlinVasilev) |
| Product Management | Pradeep Kumar Chaturvedi [pradeepkchaturvedi](https://github.com/pradeepkchaturvedi) |

148
Makefile
View File

@@ -22,26 +22,15 @@ PKG := github.com/vmware-tanzu/velero
# Where to push the docker image.
REGISTRY ?= velero
# In order to push images to an insecure registry, follow the two steps:
# 1. Set "INSECURE_REGISTRY=true"
# 2. Provide your own buildx builder instance by setting "BUILDX_INSTANCE=your-own-builder-instance"
# The builder can be created with the following command:
# cat << EOF > buildkitd.toml
# [registry."insecure-registry-ip:port"]
# http = true
# insecure = true
# EOF
# docker buildx create --name=velero-builder --driver=docker-container --bootstrap --use --config ./buildkitd.toml
# Refer to https://github.com/docker/buildx/issues/1370#issuecomment-1288516840 for more details
INSECURE_REGISTRY ?= false
GCR_REGISTRY ?= gcr.io/velero-gcp
# Image name
IMAGE ?= $(REGISTRY)/$(BIN)
GCR_IMAGE ?= $(GCR_REGISTRY)/$(BIN)
# We allow the Dockerfile to be configurable to enable the use of custom Dockerfiles
# that pull base images from different registries.
VELERO_DOCKERFILE ?= Dockerfile
VELERO_DOCKERFILE_WINDOWS ?= Dockerfile-Windows
BUILDER_IMAGE_DOCKERFILE ?= hack/build-image/Dockerfile
# Calculate the realpath of the build-image Dockerfile as we `cd` into the hack/build
@@ -65,7 +54,7 @@ endif
BUILDER_IMAGE := $(REGISTRY)/build-image:$(BUILDER_IMAGE_TAG)
BUILDER_IMAGE_CACHED := $(shell docker images -q ${BUILDER_IMAGE} 2>/dev/null )
HUGO_IMAGE := ghcr.io/gohugoio/hugo
HUGO_IMAGE := hugo-builder
# Which architecture to build - see $(ALL_ARCH) for options.
# if the 'local' rule is being run, detect the ARCH from 'go env'
@@ -79,8 +68,10 @@ TAG_LATEST ?= false
ifeq ($(TAG_LATEST), true)
IMAGE_TAGS ?= $(IMAGE):$(VERSION) $(IMAGE):latest
GCR_IMAGE_TAGS ?= $(GCR_IMAGE):$(VERSION) $(GCR_IMAGE):latest
else
IMAGE_TAGS ?= $(IMAGE):$(VERSION)
GCR_IMAGE_TAGS ?= $(GCR_IMAGE):$(VERSION)
endif
# check buildx is enabled only if docker is in path
@@ -103,30 +94,13 @@ define BUILDX_ERROR
buildx not enabled, refusing to run this recipe
see: https://velero.io/docs/main/build-from-source/#making-images-and-updating-velero for more info
endef
# comma cannot be escaped and can only be used in Make function arguments by putting into variable
comma=,
CLI_PLATFORMS ?= linux-amd64 linux-arm linux-arm64 darwin-amd64 darwin-arm64 windows-amd64 linux-ppc64le linux-s390x
BUILD_OUTPUT_TYPE ?= docker
BUILD_OS ?= linux
BUILD_ARCH ?= amd64
BUILD_WINDOWS_VERSION ?= ltsc2022
# The version of restic binary to be downloaded
RESTIC_VERSION ?= 0.15.0
ifeq ($(BUILD_OUTPUT_TYPE), docker)
ALL_OS = linux
ALL_ARCH.linux = $(word 2, $(subst -, ,$(shell go env GOOS)-$(shell go env GOARCH)))
else
ALL_OS = $(subst $(comma), ,$(BUILD_OS))
ALL_ARCH.linux = $(subst $(comma), ,$(BUILD_ARCH))
endif
ALL_ARCH.windows = $(if $(filter windows,$(ALL_OS)),amd64,)
ALL_OSVERSIONS.windows = $(if $(filter windows,$(ALL_OS)),$(BUILD_WINDOWS_VERSION),)
ALL_OS_ARCH.linux = $(foreach os, $(filter linux,$(ALL_OS)), $(foreach arch, ${ALL_ARCH.linux}, ${os}-$(arch)))
ALL_OS_ARCH.windows = $(foreach os, $(filter windows,$(ALL_OS)), $(foreach arch, $(ALL_ARCH.windows), $(foreach osversion, ${ALL_OSVERSIONS.windows}, ${os}-${osversion}-${arch})))
ALL_OS_ARCH = $(ALL_OS_ARCH.linux)$(ALL_OS_ARCH.windows)
ALL_IMAGE_TAGS = $(IMAGE_TAGS)
CLI_PLATFORMS ?= linux-amd64 linux-arm linux-arm64 darwin-amd64 darwin-arm64 windows-amd64 linux-ppc64le
BUILDX_PLATFORMS ?= $(subst -,/,$(ARCH))
BUILDX_OUTPUT_TYPE ?= docker
# set git sha and tree state
GIT_SHA = $(shell git rev-parse HEAD)
@@ -150,14 +124,17 @@ GOBIN=$$(pwd)/.go/bin
# If you want to build all containers, see the 'all-containers' rule.
all:
@$(MAKE) build
@$(MAKE) build BIN=velero-restore-helper
build-%:
@$(MAKE) --no-print-directory ARCH=$* build
@$(MAKE) --no-print-directory ARCH=$* build BIN=velero-restore-helper
all-build: $(addprefix build-, $(CLI_PLATFORMS))
all-containers:
@$(MAKE) --no-print-directory container
@$(MAKE) --no-print-directory container BIN=velero-restore-helper
local: build-dirs
# Add DEBUG=1 to enable debug locally
@@ -211,7 +188,6 @@ shell: build-dirs build-env
-v "$$(pwd)/.go/std/$(GOOS)/$(GOARCH):/usr/local/go/pkg/$(GOOS)_$(GOARCH)_static:delegated" \
-v "$$(pwd)/.go/go-build:/.cache/go-build:delegated" \
-v "$$(pwd)/.go/golangci-lint:/.cache/golangci-lint:delegated" \
-v "$$(pwd)/.go/goimports:/.cache/goimports:delegated" \
-w /github.com/vmware-tanzu/velero \
$(BUILDER_IMAGE) \
/bin/sh $(CMD)
@@ -220,38 +196,11 @@ container:
ifneq ($(BUILDX_ENABLED), true)
$(error $(BUILDX_ERROR))
endif
ifeq ($(BUILDX_INSTANCE),)
@echo creating a buildx instance
-docker buildx rm velero-builder || true
@docker buildx create --use --name=velero-builder
else
@echo using a specified buildx instance $(BUILDX_INSTANCE)
@docker buildx use $(BUILDX_INSTANCE)
endif
@mkdir -p _output
@for osarch in $(ALL_OS_ARCH); do \
$(MAKE) container-$${osarch}; \
done
ifeq ($(BUILD_OUTPUT_TYPE), registry)
@for tag in $(ALL_IMAGE_TAGS); do \
IMAGE_TAG=$${tag} $(MAKE) push-manifest; \
done
endif
container-linux-%:
@BUILDX_ARCH=$* $(MAKE) container-linux
container-linux:
@echo "building container: $(IMAGE):$(VERSION)-linux-$(BUILDX_ARCH)"
@docker buildx build --pull \
--output="type=$(BUILD_OUTPUT_TYPE)$(if $(findstring tar, $(BUILD_OUTPUT_TYPE)),$(comma)dest=_output/$(BIN)-$(VERSION)-linux-$(BUILDX_ARCH).tar,)" \
--platform="linux/$(BUILDX_ARCH)" \
$(addprefix -t , $(addsuffix "-linux-$(BUILDX_ARCH)",$(ALL_IMAGE_TAGS))) \
--output=type=$(BUILDX_OUTPUT_TYPE) \
--platform $(BUILDX_PLATFORMS) \
$(addprefix -t , $(IMAGE_TAGS)) \
$(addprefix -t , $(GCR_IMAGE_TAGS)) \
--build-arg=GOPROXY=$(GOPROXY) \
--build-arg=PKG=$(PKG) \
--build-arg=BIN=$(BIN) \
@@ -259,54 +208,15 @@ container-linux:
--build-arg=GIT_SHA=$(GIT_SHA) \
--build-arg=GIT_TREE_STATE=$(GIT_TREE_STATE) \
--build-arg=REGISTRY=$(REGISTRY) \
--provenance=false \
--sbom=false \
--build-arg=RESTIC_VERSION=$(RESTIC_VERSION) \
-f $(VELERO_DOCKERFILE) .
@echo "built container: $(IMAGE):$(VERSION)-linux-$(BUILDX_ARCH)"
container-windows-%:
@BUILDX_OSVERSION=$(firstword $(subst -, ,$*)) BUILDX_ARCH=$(lastword $(subst -, ,$*)) $(MAKE) container-windows
container-windows:
@echo "building container: $(IMAGE):$(VERSION)-windows-$(BUILDX_OSVERSION)-$(BUILDX_ARCH)"
@docker buildx build --pull \
--output="type=$(BUILD_OUTPUT_TYPE)$(if $(findstring tar, $(BUILD_OUTPUT_TYPE)),$(comma)dest=_output/$(BIN)-$(VERSION)-windows-$(BUILDX_OSVERSION)-$(BUILDX_ARCH).tar,)" \
--platform="windows/$(BUILDX_ARCH)" \
$(addprefix -t , $(addsuffix "-windows-$(BUILDX_OSVERSION)-$(BUILDX_ARCH)",$(ALL_IMAGE_TAGS))) \
--build-arg=GOPROXY=$(GOPROXY) \
--build-arg=PKG=$(PKG) \
--build-arg=BIN=$(BIN) \
--build-arg=VERSION=$(VERSION) \
--build-arg=OS_VERSION=$(BUILDX_OSVERSION) \
--build-arg=GIT_SHA=$(GIT_SHA) \
--build-arg=GIT_TREE_STATE=$(GIT_TREE_STATE) \
--build-arg=REGISTRY=$(REGISTRY) \
--provenance=false \
--sbom=false \
-f $(VELERO_DOCKERFILE_WINDOWS) .
@echo "built container: $(IMAGE):$(VERSION)-windows-$(BUILDX_OSVERSION)-$(BUILDX_ARCH)"
push-manifest:
@echo "building manifest: $(IMAGE_TAG) for $(foreach osarch, $(ALL_OS_ARCH), $(IMAGE_TAG)-${osarch})"
@docker manifest create --amend --insecure=$(INSECURE_REGISTRY) $(IMAGE_TAG) $(foreach osarch, $(ALL_OS_ARCH), $(IMAGE_TAG)-${osarch})
@set -x; \
for arch in $(ALL_ARCH.windows); do \
for osversion in $(ALL_OSVERSIONS.windows); do \
BASEIMAGE=mcr.microsoft.com/windows/nanoserver:$${osversion}; \
full_version=`docker manifest inspect --insecure=$(INSECURE_REGISTRY) $${BASEIMAGE} | jq -r '.manifests[0].platform["os.version"]'`; \
docker manifest annotate --os windows --arch $${arch} --os-version $${full_version} $(IMAGE_TAG) $(IMAGE_TAG)-windows-$${osversion}-$${arch}; \
done; \
done
@echo "pushing manifest $(IMAGE_TAG)"
@docker manifest push --purge --insecure=$(INSECURE_REGISTRY) $(IMAGE_TAG)
@echo "pushed manifest $(IMAGE_TAG):"
@docker manifest inspect --insecure=$(INSECURE_REGISTRY) $(IMAGE_TAG)
@echo "container: $(IMAGE):$(VERSION)"
ifeq ($(BUILDX_OUTPUT_TYPE)_$(REGISTRY), registry_velero)
docker pull $(IMAGE):$(VERSION)
rm -f $(BIN)-$(VERSION).tar
docker save $(IMAGE):$(VERSION) -o $(BIN)-$(VERSION).tar
gzip -f $(BIN)-$(VERSION).tar
endif
SKIP_TESTS ?=
test: build-dirs
@@ -343,7 +253,7 @@ update-crd:
build-dirs:
@mkdir -p _output/bin/$(GOOS)/$(GOARCH)
@mkdir -p .go/src/$(PKG) .go/pkg .go/bin .go/std/$(GOOS)/$(GOARCH) .go/go-build .go/golangci-lint .go/goimports
@mkdir -p .go/src/$(PKG) .go/pkg .go/bin .go/std/$(GOOS)/$(GOARCH) .go/go-build .go/golangci-lint
build-env:
@# if we have overridden the value for the build-image Dockerfile,
@@ -449,7 +359,7 @@ release:
serve-docs: build-image-hugo
docker run \
--rm \
-v "$$(pwd)/site:/project" \
-v "$$(pwd)/site:/srv/hugo" \
-it -p 1313:1313 \
$(HUGO_IMAGE) \
server --bind=0.0.0.0 --enableGitInfo=false
@@ -477,7 +387,7 @@ go-generate:
# make new-changelog CHANGELOG_BODY="Changes you have made"
new-changelog: GH_LOGIN ?= $(shell gh pr view --json author --jq .author.login 2> /dev/null)
new-changelog: GH_PR_NUMBER ?= $(shell gh pr view --json number --jq .number 2> /dev/null)
new-changelog: CHANGELOG_BODY ?= '$(shell gh pr view --json title --jq .title)'
new-changelog: CHANGELOG_BODY ?= "$(shell gh pr view --json title --jq .title)"
new-changelog:
@if [ "$(GH_LOGIN)" = "" ]; then \
echo "branch does not have PR or cli not logged in, try 'gh auth login' or 'gh pr create'"; \
@@ -485,4 +395,4 @@ new-changelog:
fi
@mkdir -p ./changelogs/unreleased/ && \
echo $(CHANGELOG_BODY) > ./changelogs/unreleased/$(GH_PR_NUMBER)-$(GH_LOGIN) && \
echo \"$(CHANGELOG_BODY)\" added to "./changelogs/unreleased/$(GH_PR_NUMBER)-$(GH_LOGIN)"
echo "\"$(CHANGELOG_BODY)\" added to ./changelogs/unreleased/$(GH_PR_NUMBER)-$(GH_LOGIN)"

View File

@@ -42,11 +42,11 @@ The following is a list of the supported Kubernetes versions for each Velero ver
| Velero version | Expected Kubernetes version compatibility | Tested on Kubernetes version |
|----------------|-------------------------------------------|-------------------------------------|
| 1.18 | 1.18-latest | 1.33.7, 1.34.1, and 1.35.0 |
| 1.17 | 1.18-latest | 1.31.7, 1.32.3, 1.33.1, and 1.34.0 |
| 1.16 | 1.18-latest | 1.31.4, 1.32.3, and 1.33.0 |
| 1.15 | 1.18-latest | 1.28.8, 1.29.8, 1.30.4 and 1.31.1 |
| 1.14 | 1.18-latest | 1.27.9, 1.28.9, and 1.29.4 |
| 1.13 | 1.18-latest | 1.26.5, 1.27.3, 1.27.8, and 1.28.3 |
| 1.12 | 1.18-latest | 1.25.7, 1.26.5, 1.26.7, and 1.27.3 |
| 1.11 | 1.18-latest | 1.23.10, 1.24.9, 1.25.5, and 1.26.1 |
Velero supports IPv4, IPv6, and dual stack environments. Support for this was tested against Velero v1.8.

View File

@@ -12,13 +12,13 @@ The Velero project maintains the following [governance document](https://github.
Security is of the highest importance and all security vulnerabilities or suspected security vulnerabilities should be reported to Velero privately, to minimize attacks against current users of Velero before they are fixed. Vulnerabilities will be investigated and patched on the next patch (or minor) release as soon as possible. This information could be kept entirely internal to the project.
If you know of a publicly disclosed security vulnerability for Velero, please **IMMEDIATELY** contact the Security Team (velero-security.pdl@broadcom.com).
If you know of a publicly disclosed security vulnerability for Velero, please **IMMEDIATELY** contact the VMware Security Team (security@vmware.com).
**IMPORTANT: Do not file public issues on GitHub for security vulnerabilities**
To report a vulnerability or a security-related issue, please contact the email address with the details of the vulnerability. The email will be fielded by the Security Team and then shared with the Velero maintainers who have committer and release permissions. Emails will be addressed within 3 business days, including a detailed plan to investigate the issue and any potential workarounds to perform in the meantime. Do not report non-security-impacting bugs through this channel. Use [GitHub issues](https://github.com/vmware-tanzu/velero/issues/new/choose) instead.
To report a vulnerability or a security-related issue, please contact the VMware email address with the details of the vulnerability. The email will be fielded by the VMware Security Team and then shared with the Velero maintainers who have committer and release permissions. Emails will be addressed within 3 business days, including a detailed plan to investigate the issue and any potential workarounds to perform in the meantime. Do not report non-security-impacting bugs through this channel. Use [GitHub issues](https://github.com/vmware-tanzu/velero/issues/new/choose) instead.
## Proposed Email Content
@@ -29,7 +29,7 @@ Provide a descriptive subject line and in the body of the email include the foll
* Basic identity information, such as your name and your affiliation or company.
* Detailed steps to reproduce the vulnerability (POC scripts, screenshots, and logs are all helpful to us).
* Description of the effects of the vulnerability on Velero and the related hardware and software configurations, so that the Security Team can reproduce it.
* Description of the effects of the vulnerability on Velero and the related hardware and software configurations, so that the VMware Security Team can reproduce it.
* How the vulnerability affects Velero usage and an estimation of the attack surface, if there is one.
* List other projects or dependencies that were used in conjunction with Velero to produce the vulnerability.
@@ -49,7 +49,7 @@ Provide a descriptive subject line and in the body of the email include the foll
## Patch, Release, and Disclosure
The Security Team will respond to vulnerability reports as follows:
The VMware Security Team will respond to vulnerability reports as follows:
@@ -62,7 +62,7 @@ The Security Team will respond to vulnerability reports as follows:
5. The Security Team will also create a [CVSS](https://www.first.org/cvss/specification-document) using the [CVSS Calculator](https://www.first.org/cvss/calculator/3.0). The Security Team makes the final call on the calculated CVSS; it is better to move quickly than making the CVSS perfect. Issues may also be reported to [Mitre](https://cve.mitre.org/) using this [scoring calculator](https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator). The CVE will initially be set to private.
6. The Security Team will work on fixing the vulnerability and perform internal testing before preparing to roll out the fix.
7. The Security Team will provide early disclosure of the vulnerability by emailing the [Velero Distributors](https://groups.google.com/u/1/g/projectvelero-distributors) mailing list. Distributors can initially plan for the vulnerability patch ahead of the fix, and later can test the fix and provide feedback to the Velero team. See the section **Early Disclosure to Velero Distributors List** for details about how to join this mailing list.
8. A public disclosure date is negotiated by the SecurityTeam, the bug submitter, and the distributors list. We prefer to fully disclose the bug as soon as possible once a user mitigation or patch is available. It is reasonable to delay disclosure when the bug or the fix is not yet fully understood, the solution is not well-tested, or for distributor coordination. The timeframe for disclosure is from immediate (especially if its already publicly known) to a few weeks. For a critical vulnerability with a straightforward mitigation, we expect the report date for the public disclosure date to be on the order of 14 business days. The Security Team holds the final say when setting a public disclosure date.
8. A public disclosure date is negotiated by the VMware SecurityTeam, the bug submitter, and the distributors list. We prefer to fully disclose the bug as soon as possible once a user mitigation or patch is available. It is reasonable to delay disclosure when the bug or the fix is not yet fully understood, the solution is not well-tested, or for distributor coordination. The timeframe for disclosure is from immediate (especially if its already publicly known) to a few weeks. For a critical vulnerability with a straightforward mitigation, we expect the report date for the public disclosure date to be on the order of 14 business days. The VMware Security Team holds the final say when setting a public disclosure date.
9. Once the fix is confirmed, the Security Team will patch the vulnerability in the next patch or minor release, and backport a patch release into all earlier supported releases. Upon release of the patched version of Velero, we will follow the **Public Disclosure Process**.
@@ -79,7 +79,7 @@ The Security Team will also publish any mitigating steps users can take until th
* Use velero-security.pdl@broadcom.com to report security concerns to the Security Team, who uses the list to privately discuss security issues and fixes prior to disclosure.
* Use security@vmware.com to report security concerns to the VMware Security Team, who uses the list to privately discuss security issues and fixes prior to disclosure.
* Join the [Velero Distributors](https://groups.google.com/u/1/g/projectvelero-distributors) mailing list for early private information and vulnerability disclosure. Early disclosure may include mitigating steps and additional information on security patch releases. See below for information on how Velero distributors or vendors can apply to join this list.
@@ -107,11 +107,11 @@ To be eligible to join the [Velero Distributors](https://groups.google.com/u/1/g
## Embargo Policy
The information that members receive on the Velero Distributors mailing list must not be made public, shared, or even hinted at anywhere beyond those who need to know within your specific team, unless you receive explicit approval to do so from the Security Team. This remains true until the public disclosure date/time agreed upon by the list. Members of the list and others cannot use the information for any reason other than to get the issue fixed for your respective distribution's users.
The information that members receive on the Velero Distributors mailing list must not be made public, shared, or even hinted at anywhere beyond those who need to know within your specific team, unless you receive explicit approval to do so from the VMware Security Team. This remains true until the public disclosure date/time agreed upon by the list. Members of the list and others cannot use the information for any reason other than to get the issue fixed for your respective distribution's users.
Before you share any information from the list with members of your team who are required to fix the issue, these team members must agree to the same terms, and only be provided with information on a need-to-know basis.
In the unfortunate event that you share information beyond what is permitted by this policy, you must urgently inform the Security Team (velero-security.pdl@broadcom.com) of exactly what information was leaked and to whom. If you continue to leak information and break the policy outlined here, you will be permanently removed from the list.
In the unfortunate event that you share information beyond what is permitted by this policy, you must urgently inform the VMware Security Team (security@vmware.com) of exactly what information was leaked and to whom. If you continue to leak information and break the policy outlined here, you will be permanently removed from the list.
@@ -123,6 +123,6 @@ Send new membership requests to projectvelero-distributors@googlegroups.com. In
## Confidentiality, integrity and availability
We consider vulnerabilities leading to the compromise of data confidentiality, elevation of privilege, or integrity to be our highest priority concerns. Availability, in particular in areas relating to DoS and resource exhaustion, is also a serious security concern. The Security Team takes all vulnerabilities, potential vulnerabilities, and suspected vulnerabilities seriously and will investigate them in an urgent and expeditious manner.
We consider vulnerabilities leading to the compromise of data confidentiality, elevation of privilege, or integrity to be our highest priority concerns. Availability, in particular in areas relating to DoS and resource exhaustion, is also a serious security concern. The VMware Security Team takes all vulnerabilities, potential vulnerabilities, and suspected vulnerabilities seriously and will investigate them in an urgent and expeditious manner.
Note that we do not currently consider the default settings for Velero to be secure-by-default. It is necessary for operators to explicitly configure settings, role based access control, and other resource related features in Velero to provide a hardened Velero environment. We will not act on any security disclosure that relates to a lack of safe defaults. Over time, we will work towards improved safe-by-default configuration, taking into account backwards compatibility.

View File

@@ -52,7 +52,7 @@ git_sha = str(local("git rev-parse HEAD", quiet = True, echo_off = True)).strip(
tilt_helper_dockerfile_header = """
# Tilt image
FROM golang:1.26 as tilt-helper
FROM golang:1.22.10 as tilt-helper
# Support live reloading with Tilt
RUN wget --output-document /restart.sh --quiet https://raw.githubusercontent.com/windmilleng/rerun-process-wrapper/master/restart.sh && \
@@ -103,6 +103,11 @@ local_resource(
deps = ["internal", "pkg/cmd"],
)
local_resource(
"restic_binary",
cmd = 'cd ' + '.' + ';mkdir -p _tiltbuild/restic; BIN=velero GOOS=linux GOARCH=amd64 GOARM="" RESTIC_VERSION=0.13.1 OUTPUT_DIR=_tiltbuild/restic ./hack/build-restic.sh',
)
# Note: we need a distro with a bash shell to exec into the Velero container
tilt_dockerfile_header = """
FROM ubuntu:22.04 as tilt
@@ -113,6 +118,7 @@ WORKDIR /
COPY --from=tilt-helper /start.sh .
COPY --from=tilt-helper /restart.sh .
COPY velero .
COPY restic/restic /usr/bin/restic
"""
dockerfile_contents = "\n".join([

View File

@@ -1,3 +1,31 @@
## v1.15.1
### Download
https://github.com/vmware-tanzu/velero/releases/tag/v1.15.1
### Container Image
`velero/velero:v1.15.1`
### Documentation
https://velero.io/docs/v1.15/
### Upgrading
https://velero.io/docs/v1.15/upgrade-to-1.15/
### All Changes
* Fix backup post hook issue #8159 (caused by #7571): always execute backup post hooks after PVBs are handled (#8517, @ywk253100)
* Fix issue #8125, log diagnostic info for data mover exposers when expose timeout (#8511, @Lyndon-Li)
* Set hinting region to use for GetBucketRegion() in pkg/repository/config/aws.go (#8505, @kaovilai)
* Fix issue #8433, add third party labels to data mover pods when the same labels exist in node-agent pods (#8501, @Lyndon-Li)
* Fix issue #8485, add an accepted time so as to count the prepare timeout (#8496, @Lyndon-Li)
* Add SecurityContext to restore-helper (#8495, @reasonerjt)
* Add nil check for updating DataUpload VolumeInfo in finalizing phase. (#8465, @blackpiglet)
* Fix issue #8391, check ErrCancelled from suffix of data mover pod's termination message (#8404, @Lyndon-Li)
* Fix issue #8394, don't call closeDataPath in VGDP callbacks, otherwise, the VGDP cleanup will hang (#8402, @Lyndon-Li)
* Reduce minimum required go toolchain in release-1.15 go.mod (#8399, @kaovilai)
* Fix issue #8539, validate uploader types when o.CRDsOnly is set to false only since CRD installation doesn't rely on uploader types (#8540, @Lyndon-Li)
## v1.15
### Download

View File

@@ -1,156 +0,0 @@
## v1.16
### Download
https://github.com/vmware-tanzu/velero/releases/tag/v1.16.0
### Container Image
`velero/velero:v1.16.0`
### Documentation
https://velero.io/docs/v1.16/
### Upgrading
https://velero.io/docs/v1.16/upgrade-to-1.16/
### Highlights
#### Windows cluster support
In v1.16, Velero supports to run in Windows clusters and backup/restore Windows workloads, either stateful or stateless:
* Hybrid build and all-in-one image: the build process is enhanced to build an all-in-one image for hybrid CPU architecture and hybrid platform. For more information, check the design https://github.com/vmware-tanzu/velero/blob/main/design/multiple-arch-build-with-windows.md
* Deployment in Windows clusters: Velero node-agent, data mover pods and maintenance jobs now support to run in both linux and Windows nodes
* Data mover backup/restore Windows workloads: Velero built-in data mover supports Windows workloads throughout its full cycle, i.e., discovery, backup, restore, pre/post hook, etc. It automatically identifies Windows workloads and schedules data mover pods to the right group of nodes
Check the epic issue https://github.com/vmware-tanzu/velero/issues/8289 for more information.
#### Parallel Item Block backup
v1.16 now supports to back up item blocks in parallel. Specifically, during backup, correlated resources are grouped in item blocks and Velero backup engine creates a thread pool to back up the item blocks in parallel. This significantly improves the backup throughput, especially when there are large scale of resources.
Pre/post hooks also belongs to item blocks, so will also run in parallel along with the item blocks.
Users are allowed to configure the parallelism through the `--item-block-worker-count` Velero server parameter. If not configured, the default parallelism is 1.
For more information, check issue https://github.com/vmware-tanzu/velero/issues/8334.
#### Data mover restore enhancement in scalability
In previous releases, for each volume of WaitForFirstConsumer mode, data mover restore is only allowed to happen in the node that the volume is attached. This severely degrades the parallelism and the balance of node resource(CPU, memory, network bandwidth) consumption for data mover restore (https://github.com/vmware-tanzu/velero/issues/8044).
In v1.16, users are allowed to configure data mover restores running and spreading evenly across all nodes in the cluster. The configuration is done through a new flag `ignoreDelayBinding` in node-agent configuration (https://github.com/vmware-tanzu/velero/issues/8242).
#### Data mover enhancements in observability
In 1.16, some observability enhancements are added:
* Output various statuses of intermediate objects for failures of data mover backup/restore (https://github.com/vmware-tanzu/velero/issues/8267)
* Output the errors when Velero fails to delete intermediate objects during clean up (https://github.com/vmware-tanzu/velero/issues/8125)
The outputs are in the same node-agent log and enabled automatically.
#### CSI snapshot backup/restore enhancement in usability
In previous releases, a unnecessary VolumeSnapshotContent object is retained for each backup and synced to other clusters sharing the same backup storage location. And during restore, the retained VolumeSnapshotContent is also restored unnecessarily.
In 1.16, the retained VolumeSnapshotContent is removed from the backup, so no unnecessary CSI objects are synced or restored.
For more information, check issue https://github.com/vmware-tanzu/velero/issues/8725.
#### Backup Repository Maintenance enhancement in resiliency and observability
In v1.16, some enhancements of backup repository maintenance are added to improve the observability and resiliency:
* A new backup repository maintenance history section, called `RecentMaintenance`, is added to the BackupRepository CR. Specifically, for each BackupRepository, including start/completion time, completion status and error message. (https://github.com/vmware-tanzu/velero/issues/7810)
* Running maintenance jobs are now recaptured after Velero server restarts. (https://github.com/vmware-tanzu/velero/issues/7753)
* The maintenance job will not be launched for readOnly BackupStorageLocation. (https://github.com/vmware-tanzu/velero/issues/8238)
* The backup repository will not try to initialize a new repository for readOnly BackupStorageLocation. (https://github.com/vmware-tanzu/velero/issues/8091)
* Users now are allowed to configure the intervals of an effective maintenance in the way of `normalGC`, `fastGC` and `eagerGC`, through the `fullMaintenanceInterval` parameter in backupRepository configuration. (https://github.com/vmware-tanzu/velero/issues/8364)
#### Volume Policy enhancement of filtering volumes by PVC labels
In v1.16, Volume Policy is extended to support filtering volumes by PVC labels. (https://github.com/vmware-tanzu/velero/issues/8256).
#### Resource Status restore per object
In v1.16, users are allowed to define whether to restore resource status per object through an annotation `velero.io/restore-status` set on the object. (https://github.com/vmware-tanzu/velero/issues/8204).
#### Velero Restore Helper binary is merged into Velero image
In v1.16, Velero banaries, i.e., velero, velero-helper and velero-restore-helper, are all included into the single Velero image. (https://github.com/vmware-tanzu/velero/issues/8484).
### Runtime and dependencies
Golang runtime: 1.23.7
kopia: 0.19.0
### Limitations/Known issues
#### Limitations of Windows support
* fs-backup is not supported for Windows workloads and so fs-backup runs only in linux nodes for linux workloads
* Backup/restore of NTFS extended attributes/advanced features are not supported, i.e., Security Descriptors, System/Hidden/ReadOnly attributes, Creation Time, NTFS Streams, etc.
### All Changes
* Add third party annotation support for maintenance job, so that the declared third party annotations could be added to the maintenance job pods (#8812, @Lyndon-Li)
* Fix issue #8803, use deterministic name to create backupRepository (#8808, @Lyndon-Li)
* Refactor restoreItem and related functions to differentiate the backup resource name and the restore target resource name. (#8797, @blackpiglet)
* ensure that PV is removed before VS is deleted (#8777, @ix-rzi)
* host_pods should not be mandatory to node-agent (#8774, @mpryc)
* Log doesn't show pv name, but displays %!s(MISSING) instead (#8771, @hu-keyu)
* Fix issue #8754, add third party annotation support for data mover (#8770, @Lyndon-Li)
* Add docs for volume policy with labels as a criteria (#8759, @shubham-pampattiwar)
* Move pvc annotation removal from CSI RIA to regular PVC RIA (#8755, @sseago)
* Add doc for maintenance history (#8747, @Lyndon-Li)
* Fix issue #8733, add doc for restorePVC (#8737, @Lyndon-Li)
* Fix issue #8426, add doc for Windows support (#8736, @Lyndon-Li)
* Fix issue #8475, refactor build-from-source doc for hybrid image build (#8729, @Lyndon-Li)
* Return directly if no pod volme backup are tracked (#8728, @ywk253100)
* Fix issue #8706, for immediate volumes, there is no selected-node annotation on PVC, so deduce the attached node from VolumeAttachment CRs (#8715, @Lyndon-Li)
* Add labels as a criteria for volume policy (#8713, @shubham-pampattiwar)
* Copy SecurityContext from Containers[0] if present for PVR (#8712, @sseago)
* Support pushing images to an insecure registry (#8703, @ywk253100)
* Modify golangci configuration to make it work. (#8695, @blackpiglet)
* Run backup post hooks inside ItemBlock synchronously (#8694, @ywk253100)
* Add docs for object level status restore (#8693, @shubham-pampattiwar)
* Clean artifacts generated during CSI B/R. (#8684, @blackpiglet)
* Don't run maintenance on the ReadOnly BackupRepositories. (#8681, @blackpiglet)
* Fix #8657: WaitGroup panic issue (#8679, @ywk253100)
* Fixes issue #8214, validate `--from-schedule` flag in create backup command to prevent empty or whitespace-only values. (#8665, @aj-2000)
* Implement parallel ItemBlock processing via backup_controller goroutines (#8659, @sseago)
* Clean up leaked CSI snapshot for incomplete backup (#8637, @raesonerjt)
* Handle update conflict when restoring the status (#8630, @ywk253100)
* Fix issue #8419, support repo maintenance job to run on Windows nodes (#8626, @Lyndon-Li)
* Always create DataUpload configmap in restore namespace (#8621, @sseago)
* Fix issue #8091, avoid to create new repo when BSL is readonly (#8615, @Lyndon-Li)
* Fix issue #8242, distribute dd evenly across nodes (#8611, @Lyndon-Li)
* Fix issue #8497, update du/dd progress on completion (#8608, @Lyndon-Li)
* Fix issue #8418, add Windows toleration to data mover pods (#8606, @Lyndon-Li)
* Check the PVB status via podvolume Backupper rather than calling API server to avoid API server issue (#8603, @ywk253100)
* Fix issue #8067, add tmp folder (/tmp for linux, C:\Windows\Temp for Windows) as an alternative of udmrepo's config file location (#8602, @Lyndon-Li)
* Data mover restore for Windows (#8594, @Lyndon-Li)
* Skip patching the PV in finalization for failed operation (#8591, @reasonerjt)
* Fix issue #8579, set event burst to block event broadcaster from filtering events (#8590, @Lyndon-Li)
* Configurable Kopia Maintenance Interval. backup-repository-configmap adds an option for configurable`fullMaintenanceInterval` where fastGC (12 hours), and eagerGC (6 hours) allowing for faster removal of deleted velero backups from kopia repo. (#8581, @kaovilai)
* Fix issue #7753, recall repo maintenance history on Velero server restart (#8580, @Lyndon-Li)
* Clear validation errors when schedule is valid (#8575, @ywk253100)
* Merge restore helper image into Velero server image (#8574, @ywk253100)
* Don't include excluded items in ItemBlocks (#8572, @sseago)
* fs uploader and block uploader support Windows nodes (#8569, @Lyndon-Li)
* Fix issue #8418, support data mover backup for Windows nodes (#8555, @Lyndon-Li)
* Fix issue #8044, allow users to ignore delay binding the restorePVC of data mover when it is in WaitForFirstConsumer mode (#8550, @Lyndon-Li)
* Fix issue #8539, validate uploader types when o.CRDsOnly is set to false only since CRD installation doesn't rely on uploader types (#8538, @Lyndon-Li)
* Fix issue #7810, add maintenance history for backupRepository CRs (#8532, @Lyndon-Li)
* Make fs-backup work on linux nodes with the new Velero deployment and disable fs-backup if the source/target pod is running in non-linux node (#8424) (#8518, @Lyndon-Li)
* Fix issue: backup schedule pause/unpause doesn't work (#8512, @ywk253100)
* Fix backup post hook issue #8159 (caused by #7571): always execute backup post hooks after PVBs are handled (#8509, @ywk253100)
* Fix issue #8267, enhance the error message when expose fails (#8508, @Lyndon-Li)
* Fix issue #8416, #8417, deploy Velero server and node-agent in linux/Windows hybrid env (#8504, @Lyndon-Li)
* Design to add label selector as a criteria for volume policy (#8503, @shubham-pampattiwar)
* Related to issue #8485, move the acceptedByNode and acceptedTimestamp to Status of DU/DD CRD (#8498, @Lyndon-Li)
* Add SecurityContext to restore-helper (#8491, @reasonerjt)
* Fix issue #8433, add third party labels to data mover pods when the same labels exist in node-agent pods (#8487, @Lyndon-Li)
* Fix issue #8485, add an accepted time so as to count the prepare timeout (#8486, @Lyndon-Li)
* Fix issue #8125, log diagnostic info for data mover exposers when expose timeout (#8482, @Lyndon-Li)
* Fix issue #8415, implement multi-arch build and Windows build (#8476, @Lyndon-Li)
* Pin kopia to 0.18.2 (#8472, @Lyndon-Li)
* Add nil check for updating DataUpload VolumeInfo in finalizing phase (#8471, @blackpiglet)
* Allowing Object-Level Resource Status Restore (#8464, @shubham-pampattiwar)
* For issue #8429. Add the design for multi-arch build and windows build (#8459, @Lyndon-Li)
* Upgrade go.mod k8s.io/ go.mod to v0.31.3 and implemented proper logger configuration for both client-go and controller-runtime libraries. This change ensures that logging format and level settings are properly applied throughout the codebase. The update improves logging consistency and control across the Velero system. (#8450, @kaovilai)
* Add Design for Allowing Object-Level Resource Status Restore (#8403, @shubham-pampattiwar)
* Fix issue #8391, check ErrCancelled from suffix of data mover pod's termination message (#8396, @Lyndon-Li)
* Fix issue #8394, don't call closeDataPath in VGDP callbacks, otherwise, the VGDP cleanup will hang (#8395, @Lyndon-Li)
* Adding support in velero Resource Policies for filtering PVs based on additional VolumeAttributes properties under CSI PVs (#8383, @mayankagg9722)
* Add --item-block-worker-count flag to velero install and server (#8380, @sseago)
* Make BackedUpItems thread safe (#8366, @sseago)
* Include --annotations flag in backup and restore create commands (#8354, @alromeros)
* Use aggregated discovery API to discovery API groups and resources (#8353, @ywk253100)
* Copy "envFrom" from Velero server when creating maintenance jobs (#8343, @evhan)
* Set hinting region to use for GetBucketRegion() in pkg/repository/config/aws.go (#8297, @kaovilai)
* Bump up version of client-go and controller-runtime (#8275, @ywk253100)
* fix(pkg/repository/maintenance): don't panic when there's no container statuses (#8271, @mcluseau)
* Add Backup warning for inclusion of NS managed by ArgoCD (#8257, @shubham-pampattiwar)
* Added tracking for deleted namespace status check in restore flow. (#8233, @sangitaray2021)

View File

@@ -1,143 +0,0 @@
## v1.17
### Download
https://github.com/vmware-tanzu/velero/releases/tag/v1.17.0
### Container Image
`velero/velero:v1.17.0`
### Documentation
https://velero.io/docs/v1.17/
### Upgrading
https://velero.io/docs/v1.17/upgrade-to-1.17/
### Highlights
#### Modernized fs-backup
In v1.17, Velero fs-backup is modernized to the micro-service architecture, which brings below benefits:
- Many features that were absent to fs-backup are now available, i.e., load concurrency control, cancel, resume on restart, etc.
- fs-backup is more robust, the running backup/restore could survive from node-agent restart; and the resource allocation is in a more granular manner, the failure of one backup/restore won't impact others.
- The resource usage of node-agent is steady, especially, the node-agent pods won't request huge memory and hold it for a long time.
Check design https://github.com/vmware-tanzu/velero/blob/main/design/vgdp-micro-service-for-fs-backup/vgdp-micro-service-for-fs-backup.md for more details.
#### fs-backup support Windows cluster
In v1.17, Velero fs-backup supports to backup/restore Windows workloads. By leveraging the new micro-service architecture for fs-backup, data mover pods could run in Windows nodes and backup/restore Windows volumes. Together with CSI snapshot data movement for Windows which is delivered in 1.16, Velero now supports Windows workload backup/restore in full scenarios.
Check design https://github.com/vmware-tanzu/velero/blob/main/design/vgdp-micro-service-for-fs-backup/vgdp-micro-service-for-fs-backup.md for more details.
#### Volume group snapshot support
In v1.17, Velero supports [volume group snapshots](https://kubernetes.io/blog/2024/12/18/kubernetes-1-32-volume-group-snapshot-beta/) which is a beta feature in Kubernetes upstream, for both CSI snapshot backup and CSI snapshot data movement. This allows a snapshot to be taken from multiple volumes at the same point-in-time to achieve write order consistency, which is helpful to achieve better data consistency when multiple volumes being backed up are correlated.
Check the document https://velero.io/docs/main/volume-group-snapshots/ for more details.
#### Priority class support
In v1.17, [Kubernetes priority class](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/#priorityclass) is supported for all modules across Velero. Specifically, users are allowed to configure priority class to Velero server, node-agent, data mover pods, backup repository maintenance jobs separately.
Check design https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/priority-class-name-support_design.md for more details.
#### Scalability and Resiliency improvements of data movers
##### Reduce excessive number of data mover pods in Pending state
In v1.17, Velero allows users to set a `PrepareQueueLength` in the node-agent configuration, data mover pods and volumes out of this number won't be created until data path quota is available, so that excessive number cluster resources won't be taken unnecessarily, which is particularly helpful for large scale environments. This improvement applies to all kinds of data movements, including fs-backup and CSI snapshot data movement.
Check design https://github.com/vmware-tanzu/velero/blob/main/design/node-agent-load-soothing.md for more details.
##### Enhancement on node-agent restart handling for data movements
In v1.17, data movements in all phases could survive from node-agent restart and resume themselves; when a data movement gets orphaned in special cases, e.g., cluster node absent, it could also be canceled appropriately after the restart. This improvement applies to all kinds of data movements, including fs-backup and CSI snapshot data movement.
Check issue https://github.com/vmware-tanzu/velero/issues/8534 for more details.
##### CSI snapshot data movement restore node-selection and node-selection by storage class
In v1.17, CSI snapshot data movement restore acquires the same node-selection capability as backup, that is, users could specify which nodes can/cannot run data mover pods for both backup and restore now. And users are also allowed to configure the node-selection per storage class, which is particularly helpful to the environments where a storage class are not usable by all cluster nodes.
Check issue https://github.com/vmware-tanzu/velero/issues/8186 and https://github.com/vmware-tanzu/velero/issues/8223 for more details.
#### Include/exclude policy support for resource policy
In v1.17, Velero resource policy supports `includeExcludePolicy` besides the existing `volumePolicy`. This allows users to set include/exclude filters for resources in a resource policy configmap, so that these filters are reusable among multiple backups.
Check the document https://velero.io/docs/main/resource-filtering/#creating-resource-policies:~:text=resources%3D%22*%22-,Resource%20policies,-Velero%20provides%20resource for more details.
### Runtime and dependencies
Golang runtime: 1.24.6
kopia: 0.21.1
### Limitations/Known issues
### Breaking changes
#### Deprecation of Restic
According to [Velero deprecation policy](https://github.com/vmware-tanzu/velero/blob/main/GOVERNANCE.md#deprecation-policy), backup of fs-backup under Restic path is removed in v1.17, so `--uploader-type=restic` is not a valid installation configuration anymore. This means you cannot create a backup under Restic path, but you can still restore from the previous backups under Restic path until v1.19.
#### Repository maintenance job configurations are removed from Velero server parameter
Since the repository maintenance job configurations are moved to repository maintenance job configMap, in v1.17 below Velero sever parameters are removed:
- --keep-latest-maintenance-jobs
- --maintenance-job-cpu-request
- --maintenance-job-mem-request
- --maintenance-job-cpu-limit
- --maintenance-job-mem-limit
### All Changes
* Add ConfigMap parameters validation for install CLI and server start. (#9200, @blackpiglet)
* Add priorityclasses to high priority restore list (#9175, @kaovilai)
* Introduced context-based logger for backend implementations (Azure, GCS, S3, and Filesystem) (#9168, @priyansh17)
* Fix issue #9140, add os=windows:NoSchedule toleration for Windows pods (#9165, @Lyndon-Li)
* Remove the repository maintenance job parameters from velero server. (#9147, @blackpiglet)
* Add include/exclude policy to resources policy (#9145, @reasonerjt)
* Add ConfigMap support for keepLatestMaintenanceJobs with CLI parameter fallback (#9135, @shubham-pampattiwar)
* Fix the dd and du's node affinity issue. (#9130, @blackpiglet)
* Remove the WaitUntilVSCHandleIsReady from vs BIA. (#9124, @blackpiglet)
* Add comprehensive Volume Group Snapshots documentation with workflow diagrams and examples (#9123, @shubham-pampattiwar)
* Fix issue #9065, add doc for node-agent prepare queue length (#9118, @Lyndon-Li)
* Fix issue #9095, update restore doc for PVC selected-node (#9117, @Lyndon-Li)
* Update CSI Snapshot Data Movement doc for issue #8534, #8185 (#9113, @Lyndon-Li)
* Fix issue #8986, refactor fs-backup doc after VGDP Micro Service for fs-backup (#9112, @Lyndon-Li)
* Return error if timeout when checking server version (#9111, @ywk253100)
* Update "Default Volumes to Fs Backup" to "File System Backup (Default)" (#9105, @shubham-pampattiwar)
* Fix issue #9077, don't block backup deletion on list VS error (#9100, @Lyndon-Li)
* Bump up Kopia to v0.21.1 (#9098, @Lyndon-Li)
* Add imagePullSecrets inheritance for VGDP pod and maintenance job. (#9096, @blackpiglet)
* Avoid checking the VS and VSC status in the backup finalizing phase. (#9092, @blackpiglet)
* Fix issue #9053, Always remove selected-node annotation during PVC restore when no node mapping exists. Breaking change: Previously, the annotation was preserved if the node existed. (#9076, @Lyndon-Li)
* Enable parameterized kubelet mount path during node-agent installation (#9074, @longxiucai)
* Fix issue #8857, support third party tolerations for data mover pods (#9072, @Lyndon-Li)
* Fix issue #8813, remove restic from the valid uploader type (#9069, @Lyndon-Li)
* Fix issue #8185, allow users to disable pod volume host path mount for node-agent (#9068, @Lyndon-Li)
* Fix #8344, add the design for a mechanism to soothe creation of data mover pods for DataUpload, DataDownload, PodVolumeBackup and PodVolumeRestore (#9067, @Lyndon-Li)
* Fix #8344, add a mechanism to soothe creation of data mover pods for DataUpload, DataDownload, PodVolumeBackup and PodVolumeRestore (#9064, @Lyndon-Li)
* Add Gauge metric for BSL availability (#9059, @reasonerjt)
* Fix missing defaultVolumesToFsBackup flag output in Velero describe backup cmd (#9056, @shubham-pampattiwar)
* Allow for proper tracking of multiple hooks per container (#9048, @sseago)
* Make the backup repository controller doesn't invalidate the BSL on restart (#9046, @blackpiglet)
* Removed username/password credential handling from newConfigCredential as azidentity.UsernamePasswordCredentialOptions is reported as deprecated. (#9041, @priyansh17)
* Remove dependency with VolumeSnapshotClass in DataUpload. (#9040, @blackpiglet)
* Fix issue #8961, cancel PVB/PVR on Velero server restart (#9031, @Lyndon-Li)
* Fix issue #8962, resume PVB/PVR during node-agent restarts (#9030, @Lyndon-Li)
* Bump kopia v0.20.1 (#9027, @Lyndon-Li)
* Fix issue #8965, support PVB/PVR's cancel state in the backup/restore (#9026, @Lyndon-Li)
* Fix Issue 8816 When specifying LabelSelector on restore, related items such as PVC and VolumeSnapshot are not included (#9024, @amastbau)
* Fix issue #8963, add legacy PVR controller for Restic path (#9022, @Lyndon-Li)
* Fix issue #8964, add Windows support for VGDP MS for fs-backup (#9021, @Lyndon-Li)
* Accommodate VGS workflows in PVC CSI plugin (#9019, @shubham-pampattiwar)
* Fix issue #8958, add VGDP MS PVB controller (#9015, @Lyndon-Li)
* Fix issue #8959, add VGDP MS PVR controller (#9014, @Lyndon-Li)
* Fix issue #8988, add data path for VGDP ms PVR (#9005, @Lyndon-Li)
* Fix issue #8988, add data path for VGDP ms pvb (#8998, @Lyndon-Li)
* Skip VS and VSC not created by backup. (#8990, @blackpiglet)
* Make ResticIdentifier optional for kopia BackupRepositories (#8987, @kaovilai)
* Fix issue #8960, implement PodVolume exposer for PVB/PVR (#8985, @Lyndon-Li)
* fix: update mc command in minio-deployment example (#8982, @vishal-chdhry)
* Fix issue #8957, add design for VGDP MS for fs-backup (#8979, @Lyndon-Li)
* Add BSL status check for backup/restore operations. (#8976, @blackpiglet)
* Mark BackupRepository not ready when BSL changed (#8975, @ywk253100)
* Add support for [distributed snapshotting](https://github.com/kubernetes-csi/external-snapshotter/tree/4cedb3f45790ac593ebfa3324c490abedf739477?tab=readme-ov-file#distributed-snapshotting) (#8969, @flx5)
* Fix issue #8534, refactor dm controllers to tolerate cancel request in more cases, e.g., node restart, node drain (#8952, @Lyndon-Li)
* The backup and restore VGDP affinity enhancement implementation. (#8949, @blackpiglet)
* Remove CSI VS and VSC metadata from backup. (#8946, @blackpiglet)
* Extend PVCAction itemblock plugin to support grouping PVCs under VGS label key (#8944, @shubham-pampattiwar)
* Copy security context from origin pod (#8943, @farodin91)
* Add support for configuring VGS label key (#8938, @shubham-pampattiwar)
* Add VolumeSnapshotContent into the RIA and the mustHave resource list. (#8924, @blackpiglet)
* Mounted cloud credentials should not be world-readable (#8919, @sseago)
* Warn for not found error in patching managed fields (#8902, @sseago)
* Fix issue 8878, relief node os deduction error checks (#8891, @Lyndon-Li)
* Skip namespace in terminating state in backup resource collection. (#8890, @blackpiglet)
* Implement PriorityClass Support (#8883, @kaovilai)
* Fix Velero adding restore-wait init container when not needed. (#8880, @kaovilai)
* Pass the logger in kopia related operations. (#8875, @hu-keyu)
* Inherit the dnsPolicy and dnsConfig from the node agent pod. This is done so that the kopia task uses the same configuration. (#8845, @flx5)
* Add design for VolumeGroupSnapshot support (#8778, @shubham-pampattiwar)
* Inherit k8s default volumeSnapshotClass. (#8719, @hu-keyu)
* CLI automatically discovers and uses cacert from BSL for download requests (#8557, @kaovilai)
* This PR aims to add s390x support to Velero binary. (#7505, @pandurangkhandeparker)

View File

@@ -1,109 +0,0 @@
## v1.18
### Download
https://github.com/vmware-tanzu/velero/releases/tag/v1.18.0
### Container Image
`velero/velero:v1.18.0`
### Documentation
https://velero.io/docs/v1.18/
### Upgrading
https://velero.io/docs/v1.18/upgrade-to-1.18/
### Highlights
#### Concurrent backup
In v1.18, Velero is capable to process multiple backups concurrently. This is a significant usability improvement, especially for multiple tenants or multiple users case, backups submitted from different users could run their backups simultaneously without interfering with each other.
Check design https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/concurrent-backup-processing.md for more details.
#### Cache volume for data movers
In v1.18, Velero allows users to configure cache volumes for data mover pods during restore for CSI snapshot data movement and fs-backup. This brings below benefits:
- Solve the problem that data mover pods fail to when pod's ephemeral disk is limited
- Solve the problem that multiple data mover pods fail to run concurrently in one node when the node's ephemeral disk is limited
- Working together with backup repository's cache limit configuration, cache volume with appropriate size helps to improve the restore throughput
Check design https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/backup-repo-cache-volume.md for more details.
#### Incremental size for data movers
In v1.18, Velero allows users to observe the incremental size of data movers backups for CSI snapshot data movement and fs-backup, so that users could visually see the data reduction due to incremental backup.
#### Wildcard support for namespaces
In v1.18, Velero allows to use Glob regular expressions for namespace filters during backup and restore, so that users could filter namespaces in a batch manner.
#### VolumePolicy for PVC phase
In v1.18, Velero VolumePolicy supports actions by PVC phase, which help users to do special operations for PVCs with a specific phase, e.g., skip PVCs in Pending/Lost status from the backup.
#### Scalability and Resiliency improvements
##### Prevent Velero server OOM Kill for large backup repositories
In v1.18, some backup repository operations are delay executed out of Velero server, so Velero server won't be OOM Killed.
#### Performance improvement for VolumePolicy
In v1.18, VolumePolicy is enhanced for large number of pods/PVCs so that the performance is significantly improved.
#### Events for data mover pod diagnostic
In v1.18, events are recorded into data mover pod diagnostic, which allows user to see more information for troubleshooting when the data mover pod fails.
### Runtime and dependencies
Golang runtime: 1.25.7
kopia: 0.22.3
### Limitations/Known issues
### Breaking changes
#### Deprecation of PVC selected node feature
According to [Velero deprecation policy](https://github.com/vmware-tanzu/velero/blob/main/GOVERNANCE.md#deprecation-policy), PVC selected node feature is deprecated in v1.18. Velero could appropriately handle PVC's selected-node annotation, so users don't need to do anything particularly.
### All Changes
* Remove backup from running list when backup fails validation (#9498, @sseago)
* Maintenance Job only uses the first element of the LoadAffinity array (#9494, @blackpiglet)
* Fix issue #9478, add diagnose info on expose peek fails (#9481, @Lyndon-Li)
* Add Role, RoleBinding, ClusterRole, and ClusterRoleBinding in restore sequence. (#9474, @blackpiglet)
* Add maintenance job and data mover pod's labels and annotations setting. (#9452, @blackpiglet)
* Fix plugin init container names exceeding DNS-1123 limit (#9445, @mpryc)
* Add PVC-to-Pod cache to improve volume policy performance (#9441, @shubham-pampattiwar)
* Remove VolumeSnapshotClass from CSI B/R process. (#9431, @blackpiglet)
* Use hookIndex for recording multiple restore exec hooks. (#9366, @blackpiglet)
* Sanitize Azure HTTP responses in BSL status messages (#9321, @shubham-pampattiwar)
* Remove labels associated with previous backups (#9206, @Joeavaikath)
* Add VolumePolicy support for PVC Phase conditions to allow skipping Pending PVCs (#9166, @claude)
* feat: Enhance BackupStorageLocation with Secret-based CA certificate support (#9141, @kaovilai)
* Add `--apply` flag to `install` command, allowing usage of Kubernetes apply to make changes to existing installs (#9132, @mjnagel)
* Fix issue #9194, add doc for GOMAXPROCS behavior change (#9420, @Lyndon-Li)
* Apply volume policies to VolumeGroupSnapshot PVC filtering (#9419, @shubham-pampattiwar)
* Fix issue #9276, add doc for cache volume support (#9418, @Lyndon-Li)
* Add Prometheus metrics for maintenance jobs (#9414, @shubham-pampattiwar)
* Fix issue #9400, connect repo first time after creation so that init params could be written (#9407, @Lyndon-Li)
* Cache volume for PVR (#9397, @Lyndon-Li)
* Cache volume support for DataDownload (#9391, @Lyndon-Li)
* don't copy securitycontext from first container if configmap found (#9389, @sseago)
* Refactor repo provider interface for static configuration (#9379, @Lyndon-Li)
* Fix issue #9365, prevent fake completion notification due to multiple update of single PVR (#9375, @Lyndon-Li)
* Add cache volume configuration (#9370, @Lyndon-Li)
* Track actual resource names for GenerateName in restore status (#9368, @shubham-pampattiwar)
* Fix managed fields patch for resources using GenerateName (#9367, @shubham-pampattiwar)
* Support cache volume for generic restore exposer and pod volume exposer (#9362, @Lyndon-Li)
* Add incrementalSize to DU/PVB for reporting new/changed size (#9357, @sseago)
* Add snapshotSize for DataDownload, PodVolumeRestore (#9354, @Lyndon-Li)
* Add cache dir configuration for udmrepo (#9353, @Lyndon-Li)
* Fix the Job build error when BackupReposiotry name longer than 63. (#9350, @blackpiglet)
* Add cache configuration to VGDP (#9342, @Lyndon-Li)
* Fix issue #9332, add bytesDone for cache files (#9333, @Lyndon-Li)
* Fix typos in documentation (#9329, @T4iFooN-IX)
* Concurrent backup processing (#9307, @sseago)
* VerifyJSONConfigs verify every elements in Data. (#9302, @blackpiglet)
* Fix issue #9267, add events to data mover prepare diagnostic (#9296, @Lyndon-Li)
* Add option for privileged fs-backup pod (#9295, @sseago)
* Fix issue #9193, don't connect repo in repo controller (#9291, @Lyndon-Li)
* Implement concurrency control for cache of native VolumeSnapshotter plugin. (#9281, @0xLeo258)
* Fix issue #7904, remove the code and doc for PVC node selection (#9269, @Lyndon-Li)
* Fix schedule controller to prevent backup queue accumulation during extended blocking scenarios by properly handling empty backup phases (#9264, @shubham-pampattiwar)
* Fix repository maintenance jobs to inherit allowlisted tolerations from Velero deployment (#9256, @shubham-pampattiwar)
* Implement wildcard namespace pattern expansion for backup namespace includes/excludes. This change adds support for wildcard patterns (*, ?, [abc], {a,b,c}) in namespace includes and excludes during backup operations (#9255, @Joeavaikath)
* Protect VolumeSnapshot field from race condition during multi-thread backup (#9248, @0xLeo258)
* Update AzureAD Microsoft Authentication Library to v1.5.0 (#9244, @priyansh17)
* Get pod list once per namespace in pvc IBA (#9226, @sseago)
* Fix issue #7725, add design for backup repo cache configuration (#9148, @Lyndon-Li)
* Fix issue #9229, don't attach backupPVC to the source node (#9233, @Lyndon-Li)
* feat: Permit specifying annotations for the BackupPVC (#9173, @clementnuss)

View File

@@ -1 +0,0 @@
Include InitContainer configured as Sidecars when validating the existence of the target containers configured for the Backup Hooks

View File

@@ -1 +0,0 @@
Support all glob wildcard characters in namespace validation

View File

@@ -1 +0,0 @@
Fix VolumePolicy PVC phase condition filter for unbound PVCs (#9507)

View File

@@ -1 +0,0 @@
Fix VolumeGroupSnapshot restore failure with Ceph RBD CSI driver by creating stub VolumeGroupSnapshotContent during restore and looking up VolumeSnapshotClass by driver for credential support

View File

@@ -1 +0,0 @@
Add block data mover design for block level incremental backup by integrating with Kubernetes CBT

View File

@@ -1 +0,0 @@
Fix issue #9343, include PV topology to data mover pod affinities

View File

@@ -1 +0,0 @@
Fix issue #9496, support customized host os

View File

@@ -1 +0,0 @@
Add custom action type to volume policies

View File

@@ -1 +0,0 @@
If BIA return updateObj with SkipFromBackupAnnotation, treat it as skip the resource from backup.

View File

@@ -1 +0,0 @@
Issue #9544: Add test coverage for S3 bucket name in MRAP ARN notation and fix bucket validation to accept ARN format

View File

@@ -1 +0,0 @@
Wildcard namespaces: Log warning on empty resolution

View File

@@ -1 +0,0 @@
Fix issue #9475, use node-selector instead of nodName for generic restore

View File

@@ -1 +0,0 @@
Fix issue #9460, flush buffer before data mover completes

View File

@@ -1 +0,0 @@
Add schedule_expected_interval_seconds metric for dynamic backup alerting thresholds (#9559)

View File

@@ -1 +0,0 @@
Add ephemeral storage limit and request support for data mover and maintenance job

View File

@@ -1 +0,0 @@
Fix DBR stuck when CSI snapshot no longer exists in cloud provider

View File

@@ -1 +0,0 @@
update go-hclog to current version

View File

@@ -1 +0,0 @@
Add check for file extraction from tarball.

View File

@@ -1 +0,0 @@
Implement original VolumeSnapshotContent deletion for legacy backups

View File

@@ -1 +0,0 @@
Fix issue #9626, let go for uninitialized repo under readonly mode

View File

@@ -1 +0,0 @@
Fix issue #9636, fix configmap lookup in non-default namespaces

View File

@@ -1 +0,0 @@
Fix issue #9641, Remove redundant ReadyToUse polling in CSI VolumeSnapshotContent delete plugin

View File

@@ -1 +0,0 @@
Fix service restore with null healthCheckNodePort in last-applied-configuration label

View File

@@ -1 +0,0 @@
Fix issue #9658, Honor --stderrthreshold when --logtostderr is enabled

View File

@@ -1 +0,0 @@
Fix issue #9659, in the case that PVB/PVR/DU/DD is cancelled before the data path is really started, call EndEvent to prevent data mover pod from crashing because of delay event distribution

View File

@@ -1 +0,0 @@
Fix issue #9666, fix node-agent node detection in multiple instances scenario

View File

@@ -1 +0,0 @@
Fix issue #9470, remove restic from repository

View File

@@ -1 +0,0 @@
Fix issue #9469, remove restic for uploader

View File

@@ -1 +0,0 @@
Fix issue #9681, fix restores and podvolumerestores list options to only list in installed namespace

View File

@@ -1 +0,0 @@
Fix issue #9428, increase repo maintenance history queue length from 3 to 25

View File

@@ -1 +0,0 @@
Fix wildcard expansion when includes is empty and excludes has wildcards

View File

@@ -1 +0,0 @@
Enhance backup deletion logic to handle tarball download failures

View File

@@ -1 +0,0 @@
Bump external-snapshotter to v8.4.0 and migrate VolumeGroupSnapshot API from v1beta1 to v1beta2 for Kubernetes 1.34+ compatibility

View File

@@ -1 +0,0 @@
Fix issue #9699, add a 2-second gap between temporary CSI VolumeSnapshotContent create and delete operations

View File

@@ -1 +0,0 @@
Update Debian base image from bookworm to trixie

View File

@@ -1 +0,0 @@
Fix issue #9703, fix CSI PVC Backup Plugin list options to only list in installed namespace

View File

@@ -1 +0,0 @@
perf: better string concatenation

View File

@@ -1 +0,0 @@
Fix issue #9709, add interfaces for CBT service and CBT bitmap

View File

@@ -1 +0,0 @@
Fix issue #9723, extend Unified Repo Interface to support block uploader

View File

@@ -1 +0,0 @@
Remove Restic build from Dockerfile, Makefile and Tiltfile.

View File

@@ -1 +0,0 @@
Remove Restic code path from PodVolumeRestore.

View File

@@ -1 +0,0 @@
fix: lint permission issue

View File

@@ -35,7 +35,7 @@ func main() {
for {
<-ticker.C
if done() {
fmt.Println("All PodVolumeRestores are done")
fmt.Println("All restic restores are done")
err := removeFolder()
if err != nil {
fmt.Println(err)
@@ -65,7 +65,6 @@ func done() bool {
doneFile := filepath.Join("/restores", child.Name(), ".velero", os.Args[1])
// #nosec G304,G703 -- doneFile is generated from internal logic and not user-controllable.
if _, err := os.Stat(doneFile); os.IsNotExist(err) {
fmt.Printf("The filesystem restore done file %s is not found yet. Retry later.\n", doneFile)
return false

View File

@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.16.5
controller-gen.kubebuilder.io/version: v0.14.0
name: backuprepositories.velero.io
spec:
group: velero.io
@@ -69,7 +69,9 @@ spec:
- ""
type: string
resticIdentifier:
description: Deprecated
description: |-
ResticIdentifier is the full restic-compatible string for identifying
this repository.
type: string
volumeNamespace:
description: |-
@@ -79,14 +81,15 @@ spec:
required:
- backupStorageLocation
- maintenanceFrequency
- resticIdentifier
- volumeNamespace
type: object
status:
description: BackupRepositoryStatus is the current status of a BackupRepository.
properties:
lastMaintenanceTime:
description: LastMaintenanceTime is the last time repo maintenance
succeeded.
description: LastMaintenanceTime is the last time maintenance was
run.
format: date-time
nullable: true
type: string
@@ -101,33 +104,6 @@ spec:
- Ready
- NotReady
type: string
recentMaintenance:
description: RecentMaintenance is status of the recent repo maintenance.
items:
properties:
completeTimestamp:
description: CompleteTimestamp is the completion time of the
repo maintenance.
format: date-time
nullable: true
type: string
message:
description: Message is a message about the current status of
the repo maintenance.
type: string
result:
description: Result is the result of the repo maintenance.
enum:
- Succeeded
- Failed
type: string
startTimestamp:
description: StartTimestamp is the start time of the repo maintenance.
format: date-time
nullable: true
type: string
type: object
type: array
type: object
type: object
served: true

View File

@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.16.5
controller-gen.kubebuilder.io/version: v0.14.0
name: backups.velero.io
spec:
group: velero.io
@@ -63,6 +63,7 @@ spec:
DefaultVolumesToRestic specifies whether restic should be used to take a
backup of all pod volumes by default.
Deprecated: this field is no longer used and will be removed entirely in future. Use DefaultVolumesToFsBackup instead.
nullable: true
type: boolean
@@ -175,13 +176,11 @@ spec:
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- key
- operator
type: object
type: array
x-kubernetes-list-type: atomic
matchLabels:
additionalProperties:
type: string
@@ -365,13 +364,11 @@ spec:
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- key
- operator
type: object
type: array
x-kubernetes-list-type: atomic
matchLabels:
additionalProperties:
type: string
@@ -428,13 +425,11 @@ spec:
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- key
- operator
type: object
type: array
x-kubernetes-list-type: atomic
matchLabels:
additionalProperties:
type: string
@@ -507,10 +502,6 @@ spec:
uploads to perform when using the uploader.
type: integer
type: object
volumeGroupSnapshotLabelKey:
description: VolumeGroupSnapshotLabelKey specifies the label key to
group PVCs under a VGS.
type: string
volumeSnapshotLocations:
description: VolumeSnapshotLocations is a list containing names of
VolumeSnapshotLocations associated with this backup.
@@ -594,8 +585,6 @@ spec:
description: Phase is the current state of the Backup.
enum:
- New
- Queued
- ReadyToStart
- FailedValidation
- InProgress
- WaitingForPluginOperations
@@ -627,11 +616,6 @@ spec:
filters that happen as items are processed.
type: integer
type: object
queuePosition:
description: |-
QueuePosition is the position of the backup in the queue.
Only relevant when Phase is "Queued"
type: integer
startTimestamp:
description: |-
StartTimestamp records the time a backup was started.

View File

@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.16.5
controller-gen.kubebuilder.io/version: v0.14.0
name: backupstoragelocations.velero.io
spec:
group: velero.io
@@ -86,13 +86,10 @@ spec:
valid secret key.
type: string
name:
default: ""
description: |-
Name of the referent.
This field is effectively required, but due to backwards compatibility is
allowed to be empty. Instances of this type with an empty value here are
almost certainly wrong.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind, uid?
type: string
optional:
description: Specify whether the Secret or its key must be defined
@@ -113,38 +110,10 @@ spec:
description: Bucket is the bucket to use for object storage.
type: string
caCert:
description: |-
CACert defines a CA bundle to use when verifying TLS connections to the provider.
Deprecated: Use CACertRef instead.
description: CACert defines a CA bundle to use when verifying
TLS connections to the provider.
format: byte
type: string
caCertRef:
description: |-
CACertRef is a reference to a Secret containing the CA certificate bundle to use
when verifying TLS connections to the provider. The Secret must be in the same
namespace as the BackupStorageLocation.
properties:
key:
description: The key of the secret to select from. Must be
a valid secret key.
type: string
name:
default: ""
description: |-
Name of the referent.
This field is effectively required, but due to backwards compatibility is
allowed to be empty. Instances of this type with an empty value here are
almost certainly wrong.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
type: string
optional:
description: Specify whether the Secret or its key must be
defined
type: boolean
required:
- key
type: object
x-kubernetes-map-type: atomic
prefix:
description: Prefix is the path inside a bucket to use for Velero
storage. Optional.
@@ -172,6 +141,7 @@ spec:
description: |-
AccessMode is an unused field.
Deprecated: there is now an AccessMode field on the Spec and this field
will be removed entirely as of v2.0.
enum:
@@ -183,6 +153,7 @@ spec:
LastSyncedRevision is the value of the `metadata/revision` file in the backup
storage location the last time the BSL's contents were synced into the cluster.
Deprecated: this field is no longer updated or used for detecting changes to
the location's contents and will be removed entirely in v2.0.
type: string

View File

@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.16.5
controller-gen.kubebuilder.io/version: v0.14.0
name: deletebackuprequests.velero.io
spec:
group: velero.io

View File

@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.16.5
controller-gen.kubebuilder.io/version: v0.14.0
name: downloadrequests.velero.io
spec:
group: velero.io

View File

@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.16.5
controller-gen.kubebuilder.io/version: v0.14.0
name: podvolumebackups.velero.io
spec:
group: velero.io
@@ -15,47 +15,38 @@ spec:
scope: Namespaced
versions:
- additionalPrinterColumns:
- description: PodVolumeBackup status such as New/InProgress
- description: Pod Volume Backup status such as New/InProgress
jsonPath: .status.phase
name: Status
type: string
- description: Time duration since this PodVolumeBackup was started
- description: Time when this backup was started
jsonPath: .status.startTimestamp
name: Started
name: Created
type: date
- description: Completed bytes
format: int64
jsonPath: .status.progress.bytesDone
name: Bytes Done
type: integer
- description: Total bytes
format: int64
jsonPath: .status.progress.totalBytes
name: Total Bytes
type: integer
- description: Incremental bytes
format: int64
jsonPath: .status.incrementalBytes
name: Incremental Bytes
priority: 10
type: integer
- description: Namespace of the pod containing the volume to be backed up
jsonPath: .spec.pod.namespace
name: Namespace
type: string
- description: Name of the pod containing the volume to be backed up
jsonPath: .spec.pod.name
name: Pod
type: string
- description: Name of the volume to be backed up
jsonPath: .spec.volume
name: Volume
type: string
- description: The type of the uploader to handle data transfer
jsonPath: .spec.uploaderType
name: Uploader Type
type: string
- description: Name of the Backup Storage Location where this backup should be
stored
jsonPath: .spec.backupStorageLocation
name: Storage Location
type: string
- description: Time duration since this PodVolumeBackup was created
jsonPath: .metadata.creationTimestamp
- jsonPath: .metadata.creationTimestamp
name: Age
type: date
- description: Name of the node where the PodVolumeBackup is processed
jsonPath: .status.node
name: Node
type: string
- description: The type of the uploader to handle data transfer
jsonPath: .spec.uploaderType
name: Uploader
type: string
name: v1
schema:
openAPIV3Schema:
@@ -85,11 +76,6 @@ spec:
BackupStorageLocation is the name of the backup storage location
where the backup repository is stored.
type: string
cancel:
description: |-
Cancel indicates request to cancel the ongoing PodVolumeBackup. It can be set
when the PodVolumeBackup is in InProgress phase
type: boolean
node:
description: Node is the name of the node that the Pod is running
on.
@@ -110,6 +96,7 @@ spec:
the event) or if no container name is specified "spec.containers[2]" (container with
index 2 in this pod). This syntax is chosen only to have some well-defined way of
referencing a part of an object.
TODO: this design is not final and this field is subject to change in the future.
type: string
kind:
description: |-
@@ -179,13 +166,6 @@ spec:
status:
description: PodVolumeBackupStatus is the current status of a PodVolumeBackup.
properties:
acceptedTimestamp:
description: |-
AcceptedTimestamp records the time the pod volume backup is to be prepared.
The server's time is used for AcceptedTimestamp
format: date-time
nullable: true
type: string
completionTimestamp:
description: |-
CompletionTimestamp records the time a backup was completed.
@@ -195,11 +175,6 @@ spec:
format: date-time
nullable: true
type: string
incrementalBytes:
description: IncrementalBytes holds the number of bytes new or changed
since the last backup
format: int64
type: integer
message:
description: Message is a message about the pod volume backup's status.
type: string
@@ -211,11 +186,7 @@ spec:
description: Phase is the current state of the PodVolumeBackup.
enum:
- New
- Accepted
- Prepared
- InProgress
- Canceling
- Canceled
- Completed
- Failed
type: string

View File

@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.16.5
controller-gen.kubebuilder.io/version: v0.14.0
name: podvolumerestores.velero.io
spec:
group: velero.io
@@ -15,40 +15,39 @@ spec:
scope: Namespaced
versions:
- additionalPrinterColumns:
- description: PodVolumeRestore status such as New/InProgress
jsonPath: .status.phase
name: Status
- description: Namespace of the pod containing the volume to be restored
jsonPath: .spec.pod.namespace
name: Namespace
type: string
- description: Time duration since this PodVolumeRestore was started
jsonPath: .status.startTimestamp
name: Started
type: date
- description: Completed bytes
format: int64
jsonPath: .status.progress.bytesDone
name: Bytes Done
type: integer
- description: Total bytes
format: int64
jsonPath: .status.progress.totalBytes
name: Total Bytes
type: integer
- description: Name of the Backup Storage Location where the backup data is stored
jsonPath: .spec.backupStorageLocation
name: Storage Location
type: string
- description: Time duration since this PodVolumeRestore was created
jsonPath: .metadata.creationTimestamp
name: Age
type: date
- description: Name of the node where the PodVolumeRestore is processed
jsonPath: .status.node
name: Node
- description: Name of the pod containing the volume to be restored
jsonPath: .spec.pod.name
name: Pod
type: string
- description: The type of the uploader to handle data transfer
jsonPath: .spec.uploaderType
name: Uploader Type
type: string
- description: Name of the volume to be restored
jsonPath: .spec.volume
name: Volume
type: string
- description: Pod Volume Restore status such as New/InProgress
jsonPath: .status.phase
name: Status
type: string
- description: Pod Volume Restore status such as New/InProgress
format: int64
jsonPath: .status.progress.totalBytes
name: TotalBytes
type: integer
- description: Pod Volume Restore status such as New/InProgress
format: int64
jsonPath: .status.progress.bytesDone
name: BytesDone
type: integer
- jsonPath: .metadata.creationTimestamp
name: Age
type: date
name: v1
schema:
openAPIV3Schema:
@@ -78,11 +77,6 @@ spec:
BackupStorageLocation is the name of the backup storage location
where the backup repository is stored.
type: string
cancel:
description: |-
Cancel indicates request to cancel the ongoing PodVolumeRestore. It can be set
when the PodVolumeRestore is in InProgress phase
type: boolean
pod:
description: Pod is a reference to the pod containing the volume to
be restored.
@@ -99,6 +93,7 @@ spec:
the event) or if no container name is specified "spec.containers[2]" (container with
index 2 in this pod). This syntax is chosen only to have some well-defined way of
referencing a part of an object.
TODO: this design is not final and this field is subject to change in the future.
type: string
kind:
description: |-
@@ -133,10 +128,6 @@ spec:
snapshotID:
description: SnapshotID is the ID of the volume snapshot to be restored.
type: string
snapshotSize:
description: SnapshotSize is the logical size in Bytes of the snapshot.
format: int64
type: integer
sourceNamespace:
description: SourceNamespace is the original namespace for namaspace
mapping.
@@ -172,13 +163,6 @@ spec:
status:
description: PodVolumeRestoreStatus is the current status of a PodVolumeRestore.
properties:
acceptedTimestamp:
description: |-
AcceptedTimestamp records the time the pod volume restore is to be prepared.
The server's time is used for AcceptedTimestamp
format: date-time
nullable: true
type: string
completionTimestamp:
description: |-
CompletionTimestamp records the time a restore was completed.
@@ -190,19 +174,11 @@ spec:
message:
description: Message is a message about the pod volume restore's status.
type: string
node:
description: Node is name of the node where the pod volume restore
is processed.
type: string
phase:
description: Phase is the current state of the PodVolumeRestore.
enum:
- New
- Accepted
- Prepared
- InProgress
- Canceling
- Canceled
- Completed
- Failed
type: string

View File

@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.16.5
controller-gen.kubebuilder.io/version: v0.14.0
name: restores.velero.io
spec:
group: velero.io
@@ -138,13 +138,11 @@ spec:
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- key
- operator
type: object
type: array
x-kubernetes-list-type: atomic
matchLabels:
additionalProperties:
type: string
@@ -293,13 +291,11 @@ spec:
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- key
- operator
type: object
type: array
x-kubernetes-list-type: atomic
matchLabels:
additionalProperties:
type: string
@@ -358,13 +354,11 @@ spec:
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- key
- operator
type: object
type: array
x-kubernetes-list-type: atomic
matchLabels:
additionalProperties:
type: string

View File

@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.16.5
controller-gen.kubebuilder.io/version: v0.14.0
name: schedules.velero.io
spec:
group: velero.io
@@ -102,6 +102,7 @@ spec:
DefaultVolumesToRestic specifies whether restic should be used to take a
backup of all pod volumes by default.
Deprecated: this field is no longer used and will be removed entirely in future. Use DefaultVolumesToFsBackup instead.
nullable: true
type: boolean
@@ -214,13 +215,11 @@ spec:
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- key
- operator
type: object
type: array
x-kubernetes-list-type: atomic
matchLabels:
additionalProperties:
type: string
@@ -406,13 +405,11 @@ spec:
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- key
- operator
type: object
type: array
x-kubernetes-list-type: atomic
matchLabels:
additionalProperties:
type: string
@@ -469,13 +466,11 @@ spec:
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- key
- operator
type: object
type: array
x-kubernetes-list-type: atomic
matchLabels:
additionalProperties:
type: string
@@ -549,10 +544,6 @@ spec:
uploads to perform when using the uploader.
type: integer
type: object
volumeGroupSnapshotLabelKey:
description: VolumeGroupSnapshotLabelKey specifies the label key
to group PVCs under a VGS.
type: string
volumeSnapshotLocations:
description: VolumeSnapshotLocations is a list containing names
of VolumeSnapshotLocations associated with this backup.

View File

@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.16.5
controller-gen.kubebuilder.io/version: v0.14.0
name: serverstatusrequests.velero.io
spec:
group: velero.io

View File

@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.16.5
controller-gen.kubebuilder.io/version: v0.14.0
name: volumesnapshotlocations.velero.io
spec:
group: velero.io
@@ -57,13 +57,10 @@ spec:
valid secret key.
type: string
name:
default: ""
description: |-
Name of the referent.
This field is effectively required, but due to backwards compatibility is
allowed to be empty. Instances of this type with an empty value here are
almost certainly wrong.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind, uid?
type: string
optional:
description: Specify whether the Secret or its key must be defined

File diff suppressed because one or more lines are too long

View File

@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.16.5
controller-gen.kubebuilder.io/version: v0.14.0
name: datadownloads.velero.io
spec:
group: velero.io
@@ -92,13 +92,6 @@ spec:
DataMover specifies the data mover to be used by the backup.
If DataMover is "" or "velero", the built-in data mover will be used.
type: string
nodeOS:
description: NodeOS is OS of the node where the DataDownload is processed.
enum:
- auto
- linux
- windows
type: string
operationTimeout:
description: |-
OperationTimeout specifies the time used to wait internal operations,
@@ -108,10 +101,6 @@ spec:
description: SnapshotID is the ID of the Velero backup snapshot to
be restored from.
type: string
snapshotSize:
description: SnapshotSize is the logical size in Bytes of the snapshot.
format: int64
type: integer
sourceNamespace:
description: |-
SourceNamespace is the original namespace where the volume is backed up from.
@@ -147,16 +136,6 @@ spec:
status:
description: DataDownloadStatus is the current status of a DataDownload.
properties:
acceptedByNode:
description: Node is name of the node where the DataUpload is prepared.
type: string
acceptedTimestamp:
description: |-
AcceptedTimestamp records the time the DataUpload is to be prepared.
The server's time is used for AcceptedTimestamp
format: date-time
nullable: true
type: string
completionTimestamp:
description: |-
CompletionTimestamp records the time a restore was completed.

View File

@@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.16.5
controller-gen.kubebuilder.io/version: v0.14.0
name: datauploads.velero.io
spec:
group: velero.io
@@ -33,12 +33,6 @@ spec:
jsonPath: .status.progress.totalBytes
name: Total Bytes
type: integer
- description: Incremental bytes
format: int64
jsonPath: .status.incrementalBytes
name: Incremental Bytes
priority: 10
type: integer
- description: Name of the Backup Storage Location where this backup should be
stored
jsonPath: .spec.backupStorageLocation
@@ -93,9 +87,6 @@ spec:
of the CSI snapshot.
nullable: true
properties:
driver:
description: Driver is the driver used by the VolumeSnapshotContent
type: string
snapshotClass:
description: SnapshotClass is the name of the snapshot class that
the volume snapshot is created with
@@ -152,17 +143,6 @@ spec:
status:
description: DataUploadStatus is the current status of a DataUpload.
properties:
acceptedByNode:
description: AcceptedByNode is name of the node where the DataUpload
is prepared.
type: string
acceptedTimestamp:
description: |-
AcceptedTimestamp records the time the DataUpload is to be prepared.
The server's time is used for AcceptedTimestamp
format: date-time
nullable: true
type: string
completionTimestamp:
description: |-
CompletionTimestamp records the time a backup was completed.
@@ -179,24 +159,12 @@ spec:
as a result of the DataUpload.
nullable: true
type: object
incrementalBytes:
description: IncrementalBytes holds the number of bytes new or changed
since the last backup
format: int64
type: integer
message:
description: Message is a message about the DataUpload's status.
type: string
node:
description: Node is name of the node where the DataUpload is processed.
type: string
nodeOS:
description: NodeOS is OS of the node where the DataUpload is processed.
enum:
- auto
- linux
- windows
type: string
path:
description: Path is the full path of the snapshot volume being backed
up.

File diff suppressed because one or more lines are too long

View File

@@ -8,7 +8,17 @@ rules:
- ""
resources:
- persistentvolumerclaims
verbs:
- get
- apiGroups:
- ""
resources:
- persistentvolumes
verbs:
- get
- apiGroups:
- ""
resources:
- pods
verbs:
- get
@@ -16,18 +26,6 @@ rules:
- velero.io
resources:
- backuprepositories
- backups
- backupstoragelocations
- datadownloads
- datauploads
- deletebackuprequests
- downloadrequests
- podvolumebackups
- podvolumerestores
- restores
- schedules
- serverstatusrequests
- volumesnapshotlocations
verbs:
- create
- delete
@@ -40,18 +38,239 @@ rules:
- velero.io
resources:
- backuprepositories/status
verbs:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
- backups
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- velero.io
resources:
- backups/status
verbs:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
- backupstoragelocations
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- velero.io
resources:
- backupstoragelocations/status
verbs:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
- datadownloads
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- velero.io
resources:
- datadownloads/status
verbs:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
- datauploads
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- velero.io
resources:
- datauploads/status
verbs:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
- deletebackuprequests
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- velero.io
resources:
- deletebackuprequests/status
verbs:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
- downloadrequests
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- velero.io
resources:
- downloadrequests/status
verbs:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
- podvolumebackups
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- velero.io
resources:
- podvolumebackups/status
verbs:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
- podvolumerestores
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- velero.io
resources:
- podvolumerestores/status
verbs:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
- restores
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- velero.io
resources:
- restores/status
verbs:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
- schedules
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- velero.io
resources:
- schedules/status
verbs:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
- serverstatusrequests
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- velero.io
resources:
- serverstatusrequests/status
verbs:
- get
- patch
- update
- apiGroups:
- velero.io
resources:
- volumesnapshotlocations
verbs:
- create
- delete
- get
- list
- patch
- update
- watch

View File

@@ -276,7 +276,7 @@ func (v *volumeHelperImpl) ShouldPerformSnapshot(obj runtime.Unstructured, group
if !boolptr.IsSetToFalse(v.snapshotVolumes) {
// If the backup.Spec.SnapshotVolumes is not set, or set to true, then should take the snapshot.
v.logger.Infof("performing snapshot action for pv %s as the snapshotVolumes is not set to false", pv.Name)
v.logger.Infof("performing snapshot action for pv %s as the snapshotVolumes is not set to false")
return true, nil
}

View File

@@ -1,70 +0,0 @@
# Apply flag for install command
## Abstract
Add an `--apply` flag to the install command that enables applying existing resources rather than creating them. This can be useful as part of the upgrade process for existing installations.
## Background
The current Velero install command creates resources but doesn't provide a direct way to apply updates to an existing installation.
Users attempting to run the install command on an existing installation receive "already exists" messages.
Upgrade steps for existing installs typically involve a three (or more) step process to apply updated CRDs (using `--dry-run` and piping to `kubectl apply`) and then updating/setting images on the Velero deployment and node-agent.
## Goals
- Provide a simple flag to enable applying resources on an existing Velero installation.
- Use server-side apply to update existing resources rather than attempting to create them.
- Maintain consistency with the regular install flow.
## Non Goals
- Implement special logic for specific version-to-version upgrades (i.e. resource deletion, etc).
- Add complex upgrade validation or pre/post-upgrade hooks.
- Provide rollback capabilities.
## High-Level Design
The `--apply` flag will be added to the Velero install command.
When this flag is set, the installation process will use server-side apply to update existing resources instead of using create on new resources.
This flag can be used as _part_ of the upgrade process, but will not always fully handle an upgrade.
## Detailed Design
The implementation adds a new boolean flag `--apply` to the install command.
This flag will be passed through to the underlying install functions where the resource creation logic resides.
When the flag is set to true:
- The `createOrApplyResource` function will use server-side apply with field manager "velero-cli" and `force=true` to update resources.
- Resources will be applied in the same order as they would be created during installation.
- Custom Resource Definitions will still be processed first, and the system will wait for them to be established before continuing.
The server-side apply approach with `force=true` ensures that resources are updated even if there are conflicts with the last applied state.
This provides a best-effort mechanism to apply resources that follows the same flow as installation but updates resources instead of creating them.
No special handling is added for specific versions or resource structures, making this a general-purpose mechanism for applying resources.
## Alternatives Considered
1. Creating a separate `upgrade` command that would duplicate much of the install command logic.
- Rejected due to code duplication and maintenance overhead.
2. Implementing version-specific upgrade logic to handle breaking changes between versions.
- Rejected as overly complex and difficult to maintain across multiple version paths.
- This could be considered again in the future, but is not in the scope of the current design.
3. Adding automatic detection of existing resources and switching to apply mode.
- Rejected as it could lead to unexpected behavior and confusion if users unintentionally apply changes to existing resources.
## Security Considerations
The apply flag maintains the same security profile as the install command.
No additional permissions are required beyond what is needed for resource creation.
The use of `force=true` with server-side apply could potentially override manual changes made to resources, but this is a necessary trade-off to ensure apply is successful.
## Compatibility
This enhancement is compatible with all existing Velero installations as it is a new opt-in flag.
It does not change any resource formats or API contracts.
The apply process is best-effort and does not guarantee compatibility between arbitrary versions of Velero.
Users should still consult release notes for any breaking changes that may require manual intervention.
This flag could be adopted by the helm chart, specifically for CRD updates, to simplify the CRD update job.
## Implementation
The implementation involves:
1. Adding support for `Apply` to the existing Kubernetes client code.
1. Adding the `--apply` flag to the install command options.
1. Changing `createResource` to `createOrApplyResource` and updating it to use server-side apply when the `apply` boolean is set.
The implementation is straightforward and follows existing code patterns.
No migration of state or special handling of specific resources is required.

View File

@@ -1,231 +0,0 @@
# Backup Repository Cache Volume Design
## Glossary & Abbreviation
**Backup Storage**: The storage to store the backup data. Check [Unified Repository design][1] for details.
**Backup Repository**: Backup repository is layered between BR data movers and Backup Storage to provide BR related features that is introduced in [Unified Repository design][1].
**Velero Generic Data Path (VGDP)**: VGDP is the collective of modules that is introduced in [Unified Repository design][1]. Velero uses these modules to finish data transfer for various purposes (i.e., PodVolume backup/restore, Volume Snapshot Data Movement). VGDP modules include uploaders and the backup repository.
**Data Mover Pods**: Intermediate pods which hold VGDP and complete the data transfer. See [VGDP Micro Service for Volume Snapshot Data Movement][2] and [VGDP Micro Service For fs-backup][3] for details.
**Repository Maintenance Pods**: Pods for [Repository Maintenance Jobs][4], which holds VGDP to run repository maintenance.
## Background
According to the [Unified Repository design][1] Velero uses selectable backup repositories for various backup/restore methods, i.e., fs-backup, volume snapshot data movement, etc. Some backup repositories may need to cache data on the client side for various repository operation, so as to accelerate the execution.
In the existing [Backup Repository Configuration][5], we allow users to configure the cache data size (`cacheLimitMB`). However, the cache data is still stored in the root file system of data mover pods/repository maintenance pods, so stored in the root file system of the node. This is not good enough, reasons:
- In many distributions, the node's system disk size is predefined, non configurable and limit, e.g., the system disk size may be 20G or less
- Velero supports concurrent data movements in each node. The cache in each of the concurrent data mover pods could quickly run out of the system disk and cause problems like pod eviction, failure of pod creation, degradation of Kubernetes QoS, etc.
We need to allow users to prepare a dedicated location, e.g., a dedictated volume, for the cache.
Not all backup repositories or not all backup repository operations require cache, we need to define the details when and how the cache is used.
## Goals
- Create a mechanism for users to configure cache volumes for various pods running VGDP
- Design the workflow to assign the cache volume pod path to backup repositories
- Describe when and how the cache volume is used
## Non-Goals
- The solution is based on [Unified Repository design][1], [VGDP Micro Service for Volume Snapshot Data Movement][2] and [VGDP Micro Service For fs-backup][3], legacy data paths are not supported. E.g., when a pod volume restore (PVR) runs with legacy Restic path, if any data is cached, the cache still resides in the root file system.
## Solution
### Cache Data
Varying on backup repositoires, cache data may include payload data or repository metadata, e.g., indexes to the payload data chunks.
Payload data is highly related to the backup data, and normally take the majority of the repository data as well as the cache data.
Repository metadata is related to the backup repository's chunking algorithm, data chunk mapping method, etc, and so the size is not proportional to the backup data size.
On the other hand for some backup repository, in extreme cases, the repository metadata may be significantly large. E.g., Kopia's indexes are per chunks, if there are huge number of small files in the repository, Kopia's index data may be in the same level of or even larger than the payload data.
However, in the cases that repository metadata data become the majority, other bottlenecks may emerge and concurrency of data movers may be significantly constrained, so the requirement to cache volumes may go away.
Therefore, for now we only consider the cache volume requirement for payload data, and leave the consideration for metadata as a future enhancement.
### Scenarios
Backup repository cache varies on backup repositories and backup repository operation during VGDP runs. Below are the scenarios when VGDP runs:
- Data Upload for Backup: this is the process to upload/write the backup data into the backup repository, e.g., DataUpload or PodVolumeBackup. The pieces of data is almost directly written to the repository, sometimes with a small group staying shortly in the local place. That is to say, there should not be large scale data cached for this scenario, so we don't prepare dedicated cache for this scenario.
- Repository Maintenance: Repository maintenance most often visits the backup repository's metadata and sometimes it needs to visit the file system directories from the backed up data. On the other hand, it is not practical to run concurrent maintenance jobs in one node. So the cache data is neither large nor affect the root file system too much. Therefore, we don't need to prepare dedicated cache for this scenario.
- Data Download for Restore: this is the process to download/read the backup data from the backup repository during restore, e.g., DataDownload or PodVolumeRestore. For backup repositories for which data are stored in remote backup storages (e.g., Kopia repository stores data in remote object stores), large scale of data are cached locally to accerlerate the restore. Therefore, we need dedicate cache volumes for this scenario.
- Backup Deletion: During this scenario, backup repository is connected, metadata is enumerated to find the repository snapshot representing the backup data. That is to say, only metadata is cached if any. Therefore, dedicated cache volumes are not required in this scenario.
The above analyses are based on the common behavior of backup repositories and they are not considering the case that backup repository metadata takes majority or siginficant proportion of the cache data.
As a conclusion of the analyses, we will create dedicated cache volumes for restore scenarios.
For other scenarios, we can add them regarded to the future changes/requirements. The mechanism to expose and connect the cache volumes should work for all scenarios. E.g., if we need to consider the backup repository metadata case, we may need cache volumes for backup and repository maintenance as well, then we can just reuse the same cache volume provision and connection mechanism to backup and repository maintenance scenarios.
### Cache Data and Lifecycle
If available, one cache volume is dedicately assigned to one data mover pod. That is, the cached data is destroyed when the data mover pod completes. Then the backup repository instance also closes.
Cache data are fully managed by the specific backup repository. So the backup repository may also have its own way to GC the cache data.
That is to say, cache data GC may be launched by the backup repository instance during the running of the data mover pod; then the left data are automatically destroyed when the data mover pod and the cache PVC are destroyed (cache PVC's `reclaimPolicy` is always `Deleted`, so once the cache PVC is destroyed, the volume will also be destroyed). So no specially logics are needed for cache data GC.
### Data Size
Cache volumes take storage space and cluster resources (PVC, PV), therefore, cache volumes should be created only when necessary and the volumes should be with reasonable size based on the cache data size:
- It is not a good bargain to have cache volumes for small backups, small backups will use resident cache location (the cache location in the root file system)
- The cache data size has a limit, the existing `cacheLimitMB` is used for this purpose. E.g., it could be set as 1024 for a 1TB backup, which means 1GB of data is cached and the old cache data exceeding this size will be cleared. Therefore, it is meaningless to set the cache volume size much larger than `cacheLimitMB`
### Cache Volume Size
The cache volume size is calculated from below factors (for Restore scenarios):
- **Limit**: The limit of the cache data, that is represented by `cacheLimitMB`, the default value is 5GB
- **backupSize**: The size of the backup as a reference to evaluate whether to create a cache volume. It doesn't mean the backup data really decides the cache data all the time, it is just a reference to evaluate the scale of the backup, small scale backups may need small cache data. Sometimes, backupSize is not irrelevant to the size of cache data, in this case, ResidentThreshold should not be set, Limit will be used directly. It is unlikely that backupSize is unavailable, but once that happens, ResidentThreshold is ignored, Limit will be used directly.
- **ResidentThreshold**: The minimum backup size that a cache volume is created
- **InflationPercentage**: Considering the overhead of the file system and the possible delay of the cache cleanup, there should be an inflation for the final volume size vs. the logical size, otherwise, the cache volume may be overrun. This inflation percentage is hardcoded, e.g., 20%.
A formula is as below:
```
cacheVolumeSize = ((backupSize != 0 ? (backupSize > residentThreshold ? limit : 0) : limit) * (100 + inflationPercentage)) / 100
```
Finally, the `cacheVolumeSize` will be rounded up to GiB considering the UX friendliness, storage friendliness and management friendliness.
### PVC/PV
The PVC for a cache volume is created in Velero namespace and a storage class is required for the cache PVC. The PVC's accessMode is `ReadWriteOnce` and volumeMode is `FileSystem`, so the storage class provided should support this specification. Otherwise, if the storageclass doesn't support either of the specifications, the data mover pod may be hang in `Pending` state until a timeout setting with the data movement (e.g. `prepareTimeout`) and the data movement will finally fail.
It is not expected that the cache volume is retained after data mover pod is deleted, so the `reclaimPolicy` for the storageclass must be `Delete`.
To detect the problems in the storageclass and fail earlier, a validation is applied to the storageclass and once the validation fails, the cache configuration will be ignored, so the data mover pod will be created without a cache volume.
### Cache Volume Configurations
Below configurations are introduced:
- **residentThresholdMB**: the minimum data size(in MB) to be processed (if available) that a cache volume is created
- **cacheStorageClass**: the name of the storage class to provision the cache PVC
Not like `cacheLimitMB` which is set to and affect the backup repository, the above two configurations are actually data mover configurations of how to create cache volumes to data mover pods; and the two configurations don't need to be per backup repository. So we add them to the node-agent Configuration.
### Sample
Below are some examples of the node-agent configMap with the configurations:
Sample-1:
```json
{
"cacheVolume": {
"storageClass": "sc-1",
"residentThresholdMB": 1024
}
}
```
Sample-2:
```json
{
"cacheVolume": {
"storageClass": "sc-1",
}
}
```
Sample-3:
```json
{
"cacheVolume": {
"residentThresholdMB": 1024
}
}
```
**sample-1**: This is a valid configuration. Restores with backup data size larger than 1G will be assigned a cache volume using storage class `sc-1`.
**sample-2**: This is a valid configuration. Data mover pods are always assigned a cache volume using storage class `sc-1`.
**sample-3**: This is not a valid configuration because the storage class is absent. Velero gives up creating a cache volume.
To create the configMap, users need to save something like the above sample to a json file and then run below command:
```
kubectl create cm <ConfigMap name> -n velero --from-file=<json file name>
```
The cache volume configurations will be visited by node-agent server, so they also need to specify the `--node-agent-configmap` to the `velero node-agent` parameters.
## Detailed Design
### Backup and Restore
The restore needs to know the backup size so as to calculate the cache volume size, some new fields are added to the DataDownload and PodVolumeRestore CRDs.
`snapshotSize` field is also added to DataDownload and PodVolumeRestore's `spec`:
```yaml
spec:
snapshotID:
description: SnapshotID is the ID of the Velero backup snapshot to
be restored from.
type: string
snapshotSize:
description: SnapshotSize is the logical size of the snapshot.
format: int64
type: integer
```
`snapshotSize` represents the total size of the backup; during restore, the value is transferred from DataUpload/PodVolumeBackup's `Status.Progress.TotalBytes` to DataDownload/PodVolumeRestore.
It is unlikely that `Status.Progress.TotalBytes` from DataUpload/PodVolumeBackup is unavailable, but once it happens, according to the above formula, `residentThresholdMB` is ignored, cache volume size is calculated directly from cache limit for the corresponding backup repository.
### Exposer
Cache volume configurations are retrieved by node-agent and passed through DataDownload/PodVolumeRestore to GenericRestore exposer/PodVolume exposer.
The exposers are responsible to calculate cache volume size, create cache PVCs and mount them to the restorePods.
If the calculated cache volume size is 0, or any of the critical parameters is missing (e.g., cache volume storage class), the exposers ignore the cache volume configuration and continue with creating restorePods without cache volumes, so no impact to the result of the restore.
Exposers mount the cache volume to a predefined directory and pass the directory to the data mover pods through the `cache-volume-path` parameter.
Below data structure is added to the exposers' expose parameters:
```go
type GenericRestoreExposeParam struct {
// RestoreSize specifies the data size for the volume to be restored
RestoreSize int64
// CacheVolume specifies the info for cache volumes
CacheVolume *CacheVolumeInfo
}
type PodVolumeExposeParam struct {
// RestoreSize specifies the data size for the volume to be restored
RestoreSize int64
// CacheVolume specifies the info for cache volumes
CacheVolume *repocache.CacheConfigs
}
type CacheConfigs struct {
// StorageClass specifies the storage class for cache volumes
StorageClass string
// Limit specifies the maximum size of the cache data
Limit int64
// ResidentThreshold specifies the minimum size of the cache data to create a cache volume
ResidentThreshold int64
}
```
### Data Mover Pods
Data mover pods retrieve the cache volume directory from `cache-volume-path` parameter and pass it to Unified Repository.
If the directory is empty, Unified Repository uses the resident location for data cache, that is, the root file system.
### Kopia Repository
Kopia repository supports cache directory configuration for both metadata and data. The existing `SetupConnectOptions` is modified to customize the `CacheDirectory`:
```go
func SetupConnectOptions(ctx context.Context, repoOptions udmrepo.RepoOptions) repo.ConnectOptions {
...
return repo.ConnectOptions{
CachingOptions: content.CachingOptions{
CacheDirectory: cacheDir,
...
},
...
}
}
```
[1]: Implemented/unified-repo-and-kopia-integration/unified-repo-and-kopia-integration.md
[2]: Implemented/vgdp-micro-service/vgdp-micro-service.md
[3]: Implemented/vgdp-micro-service-for-fs-backup/vgdp-micro-service-for-fs-backup.md
[4]: Implemented/repo_maintenance_job_config.md
[5]: Implemented/backup-repo-config.md

View File

@@ -1,417 +0,0 @@
# Design for BSL Certificate Support Enhancement
## Abstract
This design document describes the enhancement of BackupStorageLocation (BSL) certificate management in Velero, introducing a Secret-based certificate reference mechanism (`caCertRef`) alongside the existing inline certificate field (`caCert`). This enhancement provides a more secure, Kubernetes-native approach to certificate management while enabling future CLI improvements for automatic certificate discovery.
## Background
Currently, Velero supports TLS certificate verification for object storage providers through an inline `caCert` field in the BSL specification. While functional, this approach has several limitations:
- **Security**: Certificates are stored directly in the BSL YAML, potentially exposing sensitive data
- **Management**: Certificate rotation requires updating the BSL resource itself
- **CLI Usability**: Users must manually specify certificates when using CLI commands
- **Size Limitations**: Large certificate bundles can make BSL resources unwieldy
Issue #9097 and PR #8557 highlight the need for improved certificate management that addresses these concerns while maintaining backward compatibility.
## Goals
- Provide a secure, Secret-based certificate storage mechanism
- Maintain full backward compatibility with existing BSL configurations
- Enable future CLI enhancements for automatic certificate discovery
- Simplify certificate rotation and management
- Provide clear migration path for existing users
## Non-Goals
- Removing support for inline certificates immediately
- Changing the behavior of existing BSL configurations
- Implementing client-side certificate validation
- Supporting certificates from ConfigMaps or other resource types
## High-Level Design
### API Changes
#### New Field: CACertRef
```go
type ObjectStorageLocation struct {
// Existing field (now deprecated)
// +optional
// +kubebuilder:deprecatedversion:warning="caCert is deprecated, use caCertRef instead"
CACert []byte `json:"caCert,omitempty"`
// New field for Secret reference
// +optional
CACertRef *corev1api.SecretKeySelector `json:"caCertRef,omitempty"`
}
```
The `SecretKeySelector` follows standard Kubernetes patterns:
```go
type SecretKeySelector struct {
// Name of the Secret
Name string `json:"name"`
// Key within the Secret
Key string `json:"key"`
}
```
### Certificate Resolution Logic
The system follows a priority-based resolution:
1. If `caCertRef` is specified, retrieve certificate from the referenced Secret
2. If `caCert` is specified (and `caCertRef` is not), use the inline certificate
3. If neither is specified, no custom CA certificate is used
### Validation
BSL validation ensures mutual exclusivity:
```go
func (bsl *BackupStorageLocation) Validate() error {
if bsl.Spec.ObjectStorage != nil &&
bsl.Spec.ObjectStorage.CACert != nil &&
bsl.Spec.ObjectStorage.CACertRef != nil {
return errors.New("cannot specify both caCert and caCertRef in objectStorage")
}
return nil
}
```
## Detailed Design
### BSL Controller Changes
The BSL controller incorporates validation during reconciliation:
```go
func (r *backupStorageLocationReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) {
// ... existing code ...
// Validate BSL configuration
if err := location.Validate(); err != nil {
r.logger.WithError(err).Error("BSL validation failed")
return ctrl.Result{}, err
}
// ... continue reconciliation ...
}
```
### Repository Provider Integration
All repository providers implement consistent certificate handling:
```go
func configureCACert(bsl *velerov1api.BackupStorageLocation, credGetter *credentials.CredentialGetter) ([]byte, error) {
if bsl.Spec.ObjectStorage == nil {
return nil, nil
}
// Prefer caCertRef (new method)
if bsl.Spec.ObjectStorage.CACertRef != nil {
certString, err := credGetter.FromSecret.Get(bsl.Spec.ObjectStorage.CACertRef)
if err != nil {
return nil, errors.Wrap(err, "error getting CA certificate from secret")
}
return []byte(certString), nil
}
// Fall back to caCert (deprecated)
if bsl.Spec.ObjectStorage.CACert != nil {
return bsl.Spec.ObjectStorage.CACert, nil
}
return nil, nil
}
```
### CLI Certificate Discovery Integration
#### Background: PR #8557 Implementation
PR #8557 ("CLI automatically discovers and uses cacert from BSL") was merged in August 2025, introducing automatic CA certificate discovery from BackupStorageLocation for Velero CLI download operations. This eliminated the need for users to manually specify the `--cacert` flag when performing operations like `backup describe`, `backup download`, `backup logs`, and `restore logs`.
#### Current Implementation (Post PR #8557)
The CLI now automatically discovers certificates from BSL through the `pkg/cmd/util/cacert/bsl_cacert.go` module:
```go
// Current implementation only supports inline caCert
func GetCACertFromBSL(ctx context.Context, client kbclient.Client, namespace, bslName string) (string, error) {
// ... fetch BSL ...
if bsl.Spec.ObjectStorage != nil && len(bsl.Spec.ObjectStorage.CACert) > 0 {
return string(bsl.Spec.ObjectStorage.CACert), nil
}
return "", nil
}
```
#### Enhancement with caCertRef Support
This design extends the existing CLI certificate discovery to support the new `caCertRef` field:
```go
// Enhanced implementation supporting both caCert and caCertRef
func GetCACertFromBSL(ctx context.Context, client kbclient.Client, namespace, bslName string) (string, error) {
// ... fetch BSL ...
// Prefer caCertRef over inline caCert
if bsl.Spec.ObjectStorage.CACertRef != nil {
secret := &corev1api.Secret{}
key := types.NamespacedName{
Name: bsl.Spec.ObjectStorage.CACertRef.Name,
Namespace: namespace,
}
if err := client.Get(ctx, key, secret); err != nil {
return "", errors.Wrap(err, "error getting certificate secret")
}
certData, ok := secret.Data[bsl.Spec.ObjectStorage.CACertRef.Key]
if !ok {
return "", errors.Errorf("key %s not found in secret",
bsl.Spec.ObjectStorage.CACertRef.Key)
}
return string(certData), nil
}
// Fall back to inline caCert (deprecated)
if bsl.Spec.ObjectStorage.CACert != nil {
return string(bsl.Spec.ObjectStorage.CACert), nil
}
return "", nil
}
```
#### Certificate Resolution Priority
The CLI follows this priority order for certificate resolution:
1. **`--cacert` flag** - Manual override, highest priority
2. **`caCertRef`** - Secret-based certificate (recommended)
3. **`caCert`** - Inline certificate (deprecated)
4. **System certificate pool** - Default fallback
#### User Experience Improvements
With both PR #8557 and this enhancement:
```bash
# Automatic discovery - works with both caCert and caCertRef
velero backup describe my-backup
velero backup download my-backup
velero backup logs my-backup
velero restore logs my-restore
# Manual override still available
velero backup describe my-backup --cacert /custom/ca.crt
# Debug output shows certificate source
velero backup download my-backup --log-level=debug
# [DEBUG] Resolved CA certificate from BSL 'default' Secret 'storage-ca-cert' key 'ca-bundle.crt'
```
#### RBAC Considerations for CLI
CLI users need read access to Secrets when using `caCertRef`:
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: velero-cli-user
namespace: velero
rules:
- apiGroups: ["velero.io"]
resources: ["backups", "restores", "backupstoragelocations"]
verbs: ["get", "list"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get"]
# Limited to secrets referenced by BSLs
```
### Migration Strategy
#### Phase 1: Introduction (Current)
- Add `caCertRef` field
- Mark `caCert` as deprecated
- Both fields supported, mutual exclusivity enforced
#### Phase 2: Migration Period
- Documentation and tools to help users migrate
- Warning messages for `caCert` usage
- CLI enhancements to leverage `caCertRef`
#### Phase 3: Future Removal
- Remove `caCert` field in major version update
- Provide migration tool for automatic conversion
## User Experience
### Creating a BSL with Certificate Reference
1. Create a Secret containing the CA certificate:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: storage-ca-cert
namespace: velero
type: Opaque
data:
ca-bundle.crt: <base64-encoded-certificate>
```
2. Reference the Secret in BSL:
```yaml
apiVersion: velero.io/v1
kind: BackupStorageLocation
metadata:
name: default
namespace: velero
spec:
provider: aws
objectStorage:
bucket: my-bucket
caCertRef:
name: storage-ca-cert
key: ca-bundle.crt
```
### Certificate Rotation
With Secret-based certificates:
```bash
# Update the Secret with new certificate
kubectl create secret generic storage-ca-cert \
--from-file=ca-bundle.crt=new-ca.crt \
--dry-run=client -o yaml | kubectl apply -f -
# No BSL update required - changes take effect on next use
```
### CLI Usage Examples
#### Immediate Benefits
- No change required for existing workflows
- Certificate validation errors include helpful context
#### Future CLI Enhancements
```bash
# Automatic certificate discovery
velero backup download my-backup
# Manual override still available
velero backup download my-backup --cacert /custom/ca.crt
# Debug certificate resolution
velero backup download my-backup --log-level=debug
# [DEBUG] Resolved CA certificate from BSL 'default' Secret 'storage-ca-cert'
```
## Security Considerations
### Advantages of Secret-based Storage
1. **Encryption at Rest**: Secrets are encrypted in etcd
2. **RBAC Control**: Fine-grained access control via Kubernetes RBAC
3. **Audit Trail**: Secret access is auditable
4. **Separation of Concerns**: Certificates separate from configuration
### Required Permissions
The Velero server requires additional RBAC permissions:
```yaml
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get"]
# Scoped to secrets referenced by BSLs
```
## Compatibility
### Backward Compatibility
- Existing BSLs with `caCert` continue to function unchanged
- No breaking changes to API
- Gradual migration path
### Forward Compatibility
- Design allows for future enhancements:
- Multiple certificate support
- Certificate chain validation
- Automatic certificate discovery from cloud providers
## Implementation Phases
### Phase 1: Core Implementation ✓ (Current PR)
- API changes with new `caCertRef` field
- Controller validation
- Repository provider updates
- Basic testing
### Phase 2: CLI Enhancement (Future)
- Automatic certificate discovery in CLI
- Enhanced error messages
- Debug logging for certificate resolution
### Phase 3: Migration Tools (Future)
- Automated migration scripts
- Validation tools
- Documentation updates
## Testing
### Unit Tests
- BSL validation logic
- Certificate resolution in providers
- Controller behavior
### Integration Tests
- End-to-end backup/restore with `caCertRef`
- Certificate rotation scenarios
- Migration from `caCert` to `caCertRef`
### Manual Testing Scenarios
1. Create BSL with `caCertRef`
2. Perform backup/restore operations
3. Rotate certificate in Secret
4. Verify continued operation
## Documentation
### User Documentation
- Migration guide from `caCert` to `caCertRef`
- Examples for common cloud providers
- Troubleshooting guide
### API Documentation
- Updated API reference
- Deprecation notices
- Field descriptions
## Alternatives Considered
### ConfigMap-based Storage
- Pros: Similar to Secrets, simpler API
- Cons: Not designed for sensitive data, no encryption at rest
- Decision: Secrets are the Kubernetes-standard for sensitive data
### External Certificate Management
- Pros: Integration with cert-manager, etc.
- Cons: Additional complexity, dependencies
- Decision: Keep it simple, allow users to manage certificates as needed
### Immediate Removal of Inline Certificates
- Pros: Cleaner API, forces best practices
- Cons: Breaking change, migration burden
- Decision: Gradual deprecation respects existing users
## Conclusion
This design provides a secure, Kubernetes-native approach to certificate management in Velero while maintaining backward compatibility. It establishes the foundation for enhanced CLI functionality and improved user experience, addressing the concerns raised in issue #9097 and enabling the features proposed in PR #8557.
The phased approach ensures smooth migration for existing users while delivering immediate security benefits for new deployments.

View File

@@ -1,374 +0,0 @@
# Design to clean the artifacts generated in the CSI backup and restore workflows
## Terminology
* VSC: VolumeSnapshotContent
* VS: VolumeSnapshot
## Abstract
* The design aims to delete the unnecessary VSs and VSCs generated during CSI backup and restore process.
* The design stop creating related VSCs during backup syncing.
## Background
In the current CSI backup and restore workflows, please notice the CSI B/R workflows means only using the CSI snapshots in the B/R, not including the CSI snapshot data movement workflows, some generated artifacts are kept after the backup or the restore process completion.
Some of them are kept due to design, for example, the VolumeSnapshotContents generated during the backup are kept to make sure the backup deletion can clean the snapshots in the storage providers.
Some of them are kept by accident, for example, after restore, two VolumeSnapshotContents are generated for the same VolumeSnapshot. One is from the backup content, and one is dynamically generated from the restore's VolumeSnapshot.
The design aims to clean the unnecessary artifacts, and make the CSI B/R workflow more concise and reliable.
## Goals
- Clean the redundant VSC generated during CSI backup and restore.
- Remove the VSCs in the backup sync process.
## Non Goals
- There were some discussion about whether Velero backup should include VSs and VSCs not generated in during the backup. By far, the conclusion is not including them is a better option. Although that is a useful enhancement, that is not included this design.
- Delete all the CSI-related metadata files in the BSL is not the aim of this design.
## Detailed Design
### Backup
During backup, the main change is the backup-generated VSCs should not kept anymore.
The reasons is we don't need them to ensure the snapshots clean up during backup deletion. Please reference to the [Backup Deletion section](#backup-deletion) section for detail.
As a result, we can simplify the VS deletion logic in the backup. Before, we need to not only delete the VS, but also recreate a static VSC pointing a non-exiting VS.
The deletion code in VS BackupItemAction can be simplify to the following:
``` go
if backup.Status.Phase == velerov1api.BackupPhaseFinalizing ||
backup.Status.Phase == velerov1api.BackupPhaseFinalizingPartiallyFailed {
p.log.
WithField("Backup", fmt.Sprintf("%s/%s", backup.Namespace, backup.Name)).
WithField("BackupPhase", backup.Status.Phase).Debugf("Cleaning VolumeSnapshots.")
if vsc == nil {
vsc = &snapshotv1api.VolumeSnapshotContent{}
}
csi.DeleteReadyVolumeSnapshot(*vs, *vsc, p.crClient, p.log)
return item, nil, "", nil, nil
}
func DeleteReadyVolumeSnapshot(
vs snapshotv1api.VolumeSnapshot,
vsc snapshotv1api.VolumeSnapshotContent,
client crclient.Client,
logger logrus.FieldLogger,
) {
logger.Infof("Deleting Volumesnapshot %s/%s", vs.Namespace, vs.Name)
if vs.Status == nil ||
vs.Status.BoundVolumeSnapshotContentName == nil ||
len(*vs.Status.BoundVolumeSnapshotContentName) <= 0 {
logger.Errorf("VolumeSnapshot %s/%s is not ready. This is not expected.",
vs.Namespace, vs.Name)
return
}
if vs.Status != nil && vs.Status.BoundVolumeSnapshotContentName != nil {
// Patch the DeletionPolicy of the VolumeSnapshotContent to set it to Retain.
// This ensures that the volume snapshot in the storage provider is kept.
if err := SetVolumeSnapshotContentDeletionPolicy(
vsc.Name,
client,
snapshotv1api.VolumeSnapshotContentRetain,
); err != nil {
logger.Warnf("Failed to patch DeletionPolicy of volume snapshot %s/%s",
vs.Namespace, vs.Name)
return
}
if err := client.Delete(context.TODO(), &vsc); err != nil {
logger.Warnf("Failed to delete the VSC %s: %s", vsc.Name, err.Error())
}
}
if err := client.Delete(context.TODO(), &vs); err != nil {
logger.Warnf("Failed to delete volumesnapshot %s/%s: %v", vs.Namespace, vs.Name, err)
} else {
logger.Infof("Deleted volumesnapshot with volumesnapshotContent %s/%s",
vs.Namespace, vs.Name)
}
}
```
### Restore
#### Restore the VolumeSnapshotContent
The current behavior of VSC restoration is that the VSC from the backup is restore, and the restored VS also triggers creating a new VSC dynamically.
Two VSCs created for the same VS in one restore seems not right.
Skip restore the VSC from the backup is not a viable alternative, because VSC may reference to a [snapshot create secret](https://kubernetes-csi.github.io/docs/secrets-and-credentials-volume-snapshot-class.html?highlight=snapshotter-secret-name#createdelete-volumesnapshot-secret).
If the `SkipRestore` is set true in the restore action's result, the secret returned in the additional items is ignored too.
As a result, restore the VSC from the backup, and setup the VSC and the VS's relation is a better choice.
Another consideration is the VSC name should not be the same as the backed-up VSC's, because the older version Velero's restore and backup keep the VSC after completion.
There's high possibility that the restore will fail due to the VSC already exists in the cluster.
Multiple restores of the same backup will also meet the same problem.
The proposed solution is using the restore's UID and the VS's name to generate sha256 hash value as the new VSC name. Both the VS and VSC RestoreItemAction can access those UIDs, and it will avoid the conflicts issues.
The restored VS name also shares the same generated name.
The VS-referenced VSC name and the VSC's snapshot handle name are in their status.
Velero restore process purges the restore resources' metadata and status before running the RestoreItemActions.
As a result, we cannot read these information in the VS and VSC RestoreItemActions.
Fortunately, RestoreItemAction input parameters includes the `ItemFromBackup`. The status is intact in `ItemFromBackup`.
``` go
func (p *volumeSnapshotRestoreItemAction) Execute(
input *velero.RestoreItemActionExecuteInput,
) (*velero.RestoreItemActionExecuteOutput, error) {
p.log.Info("Starting VolumeSnapshotRestoreItemAction")
if boolptr.IsSetToFalse(input.Restore.Spec.RestorePVs) {
p.log.Infof("Restore %s/%s did not request for PVs to be restored.",
input.Restore.Namespace, input.Restore.Name)
return &velero.RestoreItemActionExecuteOutput{SkipRestore: true}, nil
}
var vs snapshotv1api.VolumeSnapshot
if err := runtime.DefaultUnstructuredConverter.FromUnstructured(
input.Item.UnstructuredContent(), &vs); err != nil {
return &velero.RestoreItemActionExecuteOutput{},
errors.Wrapf(err, "failed to convert input.Item from unstructured")
}
var vsFromBackup snapshotv1api.VolumeSnapshot
if err := runtime.DefaultUnstructuredConverter.FromUnstructured(
input.ItemFromBackup.UnstructuredContent(), &vsFromBackup); err != nil {
return &velero.RestoreItemActionExecuteOutput{},
errors.Wrapf(err, "failed to convert input.Item from unstructured")
}
// If cross-namespace restore is configured, change the namespace
// for VolumeSnapshot object to be restored
newNamespace, ok := input.Restore.Spec.NamespaceMapping[vs.GetNamespace()]
if !ok {
// Use original namespace
newNamespace = vs.Namespace
}
if csiutil.IsVolumeSnapshotExists(newNamespace, vs.Name, p.crClient) {
p.log.Debugf("VolumeSnapshot %s already exists in the cluster. Return without change.", vs.Namespace+"/"+vs.Name)
return &velero.RestoreItemActionExecuteOutput{UpdatedItem: input.Item}, nil
}
newVSCName := generateSha256FromRestoreAndVsUID(string(input.Restore.UID), string(vsFromBackup.UID))
// Reset Spec to convert the VolumeSnapshot from using
// the dynamic VolumeSnapshotContent to the static one.
resetVolumeSnapshotSpecForRestore(&vs, &newVSCName)
// Reset VolumeSnapshot annotation. By now, only change
// DeletionPolicy to Retain.
resetVolumeSnapshotAnnotation(&vs)
vsMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&vs)
if err != nil {
p.log.Errorf("Fail to convert VS %s to unstructured", vs.Namespace+"/"+vs.Name)
return nil, errors.WithStack(err)
}
p.log.Infof(`Returning from VolumeSnapshotRestoreItemAction with
no additionalItems`)
return &velero.RestoreItemActionExecuteOutput{
UpdatedItem: &unstructured.Unstructured{Object: vsMap},
AdditionalItems: []velero.ResourceIdentifier{},
}, nil
}
// generateSha256FromRestoreAndVsUID Use the restore UID and the VS UID to generate the new VSC name.
// By this way, VS and VSC RIA action can get the same VSC name.
func generateSha256FromRestoreAndVsUID(restoreUID string, vsUID string) string {
sha256Bytes := sha256.Sum256([]byte(restoreUID + "/" + vsUID))
return "vsc-" + hex.EncodeToString(sha256Bytes[:])
}
```
#### Restore the VolumeSnapshot
``` go
// Execute restores a VolumeSnapshotContent object without modification
// returning the snapshot lister secret, if any, as additional items to restore.
func (p *volumeSnapshotContentRestoreItemAction) Execute(
input *velero.RestoreItemActionExecuteInput,
) (*velero.RestoreItemActionExecuteOutput, error) {
if boolptr.IsSetToFalse(input.Restore.Spec.RestorePVs) {
p.log.Infof("Restore did not request for PVs to be restored %s/%s",
input.Restore.Namespace, input.Restore.Name)
return &velero.RestoreItemActionExecuteOutput{SkipRestore: true}, nil
}
p.log.Info("Starting VolumeSnapshotContentRestoreItemAction")
var vsc snapshotv1api.VolumeSnapshotContent
if err := runtime.DefaultUnstructuredConverter.FromUnstructured(
input.Item.UnstructuredContent(), &vsc); err != nil {
return &velero.RestoreItemActionExecuteOutput{},
errors.Wrapf(err, "failed to convert input.Item from unstructured")
}
var vscFromBackup snapshotv1api.VolumeSnapshotContent
if err := runtime.DefaultUnstructuredConverter.FromUnstructured(
input.ItemFromBackup.UnstructuredContent(), &vscFromBackup); err != nil {
return &velero.RestoreItemActionExecuteOutput{},
errors.Errorf(err.Error(), "failed to convert input.ItemFromBackup from unstructured")
}
// If cross-namespace restore is configured, change the namespace
// for VolumeSnapshot object to be restored
newNamespace, ok := input.Restore.Spec.NamespaceMapping[vsc.Spec.VolumeSnapshotRef.Namespace]
if ok {
// Update the referenced VS namespace to the mapping one.
vsc.Spec.VolumeSnapshotRef.Namespace = newNamespace
}
// Reset VSC name to align with VS.
vsc.Name = generateSha256FromRestoreAndVsUID(string(input.Restore.UID), string(vscFromBackup.Spec.VolumeSnapshotRef.UID))
// Reset the ResourceVersion and UID of referenced VolumeSnapshot.
vsc.Spec.VolumeSnapshotRef.ResourceVersion = ""
vsc.Spec.VolumeSnapshotRef.UID = ""
// Set the DeletionPolicy to Retain to avoid VS deletion will not trigger snapshot deletion
vsc.Spec.DeletionPolicy = snapshotv1api.VolumeSnapshotContentRetain
if vscFromBackup.Status != nil && vscFromBackup.Status.SnapshotHandle != nil {
vsc.Spec.Source.VolumeHandle = nil
vsc.Spec.Source.SnapshotHandle = vscFromBackup.Status.SnapshotHandle
} else {
p.log.Errorf("fail to get snapshot handle from VSC %s status", vsc.Name)
return nil, errors.Errorf("fail to get snapshot handle from VSC %s status", vsc.Name)
}
additionalItems := []velero.ResourceIdentifier{}
if csi.IsVolumeSnapshotContentHasDeleteSecret(&vsc) {
additionalItems = append(additionalItems,
velero.ResourceIdentifier{
GroupResource: schema.GroupResource{Group: "", Resource: "secrets"},
Name: vsc.Annotations[velerov1api.PrefixedSecretNameAnnotation],
Namespace: vsc.Annotations[velerov1api.PrefixedSecretNamespaceAnnotation],
},
)
}
vscMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&vsc)
if err != nil {
return nil, errors.WithStack(err)
}
p.log.Infof("Returning from VolumeSnapshotContentRestoreItemAction with %d additionalItems",
len(additionalItems))
return &velero.RestoreItemActionExecuteOutput{
UpdatedItem: &unstructured.Unstructured{Object: vscMap},
AdditionalItems: additionalItems,
}, nil
}
```
### Backup Sync
csi-volumesnapshotclasses.json, csi-volumesnapshotcontents.json, and csi-volumesnapshots.json are CSI-related metadata files in the BSL for each backup.
csi-volumesnapshotcontents.json and csi-volumesnapshots.json are not needed anymore, but csi-volumesnapshotclasses.json is still needed.
One concrete scenario is that a backup is created in cluster-A, then the backup is synced to cluster-B, and the backup is deleted in the cluster-B. In this case, we don't have a chance to create the VS and VSC needed VolumeSnapshotClass.
The VSC deletion workflow proposed by this design needs to create the VSC first. If the VSC's referenced VolumeSnapshotClass doesn't exist in cluster, the creation of VSC will fail.
As a result, the VolumeSnapshotClass should still be synced in the backup sync process.
### Backup Deletion
Two factors are worthy for consideration for the backup deletion change:
* Because the VSCs generated by the backup are not synced anymore, and the VSCs generated during the backup will not be kept too. The backup deletion needs to generate a VSC, then deletes it to make sure the snapshots in the storage provider are clean too.
* The VSs generated by the backup are already deleted in the backup process, we don't need a DeleteItemAction for the VS anymore. As a result, the `velero.io/csi-volumesnapshot-delete` plugin is unneeded.
For the VSC DeleteItemAction, we need to generate a VSC. Because we only care about the snapshot deletion, we don't need to create a VS associated with the VSC.
Create a static VSC, then point it to a pseudo VS, and reference to the snapshot handle should be enough.
To avoid the created VSC conflict with older version Velero B/R generated ones, the VSC name is set to `vsc-uuid`.
The following is an example of the implementation.
``` go
uuid, err := uuid.NewRandom()
if err != nil {
p.log.WithError(err).Errorf("Fail to generate the UUID to create VSC %s", snapCont.Name)
return errors.Wrapf(err, "Fail to generate the UUID to create VSC %s", snapCont.Name)
}
snapCont.Name = "vsc-" + uuid.String()
snapCont.Spec.DeletionPolicy = snapshotv1api.VolumeSnapshotContentDelete
snapCont.Spec.Source = snapshotv1api.VolumeSnapshotContentSource{
SnapshotHandle: snapCont.Status.SnapshotHandle,
}
snapCont.Spec.VolumeSnapshotRef = corev1api.ObjectReference{
APIVersion: snapshotv1api.SchemeGroupVersion.String(),
Kind: "VolumeSnapshot",
Namespace: "ns-" + string(snapCont.UID),
Name: "name-" + string(snapCont.UID),
}
snapCont.ResourceVersion = ""
if err := p.crClient.Create(context.TODO(), &snapCont); err != nil {
return errors.Wrapf(err, "fail to create VolumeSnapshotContent %s", snapCont.Name)
}
// Read resource timeout from backup annotation, if not set, use default value.
timeout, err := time.ParseDuration(
input.Backup.Annotations[velerov1api.ResourceTimeoutAnnotation])
if err != nil {
p.log.Warnf("fail to parse resource timeout annotation %s: %s",
input.Backup.Annotations[velerov1api.ResourceTimeoutAnnotation], err.Error())
timeout = 10 * time.Minute
}
p.log.Debugf("resource timeout is set to %s", timeout.String())
interval := 5 * time.Second
// Wait until VSC created and ReadyToUse is true.
if err := wait.PollUntilContextTimeout(
context.Background(),
interval,
timeout,
true,
func(ctx context.Context) (bool, error) {
tmpVSC := new(snapshotv1api.VolumeSnapshotContent)
if err := p.crClient.Get(ctx, crclient.ObjectKeyFromObject(&snapCont), tmpVSC); err != nil {
return false, errors.Wrapf(
err, "failed to get VolumeSnapshotContent %s", snapCont.Name,
)
}
if tmpVSC.Status != nil && boolptr.IsSetToTrue(tmpVSC.Status.ReadyToUse) {
return true, nil
}
return false, nil
},
); err != nil {
return errors.Wrapf(err, "fail to wait VolumeSnapshotContent %s becomes ready.", snapCont.Name)
}
```
## Security Considerations
Security is not relevant to this design.
## Compatibility
In this design, no new information is added in backup and restore. As a result, this design doesn't have any compatibility issue.
## Open Issues
Please notice the CSI snapshot backup and restore mechanism not supporting all file-store-based volume, e.g. Azure Files, EFS or vSphere CNS File Volume. Only block-based volumes are supported.
Refer to [this comment](https://github.com/vmware-tanzu/velero/issues/3151#issuecomment-2623507686) for more details.

Some files were not shown because too many files have changed in this diff Show More