Commit Graph

133 Commits

Author SHA1 Message Date
Scott Seago
91357b28c4 Move worker pool creation to backup reconcile.
ItemBlockWorkerPool is now created for each backup.

Signed-off-by: Scott Seago <sseago@redhat.com>
2025-12-02 16:38:41 -05:00
Scott Seago
d39ad6f208 run multiple backup reconcilers, only reconcile ReadyToStart backups
Signed-off-by: Scott Seago <sseago@redhat.com>
2025-12-02 16:38:41 -05:00
Daniel Jiang
249d8f581a Add include/exclude policy to resources policy
fixes #8610

This commit extends the resources policy, such that user can define
resource include exclude filters in the policy and reuse it in different backups.

Signed-off-by: Daniel Jiang <daniel.jiang@broadcom.com>
2025-08-05 15:16:59 +08:00
Lyndon-Li
34f8b73507 bump up kopia to v0.21.1
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-07-22 15:56:04 +08:00
Shubham Pampattiwar
a73a150d98 Accommodate VGS workflows in PVC CSI plugin
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

Add changelog file

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

make update

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

lint fix

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

add unit tests for getVSForPVC func

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

Use v1beta1 instead of v1 v1alpha1

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

go mod tidy

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

update updateVGSCreatedVS func to use retry on conflict

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

make update minor fix

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

fix ut assert

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

Address PR feedback

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

minor updates

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

remove unsused func and add todo for dep upgrades

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2025-07-21 11:36:40 -07:00
Matthieu MOREL
07ea14962c fix require-error rule from testifylint
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2025-06-23 15:39:54 +00:00
Shubham Pampattiwar
97a4d62d3c Extend PVCAction itemblock plugin to support grouping PVCs under VolumeGroupSnapshot label
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

Add changelog file

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

Update VGS label key and address PR feedback

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

update log level to debug for edge cases

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

Change VGS label key constant location

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

run make update

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2025-06-10 07:01:45 -07:00
Xun Jiang
1b7175394a Skip VS and VSC not created by backup.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2025-06-07 10:51:05 +08:00
Xun Jiang/Bruce Jiang
2390bc8e71 Merge pull request #8976 from blackpiglet/7785_fix
Add BSL status check for backup/restore operations.
2025-06-06 11:22:10 +08:00
Shubham Pampattiwar
d2c6b6bc3e Add support for configuring VGS label key (#8938)
Some checks failed
Run the E2E test on kind / build (push) Failing after 6m38s
Run the E2E test on kind / setup-test-matrix (push) Successful in 2s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / Build (push) Failing after 31s
Close stale issues and PRs / stale (push) Successful in 10s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 1m17s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-aws, main) (push) Failing after 1m40s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-gcp, main) (push) Failing after 49s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-microsoft-azure, main) (push) Failing after 45s
add changelog file

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2025-05-30 11:03:47 -04:00
Xun Jiang
f604a5da48 Add BSL status check for backup/restore operations.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2025-05-30 10:21:32 +08:00
Matthieu MOREL
5705bcc3f4 chore: bump golangci-lint to v2 (#8839)
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2025-04-17 02:38:19 -04:00
Scott Seago
fcfb2fd9ee Implement parallel ItemBlock processing via backup_controller goroutines
Signed-off-by: Scott Seago <sseago@redhat.com>
2025-02-12 12:03:37 -05:00
Shubham Pampattiwar
f15cde5dfd Remove mutiple single quotes from Velero backup.status.validationErrors field
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

update error message

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2024-09-30 14:20:44 -07:00
Scott Seago
9d6f4d2db5 ItemBlock model and phase 1 (single-thread) workflow changes
Signed-off-by: Scott Seago <sseago@redhat.com>
2024-09-03 19:04:18 -04:00
Matthieu MOREL
c8baaa9b11 testifylint: enable more rules (#8024)
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-07-18 10:43:16 -04:00
Matthieu MOREL
35c90f1672 testifylint: enable error-nil rule (#7670)
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-07-16 12:23:16 -04:00
Scott Seago
dc286a38fc Reuse existing plugin manager for get/put volume info
Signed-off-by: Scott Seago <sseago@redhat.com>
2024-07-12 10:15:16 -04:00
Xun Jiang
cf5dfdf42d Check whether the namespaces specified in namespace filter exist.
Check whether the namespaces specified in the
backup.Spec.IncludeNamespaces exist during backup resource collcetion
If not, log error to mark the backup as PartiallyFailed.

Signed-off-by: Xun Jiang <blackpigletbruce@gmail.com>
2024-07-04 10:02:10 +08:00
Xun Jiang/Bruce Jiang
d9c9f77860 Merge pull request #7671 from mmorel-35/testifylint/compare
testifylint: enable compares rule
2024-05-27 12:45:16 +08:00
Xun Jiang
a91d2cb036 Modify the volume helper logic.
Signed-off-by: Xun Jiang <blackpigletbruce@gmail.com>
2024-05-23 09:57:21 +08:00
Matthieu MOREL
1010b04821 testifylint: enable compares rule
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-05-16 20:18:43 +00:00
Xun Jiang/Bruce Jiang
f25c154709 Merge pull request #7569 from ywk253100/240326_namespace
Check the existence of the namespaces provided in the "--include-namespaces" option
2024-04-11 19:00:33 +08:00
Xun Jiang
5462035469 Delete the unneeded pvRestorer action in
handleSkippedPVHasRetainPolicy

According to comment, calling executePVAction aims to reset PV's
claimRef, but the reset logic was moved into resetVolumeBindingInfo
since release-1.4.

Signed-off-by: Xun Jiang <blackpigletbruce@gmail.com>
2024-03-29 14:12:12 +08:00
Xun Jiang
b06d7a467f Support update the backup VolumeInfos by the Async ops result.
1. Add PutBackupVolumeInfos method.
2. Add CompletionTimestamp in VolumeInfo.
3. Add Size in SnapshotDataMovementInfo.
4. Update CompletionTimpstmap, SnapshotHandle, RetainedSnapshot
   and Size in VolumeInfo on DataUpload Operation completes.

Signed-off-by: Xun Jiang <blackpigletbruce@gmail.com>
2024-03-28 19:52:44 +08:00
Matthieu MOREL
ef04ef6361 golangci-lint: use exclude-rules instead of skip-files and skip-dirs
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-03-27 20:17:34 +00:00
Wenkai Yin(尹文开)
35d2534e19 Check the existence of the namespaces provided in the "--include-namespaces" option
Check the existence of the namespaces provided in the "--include-namespaces" opt
ion and reports validation error if not found

Fixes #7431

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2024-03-27 18:37:03 +08:00
Wenkai Yin(尹文开)
8752c3a820 Bump up the versions of severel Kubernetes-related libs
Bump up the versions of severel Kubernetes-related libs

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2024-03-05 13:09:38 +08:00
Daniel Jiang
804b9a8d91 Merge pull request #7171 from kaovilai/tests-explicit-enableCSI
Add explicit enableCSI to TestProcessBackupCompletions
2023-12-11 14:11:37 +08:00
Xun Jiang
ca97248f2a Use VolumeInfo to help restore the PV.
Add VolumeInfo for left PVs during backup.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-12-04 08:33:37 +08:00
Tiger Kaovilai
2132506e8c Add explicit enableCSI to TestProcessBackupCompletions
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-12-01 14:22:40 -05:00
Xun Jiang
7f52321772 Generate VolumeInfo.
Remove CSI VolumeSnapshot listter and the informer.
Add download the VolumeInfos metadata for backup.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-11-22 09:40:38 +08:00
Tiger Kaovilai
8c727429c4 revert test changes
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-11-02 17:06:19 -04:00
Tiger Kaovilai
9311a4269b refactor backup snapshot status updates into UpdateBackupSnapshotsStatus() and run in backup_finalizer_controller
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-11-02 15:30:35 -04:00
Tiger Kaovilai
886e074b55 Add PatchResource unit test for backup status
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-11-01 15:28:56 -04:00
Lyndon-Li
778feba3ae fix issue 6647
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-09-04 16:55:36 +08:00
Wenkai Yin(尹文开)
89c10ddcc0 Update metrics when backup failed with validation error (#6318)
Update metrics when backup failed with validation error

Fixes #6265

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2023-06-09 11:19:26 +08:00
Xun Jiang/Bruce Jiang
9743a7ce56 Fix #6118: Do not persist VolumeSnapshot and VolumeSnapshotContent for snapshot DataMover case. (#6366)
1. Because VolumeSnapshot and VolumeSnapshotContent CRs are not kept after backup completed,
   don't persist them in the backup metadata.
2. Add some builder methods needed by CSI plugin.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-06-08 17:05:17 +08:00
Xun Jiang
7491ab1ec5 Remove Wait VolumeSnapshot to ReadyToUse logic.
Due to the logic moving to plugin, and the plugin cannot read the
Velero server's resourceTimeout setting, add the resourceTimeout
in the backup annotation to pass to plugin.
Remove VolumeSnapshotContent reset code from Velero server.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-06-02 14:43:43 +08:00
Xun Jiang
bbc1e2e151 Enable stylecheck linter and resolve found issues.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-04-25 14:37:02 +08:00
Tiger Kaovilai
da0f5d5850 Field labels for Duration, Time, client.ObjectKey, KibishiiData, VeleroCLI2Version
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-04-16 12:34:28 -04:00
Xun Jiang
56f93393d8 Modify new scope resource filters name.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-04-06 18:11:55 +08:00
lyndon
2ad43194aa Merge pull request #5900 from blackpiglet/5416_limit_concurrent_goroutine
Limit the concurrent number for backup's VolumeSnapshot operation
2023-03-21 18:07:27 +08:00
Daniel Jiang
458560795b Remove Reference of itemsnapshotter (#5997)
It will be removed as we choose to use BIAv2 for progress monitoring.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2023-03-21 09:05:29 +08:00
Xun Jiang
c677c433e0 Limit the concurrent number of go routine for VolumeSnapshot and VolumeSnapshotContent operations.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-03-16 11:42:38 +08:00
Xun Jiang
a1e4f54488 Add new resource filters can separate cluster and namespace scope resources.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-03-14 07:07:22 +00:00
Scott Seago
dd63e8182c Follow-on fixes for BIAv2 controller work
Signed-off-by: Scott Seago <sseago@redhat.com>
2023-03-13 10:29:36 -04:00
Xun Jiang/Bruce Jiang
7d7e3fff0d Refoctor backup controller with controller-runtime. (#5969)
Signed-off-by: Ming <mqiu@vmware.com>
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
Co-authored-by: Ming <mqiu@vmware.com>
2023-03-10 08:59:40 +08:00
Scott Seago
c3d1d83da5 BIAv2 async operations controller work
Signed-off-by: Scott Seago <sseago@redhat.com>
2023-03-01 11:49:17 -05:00
Zhizhen He
fb1dc110f2 Replace k8s.io/apimachinery/pkg/util/clock with k8s.io/utils/clock
Signed-off-by: Zhizhen He <hezhizhen.yi@gmail.com>
2023-02-21 11:56:17 +08:00