Commit Graph

158 Commits

Author SHA1 Message Date
lyndon-li
e6aab8ca93 add events to diagnose (#9296)
Some checks failed
Run the E2E test on kind / get-go-version (push) Failing after 1m10s
Run the E2E test on kind / build (push) Has been skipped
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / get-go-version (push) Failing after 16s
Main CI / Build (push) Has been skipped
Close stale issues and PRs / stale (push) Successful in 11s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 1m43s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-aws, main) (push) Failing after 1m5s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-gcp, main) (push) Failing after 1m2s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-microsoft-azure, main) (push) Failing after 1m2s
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-10-09 14:13:43 -04:00
Xun Jiang/Bruce Jiang
f8938e7fed VerifyJSONConfigs verify every elements in Data. (#9302)
Some checks failed
Run the E2E test on kind / get-go-version (push) Failing after 48s
Run the E2E test on kind / build (push) Has been skipped
Run the E2E test on kind / setup-test-matrix (push) Successful in 2s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / get-go-version (push) Failing after 4s
Main CI / Build (push) Has been skipped
Close stale issues and PRs / stale (push) Failing after 6s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 7s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-aws, main) (push) Failing after 3s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-gcp, main) (push) Failing after 3s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-microsoft-azure, main) (push) Failing after 4s
Add error message in the velero install CLI output if VerifyJSONConfigs fail.
Only allow one element in node-agent-configmap's Data.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2025-09-29 15:08:05 -04:00
lyndon-li
aad9dd9068 Merge branch 'main' into backup-pvc-to-different-node 2025-09-11 14:47:35 +08:00
Lyndon-Li
81c5b6692d backupPVC to different node
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-09-11 13:04:24 +08:00
Xun Jiang
e8208097ba Bump k8s library to v1.33.
Replace deprecated EventExpansion method with WithContext methods.
Modify UTs.
Align the E2E ginkgo CLI version with go.mod

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2025-09-10 17:58:38 +08:00
Xun Jiang
c62a486765 Add ConfigMap parameters validation for install CLI and server start.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2025-08-22 20:31:38 +08:00
Tiger Kaovilai
35d2cc0890 Add priority class support for Velero server and node-agent
- Add --server-priority-class-name and --node-agent-priority-class-name flags to velero install command
- Configure data mover pods (PVB/PVR/DataUpload/DataDownload) to use priority class from node-agent-configmap
- Configure maintenance jobs to use priority class from repo-maintenance-job-configmap (global config only)
- Add priority class validation with ValidatePriorityClass and GetDataMoverPriorityClassName utilities
- Update e2e tests to include PriorityClass testing utilities
- Move priority class design document to Implemented folder
- Add comprehensive unit tests for all priority class implementations
- Update documentation for priority class configuration
- Add changelog entry for #8883

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

remove unused test utils

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

feat: add unit test for getting priority class name in maintenance jobs

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

doc update

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

feat: add priority class validation for repository maintenance jobs

- Add ValidatePriorityClassWithClient function to validate priority class existence
- Integrate validation in maintenance.go when creating maintenance jobs
- Update tests to cover the new validation functionality
- Return boolean from ValidatePriorityClass to allow fallback behavior

This ensures maintenance jobs don't fail due to non-existent priority classes,
following the same pattern used for data mover pods.

Addresses feedback from:
https://github.com/vmware-tanzu/velero/pull/8883#discussion_r2238681442

Refs #8869

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

refactor: clean up priority class handling for data mover pods

- Fix comment in node_agent.go to clarify PriorityClassName is only for data mover pods
- Simplify server.go to use dataPathConfigs.PriorityClassName directly
- Remove redundant priority class logging from controllers as it's already logged during server startup
- Keep logging centralized in the node-agent server initialization

This reduces code duplication and clarifies the scope of priority class configuration.

🤖 Generated with [Claude Code](https://claude.ai/code)

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

refactor: remove GetDataMoverPriorityClassName from kube utilities

Remove GetDataMoverPriorityClassName function and its tests as priority
class is now read directly from dataPathConfigs instead of parsing from
ConfigMap. This simplifies the codebase by eliminating the need for
indirect ConfigMap parsing.

Refs #8869

🤖 Generated with [Claude Code](https://claude.ai/code)

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

refactor: remove priority class validation from install command

Remove priority class validation during install as it's redundant
since validation already occurs during server startup. Users cannot
see console logs during install, making the validation warnings
ineffective at this stage.

The validation remains in place during server and node-agent startup
where it's more appropriate and visible to users.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-06 01:36:22 -04:00
Xun Jiang/Bruce Jiang
9cb421c26f Fix the dd and du's node affinity issue. (#9130)
Some checks failed
Run the E2E test on kind / build (push) Failing after 12m11s
Run the E2E test on kind / setup-test-matrix (push) Successful in 4s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / Build (push) Failing after 27s
Close stale issues and PRs / stale (push) Successful in 12s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 1m22s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-aws, main) (push) Failing after 1m3s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-gcp, main) (push) Failing after 1m0s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-microsoft-azure, main) (push) Failing after 1m6s
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2025-08-04 16:21:35 -04:00
Xun Jiang
7b872473df Use const to represent snapshot.storage.kubernetes.io/managed-by.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2025-07-30 08:06:22 +08:00
Lyndon-Li
34f8b73507 bump up kopia to v0.21.1
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-07-22 15:56:04 +08:00
Xun Jiang
b5426516da Make the backup repository controller doesn't invalidate the BSL on restart.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2025-07-02 17:57:06 +08:00
Xun Jiang
f2133c7d22 The backup and restore data mover node selection.
Update Makefile to let the `make serve-docs` work again.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2025-07-01 16:26:12 +08:00
Matthieu MOREL
07ea14962c fix require-error rule from testifylint
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2025-06-23 15:39:54 +00:00
Lyndon-Li
5d5d4cd657 Merge branch 'main' into pod-volume-exposer 2025-06-06 14:40:26 +08:00
lyndon-li
bd23062c24 Merge branch 'main' into pod-volume-exposer 2025-06-03 14:05:11 +08:00
Lyndon-Li
5ccf22e0b0 Merge branch 'main' into vgdp-ms-pvb-data-path 2025-06-03 13:26:52 +08:00
Lyndon-Li
92c72b1a63 data path for vgdp ms pvb
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-06-03 13:25:48 +08:00
Lyndon-Li
b7d997130d issue 8960: implement PodVolume exposer for PVB/PVR
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-05-29 17:09:03 +08:00
Wenkai Yin(尹文开)
d0374fadb6 Mark BackupRepository not ready when BSL changed
Mark BackupRepository not ready when BSL changed

Fixes #8860

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2025-05-27 14:43:14 +08:00
lyndon-li
4d10047d67 issue-8878: relief node os deduction error checks (#8891)
Some checks failed
Run the E2E test on kind / build (push) Failing after 6m28s
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / Build (push) Failing after 36s
Close stale issues and PRs / stale (push) Successful in 9s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 1m5s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-aws, main) (push) Failing after 49s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-gcp, main) (push) Failing after 44s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-microsoft-azure, main) (push) Failing after 1m4s
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-05-06 12:38:05 -04:00
Matthieu MOREL
c6a420bd3a chore: define common aliases for k8s packages (#8672)
Some checks failed
Run the E2E test on kind / build (push) Failing after 6m48s
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / Build (push) Failing after 35s
Close stale issues and PRs / stale (push) Successful in 8s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 1m11s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-aws, main) (push) Failing after 47s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-gcp, main) (push) Failing after 49s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-microsoft-azure, main) (push) Failing after 43s
* lchore: define common alias for k8s packages

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

* Update .golangci.yaml

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

* Update .golangci.yaml

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

* Update .golangci.yaml

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

---------

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2025-04-22 06:14:47 -04:00
Roger Zimmermann
38a52980cc Issue #8772 ensure pv removed (#8777)
Some checks failed
Run the E2E test on kind / build (push) Failing after 6m16s
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / Build (push) Failing after 37s
Close stale issues and PRs / stale (push) Successful in 8s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 1m10s
Trivy Nightly Scan / Trivy nightly scan (velero-restore-helper, main) (push) Failing after 53s
* ensure pv has been deleted

Signed-off-by: Roger Zimmermann <roger.zimmermann@inventx.ch>

* ensure delete pv unit test

Signed-off-by: Roger Zimmermann <roger.zimmermann@inventx.ch>

* comment, errors

Signed-off-by: Roger Zimmermann <roger.zimmermann@inventx.ch>

* updated changelog
Signed-off-by: Roger Zimmermann <roger.zimmermann@inventx.ch>

Signed-off-by: Roger Zimmermann <roger.zimmermann@inventx.ch>

* pass value

Co-authored-by: Tiger Kaovilai <passawit.kaovilai@gmail.com>
Signed-off-by: Roger Zimmermann <roger.zimmermann@inventx.ch>

* function renamed as suggested

Signed-off-by: Roger Zimmermann <roger.zimmermann@inventx.ch>

---------

Signed-off-by: Roger Zimmermann <roger.zimmermann@inventx.ch>
Co-authored-by: Tiger Kaovilai <passawit.kaovilai@gmail.com>
2025-03-13 10:39:25 -04:00
Matthieu MOREL
6a6a237ba7 Bump golangci-lint from v1.57.2 to v1.64.5 (#8641)
Some checks failed
Run the E2E test on kind / build (push) Failing after 5m43s
Run the E2E test on kind / setup-test-matrix (push) Successful in 2s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
build-image / Build (push) Failing after 8s
Main CI / Build (push) Failing after 33s
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2025-03-04 13:55:29 +05:30
Lyndon-Li
bf0d909524 issue 8706: for immediate volumes, get node from volumeattachment
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-02-21 13:27:44 +08:00
Lyndon-Li
e2a7986629 issue 8706: for immediate volumes, get node from volumeattachment
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-02-20 19:19:28 +08:00
Tiger Kaovilai
a3cee616dc Upgrade go.mod k8s.io/ go.mod to v0.31.3 and set klog.SetLogger() for client-go (#8450)
Some checks failed
Run the E2E test on kind / build (push) Failing after 5m44s
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
build-image / Build (push) Failing after 10s
Main CI / Build (push) Failing after 31s
Close stale issues and PRs / stale (push) Successful in 7s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 59s
Trivy Nightly Scan / Trivy nightly scan (velero-restore-helper, main) (push) Failing after 45s
Also bumped to support upgraded k8s.io/ deps.
- controller-gen to v0.16.5
- sigs.k8s.io/controller-runtime v0.19.2

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2025-02-17 15:05:10 -05:00
Matthieu MOREL
1e54f1cb15 chore: enable var-declaration from revive (#8636)
Some checks failed
Run the E2E test on kind / build (push) Failing after 5m28s
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / Build (push) Failing after 33s
Close stale issues and PRs / stale (push) Successful in 8s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 52s
Trivy Nightly Scan / Trivy nightly scan (velero-restore-helper, main) (push) Failing after 56s
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2025-01-22 15:56:44 -05:00
Lyndon-Li
32ae4091ac add event burst
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-01-10 14:18:07 +08:00
Lyndon-Li
cfad06b701 Merge branch 'main' into restore-pvc-ignore-wait-for-first-consumer 2025-01-03 14:14:37 +08:00
Lyndon-Li
cb22dfc482 fs uploader and block uploader support Windows nodes
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-01-02 13:25:23 +08:00
Lyndon-Li
bc6414672e disable block volume data mover on windows
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-01-02 11:28:21 +08:00
Lyndon-Li
f5d13aeb17 data mover backup for Windows nodes
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-26 02:46:08 +00:00
Lyndon-Li
a56b06bab1 issue 8044: generic restore - allow to ignore WaitForFirstConsumer
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-26 10:29:15 +08:00
Lyndon-Li
4e0a0e0b72 fail fs-backup for windows nodes
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-24 14:26:02 +08:00
Lyndon-Li
a711b1067b fail fs-backup for windows nodes
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-18 10:46:00 +08:00
Wenkai Yin(尹文开)
0224d99889 Merge pull request #8482 from Lyndon-Li/data-mover-exposer-diagnostic
Some checks failed
Run the E2E test on kind / build (push) Failing after 6m17s
Run the E2E test on kind / setup-test-matrix (push) Successful in 5s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / Build (push) Failing after 1m3s
Close stale issues and PRs / stale (push) Has started running
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Has started running
Trivy Nightly Scan / Trivy nightly scan (velero-restore-helper, main) (push) Has started running
Data mover exposer diagnostic
2024-12-13 14:28:37 +08:00
Lyndon-Li
8b545532e2 issue 8267: add informative logs when expose error
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-12 11:19:26 +08:00
Lyndon-Li
b607259563 add diagnostic for data mover exposer
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-04 14:49:58 +08:00
Lyndon-Li
e4e9b18b37 add diagnostic for data mover exposer
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-04 10:28:50 +08:00
sangitaray2021
74790d9f60 Added tracking for deleted namespace status check in restore flow (#8233)
* Added tracking for deleted namespace status check in restore flow

Signed-off-by: sangitaray2021 <sangitaray@microsoft.com>

fixed unittest

Signed-off-by: sangitaray2021 <sangitaray@microsoft.com>

refactored tracker execution and caller

Signed-off-by: sangitaray2021 <sangitaray@microsoft.com>

added change log

Signed-off-by: sangitaray2021 <sangitaray@microsoft.com>

Author:    sangitaray2021 <sangitaray@microsft.com>

Author:    sangitaray2021 <sangitaray@microsoft.com>
Date:      Thu Sep 19 02:26:14 2024 +0530
Signed-off-by: sangitaray2021 <sangitaray@microsoft.com>

* fixed linter issuer

Signed-off-by: sangitaray2021 <sangitaray@microsoft.com>

* incorporated PR comments

Signed-off-by: sangitaray2021 <sangitaray@microsoft.com>

* resolved comments

Signed-off-by: sangitaray2021 <sangitaray@microsoft.com>

---------

Signed-off-by: sangitaray2021 <sangitaray@microsoft.com>
2024-11-18 13:41:07 +05:30
Wenkai Yin(尹文开)
0a4e417aab Bump up version of client-go and controller-runtime
Bump up version of client-go to v0.30.5
Bump up version of controller-runtime to v0.18.5

Fixes #8274

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2024-10-08 18:53:12 +08:00
Lyndon-Li
9deaa819aa issue 8232: ensure the ending event sinked before shutdown
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-09-23 18:35:56 +08:00
Wenkai Yin(尹文开)
670338e02f Merge pull request #8210 from kaovilai/pvc_pv_test.go-32-2
pvc_pv_test.go:32:2: other import of "k8s.io/api/core/v1"
2024-09-12 10:42:19 +08:00
Wenkai Yin(尹文开)
5b4c8cd5b1 Merge pull request #8198 from kaovilai/pes-controller
Add controller name to periodical_enqueue_source
2024-09-12 10:41:46 +08:00
Tiger Kaovilai
70168634cb pvc_pv_test.go:32:2: other import of "k8s.io/api/core/v1"
package "k8s.io/api/core/v1" is being imported more than once (ST1019)
	pvc_pv_test.go:32:2: other import of "k8s.io/api/core/v1"

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2024-09-11 17:21:04 -04:00
Tiger Kaovilai
c8aa37d852 Remove additional param, use pkg/constant
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2024-09-11 17:13:37 -04:00
Tiger Kaovilai
c643ee5fd4 Retry completion status patch for backup and restore resources
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

update to design #8063

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2024-09-10 17:01:14 -04:00
Tiger Kaovilai
5c4c66bee9 Add controller name to periodical_enqueue_source
The code changes are related to the `NewPeriodicalEnqueueSource` function in the `kube/periodical_enqueue_source.go` file. This function is used to create a new instance of the `PeriodicalEnqueueSource` struct, which is responsible for periodically enqueueing objects into a work queue.

The changes involve adding two new parameters to this function: `controllerName string` and modifying the existing `logger` parameter to include additional fields.

Here's what changed:

1. A new `controllerName` parameter was added to the `NewPeriodicalEnqueueSource` function.

These changes are to adding more context or metadata to the logging output, possibly for debugging or monitoring purposes.

The other files (`restore_operations_controller.go`, `schedule_controller.go`, and their respective test files) were modified to use this updated `NewPeriodicalEnqueueSource` function with the new `controllerName` parameter.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2024-09-09 12:07:07 -04:00
Xun Jiang
26cc41f26d Implement the Repo maintanence Job configuration design.
Remove the resource parameters from the velero server CLI.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-09-09 22:42:56 +08:00
lyndon-li
3408ffefac Merge pull request #8141 from shubham-pampattiwar/fix-backup-pvc-config
Apply backupPVCConfig to backupPod volume spec
2024-08-30 11:09:46 +08:00