71 Commits

Author SHA1 Message Date
Xun Jiang
e39374f335 Add maintenance job and data mover pod's labels and annotations setting.
Add wait in file_system_test's async test cases.
Add related documents.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2025-12-17 13:21:07 +08:00
Lyndon-Li
7dbe2b4358 cache volume for data download
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-11-04 16:59:52 +08:00
Xun Jiang
c62a486765 Add ConfigMap parameters validation for install CLI and server start.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2025-08-22 20:31:38 +08:00
Tiger Kaovilai
35d2cc0890 Add priority class support for Velero server and node-agent
- Add --server-priority-class-name and --node-agent-priority-class-name flags to velero install command
- Configure data mover pods (PVB/PVR/DataUpload/DataDownload) to use priority class from node-agent-configmap
- Configure maintenance jobs to use priority class from repo-maintenance-job-configmap (global config only)
- Add priority class validation with ValidatePriorityClass and GetDataMoverPriorityClassName utilities
- Update e2e tests to include PriorityClass testing utilities
- Move priority class design document to Implemented folder
- Add comprehensive unit tests for all priority class implementations
- Update documentation for priority class configuration
- Add changelog entry for #8883

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

remove unused test utils

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

feat: add unit test for getting priority class name in maintenance jobs

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

doc update

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

feat: add priority class validation for repository maintenance jobs

- Add ValidatePriorityClassWithClient function to validate priority class existence
- Integrate validation in maintenance.go when creating maintenance jobs
- Update tests to cover the new validation functionality
- Return boolean from ValidatePriorityClass to allow fallback behavior

This ensures maintenance jobs don't fail due to non-existent priority classes,
following the same pattern used for data mover pods.

Addresses feedback from:
https://github.com/vmware-tanzu/velero/pull/8883#discussion_r2238681442

Refs #8869

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

refactor: clean up priority class handling for data mover pods

- Fix comment in node_agent.go to clarify PriorityClassName is only for data mover pods
- Simplify server.go to use dataPathConfigs.PriorityClassName directly
- Remove redundant priority class logging from controllers as it's already logged during server startup
- Keep logging centralized in the node-agent server initialization

This reduces code duplication and clarifies the scope of priority class configuration.

🤖 Generated with [Claude Code](https://claude.ai/code)

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

refactor: remove GetDataMoverPriorityClassName from kube utilities

Remove GetDataMoverPriorityClassName function and its tests as priority
class is now read directly from dataPathConfigs instead of parsing from
ConfigMap. This simplifies the codebase by eliminating the need for
indirect ConfigMap parsing.

Refs #8869

🤖 Generated with [Claude Code](https://claude.ai/code)

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

refactor: remove priority class validation from install command

Remove priority class validation during install as it's redundant
since validation already occurs during server startup. Users cannot
see console logs during install, making the validation warnings
ineffective at this stage.

The validation remains in place during server and node-agent startup
where it's more appropriate and visible to users.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-06 01:36:22 -04:00
Xun Jiang/Bruce Jiang
9cb421c26f Fix the dd and du's node affinity issue. (#9130)
Some checks failed
Run the E2E test on kind / build (push) Failing after 12m11s
Run the E2E test on kind / setup-test-matrix (push) Successful in 4s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / Build (push) Failing after 27s
Close stale issues and PRs / stale (push) Successful in 12s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 1m22s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-aws, main) (push) Failing after 1m3s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-gcp, main) (push) Failing after 1m0s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-microsoft-azure, main) (push) Failing after 1m6s
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2025-08-04 16:21:35 -04:00
lyndon-li
06d305ea47 Issue 8344: constrain data path expose (#9064)
Some checks failed
Run the E2E test on kind / build (push) Failing after 7m38s
Run the E2E test on kind / setup-test-matrix (push) Successful in 4s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / Build (push) Failing after 39s
Close stale issues and PRs / stale (push) Successful in 22s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 1m32s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-aws, main) (push) Failing after 1m41s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-gcp, main) (push) Failing after 1m30s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-microsoft-azure, main) (push) Failing after 1m18s
* issue 8344: constrain data path exposure.

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-07-18 13:32:45 +08:00
Lyndon-Li
a752b54614 issue 8857: support third party tolerations
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-07-08 14:23:00 +08:00
Xun Jiang
f2133c7d22 The backup and restore data mover node selection.
Update Makefile to let the `make serve-docs` work again.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2025-07-01 16:26:12 +08:00
Lyndon-Li
0450290095 resume PVB/PVR on node-agent restart
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-06-25 14:24:18 +08:00
Lyndon-Li
cded6bd207 cancel pvb/pvr on velero server restarts
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-06-23 15:29:09 +08:00
Lyndon-Li
b222b88c94 dm controller refactor for cancel
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-05-22 14:56:15 +08:00
Lyndon-Li
38c927711a dm controller refactor for cancel
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-05-21 11:29:35 +08:00
Matthieu MOREL
c6a420bd3a chore: define common aliases for k8s packages (#8672)
Some checks failed
Run the E2E test on kind / build (push) Failing after 6m48s
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / Build (push) Failing after 35s
Close stale issues and PRs / stale (push) Successful in 8s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 1m11s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-aws, main) (push) Failing after 47s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-gcp, main) (push) Failing after 49s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-microsoft-azure, main) (push) Failing after 43s
* lchore: define common alias for k8s packages

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

* Update .golangci.yaml

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

* Update .golangci.yaml

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

* Update .golangci.yaml

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

---------

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2025-04-22 06:14:47 -04:00
Lyndon-Li
b170892e64 issue 8754: add third party annotation support
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-03-10 10:38:26 +08:00
Lyndon-Li
a8469126d8 data mover restore for Windows
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-01-10 08:58:32 +00:00
Lyndon-Li
cfad06b701 Merge branch 'main' into restore-pvc-ignore-wait-for-first-consumer 2025-01-03 14:14:37 +08:00
Lyndon-Li
f5d13aeb17 data mover backup for Windows nodes
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-26 02:46:08 +00:00
Lyndon-Li
a56b06bab1 issue 8044: generic restore - allow to ignore WaitForFirstConsumer
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-26 10:29:15 +08:00
Wenkai Yin(尹文开)
0224d99889 Merge pull request #8482 from Lyndon-Li/data-mover-exposer-diagnostic
Some checks failed
Run the E2E test on kind / build (push) Failing after 6m17s
Run the E2E test on kind / setup-test-matrix (push) Successful in 5s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / Build (push) Failing after 1m3s
Close stale issues and PRs / stale (push) Has started running
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Has started running
Trivy Nightly Scan / Trivy nightly scan (velero-restore-helper, main) (push) Has started running
Data mover exposer diagnostic
2024-12-13 14:28:37 +08:00
Wenkai Yin(尹文开)
26661c775f Merge pull request #8498 from Lyndon-Li/move-accept-info-to-du-dd-cr
Move the accepted info from annotations to DU/DD CR
2024-12-11 13:22:39 +08:00
Lyndon-Li
34e417bdac add diagnostic for data mover exposer
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-10 14:00:31 +08:00
lyndon-li
a1cf952b8d Issue 8433: add third party labels to data mover pods when the same labels exist in node-agent pods (#8487)
Some checks failed
Run the E2E test on kind / build (push) Failing after 6m42s
Run the E2E test on kind / setup-test-matrix (push) Successful in 8s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / Build (push) Failing after 1m26s
Close stale issues and PRs / stale (push) Successful in 14s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 1m7s
Trivy Nightly Scan / Trivy nightly scan (velero-restore-helper, main) (push) Failing after 1m19s
* issue 8433: add ask label to data mover pods

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* check existence of the same label from node-agent

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

---------

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-09 12:44:39 -05:00
Lyndon-Li
86082eb137 move the accepted info from annotations to DU/DD CR
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-09 16:39:04 +08:00
Lyndon-Li
cbdbbe26c2 fix prepare timeout issue
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-05 17:24:12 +08:00
Lyndon-Li
e4e9b18b37 add diagnostic for data mover exposer
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-04 10:28:50 +08:00
Wenkai Yin(尹文开)
0a4e417aab Bump up version of client-go and controller-runtime
Bump up version of client-go to v0.30.5
Bump up version of controller-runtime to v0.18.5

Fixes #8274

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2024-10-08 18:53:12 +08:00
Tiger Kaovilai
c8aa37d852 Remove additional param, use pkg/constant
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2024-09-11 17:13:37 -04:00
Tiger Kaovilai
5c4c66bee9 Add controller name to periodical_enqueue_source
The code changes are related to the `NewPeriodicalEnqueueSource` function in the `kube/periodical_enqueue_source.go` file. This function is used to create a new instance of the `PeriodicalEnqueueSource` struct, which is responsible for periodically enqueueing objects into a work queue.

The changes involve adding two new parameters to this function: `controllerName string` and modifying the existing `logger` parameter to include additional fields.

Here's what changed:

1. A new `controllerName` parameter was added to the `NewPeriodicalEnqueueSource` function.

These changes are to adding more context or metadata to the logging output, possibly for debugging or monitoring purposes.

The other files (`restore_operations_controller.go`, `schedule_controller.go`, and their respective test files) were modified to use this updated `NewPeriodicalEnqueueSource` function with the new `controllerName` parameter.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2024-09-09 12:07:07 -04:00
Lyndon-Li
627e2fede6 nod-agent config for pod resources
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-22 15:06:35 +08:00
Lyndon-Li
0ed1a7fc86 data mover ms smoke testing
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-15 15:06:31 +08:00
Lyndon-Li
ed0ef67c16 data mover ms smoke testing
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-14 17:29:04 +08:00
Lyndon-Li
3c0948c9be Merge branch 'main' into data-mover-ms-smoking-test 2024-08-14 00:15:56 +08:00
Lyndon-Li
04db3ba767 data mover ms node agent resume
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-12 10:45:59 +08:00
Lyndon-Li
4dea3a48e8 data mover ms smoking test
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-09 15:15:21 +08:00
lyndon-li
60f5ad5cf4 Merge branch 'main' into data-mover-ms-node-agent-resume
Signed-off-by: lyndon-li <98304688+Lyndon-Li@users.noreply.github.com>
2024-08-07 17:13:35 +08:00
Lyndon-Li
a523d10802 data mover ms node agent resume
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-06 16:25:56 +08:00
Lyndon-Li
903458b61b data mover ms new controller
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-01 15:11:13 +08:00
Lyndon-Li
514ba56ca1 data mover ms new controller
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-01 14:42:17 +08:00
Lyndon-Li
d48e9762eb data mover ms new controller
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-07-31 13:24:16 +08:00
Lyndon-Li
20676c1ae7 new data path for data mover ms
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-07-01 19:07:00 +08:00
Ming Qiu
a2c1a5a113 Fix cancel bug && adjust StartTimestamp for data mover
Signed-off-by: Ming Qiu <ming.qiu@broadcom.com>
2024-04-03 02:21:26 +00:00
lyndon-li
67bd694d1b Merge pull request #7437 from Lyndon-Li/issue-fix-7036
Issue 7036: node selection for data mover backup
2024-03-29 17:04:40 +08:00
Wenkai Yin(尹文开)
8752c3a820 Bump up the versions of severel Kubernetes-related libs
Bump up the versions of severel Kubernetes-related libs

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2024-03-05 13:09:38 +08:00
Lyndon-Li
e1bcdf0f63 issue 7308: change the data path requeue time to 5 second
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-02-22 10:10:33 +08:00
Lyndon-Li
7bf7fb9fc1 issue 7036: fail early by peek expose
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-02-18 14:34:35 +08:00
Shubham Pampattiwar
ea6c8ca127 fix finalizer typo in logs (#7204)
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2023-12-13 11:46:21 -05:00
lyndon-li
099acd2527 Merge pull request #7141 from qiuming-best/support-restore-sparse
Allow sparse option for Kopia & Restic restore
2023-12-06 18:25:34 +08:00
Ming Qiu
1a237d3e4c Update API
Signed-off-by: Ming Qiu <mqiu@vmware.com>
2023-12-06 08:59:12 +00:00
qiuming
2fa785a3dd Merge pull request #7052 from qiuming-best/data-mover-fail-early
Make data mover fail early
2023-12-05 16:33:46 +08:00
Ming Qiu
0afaa70e9b Merge branch 'main' of https://github.com/qiuming-best/velero into support-restore-sparse 2023-11-30 10:55:55 +00:00