Commit Graph
78 Commits
Author SHA1 Message Date
872f903091 Add configurable tolerations for PodVolumeBackup and data mover pods (#9575)
Run the E2E test on kind / setup-test-matrix (push) Failing after 3s
Scorecard supply-chain security / Scorecard analysis (push) Skipped
e2e-test-kind.yaml / extract (push) Failing after 6s
Run the E2E test on kind / get-go-version (push) Failing after 7s
Run the E2E test on kind / build (push) Skipped
Run the E2E test on kind / run-e2e-test (push) Skipped
push.yml / extract (push) Failing after 6s
Main CI / get-go-version (push) Failing after 7s
Main CI / Build (push) Skipped
* Remove toleration whitelist for PodVolumeBackup and data mover pods

Instead of filtering tolerations through a hardcoded allowlist
(ThirdPartyTolerations), inherit all tolerations from the node-agent
daemonset for PodVolumeBackup/Restore and DataUpload/Download pods,
and from the Velero deployment for maintenance jobs.

This enables backups and restores on nodes with custom NoExecute taints,
which was previously impossible since only two specific toleration keys
were whitelisted.

Fixes #9476

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>

* Fix codespell: replace 'whitelist' with 'allowlist' in changelog

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>

* Implement deduplication of tolerations and add unit tests for the new function

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

* Merge node-agent-configmap tolerations with third-party allowlist

Add a `tolerations` field to the node-agent-configmap so operators can
declare hosting-pod tolerations explicitly, per blackpiglet's review
feedback that tolerations shouldn't be read from the DaemonSet alone.
These are merged with (and deduplicated against) DaemonSet tolerations
matching the existing third-party allowlist
(kubernetes.azure.com/scalesetpriority, CriticalAddonsOnly), restoring
that allowlist per the follow-up suggestion to keep inheriting it
alongside the new config option.

The toleration dedup helper is moved from pkg/exposer to
pkg/util/kube (exported as DeduplicateTolerations) so it can be
shared with pkg/nodeagent without an import cycle.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

* Fix testifylint finding in TestGetTolerations

golangci-lint v2.12.0 (pinned in pr-linter-check.yml) flagged the
shared assert.Equal after the if/else as require-error: use require
for the error assertion so each branch is self-contained, matching
the pattern used elsewhere in this file.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

* Document toleration merge priority in GetTolerations

Per blackpiglet's review feedback: clarify that configured tolerations
take priority over allowlisted daemonset tolerations because they're
appended first and DeduplicateTolerations keeps only the first
occurrence of each exact (Key, Operator, Value, Effect) combination.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

---------

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Happy <yesreply@happy.engineering>
2026-09-14 18:05:01 -04:00
Lyndon-Li 5146992b5b add UT for progress message
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-09-04 16:04:38 +08:00
lyndon-liandGitHub da5bee7097 Use thread safe map for cancel recorder (#10255)
* use thread safe map for cancel recorder

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* use atomic load and store

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

---------

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-17 16:55:01 +08:00
Wenkai Yin(尹文开)andGitHub d69f6abe5c Add param to StartRestore to facilitates future expansion (#10057)
Add param to StartRestore to facilitates future expansion

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2026-07-23 10:31:42 +08:00
Xun Jiang 0bc06323bf Support change-id and volume-id in backup workflow.
* Add change-id and volume-id retrieve logic for both vks and vanilla k8s environment.
* Add change-id and volume-id support code in exposer.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2026-07-01 15:19:52 +08:00
Xun Jiang 49b670a791 Replace github.com/pkg/errors by github.com/cockroachdb/errors
Change errors.Cause to errors.Is, because github.com/cockroachdb/errors
New() function create a error with error stack with depth 1, but
github.com/pkg/errors's New() function create error with no depth.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2026-06-10 15:55:57 +08:00
Lyndon-Li 74ffe25cbe add cbt service parameters to node-agent-config
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-01 14:43:43 +08:00
Xun Jiang e39374f335 Add maintenance job and data mover pod's labels and annotations setting.
Add wait in file_system_test's async test cases.
Add related documents.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2025-12-17 13:21:07 +08:00
Lyndon-Li 6dbe772590 snapshot size in restore CRs only
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-10-22 12:41:41 +08:00
Lyndon-Li 166f50d776 add snapshot size to data mover CRs
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-10-21 15:14:38 +08:00
Xun Jiang c62a486765 Add ConfigMap parameters validation for install CLI and server start.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2025-08-22 20:31:38 +08:00
Tiger KaovilaiandClaude 35d2cc0890 Add priority class support for Velero server and node-agent
- Add --server-priority-class-name and --node-agent-priority-class-name flags to velero install command
- Configure data mover pods (PVB/PVR/DataUpload/DataDownload) to use priority class from node-agent-configmap
- Configure maintenance jobs to use priority class from repo-maintenance-job-configmap (global config only)
- Add priority class validation with ValidatePriorityClass and GetDataMoverPriorityClassName utilities
- Update e2e tests to include PriorityClass testing utilities
- Move priority class design document to Implemented folder
- Add comprehensive unit tests for all priority class implementations
- Update documentation for priority class configuration
- Add changelog entry for #8883

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

remove unused test utils

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

feat: add unit test for getting priority class name in maintenance jobs

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

doc update

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

feat: add priority class validation for repository maintenance jobs

- Add ValidatePriorityClassWithClient function to validate priority class existence
- Integrate validation in maintenance.go when creating maintenance jobs
- Update tests to cover the new validation functionality
- Return boolean from ValidatePriorityClass to allow fallback behavior

This ensures maintenance jobs don't fail due to non-existent priority classes,
following the same pattern used for data mover pods.

Addresses feedback from:
https://github.com/vmware-tanzu/velero/pull/8883#discussion_r2238681442

Refs #8869

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

refactor: clean up priority class handling for data mover pods

- Fix comment in node_agent.go to clarify PriorityClassName is only for data mover pods
- Simplify server.go to use dataPathConfigs.PriorityClassName directly
- Remove redundant priority class logging from controllers as it's already logged during server startup
- Keep logging centralized in the node-agent server initialization

This reduces code duplication and clarifies the scope of priority class configuration.

🤖 Generated with [Claude Code](https://claude.ai/code)

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

refactor: remove GetDataMoverPriorityClassName from kube utilities

Remove GetDataMoverPriorityClassName function and its tests as priority
class is now read directly from dataPathConfigs instead of parsing from
ConfigMap. This simplifies the codebase by eliminating the need for
indirect ConfigMap parsing.

Refs #8869

🤖 Generated with [Claude Code](https://claude.ai/code)

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

refactor: remove priority class validation from install command

Remove priority class validation during install as it's redundant
since validation already occurs during server startup. Users cannot
see console logs during install, making the validation warnings
ineffective at this stage.

The validation remains in place during server and node-agent startup
where it's more appropriate and visible to users.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-06 01:36:22 -04:00
Lyndon-Li 34f8b73507 bump up kopia to v0.21.1
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-07-22 15:56:04 +08:00
Shubham Pampattiwar a73a150d98 Accommodate VGS workflows in PVC CSI plugin
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

Add changelog file

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

make update

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

lint fix

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

add unit tests for getVSForPVC func

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

Use v1beta1 instead of v1 v1alpha1

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

go mod tidy

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

update updateVGSCreatedVS func to use retry on conflict

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

make update minor fix

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

fix ut assert

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

Address PR feedback

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

minor updates

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

remove unsused func and add todo for dep upgrades

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2025-07-21 11:36:40 -07:00
lyndon-liandGitHub 06d305ea47 Issue 8344: constrain data path expose (#9064)
Run the E2E test on kind / build (push) Failing after 7m38s
Run the E2E test on kind / setup-test-matrix (push) Successful in 4s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / Build (push) Failing after 39s
Close stale issues and PRs / stale (push) Successful in 22s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 1m32s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-aws, main) (push) Failing after 1m41s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-gcp, main) (push) Failing after 1m30s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-microsoft-azure, main) (push) Failing after 1m18s
* issue 8344: constrain data path exposure.

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-07-18 13:32:45 +08:00
Matthieu MOREL 07ea14962c fix require-error rule from testifylint
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2025-06-23 15:39:54 +00:00
Lyndon-Li 99c699fcb1 vgdp ms pvb controller
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-06-11 09:22:43 +00:00
Lyndon-Li 38c927711a dm controller refactor for cancel
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-05-21 11:29:35 +08:00
lyndon-liandGitHub 4d10047d67 issue-8878: relief node os deduction error checks (#8891)
Run the E2E test on kind / build (push) Failing after 6m28s
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / Build (push) Failing after 36s
Close stale issues and PRs / stale (push) Successful in 9s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 1m5s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-aws, main) (push) Failing after 49s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-gcp, main) (push) Failing after 44s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-microsoft-azure, main) (push) Failing after 1m4s
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-05-06 12:38:05 -04:00
Matthieu MORELandGitHub c6a420bd3a chore: define common aliases for k8s packages (#8672)
Run the E2E test on kind / build (push) Failing after 6m48s
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / Build (push) Failing after 35s
Close stale issues and PRs / stale (push) Successful in 8s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 1m11s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-aws, main) (push) Failing after 47s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-gcp, main) (push) Failing after 49s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-microsoft-azure, main) (push) Failing after 43s
* lchore: define common alias for k8s packages

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

* Update .golangci.yaml

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

* Update .golangci.yaml

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

* Update .golangci.yaml

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

---------

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2025-04-22 06:14:47 -04:00
Xun Jiang/Bruce JiangandGitHub 10a5b7b702 Merge pull request #8624 from mmorel-35/revive/use-any
chore: enable use-any from revive
2025-02-07 15:09:05 +08:00
Matthieu MORELandGitHub 1e54f1cb15 chore: enable var-declaration from revive (#8636)
Run the E2E test on kind / build (push) Failing after 5m28s
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / Build (push) Failing after 33s
Close stale issues and PRs / stale (push) Successful in 8s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 52s
Trivy Nightly Scan / Trivy nightly scan (velero-restore-helper, main) (push) Failing after 56s
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2025-01-22 15:56:44 -05:00
Matthieu MOREL cbba3bdde7 chore: enable use-any from revive
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2025-01-17 07:58:10 +01:00
Lyndon-Li a8469126d8 data mover restore for Windows
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-01-10 08:58:32 +00:00
Lyndon-Li cb22dfc482 fs uploader and block uploader support Windows nodes
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-01-02 13:25:23 +08:00
Wenkai Yin(尹文开)andGitHub 0224d99889 Merge pull request #8482 from Lyndon-Li/data-mover-exposer-diagnostic
Run the E2E test on kind / build (push) Failing after 6m17s
Run the E2E test on kind / setup-test-matrix (push) Successful in 5s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / Build (push) Failing after 1m3s
Close stale issues and PRs / stale (push) Has started running
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Has started running
Trivy Nightly Scan / Trivy nightly scan (velero-restore-helper, main) (push) Has started running
Data mover exposer diagnostic
2024-12-13 14:28:37 +08:00
Lyndon-Li 86082eb137 move the accepted info from annotations to DU/DD CR
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-09 16:39:04 +08:00
Lyndon-Li cbdbbe26c2 fix prepare timeout issue
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-05 17:24:12 +08:00
Lyndon-Li e4e9b18b37 add diagnostic for data mover exposer
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-04 10:28:50 +08:00
Xun Jiang 26cc41f26d Implement the Repo maintanence Job configuration design.
Remove the resource parameters from the velero server CLI.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-09-09 22:42:56 +08:00
Lyndon-Li 627e2fede6 nod-agent config for pod resources
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-22 15:06:35 +08:00
Lyndon-Li 0ed1a7fc86 data mover ms smoke testing
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-15 15:06:31 +08:00
Lyndon-Li d25a908b78 Merge branch 'main' into data-mover-ms-smoking-test 2024-08-15 11:12:44 +08:00
Lyndon-Li ed0ef67c16 data mover ms smoke testing
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-14 17:29:04 +08:00
Shubham Pampattiwar 8eac3606d9 Add support for backup PVC configuration
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

add changelog file

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

make update

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

pass backupPVCConfig to exposer as part of csi params

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2024-08-13 17:06:06 -07:00
Lyndon-Li 3c0948c9be Merge branch 'main' into data-mover-ms-smoking-test 2024-08-14 00:15:56 +08:00
Lyndon-Li 04db3ba767 data mover ms node agent resume
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-12 10:45:59 +08:00
Lyndon-Li 4dea3a48e8 data mover ms smoking test
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-09 15:15:21 +08:00
lyndon-liandGitHub 60f5ad5cf4 Merge branch 'main' into data-mover-ms-node-agent-resume
Signed-off-by: lyndon-li <98304688+Lyndon-Li@users.noreply.github.com>
2024-08-07 17:13:35 +08:00
Lyndon-Li a523d10802 data mover ms node agent resume
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-06 16:25:56 +08:00
Lyndon-Li d48e9762eb data mover ms new controller
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-07-31 13:24:16 +08:00
lyndon-liandGitHub 01aa657f0e Merge pull request #7988 from Lyndon-Li/data-mover-ms-new-exposer
New exposer for data mover ms
2024-07-24 10:11:54 +08:00
Matthieu MORELandGitHub c8baaa9b11 testifylint: enable more rules (#8024)
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-07-18 10:43:16 -04:00
Matthieu MORELandGitHub aa3fde5ea5 testifylint: enable bool-compare rule (#7623)
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-07-16 09:28:23 -04:00
Lyndon-Li 7f88d631a9 Merge branch 'main' into data-mover-ms-new-exposer 2024-07-16 15:54:29 +08:00
Lyndon-Li 49097744ee new exposer for data mover ms
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-07-15 17:18:03 +08:00
Lyndon-Li 20676c1ae7 new data path for data mover ms
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-07-01 19:07:00 +08:00
Daniel Jiang bed10c7fe6 Fix the problems found by codespell
Signed-off-by: Daniel Jiang <daniel.jiang@broadcom.com>
2024-05-24 13:32:54 +08:00
Ming Qiu a2c1a5a113 Fix cancel bug && adjust StartTimestamp for data mover
Signed-off-by: Ming Qiu <ming.qiu@broadcom.com>
2024-04-03 02:21:26 +00:00
lyndon-liandGitHub 67bd694d1b Merge pull request #7437 from Lyndon-Li/issue-fix-7036
Issue 7036: node selection for data mover backup
2024-03-29 17:04:40 +08:00