Commit Graph
42 Commits
Author SHA1 Message Date
Wenkai Yin(尹文开)andGitHub ccfdce30f9 Fall back to full restore rather than fail if fail to get the volume ID (#10465)
Run the E2E test on kind / setup-test-matrix (push) Failing after 4s
e2e-test-kind.yaml / extract (push) Failing after 7s
Run the E2E test on kind / get-go-version (push) Failing after 8s
Run the E2E test on kind / build (push) Skipped
Run the E2E test on kind / run-e2e-test (push) Skipped
push.yml / extract (push) Failing after 7s
Main CI / get-go-version (push) Failing after 8s
Main CI / Build (push) Skipped
Fall back to full restore rather than fail if fail to get the volum
e ID

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2026-09-03 14:56:35 +08:00
Lyndon-Li 83e5f9b85a UT for pv topology
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-27 18:22:47 +08:00
Lyndon-Li f6aab3b144 UT for pv topology
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-27 17:03:20 +08:00
b7d83a6f2b Cherry pick the in-place restore implementation PRs from feature branch to main (#10415)
* Update CRDs and CLI to support in-place restore (#10038)

Update CRDs(Restore, DataDownload, PodVolumeRestore) and restore create CLI to support in-place restore

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>

* Update Kopia(filesystem) uploader to support incremental and deleteExtraFile during restore (#10066)

Update Kopia(filesystem) uploader to support incremental and deleteExtraFile during restore

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>

* Update Restore Exposer and PVC CSI to support in-place restore (#10104)

1. Update Restore Exposer to support exposing with existing PV for in-place restore
2. Update PVC CSI RIA to continue the restore process for in-place restore

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>

* Update Block uploader to support increase restore (#10244)

Update Block uploader to support increase restore

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>

* Update Exposer to recreate the target PV if the volume mode is different with the restore PVC (#10257)

Update Exposer to recreate the target PV if the volume mode is different with t
he restore PVC

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>

* Preserve PVC selected-node annotation via carrier annotation for in-place restore

For in-place volume data restore, the existing PVC is deleted and
recreated. For StorageClasses with the WaitForFirstConsumer volume
binding mode, losing the volume.kubernetes.io/selected-node annotation
could let the scheduler place the recreated workload Pod in a different
zone than the original PV, leaving it stuck in ContainerCreating.

Instead of relying on RestoreItemAction execution order (the generic
PVC RIA unconditionally strips the selected-node annotation), the PVC
CSI RIA now captures the annotation from the existing PVC right before
deleting it and carries it on the target PVC via the Velero-internal
restore.velero.io/inplace-restore-selected-node annotation. The restore
engine translates the carrier back to the Kubernetes annotation after
all RestoreItemActions have run and always strips the carrier so it
never lands on the cluster.

This makes the behavior independent of RIA ordering: the Kubernetes
annotation is stripped by default on every path (including when the
target PVC does not exist and Velero falls back to provisioning a new
PVC), and preservation only happens when the CSI RIA explicitly
captured a value from the existing PVC.

Signed-off-by: chlins <chlins.zhang@gmail.com>

* Update the control path to make the in-place incremental restore with block data mover work E2E (#10410)

Update the control path to make the in-place incremental restore with block data mover work E2E

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>

---------

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
Signed-off-by: chlins <chlins.zhang@gmail.com>
Co-authored-by: chlins <chlins.zhang@gmail.com>
2026-08-26 10:21:32 -04:00
Shubham Pampattiwar cf04db2705 Use owner UID as backup-pvc-secret label value
The copied secret/configmap label value was the owner (DataUpload/
DataDownload) name, which is derived from the Backup/Restore name and
can exceed the 63-char Kubernetes label-value limit or contain invalid
characters. That would make the copy label and the cleanup selector
diverge and orphan the copied resources.

Use string(ownerObject.UID) consistently for the label value in both
copy and cleanup (backup and restore exposers). The UID is a stable,
always-valid label value.

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2026-08-18 10:28:27 -07:00
Shubham Pampattiwar 9fd5365b15 Copy namespace-scoped secrets/configmaps for restore PVC provisioning
Mirror the backup-side fix on the restore path. The generic restore
exposer creates the intermediate restore PVC in the Velero namespace
using the target PVC's StorageClass. For encrypted volumes this fails
because ceph-csi looks up the KMS token secret in the PVC's namespace
(the Velero namespace), where it does not exist.

Add SecretNames/ConfigMapNames to the RestorePVC config. When set, the
generic restore exposer copies the named secrets/configmaps from the
target namespace to the Velero namespace before creating the restore
PVC, and cleans them up in CleanUp(). Reuses the same copy/delete
helpers and label as the backup path.

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2026-08-18 10:28:27 -07:00
harshit sainiandGitHub 11a071637b Use k8s.io/api well-known label constants instead of hardcoded strings (#10279)
* refactor: use k8s.io/api well-known label constants

Several well-known Kubernetes label strings were hardcoded across the
codebase instead of using the constants already exported by
k8s.io/api/core/v1, which is an existing dependency:

  "kubernetes.io/hostname"        -> corev1api.LabelHostname
  "kubernetes.io/os"              -> corev1api.LabelOSStable
  "topology.kubernetes.io/zone"   -> corev1api.LabelTopologyZone

The local kube.NodeOSLabel and zoneLabel consts, which duplicated the
upstream values verbatim, are now defined in terms of the upstream
constants rather than repeating the literal. Both are kept: NodeOSLabel
is exported and referenced from four packages alongside NodeOSLinux and
NodeOSWindows, which have no upstream equivalent, and zoneLabel sits
beside the deprecated-label fallback it is compared against.

No functional change - every replacement is a constant with an identical
value.

Signed-off-by: Harshit saini <harshitsaini1188@gmail.com>

* Add changelog for #10279

Signed-off-by: Harshit saini <harshitsaini1188@gmail.com>

* Cover the selected-node path in createRestorePod

TestCreateRestorePod only exercised selectedNode == "", so the branch
that pins the restore pod to a node was never executed. Add a case with
a selected node and assert the resulting pod carries the hostname label
in its node selector.

Signed-off-by: Harshit saini <harshitsaini1188@gmail.com>

* Also use constants for the arch and deprecated zone labels

Extends the same replacement to the two remaining well-known labels
raised on the issue:

  "kubernetes.io/arch"                      -> corev1api.LabelArchStable
  "failure-domain.beta.kubernetes.io/zone"  -> corev1api.LabelFailureDomainBetaZone

zoneLabelDeprecated in item_backupper.go was the last local const still
repeating a literal that upstream already exports, so the zone pair now
reads consistently against k8s.io/api. The deprecation note upstream
applies to the label itself, not the constant; Velero reads that label
deliberately as the fallback for PVs created before the topology labels
existed.

Signed-off-by: Harshit saini <harshitsaini1188@gmail.com>

---------

Signed-off-by: Harshit saini <harshitsaini1188@gmail.com>
2026-08-17 13:37:44 +08:00
chlins 666d14de32 feat(resourcepolicies): support dataMover parameter in snapshot volume policy action
Signed-off-by: chlins <chlins.zhang@gmail.com>
2026-07-15 15:26:23 +08:00
lyndon-liandGitHub cfd4914123 decide restorePVC volumeMode by data mover type (#9941)
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-25 11:06:53 +08:00
Lyndon-Li eb0aa625ce use restorePV to cover retained and non-retained case
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-22 17:33:17 +08:00
Lyndon-Li 77520d0522 recall the way to rebind volume with restorePV
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-22 17:21:36 +08:00
Lyndon-Li a8cf6646e1 wait PV detachment before deleting PV
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-22 16:22:33 +08:00
lyndon-liandGitHub 55eb5f2820 Clone pv on rebind (#9913)
* clone pv on rebind

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* update fsType to cloned PV

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* clone pv on rebind

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* clone pv on rebind

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

---------

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-16 17:47:21 +08:00
Xun Jiang/Bruce JiangandGitHub 02cb85ce90 Merge pull request #9890 from blackpiglet/jxun/main/bump_deps
[main] Bump some dependancies to fix CVEs and replace some inactive maintained packages.
2026-06-11 10:08:22 +08:00
Xun Jiang 49b670a791 Replace github.com/pkg/errors by github.com/cockroachdb/errors
Change errors.Cause to errors.Is, because github.com/cockroachdb/errors
New() function create a error with error stack with depth 1, but
github.com/pkg/errors's New() function create error with no depth.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2026-06-10 15:55:57 +08:00
Lyndon-Li 0a94fbbfc5 enhance RebindVolume method for generic restore exposer
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-09 18:08:59 +08:00
Lyndon-Li e703e06eeb diagnose expose on peek error
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-01-13 16:33:14 +08:00
Lyndon-Li b1e5e4408f exposer supports cache volume
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-10-28 14:43:51 +08:00
Lyndon-Li 8d29051bbe expose supports cache volume
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-10-22 19:11:56 +08:00
lyndon-liandGitHub e6aab8ca93 add events to diagnose (#9296)
Run the E2E test on kind / get-go-version (push) Failing after 1m10s
Run the E2E test on kind / build (push) Has been skipped
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / get-go-version (push) Failing after 16s
Main CI / Build (push) Has been skipped
Close stale issues and PRs / stale (push) Successful in 11s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 1m43s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-aws, main) (push) Failing after 1m5s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-gcp, main) (push) Failing after 1m2s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-microsoft-azure, main) (push) Failing after 1m2s
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-10-09 14:13:43 -04:00
Tiger KaovilaiandClaude 35d2cc0890 Add priority class support for Velero server and node-agent
- Add --server-priority-class-name and --node-agent-priority-class-name flags to velero install command
- Configure data mover pods (PVB/PVR/DataUpload/DataDownload) to use priority class from node-agent-configmap
- Configure maintenance jobs to use priority class from repo-maintenance-job-configmap (global config only)
- Add priority class validation with ValidatePriorityClass and GetDataMoverPriorityClassName utilities
- Update e2e tests to include PriorityClass testing utilities
- Move priority class design document to Implemented folder
- Add comprehensive unit tests for all priority class implementations
- Update documentation for priority class configuration
- Add changelog entry for #8883

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

remove unused test utils

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

feat: add unit test for getting priority class name in maintenance jobs

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

doc update

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

feat: add priority class validation for repository maintenance jobs

- Add ValidatePriorityClassWithClient function to validate priority class existence
- Integrate validation in maintenance.go when creating maintenance jobs
- Update tests to cover the new validation functionality
- Return boolean from ValidatePriorityClass to allow fallback behavior

This ensures maintenance jobs don't fail due to non-existent priority classes,
following the same pattern used for data mover pods.

Addresses feedback from:
https://github.com/vmware-tanzu/velero/pull/8883#discussion_r2238681442

Refs #8869

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

refactor: clean up priority class handling for data mover pods

- Fix comment in node_agent.go to clarify PriorityClassName is only for data mover pods
- Simplify server.go to use dataPathConfigs.PriorityClassName directly
- Remove redundant priority class logging from controllers as it's already logged during server startup
- Keep logging centralized in the node-agent server initialization

This reduces code duplication and clarifies the scope of priority class configuration.

🤖 Generated with [Claude Code](https://claude.ai/code)

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

refactor: remove GetDataMoverPriorityClassName from kube utilities

Remove GetDataMoverPriorityClassName function and its tests as priority
class is now read directly from dataPathConfigs instead of parsing from
ConfigMap. This simplifies the codebase by eliminating the need for
indirect ConfigMap parsing.

Refs #8869

🤖 Generated with [Claude Code](https://claude.ai/code)

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

refactor: remove priority class validation from install command

Remove priority class validation during install as it's redundant
since validation already occurs during server startup. Users cannot
see console logs during install, making the validation warnings
ineffective at this stage.

The validation remains in place during server and node-agent startup
where it's more appropriate and visible to users.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-06 01:36:22 -04:00
Lyndon-Li 34f8b73507 bump up kopia to v0.21.1
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-07-22 15:56:04 +08:00
Lyndon-Li a752b54614 issue 8857: support third party tolerations
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-07-08 14:23:00 +08:00
Xun Jiang f2133c7d22 The backup and restore data mover node selection.
Update Makefile to let the `make serve-docs` work again.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2025-07-01 16:26:12 +08:00
Matthieu MORELandGitHub c6a420bd3a chore: define common aliases for k8s packages (#8672)
Run the E2E test on kind / build (push) Failing after 6m48s
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / Build (push) Failing after 35s
Close stale issues and PRs / stale (push) Successful in 8s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 1m11s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-aws, main) (push) Failing after 47s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-gcp, main) (push) Failing after 49s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-microsoft-azure, main) (push) Failing after 43s
* lchore: define common alias for k8s packages

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

* Update .golangci.yaml

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

* Update .golangci.yaml

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

* Update .golangci.yaml

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

---------

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2025-04-22 06:14:47 -04:00
Lyndon-Li a8469126d8 data mover restore for Windows
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-01-10 08:58:32 +00:00
Lyndon-Li a56b06bab1 issue 8044: generic restore - allow to ignore WaitForFirstConsumer
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-26 10:29:15 +08:00
Lyndon-Li 4ad9c2485a hybrid deploy
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-18 10:50:23 +08:00
Lyndon-Li 34e417bdac add diagnostic for data mover exposer
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-10 14:00:31 +08:00
Lyndon-Li b607259563 add diagnostic for data mover exposer
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-12-04 14:49:58 +08:00
Lyndon-Li 627e2fede6 nod-agent config for pod resources
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-22 15:06:35 +08:00
lyndon-liandGitHub 53b57f8bdf Merge pull request #7999 from Lyndon-Li/data-mover-ms-watcher-01
Data mover micro-service watcher
2024-07-26 10:10:07 +08:00
Lyndon-Li faa704d909 data mover ms watcher
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-07-25 10:47:52 +08:00
Lyndon-Li 7f88d631a9 Merge branch 'main' into data-mover-ms-new-exposer 2024-07-16 15:54:29 +08:00
Lyndon-Li 49097744ee new exposer for data mover ms
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-07-15 17:18:03 +08:00
Scott Seago 9614ead033 Don't consider unschedulable pods unrecoverable
Signed-off-by: Scott Seago <sseago@redhat.com>
2024-06-17 10:05:52 -04:00
lyndon-liandGitHub 67bd694d1b Merge pull request #7437 from Lyndon-Li/issue-fix-7036
Issue 7036: node selection for data mover backup
2024-03-29 17:04:40 +08:00
Wenkai Yin(尹文开) 8752c3a820 Bump up the versions of severel Kubernetes-related libs
Bump up the versions of severel Kubernetes-related libs

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2024-03-05 13:09:38 +08:00
Lyndon-Li 7bf7fb9fc1 issue 7036: fail early by peek expose
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-02-18 14:34:35 +08:00
Lyndon-Li d6134ec444 fix issue 6561
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-08-01 21:10:01 +08:00
Lyndon-Li eebb879278 data mover restore abort for existing pvc
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-07-14 10:07:33 +08:00
Lyndon-Li 2ea6ffb63f data mover restore expose
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-06-06 16:14:58 +08:00