* Add block data mover support for Velero backup/restore describe CLI.
Update output tests to accommodate RestoreType in VolumeInfo
This commit addresses the compilation and assertion errors caused by the introduction of `RestoreType` in `VolumeInfo` and the separation of `SnapshotDataMovementInfo` / `PodVolumeInfo` into their backup and restore counterparts. It fixes references across the test fixtures and the print guard conditions in `restore_describer.go`.
* Modify according to comments
* Add missing JSON tag in the VolumeInfo structures.
* Get uploaderType from the DU and DD's dataMover for the data mover volume info.
* Add IncrementalSize in the data mover volume info.
* Add existingVolumeDataPolicy and restoreType in the restore describe CLI output
* Add more UTs.
* Add some fields value setting that were previously missed.
* Fix the timestamp compare error only found in the GitHub action.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* test: Unify duplicate test harness structs in restore and delete tests
Extract the duplicated test harness (fakeRestorer, Harness struct, and
AddResource/AddItems helpers) from restore_test.go and
delete_item_action_handler_test.go into a shared pkg/test.Harness.
Backup tests are intentionally excluded: the shared AddResource strips
metadata.creationTimestamp and status (restore semantics), which would
change what the backup tests tar up and assert on.
Signed-off-by: opbot_xd <awasthikrishna23052005@gmail.com>
* ci: retry
Signed-off-by: opbot_xd <awasthikrishna23052005@gmail.com>
* ci: retry
Signed-off-by: opbot_xd <awasthikrishna23052005@gmail.com>
---------
Signed-off-by: opbot_xd <awasthikrishna23052005@gmail.com>
* Update CRDs and CLI to support in-place restore (#10038)
Update CRDs(Restore, DataDownload, PodVolumeRestore) and restore create CLI to support in-place restore
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
* Update Kopia(filesystem) uploader to support incremental and deleteExtraFile during restore (#10066)
Update Kopia(filesystem) uploader to support incremental and deleteExtraFile during restore
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
* Update Restore Exposer and PVC CSI to support in-place restore (#10104)
1. Update Restore Exposer to support exposing with existing PV for in-place restore
2. Update PVC CSI RIA to continue the restore process for in-place restore
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
* Update Block uploader to support increase restore (#10244)
Update Block uploader to support increase restore
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
* Update Exposer to recreate the target PV if the volume mode is different with the restore PVC (#10257)
Update Exposer to recreate the target PV if the volume mode is different with t
he restore PVC
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
* Preserve PVC selected-node annotation via carrier annotation for in-place restore
For in-place volume data restore, the existing PVC is deleted and
recreated. For StorageClasses with the WaitForFirstConsumer volume
binding mode, losing the volume.kubernetes.io/selected-node annotation
could let the scheduler place the recreated workload Pod in a different
zone than the original PV, leaving it stuck in ContainerCreating.
Instead of relying on RestoreItemAction execution order (the generic
PVC RIA unconditionally strips the selected-node annotation), the PVC
CSI RIA now captures the annotation from the existing PVC right before
deleting it and carries it on the target PVC via the Velero-internal
restore.velero.io/inplace-restore-selected-node annotation. The restore
engine translates the carrier back to the Kubernetes annotation after
all RestoreItemActions have run and always strips the carrier so it
never lands on the cluster.
This makes the behavior independent of RIA ordering: the Kubernetes
annotation is stripped by default on every path (including when the
target PVC does not exist and Velero falls back to provisioning a new
PVC), and preservation only happens when the CSI RIA explicitly
captured a value from the existing PVC.
Signed-off-by: chlins <chlins.zhang@gmail.com>
* Update the control path to make the in-place incremental restore with block data mover work E2E (#10410)
Update the control path to make the in-place incremental restore with block data mover work E2E
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
---------
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
Signed-off-by: chlins <chlins.zhang@gmail.com>
Co-authored-by: chlins <chlins.zhang@gmail.com>
This commit ensures the resources in the set "resourceMustHave" can only
be created in the namespace of velero deployment if it's namespace
scoped.
Signed-off-by: Daniel Jiang <daniel.jiang@broadcom.com>
This commit makes sure the object is marked as "skipped" when there's
object with same name exists in the cluster during restore. Otherwise,
such object will appeared as "failed" in the "Resource list" in the
output of "velero restore describe xxx --details"
Signed-off-by: Daniel Jiang <daniel.jiang@broadcom.com>
archive.Unmarshal returns (nil, err) when an item file contains malformed
JSON, and restoreItem dereferences its obj argument on entry, so an
additional item that fails to unmarshal must be skipped rather than passed
on. The loop only records the error and continues today; nothing covers
that, so removing the continue reintroduces a nil pointer dereference in
the restore reconciler without failing any test.
The item file is added to the tarball so the existing Stat check passes and
the unmarshal is actually reached.
Signed-off-by: chlins <chlins.zhang@gmail.com>
Stop force-including VolumeSnapshotContents via resourceMustHave on
every restore; CSI VolumeSnapshot/PVC RestoreItemActions now set
`restore.velero.io/must-include-additional-items` so bound snapshot
dependencies are restored only when their parent is restored.
Fixes: #9957
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
Let RestoreItemActions opt in via annotation to
bypass global restore filters for AdditionalItems,
mirroring the backup-side must-include behavior.
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
* Fix PodVolumeBackup metadata loss on fs-backup timeout
When a backup hits the fs-backup timeout, WaitAllPodVolumesProcessed
returned nil because PVBs were only collected from the indexer in the
done branch of the select. This discarded all PVB metadata including
already-completed PVBs, making their data unrestorable.
Move the PVB collection loop to run after the select so tracked PVBs
are always persisted regardless of timeout.
Fixes#9986
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
* Add changelog for PR #9995
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
* Filter non-completed PVBs in hasPodVolumeBackup
After preserving tracked PVBs on timeout, non-completed PVBs (in-progress
or with no snapshot ID) would cause hasPodVolumeBackup to return true,
leading the restore to skip the original PV and dynamically re-provision
it without any data to restore from.
Only match PVBs that are Completed with a valid SnapshotID.
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
* Add unit tests for hasPodVolumeBackup phase filtering
Verify that hasPodVolumeBackup only matches PVBs that are Completed
with a valid SnapshotID, and rejects in-progress, failed, or
empty-snapshot PVBs.
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
---------
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
- normalized the input to lower case for consistency
- added validations for kind collision
- add flag for unresolved kinds, and defer skip decision base on that
- move peek-and-map test cases to restore_policies_test.go
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
When a user specifies a Custom Resource Kind in a restore filter
policy (e.g., kinds: [MyCustomKind]), the discovery helper fails
to resolve it if the CRD hasn't been restored yet.
This adds a peek-and-map fallback: if a resource type in the
backup tarball doesn't match the resolved filters, Velero peeks
at the actual Kind of the first item in the tarball and matches
it against the user's original policy strings.
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
Change errors.Cause to errors.Is, because github.com/cockroachdb/errors
New() function create a error with error stack with depth 1, but
github.com/pkg/errors's New() function create error with no depth.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
Update test expectations to include createdName field for resources
with action 'created'. Also ensure namespaces track their created
names when created via EnsureNamespaceExistsAndIsReady.
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
VolumeInfo contains several sub-structures. They are filled for
different scenarios. Do not generate empty structure for the
not filled sub-structures.
Signed-off-by: Xun Jiang <jxun@vmware.com>
Use informer cache with dynamic client for Get calls on restore
When enabled, also make the Get call before create.
Add server and install parameter to allow disabling this feature,
but enable by default
Signed-off-by: Scott Seago <sseago@redhat.com>
Krishna AwasthiandGitHub
Daniel JiangandGitHub
Adam Zhang
Shubham PampattiwarandGitHub
Xun Jiang
Lyndon-Li
Matthieu MOREL
sangitaray2021andGitHub
Anshul Ahuja
Xun Jiang
Daniel Jiang
clonefetchandGitHub
Xun Jiang
Scott Seago
