Commit Graph
47 Commits
Author SHA1 Message Date
lyndon-liandGitHub 87b45ed7fd Merge pull request #10506 from Lyndon-Li/save-source-size-to-backup
Save source size to volume info
2026-09-10 14:29:48 +08:00
Lyndon-Li daed42b1d8 save source size to volume info for DU and PVB
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-09-09 17:34:17 +08:00
Lyndon-Li e52bf08c99 refactor CBT retrievement to report the concrete error
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-09-09 15:55:01 +08:00
Lyndon-Li 6b549f35b3 Merge branch 'main' into save-source-size-to-backup 2026-09-09 15:50:06 +08:00
Lyndon-Li 44f09189c2 Merge branch 'main' into report-incremental-fallback 2026-09-09 14:19:59 +08:00
Yonghui Li 9d85334d40 refactor CBT retrievement to report the concrete error
Signed-off-by: Yonghui Li <lyonghui@vmware.com>
2026-09-09 14:14:54 +08:00
Yonghui Li 8e604b17b2 add ID to repo snapshot
Signed-off-by: Yonghui Li <lyonghui@vmware.com>
2026-09-08 18:15:34 +08:00
Lyndon-Li d0884e7ce7 return source size from uploader
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-09-04 19:45:12 +08:00
Lyndon-Li 9687d1e30e Merge branch 'main' into report-incremental-fallback 2026-09-04 16:15:06 +08:00
Lyndon-Li 5146992b5b add UT for progress message
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-09-04 16:04:38 +08:00
Lyndon-Li 67923bca6f uploader report incremental fallback
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-09-04 14:17:28 +08:00
Lyndon-Li 6ec84c30f8 enhance CBT retrievement to indicate the result in error message
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-09-04 13:32:24 +08:00
Wenkai Yin(尹文开)andGitHub ccfdce30f9 Fall back to full restore rather than fail if fail to get the volume ID (#10465)
Run the E2E test on kind / setup-test-matrix (push) Failing after 4s
e2e-test-kind.yaml / extract (push) Failing after 7s
Run the E2E test on kind / get-go-version (push) Failing after 8s
Run the E2E test on kind / build (push) Skipped
Run the E2E test on kind / run-e2e-test (push) Skipped
push.yml / extract (push) Failing after 7s
Main CI / get-go-version (push) Failing after 8s
Main CI / Build (push) Skipped
Fall back to full restore rather than fail if fail to get the volum
e ID

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2026-09-03 14:56:35 +08:00
Wenkai Yin(尹文开)andGitHub 07768e7b33 Add "IncrementalBytes" field to status of DataDownload and PVR to indicate data transferred by the incremental restore (#10421)
Add "IncrementalBytes" field to status of DataDownload and PVR to indicate data transferred by the incremental restore

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2026-09-01 15:43:53 +08:00
b7d83a6f2b Cherry pick the in-place restore implementation PRs from feature branch to main (#10415)
* Update CRDs and CLI to support in-place restore (#10038)

Update CRDs(Restore, DataDownload, PodVolumeRestore) and restore create CLI to support in-place restore

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>

* Update Kopia(filesystem) uploader to support incremental and deleteExtraFile during restore (#10066)

Update Kopia(filesystem) uploader to support incremental and deleteExtraFile during restore

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>

* Update Restore Exposer and PVC CSI to support in-place restore (#10104)

1. Update Restore Exposer to support exposing with existing PV for in-place restore
2. Update PVC CSI RIA to continue the restore process for in-place restore

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>

* Update Block uploader to support increase restore (#10244)

Update Block uploader to support increase restore

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>

* Update Exposer to recreate the target PV if the volume mode is different with the restore PVC (#10257)

Update Exposer to recreate the target PV if the volume mode is different with t
he restore PVC

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>

* Preserve PVC selected-node annotation via carrier annotation for in-place restore

For in-place volume data restore, the existing PVC is deleted and
recreated. For StorageClasses with the WaitForFirstConsumer volume
binding mode, losing the volume.kubernetes.io/selected-node annotation
could let the scheduler place the recreated workload Pod in a different
zone than the original PV, leaving it stuck in ContainerCreating.

Instead of relying on RestoreItemAction execution order (the generic
PVC RIA unconditionally strips the selected-node annotation), the PVC
CSI RIA now captures the annotation from the existing PVC right before
deleting it and carries it on the target PVC via the Velero-internal
restore.velero.io/inplace-restore-selected-node annotation. The restore
engine translates the carrier back to the Kubernetes annotation after
all RestoreItemActions have run and always strips the carrier so it
never lands on the cluster.

This makes the behavior independent of RIA ordering: the Kubernetes
annotation is stripped by default on every path (including when the
target PVC does not exist and Velero falls back to provisioning a new
PVC), and preservation only happens when the CSI RIA explicitly
captured a value from the existing PVC.

Signed-off-by: chlins <chlins.zhang@gmail.com>

* Update the control path to make the in-place incremental restore with block data mover work E2E (#10410)

Update the control path to make the in-place incremental restore with block data mover work E2E

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>

---------

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
Signed-off-by: chlins <chlins.zhang@gmail.com>
Co-authored-by: chlins <chlins.zhang@gmail.com>
2026-08-26 10:21:32 -04:00
d2241adba0 Log the discovered parent snapshot ID, not the empty lookup parameter (#10305)
e2e-test-kind.yaml / extract (push) Failing after 8s
Run the E2E test on kind / get-go-version (push) Failing after 9s
Run the E2E test on kind / build (push) Skipped
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Skipped
push.yml / extract (push) Failing after 6s
Main CI / get-go-version (push) Failing after 7s
Main CI / Build (push) Skipped
getParentBackupInfo's parent-selection log lines interpolated the
parentSnapshot parameter. On the discovery branch (no explicit parent
passed by the caller) that parameter is empty by definition, so every
message about which parent was chosen, or why a run fell back to full,
printed no identifier at all -- e.g. "Using parent snapshot , start
time ...". This is the normal path for scheduled/incremental backups,
so the omission hit the common case, not an edge one.

Bind a parentID local that starts as the parameter but is overwritten
once a parent is actually resolved (explicit or discovered), and log
that instead. No behavior change.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 16:47:40 -04:00
Lyndon-Li 882ad944f3 Merge branch 'main' into block-uploader-restore-progress 2026-08-11 15:53:08 +08:00
Lyndon-Li bbb0f11f33 use source size in progress for block uploader restore
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-11 15:49:30 +08:00
Lyndon-Li c8127e243b object reader throughput improvement
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-11 14:07:59 +08:00
Lyndon-Li de87def9d9 enable object reader prefetch for block uploader
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-11 13:02:11 +08:00
Lyndon-Li a41cb11902 add prefetch options for repo interface
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-11 12:56:23 +08:00
Jay2006sawant 8ec4b22496 fix: return errors correctly in block restore validation and BatchForget
Signed-off-by: Jay2006sawant <jay242902@gmail.com>
2026-08-03 09:44:00 +05:30
lyndon-liandGitHub 23a0cbe163 add incremental size for block uploader (#10151)
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-04 16:00:45 +08:00
Lyndon-Li f22b7c86d7 upload progress every 10s
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-03 14:11:37 +08:00
Lyndon-Li c7d7801ae9 Merge branch 'main' into optimize-sub-object-description 2026-07-29 18:09:28 +08:00
Lyndon-Li d685b818ad refactor block uploader thread module
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-28 15:43:13 +08:00
Lyndon-Li 525fb9eaa2 Merge branch 'main' into optimize-sub-object-description 2026-07-28 15:14:15 +08:00
Lyndon-Li b635d3f8ed refactor block uploader thread module
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-28 13:20:34 +08:00
Lyndon-Li bc596da386 block uploader restore implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-27 14:19:22 +08:00
Lyndon-Li 00f1626f7a block uploader restore implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-24 14:02:19 +08:00
Lyndon-Li 236c45b443 block uploader restore UT
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-24 13:37:40 +08:00
Lyndon-Li 97978ed9b7 block uploader flush zero blocks
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-22 18:01:57 +08:00
Lyndon-Li 9c1d8bee71 block uploader restore data
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-22 18:00:13 +08:00
Lyndon-Li bec292e738 block uploader restore data
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-22 17:58:02 +08:00
Lyndon-Li 7bbd172684 persist source dev size
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-22 17:52:01 +08:00
Lyndon-Li 2b2aa061a8 optimize subobject description
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-20 14:17:34 +08:00
lyndon-liandGitHub a0749e7658 Block dev for restore (#10013)
Run the E2E test on kind / get-go-version (push) Failing after 1m3s
Run the E2E test on kind / build (push) Has been skipped
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / get-go-version (push) Successful in 13s
Main CI / Build (push) Failing after 22s
* add block dev operations for block uploader restore

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* add block dev operations for block uploader restore

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

---------

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-17 16:35:19 +08:00
lyndon-liandGitHub b0d7ada06a Block dev for block uploader backup (#9994)
* add block dev operations for block data mover backup

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* add block dev operations for block data mover backup

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* Add block device operations for block uploader backup

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* Add block device operations for block uploader backup

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

---------

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-16 13:44:41 +08:00
Lyndon-Li 84bee82575 block uploader backup implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-09 13:15:59 +08:00
Lyndon-Li 8d23c7e813 block uploader backup implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-30 17:03:39 +08:00
Lyndon-Li 502ec5f086 remove leading and trailing separator
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-30 16:22:55 +08:00
Lyndon-Li 39c745ef61 set totalSize from uploader
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-30 16:22:21 +08:00
Lyndon-Li df21463629 block uploader backup implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-30 16:21:59 +08:00
Lyndon-Li f38bc20a0a block uploader snapshot implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-29 15:02:10 +08:00
Lyndon-Li 5bef38dc95 block uploader snapshot implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-25 08:31:32 +00:00
Lyndon-Li f4f897f669 load object from snapshot
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-25 15:36:33 +08:00
Lyndon-Li 845bd2dc4f block uploader snapshot implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-25 15:27:18 +08:00