Commit Graph
144 Commits
Author SHA1 Message Date
d2241adba0 Log the discovered parent snapshot ID, not the empty lookup parameter (#10305)
e2e-test-kind.yaml / extract (push) Failing after 8s
Run the E2E test on kind / get-go-version (push) Failing after 9s
Run the E2E test on kind / build (push) Skipped
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Skipped
push.yml / extract (push) Failing after 6s
Main CI / get-go-version (push) Failing after 7s
Main CI / Build (push) Skipped
getParentBackupInfo's parent-selection log lines interpolated the
parentSnapshot parameter. On the discovery branch (no explicit parent
passed by the caller) that parameter is empty by definition, so every
message about which parent was chosen, or why a run fell back to full,
printed no identifier at all -- e.g. "Using parent snapshot , start
time ...". This is the normal path for scheduled/incremental backups,
so the omission hit the common case, not an edge one.

Bind a parentID local that starts as the parameter but is overwritten
once a parent is actually resolved (explicit or discovered), and log
that instead. No behavior change.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 16:47:40 -04:00
339c8edda9 Detect block uploader cancellation through wrapped errors (#10308)
* Detect block uploader cancellation through wrapped errors

Cancelling a block data mover backup was reported as a failure: the
DataUpload ended Failed with an error message and the Backup went
PartiallyFailed, for a user-requested cancel.

The cause is a sentinel equality check. block.ErrCanceled is raised in
the write loop and then wrapped twice before it reaches the provider --
once in block/uploader.go ("error backing up bdev %s") and again in
block/snapshot.go ("Failed to run uploader backup for si %v") -- so
`err == block.ErrCanceled` can never be true and the ErrorCanceled
returns are unreachable. The filesystem provider avoids this by asking
the uploader for its state (kpUploader.IsCanceled()) rather than
inspecting the error.

Use errors.Is at both the backup and restore sites.

Adds TestBlockProviderCancelThroughWrappedError, which injects the
doubly-wrapped sentinel exactly as production builds it. Note the
assertion is require.ErrorIs, not ErrorContains: provider.ErrorCanceled
and block.ErrCanceled carry identical message text, so a substring
assertion passes whether or not the sentinel was recognised -- which is
why the existing test, injecting the bare sentinel, did not catch this.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
(cherry picked from commit 9d6c5da7a893068d424b0c7896638787c636e213)
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

* Add changelog for #10308

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

* lint: fix misspelling (recognised -> recognized)

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

---------

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 15:03:33 +08:00
Lyndon-Li 882ad944f3 Merge branch 'main' into block-uploader-restore-progress 2026-08-11 15:53:08 +08:00
Lyndon-Li bbb0f11f33 use source size in progress for block uploader restore
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-11 15:49:30 +08:00
Lyndon-Li c8127e243b object reader throughput improvement
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-11 14:07:59 +08:00
Lyndon-Li de87def9d9 enable object reader prefetch for block uploader
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-11 13:02:11 +08:00
Lyndon-Li a41cb11902 add prefetch options for repo interface
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-11 12:56:23 +08:00
Jay2006sawant 8ec4b22496 fix: return errors correctly in block restore validation and BatchForget
Signed-off-by: Jay2006sawant <jay242902@gmail.com>
2026-08-03 09:44:00 +05:30
lyndon-liandGitHub 23a0cbe163 add incremental size for block uploader (#10151)
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-04 16:00:45 +08:00
Lyndon-Li f22b7c86d7 upload progress every 10s
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-03 14:11:37 +08:00
Lyndon-Li c7d7801ae9 Merge branch 'main' into optimize-sub-object-description 2026-07-29 18:09:28 +08:00
Lyndon-Li d685b818ad refactor block uploader thread module
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-28 15:43:13 +08:00
Lyndon-Li 525fb9eaa2 Merge branch 'main' into optimize-sub-object-description 2026-07-28 15:14:15 +08:00
Lyndon-Li b635d3f8ed refactor block uploader thread module
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-28 13:20:34 +08:00
Lyndon-Li bc596da386 block uploader restore implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-27 14:19:22 +08:00
Lyndon-Li 00f1626f7a block uploader restore implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-24 14:02:19 +08:00
Lyndon-Li 236c45b443 block uploader restore UT
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-24 13:37:40 +08:00
Lyndon-Li 97978ed9b7 block uploader flush zero blocks
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-22 18:01:57 +08:00
Lyndon-Li 9c1d8bee71 block uploader restore data
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-22 18:00:13 +08:00
Lyndon-Li bec292e738 block uploader restore data
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-22 17:58:02 +08:00
Lyndon-Li 7bbd172684 persist source dev size
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-22 17:52:01 +08:00
Lyndon-Li 2b2aa061a8 optimize subobject description
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-20 14:17:34 +08:00
lyndon-liandGitHub a0749e7658 Block dev for restore (#10013)
Run the E2E test on kind / get-go-version (push) Failing after 1m3s
Run the E2E test on kind / build (push) Has been skipped
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / get-go-version (push) Successful in 13s
Main CI / Build (push) Failing after 22s
* add block dev operations for block uploader restore

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* add block dev operations for block uploader restore

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

---------

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-17 16:35:19 +08:00
lyndon-liandGitHub b0d7ada06a Block dev for block uploader backup (#9994)
* add block dev operations for block data mover backup

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* add block dev operations for block data mover backup

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* Add block device operations for block uploader backup

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* Add block device operations for block uploader backup

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

---------

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-16 13:44:41 +08:00
Lyndon-Li 84bee82575 block uploader backup implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-09 13:15:59 +08:00
Lyndon-Li ce1c1803b2 Merge branch 'main' into block-uploader-backup-implementation 2026-07-09 11:39:30 +08:00
Xun Jiang/Bruce JiangandGitHub 96ae6586d0 Merge pull request #9863 from blackpiglet/jxun/change-id
Support change-id and volume-id in backup workflow.
2026-07-02 11:32:15 +08:00
Xun Jiang 0bc06323bf Support change-id and volume-id in backup workflow.
* Add change-id and volume-id retrieve logic for both vks and vanilla k8s environment.
* Add change-id and volume-id support code in exposer.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2026-07-01 15:19:52 +08:00
Lyndon-Li 8d23c7e813 block uploader backup implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-30 17:03:39 +08:00
Lyndon-Li 502ec5f086 remove leading and trailing separator
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-30 16:22:55 +08:00
Lyndon-Li 39c745ef61 set totalSize from uploader
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-30 16:22:21 +08:00
Lyndon-Li df21463629 block uploader backup implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-30 16:21:59 +08:00
Lyndon-Li 8df8709a8a block uploader snapshot implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-30 14:14:23 +08:00
Lyndon-Li 82dbef2cd9 block uploader snapshot implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-30 11:18:44 +08:00
Lyndon-Li f38bc20a0a block uploader snapshot implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-29 15:02:10 +08:00
Lyndon-Li 5bef38dc95 block uploader snapshot implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-25 08:31:32 +00:00
Lyndon-Li f4f897f669 load object from snapshot
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-25 15:36:33 +08:00
Lyndon-Li 845bd2dc4f block uploader snapshot implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-25 15:27:18 +08:00
Xun Jiang 49b670a791 Replace github.com/pkg/errors by github.com/cockroachdb/errors
Change errors.Cause to errors.Is, because github.com/cockroachdb/errors
New() function create a error with error stack with depth 1, but
github.com/pkg/errors's New() function create error with no depth.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2026-06-10 15:55:57 +08:00
Lyndon-Li d08790534a Merge branch 'main' into data-path-naming-adjustment 2026-05-22 15:08:29 +08:00
Lyndon-Li 9f0194d8fe Uploader interface fo block data mover
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-05-22 12:26:00 +08:00
Lyndon-Li cf5f5de911 Merge branch 'main' into data-path-naming-adjustment 2026-05-21 14:44:25 +08:00
Lyndon-Li d756df874f Merge branch 'main' into uploader-interface-fo-block-data-mover 2026-05-21 13:21:14 +08:00
Lyndon-Li 1d03217661 add CBT bitmap implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-05-21 09:20:24 +08:00
Lyndon-Li 2346314729 call block uploader provider creation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-05-14 16:17:51 +08:00
Lyndon-Li 0be9fc7d09 add block uploader provider
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-05-11 17:31:40 +08:00
Lyndon-Li 22be7e3218 data path naming adjustment
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-05-11 13:53:49 +08:00
Lyndon-Li 9bbadf346d modify uploader provider interface for block data mover
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-04-22 14:15:17 +08:00
Lyndon-Li 6b7df3ef4c add CBT bitmap implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-04-21 17:52:06 +08:00
Lyndon-Li 26b125769e add CBT bitmap implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-04-21 09:50:53 +00:00