Commit Graph
3017 Commits
Author SHA1 Message Date
Wenkai Yin(尹文开) 712b8caa9b Update Kopia(filesystem) uploader to support incremental and deleteExtraFile during restore (#10066)
Run the E2E test on kind / setup-test-matrix (push) Successful in 4s
e2e-test-kind.yaml / extract (push) Failing after 17s
Run the E2E test on kind / get-go-version (push) Failing after 18s
Run the E2E test on kind / build (push) Skipped
Run the E2E test on kind / run-e2e-test (push) Skipped
Update Kopia(filesystem) uploader to support incremental and deleteExtraFile during restore

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2026-07-29 16:05:53 +08:00
Wenkai Yin(尹文开) 0e94220c27 Update CRDs and CLI to support in-place restore (#10038)
Update CRDs(Restore, DataDownload, PodVolumeRestore) and restore create CLI to support in-place restore

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2026-07-29 16:05:52 +08:00
5691f7f29d Support overriding Schedule annotations via template.metadata.annotations (#10045)
Run the E2E test on kind / setup-test-matrix (push) Successful in 4s
e2e-test-kind.yaml / extract (push) Failing after 13s
Run the E2E test on kind / get-go-version (push) Failing after 14s
Run the E2E test on kind / build (push) Skipped
Run the E2E test on kind / run-e2e-test (push) Skipped
push.yml / extract (push) Successful in 14s
Main CI / get-go-version (push) Successful in 16s
Main CI / Build (push) Failing after 2m46s
* Support overriding Schedule annotations via template.metadata.annotations

Adds an Annotations field to BackupSpec.Metadata, mirroring the existing
Labels override. When Schedule.Spec.Template.Metadata.Annotations is set,
it is used for the resulting Backup's annotations instead of copying
Schedule.Annotations directly, allowing users to opt out of unwanted
annotations (e.g. ArgoCD tracking annotations) being propagated from
Schedule to Backup.

Fixes #5836

Signed-off-by: Lubron Zhan <lubronzhan@gmail.com>

* Rename changelog fragment to match PR number 10045

Signed-off-by: Lubron Zhan <lubronzhan@gmail.com>

---------

Signed-off-by: Lubron Zhan <lubronzhan@gmail.com>
Co-authored-by: Daniel Jiang <daniel.jiang@broadcom.com>
2026-07-29 15:55:51 +08:00
Scott SeagoandGitHub 041f577343 Merge branch 'main' into fix-vsc-hardcode 2026-07-28 20:21:50 -04:00
Adam Zhang 63cfddd18d add tests to cover pvc and vsc ria
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
2026-07-28 16:06:04 +08:00
Lyndon-Li d685b818ad refactor block uploader thread module
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-28 15:43:13 +08:00
Adam Zhang ef100da89b remove VolumeSnapshotContents from resourceMustHave list
Stop force-including VolumeSnapshotContents via resourceMustHave on
every restore; CSI VolumeSnapshot/PVC RestoreItemActions now set
`restore.velero.io/must-include-additional-items` so bound snapshot
dependencies are restored only when their parent is restored.

Fixes: #9957

Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
2026-07-28 14:04:15 +08:00
Lyndon-Li a389cb4560 Merge branch 'main' into block-uploader-thread-module-refactor 2026-07-28 13:21:28 +08:00
Lyndon-Li b635d3f8ed refactor block uploader thread module
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-28 13:20:34 +08:00
Adam ZhangandGitHub a43a1bce6a Add RIA must-include additional items (#10082)
Let RestoreItemActions opt in via annotation to
bypass global restore filters for AdditionalItems,
mirroring the backup-side must-include behavior.

Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
2026-07-27 16:36:55 -04:00
lyndon-liandGitHub 1711094110 Merge pull request #10071 from Lyndon-Li/block-uploader-restore-implementation
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
e2e-test-kind.yaml / extract (push) Failing after 15s
Run the E2E test on kind / get-go-version (push) Failing after 16s
Run the E2E test on kind / build (push) Skipped
Run the E2E test on kind / run-e2e-test (push) Skipped
push.yml / extract (push) Successful in 13s
Main CI / get-go-version (push) Successful in 14s
Main CI / Build (push) Failing after 1m6s
Block uploader restore implementation
2026-07-27 15:40:52 +08:00
Xun Jiang/Bruce JiangandGitHub 3905ccb0ea Backup workflow for block data mover. (#10067)
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2026-07-27 15:36:44 +08:00
Lyndon-Li bc596da386 block uploader restore implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-27 14:19:22 +08:00
Lyndon-Li d81b319abf Merge branch 'main' into block-uploader-restore-implementation 2026-07-27 14:12:06 +08:00
a12b373e4c Support set-based filter label selectors (#10064)
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
e2e-test-kind.yaml / extract (push) Failing after 12s
Run the E2E test on kind / get-go-version (push) Failing after 13s
Run the E2E test on kind / build (push) Skipped
Run the E2E test on kind / run-e2e-test (push) Skipped
push.yml / extract (push) Successful in 10s
Main CI / get-go-version (push) Successful in 11s
Main CI / Build (push) Failing after 25s
* Support set-based filter label selectors

Use matchLabels/matchExpressions in fine-grained filters.

Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>

* omit the details of resource policy for cli

The reason to not resolve and display CLI is because it may go
out of sync, we want to avoid display it to mislead users.

We may consider to cpature those information and display it
in later release.

Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>

---------

Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
Co-authored-by: Scott Seago <sseago@redhat.com>
2026-07-24 16:03:51 -04:00
Lyndon-Li 00f1626f7a block uploader restore implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-24 14:02:19 +08:00
Lyndon-Li f6f2ecf8cc Merge branch 'main' into block-uploader-restore-implementation 2026-07-24 13:38:40 +08:00
Lyndon-Li 236c45b443 block uploader restore UT
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-24 13:37:40 +08:00
e5654fa7ed Fix flaky TestKopiaObjectWriterEx_ConcurrentAsyncErrors (#10030)
The test assumed all ten Write calls succeed before any async goroutine
stores its error, but with a mock that fails instantly a goroutine can
poison the writer mid-loop, making a later Write correctly fail fast —
a timing-dependent test failure.

Rewrite the test to assert the real-world contract instead of one
schedule: a failed async block write either fails a subsequent Write
fast or surfaces at Result, and is never lost. Add a separate
deterministic case pinning the late-error schedule, holding async
writes until all writes are queued so Result alone must report the
error.

Verified with -race -count=100.

Fixes #10029

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 15:42:36 -04:00
lyndon-liandGitHub 564d213e6b Merge pull request #10035 from HajimohammadiNet/fix-install-plugin-spacing
Run the E2E test on kind / get-go-version (push) Failing after 1m0s
Run the E2E test on kind / build (push) Has been skipped
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / get-go-version (push) Successful in 14s
Main CI / Build (push) Failing after 29s
Trim plugin image entries during install
2026-07-23 14:45:21 +08:00
Shubham PampattiwarandGitHub c676fdd03e Merge pull request #10000 from shubham-pampattiwar/fix/cleanup-stale-backup-metrics
Fix stale backupLastSuccessfulTimestamp metric after schedule deletion
2026-07-22 23:26:02 -07:00
Wenkai Yin(尹文开)andGitHub d69f6abe5c Add param to StartRestore to facilitates future expansion (#10057)
Add param to StartRestore to facilitates future expansion

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2026-07-23 10:31:42 +08:00
Shubham Pampattiwar 0f01b534c3 Remove unused collectGaugeCount, assert surviving label values
Remove the unused collectGaugeCount helper and assert specific label
values survive after each deletion using testutil.ToFloat64.

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2026-07-22 10:43:13 -07:00
Shubham Pampattiwar 911cc9eb9e Fix gofmt alignment in backupReconciler struct
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2026-07-22 09:57:28 -07:00
Shubham Pampattiwar 4ea38216f5 Address review feedback: use DeleteLabelValues, extract resync method
Replace blanket Reset() with targeted DeleteLabelValues to avoid
briefly wiping metrics for schedules that still exist. Track known
schedules in a set and only delete stale entries on each resync.

Extract the wait.Until closure into resyncBackupMetrics() so tests
can call it directly without goroutine timing. Replace hand-rolled
collectGaugeCount helper with testutil.CollectAndCount.

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2026-07-22 09:17:59 -07:00
Shubham Pampattiwar b9d9dcfc38 Add integration test for updateTotalBackupMetric resync
Add a test that exercises the actual updateTotalBackupMetric goroutine
with a fake client to verify stale backupLastSuccessfulTimestamp
entries are pruned during a real resync cycle.

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2026-07-22 09:17:59 -07:00
Shubham Pampattiwar bfeccba0a8 Move metric reset inside List success block
Avoid clearing backupLastSuccessfulTimestamp on transient API errors.
The reset and re-set now only run when the backup List call succeeds,
so existing metric values remain stable across temporary failures.

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2026-07-22 09:17:59 -07:00
Shubham Pampattiwar 6ea95548d6 Remove BackupLastSuccessfulTimestampCount, use Metrics() in test
Remove the exported method that was only used in tests. Use the
existing Metrics() getter to access the gauge directly in the
backup controller test instead.

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2026-07-22 09:17:59 -07:00
Shubham Pampattiwar 8cf03998dd Reset backupLastSuccessfulTimestamp during periodic resync
The periodic backup metrics resync in updateTotalBackupMetric only set
backupLastSuccessfulTimestamp values but never removed stale entries.
When a schedule was deleted and its backups removed, the gauge persisted
until the Velero pod was restarted.

Reset the gauge before re-setting current values so that deleted
schedules are pruned automatically each resync cycle.

Fixes #9239

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2026-07-22 09:17:59 -07:00
Shubham Pampattiwar 1968bf44ee Add ResetBackupLastSuccessfulTimestamp to ServerMetrics
Add a method to reset all backupLastSuccessfulTimestamp gauge values.
This will be used by the backup controller's periodic resync to prune
stale metrics for deleted schedules.

Fixes #9239

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2026-07-22 09:17:59 -07:00
Lyndon-Li 97978ed9b7 block uploader flush zero blocks
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-22 18:01:57 +08:00
Lyndon-Li 9c1d8bee71 block uploader restore data
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-22 18:00:13 +08:00
Lyndon-Li bec292e738 block uploader restore data
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-22 17:58:02 +08:00
Lyndon-Li 7bbd172684 persist source dev size
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-22 17:52:01 +08:00
Xun Jiang/Bruce JiangandGitHub 36647b0b2f Merge pull request #9974 from blackpiglet/jxun/fips-140
Disable fips140 enforcement because Kopia doesn't support it.
2026-07-22 17:28:58 +08:00
Adam ZhangandGitHub d59e7d4eb8 Prioritize exact namespace match in restore (#10033)
Run the E2E test on kind / get-go-version (push) Failing after 1m12s
Run the E2E test on kind / build (push) Has been skipped
Run the E2E test on kind / setup-test-matrix (push) Successful in 4s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / get-go-version (push) Successful in 12s
Main CI / Build (push) Failing after 26s
* Prioritize exact namespace match in restore

Align restore pipeline with backup pipeline by
evaluating exact namespace matches before glob
patterns in namespacedFilterPolicies. This ensures
specific overrides always win regardless of list
order.

Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>

* improve test cases

add test cases for exact listed first, and excat listed last to
ensure the behavior that the order does not matter for exact
listed namespace, the rule will be always honored.

Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>

---------

Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
2026-07-20 09:18:13 -04:00
lyndon-liandGitHub b96bb73308 Merge pull request #10007 from Lyndon-Li/cancel-pvb-on-timeout
Issue 9997: cancel ongoing PVB on timeout
2026-07-20 16:31:16 +08:00
lyndon-liandGitHub 32242d2f0a Merge pull request #10005 from Lyndon-Li/fail-earlier-when-pvr-pod-is-not-ready
Issue 9973: fail earlier when PVR pod is not ready
2026-07-20 16:30:50 +08:00
hajimohammadi.netandGitHub 14c3c87292 Merge branch 'main' into fix-install-plugin-spacing 2026-07-19 09:59:41 +03:30
AmirHossein HajiMohammadi bd7b2ed690 Trim plugin image entries during install
Signed-off-by: AmirHossein HajiMohammadi <a.hajimohammadi@rahkar.team>
2026-07-18 17:13:54 +03:30
Shubham Pampattiwar 251272ae09 Address review feedback: update short names for Backup, BackupRepository, and DownloadRequest
- Backup: bkp -> bak (more common abbreviation)
- BackupRepository: br -> repo (avoids conflict with "backup & restore")
- DownloadRequest: dr -> dreq (avoids conflict with "disaster recovery")

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2026-07-17 11:44:40 -07:00
Shubham Pampattiwar deba91ab9c Add CRD short names for all Velero custom resources
Only 3 of 13 Velero CRDs had short names (bsl, vsl, ssr). This adds
short names to the remaining 10 CRDs for better kubectl usability:

  Backup=bkp, Restore=rst, Schedule=sched, BackupRepository=br,
  DeleteBackupRequest=dbr, DownloadRequest=dr, PodVolumeBackup=pvb,
  PodVolumeRestore=pvr, DataUpload=du, DataDownload=dd

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2026-07-17 11:42:09 -07:00
lyndon-liandGitHub a0749e7658 Block dev for restore (#10013)
Run the E2E test on kind / get-go-version (push) Failing after 1m3s
Run the E2E test on kind / build (push) Has been skipped
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / get-go-version (push) Successful in 13s
Main CI / Build (push) Failing after 22s
* add block dev operations for block uploader restore

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* add block dev operations for block uploader restore

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

---------

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-17 16:35:19 +08:00
Wenkai Yin(尹文开) ddac19121a Only run PVC CSI RIA for CSI snapshot
Put the checking logic at the begining to make sure only run PVC CSI RIA for CSI snapshot

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2026-07-17 13:44:41 +08:00
Chlins ZhangandGitHub 39017a205a Merge pull request #10004 from chlins/feat/data-mover-selection
Run the E2E test on kind / get-go-version (push) Failing after 55s
Run the E2E test on kind / build (push) Has been skipped
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / get-go-version (push) Successful in 15s
Main CI / Build (push) Failing after 23s
feat(resourcepolicies): support dataMover parameter in snapshot volume
2026-07-16 16:54:46 +08:00
lyndon-liandGitHub b0d7ada06a Block dev for block uploader backup (#9994)
* add block dev operations for block data mover backup

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* add block dev operations for block data mover backup

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* Add block device operations for block uploader backup

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* Add block device operations for block uploader backup

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

---------

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-16 13:44:41 +08:00
Lyndon-Li 8ade19adfc issue 9997: cancel ongoing PVB on timeout
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-15 17:29:20 +08:00
Xun Jiang d82c552aaa Add BackupType in backup.spec.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2026-07-15 08:35:06 +00:00
chlins 666d14de32 feat(resourcepolicies): support dataMover parameter in snapshot volume policy action
Signed-off-by: chlins <chlins.zhang@gmail.com>
2026-07-15 15:26:23 +08:00
Lyndon-Li a94e01760b fail earlier when PVR pod is not ready
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-15 15:24:17 +08:00