Commit Graph
34 Commits
Author SHA1 Message Date
Lyndon-Li f9ebe14e16 add backup/restore type and fallback to backup/restore describe
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-09-10 17:23:46 +08:00
0255c6b8bf Add block data mover support for Velero backup/restore describe CLI. (#10436)
e2e-test-kind.yaml / extract (push) Failing after 10s
Run the E2E test on kind / get-go-version (push) Failing after 11s
Run the E2E test on kind / build (push) Skipped
Run the E2E test on kind / setup-test-matrix (push) Failing after 3s
Run the E2E test on kind / run-e2e-test (push) Skipped
push.yml / extract (push) Failing after 7s
Main CI / get-go-version (push) Failing after 8s
Main CI / Build (push) Skipped
* Add block data mover support for Velero backup/restore describe CLI.

Update output tests to accommodate RestoreType in VolumeInfo
This commit addresses the compilation and assertion errors caused by the introduction of `RestoreType` in `VolumeInfo` and the separation of `SnapshotDataMovementInfo` / `PodVolumeInfo` into their backup and restore counterparts. It fixes references across the test fixtures and the print guard conditions in `restore_describer.go`.

* Modify according to comments
* Add missing JSON tag in the VolumeInfo structures.
* Get uploaderType from the DU and DD's dataMover for the data mover volume info.
* Add IncrementalSize in the data mover volume info.
* Add existingVolumeDataPolicy and restoreType in the restore describe CLI output
* Add more UTs.
* Add some fields value setting that were previously missed.
* Fix the timestamp compare error only found in the GitHub action.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-07 14:48:41 +08:00
2c6f45508c Report a measured zero incremental instead of erasing it (#10309)
Run the E2E test on kind / setup-test-matrix (push) Failing after 4s
e2e-test-kind.yaml / extract (push) Failing after 11s
Run the E2E test on kind / get-go-version (push) Failing after 12s
Run the E2E test on kind / build (push) Skipped
Run the E2E test on kind / run-e2e-test (push) Skipped
push.yml / extract (push) Failing after 7s
Main CI / get-go-version (push) Failing after 8s
Main CI / Build (push) Skipped
* Report a measured zero incremental instead of erasing it

A CBT incremental with an exactly zero delta -- nothing changed since
the parent -- was reported identically to a backup that moved the whole
device. `velero backup describe --details` printed only
"Moved data Size (bytes): 3221225472" with no incremental line, and
status.incrementalBytes was absent, for a run that transferred nothing.
The best possible CBT outcome displayed as the worst, and was
indistinguishable from a genuine full, a whole-device fallback, or a
backup predating incremental accounting.

The zero was being erased twice. Besides the API status fields,
datapath.BackupResult also carried omitempty, and that struct crosses a
JSON boundary from the data mover pod to the controller (see
micro_service_watcher.go), so the value was destroyed before the
controller could persist it. Every uploader always reports a figure
there, so 0 internally always means "transferred nothing" -- dropping
omitempty is sufficient and correct for that hop.

The API fields move to *int64 rather than just dropping omitempty. The
field shipped in v1.18.0-v1.18.2, so backups exist whose stored volume
info has no incrementalSize at all; with a plain int64 those unmarshal
to 0 and would render "Incremental data Size (bytes): 0", a false claim
of a perfect incremental on a run that never measured one. nil means not
measured, a pointer to 0 means measured zero. Both fields already carry
+optional, so the generated CRD schema is unchanged and no regeneration
is required.

Display gates relax from > 0 to != nil in all three places, including
volumesByPod.Add, whose signature takes *int64 now; the restore describer
passes nil, which is correct since restores measure no incremental.

Verified live: the same zero-delta scenario that reported <none> now
reports 0 and renders "Incremental data Size (bytes): 0", while an older
backup described with the new client still correctly prints no
incremental line at all.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
(cherry picked from commit 6c7aa9d588f6d5eab134d4ce19c92b838f45557c)
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

* gofmt: fix import ordering in backup_test.go

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

* Regenerate CRDs for IncrementalBytes pointer type

make update-crd was missed in the original commit. Regenerated with
the pinned controller-gen v0.16.5 to avoid unrelated version-annotation
churn across other CRDs.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

* Add changelog for #10309

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

* Address review: make IncrementalBytes a pointer to preserve backward compat

Per Lyndon-Li's review on #10309: dropping omitempty on the plain int64
field breaks compatibility with a data mover from release-1.17 or
earlier that predates IncrementalBytes and never writes the key -- the
new controller would unmarshal a zero value ("nothing transferred")
instead of recognizing the field is simply absent ("not measured").

Switch to *int64 with omitempty restored:
- an old mover's omitted key unmarshals to nil ("not measured")
- a current mover's genuine zero still serializes the key, unmarshaling
  to a non-nil pointer to 0 ("measured zero")
- nonzero values work exactly as before
- an old controller can still unmarshal a numeric value from a new mover

pkg/controller/data_upload_controller.go and pod_volume_backup_controller.go
assign the wire-struct field directly to their already-*int64,omitempty
CRD status field instead of re-wrapping it with ptr.To, since both are
now the same pointer type.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

* Fix CI: update marshal-fail test assertions for IncrementalBytes pointer

Both backup_micro_service_test.go files hardcoded the %v-formatted
zero-value BackupResult struct in an error-message assertion. Now that
IncrementalBytes is *int64, its zero value prints as <nil> instead of 0.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

---------

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-25 18:08:17 -04:00
a12b373e4c Support set-based filter label selectors (#10064)
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
e2e-test-kind.yaml / extract (push) Failing after 12s
Run the E2E test on kind / get-go-version (push) Failing after 13s
Run the E2E test on kind / build (push) Skipped
Run the E2E test on kind / run-e2e-test (push) Skipped
push.yml / extract (push) Successful in 10s
Main CI / get-go-version (push) Successful in 11s
Main CI / Build (push) Failing after 25s
* Support set-based filter label selectors

Use matchLabels/matchExpressions in fine-grained filters.

Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>

* omit the details of resource policy for cli

The reason to not resolve and display CLI is because it may go
out of sync, we want to avoid display it to mislead users.

We may consider to cpature those information and display it
in later release.

Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>

---------

Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
Co-authored-by: Scott Seago <sseago@redhat.com>
2026-07-24 16:03:51 -04:00
chlins 21d67a8622 feat(backup): add global backup volume policies
Signed-off-by: chlins <chlins.zhang@gmail.com>
2026-06-25 11:31:08 +08:00
Adam Zhang 0d719f1d8a cli support for fine-grained filter policies
add cli support for NamespacedFilterPolicies and
ClusterScopedFilterPolicy

Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
2026-06-04 13:25:16 +08:00
Scott Seago 5fc76db8c0 Add incrementalSize to DU/PVB for reporting new/changed size
Signed-off-by: Scott Seago <sseago@redhat.com>
2025-10-27 15:38:31 -04:00
Daniel Jiang 249d8f581a Add include/exclude policy to resources policy
fixes #8610

This commit extends the resources policy, such that user can define
resource include exclude filters in the policy and reuse it in different backups.

Signed-off-by: Daniel Jiang <daniel.jiang@broadcom.com>
2025-08-05 15:16:59 +08:00
Shubham PampattiwarandGitHub aa2e09c69e Update Backup describe string for DefaultVolumesToFSBackup flag (#9105)
Run the E2E test on kind / build (push) Failing after 7m54s
Run the E2E test on kind / setup-test-matrix (push) Successful in 4s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / Build (push) Failing after 36s
add changelog file

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2025-07-23 17:55:28 -04:00
Shubham PampattiwarandGitHub 60a6c7384f Fix missing defaultVolumesToFsBackup flag output in Velero describe backup cmd (#9056)
add changelog file



Show defaultVolumesToFsBackup in describe only when set by the user



minor ut fix



minor fix

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2025-07-23 09:59:51 -04:00
Lyndon-Li e4caab4086 support cancel for PVB/PVR in backups/restores
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2025-06-18 15:58:28 +08:00
Matthieu MORELandGitHub c6a420bd3a chore: define common aliases for k8s packages (#8672)
Run the E2E test on kind / build (push) Failing after 6m48s
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / Build (push) Failing after 35s
Close stale issues and PRs / stale (push) Successful in 8s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 1m11s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-aws, main) (push) Failing after 47s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-gcp, main) (push) Failing after 49s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-microsoft-azure, main) (push) Failing after 43s
* lchore: define common alias for k8s packages

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

* Update .golangci.yaml

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

* Update .golangci.yaml

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

* Update .golangci.yaml

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>

---------

Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2025-04-22 06:14:47 -04:00
Matthieu MORELandGitHub 35c90f1672 testifylint: enable error-nil rule (#7670)
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-07-16 12:23:16 -04:00
Xun Jiang df28134e25 Add result in backup VolumeInfo.
Signed-off-by: Xun Jiang <blackpigletbruce@gmail.com>
2024-07-02 00:20:36 +08:00
Xun Jiang 2eeaf4d55e Modify namespace filter logic for backup with label selector.
Signed-off-by: Xun Jiang <blackpigletbruce@gmail.com>
2024-04-18 10:30:59 +08:00
Daniel Jiang 0a280e5786 Track and persist restore volume info
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2024-04-11 17:32:18 +08:00
Xun Jiang/Bruce JiangandGitHub 7d2c749abf Merge pull request #7231 from blackpiglet/update_volumeinfo_json_tag
Don't generate empty structure.
2023-12-21 16:32:58 +08:00
Xun Jiang 9be8eb0c6d Don't generate empty structure.
VolumeInfo contains several sub-structures. They are filled for
different scenarios. Do not generate empty structure for the
not filled sub-structures.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-12-21 14:53:03 +08:00
Lyndon-Li 210838267f issue 7237: add pvc namespace to backup describe
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-12-21 10:02:27 +08:00
Lyndon-Li 0da01842ad remove csi feature check from backup describe
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-12-20 14:51:21 +08:00
Lyndon-Li 0313c2add0 issue 7214: data mover backup describe for legacy backups
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-12-18 11:07:01 +08:00
lyndon-liandGitHub 4070934f85 Merge pull request #7125 from Lyndon-Li/issue-fix-6695
Issue fix 6695: add describe for data mover backups
2023-12-07 16:23:30 +08:00
Ming Qiu 1a237d3e4c Update API
Signed-off-by: Ming Qiu <mqiu@vmware.com>
2023-12-06 08:59:12 +00:00
Lyndon-Li 72fcd84a51 csi data mover backup describe
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-12-06 10:53:09 +08:00
Lyndon-Li 8ab0c017a9 issue 6695: add backup description for data mover
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-11-27 16:19:34 +08:00
Lyndon-Li 1815c1691f Merge branch 'main' into issue-fix-6695 2023-11-27 09:46:22 +08:00
Lyndon-Li b99ac448ae issue 6695: add backup describe for CSI snapshot data movement
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-11-18 00:11:29 +08:00
Ming 481cb60493 Make Kopia file parallelism configurable
Signed-off-by: Ming <mqiu@vmware.com>
2023-10-26 02:28:36 +00:00
Wenkai Yin(尹文开)andGitHub 563f1ccee1 Merge pull request #6475 from nilesh-akhade/main
Add `--or-selector` for backup and restore command
2023-09-27 20:09:07 +08:00
Lyndon-Li dac28084a3 issue fix 6662
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-08-17 11:35:18 +08:00
Nilesh Akhade d9a7e2b6ca Add 'orLabelSelector' for backup, restore command
Signed-off-by: Nilesh Akhade <nakhade@catalogicsoftware.com>
2023-07-19 16:16:35 +05:30
allenxu404 75833eaa5b fix hook filter display issue for backup describer
Signed-off-by: allenxu404 <qix2@vmware.com>
2023-06-29 16:37:34 +08:00
Daniel Jiang 4cea533865 Add more unit test cases for cmd/util/output
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2023-06-27 00:37:31 +08:00
Daniel JiangandGitHub cd68dd369b Add Unit test cases for pkg/cmd/util/output (#6359)
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2023-06-26 08:16:36 +08:00