An in-place restore onto a different volume than the one backed up is
unsafe: an incremental (CBT) restore computes deltas against a different
volume lineage, and even a full restore would patch and write into an
unrelated volume. Verify the existing PVC is bound and still bound to
the PV recorded at backup time before any side effect, on both the CSI
data mover path (using the backed-up PVC's volume name) and the file
system path (using the PVC-to-PV mapping from the backup volume info).
The PV comparison is skipped for namespace-mapped restores, where the
target PVC is necessarily bound to a different PV (the documented
cross-namespace clone-and-restore workflow).
Signed-off-by: chlins <chlins.zhang@gmail.com>
* Enforce resource filters on cluster-wide items
When backups query all namespaces (wildcard or omitted includes),
the item collector retrieved resources in bulk, bypassing
per-namespace resource filter policies in Stage 1 collection.
This caused resources not listed in the policy to be backed up.
To preserve cluster-wide query performance while enforcing policy
rules, evaluate namespace exclusions, resource kind allowlists,
and label selectors in memory for each collected item.
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
* Optimize in-memory resource filter checks
Optimize per-item filter evaluation in the item collector:
- Precalculate GroupResource string once per resource type
- Skip filter policy evaluation when no namespaced policies exist
- Restrict in-memory filtering to cluster-wide queries
- Cache consecutive namespace lookups across collected items
- Lazily extract resource labels only when selectors are present
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
---------
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
Check the target PVC is not used by any active pod before any side
effect, on both the CSI data mover path and the file system path.
The in-use semantics align with the pvc-protection controller:
terminal-phase pods don't block, terminating pods block with a wait
hint. On the file system path, pods gated by this restore's
restore-wait init container (identified by the restore UID in its args,
and not yet terminated) are exempted: they must mount the PVC for the
node-agent to restore the data and cannot write to the volume until the
PodVolumeRestores complete. Leftover pods, controller-recreated pods,
and pods gated by a different restore still block.
Signed-off-by: chlins <chlins.zhang@gmail.com>
Get the volume ID before creating the restore PVC, otherwise the existing PV may be deleted during the creation of restore PVC
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
Add "IncrementalBytes" field to status of DataDownload and PVR to indicate data transferred by the incremental restore
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
- Replaced Expect().Should() and Expect().ShouldNot() with .To() and .ToNot() across 12 files (Task 1).
- Replaced synchronously evaluated Eventually() with Expect() in server_status_request_controller_test.go (Task 2B).
- Extracted Skip() calls inside lazy callbacks into conditional checks using slices.Contains() in enable_api_group_extentions.go (Task 3).
Signed-off-by: opbot_xd <awasthikrishna23052005@gmail.com>
* test: Unify duplicate test harness structs in restore and delete tests
Extract the duplicated test harness (fakeRestorer, Harness struct, and
AddResource/AddItems helpers) from restore_test.go and
delete_item_action_handler_test.go into a shared pkg/test.Harness.
Backup tests are intentionally excluded: the shared AddResource strips
metadata.creationTimestamp and status (restore semantics), which would
change what the backup tests tar up and assert on.
Signed-off-by: opbot_xd <awasthikrishna23052005@gmail.com>
* ci: retry
Signed-off-by: opbot_xd <awasthikrishna23052005@gmail.com>
* ci: retry
Signed-off-by: opbot_xd <awasthikrishna23052005@gmail.com>
---------
Signed-off-by: opbot_xd <awasthikrishna23052005@gmail.com>
The CLI now checks backup storage location access mode before submitting
a delete request. Also reject backups with no storage location set and
cache BSL lookups when deleting multiple backups.
Fixes#4203
Signed-off-by: PranjalManhgaye <manhgayepranjal@gmail.com>
Add comprehensive unit tests for pkg/itemblock which previously had
zero test coverage. The new tests cover AddUnstructured and FindItem
methods with 14 test cases across 3 test functions, including preferred
GVR ordering, nil item handling, namespace/name filtering, unparseable
apiVersion handling, and cluster-scoped resource matching.
Fixes#10418
Signed-off-by: opbot_xd <awasthikrishna23052005@gmail.com>
* Update CRDs and CLI to support in-place restore (#10038)
Update CRDs(Restore, DataDownload, PodVolumeRestore) and restore create CLI to support in-place restore
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
* Update Kopia(filesystem) uploader to support incremental and deleteExtraFile during restore (#10066)
Update Kopia(filesystem) uploader to support incremental and deleteExtraFile during restore
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
* Update Restore Exposer and PVC CSI to support in-place restore (#10104)
1. Update Restore Exposer to support exposing with existing PV for in-place restore
2. Update PVC CSI RIA to continue the restore process for in-place restore
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
* Update Block uploader to support increase restore (#10244)
Update Block uploader to support increase restore
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
* Update Exposer to recreate the target PV if the volume mode is different with the restore PVC (#10257)
Update Exposer to recreate the target PV if the volume mode is different with t
he restore PVC
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
* Preserve PVC selected-node annotation via carrier annotation for in-place restore
For in-place volume data restore, the existing PVC is deleted and
recreated. For StorageClasses with the WaitForFirstConsumer volume
binding mode, losing the volume.kubernetes.io/selected-node annotation
could let the scheduler place the recreated workload Pod in a different
zone than the original PV, leaving it stuck in ContainerCreating.
Instead of relying on RestoreItemAction execution order (the generic
PVC RIA unconditionally strips the selected-node annotation), the PVC
CSI RIA now captures the annotation from the existing PVC right before
deleting it and carries it on the target PVC via the Velero-internal
restore.velero.io/inplace-restore-selected-node annotation. The restore
engine translates the carrier back to the Kubernetes annotation after
all RestoreItemActions have run and always strips the carrier so it
never lands on the cluster.
This makes the behavior independent of RIA ordering: the Kubernetes
annotation is stripped by default on every path (including when the
target PVC does not exist and Velero falls back to provisioning a new
PVC), and preservation only happens when the CSI RIA explicitly
captured a value from the existing PVC.
Signed-off-by: chlins <chlins.zhang@gmail.com>
* Update the control path to make the in-place incremental restore with block data mover work E2E (#10410)
Update the control path to make the in-place incremental restore with block data mover work E2E
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
---------
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
Signed-off-by: chlins <chlins.zhang@gmail.com>
Co-authored-by: chlins <chlins.zhang@gmail.com>
* Report a measured zero incremental instead of erasing it
A CBT incremental with an exactly zero delta -- nothing changed since
the parent -- was reported identically to a backup that moved the whole
device. `velero backup describe --details` printed only
"Moved data Size (bytes): 3221225472" with no incremental line, and
status.incrementalBytes was absent, for a run that transferred nothing.
The best possible CBT outcome displayed as the worst, and was
indistinguishable from a genuine full, a whole-device fallback, or a
backup predating incremental accounting.
The zero was being erased twice. Besides the API status fields,
datapath.BackupResult also carried omitempty, and that struct crosses a
JSON boundary from the data mover pod to the controller (see
micro_service_watcher.go), so the value was destroyed before the
controller could persist it. Every uploader always reports a figure
there, so 0 internally always means "transferred nothing" -- dropping
omitempty is sufficient and correct for that hop.
The API fields move to *int64 rather than just dropping omitempty. The
field shipped in v1.18.0-v1.18.2, so backups exist whose stored volume
info has no incrementalSize at all; with a plain int64 those unmarshal
to 0 and would render "Incremental data Size (bytes): 0", a false claim
of a perfect incremental on a run that never measured one. nil means not
measured, a pointer to 0 means measured zero. Both fields already carry
+optional, so the generated CRD schema is unchanged and no regeneration
is required.
Display gates relax from > 0 to != nil in all three places, including
volumesByPod.Add, whose signature takes *int64 now; the restore describer
passes nil, which is correct since restores measure no incremental.
Verified live: the same zero-delta scenario that reported <none> now
reports 0 and renders "Incremental data Size (bytes): 0", while an older
backup described with the new client still correctly prints no
incremental line at all.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
(cherry picked from commit 6c7aa9d588f6d5eab134d4ce19c92b838f45557c)
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
* gofmt: fix import ordering in backup_test.go
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
* Regenerate CRDs for IncrementalBytes pointer type
make update-crd was missed in the original commit. Regenerated with
the pinned controller-gen v0.16.5 to avoid unrelated version-annotation
churn across other CRDs.
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
* Add changelog for #10309
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
* Address review: make IncrementalBytes a pointer to preserve backward compat
Per Lyndon-Li's review on #10309: dropping omitempty on the plain int64
field breaks compatibility with a data mover from release-1.17 or
earlier that predates IncrementalBytes and never writes the key -- the
new controller would unmarshal a zero value ("nothing transferred")
instead of recognizing the field is simply absent ("not measured").
Switch to *int64 with omitempty restored:
- an old mover's omitted key unmarshals to nil ("not measured")
- a current mover's genuine zero still serializes the key, unmarshaling
to a non-nil pointer to 0 ("measured zero")
- nonzero values work exactly as before
- an old controller can still unmarshal a numeric value from a new mover
pkg/controller/data_upload_controller.go and pod_volume_backup_controller.go
assign the wire-struct field directly to their already-*int64,omitempty
CRD status field instead of re-wrapping it with ptr.To, since both are
now the same pointer type.
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
* Fix CI: update marshal-fail test assertions for IncrementalBytes pointer
Both backup_micro_service_test.go files hardcoded the %v-formatted
zero-value BackupResult struct in an error-message assertion. Now that
IncrementalBytes is *int64, its zero value prints as <nil> instead of 0.
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
---------
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Issue #3194: Add velero client set-context-as-velero-namespace command
Saves the namespace of the current (or a specified) kubeconfig context
into the Velero client config file, so operational commands default to
it without requiring --namespace on every invocation.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: lubronzhan <lubron.zhan@broadcom.com>
* Fix CI: rename changelog to PR number, add unit tests for coverage
- changelogs/unreleased must be named <pr-number>-<username>; rename
from the 0000 placeholder to 10127 to satisfy hack/changelog-check.sh.
- Extract the command's logic into setContextAsVeleroNamespace so it's
testable without triggering os.Exit via cmd.CheckError, and add unit
tests covering: namespace read from context, context with no explicit
namespace, overwriting an existing config value, and invalid
kubeconfig path. Addresses 0% codecov patch coverage on the PR.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: lubronzhan <lubron.zhan@broadcom.com>
* Move set-namespace-from-context under client config
Shubham suggested nesting the new command under `config` for
hierarchy consistency, and renaming it since the original
set-context-as-velero-namespace name was long and ambiguous. Moves
it to `velero client config set-namespace-from-context`, matching
the existing config get/set subcommands and my follow-up naming
suggestion on the review thread.
AI-Tool-Used: Claude Code
AI-Tool-Use-Level: Category 3 (Low)
AI-Code-Category: Category 1 (Production)
Signed-off-by: lubronzhan <lubron.zhan@broadcom.com>
* Replace set-namespace-from-context with namespace-mode=auto
kaovilai noted on #10127 that a one-shot command to snapshot the
kubecontext namespace becomes redundant once a config toggle can
resolve it dynamically, and isn't much simpler than the existing
`config set namespace=...` alternative.
Drop the dedicated set-namespace-from-context subcommand and instead
teach the client Factory to resolve the operational namespace from the
current kubeconfig context on every invocation when
`namespace-mode=auto` is set via the existing generic
`config set` command. Explicit --namespace flags and VELERO_NAMESPACE
still take precedence, so the new mode only changes behavior when
neither is set.
AI-Tool-Used: Claude Code
AI-Tool-Use-Level: Category 2 (Medium)
AI-Code-Category: Category 1 (Production)
Signed-off-by: lubronzhan <lubron.zhan@broadcom.com>
* Address PR review: doc, fallback test, t.Setenv
Resolve feedback from PR #10127 review 4966054980:
- Document how to disable namespace-mode=auto (namespace-mode=)
and note the fallback to the static namespace, in namespace.md.
- Add a factory test covering the fallback to the stored/default
namespace when kubeconfig namespace resolution fails.
- Switch the VELERO_NAMESPACE override test to t.Setenv, wrapped
in a subtest so its cleanup runs before later tests execute.
AI-Tool-Used: Claude Code
AI-Tool-Use-Level: Category 2 (Medium)
AI-Code-Category: Category 2 (Non-Production)
Signed-off-by: lubronzhan <lubron.zhan@broadcom.com>
---------
Signed-off-by: lubronzhan <lubron.zhan@broadcom.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* Add readWriteOncePod backupPVC config to enable mount-level SELinux labeling
On SELinux-enabled clusters the kubelet recursively relabels every file of
the backupPVC at mount time, which can take hours on volumes with a high
file count. Kubernetes avoids this when the volume is ReadWriteOncePod and
the CSI driver advertises SELinux mount support, by mounting with
-o context= instead.
Add an opt-in per-storage-class 'readWriteOncePod' backupPVC config option
that creates the backupPVC with the ReadWriteOncePod access mode and sets
the backup pod's SecurityContext.SELinuxChangePolicy to MountOption. It is
mutually exclusive with 'readOnly', which takes precedence.
Fixes#9873
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
* Add changelog file
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
* Do not set SELinuxChangePolicy for the backup pod
Live testing on OCP 4.22 (k8s 1.35) showed that setting
SecurityContext.SELinuxChangePolicy to MountOption makes backup pod
creation fail outright when the SELinuxMount feature gate is disabled,
which is the default on current clusters:
Pod is invalid: spec.securityContext.seLinuxChangePolicy:
Unsupported value: "MountOption": supported values: "Recursive"
The field is also unnecessary. For ReadWriteOncePod volumes the kubelet
already performs mount-level SELinux labeling via the
SELinuxMountReadWriteOncePod feature gate, which has been on by default
since k8s 1.28. Setting the backupPVC access mode to ReadWriteOncePod is
sufficient on its own, and is portable to clusters where the broader
SELinuxMount gate is still off.
Verified on-cluster that the backupPVC is mounted with
context="system_u:object_r:container_file_t:s0:c22,c28" instead of the
recursive seclabel mount used without the flag.
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
---------
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
config/crd/{v1,v2alpha1}/crds/crds.go were generated files that
gzip-compressed the CRD YAML bases into committed []byte literals via
hack/crd-gen, requiring `go generate` and a dedicated CI drift check
(hack/verify-generated-crd-code.sh). This made the files large,
unreviewable in diffs, and a frequent source of merge conflicts.
Replace the generated files with config/crd/{v1,v2alpha1}/crds.go
using `//go:embed bases/*.yaml` to embed the already-committed YAML
manifests directly, decoding them the same way at init. Since Go's
go:embed can't reach outside a file's own directory tree, the crds
package now lives alongside bases/ instead of in a bases-sibling
subdirectory; import paths in pkg/install and pkg/controller were
updated accordingly.
Drop hack/crd-gen and hack/verify-generated-crd-code.sh entirely, and
trim their references from update-3generated-crd-code.sh and the
codespell skip-list. No codegen step remains, so no drift is possible.
Fixes#10328
AI-Tool-Used: Claude Code
AI-Tool-Use-Level: Category 1 (High)
AI-Code-Category: Category 1 (Production)
Signed-off-by: lubronzhan <lubron.zhan@broadcom.com>
Co-authored-by: Daniel Jiang <daniel.jiang@broadcom.com>
When CreateSnapshot fails (e.g. quota limit), the snapshot is recorded
with an empty ProviderSnapshotID. During backup deletion, velero was
calling DeleteSnapshot("") which produces unnecessary 404 API calls.
Skip the DeleteSnapshot call when ProviderSnapshotID is empty and log
a warning instead.
Fixes#9429
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Happy <yesreply@happy.engineering>
Lyndon-Li
Chlins ZhangandGitHub
Wenkai Yin(尹文开)andGitHub
Adam ZhangandGitHub
Xun Jiang/Bruce JiangandGitHub
opbot_xd
R4mboandGitHub
Krishna AwasthiandGitHub
PranjalandGitHub
Shubham PampattiwarandGitHub
