* add documentation for backup filters (#9967)
* add documentation for backup filters
Add user guide for fine grained backup filters with examples
from easy to advanced.
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
* address review comments
- enhanced example 3, explain how each item got excluded
- enhanced example 8, explain the exact match rule, and how the
ordering affecting namespace that has multiple match patterns
- cross link to restore side design
- fix the error msg to be consistent with implemenation
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
---------
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
* update backup filters example 14
update the excludeNames to match example 3 for better consistency.
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
* Support set-based filter label selectors (#10064)
* Support set-based filter label selectors
Use matchLabels/matchExpressions in fine-grained filters.
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
* omit the details of resource policy for cli
The reason to not resolve and display CLI is because it may go
out of sync, we want to avoid display it to mislead users.
We may consider to cpature those information and display it
in later release.
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
* update change log
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
---------
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
* Prioritize exact namespace match in restore
Align restore pipeline with backup pipeline by
evaluating exact namespace matches before glob
patterns in namespacedFilterPolicies. This ensures
specific overrides always win regardless of list
order.
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
* improve test cases
add test cases for exact listed first, and excat listed last to
ensure the behavior that the order does not matter for exact
listed namespace, the rule will be always honored.
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
---------
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
Added CLI for creating restore resource policies, and view
the resource policies associated with resource if present.
Only list the name of the configmap for now.
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
* restore filter enhancement
enhance restore filter with resource policies, introduce resource
policies with fine-grained control for resources in restore, both
cluster scoped resources and namespace scoped resources, with
labels, names include/exclude support with glob patterns.
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
* address review comments
- Add introductory sentence linking to the Phase 1 backup filters PR.
- Add clarification that a backup's ConfigMap may not exist on the
target cluster because it might be on a different Velero instance.
- Remove redundant explanations about backup-specific concepts
(volume policies, include/exclude policies).
- Remove the non-goal regarding restore-side `includeExcludePolicy`.
- Remove the "Interaction with Backup-Side Filters" section.
- Remove "Step 5" from the design, as additional items requested by
plugins should intentionally bypass fine-grained filter checks
(consistent with backup side Stage 2).
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
* address more review comments
- remove confusion rows regarding per-namespace kind list
- simplified CLI output to configmap name only
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
---------
Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
* Fix PodVolumeBackup metadata loss on fs-backup timeout
When a backup hits the fs-backup timeout, WaitAllPodVolumesProcessed
returned nil because PVBs were only collected from the indexer in the
done branch of the select. This discarded all PVB metadata including
already-completed PVBs, making their data unrestorable.
Move the PVB collection loop to run after the select so tracked PVBs
are always persisted regardless of timeout.
Fixes#9986
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
* Add changelog for PR #9995
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
* Filter non-completed PVBs in hasPodVolumeBackup
After preserving tracked PVBs on timeout, non-completed PVBs (in-progress
or with no snapshot ID) would cause hasPodVolumeBackup to return true,
leading the restore to skip the original PV and dynamically re-provision
it without any data to restore from.
Only match PVBs that are Completed with a valid SnapshotID.
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
* Add unit tests for hasPodVolumeBackup phase filtering
Verify that hasPodVolumeBackup only matches PVBs that are Completed
with a valid SnapshotID, and rejects in-progress, failed, or
empty-snapshot PVBs.
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
---------
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
(cherry picked from commit e593ba73f9)
Document that Velero 1.18.1+ requires external-snapshotter v8.2.0 or
later for VolumeGroupSnapshot support, since Velero upgraded from
v1beta1 to v1beta2 APIs.
Relates to #9882
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
When getNamespacesToList returns "" (cross-namespace listing), resources
from all namespaces are listed. The nsTracker.track call for each
resource's namespace would inadvertently track excluded namespaces,
causing their namespace objects to pass through filterNamespaces.
Add a ShouldInclude check before tracking a namespace from non-namespace
resources to ensure excluded namespaces are not tracked.
Add test case for includedNamespaces ["*"] with excludedNamespaces to
verify both the namespace object exclusion and resource exclusion.
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
Commit 8ac8f49b5 ("Remove wildcard check from getNamespacesToList")
removed the optimization that prevented "*" from being expanded to
individual namespace names. This caused getNamespacesToList to return
all namespace names instead of "" (cross-namespace listing), resulting
in N separate API list calls per resource type instead of 1.
On clusters with many namespaces (e.g. 178 on an ACM cluster), this
means ~35,000 API calls instead of ~200, causing backups to take
18-20 minutes for just 8 items.
Restore the "*" special case in ShouldExpandWildcards so that plain
"*" is not expanded, and restore the ShouldInclude("*") check in
getNamespacesToList so that cross-namespace listing is used.
The restore fix from 8ac8f49b5 (fromBackup flag) is preserved since
restores already return false before reaching the "*" check.
Namespace exclusion continues to work correctly: the nsTracker filters
excluded namespace objects via ShouldInclude, and backupItem filters
namespace-scoped resources at line 124 of item_backupper.go.
Fixes#9869
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
Guard the cleanupStubVGSC() call in restore finalization with a check
for VolumeGroupSnapshotHandle in volumeInfo. This avoids a spurious
warning on clusters where the v1beta2 VolumeGroupSnapshotContent CRD
is not installed, since the List call would fail even though no stubs
exist to clean up.
Fixes#9882
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
Adam ZhangandGitHub
lyndon-liandGitHub
Xun Jiang/Bruce JiangandAdam Zhang
Chlins ZhangandGitHub
Daniel Jiangandchlins
Shubham PampattiwarandGitHub
Xun JiangandXun Jiang/Bruce Jiang
Lyndon-Li