mirror of
https://github.com/vmware-tanzu/velero.git
synced 2026-08-03 13:56:07 +00:00
[Cherry-pick]Set based label selectors for 1.18 (#10072)
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
e2e-test-kind.yaml / extract (push) Successful in 12s
Run the E2E test on kind / get-go-version (push) Successful in 14s
push.yml / extract (push) Successful in 13s
Main CI / get-go-version (push) Successful in 14s
Run the E2E test on kind / build (push) Failing after 28s
Run the E2E test on kind / run-e2e-test (push) Skipped
Main CI / Build (push) Failing after 30s
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
e2e-test-kind.yaml / extract (push) Successful in 12s
Run the E2E test on kind / get-go-version (push) Successful in 14s
push.yml / extract (push) Successful in 13s
Main CI / get-go-version (push) Successful in 14s
Run the E2E test on kind / build (push) Failing after 28s
Run the E2E test on kind / run-e2e-test (push) Skipped
Main CI / Build (push) Failing after 30s
* add documentation for backup filters (#9967) * add documentation for backup filters Add user guide for fine grained backup filters with examples from easy to advanced. Signed-off-by: Adam Zhang <adam.zhang@broadcom.com> * address review comments - enhanced example 3, explain how each item got excluded - enhanced example 8, explain the exact match rule, and how the ordering affecting namespace that has multiple match patterns - cross link to restore side design - fix the error msg to be consistent with implemenation Signed-off-by: Adam Zhang <adam.zhang@broadcom.com> --------- Signed-off-by: Adam Zhang <adam.zhang@broadcom.com> * update backup filters example 14 update the excludeNames to match example 3 for better consistency. Signed-off-by: Adam Zhang <adam.zhang@broadcom.com> * Support set-based filter label selectors (#10064) * Support set-based filter label selectors Use matchLabels/matchExpressions in fine-grained filters. Signed-off-by: Adam Zhang <adam.zhang@broadcom.com> * omit the details of resource policy for cli The reason to not resolve and display CLI is because it may go out of sync, we want to avoid display it to mislead users. We may consider to cpature those information and display it in later release. Signed-off-by: Adam Zhang <adam.zhang@broadcom.com> * update change log Signed-off-by: Adam Zhang <adam.zhang@broadcom.com> --------- Signed-off-by: Adam Zhang <adam.zhang@broadcom.com>
This commit is contained in:
Adam Zhang
@@ -0,0 +1 @@
|
||||
User guide for backup fine-grained filters via resource policy, and add set based label selectors for fine-grained filters
|
||||
@@ -41,7 +41,7 @@ This creates three critical gaps for common backup scenarios:
|
||||
- Maintain full backward compatibility — existing backups with no `namespacedFilterPolicies` behave exactly as they do today
|
||||
- Define clear precedence rules for how per-namespace filters interact with global filters
|
||||
- Add corresponding validation within the Resource Policies validation pipeline using existing Velero wildcard validation functions
|
||||
- Update `velero backup describe` output to display per-namespace filter information when present
|
||||
- Update `velero backup describe` output to display the referenced ResourcePolicy ConfigMap name when configured
|
||||
- Ensure the restore process works correctly with backups produced by namespace-scoped filters, without requiring restore-side code changes in the initial phase
|
||||
|
||||
## Non-Goals
|
||||
@@ -77,7 +77,8 @@ clusterScopedFilterPolicy:
|
||||
names: ["my-app-*"]
|
||||
- kinds: [CustomResourceDefinition]
|
||||
labelSelector:
|
||||
app: my-app
|
||||
matchLabels:
|
||||
app: my-app
|
||||
namespacedFilterPolicies:
|
||||
# NEW: per-namespace filter overrides
|
||||
- namespaces:
|
||||
@@ -85,7 +86,8 @@ namespacedFilterPolicies:
|
||||
resourceFilters:
|
||||
- kinds: [ConfigMap, Secret, Deployment]
|
||||
labelSelector:
|
||||
app: my-app
|
||||
matchLabels:
|
||||
app: my-app
|
||||
- namespaces:
|
||||
- ns-b
|
||||
resourceFilters:
|
||||
@@ -93,7 +95,8 @@ namespacedFilterPolicies:
|
||||
names: [app-1, app-2]
|
||||
- kinds: [ConfigMap]
|
||||
labelSelector:
|
||||
app: my-service
|
||||
matchLabels:
|
||||
app: my-service
|
||||
```
|
||||
|
||||
All four sections coexist in the same ConfigMap. They are independent — `volumePolicies` handles volume backup strategy, `includeExcludePolicy` handles global resource type filtering, `clusterScopedFilterPolicy` handles cluster-scoped resource filtering by kind/name/label, and `namespacedFilterPolicies` handles per-namespace, per-kind overrides.
|
||||
@@ -107,7 +110,9 @@ namespacedFilterPolicies:
|
||||
- namespaces: [ns-a]
|
||||
resourceFilters:
|
||||
- kinds: [ConfigMap, Secret] # these kinds share a selector
|
||||
labelSelector: {app: my-app}
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
app: my-app
|
||||
names: ["app-*"]
|
||||
- kinds: [Deployment] # this kind has its own selector
|
||||
names: [workload-1, workload-2]
|
||||
@@ -116,6 +121,24 @@ namespacedFilterPolicies:
|
||||
|
||||
This model has one way to express filters — there is no ambiguity about how to structure the configuration. Only resource kinds listed in `resourceFilters` entries are included in the backup for the matched namespaces; unlisted kinds are implicitly excluded.
|
||||
|
||||
#### Label selectors (`matchLabels` / `matchExpressions`)
|
||||
|
||||
`labelSelector` and each entry of `orLabelSelectors` use the standard Kubernetes selector shape (same as `BackupSpec.labelSelector`):
|
||||
|
||||
```yaml
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
app: my-app
|
||||
matchExpressions:
|
||||
- key: environment
|
||||
operator: In
|
||||
values: [prod, staging]
|
||||
- key: do-not-backup
|
||||
operator: DoesNotExist
|
||||
```
|
||||
|
||||
Supported `matchExpressions` operators: `In`, `NotIn`, `Exists`, `DoesNotExist`. Prefer `In` for value-OR on one key; use `orLabelSelectors` for OR across independent multi-key groups. `labelSelector` and `orLabelSelectors` cannot co-exist in the same `resourceFilters` entry.
|
||||
|
||||
#### Catch-All Resource Filter (Empty `kinds` or `["*"]`)
|
||||
|
||||
A `ResourceFilter` entry with an empty (or omitted) `kinds` field, or a field explicitly set to `["*"]`, acts as a **catch-all**. Its `labelSelector` or `orLabelSelectors` (if provided) is applied to **all resource types in the namespace that are not already matched by a kind-specific filter entry**. If no selectors are provided, all unlisted resources are included. Using `["*"]` is highly recommended as it makes the catch-all intention explicit and self-documenting.
|
||||
@@ -319,9 +342,10 @@ resourceFilters:
|
||||
resourceFilters:
|
||||
- kinds: ["Pod"]
|
||||
labelSelector:
|
||||
"invalid label key!": "value" # invalid key syntax
|
||||
matchLabels:
|
||||
"invalid label key!": "value" # invalid key syntax
|
||||
```
|
||||
**Behavior:** Validation error during backup creation when `labels.SelectorFromSet()` fails:
|
||||
**Behavior:** Validation error during backup creation when `metav1.LabelSelectorAsSelector()` fails:
|
||||
```
|
||||
namespacedFilterPolicies[0].resourceFilters[0]: invalid label selector: "invalid label key!" is not a valid label key
|
||||
```
|
||||
@@ -340,7 +364,33 @@ This is consistent with how other discovery-dependent features handle this error
|
||||
|
||||
## ResourceFilter Field Notes
|
||||
|
||||
**`labelSelector`** supports equality-based selectors only (`key=value`). Set-based requirements (e.g., `environment in (prod, staging)`) are not supported. To match resources with any of several label combinations, use `orLabelSelectors` with multiple maps — each map is AND-evaluated internally, and the maps are OR-evaluated across the list. `labelSelector` and `orLabelSelectors` cannot co-exist in the same entry.
|
||||
**`labelSelector`** uses the standard Kubernetes shape: `matchLabels` (equality) and `matchExpressions` (set-based: `In`, `NotIn`, `Exists`, `DoesNotExist`). All requirements within one selector are AND-ed. Example:
|
||||
|
||||
```yaml
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
app: my-app
|
||||
matchExpressions:
|
||||
- key: environment
|
||||
operator: In
|
||||
values: [prod, staging]
|
||||
- key: do-not-backup
|
||||
operator: DoesNotExist
|
||||
```
|
||||
|
||||
**`orLabelSelectors`** is a list of the same selector shape. Match if **any** entry matches (AND within each entry, OR across the list). Prefer `In` for value-OR on one key; use `orLabelSelectors` for OR of independent multi-key groups. `labelSelector` and `orLabelSelectors` cannot co-exist in the same entry.
|
||||
|
||||
```yaml
|
||||
orLabelSelectors:
|
||||
- matchLabels:
|
||||
tier: frontend
|
||||
matchExpressions:
|
||||
- key: track
|
||||
operator: In
|
||||
values: [canary]
|
||||
- matchLabels:
|
||||
tier: backend
|
||||
```
|
||||
|
||||
**`names` / `excludedNames`** accept exact resource names or glob patterns. If `names` is empty, all resource names are included (subject to label filters). `excludedNames` takes precedence over `names` when a name matches both.
|
||||
|
||||
@@ -420,7 +470,8 @@ data:
|
||||
resourceFilters:
|
||||
- kinds: [ConfigMap, Secret, Deployment]
|
||||
labelSelector:
|
||||
app: my-app
|
||||
matchLabels:
|
||||
app: my-app
|
||||
# ns-b has no filter policy entry, so global filters apply (include everything)
|
||||
```
|
||||
|
||||
@@ -462,10 +513,12 @@ data:
|
||||
resourceFilters:
|
||||
- kinds: [Deployment]
|
||||
labelSelector:
|
||||
app: production-workload-1
|
||||
matchLabels:
|
||||
app: production-workload-1
|
||||
- kinds: [StatefulSet]
|
||||
labelSelector:
|
||||
app: production-workload-2
|
||||
matchLabels:
|
||||
app: production-workload-2
|
||||
```
|
||||
|
||||
### Per-Kind Exact Names
|
||||
@@ -561,7 +614,8 @@ data:
|
||||
resourceFilters:
|
||||
- kinds: ["*"] # catch-all: applies to every kind not listed below
|
||||
labelSelector:
|
||||
backup: "true" # back up any resource carrying this label
|
||||
matchLabels:
|
||||
backup: "true" # back up any resource carrying this label
|
||||
```
|
||||
|
||||
**Result:** Every resource type in `production` that has the label `backup=true` is backed up. Resources without that label are excluded. No kind enumeration is required.
|
||||
@@ -589,7 +643,8 @@ data:
|
||||
names: [db-credentials, tls-cert] # these exact Secrets by name
|
||||
- kinds: ["*"] # catch-all for all other kinds
|
||||
labelSelector:
|
||||
backup: "true" # back up by label
|
||||
matchLabels:
|
||||
backup: "true" # back up by label
|
||||
```
|
||||
|
||||
**Result:**
|
||||
@@ -666,7 +721,8 @@ data:
|
||||
names: [workload-1, workload-2]
|
||||
- kinds: [StatefulSet]
|
||||
labelSelector:
|
||||
app: my-app
|
||||
matchLabels:
|
||||
app: my-app
|
||||
- kinds: [ConfigMap, Secret]
|
||||
names: ["app-*"]
|
||||
excludedNames: ["*-tmp", "*-debug"]
|
||||
@@ -697,7 +753,7 @@ spec:
|
||||
|
||||
### `velero backup describe`
|
||||
|
||||
The output is extended to display namespace-scoped filter policies when present in the ResourcePolicy ConfigMap:
|
||||
The output displays the referenced ResourcePolicy ConfigMap name when configured on the backup. It intentionally avoids resolving and displaying the live ConfigMap contents, because the ConfigMap content in the cluster may be modified or deleted after the backup execution, which could lead to displaying out-of-sync or inaccurate information:
|
||||
|
||||
```
|
||||
Name: selective-backup
|
||||
@@ -721,46 +777,9 @@ Resources:
|
||||
|
||||
Label selector: <none>
|
||||
|
||||
Resource Policy: backup-filter-policy
|
||||
|
||||
Namespace-Scoped Filter Policies:
|
||||
ns-a:
|
||||
Resource Filters:
|
||||
ConfigMap, Secret, Deployment:
|
||||
Label selector: app=my-app
|
||||
Included names: <none>
|
||||
Excluded names: <none>
|
||||
target-namespace:
|
||||
Resource Filters:
|
||||
Deployment:
|
||||
Label selector: app=production-workload-1
|
||||
Included names: <none>
|
||||
Excluded names: <none>
|
||||
StatefulSet:
|
||||
Label selector: app=production-workload-2
|
||||
Included names: <none>
|
||||
Excluded names: <none>
|
||||
production:
|
||||
Resource Filters:
|
||||
Deployment:
|
||||
Label selector: <none>
|
||||
Included names: [api-server, worker]
|
||||
Excluded names: <none>
|
||||
<catch-all> (all other kinds):
|
||||
Label selector: backup=true
|
||||
Included names: <none>
|
||||
Excluded names: <none>
|
||||
|
||||
Fine-Grained Global Filter Policy:
|
||||
Resource Filters:
|
||||
ClusterRole, ClusterRoleBinding:
|
||||
Label selector: <none>
|
||||
Included names: [my-app-*]
|
||||
Excluded names: <none>
|
||||
CustomResourceDefinition:
|
||||
Label selector: app=my-app
|
||||
Included names: <none>
|
||||
Excluded names: <none>
|
||||
Resource policies:
|
||||
Type: configmap
|
||||
Name: backup-filter-policy
|
||||
|
||||
Storage Location: default
|
||||
|
||||
@@ -795,7 +814,7 @@ Notes:
|
||||
- Global filters (--include-resources, --selector, etc.) apply to all included namespaces
|
||||
- Namespace-scoped filters defined in --resource-policies-configmap override global filters for matching namespaces
|
||||
- Fine-grained global filter policies defined in --resource-policies-configmap override global filters for cluster-scoped resources
|
||||
- Use 'velero backup describe' to view resolved filter policies after backup creation
|
||||
- Use 'velero backup describe' to view the referenced ResourcePolicy ConfigMap name after backup creation
|
||||
```
|
||||
|
||||
### CLI Integration Points
|
||||
@@ -808,12 +827,12 @@ Notes:
|
||||
|
||||
**Help and Discovery:**
|
||||
- `velero backup create --help` includes updated filtering documentation
|
||||
- `velero backup describe` shows resolved filter policies for troubleshooting
|
||||
- `velero backup describe` shows the referenced ResourcePolicy ConfigMap name
|
||||
- Validation errors include ConfigMap field references for easy debugging
|
||||
|
||||
**Configuration Discovery:**
|
||||
- `velero backup create --help` includes namespace-scoped filtering documentation
|
||||
- `velero backup describe` shows resolved filter policies for verification
|
||||
- `velero backup describe` shows the referenced ResourcePolicy ConfigMap name for verification
|
||||
|
||||
## User Perspective
|
||||
|
||||
@@ -823,7 +842,7 @@ This design provides fine-grained, per-namespace, per-kind control over backup f
|
||||
- **For users adopting namespace-scoped filter policies**: Create a ConfigMap with the `namespacedFilterPolicies` section and reference it via `BackupSpec.ResourcePolicy` (or the existing `--resource-policies-configmap` flag). The backup will selectively include/exclude resources per namespace based on the filter rules.
|
||||
- **For users already using ResourcePolicy for volume policies**: Add the `namespacedFilterPolicies` section to the same ConfigMap. Both volume policies and namespace-scoped filters coexist.
|
||||
- **For restore from a namespace-filtered backup**: No changes to restore workflow. Restore processes whatever is in the archive. Users can use existing `RestoreSpec.IncludedNamespaces` for additional filtering at restore time.
|
||||
- **`velero backup describe` output**: Extended to show per-namespace, per-kind filter details when the ResourcePolicy ConfigMap contains `namespacedFilterPolicies`.
|
||||
- **`velero backup describe` output**: Displays the referenced ResourcePolicy ConfigMap name when configured on the backup.
|
||||
- **Validation errors**: Reported at backup start when the ResourcePolicy ConfigMap contains invalid `namespacedFilterPolicies` configurations. Consistent with how volume policy validation errors are reported today.
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
@@ -108,14 +108,16 @@ clusterScopedFilterPolicy:
|
||||
names: ["my-app-*"]
|
||||
- kinds: [CustomResourceDefinition]
|
||||
labelSelector:
|
||||
app: my-app
|
||||
matchLabels:
|
||||
app: my-app
|
||||
namespacedFilterPolicies:
|
||||
- namespaces:
|
||||
- ns-a
|
||||
resourceFilters:
|
||||
- kinds: [ConfigMap, Secret, Deployment]
|
||||
labelSelector:
|
||||
app: my-app
|
||||
matchLabels:
|
||||
app: my-app
|
||||
- namespaces:
|
||||
- ns-b
|
||||
resourceFilters:
|
||||
@@ -123,7 +125,8 @@ namespacedFilterPolicies:
|
||||
names: [app-1, app-2]
|
||||
- kinds: [ConfigMap]
|
||||
labelSelector:
|
||||
app: my-service
|
||||
matchLabels:
|
||||
app: my-service
|
||||
```
|
||||
|
||||
The restore-side ConfigMap does **not** require `volumePolicies` or `includeExcludePolicy` sections. Those are backup-specific. The YAML parser will ignore unknown fields gracefully, so a user can technically point to the same ConfigMap used for backup — the restore pipeline will only read `namespacedFilterPolicies` and `clusterScopedFilterPolicy`.
|
||||
@@ -137,7 +140,9 @@ namespacedFilterPolicies:
|
||||
- namespaces: [ns-a]
|
||||
resourceFilters:
|
||||
- kinds: [ConfigMap, Secret] # these kinds share a selector
|
||||
labelSelector: {app: my-app}
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
app: my-app
|
||||
names: ["app-*"]
|
||||
- kinds: [Deployment] # this kind has its own selector
|
||||
names: [workload-1, workload-2]
|
||||
@@ -146,6 +151,36 @@ namespacedFilterPolicies:
|
||||
|
||||
Only resource kinds listed in `resourceFilters` entries are restored for the matched namespaces; unlisted kinds are implicitly excluded (globally excluded kinds cannot be re-included — see precedence model).
|
||||
|
||||
#### Label selectors (`matchLabels` / `matchExpressions`)
|
||||
|
||||
`labelSelector` and each entry of `orLabelSelectors` use the standard Kubernetes selector shape (same as `RestoreSpec.labelSelector`):
|
||||
|
||||
```yaml
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
app: my-app
|
||||
matchExpressions:
|
||||
- key: environment
|
||||
operator: In
|
||||
values: [prod, staging]
|
||||
- key: do-not-restore
|
||||
operator: DoesNotExist
|
||||
```
|
||||
|
||||
Supported `matchExpressions` operators: `In`, `NotIn`, `Exists`, `DoesNotExist`. Prefer `In` for value-OR on one key; use `orLabelSelectors` for OR across independent multi-key groups. `labelSelector` and `orLabelSelectors` cannot co-exist in the same `resourceFilters` entry.
|
||||
|
||||
```yaml
|
||||
orLabelSelectors:
|
||||
- matchLabels:
|
||||
tier: frontend
|
||||
matchExpressions:
|
||||
- key: track
|
||||
operator: In
|
||||
values: [canary]
|
||||
- matchLabels:
|
||||
tier: backend
|
||||
```
|
||||
|
||||
#### Peek-and-Map Fallback for Unresolved Kinds
|
||||
|
||||
The `kinds` field accepts both plural resource names (e.g., `configmaps`, `mycustomkinds.mygroup.io`) and singular `Kind` names (e.g., `ConfigMap`, `MyCustomKind`).
|
||||
@@ -382,9 +417,10 @@ resourceFilters:
|
||||
resourceFilters:
|
||||
- kinds: ["Deployment"]
|
||||
labelSelector:
|
||||
"invalid label key!": "value" # invalid key syntax
|
||||
matchLabels:
|
||||
"invalid label key!": "value" # invalid key syntax
|
||||
```
|
||||
**Behavior:** Validation error during restore creation when `labels.ValidatedSelectorFromSet()` fails:
|
||||
**Behavior:** Validation error during restore creation when `metav1.LabelSelectorAsSelector()` fails:
|
||||
```
|
||||
namespacedFilterPolicies[0].resourceFilters[0]: invalid label selector: "invalid label key!" is not a valid label key
|
||||
```
|
||||
@@ -420,7 +456,8 @@ namespacedFilterPolicies:
|
||||
resourceFilters:
|
||||
- kinds: [ConfigMap, Secret] # Secret listed here is ineffective — globally excluded
|
||||
labelSelector:
|
||||
app: my-app
|
||||
matchLabels:
|
||||
app: my-app
|
||||
- kinds: [Deployment]
|
||||
```
|
||||
|
||||
@@ -461,8 +498,8 @@ After existing filter setup, the filter policies are resolved into the runtime m
|
||||
The `resolveRestoreNamespacedFilterPolicies` function:
|
||||
- For each `NamespacedFilterPolicy`, iterates its `ResourceFilters` entries
|
||||
- Resolves kind names to fully-qualified group-resource strings using the discovery helper
|
||||
- Converts `labelSelector` maps into `labels.Selector` objects using `labels.ValidatedSelectorFromSet()`
|
||||
- Converts `orLabelSelectors` maps into `[]labels.Selector`
|
||||
- Converts `labelSelector` into a `labels.Selector` via `ToMetaV1LabelSelector` + `metav1.LabelSelectorAsSelector()`
|
||||
- Converts `orLabelSelectors` into `[]labels.Selector` the same way
|
||||
- Creates `IncludesExcludes` instances for `names`/`excludedNames` patterns
|
||||
- Identifies catch-all entries (empty or `["*"]` kinds) and stores them in `catchAllFilter`
|
||||
- Builds a `resourceFilterMap` keyed by the resolved group-resource string
|
||||
@@ -537,7 +574,8 @@ data:
|
||||
resourceFilters:
|
||||
- kinds: [Deployment, ConfigMap]
|
||||
labelSelector:
|
||||
app: my-app
|
||||
matchLabels:
|
||||
app: my-app
|
||||
# ns-b has no filter policy entry, so global filters apply (restore everything)
|
||||
```
|
||||
|
||||
@@ -631,7 +669,8 @@ data:
|
||||
names: [db-credentials, tls-cert] # these exact Secrets by name
|
||||
- kinds: ["*"] # catch-all for all other kinds
|
||||
labelSelector:
|
||||
backup: "true" # restore by label
|
||||
matchLabels:
|
||||
backup: "true" # restore by label
|
||||
```
|
||||
|
||||
**Result:**
|
||||
@@ -658,7 +697,8 @@ data:
|
||||
names: ["my-app-*"]
|
||||
- kinds: [CustomResourceDefinition]
|
||||
labelSelector:
|
||||
app: my-app
|
||||
matchLabels:
|
||||
app: my-app
|
||||
namespacedFilterPolicies:
|
||||
- namespaces:
|
||||
- production
|
||||
|
||||
@@ -21,12 +21,13 @@ import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"k8s.io/apimachinery/pkg/util/sets"
|
||||
|
||||
"github.com/cockroachdb/errors"
|
||||
"github.com/gobwas/glob"
|
||||
"github.com/sirupsen/logrus"
|
||||
corev1api "k8s.io/api/core/v1"
|
||||
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||
"k8s.io/apimachinery/pkg/labels"
|
||||
"k8s.io/apimachinery/pkg/util/sets"
|
||||
crclient "sigs.k8s.io/controller-runtime/pkg/client"
|
||||
|
||||
velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
|
||||
@@ -56,13 +57,66 @@ type Action struct {
|
||||
Parameters map[string]any `yaml:"parameters,omitempty"`
|
||||
}
|
||||
|
||||
// PolicyLabelSelector mirrors metav1.LabelSelector with yaml tags for ConfigMap decode.
|
||||
// metav1.LabelSelector only has json tags, which do not populate under go.yaml.in/yaml/v3.
|
||||
type PolicyLabelSelector struct {
|
||||
MatchLabels map[string]string `yaml:"matchLabels,omitempty"`
|
||||
MatchExpressions []PolicyLabelSelectorRequirement `yaml:"matchExpressions,omitempty"`
|
||||
}
|
||||
|
||||
// PolicyLabelSelectorRequirement mirrors metav1.LabelSelectorRequirement with yaml tags.
|
||||
type PolicyLabelSelectorRequirement struct {
|
||||
Key string `yaml:"key"`
|
||||
Operator string `yaml:"operator"`
|
||||
Values []string `yaml:"values,omitempty"`
|
||||
}
|
||||
|
||||
// IsPresentLabelSelector reports whether s defines any label constraints.
|
||||
// Empty {} (nil MatchLabels and empty MatchExpressions) is treated as absent.
|
||||
func IsPresentLabelSelector(s *PolicyLabelSelector) bool {
|
||||
return s != nil && (len(s.MatchLabels) > 0 || len(s.MatchExpressions) > 0)
|
||||
}
|
||||
|
||||
// ToMetaV1LabelSelector converts the YAML mirror type to metav1.LabelSelector.
|
||||
// Conversion itself is infallible; call LabelSelectorAsSelector (or
|
||||
// SelectorFromPolicyLabelSelector) to validate operators and values.
|
||||
func ToMetaV1LabelSelector(s *PolicyLabelSelector) *metav1.LabelSelector {
|
||||
if s == nil {
|
||||
return nil
|
||||
}
|
||||
ls := &metav1.LabelSelector{MatchLabels: s.MatchLabels}
|
||||
for _, expr := range s.MatchExpressions {
|
||||
ls.MatchExpressions = append(ls.MatchExpressions, metav1.LabelSelectorRequirement{
|
||||
Key: expr.Key,
|
||||
Operator: metav1.LabelSelectorOperator(expr.Operator),
|
||||
Values: expr.Values,
|
||||
})
|
||||
}
|
||||
return ls
|
||||
}
|
||||
|
||||
// SelectorFromPolicyLabelSelector converts a present policy label selector to a
|
||||
// runtime labels.Selector. Returns (nil, nil) when s defines no constraints.
|
||||
func SelectorFromPolicyLabelSelector(s *PolicyLabelSelector) (labels.Selector, error) {
|
||||
if !IsPresentLabelSelector(s) {
|
||||
return nil, nil
|
||||
}
|
||||
return metav1.LabelSelectorAsSelector(ToMetaV1LabelSelector(s))
|
||||
}
|
||||
|
||||
// validatePolicyLabelSelector converts and validates a policy label selector.
|
||||
func validatePolicyLabelSelector(s *PolicyLabelSelector) error {
|
||||
_, err := SelectorFromPolicyLabelSelector(s)
|
||||
return err
|
||||
}
|
||||
|
||||
// ResourceFilter defines a filter for specific resource kinds.
|
||||
type ResourceFilter struct {
|
||||
Kinds []string `yaml:"kinds"`
|
||||
LabelSelector map[string]string `yaml:"labelSelector,omitempty"`
|
||||
OrLabelSelectors []map[string]string `yaml:"orLabelSelectors,omitempty"`
|
||||
Names []string `yaml:"names,omitempty"`
|
||||
ExcludedNames []string `yaml:"excludedNames,omitempty"`
|
||||
Kinds []string `yaml:"kinds"`
|
||||
LabelSelector *PolicyLabelSelector `yaml:"labelSelector,omitempty"`
|
||||
OrLabelSelectors []*PolicyLabelSelector `yaml:"orLabelSelectors,omitempty"`
|
||||
Names []string `yaml:"names,omitempty"`
|
||||
ExcludedNames []string `yaml:"excludedNames,omitempty"`
|
||||
}
|
||||
|
||||
// IsCatchAll returns true if the filter is a catch-all entry (empty kinds or ["*"])
|
||||
@@ -561,9 +615,17 @@ func (p *Policies) validateNamespacedFilterPolicies() error {
|
||||
seenKinds[kind] = j
|
||||
}
|
||||
|
||||
if len(rf.LabelSelector) > 0 && len(rf.OrLabelSelectors) > 0 {
|
||||
if IsPresentLabelSelector(rf.LabelSelector) && len(rf.OrLabelSelectors) > 0 {
|
||||
return fmt.Errorf("namespacedFilterPolicies[%d].resourceFilters[%d]: labelSelector and orLabelSelectors cannot co-exist", i, j)
|
||||
}
|
||||
if err := validatePolicyLabelSelector(rf.LabelSelector); err != nil {
|
||||
return fmt.Errorf("namespacedFilterPolicies[%d].resourceFilters[%d]: invalid label selector: %w", i, j, err)
|
||||
}
|
||||
for k, ols := range rf.OrLabelSelectors {
|
||||
if err := validatePolicyLabelSelector(ols); err != nil {
|
||||
return fmt.Errorf("namespacedFilterPolicies[%d].resourceFilters[%d].orLabelSelectors[%d]: invalid label selector: %w", i, j, k, err)
|
||||
}
|
||||
}
|
||||
|
||||
// Validate glob patterns for names and excludedNames using gobwas/glob
|
||||
for k, pattern := range rf.Names {
|
||||
@@ -613,9 +675,17 @@ func (p *Policies) validateClusterScopedFilterPolicy() error {
|
||||
seenKinds[kind] = j
|
||||
}
|
||||
|
||||
if len(rf.LabelSelector) > 0 && len(rf.OrLabelSelectors) > 0 {
|
||||
if IsPresentLabelSelector(rf.LabelSelector) && len(rf.OrLabelSelectors) > 0 {
|
||||
return fmt.Errorf("clusterScopedFilterPolicy.resourceFilters[%d]: labelSelector and orLabelSelectors cannot co-exist", j)
|
||||
}
|
||||
if err := validatePolicyLabelSelector(rf.LabelSelector); err != nil {
|
||||
return fmt.Errorf("clusterScopedFilterPolicy.resourceFilters[%d]: invalid label selector: %w", j, err)
|
||||
}
|
||||
for k, ols := range rf.OrLabelSelectors {
|
||||
if err := validatePolicyLabelSelector(ols); err != nil {
|
||||
return fmt.Errorf("clusterScopedFilterPolicy.resourceFilters[%d].orLabelSelectors[%d]: invalid label selector: %w", j, k, err)
|
||||
}
|
||||
}
|
||||
|
||||
for k, pattern := range rf.Names {
|
||||
if _, err := glob.Compile(pattern); err != nil {
|
||||
|
||||
@@ -25,6 +25,7 @@ import (
|
||||
corev1api "k8s.io/api/core/v1"
|
||||
"k8s.io/apimachinery/pkg/api/resource"
|
||||
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||
"k8s.io/apimachinery/pkg/labels"
|
||||
"k8s.io/client-go/kubernetes/scheme"
|
||||
"sigs.k8s.io/controller-runtime/pkg/client/fake"
|
||||
|
||||
@@ -2027,7 +2028,8 @@ namespacedFilterPolicies:
|
||||
resourceFilters:
|
||||
- kinds: ["Pod", "ConfigMap"]
|
||||
labelSelector:
|
||||
app: web
|
||||
matchLabels:
|
||||
app: web
|
||||
names: ["app-*"]
|
||||
- kinds: ["Secret"]
|
||||
excludedNames: ["temp-*"]`,
|
||||
@@ -2041,8 +2043,10 @@ namespacedFilterPolicies:
|
||||
resourceFilters:
|
||||
- kinds: ["Pod"]
|
||||
orLabelSelectors:
|
||||
- env: prod
|
||||
- env: staging`,
|
||||
- matchLabels:
|
||||
env: prod
|
||||
- matchLabels:
|
||||
env: staging`,
|
||||
wantErr: false,
|
||||
},
|
||||
{
|
||||
@@ -2084,7 +2088,8 @@ namespacedFilterPolicies:
|
||||
resourceFilters:
|
||||
- kinds: ["*"]
|
||||
labelSelector:
|
||||
app: web`,
|
||||
matchLabels:
|
||||
app: web`,
|
||||
wantErr: false,
|
||||
},
|
||||
{
|
||||
@@ -2095,10 +2100,12 @@ namespacedFilterPolicies:
|
||||
resourceFilters:
|
||||
- kinds: ["*"]
|
||||
labelSelector:
|
||||
app: web
|
||||
matchLabels:
|
||||
app: web
|
||||
- kinds: ["*"]
|
||||
labelSelector:
|
||||
app: db`,
|
||||
matchLabels:
|
||||
app: db`,
|
||||
wantErr: true,
|
||||
errMsg: "only one catch-all resource filter is allowed",
|
||||
},
|
||||
@@ -2110,10 +2117,12 @@ namespacedFilterPolicies:
|
||||
resourceFilters:
|
||||
- kinds: []
|
||||
labelSelector:
|
||||
app: web
|
||||
matchLabels:
|
||||
app: web
|
||||
- kinds: ["*"]
|
||||
labelSelector:
|
||||
app: db`,
|
||||
matchLabels:
|
||||
app: db`,
|
||||
wantErr: true,
|
||||
errMsg: "only one catch-all resource filter is allowed",
|
||||
},
|
||||
@@ -2125,10 +2134,12 @@ namespacedFilterPolicies:
|
||||
resourceFilters:
|
||||
- kinds: []
|
||||
labelSelector:
|
||||
app: web
|
||||
matchLabels:
|
||||
app: web
|
||||
- kinds: []
|
||||
labelSelector:
|
||||
app: db`,
|
||||
matchLabels:
|
||||
app: db`,
|
||||
wantErr: true,
|
||||
errMsg: "only one catch-all resource filter is allowed",
|
||||
},
|
||||
@@ -2141,7 +2152,8 @@ namespacedFilterPolicies:
|
||||
- kinds: []
|
||||
names: ["app-*"]
|
||||
labelSelector:
|
||||
app: web`,
|
||||
matchLabels:
|
||||
app: web`,
|
||||
wantErr: true,
|
||||
errMsg: "names or excludedNames cannot be specified for catch-all filters",
|
||||
},
|
||||
@@ -2154,7 +2166,8 @@ namespacedFilterPolicies:
|
||||
- kinds: []
|
||||
excludedNames: ["app-*"]
|
||||
labelSelector:
|
||||
app: web`,
|
||||
matchLabels:
|
||||
app: web`,
|
||||
wantErr: true,
|
||||
errMsg: "names or excludedNames cannot be specified for catch-all filters",
|
||||
},
|
||||
@@ -2186,9 +2199,11 @@ namespacedFilterPolicies:
|
||||
resourceFilters:
|
||||
- kinds: ["Pod"]
|
||||
labelSelector:
|
||||
app: web
|
||||
matchLabels:
|
||||
app: web
|
||||
orLabelSelectors:
|
||||
- env: prod`,
|
||||
- matchLabels:
|
||||
env: prod`,
|
||||
wantErr: true,
|
||||
errMsg: "labelSelector and orLabelSelectors cannot co-exist",
|
||||
},
|
||||
@@ -2272,7 +2287,8 @@ namespacedFilterPolicies:
|
||||
resourceFilters:
|
||||
- kinds: ["Pod"]
|
||||
labelSelector:
|
||||
app: web`
|
||||
matchLabels:
|
||||
app: web`
|
||||
|
||||
resPolicies, err := unmarshalResourcePolicies(&yamlData)
|
||||
require.NoError(t, err)
|
||||
@@ -2290,7 +2306,135 @@ namespacedFilterPolicies:
|
||||
|
||||
rf := policy.ResourceFilters[0]
|
||||
assert.Equal(t, []string{"Pod"}, rf.Kinds)
|
||||
assert.Equal(t, map[string]string{"app": "web"}, rf.LabelSelector)
|
||||
assert.Equal(t, &PolicyLabelSelector{MatchLabels: map[string]string{"app": "web"}}, rf.LabelSelector)
|
||||
}
|
||||
|
||||
func TestPolicyLabelSelectorSetBased(t *testing.T) {
|
||||
t.Run("yaml decode matchLabels and matchExpressions", func(t *testing.T) {
|
||||
yamlData := `version: v1
|
||||
namespacedFilterPolicies:
|
||||
- namespaces: ["ns1"]
|
||||
resourceFilters:
|
||||
- kinds: ["Pod"]
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
app: web
|
||||
matchExpressions:
|
||||
- key: environment
|
||||
operator: In
|
||||
values: [prod, staging]
|
||||
- key: do-not-backup
|
||||
operator: DoesNotExist`
|
||||
|
||||
resPolicies, err := unmarshalResourcePolicies(&yamlData)
|
||||
require.NoError(t, err)
|
||||
|
||||
policies := &Policies{}
|
||||
require.NoError(t, policies.BuildPolicy(resPolicies))
|
||||
require.NoError(t, policies.Validate())
|
||||
|
||||
rf := policies.GetNamespacedFilterPolicies()[0].ResourceFilters[0]
|
||||
require.NotNil(t, rf.LabelSelector)
|
||||
assert.Equal(t, map[string]string{"app": "web"}, rf.LabelSelector.MatchLabels)
|
||||
require.Len(t, rf.LabelSelector.MatchExpressions, 2)
|
||||
assert.Equal(t, "environment", rf.LabelSelector.MatchExpressions[0].Key)
|
||||
assert.Equal(t, "In", rf.LabelSelector.MatchExpressions[0].Operator)
|
||||
assert.Equal(t, []string{"prod", "staging"}, rf.LabelSelector.MatchExpressions[0].Values)
|
||||
assert.Equal(t, "do-not-backup", rf.LabelSelector.MatchExpressions[1].Key)
|
||||
assert.Equal(t, "DoesNotExist", rf.LabelSelector.MatchExpressions[1].Operator)
|
||||
})
|
||||
|
||||
t.Run("empty labelSelector is no filter", func(t *testing.T) {
|
||||
yamlData := `version: v1
|
||||
namespacedFilterPolicies:
|
||||
- namespaces: ["ns1"]
|
||||
resourceFilters:
|
||||
- kinds: ["Pod"]
|
||||
labelSelector: {}`
|
||||
|
||||
resPolicies, err := unmarshalResourcePolicies(&yamlData)
|
||||
require.NoError(t, err)
|
||||
|
||||
policies := &Policies{}
|
||||
require.NoError(t, policies.BuildPolicy(resPolicies))
|
||||
require.NoError(t, policies.Validate())
|
||||
|
||||
rf := policies.GetNamespacedFilterPolicies()[0].ResourceFilters[0]
|
||||
assert.False(t, IsPresentLabelSelector(rf.LabelSelector))
|
||||
})
|
||||
|
||||
t.Run("invalid operator rejected", func(t *testing.T) {
|
||||
yamlData := `version: v1
|
||||
namespacedFilterPolicies:
|
||||
- namespaces: ["ns1"]
|
||||
resourceFilters:
|
||||
- kinds: ["Pod"]
|
||||
labelSelector:
|
||||
matchExpressions:
|
||||
- key: environment
|
||||
operator: Equals
|
||||
values: [prod]`
|
||||
|
||||
resPolicies, err := unmarshalResourcePolicies(&yamlData)
|
||||
require.NoError(t, err)
|
||||
|
||||
policies := &Policies{}
|
||||
require.NoError(t, policies.BuildPolicy(resPolicies))
|
||||
err = policies.Validate()
|
||||
require.Error(t, err)
|
||||
assert.Contains(t, err.Error(), "invalid label selector")
|
||||
})
|
||||
|
||||
t.Run("NotIn Exists operators validate", func(t *testing.T) {
|
||||
yamlData := `version: v1
|
||||
clusterScopedFilterPolicy:
|
||||
resourceFilters:
|
||||
- kinds: ["ClusterRole"]
|
||||
labelSelector:
|
||||
matchExpressions:
|
||||
- key: tier
|
||||
operator: NotIn
|
||||
values: [debug]
|
||||
- key: managed-by
|
||||
operator: Exists`
|
||||
|
||||
resPolicies, err := unmarshalResourcePolicies(&yamlData)
|
||||
require.NoError(t, err)
|
||||
|
||||
policies := &Policies{}
|
||||
require.NoError(t, policies.BuildPolicy(resPolicies))
|
||||
require.NoError(t, policies.Validate())
|
||||
})
|
||||
|
||||
t.Run("ToMetaV1LabelSelector and IsPresentLabelSelector", func(t *testing.T) {
|
||||
assert.False(t, IsPresentLabelSelector(nil))
|
||||
assert.False(t, IsPresentLabelSelector(&PolicyLabelSelector{}))
|
||||
assert.True(t, IsPresentLabelSelector(&PolicyLabelSelector{MatchLabels: map[string]string{"a": "b"}}))
|
||||
|
||||
ls := ToMetaV1LabelSelector(&PolicyLabelSelector{
|
||||
MatchLabels: map[string]string{"app": "web"},
|
||||
MatchExpressions: []PolicyLabelSelectorRequirement{
|
||||
{Key: "env", Operator: "In", Values: []string{"prod"}},
|
||||
},
|
||||
})
|
||||
require.NotNil(t, ls)
|
||||
assert.Equal(t, map[string]string{"app": "web"}, ls.MatchLabels)
|
||||
require.Len(t, ls.MatchExpressions, 1)
|
||||
assert.Equal(t, metav1.LabelSelectorOpIn, ls.MatchExpressions[0].Operator)
|
||||
|
||||
assert.Nil(t, ToMetaV1LabelSelector(nil))
|
||||
|
||||
sel, err := SelectorFromPolicyLabelSelector(&PolicyLabelSelector{
|
||||
MatchLabels: map[string]string{"app": "web"},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
require.NotNil(t, sel)
|
||||
assert.True(t, sel.Matches(labels.Set{"app": "web"}))
|
||||
|
||||
emptySel, err := SelectorFromPolicyLabelSelector(&PolicyLabelSelector{})
|
||||
require.NoError(t, err)
|
||||
assert.Nil(t, emptySel)
|
||||
})
|
||||
}
|
||||
|
||||
func TestClusterScopedFilterPoliciesAccessor(t *testing.T) {
|
||||
@@ -2394,7 +2538,8 @@ clusterScopedFilterPolicy:
|
||||
resourceFilters:
|
||||
- kinds: ["ClusterRole", "ClusterRoleBinding"]
|
||||
labelSelector:
|
||||
app: my-app`,
|
||||
matchLabels:
|
||||
app: my-app`,
|
||||
wantErr: false,
|
||||
},
|
||||
{
|
||||
@@ -2404,8 +2549,10 @@ clusterScopedFilterPolicy:
|
||||
resourceFilters:
|
||||
- kinds: ["CustomResourceDefinition"]
|
||||
orLabelSelectors:
|
||||
- app: my-app
|
||||
- app: other-app`,
|
||||
- matchLabels:
|
||||
app: my-app
|
||||
- matchLabels:
|
||||
app: other-app`,
|
||||
wantErr: false,
|
||||
},
|
||||
{
|
||||
@@ -2443,7 +2590,8 @@ clusterScopedFilterPolicy:
|
||||
resourceFilters:
|
||||
- kinds: ["*"]
|
||||
labelSelector:
|
||||
app: my-app`,
|
||||
matchLabels:
|
||||
app: my-app`,
|
||||
wantErr: true,
|
||||
errMsg: "kinds must be specified",
|
||||
},
|
||||
@@ -2456,7 +2604,8 @@ clusterScopedFilterPolicy:
|
||||
names: ["my-app-*"]
|
||||
- kinds: ["ClusterRole"]
|
||||
labelSelector:
|
||||
app: other`,
|
||||
matchLabels:
|
||||
app: other`,
|
||||
wantErr: true,
|
||||
errMsg: `kind "ClusterRole" appears in both`,
|
||||
},
|
||||
@@ -2467,9 +2616,11 @@ clusterScopedFilterPolicy:
|
||||
resourceFilters:
|
||||
- kinds: ["ClusterRole"]
|
||||
labelSelector:
|
||||
app: my-app
|
||||
matchLabels:
|
||||
app: my-app
|
||||
orLabelSelectors:
|
||||
- app: other`,
|
||||
- matchLabels:
|
||||
app: other`,
|
||||
wantErr: true,
|
||||
errMsg: "labelSelector and orLabelSelectors cannot co-exist",
|
||||
},
|
||||
|
||||
@@ -1409,22 +1409,20 @@ func resolveClusterScopedFilterPolicy(
|
||||
}
|
||||
|
||||
func resolveResourceFilter(rf resourcepolicies.ResourceFilter) (*ResolvedResourceFilter, error) {
|
||||
var selector labels.Selector
|
||||
if len(rf.LabelSelector) > 0 {
|
||||
var err error
|
||||
selector, err = labels.ValidatedSelectorFromSet(labels.Set(rf.LabelSelector))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid label selector in resource filter: %w", err)
|
||||
}
|
||||
selector, err := resourcepolicies.SelectorFromPolicyLabelSelector(rf.LabelSelector)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid label selector in resource filter: %w", err)
|
||||
}
|
||||
|
||||
var orSelectors []labels.Selector
|
||||
for _, ols := range rf.OrLabelSelectors {
|
||||
s, err := labels.ValidatedSelectorFromSet(labels.Set(ols))
|
||||
s, err := resourcepolicies.SelectorFromPolicyLabelSelector(ols)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid OR label selector in resource filter: %w", err)
|
||||
}
|
||||
orSelectors = append(orSelectors, s)
|
||||
if s != nil {
|
||||
orSelectors = append(orSelectors, s)
|
||||
}
|
||||
}
|
||||
|
||||
var nameIE *collections.IncludesExcludes
|
||||
|
||||
+77
-15
@@ -5749,7 +5749,7 @@ func TestResolveResourceFilter(t *testing.T) {
|
||||
{
|
||||
name: "valid label selector",
|
||||
rf: resourcepolicies.ResourceFilter{
|
||||
LabelSelector: map[string]string{"app": "foo"},
|
||||
LabelSelector: &resourcepolicies.PolicyLabelSelector{MatchLabels: map[string]string{"app": "foo"}},
|
||||
},
|
||||
expectErr: false,
|
||||
checkResult: func(t *testing.T, r *ResolvedResourceFilter) {
|
||||
@@ -5762,16 +5762,16 @@ func TestResolveResourceFilter(t *testing.T) {
|
||||
{
|
||||
name: "invalid label selector",
|
||||
rf: resourcepolicies.ResourceFilter{
|
||||
LabelSelector: map[string]string{"invalid/label/key": "value"},
|
||||
LabelSelector: &resourcepolicies.PolicyLabelSelector{MatchLabels: map[string]string{"invalid/label/key": "value"}},
|
||||
},
|
||||
expectErr: true,
|
||||
},
|
||||
{
|
||||
name: "valid or label selectors",
|
||||
rf: resourcepolicies.ResourceFilter{
|
||||
OrLabelSelectors: []map[string]string{
|
||||
{"app": "foo"},
|
||||
{"app": "bar"},
|
||||
OrLabelSelectors: []*resourcepolicies.PolicyLabelSelector{
|
||||
{MatchLabels: map[string]string{"app": "foo"}},
|
||||
{MatchLabels: map[string]string{"app": "bar"}},
|
||||
},
|
||||
},
|
||||
expectErr: false,
|
||||
@@ -5784,8 +5784,8 @@ func TestResolveResourceFilter(t *testing.T) {
|
||||
{
|
||||
name: "invalid or label selectors",
|
||||
rf: resourcepolicies.ResourceFilter{
|
||||
OrLabelSelectors: []map[string]string{
|
||||
{"invalid/label/key": "value"},
|
||||
OrLabelSelectors: []*resourcepolicies.PolicyLabelSelector{
|
||||
{MatchLabels: map[string]string{"invalid/label/key": "value"}},
|
||||
},
|
||||
},
|
||||
expectErr: true,
|
||||
@@ -5805,6 +5805,68 @@ func TestResolveResourceFilter(t *testing.T) {
|
||||
assert.False(t, r.NameIE.ShouldInclude("exc1"))
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "empty labelSelector is no filter",
|
||||
rf: resourcepolicies.ResourceFilter{
|
||||
LabelSelector: &resourcepolicies.PolicyLabelSelector{},
|
||||
},
|
||||
expectErr: false,
|
||||
checkResult: func(t *testing.T, r *ResolvedResourceFilter) {
|
||||
t.Helper()
|
||||
require.NotNil(t, r)
|
||||
assert.Nil(t, r.LabelSelector)
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "set-based In and DoesNotExist",
|
||||
rf: resourcepolicies.ResourceFilter{
|
||||
LabelSelector: &resourcepolicies.PolicyLabelSelector{
|
||||
MatchExpressions: []resourcepolicies.PolicyLabelSelectorRequirement{
|
||||
{Key: "environment", Operator: "In", Values: []string{"prod", "staging"}},
|
||||
{Key: "do-not-backup", Operator: "DoesNotExist"},
|
||||
},
|
||||
},
|
||||
},
|
||||
expectErr: false,
|
||||
checkResult: func(t *testing.T, r *ResolvedResourceFilter) {
|
||||
t.Helper()
|
||||
require.NotNil(t, r.LabelSelector)
|
||||
assert.True(t, r.LabelSelector.Matches(labels.Set{"environment": "prod"}))
|
||||
assert.True(t, r.LabelSelector.Matches(labels.Set{"environment": "staging"}))
|
||||
assert.False(t, r.LabelSelector.Matches(labels.Set{"environment": "dev"}))
|
||||
assert.False(t, r.LabelSelector.Matches(labels.Set{"environment": "prod", "do-not-backup": "true"}))
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "set-based NotIn and Exists",
|
||||
rf: resourcepolicies.ResourceFilter{
|
||||
LabelSelector: &resourcepolicies.PolicyLabelSelector{
|
||||
MatchExpressions: []resourcepolicies.PolicyLabelSelectorRequirement{
|
||||
{Key: "tier", Operator: "NotIn", Values: []string{"debug"}},
|
||||
{Key: "app", Operator: "Exists"},
|
||||
},
|
||||
},
|
||||
},
|
||||
expectErr: false,
|
||||
checkResult: func(t *testing.T, r *ResolvedResourceFilter) {
|
||||
t.Helper()
|
||||
require.NotNil(t, r.LabelSelector)
|
||||
assert.True(t, r.LabelSelector.Matches(labels.Set{"app": "web", "tier": "frontend"}))
|
||||
assert.False(t, r.LabelSelector.Matches(labels.Set{"app": "web", "tier": "debug"}))
|
||||
assert.False(t, r.LabelSelector.Matches(labels.Set{"tier": "frontend"}))
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "invalid operator",
|
||||
rf: resourcepolicies.ResourceFilter{
|
||||
LabelSelector: &resourcepolicies.PolicyLabelSelector{
|
||||
MatchExpressions: []resourcepolicies.PolicyLabelSelectorRequirement{
|
||||
{Key: "env", Operator: "Equals", Values: []string{"prod"}},
|
||||
},
|
||||
},
|
||||
},
|
||||
expectErr: true,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
@@ -5842,11 +5904,11 @@ func TestResolveClusterScopedFilterPolicy(t *testing.T) {
|
||||
ResourceFilters: []resourcepolicies.ResourceFilter{
|
||||
{
|
||||
Kinds: []string{"pods", "secrets"},
|
||||
LabelSelector: map[string]string{"app": "foo"},
|
||||
LabelSelector: &resourcepolicies.PolicyLabelSelector{MatchLabels: map[string]string{"app": "foo"}},
|
||||
},
|
||||
{
|
||||
Kinds: []string{"invalid-kind"},
|
||||
LabelSelector: map[string]string{"invalid/label/key": "value"},
|
||||
LabelSelector: &resourcepolicies.PolicyLabelSelector{MatchLabels: map[string]string{"invalid/label/key": "value"}},
|
||||
},
|
||||
},
|
||||
}
|
||||
@@ -5860,7 +5922,7 @@ func TestResolveClusterScopedFilterPolicy(t *testing.T) {
|
||||
ResourceFilters: []resourcepolicies.ResourceFilter{
|
||||
{
|
||||
Kinds: []string{"pods", "secrets"},
|
||||
LabelSelector: map[string]string{"app": "foo"},
|
||||
LabelSelector: &resourcepolicies.PolicyLabelSelector{MatchLabels: map[string]string{"app": "foo"}},
|
||||
},
|
||||
},
|
||||
}
|
||||
@@ -5908,11 +5970,11 @@ func TestResolveNamespacedFilterPolicies(t *testing.T) {
|
||||
ResourceFilters: []resourcepolicies.ResourceFilter{
|
||||
{
|
||||
Kinds: []string{"pods"},
|
||||
LabelSelector: map[string]string{"app": "foo"},
|
||||
LabelSelector: &resourcepolicies.PolicyLabelSelector{MatchLabels: map[string]string{"app": "foo"}},
|
||||
},
|
||||
{
|
||||
Kinds: []string{"*"},
|
||||
LabelSelector: map[string]string{"catch": "all"},
|
||||
LabelSelector: &resourcepolicies.PolicyLabelSelector{MatchLabels: map[string]string{"catch": "all"}},
|
||||
},
|
||||
},
|
||||
},
|
||||
@@ -5940,7 +6002,7 @@ func TestResolveNamespacedFilterPolicies(t *testing.T) {
|
||||
ResourceFilters: []resourcepolicies.ResourceFilter{
|
||||
{
|
||||
Kinds: []string{"pods"},
|
||||
LabelSelector: map[string]string{"invalid/label/key": "value"},
|
||||
LabelSelector: &resourcepolicies.PolicyLabelSelector{MatchLabels: map[string]string{"invalid/label/key": "value"}},
|
||||
},
|
||||
},
|
||||
},
|
||||
@@ -6024,7 +6086,7 @@ func TestBackupWithResPoliciesLogs(t *testing.T) {
|
||||
ResourceFilters: []resourcepolicies.ResourceFilter{
|
||||
{
|
||||
Kinds: []string{"pods"},
|
||||
LabelSelector: map[string]string{"invalid/label/key": "value"},
|
||||
LabelSelector: &resourcepolicies.PolicyLabelSelector{MatchLabels: map[string]string{"invalid/label/key": "value"}},
|
||||
},
|
||||
},
|
||||
}
|
||||
@@ -6043,7 +6105,7 @@ func TestBackupWithResPoliciesLogs(t *testing.T) {
|
||||
ResourceFilters: []resourcepolicies.ResourceFilter{
|
||||
{
|
||||
Kinds: []string{"pods"},
|
||||
LabelSelector: map[string]string{"invalid/label/key": "value"},
|
||||
LabelSelector: &resourcepolicies.PolicyLabelSelector{MatchLabels: map[string]string{"invalid/label/key": "value"}},
|
||||
},
|
||||
},
|
||||
},
|
||||
|
||||
@@ -21,7 +21,6 @@ import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
@@ -31,7 +30,6 @@ import (
|
||||
|
||||
"github.com/cockroachdb/errors"
|
||||
snapshotv1api "github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1"
|
||||
"github.com/sirupsen/logrus"
|
||||
|
||||
"github.com/fatih/color"
|
||||
kbclient "sigs.k8s.io/controller-runtime/pkg/client"
|
||||
@@ -94,9 +92,6 @@ func DescribeBackup(
|
||||
if backup.Spec.ResourcePolicy != nil {
|
||||
d.Println()
|
||||
DescribeResourcePolicies(d, backup.Spec.ResourcePolicy)
|
||||
|
||||
// Display fine-grained filter policies if they exist
|
||||
DescribeFineGrainedFilterPolicies(ctx, kbClient, d, backup)
|
||||
}
|
||||
|
||||
DescribeGlobalVolumePolicy(d, backup)
|
||||
@@ -151,119 +146,6 @@ func DescribeGlobalVolumePolicy(d *Describer, backup *velerov1api.Backup) {
|
||||
d.Printf("\tName:\t%s\n", name)
|
||||
}
|
||||
|
||||
// DescribeFineGrainedFilterPolicies describes cluster-scoped and namespace-scoped filter policies if present
|
||||
func DescribeFineGrainedFilterPolicies(ctx context.Context, kbClient kbclient.Client, d *Describer, backup *velerov1api.Backup) {
|
||||
if backup.Spec.ResourcePolicy == nil {
|
||||
return
|
||||
}
|
||||
|
||||
// Create a discard logger for the resource policies function since this is CLI output context
|
||||
discardLogger := logrus.New()
|
||||
discardLogger.Out = io.Discard
|
||||
|
||||
resourcePolicies, err := resourcepolicies.GetResourcePoliciesFromBackup(*backup, kbClient, discardLogger)
|
||||
if err != nil {
|
||||
// Don't fail the describe if we can't read policies, just skip
|
||||
return
|
||||
}
|
||||
|
||||
if resourcePolicies == nil {
|
||||
return
|
||||
}
|
||||
|
||||
clusterScopedFilterPolicy := resourcePolicies.GetClusterScopedFilterPolicy()
|
||||
if clusterScopedFilterPolicy != nil {
|
||||
d.Printf("\nCluster Scoped Filter Policy:\n")
|
||||
d.Printf(" Resource Filters:\n")
|
||||
for _, rf := range clusterScopedFilterPolicy.ResourceFilters {
|
||||
kindsStr := strings.Join(rf.Kinds, ", ")
|
||||
d.Printf(" %s:\n", kindsStr)
|
||||
|
||||
// Label selector
|
||||
if len(rf.LabelSelector) > 0 {
|
||||
selectorStr := formatLabelMap(rf.LabelSelector)
|
||||
d.Printf(" Label selector: %s\n", selectorStr)
|
||||
} else if len(rf.OrLabelSelectors) > 0 {
|
||||
var orStrs []string
|
||||
for _, ols := range rf.OrLabelSelectors {
|
||||
orStrs = append(orStrs, formatLabelMap(ols))
|
||||
}
|
||||
d.Printf(" OR label selectors: [%s]\n", strings.Join(orStrs, ", "))
|
||||
} else {
|
||||
d.Printf(" Label selector: <none>\n")
|
||||
}
|
||||
|
||||
// Name patterns
|
||||
if len(rf.Names) > 0 {
|
||||
d.Printf(" Included names: [%s]\n", strings.Join(rf.Names, ", "))
|
||||
} else {
|
||||
d.Printf(" Included names: <none>\n")
|
||||
}
|
||||
|
||||
if len(rf.ExcludedNames) > 0 {
|
||||
d.Printf(" Excluded names: [%s]\n", strings.Join(rf.ExcludedNames, ", "))
|
||||
} else {
|
||||
d.Printf(" Excluded names: <none>\n")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
nfPolicies := resourcePolicies.GetNamespacedFilterPolicies()
|
||||
if len(nfPolicies) > 0 {
|
||||
d.Printf("\nNamespace-Scoped Filter Policies:\n")
|
||||
for _, policy := range nfPolicies {
|
||||
for _, ns := range policy.Namespaces {
|
||||
d.Printf(" %s:\n", ns)
|
||||
d.Printf(" Resource Filters:\n")
|
||||
for _, rf := range policy.ResourceFilters {
|
||||
var kindsStr string
|
||||
if rf.IsCatchAll() {
|
||||
kindsStr = "<catch-all> (all other kinds)"
|
||||
} else {
|
||||
kindsStr = strings.Join(rf.Kinds, ", ")
|
||||
}
|
||||
d.Printf(" %s:\n", kindsStr)
|
||||
|
||||
// Label selector
|
||||
if len(rf.LabelSelector) > 0 {
|
||||
selectorStr := formatLabelMap(rf.LabelSelector)
|
||||
d.Printf(" Label selector: %s\n", selectorStr)
|
||||
} else if len(rf.OrLabelSelectors) > 0 {
|
||||
var orStrs []string
|
||||
for _, ols := range rf.OrLabelSelectors {
|
||||
orStrs = append(orStrs, formatLabelMap(ols))
|
||||
}
|
||||
d.Printf(" OR label selectors: [%s]\n", strings.Join(orStrs, ", "))
|
||||
} else {
|
||||
d.Printf(" Label selector: <none>\n")
|
||||
}
|
||||
|
||||
// Name patterns
|
||||
if len(rf.Names) > 0 {
|
||||
d.Printf(" Included names: [%s]\n", strings.Join(rf.Names, ", "))
|
||||
} else {
|
||||
d.Printf(" Included names: <none>\n")
|
||||
}
|
||||
|
||||
if len(rf.ExcludedNames) > 0 {
|
||||
d.Printf(" Excluded names: [%s]\n", strings.Join(rf.ExcludedNames, ", "))
|
||||
} else {
|
||||
d.Printf(" Excluded names: <none>\n")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func formatLabelMap(labelMap map[string]string) string {
|
||||
var pairs []string
|
||||
for k, v := range labelMap {
|
||||
pairs = append(pairs, fmt.Sprintf("%s=%s", k, v))
|
||||
}
|
||||
return strings.Join(pairs, ",")
|
||||
}
|
||||
|
||||
// DescribeUploaderConfigForBackup describes uploader config in human-readable format
|
||||
func DescribeUploaderConfigForBackup(d *Describer, spec velerov1api.BackupSpec) {
|
||||
d.Printf("Uploader config:\n")
|
||||
|
||||
@@ -18,7 +18,6 @@ package output
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"testing"
|
||||
"text/tabwriter"
|
||||
"time"
|
||||
@@ -26,8 +25,6 @@ import (
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
corev1api "k8s.io/api/core/v1"
|
||||
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||
"sigs.k8s.io/controller-runtime/pkg/client/fake"
|
||||
|
||||
"github.com/vmware-tanzu/velero/internal/volume"
|
||||
velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
|
||||
@@ -897,85 +894,3 @@ func TestDescribeBackupItemOperation(t *testing.T) {
|
||||
d.out.Flush()
|
||||
assert.Equal(t, expected, d.buf.String())
|
||||
}
|
||||
|
||||
func TestDescribeFineGrainedFilterPolicies(t *testing.T) {
|
||||
yamlData := `
|
||||
version: v1
|
||||
clusterScopedFilterPolicy:
|
||||
resourceFilters:
|
||||
- kinds: ["StorageClass"]
|
||||
labelSelector: {"app": "velero"}
|
||||
- kinds: ["ClusterRole"]
|
||||
orLabelSelectors:
|
||||
- {"app": "velero"}
|
||||
- {"app": "test"}
|
||||
names: ["role1"]
|
||||
excludedNames: ["role2"]
|
||||
namespacedFilterPolicies:
|
||||
- namespaces: ["ns1", "ns2"]
|
||||
resourceFilters:
|
||||
- kinds: ["Pod", "ConfigMap"]
|
||||
labelSelector: {"app": "velero"}
|
||||
- kinds: ["*"]
|
||||
`
|
||||
cm := &corev1api.ConfigMap{
|
||||
ObjectMeta: metav1.ObjectMeta{
|
||||
Name: "test-policy",
|
||||
Namespace: "velero",
|
||||
},
|
||||
Data: map[string]string{
|
||||
"policy.yaml": yamlData,
|
||||
},
|
||||
}
|
||||
|
||||
client := fake.NewClientBuilder().WithRuntimeObjects(cm).Build()
|
||||
|
||||
backup := builder.ForBackup("velero", "test-backup").
|
||||
ResourcePolicies("test-policy").Result()
|
||||
|
||||
d := &Describer{
|
||||
Prefix: "",
|
||||
out: &tabwriter.Writer{},
|
||||
buf: &bytes.Buffer{},
|
||||
}
|
||||
d.out.Init(d.buf, 0, 8, 2, ' ', 0)
|
||||
|
||||
DescribeFineGrainedFilterPolicies(context.Background(), client, d, backup)
|
||||
d.out.Flush()
|
||||
|
||||
expected := `
|
||||
Cluster Scoped Filter Policy:
|
||||
Resource Filters:
|
||||
StorageClass:
|
||||
Label selector: app=velero
|
||||
Included names: <none>
|
||||
Excluded names: <none>
|
||||
ClusterRole:
|
||||
OR label selectors: [app=velero, app=test]
|
||||
Included names: [role1]
|
||||
Excluded names: [role2]
|
||||
|
||||
Namespace-Scoped Filter Policies:
|
||||
ns1:
|
||||
Resource Filters:
|
||||
Pod, ConfigMap:
|
||||
Label selector: app=velero
|
||||
Included names: <none>
|
||||
Excluded names: <none>
|
||||
<catch-all> (all other kinds):
|
||||
Label selector: <none>
|
||||
Included names: <none>
|
||||
Excluded names: <none>
|
||||
ns2:
|
||||
Resource Filters:
|
||||
Pod, ConfigMap:
|
||||
Label selector: app=velero
|
||||
Included names: <none>
|
||||
Excluded names: <none>
|
||||
<catch-all> (all other kinds):
|
||||
Label selector: <none>
|
||||
Included names: <none>
|
||||
Excluded names: <none>
|
||||
`
|
||||
assert.Equal(t, expected, d.buf.String())
|
||||
}
|
||||
|
||||
@@ -21,10 +21,8 @@ import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"strings"
|
||||
|
||||
"github.com/sirupsen/logrus"
|
||||
corev1api "k8s.io/api/core/v1"
|
||||
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||
|
||||
@@ -57,7 +55,6 @@ func DescribeBackupInSF(
|
||||
|
||||
if backup.Spec.ResourcePolicy != nil {
|
||||
DescribeResourcePoliciesInSF(d, backup.Spec.ResourcePolicy)
|
||||
DescribeFineGrainedFilterPoliciesInSF(ctx, kbClient, d, backup)
|
||||
}
|
||||
|
||||
DescribeGlobalVolumePolicyInSF(d, backup)
|
||||
@@ -228,88 +225,6 @@ func DescribeBackupSpecInSF(d *StructuredDescriber, spec velerov1api.BackupSpec)
|
||||
d.Describe("spec", backupSpecInfo)
|
||||
}
|
||||
|
||||
// DescribeFineGrainedFilterPoliciesInSF adds the clusterScopedFilterPolicy
|
||||
// and namespacedFilterPolicies sections to the structured describer output when present
|
||||
// in the ResourcePolicy ConfigMap referenced by the backup.
|
||||
func DescribeFineGrainedFilterPoliciesInSF(ctx context.Context, kbClient kbclient.Client, d *StructuredDescriber, backup *velerov1api.Backup) {
|
||||
if backup.Spec.ResourcePolicy == nil {
|
||||
return
|
||||
}
|
||||
|
||||
discardLogger := logrus.New()
|
||||
discardLogger.Out = io.Discard
|
||||
|
||||
resPolicies, err := resourcepolicies.GetResourcePoliciesFromBackup(*backup, kbClient, discardLogger)
|
||||
if err != nil || resPolicies == nil {
|
||||
return
|
||||
}
|
||||
|
||||
clusterScopedFilterPolicy := resPolicies.GetClusterScopedFilterPolicy()
|
||||
if clusterScopedFilterPolicy != nil {
|
||||
var clusterScopedFilters []map[string]any
|
||||
for _, rf := range clusterScopedFilterPolicy.ResourceFilters {
|
||||
entry := map[string]any{
|
||||
"kinds": rf.Kinds,
|
||||
}
|
||||
if len(rf.LabelSelector) > 0 {
|
||||
entry["labelSelector"] = rf.LabelSelector
|
||||
}
|
||||
if len(rf.OrLabelSelectors) > 0 {
|
||||
entry["orLabelSelectors"] = rf.OrLabelSelectors
|
||||
}
|
||||
if len(rf.Names) > 0 {
|
||||
entry["names"] = rf.Names
|
||||
}
|
||||
if len(rf.ExcludedNames) > 0 {
|
||||
entry["excludedNames"] = rf.ExcludedNames
|
||||
}
|
||||
clusterScopedFilters = append(clusterScopedFilters, entry)
|
||||
}
|
||||
d.Describe("clusterScopedFilterPolicy", map[string]any{
|
||||
"resourceFilters": clusterScopedFilters,
|
||||
})
|
||||
}
|
||||
|
||||
nfPolicies := resPolicies.GetNamespacedFilterPolicies()
|
||||
if len(nfPolicies) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
var structuredPolicies []map[string]any
|
||||
for _, policy := range nfPolicies {
|
||||
for _, ns := range policy.Namespaces {
|
||||
var rfEntries []map[string]any
|
||||
for _, rf := range policy.ResourceFilters {
|
||||
entry := map[string]any{}
|
||||
if rf.IsCatchAll() {
|
||||
entry["kinds"] = []string{}
|
||||
entry["isCatchAll"] = true
|
||||
} else {
|
||||
entry["kinds"] = rf.Kinds
|
||||
}
|
||||
if len(rf.LabelSelector) > 0 {
|
||||
entry["labelSelector"] = rf.LabelSelector
|
||||
}
|
||||
if len(rf.OrLabelSelectors) > 0 {
|
||||
entry["orLabelSelectors"] = rf.OrLabelSelectors
|
||||
}
|
||||
if len(rf.Names) > 0 {
|
||||
entry["names"] = rf.Names
|
||||
}
|
||||
if len(rf.ExcludedNames) > 0 {
|
||||
entry["excludedNames"] = rf.ExcludedNames
|
||||
}
|
||||
rfEntries = append(rfEntries, entry)
|
||||
}
|
||||
structuredPolicies = append(structuredPolicies, map[string]any{
|
||||
"namespace": ns,
|
||||
"resourceFilters": rfEntries,
|
||||
})
|
||||
}
|
||||
}
|
||||
d.Describe("namespacedFilterPolicies", structuredPolicies)
|
||||
}
|
||||
|
||||
// DescribeBackupStatusInSF describes a backup status in structured format.
|
||||
func DescribeBackupStatusInSF(ctx context.Context, kbClient kbclient.Client, d *StructuredDescriber, backup *velerov1api.Backup, details bool,
|
||||
insecureSkipTLSVerify bool, caCertPath string, podVolumeBackups []velerov1api.PodVolumeBackup) {
|
||||
|
||||
@@ -17,7 +17,6 @@ limitations under the License.
|
||||
package output
|
||||
|
||||
import (
|
||||
"context"
|
||||
"reflect"
|
||||
"testing"
|
||||
"time"
|
||||
@@ -25,8 +24,6 @@ import (
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
corev1api "k8s.io/api/core/v1"
|
||||
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||
"sigs.k8s.io/controller-runtime/pkg/client/fake"
|
||||
|
||||
"github.com/vmware-tanzu/velero/internal/volume"
|
||||
velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
|
||||
@@ -731,96 +728,3 @@ func TestDescribeDeleteBackupRequestsInSF(t *testing.T) {
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestDescribeFineGrainedFilterPoliciesInSF(t *testing.T) {
|
||||
yamlData := `
|
||||
version: v1
|
||||
clusterScopedFilterPolicy:
|
||||
resourceFilters:
|
||||
- kinds: ["StorageClass"]
|
||||
labelSelector: {"app": "velero"}
|
||||
- kinds: ["ClusterRole"]
|
||||
orLabelSelectors:
|
||||
- {"app": "velero"}
|
||||
- {"app": "test"}
|
||||
names: ["role1"]
|
||||
excludedNames: ["role2"]
|
||||
namespacedFilterPolicies:
|
||||
- namespaces: ["ns1", "ns2"]
|
||||
resourceFilters:
|
||||
- kinds: ["Pod", "ConfigMap"]
|
||||
labelSelector: {"app": "velero"}
|
||||
- kinds: ["*"]
|
||||
`
|
||||
cm := &corev1api.ConfigMap{
|
||||
ObjectMeta: metav1.ObjectMeta{
|
||||
Name: "test-policy",
|
||||
Namespace: "velero",
|
||||
},
|
||||
Data: map[string]string{
|
||||
"policy.yaml": yamlData,
|
||||
},
|
||||
}
|
||||
|
||||
client := fake.NewClientBuilder().WithRuntimeObjects(cm).Build()
|
||||
|
||||
backup := builder.ForBackup("velero", "test-backup").
|
||||
ResourcePolicies("test-policy").Result()
|
||||
|
||||
sd := &StructuredDescriber{
|
||||
output: make(map[string]any),
|
||||
format: "",
|
||||
}
|
||||
|
||||
DescribeFineGrainedFilterPoliciesInSF(context.Background(), client, sd, backup)
|
||||
|
||||
expect := map[string]any{
|
||||
"clusterScopedFilterPolicy": map[string]any{
|
||||
"resourceFilters": []map[string]any{
|
||||
{
|
||||
"kinds": []string{"StorageClass"},
|
||||
"labelSelector": map[string]string{"app": "velero"},
|
||||
},
|
||||
{
|
||||
"kinds": []string{"ClusterRole"},
|
||||
"orLabelSelectors": []map[string]string{
|
||||
{"app": "velero"},
|
||||
{"app": "test"},
|
||||
},
|
||||
"names": []string{"role1"},
|
||||
"excludedNames": []string{"role2"},
|
||||
},
|
||||
},
|
||||
},
|
||||
"namespacedFilterPolicies": []map[string]any{
|
||||
{
|
||||
"namespace": "ns1",
|
||||
"resourceFilters": []map[string]any{
|
||||
{
|
||||
"kinds": []string{"Pod", "ConfigMap"},
|
||||
"labelSelector": map[string]string{"app": "velero"},
|
||||
},
|
||||
{
|
||||
"kinds": []string{},
|
||||
"isCatchAll": true,
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
"namespace": "ns2",
|
||||
"resourceFilters": []map[string]any{
|
||||
{
|
||||
"kinds": []string{"Pod", "ConfigMap"},
|
||||
"labelSelector": map[string]string{"app": "velero"},
|
||||
},
|
||||
{
|
||||
"kinds": []string{},
|
||||
"isCatchAll": true,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
assert.True(t, reflect.DeepEqual(sd.output, expect))
|
||||
}
|
||||
|
||||
@@ -638,21 +638,19 @@ func resolveRestoreNamespacedFilterPolicies(
|
||||
func resolveResourceFilter(
|
||||
rf resourcepolicies.ResourceFilter,
|
||||
) (*resolvedResourceFilter, error) {
|
||||
var selector labels.Selector
|
||||
if len(rf.LabelSelector) > 0 {
|
||||
var err error
|
||||
selector, err = labels.ValidatedSelectorFromSet(labels.Set(rf.LabelSelector))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid label selector in resource filter: %w", err)
|
||||
}
|
||||
selector, err := resourcepolicies.SelectorFromPolicyLabelSelector(rf.LabelSelector)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid label selector in resource filter: %w", err)
|
||||
}
|
||||
var orSelectors []labels.Selector
|
||||
for _, ols := range rf.OrLabelSelectors {
|
||||
s, err := labels.ValidatedSelectorFromSet(labels.Set(ols))
|
||||
s, err := resourcepolicies.SelectorFromPolicyLabelSelector(ols)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid OR label selector in resource filter: %w", err)
|
||||
}
|
||||
orSelectors = append(orSelectors, s)
|
||||
if s != nil {
|
||||
orSelectors = append(orSelectors, s)
|
||||
}
|
||||
}
|
||||
var nameIE *collections.IncludesExcludes
|
||||
if len(rf.Names) > 0 || len(rf.ExcludedNames) > 0 {
|
||||
|
||||
@@ -170,7 +170,8 @@ namespacedFilterPolicies:
|
||||
- kinds:
|
||||
- '*'
|
||||
labelSelector:
|
||||
app: test
|
||||
matchLabels:
|
||||
app: test
|
||||
`,
|
||||
tarball: test.NewTarWriter(t).
|
||||
AddItems("pods",
|
||||
|
||||
@@ -0,0 +1,856 @@
|
||||
---
|
||||
title: "Fine-Grained Backup Filters"
|
||||
layout: docs
|
||||
---
|
||||
|
||||
This guide explains how to use Velero's **fine-grained backup filters**: per-namespace, per-kind rules with independent label selectors and resource name patterns. Configuration lives in the same **ResourcePolicy ConfigMap** you may already use for volume policies.
|
||||
|
||||
For architecture and pipeline details, see the [design document](https://github.com/velero-io/velero/blob/main/design/backup-filter-enhancement/fine-grained-backup-filters-design.md).
|
||||
|
||||
---
|
||||
|
||||
## Introduction
|
||||
|
||||
Velero's global backup filters apply the same namespace list, resource types, and label selector to every namespace in a backup. That works for many clusters, but common scenarios need more control:
|
||||
|
||||
- **Different namespaces, different strategies** — back up everything in a database namespace, but only Deployments and ConfigMaps in a frontend namespace.
|
||||
- **Filter by resource name** — back up `app-config` and `app-secret` without also capturing `monitoring-config`.
|
||||
- **Different labels per kind** — Deployments labeled `app=workload-1` and StatefulSets labeled `app=workload-2` in the same namespace.
|
||||
|
||||
Fine-grained filters add two optional sections to the ResourcePolicy ConfigMap:
|
||||
|
||||
| Section | Scope | Behavior |
|
||||
|---------|-------|----------|
|
||||
| `namespacedFilterPolicies` | Namespaces you match (exact name or glob) | **Exclusive allowlist** — only resource kinds listed in `resourceFilters` (or covered by a catch-all) are backed up from those namespaces |
|
||||
| `clusterScopedFilterPolicy` | Cluster-scoped resources globally | **Refinement overlay** — listed kinds get per-kind label and name rules; unlisted cluster-scoped kinds still use global BackupSpec filters |
|
||||
|
||||
**No new BackupSpec CRD fields** are required. Reference the policy from `Backup.spec.resourcePolicy` or `velero backup create --resource-policies-configmap`.
|
||||
|
||||
**Backward compatible:** if you omit both new sections, backups behave exactly as they do today.
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites and wiring
|
||||
|
||||
### What you need
|
||||
|
||||
- Velero installed with backup filters support (see your Velero release notes).
|
||||
- A ResourcePolicy ConfigMap in the Velero namespace (`velero` by default).
|
||||
- Permission to create Backups (or Schedules) that reference the ConfigMap.
|
||||
|
||||
### End-to-end pattern
|
||||
|
||||
Every example below follows the same three steps:
|
||||
|
||||
1. **Create or update** a ConfigMap with `data.policy` containing `version: v1` and your filter rules.
|
||||
2. **Create a Backup** (or Schedule) that includes the target namespaces and references the ConfigMap.
|
||||
3. **Verify** with `velero backup describe` and inspect backup contents or logs.
|
||||
|
||||
### Minimal skeleton
|
||||
|
||||
Use this once; later examples show only the `policy:` body.
|
||||
|
||||
**ResourcePolicy ConfigMap:**
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: my-backup-filter-policy
|
||||
namespace: velero
|
||||
data:
|
||||
policy: |
|
||||
version: v1
|
||||
namespacedFilterPolicies:
|
||||
- namespaces:
|
||||
- my-namespace
|
||||
resourceFilters:
|
||||
- kinds: [ConfigMap]
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
app: my-app
|
||||
```
|
||||
|
||||
**Backup:**
|
||||
|
||||
```yaml
|
||||
apiVersion: velero.io/v1
|
||||
kind: Backup
|
||||
metadata:
|
||||
name: my-backup
|
||||
namespace: velero
|
||||
spec:
|
||||
includedNamespaces:
|
||||
- my-namespace
|
||||
resourcePolicy:
|
||||
kind: configmap
|
||||
name: my-backup-filter-policy
|
||||
storageLocation: default
|
||||
```
|
||||
|
||||
**CLI equivalent:**
|
||||
|
||||
```bash
|
||||
velero backup create my-backup \
|
||||
--include-namespaces my-namespace \
|
||||
--resource-policies-configmap my-backup-filter-policy
|
||||
```
|
||||
|
||||
**Verify:**
|
||||
|
||||
```bash
|
||||
velero backup describe my-backup
|
||||
velero backup describe my-backup -o json | jq '.namespacedFilterPolicies'
|
||||
```
|
||||
|
||||
### Important: do not mix old-style BackupSpec resource filters
|
||||
|
||||
When `namespacedFilterPolicies` or `clusterScopedFilterPolicy` is present in the ResourcePolicy, **do not** set these on the Backup:
|
||||
|
||||
- `spec.includedResources` / `spec.excludedResources`
|
||||
- `spec.includeClusterResources`
|
||||
|
||||
Use `includeExcludePolicy` inside the ResourcePolicy ConfigMap for global resource-type include/exclude instead. Velero rejects backups that combine the new policy sections with old-style fields.
|
||||
|
||||
Schedules follow the same rule: configure filters in the ResourcePolicy ConfigMap, not via deprecated resource filter fields on the Schedule template.
|
||||
|
||||
---
|
||||
|
||||
## Examples
|
||||
|
||||
Each example includes: **goal**, **policy YAML**, **backup notes**, **expected outcome**, and **how to verify**.
|
||||
|
||||
---
|
||||
|
||||
### Example 0 — Baseline (no new filters)
|
||||
|
||||
**Goal:** Confirm that namespaces without a `namespacedFilterPolicies` entry still use global BackupSpec filters.
|
||||
|
||||
**Policy:** Omit `namespacedFilterPolicies` and `clusterScopedFilterPolicy` entirely (or use a ConfigMap with only `volumePolicies` / `includeExcludePolicy`).
|
||||
|
||||
**Backup:**
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
includedNamespaces:
|
||||
- ns-a
|
||||
- ns-b
|
||||
- production
|
||||
# No resourcePolicy — global filters only
|
||||
```
|
||||
|
||||
**Expected outcome:** All resources in included namespaces follow `includedNamespaces`, `labelSelector`, `includedResources`, and related global fields — same as before this feature.
|
||||
|
||||
**Verify:** `velero backup describe` shows no namespace-scoped filter policies section.
|
||||
|
||||
---
|
||||
|
||||
### Example 1 — Per-namespace kinds and labels
|
||||
|
||||
**Goal:** In `ns-a`, back up only ConfigMaps, Secrets, Deployments, and Pods with `app=my-app`. In `ns-b`, use global filters (no policy entry for that namespace).
|
||||
|
||||
**Policy:**
|
||||
|
||||
```yaml
|
||||
version: v1
|
||||
namespacedFilterPolicies:
|
||||
- namespaces:
|
||||
- ns-a
|
||||
resourceFilters:
|
||||
- kinds: [ConfigMap, Secret, Deployment, Pod]
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
app: my-app
|
||||
```
|
||||
|
||||
**Backup:**
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
includedNamespaces:
|
||||
- ns-a
|
||||
- ns-b
|
||||
resourcePolicy:
|
||||
kind: configmap
|
||||
name: per-namespace-resource-filter-policy # or your ConfigMap name
|
||||
```
|
||||
|
||||
**Expected outcome:**
|
||||
|
||||
- **ns-a:** Only listed kinds with label `app=my-app` (e.g. `app-config`, `app-secret`, `app-deployment`). Resources like `monitoring-config` (different labels) are excluded.
|
||||
- **ns-b:** Everything allowed by global filters (no namespace policy match).
|
||||
|
||||
**Verify:** `velero backup describe` lists resolved filters for `ns-a`.
|
||||
|
||||
---
|
||||
|
||||
### Example 2 — Exact resource names
|
||||
|
||||
**Goal:** Back up only two ConfigMaps by exact name, optionally requiring a label.
|
||||
|
||||
**Policy:**
|
||||
|
||||
```yaml
|
||||
version: v1
|
||||
namespacedFilterPolicies:
|
||||
- namespaces:
|
||||
- target-namespace
|
||||
resourceFilters:
|
||||
- kinds: [ConfigMap]
|
||||
names: [vm-1, vm-2]
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
resource-type: VirtualMachine
|
||||
```
|
||||
|
||||
**Backup:** `includedNamespaces: [target-namespace]` plus `resourcePolicy` reference.
|
||||
|
||||
**Expected outcome:** Only `vm-1` and `vm-2` ConfigMaps with `resource-type=VirtualMachine`. `vm-3` and other ConfigMaps are excluded.
|
||||
|
||||
**Verify:** Backup archive contains exactly those two ConfigMaps in `target-namespace`.
|
||||
|
||||
---
|
||||
|
||||
### Example 3 — Glob name patterns with exclusions
|
||||
|
||||
**Goal:** Back up `app-*` ConfigMaps and Secrets in `production`, but exclude temporary and debug names.
|
||||
|
||||
**Policy:**
|
||||
|
||||
```yaml
|
||||
version: v1
|
||||
namespacedFilterPolicies:
|
||||
- namespaces:
|
||||
- production
|
||||
resourceFilters:
|
||||
- kinds: [ConfigMap, Secret]
|
||||
names: ["app-*"]
|
||||
excludedNames: ["*-tmp-*", "*-debug-*", "*-tmp", "*-debug"]
|
||||
```
|
||||
|
||||
**Expected outcome:**
|
||||
|
||||
- **Included:** `app-config`, `app-cache-config`, `app-secret`, `app-db-secret`
|
||||
- **Excluded:** `app-tmp-config`, `app-debug-config` (excluded by `excludedNames`), and `monitoring-tmp-secret` (excluded because it does not match the `names: ["app-*"]` allowlist)
|
||||
|
||||
`excludedNames` takes precedence over `names` when both match.
|
||||
|
||||
**Verify:** Inspect backup item list.
|
||||
|
||||
---
|
||||
|
||||
### Example 4 — Per-kind label selectors
|
||||
|
||||
**Goal:** Apply different label rules to different resource types in the same namespace.
|
||||
|
||||
**Policy:**
|
||||
|
||||
```yaml
|
||||
version: v1
|
||||
namespacedFilterPolicies:
|
||||
- namespaces:
|
||||
- target-namespace
|
||||
resourceFilters:
|
||||
- kinds: [ConfigMap]
|
||||
orLabelSelectors:
|
||||
- matchLabels:
|
||||
app: production-workload-1
|
||||
component: vm-group
|
||||
- matchLabels:
|
||||
app: production-workload-2
|
||||
component: vm-service
|
||||
```
|
||||
|
||||
**Expected outcome:** ConfigMaps matching either label combination are backed up; other ConfigMaps in the namespace are not (for this kind).
|
||||
|
||||
**Note:** Prefer `matchExpressions` with `In` for value-OR on a single key (see next example). Use `orLabelSelectors` when you need OR across **independent multi-key groups**. `labelSelector` and `orLabelSelectors` cannot appear in the same `resourceFilters` entry.
|
||||
|
||||
---
|
||||
|
||||
### Example 4b — Set-based label selectors (`matchExpressions`)
|
||||
|
||||
**Goal:** Back up Deployments and Pods that are in `prod` or `staging`, belong to `app=my-app`, and do **not** carry a skip label.
|
||||
|
||||
**Policy:**
|
||||
|
||||
```yaml
|
||||
version: v1
|
||||
namespacedFilterPolicies:
|
||||
- namespaces:
|
||||
- production
|
||||
resourceFilters:
|
||||
- kinds: [Deployment, Pod]
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
app: my-app
|
||||
matchExpressions:
|
||||
- key: environment
|
||||
operator: In
|
||||
values: [prod, staging]
|
||||
- key: do-not-backup
|
||||
operator: DoesNotExist
|
||||
```
|
||||
|
||||
**Supported operators:** `In`, `NotIn`, `Exists`, `DoesNotExist` (same as Kubernetes / Velero global `--selector`).
|
||||
|
||||
**Other useful patterns:**
|
||||
|
||||
```yaml
|
||||
# Exclude environments
|
||||
matchExpressions:
|
||||
- key: environment
|
||||
operator: NotIn
|
||||
values: [dev, test]
|
||||
|
||||
# Require a label key to be present (any value)
|
||||
matchExpressions:
|
||||
- key: tier
|
||||
operator: Exists
|
||||
```
|
||||
|
||||
**Expected outcome:** Only Deployments/Pods with `app=my-app`, `environment` in `{prod, staging}`, and without `do-not-backup` are backed up.
|
||||
|
||||
---
|
||||
|
||||
### Example 5 — OR label selectors across kinds
|
||||
|
||||
**Goal:** Back up ConfigMaps, Secrets, or Deployments that match any of several label conditions.
|
||||
|
||||
**Policy:**
|
||||
|
||||
```yaml
|
||||
version: v1
|
||||
namespacedFilterPolicies:
|
||||
- namespaces:
|
||||
- ns-a
|
||||
resourceFilters:
|
||||
- kinds: [ConfigMap, Secret]
|
||||
orLabelSelectors:
|
||||
- matchLabels:
|
||||
app: my-app
|
||||
- matchLabels:
|
||||
app: monitoring
|
||||
- kinds: [Deployment]
|
||||
orLabelSelectors:
|
||||
- matchLabels:
|
||||
app: my-app
|
||||
- matchLabels:
|
||||
app: monitoring
|
||||
- matchLabels:
|
||||
component: backend
|
||||
```
|
||||
|
||||
**Expected outcome:** Resources included if they match **any** selector in `orLabelSelectors` for their kind (AND within each selector, OR across the list).
|
||||
|
||||
---
|
||||
|
||||
### Example 6 — Multiple criteria on one kind
|
||||
|
||||
**Goal:** Combine exact names with OR label selectors for a single kind.
|
||||
|
||||
**Policy:**
|
||||
|
||||
```yaml
|
||||
version: v1
|
||||
namespacedFilterPolicies:
|
||||
- namespaces:
|
||||
- target-namespace
|
||||
resourceFilters:
|
||||
- kinds: [ConfigMap]
|
||||
names: [vm-1, vm-2]
|
||||
orLabelSelectors:
|
||||
- matchLabels:
|
||||
resource-type: VirtualMachine
|
||||
- matchLabels:
|
||||
component: vm-group
|
||||
- matchLabels:
|
||||
component: vm-service
|
||||
```
|
||||
|
||||
**Expected outcome:** Only `vm-1` and `vm-2` that also satisfy one of the label OR branches.
|
||||
|
||||
---
|
||||
|
||||
### Example 7 — One policy entry, multiple namespaces
|
||||
|
||||
**Goal:** Apply the same rules to `ns-a`, `ns-b`, and `production` in a single policy block.
|
||||
|
||||
**Policy:**
|
||||
|
||||
```yaml
|
||||
version: v1
|
||||
namespacedFilterPolicies:
|
||||
- namespaces:
|
||||
- ns-a
|
||||
- ns-b
|
||||
- production
|
||||
resourceFilters:
|
||||
- kinds: [ConfigMap]
|
||||
- kinds: [Deployment]
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
tier: web
|
||||
```
|
||||
|
||||
**Expected outcome:**
|
||||
|
||||
- All ConfigMaps in those namespaces (no label filter on that entry).
|
||||
- Deployments with `tier=web` only.
|
||||
|
||||
---
|
||||
|
||||
### Example 8 — Namespace glob patterns and ordering
|
||||
|
||||
**Goal:** Different backup breadth for `team-frontend-prod`, `team-frontend-dev`, and `team-backend-test` using glob patterns.
|
||||
|
||||
**Policy (correct order — most specific first):**
|
||||
|
||||
```yaml
|
||||
version: v1
|
||||
namespacedFilterPolicies:
|
||||
- namespaces:
|
||||
- "team-frontend-*"
|
||||
resourceFilters:
|
||||
- kinds: [Deployment, Service, ConfigMap]
|
||||
- namespaces:
|
||||
- "team-*"
|
||||
resourceFilters:
|
||||
- kinds: [Deployment, Service]
|
||||
- namespaces:
|
||||
- team-frontend-prod # exact match
|
||||
resourceFilters:
|
||||
- kinds: [Deployment, Service, ConfigMap, Secret, PersistentVolumeClaim]
|
||||
```
|
||||
|
||||
**Expected outcome:**
|
||||
|
||||
| Namespace | Matched policy | Kinds backed up |
|
||||
|-----------|----------------|-----------------|
|
||||
| `team-frontend-prod` | First entry (exact) | 5 kinds |
|
||||
| `team-frontend-dev` | `team-frontend-*` | 3 kinds |
|
||||
| `team-backend-test` | `team-*` | 2 kinds |
|
||||
|
||||
**Wrong order (avoid):** If `team-*` is listed **before** `team-frontend-*`, then `team-frontend-dev` matches the broader `team-*` rule first and only Deployments and Services are backed up — the more specific `team-frontend-*` rule is never reached.
|
||||
|
||||
Velero evaluates namespaces by looking for an **exact match** first, and then evaluates glob patterns in **definition order** (first-match wins). Because `team-frontend-prod` is an exact match in this policy, its evaluation is unaffected by glob ordering. However, for namespaces relying on glob patterns like `team-frontend-dev`, the order of the glob patterns is critical.
|
||||
|
||||
**Backup:** Include all relevant namespaces in `includedNamespaces` (they must still pass the global namespace filter).
|
||||
|
||||
---
|
||||
|
||||
### Example 9 — Catch-all by label
|
||||
|
||||
**Goal:** Back up any resource kind that has a given label, without listing every kind. Kind-specific entries override the catch-all.
|
||||
|
||||
**Policy (recommended explicit form):**
|
||||
|
||||
```yaml
|
||||
version: v1
|
||||
namespacedFilterPolicies:
|
||||
- namespaces:
|
||||
- ns-a
|
||||
resourceFilters:
|
||||
- kinds: ["*"] # catch-all
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
app: common-app
|
||||
- kinds: [ConfigMap, Secret] # override for these kinds
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
app: specialized-app
|
||||
```
|
||||
|
||||
**Equivalent:** `kinds: []` (empty) also denotes a catch-all; `kinds: ["*"]` is preferred for readability.
|
||||
|
||||
**Rules:**
|
||||
|
||||
- At most **one** catch-all per namespace policy entry.
|
||||
- Catch-all entries **cannot** use `names` or `excludedNames` — use kind-specific entries for name filtering.
|
||||
- Catch-all does **not** inherit `BackupSpec.labelSelector`; set `labelSelector` or `orLabelSelectors` on the catch-all entry explicitly.
|
||||
|
||||
**Expected outcome:** ConfigMaps and Secrets use `app=specialized-app`; all other kinds listed only via catch-all use `app=common-app`.
|
||||
|
||||
---
|
||||
|
||||
### Example 10 — Catch-all with per-kind name overrides
|
||||
|
||||
**Goal:** Pin critical Deployments and Secrets by exact name; back up everything else with a label convention.
|
||||
|
||||
**Policy:**
|
||||
|
||||
```yaml
|
||||
version: v1
|
||||
namespacedFilterPolicies:
|
||||
- namespaces:
|
||||
- ns-a
|
||||
resourceFilters:
|
||||
- kinds: [Deployment]
|
||||
names: [api-server, worker]
|
||||
- kinds: [Secret]
|
||||
names: [db-credentials, tls-cert]
|
||||
- kinds: ["*"]
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
backup: "true"
|
||||
```
|
||||
|
||||
**Expected outcome:**
|
||||
|
||||
- Deployments: only `api-server` and `worker`
|
||||
- Secrets: only `db-credentials` and `tls-cert`
|
||||
- Other kinds (ConfigMap, Service, …): resources with `backup=true` only
|
||||
|
||||
**Verify:** `other-deployment` and `no-backup-label-config` should be absent; `backup-labeled-config` and `catch-all-labeled-service` should be present.
|
||||
|
||||
---
|
||||
|
||||
### Example 11 — Override-only catch-all (no label on catch-all)
|
||||
|
||||
**Goal:** Apply a strict name filter to one kind while including all other kinds without listing them or adding labels.
|
||||
|
||||
**Policy:**
|
||||
|
||||
```yaml
|
||||
version: v1
|
||||
namespacedFilterPolicies:
|
||||
- namespaces:
|
||||
- ns-a
|
||||
resourceFilters:
|
||||
- kinds: [Secret]
|
||||
names: [app-secret]
|
||||
- kinds: ["*"] # no labelSelector — all other kinds included
|
||||
```
|
||||
|
||||
**Expected outcome:**
|
||||
|
||||
- Secrets: only `app-secret`
|
||||
- Other kinds in `ns-a`: all instances included (subject to global filters and allowlist semantics for listed vs unlisted kinds via catch-all)
|
||||
|
||||
Use this when you need a narrow exception for one type and broad inclusion for the rest of the namespace.
|
||||
|
||||
---
|
||||
|
||||
### Example 12 — Cluster-scoped refinement
|
||||
|
||||
**Goal:** Refine which cluster-scoped resources are backed up by name and label, without replacing global cluster-scoped inclusion.
|
||||
|
||||
**Policy:**
|
||||
|
||||
```yaml
|
||||
version: v1
|
||||
clusterScopedFilterPolicy:
|
||||
resourceFilters:
|
||||
- kinds: [StorageClass]
|
||||
names: ["my-app-*"]
|
||||
- kinds: [ClusterRole, ClusterRoleBinding]
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
app: my-app
|
||||
```
|
||||
|
||||
**Backup (required):** You must still include cluster-scoped kinds on the Backup:
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
includedNamespaces:
|
||||
- ns-a
|
||||
includedClusterScopedResources:
|
||||
- storageclasses
|
||||
- clusterroles
|
||||
- clusterrolebindings
|
||||
resourcePolicy:
|
||||
kind: configmap
|
||||
name: cluster-scoped-filter-policy
|
||||
```
|
||||
|
||||
**Expected outcome (full overlay):**
|
||||
|
||||
- StorageClasses matching `my-app-*` only
|
||||
- ClusterRoles and ClusterRoleBindings with `app=my-app` only
|
||||
- Namespace-scoped resources in `ns-a`: global filters (no `namespacedFilterPolicies` in this example)
|
||||
|
||||
**Partial overlay:** If `includedClusterScopedResources` lists only `clusterroles` and `clusterrolebindings`, StorageClasses are **not** backed up even if listed in `clusterScopedFilterPolicy` — global inclusion is evaluated first.
|
||||
|
||||
**Differences from namespace policies:**
|
||||
|
||||
- **Not** an allowlist — unlisted cluster-scoped kinds fall back to global filters.
|
||||
- **No catch-all** — `kinds: []` or `kinds: ["*"]` is invalid and fails validation.
|
||||
|
||||
---
|
||||
|
||||
### Example 13 — Global `includeExcludePolicy` and namespace filters
|
||||
|
||||
**Goal:** Set a global resource-type baseline, then refine per namespace. Understand that global **exclusions** cannot be overridden per namespace.
|
||||
|
||||
**Policy:**
|
||||
|
||||
```yaml
|
||||
version: v1
|
||||
includeExcludePolicy:
|
||||
includedNamespaceScopedResources:
|
||||
- configmaps
|
||||
- secrets
|
||||
- deployments
|
||||
- services
|
||||
namespacedFilterPolicies:
|
||||
- namespaces:
|
||||
- ns-a
|
||||
resourceFilters:
|
||||
- kinds: [ConfigMap, Secret]
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
app: my-app
|
||||
- namespaces:
|
||||
- production
|
||||
resourceFilters:
|
||||
- kinds: [ConfigMap]
|
||||
names: ["app-*"]
|
||||
```
|
||||
|
||||
**Expected outcome:**
|
||||
|
||||
- **ns-a:** ConfigMaps and Secrets with `app=my-app` (within global allowlist)
|
||||
- **production:** ConfigMaps matching `app-*` pattern
|
||||
- **Other included namespaces:** Only kinds allowed by `includeExcludePolicy` (no per-namespace override)
|
||||
|
||||
**Global exclusion wins (important):**
|
||||
|
||||
```yaml
|
||||
includeExcludePolicy:
|
||||
excludedNamespaceScopedResources:
|
||||
- secrets
|
||||
namespacedFilterPolicies:
|
||||
- namespaces:
|
||||
- ns-a
|
||||
resourceFilters:
|
||||
- kinds: [ConfigMap, Secret, Deployment]
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
app: my-app
|
||||
```
|
||||
|
||||
**Result:** No Secrets in the backup — the namespace policy cannot re-include a globally excluded kind. Velero logs a warning at backup start if you list an excluded kind in `namespacedFilterPolicies`.
|
||||
|
||||
**Backup tip:** Do not set `includedResources` on the Backup; use `includeExcludePolicy` in the ConfigMap instead.
|
||||
|
||||
---
|
||||
|
||||
### Example 14 — Volume policies and namespace filters together
|
||||
|
||||
**Goal:** Use volume snapshot/fs-backup rules and namespace filters in one ConfigMap.
|
||||
|
||||
**Policy:**
|
||||
|
||||
```yaml
|
||||
version: v1
|
||||
volumePolicies:
|
||||
- conditions:
|
||||
capacity: "0,10Gi"
|
||||
storageClass:
|
||||
- standard
|
||||
action:
|
||||
type: fs-backup
|
||||
- conditions:
|
||||
capacity: "10Gi,100Gi"
|
||||
action:
|
||||
type: snapshot
|
||||
namespacedFilterPolicies:
|
||||
- namespaces:
|
||||
- production
|
||||
resourceFilters:
|
||||
- kinds: [ConfigMap]
|
||||
names: ["app-*"]
|
||||
excludedNames: ["*-tmp-*", "*-debug-*", "*-tmp", "*-debug"]
|
||||
- kinds: [Secret]
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
workload: application
|
||||
```
|
||||
|
||||
**Expected outcome:** Volume actions apply to PVCs per `volumePolicies`; resource inclusion follows `namespacedFilterPolicies`. The sections are independent.
|
||||
|
||||
---
|
||||
|
||||
### Example 15 — `velero.io/exclude-from-backup=true` always wins
|
||||
|
||||
**Goal:** Ensure explicitly excluded resources never appear in the backup, even when they match namespace filters or catch-all rules.
|
||||
|
||||
**Policy:**
|
||||
|
||||
```yaml
|
||||
version: v1
|
||||
namespacedFilterPolicies:
|
||||
- namespaces:
|
||||
- ns-a
|
||||
resourceFilters:
|
||||
- kinds: [ConfigMap, Secret]
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
app: my-app
|
||||
- kinds: ["*"]
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
app: my-app
|
||||
```
|
||||
|
||||
**On resources to exclude**, set:
|
||||
|
||||
```yaml
|
||||
metadata:
|
||||
labels:
|
||||
velero.io/exclude-from-backup: "true"
|
||||
```
|
||||
|
||||
**Expected outcome:** Resources with `app=my-app` **and** `velero.io/exclude-from-backup=true` are excluded. Same rule applies to cluster-scoped resources refined by `clusterScopedFilterPolicy`.
|
||||
|
||||
---
|
||||
|
||||
## Concepts reference
|
||||
|
||||
### `resourceFilters` fields
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `kinds` | Resource type names (e.g. `ConfigMap`, `deployments`). Empty or `["*"]` = catch-all (namespace policies only). |
|
||||
| `labelSelector` | Kubernetes-style selector with `matchLabels` and/or `matchExpressions` (`In`, `NotIn`, `Exists`, `DoesNotExist`). All requirements are AND-ed. |
|
||||
| `orLabelSelectors` | List of selectors; match if **any** entry matches (AND within each, OR across the list). Use for OR of multi-key groups; prefer `In` for value-OR on one key. Mutually exclusive with `labelSelector`. |
|
||||
| `names` | Exact names or glob patterns to include. |
|
||||
| `excludedNames` | Patterns to exclude; wins over `names` when both match. |
|
||||
|
||||
Only kinds listed in `resourceFilters` (or covered by catch-all) are collected from namespaces matched by `namespacedFilterPolicies`.
|
||||
|
||||
### Glob pattern syntax
|
||||
|
||||
Name and namespace patterns use the same glob style as elsewhere in Velero (`gobwas/glob`):
|
||||
|
||||
- Supported: `*`, `?`, `[abc]`, `[a-z]`
|
||||
- Not supported: `**`, regex, `|`, `()`, `!`, `{}`, `,`
|
||||
|
||||
Examples: `app-*`, `team-frontend-*`, `*-tmp`.
|
||||
|
||||
### Precedence cheat sheet
|
||||
|
||||
**Namespaces**
|
||||
|
||||
1. `BackupSpec.excludedNamespaces` — excluded namespaces are never backed up; namespace policies cannot override this.
|
||||
2. `namespacedFilterPolicies` — first matching pattern (exact match checked before globs in pattern order).
|
||||
3. No match — use global BackupSpec + `includeExcludePolicy`.
|
||||
|
||||
**Namespace-scoped resources (when a namespace policy matches)**
|
||||
|
||||
1. Global `includeExcludePolicy` exclusions (e.g. `excludedNamespaceScopedResources`) apply first.
|
||||
2. Only kinds in `resourceFilters` (or catch-all) are allowlisted for collection.
|
||||
3. Per-kind `labelSelector` / `orLabelSelectors` for API list calls.
|
||||
4. Per-kind `names` / `excludedNames` at backup write time.
|
||||
5. Label `velero.io/exclude-from-backup=true` always excludes.
|
||||
|
||||
**Cluster-scoped resources**
|
||||
|
||||
1. Must be allowed by `includedClusterScopedResources` / global cluster settings.
|
||||
2. If `clusterScopedFilterPolicy` lists the kind, apply its label and name rules.
|
||||
3. If not listed in `clusterScopedFilterPolicy`, use global BackupSpec filters.
|
||||
4. `velero.io/exclude-from-backup=true` always excludes.
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
nsGlobal[BackupSpec namespace include/exclude]
|
||||
nsPolicy{namespacedFilterPolicies match?}
|
||||
nsAllow[Allowlist kinds + per-kind filters]
|
||||
nsGlobalFallback[Global BackupSpec + includeExcludePolicy]
|
||||
|
||||
nsGlobal --> nsPolicy
|
||||
nsPolicy -->|yes| nsAllow
|
||||
nsPolicy -->|no| nsGlobalFallback
|
||||
|
||||
csInclude[includedClusterScopedResources]
|
||||
csPolicy{kind in clusterScopedFilterPolicy?}
|
||||
csRefine[Per-kind label and name rules]
|
||||
csGlobal[Global cluster filters]
|
||||
|
||||
csInclude --> csPolicy
|
||||
csPolicy -->|yes| csRefine
|
||||
csPolicy -->|no| csGlobal
|
||||
```
|
||||
|
||||
### Catch-all summary
|
||||
|
||||
| Rule | Detail |
|
||||
|------|--------|
|
||||
| Syntax | `kinds: ["*"]` or `kinds: []` |
|
||||
| Count | At most one catch-all per `namespacedFilterPolicies` entry |
|
||||
| Names | `names` / `excludedNames` not allowed on catch-all |
|
||||
| Override | Kind-specific entries take precedence over catch-all |
|
||||
| Label inheritance | Does not use `BackupSpec.labelSelector` |
|
||||
| Cluster-scoped | Catch-all **not** supported in `clusterScopedFilterPolicy` |
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting and validation
|
||||
|
||||
### Verify a backup
|
||||
|
||||
```bash
|
||||
velero backup describe BACKUP_NAME
|
||||
velero backup logs BACKUP_NAME
|
||||
velero backup describe BACKUP_NAME -o json | jq '.namespacedFilterPolicies'
|
||||
velero backup describe BACKUP_NAME -o json | jq '.clusterScopedFilterPolicy'
|
||||
```
|
||||
|
||||
Catch-all entries appear as `<catch-all> (all other kinds)` in text output, or `"isCatchAll": true` in JSON.
|
||||
|
||||
### Common misconfigurations
|
||||
|
||||
| Symptom | Likely cause | Fix |
|
||||
|---------|----------------|-----|
|
||||
| Fewer resources than expected in `team-frontend-prod` | Broad namespace pattern listed before specific one | Reorder policies: most specific `namespaces` first |
|
||||
| Namespace policy lists Secrets but none in backup | `includeExcludePolicy` excludes `secrets` globally | Remove global exclusion or accept no Secrets |
|
||||
| `ClusterRole` in namespace policy has no effect | Cluster-scoped kind in `namespacedFilterPolicies` | Move rule to `clusterScopedFilterPolicy`; check logs for warning |
|
||||
| Backup fails at creation with filter message | Old-style `includedResources` with new policies | Move resource types to `includeExcludePolicy` in ConfigMap |
|
||||
| Catch-all does not use backup-wide label | By design | Set `labelSelector` on the catch-all entry |
|
||||
| Cluster-scoped policy validation error on `kinds: ["*"]` | Catch-all not allowed for cluster policy | List each cluster-scoped kind explicitly |
|
||||
|
||||
### Velero logs
|
||||
|
||||
```bash
|
||||
kubectl logs -n velero deployment/velero | grep -i "namespacedFilterPolicies\|clusterScopedFilterPolicy"
|
||||
kubectl logs -n velero deployment/velero | grep "globally excluded by includeExcludePolicy"
|
||||
kubectl logs -n velero deployment/velero | grep "cluster-scoped"
|
||||
```
|
||||
|
||||
### Validation errors (policy ConfigMap)
|
||||
|
||||
Velero validates the ResourcePolicy when a backup starts. Common errors:
|
||||
|
||||
| Error (summary) | Cause |
|
||||
|-----------------|--------|
|
||||
| `at least one namespace must be specified` | Empty `namespaces: []` |
|
||||
| `at least one resourceFilter must be specified` | Empty `resourceFilters: []` |
|
||||
| `names or excludedNames cannot be specified for catch-all filters` | Name patterns on catch-all entry |
|
||||
| `only one catch-all resource filter is allowed` | Multiple catch-alls in one policy entry |
|
||||
| `kind "X" appears in both resourceFilters[...]` | Same kind in two entries |
|
||||
| `labelSelector and orLabelSelectors cannot co-exist` | Both set in one entry |
|
||||
| `invalid label selector` | Bad operator, values, or label key/value syntax |
|
||||
| `duplicate namespace pattern` | Same namespace string in two policy entries |
|
||||
| `invalid glob pattern` | Bad characters in namespace or name pattern |
|
||||
| `clusterScopedFilterPolicy... kinds must be specified (catch-all is not supported)` | Empty or `["*"]` kinds in cluster policy |
|
||||
| `include-resources, exclude-resources... cannot be used with namespace-scoped or cluster-scoped global filter policies` | Old-style BackupSpec filters with new policy |
|
||||
|
||||
### Silent edge cases (no error)
|
||||
|
||||
- Namespace pattern matches no existing namespace — policy loaded but never applied.
|
||||
- Kind listed but no instances in namespace — empty result, backup still succeeds.
|
||||
- `excludedNames` narrows `names` — e.g. `names: ["app-*"]` + `excludedNames: ["app-config"]` excludes `app-config` only.
|
||||
|
||||
---
|
||||
|
||||
## Restore behavior
|
||||
|
||||
Restore is unchanged: it restores whatever is in the backup archive. Resources excluded by fine-grained filters are simply absent. Use `Restore.spec.includedNamespaces` (and existing restore filters) to limit what you restore from a partial backup.
|
||||
|
||||
Fine-grained resource filtering is also available on the restore path using `namespacedFilterPolicies` and `clusterScopedFilterPolicy`. For details on the restore-side policies, see the [Fine-grained restore filters design](https://github.com/vmware-tanzu/velero/blob/main/design/restore-filter-enhancement/fine-grained-restore-filters-design.md).
|
||||
|
||||
---
|
||||
|
||||
## Related links
|
||||
|
||||
- [Fine-grained backup filters design](https://github.com/velero-io/velero/blob/main/design/backup-filter-enhancement/fine-grained-backup-filters-design.md)
|
||||
@@ -33,6 +33,8 @@ toc:
|
||||
url: /enable-api-group-versions-feature
|
||||
- page: Resource filtering
|
||||
url: /resource-filtering
|
||||
- page: Fine-Grained Backup Filters
|
||||
url: /fine-grained-backup-filters
|
||||
- page: Namespace glob patterns
|
||||
url: /namespace-glob-patterns
|
||||
- page: Backup reference
|
||||
|
||||
Reference in New Issue
Block a user