Compare commits

...

17 Commits

Author SHA1 Message Date
Xun Jiang/Bruce Jiang
6adcf06b5b Merge pull request #9572 from blackpiglet/xj014661/v1.18/CVE-2026-24051
Some checks failed
Run the E2E test on kind / get-go-version (push) Failing after 1m0s
Run the E2E test on kind / build (push) Has been skipped
Run the E2E test on kind / setup-test-matrix (push) Successful in 2s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / get-go-version (push) Failing after 15s
Main CI / Build (push) Has been skipped
Xj014661/v1.18/CVE 2026 24051
2026-03-02 17:42:25 +08:00
Xun Jiang
ffa65605a6 Bump go.opentelemetry.io/otel/sdk to 1.40.0 to fix CVE-2026-24051.
Bump Linux base image to paketobuildpacks/run-jammy-tiny:0.2.104

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2026-03-02 17:09:23 +08:00
Xun Jiang/Bruce Jiang
bd8dfe9ee2 Merge branch 'vmware-tanzu:release-1.18' into release-1.18 2026-03-02 17:04:43 +08:00
lyndon-li
54783fbe28 Merge pull request #9546 from Lyndon-Li/release-1.18
Some checks failed
Run the E2E test on kind / get-go-version (push) Failing after 1h5m30s
Run the E2E test on kind / setup-test-matrix (push) Successful in 4s
Run the E2E test on kind / build (push) Has been skipped
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / get-go-version (push) Failing after 21s
Main CI / Build (push) Has been skipped
Update base image for 1.18.0
2026-02-13 17:11:09 +08:00
Lyndon-Li
cb5f56265a update base image for 1.18.0
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-02-13 16:43:12 +08:00
Tiger Kaovilai
0c7b89a44e Fix VolumePolicy PVC phase condition filter for unbound PVCs
Use typed error approach: Make GetPVForPVC return ErrPVNotFoundForPVC
when PV is not expected to be found (unbound PVC), then use errors.Is
to check for this error type. When a matching policy exists (e.g.,
pvcPhase: [Pending, Lost] with action: skip), apply the action without
error. When no policy matches, return the original error to preserve
default behavior.

Changes:
- Add ErrPVNotFoundForPVC sentinel error to pvc_pv.go
- Update ShouldPerformSnapshot to handle unbound PVCs with policies
- Update ShouldPerformFSBackup to handle unbound PVCs with policies
- Update item_backupper.go to handle Lost PVCs in tracking functions
- Remove checkPVCOnlySkip helper (no longer needed)
- Update tests to reflect new behavior

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-12 16:18:56 +08:00
Xun Jiang/Bruce Jiang
aa89713559 Merge pull request #9537 from kaovilai/1.18-9508
Some checks failed
Run the E2E test on kind / get-go-version (push) Failing after 1m33s
Run the E2E test on kind / build (push) Has been skipped
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / get-go-version (push) Failing after 29s
Main CI / Build (push) Has been skipped
release-1.18: Fix VolumePolicy PVC phase condition filter for unbound PVCs
2026-02-12 11:41:25 +08:00
Xun Jiang/Bruce Jiang
5db4c65a92 Merge branch 'release-1.18' into 1.18-9508 2026-02-12 11:32:06 +08:00
Xun Jiang/Bruce Jiang
87db850f66 Merge pull request #9539 from Joeavaikath/1.18-9502
release-1.18: Support all glob wildcard characters in namespace validation (#9502)
2026-02-12 10:50:55 +08:00
Xun Jiang/Bruce Jiang
c7631fc4a4 Merge branch 'release-1.18' into 1.18-9502 2026-02-12 10:41:26 +08:00
Wenkai Yin(尹文开)
9a37478cc2 Merge pull request #9538 from vmware-tanzu/topic/xj014661/update_migration_test_cases
Update the migration and upgrade test cases.
2026-02-12 10:19:43 +08:00
Tiger Kaovilai
5b54ccd2e0 Fix VolumePolicy PVC phase condition filter for unbound PVCs
Use typed error approach: Make GetPVForPVC return ErrPVNotFoundForPVC
when PV is not expected to be found (unbound PVC), then use errors.Is
to check for this error type. When a matching policy exists (e.g.,
pvcPhase: [Pending, Lost] with action: skip), apply the action without
error. When no policy matches, return the original error to preserve
default behavior.

Changes:
- Add ErrPVNotFoundForPVC sentinel error to pvc_pv.go
- Update ShouldPerformSnapshot to handle unbound PVCs with policies
- Update ShouldPerformFSBackup to handle unbound PVCs with policies
- Update item_backupper.go to handle Lost PVCs in tracking functions
- Remove checkPVCOnlySkip helper (no longer needed)
- Update tests to reflect new behavior

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-11 15:29:14 -05:00
Joseph Antony Vaikath
43b926a58b Support all glob wildcard characters in namespace validation (#9502)
* Support all glob wildcard characters in namespace validation

Expand namespace validation to allow all valid glob pattern characters
(*, ?, {}, [], ,) by replacing them with valid characters during RFC 1123
validation. The actual glob pattern validation is handled separately by
the wildcard package.

Also add validation to reject unsupported characters (|, (), !) that are
not valid in glob patterns, and update terminology from "regex" to "glob"
for clarity since this implementation uses glob patterns, not regex.

Changes:
- Replace all glob wildcard characters in validateNamespaceName
- Add test coverage for valid glob patterns in includes/excludes
- Add test coverage for unsupported characters
- Reject exclamation mark (!) in wildcard patterns
- Clarify comments and error messages about glob vs regex

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Signed-off-by: Joseph <jvaikath@redhat.com>

* Changelog

Signed-off-by: Joseph <jvaikath@redhat.com>

* Add documentation: glob patterns are now accepted

Signed-off-by: Joseph <jvaikath@redhat.com>

* Error message fix

Signed-off-by: Joseph <jvaikath@redhat.com>

* Remove negation glob char test

Signed-off-by: Joseph <jvaikath@redhat.com>

* Add bracket pattern validation for namespace glob patterns

Extends wildcard validation to support square bracket patterns [] used in glob character classes. Validates bracket syntax including empty brackets, unclosed brackets, and unmatched brackets. Extracts ValidateNamespaceName as a public function to enable reuse in namespace validation logic.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Signed-off-by: Joseph <jvaikath@redhat.com>

* Reduce scope to *, ?, [ and ]

Signed-off-by: Joseph <jvaikath@redhat.com>

* Fix tests

Signed-off-by: Joseph <jvaikath@redhat.com>

* Add namespace glob patterns documentation page

Adds dedicated documentation explaining supported glob patterns
for namespace include/exclude filtering to help users understand
the wildcard syntax.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Signed-off-by: Joseph <jvaikath@redhat.com>

* Fix build-image Dockerfile envtest download

Replace inaccessible go.kubebuilder.io URL with setup-envtest and update envtest version to 1.33.0 to match Kubernetes v0.33.3 dependencies.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Signed-off-by: Joseph <jvaikath@redhat.com>

* kubebuilder binaries mv

Signed-off-by: Joseph <jvaikath@redhat.com>

* Reject brace patterns and update documentation

Add {, }, and , to unsupported characters list to explicitly reject
brace expansion patterns. Remove { from wildcard detection since these
patterns are not supported in the 1.18 release.

Update all documentation to show supported patterns inline (*, ?, [abc])
with clickable links to the detailed namespace-glob-patterns page.
Simplify YAML comments by removing non-clickable URLs.

Update tests to expect errors when brace patterns are used.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Signed-off-by: Joseph <jvaikath@redhat.com>

* Document brace expansion as unsupported

Add {} and , to the unsupported patterns section to clarify that
brace expansion patterns like {a,b,c} are not supported.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Signed-off-by: Joseph <jvaikath@redhat.com>

* Update tests to expect brace pattern rejection

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Signed-off-by: Joseph <jvaikath@redhat.com>

---------

Signed-off-by: Joseph <jvaikath@redhat.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-11 15:14:06 -05:00
Xun Jiang
9bfc78e769 Update the migration and upgrade test cases.
Some checks failed
Run the E2E test on kind / get-go-version (push) Failing after 1m34s
Run the E2E test on kind / build (push) Has been skipped
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Modify Dockerfile to fix GitHub CI action error.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2026-02-11 14:56:55 +08:00
Xun Jiang/Bruce Jiang
c9e26256fa Bump Golang version to v1.25.7 (#9536)
Some checks failed
Run the E2E test on kind / get-go-version (push) Failing after 1m26s
Run the E2E test on kind / build (push) Has been skipped
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / get-go-version (push) Failing after 16s
Main CI / Build (push) Has been skipped
Fix test case issue and add UT.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2026-02-10 15:50:23 -05:00
Wenkai Yin(尹文开)
6e315c32e2 Merge pull request #9530 from Lyndon-Li/release-1.18
Some checks failed
Run the E2E test on kind / get-go-version (push) Failing after 1m55s
Run the E2E test on kind / build (push) Has been skipped
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / get-go-version (push) Failing after 12s
Main CI / Build (push) Has been skipped
[1.18] Move implemented design for 1.18
2026-02-09 11:18:40 +08:00
Lyndon-Li
91cbc40956 move implemented design for 1.18
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-02-06 18:56:05 +08:00
40 changed files with 1312 additions and 375 deletions

View File

@@ -13,7 +13,7 @@
# limitations under the License.
# Velero binary build section
FROM --platform=$BUILDPLATFORM golang:1.25-bookworm AS velero-builder
FROM --platform=$BUILDPLATFORM golang:1.25.7-bookworm AS velero-builder
ARG GOPROXY
ARG BIN
@@ -49,7 +49,7 @@ RUN mkdir -p /output/usr/bin && \
go clean -modcache -cache
# Restic binary build section
FROM --platform=$BUILDPLATFORM golang:1.25-bookworm AS restic-builder
FROM --platform=$BUILDPLATFORM golang:1.25.7-bookworm AS restic-builder
ARG GOPROXY
ARG BIN
@@ -73,7 +73,7 @@ RUN mkdir -p /output/usr/bin && \
go clean -modcache -cache
# Velero image packing section
FROM paketobuildpacks/run-jammy-tiny:latest
FROM paketobuildpacks/run-jammy-tiny:0.2.104
LABEL maintainer="Xun Jiang <jxun@vmware.com>"

View File

@@ -15,7 +15,7 @@
ARG OS_VERSION=1809
# Velero binary build section
FROM --platform=$BUILDPLATFORM golang:1.25-bookworm AS velero-builder
FROM --platform=$BUILDPLATFORM golang:1.25.7-bookworm AS velero-builder
ARG GOPROXY
ARG BIN

View File

@@ -52,7 +52,7 @@ git_sha = str(local("git rev-parse HEAD", quiet = True, echo_off = True)).strip(
tilt_helper_dockerfile_header = """
# Tilt image
FROM golang:1.25 as tilt-helper
FROM golang:1.25.7 as tilt-helper
# Support live reloading with Tilt
RUN wget --output-document /restart.sh --quiet https://raw.githubusercontent.com/windmilleng/rerun-process-wrapper/master/restart.sh && \

View File

@@ -16,7 +16,7 @@ https://velero.io/docs/v1.18/upgrade-to-1.18/
#### Concurrent backup
In v1.18, Velero is capable to process multiple backups concurrently. This is a significant usability improvement, especially for multiple tenants or multiple users case, backups submitted from different users could run their backups simultaneously without interfering with each other.
Check design https://github.com/vmware-tanzu/velero/blob/main/design/concurrent-backup-processing.md for more details.
Check design https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/concurrent-backup-processing.md for more details.
#### Cache volume for data movers
In v1.18, Velero allows users to configure cache volumes for data mover pods during restore for CSI snapshot data movement and fs-backup. This brings below benefits:
@@ -24,7 +24,7 @@ In v1.18, Velero allows users to configure cache volumes for data mover pods dur
- Solve the problem that multiple data mover pods fail to run concurrently in one node when the node's ephemeral disk is limited
- Working together with backup repository's cache limit configuration, cache volume with appropriate size helps to improve the restore throughput
Check design https://github.com/vmware-tanzu/velero/blob/main/design/backup-repo-cache-volume.md for more details.
Check design https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/backup-repo-cache-volume.md for more details.
#### Incremental size for data movers
In v1.18, Velero allows users to observe the incremental size of data movers backups for CSI snapshot data movement and fs-backup, so that users could visually see the data reduction due to incremental backup.

View File

@@ -0,0 +1 @@
Fix VolumePolicy PVC phase condition filter for unbound PVCs (#9507)

View File

@@ -0,0 +1 @@
Fix VolumePolicy PVC phase condition filter for unbound PVCs (#9507)

View File

@@ -0,0 +1 @@
Support all glob wildcard characters in namespace validation

14
go.mod
View File

@@ -1,6 +1,6 @@
module github.com/vmware-tanzu/velero
go 1.25.0
go 1.25.7
require (
cloud.google.com/go/storage v1.57.2
@@ -171,11 +171,11 @@ require (
go.opentelemetry.io/contrib/detectors/gcp v1.38.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 // indirect
go.opentelemetry.io/otel v1.38.0 // indirect
go.opentelemetry.io/otel/metric v1.38.0 // indirect
go.opentelemetry.io/otel/sdk v1.38.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.38.0 // indirect
go.opentelemetry.io/otel/trace v1.38.0 // indirect
go.opentelemetry.io/otel v1.40.0 // indirect
go.opentelemetry.io/otel/metric v1.40.0 // indirect
go.opentelemetry.io/otel/sdk v1.40.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.40.0 // indirect
go.opentelemetry.io/otel/trace v1.40.0 // indirect
go.starlark.net v0.0.0-20230525235612-a134d8f9ddca // indirect
go.uber.org/multierr v1.11.0 // indirect
go.yaml.in/yaml/v2 v2.4.3 // indirect
@@ -183,7 +183,7 @@ require (
golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 // indirect
golang.org/x/net v0.47.0 // indirect
golang.org/x/sync v0.18.0 // indirect
golang.org/x/sys v0.38.0 // indirect
golang.org/x/sys v0.40.0 // indirect
golang.org/x/term v0.37.0 // indirect
golang.org/x/time v0.14.0 // indirect
golang.org/x/tools v0.38.0 // indirect

24
go.sum
View File

@@ -748,18 +748,18 @@ go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.6
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0/go.mod h1:snMWehoOh2wsEwnvvwtDyFCxVeDAODenXHtn5vzrKjo=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 h1:F7Jx+6hwnZ41NSFTO5q4LYDtJRXBf2PD0rNBkeB/lus=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0/go.mod h1:UHB22Z8QsdRDrnAtX4PntOl36ajSxcdUMt1sF7Y6E7Q=
go.opentelemetry.io/otel v1.38.0 h1:RkfdswUDRimDg0m2Az18RKOsnI8UDzppJAtj01/Ymk8=
go.opentelemetry.io/otel v1.38.0/go.mod h1:zcmtmQ1+YmQM9wrNsTGV/q/uyusom3P8RxwExxkZhjM=
go.opentelemetry.io/otel v1.40.0 h1:oA5YeOcpRTXq6NN7frwmwFR0Cn3RhTVZvXsP4duvCms=
go.opentelemetry.io/otel v1.40.0/go.mod h1:IMb+uXZUKkMXdPddhwAHm6UfOwJyh4ct1ybIlV14J0g=
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.36.0 h1:rixTyDGXFxRy1xzhKrotaHy3/KXdPhlWARrCgK+eqUY=
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.36.0/go.mod h1:dowW6UsM9MKbJq5JTz2AMVp3/5iW5I/TStsk8S+CfHw=
go.opentelemetry.io/otel/metric v1.38.0 h1:Kl6lzIYGAh5M159u9NgiRkmoMKjvbsKtYRwgfrA6WpA=
go.opentelemetry.io/otel/metric v1.38.0/go.mod h1:kB5n/QoRM8YwmUahxvI3bO34eVtQf2i4utNVLr9gEmI=
go.opentelemetry.io/otel/sdk v1.38.0 h1:l48sr5YbNf2hpCUj/FoGhW9yDkl+Ma+LrVl8qaM5b+E=
go.opentelemetry.io/otel/sdk v1.38.0/go.mod h1:ghmNdGlVemJI3+ZB5iDEuk4bWA3GkTpW+DOoZMYBVVg=
go.opentelemetry.io/otel/sdk/metric v1.38.0 h1:aSH66iL0aZqo//xXzQLYozmWrXxyFkBJ6qT5wthqPoM=
go.opentelemetry.io/otel/sdk/metric v1.38.0/go.mod h1:dg9PBnW9XdQ1Hd6ZnRz689CbtrUp0wMMs9iPcgT9EZA=
go.opentelemetry.io/otel/trace v1.38.0 h1:Fxk5bKrDZJUH+AMyyIXGcFAPah0oRcT+LuNtJrmcNLE=
go.opentelemetry.io/otel/trace v1.38.0/go.mod h1:j1P9ivuFsTceSWe1oY+EeW3sc+Pp42sO++GHkg4wwhs=
go.opentelemetry.io/otel/metric v1.40.0 h1:rcZe317KPftE2rstWIBitCdVp89A2HqjkxR3c11+p9g=
go.opentelemetry.io/otel/metric v1.40.0/go.mod h1:ib/crwQH7N3r5kfiBZQbwrTge743UDc7DTFVZrrXnqc=
go.opentelemetry.io/otel/sdk v1.40.0 h1:KHW/jUzgo6wsPh9At46+h4upjtccTmuZCFAc9OJ71f8=
go.opentelemetry.io/otel/sdk v1.40.0/go.mod h1:Ph7EFdYvxq72Y8Li9q8KebuYUr2KoeyHx0DRMKrYBUE=
go.opentelemetry.io/otel/sdk/metric v1.40.0 h1:mtmdVqgQkeRxHgRv4qhyJduP3fYJRMX4AtAlbuWdCYw=
go.opentelemetry.io/otel/sdk/metric v1.40.0/go.mod h1:4Z2bGMf0KSK3uRjlczMOeMhKU2rhUqdWNoKcYrtcBPg=
go.opentelemetry.io/otel/trace v1.40.0 h1:WA4etStDttCSYuhwvEa8OP8I5EWu24lkOzp+ZYblVjw=
go.opentelemetry.io/otel/trace v1.40.0/go.mod h1:zeAhriXecNGP/s2SEG3+Y8X9ujcJOTqQ5RgdEJcawiA=
go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5/go.mod h1:nmDLcffg48OtT/PSW0Hg7FvpRQsQh5OSqIylirxKC7o=
go.starlark.net v0.0.0-20201006213952-227f4aabceb5/go.mod h1:f0znQkUKRrkk36XxWbGjMqQM8wGv/xHBVE2qc3B5oFU=
go.starlark.net v0.0.0-20230525235612-a134d8f9ddca h1:VdD38733bfYv5tUZwEIskMM93VanwNIi5bIKnDrJdEY=
@@ -969,8 +969,8 @@ golang.org/x/sys v0.0.0-20211019181941-9d821ace8654/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc=
golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/sys v0.40.0 h1:DBZZqJ2Rkml6QMQsZywtnjnnGvHza6BTfYFWY9kjEWQ=
golang.org/x/sys v0.40.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210220032956-6a3ed077a48d/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=

View File

@@ -12,7 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
FROM --platform=$TARGETPLATFORM golang:1.25-bookworm
FROM --platform=$TARGETPLATFORM golang:1.25.7-bookworm
ARG GOPROXY
@@ -21,9 +21,11 @@ ENV GO111MODULE=on
ENV GOPROXY=${GOPROXY}
# kubebuilder test bundle is separated from kubebuilder. Need to setup it for CI test.
RUN curl -sSLo envtest-bins.tar.gz https://go.kubebuilder.io/test-tools/1.22.1/linux/$(go env GOARCH) && \
mkdir /usr/local/kubebuilder && \
tar -C /usr/local/kubebuilder --strip-components=1 -zvxf envtest-bins.tar.gz
# Using setup-envtest to download envtest binaries
RUN go install sigs.k8s.io/controller-runtime/tools/setup-envtest@latest && \
mkdir -p /usr/local/kubebuilder/bin && \
ENVTEST_ASSETS_DIR=$(setup-envtest use 1.33.0 --bin-dir /usr/local/kubebuilder/bin -p path) && \
cp -r ${ENVTEST_ASSETS_DIR}/* /usr/local/kubebuilder/bin/
RUN wget --quiet https://github.com/kubernetes-sigs/kubebuilder/releases/download/v3.2.0/kubebuilder_linux_$(go env GOARCH) && \
mv kubebuilder_linux_$(go env GOARCH) /usr/local/kubebuilder/bin/kubebuilder && \

View File

@@ -134,6 +134,7 @@ func (v *volumeHelperImpl) ShouldPerformSnapshot(obj runtime.Unstructured, group
pv := new(corev1api.PersistentVolume)
var err error
var pvNotFoundErr error
if groupResource == kuberesource.PersistentVolumeClaims {
if err = runtime.DefaultUnstructuredConverter.FromUnstructured(obj.UnstructuredContent(), &pvc); err != nil {
v.logger.WithError(err).Error("fail to convert unstructured into PVC")
@@ -142,8 +143,10 @@ func (v *volumeHelperImpl) ShouldPerformSnapshot(obj runtime.Unstructured, group
pv, err = kubeutil.GetPVForPVC(pvc, v.client)
if err != nil {
v.logger.WithError(err).Errorf("fail to get PV for PVC %s", pvc.Namespace+"/"+pvc.Name)
return false, err
// Any error means PV not available - save to return later if no policy matches
v.logger.Debugf("PV not found for PVC %s: %v", pvc.Namespace+"/"+pvc.Name, err)
pvNotFoundErr = err
pv = nil
}
}
@@ -158,7 +161,7 @@ func (v *volumeHelperImpl) ShouldPerformSnapshot(obj runtime.Unstructured, group
vfd := resourcepolicies.NewVolumeFilterData(pv, nil, pvc)
action, err := v.volumePolicy.GetMatchAction(vfd)
if err != nil {
v.logger.WithError(err).Errorf("fail to get VolumePolicy match action for PV %s", pv.Name)
v.logger.WithError(err).Errorf("fail to get VolumePolicy match action for %+v", vfd)
return false, err
}
@@ -167,15 +170,21 @@ func (v *volumeHelperImpl) ShouldPerformSnapshot(obj runtime.Unstructured, group
// If there is no match action, go on to the next check.
if action != nil {
if action.Type == resourcepolicies.Snapshot {
v.logger.Infof(fmt.Sprintf("performing snapshot action for pv %s", pv.Name))
v.logger.Infof("performing snapshot action for %+v", vfd)
return true, nil
} else {
v.logger.Infof("Skip snapshot action for pv %s as the action type is %s", pv.Name, action.Type)
v.logger.Infof("Skip snapshot action for %+v as the action type is %s", vfd, action.Type)
return false, nil
}
}
}
// If resource is PVC, and PV is nil (e.g., Pending/Lost PVC with no matching policy), return the original error
if groupResource == kuberesource.PersistentVolumeClaims && pv == nil && pvNotFoundErr != nil {
v.logger.WithError(pvNotFoundErr).Errorf("fail to get PV for PVC %s", pvc.Namespace+"/"+pvc.Name)
return false, pvNotFoundErr
}
// If this PV is claimed, see if we've already taken a (pod volume backup)
// snapshot of the contents of this PV. If so, don't take a snapshot.
if pv.Spec.ClaimRef != nil {
@@ -209,7 +218,7 @@ func (v *volumeHelperImpl) ShouldPerformSnapshot(obj runtime.Unstructured, group
return true, nil
}
v.logger.Infof(fmt.Sprintf("skipping snapshot action for pv %s possibly due to no volume policy setting or snapshotVolumes is false", pv.Name))
v.logger.Infof("skipping snapshot action for pv %s possibly due to no volume policy setting or snapshotVolumes is false", pv.Name)
return false, nil
}
@@ -219,6 +228,7 @@ func (v volumeHelperImpl) ShouldPerformFSBackup(volume corev1api.Volume, pod cor
return false, nil
}
var pvNotFoundErr error
if v.volumePolicy != nil {
var resource any
var err error
@@ -230,10 +240,13 @@ func (v volumeHelperImpl) ShouldPerformFSBackup(volume corev1api.Volume, pod cor
v.logger.WithError(err).Errorf("fail to get PVC for pod %s", pod.Namespace+"/"+pod.Name)
return false, err
}
resource, err = kubeutil.GetPVForPVC(pvc, v.client)
pvResource, err := kubeutil.GetPVForPVC(pvc, v.client)
if err != nil {
v.logger.WithError(err).Errorf("fail to get PV for PVC %s", pvc.Namespace+"/"+pvc.Name)
return false, err
// Any error means PV not available - save to return later if no policy matches
v.logger.Debugf("PV not found for PVC %s: %v", pvc.Namespace+"/"+pvc.Name, err)
pvNotFoundErr = err
} else {
resource = pvResource
}
}
@@ -260,6 +273,12 @@ func (v volumeHelperImpl) ShouldPerformFSBackup(volume corev1api.Volume, pod cor
return false, nil
}
}
// If no policy matched and PV was not found, return the original error
if pvNotFoundErr != nil {
v.logger.WithError(pvNotFoundErr).Errorf("fail to get PV for PVC %s", pvc.Namespace+"/"+pvc.Name)
return false, pvNotFoundErr
}
}
if v.shouldPerformFSBackupLegacy(volume, pod) {

View File

@@ -286,7 +286,7 @@ func TestVolumeHelperImpl_ShouldPerformSnapshot(t *testing.T) {
expectedErr: false,
},
{
name: "PVC not having PV, return false and error case PV not found",
name: "PVC not having PV, return false and error when no matching policy",
inputObj: builder.ForPersistentVolumeClaim("default", "example-pvc").StorageClass("gp2-csi").Result(),
groupResource: kuberesource.PersistentVolumeClaims,
resourcePolicies: &resourcepolicies.ResourcePolicies{
@@ -1234,3 +1234,312 @@ func TestNewVolumeHelperImplWithCache_UsesCache(t *testing.T) {
require.NoError(t, err)
require.False(t, shouldSnapshot, "Expected snapshot to be skipped due to fs-backup selection via cache")
}
// TestVolumeHelperImpl_ShouldPerformSnapshot_UnboundPVC tests that Pending and Lost PVCs with
// phase-based skip policies don't cause errors when GetPVForPVC would fail.
func TestVolumeHelperImpl_ShouldPerformSnapshot_UnboundPVC(t *testing.T) {
testCases := []struct {
name string
inputPVC *corev1api.PersistentVolumeClaim
resourcePolicies *resourcepolicies.ResourcePolicies
shouldSnapshot bool
expectedErr bool
}{
{
name: "Pending PVC with phase-based skip policy should not error and return false",
inputPVC: builder.ForPersistentVolumeClaim("ns", "pvc-pending").
StorageClass("non-existent-class").
Phase(corev1api.ClaimPending).
Result(),
resourcePolicies: &resourcepolicies.ResourcePolicies{
Version: "v1",
VolumePolicies: []resourcepolicies.VolumePolicy{
{
Conditions: map[string]any{
"pvcPhase": []string{"Pending"},
},
Action: resourcepolicies.Action{
Type: resourcepolicies.Skip,
},
},
},
},
shouldSnapshot: false,
expectedErr: false,
},
{
name: "Pending PVC without matching skip policy should error (no PV)",
inputPVC: builder.ForPersistentVolumeClaim("ns", "pvc-pending-no-policy").
StorageClass("non-existent-class").
Phase(corev1api.ClaimPending).
Result(),
resourcePolicies: &resourcepolicies.ResourcePolicies{
Version: "v1",
VolumePolicies: []resourcepolicies.VolumePolicy{
{
Conditions: map[string]any{
"storageClass": []string{"gp2-csi"},
},
Action: resourcepolicies.Action{
Type: resourcepolicies.Skip,
},
},
},
},
shouldSnapshot: false,
expectedErr: true,
},
{
name: "Lost PVC with phase-based skip policy should not error and return false",
inputPVC: builder.ForPersistentVolumeClaim("ns", "pvc-lost").
StorageClass("some-class").
Phase(corev1api.ClaimLost).
Result(),
resourcePolicies: &resourcepolicies.ResourcePolicies{
Version: "v1",
VolumePolicies: []resourcepolicies.VolumePolicy{
{
Conditions: map[string]any{
"pvcPhase": []string{"Lost"},
},
Action: resourcepolicies.Action{
Type: resourcepolicies.Skip,
},
},
},
},
shouldSnapshot: false,
expectedErr: false,
},
{
name: "Lost PVC with policy for Pending and Lost should not error and return false",
inputPVC: builder.ForPersistentVolumeClaim("ns", "pvc-lost").
StorageClass("some-class").
Phase(corev1api.ClaimLost).
Result(),
resourcePolicies: &resourcepolicies.ResourcePolicies{
Version: "v1",
VolumePolicies: []resourcepolicies.VolumePolicy{
{
Conditions: map[string]any{
"pvcPhase": []string{"Pending", "Lost"},
},
Action: resourcepolicies.Action{
Type: resourcepolicies.Skip,
},
},
},
},
shouldSnapshot: false,
expectedErr: false,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
fakeClient := velerotest.NewFakeControllerRuntimeClient(t)
var p *resourcepolicies.Policies
if tc.resourcePolicies != nil {
p = &resourcepolicies.Policies{}
err := p.BuildPolicy(tc.resourcePolicies)
require.NoError(t, err)
}
vh := NewVolumeHelperImpl(
p,
ptr.To(true),
logrus.StandardLogger(),
fakeClient,
false,
false,
)
obj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.inputPVC)
require.NoError(t, err)
actualShouldSnapshot, actualError := vh.ShouldPerformSnapshot(&unstructured.Unstructured{Object: obj}, kuberesource.PersistentVolumeClaims)
if tc.expectedErr {
require.Error(t, actualError, "Want error; Got nil error")
return
}
require.NoError(t, actualError)
require.Equalf(t, tc.shouldSnapshot, actualShouldSnapshot, "Want shouldSnapshot as %t; Got shouldSnapshot as %t", tc.shouldSnapshot, actualShouldSnapshot)
})
}
}
// TestVolumeHelperImpl_ShouldPerformFSBackup_UnboundPVC tests that Pending and Lost PVCs with
// phase-based skip policies don't cause errors when GetPVForPVC would fail.
func TestVolumeHelperImpl_ShouldPerformFSBackup_UnboundPVC(t *testing.T) {
testCases := []struct {
name string
pod *corev1api.Pod
pvc *corev1api.PersistentVolumeClaim
resourcePolicies *resourcepolicies.ResourcePolicies
shouldFSBackup bool
expectedErr bool
}{
{
name: "Pending PVC with phase-based skip policy should not error and return false",
pod: builder.ForPod("ns", "pod-1").
Volumes(
&corev1api.Volume{
Name: "vol-pending",
VolumeSource: corev1api.VolumeSource{
PersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{
ClaimName: "pvc-pending",
},
},
}).Result(),
pvc: builder.ForPersistentVolumeClaim("ns", "pvc-pending").
StorageClass("non-existent-class").
Phase(corev1api.ClaimPending).
Result(),
resourcePolicies: &resourcepolicies.ResourcePolicies{
Version: "v1",
VolumePolicies: []resourcepolicies.VolumePolicy{
{
Conditions: map[string]any{
"pvcPhase": []string{"Pending"},
},
Action: resourcepolicies.Action{
Type: resourcepolicies.Skip,
},
},
},
},
shouldFSBackup: false,
expectedErr: false,
},
{
name: "Pending PVC without matching skip policy should error (no PV)",
pod: builder.ForPod("ns", "pod-1").
Volumes(
&corev1api.Volume{
Name: "vol-pending",
VolumeSource: corev1api.VolumeSource{
PersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{
ClaimName: "pvc-pending-no-policy",
},
},
}).Result(),
pvc: builder.ForPersistentVolumeClaim("ns", "pvc-pending-no-policy").
StorageClass("non-existent-class").
Phase(corev1api.ClaimPending).
Result(),
resourcePolicies: &resourcepolicies.ResourcePolicies{
Version: "v1",
VolumePolicies: []resourcepolicies.VolumePolicy{
{
Conditions: map[string]any{
"storageClass": []string{"gp2-csi"},
},
Action: resourcepolicies.Action{
Type: resourcepolicies.Skip,
},
},
},
},
shouldFSBackup: false,
expectedErr: true,
},
{
name: "Lost PVC with phase-based skip policy should not error and return false",
pod: builder.ForPod("ns", "pod-1").
Volumes(
&corev1api.Volume{
Name: "vol-lost",
VolumeSource: corev1api.VolumeSource{
PersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{
ClaimName: "pvc-lost",
},
},
}).Result(),
pvc: builder.ForPersistentVolumeClaim("ns", "pvc-lost").
StorageClass("some-class").
Phase(corev1api.ClaimLost).
Result(),
resourcePolicies: &resourcepolicies.ResourcePolicies{
Version: "v1",
VolumePolicies: []resourcepolicies.VolumePolicy{
{
Conditions: map[string]any{
"pvcPhase": []string{"Lost"},
},
Action: resourcepolicies.Action{
Type: resourcepolicies.Skip,
},
},
},
},
shouldFSBackup: false,
expectedErr: false,
},
{
name: "Lost PVC with policy for Pending and Lost should not error and return false",
pod: builder.ForPod("ns", "pod-1").
Volumes(
&corev1api.Volume{
Name: "vol-lost",
VolumeSource: corev1api.VolumeSource{
PersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{
ClaimName: "pvc-lost",
},
},
}).Result(),
pvc: builder.ForPersistentVolumeClaim("ns", "pvc-lost").
StorageClass("some-class").
Phase(corev1api.ClaimLost).
Result(),
resourcePolicies: &resourcepolicies.ResourcePolicies{
Version: "v1",
VolumePolicies: []resourcepolicies.VolumePolicy{
{
Conditions: map[string]any{
"pvcPhase": []string{"Pending", "Lost"},
},
Action: resourcepolicies.Action{
Type: resourcepolicies.Skip,
},
},
},
},
shouldFSBackup: false,
expectedErr: false,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
fakeClient := velerotest.NewFakeControllerRuntimeClient(t, tc.pvc)
require.NoError(t, fakeClient.Create(t.Context(), tc.pod))
var p *resourcepolicies.Policies
if tc.resourcePolicies != nil {
p = &resourcepolicies.Policies{}
err := p.BuildPolicy(tc.resourcePolicies)
require.NoError(t, err)
}
vh := NewVolumeHelperImpl(
p,
ptr.To(true),
logrus.StandardLogger(),
fakeClient,
false,
false,
)
actualShouldFSBackup, actualError := vh.ShouldPerformFSBackup(tc.pod.Spec.Volumes[0], *tc.pod)
if tc.expectedErr {
require.Error(t, actualError, "Want error; Got nil error")
return
}
require.NoError(t, actualError)
require.Equalf(t, tc.shouldFSBackup, actualShouldFSBackup, "Want shouldFSBackup as %t; Got shouldFSBackup as %t", tc.shouldFSBackup, actualShouldFSBackup)
})
}
}

View File

@@ -687,15 +687,14 @@ func (ib *itemBackupper) getMatchAction(obj runtime.Unstructured, groupResource
return nil, errors.WithStack(err)
}
pvName := pvc.Spec.VolumeName
if pvName == "" {
return nil, errors.Errorf("PVC has no volume backing this claim")
}
pv := &corev1api.PersistentVolume{}
if err := ib.kbClient.Get(context.Background(), kbClient.ObjectKey{Name: pvName}, pv); err != nil {
return nil, errors.WithStack(err)
var pv *corev1api.PersistentVolume
if pvName := pvc.Spec.VolumeName; pvName != "" {
pv = &corev1api.PersistentVolume{}
if err := ib.kbClient.Get(context.Background(), kbClient.ObjectKey{Name: pvName}, pv); err != nil {
return nil, errors.WithStack(err)
}
}
// If pv is nil for unbound PVCs - policy matching will use PVC-only conditions
vfd := resourcepolicies.NewVolumeFilterData(pv, nil, pvc)
return ib.backupRequest.ResPolicies.GetMatchAction(vfd)
}
@@ -709,7 +708,10 @@ func (ib *itemBackupper) trackSkippedPV(obj runtime.Unstructured, groupResource
if name, err := getPVName(obj, groupResource); len(name) > 0 && err == nil {
ib.backupRequest.SkippedPVTracker.Track(name, approach, reason)
} else if err != nil {
log.WithError(err).Warnf("unable to get PV name, skip tracking.")
// Log at info level for tracking purposes. This is not an error because
// it's expected for some resources (e.g., PVCs in Pending or Lost phase)
// to not have a PV name. This occurs when volume policy skips unbound PVCs.
log.WithError(err).Infof("unable to get PV name, skip tracking.")
}
}
@@ -719,6 +721,17 @@ func (ib *itemBackupper) unTrackSkippedPV(obj runtime.Unstructured, groupResourc
if name, err := getPVName(obj, groupResource); len(name) > 0 && err == nil {
ib.backupRequest.SkippedPVTracker.Untrack(name)
} else if err != nil {
// For PVCs in Pending or Lost phase, it's expected that there's no PV name.
// Log at debug level instead of warning to reduce noise.
if groupResource == kuberesource.PersistentVolumeClaims {
pvc := new(corev1api.PersistentVolumeClaim)
if convErr := runtime.DefaultUnstructuredConverter.FromUnstructured(obj.UnstructuredContent(), pvc); convErr == nil {
if pvc.Status.Phase == corev1api.ClaimPending || pvc.Status.Phase == corev1api.ClaimLost {
log.WithError(err).Debugf("unable to get PV name for %s PVC, skip untracking.", pvc.Status.Phase)
return
}
}
}
log.WithError(err).Warnf("unable to get PV name, skip untracking.")
}
}

View File

@@ -17,12 +17,15 @@ limitations under the License.
package backup
import (
"bytes"
"testing"
"github.com/sirupsen/logrus"
"github.com/stretchr/testify/require"
"k8s.io/apimachinery/pkg/runtime/schema"
ctrlfake "sigs.k8s.io/controller-runtime/pkg/client/fake"
"github.com/vmware-tanzu/velero/internal/resourcepolicies"
"github.com/vmware-tanzu/velero/pkg/kuberesource"
"github.com/stretchr/testify/assert"
@@ -269,3 +272,225 @@ func TestAddVolumeInfo(t *testing.T) {
})
}
}
func TestGetMatchAction_PendingLostPVC(t *testing.T) {
scheme := runtime.NewScheme()
require.NoError(t, corev1api.AddToScheme(scheme))
// Create resource policies that skip Pending/Lost PVCs
resPolicies := &resourcepolicies.ResourcePolicies{
Version: "v1",
VolumePolicies: []resourcepolicies.VolumePolicy{
{
Conditions: map[string]any{
"pvcPhase": []string{"Pending", "Lost"},
},
Action: resourcepolicies.Action{
Type: resourcepolicies.Skip,
},
},
},
}
policies := &resourcepolicies.Policies{}
err := policies.BuildPolicy(resPolicies)
require.NoError(t, err)
testCases := []struct {
name string
pvc *corev1api.PersistentVolumeClaim
pv *corev1api.PersistentVolume
expectedAction *resourcepolicies.Action
expectError bool
}{
{
name: "Pending PVC with no VolumeName should match pvcPhase policy",
pvc: builder.ForPersistentVolumeClaim("ns", "pending-pvc").
StorageClass("test-sc").
Phase(corev1api.ClaimPending).
Result(),
pv: nil,
expectedAction: &resourcepolicies.Action{Type: resourcepolicies.Skip},
expectError: false,
},
{
name: "Lost PVC with no VolumeName should match pvcPhase policy",
pvc: builder.ForPersistentVolumeClaim("ns", "lost-pvc").
StorageClass("test-sc").
Phase(corev1api.ClaimLost).
Result(),
pv: nil,
expectedAction: &resourcepolicies.Action{Type: resourcepolicies.Skip},
expectError: false,
},
{
name: "Bound PVC with VolumeName and matching PV should not match pvcPhase policy",
pvc: builder.ForPersistentVolumeClaim("ns", "bound-pvc").
StorageClass("test-sc").
VolumeName("test-pv").
Phase(corev1api.ClaimBound).
Result(),
pv: builder.ForPersistentVolume("test-pv").StorageClass("test-sc").Result(),
expectedAction: nil,
expectError: false,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
// Build fake client with PV if present
clientBuilder := ctrlfake.NewClientBuilder().WithScheme(scheme)
if tc.pv != nil {
clientBuilder = clientBuilder.WithObjects(tc.pv)
}
fakeClient := clientBuilder.Build()
ib := &itemBackupper{
kbClient: fakeClient,
backupRequest: &Request{
ResPolicies: policies,
},
}
// Convert PVC to unstructured
pvcData, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.pvc)
require.NoError(t, err)
obj := &unstructured.Unstructured{Object: pvcData}
action, err := ib.getMatchAction(obj, kuberesource.PersistentVolumeClaims, csiBIAPluginName)
if tc.expectError {
require.Error(t, err)
} else {
require.NoError(t, err)
}
if tc.expectedAction == nil {
assert.Nil(t, action)
} else {
require.NotNil(t, action)
assert.Equal(t, tc.expectedAction.Type, action.Type)
}
})
}
}
func TestTrackSkippedPV_PendingLostPVC(t *testing.T) {
testCases := []struct {
name string
pvc *corev1api.PersistentVolumeClaim
}{
{
name: "Pending PVC should log at info level",
pvc: builder.ForPersistentVolumeClaim("ns", "pending-pvc").
Phase(corev1api.ClaimPending).
Result(),
},
{
name: "Lost PVC should log at info level",
pvc: builder.ForPersistentVolumeClaim("ns", "lost-pvc").
Phase(corev1api.ClaimLost).
Result(),
},
{
name: "Bound PVC without VolumeName should log at info level",
pvc: builder.ForPersistentVolumeClaim("ns", "bound-pvc").
Phase(corev1api.ClaimBound).
Result(),
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
ib := &itemBackupper{
backupRequest: &Request{
SkippedPVTracker: NewSkipPVTracker(),
},
}
// Set up log capture
logOutput := &bytes.Buffer{}
logger := logrus.New()
logger.SetOutput(logOutput)
logger.SetLevel(logrus.DebugLevel)
// Convert PVC to unstructured
pvcData, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.pvc)
require.NoError(t, err)
obj := &unstructured.Unstructured{Object: pvcData}
ib.trackSkippedPV(obj, kuberesource.PersistentVolumeClaims, "", "test reason", logger)
logStr := logOutput.String()
assert.Contains(t, logStr, "level=info")
assert.Contains(t, logStr, "unable to get PV name, skip tracking.")
})
}
}
func TestUnTrackSkippedPV_PendingLostPVC(t *testing.T) {
testCases := []struct {
name string
pvc *corev1api.PersistentVolumeClaim
expectWarningLog bool
expectDebugMessage string
}{
{
name: "Pending PVC should log at debug level, not warning",
pvc: builder.ForPersistentVolumeClaim("ns", "pending-pvc").
Phase(corev1api.ClaimPending).
Result(),
expectWarningLog: false,
expectDebugMessage: "unable to get PV name for Pending PVC, skip untracking.",
},
{
name: "Lost PVC should log at debug level, not warning",
pvc: builder.ForPersistentVolumeClaim("ns", "lost-pvc").
Phase(corev1api.ClaimLost).
Result(),
expectWarningLog: false,
expectDebugMessage: "unable to get PV name for Lost PVC, skip untracking.",
},
{
name: "Bound PVC without VolumeName should log warning",
pvc: builder.ForPersistentVolumeClaim("ns", "bound-pvc").
Phase(corev1api.ClaimBound).
Result(),
expectWarningLog: true,
expectDebugMessage: "",
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
ib := &itemBackupper{
backupRequest: &Request{
SkippedPVTracker: NewSkipPVTracker(),
},
}
// Set up log capture
logOutput := &bytes.Buffer{}
logger := logrus.New()
logger.SetOutput(logOutput)
logger.SetLevel(logrus.DebugLevel)
// Convert PVC to unstructured
pvcData, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.pvc)
require.NoError(t, err)
obj := &unstructured.Unstructured{Object: pvcData}
ib.unTrackSkippedPV(obj, kuberesource.PersistentVolumeClaims, logger)
logStr := logOutput.String()
if tc.expectWarningLog {
assert.Contains(t, logStr, "level=warning")
assert.Contains(t, logStr, "unable to get PV name, skip untracking.")
} else {
assert.NotContains(t, logStr, "level=warning")
if tc.expectDebugMessage != "" {
assert.Contains(t, logStr, "level=debug")
assert.Contains(t, logStr, tc.expectDebugMessage)
}
}
})
}
}

View File

@@ -210,11 +210,9 @@ func resultsKey(ns, name string) string {
func (b *backupper) getMatchAction(resPolicies *resourcepolicies.Policies, pvc *corev1api.PersistentVolumeClaim, volume *corev1api.Volume) (*resourcepolicies.Action, error) {
if pvc != nil {
pv := new(corev1api.PersistentVolume)
err := b.crClient.Get(context.TODO(), ctrlclient.ObjectKey{Name: pvc.Spec.VolumeName}, pv)
if err != nil {
return nil, errors.Wrapf(err, "error getting pv for pvc %s", pvc.Spec.VolumeName)
}
// Ignore err, if the PV is not available (Pending/Lost PVC or PV fetch failed) - try matching with PVC only
// GetPVForPVC returns nil for all error cases
pv, _ := kube.GetPVForPVC(pvc, b.crClient)
vfd := resourcepolicies.NewVolumeFilterData(pv, nil, pvc)
return resPolicies.GetMatchAction(vfd)
}

View File

@@ -309,8 +309,8 @@ func createNodeObj() *corev1api.Node {
func TestBackupPodVolumes(t *testing.T) {
scheme := runtime.NewScheme()
velerov1api.AddToScheme(scheme)
corev1api.AddToScheme(scheme)
require.NoError(t, velerov1api.AddToScheme(scheme))
require.NoError(t, corev1api.AddToScheme(scheme))
log := logrus.New()
tests := []struct {
@@ -778,7 +778,7 @@ func TestWaitAllPodVolumesProcessed(t *testing.T) {
backuper := newBackupper(c.ctx, log, nil, nil, informer, nil, "", &velerov1api.Backup{})
if c.pvb != nil {
backuper.pvbIndexer.Add(c.pvb)
require.NoError(t, backuper.pvbIndexer.Add(c.pvb))
backuper.wg.Add(1)
}
@@ -833,3 +833,185 @@ func TestPVCBackupSummary(t *testing.T) {
assert.Empty(t, pbs.Skipped)
assert.Len(t, pbs.Backedup, 2)
}
func TestGetMatchAction_PendingPVC(t *testing.T) {
// Create resource policies that skip Pending/Lost PVCs
resPolicies := &resourcepolicies.ResourcePolicies{
Version: "v1",
VolumePolicies: []resourcepolicies.VolumePolicy{
{
Conditions: map[string]any{
"pvcPhase": []string{"Pending", "Lost"},
},
Action: resourcepolicies.Action{
Type: resourcepolicies.Skip,
},
},
},
}
policies := &resourcepolicies.Policies{}
err := policies.BuildPolicy(resPolicies)
require.NoError(t, err)
testCases := []struct {
name string
pvc *corev1api.PersistentVolumeClaim
volume *corev1api.Volume
pv *corev1api.PersistentVolume
expectedAction *resourcepolicies.Action
expectError bool
}{
{
name: "Pending PVC with pvcPhase skip policy should return skip action",
pvc: builder.ForPersistentVolumeClaim("ns", "pending-pvc").
StorageClass("test-sc").
Phase(corev1api.ClaimPending).
Result(),
volume: &corev1api.Volume{
Name: "test-volume",
VolumeSource: corev1api.VolumeSource{
PersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{
ClaimName: "pending-pvc",
},
},
},
pv: nil,
expectedAction: &resourcepolicies.Action{Type: resourcepolicies.Skip},
expectError: false,
},
{
name: "Lost PVC with pvcPhase skip policy should return skip action",
pvc: builder.ForPersistentVolumeClaim("ns", "lost-pvc").
StorageClass("test-sc").
Phase(corev1api.ClaimLost).
Result(),
volume: &corev1api.Volume{
Name: "test-volume",
VolumeSource: corev1api.VolumeSource{
PersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{
ClaimName: "lost-pvc",
},
},
},
pv: nil,
expectedAction: &resourcepolicies.Action{Type: resourcepolicies.Skip},
expectError: false,
},
{
name: "Bound PVC with matching PV should not match pvcPhase policy",
pvc: builder.ForPersistentVolumeClaim("ns", "bound-pvc").
StorageClass("test-sc").
VolumeName("test-pv").
Phase(corev1api.ClaimBound).
Result(),
volume: &corev1api.Volume{
Name: "test-volume",
VolumeSource: corev1api.VolumeSource{
PersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{
ClaimName: "bound-pvc",
},
},
},
pv: builder.ForPersistentVolume("test-pv").StorageClass("test-sc").Result(),
expectedAction: nil,
expectError: false,
},
{
name: "Pending PVC with no matching policy should return nil action",
pvc: builder.ForPersistentVolumeClaim("ns", "pending-pvc-no-match").
StorageClass("test-sc").
Phase(corev1api.ClaimPending).
Result(),
volume: &corev1api.Volume{
Name: "test-volume",
VolumeSource: corev1api.VolumeSource{
PersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{
ClaimName: "pending-pvc-no-match",
},
},
},
pv: nil,
expectedAction: &resourcepolicies.Action{Type: resourcepolicies.Skip}, // Will match the pvcPhase policy
expectError: false,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
// Build fake client with PV if present
var objs []runtime.Object
if tc.pv != nil {
objs = append(objs, tc.pv)
}
fakeClient := velerotest.NewFakeControllerRuntimeClient(t, objs...)
b := &backupper{
crClient: fakeClient,
}
action, err := b.getMatchAction(policies, tc.pvc, tc.volume)
if tc.expectError {
require.Error(t, err)
} else {
require.NoError(t, err)
}
if tc.expectedAction == nil {
assert.Nil(t, action)
} else {
require.NotNil(t, action)
assert.Equal(t, tc.expectedAction.Type, action.Type)
}
})
}
}
func TestGetMatchAction_PVCWithoutPVLookupError(t *testing.T) {
// Test that when a PVC has a VolumeName but the PV doesn't exist,
// the function ignores the error and tries to match with PVC only
resPolicies := &resourcepolicies.ResourcePolicies{
Version: "v1",
VolumePolicies: []resourcepolicies.VolumePolicy{
{
Conditions: map[string]any{
"pvcPhase": []string{"Pending"},
},
Action: resourcepolicies.Action{
Type: resourcepolicies.Skip,
},
},
},
}
policies := &resourcepolicies.Policies{}
err := policies.BuildPolicy(resPolicies)
require.NoError(t, err)
// Pending PVC without a matching PV in the cluster
pvc := builder.ForPersistentVolumeClaim("ns", "pending-pvc").
StorageClass("test-sc").
Phase(corev1api.ClaimPending).
Result()
volume := &corev1api.Volume{
Name: "test-volume",
VolumeSource: corev1api.VolumeSource{
PersistentVolumeClaim: &corev1api.PersistentVolumeClaimVolumeSource{
ClaimName: "pending-pvc",
},
},
}
// Empty client - no PV exists
fakeClient := velerotest.NewFakeControllerRuntimeClient(t)
b := &backupper{
crClient: fakeClient,
}
// Should succeed even though PV lookup would fail
// because the function ignores PV lookup errors and uses PVC-only matching
action, err := b.getMatchAction(policies, pvc, volume)
require.NoError(t, err)
require.NotNil(t, action)
assert.Equal(t, resourcepolicies.Skip, action.Type)
}

View File

@@ -666,10 +666,22 @@ func validateNamespaceName(ns string) []error {
return nil
}
// Kubernetes does not allow asterisks in namespaces but Velero uses them as
// wildcards. Replace asterisks with an arbitrary letter to pass Kubernetes
// validation.
tmpNamespace := strings.ReplaceAll(ns, "*", "x")
// Validate the namespace name to ensure it is a valid wildcard pattern
if err := wildcard.ValidateNamespaceName(ns); err != nil {
return []error{err}
}
// Kubernetes does not allow wildcard characters in namespaces but Velero uses them
// for glob patterns. Replace wildcard characters with valid characters to pass
// Kubernetes validation.
tmpNamespace := ns
// Replace glob wildcard characters with valid alphanumeric characters
// Note: Validation of wildcard patterns is handled by the wildcard package.
tmpNamespace = strings.ReplaceAll(tmpNamespace, "*", "x") // matches any sequence
tmpNamespace = strings.ReplaceAll(tmpNamespace, "?", "x") // matches single character
tmpNamespace = strings.ReplaceAll(tmpNamespace, "[", "x") // character class start
tmpNamespace = strings.ReplaceAll(tmpNamespace, "]", "x") // character class end
if errMsgs := validation.ValidateNamespaceName(tmpNamespace, false); errMsgs != nil {
for _, msg := range errMsgs {

View File

@@ -289,6 +289,54 @@ func TestValidateNamespaceIncludesExcludes(t *testing.T) {
excludes: []string{"bar"},
wantErr: true,
},
{
name: "glob characters in includes should not error",
includes: []string{"kube-*", "test-?", "ns-[0-9]"},
excludes: []string{},
wantErr: false,
},
{
name: "glob characters in excludes should not error",
includes: []string{"default"},
excludes: []string{"test-*", "app-?", "ns-[1-5]"},
wantErr: false,
},
{
name: "character class in includes should not error",
includes: []string{"ns-[abc]", "test-[0-9]"},
excludes: []string{},
wantErr: false,
},
{
name: "mixed glob patterns should not error",
includes: []string{"kube-*", "test-?"},
excludes: []string{"*-test", "debug-[0-9]"},
wantErr: false,
},
{
name: "pipe character in includes should error",
includes: []string{"namespace|other"},
excludes: []string{},
wantErr: true,
},
{
name: "parentheses in includes should error",
includes: []string{"namespace(prod)", "test-(dev)"},
excludes: []string{},
wantErr: true,
},
{
name: "exclamation mark in includes should error",
includes: []string{"!namespace", "test!"},
excludes: []string{},
wantErr: true,
},
{
name: "unsupported characters in excludes should error",
includes: []string{"default"},
excludes: []string{"test|prod", "app(staging)"},
wantErr: true,
},
}
for _, tc := range tests {
@@ -1082,16 +1130,6 @@ func TestExpandIncludesExcludes(t *testing.T) {
expectedWildcardExpanded: true,
expectError: false,
},
{
name: "brace wildcard pattern",
includes: []string{"app-{prod,dev}"},
excludes: []string{},
activeNamespaces: []string{"app-prod", "app-dev", "app-test", "default"},
expectedIncludes: []string{"app-prod", "app-dev"},
expectedExcludes: []string{},
expectedWildcardExpanded: true,
expectError: false,
},
{
name: "empty activeNamespaces with wildcards",
includes: []string{"kube-*"},
@@ -1233,13 +1271,6 @@ func TestResolveNamespaceList(t *testing.T) {
expectedNamespaces: []string{"kube-system", "kube-public"},
preExpandWildcards: true,
},
{
name: "complex wildcard pattern",
includes: []string{"app-{prod,dev}", "kube-*"},
excludes: []string{"*-test"},
activeNamespaces: []string{"app-prod", "app-dev", "app-test", "kube-system", "kube-test", "default"},
expectedNamespaces: []string{"app-prod", "app-dev", "kube-system"},
},
{
name: "question mark wildcard pattern",
includes: []string{"ns-?"},

View File

@@ -417,19 +417,19 @@ func MakePodPVCAttachment(volumeName string, volumeMode *corev1api.PersistentVol
return volumeMounts, volumeDevices, volumePath
}
// GetPVForPVC returns the PersistentVolume backing a PVC
// returns PV, error.
// PV will be nil on error
func GetPVForPVC(
pvc *corev1api.PersistentVolumeClaim,
crClient crclient.Client,
) (*corev1api.PersistentVolume, error) {
if pvc.Spec.VolumeName == "" {
return nil, errors.Errorf("PVC %s/%s has no volume backing this claim",
pvc.Namespace, pvc.Name)
return nil, errors.Errorf("PVC %s/%s has no volume backing this claim", pvc.Namespace, pvc.Name)
}
if pvc.Status.Phase != corev1api.ClaimBound {
// TODO: confirm if this PVC should be snapshotted if it has no PV bound
return nil,
errors.Errorf("PVC %s/%s is in phase %v and is not bound to a volume",
pvc.Namespace, pvc.Name, pvc.Status.Phase)
return nil, errors.Errorf("PVC %s/%s is in phase %v and is not bound to a volume",
pvc.Namespace, pvc.Name, pvc.Status.Phase)
}
pv := &corev1api.PersistentVolume{}

View File

@@ -31,70 +31,77 @@ func ShouldExpandWildcards(includes []string, excludes []string) bool {
}
// containsWildcardPattern checks if a pattern contains any wildcard symbols
// Supported patterns: *, ?, [abc], {a,b,c}
// Supported patterns: *, ?, [abc]
// Note: . and + are treated as literal characters (not wildcards)
// Note: ** and consecutive asterisks are NOT supported (will cause validation error)
func containsWildcardPattern(pattern string) bool {
return strings.ContainsAny(pattern, "*?[{")
return strings.ContainsAny(pattern, "*?[")
}
func validateWildcardPatterns(patterns []string) error {
for _, pattern := range patterns {
// Check for invalid regex-only patterns that we don't support
if strings.ContainsAny(pattern, "|()") {
return errors.New("wildcard pattern contains unsupported regex symbols: |, (, )")
}
// Check for consecutive asterisks (2 or more)
if strings.Contains(pattern, "**") {
return errors.New("wildcard pattern contains consecutive asterisks (only single * allowed)")
}
// Check for malformed brace patterns
if err := validateBracePatterns(pattern); err != nil {
if err := ValidateNamespaceName(pattern); err != nil {
return err
}
}
return nil
}
func ValidateNamespaceName(pattern string) error {
// Check for invalid characters that are not supported in glob patterns
if strings.ContainsAny(pattern, "|()!{},") {
return errors.New("wildcard pattern contains unsupported characters: |, (, ), !, {, }, ,")
}
// Check for consecutive asterisks (2 or more)
if strings.Contains(pattern, "**") {
return errors.New("wildcard pattern contains consecutive asterisks (only single * allowed)")
}
// Check for malformed brace patterns
if err := validateBracePatterns(pattern); err != nil {
return err
}
return nil
}
// validateBracePatterns checks for malformed brace patterns like unclosed braces or empty braces
// Also validates bracket patterns [] for character classes
func validateBracePatterns(pattern string) error {
depth := 0
bracketDepth := 0
for i := 0; i < len(pattern); i++ {
if pattern[i] == '{' {
braceStart := i
depth++
if pattern[i] == '[' {
bracketStart := i
bracketDepth++
// Scan ahead to find the matching closing brace and validate content
for j := i + 1; j < len(pattern) && depth > 0; j++ {
if pattern[j] == '{' {
depth++
} else if pattern[j] == '}' {
depth--
if depth == 0 {
// Found matching closing brace - validate content
content := pattern[braceStart+1 : j]
if strings.Trim(content, ", \t") == "" {
return errors.New("wildcard pattern contains empty brace pattern '{}'")
// Scan ahead to find the matching closing bracket and validate content
for j := i + 1; j < len(pattern) && bracketDepth > 0; j++ {
if pattern[j] == ']' {
bracketDepth--
if bracketDepth == 0 {
// Found matching closing bracket - validate content
content := pattern[bracketStart+1 : j]
if content == "" {
return errors.New("wildcard pattern contains empty bracket pattern '[]'")
}
// Skip to the closing brace
// Skip to the closing bracket
i = j
break
}
}
}
// If we exited the loop without finding a match (depth > 0), brace is unclosed
if depth > 0 {
return errors.New("wildcard pattern contains unclosed brace '{'")
// If we exited the loop without finding a match (bracketDepth > 0), bracket is unclosed
if bracketDepth > 0 {
return errors.New("wildcard pattern contains unclosed bracket '['")
}
// i is now positioned at the closing brace; the outer loop will increment it
} else if pattern[i] == '}' {
// Found a closing brace without a matching opening brace
return errors.New("wildcard pattern contains unmatched closing brace '}'")
// i is now positioned at the closing bracket; the outer loop will increment it
} else if pattern[i] == ']' {
// Found a closing bracket without a matching opening bracket
return errors.New("wildcard pattern contains unmatched closing bracket ']'")
}
}

View File

@@ -90,7 +90,7 @@ func TestShouldExpandWildcards(t *testing.T) {
name: "brace alternatives wildcard",
includes: []string{"ns{prod,staging}"},
excludes: []string{},
expected: true, // brace alternatives are considered wildcard
expected: false, // brace alternatives are not supported
},
{
name: "dot is literal - not wildcard",
@@ -237,9 +237,9 @@ func TestExpandWildcards(t *testing.T) {
activeNamespaces: []string{"app-prod", "app-staging", "app-dev", "db-prod"},
includes: []string{"app-{prod,staging}"},
excludes: []string{},
expectedIncludes: []string{"app-prod", "app-staging"}, // {prod,staging} matches either
expectedIncludes: nil,
expectedExcludes: nil,
expectError: false,
expectError: true,
},
{
name: "literal dot and plus patterns",
@@ -259,33 +259,6 @@ func TestExpandWildcards(t *testing.T) {
expectedExcludes: nil,
expectError: true, // |, (, ) are not supported
},
{
name: "unclosed brace patterns should error",
activeNamespaces: []string{"app-prod"},
includes: []string{"app-{prod,staging"},
excludes: []string{},
expectedIncludes: nil,
expectedExcludes: nil,
expectError: true, // unclosed brace
},
{
name: "empty brace patterns should error",
activeNamespaces: []string{"app-prod"},
includes: []string{"app-{}"},
excludes: []string{},
expectedIncludes: nil,
expectedExcludes: nil,
expectError: true, // empty braces
},
{
name: "unmatched closing brace should error",
activeNamespaces: []string{"app-prod"},
includes: []string{"app-prod}"},
excludes: []string{},
expectedIncludes: nil,
expectedExcludes: nil,
expectError: true, // unmatched closing brace
},
}
for _, tt := range tests {
@@ -354,13 +327,6 @@ func TestExpandWildcardsPrivate(t *testing.T) {
expected: []string{}, // returns empty slice, not nil
expectError: false,
},
{
name: "brace patterns work correctly",
patterns: []string{"app-{prod,staging}"},
activeNamespaces: []string{"app-prod", "app-staging", "app-dev", "app-{prod,staging}"},
expected: []string{"app-prod", "app-staging"}, // brace patterns do expand
expectError: false,
},
{
name: "duplicate matches from multiple patterns",
patterns: []string{"app-*", "*-prod"},
@@ -389,20 +355,6 @@ func TestExpandWildcardsPrivate(t *testing.T) {
expected: []string{"nsa", "nsb", "nsc"}, // [a-c] matches a to c
expectError: false,
},
{
name: "negated character class",
patterns: []string{"ns[!abc]"},
activeNamespaces: []string{"nsa", "nsb", "nsc", "nsd", "ns1"},
expected: []string{"nsd", "ns1"}, // [!abc] matches anything except a, b, c
expectError: false,
},
{
name: "brace alternatives",
patterns: []string{"app-{prod,test}"},
activeNamespaces: []string{"app-prod", "app-test", "app-staging", "db-prod"},
expected: []string{"app-prod", "app-test"}, // {prod,test} matches either
expectError: false,
},
{
name: "double asterisk should error",
patterns: []string{"**"},
@@ -410,13 +362,6 @@ func TestExpandWildcardsPrivate(t *testing.T) {
expected: nil,
expectError: true, // ** is not allowed
},
{
name: "literal dot and plus",
patterns: []string{"app.prod", "service+"},
activeNamespaces: []string{"app.prod", "appXprod", "service+", "service"},
expected: []string{"app.prod", "service+"}, // . and + are literal
expectError: false,
},
{
name: "unsupported regex symbols should error",
patterns: []string{"ns(1|2)"},
@@ -468,153 +413,101 @@ func TestValidateBracePatterns(t *testing.T) {
expectError bool
errorMsg string
}{
// Valid patterns
// Valid square bracket patterns
{
name: "valid single brace pattern",
pattern: "app-{prod,staging}",
name: "valid square bracket pattern",
pattern: "ns[abc]",
expectError: false,
},
{
name: "valid brace with single option",
pattern: "app-{prod}",
name: "valid square bracket pattern with range",
pattern: "ns[a-z]",
expectError: false,
},
{
name: "valid brace with three options",
pattern: "app-{prod,staging,dev}",
name: "valid square bracket pattern with numbers",
pattern: "ns[0-9]",
expectError: false,
},
{
name: "valid pattern with text before and after brace",
pattern: "prefix-{a,b}-suffix",
name: "valid square bracket pattern with mixed",
pattern: "ns[a-z0-9]",
expectError: false,
},
{
name: "valid pattern with no braces",
pattern: "app-prod",
name: "valid square bracket pattern with single character",
pattern: "ns[a]",
expectError: false,
},
{
name: "valid pattern with asterisk",
pattern: "app-*",
name: "valid square bracket pattern with text before and after",
pattern: "prefix-[abc]-suffix",
expectError: false,
},
// Unclosed opening brackets
{
name: "valid brace with spaces around content",
pattern: "app-{ prod , staging }",
expectError: false,
name: "unclosed opening bracket at end",
pattern: "ns[abc",
expectError: true,
errorMsg: "unclosed bracket",
},
{
name: "valid brace with numbers",
pattern: "ns-{1,2,3}",
expectError: false,
name: "unclosed opening bracket at start",
pattern: "[abc",
expectError: true,
errorMsg: "unclosed bracket",
},
{
name: "valid brace with hyphens in options",
pattern: "{app-prod,db-staging}",
expectError: false,
name: "unclosed opening bracket in middle",
pattern: "ns[abc-test",
expectError: true,
errorMsg: "unclosed bracket",
},
// Unclosed opening braces
// Unmatched closing brackets
{
name: "unclosed opening brace at end",
pattern: "app-{prod,staging",
name: "unmatched closing bracket at end",
pattern: "ns-abc]",
expectError: true,
errorMsg: "unclosed brace",
errorMsg: "unmatched closing bracket",
},
{
name: "unclosed opening brace at start",
pattern: "{prod,staging",
name: "unmatched closing bracket at start",
pattern: "]ns-abc",
expectError: true,
errorMsg: "unclosed brace",
errorMsg: "unmatched closing bracket",
},
{
name: "unclosed opening brace in middle",
pattern: "app-{prod-test",
name: "unmatched closing bracket in middle",
pattern: "ns-]abc",
expectError: true,
errorMsg: "unclosed brace",
errorMsg: "unmatched closing bracket",
},
{
name: "multiple unclosed braces",
pattern: "app-{prod-{staging",
name: "extra closing bracket after valid pair",
pattern: "ns[abc]]",
expectError: true,
errorMsg: "unclosed brace",
errorMsg: "unmatched closing bracket",
},
// Unmatched closing braces
// Empty bracket patterns
{
name: "unmatched closing brace at end",
pattern: "app-prod}",
name: "completely empty brackets",
pattern: "ns[]",
expectError: true,
errorMsg: "unmatched closing brace",
errorMsg: "empty bracket pattern",
},
{
name: "unmatched closing brace at start",
pattern: "}app-prod",
name: "empty brackets at start",
pattern: "[]ns",
expectError: true,
errorMsg: "unmatched closing brace",
errorMsg: "empty bracket pattern",
},
{
name: "unmatched closing brace in middle",
pattern: "app-}prod",
name: "empty brackets standalone",
pattern: "[]",
expectError: true,
errorMsg: "unmatched closing brace",
},
{
name: "extra closing brace after valid pair",
pattern: "app-{prod,staging}}",
expectError: true,
errorMsg: "unmatched closing brace",
},
// Empty brace patterns
{
name: "completely empty braces",
pattern: "app-{}",
expectError: true,
errorMsg: "empty brace pattern",
},
{
name: "braces with only spaces",
pattern: "app-{ }",
expectError: true,
errorMsg: "empty brace pattern",
},
{
name: "braces with only comma",
pattern: "app-{,}",
expectError: true,
errorMsg: "empty brace pattern",
},
{
name: "braces with only commas",
pattern: "app-{,,,}",
expectError: true,
errorMsg: "empty brace pattern",
},
{
name: "braces with commas and spaces",
pattern: "app-{ , , }",
expectError: true,
errorMsg: "empty brace pattern",
},
{
name: "braces with tabs and commas",
pattern: "app-{\t,\t}",
expectError: true,
errorMsg: "empty brace pattern",
},
{
name: "empty braces at start",
pattern: "{}app-prod",
expectError: true,
errorMsg: "empty brace pattern",
},
{
name: "empty braces standalone",
pattern: "{}",
expectError: true,
errorMsg: "empty brace pattern",
errorMsg: "empty bracket pattern",
},
// Edge cases
@@ -623,58 +516,6 @@ func TestValidateBracePatterns(t *testing.T) {
pattern: "",
expectError: false,
},
{
name: "pattern with only opening brace",
pattern: "{",
expectError: true,
errorMsg: "unclosed brace",
},
{
name: "pattern with only closing brace",
pattern: "}",
expectError: true,
errorMsg: "unmatched closing brace",
},
{
name: "valid brace with special characters inside",
pattern: "app-{prod-1,staging_2,dev.3}",
expectError: false,
},
{
name: "brace with asterisk inside option",
pattern: "app-{prod*,staging}",
expectError: false,
},
{
name: "multiple valid brace patterns",
pattern: "{app,db}-{prod,staging}",
expectError: false,
},
{
name: "brace with single character",
pattern: "app-{a}",
expectError: false,
},
{
name: "brace with trailing comma but has content",
pattern: "app-{prod,staging,}",
expectError: false, // Has content, so it's valid
},
{
name: "brace with leading comma but has content",
pattern: "app-{,prod,staging}",
expectError: false, // Has content, so it's valid
},
{
name: "brace with leading comma but has content",
pattern: "app-{{,prod,staging}",
expectError: true, // unclosed brace
},
{
name: "brace with leading comma but has content",
pattern: "app-{,prod,staging}}",
expectError: true, // unmatched closing brace
},
}
for _, tt := range tests {
@@ -723,20 +564,6 @@ func TestExpandWildcardsEdgeCases(t *testing.T) {
assert.ElementsMatch(t, []string{"ns-1", "ns_2", "ns.3", "ns@4"}, result)
})
t.Run("complex glob combinations", func(t *testing.T) {
activeNamespaces := []string{"app1-prod", "app2-prod", "app1-test", "db-prod", "service"}
result, err := expandWildcards([]string{"app?-{prod,test}"}, activeNamespaces)
require.NoError(t, err)
assert.ElementsMatch(t, []string{"app1-prod", "app2-prod", "app1-test"}, result)
})
t.Run("escaped characters", func(t *testing.T) {
activeNamespaces := []string{"app*", "app-prod", "app?test", "app-test"}
result, err := expandWildcards([]string{"app\\*"}, activeNamespaces)
require.NoError(t, err)
assert.ElementsMatch(t, []string{"app*"}, result)
})
t.Run("mixed literal and wildcard patterns", func(t *testing.T) {
activeNamespaces := []string{"app.prod", "app-prod", "app_prod", "test.ns"}
result, err := expandWildcards([]string{"app.prod", "app?prod"}, activeNamespaces)
@@ -777,12 +604,8 @@ func TestExpandWildcardsEdgeCases(t *testing.T) {
shouldError bool
}{
{"unclosed bracket", "ns[abc", true},
{"unclosed brace", "app-{prod,staging", true},
{"nested unclosed", "ns[a{bc", true},
{"valid bracket", "ns[abc]", false},
{"valid brace", "app-{prod,staging}", false},
{"empty bracket", "ns[]", true}, // empty brackets are invalid
{"empty brace", "app-{}", true}, // empty braces are invalid
}
for _, tt := range tests {

View File

@@ -16,6 +16,8 @@ Backup belongs to the API group version `velero.io/v1`.
Here is a sample `Backup` object with each of the fields documented:
**Note:** Namespace includes/excludes support glob patterns (`*`, `?`, `[abc]`). See [Namespace Glob Patterns](../namespace-glob-patterns) for more details.
```yaml
# Standard Kubernetes API Version declaration. Required.
apiVersion: velero.io/v1
@@ -42,11 +44,12 @@ spec:
resourcePolicy:
kind: configmap
name: resource-policy-configmap
# Array of namespaces to include in the backup. If unspecified, all namespaces are included.
# Optional.
# Array of namespaces to include in the backup. Accepts glob patterns (*, ?, [abc]).
# Note: '*' alone is reserved for empty fields, which means all namespaces.
# If unspecified, all namespaces are included. Optional.
includedNamespaces:
- '*'
# Array of namespaces to exclude from the backup. Optional.
# Array of namespaces to exclude from the backup. Accepts glob patterns (*, ?, [abc]). Optional.
excludedNamespaces:
- some-namespace
# Array of resources to include in the backup. Resources may be shortcuts (for example 'po' for 'pods')

View File

@@ -16,6 +16,8 @@ Restore belongs to the API group version `velero.io/v1`.
Here is a sample `Restore` object with each of the fields documented:
**Note:** Namespace includes/excludes support glob patterns (`*`, `?`, `[abc]`). See [Namespace Glob Patterns](../namespace-glob-patterns) for more details.
```yaml
# Standard Kubernetes API Version declaration. Required.
apiVersion: velero.io/v1
@@ -45,11 +47,11 @@ spec:
writeSparseFiles: true
# ParallelFilesDownload is the concurrency number setting for restore
parallelFilesDownload: 10
# Array of namespaces to include in the restore. If unspecified, all namespaces are included.
# Optional.
# Array of namespaces to include in the restore. Accepts glob patterns (*, ?, [abc]).
# If unspecified, all namespaces are included. Optional.
includedNamespaces:
- '*'
# Array of namespaces to exclude from the restore. Optional.
# Array of namespaces to exclude from the restore. Accepts glob patterns (*, ?, [abc]). Optional.
excludedNamespaces:
- some-namespace
# Array of resources to include in the restore. Resources may be shortcuts (for example 'po' for 'pods')

View File

@@ -0,0 +1,71 @@
---
title: "Namespace Glob Patterns"
layout: docs
---
When using `--include-namespaces` and `--exclude-namespaces` flags with backup and restore commands, you can use glob patterns to match multiple namespaces.
## Supported Patterns
Velero supports the following glob pattern characters:
- `*` - Matches any sequence of characters
```bash
velero backup create my-backup --include-namespaces "app-*"
# Matches: app-prod, app-staging, app-dev, etc.
```
- `?` - Matches exactly one character
```bash
velero backup create my-backup --include-namespaces "ns?"
# Matches: ns1, ns2, nsa, but NOT ns10
```
- `[abc]` - Matches any single character in the brackets
```bash
velero backup create my-backup --include-namespaces "ns[123]"
# Matches: ns1, ns2, ns3
```
- `[a-z]` - Matches any single character in the range
```bash
velero backup create my-backup --include-namespaces "ns[a-c]"
# Matches: nsa, nsb, nsc
```
## Unsupported Patterns
The following patterns are **not supported** and will cause validation errors:
- `**` - Consecutive asterisks
- `|` - Alternation (regex operator)
- `()` - Grouping (regex operators)
- `!` - Negation
- `{}` - Brace expansion
- `,` - Comma (used in brace expansion)
## Special Cases
- `*` alone means "all namespaces" and is not expanded
- Empty brackets `[]` are invalid
- Unmatched or unclosed brackets will cause validation errors
## Examples
Combine patterns with include and exclude flags:
```bash
# Backup all production namespaces except test
velero backup create prod-backup \
--include-namespaces "*-prod" \
--exclude-namespaces "test-*"
# Backup specific numbered namespaces
velero backup create numbered-backup \
--include-namespaces "app-[0-9]"
# Restore namespaces matching multiple patterns
velero restore create my-restore \
--from-backup my-backup \
--include-namespaces "frontend-*,backend-*"
```

View File

@@ -17,7 +17,11 @@ Wildcard takes precedence when both a wildcard and specific resource are include
### --include-namespaces
Namespaces to include. Default is `*`, all namespaces.
Namespaces to include. Accepts glob patterns (`*`, `?`, `[abc]`). Default is `*`, all namespaces.
See [Namespace Glob Patterns](namespace-glob-patterns) for more details on supported patterns.
Note: `*` alone is reserved for empty fields, which means all namespaces.
* Backup a namespace and it's objects.
@@ -158,7 +162,9 @@ Wildcard excludes are ignored.
### --exclude-namespaces
Namespaces to exclude.
Namespaces to exclude. Accepts glob patterns (`*`, `?`, `[abc]`).
See [Namespace Glob Patterns](namespace-glob-patterns.md) for more details on supported patterns.
* Exclude kube-system from the cluster backup.

View File

@@ -16,6 +16,8 @@ Backup belongs to the API group version `velero.io/v1`.
Here is a sample `Backup` object with each of the fields documented:
**Note:** Namespace includes/excludes support glob patterns (`*`, `?`, `[abc]`). See [Namespace Glob Patterns](../namespace-glob-patterns) for more details.
```yaml
# Standard Kubernetes API Version declaration. Required.
apiVersion: velero.io/v1
@@ -42,11 +44,11 @@ spec:
resourcePolicy:
kind: configmap
name: resource-policy-configmap
# Array of namespaces to include in the backup. If unspecified, all namespaces are included.
# Optional.
# Array of namespaces to include in the backup. Accepts glob patterns (*, ?, [abc]).
# If unspecified, all namespaces are included. Optional.
includedNamespaces:
- '*'
# Array of namespaces to exclude from the backup. Optional.
# Array of namespaces to exclude from the backup. Accepts glob patterns (*, ?, [abc]). Optional.
excludedNamespaces:
- some-namespace
# Array of resources to include in the backup. Resources may be shortcuts (for example 'po' for 'pods')

View File

@@ -16,6 +16,8 @@ Restore belongs to the API group version `velero.io/v1`.
Here is a sample `Restore` object with each of the fields documented:
**Note:** Namespace includes/excludes support glob patterns (`*`, `?`, `[abc]`). See [Namespace Glob Patterns](../namespace-glob-patterns) for more details.
```yaml
# Standard Kubernetes API Version declaration. Required.
apiVersion: velero.io/v1
@@ -45,11 +47,11 @@ spec:
writeSparseFiles: true
# ParallelFilesDownload is the concurrency number setting for restore
parallelFilesDownload: 10
# Array of namespaces to include in the restore. If unspecified, all namespaces are included.
# Optional.
# Array of namespaces to include in the restore. Accepts glob patterns (*, ?, [abc]).
# If unspecified, all namespaces are included. Optional.
includedNamespaces:
- '*'
# Array of namespaces to exclude from the restore. Optional.
# Array of namespaces to exclude from the restore. Accepts glob patterns (*, ?, [abc]). Optional.
excludedNamespaces:
- some-namespace
# Array of resources to include in the restore. Resources may be shortcuts (for example 'po' for 'pods')

View File

@@ -0,0 +1,71 @@
---
title: "Namespace Glob Patterns"
layout: docs
---
When using `--include-namespaces` and `--exclude-namespaces` flags with backup and restore commands, you can use glob patterns to match multiple namespaces.
## Supported Patterns
Velero supports the following glob pattern characters:
- `*` - Matches any sequence of characters
```bash
velero backup create my-backup --include-namespaces "app-*"
# Matches: app-prod, app-staging, app-dev, etc.
```
- `?` - Matches exactly one character
```bash
velero backup create my-backup --include-namespaces "ns?"
# Matches: ns1, ns2, nsa, but NOT ns10
```
- `[abc]` - Matches any single character in the brackets
```bash
velero backup create my-backup --include-namespaces "ns[123]"
# Matches: ns1, ns2, ns3
```
- `[a-z]` - Matches any single character in the range
```bash
velero backup create my-backup --include-namespaces "ns[a-c]"
# Matches: nsa, nsb, nsc
```
## Unsupported Patterns
The following patterns are **not supported** and will cause validation errors:
- `**` - Consecutive asterisks
- `|` - Alternation (regex operator)
- `()` - Grouping (regex operators)
- `!` - Negation
- `{}` - Brace expansion
- `,` - Comma (used in brace expansion)
## Special Cases
- `*` alone means "all namespaces" and is not expanded
- Empty brackets `[]` are invalid
- Unmatched or unclosed brackets will cause validation errors
## Examples
Combine patterns with include and exclude flags:
```bash
# Backup all production namespaces except test
velero backup create prod-backup \
--include-namespaces "*-prod" \
--exclude-namespaces "test-*"
# Backup specific numbered namespaces
velero backup create numbered-backup \
--include-namespaces "app-[0-9]"
# Restore namespaces matching multiple patterns
velero restore create my-restore \
--from-backup my-backup \
--include-namespaces "frontend-*,backend-*"
```

View File

@@ -17,7 +17,11 @@ Wildcard takes precedence when both a wildcard and specific resource are include
### --include-namespaces
Namespaces to include. Default is `*`, all namespaces.
Namespaces to include. Accepts glob patterns (`*`, `?`, `[abc]`). Default is `*`, all namespaces.
See [Namespace Glob Patterns](namespace-glob-patterns) for more details on supported patterns.
Note: `*` alone is reserved for empty fields, which means all namespaces.
* Backup a namespace and it's objects.
@@ -158,7 +162,9 @@ Wildcard excludes are ignored.
### --exclude-namespaces
Namespaces to exclude.
Namespaces to exclude. Accepts glob patterns (`*`, `?`, `[abc]`).
See [Namespace Glob Patterns](namespace-glob-patterns) for more details on supported patterns.
* Exclude kube-system from the cluster backup.

View File

@@ -33,6 +33,8 @@ toc:
url: /enable-api-group-versions-feature
- page: Resource filtering
url: /resource-filtering
- page: Namespace glob patterns
url: /namespace-glob-patterns
- page: Backup reference
url: /backup-reference
- page: Backup hooks

View File

@@ -33,6 +33,8 @@ toc:
url: /enable-api-group-versions-feature
- page: Resource filtering
url: /resource-filtering
- page: Namespace glob patterns
url: /namespace-glob-patterns
- page: Backup reference
url: /backup-reference
- page: Backup hooks

View File

@@ -76,7 +76,7 @@ HAS_VSPHERE_PLUGIN ?= false
RESTORE_HELPER_IMAGE ?=
#Released version only
UPGRADE_FROM_VELERO_VERSION ?= v1.15.2,v1.16.2
UPGRADE_FROM_VELERO_VERSION ?= v1.16.2,v1.17.2
# UPGRADE_FROM_VELERO_CLI can has the same format(a list divided by comma) with UPGRADE_FROM_VELERO_VERSION
# Upgrade tests will be executed sequently according to the list by UPGRADE_FROM_VELERO_VERSION
@@ -85,7 +85,7 @@ UPGRADE_FROM_VELERO_VERSION ?= v1.15.2,v1.16.2
# to the end, nil string will be set if UPGRADE_FROM_VELERO_CLI is shorter than UPGRADE_FROM_VELERO_VERSION
UPGRADE_FROM_VELERO_CLI ?=
MIGRATE_FROM_VELERO_VERSION ?= v1.16.2,$(VERSION)
MIGRATE_FROM_VELERO_VERSION ?= v1.17.2,$(VERSION)
MIGRATE_FROM_VELERO_CLI ?=
VELERO_NAMESPACE ?= velero

View File

@@ -365,7 +365,7 @@ func VersionNoOlderThan(version string, targetVersion string) (bool, error) {
matches := tagRe.FindStringSubmatch(targetVersion)
targetMajor := matches[1]
targetMinor := matches[2]
if major > targetMajor && minor >= targetMinor {
if major >= targetMajor && minor >= targetMinor {
return true, nil
} else {
return false, nil

View File

@@ -0,0 +1,65 @@
/*
Copyright the Velero contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package velero
import (
"testing"
"github.com/stretchr/testify/require"
)
func Test_VersionNoOlderThan(t *testing.T) {
type versionTest struct {
caseName string
version string
targetVersion string
result bool
err error
}
tests := []versionTest{
{
caseName: "branch version compare",
version: "release-1.18",
targetVersion: "v1.16",
result: true,
err: nil,
},
{
caseName: "tag version compare",
version: "v1.18.0",
targetVersion: "v1.16",
result: true,
err: nil,
},
{
caseName: "main version compare",
version: "main",
targetVersion: "v1.15",
result: true,
err: nil,
},
}
for _, test := range tests {
t.Run(test.caseName, func(t *testing.T) {
res, err := VersionNoOlderThan(test.version, test.targetVersion)
require.Equal(t, test.err, err)
require.Equal(t, test.result, res)
})
}
}

View File

@@ -99,6 +99,15 @@ var ImagesMatrix = map[string]map[string][]string{
"velero": {"velero/velero:v1.16.2"},
"velero-restore-helper": {"velero/velero:v1.16.2"},
},
"v1.17": {
"aws": {"velero/velero-plugin-for-aws:v1.13.2"},
"azure": {"velero/velero-plugin-for-microsoft-azure:v1.13.2"},
"vsphere": {"vsphereveleroplugin/velero-plugin-for-vsphere:v1.5.2"},
"gcp": {"velero/velero-plugin-for-gcp:v1.13.2"},
"datamover": {"velero/velero-plugin-for-aws:v1.13.2"},
"velero": {"velero/velero:v1.17.2"},
"velero-restore-helper": {"velero/velero:v1.17.2"},
},
"main": {
"aws": {"velero/velero-plugin-for-aws:main"},
"azure": {"velero/velero-plugin-for-microsoft-azure:main"},
@@ -128,16 +137,13 @@ func SetImagesToDefaultValues(config VeleroConfig, version string) (VeleroConfig
ret.Plugins = ""
versionWithoutPatch := "main"
if version != "main" {
versionWithoutPatch = semver.MajorMinor(version)
}
versionWithoutPatch := getVersionWithoutPatch(version)
// Read migration case needs images from the PluginsMatrix map.
images, ok := ImagesMatrix[versionWithoutPatch]
if !ok {
return config, fmt.Errorf("fail to read the images for version %s from the ImagesMatrix",
versionWithoutPatch)
fmt.Printf("Cannot read the images for version %s from the ImagesMatrix. Use the original values.\n", versionWithoutPatch)
return config, nil
}
ret.VeleroImage = images[Velero][0]
@@ -164,6 +170,27 @@ func SetImagesToDefaultValues(config VeleroConfig, version string) (VeleroConfig
return ret, nil
}
func getVersionWithoutPatch(version string) string {
versionWithoutPatch := ""
mainRe := regexp.MustCompile(`^main$`)
releaseRe := regexp.MustCompile(`^release-(\d+)\.(\d+)(-dev)?$`)
switch {
case mainRe.MatchString(version):
versionWithoutPatch = "main"
case releaseRe.MatchString(version):
matches := releaseRe.FindStringSubmatch(version)
versionWithoutPatch = fmt.Sprintf("v%s.%s", matches[1], matches[2])
default:
versionWithoutPatch = semver.MajorMinor(version)
}
fmt.Println("The version is ", versionWithoutPatch)
return versionWithoutPatch
}
func getPluginsByVersion(version string, cloudProvider string, needDataMoverPlugin bool) ([]string, error) {
var cloudMap map[string][]string
arr := strings.Split(version, ".")

View File

@@ -0,0 +1,54 @@
/*
Copyright the Velero contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package velero
import (
"testing"
"github.com/stretchr/testify/require"
)
func Test_getVersionWithoutPatch(t *testing.T) {
versionTests := []struct {
caseName string
version string
result string
}{
{
caseName: "main version",
version: "main",
result: "main",
},
{
caseName: "release version",
version: "release-1.18-dev",
result: "v1.18",
},
{
caseName: "tag version",
version: "v1.17.2",
result: "v1.17",
},
}
for _, test := range versionTests {
t.Run(test.caseName, func(t *testing.T) {
res := getVersionWithoutPatch(test.version)
require.Equal(t, test.result, res)
})
}
}