mirror of
https://github.com/vmware-tanzu/velero.git
synced 2026-07-28 02:52:41 +00:00
Merge branch 'main' into block-uploader-restore-implementation
This commit is contained in:
Lyndon-Li
@@ -62,8 +62,28 @@ jobs:
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
run: |
|
||||
DOCKERFILE_SHA=$(curl -s -H "Authorization: Bearer $GH_TOKEN" https://api.github.com/repos/bitnami/containers/commits?path=bitnami/minio/2026/debian-12/Dockerfile\&per_page=1 | jq -r '.[0].sha')
|
||||
echo "dockerfile_sha=${DOCKERFILE_SHA}" >> $GITHUB_OUTPUT
|
||||
set -euo pipefail
|
||||
|
||||
url="https://api.github.com/repos/bitnami/containers/commits?path=bitnami/minio/2026/debian-12/Dockerfile&per_page=1"
|
||||
|
||||
response="$(curl --fail-with-body -sS \
|
||||
--retry 5 \
|
||||
--retry-delay 2 \
|
||||
--retry-all-errors \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "Authorization: Bearer ${GH_TOKEN}" \
|
||||
-H "X-GitHub-Api-Version: 2022-11-28" \
|
||||
"$url")"
|
||||
|
||||
DOCKERFILE_SHA="$(echo "$response" | jq -r '.[0].sha // empty')"
|
||||
|
||||
if [ -z "$DOCKERFILE_SHA" ]; then
|
||||
echo "Failed to resolve Bitnami MinIO Dockerfile SHA from GitHub API response"
|
||||
echo "$response"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "dockerfile_sha=${DOCKERFILE_SHA}" >> "$GITHUB_OUTPUT"
|
||||
- name: Cache MinIO Image
|
||||
uses: actions/cache@v4
|
||||
id: minio-cache
|
||||
|
||||
+10
-10
@@ -8,16 +8,16 @@ This document defines the project governance for Velero.
|
||||
|
||||
## Code Repositories
|
||||
|
||||
The following code repositories are governed by Velero community and maintained under the `vmware-tanzu\Velero` organization.
|
||||
The following code repositories are governed by Velero community and maintained under the `velero-io` organization.
|
||||
|
||||
* **[Velero](https://github.com/vmware-tanzu/velero):** Main Velero codebase
|
||||
* **[Velero](https://github.com/velero-io/velero):** Main Velero codebase
|
||||
* **[Helm Chart](https://github.com/vmware-tanzu/helm-charts/tree/main/charts/velero):** The Helm chart for the Velero server component
|
||||
* **[Velero CSI Plugin](https://github.com/vmware-tanzu/velero-plugin-for-csi):** This repository contains Velero plugins for snapshotting CSI backed PVCs using the CSI beta snapshot APIs
|
||||
* **[Velero Plugin for vSphere](https://github.com/vmware-tanzu/velero-plugin-for-vsphere):** This repository contains the Velero Plugin for vSphere. This plugin is a volume snapshotter plugin that provides crash-consistent snapshots of vSphere block volumes and backup of volume data into S3 compatible storage.
|
||||
* **[Velero Plugin for AWS](https://github.com/vmware-tanzu/velero-plugin-for-aws):** This repository contains the plugins to support running Velero on AWS, including the object store plugin and the volume snapshotter plugin
|
||||
* **[Velero Plugin for GCP](https://github.com/vmware-tanzu/velero-plugin-for-gcp):** This repository contains the plugins to support running Velero on GCP, including the object store plugin and the volume snapshotter plugin
|
||||
* **[Velero Plugin for Azure](https://github.com/vmware-tanzu/velero-plugin-for-microsoft-azure):** This repository contains the plugins to support running Velero on Azure, including the object store plugin and the volume snapshotter plugin
|
||||
* **[Velero Plugin Example](https://github.com/vmware-tanzu/velero-plugin-example):** This repository contains example plugins for Velero
|
||||
* **[Velero Plugin for AWS](https://github.com/velero-io/velero-plugin-for-aws):** This repository contains the plugins to support running Velero on AWS, including the object store plugin and the volume snapshotter plugin
|
||||
* **[Velero Plugin for GCP](https://github.com/velero-io/velero-plugin-for-gcp):** This repository contains the plugins to support running Velero on GCP, including the object store plugin and the volume snapshotter plugin
|
||||
* **[Velero Plugin for Azure](https://github.com/velero-io/velero-plugin-for-microsoft-azure):** This repository contains the plugins to support running Velero on Azure, including the object store plugin and the volume snapshotter plugin
|
||||
* **[Velero Plugin Example](https://github.com/velero-io/velero-plugin-example):** This repository contains example plugins for Velero
|
||||
|
||||
|
||||
## Community Roles
|
||||
@@ -67,12 +67,12 @@ interested in implementing the proposal should be either deeply engaged in the
|
||||
proposal process or be an author of the proposal.
|
||||
|
||||
The proposal should be documented as a separated markdown file pushed to the root of the
|
||||
`design` folder in the [Velero](https://github.com/vmware-tanzu/velero/tree/main/design)
|
||||
`design` folder in the [Velero](https://github.com/velero-io/velero/tree/main/design)
|
||||
repository via PR. The name of the file should follow the name pattern `<short
|
||||
meaningful words joined by '-'>_design.md`, e.g:
|
||||
`restore-hooks-design.md`.
|
||||
|
||||
Use the [Proposal Template](https://github.com/vmware-tanzu/velero/blob/main/design/_template.md) as a starting point.
|
||||
Use the [Proposal Template](https://github.com/velero-io/velero/blob/main/design/_template.md) as a starting point.
|
||||
|
||||
### Proposal Lifecycle
|
||||
|
||||
@@ -88,7 +88,7 @@ To maintain velocity in a project as busy as Velero, the concept of [Lazy
|
||||
Consensus](http://en.osswiki.info/concepts/lazy_consensus) is practiced. Ideas
|
||||
and / or proposals should be shared by maintainers via
|
||||
GitHub with the appropriate maintainer groups (e.g.,
|
||||
`@vmware-tanzu/velero-maintainers`) tagged. Out of respect for other contributors,
|
||||
`@velero-io/velero-maintainers`) tagged. Out of respect for other contributors,
|
||||
major changes should also be accompanied by a ping on Slack or a note on the
|
||||
Velero mailing list as appropriate. Author(s) of proposal, Pull Requests,
|
||||
issues, etc. will give a time period of no less than five (5) working days for
|
||||
@@ -111,7 +111,7 @@ Lazy consensus does _not_ apply to the process of:
|
||||
|
||||
### Deprecation Process
|
||||
|
||||
Any contributor may introduce a request to deprecate a feature or an option of a feature by opening a feature request issue in the vmware-tanzu/velero GitHub project. The issue should describe why the feature is no longer needed or has become detrimental to Velero, as well as whether and how it has been superseded. The submitter should give as much detail as possible.
|
||||
Any contributor may introduce a request to deprecate a feature or an option of a feature by opening a feature request issue in the velero-io/velero GitHub project. The issue should describe why the feature is no longer needed or has become detrimental to Velero, as well as whether and how it has been superseded. The submitter should give as much detail as possible.
|
||||
|
||||
Once the issue is filed, a one-month discussion period begins. Discussions take place within the issue itself as well as in the community meetings. The person who opens the issue, or a maintainer, should add the date and time marking the end of the discussion period in a comment on the issue as soon as possible after it is opened. A decision on the issue needs to be made within this one-month period.
|
||||
|
||||
|
||||
@@ -155,6 +155,11 @@ GOARCH = $(word 2, $(platform_temp))
|
||||
GOPROXY ?= https://proxy.golang.org
|
||||
GOBIN=$$(pwd)/.go/bin
|
||||
|
||||
# Keep these build-image tool versions in sync with go.mod so the CLI/library
|
||||
# pair doesn't drift (see https://github.com/velero-io/velero/issues/10023).
|
||||
PROTOC_GEN_GO_VERSION := $(shell go list -m -f '{{.Version}}' google.golang.org/protobuf)
|
||||
GOIMPORTS_VERSION := $(shell go list -m -f '{{.Version}}' golang.org/x/tools)
|
||||
|
||||
# If you want to build all binaries, see the 'all-build' rule.
|
||||
# If you want to build all containers, see the 'all-containers' rule.
|
||||
all:
|
||||
@@ -395,9 +400,9 @@ ifeq ($(BUILDX_ENABLED), true)
|
||||
ifneq ($(CONTAINER_TOOL),docker)
|
||||
$(error $(DOCKER_ONLY_ERROR))
|
||||
endif
|
||||
@cd hack/build-image && $(CONTAINER_TOOL) buildx build --build-arg=GOPROXY=$(GOPROXY) --output=type=docker --pull -t $(BUILDER_IMAGE) -f $(BUILDER_IMAGE_DOCKERFILE_REALPATH) .
|
||||
@cd hack/build-image && $(CONTAINER_TOOL) buildx build --build-arg=GOPROXY=$(GOPROXY) --build-arg=PROTOC_GEN_GO_VERSION=$(PROTOC_GEN_GO_VERSION) --build-arg=GOIMPORTS_VERSION=$(GOIMPORTS_VERSION) --output=type=docker --pull -t $(BUILDER_IMAGE) -f $(BUILDER_IMAGE_DOCKERFILE_REALPATH) .
|
||||
else
|
||||
@cd hack/build-image && $(CONTAINER_TOOL) build --build-arg=GOPROXY=$(GOPROXY) --pull -t $(BUILDER_IMAGE) -f $(BUILDER_IMAGE_DOCKERFILE_REALPATH) .
|
||||
@cd hack/build-image && $(CONTAINER_TOOL) build --build-arg=GOPROXY=$(GOPROXY) --build-arg=PROTOC_GEN_GO_VERSION=$(PROTOC_GEN_GO_VERSION) --build-arg=GOIMPORTS_VERSION=$(GOIMPORTS_VERSION) --pull -t $(BUILDER_IMAGE) -f $(BUILDER_IMAGE_DOCKERFILE_REALPATH) .
|
||||
endif
|
||||
$(eval new_id=$(shell $(CONTAINER_TOOL) image inspect --format '{{ .ID }}' ${BUILDER_IMAGE} 2>/dev/null))
|
||||
@if [ "$(old_id)" != "" ] && [ "$(old_id)" != "$(new_id)" ]; then \
|
||||
|
||||
+3
-3
@@ -5,7 +5,7 @@ Velero is an open source tool with a growing community devoted to safe backup an
|
||||
|
||||
## Supported Versions
|
||||
|
||||
The Velero project maintains the following [governance document](https://github.com/vmware-tanzu/velero/blob/main/GOVERNANCE.md), [release document](https://github.com/vmware-tanzu/velero/blob/f42c63af1b9af445e38f78a7256b1c48ef79c10e/site/docs/main/release-instructions.md), and [support document](https://velero.io/docs/main/support-process/). Please refer to these for release and related details. Only the most recent version of Velero is supported. Each [release](https://github.com/vmware-tanzu/velero/releases) includes information about upgrading to the latest version.
|
||||
The Velero project maintains the following [governance document](https://github.com/velero-io/velero/blob/main/GOVERNANCE.md), [release document](https://github.com/velero-io/velero/blob/f42c63af1b9af445e38f78a7256b1c48ef79c10e/site/docs/main/release-instructions.md), and [support document](https://velero.io/docs/main/support-process/). Please refer to these for release and related details. Only the most recent version of Velero is supported. Each [release](https://github.com/velero-io/velero/releases) includes information about upgrading to the latest version.
|
||||
|
||||
|
||||
## Reporting a Vulnerability - Private Disclosure Process
|
||||
@@ -18,7 +18,7 @@ If you know of a publicly disclosed security vulnerability for Velero, please **
|
||||
|
||||
**IMPORTANT: Do not file public issues on GitHub for security vulnerabilities**
|
||||
|
||||
To report a vulnerability or a security-related issue, please contact the email address with the details of the vulnerability. The email will be fielded by the Security Team and then shared with the Velero maintainers who have committer and release permissions. Emails will be addressed within 3 business days, including a detailed plan to investigate the issue and any potential workarounds to perform in the meantime. Do not report non-security-impacting bugs through this channel. Use [GitHub issues](https://github.com/vmware-tanzu/velero/issues/new/choose) instead.
|
||||
To report a vulnerability or a security-related issue, please contact the email address with the details of the vulnerability. The email will be fielded by the Security Team and then shared with the Velero maintainers who have committer and release permissions. Emails will be addressed within 3 business days, including a detailed plan to investigate the issue and any potential workarounds to perform in the meantime. Do not report non-security-impacting bugs through this channel. Use [GitHub issues](https://github.com/velero-io/velero/issues/new/choose) instead.
|
||||
|
||||
|
||||
## Proposed Email Content
|
||||
@@ -68,7 +68,7 @@ The Security Team will respond to vulnerability reports as follows:
|
||||
|
||||
## Public Disclosure Process
|
||||
|
||||
The Security Team publishes a [public advisory](https://github.com/vmware-tanzu/velero/security/advisories) to the Velero community via GitHub. In most cases, additional communication via Slack, Twitter, mailing lists, blog and other channels will assist in educating Velero users and rolling out the patched release to affected users.
|
||||
The Security Team publishes a [public advisory](https://github.com/velero-io/velero/security/advisories) to the Velero community via GitHub. In most cases, additional communication via Slack, Twitter, mailing lists, blog and other channels will assist in educating Velero users and rolling out the patched release to affected users.
|
||||
|
||||
The Security Team will also publish any mitigating steps users can take until the fix can be applied to their Velero instances. Velero distributors will handle creating and publishing their own security advisories.
|
||||
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
Fix stale backupLastSuccessfulTimestamp metric after schedule deletion
|
||||
@@ -0,0 +1 @@
|
||||
Trim whitespace around plugin image entries during install.
|
||||
@@ -49,7 +49,7 @@ require (
|
||||
golang.org/x/sys v0.46.0
|
||||
golang.org/x/text v0.37.0
|
||||
google.golang.org/api v0.283.0
|
||||
google.golang.org/grpc v1.81.1
|
||||
google.golang.org/grpc v1.82.1
|
||||
google.golang.org/protobuf v1.36.12-0.20260120151049-f2248ac996af
|
||||
k8s.io/api v0.36.0
|
||||
k8s.io/apiextensions-apiserver v0.36.0
|
||||
@@ -76,7 +76,7 @@ require (
|
||||
github.com/Azure/azure-sdk-for-go/sdk/internal v1.12.0 // indirect
|
||||
github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 // indirect
|
||||
github.com/AzureAD/microsoft-authentication-library-for-go v1.7.2 // indirect
|
||||
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.31.0 // indirect
|
||||
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.32.0 // indirect
|
||||
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.55.0 // indirect
|
||||
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.55.0 // indirect
|
||||
github.com/Masterminds/semver/v3 v3.5.0 // indirect
|
||||
@@ -189,7 +189,7 @@ require (
|
||||
github.com/zeebo/blake3 v0.2.4 // indirect
|
||||
github.com/zeebo/xxh3 v1.1.0 // indirect
|
||||
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
|
||||
go.opentelemetry.io/contrib/detectors/gcp v1.42.0 // indirect
|
||||
go.opentelemetry.io/contrib/detectors/gcp v1.43.0 // indirect
|
||||
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.67.0 // indirect
|
||||
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.67.0 // indirect
|
||||
go.opentelemetry.io/otel v1.44.0 // indirect
|
||||
|
||||
@@ -48,8 +48,8 @@ github.com/AzureAD/microsoft-authentication-library-for-go v1.7.2 h1:RHK7bS+HQMs
|
||||
github.com/AzureAD/microsoft-authentication-library-for-go v1.7.2/go.mod h1:HKpQxkWaGLJ+D/5H8QRpyQXA1eKjxkFlOMwck5+33Jk=
|
||||
github.com/GehirnInc/crypt v0.0.0-20230320061759-8cc1b52080c5 h1:IEjq88XO4PuBDcvmjQJcQGg+w+UaafSy8G5Kcb5tBhI=
|
||||
github.com/GehirnInc/crypt v0.0.0-20230320061759-8cc1b52080c5/go.mod h1:exZ0C/1emQJAw5tHOaUDyY1ycttqBAPcxuzf7QbY6ec=
|
||||
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.31.0 h1:DHa2U07rk8syqvCge0QIGMCE1WxGj9njT44GH7zNJLQ=
|
||||
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.31.0/go.mod h1:P4WPRUkOhJC13W//jWpyfJNDAIpvRbAUIYLX/4jtlE0=
|
||||
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.32.0 h1:rIkQfkCOVKc1OiRCNcSDD8ml5RJlZbH/Xsq7lbpynwc=
|
||||
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.32.0/go.mod h1:RD2SsorTmYhF6HkTmDw7KmPYQk8OBYwTkuasChwv7R4=
|
||||
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.55.0 h1:UnDZ/zFfG1JhH/DqxIZYU/1CUAlTUScoXD/LcM2Ykk8=
|
||||
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.55.0/go.mod h1:IA1C1U7jO/ENqm/vhi7V9YYpBsp+IMyqNrEN94N7tVc=
|
||||
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/cloudmock v0.55.0 h1:7t/qx5Ost0s0wbA/VDrByOooURhp+ikYwv20i9Y07TQ=
|
||||
@@ -466,8 +466,8 @@ github.com/zeebo/xxh3 v1.1.0 h1:s7DLGDK45Dyfg7++yxI0khrfwq9661w9EN78eP/UZVs=
|
||||
github.com/zeebo/xxh3 v1.1.0/go.mod h1:IisAie1LELR4xhVinxWS5+zf1lA4p0MW4T+w+W07F5s=
|
||||
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
|
||||
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
|
||||
go.opentelemetry.io/contrib/detectors/gcp v1.42.0 h1:kpt2PEJuOuqYkPcktfJqWWDjTEd/FNgrxcniL7kQrXQ=
|
||||
go.opentelemetry.io/contrib/detectors/gcp v1.42.0/go.mod h1:W9zQ439utxymRrXsUOzZbFX4JhLxXU4+ZnCt8GG7yA8=
|
||||
go.opentelemetry.io/contrib/detectors/gcp v1.43.0 h1:62yY3dT7/ShwOxzA0RsKRgshBmfElKI4d/Myu2OxDFU=
|
||||
go.opentelemetry.io/contrib/detectors/gcp v1.43.0/go.mod h1:RyaZMFY7yi1kAs45S6mbFGz8O8rqB0dTY14uzvG4LCs=
|
||||
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.67.0 h1:yI1/OhfEPy7J9eoa6Sj051C7n5dvpj0QX8g4sRchg04=
|
||||
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.67.0/go.mod h1:NoUCKYWK+3ecatC4HjkRktREheMeEtrXoQxrqYFeHSc=
|
||||
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.67.0 h1:OyrsyzuttWTSur2qN/Lm0m2a8yqyIjUVBZcxFPuXq2o=
|
||||
@@ -564,8 +564,8 @@ google.golang.org/genproto/googleapis/api v0.0.0-20260526163538-3dc84a4a5aaa h1:
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20260526163538-3dc84a4a5aaa/go.mod h1:q4lMZS6kskjT5HvCPrnnypcDPVJqT/f4nfxmkE7gryY=
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20260526163538-3dc84a4a5aaa h1:mZHHdPZl0dbGHCflZgAq/Q468DWVFcU2whhB2KAo8fk=
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20260526163538-3dc84a4a5aaa/go.mod h1:4Hqkh8ycfw05ld/3BWL7rJOSfebL2Q+DVDeRgYgxUU8=
|
||||
google.golang.org/grpc v1.81.1 h1:VnnIIZ88UzOOKLukQi+ImGz8O1Wdp8nAGGnvOfEIWQQ=
|
||||
google.golang.org/grpc v1.81.1/go.mod h1:xGH9GfzOyMTGIOXBJmXt+BX/V0kcdQbdcuwQ/zNw42I=
|
||||
google.golang.org/grpc v1.82.1 h1:NnAxzGRA0677vCa4BUkOAnO5+FfQqVl9iUXeD0IqcGE=
|
||||
google.golang.org/grpc v1.82.1/go.mod h1:yzTZ1TB1Z3SG+LIYaI+WiE8D5+PZ3ArnrSp8zF3+/ZA=
|
||||
google.golang.org/protobuf v1.36.12-0.20260120151049-f2248ac996af h1:+5/Sw3GsDNlEmu7TfklWKPdQ0Ykja5VEmq2i817+jbI=
|
||||
google.golang.org/protobuf v1.36.12-0.20260120151049-f2248ac996af/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
|
||||
@@ -15,6 +15,8 @@
|
||||
FROM --platform=$TARGETPLATFORM golang:1.26-trixie
|
||||
|
||||
ARG GOPROXY
|
||||
ARG PROTOC_GEN_GO_VERSION
|
||||
ARG GOIMPORTS_VERSION
|
||||
|
||||
ENV GO111MODULE=on
|
||||
# Use a proxy for go modules to reduce the likelihood of various hosts being down and breaking the build
|
||||
@@ -34,9 +36,9 @@ RUN wget --quiet https://github.com/kubernetes-sigs/kubebuilder/releases/downloa
|
||||
# get controller-tools
|
||||
RUN go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.16.5
|
||||
|
||||
# get goimports (the revision is pinned so we don't indiscriminately update, but the particular commit
|
||||
# is not important)
|
||||
RUN go install golang.org/x/tools/cmd/goimports@v0.33.0
|
||||
# get goimports, version derived from go.mod's golang.org/x/tools requirement
|
||||
# (see https://github.com/velero-io/velero/issues/10023)
|
||||
RUN go install golang.org/x/tools/cmd/goimports@${GOIMPORTS_VERSION}
|
||||
|
||||
# get protoc compiler and golang plugin
|
||||
WORKDIR /root
|
||||
@@ -71,7 +73,7 @@ RUN ARCH=$(go env GOARCH) && \
|
||||
chmod a+x /usr/include/google/protobuf && \
|
||||
chmod a+r -R /usr/include/google && \
|
||||
chmod +x /usr/bin/protoc
|
||||
RUN go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.33.0 \
|
||||
RUN go install google.golang.org/protobuf/cmd/protoc-gen-go@${PROTOC_GEN_GO_VERSION} \
|
||||
&& go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.3.0
|
||||
|
||||
# get goreleaser
|
||||
|
||||
@@ -84,33 +84,34 @@ var autoExcludeClusterScopedResources = []string{
|
||||
}
|
||||
|
||||
type backupReconciler struct {
|
||||
ctx context.Context
|
||||
logger logrus.FieldLogger
|
||||
discoveryHelper discovery.Helper
|
||||
backupper pkgbackup.Backupper
|
||||
kbClient kbclient.Client
|
||||
clock clock.WithTickerAndDelayedExecution
|
||||
backupLogLevel logrus.Level
|
||||
newPluginManager func(logrus.FieldLogger) clientmgmt.Manager
|
||||
backupTracker BackupTracker
|
||||
defaultBackupLocation string
|
||||
defaultVolumesToFsBackup bool
|
||||
defaultBackupTTL time.Duration
|
||||
defaultVGSLabelKey string
|
||||
defaultCSISnapshotTimeout time.Duration
|
||||
resourceTimeout time.Duration
|
||||
defaultItemOperationTimeout time.Duration
|
||||
defaultSnapshotLocations map[string]string
|
||||
metrics *metrics.ServerMetrics
|
||||
backupStoreGetter persistence.ObjectBackupStoreGetter
|
||||
formatFlag logging.Format
|
||||
credentialFileStore credentials.FileStore
|
||||
maxConcurrentK8SConnections int
|
||||
defaultSnapshotMoveData bool
|
||||
globalCRClient kbclient.Client
|
||||
itemBlockWorkerCount int
|
||||
concurrentBackups int
|
||||
globalVolumePoliciesConfigMap string
|
||||
ctx context.Context
|
||||
logger logrus.FieldLogger
|
||||
discoveryHelper discovery.Helper
|
||||
backupper pkgbackup.Backupper
|
||||
kbClient kbclient.Client
|
||||
clock clock.WithTickerAndDelayedExecution
|
||||
backupLogLevel logrus.Level
|
||||
newPluginManager func(logrus.FieldLogger) clientmgmt.Manager
|
||||
backupTracker BackupTracker
|
||||
defaultBackupLocation string
|
||||
defaultVolumesToFsBackup bool
|
||||
defaultBackupTTL time.Duration
|
||||
defaultVGSLabelKey string
|
||||
defaultCSISnapshotTimeout time.Duration
|
||||
resourceTimeout time.Duration
|
||||
defaultItemOperationTimeout time.Duration
|
||||
defaultSnapshotLocations map[string]string
|
||||
metrics *metrics.ServerMetrics
|
||||
backupStoreGetter persistence.ObjectBackupStoreGetter
|
||||
formatFlag logging.Format
|
||||
credentialFileStore credentials.FileStore
|
||||
maxConcurrentK8SConnections int
|
||||
defaultSnapshotMoveData bool
|
||||
globalCRClient kbclient.Client
|
||||
itemBlockWorkerCount int
|
||||
concurrentBackups int
|
||||
globalVolumePoliciesConfigMap string
|
||||
knownSchedulesWithSuccessfulBackup sets.Set[string]
|
||||
}
|
||||
|
||||
func NewBackupReconciler(
|
||||
@@ -204,28 +205,43 @@ func (b *backupReconciler) updateTotalBackupMetric() {
|
||||
time.Sleep(5 * time.Second)
|
||||
|
||||
wait.Until(
|
||||
func() {
|
||||
// recompute backup_total metric
|
||||
backups := &velerov1api.BackupList{}
|
||||
err := b.kbClient.List(context.Background(), backups, &kbclient.ListOptions{LabelSelector: labels.Everything()})
|
||||
if err != nil {
|
||||
b.logger.Error(err, "Error computing backup_total metric")
|
||||
} else {
|
||||
b.metrics.SetBackupTotal(int64(len(backups.Items)))
|
||||
}
|
||||
|
||||
// recompute backup_last_successful_timestamp metric for each
|
||||
// schedule (including the empty schedule, i.e. ad-hoc backups)
|
||||
for schedule, timestamp := range getLastSuccessBySchedule(backups.Items) {
|
||||
b.metrics.SetBackupLastSuccessfulTimestamp(schedule, timestamp)
|
||||
}
|
||||
},
|
||||
b.resyncBackupMetrics,
|
||||
backupResyncPeriod,
|
||||
b.ctx.Done(),
|
||||
)
|
||||
}()
|
||||
}
|
||||
|
||||
func (b *backupReconciler) resyncBackupMetrics() {
|
||||
backups := &velerov1api.BackupList{}
|
||||
err := b.kbClient.List(context.Background(), backups, &kbclient.ListOptions{LabelSelector: labels.Everything()})
|
||||
if err != nil {
|
||||
b.logger.Error(err, "Error computing backup_total metric")
|
||||
return
|
||||
}
|
||||
|
||||
b.metrics.SetBackupTotal(int64(len(backups.Items)))
|
||||
|
||||
currentSchedules := getLastSuccessBySchedule(backups.Items)
|
||||
for schedule, timestamp := range currentSchedules {
|
||||
b.metrics.SetBackupLastSuccessfulTimestamp(schedule, timestamp)
|
||||
}
|
||||
|
||||
// Remove metrics for schedules that no longer have successful backups
|
||||
if b.knownSchedulesWithSuccessfulBackup != nil {
|
||||
for schedule := range b.knownSchedulesWithSuccessfulBackup {
|
||||
if _, exists := currentSchedules[schedule]; !exists {
|
||||
b.metrics.DeleteBackupLastSuccessfulTimestamp(schedule)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
b.knownSchedulesWithSuccessfulBackup = sets.New[string]()
|
||||
for schedule := range currentSchedules {
|
||||
b.knownSchedulesWithSuccessfulBackup.Insert(schedule)
|
||||
}
|
||||
}
|
||||
|
||||
// getLastSuccessBySchedule finds the most recent completed backup for each schedule
|
||||
// and returns a map of schedule name -> completion time of the most recent completed
|
||||
// backup. This map includes an entry for ad-hoc/non-scheduled backups, where the key
|
||||
|
||||
@@ -31,6 +31,7 @@ import (
|
||||
"github.com/google/go-cmp/cmp"
|
||||
"github.com/google/go-cmp/cmp/cmpopts"
|
||||
snapshotv1api "github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1"
|
||||
"github.com/prometheus/client_golang/prometheus/testutil"
|
||||
"github.com/sirupsen/logrus"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/mock"
|
||||
@@ -2041,6 +2042,48 @@ func Test_getLastSuccessBySchedule(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// Test_resyncBackupMetrics_prunesStaleTimestamps verifies that resyncBackupMetrics
|
||||
// removes backupLastSuccessfulTimestamp entries for schedules that no longer have
|
||||
// any completed backups (e.g. after the schedule and its backups are deleted).
|
||||
func Test_resyncBackupMetrics_prunesStaleTimestamps(t *testing.T) {
|
||||
baseTime, err := time.Parse(time.RFC1123, time.RFC1123)
|
||||
require.NoError(t, err)
|
||||
|
||||
m := metrics.NewServerMetrics()
|
||||
gauge := m.Metrics()["backup_last_successful_timestamp"]
|
||||
|
||||
activeBackup := builder.ForBackup("velero", "b1").
|
||||
ObjectMeta(builder.WithLabels(velerov1api.ScheduleNameLabel, "active-schedule")).
|
||||
Phase(velerov1api.BackupPhaseCompleted).
|
||||
CompletionTimestamp(baseTime).
|
||||
Result()
|
||||
|
||||
deletedBackup := builder.ForBackup("velero", "b2").
|
||||
ObjectMeta(builder.WithLabels(velerov1api.ScheduleNameLabel, "deleted-schedule")).
|
||||
Phase(velerov1api.BackupPhaseCompleted).
|
||||
CompletionTimestamp(baseTime).
|
||||
Result()
|
||||
|
||||
fakeClient := velerotest.NewFakeControllerRuntimeClient(t, activeBackup, deletedBackup)
|
||||
|
||||
c := &backupReconciler{
|
||||
kbClient: fakeClient,
|
||||
logger: logrus.StandardLogger(),
|
||||
metrics: m,
|
||||
}
|
||||
|
||||
// First resync: sets metrics for both schedules
|
||||
c.resyncBackupMetrics()
|
||||
assert.Equal(t, 2, testutil.CollectAndCount(gauge))
|
||||
|
||||
// Simulate schedule deletion: remove the backup for "deleted-schedule"
|
||||
require.NoError(t, fakeClient.Delete(t.Context(), deletedBackup))
|
||||
|
||||
// Second resync: prunes "deleted-schedule" metric, keeps "active-schedule"
|
||||
c.resyncBackupMetrics()
|
||||
assert.Equal(t, 1, testutil.CollectAndCount(gauge))
|
||||
}
|
||||
|
||||
// Unit tests to make sure that the backup's status is updated correctly during reconcile.
|
||||
// To clear up confusion whether status can be updated with Patch alone without status writer and not kbClient.Status().Patch()
|
||||
func TestPatchResourceWorksWithStatus(t *testing.T) {
|
||||
|
||||
@@ -454,7 +454,7 @@ func (r *DataDownloadReconciler) startCancelableDataPath(asyncBR datapath.AsyncB
|
||||
|
||||
if err := asyncBR.StartRestore(dd.Spec.SnapshotID, datapath.AccessPoint{
|
||||
ByPath: res.ByPod.VolumeName,
|
||||
}, dd.Spec.DataMoverConfig); err != nil {
|
||||
}, dd.Spec.DataMoverConfig, nil); err != nil {
|
||||
return errors.Wrapf(err, "error starting async restore for pod %s, volume %s", res.ByPod.HostingPod.Name, res.ByPod.VolumeName)
|
||||
}
|
||||
|
||||
@@ -1096,7 +1096,7 @@ func (r *DataDownloadReconciler) resumeCancellableDataPath(ctx context.Context,
|
||||
|
||||
if err := asyncBR.StartRestore(dd.Spec.SnapshotID, datapath.AccessPoint{
|
||||
ByPath: res.ByPod.VolumeName,
|
||||
}, nil); err != nil {
|
||||
}, nil, nil); err != nil {
|
||||
return errors.Wrapf(err, "error to resume asyncBR watcher for dd %s", dd.Name)
|
||||
}
|
||||
|
||||
|
||||
@@ -529,7 +529,7 @@ func TestDataDownloadReconcile(t *testing.T) {
|
||||
}
|
||||
|
||||
if test.mockStart {
|
||||
asyncBR.On("StartRestore", mock.Anything, mock.Anything, mock.Anything).Return(test.mockStartErr)
|
||||
asyncBR.On("StartRestore", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(test.mockStartErr)
|
||||
}
|
||||
|
||||
if test.mockCancel {
|
||||
@@ -1288,7 +1288,7 @@ func TestResumeCancellableRestore(t *testing.T) {
|
||||
}
|
||||
|
||||
if test.mockStart {
|
||||
mockAsyncBR.On("StartRestore", mock.Anything, mock.Anything, mock.Anything).Return(test.startWatcherErr)
|
||||
mockAsyncBR.On("StartRestore", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(test.startWatcherErr)
|
||||
}
|
||||
|
||||
if test.mockClose {
|
||||
|
||||
@@ -348,7 +348,7 @@ func (f *fakeFSBR) StartBackup(source datapath.AccessPoint, uploaderConfigs map[
|
||||
return f.startErr
|
||||
}
|
||||
|
||||
func (f *fakeFSBR) StartRestore(snapshotID string, target datapath.AccessPoint, uploaderConfigs map[string]string) error {
|
||||
func (f *fakeFSBR) StartRestore(snapshotID string, target datapath.AccessPoint, uploaderConfigs map[string]string, param any) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
|
||||
@@ -528,7 +528,7 @@ func (r *PodVolumeRestoreReconciler) startCancelableDataPath(asyncBR datapath.As
|
||||
|
||||
if err := asyncBR.StartRestore(pvr.Spec.SnapshotID, datapath.AccessPoint{
|
||||
ByPath: res.ByPod.VolumeName,
|
||||
}, pvr.Spec.UploaderSettings); err != nil {
|
||||
}, pvr.Spec.UploaderSettings, nil); err != nil {
|
||||
return errors.Wrapf(err, "error starting async restore for pod %s, volume %s", res.ByPod.HostingPod.Name, res.ByPod.VolumeName)
|
||||
}
|
||||
|
||||
@@ -1146,7 +1146,7 @@ func (r *PodVolumeRestoreReconciler) resumeCancellableDataPath(ctx context.Conte
|
||||
|
||||
if err := asyncBR.StartRestore(pvr.Spec.SnapshotID, datapath.AccessPoint{
|
||||
ByPath: res.ByPod.VolumeName,
|
||||
}, pvr.Spec.UploaderSettings); err != nil {
|
||||
}, pvr.Spec.UploaderSettings, nil); err != nil {
|
||||
return errors.Wrapf(err, "error to resume asyncBR watcher for PVR %s", pvr.Name)
|
||||
}
|
||||
|
||||
|
||||
@@ -1099,7 +1099,7 @@ func TestPodVolumeRestoreReconcile(t *testing.T) {
|
||||
}
|
||||
|
||||
if test.mockStart {
|
||||
asyncBR.On("StartRestore", mock.Anything, mock.Anything, mock.Anything).Return(test.mockStartErr)
|
||||
asyncBR.On("StartRestore", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(test.mockStartErr)
|
||||
}
|
||||
|
||||
if test.mockCancel {
|
||||
@@ -1901,7 +1901,7 @@ func TestResumeCancellablePodVolumeRestore(t *testing.T) {
|
||||
}
|
||||
|
||||
if test.mockStart {
|
||||
mockAsyncBR.On("StartRestore", mock.Anything, mock.Anything, mock.Anything).Return(test.startWatcherErr)
|
||||
mockAsyncBR.On("StartRestore", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(test.startWatcherErr)
|
||||
}
|
||||
|
||||
if test.mockClose {
|
||||
|
||||
@@ -180,7 +180,7 @@ func (r *RestoreMicroService) RunCancelableDataPath(ctx context.Context) (string
|
||||
}
|
||||
log.Info("fs init")
|
||||
|
||||
if err := dp.StartRestore(dd.Spec.SnapshotID, r.sourceTargetPath, dd.Spec.DataMoverConfig); err != nil {
|
||||
if err := dp.StartRestore(dd.Spec.SnapshotID, r.sourceTargetPath, dd.Spec.DataMoverConfig, &datapath.RestoreStartParam{}); err != nil {
|
||||
return "", errors.Wrap(err, "error starting data path restore")
|
||||
}
|
||||
|
||||
|
||||
@@ -355,12 +355,12 @@ func TestRunCancelableRestore(t *testing.T) {
|
||||
|
||||
if test.startErr != nil {
|
||||
fsBR.On("Init", mock.Anything, mock.Anything).Return(nil)
|
||||
fsBR.On("StartRestore", mock.Anything, mock.Anything, mock.Anything).Return(test.startErr)
|
||||
fsBR.On("StartRestore", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(test.startErr)
|
||||
}
|
||||
|
||||
if test.dataPathStarted {
|
||||
fsBR.On("Init", mock.Anything, mock.Anything).Return(nil)
|
||||
fsBR.On("StartRestore", mock.Anything, mock.Anything, mock.Anything).Return(nil)
|
||||
fsBR.On("StartRestore", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(nil)
|
||||
}
|
||||
|
||||
return fsBR
|
||||
|
||||
@@ -59,6 +59,10 @@ type BackupStartParam struct {
|
||||
SnapshotID string
|
||||
}
|
||||
|
||||
// RestoreStartParam define the input param for restore start
|
||||
type RestoreStartParam struct {
|
||||
}
|
||||
|
||||
type generalDataPath struct {
|
||||
ctx context.Context
|
||||
cancel context.CancelFunc
|
||||
@@ -221,7 +225,7 @@ func (dp *generalDataPath) StartBackup(source AccessPoint, uploaderConfig map[st
|
||||
return nil
|
||||
}
|
||||
|
||||
func (dp *generalDataPath) StartRestore(snapshotID string, target AccessPoint, uploaderConfigs map[string]string) error {
|
||||
func (dp *generalDataPath) StartRestore(snapshotID string, target AccessPoint, uploaderConfigs map[string]string, param any) error {
|
||||
if !dp.initialized {
|
||||
return errors.New("data path is not initialized")
|
||||
}
|
||||
|
||||
@@ -190,7 +190,7 @@ func TestAsyncRestore(t *testing.T) {
|
||||
dp.initialized = true
|
||||
dp.callbacks = test.callbacks
|
||||
|
||||
err := dp.StartRestore(test.snapshot, AccessPoint{ByPath: test.path}, map[string]string{})
|
||||
err := dp.StartRestore(test.snapshot, AccessPoint{ByPath: test.path}, map[string]string{}, &RestoreStartParam{})
|
||||
require.NoError(t, err)
|
||||
|
||||
<-finish
|
||||
|
||||
@@ -221,7 +221,7 @@ func (ms *microServiceBRWatcher) StartBackup(source AccessPoint, uploaderConfig
|
||||
return nil
|
||||
}
|
||||
|
||||
func (ms *microServiceBRWatcher) StartRestore(snapshotID string, target AccessPoint, uploaderConfigs map[string]string) error {
|
||||
func (ms *microServiceBRWatcher) StartRestore(snapshotID string, target AccessPoint, uploaderConfigs map[string]string, param any) error {
|
||||
ms.log.Infof("Start watching restore ms to target %s, from snapshot %s", target.ByPath, snapshotID)
|
||||
|
||||
ms.startWatch()
|
||||
|
||||
@@ -60,17 +60,17 @@ func (_m *AsyncBR) StartBackup(source datapath.AccessPoint, dataMoverConfig map[
|
||||
return r0
|
||||
}
|
||||
|
||||
// StartRestore provides a mock function with given fields: snapshotID, target, dataMoverConfig
|
||||
func (_m *AsyncBR) StartRestore(snapshotID string, target datapath.AccessPoint, dataMoverConfig map[string]string) error {
|
||||
ret := _m.Called(snapshotID, target, dataMoverConfig)
|
||||
// StartRestore provides a mock function with given fields: snapshotID, target, dataMoverConfig, param
|
||||
func (_m *AsyncBR) StartRestore(snapshotID string, target datapath.AccessPoint, dataMoverConfig map[string]string, param interface{}) error {
|
||||
ret := _m.Called(snapshotID, target, dataMoverConfig, param)
|
||||
|
||||
if len(ret) == 0 {
|
||||
panic("no return value specified for StartRestore")
|
||||
}
|
||||
|
||||
var r0 error
|
||||
if rf, ok := ret.Get(0).(func(string, datapath.AccessPoint, map[string]string) error); ok {
|
||||
r0 = rf(snapshotID, target, dataMoverConfig)
|
||||
if rf, ok := ret.Get(0).(func(string, datapath.AccessPoint, map[string]string, interface{}) error); ok {
|
||||
r0 = rf(snapshotID, target, dataMoverConfig, param)
|
||||
} else {
|
||||
r0 = ret.Error(0)
|
||||
}
|
||||
|
||||
@@ -66,7 +66,7 @@ type AsyncBR interface {
|
||||
StartBackup(source AccessPoint, dataMoverConfig map[string]string, param any) error
|
||||
|
||||
// StartRestore starts an asynchronous data path instance for restore
|
||||
StartRestore(snapshotID string, target AccessPoint, dataMoverConfig map[string]string) error
|
||||
StartRestore(snapshotID string, target AccessPoint, dataMoverConfig map[string]string, param any) error
|
||||
|
||||
// Cancel cancels an asynchronous data path instance
|
||||
Cancel()
|
||||
|
||||
@@ -139,7 +139,10 @@ func WithPodVolumeOperationTimeout(val time.Duration) podTemplateOption {
|
||||
|
||||
func WithPlugins(plugins []string) podTemplateOption {
|
||||
return func(c *podTemplateConfig) {
|
||||
c.plugins = plugins
|
||||
c.plugins = make([]string, 0, len(plugins))
|
||||
for _, plugin := range plugins {
|
||||
c.plugins = append(c.plugins, strings.TrimSpace(plugin))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -60,6 +60,15 @@ func TestDeployment(t *testing.T) {
|
||||
assert.Len(t, deploy.Spec.Template.Spec.Containers[0].Args, 2)
|
||||
assert.Equal(t, "--features=EnableCSI,foo,bar,baz", deploy.Spec.Template.Spec.Containers[0].Args[1])
|
||||
|
||||
deploy = Deployment("velero", WithPlugins([]string{
|
||||
"harbor-repo.vmware.com/harbor-ci/velero/velero-plugin-for-aws:v1.2.0",
|
||||
" \n vsphereveleroplugin/velero-plugin-for-vsphere:v1.1.1 ",
|
||||
}))
|
||||
assert.Len(t, deploy.Spec.Template.Spec.InitContainers, 2)
|
||||
assert.Equal(t, "harbor-repo.vmware.com/harbor-ci/velero/velero-plugin-for-aws:v1.2.0", deploy.Spec.Template.Spec.InitContainers[0].Image)
|
||||
assert.Equal(t, "vsphereveleroplugin/velero-plugin-for-vsphere:v1.1.1", deploy.Spec.Template.Spec.InitContainers[1].Image)
|
||||
assert.Equal(t, "vsphereveleroplugin-velero-plugin-for-vsphere", deploy.Spec.Template.Spec.InitContainers[1].Name)
|
||||
|
||||
deploy = Deployment("velero", WithUploaderType("kopia"))
|
||||
assert.Len(t, deploy.Spec.Template.Spec.Containers[0].Args, 2)
|
||||
assert.Equal(t, "--uploader-type=kopia", deploy.Spec.Template.Spec.Containers[0].Args[1])
|
||||
|
||||
@@ -758,6 +758,14 @@ func (m *ServerMetrics) RegisterPodVolumeOpLatencyGauge(node, pvbName, opName, b
|
||||
}
|
||||
}
|
||||
|
||||
// DeleteBackupLastSuccessfulTimestamp removes the backupLastSuccessfulTimestamp
|
||||
// metric for a single schedule.
|
||||
func (m *ServerMetrics) DeleteBackupLastSuccessfulTimestamp(scheduleName string) {
|
||||
if g, ok := m.metrics[backupLastSuccessfulTimestamp].(*prometheus.GaugeVec); ok {
|
||||
g.DeleteLabelValues(scheduleName)
|
||||
}
|
||||
}
|
||||
|
||||
// SetBackupTarballSizeBytesGauge records the size, in bytes, of a backup tarball.
|
||||
func (m *ServerMetrics) SetBackupTarballSizeBytesGauge(backupSchedule string, size int64) {
|
||||
if g, ok := m.metrics[backupTarballSizeBytesGauge].(*prometheus.GaugeVec); ok {
|
||||
|
||||
@@ -21,6 +21,7 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/prometheus/client_golang/prometheus"
|
||||
"github.com/prometheus/client_golang/prometheus/testutil"
|
||||
dto "github.com/prometheus/client_model/go"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
@@ -457,6 +458,32 @@ func getHistogramCount(t *testing.T, vec *prometheus.HistogramVec, scheduleLabel
|
||||
return 0
|
||||
}
|
||||
|
||||
// TestDeleteBackupLastSuccessfulTimestamp verifies that DeleteBackupLastSuccessfulTimestamp
|
||||
// removes only the specified schedule's metric.
|
||||
func TestDeleteBackupLastSuccessfulTimestamp(t *testing.T) {
|
||||
m := NewServerMetrics()
|
||||
|
||||
now := time.Now()
|
||||
m.SetBackupLastSuccessfulTimestamp("schedule-1", now)
|
||||
m.SetBackupLastSuccessfulTimestamp("schedule-2", now.Add(-time.Hour))
|
||||
m.SetBackupLastSuccessfulTimestamp("", now.Add(-2*time.Hour))
|
||||
|
||||
g := m.metrics[backupLastSuccessfulTimestamp].(*prometheus.GaugeVec)
|
||||
assert.Equal(t, 3, testutil.CollectAndCount(g))
|
||||
|
||||
m.DeleteBackupLastSuccessfulTimestamp("schedule-1")
|
||||
assert.Equal(t, 2, testutil.CollectAndCount(g))
|
||||
assert.Equal(t, float64(now.Add(-time.Hour).Unix()), testutil.ToFloat64(g.WithLabelValues("schedule-2")))
|
||||
assert.Equal(t, float64(now.Add(-2*time.Hour).Unix()), testutil.ToFloat64(g.WithLabelValues("")))
|
||||
|
||||
m.DeleteBackupLastSuccessfulTimestamp("schedule-2")
|
||||
assert.Equal(t, 1, testutil.CollectAndCount(g))
|
||||
assert.Equal(t, float64(now.Add(-2*time.Hour).Unix()), testutil.ToFloat64(g.WithLabelValues("")))
|
||||
|
||||
m.DeleteBackupLastSuccessfulTimestamp("")
|
||||
assert.Equal(t, 0, testutil.CollectAndCount(g))
|
||||
}
|
||||
|
||||
// TestRepoMaintenanceMetrics verifies that repo maintenance metrics are properly recorded.
|
||||
func TestRepoMaintenanceMetrics(t *testing.T) {
|
||||
tests := []struct {
|
||||
|
||||
@@ -184,7 +184,7 @@ func (r *RestoreMicroService) RunCancelableDataPath(ctx context.Context) (string
|
||||
|
||||
log.Info("Async fs br init")
|
||||
|
||||
if err := fsRestore.StartRestore(pvr.Spec.SnapshotID, r.sourceTargetPath, pvr.Spec.UploaderSettings); err != nil {
|
||||
if err := fsRestore.StartRestore(pvr.Spec.SnapshotID, r.sourceTargetPath, pvr.Spec.UploaderSettings, &datapath.RestoreStartParam{}); err != nil {
|
||||
return "", errors.Wrap(err, "error starting data path restore")
|
||||
}
|
||||
|
||||
|
||||
@@ -436,12 +436,12 @@ func TestRunCancelableDataPathRestore(t *testing.T) {
|
||||
|
||||
if test.startErr != nil {
|
||||
fsBR.On("Init", mock.Anything, mock.Anything).Return(nil)
|
||||
fsBR.On("StartRestore", mock.Anything, mock.Anything, mock.Anything).Return(test.startErr)
|
||||
fsBR.On("StartRestore", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(test.startErr)
|
||||
}
|
||||
|
||||
if test.dataPathStarted {
|
||||
fsBR.On("Init", mock.Anything, mock.Anything).Return(nil)
|
||||
fsBR.On("StartRestore", mock.Anything, mock.Anything, mock.Anything).Return(nil)
|
||||
fsBR.On("StartRestore", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(nil)
|
||||
}
|
||||
|
||||
return fsBR
|
||||
|
||||
@@ -1208,6 +1208,10 @@ func TestKopiaObjectWriterEx_MixedWriteAndWriteAt(t *testing.T) {
|
||||
assert.Equal(t, int64(3072), kow.entries[3].Start)
|
||||
}
|
||||
|
||||
// TestKopiaObjectWriterEx_ConcurrentAsyncErrors verifies the async error contract
|
||||
// under real scheduling: once an async block write fails, the error either fails a
|
||||
// subsequent Write call fast or surfaces at Result — it is never lost. Which of the
|
||||
// two happens first depends on goroutine scheduling, and both are correct.
|
||||
func TestKopiaObjectWriterEx_ConcurrentAsyncErrors(t *testing.T) {
|
||||
mockRepoWriter := repomocks.NewMockRepositoryWriter(t)
|
||||
mockWriter := repomocks.NewWriter(t)
|
||||
@@ -1231,14 +1235,65 @@ func TestKopiaObjectWriterEx_ConcurrentAsyncErrors(t *testing.T) {
|
||||
|
||||
data := make([]byte, 1024)
|
||||
|
||||
// Issue multiple writes so they all spawn async goroutines
|
||||
// First few writes shouldn't fail immediately until getWriteError catches the asynchronous fault
|
||||
// Issue multiple writes so they all spawn async goroutines. A later Write may
|
||||
// observe the stored async error and fail fast — that is correct behavior.
|
||||
for i := 0; i < 10; i++ {
|
||||
l, err := kow.Write(data)
|
||||
if err != nil {
|
||||
assert.Contains(t, err.Error(), "simulated async error")
|
||||
break
|
||||
}
|
||||
assert.Equal(t, 1024, l)
|
||||
}
|
||||
|
||||
// Regardless of whether a Write observed the error first, Result must report it.
|
||||
id, err := kow.Result()
|
||||
|
||||
require.Error(t, err)
|
||||
assert.Contains(t, err.Error(), "simulated async error")
|
||||
assert.Equal(t, udmrepo.ID(""), id)
|
||||
}
|
||||
|
||||
// TestKopiaObjectWriterEx_AsyncErrorSurfacesAtResult pins the late-error schedule:
|
||||
// async writes are held until all writes have been queued, so no Write call observes
|
||||
// the failure and Result alone must report it.
|
||||
func TestKopiaObjectWriterEx_AsyncErrorSurfacesAtResult(t *testing.T) {
|
||||
mockRepoWriter := repomocks.NewMockRepositoryWriter(t)
|
||||
mockWriter := repomocks.NewWriter(t)
|
||||
|
||||
releaseWrites := make(chan struct{})
|
||||
mockWriter.On("Write", mock.Anything).Run(func(mock.Arguments) {
|
||||
<-releaseWrites
|
||||
}).Return(0, errors.New("simulated async error"))
|
||||
mockWriter.On("Close").Return(nil)
|
||||
|
||||
mockRepoWriter.On("NewObjectWriter", mock.Anything, mock.Anything).Return(mockWriter)
|
||||
|
||||
sem := make(chan struct{}, 10)
|
||||
buf := freelist.New(10*1024, 1024)
|
||||
|
||||
kow := &kopiaObjectWriterEx{
|
||||
ctx: context.Background(),
|
||||
rawRepoWriter: mockRepoWriter,
|
||||
blockSize: 1024,
|
||||
asyncWritesSem: sem,
|
||||
asyncBuffer: buf,
|
||||
logger: velerotest.NewLogger(),
|
||||
}
|
||||
|
||||
data := make([]byte, 1024)
|
||||
|
||||
// All async writes block on releaseWrites, so no error can be stored yet and
|
||||
// every Write must succeed.
|
||||
for i := 0; i < 10; i++ {
|
||||
l, err := kow.Write(data)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, 1024, l)
|
||||
}
|
||||
|
||||
close(releaseWrites)
|
||||
|
||||
// Result waits for the async writers to finish and must report their error.
|
||||
id, err := kow.Result()
|
||||
|
||||
require.Error(t, err)
|
||||
|
||||
@@ -67,7 +67,7 @@ On source cluster, Velero needs to manipulate CSI snapshots through the CSI volu
|
||||
|
||||
To integrate Velero with the CSI volume snapshot APIs, you must enable the `EnableCSI` feature flag.
|
||||
|
||||
From release-1.14, the `github.com/vmware-tanzu/velero-plugin-for-csi` repository, which is the Velero CSI plugin, is merged into the `github.com/velero-io/velero` repository.
|
||||
From release-1.14, the `github.com/velero-io/velero-plugin-for-csi` repository, which is the Velero CSI plugin, is merged into the `github.com/velero-io/velero` repository.
|
||||
The reasons to merge the CSI plugin are:
|
||||
* The VolumeSnapshot data mover depends on the CSI plugin, it's reasonabe to integrate them.
|
||||
* This change reduces the Velero deploying complexity.
|
||||
|
||||
@@ -8,7 +8,7 @@ Integrating Container Storage Interface (CSI) snapshot support into Velero enabl
|
||||
By supporting CSI snapshot APIs, Velero can support any volume provider that has a CSI driver, without requiring a Velero-specific plugin to be available. This page gives an overview of how to add support for CSI snapshots to Velero.
|
||||
|
||||
## Notice
|
||||
From release-1.14, the `github.com/vmware-tanzu/velero-plugin-for-csi` repository, which is the Velero CSI plugin, is merged into the `github.com/velero-io/velero` repository.
|
||||
From release-1.14, the `github.com/velero-io/velero-plugin-for-csi` repository, which is the Velero CSI plugin, is merged into the `github.com/velero-io/velero` repository.
|
||||
The reasons to merge the CSI plugin are:
|
||||
* The VolumeSnapshot data mover depends on the CSI plugin, it's reasonabe to integrate them.
|
||||
* This change reduces the Velero deploying complexity.
|
||||
|
||||
@@ -595,7 +595,7 @@ namespacedFilterPolicies:
|
||||
resourceFilters:
|
||||
- kinds: [ConfigMap]
|
||||
names: ["app-*"]
|
||||
excludedNames: ["*-tmp", "*-debug"]
|
||||
excludedNames: ["*-tmp-*", "*-debug-*", "*-tmp", "*-debug"]
|
||||
- kinds: [Secret]
|
||||
labelSelector:
|
||||
workload: application
|
||||
@@ -778,7 +778,7 @@ Velero validates the ResourcePolicy when a backup starts. Common errors:
|
||||
|
||||
Restore is unchanged: it restores whatever is in the backup archive. Resources excluded by fine-grained filters are simply absent. Use `Restore.spec.includedNamespaces` (and existing restore filters) to limit what you restore from a partial backup.
|
||||
|
||||
Fine-grained resource filtering is also available on the restore path using `namespacedFilterPolicies` and `clusterScopedFilterPolicy`. For details on the restore-side policies, see the [Fine-grained restore filters design](https://github.com/vmware-tanzu/velero/blob/main/design/restore-filter-enhancement/fine-grained-restore-filters-design.md).
|
||||
Fine-grained resource filtering is also available on the restore path using `namespacedFilterPolicies` and `clusterScopedFilterPolicy`. For details on the restore-side policies, see the [Fine-grained restore filters design](https://github.com/velero-io/velero/blob/main/design/restore-filter-enhancement/fine-grained-restore-filters-design.md).
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -19,11 +19,11 @@ Plugins the Velero core team is responsible include all those listed in [the Vel
|
||||
1. Once the PR is merged, checkout the upstream `main` branch. Your local upstream might be named `upstream` or `origin`, so use this command: `git checkout <upstream-name>/main`.
|
||||
1. Tag the git version - `git tag v<version>`.
|
||||
1. Push the git tag - `git push --tags <upstream-name>` to trigger the image build.
|
||||
2. Wait for the container images to build. You may check the progress of the GH action that triggers the image build at `https://github.com/vmware-tanzu/<plugin-name>/actions`
|
||||
2. Wait for the container images to build. You may check the progress of the GH action that triggers the image build at `https://github.com/velero-io/<plugin-name>/actions`
|
||||
3. Verify that an image with the new tag is available at `https://hub.docker.com/repository/docker/velero/<plugin-name>/`.
|
||||
4. Run the Velero [e2e tests][2] using the new image. Until it is made configurable, you will have to edit the [plugin version][1] in the test.
|
||||
### Release
|
||||
1. If all e2e tests pass, go to the GitHub release page of the plugin (`https://github.com/vmware-tanzu/<plugin-name>/releases`) and manually create a release for the new tag.
|
||||
1. If all e2e tests pass, go to the GitHub release page of the plugin (`https://github.com/velero-io/<plugin-name>/releases`) and manually create a release for the new tag.
|
||||
1. Copy and paste the content of the new changelog file into the release description field.
|
||||
|
||||
[1]: https://github.com/velero-io/velero/blob/c8dfd648bbe85db0184ea53296de4220895497e6/test/e2e/velero_utils.go#L27
|
||||
|
||||
@@ -40,4 +40,4 @@ Generally speaking, new GitHub issues will fall into one of several categories.
|
||||
- If the issue ends up being a feature request or a bug, update the title and follow the appropriate process for it
|
||||
- If the reporter becomes unresponsive after multiple pings, close out the issue due to inactivity and comment that the user can always reach out again as needed
|
||||
|
||||
[0]: https://github.com/vmware-tanzu?q=velero&type=&language=
|
||||
[0]: https://github.com/velero-io?q=velero&type=&language=
|
||||
|
||||
+2
-1
@@ -48,6 +48,7 @@ GOBIN := $(REPO_ROOT)/.go/bin
|
||||
TOOLS_BIN_DIR := $(TOOLS_DIR)/$(BIN_DIR)
|
||||
|
||||
GINKGO := $(GOBIN)/ginkgo
|
||||
GINKGO_VERSION := $(shell go list -m -f '{{.Version}}' github.com/onsi/ginkgo/v2 2>/dev/null)
|
||||
|
||||
KUSTOMIZE := $(TOOLS_BIN_DIR)/kustomize
|
||||
|
||||
@@ -186,7 +187,7 @@ ginkgo: ${GOBIN}/ginkgo
|
||||
|
||||
# This target does not run if ginkgo is already in $GOBIN
|
||||
${GOBIN}/ginkgo:
|
||||
GOBIN=${GOBIN} go install github.com/onsi/ginkgo/v2/ginkgo@v2.22.0
|
||||
GOBIN=${GOBIN} go install github.com/onsi/ginkgo/v2/ginkgo@${GINKGO_VERSION}
|
||||
|
||||
.PHONY: run-e2e
|
||||
run-e2e: ginkgo
|
||||
|
||||
Reference in New Issue
Block a user