Compare commits

..

52 Commits

Author SHA1 Message Date
lyndon
7416504e3a Merge pull request #5905 from danfengliu/cherrypick-rollback-velero-client-factory-code
[Cherrypick 1.10] rollback velero client factory code
2023-02-23 15:03:54 +08:00
lyndon
f80c947a19 Merge branch 'release-1.10' into cherrypick-rollback-velero-client-factory-code 2023-02-23 14:50:06 +08:00
lyndon
f5124ecf7a Merge pull request #5909 from Lyndon-Li/release-1.10
Fix main CI problem 02
2023-02-23 12:40:10 +08:00
Lyndon-Li
81e52c5f70 fix main CI problem 02
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-02-23 12:28:09 +08:00
lyndon
7e031a7936 Merge pull request #5908 from Lyndon-Li/release-1.10
Fix main CI disk space error
2023-02-23 11:15:50 +08:00
Lyndon-Li
e6d9f2ef40 fix main CI disk space error
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-02-23 10:59:09 +08:00
danfengl
19076eec1d Roll back pkg client code for Velero server
Signed-off-by: danfengl <danfengl@vmware.com>
2023-02-22 14:09:23 +00:00
danfengl
1084cc01c3 Merge branch 'release-1.10' of https://github.com/vmware-tanzu/velero into release-1.10 2023-02-22 14:03:30 +00:00
qiuming
dcf1b00569 Merge pull request #5897 from Lyndon-Li/release-1.10
1.10.2 change log
2023-02-22 10:35:30 +08:00
Lyndon-Li
4d2e1ccdce 1.10.2 change log
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-02-22 10:00:32 +08:00
lyndon
b665614f94 Merge pull request #5891 from blackpiglet/release-1.10
Update distroless image and fix CVE-2022-41717 for release-1.10
2023-02-21 15:25:29 +08:00
Xun Jiang
82ef5317c5 Update distroless image and fix CVE-2022-41717 for release-1.10
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-02-21 15:10:21 +08:00
lyndon
2b6a55958d Merge pull request #5890 from qiuming-best/release-1.10
[cherry-pick]Set Kopia IgnoreUnknownTypes in ErrorHandlingPolicy to True for ignoring backup unknown file type
2023-02-21 15:09:02 +08:00
Ming
7698482256 Set Kopia IgnoreUnknownTypes in ErrorHandlingPolicy to True for ignoring backup unknown file type
Signed-off-by: Ming <mqiu@vmware.com>
2023-02-21 06:47:59 +00:00
lyndon
dbb1bc2912 Merge pull request #5879 from reasonerjt/cp-backup-result-1.10
[Cherrypick-1.10]Publish backup results extracted from backup logs
2023-02-21 13:57:29 +08:00
lyndon
b192665024 Merge branch 'release-1.10' into cp-backup-result-1.10 2023-02-21 13:42:26 +08:00
lyndon
cecbde4423 Merge pull request #5888 from blackpiglet/release-1.10
[cherry-pick][release-1.10]Add labels for velero installed namespace to support PSA
2023-02-21 12:51:10 +08:00
Xun Jiang
aa2287d100 Add labels for created namespace during velero installation to adopt k8s v1.25's PSS and PSA.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-02-21 11:28:30 +08:00
Wenkai Yin(尹文开)
4512160be4 Restore finalizer and managedFields (#5877)
Restore finalizer and managedFields of metadata during the restoration

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2023-02-20 16:56:05 +08:00
Anshul Ahuja
9973e2e0c4 Publish backup results extracted from backup logs
Signed-off-by: Anshul Ahuja <anshulahuja@microsoft.com>
2023-02-20 11:08:36 +08:00
Xun Jiang/Bruce Jiang
42fb499b9b Merge pull request #5869 from ywk253100/230217_secret
[cherry-pick]Add secret restore item action to handle service account token secret
2023-02-17 17:12:08 +08:00
Wenkai Yin(尹文开)
3fc787f630 Add secret restore item action to handle service account token secret
Add secret restore item action to handle service account token secret:
1. Skip the restoration for the auto-created service account token secret
2. Remove several fields for non-auto-created service account token secret to make sure the secret can be restored

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2023-02-17 10:50:02 +08:00
lyndon
1e015ecb15 Merge pull request #5830 from kaovilai/pvb-pvr-should-fail-during-startup-release-1.10
release-1.10: Use updated PVB/PVR for patching Failed Phase during startup
2023-02-08 10:05:11 +08:00
Tiger Kaovilai
820fe8a559 move UpdatePVXStatusToFailed to controller pkg
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-02-07 10:21:22 -05:00
Tiger Kaovilai
dc2f12743b Use updated PVB/PVR for patching Failed Phase during startup
Use the same pvb/pvr update functions across pkg/controller and pkg/cli/nodeagent for consistency of behavior

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-02-03 14:49:09 -05:00
danfengliu
948b3790d5 Merge pull request #5817 from blackpiglet/release-1.10
Fix Restic v0.14.0 HIGH grade CVEs.
2023-02-02 11:00:50 +08:00
Xun Jiang
29ebd16253 Fix Restic v0.14.0 HIGH grade CVEs.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-02-02 10:46:09 +08:00
qiuming
3de7951161 Merge pull request #5813 from Lyndon-Li/release-1.10
add change log
2023-02-01 16:26:46 +08:00
Lyndon-Li
3070198307 add change log
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-02-01 16:00:05 +08:00
Xun Jiang/Bruce Jiang
4806db925f Merge pull request #5811 from Lyndon-Li/release-1.10
bump up golang net
2023-02-01 15:15:31 +08:00
Lyndon-Li
203e9560d1 bump up golang net
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-02-01 13:42:06 +08:00
lyndon
e4d2a83917 Merge pull request #5785 from Lyndon-Li/release-1.10
Add 1.10.1 changelog
2023-01-19 12:07:25 +08:00
Lyndon-Li
8dcc720641 add 1.10.1 changelog
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-01-19 11:50:57 +08:00
Xun Jiang/Bruce Jiang
d594cc5217 Merge pull request #5780 from Lyndon-Li/release-1.10
Bump up to golang 1.18.10
2023-01-18 17:18:52 +08:00
Lyndon-Li
0a114c50c3 bump up to golang 1.18.10
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-01-18 16:56:12 +08:00
qiuming
fa162a31bc Merge pull request #5770 from blackpiglet/release-1.10
[cherry-pick][v1.10] Remove container-builder-env section and add pr-containers action
2023-01-18 14:02:46 +08:00
Xun Jiang
bc7d1d0f82 Add GOARM in Restic builder. Add PR container build action.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-01-18 10:02:04 +08:00
Xun Jiang
fce9669021 Remove container-builder-env section.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-01-16 19:33:44 +08:00
Xun Jiang
118a4e2f72 Remove container-builder-env section.
​
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-01-16 07:37:02 +00:00
qiuming
62287da133 Merge pull request #5765 from blackpiglet/release-1.10
[cherry-pick][v1.10] Add Restic builder in Dockerfile
2023-01-13 16:34:45 +08:00
Xun Jiang
0f9f5f0b71 Fix Dockerfile issue.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-01-13 14:50:41 +08:00
Xun Jiang
1b309ef61f Add Restic builder in Dockerfile.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-01-13 14:23:05 +08:00
lyndon
57ffffccab Merge pull request #5708 from dymurray/panic110
Prevent nil panic on exec restore hooks (#5675)
2023-01-03 10:06:17 +08:00
lyndon
8bee9c9f71 Merge branch 'release-1.10' into panic110 2023-01-03 09:47:44 +08:00
qiuming
b73914d1cc Merge pull request #5714 from Lyndon-Li/release-1.10
[1.10] Fix issue 5696
2022-12-22 10:48:43 +08:00
Lyndon-Li
1b846103dc fix issue 5696
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-12-21 12:47:00 +08:00
lyndon
f2fe0f6b17 Merge pull request #5711 from qiuming-best/release-1.10
Fix error with Restic backup empty volumes
2022-12-20 10:57:57 +08:00
Ming
2a0987c714 Fix error with Restic backup empty volumes
Signed-off-by: Ming <mqiu@vmware.com>
2022-12-20 02:44:27 +00:00
Dylan Murray
7b15b0ab5b Fix changelog
Signed-off-by: Dylan Murray <dymurray@redhat.com>
2022-12-19 10:00:14 -05:00
Dylan Murray
f41d464c47 Prevent nil panic on exec restore hooks (#5675)
* Prevent nil panic on exec restore hooks

Signed-off-by: Dylan Murray <dymurray@redhat.com>
(cherry picked from commit 55873c1c37)
2022-12-19 09:54:23 -05:00
Xun Jiang/Bruce Jiang
d1945d1db3 Merge pull request #5655 from qiuming-best/release-1.10
[Cherry-Pick]Fix CVEs scanned by trivy
2022-12-06 16:25:38 +08:00
Ming
e0642125cd Fix CVEs scanned by trivy
Signed-off-by: Ming <mqiu@vmware.com>
2022-12-02 09:59:19 +00:00
61 changed files with 1763 additions and 167 deletions

View File

@@ -14,7 +14,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1.18.8
go-version: 1.18.10
id: go
# Look for a CLI that's made for this PR
- name: Fetch built CLI

View File

@@ -14,7 +14,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1.18.8
go-version: 1.18.10
id: go
# Look for a CLI that's made for this PR
- name: Fetch built CLI
@@ -72,7 +72,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1.18.8
go-version: 1.18.10
id: go
- name: Check out the code
uses: actions/checkout@v2

View File

@@ -8,7 +8,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1.18.8
go-version: 1.18.10
id: go
- name: Check out the code
uses: actions/checkout@v2

37
.github/workflows/pr-containers.yml vendored Normal file
View File

@@ -0,0 +1,37 @@
name: build Velero containers on Dockerfile change
on:
pull_request:
branches:
- 'main'
- 'release-**'
paths:
- 'Dockerfile'
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
name: Checkout
- name: Set up QEMU
id: qemu
uses: docker/setup-qemu-action@v1
with:
platforms: all
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v1
with:
version: latest
# Although this action also calls docker-push.sh, it is not triggered
# by push, so BRANCH and TAG are empty by default. docker-push.sh will
# only build Velero image without pushing.
- name: Make Velero container without pushing to registry.
if: github.repository == 'vmware-tanzu/velero'
run: |
./hack/docker-push.sh

View File

@@ -18,7 +18,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1.18.8
go-version: 1.18.10
id: go
- uses: actions/checkout@v3
@@ -99,4 +99,7 @@ jobs:
- name: Publish container image to GCR
if: github.repository == 'vmware-tanzu/velero'
run: |
sudo swapoff -a
sudo rm -f /mnt/swapfile
docker image prune -a --force
REGISTRY=gcr.io/velero-gcp ./hack/docker-push.sh

View File

@@ -11,48 +11,68 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
FROM --platform=$BUILDPLATFORM golang:1.18.8-bullseye as builder-env
# Velero binary build section
FROM --platform=$BUILDPLATFORM golang:1.18.10 as velero-builder
ARG GOPROXY
ARG BIN
ARG PKG
ARG VERSION
ARG REGISTRY
ARG GIT_SHA
ARG GIT_TREE_STATE
ARG REGISTRY
ARG TARGETOS
ARG TARGETARCH
ARG TARGETVARIANT
ENV CGO_ENABLED=0 \
GO111MODULE=on \
GOPROXY=${GOPROXY} \
GOOS=${TARGETOS} \
GOARCH=${TARGETARCH} \
GOARM=${TARGETVARIANT} \
LDFLAGS="-X ${PKG}/pkg/buildinfo.Version=${VERSION} -X ${PKG}/pkg/buildinfo.GitSHA=${GIT_SHA} -X ${PKG}/pkg/buildinfo.GitTreeState=${GIT_TREE_STATE} -X ${PKG}/pkg/buildinfo.ImageRegistry=${REGISTRY}"
WORKDIR /go/src/github.com/vmware-tanzu/velero
COPY . /go/src/github.com/vmware-tanzu/velero
FROM --platform=$BUILDPLATFORM builder-env as builder
ARG TARGETOS
ARG TARGETARCH
ARG TARGETVARIANT
ARG PKG
ARG BIN
ARG RESTIC_VERSION
ENV GOOS=${TARGETOS} \
GOARCH=${TARGETARCH} \
GOARM=${TARGETVARIANT}
RUN mkdir -p /output/usr/bin && \
export GOARM=$( echo "${GOARM}" | cut -c2-) && \
bash ./hack/build-restic.sh && \
go build -o /output/${BIN} \
-ldflags "${LDFLAGS}" ${PKG}/cmd/${BIN}
FROM gcr.io/distroless/base-debian11@sha256:4b22ca3c68018333c56f8dddcf1f8b55f32889f2dd12d28ab60856eba1130d04
# Restic binary build section
FROM --platform=$BUILDPLATFORM golang:1.19.4-bullseye as restic-builder
ARG BIN
ARG TARGETOS
ARG TARGETARCH
ARG TARGETVARIANT
ARG RESTIC_VERSION
env CGO_ENABLED=0 \
GO111MODULE=on \
GOPROXY=${GOPROXY} \
GOOS=${TARGETOS} \
GOARCH=${TARGETARCH} \
GOARM=${TARGETVARIANT}
COPY . /go/src/github.com/vmware-tanzu/velero
RUN mkdir -p /output/usr/bin && \
export GOARM=$(echo "${GOARM}" | cut -c2-) && \
/go/src/github.com/vmware-tanzu/velero/hack/build-restic.sh
# Velero image packing section
FROM gcr.io/distroless/base-debian11@sha256:db7ea5913b13bb5fc4a5f5ee5a1fab693d262846d3e7b28efd3f8f62f835c161
LABEL maintainer="Nolan Brubaker <brubakern@vmware.com>"
COPY --from=builder /output /
COPY --from=velero-builder /output /
COPY --from=restic-builder /output /
USER nonroot:nonroot

View File

@@ -82,7 +82,7 @@ see: https://velero.io/docs/main/build-from-source/#making-images-and-updating-v
endef
# The version of restic binary to be downloaded
RESTIC_VERSION ?= 0.13.1
RESTIC_VERSION ?= 0.14.0
CLI_PLATFORMS ?= linux-amd64 linux-arm linux-arm64 darwin-amd64 darwin-arm64 windows-amd64 linux-ppc64le
BUILDX_PLATFORMS ?= $(subst -,/,$(ARCH))
@@ -120,7 +120,7 @@ build-%:
all-build: $(addprefix build-, $(CLI_PLATFORMS))
all-containers: container-builder-env
all-containers:
@$(MAKE) --no-print-directory container
@$(MAKE) --no-print-directory container BIN=velero-restore-helper
@@ -178,20 +178,6 @@ shell: build-dirs build-env
$(BUILDER_IMAGE) \
/bin/sh $(CMD)
container-builder-env:
ifneq ($(BUILDX_ENABLED), true)
$(error $(BUILDX_ERROR))
endif
@docker buildx build \
--target=builder-env \
--build-arg=GOPROXY=$(GOPROXY) \
--build-arg=PKG=$(PKG) \
--build-arg=VERSION=$(VERSION) \
--build-arg=GIT_SHA=$(GIT_SHA) \
--build-arg=GIT_TREE_STATE=$(GIT_TREE_STATE) \
--build-arg=REGISTRY=$(REGISTRY) \
-f $(VELERO_DOCKERFILE) .
container:
ifneq ($(BUILDX_ENABLED), true)
$(error $(BUILDX_ERROR))
@@ -200,6 +186,7 @@ endif
--output=type=$(BUILDX_OUTPUT_TYPE) \
--platform $(BUILDX_PLATFORMS) \
$(addprefix -t , $(IMAGE_TAGS)) \
--build-arg=GOPROXY=$(GOPROXY) \
--build-arg=PKG=$(PKG) \
--build-arg=BIN=$(BIN) \
--build-arg=VERSION=$(VERSION) \

View File

@@ -50,7 +50,7 @@ git_sha = str(local("git rev-parse HEAD", quiet = True, echo_off = True)).strip(
tilt_helper_dockerfile_header = """
# Tilt image
FROM golang:1.18.8 as tilt-helper
FROM golang:1.18.10 as tilt-helper
# Support live reloading with Tilt
RUN wget --output-document /restart.sh --quiet https://raw.githubusercontent.com/windmilleng/rerun-process-wrapper/master/restart.sh && \

View File

@@ -1,4 +1,54 @@
## v1.10.0
## v1.10.2
### 2023-02-22
### Download
https://github.com/vmware-tanzu/velero/releases/tag/v1.10.2
### Container Image
`velero/velero:v1.10.2`
### Documentation
https://velero.io/docs/v1.10/
### Upgrading
https://velero.io/docs/v1.10/upgrade-to-1.10/
### All changes
* Update distroless image and fix CVE-2022-41717 for release-1.10 (#5891, @blackpiglet)
* Set Kopia IgnoreUnknownTypes in ErrorHandlingPolicy to True for ignoring backup unknown file type (#5890, @qiuiming-best)
* Add labels for velero installed namespace to support PSA. (#5888, @blackpiglet)
* Publish backupresults json to enhance error info during backups. (#5879, @anshulahuja98)
* Restore finalizer and managedFields of metadata during the restoration (#5877, @ywk253100)
* Add secret restore item action to handle service account token secret (#5869, @ywk253100)
* Correct PVB/PVR Failed Phase patching during startup (#5830, @kaovilai)
## v1.10.1
### 2023-01-19
### Download
https://github.com/vmware-tanzu/velero/releases/tag/v1.10.1
### Container Image
`velero/velero:v1.10.1`
### Documentation
https://velero.io/docs/v1.10/
### Upgrading
https://velero.io/docs/v1.10/upgrade-to-1.10/
### All changes
* Fix Restic v0.14.0 HIGH grade CVEs. (#5817, @blackpiglet)
* Bump up golang net to fix CVE-2022-41721 (#5811, @Lyndon-Li)
* Bump up golang to 1.18.10 for Velero (#5780, @Lyndon-Li)
* Add PR container build action, which will not push image. Add GOARM parameter. Remove container-builder-env section. (#5770, @blackpiglet)
* Add Restic builder in Dockerfile, and keep the used built Golang image version in accordance with upstream Restic. (#5765, @blackpiglet)
* Fix issue 5696, check if the repo is still openable before running the prune and forget operation, if not, try to reconnect the repo (#5714, @Lyndon-Li)
* Fix error with Restic backup empty volumes (#5711, @qiuming-best)
* Prevent nil panic on exec restore hooks (#5708, @dymurray)
* Fix CVEs scanned by trivy (#5655, @qiuming-best)
## v1.10.0
### 2022-11-23
### Download

12
go.mod
View File

@@ -34,8 +34,8 @@ require (
github.com/spf13/pflag v1.0.5
github.com/stretchr/testify v1.7.1
github.com/vmware-tanzu/crash-diagnostics v0.3.7
golang.org/x/mod v0.6.0-dev.0.20220106191415-9b9b3d81d5e3
golang.org/x/net v0.0.0-20220615171555-694bf12d69de
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4
golang.org/x/net v0.4.0
golang.org/x/oauth2 v0.0.0-20220608161450-d0670ef3b1eb
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c
google.golang.org/api v0.74.0
@@ -132,9 +132,9 @@ require (
go.uber.org/zap v1.21.0 // indirect
golang.org/x/crypto v0.0.0-20220315160706-3147a52a75dd // indirect
golang.org/x/exp v0.0.0-20210916165020-5cb4fee858ee // indirect
golang.org/x/sys v0.0.0-20220614162138-6c1b26c55098 // indirect
golang.org/x/term v0.0.0-20220526004731-065cf7ba2467 // indirect
golang.org/x/text v0.3.7 // indirect
golang.org/x/sys v0.3.0 // indirect
golang.org/x/term v0.3.0 // indirect
golang.org/x/text v0.5.0 // indirect
golang.org/x/time v0.0.0-20220609170525-579cf78fd858 // indirect
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
gomodules.xyz/jsonpatch/v2 v2.2.0 // indirect
@@ -150,5 +150,3 @@ require (
sigs.k8s.io/json v0.0.0-20220525155127-227cbc7cc124 // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.2.1 // indirect
)
replace github.com/gogo/protobuf => github.com/gogo/protobuf v1.3.2

25
go.sum
View File

@@ -293,6 +293,9 @@ github.com/gobwas/glob v0.2.3/go.mod h1:d3Ez4x06l9bZtSvzIay5+Yzi0fmZzPgnTbPcKjJA
github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
github.com/gofrs/flock v0.8.1 h1:+gYjHKf32LDeiEEFhQaotPbLuUXjY5ZqxKgXy7n59aw=
github.com/gofrs/flock v0.8.1/go.mod h1:F1TvTiK9OcQqauNUHlbJvyl9Qa1QvF/gOUDKA14jxHU=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
github.com/gogo/protobuf v1.3.1/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/golang-jwt/jwt/v4 v4.0.0/go.mod h1:/xlHOz8bRuivTWchD4jCa+NbatV+wEUSzwAxVc6locg=
@@ -475,6 +478,8 @@ github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/X
github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=
github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00=
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/compress v1.15.1 h1:y9FcTHGyrebwfP0ZZqFiaxTaiDnUrGkJkI+f583BL1A=
@@ -842,8 +847,9 @@ golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.6.0-dev.0.20220106191415-9b9b3d81d5e3 h1:kQgndtyPBW/JIYERgdxfwMYh3AVStj88WQTlNDi2a+o=
golang.org/x/mod v0.6.0-dev.0.20220106191415-9b9b3d81d5e3/go.mod h1:3p9vT2HGsQu2K1YbXdKPJLVgG5VJdoTa1poYQBtP1AY=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4 h1:6zppjxzCulZykYSLyVDYbneBfbaBIQPYMevg0bEwv2s=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/net v0.0.0-20180530234432-1e491301e022/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -904,8 +910,8 @@ golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qx
golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
golang.org/x/net v0.0.0-20220325170049-de3da57026de/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
golang.org/x/net v0.0.0-20220615171555-694bf12d69de h1:ogOG2+P6LjO2j55AkRScrkB2BFpd+Z8TY2wcM0Z3MGo=
golang.org/x/net v0.0.0-20220615171555-694bf12d69de/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.4.0 h1:Q5QPcMlvfxFTAPV0+07Xz/MpK9NTXu2VDUuy0FeMfaU=
golang.org/x/net v0.4.0/go.mod h1:MBQ8lrhLObU/6UmLb4fmbmk5OcyYmqtbGd/9yIeKjEE=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -1027,14 +1033,14 @@ golang.org/x/sys v0.0.0-20220209214540-3681064d5158/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220227234510-4e6760a101f9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220328115105-d36c6a25d886/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220412211240-33da011f77ad/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220614162138-6c1b26c55098 h1:PgOr27OhUx2IRqGJ2RxAWI4dJQ7bi9cSrB82uzFzfUA=
golang.org/x/sys v0.0.0-20220614162138-6c1b26c55098/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.3.0 h1:w8ZOecv6NaNa/zC8944JTU3vz4u6Lagfk4RPQxv92NQ=
golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210220032956-6a3ed077a48d/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.0.0-20220526004731-065cf7ba2467 h1:CBpWXWQpIRjzmkkA+M7q9Fqnwd2mZr3AFqexg8YTfoM=
golang.org/x/term v0.0.0-20220526004731-065cf7ba2467/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.3.0 h1:qoo4akIqOcDME5bhc/NgxUdovd6BSS2uMsVjB56q1xI=
golang.org/x/term v0.3.0/go.mod h1:q750SLmJuPmVoN1blW3UFBPREJfb1KmY3vwxfr+nFDA=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -1043,8 +1049,9 @@ golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.5.0 h1:OLmvp0KP+FVG99Ct/qFiL/Fhk4zp4QQnZ7b2U+5piUM=
golang.org/x/text v0.5.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
@@ -1055,8 +1062,10 @@ golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac/go.mod h1:tRJNPiyCQ0inRvYxb
golang.org/x/time v0.0.0-20220210224613-90d013bbcef8/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20220609170525-579cf78fd858 h1:Dpdu/EMxGMFgq0CeYMh4fazTD2vtlZRYE7wyynxJb9U=
golang.org/x/time v0.0.0-20220609170525-579cf78fd858/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20181011042414-1f849cf54d09/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20181030221726-6c7e314b6563/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=

View File

@@ -12,7 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
FROM --platform=linux/amd64 golang:1.18.8-bullseye
FROM --platform=linux/amd64 golang:1.18.10-bullseye
ARG GOPROXY

View File

@@ -50,6 +50,8 @@ fi
mkdir ${build_path}/restic
git clone -b v${RESTIC_VERSION} https://github.com/restic/restic.git ${build_path}/restic
pushd ${build_path}/restic
git apply /go/src/github.com/vmware-tanzu/velero/hack/modify_acces_denied_code.txt
git apply /go/src/github.com/vmware-tanzu/velero/hack/fix_restic_cve.txt
go run build.go --goos "${GOOS}" --goarch "${GOARCH}" --goarm "${GOARM}" -o ${restic_bin}
chmod +x ${restic_bin}
popd

View File

@@ -56,6 +56,18 @@ elif [[ "$triggeredBy" == "tags" ]]; then
TAG=$(echo $GITHUB_REF | cut -d / -f 3)
fi
# if both BRANCH and TAG are empty, then it's triggered by PR. Use target branch instead.
# BRANCH is needed in docker buildx command to set as image tag.
# When action is triggered by PR, just build container without pushing, so set type to local.
# When action is triggered by PUSH, need to push container, so set type to registry.
if [[ -z $BRANCH && -z $TAG ]]; then
echo "Test Velero container build without pushing, when Dockerfile is changed by PR."
BRANCH="${GITHUB_BASE_REF}-container"
OUTPUT_TYPE="local,dest=."
else
OUTPUT_TYPE="registry"
fi
TAG_LATEST=false
if [[ ! -z "$TAG" ]]; then
echo "We're building tag $TAG"
@@ -90,11 +102,9 @@ echo "BUILDX_PLATFORMS: $BUILDX_PLATFORMS"
echo "Building and pushing container images."
# The use of "registry" as the buildx output type below instructs
# Docker to push the image
VERSION="$VERSION" \
TAG_LATEST="$TAG_LATEST" \
BUILDX_PLATFORMS="$BUILDX_PLATFORMS" \
BUILDX_OUTPUT_TYPE="registry" \
BUILDX_OUTPUT_TYPE=$OUTPUT_TYPE \
make all-containers

98
hack/fix_restic_cve.txt Normal file
View File

@@ -0,0 +1,98 @@
diff --git a/go.mod b/go.mod
index d819a6be7..41125958a 100644
--- a/go.mod
+++ b/go.mod
@@ -35,12 +35,12 @@ require (
github.com/spf13/cobra v1.5.0
github.com/spf13/pflag v1.0.5
golang.org/x/crypto v0.0.0-20220817201139-bc19a97f63c8
- golang.org/x/net v0.0.0-20220822230855-b0a4917ee28c
+ golang.org/x/net v0.4.0
golang.org/x/oauth2 v0.0.0-20220622183110-fd043fe589d2
golang.org/x/sync v0.0.0-20220819030929-7fc1605a5dde
- golang.org/x/sys v0.0.0-20220818161305-2296e01440c6
- golang.org/x/term v0.0.0-20220722155259-a9ba230a4035
- golang.org/x/text v0.3.7
+ golang.org/x/sys v0.3.0
+ golang.org/x/term v0.3.0
+ golang.org/x/text v0.5.0
google.golang.org/api v0.93.0
google.golang.org/genproto v0.0.0-20220822174746-9e6da59bd2fc // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
diff --git a/go.sum b/go.sum
index 959651048..b7a4358d5 100644
--- a/go.sum
+++ b/go.sum
@@ -319,6 +319,7 @@ github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9de
github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
+github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
@@ -373,6 +374,7 @@ golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
+golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -418,8 +420,8 @@ golang.org/x/net v0.0.0-20220607020251-c690dde0001d/go.mod h1:XRhObCWvk6IyKnWLug
golang.org/x/net v0.0.0-20220617184016-355a448f1bc9/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.0.0-20220624214902-1bab6f366d9e/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
-golang.org/x/net v0.0.0-20220822230855-b0a4917ee28c h1:JVAXQ10yGGVbSyoer5VILysz6YKjdNT2bsvlayjqhes=
-golang.org/x/net v0.0.0-20220822230855-b0a4917ee28c/go.mod h1:YDH+HFinaLZZlnHAfSS6ZXJJ9M9t4Dl22yv3iI2vPwk=
+golang.org/x/net v0.4.0 h1:Q5QPcMlvfxFTAPV0+07Xz/MpK9NTXu2VDUuy0FeMfaU=
+golang.org/x/net v0.4.0/go.mod h1:MBQ8lrhLObU/6UmLb4fmbmk5OcyYmqtbGd/9yIeKjEE=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -454,6 +456,7 @@ golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220601150217-0de741cfad7f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220819030929-7fc1605a5dde h1:ejfdSekXMDxDLbRrJMwUk6KnSLZ2McaUCVcIKM+N6jc=
golang.org/x/sync v0.0.0-20220819030929-7fc1605a5dde/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@@ -522,13 +525,12 @@ golang.org/x/sys v0.0.0-20220624220833-87e55d714810/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220704084225-05e143d24a9e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20220728004956-3c1f35247d10/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20220818161305-2296e01440c6 h1:Sx/u41w+OwrInGdEckYmEuU5gHoGSL4QbDz3S9s6j4U=
-golang.org/x/sys v0.0.0-20220818161305-2296e01440c6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.3.0 h1:w8ZOecv6NaNa/zC8944JTU3vz4u6Lagfk4RPQxv92NQ=
+golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
-golang.org/x/term v0.0.0-20220722155259-a9ba230a4035 h1:Q5284mrmYTpACcm+eAKjKJH48BBwSyfJqmmGDTtT8Vc=
-golang.org/x/term v0.0.0-20220722155259-a9ba230a4035/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
+golang.org/x/term v0.3.0 h1:qoo4akIqOcDME5bhc/NgxUdovd6BSS2uMsVjB56q1xI=
+golang.org/x/term v0.3.0/go.mod h1:q750SLmJuPmVoN1blW3UFBPREJfb1KmY3vwxfr+nFDA=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -537,8 +539,9 @@ golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
-golang.org/x/text v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
+golang.org/x/text v0.5.0 h1:OLmvp0KP+FVG99Ct/qFiL/Fhk4zp4QQnZ7b2U+5piUM=
+golang.org/x/text v0.5.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
@@ -593,6 +596,7 @@ golang.org/x/tools v0.1.2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.3/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.4/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
+golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=

View File

@@ -0,0 +1,13 @@
diff --git a/internal/backend/s3/s3.go b/internal/backend/s3/s3.go
index 0b3816c06..eec10f9c7 100644
--- a/internal/backend/s3/s3.go
+++ b/internal/backend/s3/s3.go
@@ -164,7 +164,7 @@ func isAccessDenied(err error) bool {
debug.Log("isAccessDenied(%T, %#v)", err, err)
var e minio.ErrorResponse
- return errors.As(err, &e) && e.Code == "Access Denied"
+ return errors.As(err, &e) && e.Code == "AccessDenied"
}
// IsNotExist returns true if the error is caused by a not existing file.

View File

@@ -146,3 +146,10 @@ func WithGenerateName(val string) func(obj metav1.Object) {
obj.SetGenerateName(val)
}
}
// WithManagedFields is a functional option that applies the specified managed fields to an object.
func WithManagedFields(val []metav1.ManagedFieldsEntry) func(obj metav1.Object) {
return func(obj metav1.Object) {
obj.SetManagedFields(val)
}
}

View File

@@ -27,20 +27,15 @@ import (
"github.com/vmware-tanzu/velero/pkg/buildinfo"
)
func buildConfigFromFlags(context, kubeconfigPath string, precedence []string) (*rest.Config, error) {
return clientcmd.NewNonInteractiveDeferredLoadingClientConfig(
&clientcmd.ClientConfigLoadingRules{ExplicitPath: kubeconfigPath, Precedence: precedence},
&clientcmd.ConfigOverrides{
CurrentContext: context,
}).ClientConfig()
}
// Config returns a *rest.Config, using either the kubeconfig (if specified) or an in-cluster
// configuration.
func Config(kubeconfig, kubecontext, baseName string, qps float32, burst int) (*rest.Config, error) {
loadingRules := clientcmd.NewDefaultClientConfigLoadingRules()
loadingRules.ExplicitPath = kubeconfig
clientConfig, err := buildConfigFromFlags(kubecontext, kubeconfig, loadingRules.Precedence)
configOverrides := &clientcmd.ConfigOverrides{CurrentContext: kubecontext}
kubeConfig := clientcmd.NewNonInteractiveDeferredLoadingClientConfig(loadingRules, configOverrides)
clientConfig, err := kubeConfig.ClientConfig()
if err != nil {
return nil, errors.Wrap(err, "error finding Kubernetes API server config in --kubeconfig, $KUBECONFIG, or in-cluster configuration")
}

View File

@@ -77,11 +77,10 @@ type factory struct {
}
// NewFactory returns a Factory.
func NewFactory(baseName, kubecontext string, config VeleroConfig) Factory {
func NewFactory(baseName string, config VeleroConfig) Factory {
f := &factory{
flags: pflag.NewFlagSet("", pflag.ContinueOnError),
baseName: baseName,
kubecontext: kubecontext,
flags: pflag.NewFlagSet("", pflag.ContinueOnError),
baseName: baseName,
}
f.namespace = os.Getenv("VELERO_NAMESPACE")
@@ -97,7 +96,7 @@ func NewFactory(baseName, kubecontext string, config VeleroConfig) Factory {
f.flags.StringVar(&f.kubeconfig, "kubeconfig", "", "Path to the kubeconfig file to use to talk to the Kubernetes apiserver. If unset, try the environment variable KUBECONFIG, as well as in-cluster configuration")
f.flags.StringVarP(&f.namespace, "namespace", "n", f.namespace, "The namespace in which Velero should operate")
//f.flags.StringVar(&f.kubecontext, "kubecontext", "", "The context to use to talk to the Kubernetes apiserver. If unset defaults to whatever your current-context is (kubectl config current-context)")
f.flags.StringVar(&f.kubecontext, "kubecontext", "", "The context to use to talk to the Kubernetes apiserver. If unset defaults to whatever your current-context is (kubectl config current-context)")
return f
}
@@ -128,6 +127,7 @@ func (f *factory) KubeClient() (kubernetes.Interface, error) {
return nil, err
}
kubeClient, err := kubernetes.NewForConfig(clientConfig)
if err != nil {
return nil, errors.WithStack(err)
}

View File

@@ -31,14 +31,14 @@ func TestFactory(t *testing.T) {
// Env variable should set the namespace if no config or argument are used
os.Setenv("VELERO_NAMESPACE", "env-velero")
f := NewFactory("velero", "", make(map[string]interface{}))
f := NewFactory("velero", make(map[string]interface{}))
assert.Equal(t, "env-velero", f.Namespace())
os.Unsetenv("VELERO_NAMESPACE")
// Argument should change the namespace
f = NewFactory("velero", "", make(map[string]interface{}))
f = NewFactory("velero", make(map[string]interface{}))
s := "flag-velero"
flags := new(pflag.FlagSet)
@@ -50,7 +50,7 @@ func TestFactory(t *testing.T) {
// An argument overrides the env variable if both are set.
os.Setenv("VELERO_NAMESPACE", "env-velero")
f = NewFactory("velero", "", make(map[string]interface{}))
f = NewFactory("velero", make(map[string]interface{}))
flags = new(pflag.FlagSet)
f.BindFlags(flags)

View File

@@ -303,11 +303,10 @@ func (s *nodeAgentServer) markInProgressPVBsFailed(client ctrlclient.Client) {
s.logger.Debugf("the node of podvolumebackup %q is %q, not %q, skip", pvb.GetName(), pvb.Spec.Node, s.nodeName)
continue
}
original := pvb.DeepCopy()
pvb.Status.Phase = velerov1api.PodVolumeBackupPhaseFailed
pvb.Status.Message = fmt.Sprintf("get a podvolumebackup with status %q during the server starting, mark it as %q", velerov1api.PodVolumeBackupPhaseInProgress, pvb.Status.Phase)
pvb.Status.CompletionTimestamp = &metav1.Time{Time: time.Now()}
if err := client.Patch(s.ctx, &pvbs.Items[i], ctrlclient.MergeFrom(original)); err != nil {
if err := controller.UpdatePVBStatusToFailed(client, s.ctx, &pvbs.Items[i],
fmt.Sprintf("get a podvolumebackup with status %q during the server starting, mark it as %q", velerov1api.PodVolumeBackupPhaseInProgress, velerov1api.PodVolumeBackupPhaseFailed),
time.Now()); err != nil {
s.logger.WithError(errors.WithStack(err)).Errorf("failed to patch podvolumebackup %q", pvb.GetName())
continue
}
@@ -341,11 +340,9 @@ func (s *nodeAgentServer) markInProgressPVRsFailed(client ctrlclient.Client) {
continue
}
original := pvr.DeepCopy()
pvr.Status.Phase = velerov1api.PodVolumeRestorePhaseFailed
pvr.Status.Message = fmt.Sprintf("get a podvolumerestore with status %q during the server starting, mark it as %q", velerov1api.PodVolumeRestorePhaseInProgress, pvr.Status.Phase)
pvr.Status.CompletionTimestamp = &metav1.Time{Time: time.Now()}
if err := client.Patch(s.ctx, &pvrs.Items[i], ctrlclient.MergeFrom(original)); err != nil {
if err := controller.UpdatePVRStatusToFailed(client, s.ctx, &pvrs.Items[i],
fmt.Sprintf("get a podvolumerestore with status %q during the server starting, mark it as %q", velerov1api.PodVolumeRestorePhaseInProgress, velerov1api.PodVolumeRestorePhaseFailed),
time.Now()); err != nil {
s.logger.WithError(errors.WithStack(err)).Errorf("failed to patch podvolumerestore %q", pvr.GetName())
continue
}

View File

@@ -57,7 +57,8 @@ func NewCommand(f client.Factory) *cobra.Command {
RegisterRestoreItemAction("velero.io/crd-preserve-fields", newCRDV1PreserveUnknownFieldsItemAction).
RegisterRestoreItemAction("velero.io/change-pvc-node-selector", newChangePVCNodeSelectorItemAction(f)).
RegisterRestoreItemAction("velero.io/apiservice", newAPIServiceRestoreItemAction).
RegisterRestoreItemAction("velero.io/admission-webhook-configuration", newAdmissionWebhookConfigurationAction)
RegisterRestoreItemAction("velero.io/admission-webhook-configuration", newAdmissionWebhookConfigurationAction).
RegisterRestoreItemAction("velero.io/secret", newSecretRestoreItemAction(f))
if !features.IsEnabled(velerov1api.APIGroupVersionsFeatureFlag) {
// Do not register crd-remap-version BIA if the API Group feature flag is enabled, so that the v1 CRD can be backed up
pluginServer = pluginServer.RegisterBackupItemAction("velero.io/crd-remap-version", newRemapCRDVersionAction(f))
@@ -220,3 +221,13 @@ func newAPIServiceRestoreItemAction(logger logrus.FieldLogger) (interface{}, err
func newAdmissionWebhookConfigurationAction(logger logrus.FieldLogger) (interface{}, error) {
return restore.NewAdmissionWebhookConfigurationAction(logger), nil
}
func newSecretRestoreItemAction(f client.Factory) plugincommon.HandlerInitializer {
return func(logger logrus.FieldLogger) (interface{}, error) {
client, err := f.KubebuilderClient()
if err != nil {
return nil, err
}
return restore.NewSecretAction(logger, client), nil
}
}

View File

@@ -499,9 +499,9 @@ func (s *server) veleroResourcesExist() error {
// - VolumeSnapshots are needed to create PVCs using the VolumeSnapshot as their data source.
// - PVs go before PVCs because PVCs depend on them.
// - PVCs go before pods or controllers so they can be mounted as volumes.
// - Service accounts go before secrets so service account token secrets can be filled automatically.
// - Secrets and config maps go before pods or controllers so they can be mounted
// as volumes.
// - Service accounts go before pods or controllers so pods can use them.
// - Limit ranges go before pods or controllers so pods can use them.
// - Pods go before controllers so they can be explicitly restored and potentially
// have pod volume restores run before controllers adopt the pods.
@@ -525,9 +525,9 @@ var defaultRestorePriorities = restore.Priorities{
"volumesnapshots.snapshot.storage.k8s.io",
"persistentvolumes",
"persistentvolumeclaims",
"serviceaccounts",
"secrets",
"configmaps",
"serviceaccounts",
"limitranges",
"pods",
// we fully qualify replicasets.apps because prior to Kubernetes 1.16, replicasets also

View File

@@ -32,7 +32,7 @@ import (
velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
"github.com/vmware-tanzu/velero/pkg/cmd/util/downloadrequest"
clientset "github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned"
pkgrestore "github.com/vmware-tanzu/velero/pkg/restore"
"github.com/vmware-tanzu/velero/pkg/util/results"
)
func DescribeRestore(ctx context.Context, kbClient kbclient.Client, restore *velerov1api.Restore, podVolumeRestores []velerov1api.PodVolumeRestore, details bool, veleroClient clientset.Interface, insecureSkipTLSVerify bool, caCertFile string) string {
@@ -167,7 +167,7 @@ func describeRestoreResults(ctx context.Context, kbClient kbclient.Client, d *De
}
var buf bytes.Buffer
var resultMap map[string]pkgrestore.Result
var resultMap map[string]results.Result
if err := downloadrequest.Stream(ctx, kbClient, restore.Namespace, restore.Name, velerov1api.DownloadTargetKindRestoreResults, &buf, downloadRequestTimeout, insecureSkipTLSVerify, caCertPath); err != nil {
d.Printf("Warnings:\t<error getting warnings: %v>\n\nErrors:\t<error getting errors: %v>\n", err, err)
@@ -189,7 +189,7 @@ func describeRestoreResults(ctx context.Context, kbClient kbclient.Client, d *De
}
}
func describeRestoreResult(d *Describer, name string, result pkgrestore.Result) {
func describeRestoreResult(d *Describer, name string, result results.Result) {
d.Printf("%s:\n", name)
d.DescribeSlice(1, "Velero", result.Velero)
d.DescribeSlice(1, "Cluster", result.Cluster)

View File

@@ -91,7 +91,7 @@ operations can also be performed as 'velero backup get' and 'velero schedule cre
},
}
f := client.NewFactory(name, "", config)
f := client.NewFactory(name, config)
f.BindFlags(c.PersistentFlags())
// Bind features directly to the root command so it's available to all callers.

View File

@@ -43,6 +43,8 @@ import (
"k8s.io/apimachinery/pkg/util/wait"
"k8s.io/client-go/tools/cache"
"github.com/vmware-tanzu/velero/pkg/util/results"
"github.com/vmware-tanzu/velero/pkg/util/csi"
snapshotv1api "github.com/kubernetes-csi/external-snapshotter/client/v4/apis/volumesnapshot/v1"
@@ -603,7 +605,7 @@ func (c *backupController) runBackup(backup *pkgbackup.Request) error {
logger := logging.DefaultLogger(c.backupLogLevel, c.formatFlag)
logger.Out = io.MultiWriter(os.Stdout, gzippedLogFile)
logCounter := logging.NewLogCounterHook()
logCounter := logging.NewLogHook()
logger.Hooks.Add(logCounter)
backupLog := logger.WithField(Backup, kubeutil.NamespaceAndName(backup))
@@ -725,6 +727,13 @@ func (c *backupController) runBackup(backup *pkgbackup.Request) error {
recordBackupMetrics(backupLog, backup.Backup, backupFile, c.metrics)
backupWarnings := logCounter.GetEntries(logrus.WarnLevel)
backupErrors := logCounter.GetEntries(logrus.ErrorLevel)
results := map[string]results.Result{
"warnings": backupWarnings,
"errors": backupErrors,
}
if err := gzippedLogFile.Close(); err != nil {
c.logger.WithField(Backup, kubeutil.NamespaceAndName(backup)).WithError(err).Error("error closing gzippedLogFile")
}
@@ -750,7 +759,7 @@ func (c *backupController) runBackup(backup *pkgbackup.Request) error {
return err
}
if errs := persistBackup(backup, backupFile, logFile, backupStore, volumeSnapshots, volumeSnapshotContents, volumeSnapshotClasses); len(errs) > 0 {
if errs := persistBackup(backup, backupFile, logFile, backupStore, volumeSnapshots, volumeSnapshotContents, volumeSnapshotClasses, results); len(errs) > 0 {
fatalErrs = append(fatalErrs, errs...)
}
@@ -797,6 +806,7 @@ func persistBackup(backup *pkgbackup.Request,
csiVolumeSnapshots []snapshotv1api.VolumeSnapshot,
csiVolumeSnapshotContents []snapshotv1api.VolumeSnapshotContent,
csiVolumesnapshotClasses []snapshotv1api.VolumeSnapshotClass,
results map[string]results.Result,
) []error {
persistErrs := []error{}
backupJSON := new(bytes.Buffer)
@@ -835,6 +845,11 @@ func persistBackup(backup *pkgbackup.Request,
persistErrs = append(persistErrs, errs...)
}
backupResult, errs := encodeToJSONGzip(results, "backup results")
if errs != nil {
persistErrs = append(persistErrs, errs...)
}
if len(persistErrs) > 0 {
// Don't upload the JSON files or backup tarball if encoding to json fails.
backupJSON = nil
@@ -844,6 +859,7 @@ func persistBackup(backup *pkgbackup.Request,
csiSnapshotJSON = nil
csiSnapshotContentsJSON = nil
csiSnapshotClassesJSON = nil
backupResult = nil
}
backupInfo := persistence.BackupInfo{
@@ -851,6 +867,7 @@ func persistBackup(backup *pkgbackup.Request,
Metadata: backupJSON,
Contents: backupContents,
Log: backupLog,
BackupResults: backupResult,
PodVolumeBackups: podVolumeBackups,
VolumeSnapshots: nativeVolumeSnapshots,
BackupResourceList: backupResourceList,

View File

@@ -279,19 +279,22 @@ func (r *PodVolumeBackupReconciler) getParentSnapshot(ctx context.Context, log l
}
func (r *PodVolumeBackupReconciler) updateStatusToFailed(ctx context.Context, pvb *velerov1api.PodVolumeBackup, err error, msg string, log logrus.FieldLogger) (ctrl.Result, error) {
original := pvb.DeepCopy()
pvb.Status.Phase = velerov1api.PodVolumeBackupPhaseFailed
pvb.Status.Message = errors.WithMessage(err, msg).Error()
pvb.Status.CompletionTimestamp = &metav1.Time{Time: r.Clock.Now()}
if err = r.Client.Patch(ctx, pvb, client.MergeFrom(original)); err != nil {
if err = UpdatePVBStatusToFailed(r.Client, ctx, pvb, errors.WithMessage(err, msg).Error(), r.Clock.Now()); err != nil {
log.WithError(err).Error("error updating PodVolumeBackup status")
return ctrl.Result{}, err
}
return ctrl.Result{}, nil
}
func UpdatePVBStatusToFailed(c client.Client, ctx context.Context, pvb *velerov1api.PodVolumeBackup, errString string, time time.Time) error {
original := pvb.DeepCopy()
pvb.Status.Phase = velerov1api.PodVolumeBackupPhaseFailed
pvb.Status.Message = errString
pvb.Status.CompletionTimestamp = &metav1.Time{Time: time}
return c.Patch(ctx, pvb, client.MergeFrom(original))
}
func (r *PodVolumeBackupReconciler) NewBackupProgressUpdater(pvb *velerov1api.PodVolumeBackup, log logrus.FieldLogger, ctx context.Context) *BackupProgressUpdater {
return &BackupProgressUpdater{pvb, log, ctx, r.Client}
}

View File

@@ -22,6 +22,7 @@ import (
"io/ioutil"
"os"
"path/filepath"
"time"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
@@ -122,11 +123,7 @@ func (c *PodVolumeRestoreReconciler) Reconcile(ctx context.Context, req ctrl.Req
}
if err = c.processRestore(ctx, pvr, pod, log); err != nil {
original = pvr.DeepCopy()
pvr.Status.Phase = velerov1api.PodVolumeRestorePhaseFailed
pvr.Status.Message = err.Error()
pvr.Status.CompletionTimestamp = &metav1.Time{Time: c.clock.Now()}
if e := c.Patch(ctx, pvr, client.MergeFrom(original)); e != nil {
if e := UpdatePVRStatusToFailed(c, ctx, pvr, err.Error(), c.clock.Now()); e != nil {
log.WithError(err).Error("Unable to update status to failed")
}
@@ -145,6 +142,15 @@ func (c *PodVolumeRestoreReconciler) Reconcile(ctx context.Context, req ctrl.Req
return ctrl.Result{}, nil
}
func UpdatePVRStatusToFailed(c client.Client, ctx context.Context, pvr *velerov1api.PodVolumeRestore, errString string, time time.Time) error {
original := pvr.DeepCopy()
pvr.Status.Phase = velerov1api.PodVolumeRestorePhaseFailed
pvr.Status.Message = errString
pvr.Status.CompletionTimestamp = &metav1.Time{Time: time}
return c.Patch(ctx, pvr, client.MergeFrom(original))
}
func (c *PodVolumeRestoreReconciler) shouldProcess(ctx context.Context, log logrus.FieldLogger, pvr *velerov1api.PodVolumeRestore) (bool, *corev1api.Pod, error) {
if !isPVRNew(pvr) {
log.Debug("PodVolumeRestore is not new, skip")

View File

@@ -53,6 +53,7 @@ import (
"github.com/vmware-tanzu/velero/pkg/util/collections"
kubeutil "github.com/vmware-tanzu/velero/pkg/util/kube"
"github.com/vmware-tanzu/velero/pkg/util/logging"
"github.com/vmware-tanzu/velero/pkg/util/results"
"sigs.k8s.io/controller-runtime/pkg/client"
)
@@ -342,10 +343,12 @@ func (c *restoreController) validateAndComplete(restore *api.Restore, pluginMana
}
for _, resource := range restoreHooks {
for _, h := range resource.RestoreHooks {
for _, container := range h.Init.InitContainers {
err = hook.ValidateContainer(container.Raw)
if err != nil {
restore.Status.ValidationErrors = append(restore.Status.ValidationErrors, err.Error())
if h.Init != nil {
for _, container := range h.Init.InitContainers {
err = hook.ValidateContainer(container.Raw)
if err != nil {
restore.Status.ValidationErrors = append(restore.Status.ValidationErrors, err.Error())
}
}
}
}
@@ -572,7 +575,7 @@ func (c *restoreController) runValidatedRestore(restore *api.Restore, info backu
restore.Status.Errors += len(e)
}
m := map[string]pkgrestore.Result{
m := map[string]results.Result{
"warnings": restoreWarnings,
"errors": restoreErrors,
}
@@ -584,7 +587,7 @@ func (c *restoreController) runValidatedRestore(restore *api.Restore, info backu
return nil
}
func putResults(restore *api.Restore, results map[string]pkgrestore.Result, backupStore persistence.BackupStore) error {
func putResults(restore *api.Restore, results map[string]results.Result, backupStore persistence.BackupStore) error {
buf := new(bytes.Buffer)
gzw := gzip.NewWriter(buf)
defer gzw.Close()

View File

@@ -51,6 +51,7 @@ import (
pkgrestore "github.com/vmware-tanzu/velero/pkg/restore"
velerotest "github.com/vmware-tanzu/velero/pkg/test"
"github.com/vmware-tanzu/velero/pkg/util/logging"
"github.com/vmware-tanzu/velero/pkg/util/results"
"github.com/vmware-tanzu/velero/pkg/volume"
)
@@ -506,7 +507,7 @@ func TestProcessQueueItem(t *testing.T) {
sharedInformers.Velero().V1().Backups().Informer().GetStore().Add(test.backup)
}
var warnings, errors pkgrestore.Result
var warnings, errors results.Result
if test.restorerError != nil {
errors.Namespaces = map[string][]string{"ns-1": {test.restorerError.Error()}}
}
@@ -864,12 +865,12 @@ func (r *fakeRestorer) Restore(
actions []riav1.RestoreItemAction,
snapshotLocationLister listers.VolumeSnapshotLocationLister,
volumeSnapshotterGetter pkgrestore.VolumeSnapshotterGetter,
) (pkgrestore.Result, pkgrestore.Result) {
) (results.Result, results.Result) {
res := r.Called(info.Log, info.Restore, info.Backup, info.BackupReader, actions)
r.calledWithArg = *info.Restore
return res.Get(0).(pkgrestore.Result), res.Get(1).(pkgrestore.Result)
return res.Get(0).(results.Result), res.Get(1).(results.Result)
}
func (r *fakeRestorer) RestoreWithResolvers(req pkgrestore.Request,
@@ -877,11 +878,11 @@ func (r *fakeRestorer) RestoreWithResolvers(req pkgrestore.Request,
itemSnapshotterResolver framework.ItemSnapshotterResolver,
snapshotLocationLister listers.VolumeSnapshotLocationLister,
volumeSnapshotterGetter pkgrestore.VolumeSnapshotterGetter,
) (pkgrestore.Result, pkgrestore.Result) {
) (results.Result, results.Result) {
res := r.Called(req.Log, req.Restore, req.Backup, req.BackupReader, resolver, itemSnapshotterResolver,
snapshotLocationLister, volumeSnapshotterGetter)
r.calledWithArg = *req.Restore
return res.Get(0).(pkgrestore.Result), res.Get(1).(pkgrestore.Result)
return res.Get(0).(results.Result), res.Get(1).(results.Result)
}

View File

@@ -136,13 +136,18 @@ func ClusterRoleBinding(namespace string) *rbacv1.ClusterRoleBinding {
}
func Namespace(namespace string) *corev1.Namespace {
return &corev1.Namespace{
ns := &corev1.Namespace{
ObjectMeta: objectMeta("", namespace),
TypeMeta: metav1.TypeMeta{
Kind: "Namespace",
APIVersion: corev1.SchemeGroupVersion.String(),
},
}
ns.Labels["pod-security.kubernetes.io/enforce"] = "privileged"
ns.Labels["pod-security.kubernetes.io/enforce-version"] = "latest"
return ns
}
func BackupStorageLocation(namespace, provider, bucket, prefix string, config map[string]string, caCert []byte) *velerov1api.BackupStorageLocation {

View File

@@ -40,6 +40,11 @@ func TestResources(t *testing.T) {
ns := Namespace("velero")
assert.Equal(t, "velero", ns.Name)
// For k8s version v1.25 and later, need to add the following labels to make
// velero installation namespace has privileged version to work with
// PSA(Pod Security Admission) and PSS(Pod Security Standards).
assert.Equal(t, ns.Labels["pod-security.kubernetes.io/enforce"], "privileged")
assert.Equal(t, ns.Labels["pod-security.kubernetes.io/enforce-version"], "latest")
crb := ClusterRoleBinding(DefaultVeleroNamespace)
// The CRB is a cluster-scoped resource

View File

@@ -42,6 +42,7 @@ type BackupInfo struct {
Metadata,
Contents,
Log,
BackupResults,
PodVolumeBackups,
VolumeSnapshots,
ItemSnapshots,
@@ -261,6 +262,7 @@ func (s *objectBackupStore) PutBackup(info BackupInfo) error {
s.layout.getCSIVolumeSnapshotKey(info.Name): info.CSIVolumeSnapshots,
s.layout.getCSIVolumeSnapshotContentsKey(info.Name): info.CSIVolumeSnapshotContents,
s.layout.getCSIVolumeSnapshotClassesKey(info.Name): info.CSIVolumeSnapshotClasses,
s.layout.getBackupResultsKey(info.Name): info.BackupResults,
}
for key, reader := range backupObjs {

View File

@@ -116,3 +116,7 @@ func (l *ObjectStoreLayout) getCSIVolumeSnapshotContentsKey(backup string) strin
func (l *ObjectStoreLayout) getCSIVolumeSnapshotClassesKey(backup string) string {
return path.Join(l.subdirs["backups"], backup, fmt.Sprintf("%s-csi-volumesnapshotclasses.json.gz", backup))
}
func (l *ObjectStoreLayout) getBackupResultsKey(backup string) string {
return path.Join(l.subdirs["backups"], backup, fmt.Sprintf("%s-results.gz", backup))
}

View File

@@ -172,6 +172,11 @@ func (m *manager) PruneRepo(repo *velerov1api.BackupRepository) error {
if err != nil {
return errors.WithStack(err)
}
if err := prd.BoostRepoConnect(context.Background(), param); err != nil {
return errors.WithStack(err)
}
return prd.PruneRepo(context.Background(), param)
}
@@ -207,6 +212,11 @@ func (m *manager) Forget(ctx context.Context, snapshot SnapshotIdentifier) error
if err != nil {
return errors.WithStack(err)
}
if err := prd.BoostRepoConnect(context.Background(), param); err != nil {
return errors.WithStack(err)
}
return prd.Forget(context.Background(), snapshot.SnapshotID, param)
}

View File

@@ -43,6 +43,11 @@ type Provider interface {
// is already initialized, or do nothing if the repository is already connected
PrepareRepo(ctx context.Context, param RepoParam) error
// BoostRepoConnect is used to re-ensure the local connection to the repo,
// so that the followed operations could succeed in some environment reset
// scenarios, for example, pod restart
BoostRepoConnect(ctx context.Context, param RepoParam) error
// PruneRepo does a full prune/maintenance of the repository
PruneRepo(ctx context.Context, param RepoParam) error

View File

@@ -62,6 +62,10 @@ func (r *resticRepositoryProvider) PrepareRepo(ctx context.Context, param RepoPa
return nil
}
func (r *resticRepositoryProvider) BoostRepoConnect(ctx context.Context, param RepoParam) error {
return nil
}
func (r *resticRepositoryProvider) PruneRepo(ctx context.Context, param RepoParam) error {
return r.svc.PruneRepo(param.BackupLocation, param.BackupRepo)
}

View File

@@ -200,6 +200,37 @@ func (urp *unifiedRepoProvider) PrepareRepo(ctx context.Context, param RepoParam
return nil
}
func (urp *unifiedRepoProvider) BoostRepoConnect(ctx context.Context, param RepoParam) error {
log := urp.log.WithFields(logrus.Fields{
"BSL name": param.BackupLocation.Name,
"repo name": param.BackupRepo.Name,
"repo UID": param.BackupRepo.UID,
})
log.Debug("Start to boost repo connect")
repoOption, err := udmrepo.NewRepoOptions(
udmrepo.WithPassword(urp, param),
udmrepo.WithConfigFile(urp.workPath, string(param.BackupRepo.UID)),
udmrepo.WithDescription(repoConnectDesc),
)
if err != nil {
return errors.Wrap(err, "error to get repo options")
}
bkRepo, err := urp.repoService.Open(ctx, *repoOption)
if err == nil {
if c := bkRepo.Close(ctx); c != nil {
log.WithError(c).Error("Failed to close repo")
}
return nil
}
return urp.ConnectToRepo(ctx, param)
}
func (urp *unifiedRepoProvider) PruneRepo(ctx context.Context, param RepoParam) error {
log := urp.log.WithFields(logrus.Fields{
"BSL name": param.BackupLocation.Name,

View File

@@ -214,7 +214,7 @@ func userPriorityConfigMap() (*corev1.ConfigMap, error) {
return nil, errors.Wrap(err, "reading client config file")
}
fc := client.NewFactory("APIGroupVersionsRestore", "", cfg)
fc := client.NewFactory("APIGroupVersionsRestore", cfg)
kc, err := fc.KubeClient()
if err != nil {

View File

@@ -67,6 +67,7 @@ import (
"github.com/vmware-tanzu/velero/pkg/util/collections"
"github.com/vmware-tanzu/velero/pkg/util/filesystem"
"github.com/vmware-tanzu/velero/pkg/util/kube"
. "github.com/vmware-tanzu/velero/pkg/util/results"
"github.com/vmware-tanzu/velero/pkg/volume"
)
@@ -1421,6 +1422,24 @@ func (ctx *restoreContext) restoreItem(obj *unstructured.Unstructured, groupReso
}
}
// restore the managedFields
withoutManagedFields := createdObj.DeepCopy()
createdObj.SetManagedFields(obj.GetManagedFields())
patchBytes, err := generatePatch(withoutManagedFields, createdObj)
if err != nil {
ctx.log.Errorf("error generating patch for managed fields %s: %v", kube.NamespaceAndName(obj), err)
errs.Add(namespace, err)
return warnings, errs
}
if patchBytes != nil {
if _, err = resourceClient.Patch(name, patchBytes); err != nil {
ctx.log.Errorf("error patch for managed fields %s: %v", kube.NamespaceAndName(obj), err)
errs.Add(namespace, err)
return warnings, errs
}
ctx.log.Infof("the managed fields for %s is patched", kube.NamespaceAndName(obj))
}
if groupResource == kuberesource.Pods {
pod := new(v1.Pod)
if err := runtime.DefaultUnstructuredConverter.FromUnstructured(obj.UnstructuredContent(), pod); err != nil {
@@ -1720,8 +1739,8 @@ func resetMetadata(obj *unstructured.Unstructured) (*unstructured.Unstructured,
for k := range metadata {
switch k {
case "name", "namespace", "labels", "annotations":
default:
case "generateName", "selfLink", "uid", "resourceVersion", "generation", "creationTimestamp", "deletionTimestamp",
"deletionGracePeriodSeconds", "ownerReferences":
delete(metadata, k)
}
}

View File

@@ -56,6 +56,7 @@ import (
testutil "github.com/vmware-tanzu/velero/pkg/test"
"github.com/vmware-tanzu/velero/pkg/util/kube"
kubeutil "github.com/vmware-tanzu/velero/pkg/util/kube"
. "github.com/vmware-tanzu/velero/pkg/util/results"
"github.com/vmware-tanzu/velero/pkg/volume"
)
@@ -867,7 +868,7 @@ func TestRestoreItems(t *testing.T) {
want []*test.APIResource
}{
{
name: "metadata other than namespace/name/labels/annotations gets removed",
name: "metadata uid/resourceVersion/etc. gets removed",
restore: defaultRestore().Result(),
backup: defaultBackup().Result(),
tarball: test.NewTarWriter(t).
@@ -877,6 +878,7 @@ func TestRestoreItems(t *testing.T) {
builder.WithLabels("key-1", "val-1"),
builder.WithAnnotations("key-1", "val-1"),
builder.WithFinalizers("finalizer-1"),
builder.WithUID("uid"),
).
Result(),
).
@@ -890,6 +892,7 @@ func TestRestoreItems(t *testing.T) {
ObjectMeta(
builder.WithLabels("key-1", "val-1", "velero.io/backup-name", "backup-1", "velero.io/restore-name", "restore-1"),
builder.WithAnnotations("key-1", "val-1"),
builder.WithFinalizers("finalizer-1"),
).
Result(),
),
@@ -1107,6 +1110,53 @@ func TestRestoreItems(t *testing.T) {
}),
},
},
{
name: "metadata managedFields gets restored",
restore: defaultRestore().Result(),
backup: defaultBackup().Result(),
tarball: test.NewTarWriter(t).
AddItems("pods",
builder.ForPod("ns-1", "pod-1").
ObjectMeta(
builder.WithManagedFields([]metav1.ManagedFieldsEntry{
{
Manager: "kubectl",
Operation: "Apply",
APIVersion: "v1",
FieldsType: "FieldsV1",
FieldsV1: &metav1.FieldsV1{
Raw: []byte(`{"f:data": {"f:key":{}}}`),
},
},
}),
).
Result(),
).
Done(),
apiResources: []*test.APIResource{
test.Pods(),
},
want: []*test.APIResource{
test.Pods(
builder.ForPod("ns-1", "pod-1").
ObjectMeta(
builder.WithLabels("velero.io/backup-name", "backup-1", "velero.io/restore-name", "restore-1"),
builder.WithManagedFields([]metav1.ManagedFieldsEntry{
{
Manager: "kubectl",
Operation: "Apply",
APIVersion: "v1",
FieldsType: "FieldsV1",
FieldsV1: &metav1.FieldsV1{
Raw: []byte(`{"f:data": {"f:key":{}}}`),
},
},
}),
).
Result(),
),
},
},
}
for _, tc := range tests {
@@ -2823,10 +2873,16 @@ func TestResetMetadata(t *testing.T) {
expectedErr: true,
},
{
name: "keep name, namespace, labels, annotations only",
obj: NewTestUnstructured().WithMetadata("name", "blah", "namespace", "labels", "annotations", "foo").Unstructured,
name: "keep name, namespace, labels, annotations, managedFields, finalizers",
obj: NewTestUnstructured().WithMetadata("name", "namespace", "labels", "annotations", "managedFields", "finalizers").Unstructured,
expectedErr: false,
expectedRes: NewTestUnstructured().WithMetadata("name", "namespace", "labels", "annotations").Unstructured,
expectedRes: NewTestUnstructured().WithMetadata("name", "namespace", "labels", "annotations", "managedFields", "finalizers").Unstructured,
},
{
name: "remove uid, ownerReferences",
obj: NewTestUnstructured().WithMetadata("name", "namespace", "uid", "ownerReferences").Unstructured,
expectedErr: false,
expectedRes: NewTestUnstructured().WithMetadata("name", "namespace").Unstructured,
},
{
name: "keep status",

View File

@@ -0,0 +1,107 @@
/*
Copyright The Velero Contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package restore
import (
"context"
"fmt"
"strings"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"github.com/vmware-tanzu/velero/pkg/plugin/velero"
"github.com/vmware-tanzu/velero/pkg/util/kube"
)
// SecretAction is a restore item action for secrets
type SecretAction struct {
logger logrus.FieldLogger
client client.Client
}
// NewSecretAction creates a new SecretAction instance
func NewSecretAction(logger logrus.FieldLogger, client client.Client) *SecretAction {
return &SecretAction{
logger: logger,
client: client,
}
}
// AppliesTo indicates which resources this action applies
func (s *SecretAction) AppliesTo() (velero.ResourceSelector, error) {
return velero.ResourceSelector{
IncludedResources: []string{"secrets"},
}, nil
}
// Execute the action
func (s *SecretAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {
s.logger.Info("Executing SecretAction")
defer s.logger.Info("Done executing SecretAction")
var secret corev1.Secret
if err := runtime.DefaultUnstructuredConverter.FromUnstructured(input.Item.UnstructuredContent(), &secret); err != nil {
return nil, errors.Wrap(err, "unable to convert secret from runtime.Unstructured")
}
log := s.logger.WithField("secret", kube.NamespaceAndName(&secret))
if secret.Type != corev1.SecretTypeServiceAccountToken {
log.Debug("No match found - including this secret")
return &velero.RestoreItemActionExecuteOutput{
UpdatedItem: input.Item,
}, nil
}
// The auto created service account token secret will be created by kube controller automatically again(before Kubernetes v1.22), no need to restore.
// This will cause the patch operation of managedFields failed if we restore it as the secret is removed immediately
// after restoration and the patch operation reports not found error.
list := &corev1.ServiceAccountList{}
if err := s.client.List(context.Background(), list, &client.ListOptions{Namespace: secret.Namespace}); err != nil {
return nil, errors.Wrap(err, "unable to list the service accounts")
}
for _, sa := range list.Items {
if strings.HasPrefix(secret.Name, fmt.Sprintf("%s-token-", sa.Name)) {
log.Debug("auto created service account token secret found - excluding this secret")
return &velero.RestoreItemActionExecuteOutput{
UpdatedItem: input.Item,
SkipRestore: true,
}, nil
}
}
log.Debug("service account token secret(not auto created) found - remove some fields from this secret")
// If the annotation and data are not removed, the secret cannot be restored successfully.
// The kube controller will fill the annotation and data with new value automatically:
// https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets
delete(secret.Annotations, "kubernetes.io/service-account.uid")
delete(secret.Data, "token")
delete(secret.Data, "ca.crt")
res, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&secret)
if err != nil {
return nil, errors.Wrap(err, "unable to convert secret to runtime.Unstructured")
}
return &velero.RestoreItemActionExecuteOutput{
UpdatedItem: &unstructured.Unstructured{Object: res},
}, nil
}

View File

@@ -0,0 +1,142 @@
/*
Copyright The Velero Contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package restore
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/client/fake"
"github.com/vmware-tanzu/velero/pkg/plugin/velero"
"github.com/vmware-tanzu/velero/pkg/test"
)
func TestSecretActionAppliesTo(t *testing.T) {
action := NewSecretAction(test.NewLogger(), nil)
actual, err := action.AppliesTo()
require.NoError(t, err)
assert.Equal(t, velero.ResourceSelector{IncludedResources: []string{"secrets"}}, actual)
}
func TestSecretActionExecute(t *testing.T) {
tests := []struct {
name string
input *corev1.Secret
serviceAccount *corev1.ServiceAccount
skipped bool
output *corev1.Secret
}{
{
name: "not service account token secret",
input: &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Namespace: "foo",
Name: "default-token-sfafa",
},
Type: corev1.SecretTypeOpaque,
},
skipped: false,
output: &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Namespace: "foo",
Name: "default-token-sfafa",
},
Type: corev1.SecretTypeOpaque,
},
},
{
name: "auto created service account token",
input: &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Namespace: "foo",
Name: "default-token-sfafa",
},
Type: corev1.SecretTypeServiceAccountToken,
},
serviceAccount: &corev1.ServiceAccount{
ObjectMeta: metav1.ObjectMeta{
Namespace: "foo",
Name: "default",
},
},
skipped: true,
},
{
name: "not auto created service account token",
input: &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Namespace: "foo",
Name: "my-token",
Annotations: map[string]string{
"kubernetes.io/service-account.uid": "uid",
"key": "value",
},
},
Type: corev1.SecretTypeServiceAccountToken,
Data: map[string][]byte{
"token": []byte("token"),
"ca.crt": []byte("ca"),
"key": []byte("value"),
},
},
skipped: false,
output: &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Namespace: "foo",
Name: "my-token",
Annotations: map[string]string{
"key": "value",
},
},
Type: corev1.SecretTypeServiceAccountToken,
Data: map[string][]byte{
"key": []byte("value"),
},
},
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
secretUnstructured, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.input)
require.NoError(t, err)
var serviceAccounts []client.Object
if tc.serviceAccount != nil {
serviceAccounts = append(serviceAccounts, tc.serviceAccount)
}
client := fake.NewClientBuilder().WithObjects(serviceAccounts...).Build()
action := NewSecretAction(test.NewLogger(), client)
res, err := action.Execute(&velero.RestoreItemActionExecuteInput{
Item: &unstructured.Unstructured{Object: secretUnstructured},
})
require.NoError(t, err)
assert.Equal(t, tc.skipped, res.SkipRestore)
if !tc.skipped {
r, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.output)
require.NoError(t, err)
assert.EqualValues(t, &unstructured.Unstructured{Object: r}, res.UpdatedItem)
}
})
}
}

View File

@@ -18,6 +18,7 @@ package restore
import (
"encoding/json"
"fmt"
"strconv"
"github.com/pkg/errors"
@@ -83,10 +84,10 @@ func deleteNodePorts(service *corev1api.Service) error {
// find any NodePorts whose values were explicitly specified according
// to the last-applied-config annotation. We'll retain these values, and
// clear out any other (presumably auto-assigned) NodePort values.
explicitNodePorts := sets.NewString()
unnamedPortInts := sets.NewInt()
lastAppliedConfig, ok := service.Annotations[annotationLastAppliedConfig]
if ok {
explicitNodePorts := sets.NewString()
unnamedPortInts := sets.NewInt()
appliedServiceUnstructured := new(map[string]interface{})
if err := json.Unmarshal([]byte(lastAppliedConfig), appliedServiceUnstructured); err != nil {
return errors.WithStack(err)
@@ -134,19 +135,58 @@ func deleteNodePorts(service *corev1api.Service) error {
}
}
}
for i, port := range service.Spec.Ports {
if port.Name != "" {
if !explicitNodePorts.Has(port.Name) {
service.Spec.Ports[i].NodePort = 0
}
} else {
if !unnamedPortInts.Has(int(port.NodePort)) {
service.Spec.Ports[i].NodePort = 0
}
}
}
return nil
}
for i, port := range service.Spec.Ports {
if port.Name != "" {
if !explicitNodePorts.Has(port.Name) {
service.Spec.Ports[i].NodePort = 0
explicitNodePorts := sets.NewString()
for _, entry := range service.GetManagedFields() {
if entry.FieldsV1 == nil {
continue
}
fields := new(map[string]interface{})
if err := json.Unmarshal(entry.FieldsV1.Raw, fields); err != nil {
return errors.WithStack(err)
}
ports, exist, err := unstructured.NestedMap(*fields, "f:spec", "f:ports")
if err != nil {
return errors.WithStack(err)
}
if !exist {
continue
}
for key, port := range ports {
p, ok := port.(map[string]interface{})
if !ok {
continue
}
} else {
if !unnamedPortInts.Has(int(port.NodePort)) {
service.Spec.Ports[i].NodePort = 0
if _, exist := p["f:nodePort"]; exist {
explicitNodePorts.Insert(key)
}
}
}
for i, port := range service.Spec.Ports {
k := portKey(port)
if !explicitNodePorts.Has(k) {
service.Spec.Ports[i].NodePort = 0
}
}
return nil
}
func portKey(port corev1api.ServicePort) string {
return fmt.Sprintf(`k:{"port":%d,"protocol":"%s"}`, port.Port, port.Protocol)
}

View File

@@ -368,6 +368,124 @@ func TestServiceActionExecute(t *testing.T) {
},
},
},
{
name: "nodePort should be delete when not specified in managedFields",
obj: corev1api.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "svc-1",
ManagedFields: []metav1.ManagedFieldsEntry{
{
FieldsV1: &metav1.FieldsV1{
Raw: []byte(`{"f:spec":{"f:ports":{"k:{\"port\":443,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{}},"k:{\"port\":80,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{}}},"f:selector":{},"f:type":{}}}`),
},
},
},
},
Spec: corev1api.ServiceSpec{
Ports: []corev1api.ServicePort{
{
Name: "http",
Port: 80,
Protocol: "TCP",
},
{
Name: "https",
Port: 443,
Protocol: "TCP",
},
},
},
},
restore: builder.ForRestore(api.DefaultNamespace, "").Result(),
expectedRes: corev1api.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "svc-1",
ManagedFields: []metav1.ManagedFieldsEntry{
{
FieldsV1: &metav1.FieldsV1{
Raw: []byte(`{"f:spec":{"f:ports":{"k:{\"port\":443,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{}},"k:{\"port\":80,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{}}},"f:selector":{},"f:type":{}}}`),
},
},
},
},
Spec: corev1api.ServiceSpec{
Ports: []corev1api.ServicePort{
{
Name: "http",
Port: 80,
NodePort: 0,
Protocol: "TCP",
},
{
Name: "https",
Port: 443,
NodePort: 0,
Protocol: "TCP",
},
},
},
},
},
{
name: "nodePort should be preserved when specified in managedFields",
obj: corev1api.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "svc-1",
ManagedFields: []metav1.ManagedFieldsEntry{
{
FieldsV1: &metav1.FieldsV1{
Raw: []byte(`{"f:spec":{"f:ports":{"k:{\"port\":443,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:nodePort":{},"f:port":{}},"k:{\"port\":80,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:nodePort":{},"f:port":{}}},"f:selector":{},"f:type":{}}}`),
},
},
},
},
Spec: corev1api.ServiceSpec{
Ports: []corev1api.ServicePort{
{
Name: "http",
Port: 80,
NodePort: 30000,
Protocol: "TCP",
},
{
Name: "https",
Port: 443,
NodePort: 30002,
Protocol: "TCP",
},
},
},
},
restore: builder.ForRestore(api.DefaultNamespace, "").Result(),
expectedRes: corev1api.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "svc-1",
ManagedFields: []metav1.ManagedFieldsEntry{
{
FieldsV1: &metav1.FieldsV1{
Raw: []byte(`{"f:spec":{"f:ports":{"k:{\"port\":443,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:nodePort":{},"f:port":{}},"k:{\"port\":80,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:nodePort":{},"f:port":{}}},"f:selector":{},"f:type":{}}}`),
},
},
},
},
Spec: corev1api.ServiceSpec{
Ports: []corev1api.ServicePort{
{
Name: "http",
Port: 80,
NodePort: 30000,
Protocol: "TCP",
},
{
Name: "https",
Port: 443,
NodePort: 30002,
Protocol: "TCP",
},
},
},
},
},
}
for _, test := range tests {

View File

@@ -20,6 +20,8 @@ import (
"sync/atomic"
"time"
"github.com/sirupsen/logrus"
"github.com/vmware-tanzu/velero/pkg/uploader"
)
@@ -63,6 +65,7 @@ type KopiaProgress struct {
processedBytes int64 // which statistic all bytes has been processed currently
outputThrottle Throttle // which control the frequency of update progress
Updater uploader.ProgressUpdater //which kopia progress will call the UpdateProgress interface, the third party will implement the interface to do the progress update
Log logrus.FieldLogger // output info into log when backup
}
//UploadedBytes the total bytes has uploaded currently
@@ -77,8 +80,10 @@ func (p *KopiaProgress) UploadedBytes(numBytes int64) {
func (p *KopiaProgress) Error(path string, err error, isIgnored bool) {
if isIgnored {
atomic.AddInt32(&p.ignoredErrorCount, 1)
p.Log.Warnf("Ignored error when processing %v: %v", path, err)
} else {
atomic.AddInt32(&p.fatalErrorCount, 1)
p.Log.Errorf("Error when processing %v: %v", path, err)
}
}

View File

@@ -18,6 +18,7 @@ package kopia
import (
"context"
"fmt"
"io/ioutil"
"math"
"os"
@@ -61,8 +62,14 @@ type SnapshotUploader interface {
) (*snapshot.Manifest, error)
}
func newOptionalInt(b policy.OptionalInt) *policy.OptionalInt {
return &b
func newOptionalInt(b int) *policy.OptionalInt {
ob := policy.OptionalInt(b)
return &ob
}
func newOptionalBool(b bool) *policy.OptionalBool {
ob := policy.OptionalBool(b)
return &ob
}
//setupDefaultPolicy set default policy for kopia
@@ -75,11 +82,14 @@ func setupDefaultPolicy(ctx context.Context, rep repo.RepositoryWriter, sourceIn
CompressorName: "none",
},
UploadPolicy: policy.UploadPolicy{
MaxParallelFileReads: newOptionalInt(policy.OptionalInt(runtime.NumCPU())),
MaxParallelFileReads: newOptionalInt(runtime.NumCPU()),
},
SchedulingPolicy: policy.SchedulingPolicy{
Manual: true,
},
ErrorHandlingPolicy: policy.ErrorHandlingPolicy{
IgnoreUnknownTypes: newOptionalBool(true),
},
})
}
@@ -212,19 +222,23 @@ func SnapshotSource(
return "", 0, errors.Wrapf(err, "Failed to flush kopia repository")
}
log.Infof("Created snapshot with root %v and ID %v in %v", manifest.RootObjectID(), manifest.ID, time.Since(snapshotStartTime).Truncate(time.Second))
return reportSnapshotStatus(manifest)
return reportSnapshotStatus(manifest, policyTree)
}
func reportSnapshotStatus(manifest *snapshot.Manifest) (string, int64, error) {
func reportSnapshotStatus(manifest *snapshot.Manifest, policyTree *policy.Tree) (string, int64, error) {
manifestID := manifest.ID
snapSize := manifest.Stats.TotalFileSize
var errs []string
if ds := manifest.RootEntry.DirSummary; ds != nil {
for _, ent := range ds.FailedEntries {
errs = append(errs, ent.Error)
policy := policyTree.DefinedPolicy()
if !(policy != nil && *policy.ErrorHandlingPolicy.IgnoreUnknownTypes == true && strings.Contains(ent.Error, fs.ErrUnknown.Error())) {
errs = append(errs, fmt.Sprintf("Error when processing %v: %v", ent.EntryPath, ent.Error))
}
}
}
if len(errs) != 0 {
return "", 0, errors.New(strings.Join(errs, "\n"))
}

View File

@@ -119,10 +119,11 @@ func (kp *kopiaProvider) RunBackup(
})
repoWriter := kopia.NewShimRepo(kp.bkRepo)
kpUploader := snapshotfs.NewUploader(repoWriter)
prorgess := new(kopia.KopiaProgress)
prorgess.InitThrottle(backupProgressCheckInterval)
prorgess.Updater = updater
kpUploader.Progress = prorgess
progress := new(kopia.KopiaProgress)
progress.InitThrottle(backupProgressCheckInterval)
progress.Updater = updater
progress.Log = log
kpUploader.Progress = progress
quit := make(chan struct{})
log.Info("Starting backup")
go kp.CheckContext(ctx, quit, nil, kpUploader)

View File

@@ -137,7 +137,7 @@ func (rp *resticProvider) RunBackup(
summary, stderrBuf, err := restic.RunBackup(backupCmd, log, updater)
if err != nil {
if strings.Contains(err.Error(), "snapshot is empty") {
if strings.Contains(stderrBuf, "snapshot is empty") {
log.Debugf("Restic backup got empty dir with %s path", path)
return "", true, nil
}

View File

@@ -17,45 +17,88 @@ limitations under the License.
package logging
import (
"errors"
"fmt"
"sync"
"github.com/sirupsen/logrus"
"github.com/vmware-tanzu/velero/pkg/util/results"
)
// LogCounterHook is a logrus hook that counts the number of log
// statements that have been written at each logrus level.
type LogCounterHook struct {
mu sync.RWMutex
counts map[logrus.Level]int
// LogHook is a logrus hook that counts the number of log
// statements that have been written at each logrus level. It also
// maintains log entries at each logrus level in result structure.
type LogHook struct {
mu sync.RWMutex
counts map[logrus.Level]int
entries map[logrus.Level]*results.Result
}
// NewLogCounterHook returns a pointer to an initialized LogCounterHook.
func NewLogCounterHook() *LogCounterHook {
return &LogCounterHook{
counts: make(map[logrus.Level]int),
// NewLogHook returns a pointer to an initialized LogHook.
func NewLogHook() *LogHook {
return &LogHook{
counts: make(map[logrus.Level]int),
entries: make(map[logrus.Level]*results.Result),
}
}
// Levels returns the logrus levels that the hook should be fired for.
func (h *LogCounterHook) Levels() []logrus.Level {
func (h *LogHook) Levels() []logrus.Level {
return logrus.AllLevels
}
// Fire executes the hook's logic.
func (h *LogCounterHook) Fire(entry *logrus.Entry) error {
func (h *LogHook) Fire(entry *logrus.Entry) error {
h.mu.Lock()
defer h.mu.Unlock()
h.counts[entry.Level]++
if h.entries[entry.Level] == nil {
h.entries[entry.Level] = &results.Result{}
}
namespace, isNamespacePresent := entry.Data["namespace"]
errorField, isErrorFieldPresent := entry.Data["error"]
resourceField, isResourceFieldPresent := entry.Data["resource"]
nameField, isNameFieldPresent := entry.Data["name"]
entryMessage := ""
if isResourceFieldPresent {
entryMessage = fmt.Sprintf("%s resource: /%s", entryMessage, resourceField.(string))
}
if isNameFieldPresent {
entryMessage = fmt.Sprintf("%s name: /%s", entryMessage, nameField.(string))
}
if isErrorFieldPresent {
entryMessage = fmt.Sprintf("%s error: /%s", entryMessage, errorField.(error).Error())
}
if isNamespacePresent {
h.entries[entry.Level].Add(namespace.(string), errors.New(entryMessage))
} else {
h.entries[entry.Level].AddVeleroError(errors.New(entryMessage))
}
return nil
}
// GetCount returns the number of log statements that have been
// written at the specific level provided.
func (h *LogCounterHook) GetCount(level logrus.Level) int {
func (h *LogHook) GetCount(level logrus.Level) int {
h.mu.RLock()
defer h.mu.RUnlock()
return h.counts[level]
}
// GetEntries returns the log statements that have been
// written at the specific level provided.
func (h *LogHook) GetEntries(level logrus.Level) results.Result {
h.mu.RLock()
defer h.mu.RUnlock()
response, isPresent := h.entries[level]
if isPresent {
return *response
}
return results.Result{}
}

View File

@@ -14,10 +14,10 @@ See the License for the specific language governing permissions and
limitations under the License.
*/
package restore
package results
// Result is a collection of messages that were generated during
// execution of a restore. This will typically store either
// execution of a backup or restore. This will typically store either
// warning or error messages.
type Result struct {
// Velero is a slice of messages related to the operation of Velero
@@ -25,12 +25,12 @@ type Result struct {
// cloud, reading a backup file, etc.)
Velero []string `json:"velero,omitempty"`
// Cluster is a slice of messages related to restoring cluster-
// scoped resources.
// Cluster is a slice of messages related to backup or restore of
// cluster-scoped resources.
Cluster []string `json:"cluster,omitempty"`
// Namespaces is a map of namespace name to slice of messages
// related to restoring namespace-scoped resources.
// related to backup or restore namespace-scoped resources.
Namespaces map[string][]string `json:"namespaces,omitempty"`
}

View File

@@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License.
*/
package restore
package results
import (
"testing"

View File

@@ -0,0 +1,25 @@
/*
Copyright 2017 the Velero contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package client
// Make sure we import the client-go auth provider plugins.
import (
_ "k8s.io/client-go/plugin/pkg/client/auth/azure"
_ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
_ "k8s.io/client-go/plugin/pkg/client/auth/oidc"
)

View File

@@ -0,0 +1,70 @@
/*
Copyright 2017, 2019 the Velero contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package client
import (
"fmt"
"runtime"
"github.com/pkg/errors"
"k8s.io/client-go/rest"
"k8s.io/client-go/tools/clientcmd"
"github.com/vmware-tanzu/velero/pkg/buildinfo"
)
func buildConfigFromFlags(context, kubeconfigPath string, precedence []string) (*rest.Config, error) {
return clientcmd.NewNonInteractiveDeferredLoadingClientConfig(
&clientcmd.ClientConfigLoadingRules{ExplicitPath: kubeconfigPath, Precedence: precedence},
&clientcmd.ConfigOverrides{
CurrentContext: context,
}).ClientConfig()
}
// Config returns a *rest.Config, using either the kubeconfig (if specified) or an in-cluster
// configuration.
func Config(kubeconfig, kubecontext, baseName string, qps float32, burst int) (*rest.Config, error) {
loadingRules := clientcmd.NewDefaultClientConfigLoadingRules()
loadingRules.ExplicitPath = kubeconfig
clientConfig, err := buildConfigFromFlags(kubecontext, kubeconfig, loadingRules.Precedence)
if err != nil {
return nil, errors.Wrap(err, "error finding Kubernetes API server config in --kubeconfig, $KUBECONFIG, or in-cluster configuration")
}
if qps > 0.0 {
clientConfig.QPS = qps
}
if burst > 0 {
clientConfig.Burst = burst
}
clientConfig.UserAgent = buildUserAgent(
baseName,
buildinfo.Version,
buildinfo.FormattedGitSHA(),
runtime.GOOS,
runtime.GOARCH,
)
return clientConfig, nil
}
// buildUserAgent builds a User-Agent string from given args.
func buildUserAgent(command, version, formattedSha, os, arch string) string {
return fmt.Sprintf(
"%s/%s (%s/%s) %s", command, version, os, arch, formattedSha)
}

View File

@@ -0,0 +1,51 @@
/*
Copyright 2018 the Velero contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package client
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestBuildUserAgent(t *testing.T) {
tests := []struct {
name string
command string
os string
arch string
gitSha string
version string
expected string
}{
{
name: "Test general interpolation in correct order",
command: "velero",
os: "darwin",
arch: "amd64",
gitSha: "abc123",
version: "v0.1.1",
expected: "velero/v0.1.1 (darwin/amd64) abc123",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
resp := buildUserAgent(test.command, test.version, test.gitSha, test.os, test.arch)
assert.Equal(t, resp, test.expected)
})
}
}

View File

@@ -0,0 +1,151 @@
/*
Copyright 2021 the Velero contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package client
import (
"encoding/json"
"os"
"path/filepath"
"strconv"
"strings"
"github.com/pkg/errors"
)
const (
ConfigKeyNamespace = "namespace"
ConfigKeyFeatures = "features"
ConfigKeyCACert = "cacert"
ConfigKeyColorized = "colorized"
)
// VeleroConfig is a map of strings to interface{} for deserializing Velero client config options.
// The alias is a way to attach type-asserting convenience methods.
type VeleroConfig map[string]interface{}
// LoadConfig loads the Velero client configuration file and returns it as a VeleroConfig. If the
// file does not exist, an empty map is returned.
func LoadConfig() (VeleroConfig, error) {
fileName := configFileName()
_, err := os.Stat(fileName)
if os.IsNotExist(err) {
// If the file isn't there, just return an empty map
return VeleroConfig{}, nil
}
if err != nil {
// For any other Stat() error, return it
return nil, errors.WithStack(err)
}
configFile, err := os.Open(fileName)
if err != nil {
return nil, errors.WithStack(err)
}
defer configFile.Close()
var config VeleroConfig
if err := json.NewDecoder(configFile).Decode(&config); err != nil {
return nil, errors.WithStack(err)
}
return config, nil
}
// SaveConfig saves the passed in config map to the Velero client configuration file.
func SaveConfig(config VeleroConfig) error {
fileName := configFileName()
// Try to make the directory in case it doesn't exist
dir := filepath.Dir(fileName)
if err := os.MkdirAll(dir, 0700); err != nil {
return errors.WithStack(err)
}
configFile, err := os.OpenFile(fileName, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0600)
if err != nil {
return errors.WithStack(err)
}
defer configFile.Close()
return json.NewEncoder(configFile).Encode(&config)
}
func (c VeleroConfig) Namespace() string {
val, ok := c[ConfigKeyNamespace]
if !ok {
return ""
}
ns, ok := val.(string)
if !ok {
return ""
}
return ns
}
func (c VeleroConfig) Features() []string {
val, ok := c[ConfigKeyFeatures]
if !ok {
return []string{}
}
features, ok := val.(string)
if !ok {
return []string{}
}
return strings.Split(features, ",")
}
func (c VeleroConfig) Colorized() bool {
val, ok := c[ConfigKeyColorized]
if !ok {
return true
}
valString, ok := val.(string)
if !ok {
return true
}
colorized, err := strconv.ParseBool(valString)
if err != nil {
return true
}
return colorized
}
func (c VeleroConfig) CACertFile() string {
val, ok := c[ConfigKeyCACert]
if !ok {
return ""
}
caCertFile, ok := val.(string)
if !ok {
return ""
}
return caCertFile
}
func configFileName() string {
return filepath.Join(os.Getenv("HOME"), ".config", "velero", "config.json")
}

View File

@@ -0,0 +1,34 @@
/*
Copyright 2021 the Velero contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package client
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestVeleroConfig(t *testing.T) {
c := VeleroConfig{
"namespace": "foo",
"features": "feature1,feature2",
}
assert.Equal(t, "foo", c.Namespace())
assert.Equal(t, []string{"feature1", "feature2"}, c.Features())
assert.Equal(t, true, c.Colorized())
}

View File

@@ -0,0 +1,141 @@
/*
Copyright 2017 the Velero contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package client
import (
"context"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/runtime/schema"
"k8s.io/apimachinery/pkg/types"
"k8s.io/apimachinery/pkg/watch"
"k8s.io/client-go/dynamic"
)
// DynamicFactory contains methods for retrieving dynamic clients for GroupVersionResources and
// GroupVersionKinds.
type DynamicFactory interface {
// ClientForGroupVersionResource returns a Dynamic client for the given group/version
// and resource for the given namespace.
ClientForGroupVersionResource(gv schema.GroupVersion, resource metav1.APIResource, namespace string) (Dynamic, error)
}
// dynamicFactory implements DynamicFactory.
type dynamicFactory struct {
dynamicClient dynamic.Interface
}
// NewDynamicFactory returns a new ClientPool-based dynamic factory.
func NewDynamicFactory(dynamicClient dynamic.Interface) DynamicFactory {
return &dynamicFactory{dynamicClient: dynamicClient}
}
func (f *dynamicFactory) ClientForGroupVersionResource(gv schema.GroupVersion, resource metav1.APIResource, namespace string) (Dynamic, error) {
return &dynamicResourceClient{
resourceClient: f.dynamicClient.Resource(gv.WithResource(resource.Name)).Namespace(namespace),
}, nil
}
// Creator creates an object.
type Creator interface {
// Create creates an object.
Create(obj *unstructured.Unstructured) (*unstructured.Unstructured, error)
}
// Lister lists objects.
type Lister interface {
// List lists all the objects of a given resource.
List(metav1.ListOptions) (*unstructured.UnstructuredList, error)
}
// Watcher watches objects.
type Watcher interface {
// Watch watches for changes to objects of a given resource.
Watch(metav1.ListOptions) (watch.Interface, error)
}
// Getter gets an object.
type Getter interface {
// Get fetches an object by name.
Get(name string, opts metav1.GetOptions) (*unstructured.Unstructured, error)
}
// Patcher patches an object.
type Patcher interface {
//Patch patches the named object using the provided patch bytes, which are expected to be in JSON merge patch format. The patched object is returned.
Patch(name string, data []byte) (*unstructured.Unstructured, error)
}
// Deletor deletes an object.
type Deletor interface {
//Patch patches the named object using the provided patch bytes, which are expected to be in JSON merge patch format. The patched object is returned.
Delete(name string, opts metav1.DeleteOptions) error
}
// StatusUpdater updates status field of a object
type StatusUpdater interface {
UpdateStatus(obj *unstructured.Unstructured, opts metav1.UpdateOptions) (*unstructured.Unstructured, error)
}
// Dynamic contains client methods that Velero needs for backing up and restoring resources.
type Dynamic interface {
Creator
Lister
Watcher
Getter
Patcher
Deletor
StatusUpdater
}
// dynamicResourceClient implements Dynamic.
type dynamicResourceClient struct {
resourceClient dynamic.ResourceInterface
}
var _ Dynamic = &dynamicResourceClient{}
func (d *dynamicResourceClient) Create(obj *unstructured.Unstructured) (*unstructured.Unstructured, error) {
return d.resourceClient.Create(context.TODO(), obj, metav1.CreateOptions{})
}
func (d *dynamicResourceClient) List(options metav1.ListOptions) (*unstructured.UnstructuredList, error) {
return d.resourceClient.List(context.TODO(), options)
}
func (d *dynamicResourceClient) Watch(options metav1.ListOptions) (watch.Interface, error) {
return d.resourceClient.Watch(context.TODO(), options)
}
func (d *dynamicResourceClient) Get(name string, opts metav1.GetOptions) (*unstructured.Unstructured, error) {
return d.resourceClient.Get(context.TODO(), name, opts)
}
func (d *dynamicResourceClient) Patch(name string, data []byte) (*unstructured.Unstructured, error) {
return d.resourceClient.Patch(context.TODO(), name, types.MergePatchType, data, metav1.PatchOptions{})
}
func (d *dynamicResourceClient) Delete(name string, opts metav1.DeleteOptions) error {
return d.resourceClient.Delete(context.TODO(), name, opts)
}
func (d *dynamicResourceClient) UpdateStatus(obj *unstructured.Unstructured, opts metav1.UpdateOptions) (*unstructured.Unstructured, error) {
return d.resourceClient.UpdateStatus(context.TODO(), obj, opts)
}

View File

@@ -0,0 +1,185 @@
/*
Copyright 2017, 2019 the Velero contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package client
import (
"os"
apiextv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
apiextv1beta1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1"
k8scheme "k8s.io/client-go/kubernetes/scheme"
kbclient "sigs.k8s.io/controller-runtime/pkg/client"
"github.com/pkg/errors"
"github.com/spf13/pflag"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/client-go/dynamic"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/rest"
velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
clientset "github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned"
)
// Factory knows how to create a VeleroClient and Kubernetes client.
type Factory interface {
// BindFlags binds common flags (--kubeconfig, --namespace) to the passed-in FlagSet.
BindFlags(flags *pflag.FlagSet)
// Client returns a VeleroClient. It uses the following priority to specify the cluster
// configuration: --kubeconfig flag, KUBECONFIG environment variable, in-cluster configuration.
Client() (clientset.Interface, error)
// KubeClient returns a Kubernetes client. It uses the following priority to specify the cluster
// configuration: --kubeconfig flag, KUBECONFIG environment variable, in-cluster configuration.
KubeClient() (kubernetes.Interface, error)
// DynamicClient returns a Kubernetes dynamic client. It uses the following priority to specify the cluster
// configuration: --kubeconfig flag, KUBECONFIG environment variable, in-cluster configuration.
DynamicClient() (dynamic.Interface, error)
// KubebuilderClient returns a client for the controller runtime framework. It adds Kubernetes and Velero
// types to its scheme. It uses the following priority to specify the cluster
// configuration: --kubeconfig flag, KUBECONFIG environment variable, in-cluster configuration.
KubebuilderClient() (kbclient.Client, error)
// SetBasename changes the basename for an already-constructed client.
// This is useful for generating clients that require a different user-agent string below the root `velero`
// command, such as the server subcommand.
SetBasename(string)
// SetClientQPS sets the Queries Per Second for a client.
SetClientQPS(float32)
// SetClientBurst sets the Burst for a client.
SetClientBurst(int)
// ClientConfig returns a rest.Config struct used for client-go clients.
ClientConfig() (*rest.Config, error)
// Namespace returns the namespace which the Factory will create clients for.
Namespace() string
}
type factory struct {
flags *pflag.FlagSet
kubeconfig string
kubecontext string
baseName string
namespace string
clientQPS float32
clientBurst int
}
// NewFactory returns a Factory.
func NewFactory(baseName, kubecontext string, config VeleroConfig) Factory {
f := &factory{
flags: pflag.NewFlagSet("", pflag.ContinueOnError),
baseName: baseName,
kubecontext: kubecontext,
}
f.namespace = os.Getenv("VELERO_NAMESPACE")
if config.Namespace() != "" {
f.namespace = config.Namespace()
}
// We didn't get the namespace via env var or config file, so use the default.
// Command line flags will override when BindFlags is called.
if f.namespace == "" {
f.namespace = velerov1api.DefaultNamespace
}
f.flags.StringVar(&f.kubeconfig, "kubeconfig", "", "Path to the kubeconfig file to use to talk to the Kubernetes apiserver. If unset, try the environment variable KUBECONFIG, as well as in-cluster configuration")
f.flags.StringVarP(&f.namespace, "namespace", "n", f.namespace, "The namespace in which Velero should operate")
//f.flags.StringVar(&f.kubecontext, "kubecontext", "", "The context to use to talk to the Kubernetes apiserver. If unset defaults to whatever your current-context is (kubectl config current-context)")
return f
}
func (f *factory) BindFlags(flags *pflag.FlagSet) {
flags.AddFlagSet(f.flags)
}
func (f *factory) ClientConfig() (*rest.Config, error) {
return Config(f.kubeconfig, f.kubecontext, f.baseName, f.clientQPS, f.clientBurst)
}
func (f *factory) Client() (clientset.Interface, error) {
clientConfig, err := f.ClientConfig()
if err != nil {
return nil, err
}
veleroClient, err := clientset.NewForConfig(clientConfig)
if err != nil {
return nil, errors.WithStack(err)
}
return veleroClient, nil
}
func (f *factory) KubeClient() (kubernetes.Interface, error) {
clientConfig, err := f.ClientConfig()
if err != nil {
return nil, err
}
kubeClient, err := kubernetes.NewForConfig(clientConfig)
if err != nil {
return nil, errors.WithStack(err)
}
return kubeClient, nil
}
func (f *factory) DynamicClient() (dynamic.Interface, error) {
clientConfig, err := f.ClientConfig()
if err != nil {
return nil, err
}
dynamicClient, err := dynamic.NewForConfig(clientConfig)
if err != nil {
return nil, errors.WithStack(err)
}
return dynamicClient, nil
}
func (f *factory) KubebuilderClient() (kbclient.Client, error) {
clientConfig, err := f.ClientConfig()
if err != nil {
return nil, err
}
scheme := runtime.NewScheme()
velerov1api.AddToScheme(scheme)
k8scheme.AddToScheme(scheme)
apiextv1beta1.AddToScheme(scheme)
apiextv1.AddToScheme(scheme)
kubebuilderClient, err := kbclient.New(clientConfig, kbclient.Options{
Scheme: scheme,
})
if err != nil {
return nil, err
}
return kubebuilderClient, nil
}
func (f *factory) SetBasename(name string) {
f.baseName = name
}
func (f *factory) SetClientQPS(qps float32) {
f.clientQPS = qps
}
func (f *factory) SetClientBurst(burst int) {
f.clientBurst = burst
}
func (f *factory) Namespace() string {
return f.namespace
}

View File

@@ -0,0 +1,61 @@
/*
Copyright 2019 the Velero contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package client
import (
"os"
"testing"
"github.com/spf13/pflag"
"github.com/stretchr/testify/assert"
)
// TestFactory tests the client.Factory interface.
func TestFactory(t *testing.T) {
// Velero client configuration is currently omitted due to requiring a
// test filesystem in pkg/test. This causes an import cycle as pkg/test
// uses pkg/client's interfaces to implement fakes
// Env variable should set the namespace if no config or argument are used
os.Setenv("VELERO_NAMESPACE", "env-velero")
f := NewFactory("velero", "", make(map[string]interface{}))
assert.Equal(t, "env-velero", f.Namespace())
os.Unsetenv("VELERO_NAMESPACE")
// Argument should change the namespace
f = NewFactory("velero", "", make(map[string]interface{}))
s := "flag-velero"
flags := new(pflag.FlagSet)
f.BindFlags(flags)
flags.Parse([]string{"--namespace", s})
assert.Equal(t, s, f.Namespace())
// An argument overrides the env variable if both are set.
os.Setenv("VELERO_NAMESPACE", "env-velero")
f = NewFactory("velero", "", make(map[string]interface{}))
flags = new(pflag.FlagSet)
f.BindFlags(flags)
flags.Parse([]string{"--namespace", s})
assert.Equal(t, s, f.Namespace())
os.Unsetenv("VELERO_NAMESPACE")
}

View File

@@ -20,7 +20,7 @@ import (
"k8s.io/client-go/kubernetes"
kbclient "sigs.k8s.io/controller-runtime/pkg/client"
"github.com/vmware-tanzu/velero/pkg/client"
"github.com/vmware-tanzu/velero/test/e2e/pkg/client"
)
// TestClient contains different API clients that are in use throughout