Compare commits

..

26 Commits

Author SHA1 Message Date
lyndon
7416504e3a Merge pull request #5905 from danfengliu/cherrypick-rollback-velero-client-factory-code
[Cherrypick 1.10] rollback velero client factory code
2023-02-23 15:03:54 +08:00
lyndon
f80c947a19 Merge branch 'release-1.10' into cherrypick-rollback-velero-client-factory-code 2023-02-23 14:50:06 +08:00
lyndon
f5124ecf7a Merge pull request #5909 from Lyndon-Li/release-1.10
Fix main CI problem 02
2023-02-23 12:40:10 +08:00
Lyndon-Li
81e52c5f70 fix main CI problem 02
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-02-23 12:28:09 +08:00
lyndon
7e031a7936 Merge pull request #5908 from Lyndon-Li/release-1.10
Fix main CI disk space error
2023-02-23 11:15:50 +08:00
Lyndon-Li
e6d9f2ef40 fix main CI disk space error
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-02-23 10:59:09 +08:00
danfengl
19076eec1d Roll back pkg client code for Velero server
Signed-off-by: danfengl <danfengl@vmware.com>
2023-02-22 14:09:23 +00:00
danfengl
1084cc01c3 Merge branch 'release-1.10' of https://github.com/vmware-tanzu/velero into release-1.10 2023-02-22 14:03:30 +00:00
qiuming
dcf1b00569 Merge pull request #5897 from Lyndon-Li/release-1.10
1.10.2 change log
2023-02-22 10:35:30 +08:00
Lyndon-Li
4d2e1ccdce 1.10.2 change log
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-02-22 10:00:32 +08:00
lyndon
b665614f94 Merge pull request #5891 from blackpiglet/release-1.10
Update distroless image and fix CVE-2022-41717 for release-1.10
2023-02-21 15:25:29 +08:00
Xun Jiang
82ef5317c5 Update distroless image and fix CVE-2022-41717 for release-1.10
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-02-21 15:10:21 +08:00
lyndon
2b6a55958d Merge pull request #5890 from qiuming-best/release-1.10
[cherry-pick]Set Kopia IgnoreUnknownTypes in ErrorHandlingPolicy to True for ignoring backup unknown file type
2023-02-21 15:09:02 +08:00
Ming
7698482256 Set Kopia IgnoreUnknownTypes in ErrorHandlingPolicy to True for ignoring backup unknown file type
Signed-off-by: Ming <mqiu@vmware.com>
2023-02-21 06:47:59 +00:00
lyndon
dbb1bc2912 Merge pull request #5879 from reasonerjt/cp-backup-result-1.10
[Cherrypick-1.10]Publish backup results extracted from backup logs
2023-02-21 13:57:29 +08:00
lyndon
b192665024 Merge branch 'release-1.10' into cp-backup-result-1.10 2023-02-21 13:42:26 +08:00
lyndon
cecbde4423 Merge pull request #5888 from blackpiglet/release-1.10
[cherry-pick][release-1.10]Add labels for velero installed namespace to support PSA
2023-02-21 12:51:10 +08:00
Xun Jiang
aa2287d100 Add labels for created namespace during velero installation to adopt k8s v1.25's PSS and PSA.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-02-21 11:28:30 +08:00
Wenkai Yin(尹文开)
4512160be4 Restore finalizer and managedFields (#5877)
Restore finalizer and managedFields of metadata during the restoration

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2023-02-20 16:56:05 +08:00
Anshul Ahuja
9973e2e0c4 Publish backup results extracted from backup logs
Signed-off-by: Anshul Ahuja <anshulahuja@microsoft.com>
2023-02-20 11:08:36 +08:00
Xun Jiang/Bruce Jiang
42fb499b9b Merge pull request #5869 from ywk253100/230217_secret
[cherry-pick]Add secret restore item action to handle service account token secret
2023-02-17 17:12:08 +08:00
Wenkai Yin(尹文开)
3fc787f630 Add secret restore item action to handle service account token secret
Add secret restore item action to handle service account token secret:
1. Skip the restoration for the auto-created service account token secret
2. Remove several fields for non-auto-created service account token secret to make sure the secret can be restored

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2023-02-17 10:50:02 +08:00
lyndon
1e015ecb15 Merge pull request #5830 from kaovilai/pvb-pvr-should-fail-during-startup-release-1.10
release-1.10: Use updated PVB/PVR for patching Failed Phase during startup
2023-02-08 10:05:11 +08:00
Tiger Kaovilai
820fe8a559 move UpdatePVXStatusToFailed to controller pkg
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-02-07 10:21:22 -05:00
Tiger Kaovilai
dc2f12743b Use updated PVB/PVR for patching Failed Phase during startup
Use the same pvb/pvr update functions across pkg/controller and pkg/cli/nodeagent for consistency of behavior

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-02-03 14:49:09 -05:00
Xun Jiang
118a4e2f72 Remove container-builder-env section.
​
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-01-16 07:37:02 +00:00
46 changed files with 1481 additions and 131 deletions

View File

@@ -99,4 +99,7 @@ jobs:
- name: Publish container image to GCR
if: github.repository == 'vmware-tanzu/velero'
run: |
sudo swapoff -a
sudo rm -f /mnt/swapfile
docker image prune -a --force
REGISTRY=gcr.io/velero-gcp ./hack/docker-push.sh

View File

@@ -66,7 +66,7 @@ RUN mkdir -p /output/usr/bin && \
/go/src/github.com/vmware-tanzu/velero/hack/build-restic.sh
# Velero image packing section
FROM gcr.io/distroless/base-debian11@sha256:99133cb0878bb1f84d1753957c6fd4b84f006f2798535de22ebf7ba170bbf434
FROM gcr.io/distroless/base-debian11@sha256:db7ea5913b13bb5fc4a5f5ee5a1fab693d262846d3e7b28efd3f8f62f835c161
LABEL maintainer="Nolan Brubaker <brubakern@vmware.com>"

View File

@@ -1,3 +1,27 @@
## v1.10.2
### 2023-02-22
### Download
https://github.com/vmware-tanzu/velero/releases/tag/v1.10.2
### Container Image
`velero/velero:v1.10.2`
### Documentation
https://velero.io/docs/v1.10/
### Upgrading
https://velero.io/docs/v1.10/upgrade-to-1.10/
### All changes
* Update distroless image and fix CVE-2022-41717 for release-1.10 (#5891, @blackpiglet)
* Set Kopia IgnoreUnknownTypes in ErrorHandlingPolicy to True for ignoring backup unknown file type (#5890, @qiuiming-best)
* Add labels for velero installed namespace to support PSA. (#5888, @blackpiglet)
* Publish backupresults json to enhance error info during backups. (#5879, @anshulahuja98)
* Restore finalizer and managedFields of metadata during the restoration (#5877, @ywk253100)
* Add secret restore item action to handle service account token secret (#5869, @ywk253100)
* Correct PVB/PVR Failed Phase patching during startup (#5830, @kaovilai)
## v1.10.1
### 2023-01-19

10
go.mod
View File

@@ -35,7 +35,7 @@ require (
github.com/stretchr/testify v1.7.1
github.com/vmware-tanzu/crash-diagnostics v0.3.7
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4
golang.org/x/net v0.1.1-0.20221104162952-702349b0e862
golang.org/x/net v0.4.0
golang.org/x/oauth2 v0.0.0-20220608161450-d0670ef3b1eb
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c
google.golang.org/api v0.74.0
@@ -132,9 +132,9 @@ require (
go.uber.org/zap v1.21.0 // indirect
golang.org/x/crypto v0.0.0-20220315160706-3147a52a75dd // indirect
golang.org/x/exp v0.0.0-20210916165020-5cb4fee858ee // indirect
golang.org/x/sys v0.1.0 // indirect
golang.org/x/term v0.1.0 // indirect
golang.org/x/text v0.4.0 // indirect
golang.org/x/sys v0.3.0 // indirect
golang.org/x/term v0.3.0 // indirect
golang.org/x/text v0.5.0 // indirect
golang.org/x/time v0.0.0-20220609170525-579cf78fd858 // indirect
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
gomodules.xyz/jsonpatch/v2 v2.2.0 // indirect
@@ -150,5 +150,3 @@ require (
sigs.k8s.io/json v0.0.0-20220525155127-227cbc7cc124 // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.2.1 // indirect
)
replace github.com/gogo/protobuf => github.com/gogo/protobuf v1.3.2

23
go.sum
View File

@@ -293,6 +293,9 @@ github.com/gobwas/glob v0.2.3/go.mod h1:d3Ez4x06l9bZtSvzIay5+Yzi0fmZzPgnTbPcKjJA
github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
github.com/gofrs/flock v0.8.1 h1:+gYjHKf32LDeiEEFhQaotPbLuUXjY5ZqxKgXy7n59aw=
github.com/gofrs/flock v0.8.1/go.mod h1:F1TvTiK9OcQqauNUHlbJvyl9Qa1QvF/gOUDKA14jxHU=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
github.com/gogo/protobuf v1.3.1/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/golang-jwt/jwt/v4 v4.0.0/go.mod h1:/xlHOz8bRuivTWchD4jCa+NbatV+wEUSzwAxVc6locg=
@@ -475,6 +478,8 @@ github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/X
github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=
github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00=
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/compress v1.15.1 h1:y9FcTHGyrebwfP0ZZqFiaxTaiDnUrGkJkI+f583BL1A=
@@ -905,8 +910,8 @@ golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qx
golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
golang.org/x/net v0.0.0-20220325170049-de3da57026de/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
golang.org/x/net v0.1.1-0.20221104162952-702349b0e862 h1:KrLJ+iz8J6j6VVr/OCfULAcK+xozUmWE43fKpMR4MlI=
golang.org/x/net v0.1.1-0.20221104162952-702349b0e862/go.mod h1:Cx3nUiGt4eDBEyega/BKRp+/AlGL8hYe7U9odMt2Cco=
golang.org/x/net v0.4.0 h1:Q5QPcMlvfxFTAPV0+07Xz/MpK9NTXu2VDUuy0FeMfaU=
golang.org/x/net v0.4.0/go.mod h1:MBQ8lrhLObU/6UmLb4fmbmk5OcyYmqtbGd/9yIeKjEE=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -1028,14 +1033,14 @@ golang.org/x/sys v0.0.0-20220209214540-3681064d5158/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220227234510-4e6760a101f9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220328115105-d36c6a25d886/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220412211240-33da011f77ad/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.1.0 h1:kunALQeHf1/185U1i0GOB/fy1IPRDDpuoOOqRReG57U=
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.3.0 h1:w8ZOecv6NaNa/zC8944JTU3vz4u6Lagfk4RPQxv92NQ=
golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210220032956-6a3ed077a48d/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.1.0 h1:g6Z6vPFA9dYBAF7DWcH6sCcOntplXsDKcliusYijMlw=
golang.org/x/term v0.1.0/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.3.0 h1:qoo4akIqOcDME5bhc/NgxUdovd6BSS2uMsVjB56q1xI=
golang.org/x/term v0.3.0/go.mod h1:q750SLmJuPmVoN1blW3UFBPREJfb1KmY3vwxfr+nFDA=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -1045,8 +1050,8 @@ golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.4.0 h1:BrVqGRd7+k1DiOgtnFvAkoQEWQvBc25ouMJM6429SFg=
golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.5.0 h1:OLmvp0KP+FVG99Ct/qFiL/Fhk4zp4QQnZ7b2U+5piUM=
golang.org/x/text v0.5.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
@@ -1057,8 +1062,10 @@ golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac/go.mod h1:tRJNPiyCQ0inRvYxb
golang.org/x/time v0.0.0-20220210224613-90d013bbcef8/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20220609170525-579cf78fd858 h1:Dpdu/EMxGMFgq0CeYMh4fazTD2vtlZRYE7wyynxJb9U=
golang.org/x/time v0.0.0-20220609170525-579cf78fd858/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20181011042414-1f849cf54d09/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20181030221726-6c7e314b6563/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=

View File

@@ -1,5 +1,5 @@
diff --git a/go.mod b/go.mod
index d819a6be7..0629ed935 100644
index d819a6be7..41125958a 100644
--- a/go.mod
+++ b/go.mod
@@ -35,12 +35,12 @@ require (
@@ -7,20 +7,20 @@ index d819a6be7..0629ed935 100644
github.com/spf13/pflag v1.0.5
golang.org/x/crypto v0.0.0-20220817201139-bc19a97f63c8
- golang.org/x/net v0.0.0-20220822230855-b0a4917ee28c
+ golang.org/x/net v0.1.1-0.20221104162952-702349b0e862
+ golang.org/x/net v0.4.0
golang.org/x/oauth2 v0.0.0-20220622183110-fd043fe589d2
golang.org/x/sync v0.0.0-20220819030929-7fc1605a5dde
- golang.org/x/sys v0.0.0-20220818161305-2296e01440c6
- golang.org/x/term v0.0.0-20220722155259-a9ba230a4035
- golang.org/x/text v0.3.7
+ golang.org/x/sys v0.1.0
+ golang.org/x/term v0.1.0
+ golang.org/x/text v0.4.0
+ golang.org/x/sys v0.3.0
+ golang.org/x/term v0.3.0
+ golang.org/x/text v0.5.0
google.golang.org/api v0.93.0
google.golang.org/genproto v0.0.0-20220822174746-9e6da59bd2fc // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
diff --git a/go.sum b/go.sum
index 959651048..8dea7af8a 100644
index 959651048..b7a4358d5 100644
--- a/go.sum
+++ b/go.sum
@@ -319,6 +319,7 @@ github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9de
@@ -45,8 +45,8 @@ index 959651048..8dea7af8a 100644
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
-golang.org/x/net v0.0.0-20220822230855-b0a4917ee28c h1:JVAXQ10yGGVbSyoer5VILysz6YKjdNT2bsvlayjqhes=
-golang.org/x/net v0.0.0-20220822230855-b0a4917ee28c/go.mod h1:YDH+HFinaLZZlnHAfSS6ZXJJ9M9t4Dl22yv3iI2vPwk=
+golang.org/x/net v0.1.1-0.20221104162952-702349b0e862 h1:KrLJ+iz8J6j6VVr/OCfULAcK+xozUmWE43fKpMR4MlI=
+golang.org/x/net v0.1.1-0.20221104162952-702349b0e862/go.mod h1:Cx3nUiGt4eDBEyega/BKRp+/AlGL8hYe7U9odMt2Cco=
+golang.org/x/net v0.4.0 h1:Q5QPcMlvfxFTAPV0+07Xz/MpK9NTXu2VDUuy0FeMfaU=
+golang.org/x/net v0.4.0/go.mod h1:MBQ8lrhLObU/6UmLb4fmbmk5OcyYmqtbGd/9yIeKjEE=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -65,14 +65,14 @@ index 959651048..8dea7af8a 100644
-golang.org/x/sys v0.0.0-20220728004956-3c1f35247d10/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20220818161305-2296e01440c6 h1:Sx/u41w+OwrInGdEckYmEuU5gHoGSL4QbDz3S9s6j4U=
-golang.org/x/sys v0.0.0-20220818161305-2296e01440c6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.1.0 h1:kunALQeHf1/185U1i0GOB/fy1IPRDDpuoOOqRReG57U=
+golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.3.0 h1:w8ZOecv6NaNa/zC8944JTU3vz4u6Lagfk4RPQxv92NQ=
+golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
-golang.org/x/term v0.0.0-20220722155259-a9ba230a4035 h1:Q5284mrmYTpACcm+eAKjKJH48BBwSyfJqmmGDTtT8Vc=
-golang.org/x/term v0.0.0-20220722155259-a9ba230a4035/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
+golang.org/x/term v0.1.0 h1:g6Z6vPFA9dYBAF7DWcH6sCcOntplXsDKcliusYijMlw=
+golang.org/x/term v0.1.0/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
+golang.org/x/term v0.3.0 h1:qoo4akIqOcDME5bhc/NgxUdovd6BSS2uMsVjB56q1xI=
+golang.org/x/term v0.3.0/go.mod h1:q750SLmJuPmVoN1blW3UFBPREJfb1KmY3vwxfr+nFDA=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -82,8 +82,8 @@ index 959651048..8dea7af8a 100644
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
-golang.org/x/text v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
+golang.org/x/text v0.4.0 h1:BrVqGRd7+k1DiOgtnFvAkoQEWQvBc25ouMJM6429SFg=
+golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
+golang.org/x/text v0.5.0 h1:OLmvp0KP+FVG99Ct/qFiL/Fhk4zp4QQnZ7b2U+5piUM=
+golang.org/x/text v0.5.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
@@ -95,3 +95,4 @@ index 959651048..8dea7af8a 100644
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=

View File

@@ -146,3 +146,10 @@ func WithGenerateName(val string) func(obj metav1.Object) {
obj.SetGenerateName(val)
}
}
// WithManagedFields is a functional option that applies the specified managed fields to an object.
func WithManagedFields(val []metav1.ManagedFieldsEntry) func(obj metav1.Object) {
return func(obj metav1.Object) {
obj.SetManagedFields(val)
}
}

View File

@@ -27,20 +27,15 @@ import (
"github.com/vmware-tanzu/velero/pkg/buildinfo"
)
func buildConfigFromFlags(context, kubeconfigPath string, precedence []string) (*rest.Config, error) {
return clientcmd.NewNonInteractiveDeferredLoadingClientConfig(
&clientcmd.ClientConfigLoadingRules{ExplicitPath: kubeconfigPath, Precedence: precedence},
&clientcmd.ConfigOverrides{
CurrentContext: context,
}).ClientConfig()
}
// Config returns a *rest.Config, using either the kubeconfig (if specified) or an in-cluster
// configuration.
func Config(kubeconfig, kubecontext, baseName string, qps float32, burst int) (*rest.Config, error) {
loadingRules := clientcmd.NewDefaultClientConfigLoadingRules()
loadingRules.ExplicitPath = kubeconfig
clientConfig, err := buildConfigFromFlags(kubecontext, kubeconfig, loadingRules.Precedence)
configOverrides := &clientcmd.ConfigOverrides{CurrentContext: kubecontext}
kubeConfig := clientcmd.NewNonInteractiveDeferredLoadingClientConfig(loadingRules, configOverrides)
clientConfig, err := kubeConfig.ClientConfig()
if err != nil {
return nil, errors.Wrap(err, "error finding Kubernetes API server config in --kubeconfig, $KUBECONFIG, or in-cluster configuration")
}

View File

@@ -77,11 +77,10 @@ type factory struct {
}
// NewFactory returns a Factory.
func NewFactory(baseName, kubecontext string, config VeleroConfig) Factory {
func NewFactory(baseName string, config VeleroConfig) Factory {
f := &factory{
flags: pflag.NewFlagSet("", pflag.ContinueOnError),
baseName: baseName,
kubecontext: kubecontext,
flags: pflag.NewFlagSet("", pflag.ContinueOnError),
baseName: baseName,
}
f.namespace = os.Getenv("VELERO_NAMESPACE")
@@ -97,7 +96,7 @@ func NewFactory(baseName, kubecontext string, config VeleroConfig) Factory {
f.flags.StringVar(&f.kubeconfig, "kubeconfig", "", "Path to the kubeconfig file to use to talk to the Kubernetes apiserver. If unset, try the environment variable KUBECONFIG, as well as in-cluster configuration")
f.flags.StringVarP(&f.namespace, "namespace", "n", f.namespace, "The namespace in which Velero should operate")
//f.flags.StringVar(&f.kubecontext, "kubecontext", "", "The context to use to talk to the Kubernetes apiserver. If unset defaults to whatever your current-context is (kubectl config current-context)")
f.flags.StringVar(&f.kubecontext, "kubecontext", "", "The context to use to talk to the Kubernetes apiserver. If unset defaults to whatever your current-context is (kubectl config current-context)")
return f
}
@@ -128,6 +127,7 @@ func (f *factory) KubeClient() (kubernetes.Interface, error) {
return nil, err
}
kubeClient, err := kubernetes.NewForConfig(clientConfig)
if err != nil {
return nil, errors.WithStack(err)
}

View File

@@ -31,14 +31,14 @@ func TestFactory(t *testing.T) {
// Env variable should set the namespace if no config or argument are used
os.Setenv("VELERO_NAMESPACE", "env-velero")
f := NewFactory("velero", "", make(map[string]interface{}))
f := NewFactory("velero", make(map[string]interface{}))
assert.Equal(t, "env-velero", f.Namespace())
os.Unsetenv("VELERO_NAMESPACE")
// Argument should change the namespace
f = NewFactory("velero", "", make(map[string]interface{}))
f = NewFactory("velero", make(map[string]interface{}))
s := "flag-velero"
flags := new(pflag.FlagSet)
@@ -50,7 +50,7 @@ func TestFactory(t *testing.T) {
// An argument overrides the env variable if both are set.
os.Setenv("VELERO_NAMESPACE", "env-velero")
f = NewFactory("velero", "", make(map[string]interface{}))
f = NewFactory("velero", make(map[string]interface{}))
flags = new(pflag.FlagSet)
f.BindFlags(flags)

View File

@@ -303,11 +303,10 @@ func (s *nodeAgentServer) markInProgressPVBsFailed(client ctrlclient.Client) {
s.logger.Debugf("the node of podvolumebackup %q is %q, not %q, skip", pvb.GetName(), pvb.Spec.Node, s.nodeName)
continue
}
original := pvb.DeepCopy()
pvb.Status.Phase = velerov1api.PodVolumeBackupPhaseFailed
pvb.Status.Message = fmt.Sprintf("get a podvolumebackup with status %q during the server starting, mark it as %q", velerov1api.PodVolumeBackupPhaseInProgress, pvb.Status.Phase)
pvb.Status.CompletionTimestamp = &metav1.Time{Time: time.Now()}
if err := client.Patch(s.ctx, &pvbs.Items[i], ctrlclient.MergeFrom(original)); err != nil {
if err := controller.UpdatePVBStatusToFailed(client, s.ctx, &pvbs.Items[i],
fmt.Sprintf("get a podvolumebackup with status %q during the server starting, mark it as %q", velerov1api.PodVolumeBackupPhaseInProgress, velerov1api.PodVolumeBackupPhaseFailed),
time.Now()); err != nil {
s.logger.WithError(errors.WithStack(err)).Errorf("failed to patch podvolumebackup %q", pvb.GetName())
continue
}
@@ -341,11 +340,9 @@ func (s *nodeAgentServer) markInProgressPVRsFailed(client ctrlclient.Client) {
continue
}
original := pvr.DeepCopy()
pvr.Status.Phase = velerov1api.PodVolumeRestorePhaseFailed
pvr.Status.Message = fmt.Sprintf("get a podvolumerestore with status %q during the server starting, mark it as %q", velerov1api.PodVolumeRestorePhaseInProgress, pvr.Status.Phase)
pvr.Status.CompletionTimestamp = &metav1.Time{Time: time.Now()}
if err := client.Patch(s.ctx, &pvrs.Items[i], ctrlclient.MergeFrom(original)); err != nil {
if err := controller.UpdatePVRStatusToFailed(client, s.ctx, &pvrs.Items[i],
fmt.Sprintf("get a podvolumerestore with status %q during the server starting, mark it as %q", velerov1api.PodVolumeRestorePhaseInProgress, velerov1api.PodVolumeRestorePhaseFailed),
time.Now()); err != nil {
s.logger.WithError(errors.WithStack(err)).Errorf("failed to patch podvolumerestore %q", pvr.GetName())
continue
}

View File

@@ -57,7 +57,8 @@ func NewCommand(f client.Factory) *cobra.Command {
RegisterRestoreItemAction("velero.io/crd-preserve-fields", newCRDV1PreserveUnknownFieldsItemAction).
RegisterRestoreItemAction("velero.io/change-pvc-node-selector", newChangePVCNodeSelectorItemAction(f)).
RegisterRestoreItemAction("velero.io/apiservice", newAPIServiceRestoreItemAction).
RegisterRestoreItemAction("velero.io/admission-webhook-configuration", newAdmissionWebhookConfigurationAction)
RegisterRestoreItemAction("velero.io/admission-webhook-configuration", newAdmissionWebhookConfigurationAction).
RegisterRestoreItemAction("velero.io/secret", newSecretRestoreItemAction(f))
if !features.IsEnabled(velerov1api.APIGroupVersionsFeatureFlag) {
// Do not register crd-remap-version BIA if the API Group feature flag is enabled, so that the v1 CRD can be backed up
pluginServer = pluginServer.RegisterBackupItemAction("velero.io/crd-remap-version", newRemapCRDVersionAction(f))
@@ -220,3 +221,13 @@ func newAPIServiceRestoreItemAction(logger logrus.FieldLogger) (interface{}, err
func newAdmissionWebhookConfigurationAction(logger logrus.FieldLogger) (interface{}, error) {
return restore.NewAdmissionWebhookConfigurationAction(logger), nil
}
func newSecretRestoreItemAction(f client.Factory) plugincommon.HandlerInitializer {
return func(logger logrus.FieldLogger) (interface{}, error) {
client, err := f.KubebuilderClient()
if err != nil {
return nil, err
}
return restore.NewSecretAction(logger, client), nil
}
}

View File

@@ -499,9 +499,9 @@ func (s *server) veleroResourcesExist() error {
// - VolumeSnapshots are needed to create PVCs using the VolumeSnapshot as their data source.
// - PVs go before PVCs because PVCs depend on them.
// - PVCs go before pods or controllers so they can be mounted as volumes.
// - Service accounts go before secrets so service account token secrets can be filled automatically.
// - Secrets and config maps go before pods or controllers so they can be mounted
// as volumes.
// - Service accounts go before pods or controllers so pods can use them.
// - Limit ranges go before pods or controllers so pods can use them.
// - Pods go before controllers so they can be explicitly restored and potentially
// have pod volume restores run before controllers adopt the pods.
@@ -525,9 +525,9 @@ var defaultRestorePriorities = restore.Priorities{
"volumesnapshots.snapshot.storage.k8s.io",
"persistentvolumes",
"persistentvolumeclaims",
"serviceaccounts",
"secrets",
"configmaps",
"serviceaccounts",
"limitranges",
"pods",
// we fully qualify replicasets.apps because prior to Kubernetes 1.16, replicasets also

View File

@@ -32,7 +32,7 @@ import (
velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
"github.com/vmware-tanzu/velero/pkg/cmd/util/downloadrequest"
clientset "github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned"
pkgrestore "github.com/vmware-tanzu/velero/pkg/restore"
"github.com/vmware-tanzu/velero/pkg/util/results"
)
func DescribeRestore(ctx context.Context, kbClient kbclient.Client, restore *velerov1api.Restore, podVolumeRestores []velerov1api.PodVolumeRestore, details bool, veleroClient clientset.Interface, insecureSkipTLSVerify bool, caCertFile string) string {
@@ -167,7 +167,7 @@ func describeRestoreResults(ctx context.Context, kbClient kbclient.Client, d *De
}
var buf bytes.Buffer
var resultMap map[string]pkgrestore.Result
var resultMap map[string]results.Result
if err := downloadrequest.Stream(ctx, kbClient, restore.Namespace, restore.Name, velerov1api.DownloadTargetKindRestoreResults, &buf, downloadRequestTimeout, insecureSkipTLSVerify, caCertPath); err != nil {
d.Printf("Warnings:\t<error getting warnings: %v>\n\nErrors:\t<error getting errors: %v>\n", err, err)
@@ -189,7 +189,7 @@ func describeRestoreResults(ctx context.Context, kbClient kbclient.Client, d *De
}
}
func describeRestoreResult(d *Describer, name string, result pkgrestore.Result) {
func describeRestoreResult(d *Describer, name string, result results.Result) {
d.Printf("%s:\n", name)
d.DescribeSlice(1, "Velero", result.Velero)
d.DescribeSlice(1, "Cluster", result.Cluster)

View File

@@ -91,7 +91,7 @@ operations can also be performed as 'velero backup get' and 'velero schedule cre
},
}
f := client.NewFactory(name, "", config)
f := client.NewFactory(name, config)
f.BindFlags(c.PersistentFlags())
// Bind features directly to the root command so it's available to all callers.

View File

@@ -43,6 +43,8 @@ import (
"k8s.io/apimachinery/pkg/util/wait"
"k8s.io/client-go/tools/cache"
"github.com/vmware-tanzu/velero/pkg/util/results"
"github.com/vmware-tanzu/velero/pkg/util/csi"
snapshotv1api "github.com/kubernetes-csi/external-snapshotter/client/v4/apis/volumesnapshot/v1"
@@ -603,7 +605,7 @@ func (c *backupController) runBackup(backup *pkgbackup.Request) error {
logger := logging.DefaultLogger(c.backupLogLevel, c.formatFlag)
logger.Out = io.MultiWriter(os.Stdout, gzippedLogFile)
logCounter := logging.NewLogCounterHook()
logCounter := logging.NewLogHook()
logger.Hooks.Add(logCounter)
backupLog := logger.WithField(Backup, kubeutil.NamespaceAndName(backup))
@@ -725,6 +727,13 @@ func (c *backupController) runBackup(backup *pkgbackup.Request) error {
recordBackupMetrics(backupLog, backup.Backup, backupFile, c.metrics)
backupWarnings := logCounter.GetEntries(logrus.WarnLevel)
backupErrors := logCounter.GetEntries(logrus.ErrorLevel)
results := map[string]results.Result{
"warnings": backupWarnings,
"errors": backupErrors,
}
if err := gzippedLogFile.Close(); err != nil {
c.logger.WithField(Backup, kubeutil.NamespaceAndName(backup)).WithError(err).Error("error closing gzippedLogFile")
}
@@ -750,7 +759,7 @@ func (c *backupController) runBackup(backup *pkgbackup.Request) error {
return err
}
if errs := persistBackup(backup, backupFile, logFile, backupStore, volumeSnapshots, volumeSnapshotContents, volumeSnapshotClasses); len(errs) > 0 {
if errs := persistBackup(backup, backupFile, logFile, backupStore, volumeSnapshots, volumeSnapshotContents, volumeSnapshotClasses, results); len(errs) > 0 {
fatalErrs = append(fatalErrs, errs...)
}
@@ -797,6 +806,7 @@ func persistBackup(backup *pkgbackup.Request,
csiVolumeSnapshots []snapshotv1api.VolumeSnapshot,
csiVolumeSnapshotContents []snapshotv1api.VolumeSnapshotContent,
csiVolumesnapshotClasses []snapshotv1api.VolumeSnapshotClass,
results map[string]results.Result,
) []error {
persistErrs := []error{}
backupJSON := new(bytes.Buffer)
@@ -835,6 +845,11 @@ func persistBackup(backup *pkgbackup.Request,
persistErrs = append(persistErrs, errs...)
}
backupResult, errs := encodeToJSONGzip(results, "backup results")
if errs != nil {
persistErrs = append(persistErrs, errs...)
}
if len(persistErrs) > 0 {
// Don't upload the JSON files or backup tarball if encoding to json fails.
backupJSON = nil
@@ -844,6 +859,7 @@ func persistBackup(backup *pkgbackup.Request,
csiSnapshotJSON = nil
csiSnapshotContentsJSON = nil
csiSnapshotClassesJSON = nil
backupResult = nil
}
backupInfo := persistence.BackupInfo{
@@ -851,6 +867,7 @@ func persistBackup(backup *pkgbackup.Request,
Metadata: backupJSON,
Contents: backupContents,
Log: backupLog,
BackupResults: backupResult,
PodVolumeBackups: podVolumeBackups,
VolumeSnapshots: nativeVolumeSnapshots,
BackupResourceList: backupResourceList,

View File

@@ -279,19 +279,22 @@ func (r *PodVolumeBackupReconciler) getParentSnapshot(ctx context.Context, log l
}
func (r *PodVolumeBackupReconciler) updateStatusToFailed(ctx context.Context, pvb *velerov1api.PodVolumeBackup, err error, msg string, log logrus.FieldLogger) (ctrl.Result, error) {
original := pvb.DeepCopy()
pvb.Status.Phase = velerov1api.PodVolumeBackupPhaseFailed
pvb.Status.Message = errors.WithMessage(err, msg).Error()
pvb.Status.CompletionTimestamp = &metav1.Time{Time: r.Clock.Now()}
if err = r.Client.Patch(ctx, pvb, client.MergeFrom(original)); err != nil {
if err = UpdatePVBStatusToFailed(r.Client, ctx, pvb, errors.WithMessage(err, msg).Error(), r.Clock.Now()); err != nil {
log.WithError(err).Error("error updating PodVolumeBackup status")
return ctrl.Result{}, err
}
return ctrl.Result{}, nil
}
func UpdatePVBStatusToFailed(c client.Client, ctx context.Context, pvb *velerov1api.PodVolumeBackup, errString string, time time.Time) error {
original := pvb.DeepCopy()
pvb.Status.Phase = velerov1api.PodVolumeBackupPhaseFailed
pvb.Status.Message = errString
pvb.Status.CompletionTimestamp = &metav1.Time{Time: time}
return c.Patch(ctx, pvb, client.MergeFrom(original))
}
func (r *PodVolumeBackupReconciler) NewBackupProgressUpdater(pvb *velerov1api.PodVolumeBackup, log logrus.FieldLogger, ctx context.Context) *BackupProgressUpdater {
return &BackupProgressUpdater{pvb, log, ctx, r.Client}
}

View File

@@ -22,6 +22,7 @@ import (
"io/ioutil"
"os"
"path/filepath"
"time"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
@@ -122,11 +123,7 @@ func (c *PodVolumeRestoreReconciler) Reconcile(ctx context.Context, req ctrl.Req
}
if err = c.processRestore(ctx, pvr, pod, log); err != nil {
original = pvr.DeepCopy()
pvr.Status.Phase = velerov1api.PodVolumeRestorePhaseFailed
pvr.Status.Message = err.Error()
pvr.Status.CompletionTimestamp = &metav1.Time{Time: c.clock.Now()}
if e := c.Patch(ctx, pvr, client.MergeFrom(original)); e != nil {
if e := UpdatePVRStatusToFailed(c, ctx, pvr, err.Error(), c.clock.Now()); e != nil {
log.WithError(err).Error("Unable to update status to failed")
}
@@ -145,6 +142,15 @@ func (c *PodVolumeRestoreReconciler) Reconcile(ctx context.Context, req ctrl.Req
return ctrl.Result{}, nil
}
func UpdatePVRStatusToFailed(c client.Client, ctx context.Context, pvr *velerov1api.PodVolumeRestore, errString string, time time.Time) error {
original := pvr.DeepCopy()
pvr.Status.Phase = velerov1api.PodVolumeRestorePhaseFailed
pvr.Status.Message = errString
pvr.Status.CompletionTimestamp = &metav1.Time{Time: time}
return c.Patch(ctx, pvr, client.MergeFrom(original))
}
func (c *PodVolumeRestoreReconciler) shouldProcess(ctx context.Context, log logrus.FieldLogger, pvr *velerov1api.PodVolumeRestore) (bool, *corev1api.Pod, error) {
if !isPVRNew(pvr) {
log.Debug("PodVolumeRestore is not new, skip")

View File

@@ -53,6 +53,7 @@ import (
"github.com/vmware-tanzu/velero/pkg/util/collections"
kubeutil "github.com/vmware-tanzu/velero/pkg/util/kube"
"github.com/vmware-tanzu/velero/pkg/util/logging"
"github.com/vmware-tanzu/velero/pkg/util/results"
"sigs.k8s.io/controller-runtime/pkg/client"
)
@@ -574,7 +575,7 @@ func (c *restoreController) runValidatedRestore(restore *api.Restore, info backu
restore.Status.Errors += len(e)
}
m := map[string]pkgrestore.Result{
m := map[string]results.Result{
"warnings": restoreWarnings,
"errors": restoreErrors,
}
@@ -586,7 +587,7 @@ func (c *restoreController) runValidatedRestore(restore *api.Restore, info backu
return nil
}
func putResults(restore *api.Restore, results map[string]pkgrestore.Result, backupStore persistence.BackupStore) error {
func putResults(restore *api.Restore, results map[string]results.Result, backupStore persistence.BackupStore) error {
buf := new(bytes.Buffer)
gzw := gzip.NewWriter(buf)
defer gzw.Close()

View File

@@ -51,6 +51,7 @@ import (
pkgrestore "github.com/vmware-tanzu/velero/pkg/restore"
velerotest "github.com/vmware-tanzu/velero/pkg/test"
"github.com/vmware-tanzu/velero/pkg/util/logging"
"github.com/vmware-tanzu/velero/pkg/util/results"
"github.com/vmware-tanzu/velero/pkg/volume"
)
@@ -506,7 +507,7 @@ func TestProcessQueueItem(t *testing.T) {
sharedInformers.Velero().V1().Backups().Informer().GetStore().Add(test.backup)
}
var warnings, errors pkgrestore.Result
var warnings, errors results.Result
if test.restorerError != nil {
errors.Namespaces = map[string][]string{"ns-1": {test.restorerError.Error()}}
}
@@ -864,12 +865,12 @@ func (r *fakeRestorer) Restore(
actions []riav1.RestoreItemAction,
snapshotLocationLister listers.VolumeSnapshotLocationLister,
volumeSnapshotterGetter pkgrestore.VolumeSnapshotterGetter,
) (pkgrestore.Result, pkgrestore.Result) {
) (results.Result, results.Result) {
res := r.Called(info.Log, info.Restore, info.Backup, info.BackupReader, actions)
r.calledWithArg = *info.Restore
return res.Get(0).(pkgrestore.Result), res.Get(1).(pkgrestore.Result)
return res.Get(0).(results.Result), res.Get(1).(results.Result)
}
func (r *fakeRestorer) RestoreWithResolvers(req pkgrestore.Request,
@@ -877,11 +878,11 @@ func (r *fakeRestorer) RestoreWithResolvers(req pkgrestore.Request,
itemSnapshotterResolver framework.ItemSnapshotterResolver,
snapshotLocationLister listers.VolumeSnapshotLocationLister,
volumeSnapshotterGetter pkgrestore.VolumeSnapshotterGetter,
) (pkgrestore.Result, pkgrestore.Result) {
) (results.Result, results.Result) {
res := r.Called(req.Log, req.Restore, req.Backup, req.BackupReader, resolver, itemSnapshotterResolver,
snapshotLocationLister, volumeSnapshotterGetter)
r.calledWithArg = *req.Restore
return res.Get(0).(pkgrestore.Result), res.Get(1).(pkgrestore.Result)
return res.Get(0).(results.Result), res.Get(1).(results.Result)
}

View File

@@ -136,13 +136,18 @@ func ClusterRoleBinding(namespace string) *rbacv1.ClusterRoleBinding {
}
func Namespace(namespace string) *corev1.Namespace {
return &corev1.Namespace{
ns := &corev1.Namespace{
ObjectMeta: objectMeta("", namespace),
TypeMeta: metav1.TypeMeta{
Kind: "Namespace",
APIVersion: corev1.SchemeGroupVersion.String(),
},
}
ns.Labels["pod-security.kubernetes.io/enforce"] = "privileged"
ns.Labels["pod-security.kubernetes.io/enforce-version"] = "latest"
return ns
}
func BackupStorageLocation(namespace, provider, bucket, prefix string, config map[string]string, caCert []byte) *velerov1api.BackupStorageLocation {

View File

@@ -40,6 +40,11 @@ func TestResources(t *testing.T) {
ns := Namespace("velero")
assert.Equal(t, "velero", ns.Name)
// For k8s version v1.25 and later, need to add the following labels to make
// velero installation namespace has privileged version to work with
// PSA(Pod Security Admission) and PSS(Pod Security Standards).
assert.Equal(t, ns.Labels["pod-security.kubernetes.io/enforce"], "privileged")
assert.Equal(t, ns.Labels["pod-security.kubernetes.io/enforce-version"], "latest")
crb := ClusterRoleBinding(DefaultVeleroNamespace)
// The CRB is a cluster-scoped resource

View File

@@ -42,6 +42,7 @@ type BackupInfo struct {
Metadata,
Contents,
Log,
BackupResults,
PodVolumeBackups,
VolumeSnapshots,
ItemSnapshots,
@@ -261,6 +262,7 @@ func (s *objectBackupStore) PutBackup(info BackupInfo) error {
s.layout.getCSIVolumeSnapshotKey(info.Name): info.CSIVolumeSnapshots,
s.layout.getCSIVolumeSnapshotContentsKey(info.Name): info.CSIVolumeSnapshotContents,
s.layout.getCSIVolumeSnapshotClassesKey(info.Name): info.CSIVolumeSnapshotClasses,
s.layout.getBackupResultsKey(info.Name): info.BackupResults,
}
for key, reader := range backupObjs {

View File

@@ -116,3 +116,7 @@ func (l *ObjectStoreLayout) getCSIVolumeSnapshotContentsKey(backup string) strin
func (l *ObjectStoreLayout) getCSIVolumeSnapshotClassesKey(backup string) string {
return path.Join(l.subdirs["backups"], backup, fmt.Sprintf("%s-csi-volumesnapshotclasses.json.gz", backup))
}
func (l *ObjectStoreLayout) getBackupResultsKey(backup string) string {
return path.Join(l.subdirs["backups"], backup, fmt.Sprintf("%s-results.gz", backup))
}

View File

@@ -214,7 +214,7 @@ func userPriorityConfigMap() (*corev1.ConfigMap, error) {
return nil, errors.Wrap(err, "reading client config file")
}
fc := client.NewFactory("APIGroupVersionsRestore", "", cfg)
fc := client.NewFactory("APIGroupVersionsRestore", cfg)
kc, err := fc.KubeClient()
if err != nil {

View File

@@ -67,6 +67,7 @@ import (
"github.com/vmware-tanzu/velero/pkg/util/collections"
"github.com/vmware-tanzu/velero/pkg/util/filesystem"
"github.com/vmware-tanzu/velero/pkg/util/kube"
. "github.com/vmware-tanzu/velero/pkg/util/results"
"github.com/vmware-tanzu/velero/pkg/volume"
)
@@ -1421,6 +1422,24 @@ func (ctx *restoreContext) restoreItem(obj *unstructured.Unstructured, groupReso
}
}
// restore the managedFields
withoutManagedFields := createdObj.DeepCopy()
createdObj.SetManagedFields(obj.GetManagedFields())
patchBytes, err := generatePatch(withoutManagedFields, createdObj)
if err != nil {
ctx.log.Errorf("error generating patch for managed fields %s: %v", kube.NamespaceAndName(obj), err)
errs.Add(namespace, err)
return warnings, errs
}
if patchBytes != nil {
if _, err = resourceClient.Patch(name, patchBytes); err != nil {
ctx.log.Errorf("error patch for managed fields %s: %v", kube.NamespaceAndName(obj), err)
errs.Add(namespace, err)
return warnings, errs
}
ctx.log.Infof("the managed fields for %s is patched", kube.NamespaceAndName(obj))
}
if groupResource == kuberesource.Pods {
pod := new(v1.Pod)
if err := runtime.DefaultUnstructuredConverter.FromUnstructured(obj.UnstructuredContent(), pod); err != nil {
@@ -1720,8 +1739,8 @@ func resetMetadata(obj *unstructured.Unstructured) (*unstructured.Unstructured,
for k := range metadata {
switch k {
case "name", "namespace", "labels", "annotations":
default:
case "generateName", "selfLink", "uid", "resourceVersion", "generation", "creationTimestamp", "deletionTimestamp",
"deletionGracePeriodSeconds", "ownerReferences":
delete(metadata, k)
}
}

View File

@@ -56,6 +56,7 @@ import (
testutil "github.com/vmware-tanzu/velero/pkg/test"
"github.com/vmware-tanzu/velero/pkg/util/kube"
kubeutil "github.com/vmware-tanzu/velero/pkg/util/kube"
. "github.com/vmware-tanzu/velero/pkg/util/results"
"github.com/vmware-tanzu/velero/pkg/volume"
)
@@ -867,7 +868,7 @@ func TestRestoreItems(t *testing.T) {
want []*test.APIResource
}{
{
name: "metadata other than namespace/name/labels/annotations gets removed",
name: "metadata uid/resourceVersion/etc. gets removed",
restore: defaultRestore().Result(),
backup: defaultBackup().Result(),
tarball: test.NewTarWriter(t).
@@ -877,6 +878,7 @@ func TestRestoreItems(t *testing.T) {
builder.WithLabels("key-1", "val-1"),
builder.WithAnnotations("key-1", "val-1"),
builder.WithFinalizers("finalizer-1"),
builder.WithUID("uid"),
).
Result(),
).
@@ -890,6 +892,7 @@ func TestRestoreItems(t *testing.T) {
ObjectMeta(
builder.WithLabels("key-1", "val-1", "velero.io/backup-name", "backup-1", "velero.io/restore-name", "restore-1"),
builder.WithAnnotations("key-1", "val-1"),
builder.WithFinalizers("finalizer-1"),
).
Result(),
),
@@ -1107,6 +1110,53 @@ func TestRestoreItems(t *testing.T) {
}),
},
},
{
name: "metadata managedFields gets restored",
restore: defaultRestore().Result(),
backup: defaultBackup().Result(),
tarball: test.NewTarWriter(t).
AddItems("pods",
builder.ForPod("ns-1", "pod-1").
ObjectMeta(
builder.WithManagedFields([]metav1.ManagedFieldsEntry{
{
Manager: "kubectl",
Operation: "Apply",
APIVersion: "v1",
FieldsType: "FieldsV1",
FieldsV1: &metav1.FieldsV1{
Raw: []byte(`{"f:data": {"f:key":{}}}`),
},
},
}),
).
Result(),
).
Done(),
apiResources: []*test.APIResource{
test.Pods(),
},
want: []*test.APIResource{
test.Pods(
builder.ForPod("ns-1", "pod-1").
ObjectMeta(
builder.WithLabels("velero.io/backup-name", "backup-1", "velero.io/restore-name", "restore-1"),
builder.WithManagedFields([]metav1.ManagedFieldsEntry{
{
Manager: "kubectl",
Operation: "Apply",
APIVersion: "v1",
FieldsType: "FieldsV1",
FieldsV1: &metav1.FieldsV1{
Raw: []byte(`{"f:data": {"f:key":{}}}`),
},
},
}),
).
Result(),
),
},
},
}
for _, tc := range tests {
@@ -2823,10 +2873,16 @@ func TestResetMetadata(t *testing.T) {
expectedErr: true,
},
{
name: "keep name, namespace, labels, annotations only",
obj: NewTestUnstructured().WithMetadata("name", "blah", "namespace", "labels", "annotations", "foo").Unstructured,
name: "keep name, namespace, labels, annotations, managedFields, finalizers",
obj: NewTestUnstructured().WithMetadata("name", "namespace", "labels", "annotations", "managedFields", "finalizers").Unstructured,
expectedErr: false,
expectedRes: NewTestUnstructured().WithMetadata("name", "namespace", "labels", "annotations").Unstructured,
expectedRes: NewTestUnstructured().WithMetadata("name", "namespace", "labels", "annotations", "managedFields", "finalizers").Unstructured,
},
{
name: "remove uid, ownerReferences",
obj: NewTestUnstructured().WithMetadata("name", "namespace", "uid", "ownerReferences").Unstructured,
expectedErr: false,
expectedRes: NewTestUnstructured().WithMetadata("name", "namespace").Unstructured,
},
{
name: "keep status",

View File

@@ -0,0 +1,107 @@
/*
Copyright The Velero Contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package restore
import (
"context"
"fmt"
"strings"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"github.com/vmware-tanzu/velero/pkg/plugin/velero"
"github.com/vmware-tanzu/velero/pkg/util/kube"
)
// SecretAction is a restore item action for secrets
type SecretAction struct {
logger logrus.FieldLogger
client client.Client
}
// NewSecretAction creates a new SecretAction instance
func NewSecretAction(logger logrus.FieldLogger, client client.Client) *SecretAction {
return &SecretAction{
logger: logger,
client: client,
}
}
// AppliesTo indicates which resources this action applies
func (s *SecretAction) AppliesTo() (velero.ResourceSelector, error) {
return velero.ResourceSelector{
IncludedResources: []string{"secrets"},
}, nil
}
// Execute the action
func (s *SecretAction) Execute(input *velero.RestoreItemActionExecuteInput) (*velero.RestoreItemActionExecuteOutput, error) {
s.logger.Info("Executing SecretAction")
defer s.logger.Info("Done executing SecretAction")
var secret corev1.Secret
if err := runtime.DefaultUnstructuredConverter.FromUnstructured(input.Item.UnstructuredContent(), &secret); err != nil {
return nil, errors.Wrap(err, "unable to convert secret from runtime.Unstructured")
}
log := s.logger.WithField("secret", kube.NamespaceAndName(&secret))
if secret.Type != corev1.SecretTypeServiceAccountToken {
log.Debug("No match found - including this secret")
return &velero.RestoreItemActionExecuteOutput{
UpdatedItem: input.Item,
}, nil
}
// The auto created service account token secret will be created by kube controller automatically again(before Kubernetes v1.22), no need to restore.
// This will cause the patch operation of managedFields failed if we restore it as the secret is removed immediately
// after restoration and the patch operation reports not found error.
list := &corev1.ServiceAccountList{}
if err := s.client.List(context.Background(), list, &client.ListOptions{Namespace: secret.Namespace}); err != nil {
return nil, errors.Wrap(err, "unable to list the service accounts")
}
for _, sa := range list.Items {
if strings.HasPrefix(secret.Name, fmt.Sprintf("%s-token-", sa.Name)) {
log.Debug("auto created service account token secret found - excluding this secret")
return &velero.RestoreItemActionExecuteOutput{
UpdatedItem: input.Item,
SkipRestore: true,
}, nil
}
}
log.Debug("service account token secret(not auto created) found - remove some fields from this secret")
// If the annotation and data are not removed, the secret cannot be restored successfully.
// The kube controller will fill the annotation and data with new value automatically:
// https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets
delete(secret.Annotations, "kubernetes.io/service-account.uid")
delete(secret.Data, "token")
delete(secret.Data, "ca.crt")
res, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&secret)
if err != nil {
return nil, errors.Wrap(err, "unable to convert secret to runtime.Unstructured")
}
return &velero.RestoreItemActionExecuteOutput{
UpdatedItem: &unstructured.Unstructured{Object: res},
}, nil
}

View File

@@ -0,0 +1,142 @@
/*
Copyright The Velero Contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package restore
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/client/fake"
"github.com/vmware-tanzu/velero/pkg/plugin/velero"
"github.com/vmware-tanzu/velero/pkg/test"
)
func TestSecretActionAppliesTo(t *testing.T) {
action := NewSecretAction(test.NewLogger(), nil)
actual, err := action.AppliesTo()
require.NoError(t, err)
assert.Equal(t, velero.ResourceSelector{IncludedResources: []string{"secrets"}}, actual)
}
func TestSecretActionExecute(t *testing.T) {
tests := []struct {
name string
input *corev1.Secret
serviceAccount *corev1.ServiceAccount
skipped bool
output *corev1.Secret
}{
{
name: "not service account token secret",
input: &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Namespace: "foo",
Name: "default-token-sfafa",
},
Type: corev1.SecretTypeOpaque,
},
skipped: false,
output: &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Namespace: "foo",
Name: "default-token-sfafa",
},
Type: corev1.SecretTypeOpaque,
},
},
{
name: "auto created service account token",
input: &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Namespace: "foo",
Name: "default-token-sfafa",
},
Type: corev1.SecretTypeServiceAccountToken,
},
serviceAccount: &corev1.ServiceAccount{
ObjectMeta: metav1.ObjectMeta{
Namespace: "foo",
Name: "default",
},
},
skipped: true,
},
{
name: "not auto created service account token",
input: &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Namespace: "foo",
Name: "my-token",
Annotations: map[string]string{
"kubernetes.io/service-account.uid": "uid",
"key": "value",
},
},
Type: corev1.SecretTypeServiceAccountToken,
Data: map[string][]byte{
"token": []byte("token"),
"ca.crt": []byte("ca"),
"key": []byte("value"),
},
},
skipped: false,
output: &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
Namespace: "foo",
Name: "my-token",
Annotations: map[string]string{
"key": "value",
},
},
Type: corev1.SecretTypeServiceAccountToken,
Data: map[string][]byte{
"key": []byte("value"),
},
},
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
secretUnstructured, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.input)
require.NoError(t, err)
var serviceAccounts []client.Object
if tc.serviceAccount != nil {
serviceAccounts = append(serviceAccounts, tc.serviceAccount)
}
client := fake.NewClientBuilder().WithObjects(serviceAccounts...).Build()
action := NewSecretAction(test.NewLogger(), client)
res, err := action.Execute(&velero.RestoreItemActionExecuteInput{
Item: &unstructured.Unstructured{Object: secretUnstructured},
})
require.NoError(t, err)
assert.Equal(t, tc.skipped, res.SkipRestore)
if !tc.skipped {
r, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.output)
require.NoError(t, err)
assert.EqualValues(t, &unstructured.Unstructured{Object: r}, res.UpdatedItem)
}
})
}
}

View File

@@ -18,6 +18,7 @@ package restore
import (
"encoding/json"
"fmt"
"strconv"
"github.com/pkg/errors"
@@ -83,10 +84,10 @@ func deleteNodePorts(service *corev1api.Service) error {
// find any NodePorts whose values were explicitly specified according
// to the last-applied-config annotation. We'll retain these values, and
// clear out any other (presumably auto-assigned) NodePort values.
explicitNodePorts := sets.NewString()
unnamedPortInts := sets.NewInt()
lastAppliedConfig, ok := service.Annotations[annotationLastAppliedConfig]
if ok {
explicitNodePorts := sets.NewString()
unnamedPortInts := sets.NewInt()
appliedServiceUnstructured := new(map[string]interface{})
if err := json.Unmarshal([]byte(lastAppliedConfig), appliedServiceUnstructured); err != nil {
return errors.WithStack(err)
@@ -134,19 +135,58 @@ func deleteNodePorts(service *corev1api.Service) error {
}
}
}
for i, port := range service.Spec.Ports {
if port.Name != "" {
if !explicitNodePorts.Has(port.Name) {
service.Spec.Ports[i].NodePort = 0
}
} else {
if !unnamedPortInts.Has(int(port.NodePort)) {
service.Spec.Ports[i].NodePort = 0
}
}
}
return nil
}
for i, port := range service.Spec.Ports {
if port.Name != "" {
if !explicitNodePorts.Has(port.Name) {
service.Spec.Ports[i].NodePort = 0
explicitNodePorts := sets.NewString()
for _, entry := range service.GetManagedFields() {
if entry.FieldsV1 == nil {
continue
}
fields := new(map[string]interface{})
if err := json.Unmarshal(entry.FieldsV1.Raw, fields); err != nil {
return errors.WithStack(err)
}
ports, exist, err := unstructured.NestedMap(*fields, "f:spec", "f:ports")
if err != nil {
return errors.WithStack(err)
}
if !exist {
continue
}
for key, port := range ports {
p, ok := port.(map[string]interface{})
if !ok {
continue
}
} else {
if !unnamedPortInts.Has(int(port.NodePort)) {
service.Spec.Ports[i].NodePort = 0
if _, exist := p["f:nodePort"]; exist {
explicitNodePorts.Insert(key)
}
}
}
for i, port := range service.Spec.Ports {
k := portKey(port)
if !explicitNodePorts.Has(k) {
service.Spec.Ports[i].NodePort = 0
}
}
return nil
}
func portKey(port corev1api.ServicePort) string {
return fmt.Sprintf(`k:{"port":%d,"protocol":"%s"}`, port.Port, port.Protocol)
}

View File

@@ -368,6 +368,124 @@ func TestServiceActionExecute(t *testing.T) {
},
},
},
{
name: "nodePort should be delete when not specified in managedFields",
obj: corev1api.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "svc-1",
ManagedFields: []metav1.ManagedFieldsEntry{
{
FieldsV1: &metav1.FieldsV1{
Raw: []byte(`{"f:spec":{"f:ports":{"k:{\"port\":443,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{}},"k:{\"port\":80,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{}}},"f:selector":{},"f:type":{}}}`),
},
},
},
},
Spec: corev1api.ServiceSpec{
Ports: []corev1api.ServicePort{
{
Name: "http",
Port: 80,
Protocol: "TCP",
},
{
Name: "https",
Port: 443,
Protocol: "TCP",
},
},
},
},
restore: builder.ForRestore(api.DefaultNamespace, "").Result(),
expectedRes: corev1api.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "svc-1",
ManagedFields: []metav1.ManagedFieldsEntry{
{
FieldsV1: &metav1.FieldsV1{
Raw: []byte(`{"f:spec":{"f:ports":{"k:{\"port\":443,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{}},"k:{\"port\":80,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{}}},"f:selector":{},"f:type":{}}}`),
},
},
},
},
Spec: corev1api.ServiceSpec{
Ports: []corev1api.ServicePort{
{
Name: "http",
Port: 80,
NodePort: 0,
Protocol: "TCP",
},
{
Name: "https",
Port: 443,
NodePort: 0,
Protocol: "TCP",
},
},
},
},
},
{
name: "nodePort should be preserved when specified in managedFields",
obj: corev1api.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "svc-1",
ManagedFields: []metav1.ManagedFieldsEntry{
{
FieldsV1: &metav1.FieldsV1{
Raw: []byte(`{"f:spec":{"f:ports":{"k:{\"port\":443,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:nodePort":{},"f:port":{}},"k:{\"port\":80,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:nodePort":{},"f:port":{}}},"f:selector":{},"f:type":{}}}`),
},
},
},
},
Spec: corev1api.ServiceSpec{
Ports: []corev1api.ServicePort{
{
Name: "http",
Port: 80,
NodePort: 30000,
Protocol: "TCP",
},
{
Name: "https",
Port: 443,
NodePort: 30002,
Protocol: "TCP",
},
},
},
},
restore: builder.ForRestore(api.DefaultNamespace, "").Result(),
expectedRes: corev1api.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "svc-1",
ManagedFields: []metav1.ManagedFieldsEntry{
{
FieldsV1: &metav1.FieldsV1{
Raw: []byte(`{"f:spec":{"f:ports":{"k:{\"port\":443,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:nodePort":{},"f:port":{}},"k:{\"port\":80,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:nodePort":{},"f:port":{}}},"f:selector":{},"f:type":{}}}`),
},
},
},
},
Spec: corev1api.ServiceSpec{
Ports: []corev1api.ServicePort{
{
Name: "http",
Port: 80,
NodePort: 30000,
Protocol: "TCP",
},
{
Name: "https",
Port: 443,
NodePort: 30002,
Protocol: "TCP",
},
},
},
},
},
}
for _, test := range tests {

View File

@@ -20,6 +20,8 @@ import (
"sync/atomic"
"time"
"github.com/sirupsen/logrus"
"github.com/vmware-tanzu/velero/pkg/uploader"
)
@@ -63,6 +65,7 @@ type KopiaProgress struct {
processedBytes int64 // which statistic all bytes has been processed currently
outputThrottle Throttle // which control the frequency of update progress
Updater uploader.ProgressUpdater //which kopia progress will call the UpdateProgress interface, the third party will implement the interface to do the progress update
Log logrus.FieldLogger // output info into log when backup
}
//UploadedBytes the total bytes has uploaded currently
@@ -77,8 +80,10 @@ func (p *KopiaProgress) UploadedBytes(numBytes int64) {
func (p *KopiaProgress) Error(path string, err error, isIgnored bool) {
if isIgnored {
atomic.AddInt32(&p.ignoredErrorCount, 1)
p.Log.Warnf("Ignored error when processing %v: %v", path, err)
} else {
atomic.AddInt32(&p.fatalErrorCount, 1)
p.Log.Errorf("Error when processing %v: %v", path, err)
}
}

View File

@@ -18,6 +18,7 @@ package kopia
import (
"context"
"fmt"
"io/ioutil"
"math"
"os"
@@ -61,8 +62,14 @@ type SnapshotUploader interface {
) (*snapshot.Manifest, error)
}
func newOptionalInt(b policy.OptionalInt) *policy.OptionalInt {
return &b
func newOptionalInt(b int) *policy.OptionalInt {
ob := policy.OptionalInt(b)
return &ob
}
func newOptionalBool(b bool) *policy.OptionalBool {
ob := policy.OptionalBool(b)
return &ob
}
//setupDefaultPolicy set default policy for kopia
@@ -75,11 +82,14 @@ func setupDefaultPolicy(ctx context.Context, rep repo.RepositoryWriter, sourceIn
CompressorName: "none",
},
UploadPolicy: policy.UploadPolicy{
MaxParallelFileReads: newOptionalInt(policy.OptionalInt(runtime.NumCPU())),
MaxParallelFileReads: newOptionalInt(runtime.NumCPU()),
},
SchedulingPolicy: policy.SchedulingPolicy{
Manual: true,
},
ErrorHandlingPolicy: policy.ErrorHandlingPolicy{
IgnoreUnknownTypes: newOptionalBool(true),
},
})
}
@@ -212,19 +222,23 @@ func SnapshotSource(
return "", 0, errors.Wrapf(err, "Failed to flush kopia repository")
}
log.Infof("Created snapshot with root %v and ID %v in %v", manifest.RootObjectID(), manifest.ID, time.Since(snapshotStartTime).Truncate(time.Second))
return reportSnapshotStatus(manifest)
return reportSnapshotStatus(manifest, policyTree)
}
func reportSnapshotStatus(manifest *snapshot.Manifest) (string, int64, error) {
func reportSnapshotStatus(manifest *snapshot.Manifest, policyTree *policy.Tree) (string, int64, error) {
manifestID := manifest.ID
snapSize := manifest.Stats.TotalFileSize
var errs []string
if ds := manifest.RootEntry.DirSummary; ds != nil {
for _, ent := range ds.FailedEntries {
errs = append(errs, ent.Error)
policy := policyTree.DefinedPolicy()
if !(policy != nil && *policy.ErrorHandlingPolicy.IgnoreUnknownTypes == true && strings.Contains(ent.Error, fs.ErrUnknown.Error())) {
errs = append(errs, fmt.Sprintf("Error when processing %v: %v", ent.EntryPath, ent.Error))
}
}
}
if len(errs) != 0 {
return "", 0, errors.New(strings.Join(errs, "\n"))
}

View File

@@ -119,10 +119,11 @@ func (kp *kopiaProvider) RunBackup(
})
repoWriter := kopia.NewShimRepo(kp.bkRepo)
kpUploader := snapshotfs.NewUploader(repoWriter)
prorgess := new(kopia.KopiaProgress)
prorgess.InitThrottle(backupProgressCheckInterval)
prorgess.Updater = updater
kpUploader.Progress = prorgess
progress := new(kopia.KopiaProgress)
progress.InitThrottle(backupProgressCheckInterval)
progress.Updater = updater
progress.Log = log
kpUploader.Progress = progress
quit := make(chan struct{})
log.Info("Starting backup")
go kp.CheckContext(ctx, quit, nil, kpUploader)

View File

@@ -17,45 +17,88 @@ limitations under the License.
package logging
import (
"errors"
"fmt"
"sync"
"github.com/sirupsen/logrus"
"github.com/vmware-tanzu/velero/pkg/util/results"
)
// LogCounterHook is a logrus hook that counts the number of log
// statements that have been written at each logrus level.
type LogCounterHook struct {
mu sync.RWMutex
counts map[logrus.Level]int
// LogHook is a logrus hook that counts the number of log
// statements that have been written at each logrus level. It also
// maintains log entries at each logrus level in result structure.
type LogHook struct {
mu sync.RWMutex
counts map[logrus.Level]int
entries map[logrus.Level]*results.Result
}
// NewLogCounterHook returns a pointer to an initialized LogCounterHook.
func NewLogCounterHook() *LogCounterHook {
return &LogCounterHook{
counts: make(map[logrus.Level]int),
// NewLogHook returns a pointer to an initialized LogHook.
func NewLogHook() *LogHook {
return &LogHook{
counts: make(map[logrus.Level]int),
entries: make(map[logrus.Level]*results.Result),
}
}
// Levels returns the logrus levels that the hook should be fired for.
func (h *LogCounterHook) Levels() []logrus.Level {
func (h *LogHook) Levels() []logrus.Level {
return logrus.AllLevels
}
// Fire executes the hook's logic.
func (h *LogCounterHook) Fire(entry *logrus.Entry) error {
func (h *LogHook) Fire(entry *logrus.Entry) error {
h.mu.Lock()
defer h.mu.Unlock()
h.counts[entry.Level]++
if h.entries[entry.Level] == nil {
h.entries[entry.Level] = &results.Result{}
}
namespace, isNamespacePresent := entry.Data["namespace"]
errorField, isErrorFieldPresent := entry.Data["error"]
resourceField, isResourceFieldPresent := entry.Data["resource"]
nameField, isNameFieldPresent := entry.Data["name"]
entryMessage := ""
if isResourceFieldPresent {
entryMessage = fmt.Sprintf("%s resource: /%s", entryMessage, resourceField.(string))
}
if isNameFieldPresent {
entryMessage = fmt.Sprintf("%s name: /%s", entryMessage, nameField.(string))
}
if isErrorFieldPresent {
entryMessage = fmt.Sprintf("%s error: /%s", entryMessage, errorField.(error).Error())
}
if isNamespacePresent {
h.entries[entry.Level].Add(namespace.(string), errors.New(entryMessage))
} else {
h.entries[entry.Level].AddVeleroError(errors.New(entryMessage))
}
return nil
}
// GetCount returns the number of log statements that have been
// written at the specific level provided.
func (h *LogCounterHook) GetCount(level logrus.Level) int {
func (h *LogHook) GetCount(level logrus.Level) int {
h.mu.RLock()
defer h.mu.RUnlock()
return h.counts[level]
}
// GetEntries returns the log statements that have been
// written at the specific level provided.
func (h *LogHook) GetEntries(level logrus.Level) results.Result {
h.mu.RLock()
defer h.mu.RUnlock()
response, isPresent := h.entries[level]
if isPresent {
return *response
}
return results.Result{}
}

View File

@@ -14,10 +14,10 @@ See the License for the specific language governing permissions and
limitations under the License.
*/
package restore
package results
// Result is a collection of messages that were generated during
// execution of a restore. This will typically store either
// execution of a backup or restore. This will typically store either
// warning or error messages.
type Result struct {
// Velero is a slice of messages related to the operation of Velero
@@ -25,12 +25,12 @@ type Result struct {
// cloud, reading a backup file, etc.)
Velero []string `json:"velero,omitempty"`
// Cluster is a slice of messages related to restoring cluster-
// scoped resources.
// Cluster is a slice of messages related to backup or restore of
// cluster-scoped resources.
Cluster []string `json:"cluster,omitempty"`
// Namespaces is a map of namespace name to slice of messages
// related to restoring namespace-scoped resources.
// related to backup or restore namespace-scoped resources.
Namespaces map[string][]string `json:"namespaces,omitempty"`
}

View File

@@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
limitations under the License.
*/
package restore
package results
import (
"testing"

View File

@@ -0,0 +1,25 @@
/*
Copyright 2017 the Velero contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package client
// Make sure we import the client-go auth provider plugins.
import (
_ "k8s.io/client-go/plugin/pkg/client/auth/azure"
_ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
_ "k8s.io/client-go/plugin/pkg/client/auth/oidc"
)

View File

@@ -0,0 +1,70 @@
/*
Copyright 2017, 2019 the Velero contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package client
import (
"fmt"
"runtime"
"github.com/pkg/errors"
"k8s.io/client-go/rest"
"k8s.io/client-go/tools/clientcmd"
"github.com/vmware-tanzu/velero/pkg/buildinfo"
)
func buildConfigFromFlags(context, kubeconfigPath string, precedence []string) (*rest.Config, error) {
return clientcmd.NewNonInteractiveDeferredLoadingClientConfig(
&clientcmd.ClientConfigLoadingRules{ExplicitPath: kubeconfigPath, Precedence: precedence},
&clientcmd.ConfigOverrides{
CurrentContext: context,
}).ClientConfig()
}
// Config returns a *rest.Config, using either the kubeconfig (if specified) or an in-cluster
// configuration.
func Config(kubeconfig, kubecontext, baseName string, qps float32, burst int) (*rest.Config, error) {
loadingRules := clientcmd.NewDefaultClientConfigLoadingRules()
loadingRules.ExplicitPath = kubeconfig
clientConfig, err := buildConfigFromFlags(kubecontext, kubeconfig, loadingRules.Precedence)
if err != nil {
return nil, errors.Wrap(err, "error finding Kubernetes API server config in --kubeconfig, $KUBECONFIG, or in-cluster configuration")
}
if qps > 0.0 {
clientConfig.QPS = qps
}
if burst > 0 {
clientConfig.Burst = burst
}
clientConfig.UserAgent = buildUserAgent(
baseName,
buildinfo.Version,
buildinfo.FormattedGitSHA(),
runtime.GOOS,
runtime.GOARCH,
)
return clientConfig, nil
}
// buildUserAgent builds a User-Agent string from given args.
func buildUserAgent(command, version, formattedSha, os, arch string) string {
return fmt.Sprintf(
"%s/%s (%s/%s) %s", command, version, os, arch, formattedSha)
}

View File

@@ -0,0 +1,51 @@
/*
Copyright 2018 the Velero contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package client
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestBuildUserAgent(t *testing.T) {
tests := []struct {
name string
command string
os string
arch string
gitSha string
version string
expected string
}{
{
name: "Test general interpolation in correct order",
command: "velero",
os: "darwin",
arch: "amd64",
gitSha: "abc123",
version: "v0.1.1",
expected: "velero/v0.1.1 (darwin/amd64) abc123",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
resp := buildUserAgent(test.command, test.version, test.gitSha, test.os, test.arch)
assert.Equal(t, resp, test.expected)
})
}
}

View File

@@ -0,0 +1,151 @@
/*
Copyright 2021 the Velero contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package client
import (
"encoding/json"
"os"
"path/filepath"
"strconv"
"strings"
"github.com/pkg/errors"
)
const (
ConfigKeyNamespace = "namespace"
ConfigKeyFeatures = "features"
ConfigKeyCACert = "cacert"
ConfigKeyColorized = "colorized"
)
// VeleroConfig is a map of strings to interface{} for deserializing Velero client config options.
// The alias is a way to attach type-asserting convenience methods.
type VeleroConfig map[string]interface{}
// LoadConfig loads the Velero client configuration file and returns it as a VeleroConfig. If the
// file does not exist, an empty map is returned.
func LoadConfig() (VeleroConfig, error) {
fileName := configFileName()
_, err := os.Stat(fileName)
if os.IsNotExist(err) {
// If the file isn't there, just return an empty map
return VeleroConfig{}, nil
}
if err != nil {
// For any other Stat() error, return it
return nil, errors.WithStack(err)
}
configFile, err := os.Open(fileName)
if err != nil {
return nil, errors.WithStack(err)
}
defer configFile.Close()
var config VeleroConfig
if err := json.NewDecoder(configFile).Decode(&config); err != nil {
return nil, errors.WithStack(err)
}
return config, nil
}
// SaveConfig saves the passed in config map to the Velero client configuration file.
func SaveConfig(config VeleroConfig) error {
fileName := configFileName()
// Try to make the directory in case it doesn't exist
dir := filepath.Dir(fileName)
if err := os.MkdirAll(dir, 0700); err != nil {
return errors.WithStack(err)
}
configFile, err := os.OpenFile(fileName, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0600)
if err != nil {
return errors.WithStack(err)
}
defer configFile.Close()
return json.NewEncoder(configFile).Encode(&config)
}
func (c VeleroConfig) Namespace() string {
val, ok := c[ConfigKeyNamespace]
if !ok {
return ""
}
ns, ok := val.(string)
if !ok {
return ""
}
return ns
}
func (c VeleroConfig) Features() []string {
val, ok := c[ConfigKeyFeatures]
if !ok {
return []string{}
}
features, ok := val.(string)
if !ok {
return []string{}
}
return strings.Split(features, ",")
}
func (c VeleroConfig) Colorized() bool {
val, ok := c[ConfigKeyColorized]
if !ok {
return true
}
valString, ok := val.(string)
if !ok {
return true
}
colorized, err := strconv.ParseBool(valString)
if err != nil {
return true
}
return colorized
}
func (c VeleroConfig) CACertFile() string {
val, ok := c[ConfigKeyCACert]
if !ok {
return ""
}
caCertFile, ok := val.(string)
if !ok {
return ""
}
return caCertFile
}
func configFileName() string {
return filepath.Join(os.Getenv("HOME"), ".config", "velero", "config.json")
}

View File

@@ -0,0 +1,34 @@
/*
Copyright 2021 the Velero contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package client
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestVeleroConfig(t *testing.T) {
c := VeleroConfig{
"namespace": "foo",
"features": "feature1,feature2",
}
assert.Equal(t, "foo", c.Namespace())
assert.Equal(t, []string{"feature1", "feature2"}, c.Features())
assert.Equal(t, true, c.Colorized())
}

View File

@@ -0,0 +1,141 @@
/*
Copyright 2017 the Velero contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package client
import (
"context"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/runtime/schema"
"k8s.io/apimachinery/pkg/types"
"k8s.io/apimachinery/pkg/watch"
"k8s.io/client-go/dynamic"
)
// DynamicFactory contains methods for retrieving dynamic clients for GroupVersionResources and
// GroupVersionKinds.
type DynamicFactory interface {
// ClientForGroupVersionResource returns a Dynamic client for the given group/version
// and resource for the given namespace.
ClientForGroupVersionResource(gv schema.GroupVersion, resource metav1.APIResource, namespace string) (Dynamic, error)
}
// dynamicFactory implements DynamicFactory.
type dynamicFactory struct {
dynamicClient dynamic.Interface
}
// NewDynamicFactory returns a new ClientPool-based dynamic factory.
func NewDynamicFactory(dynamicClient dynamic.Interface) DynamicFactory {
return &dynamicFactory{dynamicClient: dynamicClient}
}
func (f *dynamicFactory) ClientForGroupVersionResource(gv schema.GroupVersion, resource metav1.APIResource, namespace string) (Dynamic, error) {
return &dynamicResourceClient{
resourceClient: f.dynamicClient.Resource(gv.WithResource(resource.Name)).Namespace(namespace),
}, nil
}
// Creator creates an object.
type Creator interface {
// Create creates an object.
Create(obj *unstructured.Unstructured) (*unstructured.Unstructured, error)
}
// Lister lists objects.
type Lister interface {
// List lists all the objects of a given resource.
List(metav1.ListOptions) (*unstructured.UnstructuredList, error)
}
// Watcher watches objects.
type Watcher interface {
// Watch watches for changes to objects of a given resource.
Watch(metav1.ListOptions) (watch.Interface, error)
}
// Getter gets an object.
type Getter interface {
// Get fetches an object by name.
Get(name string, opts metav1.GetOptions) (*unstructured.Unstructured, error)
}
// Patcher patches an object.
type Patcher interface {
//Patch patches the named object using the provided patch bytes, which are expected to be in JSON merge patch format. The patched object is returned.
Patch(name string, data []byte) (*unstructured.Unstructured, error)
}
// Deletor deletes an object.
type Deletor interface {
//Patch patches the named object using the provided patch bytes, which are expected to be in JSON merge patch format. The patched object is returned.
Delete(name string, opts metav1.DeleteOptions) error
}
// StatusUpdater updates status field of a object
type StatusUpdater interface {
UpdateStatus(obj *unstructured.Unstructured, opts metav1.UpdateOptions) (*unstructured.Unstructured, error)
}
// Dynamic contains client methods that Velero needs for backing up and restoring resources.
type Dynamic interface {
Creator
Lister
Watcher
Getter
Patcher
Deletor
StatusUpdater
}
// dynamicResourceClient implements Dynamic.
type dynamicResourceClient struct {
resourceClient dynamic.ResourceInterface
}
var _ Dynamic = &dynamicResourceClient{}
func (d *dynamicResourceClient) Create(obj *unstructured.Unstructured) (*unstructured.Unstructured, error) {
return d.resourceClient.Create(context.TODO(), obj, metav1.CreateOptions{})
}
func (d *dynamicResourceClient) List(options metav1.ListOptions) (*unstructured.UnstructuredList, error) {
return d.resourceClient.List(context.TODO(), options)
}
func (d *dynamicResourceClient) Watch(options metav1.ListOptions) (watch.Interface, error) {
return d.resourceClient.Watch(context.TODO(), options)
}
func (d *dynamicResourceClient) Get(name string, opts metav1.GetOptions) (*unstructured.Unstructured, error) {
return d.resourceClient.Get(context.TODO(), name, opts)
}
func (d *dynamicResourceClient) Patch(name string, data []byte) (*unstructured.Unstructured, error) {
return d.resourceClient.Patch(context.TODO(), name, types.MergePatchType, data, metav1.PatchOptions{})
}
func (d *dynamicResourceClient) Delete(name string, opts metav1.DeleteOptions) error {
return d.resourceClient.Delete(context.TODO(), name, opts)
}
func (d *dynamicResourceClient) UpdateStatus(obj *unstructured.Unstructured, opts metav1.UpdateOptions) (*unstructured.Unstructured, error) {
return d.resourceClient.UpdateStatus(context.TODO(), obj, opts)
}

View File

@@ -0,0 +1,185 @@
/*
Copyright 2017, 2019 the Velero contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package client
import (
"os"
apiextv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
apiextv1beta1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1"
k8scheme "k8s.io/client-go/kubernetes/scheme"
kbclient "sigs.k8s.io/controller-runtime/pkg/client"
"github.com/pkg/errors"
"github.com/spf13/pflag"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/client-go/dynamic"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/rest"
velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
clientset "github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned"
)
// Factory knows how to create a VeleroClient and Kubernetes client.
type Factory interface {
// BindFlags binds common flags (--kubeconfig, --namespace) to the passed-in FlagSet.
BindFlags(flags *pflag.FlagSet)
// Client returns a VeleroClient. It uses the following priority to specify the cluster
// configuration: --kubeconfig flag, KUBECONFIG environment variable, in-cluster configuration.
Client() (clientset.Interface, error)
// KubeClient returns a Kubernetes client. It uses the following priority to specify the cluster
// configuration: --kubeconfig flag, KUBECONFIG environment variable, in-cluster configuration.
KubeClient() (kubernetes.Interface, error)
// DynamicClient returns a Kubernetes dynamic client. It uses the following priority to specify the cluster
// configuration: --kubeconfig flag, KUBECONFIG environment variable, in-cluster configuration.
DynamicClient() (dynamic.Interface, error)
// KubebuilderClient returns a client for the controller runtime framework. It adds Kubernetes and Velero
// types to its scheme. It uses the following priority to specify the cluster
// configuration: --kubeconfig flag, KUBECONFIG environment variable, in-cluster configuration.
KubebuilderClient() (kbclient.Client, error)
// SetBasename changes the basename for an already-constructed client.
// This is useful for generating clients that require a different user-agent string below the root `velero`
// command, such as the server subcommand.
SetBasename(string)
// SetClientQPS sets the Queries Per Second for a client.
SetClientQPS(float32)
// SetClientBurst sets the Burst for a client.
SetClientBurst(int)
// ClientConfig returns a rest.Config struct used for client-go clients.
ClientConfig() (*rest.Config, error)
// Namespace returns the namespace which the Factory will create clients for.
Namespace() string
}
type factory struct {
flags *pflag.FlagSet
kubeconfig string
kubecontext string
baseName string
namespace string
clientQPS float32
clientBurst int
}
// NewFactory returns a Factory.
func NewFactory(baseName, kubecontext string, config VeleroConfig) Factory {
f := &factory{
flags: pflag.NewFlagSet("", pflag.ContinueOnError),
baseName: baseName,
kubecontext: kubecontext,
}
f.namespace = os.Getenv("VELERO_NAMESPACE")
if config.Namespace() != "" {
f.namespace = config.Namespace()
}
// We didn't get the namespace via env var or config file, so use the default.
// Command line flags will override when BindFlags is called.
if f.namespace == "" {
f.namespace = velerov1api.DefaultNamespace
}
f.flags.StringVar(&f.kubeconfig, "kubeconfig", "", "Path to the kubeconfig file to use to talk to the Kubernetes apiserver. If unset, try the environment variable KUBECONFIG, as well as in-cluster configuration")
f.flags.StringVarP(&f.namespace, "namespace", "n", f.namespace, "The namespace in which Velero should operate")
//f.flags.StringVar(&f.kubecontext, "kubecontext", "", "The context to use to talk to the Kubernetes apiserver. If unset defaults to whatever your current-context is (kubectl config current-context)")
return f
}
func (f *factory) BindFlags(flags *pflag.FlagSet) {
flags.AddFlagSet(f.flags)
}
func (f *factory) ClientConfig() (*rest.Config, error) {
return Config(f.kubeconfig, f.kubecontext, f.baseName, f.clientQPS, f.clientBurst)
}
func (f *factory) Client() (clientset.Interface, error) {
clientConfig, err := f.ClientConfig()
if err != nil {
return nil, err
}
veleroClient, err := clientset.NewForConfig(clientConfig)
if err != nil {
return nil, errors.WithStack(err)
}
return veleroClient, nil
}
func (f *factory) KubeClient() (kubernetes.Interface, error) {
clientConfig, err := f.ClientConfig()
if err != nil {
return nil, err
}
kubeClient, err := kubernetes.NewForConfig(clientConfig)
if err != nil {
return nil, errors.WithStack(err)
}
return kubeClient, nil
}
func (f *factory) DynamicClient() (dynamic.Interface, error) {
clientConfig, err := f.ClientConfig()
if err != nil {
return nil, err
}
dynamicClient, err := dynamic.NewForConfig(clientConfig)
if err != nil {
return nil, errors.WithStack(err)
}
return dynamicClient, nil
}
func (f *factory) KubebuilderClient() (kbclient.Client, error) {
clientConfig, err := f.ClientConfig()
if err != nil {
return nil, err
}
scheme := runtime.NewScheme()
velerov1api.AddToScheme(scheme)
k8scheme.AddToScheme(scheme)
apiextv1beta1.AddToScheme(scheme)
apiextv1.AddToScheme(scheme)
kubebuilderClient, err := kbclient.New(clientConfig, kbclient.Options{
Scheme: scheme,
})
if err != nil {
return nil, err
}
return kubebuilderClient, nil
}
func (f *factory) SetBasename(name string) {
f.baseName = name
}
func (f *factory) SetClientQPS(qps float32) {
f.clientQPS = qps
}
func (f *factory) SetClientBurst(burst int) {
f.clientBurst = burst
}
func (f *factory) Namespace() string {
return f.namespace
}

View File

@@ -0,0 +1,61 @@
/*
Copyright 2019 the Velero contributors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package client
import (
"os"
"testing"
"github.com/spf13/pflag"
"github.com/stretchr/testify/assert"
)
// TestFactory tests the client.Factory interface.
func TestFactory(t *testing.T) {
// Velero client configuration is currently omitted due to requiring a
// test filesystem in pkg/test. This causes an import cycle as pkg/test
// uses pkg/client's interfaces to implement fakes
// Env variable should set the namespace if no config or argument are used
os.Setenv("VELERO_NAMESPACE", "env-velero")
f := NewFactory("velero", "", make(map[string]interface{}))
assert.Equal(t, "env-velero", f.Namespace())
os.Unsetenv("VELERO_NAMESPACE")
// Argument should change the namespace
f = NewFactory("velero", "", make(map[string]interface{}))
s := "flag-velero"
flags := new(pflag.FlagSet)
f.BindFlags(flags)
flags.Parse([]string{"--namespace", s})
assert.Equal(t, s, f.Namespace())
// An argument overrides the env variable if both are set.
os.Setenv("VELERO_NAMESPACE", "env-velero")
f = NewFactory("velero", "", make(map[string]interface{}))
flags = new(pflag.FlagSet)
f.BindFlags(flags)
flags.Parse([]string{"--namespace", s})
assert.Equal(t, s, f.Namespace())
os.Unsetenv("VELERO_NAMESPACE")
}

View File

@@ -20,7 +20,7 @@ import (
"k8s.io/client-go/kubernetes"
kbclient "sigs.k8s.io/controller-runtime/pkg/client"
"github.com/vmware-tanzu/velero/pkg/client"
"github.com/vmware-tanzu/velero/test/e2e/pkg/client"
)
// TestClient contains different API clients that are in use throughout