Commit Graph

2621 Commits

Author SHA1 Message Date
Steve Kriss
c7f283c7fa wait for informer caches to sync before running controllers
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-03-03 15:55:17 -07:00
Steve Kriss
ac7a84cc58 update generated CRDs
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-03-02 10:55:08 -07:00
Steve Kriss
ea418ceda7 update generated CRDs
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-03-02 10:55:08 -07:00
Steve Kriss
9e19c1d3bd fix goimport noise in prep for bump to latest
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-03-02 10:55:08 -07:00
Ashish Amarnath
31dca0ea1b add GroupResources for volumesnapshot types (#2288)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-02-24 11:47:35 -08:00
Nolan Brubaker
766ab5d3ee Account for possible missing schemas on v1 CRDs (#2264)
* Account for possible missing schemas on v1 CRDs

If a v1beta1 CRD without a Schema was submitted to a Kubernets v1.16
cluster, then Kubernetes will server it back as a v1 CRD without a
schema.

However, when Velero tries to restore this document, the request will be
rejected as a v1 CRD must have a schema.

This commit has some defensive coding on the restore side, as well as
potential fixes on the backup side for getting around this.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Back up nonstructural CRDs as v1beta1

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add tests for remapping plugin

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add builders for v1 CRDs

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Address review feedback

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Remove extraneous log message

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add changelog

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-02-24 12:07:50 -07:00
Ashish Amarnath
e681759178 export restic pod annotation (#2283)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-02-19 12:37:40 -07:00
JenTing Hsiao
b4446bd358 Fix restic backup volume snapshot to the second location failed (#2244)
* Fix restic backup volume snapshot to the second location failed

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
2020-02-19 11:01:21 -08:00
Mayank
1a1372550d Use PV name returned from volumesnapshotter while creating a PV (#2216)
* Using PV name returned from volumesnapshotter while creating a PV

Signed-off-by: mayank <mayank.patel@mayadata.io>
2020-02-10 13:17:15 -07:00
Steve Kriss
44d3f91e97 backup describer: reword resource list not found msg
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-02-06 11:40:15 -07:00
Brad Koehn
b25fea3bea properly handle empty restic stdout during progress updates (#2231)
* properly handle empty restic stdout during progress updates

Signed-off-by: Brad Koehn <brad@koehn.com>
2020-02-03 12:49:23 -07:00
Steve Kriss
08c549a092 Restore result refactoring (#2234)
* move Result helper funcs to be methods

Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-02-03 10:56:57 -08:00
Carlos Tadeu Panato Junior
c9bc6646a5 use GA topology labels for PVs (#2219)
* use GA toplogy labels for PVs

Signed-off-by: Carlos Panato <ctadeu@gmail.com>
2020-02-03 10:47:18 -08:00
Ashish Amarnath
21264a11ec Merge pull request #2218 from cpanato/GH-1168
remove schedule validation
2020-01-31 13:00:24 -08:00
Steve Kriss
37011ee5ac bug fix: back up server-preferred version of CRD (#2230)
* bug fix: back up server-preferred version of CRD

Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-01-31 08:45:46 -07:00
Carlos Panato
42b6126458 update variable name
Signed-off-by: Carlos Panato <ctadeu@gmail.com>
2020-01-31 09:22:46 +01:00
Nolan Brubaker
6745979a7b Wait for CRDs to be ready before restoring CRs (#1937)
* Wait for CRDs to be available and ready

When restoring CRDs, we should wait for the definition to be ready and
available before moving on to restoring specific CRs.

While the CRDs are often ready by the time we get to restoring a CR,
there is a race condition where the CRD isn't ready.

This change waits on each CRD at restore time.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-01-30 09:19:13 -08:00
Carlos Panato
f7adc4dfd1 check if schedule have a backup
Signed-off-by: Carlos Panato <ctadeu@gmail.com>
2020-01-29 18:24:32 +01:00
Carlos Panato
445b5b781e remove schedule validation
Signed-off-by: Carlos Panato <ctadeu@gmail.com>
2020-01-29 18:24:32 +01:00
JenTing Hsiao
b19097f825 Pruning unknown fields (#2187)
* Pruning unknown fields

In CRD apiversion v1beta1, default preserveUnknownFields=true.

In CRD apiversion v1, the preserveUnknownFields can only be false.
Otherwise, the k8s validation bumps out error message for the
invalid preserveUnknownFields value.

Deploy Velero on k8s 1.16+ with CRD apiversion v1beta1, the
k8s cluster converts apiversion from v1beta1 to v1 automatically.

Fully backup and restore the cluster, restore bumps out error message
due to the preserveUnknownFields=true is not allowed on k8s 1.16+.

Since the CRD structural schema had been defined, enable the preserveUnknownFields
to false to solves the restore bumps out error message on k8s 1.16+.

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Add changelog

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
2020-01-28 14:04:28 -05:00
Nolan Brubaker
5b1280c2cd Fix restore issues for CRD v1 on Kubernetes v1.16 clusters (#2197)
Fix restore issues for CRD v1 on Kubernetes v1.16 clusters

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-01-27 13:59:08 -08:00
Ashish Amarnath
ec22f2c88d Merge pull request #2196 from skriss/fix-1777
recompute backup_last_successful_timestamp metric during resync
2020-01-21 17:01:51 -08:00
Ashish Amarnath
421dcd4e8d Merge pull request #2201 from skriss/fix-2121
fix race condition in waiting for restic restores to complete
2020-01-21 15:16:08 -08:00
Steve Kriss
71201fe929 extract helper func and add unit tests
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-01-21 13:21:28 -07:00
Steve Kriss
ae316193ea move log statement
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-01-21 12:36:46 -07:00
Steve Kriss
3b80e00d62 fix race condition in waiting for restic restores to complete
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-01-15 16:09:54 -07:00
Nolan Brubaker
ee0cbcf11e Merge pull request #2199 from jnaulty/jnaulty/improve-config-file-perms
Restrict file permissions for config file/dir
2020-01-15 15:13:22 -05:00
John Naulty
254a5eebb5 Restrict file permissions for config file/dir
Velero client config file should have restricted file permissions to be
read/write-able for the user that creates it--similiar to files like
`.ssh/id_rsa`

Refer to OTG-CONFIG-009: Test File Permission
> Impoper file permission configuration may result in privilledge
escalation, information explousure, DLL injection, or unauthorized file
access.
Therefore, files permission must be properly configured with minium
access permission by default.

[source](https://www.owasp.org/index.php/Test_File_Permission_(OTG-CONFIG-009))

Ticket: #1758
Signed-off-by: John Naulty <johnnaulty@bitgo.com>
2020-01-15 09:41:25 -08:00
Scott Seago
aa44cf1c32 Check for nil LastMaintenanceTime in dueForMaintenance (#2200)
* Check for nil LastMaintenanceTime in dueForMaintenance

ResticRepository.dueForMaintenance causes a panic in the velero pod
("invalid memory address or nil pointer dereference") if
repository.Status.LastMaintenanceTime is nil. This fix returns 'true'
if it's nil, so the repository is due for maintenance if LastMaintenanceTime
is nil *or* the time elapsed since the last maintenance is greater than
repository.Spec.MaintenanceFrequency.Duration

Signed-off-by: Scott Seago <sseago@redhat.com>

* changelog for PR#2200

Signed-off-by: Scott Seago <sseago@redhat.com>
2020-01-15 12:06:16 -05:00
Steve Kriss
9fa302aa8b recompute backup_last_successful_timestamp metric during resync
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-01-14 14:11:21 -07:00
Steve Kriss
b2acd3b683 bump go-hclog and prometheus dependency versions in prep for go modules (#2188)
* update revision of go-hclog to match go.mod requirement

Signed-off-by: Steve Kriss <krisss@vmware.com>

* update prometheus dep to prepare for go module migration

Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-01-14 11:29:37 -05:00
Steve Kriss
04d8b47d3e update to k8s 1.17 deps and fix breaks (#2180)
* update to k8s 1.17 deps

Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-01-10 08:55:58 -08:00
JenTing Hsiao
97b33402a4 Assign default CPU/Memory resource value if not present (#2176)
Install restic with CPU/Memory limits is optional.
If velero cannot parse resource requirements, use default value instead.

After that, the administrator won't get confused that something recovered failed.

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
2020-01-09 08:48:35 -07:00
Shuwei Hao
c832e52905 add ALIBABA_CLOUD_CREDENTIALS_FILE env in deployment and daemonset installation (#2166)
Signed-off-by: haoshuwei <haoshuwei24@gmail.com>
2020-01-06 14:56:23 -08:00
Ashish Amarnath
408c0dfb5a Add documentation for velero install cli (#2155)
* Add documentation for velero install cli

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* mirror doc changes into v1.2.0

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-01-06 13:13:02 -07:00
JenTing Hsiao
93e4737258 Fix incorrect output log format (#2165)
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
2020-01-06 09:05:38 -07:00
Steve Kriss
be140985c5 bug fix: only prioritize restoring replicasets.apps (#2157)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-12-19 09:44:08 -08:00
John Naulty Jr
cd860771c2 Add error return to pvRenamer func (#2138)
Migrate logic from NewUUID function into the pvRenamer function.

PR #2133 switched to a new NewUUID function that returns an error, but
the invocation of that function needs to happen within the pvRenamer
closure. Because the new function returns an error, the pvRenamer should
return the error, the signature needs to be changed and the return
checked.

Signed-off-by: John Naulty <johnnaulty@bitgo.com>
2019-12-17 10:23:58 -08:00
John Naulty
ff889283b8 Switch from satori/go.uuid to gofrs/uuid
satori/go.uuid has a known issue with random uuid generation.
gofrs/uuid is still maintained and has fixed the random uuid generation
issue present in satori/go.uuid

Signed-off-by: John Naulty <johnnaulty@bitgo.com>
2019-12-14 10:27:00 -08:00
Steve Kriss
7c60829f38 bug fix: don't restore cluster-scoped resources by default when restoring specific namespaces (#2118)
* add failing test case for bug

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-12-12 12:45:41 -08:00
Steve Kriss
96297ea437 add replicasets.apps to prioritized restore resources (#2120)
* add replicasets.apps to prioritized restore resources

Signed-off-by: Steve Kriss <krisss@vmware.com>

* changelog

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-12-11 15:32:33 -05:00
Dinesh Yadav
83ef4eb4d0 Support AWS_PROFILE for restic backups/restore operations (#2096)
* Support AWS_PROFILE for restic backups/restore operations

It enables Velero to switch credentials if multiple s3-compatible
backupLocations are present.

Signed-off-by: dinesh <dinesh1042@gmail.com>

* better comments and fixing typos

Signed-off-by: dinesh <dinesh1042@gmail.com>

* add changelog entry

Signed-off-by: dinesh <dinesh1042@gmail.com>
2019-12-09 09:46:02 -05:00
Steve Kriss
f42406723c bug fix: deep-copy backup's labels for snapshot tags (#2075)
* bug fix: deep-copy backup's labels for snapshot tags

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-12-05 12:54:19 -08:00
Roman Klimenko
87be775139 Add support for private registry with custom port in restic-helper image (#1999)
* Add support for private registry with custom port in restic-helper image definition

Signed-off-by: Roman Klimenko <cognoz@ya.ru>
2019-12-04 08:01:39 -08:00
Steve Kriss
e5a85f140a use field selector on name when watching download requests (#2065)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-11-14 16:50:24 -08:00
Steve Kriss
5a31a78df5 add better error message when cluster config can't be found
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-11-08 15:01:12 -07:00
Antony S Bett
121dc02d2f Ensure object store plugin processes are cleaned up after restore and after BSL validation during server start up (#2041)
* Ensure child object store plugin processes are cleaned up during restore

Signed-off-by: Antony Bett <antony.bett@gmail.com>
2019-11-05 08:58:08 -08:00
Steve Kriss
4fb1bc2ef3 restic: don't try to restore PVBs with no snapshotID (#2031)
* restic: don't try to restore PVBs with no snapshotID

Signed-off-by: Steve Kriss <krisss@vmware.com>

* changelog

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-11-04 16:18:08 -07:00
Adnan Abdulhussein
636a5b9db6 add velero install --crds-only to easily update CRDs (#2038)
* add velero install --crds-only to easily update CRDs

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* make update

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-11-04 14:36:04 -08:00
Steve Kriss
83752d28d7 Restic restore fixes (#2013)
* restic: use restore cmd's env when getting snapshot size

Signed-off-by: Steve Kriss <krisss@vmware.com>

* restic: remove code that considers 0-byte backups an error

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-10-29 07:42:12 -07:00