Commit Graph

147 Commits

Author SHA1 Message Date
Andrew Lavery
a368370bef k8s 1.18 import (#2651)
* k8s 1.18 import wip

backup, cmd, controller, generated, restic, restore, serverstatusrequest, test and util

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* go mod tidy

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* add changelog file

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* go fmt

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* update code-generator and controller-gen in CI

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* checkout proper code-generator version, regen

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* fix remaining calls

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* regenerate CRDs with ./hack/update-generated-crd-code.sh

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* use existing context in restic and server

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* fix test cases by resetting resource version

also use main library go context, not golang.org/x/net/context, in pkg/restore/restore.go

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* clarify changelog message

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* use github.com/kubernetes-csi/external-snapshotter/v2@v2.2.0-rc1

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* run 'go mod tidy' to remove old external-snapshotter version

Signed-off-by: Andrew Lavery <laverya@umich.edu>
2020-07-16 12:21:37 -04:00
Ashish Amarnath
3d3b9e312a Revert "🐛 fix file perissions on the manifest JSON in backup archive (#2685)" (#2700)
dae5230aae

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-10 14:07:05 -07:00
Ashish Amarnath
dae5230aae 🐛 fix file perissions on the manifest JSON in backup archive (#2685)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-08 06:37:59 -07:00
Ashish Amarnath
b4465e92b8 🐛 Use CRD version prior to remap_crd_version backup item action (#2683)
* 🐛 preserve crd version before remapping

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-07 10:59:41 -07:00
Ashish Amarnath
e5e7c025dd fix copyright boilerplate
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-18 14:14:15 -07:00
Ashish Amarnath
63f7690f44 more tests
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-16 14:46:36 -07:00
Ashish Amarnath
b0fd3d35c1 rename field
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-15 15:26:44 -07:00
Ashish Amarnath
b71173228a add velero server flag to allow default restic use on all pod volumes
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-15 14:46:06 -07:00
Ashish Amarnath
8a2a852b87 use backup's defaultRestic flag to identify pod volumes using restic
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-15 14:46:06 -07:00
Nolan Brubaker
d9d9dc60da Log when a hook timeout duration can't be parsed (#2610)
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-06-05 12:45:50 -07:00
Steve Kriss
1c80ba903e don't error during backup when additional items returned by plugin don't exist (#2595)
* log a warning instead of erroring if additional item can't be found

Signed-off-by: Steve Kriss <krisss@vmware.com>

* always show backup warning/error count in get/describe

Signed-off-by: Steve Kriss <krisss@vmware.com>

* changelog

Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-06-04 16:29:55 -07:00
Steve Kriss
92693ad422 update backup struct with progress updates
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-05-21 15:40:59 -06:00
Nolan Brubaker
abae81ddc8 Merge pull request #2440 from skriss/backup-progress
report backup progress
2020-05-07 14:51:20 -04:00
Nolan Brubaker
8671a639c9 Backup entire v1beta1 CRD instead of just changing version string (#2478)
* Switch to backing up v1beta1 CRDs from API server

Instead of simply switching out the APIVersion string on a v1
CustomResourceDefinition object, re-download the object from the API
server entirely to get the correct fields.

This should fix validation errors upon restore.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Fix existing tests

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add full example CRDs to automated tests

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Move beta CRD lookup into helper function

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add case for preserveUnknownFields CRDs

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add PreserveUnknownFields case and refactor execute

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add older prometheus CRD test cases

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add changelog

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-05-05 13:21:28 -07:00
Steve Kriss
4b0f654a1e add progress info to backup status
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-05-01 15:30:50 -06:00
Steve Kriss
30ca0e4322 split out collecting items from backing up items
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-05-01 15:30:42 -06:00
Steve Kriss
19c52434b4 simplify group/resource/item backupper structure
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-05-01 15:29:01 -06:00
Rafael Brito
0d97f9400e Backup all groups and versions with backward compatibility (#2373)
* Backup all API Groups versions while keeping backward compatibility

Signed-off-by: Rafael Brito <rbrito@vmware.com>

* Backup all API Groups versions while keeping backward compatibility

Signed-off-by: Rafael Brito <rbrito@vmware.com>

* Adding feature flag to enable backup of multiple API group versions

Signed-off-by: Rafael Brito <rbrito@vmware.com>
2020-05-01 15:54:57 -04:00
Steve Kriss
15b2a1c9c6 bug fix: don't remove unresolvable includes from includes-excludes lists (#2462)
* bug fix: don't remove unresolvable includes from includes-excludes lists

Signed-off-by: Steve Kriss <krisss@vmware.com>

* changelog

Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-04-27 17:17:39 -04:00
Mayank
962f5d6859 Skipping validation for volumesnapshotlocation for backup if snapshotvolume set to false (#2450)
* Disabling validation for volumesnapshotlocation if the backup has snapshotvolume set to false

Signed-off-by: mayank <mayank.patel@mayadata.io>

* adding a changelog

Signed-off-by: mayank <mayank.patel@mayadata.io>

* addressing review comment

Signed-off-by: mayank <mayank.patel@mayadata.io>
2020-04-24 09:46:20 -07:00
Steve Kriss
ca58d7ab90 change dynamic client's List return type to UnstructuredList
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-04-15 12:41:47 -06:00
Steve Kriss
1826b6b67b bug fix: populate namespace in logs for backup errors (#2438)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-04-15 10:31:40 -07:00
Nolan Brubaker
040f6806f1 Fix int/float conversion issues in CRD version remapping plugin (#2322)
* Add builders for CRD schemas

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add test case for #2319

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add failing test case

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Remove unnecessary print and temporary variable

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add some options for fixing the test case

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Switch to a JSON middle step to "fix" conversions

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add comment and changelog

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-03-10 09:24:22 -06:00
Steve Kriss
9e19c1d3bd fix goimport noise in prep for bump to latest
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-03-02 10:55:08 -07:00
Nolan Brubaker
766ab5d3ee Account for possible missing schemas on v1 CRDs (#2264)
* Account for possible missing schemas on v1 CRDs

If a v1beta1 CRD without a Schema was submitted to a Kubernets v1.16
cluster, then Kubernetes will server it back as a v1 CRD without a
schema.

However, when Velero tries to restore this document, the request will be
rejected as a v1 CRD must have a schema.

This commit has some defensive coding on the restore side, as well as
potential fixes on the backup side for getting around this.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Back up nonstructural CRDs as v1beta1

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add tests for remapping plugin

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add builders for v1 CRDs

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Address review feedback

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Remove extraneous log message

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add changelog

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-02-24 12:07:50 -07:00
Carlos Tadeu Panato Junior
c9bc6646a5 use GA topology labels for PVs (#2219)
* use GA toplogy labels for PVs

Signed-off-by: Carlos Panato <ctadeu@gmail.com>
2020-02-03 10:47:18 -08:00
Steve Kriss
37011ee5ac bug fix: back up server-preferred version of CRD (#2230)
* bug fix: back up server-preferred version of CRD

Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-01-31 08:45:46 -07:00
JenTing Hsiao
93e4737258 Fix incorrect output log format (#2165)
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
2020-01-06 09:05:38 -07:00
Steve Kriss
f42406723c bug fix: deep-copy backup's labels for snapshot tags (#2075)
* bug fix: deep-copy backup's labels for snapshot tags

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-12-05 12:54:19 -08:00
Steve Kriss
92ea44e8e6 tweak test case names for clarity (#1976)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-10-17 12:39:42 -07:00
Scott Seago
970f17c1ef If includeClusterResources is nil/auto, pull in necessary CRDs (#1831)
Related issue: https://github.com/heptio/velero/issues/1830

This accomplishes everything
that's needed, although there might be room for improvement in avoiding
a GET call for matching CRDs for each resource backed up. An alternative
could be a single call to get all CRDs prior to iterating over resources
and passing this into the backupResource function.

Signed-off-by: Scott Seago <sseago@redhat.com>
2019-10-17 12:31:04 -06:00
Steve Kriss
557d170541 restic: only backup ReadWriteMany PVC's once per velero backup (#1896)
* restic: only backup ReadWriteMany PVC's once per velero backup

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-10-08 17:16:35 -04:00
Steve Kriss
48792ece1f Updates for org move to vmware-tanzu (#1920)
* update import paths to github.com/vmware-tanzu/...

Signed-off-by: Steve Kriss <krisss@vmware.com>

* update other GH org refs to vmware-tanzu

Signed-off-by: Steve Kriss <krisss@vmware.com>

* site and docs: update GH org to vmware-tanzu

Signed-off-by: Steve Kriss <krisss@vmware.com>

* update travis badge links on docs readmes

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-09-30 17:26:56 -04:00
Christopher Angelo Phillips
ca6c0a5a3d Increment logging for item backupper (#1904)
Signed-off-by: Christopher Phillips <cphillips918@gmail.com>
2019-09-27 13:17:54 -07:00
Adnan Abdulhussein
26e06dae53 fix excluding additional items with the exclude-from-backup label (#1843)
* fix excluding additional items with the exclude-from-backup label

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-09-10 11:45:55 -07:00
Adnan Abdulhussein
faef20d45a ensure backup resource list per gvk is sorted
Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-08-08 13:53:40 -07:00
Steve Kriss
c69fa4d0e1 bug fix: upload pod volume backups from *all* pods to obj storage
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-08-08 09:27:46 -06:00
Steve Kriss
ffa3251efc move contents of pkg/util/test into pkg/test
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-08-07 08:26:00 -06:00
Adnan Abdulhussein
4e1b1f9457 use Backup CR labels as tags for snapshots (#1729)
* use Backup CR labels as tags for snapshots

This allows users to define custom tags to be added to snapshots, by
specifying custom labels on the Backup CR with the `velero backup create
--labels` flag.

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-08-06 13:01:36 -07:00
Adnan Abdulhussein
07525bd593 store backup resource list metadata in object storage (#1709)
* move backedUpItems to pkg/backup.Request struct

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* construct resource itemKey field from gvk

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* store backup resource list metadata in object storage

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* remove debug log

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* fix formatting

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* add missing license blocks and split BackupInfo struct lines

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* add test for checking BackedUpItems matches tarball contents

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* add comment to explain test

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-08-05 11:15:55 -06:00
Steve Kriss
4543258970 Update tests to use object builders in pkg/builder (#1707)
* add pkg/builder with BackupBuilder and ObjectMeta functional options

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-07-31 07:46:48 -07:00
Steve Kriss
bf247836e6 Migrate restic backup tests (#1703)
* migrate pkg/backup restic tests to new structure

Signed-off-by: Steve Kriss <krisss@vmware.com>

* rename backup_new_test.go to backup_test.go

Signed-off-by: Steve Kriss <krisss@vmware.com>

* use pod volume backup builder

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-07-29 15:29:44 -07:00
Adnan Abdulhussein
52d97e7bd7 proposal: expose list of backed up resources in backup details (#1682)
* proposal: expose list of backed up resources in backup details

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-07-29 14:00:56 -04:00
KubeKween
3b9af8c654 Store PodVolumeBackups in obj storage & use as source of truth (#1577)
* Store PodVolumeBackups in object storage

Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-07-24 15:51:20 -04:00
Steve Kriss
1a339f06ac migrate restore resource filtering tests
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-06-24 15:53:15 -06:00
Adnan Abdulhussein
b0bdaeea73 ensure correct backup item actions run with namespace selector (#1601)
* ensure correct backup item actions run with namespace selector

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* changelog

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* don't run backup item actions for namespace resources

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* simplify cluster-scope resources checks

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-06-24 15:50:25 -06:00
Nolan Brubaker
08fe7be851 Fix test conflicts from #1595 and #1598 (#1602)
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-06-24 15:34:56 -06:00
Nolan Brubaker
67512a3808 Merge pull request #1595 from skriss/restore-test-refactor-1
initial refactoring for pkg/restore tests
2019-06-24 17:08:51 -04:00
Steve Kriss
5634a4f463 migrate backup hooks tests (#1591)
* migrate hooks tests

Signed-off-by: Steve Kriss <krisss@vmware.com>

* add more test cases

Signed-off-by: Steve Kriss <krisss@vmware.com>

* refactor to strongly typed expectation, add pre+post hook test case

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-06-24 12:23:17 -07:00
Steve Kriss
19052994ed replace TestBackup with pkg/backup.Builder (#1593)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-06-21 17:08:08 -04:00