Commit Graph

108 Commits

Author SHA1 Message Date
Ron Green
861cc78bcd refactor(external-snapshotter): bump to v4
Signed-off-by: Ron Green <11993626+georgettica@users.noreply.github.com>
2021-01-26 13:06:27 +02:00
Carlisia
bd10b7660c Improvements to BSL logic
Signed-off-by: Carlisia <carlisia@vmware.com>
2020-12-09 13:25:01 -08:00
JenTing Hsiao
9dd158d13d feat: support configure BSL CR to indicate which one is the default (#3092)
* Add default field to BSL CRD

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Add a new flag `--default` under `velero backup-location create`

add a new flag `--default` under `velero backup-location create`
to specify this new location to be the new default BSL.

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Add a new default field under `velero backup-location get`

add a new default field under `velero backup-location get` to indicate
which BSL is the default one.

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Add a new sub-command and flag under `velero backup-location`

Add a new sub-command called `velero backup-location set` sub-command
and a new flag `velero backup-cation set --default` to configure which
BSL is the default one.

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Add new flag to get the default backup-location

Add a new flag `--default` under `velero backup-location get`
to displays the current default BSL.

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Configures default BSL in BSL controller

When upgrade the BSL CRDs, none of the BSL has been labeled as default.
Sets the BSL default field to true if the BSL name matches to the default BSL setting.

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Configures the default BSL in BSL controller for velero upgrade

When upgrade the BSL CRDs, none of the BSL be marked as the default.
Sets the BSL `.spec.default: true` if the BSL name matches against the
`velero server --default-backup-storage-location`.

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Add unit test to test default BSL behavior

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Update check which one is the default BSL in backup/backup_sync/restore controller

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Add changelog

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Update docs locations.md and upgrade-to-1.6.md

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
2020-12-08 16:38:29 -05:00
Bridget McErlean
a877354750 Don't fail backup deletion if downloading tarball fails (#2993)
* Don't fail backup if downloading tarball fails

Previously, we would always attempt to download the tarball for a backup
for processing DeleteItemAction plugins, even if there weren't any.
This caused an issue for some users in the case where the backup tarball
had been deleted from object storage as the backup deletion would fail.

Now, we only attempt to download the tarball in the case where there are
DeleteItemAction plugins. If downloading that tarball fails, we log
the error, skip the processing of the DeleteItemAction plugins and
proceed with the rest of the deletion.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Skip file removal in closeAndRemoveFile if nil

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2020-11-30 10:58:34 -08:00
Ashish Amarnath
1be97a2b04 🏃‍♂ Improve log message clarity (#3047)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-11-02 13:39:32 -08:00
Carlisia Thompson
e69fac153b Centralize + rename controller names and list (#2936)
* Centralize + rename controller names and list

Signed-off-by: Carlisia <carlisia@vmware.com>

* Rename file

Signed-off-by: Carlisia <carlisia@vmware.com>

* Reset restic-repo name

Signed-off-by: Carlisia <carlisia@vmware.com>

* Reset gc controller name

Signed-off-by: Carlisia <carlisia@vmware.com>
2020-10-06 13:58:56 -04:00
kathpeony
243ac62e3f Add backupValidationFailureTotal to metrics (#2714)
* Add backValidationFailureTotal to metrics

Signed-off-by: Kathrin Mao <kathrin.mao@sap.com>
2020-07-16 10:13:17 -07:00
Andrew Lavery
a368370bef k8s 1.18 import (#2651)
* k8s 1.18 import wip

backup, cmd, controller, generated, restic, restore, serverstatusrequest, test and util

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* go mod tidy

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* add changelog file

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* go fmt

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* update code-generator and controller-gen in CI

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* checkout proper code-generator version, regen

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* fix remaining calls

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* regenerate CRDs with ./hack/update-generated-crd-code.sh

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* use existing context in restic and server

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* fix test cases by resetting resource version

also use main library go context, not golang.org/x/net/context, in pkg/restore/restore.go

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* clarify changelog message

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* use github.com/kubernetes-csi/external-snapshotter/v2@v2.2.0-rc1

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* run 'go mod tidy' to remove old external-snapshotter version

Signed-off-by: Andrew Lavery <laverya@umich.edu>
2020-07-16 12:21:37 -04:00
Carlisia Campos
4048c020a8 Convert manifests + BSL api client to kubebuilder (#2561)
* kubebuilder init - minimalist version

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add back main.go, apparently kb needs it

Signed-off-by: Carlisia <carlisia@vmware.com>

* Tweak makefile to accomodate kubebuilder expectations

Signed-off-by: Carlisia <carlisia@vmware.com>

* Port BSL to kubebuilder api client

Signed-off-by: Carlisia <carlisia@vmware.com>

* s/cache/client bc client fetches from cache
And other naming improvements

Signed-off-by: Carlisia <carlisia@vmware.com>

* So, .GetAPIReader is how we bypass the cache
In this case, the cache hasn't started yet

Signed-off-by: Carlisia <carlisia@vmware.com>

* Oh that's what this code was for... adding back

We still need to embed the CRDs as binary data in the Velero binary to
access the generated CRDs at runtime.

Signed-off-by: Carlisia <carlisia@vmware.com>

* Tie in CRD/code generation w/ existing scripts

Signed-off-by: Carlisia <carlisia@vmware.com>

* Mostly result of running update-fmt, updated file formatting

Signed-off-by: Carlisia <carlisia@vmware.com>

* Just a copyright fix

Signed-off-by: Carlisia <carlisia@vmware.com>

* All the test fixes

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add changelog + some cleanup

Signed-off-by: Carlisia <carlisia@vmware.com>

* Update backup manifest

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove unneeded auto-generated files

Signed-off-by: Carlisia <carlisia@vmware.com>

* Keep everything in the same (existing) package

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix/clean scripts, generated code, and calls

Deleting the entire `generated` directory and running `make update`
works. Modifying an api and running `make verify` works as expected.

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up schema and client calls + code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Move all code gen to inside builder container

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix imports/aliases

Signed-off-by: Carlisia <carlisia@vmware.com>

* More code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add waitforcachesync

Signed-off-by: Carlisia <carlisia@vmware.com>

* Have manager register ALL controllers

This will allow for proper cache management.

Signed-off-by: Carlisia <carlisia@vmware.com>

* Status subresource is now enabled; cleanup

Signed-off-by: Carlisia <carlisia@vmware.com>

* More code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up

Signed-off-by: Carlisia <carlisia@vmware.com>

* Manager registers ALL controllers for restic too

Signed-off-by: Carlisia <carlisia@vmware.com>

* More code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add deprecation warning/todo

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add documentation

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add helpful comments

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* More idiomatic Runnable

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up imports

Signed-off-by: Carlisia <carlisia@vmware.com>
2020-06-24 12:55:18 -04:00
Ashish Amarnath
e5e7c025dd fix copyright boilerplate
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-18 14:14:15 -07:00
Ashish Amarnath
7abd2c6db9 doc updates
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-18 13:57:59 -07:00
Ashish Amarnath
b0fd3d35c1 rename field
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-15 15:26:44 -07:00
Ashish Amarnath
8a2a852b87 use backup's defaultRestic flag to identify pod volumes using restic
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-15 14:46:06 -07:00
Steve Kriss
5b52fd3efd re-instantiate backup store just before persisting artifacts (#2550)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-05-27 16:03:52 -07:00
Nolan Brubaker
e91dfe3d91 Remove comment that has been turned into an issue (#2500)
Logged comment as issue #2499

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-05-08 09:27:01 -07:00
Nolan Brubaker
de8962ea18 Bug fix: Calculate label using backup name for CSI objects (#2510)
* Use a helper function when querying w/ backup label

Setting or querying for a backup label name should always pass the value
through the GetValidName function. This change passes query uses of the
backup label value through the GetValidName function by introducing 2
new helpers, one for making a Selector, one for making a ListOptions.

It also removes functions returning the same data, but under
unecessarily specific names.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Document using the label.GetValidName function

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Update copyright year

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Clarify labels.GetValidName and annotations

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Move functions to pkg/label

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Fix function comments

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-05-07 11:56:13 -07:00
Ashish Amarnath
c49afd44bd Use label selector to list VSCs to persist (#2502)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-05-06 11:04:34 -07:00
Rafael Brito
0d97f9400e Backup all groups and versions with backward compatibility (#2373)
* Backup all API Groups versions while keeping backward compatibility

Signed-off-by: Rafael Brito <rbrito@vmware.com>

* Backup all API Groups versions while keeping backward compatibility

Signed-off-by: Rafael Brito <rbrito@vmware.com>

* Adding feature flag to enable backup of multiple API group versions

Signed-off-by: Rafael Brito <rbrito@vmware.com>
2020-05-01 15:54:57 -04:00
Mayank
962f5d6859 Skipping validation for volumesnapshotlocation for backup if snapshotvolume set to false (#2450)
* Disabling validation for volumesnapshotlocation if the backup has snapshotvolume set to false

Signed-off-by: mayank <mayank.patel@mayadata.io>

* adding a changelog

Signed-off-by: mayank <mayank.patel@mayadata.io>

* addressing review comment

Signed-off-by: mayank <mayank.patel@mayadata.io>
2020-04-24 09:46:20 -07:00
Nolan Brubaker
aa3abefdbf Address review feedback
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
16cf2780d8 Fetch contents based on volumesnapshots found
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
bc04c568cd Return list of errors for encoding/zipping
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
eefd12b3e4 Rename CSI variables to be more descriptive
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
b567859655 Add data description for encoding JSON in case of error
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
d31951c81c Add CSI feature flag constant
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
15d4c11305 Wrap CSI lister access in a nil check
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
6c3fc8c780 Addresss simple review feedback
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
9372eaa31b Reduce duplication in object serialization
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
9c1a2e884b Update comments
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
5202885349 Remove unused CSI client variable
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
c71318bf19 Address more review feedback
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:17 -04:00
Nolan Brubaker
aff529e5d5 Upload CSI volumesnapshots associated with backup
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:17 -04:00
Nolan Brubaker
6a7beaf5ce Plumb CSI listers through to backup controller
Account for having CSI enabled or not, as well as having the snapshot
CRDs installed in the kubernetes cluster.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:17 -04:00
Nolan Brubaker
232e1d8927 Add clients for CSI snapshots
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:17 -04:00
Nolan Brubaker
b4be7eccb9 Document locations for CSI support
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:17 -04:00
Nolan Brubaker
67322d555e Write kubernetes version info to annoations
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-14 17:11:04 -04:00
Rafael Brito
f6cd53c6cb Adding k8s version labels on backup CRD #2342 (#2346)
Signed-off-by: Rafael Brito <rbrito@vmware.com>
2020-03-19 09:18:05 -07:00
Steve Kriss
c7f283c7fa wait for informer caches to sync before running controllers
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-03-03 15:55:17 -07:00
Steve Kriss
71201fe929 extract helper func and add unit tests
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-01-21 13:21:28 -07:00
Steve Kriss
9fa302aa8b recompute backup_last_successful_timestamp metric during resync
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-01-14 14:11:21 -07:00
Adnan Abdulhussein
e3d64d9dd9 use pointer types for metav1.Time fields (#1951)
* use pointer types for metav1.Time fields

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* simpler metav1.Time ptrs

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* remove test debug println

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-10-14 10:20:28 -06:00
Steve Kriss
48792ece1f Updates for org move to vmware-tanzu (#1920)
* update import paths to github.com/vmware-tanzu/...

Signed-off-by: Steve Kriss <krisss@vmware.com>

* update other GH org refs to vmware-tanzu

Signed-off-by: Steve Kriss <krisss@vmware.com>

* site and docs: update GH org to vmware-tanzu

Signed-off-by: Steve Kriss <krisss@vmware.com>

* update travis badge links on docs readmes

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-09-30 17:26:56 -04:00
Adnan Abdulhussein
07525bd593 store backup resource list metadata in object storage (#1709)
* move backedUpItems to pkg/backup.Request struct

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* construct resource itemKey field from gvk

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* store backup resource list metadata in object storage

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* remove debug log

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* fix formatting

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* add missing license blocks and split BackupInfo struct lines

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* add test for checking BackedUpItems matches tarball contents

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* add comment to explain test

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-08-05 11:15:55 -06:00
carthewd
22eca22ac8 Add JSON as an option for log output (#1654)
* Add JSON as an option for log output

Signed-off-by: Donovan Carthew <donovan.carthew@gmail.com>
2019-07-30 16:29:34 -07:00
KubeKween
3b9af8c654 Store PodVolumeBackups in obj storage & use as source of truth (#1577)
* Store PodVolumeBackups in object storage

Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-07-24 15:51:20 -04:00
Steve Kriss
411d44a673 allow individual backup storage locations to be read-only (#1517)
* allow individual backup storage locations to be read-only

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-05-29 14:21:25 -04:00
Anshul Chandra
4e12b08953 Validate restore name label length
Velero should handle cases when the label length exceeds 63 characters.

- if the length of the backup/restore name is <= 63 characters, use it as the value of the label
- if it's > 63 characters, take the SHA256 hash of the name. the value of
  the label will be the first 57 characters of the backup/restore name
  plus the first six characters of the SHA256 hash.

Fixes heptio#1021

Signed-off-by: Anshul Chandra <anshulc@vmware.com>
2019-05-01 10:25:11 -07:00
Steve Kriss
8392e6d83f add PartiallyFailed phase for backups, log+continue on errors (#1386)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-04-26 09:14:26 -07:00
Carlisia
6f496a8921 Bug fix
Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-04-24 14:18:24 -07:00
Carlisia
7d28f82540 Address code review
Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-04-24 10:54:43 -07:00