Commit Graph

130 Commits

Author SHA1 Message Date
Bridget McErlean
b9a8c0b254 Pass configured BSL credential to plugin via config (#3442)
* Load credentials and pass to ObjectStorage plugins

Update NewObjectBackupStore to take a CredentialsGetter which can be
used to get the credentials for a BackupStorageLocation if it has been
configured with a Credential. If the BSL has a credential, use that
SecretKeySelector to fetch the secret, write the contents to a temp file
and then pass that file through to the plugin via the config map using
the key `credentialsFile`. This relies on the plugin being able to use
this new config field.

This does not yet handle VolumeSnapshotLocations or ResticRepositories.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Address code reviews

Add godocs and comments.
Improve formatting and test names.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Address code reviews

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-03-04 13:43:15 -08:00
Pranav Gaikwad
c46fe71b12 Restore progress reporting (#3125)
* restore progress reporting

Signed-off-by: Pranav Gaikwad <pgaikwad@redhat.com>

* add restore statistics to describe restore

Signed-off-by: Pranav Gaikwad <pgaikwad@redhat.com>

* address feedback, include namespaces in the count

Signed-off-by: Pranav Gaikwad <pgaikwad@redhat.com>
2021-03-04 16:21:44 -05:00
Suraj Banakar
ff1a31db4a Support cli uninstall (#3399)
* Add uninstall cmd
- init fn to uninstall velero
- abstract dynamic client creation to a separate fn
- creates a separate client per unstructured resource
- add delete client for CRDs
- export appendUnstructured
- add uninstall command to main cmd
- export `podTemplateOption`
- uninstall resources in the reverse order of installation
- fallback to `velero` if no ns is provided during uninstall
- skip deletion if the resource doesn't exist
- handle resource not found error
- match log formatting with cli install logs
- add Delete fn to fake client
- fix import order
- add changelog
- add comment doc for CreateClient fn

Signed-off-by: Suraj Banakar <suraj@infracloud.io>

* Re-use uninstall code from test suite
- move helper functions out of test suite
- this is to prevent cyclic imports
- move uninstall helpers to uninstall cmd
- call them from test suite
- revert export of variables/fns from install code
- because not required anymore

Signed-off-by: Suraj Banakar <suraj@infracloud.io>

* Revert `PodTemplateOption` -> `podTemplateOption`

Signed-off-by: Suraj Banakar <suraj@infracloud.io>

* Use uninstall helper under VeleroUninstall
- as a wrapper
- fix import related errors in test suite

Signed-off-by: Suraj Banakar <suraj@infracloud.io>
2021-03-04 14:16:40 -05:00
Bridget McErlean
9dbd238c89 Use controller-runtime client to get restic secrets (#3320)
* Use kubebuilder client for fetching restic secrets

Instead of using a SecretInformer for fetching secrets for restic, use
the cached client provided by the controller-runtime manager.

In order to use this client, the scheme for Secrets must be added to the
scheme used by the manager so this is added when creating the manager in
both the velero and restic servers.

This change also refactors some of the tests to add a shared utility for
creating a fake controller-runtime client which is now used among all
tests which use that client. This has been added to ensure that all
tests use the same client with the same scheme.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Add builder for SecretKeySelector

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-02-18 10:30:52 -08:00
Ashish Amarnath
ecab583680 🐛 Do not run ItemAction plugins for unresolvable types for all types (#3059)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-11-11 09:50:57 -05:00
Nolan Brubaker
718a94ad05 Invoke DeleteItemActions on backup deletion (#2815)
* Add serving and listing support

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-08-20 17:24:29 -07:00
Benoit Gagnon
5d2c9e2ba1 Override logrus.ErrorKey when json logging is enabled (#2830)
* override logrus.ErrorKey when json logging is enabled

Signed-off-by: Benoit Gagnon <benoit.gagnon@ubisoft.com>

* document the logrus.ErrorKey override

Signed-off-by: Benoit Gagnon <benoit.gagnon@ubisoft.com>

* add changelog entry

Signed-off-by: Benoit Gagnon <benoit.gagnon@ubisoft.com>
2020-08-18 13:53:45 -04:00
Piper Dougherty
19e65689ef Add the ability to set the allowPrivilegeEscalation property on the Restic restore helper via plugin ConfigMap (#2792)
* Add the ability to set the `allowPrivilegeEscalation` security context attribute on the Restic restore helper init container.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>

* Add changelog.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>

* Fix old tests and add tests for new allowPrivilegeEscalation config option.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>

* Correct spelling in changelog.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>

* Switch to boolptr type.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>

* Reorder imports for sanity.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>
2020-08-06 13:08:36 -04:00
Ashish Amarnath
2636730ef2 fix copyright year
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-22 12:11:59 -07:00
Martin Odstrčilík
86efd1577e add support for setting SecurityContext (user, group) for restic restore (#2621)
* add support for setting SecurityContext (user, group) for restic restore

Signed-off-by: Martin Odstrcilik <martin.odstrcilik@gmail.com>
2020-07-22 12:10:25 -07:00
Andrew Lavery
a368370bef k8s 1.18 import (#2651)
* k8s 1.18 import wip

backup, cmd, controller, generated, restic, restore, serverstatusrequest, test and util

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* go mod tidy

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* add changelog file

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* go fmt

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* update code-generator and controller-gen in CI

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* checkout proper code-generator version, regen

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* fix remaining calls

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* regenerate CRDs with ./hack/update-generated-crd-code.sh

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* use existing context in restic and server

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* fix test cases by resetting resource version

also use main library go context, not golang.org/x/net/context, in pkg/restore/restore.go

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* clarify changelog message

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* use github.com/kubernetes-csi/external-snapshotter/v2@v2.2.0-rc1

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* run 'go mod tidy' to remove old external-snapshotter version

Signed-off-by: Andrew Lavery <laverya@umich.edu>
2020-07-16 12:21:37 -04:00
Steve Kriss
5cc6c12eb4 don't return an error from log hook if value is not an error type
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-04-30 15:30:22 -06:00
Steve Kriss
15b2a1c9c6 bug fix: don't remove unresolvable includes from includes-excludes lists (#2462)
* bug fix: don't remove unresolvable includes from includes-excludes lists

Signed-off-by: Steve Kriss <krisss@vmware.com>

* changelog

Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-04-27 17:17:39 -04:00
Nolan Brubaker
040f6806f1 Fix int/float conversion issues in CRD version remapping plugin (#2322)
* Add builders for CRD schemas

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add test case for #2319

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add failing test case

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Remove unnecessary print and temporary variable

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add some options for fixing the test case

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Switch to a JSON middle step to "fix" conversions

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add comment and changelog

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-03-10 09:24:22 -06:00
Nolan Brubaker
6745979a7b Wait for CRDs to be ready before restoring CRs (#1937)
* Wait for CRDs to be available and ready

When restoring CRDs, we should wait for the definition to be ready and
available before moving on to restoring specific CRs.

While the CRDs are often ready by the time we get to restoring a CR,
there is a race condition where the CRD isn't ready.

This change waits on each CRD at restore time.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-01-30 09:19:13 -08:00
Steve Kriss
3b80e00d62 fix race condition in waiting for restic restores to complete
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-01-15 16:09:54 -07:00
Steve Kriss
48792ece1f Updates for org move to vmware-tanzu (#1920)
* update import paths to github.com/vmware-tanzu/...

Signed-off-by: Steve Kriss <krisss@vmware.com>

* update other GH org refs to vmware-tanzu

Signed-off-by: Steve Kriss <krisss@vmware.com>

* site and docs: update GH org to vmware-tanzu

Signed-off-by: Steve Kriss <krisss@vmware.com>

* update travis badge links on docs readmes

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-09-30 17:26:56 -04:00
Adnan Abdulhussein
7b7b96de74 velero install: support velero/restic pod cpu/mem requests with unbounded limits (#1745)
* support velero/restic pod cpu/mem requests with unbounded limits

explicitly treat "0" value as unbounded

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* update flag documentation

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* changelog

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>

* update restic configmap docs

Signed-off-by: Adnan Abdulhussein <aadnan@vmware.com>
2019-08-09 12:54:04 -06:00
Steve Kriss
ffa3251efc move contents of pkg/util/test into pkg/test
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-08-07 08:26:00 -06:00
Nolan Brubaker
a4e70456a1 Add resource limits to restic init container (#1677)
* Add resource limits to restic init container

Fixes #1201

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Start restic restore item action tests

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Get initial tests for restore action working

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add new test case

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Move resource parsing into a shared function

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Fetch request/limits from plugin's ConfigMap

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Use builders

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Use moved ParseResourceRequirements function

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Move init container building inline

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Drop CPU limit down a bit and clarify error message

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Fix godoc

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add resource requirements to doc

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-08-05 13:18:11 -06:00
Steve Kriss
4543258970 Update tests to use object builders in pkg/builder (#1707)
* add pkg/builder with BackupBuilder and ObjectMeta functional options

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-07-31 07:46:48 -07:00
carthewd
22eca22ac8 Add JSON as an option for log output (#1654)
* Add JSON as an option for log output

Signed-off-by: Donovan Carthew <donovan.carthew@gmail.com>
2019-07-30 16:29:34 -07:00
Nolan Brubaker
63964fc6f9 Append "/mount" to directory for CSI volumes (#1615)
CSI volumes are mounted one level deeper than "native" kubernetes
volumes, and this needs to be appended for proper restic support.

Fixes #1313.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-07-10 15:16:21 -07:00
Steve Kriss
19052994ed replace TestBackup with pkg/backup.Builder (#1593)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-06-21 17:08:08 -04:00
Steve Kriss
bc7ee686d7 Merge pull request #1428 from guilhem/glob
Add ability to use wildcard in includes/excludes
2019-06-05 09:53:37 -06:00
Guilhem Lettron
81a26e4aad Add ability to use wildcard in includes/excludes
Use https://github.com/gobwas/glob as glob engine
Fix #373

Signed-off-by: Guilhem Lettron <guilhem@barpilot.io>
2019-06-05 16:28:25 +02:00
Steve Kriss
411d44a673 allow individual backup storage locations to be read-only (#1517)
* allow individual backup storage locations to be read-only

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-05-29 14:21:25 -04:00
Steve Kriss
8392e6d83f add PartiallyFailed phase for backups, log+continue on errors (#1386)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-04-26 09:14:26 -07:00
Nolan Brubaker
58bb7ed3aa Merge pull request #1390 from carlisia/c-check-for-backup-III
Fail backup if it already exists in object storage
2019-04-24 17:31:48 -04:00
Steve Kriss
3155bb159f remove VolumeBackupInfo from API pkg (#1388)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-04-24 13:16:03 -07:00
Andy Goldstein
01d0b026e9 Fail backup if it already exists in object storage
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2019-04-22 15:22:58 -07:00
Nolan Brubaker
6f474016a6 Add velero install command (#1287)
Add velero install command

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-04-15 14:10:11 -07:00
Steve Kriss
c59544cb79 remove backup.status.volumeBackups and all related code
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-04-15 10:17:40 -06:00
Fábio Franco Uechi
40c7fbce09 Gracefully handle failed API groups from the discovery API (#1293)
* log details and continue executing if error of type ErrGroupDiscoveryFailed is returned by discovery API

Signed-off-by: fabito <fuechi@ciandt.com>
2019-04-01 14:27:37 -04:00
Steve Kriss
bb9c3f6a1a rename BlockStore to VolumeSnapshotter
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-27 14:55:28 -06:00
Nolan Brubaker
3f2c28f6bb Merge pull request #1301 from skriss/plugins-error-location
log error locations from plugin logger and don't overwrite in client
2019-03-27 11:21:52 -04:00
Steve Kriss
7b0d8217de send plugin error stack traces over gRPC and log error locations
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-27 08:34:03 -06:00
Steve Kriss
ff2db31b32 log error locations from plugin logger and don't overwrite in client
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-20 16:13:37 -06:00
Steve Kriss
a111eed2af update license headers to Velero contributors (#1302)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-20 12:32:48 -07:00
Steve Kriss
250f109c41 delete pkg/util/collections/map_utils.go & tests
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:21:49 -07:00
Nolan Brubaker
890202f2e4 Wait for PV/namespace to delete before restore
If a PV already exists, wait for it, it's associated PVC, and associated
namespace to be deleted before attempting to restore it.

If a namespace already exists, wait for it to be deleted before
attempting to restore it.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-02-06 15:43:50 -05:00
Nolan Brubaker
43714caaec Rename Ark to Velero!!!
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-02-04 17:35:22 -05:00
Andy Goldstein
bca585162f Run 'make update' to update formatting
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-10-23 13:08:42 -04:00
Steve Kriss
90d9be59d3 support restoring/deleting legacy backups with .status.volumeBackups
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-23 09:55:40 -06:00
Steve Kriss
df07b7dc9f update backup code to work with volume snapshot locations
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-17 13:36:50 -06:00
Carlisia
1aa712d236 Default and validate VolumeSnapshotLocations
Signed-off-by: Carlisia <carlisia@grokkingtech.io>
2018-10-17 13:33:22 -06:00
Steve Kriss
7d8813a96c move ObjectStore mock into pkg/cloudprovider/mocks
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-06 10:53:58 -06:00
Steve Kriss
f0edf7335f add a BackupStore to pkg/persistence that supports prefixes
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-06 10:53:58 -06:00
Carlisia
2750aa71b9 Use backup storage location during restore
Closes #740

Signed-off-by: Carlisia <carlisia@grokkingtech.io>
2018-08-28 13:19:21 -07:00
Nolan Brubaker
c6f488f75f Use backup location in the backup controller
Fixes #739

Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-08-28 13:19:20 -07:00