Commit Graph

44 Commits

Author SHA1 Message Date
Carlisia Thompson
e69fac153b Centralize + rename controller names and list (#2936)
* Centralize + rename controller names and list

Signed-off-by: Carlisia <carlisia@vmware.com>

* Rename file

Signed-off-by: Carlisia <carlisia@vmware.com>

* Reset restic-repo name

Signed-off-by: Carlisia <carlisia@vmware.com>

* Reset gc controller name

Signed-off-by: Carlisia <carlisia@vmware.com>
2020-10-06 13:58:56 -04:00
Carlisia Thompson
be1cd03023 Refactor BSL related code (#2870)
* Refactor BSL related code

Signed-off-by: Carlisia <carlisia@vmware.com>

* Increase # of max concurrent reconciles

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up for better, more precise interfaces

Signed-off-by: Carlisia <carlisia@vmware.com>

* Minor clean up - code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add import and fix CI

Signed-off-by: Carlisia <carlisia@vmware.com>
2020-09-02 13:12:37 -04:00
Andrew Lavery
a368370bef k8s 1.18 import (#2651)
* k8s 1.18 import wip

backup, cmd, controller, generated, restic, restore, serverstatusrequest, test and util

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* go mod tidy

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* add changelog file

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* go fmt

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* update code-generator and controller-gen in CI

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* checkout proper code-generator version, regen

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* fix remaining calls

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* regenerate CRDs with ./hack/update-generated-crd-code.sh

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* use existing context in restic and server

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* fix test cases by resetting resource version

also use main library go context, not golang.org/x/net/context, in pkg/restore/restore.go

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* clarify changelog message

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* use github.com/kubernetes-csi/external-snapshotter/v2@v2.2.0-rc1

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* run 'go mod tidy' to remove old external-snapshotter version

Signed-off-by: Andrew Lavery <laverya@umich.edu>
2020-07-16 12:21:37 -04:00
Carlisia Campos
dbd0aa4915 Add a BSL controller to handle validation + update BSL status phase (#2674)
* Add BSL controller

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add changelog

Signed-off-by: Carlisia <carlisia@vmware.com>

* Make update

Signed-off-by: Carlisia <carlisia@vmware.com>

* Update docs

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add kubebuilder dependency

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add export

Signed-off-by: Carlisia <carlisia@vmware.com>

* add kubebuilder binaries into velero builder image

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* Reset velero dockerfile

Signed-off-by: Carlisia <carlisia@vmware.com>

* Consolidate all logic

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add copyright header

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up + add "last validated" column

Signed-off-by: Carlisia <carlisia@vmware.com>

* Better tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add more tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* Better logging

Signed-off-by: Carlisia <carlisia@vmware.com>

* Format

Signed-off-by: Carlisia <carlisia@vmware.com>

* Code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove redundant logic

Signed-off-by: Carlisia <carlisia@vmware.com>

Co-authored-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-14 17:47:00 -04:00
Carlisia Campos
4048c020a8 Convert manifests + BSL api client to kubebuilder (#2561)
* kubebuilder init - minimalist version

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add back main.go, apparently kb needs it

Signed-off-by: Carlisia <carlisia@vmware.com>

* Tweak makefile to accomodate kubebuilder expectations

Signed-off-by: Carlisia <carlisia@vmware.com>

* Port BSL to kubebuilder api client

Signed-off-by: Carlisia <carlisia@vmware.com>

* s/cache/client bc client fetches from cache
And other naming improvements

Signed-off-by: Carlisia <carlisia@vmware.com>

* So, .GetAPIReader is how we bypass the cache
In this case, the cache hasn't started yet

Signed-off-by: Carlisia <carlisia@vmware.com>

* Oh that's what this code was for... adding back

We still need to embed the CRDs as binary data in the Velero binary to
access the generated CRDs at runtime.

Signed-off-by: Carlisia <carlisia@vmware.com>

* Tie in CRD/code generation w/ existing scripts

Signed-off-by: Carlisia <carlisia@vmware.com>

* Mostly result of running update-fmt, updated file formatting

Signed-off-by: Carlisia <carlisia@vmware.com>

* Just a copyright fix

Signed-off-by: Carlisia <carlisia@vmware.com>

* All the test fixes

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add changelog + some cleanup

Signed-off-by: Carlisia <carlisia@vmware.com>

* Update backup manifest

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove unneeded auto-generated files

Signed-off-by: Carlisia <carlisia@vmware.com>

* Keep everything in the same (existing) package

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix/clean scripts, generated code, and calls

Deleting the entire `generated` directory and running `make update`
works. Modifying an api and running `make verify` works as expected.

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up schema and client calls + code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Move all code gen to inside builder container

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix imports/aliases

Signed-off-by: Carlisia <carlisia@vmware.com>

* More code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add waitforcachesync

Signed-off-by: Carlisia <carlisia@vmware.com>

* Have manager register ALL controllers

This will allow for proper cache management.

Signed-off-by: Carlisia <carlisia@vmware.com>

* Status subresource is now enabled; cleanup

Signed-off-by: Carlisia <carlisia@vmware.com>

* More code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up

Signed-off-by: Carlisia <carlisia@vmware.com>

* Manager registers ALL controllers for restic too

Signed-off-by: Carlisia <carlisia@vmware.com>

* More code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add deprecation warning/todo

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add documentation

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add helpful comments

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* More idiomatic Runnable

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up imports

Signed-off-by: Carlisia <carlisia@vmware.com>
2020-06-24 12:55:18 -04:00
Ashish Amarnath
577e87d1b8 Sync CSI API objects during backup sync (#2496)
* add changelog

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* Sync CSI API volumesnapshotcontents during backup sync

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-05-06 08:00:05 -06:00
Steve Kriss
c7f283c7fa wait for informer caches to sync before running controllers
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-03-03 15:55:17 -07:00
Antony S Bett
0450567bba Allow backup storage locations to specify backup sync period or toggle off sync (#1936)
* Allow backup storage locations to specify backup sync period or toggle off sync

Signed-off-by: Antony Bett <antony.bett@gmail.com>

* Add --backup-sync-period flag to backup location create command

Signed-off-by: Antony Bett <antony.bett@gmail.com>
2019-10-24 08:32:45 -06:00
Steve Kriss
48792ece1f Updates for org move to vmware-tanzu (#1920)
* update import paths to github.com/vmware-tanzu/...

Signed-off-by: Steve Kriss <krisss@vmware.com>

* update other GH org refs to vmware-tanzu

Signed-off-by: Steve Kriss <krisss@vmware.com>

* site and docs: update GH org to vmware-tanzu

Signed-off-by: Steve Kriss <krisss@vmware.com>

* update travis badge links on docs readmes

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-09-30 17:26:56 -04:00
Steve Kriss
d96186473c sync controller: replace revision file with full diff each interval (#1892)
* backup sync controller: replace revision file with full diff each interval

Signed-off-by: Steve Kriss <krisss@vmware.com>

* remove getting/setting of metadata/revision file

Signed-off-by: Steve Kriss <krisss@vmware.com>

* changelog

Signed-off-by: Steve Kriss <krisss@vmware.com>

* tweak logging

Signed-off-by: Steve Kriss <krisss@vmware.com>

* don't keep podVolumeBackup log field around after syncing PVBs

Signed-off-by: Steve Kriss <krisss@vmware.com>

* update generated CRDs

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-09-27 16:23:11 -04:00
Steve Kriss
5ef0a84128 create pod volume backups in backup's namespace during sync (#1760)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-08-13 10:20:25 -07:00
Steve Kriss
6522ba7c42 fix bugs preventing pod volume backups from syncing properly
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-08-06 09:23:49 -06:00
KubeKween
3b9af8c654 Store PodVolumeBackups in obj storage & use as source of truth (#1577)
* Store PodVolumeBackups in object storage

Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-07-24 15:51:20 -04:00
Anshul Chandra
4e12b08953 Validate restore name label length
Velero should handle cases when the label length exceeds 63 characters.

- if the length of the backup/restore name is <= 63 characters, use it as the value of the label
- if it's > 63 characters, take the SHA256 hash of the name. the value of
  the label will be the first 57 characters of the backup/restore name
  plus the first six characters of the SHA256 hash.

Fixes heptio#1021

Signed-off-by: Anshul Chandra <anshulc@vmware.com>
2019-05-01 10:25:11 -07:00
Steve Kriss
7f36f78aee remove code that removes legacy GC finalizer from backups
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-04-15 10:17:03 -06:00
Steve Kriss
a111eed2af update license headers to Velero contributors (#1302)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-20 12:32:48 -07:00
Carlisia
a71e43b2b7 Split velero plugin client into its own package
Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-03-19 16:05:37 -07:00
Steve Kriss
c0a55e136b logging tweaks for clarity
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-05 10:08:33 -07:00
Nolan Brubaker
43714caaec Rename Ark to Velero!!!
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-02-04 17:35:22 -05:00
Steve Kriss
40e33020d8 Merge pull request #1022 from shubheksha/fix/945-restore-controller-remove-checking-backup-storage-directly
Remove code that checks backup storage directly from restore controller
2018-11-06 15:53:13 -08:00
Steve Kriss
6c9e1f187f lower some noisy logs to debug level
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-11-02 15:55:47 -06:00
Shubheksha Jalan
defb8aa856 remove code that checks directly for a backup from restore controller
Signed-off-by: Shubheksha Jalan <jshubheksha@gmail.com>
2018-11-02 23:35:04 +05:30
Steve Kriss
516422c2c4 sync controller: fill in missing .spec.storageLocation
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-23 16:26:07 -06:00
Andy Goldstein
bca585162f Run 'make update' to update formatting
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-10-23 13:08:42 -04:00
Steve Kriss
da9ed38c63 store volume snapshot info as JSON in backup storage
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-17 13:39:49 -06:00
Steve Kriss
eb709b8f70 only sync a backup location if it's changed since last sync
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-03 08:05:06 -06:00
Steve Kriss
f0edf7335f add a BackupStore to pkg/persistence that supports prefixes
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-06 10:53:58 -06:00
Steve Kriss
af64069d65 create pkg/persistence and move relevant code from pkg/cloudprovider into it
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-06 09:05:40 -06:00
Steve Kriss
729d733986 controllers: take a newPluginManager func in constructors
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
Steve Kriss
133dc185ca backup sync: process the default location first
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
Steve Kriss
0e94fa37f9 update sync controller for backup locations
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
Carlisia
8ce513acbd Only delete unused backup if they are complete
Fixes #705

Signed-off-by: Carlisia <carlisia@grokkingtech.io>
2018-07-31 09:12:12 -07:00
Andy Goldstein
130512187a Refactor plugin management
Refactor plugin management:
- support multiple plugins per executable
- support restarting a plugin process in the event it terminates
- simplify plugin lifecycle management by using separate managers for
  each scope (server vs backup vs restore)

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-07-31 08:34:57 -07:00
Carlisia
d2e629f5a8 Delete backups from etcd if they're not in storage
This fixes #309.

Signed-off-by: Carlisia <carlisia@grokkingtech.io>

Co-authored-by: Steve Kriss <steve@heptio.com>
2018-07-25 14:53:50 -07:00
Carlisia Pinto
5b89f7b6db Skip backup sync if it already exists in k8s
Signed-off-by: Carlisia Pinto <carlisia@grokkingtech.io>
2018-07-09 21:55:24 -07:00
Steve Kriss
6754955bcd when syncing backups, set their namespace to current cluster's Ark ns
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-07 09:12:51 -07:00
Nolan Brubaker
300a010959 Remove finalizers from backups before syncing
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-04-18 17:32:39 -04:00
Steve Kriss
1503796419 make language and casing consistent across copyrights
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-01-02 10:51:49 -08:00
Andy Goldstein
1210cb36e1 Merge pull request #253 from skriss/update-license
Update all license headers
2017-12-19 19:26:54 -05:00
Steve Kriss
3641c2c043 update all license headers
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-15 13:38:12 -08:00
Steve Kriss
56c9d68137 switch to logrus.FieldLogger and arktest.NewLogger() everywhere
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-13 09:38:00 -08:00
Andy Goldstein
15fe87aea3 Fix clientset imports
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-30 13:31:03 -04:00
Steve Kriss
914165095a switch logging to logrus and errors to pkg/errors
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-05 08:49:33 -07:00
Andy Goldstein
2fe501f527 Initial commit
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-08-02 13:27:17 -04:00