Commit Graph

274 Commits

Author SHA1 Message Date
Mateusz Gozdek
dbc83af77b Fix various typos found by codespell (#3057)
By running the following command:

codespell -S .git,*.png,*.jpg,*.woff,*.ttf,*.gif,*.ico -L \
iam,aks,ist,bridget,ue

Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
2020-11-10 11:48:35 -05:00
Ashish Amarnath
1be97a2b04 🏃‍♂ Improve log message clarity (#3047)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-11-02 13:39:32 -08:00
Piper Dougherty
60ff351269 Adding fix for restic init container index on restores. (#3011)
* Adding handling of restic-wait init container at any order with warning.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>

* Adding newline at end of files to match convention.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>

* Formatting.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>

* Update copyright year on modified files.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>
2020-10-21 15:15:03 -07:00
Antony S Bett
35d25c81ec Fix BSL controller to avoid invoking init() on all BSLs regardless of ValidationFrequency (#2992)
Signed-off-by: Bett, Antony <antony.bett@dell.com>
2020-10-13 12:10:32 -07:00
Carlisia Thompson
e69fac153b Centralize + rename controller names and list (#2936)
* Centralize + rename controller names and list

Signed-off-by: Carlisia <carlisia@vmware.com>

* Rename file

Signed-off-by: Carlisia <carlisia@vmware.com>

* Reset restic-repo name

Signed-off-by: Carlisia <carlisia@vmware.com>

* Reset gc controller name

Signed-off-by: Carlisia <carlisia@vmware.com>
2020-10-06 13:58:56 -04:00
Carlisia Thompson
be1cd03023 Refactor BSL related code (#2870)
* Refactor BSL related code

Signed-off-by: Carlisia <carlisia@vmware.com>

* Increase # of max concurrent reconciles

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up for better, more precise interfaces

Signed-off-by: Carlisia <carlisia@vmware.com>

* Minor clean up - code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add import and fix CI

Signed-off-by: Carlisia <carlisia@vmware.com>
2020-09-02 13:12:37 -04:00
Carlisia Campos
c952932f1b Migrate ServerStatusRequest controller and resource to kubebuilder (#2838)
* Convert ServerStatusRequest controller to controller-runtime

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add select stm

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fixed status patch bug

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add mgr start

Signed-off-by: Carlisia <carlisia@vmware.com>

* Trying to sync

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean async now

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up + move context out

Signed-off-by: Carlisia <carlisia@vmware.com>

* Bug: not closing the channel

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up some tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* Much better way to fetch an update using a backoff loop

Signed-off-by: Carlisia <carlisia@vmware.com>

* Even better way to retry: use apimachinery lib

Signed-off-by: Carlisia <carlisia@vmware.com>

* Refactor controller + add test

Signed-off-by: Carlisia <carlisia@vmware.com>

* partially fix unit tests

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* Fix and add tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add changelog

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add ability to disable the controller + cleanups

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix bug w/ disabling controllers + fix test + clean up

Signed-off-by: Carlisia <carlisia@vmware.com>

* Move role.yaml to the correct folder

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add sample serverstatusrequest.yaml

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add requeue + better formatting

Signed-off-by: Carlisia <carlisia@vmware.com>

* Increase # of max concurrent reconciles

Signed-off-by: Carlisia <carlisia@vmware.com>

Co-authored-by: Ashish Amarnath <ashisham@vmware.com>
2020-09-01 14:15:23 -07:00
Nolan Brubaker
718a94ad05 Invoke DeleteItemActions on backup deletion (#2815)
* Add serving and listing support

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-08-20 17:24:29 -07:00
Ashish Amarnath
91ccc4adb2 Add metrics for restic back up operation (#2719)
* add metrics for restic back up operation

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* changelog

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-22 15:07:52 -04:00
Thejas Babu
d0d143e119 Add StartTimestamp and CompletionTimestamp in Restore Status (#2748)
Signed-off-by: thejas <thejasb99@gmail.com>
2020-07-22 11:40:39 -07:00
Nolan Brubaker
f42c63af1b Address insensitive language (#2677)
* Change master to main in most uses

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-07-17 14:59:51 -07:00
kathpeony
243ac62e3f Add backupValidationFailureTotal to metrics (#2714)
* Add backValidationFailureTotal to metrics

Signed-off-by: Kathrin Mao <kathrin.mao@sap.com>
2020-07-16 10:13:17 -07:00
Andrew Lavery
a368370bef k8s 1.18 import (#2651)
* k8s 1.18 import wip

backup, cmd, controller, generated, restic, restore, serverstatusrequest, test and util

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* go mod tidy

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* add changelog file

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* go fmt

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* update code-generator and controller-gen in CI

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* checkout proper code-generator version, regen

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* fix remaining calls

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* regenerate CRDs with ./hack/update-generated-crd-code.sh

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* use existing context in restic and server

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* fix test cases by resetting resource version

also use main library go context, not golang.org/x/net/context, in pkg/restore/restore.go

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* clarify changelog message

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* use github.com/kubernetes-csi/external-snapshotter/v2@v2.2.0-rc1

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* run 'go mod tidy' to remove old external-snapshotter version

Signed-off-by: Andrew Lavery <laverya@umich.edu>
2020-07-16 12:21:37 -04:00
Carlisia Campos
13e1eeabfe Return early from a BSL controller (#2709)
Signed-off-by: Carlisia <carlisia@vmware.com>
2020-07-14 15:09:11 -07:00
Carlisia Campos
dbd0aa4915 Add a BSL controller to handle validation + update BSL status phase (#2674)
* Add BSL controller

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add changelog

Signed-off-by: Carlisia <carlisia@vmware.com>

* Make update

Signed-off-by: Carlisia <carlisia@vmware.com>

* Update docs

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add kubebuilder dependency

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add export

Signed-off-by: Carlisia <carlisia@vmware.com>

* add kubebuilder binaries into velero builder image

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* Reset velero dockerfile

Signed-off-by: Carlisia <carlisia@vmware.com>

* Consolidate all logic

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add copyright header

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up + add "last validated" column

Signed-off-by: Carlisia <carlisia@vmware.com>

* Better tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add more tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* Better logging

Signed-off-by: Carlisia <carlisia@vmware.com>

* Format

Signed-off-by: Carlisia <carlisia@vmware.com>

* Code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove redundant logic

Signed-off-by: Carlisia <carlisia@vmware.com>

Co-authored-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-14 17:47:00 -04:00
Carlisia Campos
4048c020a8 Convert manifests + BSL api client to kubebuilder (#2561)
* kubebuilder init - minimalist version

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add back main.go, apparently kb needs it

Signed-off-by: Carlisia <carlisia@vmware.com>

* Tweak makefile to accomodate kubebuilder expectations

Signed-off-by: Carlisia <carlisia@vmware.com>

* Port BSL to kubebuilder api client

Signed-off-by: Carlisia <carlisia@vmware.com>

* s/cache/client bc client fetches from cache
And other naming improvements

Signed-off-by: Carlisia <carlisia@vmware.com>

* So, .GetAPIReader is how we bypass the cache
In this case, the cache hasn't started yet

Signed-off-by: Carlisia <carlisia@vmware.com>

* Oh that's what this code was for... adding back

We still need to embed the CRDs as binary data in the Velero binary to
access the generated CRDs at runtime.

Signed-off-by: Carlisia <carlisia@vmware.com>

* Tie in CRD/code generation w/ existing scripts

Signed-off-by: Carlisia <carlisia@vmware.com>

* Mostly result of running update-fmt, updated file formatting

Signed-off-by: Carlisia <carlisia@vmware.com>

* Just a copyright fix

Signed-off-by: Carlisia <carlisia@vmware.com>

* All the test fixes

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add changelog + some cleanup

Signed-off-by: Carlisia <carlisia@vmware.com>

* Update backup manifest

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove unneeded auto-generated files

Signed-off-by: Carlisia <carlisia@vmware.com>

* Keep everything in the same (existing) package

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix/clean scripts, generated code, and calls

Deleting the entire `generated` directory and running `make update`
works. Modifying an api and running `make verify` works as expected.

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up schema and client calls + code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Move all code gen to inside builder container

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix imports/aliases

Signed-off-by: Carlisia <carlisia@vmware.com>

* More code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add waitforcachesync

Signed-off-by: Carlisia <carlisia@vmware.com>

* Have manager register ALL controllers

This will allow for proper cache management.

Signed-off-by: Carlisia <carlisia@vmware.com>

* Status subresource is now enabled; cleanup

Signed-off-by: Carlisia <carlisia@vmware.com>

* More code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up

Signed-off-by: Carlisia <carlisia@vmware.com>

* Manager registers ALL controllers for restic too

Signed-off-by: Carlisia <carlisia@vmware.com>

* More code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add deprecation warning/todo

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add documentation

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add helpful comments

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* More idiomatic Runnable

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up imports

Signed-off-by: Carlisia <carlisia@vmware.com>
2020-06-24 12:55:18 -04:00
Ashish Amarnath
e5e7c025dd fix copyright boilerplate
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-18 14:14:15 -07:00
Ashish Amarnath
7abd2c6db9 doc updates
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-18 13:57:59 -07:00
Ashish Amarnath
63f7690f44 more tests
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-16 14:46:36 -07:00
Ashish Amarnath
b0fd3d35c1 rename field
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-15 15:26:44 -07:00
Ashish Amarnath
8a2a852b87 use backup's defaultRestic flag to identify pod volumes using restic
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-15 14:46:06 -07:00
Christoph Blecker
ceae7a2fdc When creating backup from schedule, allow default name (#2569)
* When creating backup from schedule, allow default name

Signed-off-by: Christoph Blecker <admin@toph.ca>
2020-05-28 12:31:25 -07:00
Steve Kriss
5b52fd3efd re-instantiate backup store just before persisting artifacts (#2550)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-05-27 16:03:52 -07:00
Nolan Brubaker
e91dfe3d91 Remove comment that has been turned into an issue (#2500)
Logged comment as issue #2499

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-05-08 09:27:01 -07:00
Nolan Brubaker
de8962ea18 Bug fix: Calculate label using backup name for CSI objects (#2510)
* Use a helper function when querying w/ backup label

Setting or querying for a backup label name should always pass the value
through the GetValidName function. This change passes query uses of the
backup label value through the GetValidName function by introducing 2
new helpers, one for making a Selector, one for making a ListOptions.

It also removes functions returning the same data, but under
unecessarily specific names.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Document using the label.GetValidName function

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Update copyright year

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Clarify labels.GetValidName and annotations

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Move functions to pkg/label

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Fix function comments

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-05-07 11:56:13 -07:00
Ashish Amarnath
c49afd44bd Use label selector to list VSCs to persist (#2502)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-05-06 11:04:34 -07:00
Ashish Amarnath
577e87d1b8 Sync CSI API objects during backup sync (#2496)
* add changelog

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* Sync CSI API volumesnapshotcontents during backup sync

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-05-06 08:00:05 -06:00
Ashish Amarnath
6b5a084f32 Delete dangling volumesnapshotcontents from velero backups (#2480)
* Delete dangling volumesnapshotcontents from velero backups

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* add changelog

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* handle not found errors from VSC delete

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* clean up unit tests

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-05-04 15:41:27 -04:00
Ashish Amarnath
bf52caf658 Delete CSI volumesnapshots created as part of a backup in its deletion (#2411)
* delete CSI volumesnapshots in backup on backup delete

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* set vsc deletionpolicy to delete prior to vs deletion

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* add more unit tests

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* add change log

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* passes build need to pass unit tests

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* remove GetNamespacesInBackup

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* fix unit tests

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* check feature flag instead of nil check

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* cleanup

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-05-01 16:05:20 -04:00
Rafael Brito
0d97f9400e Backup all groups and versions with backward compatibility (#2373)
* Backup all API Groups versions while keeping backward compatibility

Signed-off-by: Rafael Brito <rbrito@vmware.com>

* Backup all API Groups versions while keeping backward compatibility

Signed-off-by: Rafael Brito <rbrito@vmware.com>

* Adding feature flag to enable backup of multiple API group versions

Signed-off-by: Rafael Brito <rbrito@vmware.com>
2020-05-01 15:54:57 -04:00
Mayank
962f5d6859 Skipping validation for volumesnapshotlocation for backup if snapshotvolume set to false (#2450)
* Disabling validation for volumesnapshotlocation if the backup has snapshotvolume set to false

Signed-off-by: mayank <mayank.patel@mayadata.io>

* adding a changelog

Signed-off-by: mayank <mayank.patel@mayadata.io>

* addressing review comment

Signed-off-by: mayank <mayank.patel@mayadata.io>
2020-04-24 09:46:20 -07:00
Nolan Brubaker
aa3abefdbf Address review feedback
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
16cf2780d8 Fetch contents based on volumesnapshots found
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
bc04c568cd Return list of errors for encoding/zipping
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
eefd12b3e4 Rename CSI variables to be more descriptive
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
b567859655 Add data description for encoding JSON in case of error
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
d31951c81c Add CSI feature flag constant
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
15d4c11305 Wrap CSI lister access in a nil check
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
6c3fc8c780 Addresss simple review feedback
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
9372eaa31b Reduce duplication in object serialization
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
9c1a2e884b Update comments
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
5202885349 Remove unused CSI client variable
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:18 -04:00
Nolan Brubaker
c71318bf19 Address more review feedback
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:17 -04:00
Nolan Brubaker
aff529e5d5 Upload CSI volumesnapshots associated with backup
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:17 -04:00
Nolan Brubaker
6a7beaf5ce Plumb CSI listers through to backup controller
Account for having CSI enabled or not, as well as having the snapshot
CRDs installed in the kubernetes cluster.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:17 -04:00
Nolan Brubaker
232e1d8927 Add clients for CSI snapshots
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:17 -04:00
Nolan Brubaker
b4be7eccb9 Document locations for CSI support
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-20 13:49:17 -04:00
Nolan Brubaker
67322d555e Write kubernetes version info to annoations
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-04-14 17:11:04 -04:00
Samuel Lucidi
4d49b5971c Support setting a custom CA bundle to use with a BackupStorageLocation (#2353)
* Support setting a custom CA certificate for a BSL

Signed-off-by: Sam Lucidi <slucidi@redhat.com>

* update CRDS

Signed-off-by: Sam Lucidi <slucidi@redhat.com>

* Add changelog for #2353

Signed-off-by: Sam Lucidi <slucidi@redhat.com>

* Clean up temp file from TestTempCACertFile

Signed-off-by: Sam Lucidi <slucidi@redhat.com>
2020-03-24 15:50:48 -06:00
Rafael Brito
f6cd53c6cb Adding k8s version labels on backup CRD #2342 (#2346)
Signed-off-by: Rafael Brito <rbrito@vmware.com>
2020-03-19 09:18:05 -07:00