Commit Graph

317 Commits

Author SHA1 Message Date
Nolan Brubaker
e9ece0f7b5 Implement DeleteItemAction plugin support (#2808)
* Add DeleteItemAction struct & protobuf definition

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-08-18 12:16:26 -07:00
Benoit Gagnon
5d2c9e2ba1 Override logrus.ErrorKey when json logging is enabled (#2830)
* override logrus.ErrorKey when json logging is enabled

Signed-off-by: Benoit Gagnon <benoit.gagnon@ubisoft.com>

* document the logrus.ErrorKey override

Signed-off-by: Benoit Gagnon <benoit.gagnon@ubisoft.com>

* add changelog entry

Signed-off-by: Benoit Gagnon <benoit.gagnon@ubisoft.com>
2020-08-18 13:53:45 -04:00
runzexia
3a4e441af8 add kindfor func to get apiresource from gvk (#2764)
* add kindfor func to get apiresource from gvk

Signed-off-by: RyderXia <ryder.xia@sap.com>

* impl interface & changelog

Signed-off-by: RyderXia <ryder.xia@sap.com>
2020-08-17 15:52:33 -07:00
Tony Batard
c663ce15ab Hugo migration (#2720)
* Move files to a Hugo structure

Signed-off-by: Tony Batard <tbatard@pivotal.io>
2020-08-13 09:09:15 -07:00
Phuong N. Hoang
14170b52a8 Enhance Backup to backup resources in specific order. (#2724)
Signed-off-by: Phuong Hoang <phuong.n.hoang@dell.com>

Co-authored-by: Phuong Hoang <phuong.n.hoang@dell.com>
2020-08-12 17:17:31 -07:00
Ashish Amarnath
d4bbd7b817 🐛 Patch generated yaml for restore CRD as a hacky workaround (#2814)
* 🐛 Patch generated yaml for restore CRD as a hacky workaround

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* changelog

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-08-12 18:43:29 -04:00
Ashish Amarnath
9eca0fcbff Pass default-volumes-to-restic flag from create schedule to backup (#2776)
* Pass default-volumes-to-restic flag from create schedule to backup

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-08-12 12:09:07 -07:00
Ashish Amarnath
e391e43192 🐛 Supply command to run restic-wait init container
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-08-11 11:05:30 -07:00
Ashish Amarnath
a68e5fc330 🏃‍♂️ Setup crd validation github action on k8s versions (#2805)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-08-11 10:45:12 -07:00
Ashish Amarnath
5d6da6517b Implement restore hooks injecting init containers into pod spec (#2787)
*  Implement restore hooks injecting init containers into pod spec

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-08-11 10:38:44 -07:00
Ashish Amarnath
9b9bb62968 🐛 Make init and exec restore hooks as omitempty in restore hookSpec (#2793)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-08-11 10:13:41 -07:00
Benoit Gagnon
0e8a7a23cb always use groupResource.String() when logging (fixes #2795) (#2796)
Signed-off-by: Benoit Gagnon <benoit.gagnon@ubisoft.com>
2020-08-06 10:10:59 -07:00
Piper Dougherty
19e65689ef Add the ability to set the allowPrivilegeEscalation property on the Restic restore helper via plugin ConfigMap (#2792)
* Add the ability to set the `allowPrivilegeEscalation` security context attribute on the Restic restore helper init container.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>

* Add changelog.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>

* Fix old tests and add tests for new allowPrivilegeEscalation config option.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>

* Correct spelling in changelog.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>

* Switch to boolptr type.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>

* Reorder imports for sanity.

Signed-off-by: Piper Dougherty <doughertypiper@gmail.com>
2020-08-06 13:08:36 -04:00
Rob Reus
db139cf07c Refactor image builds to use buildx for multi arch image building (#2754)
* Refactor image builds to use buildx for multi arch image building

Signed-off-by: Rob Reus <rob@devrobs.nl>

* Adding image build sanity checks to Makefile

Signed-off-by: Rob Reus <rob@devrobs.nl>

* Making locally building of docker images possible

Signed-off-by: Rob Reus <rob@devrobs.nl>

* Adding docs on building container images using buildx

Signed-off-by: Rob Reus <rob@devrobs.nl>

* Adding changelog and implementing feedback from PR

Signed-off-by: Rob Reus <rob@devrobs.nl>

* Making GOPROXY used in the build containers configurable

Signed-off-by: Rob Reus <rob@devrobs.nl>
2020-08-04 11:40:05 -07:00
JenTing Hsiao
1fdb647c7f Add cacert flag for velero backup-location create (#2778)
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
2020-07-30 11:33:45 -07:00
Ashish Amarnath
9de644f1a4 Add types to implement restore hooks (#2761)
* Add types to implement restore hooks

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-29 13:29:40 -07:00
Ashish Amarnath
028818a053 exclude vols mounting secrets and configmaps from defaultVolumesToRestic (#2762)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-27 20:27:49 -07:00
Ashish Amarnath
bc25e789e0 Add constants for restore hook annotation keys (#2750)
* add annotation key constants for restore hooks

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-24 13:05:27 -07:00
Andrew Reed
9011b192e9 Add hooks fields to restore context (#2755)
* Add hooks fields to restore context

Signed-off-by: Andrew Reed <andrew@replicated.com>

* Changelog

Signed-off-by: Andrew Reed <andrew@replicated.com>
2020-07-24 11:43:44 -07:00
Nolan Brubaker
bbcbde084d Create hook package (#2734)
* Move pkg/backup/item_hook_handler to internal/hoo

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>

* Add internal packages to test target

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-07-22 14:26:14 -07:00
Martin Odstrčilík
86efd1577e add support for setting SecurityContext (user, group) for restic restore (#2621)
* add support for setting SecurityContext (user, group) for restic restore

Signed-off-by: Martin Odstrcilik <martin.odstrcilik@gmail.com>
2020-07-22 12:10:25 -07:00
Ashish Amarnath
91ccc4adb2 Add metrics for restic back up operation (#2719)
* add metrics for restic back up operation

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* changelog

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-22 15:07:52 -04:00
Thejas Babu
d0d143e119 Add StartTimestamp and CompletionTimestamp in Restore Status (#2748)
Signed-off-by: thejas <thejasb99@gmail.com>
2020-07-22 11:40:39 -07:00
Nolan Brubaker
caeb4ae404 Update changelogs for v1.4.2 (#2705)
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-07-21 11:14:41 -07:00
fvsqr
01e2dcb364 StorageGrid compatibility (#2712)
* remove explicit Accept-Encoding header

For StorageGrid compatibility the Accept-Encoding header should not be set, otherwise StorageGrid compresses the already compressed log files which are only decompressed by the client once

Signed-off-by: fvsqr <48791253+fvsqr@users.noreply.github.com>

* Removed explicit gzip Accept-Encoding header

For StorageGrid compatibility the Accept-Encoding header should not be set, otherwise StorageGrid compresses the already compressed log files which are only decompressed by the client once.
Unclear, how this affects Backup endpoints from Azure or GCP

Signed-off-by: fvsqr <48791253+fvsqr@users.noreply.github.com>

* Create 2712-fvsqr

Signed-off-by: fvsqr <48791253+fvsqr@users.noreply.github.com>
2020-07-20 13:11:26 -04:00
Nolan Brubaker
f42c63af1b Address insensitive language (#2677)
* Change master to main in most uses

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-07-17 14:59:51 -07:00
kathpeony
243ac62e3f Add backupValidationFailureTotal to metrics (#2714)
* Add backValidationFailureTotal to metrics

Signed-off-by: Kathrin Mao <kathrin.mao@sap.com>
2020-07-16 10:13:17 -07:00
Andrew Lavery
a368370bef k8s 1.18 import (#2651)
* k8s 1.18 import wip

backup, cmd, controller, generated, restic, restore, serverstatusrequest, test and util

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* go mod tidy

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* add changelog file

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* go fmt

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* update code-generator and controller-gen in CI

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* checkout proper code-generator version, regen

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* fix remaining calls

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* regenerate CRDs with ./hack/update-generated-crd-code.sh

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* use existing context in restic and server

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* fix test cases by resetting resource version

also use main library go context, not golang.org/x/net/context, in pkg/restore/restore.go

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* clarify changelog message

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* use github.com/kubernetes-csi/external-snapshotter/v2@v2.2.0-rc1

Signed-off-by: Andrew Lavery <laverya@umich.edu>

* run 'go mod tidy' to remove old external-snapshotter version

Signed-off-by: Andrew Lavery <laverya@umich.edu>
2020-07-16 12:21:37 -04:00
Carlisia Campos
dbd0aa4915 Add a BSL controller to handle validation + update BSL status phase (#2674)
* Add BSL controller

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add changelog

Signed-off-by: Carlisia <carlisia@vmware.com>

* Make update

Signed-off-by: Carlisia <carlisia@vmware.com>

* Update docs

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add kubebuilder dependency

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add export

Signed-off-by: Carlisia <carlisia@vmware.com>

* add kubebuilder binaries into velero builder image

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>

* Reset velero dockerfile

Signed-off-by: Carlisia <carlisia@vmware.com>

* Consolidate all logic

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add copyright header

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up + add "last validated" column

Signed-off-by: Carlisia <carlisia@vmware.com>

* Better tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add more tests

Signed-off-by: Carlisia <carlisia@vmware.com>

* Better logging

Signed-off-by: Carlisia <carlisia@vmware.com>

* Format

Signed-off-by: Carlisia <carlisia@vmware.com>

* Code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove redundant logic

Signed-off-by: Carlisia <carlisia@vmware.com>

Co-authored-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-14 17:47:00 -04:00
Ashish Amarnath
3d3b9e312a Revert "🐛 fix file perissions on the manifest JSON in backup archive (#2685)" (#2700)
dae5230aae

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-10 14:07:05 -07:00
Nolan Brubaker
54aa75a4cd Adjust restic timeout and pod values up (#2696)
* Adjust restic timeout and pod values up

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2020-07-08 14:59:19 -07:00
Ashish Amarnath
dae5230aae 🐛 fix file perissions on the manifest JSON in backup archive (#2685)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-08 06:37:59 -07:00
Ashish Amarnath
b4465e92b8 🐛 Use CRD version prior to remap_crd_version backup item action (#2683)
* 🐛 preserve crd version before remapping

Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-07-07 10:59:41 -07:00
Duffie Cooley
2d48ac79c2 Image handling (#2620)
* Fix Image Handling

Signed-off-by: Duffie Cooley <cooleyd@vmware.com>

Co-authored-by: Carlisia <carlisia@vmware.com>
2020-07-06 15:15:38 -07:00
Daniel Thrasher
94a9522f75 updated acceptable values on cron schedule for day of the week from 0-7 to 0-6 (#2676)
* updated acceptable values on cron schedule for day of the week from 0-7 to 0-6

Signed-off-by: Daniel Thrasher <dannythrasher@gmail.com>

* added a changelog file to changelog directory

Signed-off-by: Daniel Thrasher <dannythrasher@gmail.com>

Co-authored-by: Daniel Thrasher <dannythrasher@gmail.com>
2020-07-06 17:14:44 -04:00
Tony Batard
b968813058 Add linter (#2615)
* Add linter to Makefile and build image

* Also make it part of verify step

Signed-off-by: Tony Batard <tbatard@pivotal.io>

* clean up of Makefile and permissions for .go/golangci-lint

Signed-off-by: Duffie Cooley <cooleyd@vmware.com>

* changed verify-lint.sh to lint.sh to avoid breaking ci

Signed-off-by: mtritabaugh <mtritabaugh@vmware.com>

* Add changelog

Signed-off-by: Tony Batard <tbatard@pivotal.io>

* Add LINTERS option to run only specific linters

* e.g. make lint LINTERS=unused,deadcode

Signed-off-by: Tony Batard <tbatard@pivotal.io>

* adding timeout to golangci-lint, and checking cache

Signed-off-by: Matyas Danter <mdanter@vmware.com>

* Fixed some formatting and added comments

Signed-off-by: Matyas Danter <mdanter@vmware.com>

* modifying lint script to use golangci.yaml

Signed-off-by: Matyas Danter <mdanter@vmware.com>

* update to move default linters to Makefile

Signed-off-by: mtritabaugh <mtritabaugh@vmware.com>

* Adding documentation for lint make targets.

Signed-off-by: Matyas Danter <mdanter@vmware.com>

* Update Copyright with current year

Signed-off-by: Tony Batard <tbatard@pivotal.io>

* initial git workflow commit

Signed-off-by: mtritabaugh <mtritabaugh@vmware.com>

* Added lint-all target and implemented -n as default

* Added a local-lint-all and lint-all target that will show lint errors
for all of the codebase
* changed the default of lint and local-lint to only show new lint
errors

Signed-off-by: Duffie Cooley <cooleyd@vmware.com>

* updated docs to reflect new target

Signed-off-by: mtritabaugh <mtritabaugh@vmware.com>

Co-authored-by: Duffie Cooley <cooleyd@vmware.com>
Co-authored-by: mtritabaugh <mtritabaugh@vmware.com>
Co-authored-by: Matyas Danter <mdanter@vmware.com>
2020-06-30 12:51:10 -04:00
Carlisia Campos
4048c020a8 Convert manifests + BSL api client to kubebuilder (#2561)
* kubebuilder init - minimalist version

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add back main.go, apparently kb needs it

Signed-off-by: Carlisia <carlisia@vmware.com>

* Tweak makefile to accomodate kubebuilder expectations

Signed-off-by: Carlisia <carlisia@vmware.com>

* Port BSL to kubebuilder api client

Signed-off-by: Carlisia <carlisia@vmware.com>

* s/cache/client bc client fetches from cache
And other naming improvements

Signed-off-by: Carlisia <carlisia@vmware.com>

* So, .GetAPIReader is how we bypass the cache
In this case, the cache hasn't started yet

Signed-off-by: Carlisia <carlisia@vmware.com>

* Oh that's what this code was for... adding back

We still need to embed the CRDs as binary data in the Velero binary to
access the generated CRDs at runtime.

Signed-off-by: Carlisia <carlisia@vmware.com>

* Tie in CRD/code generation w/ existing scripts

Signed-off-by: Carlisia <carlisia@vmware.com>

* Mostly result of running update-fmt, updated file formatting

Signed-off-by: Carlisia <carlisia@vmware.com>

* Just a copyright fix

Signed-off-by: Carlisia <carlisia@vmware.com>

* All the test fixes

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add changelog + some cleanup

Signed-off-by: Carlisia <carlisia@vmware.com>

* Update backup manifest

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove unneeded auto-generated files

Signed-off-by: Carlisia <carlisia@vmware.com>

* Keep everything in the same (existing) package

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix/clean scripts, generated code, and calls

Deleting the entire `generated` directory and running `make update`
works. Modifying an api and running `make verify` works as expected.

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up schema and client calls + code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Move all code gen to inside builder container

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix imports/aliases

Signed-off-by: Carlisia <carlisia@vmware.com>

* More code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add waitforcachesync

Signed-off-by: Carlisia <carlisia@vmware.com>

* Have manager register ALL controllers

This will allow for proper cache management.

Signed-off-by: Carlisia <carlisia@vmware.com>

* Status subresource is now enabled; cleanup

Signed-off-by: Carlisia <carlisia@vmware.com>

* More code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up

Signed-off-by: Carlisia <carlisia@vmware.com>

* Manager registers ALL controllers for restic too

Signed-off-by: Carlisia <carlisia@vmware.com>

* More code reviews

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add deprecation warning/todo

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add documentation

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add helpful comments

Signed-off-by: Carlisia <carlisia@vmware.com>

* Address code review

Signed-off-by: Carlisia <carlisia@vmware.com>

* More idiomatic Runnable

Signed-off-by: Carlisia <carlisia@vmware.com>

* Clean up imports

Signed-off-by: Carlisia <carlisia@vmware.com>
2020-06-24 12:55:18 -04:00
Carlisia Campos
0b8e2cbbe5 Improve velero download doc (#2660)
* Improve velero download doc

Signed-off-by: Carlisia <carlisia@vmware.com>

* Add changelog

Signed-off-by: Carlisia <carlisia@vmware.com>

* Better message

Signed-off-by: Carlisia <carlisia@vmware.com>
2020-06-23 11:12:01 -07:00
Ashish Amarnath
fcf0f3e5c2 move csi pluing out of prototype (#2636)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-19 13:27:00 -07:00
Nolan Brubaker
ae1f4e284f Merge pull request #2611 from ashish-amarnath/restic-by-default
  Implement option to back up all volumes by default with restic
2020-06-19 12:42:25 -04:00
Nolan Brubaker
af79f96513 Merge pull request #2638 from adamrushuk/update-docs
Update basic-install and release-instructions documentation
2020-06-17 15:20:56 -04:00
Jess G
c58331d408 add a supported provider: Storj object storage (#2635)
* add storj as provider

Signed-off-by: Jessica Grebenschikov <Jessica.greben1+github@gmail.com>

* update plugin list and master docs

Signed-off-by: Jessica Grebenschikov <Jessica.greben1+github@gmail.com>

* add changelog

Signed-off-by: Jessica Grebenschikov <Jessica.greben1+github@gmail.com>

* fix name of changelog

Signed-off-by: Jessica Grebenschikov <Jessica.greben1+github@gmail.com>

* add newline

Signed-off-by: Jessica Grebenschikov <Jessica.greben1+github@gmail.com>
2020-06-17 11:19:10 -07:00
Adam Rush
2af9d6a52a Changed pr number for changelog
Signed-off-by: Adam Rush <adamrushuk@gmail.com>
2020-06-17 09:04:51 +01:00
Adam Rush
e7b413c7de Update basic-install and release-instructions documentation for Windows Chocolatey package
Signed-off-by: Adam Rush <adamrushuk@gmail.com>
2020-06-17 08:58:53 +01:00
Nolan Brubaker
0bdd6ef5a4 Merge pull request #2629 from adamrushuk/master
Update basic-install.md: Add windows cli installation option via chocolatey
2020-06-16 17:24:00 -04:00
Ashish Amarnath
dd11b175ec changelog
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-06-15 14:46:06 -07:00
Adam Rush
d1b188421f changed pr number for changelog
Signed-off-by: Adam Rush <adamrushuk@gmail.com>
2020-06-13 13:01:58 +01:00
Adam Rush
a728ea8015 add windows cli installation option via chocolatey
Signed-off-by: Adam Rush <adamrushuk@gmail.com>
2020-06-13 12:53:17 +01:00
Tony Batard
9518ac898a Add changelog
Signed-off-by: Tony Batard <tbatard@pivotal.io>
2020-06-11 14:57:28 -07:00
Steve Kriss
a5346c1a87 azure: support aad-pod-identity auth when using restic (#2602)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2020-06-08 11:17:43 -07:00