Commit Graph

52 Commits

Author SHA1 Message Date
Andy Goldstein
ea83ed32f5 Merge pull request #470 from skriss/backup-cluster-roles-and-bindings
backups: include clusterroles/bindings that reference serviceaccounts
2018-05-09 13:21:13 -04:00
Steve Kriss
041cfc2173 backups: include clusterroles/bindings that reference serviceaccounts
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-09 10:03:55 -07:00
Nolan Brubaker
923870390b Skip completed jobs and pods when restoring
Completed jobs and pods may be useful in the backup for auditing
purposes, but don't recreate them when restoring.

Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-05-02 12:16:54 -04:00
Steve Kriss
97e52f2b3c add timeout arg to rest.NewRequest()
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-04-20 12:58:53 -07:00
Andy Goldstein
4328b67f93 backup delete/describe enhancements
Always request DeleteBackupRequests for a given backup so we can show
failed deletion attempts if you try to delete a backup that has PV
snapshots when Ark doesn't have a persistentVolumeProvider configured.

When creating a DeleteBackupRequest, include a label for the UID so we
can match based on name and UID when associated DeleteBackupRequests
with a given backup.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-04-05 14:21:45 -04:00
Andy Goldstein
74f60b1ee1 Switch backup finalizer to DeleteBackupRequest
We ran into a lot of problems using a finalizer on the backup to allow
the Ark server to clean up all associated backup data when deleting a
backup.

Users also found it less than desirable that deleting the heptio-ark
namespace resulted in all the backup data being deleted.

This removes the finalizer and replaces it with an explicit
DeleteBackupRequest that is created as a means of requesting the
deletion of a backup and all its associated data. This is what `ark
backup delete` does.

If you use kubectl to delete a backup or to delete the heptio-ark
namespace, this no longer deletes associated backups. Additionally, as
long as the heptio-ark namespace still exists, the Ark server's
BackupSyncController will continually sync backups into the heptio-ark
namespace from object storage.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-04-05 11:16:15 -04:00
Steve Kriss
a05ae1a7cf add useful Ark tags to snapshots
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-13 12:17:50 -07:00
Sanket Patel
7257a75f82 Add events to nonRestorableResources and cohabitatingResources
Fixes #367
Fixes #368

Signed-off-by: Sanket Patel <sanketpatel.301090@gmail.com>
2018-03-12 19:55:32 -05:00
Steve Kriss
26944cf9a2 only log if hooks are actually executing
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-07 09:29:27 -08:00
Andy Goldstein
de785af89d Support pre and post hooks.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-01-05 10:47:34 -05:00
Steve Kriss
656428d0b9 Merge pull request #250 from ncdc/backup-controller-do-as-much-as-possible
BackupController: do as much as possible
2018-01-03 12:10:51 -08:00
Andy Goldstein
1b124a3146 Log backup item action error when it occurs
If a backup item action errors, log the error as soon as it occurs, so
it's clear when the error happened. Also include information about the
groupResource, namespace, and name of the item in the error.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-01-03 13:20:05 -05:00
Steve Kriss
1503796419 make language and casing consistent across copyrights
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-01-02 10:51:49 -08:00
Andy Goldstein
0fc087c967 Flatten aggregated errors
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-12-21 11:56:33 -05:00
Andy Goldstein
fea701a03b Merge pull request #256 from skriss/unbound-pvc-fix
don't error if PVC is not bound to a PV
2017-12-19 19:37:28 -05:00
Andy Goldstein
1210cb36e1 Merge pull request #253 from skriss/update-license
Update all license headers
2017-12-19 19:26:54 -05:00
Steve Kriss
79c0e5a09a don't error if PVC is not bound to a PV
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-19 14:10:58 -08:00
Steve Kriss
3641c2c043 update all license headers
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-15 13:38:12 -08:00
Steve Kriss
56c9d68137 switch to logrus.FieldLogger and arktest.NewLogger() everywhere
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-13 09:38:00 -08:00
Andy Goldstein
bd8f433188 Merge pull request #229 from skriss/plugin-docs
add godoc for ResourceSelector
2017-11-30 11:25:53 -05:00
Steve Kriss
94ac3b3529 godoc for ResourceSelector
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-30 08:20:27 -08:00
Steve Kriss
cbcd15d603 add/update documentation for plugins
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-29 14:24:23 -08:00
Andy Goldstein
c700455272 Support custom volume snapshots & restores
The main Ark code was hard-coding specific support for AWS, GCE, and
Azure volume snapshots and restores, and anything else was considered
unsupported.

Add GetVolumeID and SetVolumeID to the BlockStore interface, to allow
block store plugins to handle volume snapshots and restores.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-29 13:19:40 -05:00
Steve Kriss
179b95c81d convert restorers to plugins
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-28 10:58:12 -08:00
Steve Kriss
0f2d1ab82b migrate backup actions to plugins
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-21 10:03:03 -08:00
Andy Goldstein
af189fd5f4 Add error & logSource hooks to backup & restore loggers
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-14 14:19:52 -05:00
Andy Goldstein
737b6d932a Add more group-resource level logging
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-08 12:08:57 -05:00
Andy Goldstein
efa4e57d1e More namespace special casing
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-03 16:34:07 -04:00
Devan Goodwin
9471f9da3c Fix missing namespace metadata after restore.
If backing up specific namespaces with "auto" cluster resources, the
actual namespace objects themselves were not being included in the
backup. Restore would create them but any labels or metadata would be
lost.

Instead handle the special case of namespace as a cluster level resource
we may still need, even if excluding most cluster level resources.

Signed-off-by: Devan Goodwin <dgoodwin@redhat.com>
2017-11-03 12:12:03 -03:00
Andy Goldstein
c1bc52eb65 Update gofmt
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-30 13:31:03 -04:00
Andy Goldstein
293674c40e React to upstream spdy executor changes
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-30 13:31:03 -04:00
Andy Goldstein
43449885a1 React to move to k8s.io/api
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-30 13:30:27 -04:00
Akhil Thampy
1ac0303283 Fix #137. Remove clock from volume_snapshot_action.
Signed-off-by: Akhil Thampy <akhilthampy@yahoo.com>
2017-10-26 13:53:52 -05:00
Andy Goldstein
901f8e1302 Add pod exec backup hooks
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-20 10:20:59 -04:00
Devan Goodwin
ed0194c09b Restructure backups for resource prioritization.
Previously the directory structure separated resources depending on
whether or not they were cluster or namespace scoped. All cluster
resources were restored first, then all namespace resources. Priority
did not apply across both and you could not order any namespace
resources before any cluster resources.

This restructure sorts firstly on resource type.

resources/serviceaccounts/namespaces/ns1.json
resources/nodes/cluster/node1.json

This will break old backups as the format is no longer consistent as
announced on the Google group.

Signed-off-by: Devan Goodwin <dgoodwin@redhat.com>
2017-10-17 08:49:05 -03:00
Steve Kriss
4085e473e1 additional deployment/networkpolicy cases for TestBackupResource
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-11 15:05:12 -07:00
Steve Kriss
07b33ace57 add --include-cluster-resources flag to backup create cmd
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-11 15:05:12 -07:00
Steve Kriss
9438a8670a when backing up PVCs, also back up claimed PVs
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-11 09:54:57 -07:00
Steve Kriss
fa427ebcfd switch to logrus for per-backup/restore logs
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-10 14:11:21 -07:00
Steve Kriss
82655b8359 tweak volume snapshot action logging messages
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-10 11:10:50 -07:00
ashish-amarnath
6dc81b42f5 Merge branch 'master' into support-multi-az-pvs
Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
2017-10-06 23:24:41 -07:00
Ashish Amarnath
9fc9dbb413 Preserve PV's AZ info when snapshotting and restoring PVs.
- Read PV's AZ info from fault-domain label of the PV object for snapshotting.
- Store PV's AZ info in the VolumeInfo.
- Add tests for reading the label from the PV object.
- Remove availability zone validation in AWS and GCP BlockStorageAdaptor.
- Add volumeAZ as a parameter to methods in the BlockStorageAdapter interface.
- Get AZ from VolumeInfo when restoring PV snapshot.
- Remove references to PV availability zone in docs.

Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
2017-10-06 23:24:41 -07:00
Steve Kriss
914165095a switch logging to logrus and errors to pkg/errors
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-05 08:49:33 -07:00
Steve Kriss
907ae6c5b0 move groupresource resolution into discovery helper
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-09-11 14:41:50 -07:00
Andy Goldstein
3bb02ac949 Use ioutil.Discard in unit test
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-11 15:40:49 -04:00
Andy Goldstein
71fadf381e Clarify Backup() param names
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-11 15:40:49 -04:00
Andy Goldstein
a583f22ea9 Add ActionContext so backup actions can log
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-11 15:40:48 -04:00
Andy Goldstein
9848a7a55b Save backup log file to object storage
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-11 15:40:48 -04:00
Steve Kriss
768aed4ddd make --snapshot-volumes, --restore-volumes optional with sensible default behavior based on PVProvider existence
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-08-23 09:47:52 -07:00
Steve Kriss
ebc06fd632 make PVProvider optional in server config; disallow snap/restore PVs when not provided
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-08-23 09:46:15 -07:00