Commit Graph

361 Commits

Author SHA1 Message Date
Steve Kriss
51928e9177 use typed structs for decoding patch JSON in unit tests
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-07 09:22:20 -07:00
Nolan Brubaker
923870390b Skip completed jobs and pods when restoring
Completed jobs and pods may be useful in the backup for auditing
purposes, but don't recreate them when restoring.

Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-05-02 12:16:54 -04:00
Andy Goldstein
18d6b233da Merge pull request #453 from nrb/fix-444
Make empty excludes string more accurate
2018-04-24 14:53:33 -04:00
Nolan Brubaker
05cb059b1a Make empty excludes string more accurate
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-04-24 14:34:56 -04:00
Steve Kriss
989169dcfe remove unused kubernetes/pkg/util/version dependency
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-04-20 12:59:22 -07:00
Nolan Brubaker
300a010959 Remove finalizers from backups before syncing
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-04-18 17:32:39 -04:00
Andy Goldstein
258f3e011e Merge pull request #409 from skriss/fix-398
get backup name from restore spec when handling restore DownloadRequests
2018-04-10 13:29:07 -04:00
Steve Kriss
0d2b49acce get backup name from restore spec when handling restore DownloadRequests
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-04-09 10:50:10 -07:00
Steve Kriss
5bb47d2b7a extract a helper for comparing slices of Actions reliably
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-04-09 09:39:33 -07:00
Andy Goldstein
74f60b1ee1 Switch backup finalizer to DeleteBackupRequest
We ran into a lot of problems using a finalizer on the backup to allow
the Ark server to clean up all associated backup data when deleting a
backup.

Users also found it less than desirable that deleting the heptio-ark
namespace resulted in all the backup data being deleted.

This removes the finalizer and replaces it with an explicit
DeleteBackupRequest that is created as a means of requesting the
deletion of a backup and all its associated data. This is what `ark
backup delete` does.

If you use kubectl to delete a backup or to delete the heptio-ark
namespace, this no longer deletes associated backups. Additionally, as
long as the heptio-ark namespace still exists, the Ark server's
BackupSyncController will continually sync backups into the heptio-ark
namespace from object storage.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-04-05 11:16:15 -04:00
Steve Kriss
a05ae1a7cf add useful Ark tags to snapshots
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-03-13 12:17:50 -07:00
Andy Goldstein
d24fb232cc Allow forced backup deletion
Add --force and --confirm to `ark backup delete` to support forced
backup deletion. This forcibly removes the Ark GC finalizer (if it's
present) from a backup and will orphan any resources associated with the
backup, such as backup tarballs in object storage, persistent volume
snapshots, and restores for the backup.

If a backup has a deletion timestamp, display `Deleting` in `ark backup
describe` and `ark backup get`.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-02-26 16:25:22 -05:00
Andy Goldstein
6b0b6373da Merge pull request #252 from skriss/delete-backup
add delete backup cmd using finalizer and simplify GC process
2018-01-02 14:26:03 -05:00
Steve Kriss
1503796419 make language and casing consistent across copyrights
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-01-02 10:51:49 -08:00
Steve Kriss
1c974782fa disable GC and backup deletion if Kubernetes is less than v1.7.5
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-22 10:25:32 -08:00
Steve Kriss
f5123794e0 add delete backup cmd using finalizer and simplify GC process
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-21 10:47:37 -08:00
Andy Goldstein
1210cb36e1 Merge pull request #253 from skriss/update-license
Update all license headers
2017-12-19 19:26:54 -05:00
Steve Kriss
3641c2c043 update all license headers
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-15 13:38:12 -08:00
Steve Kriss
56c9d68137 switch to logrus.FieldLogger and arktest.NewLogger() everywhere
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-13 09:38:00 -08:00
Steve Kriss
4aea9b9a2c switch from Update() to Patch()
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-12-12 15:23:45 -08:00
Steve Kriss
038fa39451 remove ListSnapshots and GetAllSnapshots
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-29 13:59:29 -08:00
Steve Kriss
5b8562e73c Merge pull request #222 from ncdc/fix-log-location-hook
Fix log location hook prefix stripping
2017-11-29 11:07:43 -08:00
Andy Goldstein
c700455272 Support custom volume snapshots & restores
The main Ark code was hard-coding specific support for AWS, GCE, and
Azure volume snapshots and restores, and anything else was considered
unsupported.

Add GetVolumeID and SetVolumeID to the BlockStore interface, to allow
block store plugins to handle volume snapshots and restores.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-29 13:19:40 -05:00
Andy Goldstein
99f67db39d Fix log location hook prefix stripping
The log location hook was matching github.com/heptio/ark and stripping
off that + 1 more char. This meant that
github.com/heptio/ark-plugin-example/foo.go was being listed as
plugin-example/foo.go instead of
github.com/heptio/ark-plugin-example/foo.go.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-29 12:56:01 -05:00
Andy Goldstein
526b604237 Add boolptr helper package
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-29 12:25:10 -05:00
Steve Kriss
179b95c81d convert restorers to plugins
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-28 10:58:12 -08:00
Andy Goldstein
4c481f4d23 Make logSource more concise
Strip off leading .../github.com/heptio/ark/

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-14 14:21:19 -05:00
Steve Kriss
8ba5a29679 add a logger that can be used within plugins to emit logs into Ark server
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-14 10:38:26 -08:00
Steve Kriss
24ce316788 switch built-in cloud providers to run as plugins
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-14 09:47:36 -08:00
Steve Kriss
21e2019540 rename Block/ObjectStoreAdapter -> Block/ObjectStore
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-08 16:58:47 -08:00
Andy Goldstein
b2d80471ac Move restore warnings/errors to object storage
If you have a large number of warnings and/or errors, the restore
object's size can exceed the maximum allowed by etcd. Move them to
object storage, and add a new describe command to fetch and display them
on the fly.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-11-02 15:53:59 -04:00
Steve Kriss
e7703d88ec switch UploadBackup, UploadRestoreLog, PutObject to take io.Reader
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-11-01 15:48:13 -07:00
Andy Goldstein
15fe87aea3 Fix clientset imports
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-30 13:31:03 -04:00
Andy Goldstein
43449885a1 React to move to k8s.io/api
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-30 13:30:27 -04:00
Andy Goldstein
901f8e1302 Add pod exec backup hooks
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-10-20 10:20:59 -04:00
Steve Kriss
07b33ace57 add --include-cluster-resources flag to backup create cmd
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-11 15:05:12 -07:00
Steve Kriss
9438a8670a when backing up PVCs, also back up claimed PVs
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-11 09:54:57 -07:00
Steve Kriss
7c0d9dcfce add missing licenses
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-10 13:14:03 -07:00
Steve Kriss
e7c62b5f38 add file/line numbers to logs
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-10 13:14:03 -07:00
ashish-amarnath
6dc81b42f5 Merge branch 'master' into support-multi-az-pvs
Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
2017-10-06 23:24:41 -07:00
Ashish Amarnath
9fc9dbb413 Preserve PV's AZ info when snapshotting and restoring PVs.
- Read PV's AZ info from fault-domain label of the PV object for snapshotting.
- Store PV's AZ info in the VolumeInfo.
- Add tests for reading the label from the PV object.
- Remove availability zone validation in AWS and GCP BlockStorageAdaptor.
- Add volumeAZ as a parameter to methods in the BlockStorageAdapter interface.
- Get AZ from VolumeInfo when restoring PV snapshot.
- Remove references to PV availability zone in docs.

Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
2017-10-06 23:24:41 -07:00
Steve Kriss
9ac71b03cc add logrus hook to log error file/line/function from stack traces
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-05 08:49:33 -07:00
Steve Kriss
914165095a switch logging to logrus and errors to pkg/errors
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-10-05 08:49:33 -07:00
Steve Kriss
acb3babb87 when restoring azureDisk from snapshot, update the diskURI with the new diskName
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-09-13 16:10:30 -07:00
Andy Goldstein
273b563c86 Add per-restore logs
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-13 09:51:40 -04:00
Andy Goldstein
d0b7880881 Merge pull request #78 from skriss/restore-resource-inc-exc
Add --include-resources, --exclude-resources for restore operations
2017-09-12 15:53:21 -04:00
Steve Kriss
4dfce17de5 add resource includes/excludes for Restores
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-09-12 12:43:23 -07:00
Justin Nauman
98d4660d27 Adding in support for backup download
- Adding in support for a new `download` subcommand of backup
- Adjusted signing to allow for multiple types
- Adding in git sha version during build more granular version
  debugging

Signed-off-by: Justin Nauman <justin.r.nauman@gmail.com>
2017-09-11 20:17:33 -05:00
Steve Kriss
907ae6c5b0 move groupresource resolution into discovery helper
Signed-off-by: Steve Kriss <steve@heptio.com>
2017-09-11 14:41:50 -07:00
Andy Goldstein
50a95d052e Delete all objects in backup dir
Delete all objects in backup "dir" when deleting a backup, instead of
hard-coding individual file names/types. This way, we'll be able to
delete log files and anything else we add without having to update our
deletion code.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2017-09-11 15:40:48 -04:00