Steve Kriss
cb321db21f
azure: refactor to not use helpers/ pkg, validate all env/config inputs
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-08-28 13:19:21 -07:00
Steve Kriss
9d7ea7483c
azure: support different RGs/storage accounts per backup location
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-08-28 13:19:21 -07:00
Steve Kriss
cd4e9f5336
azure: fix for breaking change in blob.GetSASURI
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-08-28 13:19:21 -07:00
Steve Kriss
0e94fa37f9
update sync controller for backup locations
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-08-28 13:19:21 -07:00
Steve Kriss
1c26fbde32
remove SnapshotService, replace with direct BlockStore usage
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-07-31 08:58:11 -07:00
Andy Goldstein
130512187a
Refactor plugin management
...
Refactor plugin management:
- support multiple plugins per executable
- support restarting a plugin process in the event it terminates
- simplify plugin lifecycle management by using separate managers for
each scope (server vs backup vs restore)
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com >
2018-07-31 08:34:57 -07:00
Dave Parfitt
06d6665abb
check s3URL scheme upon AWS ObjectStore Init()
...
Signed-off-by: Dave Parfitt <diparfitt@gmail.com >
2018-07-24 14:01:05 -04:00
Steve Kriss
85a61b8e8d
return nil error if 404 encountered when deleting snapshots
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-07-20 11:42:34 -07:00
Mike Arpaia
3efe677007
Update Ark library code to work with Kubernetes 1.11
...
Signed-off-by: Mike Arpaia <mike@arpaia.co >
2018-07-16 20:46:20 -06:00
Steve Kriss
5ad21854f7
restic: if S3, get bucket's region up-front
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-06-21 17:20:53 -07:00
Steve Kriss
170034787d
Merge pull request #513 from ncdc/510-follow-ups
...
Fix copyright date, add happy path test case
2018-05-23 08:54:22 -07:00
Andy Goldstein
18e2401e79
Merge pull request #455 from skriss/remove-aws-region
...
AWS: remove region requirement from config for backupStorageProvider
2018-05-23 11:02:57 -04:00
Andy Goldstein
849297e623
Fix copyright date, add happy path test case
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com >
2018-05-23 09:59:32 -04:00
Nolan Brubaker
09bbe072cd
Check errors when closing GCP objects
...
Writing to GCP's object store is any async operation, so errors need to
be checked both on write and close calls, since errors like permission
violations aren't reported until a close.
Signed-off-by: Nolan Brubaker <nolan@heptio.com >
2018-05-22 17:38:49 -04:00
Steve Kriss
b029860b46
AWS: remove region requirement from config for backupStorageProvider
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-05-22 11:59:53 -07:00
Steve Kriss
e81de2491f
remove duplicate github.com/satori/go.uuid dependency
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-05-07 10:02:59 -07:00
Steve Kriss
72d7e5e01d
azure: pass full snapshot name when restoring disk
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-04-23 12:51:05 -07:00
Steve Kriss
a466208975
add backwards-compatibility for azure snapshot ID
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-04-23 11:34:37 -07:00
Steve Kriss
0d2b49acce
get backup name from restore spec when handling restore DownloadRequests
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-04-09 10:50:10 -07:00
Andy Goldstein
3172e907a1
Merge pull request #341 from skriss/snapshot-tags
...
Add tags to snapshots, restored volumes
2018-03-14 12:50:40 -04:00
Andy Goldstein
07fcc927b7
Merge pull request #359 from skriss/reduce-azure-polling-interval
...
Azure: reduce client polling interval from default 60s to 5s
2018-03-14 12:17:54 -04:00
Steve Kriss
45cee7d222
Azure: reduce client polling interval from default 60s to 5s
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-03-14 09:07:20 -07:00
Steve Kriss
b2b16b3c85
Azure: fix bug preventing snapshot deletion
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-03-13 17:03:14 -07:00
Steve Kriss
0388845bca
AWS: tag snapshots during create
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-03-13 12:17:50 -07:00
Steve Kriss
a05ae1a7cf
add useful Ark tags to snapshots
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-03-13 12:17:50 -07:00
Steve Kriss
df985bca74
Azure: copy tags from volume to snapshot, and snapshot to volume
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-03-13 12:17:50 -07:00
Steve Kriss
cab904570f
GCP: copy tags from volume to snapshot, and snapshot to volume
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-03-13 11:52:09 -07:00
Steve Kriss
9673e9d158
AWS: copy tags from volume to snapshot, and snapshot to volume
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-03-13 11:52:09 -07:00
Steve Kriss
409f17361d
Azure: store snapshot URI to support cross-resource group restores
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-03-13 09:18:20 -07:00
Andy Goldstein
331e0c28cc
Remove Azure location requirement
...
Instead of requiring the Ark admin to specify a "location" in the azure
persistentVolumeProvider config (meaning only a single location is
supported), get info about the disk (for its location) when creating a
snapshot, and get info about the snapshot (for its location) when
creating a disk from a snapshot.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com >
2018-03-05 15:20:58 -05:00
Steve Kriss
cbff5ef53b
extract GCP project ID from creds file, remove from config
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-03-02 09:15:12 -08:00
Steve Kriss
af81f589f6
update GCP block store to use v1 library, and tag snapshots on create
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-01-08 09:23:32 -08:00
Steve Kriss
b53ee44176
update GCP object store to use new/recommended lib
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-01-05 14:21:13 -08:00
Steve Kriss
656428d0b9
Merge pull request #250 from ncdc/backup-controller-do-as-much-as-possible
...
BackupController: do as much as possible
2018-01-03 12:10:51 -08:00
Steve Kriss
1503796419
make language and casing consistent across copyrights
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2018-01-02 10:51:49 -08:00
Andy Goldstein
1e581f1ead
BackupController: do as much as possible
...
When running a backup, try to do as much as possible, collecting errors
along the way, and return an aggregate at the end. This way, if a backup
fails for most reasons, we'll be able to upload the backup log file to
object storage, which wasn't happening before.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com >
2017-12-20 14:49:27 -05:00
Andy Goldstein
1210cb36e1
Merge pull request #253 from skriss/update-license
...
Update all license headers
2017-12-19 19:26:54 -05:00
Steve Kriss
3641c2c043
update all license headers
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2017-12-15 13:38:12 -08:00
Steve Kriss
56c9d68137
switch to logrus.FieldLogger and arktest.NewLogger() everywhere
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2017-12-13 09:38:00 -08:00
Steve Kriss
cbcd15d603
add/update documentation for plugins
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2017-11-29 14:24:23 -08:00
Steve Kriss
038fa39451
remove ListSnapshots and GetAllSnapshots
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2017-11-29 13:59:29 -08:00
Andy Goldstein
c700455272
Support custom volume snapshots & restores
...
The main Ark code was hard-coding specific support for AWS, GCE, and
Azure volume snapshots and restores, and anything else was considered
unsupported.
Add GetVolumeID and SetVolumeID to the BlockStore interface, to allow
block store plugins to handle volume snapshots and restores.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com >
2017-11-29 13:19:40 -05:00
Steve Kriss
24ce316788
switch built-in cloud providers to run as plugins
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2017-11-14 09:47:36 -08:00
Steve Kriss
21e2019540
rename Block/ObjectStoreAdapter -> Block/ObjectStore
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2017-11-08 16:58:47 -08:00
Andy Goldstein
b2d80471ac
Move restore warnings/errors to object storage
...
If you have a large number of warnings and/or errors, the restore
object's size can exceed the maximum allowed by etcd. Move them to
object storage, and add a new describe command to fetch and display them
on the fly.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com >
2017-11-02 15:53:59 -04:00
Steve Kriss
e7703d88ec
switch UploadBackup, UploadRestoreLog, PutObject to take io.Reader
...
Signed-off-by: Steve Kriss <steve@heptio.com >
2017-11-01 15:48:13 -07:00
Andy Goldstein
c1bc52eb65
Update gofmt
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com >
2017-10-30 13:31:03 -04:00
Andy Goldstein
15fe87aea3
Fix clientset imports
...
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com >
2017-10-30 13:31:03 -04:00
ashish-amarnath
6dc81b42f5
Merge branch 'master' into support-multi-az-pvs
...
Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com >
2017-10-06 23:24:41 -07:00
Ashish Amarnath
9fc9dbb413
Preserve PV's AZ info when snapshotting and restoring PVs.
...
- Read PV's AZ info from fault-domain label of the PV object for snapshotting.
- Store PV's AZ info in the VolumeInfo.
- Add tests for reading the label from the PV object.
- Remove availability zone validation in AWS and GCP BlockStorageAdaptor.
- Add volumeAZ as a parameter to methods in the BlockStorageAdapter interface.
- Get AZ from VolumeInfo when restoring PV snapshot.
- Remove references to PV availability zone in docs.
Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com >
2017-10-06 23:24:41 -07:00