Commit Graph

187 Commits

Author SHA1 Message Date
Steve Kriss
8bc7e4f6aa store backups & restores in backups/, restores/ subdirs in obj storage
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-25 09:56:41 -06:00
Steve Kriss
d34994cb5f set azure restic env vars based on default backup location's config
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-05 14:31:42 -06:00
Steve Kriss
b31e25bf6e server: remove unused code, replace deprecated func
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
Steve Kriss
729d733986 controllers: take a newPluginManager func in constructors
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
Steve Kriss
133dc185ca backup sync: process the default location first
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
Steve Kriss
6f7bfe545d remove Config CRD's BackupStorageProvider & other obsolete code
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
Steve Kriss
bd4d97b9e4 move server's defaultBackupLocation into config struct
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
Steve Kriss
0e94fa37f9 update sync controller for backup locations
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
Carlisia
2750aa71b9 Use backup storage location during restore
Closes #740

Signed-off-by: Carlisia <carlisia@grokkingtech.io>
2018-08-28 13:19:21 -07:00
Steve Kriss
20f89fbcef use the default backup storage location for restic
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
Steve Kriss
cf7c8587f0 download request: fix setting of log level for plugin manager
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:21 -07:00
Steve Kriss
3234124afe backup deletion: fix setting of log level in plugin manager
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:20 -07:00
Steve Kriss
8f5346150c download request controller: use backup location for object store
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:20 -07:00
Steve Kriss
bab08ed1a6 backup deletion controller: use backup location for object store
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:19:20 -07:00
Nolan Brubaker
c6f488f75f Use backup location in the backup controller
Fixes #739

Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-08-28 13:19:20 -07:00
Steve Kriss
adbcd3703b add --default-backup-storage-location flag to server cmd
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-28 13:14:51 -07:00
Carlisia
9508e4a20e Switch Config CRD elements to server flags
Signed-off-by: Carlisia <carlisia@grokkingtech.io>
2018-08-16 08:51:46 -07:00
Nolan Brubaker
eb6f742b5d Document CRD not found errors
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-08-02 16:41:28 -04:00
Steve Kriss
1c26fbde32 remove SnapshotService, replace with direct BlockStore usage
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-31 08:58:11 -07:00
Andy Goldstein
130512187a Refactor plugin management
Refactor plugin management:
- support multiple plugins per executable
- support restarting a plugin process in the event it terminates
- simplify plugin lifecycle management by using separate managers for
  each scope (server vs backup vs restore)

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-07-31 08:34:57 -07:00
Steve Kriss
131afb571e Merge pull request #607 from nrb/restore-metrics
Restore metrics
2018-07-27 12:25:56 -07:00
Nolan Brubaker
82f1cd87dc Merge pull request #680 from carlisia/c-delete-from-etcd
Delete backups from etcd if they're not in storage
2018-07-25 18:25:52 -04:00
Carlisia
d2e629f5a8 Delete backups from etcd if they're not in storage
This fixes #309.

Signed-off-by: Carlisia <carlisia@grokkingtech.io>

Co-authored-by: Steve Kriss <steve@heptio.com>
2018-07-25 14:53:50 -07:00
Nolan Brubaker
7cebfe2df0 Add restore attempt and success/failure counters
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-07-25 11:22:43 -04:00
Nolan Brubaker
b0af81e780 Merge pull request #689 from lemaral/init-schedule-metrics
Init schedule Prometheus metrics
2018-07-23 16:16:26 -04:00
Carlisia
c47a364ab3 Merge pull request #687 from skriss/fix-delete-issues
fix delete issues
2018-07-23 10:40:14 -07:00
Alex Lemaresquier
450fa72fbb Initialize schedule Prometheus metrics to have them created beforehand (see https://prometheus.io/docs/practices/instrumentation/#avoid-missing-metrics)
Signed-off-by: Alex Lemaresquier <alex+github@lemaresquier.org>
2018-07-20 23:30:24 +02:00
Steve Kriss
78cbdf95f3 delete old deletion requests for backup when processing a new one
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-20 11:42:42 -07:00
Steve Kriss
1df9a8a38d exit server if not all Ark CRDs exist at startup
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-18 14:30:42 -07:00
Nolan Brubaker
e11634bfbc Merge pull request #676 from skriss/dont-create-namespace
require namespace for backups/etc. to exist at server startup
2018-07-17 14:26:04 -04:00
Mike Arpaia
3efe677007 Update Ark library code to work with Kubernetes 1.11
Signed-off-by: Mike Arpaia <mike@arpaia.co>
2018-07-16 20:46:20 -06:00
Steve Kriss
d195512062 require namespace for backups/etc. to exist at server startup
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-07-13 13:50:12 -07:00
Steve Kriss
22e8f23e2c replace ark restic repo init cmd with automatic initialization of repos
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-27 13:50:37 -07:00
Steve Kriss
f2072e5868 use repo identifier from ResticRepo CR when backing up/restoring
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 10:42:53 -07:00
Steve Kriss
65ed8da4b7 add ResticRepository CRD and move repo-level actions to a controller
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-20 09:21:43 -07:00
Andy Goldstein
cc9140b3cc Log to stdout instead of stderr
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-06-14 14:02:12 -04:00
Ashish Amarnath
83658e891e Add a metrics package to add and expose metrics
* add a metrics package to handle metric registration and publishing
* add a metricsAddress field to the server struct
* make metrics a part of the server
* start a metrics endpoint as part of starting the controllers
* instrument backup_controller to report metrics
* update cli-reference docs
* update example deployments with prometheus annotations
* update 'pkg/install' tooling with prometheus annotations

Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
2018-06-13 13:17:08 -07:00
Steve Kriss
50d4084fac add restic integration for doing pod volume backups/restores
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-06 09:48:10 -07:00
Andy Goldstein
18e2401e79 Merge pull request #455 from skriss/remove-aws-region
AWS: remove region requirement from config for backupStorageProvider
2018-05-23 11:02:57 -04:00
Steve Kriss
b029860b46 AWS: remove region requirement from config for backupStorageProvider
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-22 11:59:53 -07:00
Steve Kriss
20f56e9868 cleanup plugin subprocesses on server exit
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-11 09:48:46 -07:00
Steve Kriss
09c20b51e6 shutdown gracefully on SIGINT/SIGTERM
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-10 16:22:14 -07:00
Steve Kriss
6754955bcd when syncing backups, set their namespace to current cluster's Ark ns
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-07 09:12:51 -07:00
Andy Goldstein
258f3e011e Merge pull request #409 from skriss/fix-398
get backup name from restore spec when handling restore DownloadRequests
2018-04-10 13:29:07 -04:00
Steve Kriss
0d2b49acce get backup name from restore spec when handling restore DownloadRequests
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-04-09 10:50:10 -07:00
Andy Goldstein
3f2e222ae4 Don't allow deletion of in-progress backups
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-04-06 13:08:39 -04:00
Andy Goldstein
74f60b1ee1 Switch backup finalizer to DeleteBackupRequest
We ran into a lot of problems using a finalizer on the backup to allow
the Ark server to clean up all associated backup data when deleting a
backup.

Users also found it less than desirable that deleting the heptio-ark
namespace resulted in all the backup data being deleted.

This removes the finalizer and replaces it with an explicit
DeleteBackupRequest that is created as a means of requesting the
deletion of a backup and all its associated data. This is what `ark
backup delete` does.

If you use kubectl to delete a backup or to delete the heptio-ark
namespace, this no longer deletes associated backups. Additionally, as
long as the heptio-ark namespace still exists, the Ark server's
BackupSyncController will continually sync backups into the heptio-ark
namespace from object storage.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-04-05 11:16:15 -04:00
Shubham
73499c2cff Add limitranges to defaultResourcePriorities
This commit adds limitranges to defaultResourcePriorities as
suggested in #385.

This is done so that pods are not restored before the LimitRange
objects, because that would lead to pods not honoring the requests
and limits set in LimitRange objects.

Fixes #385

Signed-off-by: Shubham <shubham@linux.com>
2018-03-17 21:13:44 +05:30
Andy Goldstein
e618e0e456 server: allow configurable plugin dir
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-02-27 14:50:38 -05:00
Blake
465c60b408 Add global kubecontext CLI flag
Signed-off-by: Blake <blake.barnett@postmates.com>
2018-02-26 10:47:39 -08:00