From 3646fcce46f5ce437eb39c5d07eecc409b5063dc Mon Sep 17 00:00:00 2001 From: JENNIFER RONDEAU Date: Mon, 14 May 2018 07:54:13 -0700 Subject: [PATCH 1/2] add docs site build to gitignore Signed-off-by: JENNIFER RONDEAU --- .gitignore | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.gitignore b/.gitignore index 23306f1e5..92c0a6f6c 100644 --- a/.gitignore +++ b/.gitignore @@ -36,3 +36,5 @@ debug .push-* .vscode *.diff + +_site/ From cfdcd65f418e12a4a25b4f676b0dafca605f85a6 Mon Sep 17 00:00:00 2001 From: JENNIFER RONDEAU Date: Thu, 5 Jul 2018 11:33:22 -0400 Subject: [PATCH 2/2] reorganize and edit docs for upcoming release Signed-off-by: JENNIFER RONDEAU --- README.md | 174 +++------------------------------------------ docs/about.md | 45 +++++------- docs/quickstart.md | 149 ++++++++++++++++++++++++++++++++++++++ docs/restic.md | 96 +++++++++++++------------ 4 files changed, 226 insertions(+), 238 deletions(-) create mode 100644 docs/quickstart.md diff --git a/README.md b/README.md index 266a00da7..1cb7d9900 100644 --- a/README.md +++ b/README.md @@ -19,155 +19,7 @@ Ark consists of: ## More information -[The documentation][29] provides detailed information about building from source, architecture, extending Ark, and more. - -## Getting started - -The following example sets up the Ark server and client, then backs up and restores a sample application. - -For simplicity, the example uses Minio, an S3-compatible storage service that runs locally on your cluster. See [Set up Ark with your cloud provider][3] for how to run on a cloud provider. - -### Prerequisites - -* Access to a Kubernetes cluster, version 1.7 or later. Version 1.7.5 or later is required to run `ark backup delete`. -* A DNS server on the cluster -* `kubectl` installed - -### Download - -Clone or fork the Ark repository: - -``` -git clone git@github.com:heptio/ark.git -``` - -NOTE: Make sure to check out the appropriate version. We recommend that you check out the latest tagged version. The master branch is under active development and might not be stable. - -### Set up server - -1. Start the server and the local storage service. In the root directory of Ark, run: - - ```bash - kubectl apply -f examples/common/00-prereqs.yaml - kubectl apply -f examples/minio/ - ``` - - NOTE: If you get an error about Config creation, wait for a minute, then run the commands again. - -1. Deploy the example nginx application: - - ```bash - kubectl apply -f examples/nginx-app/base.yaml - ``` - -1. Check to see that both the Ark and nginx deployments are successfully created: - - ``` - kubectl get deployments -l component=ark --namespace=heptio-ark - kubectl get deployments --namespace=nginx-example - ``` - -### Install client - -For this example, we recommend that you [download a pre-built release][26]. - -You can also [build from source][7]. - -Make sure that you install somewhere in your `$PATH`. - -### Back up - -1. Create a backup for any object that matches the `app=nginx` label selector: - - ``` - ark backup create nginx-backup --selector app=nginx - ``` - - Alternatively if you want to backup all objects *except* those matching the label `backup=ignore`: - - ``` - ark backup create nginx-backup --selector 'backup notin (ignore)' - ``` - -1. Simulate a disaster: - - ``` - kubectl delete namespace nginx-example - ``` - -1. To check that the nginx deployment and service are gone, run: - - ``` - kubectl get deployments --namespace=nginx-example - kubectl get services --namespace=nginx-example - kubectl get namespace/nginx-example - ``` - - You should get no results. - - NOTE: You might need to wait for a few minutes for the namespace to be fully cleaned up. - -### Restore - -1. Run: - - ``` - ark restore create --from-backup nginx-backup - ``` - -1. Run: - - ``` - ark restore get - ``` - - After the restore finishes, the output looks like the following: - - ``` - NAME BACKUP STATUS WARNINGS ERRORS CREATED SELECTOR - nginx-backup-20170727200524 nginx-backup Completed 0 0 2017-07-27 20:05:24 +0000 UTC - ``` - -NOTE: The restore can take a few moments to finish. During this time, the `STATUS` column reads `InProgress`. - -After a successful restore, the `STATUS` column is `Completed`, and `WARNINGS` and `ERRORS` are 0. All objects in the `nginx-example` namespace should be just as they were before you deleted them. - -If there are errors or warnings, you can look at them in detail: - -``` -ark restore describe -``` - -For more information, see [the debugging information][18]. - -### Clean up - -If you want to delete any backups you created, including data in object storage and persistent -volume snapshots, you can run: - -``` -ark backup delete BACKUP_NAME -``` - -This asks the Ark server to delete all backup data associated with `BACKUP_NAME`. You need to do -this for each backup you want to permanently delete. A future version of Ark will allow you to -delete multiple backups by name or label selector. - -Once fully removed, the backup is no longer visible when you run: - -``` -ark backup get BACKUP_NAME -``` - -If you want to uninstall Ark but preserve the backup data in object storage and persistent volume -snapshots, it is safe to remove the `heptio-ark` namespace and everything else created for this -example: - -``` -kubectl delete -f examples/common/ -kubectl delete -f examples/minio/ -kubectl delete -f examples/nginx-app/base.yaml -``` +[The documentation][29] provides a getting started guide, plus information about building from source, architecture, extending Ark, and more. ## Troubleshooting @@ -179,12 +31,12 @@ Thanks for taking the time to join our community and start contributing! Feedback and discussion is available on [the mailing list][24]. -#### Before you start +### Before you start * Please familiarize yourself with the [Code of Conduct][8] before contributing. * See [CONTRIBUTING.md][5] for instructions on the developer certificate of origin that we require. -#### Pull requests +### Pull requests * We welcome pull requests. Feel free to dig through the [issues][4] and jump in. @@ -195,30 +47,22 @@ See [the list of releases][6] to find out about feature changes. [0]: https://github.com/heptio [1]: https://travis-ci.org/heptio/ark.svg?branch=master [2]: https://travis-ci.org/heptio/ark -[3]: /docs/cloud-common.md + [4]: https://github.com/heptio/ark/issues [5]: https://github.com/heptio/ark/blob/master/CONTRIBUTING.md [6]: https://github.com/heptio/ark/releases -[7]: /docs/build-from-scratch.md + [8]: https://github.com/heptio/ark/blob/master/CODE_OF_CONDUCT.md [9]: https://kubernetes.io/docs/setup/ [10]: https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-with-homebrew-on-macos [11]: https://kubernetes.io/docs/tasks/tools/install-kubectl/#tabset-1 [12]: https://github.com/kubernetes/kubernetes/blob/master/cluster/addons/dns/README.md -[13]: /docs/output-file-format.md [14]: https://github.com/kubernetes/kubernetes -[15]: https://aws.amazon.com/ -[16]: https://cloud.google.com/ -[17]: https://azure.microsoft.com/ -[18]: /docs/debugging-restores.md -[19]: /docs/img/backup-process.png -[20]: https://kubernetes.io/docs/concepts/api-extension/custom-resources/#customresourcedefinitions -[21]: https://kubernetes.io/docs/concepts/api-extension/custom-resources/#custom-controllers -[22]: https://github.com/coreos/etcd + + [24]: http://j.hept.io/ark-list [25]: https://kubernetes.slack.com/messages/ark-dr -[26]: https://github.com/heptio/ark/releases -[27]: /docs/hooks.md -[28]: /docs/plugins.md + + [29]: https://heptio.github.io/ark/ [30]: /docs/troubleshooting.md diff --git a/docs/about.md b/docs/about.md index 969e49594..44a57ce21 100644 --- a/docs/about.md +++ b/docs/about.md @@ -1,22 +1,12 @@ -# About Heptio Ark +# How Ark Works -Heptio Ark provides customizable degrees of recovery for all Kubernetes objects (Pods, Deployments, Jobs, Custom Resource Definitions, etc.), as well as for persistent volumes. This recovery can be cluster-wide, or fine-tuned according to object type, namespace, or labels. +Each Ark operation -- on-demand backup, scheduled backup, restore -- is a custom resource, defined with a Kubernetes [Custom Resource Definition (CRD)][20] and stored in [etcd][22]. The config custom resource specifies core information and options such as cloud provider settings. Ark also includes controllers that process the custom resources to perform backups, restores, and all related operations. + +You can back up or restore all objects in your cluster, or you can filter objects by type, namespace, and/or label. Ark is ideal for the disaster recovery use case, as well as for snapshotting your application state, prior to performing system operations on your cluster (e.g. upgrades). -## Features - -Ark provides the following operations: - -* On-demand backups -* Scheduled backups -* Restores - -Each operation is a custom resource, defined with a Kubernetes [Custom Resource Definition (CRD)][20] and stored in [etcd][22]. An additional custom resource, Config, specifies required information and customized options, such as cloud provider settings. These resources are handled by [custom controllers][21] when their corresponding requests are submitted to the Kubernetes API server. - -Each controller watches its custom resource for API requests (Ark operations), performs validations, and handles the logic for interacting with the cloud provider API -- for example, managing object storage and persistent volumes. - -### On-demand backups +## On-demand backups The **backup** operation: @@ -29,25 +19,23 @@ need to tell a database to flush its in-memory buffers to disk before taking a s Note that cluster backups are not strictly atomic. If Kubernetes objects are being created or edited at the time of backup, they might not be included in the backup. The odds of capturing inconsistent information are low, but it is possible. -### Scheduled backups +## Scheduled backups The **schedule** operation allows you to back up your data at recurring intervals. The first backup is performed when the schedule is first created, and subsequent backups happen at the schedule's specified interval. These intervals are specified by a Cron expression. -A Schedule acts as a wrapper for Backups; when triggered, it creates them behind the scenes. - Scheduled backups are saved with the name `-`, where `` is formatted as *YYYYMMDDhhmmss*. -### Restores +## Restores -The **restore** operation allows you to restore all of the objects and persistent volumes from a previously created Backup. Heptio Ark supports multiple namespace remapping--for example, in a single restore, objects in namespace "abc" can be recreated under namespace "def", and the ones in "123" under "456". +The **restore** operation allows you to restore all of the objects and persistent volumes from a previously created backup. You can also restore only a filtered subset of objects and persistent volumes. Ark supports multiple namespace remapping--for example, in a single restore, objects in namespace "abc" can be recreated under namespace "def", and the objects in namespace "123" under "456". -Kubernetes objects that have been restored can be identified with a label that looks like `ark-restore=-`, where `` is formatted as *YYYYMMDDhhmmss*. +The default name of a restore is `-`, where `` is formatted as *YYYYMMDDhhmmss*. You can also specify a custom name. A restored object also includes a label with key `ark-restore` and value ``. You can also run the Ark server in restore-only mode, which disables backup, schedule, and garbage collection functionality during disaster recovery. ## Backup workflow -Here's what happens when you run `ark backup create test-backup`: +When you run `ark backup create test-backup`: 1. The Ark client makes a call to the Kubernetes API server to create a `Backup` object. @@ -57,24 +45,27 @@ Here's what happens when you run `ark backup create test-backup`: 1. The `BackupController` makes a call to the object storage service -- for example, AWS S3 -- to upload the backup file. -By default `ark backup create` makes disk snapshots of any persistent volumes. You can adjust the snapshots by specifying additional flags. See [the CLI help][30] for more information. Snapshots can be disabled with the option `--snapshot-volumes=false`. +By default, `ark backup create` makes disk snapshots of any persistent volumes. You can adjust the snapshots by specifying additional flags. See [the CLI help][30] for more information. Snapshots can be disabled with the option `--snapshot-volumes=false`. ![19] ## Set a backup to expire -When you create a backup, you can specify a TTL by adding the flag `--ttl `. If Ark sees that an existing Backup resource is expired, it removes: +When you create a backup, you can specify a TTL by adding the flag `--ttl `. If Ark sees that an existing backup resource is expired, it removes: -* The Backup resource +* The backup resource * The backup file from cloud object storage * All PersistentVolume snapshots * All associated Restores ## Object storage sync -Heptio Ark treats object storage as the source of truth. It continuously checks to see that the correct Backup resources are always present. If there is a properly formatted backup file in the storage bucket, but no corresponding Backup resources in the Kubernetes API, Ark synchronizes the information from object storage to Kubernetes. +Heptio Ark treats object storage as the source of truth. It continuously checks to see that the correct backup resources are always present. If there is a properly formatted backup file in the storage bucket, but no corresponding backup resource in the Kubernetes API, Ark synchronizes the information from object storage to Kubernetes. -This allows restore functionality to work in a cluster migration scenario, where the original Backup objects do not exist in the new cluster. See the tutorials for details. +This allows restore functionality to work in a cluster migration scenario, where the original backup objects do not exist in the new cluster. [19]: /img/backup-process.png +[20]: https://kubernetes.io/docs/concepts/api-extension/custom-resources/#customresourcedefinitions +[21]: https://kubernetes.io/docs/concepts/api-extension/custom-resources/#custom-controllers +[22]: https://github.com/coreos/etcd [30]: https://github.com/heptio/ark/blob/master/docs/cli-reference/ark_create_backup.md \ No newline at end of file diff --git a/docs/quickstart.md b/docs/quickstart.md new file mode 100644 index 000000000..f664059bd --- /dev/null +++ b/docs/quickstart.md @@ -0,0 +1,149 @@ +## Getting started + +The following example sets up the Ark server and client, then backs up and restores a sample application. + +For simplicity, the example uses Minio, an S3-compatible storage service that runs locally on your cluster. See [Set up Ark with your cloud provider][3] for how to run on a cloud provider. + +### Prerequisites + +* Access to a Kubernetes cluster, version 1.7 or later. Version 1.7.5 or later is required to run `ark backup delete`. +* A DNS server on the cluster +* `kubectl` installed + +### Download + +Clone or fork the Ark repository: + +``` +git clone git@github.com:heptio/ark.git +``` + +NOTE: Make sure to check out the appropriate version. We recommend that you check out the latest tagged version. The master branch is under active development and might not be stable. + +### Set up server + +1. Start the server and the local storage service. In the root directory of Ark, run: + + ```bash + kubectl apply -f examples/common/00-prereqs.yaml + kubectl apply -f examples/minio/ + ``` + + NOTE: If you get an error about Config creation, wait for a minute, then run the commands again. + +1. Deploy the example nginx application: + + ```bash + kubectl apply -f examples/nginx-app/base.yaml + ``` + +1. Check to see that both the Ark and nginx deployments are successfully created: + + ``` + kubectl get deployments -l component=ark --namespace=heptio-ark + kubectl get deployments --namespace=nginx-example + ``` + +### Install client + +[Download the client][26]. + +Make sure that you install somewhere in your PATH. + +### Back up + +1. Create a backup for any object that matches the `app=nginx` label selector: + + ``` + ark backup create nginx-backup --selector app=nginx + ``` + + Alternatively if you want to backup all objects *except* those matching the label `backup=ignore`: + + ``` + ark backup create nginx-backup --selector 'backup notin (ignore)' + ``` + +1. Simulate a disaster: + + ``` + kubectl delete namespace nginx-example + ``` + +1. To check that the nginx deployment and service are gone, run: + + ``` + kubectl get deployments --namespace=nginx-example + kubectl get services --namespace=nginx-example + kubectl get namespace/nginx-example + ``` + + You should get no results. + + NOTE: You might need to wait for a few minutes for the namespace to be fully cleaned up. + +### Restore + +1. Run: + + ``` + ark restore create --from-backup nginx-backup + ``` + +1. Run: + + ``` + ark restore get + ``` + + After the restore finishes, the output looks like the following: + + ``` + NAME BACKUP STATUS WARNINGS ERRORS CREATED SELECTOR + nginx-backup-20170727200524 nginx-backup Completed 0 0 2017-07-27 20:05:24 +0000 UTC + ``` + +NOTE: The restore can take a few moments to finish. During this time, the `STATUS` column reads `InProgress`. + +After a successful restore, the `STATUS` column is `Completed`, and `WARNINGS` and `ERRORS` are 0. All objects in the `nginx-example` namespace should be just as they were before you deleted them. + +If there are errors or warnings, you can look at them in detail: + +``` +ark restore describe +``` + +For more information, see [the debugging information][18]. + +### Clean up + +If you want to delete any backups you created, including data in object storage and persistent +volume snapshots, you can run: + +``` +ark backup delete BACKUP_NAME +``` + +This asks the Ark server to delete all backup data associated with `BACKUP_NAME`. You need to do +this for each backup you want to permanently delete. A future version of Ark will allow you to +delete multiple backups by name or label selector. + +Once fully removed, the backup is no longer visible when you run: + +``` +ark backup get BACKUP_NAME +``` + +If you want to uninstall Ark but preserve the backup data in object storage and persistent volume +snapshots, it is safe to remove the `heptio-ark` namespace and everything else created for this +example: + +``` +kubectl delete -f examples/common/ +kubectl delete -f examples/minio/ +kubectl delete -f examples/nginx-app/base.yaml +``` + +[3]: /docs/cloud-common.md +[18]: /docs/debugging-restores.md +[26]: https://github.com/heptio/ark/releases \ No newline at end of file diff --git a/docs/restic.md b/docs/restic.md index f3b54be76..cee91ee53 100644 --- a/docs/restic.md +++ b/docs/restic.md @@ -1,41 +1,43 @@ # Restic Integration -As of v0.9.0, Ark has support for backing up and restoring Kubernetes volumes using a free open-source backup tool called +As of version 0.9.0, Ark has support for backing up and restoring Kubernetes volumes using a free open-source backup tool called [restic][1]. -Ark has always allowed you to take snapshots of persistent volumes as part of your backups, provided you’re using one of +Ark has always allowed you to take snapshots of persistent volumes as part of your backups if you’re using one of the supported cloud providers’ block storage offerings (Amazon EBS Volumes, Azure Managed Disks, Google Persistent Disks). -And, since v0.6.0, we’ve had a plugin model which enables anyone to easily implement additional object and block storage -backends, outside of the main Ark repository. +Starting with version 0.6.0, we provide a plugin model that enables anyone to implement additional object and block storage +backends, outside the main Ark repository. We integrated restic with Ark so that users have an out-of-the-box solution for backing up and restoring almost any type of Kubernetes -volume*. We view this as a new capability for Ark, rather than a replacement for existing functionality. If you're running on AWS, and -taking EBS snapshots as part of your regular Ark backups, great! There's no need to switch to using restic. However, if you've -been waiting for a snapshot plugin to be implemented for your storage platform, or you're using EFS, AzureFile, NFS, emptyDir, -local, or any other volume type that doesn't have a native snapshot concept, restic may be for you. +volume*. This is a new capability for Ark, not a replacement for existing functionality. If you're running on AWS, and +taking EBS snapshots as part of your regular Ark backups, there's no need to switch to using restic. However, if you've +been waiting for a snapshot plugin for your storage platform, or if you're using EFS, AzureFile, NFS, emptyDir, +local, or any other volume type that doesn't have a native snapshot concept, restic might be for you. -Additionally, since restic is not tied to a specific storage platform, this integration paves the way for future work to enable +Restic is not tied to a specific storage platform, which means that this integration also paves the way for future work to enable cross-volume-type data migrations. Stay tuned as this evolves! -\* hostPath volumes are not supported, but the [new local volume type][4] is! +\* hostPath volumes are not supported, but the [new local volume type][4] is supported. ## Setup -This setup guide assumes you already have a working Ark v0.8.1+ installation. If not, go [here][2] for instructions. It -also assumes you have the [latest release tag][3] of the Ark repo cloned locally. +### Prerequisites -### Pre-Install Steps - Upgrading from a Previous v0.9 Alpha +- A working install of Ark version 0.8.1 or later. See [Set up Ark][2] +- A local clone of [the latest release tag of the Ark repository][3] -If you're upgrading from a previous pre-release version of Ark with restic, before you proceed you'll need to: -- manually delete all of the repositories/data from your existing restic bucket -- delete all Ark backups from your cluster using `ark backup delete` -- delete all secrets named `ark-restic-credentials` across all namespaces in your cluster +#### Additional steps if upgrading from version 0.9 alpha + +- Manually delete all of the repositories/data from your existing restic bucket +- Delete all Ark backups from your cluster using `ark backup delete` +- Delete all secrets named `ark-restic-credentials` across all namespaces in your cluster ### Instructions -1. Download an updated Ark client from the [latest release][3], and move it to a location within your PATH. +1. Download an updated Ark client from the [latest release][3], and move it to a location in your PATH. 1. From the Ark root directory, run the following to create new custom resource definitions: + ```bash kubectl apply -f examples/common/00-prereqs.yaml ``` @@ -47,15 +49,16 @@ If you're upgrading from a previous pre-release version of Ark with restic, befo - GCP: `kubectl apply -f examples/gcp/20-restic-daemonset.yaml` - Minio: `kubectl apply -f examples/minio/30-restic-daemonset.yaml` -1. Update the image tag on the Ark daemonset and deployment to match the release version you used in Step 1 (e.g. `v0.9.0-alpha.3`): +1. Update the image tag on the Ark daemonset and deployment to match the release version you're working with -- for example, `v0.9.0`: + ```bash kubectl -n heptio-ark set image deployment/ark ark=gcr.io/heptio-images/ark: kubectl -n heptio-ark set image daemonset/restic ark=gcr.io/heptio-images/ark: ``` 1. Create a new bucket for restic to store its data in, and give the `heptio-ark` IAM user access to it, similarly to -the main Ark bucket you've already set up. Note that for now, this must be a different bucket than the main Ark bucket, -but we plan to remove this limitation in a future release. +the main Ark bucket you've already set up. Note that this must be a different bucket than the main Ark bucket. +We plan to remove this limitation in a future release. 1. Uncomment `resticLocation` in your Ark config and set the value appropriately, then apply: @@ -69,14 +72,15 @@ but we plan to remove this limitation in a future release. You're now ready to use Ark with restic. -## Backing Up +## Back up + +1. Run the following for each pod that contains a volume to back up: -1. Run the following for each pod containing a volume that you'd like to back up using restic: ```bash kubectl -n YOUR_POD_NAMESPACE annotate pod/YOUR_POD_NAME backup.ark.heptio.com/backup-volumes=YOUR_VOLUME_NAME_1,YOUR_VOLUME_NAME_2,... ``` - The volume names specified in the annotation should be the names of the volumes within the pod spec. + where the volume names are the names of the volumes in the pod spec. For example, for the following pod: @@ -108,21 +112,21 @@ You're now ready to use Ark with restic. kubectl -n foo annotate pod/sample backup.ark.heptio.com/backup-volumes=pvc-volume,emptydir-volume ``` + This annotation can also be provided in a pod template spec if you use a controller to manage your pods. - This annotation can also be provided in the pod template spec if using a deployment, daemonset, etc. - to manage your pods. +1. Take an Ark backup: -1. Take an Ark backup as usual: ```bash ark backup create NAME OPTIONS... ``` -1. When the backup has completed, view information about your pod volume backups: +1. When the backup completes, view information about the backups: + ```bash kubectl -n heptio-ark get podvolumebackups -l ark.heptio.com/backup-name=YOUR_BACKUP_NAME -o yaml ``` -## Restoring +## Restore Restore from your Ark backup as usual: @@ -132,28 +136,27 @@ ark restore create --from-backup BACKUP_NAME OPTIONS... ## Limitations -There are several limitations that users should be aware of: - -- As mentioned previously, you cannot use the main Ark bucket for storing restic backups. We plan to address this +- You cannot use the main Ark bucket for storing restic backups. We plan to address this issue in a future release. -- `hostPath` volumes are not supported. [Local persistent volumes][4] are, though! +- `hostPath` volumes are not supported. [Local persistent volumes][4] are supported. - Those of you familiar with [restic][1] may know that it encrypts all of its data. We've decided to use a static, common encryption key for all restic repositories created by Ark. **This means that anyone who has access to your -bucket can decrypt your restic backup data**. You should ensure that you are limiting access to the restic bucket +bucket can decrypt your restic backup data**. Make sure that you limit access to the restic bucket appropriately. We plan to implement full Ark backup encryption, including securing the restic encryption keys, in a future release. ## Troubleshooting -If something's not working as expected, there are several places you can go to for more information on what might -be happening. +Run the following checks: Are your Ark server and daemonset pods running? + ```bash kubectl get pods -n heptio-ark ``` Does your restic repository exist, and is it ready? + ```bash ark restic repo get @@ -161,6 +164,7 @@ ark restic repo get REPO_NAME -o yaml ``` Are there any errors in your Ark backup/restore? + ```bash ark backup describe BACKUP_NAME ark backup logs BACKUP_NAME @@ -170,6 +174,7 @@ ark restore logs RESTORE_NAME ``` What is the status of your pod volume backups/restores? + ```bash kubectl -n heptio-ark get podvolumebackups -l ark.heptio.com/backup-name=BACKUP_NAME -o yaml @@ -177,25 +182,25 @@ kubectl -n heptio-ark get podvolumerestores -l ark.heptio.com/restore-name=RESTO ``` Is there any useful information in the Ark server or daemon pod logs? + ```bash kubectl -n heptio-ark logs deploy/ark kubectl -n heptio-ark logs DAEMON_POD_NAME ``` + **NOTE**: You can increase the verbosity of the pod logs by adding `--log-level=debug` as an argument to the container command in the deployment/daemonset pod template spec. -## Details of Backup & Restore Process with Restic +## How backup and restore work with restic -This section describes in detail how backups/restores with Ark and restic work. - -As background, we've introduced three new custom resource definitions and associated controllers: +We introduced three custom resource definitions and associated controllers: - `ResticRepository` - represents/manages the lifecycle of Ark's [restic repositories][5]. Ark creates -a restic repository per namespace, when the first restic backup for a namespace is requested. The controller +a restic repository per namespace when the first restic backup for a namespace is requested. The controller for this custom resource executes restic repository lifecycle commands -- `restic init`, `restic check`, and `restic prune`. - You can see information about your Ark restic repos with `ark restic repo get`. + You can see information about your Ark restic repositories by running `ark restic repo get`. - `PodVolumeBackup` - represents a restic backup of a volume in a pod. The main Ark backup process creates one or more of these when it finds an annotated pod. Each node in the cluster runs a controller for this @@ -207,7 +212,7 @@ or more of these when it encounters a pod that has associated restic backups. Ea controller for this resource (in the same daemonset as above) that handles the `PodVolumeRestores` for pods on that node. The controller executes `restic restore` commands to restore pod volume data. -### Backups +### Backup 1. The main Ark backup process checks each pod that it's backing up for the annotation specifying a restic backup should be taken (`backup.ark.heptio.com/backup-volumes`) @@ -224,8 +229,7 @@ should be taken (`backup.ark.heptio.com/backup-volumes`) 1. As each `PodVolumeBackup` finishes, the main Ark process captures its restic snapshot ID and adds it as an annotation to the copy of the pod JSON that's stored in the Ark backup. This will be used for restores, as seen in the next section. - -### Restores +### Restore 1. The main Ark restore process checks each pod that it's restoring for annotations specifying a restic backup exists for a volume in the pod (`snapshot.ark.heptio.com/`) @@ -254,7 +258,7 @@ on to running other init containers/the main containers. [1]: https://github.com/restic/restic -[2]: https://heptio.github.io/ark/v0.8.1/cloud-common +[2]: cloud-common.md [3]: https://github.com/heptio/ark/releases/ [4]: https://kubernetes.io/docs/concepts/storage/volumes/#local [5]: http://restic.readthedocs.io/en/latest/100_references.html#terminology \ No newline at end of file