mirror of
https://github.com/vmware-tanzu/velero.git
synced 2026-09-05 07:37:07 +00:00
Initial commit
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
This commit is contained in:
Andy Goldstein
@@ -0,0 +1,29 @@
|
||||
# Table of Contents
|
||||
|
||||
## User Guide
|
||||
|
||||
* [Concepts][1]
|
||||
* [Build from scratch][0]
|
||||
* [Cloud provider specifics][9]
|
||||
* [Debugging restores][4]
|
||||
|
||||
## Reference
|
||||
* [CLI reference][2]
|
||||
* [Config definition][5]
|
||||
* [Output file format][6]
|
||||
* [Sample YAML files][3]
|
||||
|
||||
## Scenarios
|
||||
* [Disaster recovery][7]
|
||||
* [Cluster migration][8]
|
||||
|
||||
[0]: build-from-scratch.md
|
||||
[1]: concepts.md
|
||||
[2]: cli-reference
|
||||
[3]: /examples
|
||||
[4]: debugging-restores.md
|
||||
[5]: config-definition.md
|
||||
[6]: output-file-format.md
|
||||
[7]: use-cases.md#disaster-recovery
|
||||
[8]: use-cases.md#cluster-migration
|
||||
[9]: cloud-provider-specifics.md
|
||||
@@ -0,0 +1,56 @@
|
||||
# Build From Scratch
|
||||
|
||||
While the [README][0] pulls from the Heptio image registry, you can also build your own Heptio Ark container with the following steps:
|
||||
|
||||
* [0. Prerequisites][1]
|
||||
* [1. Download][2]
|
||||
* [2. Build][3]
|
||||
* [3. Run][7]
|
||||
|
||||
## 0. Prerequisites
|
||||
|
||||
In addition to the handling the prerequisites mentioned in the [Quickstart][4], you should have [Go][5] installed (minimum version 1.8).
|
||||
|
||||
## 1. Download
|
||||
|
||||
Install with go:
|
||||
```
|
||||
go get github.com/heptio/ark
|
||||
```
|
||||
The files are installed in `$GOPATH/src/github.com/heptio/ark`.
|
||||
|
||||
## 2. Build
|
||||
|
||||
Set the `$REGISTRY` environment variable (used in the `Makefile`) if you want to push the Heptio Ark images to your own registry. This allows any node in your cluster to pull your locally built image.
|
||||
|
||||
`$PROJECT` and `$VERSION` environment variables are also specified in the `Makefile`, and can be similarly modified as desired.
|
||||
|
||||
Run the following in the Ark root directory to build your container with the tag `$REGISTRY/$PROJECT:$VERSION`:
|
||||
```
|
||||
sudo make all
|
||||
```
|
||||
|
||||
To push your image to a registry, use `make push`.
|
||||
|
||||
## 3. Run
|
||||
When running Heptio Ark, you will need to account for the following (all of which are handled in the [`/examples`][6] manifests):
|
||||
* Appropriate RBAC permissions in the cluster
|
||||
* *Read access* for all data from the source cluster and namespaces
|
||||
* *Write access* to the target cluster and namespaces
|
||||
* Cloud provider credentials
|
||||
* *Read/write access* to volumes
|
||||
* *Read/write access* to object storage for backup data
|
||||
* A [Config object][8] definition for the Ark server
|
||||
|
||||
See [Cloud Provider Specifics][9] for a more detailed guide.
|
||||
|
||||
[0]: ../README.md
|
||||
[1]: #0-prerequisites
|
||||
[2]: #1-download
|
||||
[3]: #2-build
|
||||
[4]: ../README.md#quickstart
|
||||
[5]: https://golang.org/doc/install
|
||||
[6]: /examples
|
||||
[7]: #3-run
|
||||
[8]: reference.md#ark-config-definition
|
||||
[9]: cloud-provider-specifics.md
|
||||
@@ -0,0 +1,21 @@
|
||||
# Command line reference
|
||||
|
||||
The Ark client provides a CLI that allows you to initiate ad-hoc backups, scheduled backups, or restores.
|
||||
|
||||
*The files in this directory enumerate each of the possible `ark` commands and their flags. Note that you can also find this info with the CLI itself, using the `--help` flag.*
|
||||
|
||||
## Running the client
|
||||
|
||||
While it is possible to build and run the `ark` executable yourself, it is recommended to use the containerized version. Use the alias described in the quickstart:
|
||||
|
||||
```
|
||||
alias ark='docker run --rm -v $(dirname $KUBECONFIG):/kubeconfig -e KUBECONFIG=/kubeconfig/$(basename $KUBECONFIG) gcr.io/heptio-images/ark:latest'
|
||||
```
|
||||
|
||||
Assuming that your `KUBECONFIG` variable is set, this alias takes care of specifying the appropriate Kubernetes cluster credentials for you.
|
||||
|
||||
## Kubernetes cluster credentials
|
||||
In general, Ark will search for your cluster credentials in the following order:
|
||||
* `--kubeconfig` command line flag
|
||||
* `$KUBECONFIG` environment variable
|
||||
* In-cluster credentials--this only works when you are running Ark in a pod
|
||||
@@ -0,0 +1,32 @@
|
||||
## ark
|
||||
|
||||
Back up and restore Kubernetes cluster resources.
|
||||
|
||||
### Synopsis
|
||||
|
||||
|
||||
Heptio Ark is a tool for managing disaster recovery, specifically for
|
||||
Kubernetes cluster resources. It provides a simple, configurable,
|
||||
and operationally robust way to back up your application state and
|
||||
associated data.
|
||||
|
||||
### Options
|
||||
|
||||
```
|
||||
--alsologtostderr log to standard error as well as files
|
||||
--kubeconfig string Path to the kubeconfig file to use to talk to the Kubernetes apiserver. If unset, try the environment variable KUBECONFIG, as well as in-cluster configuration
|
||||
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
|
||||
--log_dir string If non-empty, write log files in this directory
|
||||
--logtostderr log to standard error instead of files
|
||||
--stderrthreshold severity logs at or above this threshold go to stderr (default 2)
|
||||
-v, --v Level log level for V logs
|
||||
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
|
||||
```
|
||||
|
||||
### SEE ALSO
|
||||
* [ark backup](ark_backup.md) - Work with backups
|
||||
* [ark restore](ark_restore.md) - Work with restores
|
||||
* [ark schedule](ark_schedule.md) - Work with schedules
|
||||
* [ark server](ark_server.md) - Run the ark server
|
||||
* [ark version](ark_version.md) - Print the ark version and associated image
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
## ark backup
|
||||
|
||||
Work with backups
|
||||
|
||||
### Synopsis
|
||||
|
||||
|
||||
Work with backups
|
||||
|
||||
### Options inherited from parent commands
|
||||
|
||||
```
|
||||
--alsologtostderr log to standard error as well as files
|
||||
--kubeconfig string Path to the kubeconfig file to use to talk to the Kubernetes apiserver. If unset, try the environment variable KUBECONFIG, as well as in-cluster configuration
|
||||
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
|
||||
--log_dir string If non-empty, write log files in this directory
|
||||
--logtostderr log to standard error instead of files
|
||||
--stderrthreshold severity logs at or above this threshold go to stderr (default 2)
|
||||
-v, --v Level log level for V logs
|
||||
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
|
||||
```
|
||||
|
||||
### SEE ALSO
|
||||
* [ark](ark.md) - Back up and restore Kubernetes cluster resources.
|
||||
* [ark backup create](ark_backup_create.md) - Create a backup
|
||||
* [ark backup get](ark_backup_get.md) - Get backups
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
## ark backup create
|
||||
|
||||
Create a backup
|
||||
|
||||
### Synopsis
|
||||
|
||||
|
||||
Create a backup
|
||||
|
||||
```
|
||||
ark backup create NAME
|
||||
```
|
||||
|
||||
### Options
|
||||
|
||||
```
|
||||
--exclude-namespaces stringArray namespaces to exclude from the backup
|
||||
--exclude-resources stringArray resources to exclude from the backup, formatted as resource.group, such as storageclasses.storage.k8s.io
|
||||
--include-namespaces stringArray namespaces to include in the backup (use '*' for all namespaces) (default *)
|
||||
--include-resources stringArray resources to include in the backup, formatted as resource.group, such as storageclasses.storage.k8s.io (use '*' for all resources)
|
||||
--label-columns stringArray a comma-separated list of labels to be displayed as columns
|
||||
--labels mapStringString labels to apply to the backup
|
||||
-o, --output string Output display format. For create commands, display the object but do not send it to the server. Valid formats are 'table', 'json', and 'yaml'.
|
||||
-l, --selector labelSelector only back up resources matching this label selector (default <none>)
|
||||
--show-labels show labels in the last column
|
||||
--snapshot-volumes take snapshots of PersistentVolumes as part of the backup
|
||||
--ttl duration how long before the backup can be garbage collected (default 24h0m0s)
|
||||
```
|
||||
|
||||
### Options inherited from parent commands
|
||||
|
||||
```
|
||||
--alsologtostderr log to standard error as well as files
|
||||
--kubeconfig string Path to the kubeconfig file to use to talk to the Kubernetes apiserver. If unset, try the environment variable KUBECONFIG, as well as in-cluster configuration
|
||||
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
|
||||
--log_dir string If non-empty, write log files in this directory
|
||||
--logtostderr log to standard error instead of files
|
||||
--stderrthreshold severity logs at or above this threshold go to stderr (default 2)
|
||||
-v, --v Level log level for V logs
|
||||
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
|
||||
```
|
||||
|
||||
### SEE ALSO
|
||||
* [ark backup](ark_backup.md) - Work with backups
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
## ark backup get
|
||||
|
||||
Get backups
|
||||
|
||||
### Synopsis
|
||||
|
||||
|
||||
Get backups
|
||||
|
||||
```
|
||||
ark backup get
|
||||
```
|
||||
|
||||
### Options
|
||||
|
||||
```
|
||||
--label-columns stringArray a comma-separated list of labels to be displayed as columns
|
||||
-o, --output string Output display format. For create commands, display the object but do not send it to the server. Valid formats are 'table', 'json', and 'yaml'. (default "table")
|
||||
-l, --selector string only show items matching this label selector
|
||||
--show-labels show labels in the last column
|
||||
```
|
||||
|
||||
### Options inherited from parent commands
|
||||
|
||||
```
|
||||
--alsologtostderr log to standard error as well as files
|
||||
--kubeconfig string Path to the kubeconfig file to use to talk to the Kubernetes apiserver. If unset, try the environment variable KUBECONFIG, as well as in-cluster configuration
|
||||
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
|
||||
--log_dir string If non-empty, write log files in this directory
|
||||
--logtostderr log to standard error instead of files
|
||||
--stderrthreshold severity logs at or above this threshold go to stderr (default 2)
|
||||
-v, --v Level log level for V logs
|
||||
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
|
||||
```
|
||||
|
||||
### SEE ALSO
|
||||
* [ark backup](ark_backup.md) - Work with backups
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
## ark restore
|
||||
|
||||
Work with restores
|
||||
|
||||
### Synopsis
|
||||
|
||||
|
||||
Work with restores
|
||||
|
||||
### Options inherited from parent commands
|
||||
|
||||
```
|
||||
--alsologtostderr log to standard error as well as files
|
||||
--kubeconfig string Path to the kubeconfig file to use to talk to the Kubernetes apiserver. If unset, try the environment variable KUBECONFIG, as well as in-cluster configuration
|
||||
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
|
||||
--log_dir string If non-empty, write log files in this directory
|
||||
--logtostderr log to standard error instead of files
|
||||
--stderrthreshold severity logs at or above this threshold go to stderr (default 2)
|
||||
-v, --v Level log level for V logs
|
||||
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
|
||||
```
|
||||
|
||||
### SEE ALSO
|
||||
* [ark](ark.md) - Back up and restore Kubernetes cluster resources.
|
||||
* [ark restore create](ark_restore_create.md) - Create a restore
|
||||
* [ark restore delete](ark_restore_delete.md) - Delete a restore
|
||||
* [ark restore get](ark_restore_get.md) - get restores
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
## ark restore create
|
||||
|
||||
Create a restore
|
||||
|
||||
### Synopsis
|
||||
|
||||
|
||||
Create a restore
|
||||
|
||||
```
|
||||
ark restore create BACKUP
|
||||
```
|
||||
|
||||
### Options
|
||||
|
||||
```
|
||||
--label-columns stringArray a comma-separated list of labels to be displayed as columns
|
||||
--labels mapStringString labels to apply to the restore
|
||||
--namespace-mappings mapStringString namespace mappings from name in the backup to desired restored name in the form src1:dst1,src2:dst2,...
|
||||
--namespaces stringArray comma-separated list of namespaces to restore
|
||||
-o, --output string Output display format. For create commands, display the object but do not send it to the server. Valid formats are 'table', 'json', and 'yaml'.
|
||||
--restore-volumes whether to restore volumes from snapshots
|
||||
-l, --selector labelSelector only restore resources matching this label selector (default <none>)
|
||||
--show-labels show labels in the last column
|
||||
```
|
||||
|
||||
### Options inherited from parent commands
|
||||
|
||||
```
|
||||
--alsologtostderr log to standard error as well as files
|
||||
--kubeconfig string Path to the kubeconfig file to use to talk to the Kubernetes apiserver. If unset, try the environment variable KUBECONFIG, as well as in-cluster configuration
|
||||
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
|
||||
--log_dir string If non-empty, write log files in this directory
|
||||
--logtostderr log to standard error instead of files
|
||||
--stderrthreshold severity logs at or above this threshold go to stderr (default 2)
|
||||
-v, --v Level log level for V logs
|
||||
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
|
||||
```
|
||||
|
||||
### SEE ALSO
|
||||
* [ark restore](ark_restore.md) - Work with restores
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
## ark restore delete
|
||||
|
||||
Delete a restore
|
||||
|
||||
### Synopsis
|
||||
|
||||
|
||||
Delete a restore
|
||||
|
||||
```
|
||||
ark restore delete NAME
|
||||
```
|
||||
|
||||
### Options inherited from parent commands
|
||||
|
||||
```
|
||||
--alsologtostderr log to standard error as well as files
|
||||
--kubeconfig string Path to the kubeconfig file to use to talk to the Kubernetes apiserver. If unset, try the environment variable KUBECONFIG, as well as in-cluster configuration
|
||||
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
|
||||
--log_dir string If non-empty, write log files in this directory
|
||||
--logtostderr log to standard error instead of files
|
||||
--stderrthreshold severity logs at or above this threshold go to stderr (default 2)
|
||||
-v, --v Level log level for V logs
|
||||
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
|
||||
```
|
||||
|
||||
### SEE ALSO
|
||||
* [ark restore](ark_restore.md) - Work with restores
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
## ark restore get
|
||||
|
||||
get restores
|
||||
|
||||
### Synopsis
|
||||
|
||||
|
||||
get restores
|
||||
|
||||
```
|
||||
ark restore get
|
||||
```
|
||||
|
||||
### Options
|
||||
|
||||
```
|
||||
--label-columns stringArray a comma-separated list of labels to be displayed as columns
|
||||
-o, --output string Output display format. For create commands, display the object but do not send it to the server. Valid formats are 'table', 'json', and 'yaml'. (default "table")
|
||||
-l, --selector string only show items matching this label selector
|
||||
--show-labels show labels in the last column
|
||||
```
|
||||
|
||||
### Options inherited from parent commands
|
||||
|
||||
```
|
||||
--alsologtostderr log to standard error as well as files
|
||||
--kubeconfig string Path to the kubeconfig file to use to talk to the Kubernetes apiserver. If unset, try the environment variable KUBECONFIG, as well as in-cluster configuration
|
||||
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
|
||||
--log_dir string If non-empty, write log files in this directory
|
||||
--logtostderr log to standard error instead of files
|
||||
--stderrthreshold severity logs at or above this threshold go to stderr (default 2)
|
||||
-v, --v Level log level for V logs
|
||||
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
|
||||
```
|
||||
|
||||
### SEE ALSO
|
||||
* [ark restore](ark_restore.md) - Work with restores
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
## ark schedule
|
||||
|
||||
Work with schedules
|
||||
|
||||
### Synopsis
|
||||
|
||||
|
||||
Work with schedules
|
||||
|
||||
### Options inherited from parent commands
|
||||
|
||||
```
|
||||
--alsologtostderr log to standard error as well as files
|
||||
--kubeconfig string Path to the kubeconfig file to use to talk to the Kubernetes apiserver. If unset, try the environment variable KUBECONFIG, as well as in-cluster configuration
|
||||
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
|
||||
--log_dir string If non-empty, write log files in this directory
|
||||
--logtostderr log to standard error instead of files
|
||||
--stderrthreshold severity logs at or above this threshold go to stderr (default 2)
|
||||
-v, --v Level log level for V logs
|
||||
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
|
||||
```
|
||||
|
||||
### SEE ALSO
|
||||
* [ark](ark.md) - Back up and restore Kubernetes cluster resources.
|
||||
* [ark schedule create](ark_schedule_create.md) - Create a schedule
|
||||
* [ark schedule delete](ark_schedule_delete.md) - Delete a schedule
|
||||
* [ark schedule get](ark_schedule_get.md) - Get schedules
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
## ark schedule create
|
||||
|
||||
Create a schedule
|
||||
|
||||
### Synopsis
|
||||
|
||||
|
||||
Create a schedule
|
||||
|
||||
```
|
||||
ark schedule create NAME
|
||||
```
|
||||
|
||||
### Options
|
||||
|
||||
```
|
||||
--exclude-namespaces stringArray namespaces to exclude from the backup
|
||||
--exclude-resources stringArray resources to exclude from the backup, formatted as resource.group, such as storageclasses.storage.k8s.io
|
||||
--include-namespaces stringArray namespaces to include in the backup (use '*' for all namespaces) (default *)
|
||||
--include-resources stringArray resources to include in the backup, formatted as resource.group, such as storageclasses.storage.k8s.io (use '*' for all resources)
|
||||
--label-columns stringArray a comma-separated list of labels to be displayed as columns
|
||||
--labels mapStringString labels to apply to the backup
|
||||
-o, --output string Output display format. For create commands, display the object but do not send it to the server. Valid formats are 'table', 'json', and 'yaml'.
|
||||
--schedule string a cron expression specifying a recurring schedule for this backup to run
|
||||
-l, --selector labelSelector only back up resources matching this label selector (default <none>)
|
||||
--show-labels show labels in the last column
|
||||
--snapshot-volumes take snapshots of PersistentVolumes as part of the backup
|
||||
--ttl duration how long before the backup can be garbage collected (default 24h0m0s)
|
||||
```
|
||||
|
||||
### Options inherited from parent commands
|
||||
|
||||
```
|
||||
--alsologtostderr log to standard error as well as files
|
||||
--kubeconfig string Path to the kubeconfig file to use to talk to the Kubernetes apiserver. If unset, try the environment variable KUBECONFIG, as well as in-cluster configuration
|
||||
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
|
||||
--log_dir string If non-empty, write log files in this directory
|
||||
--logtostderr log to standard error instead of files
|
||||
--stderrthreshold severity logs at or above this threshold go to stderr (default 2)
|
||||
-v, --v Level log level for V logs
|
||||
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
|
||||
```
|
||||
|
||||
### SEE ALSO
|
||||
* [ark schedule](ark_schedule.md) - Work with schedules
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
## ark schedule delete
|
||||
|
||||
Delete a schedule
|
||||
|
||||
### Synopsis
|
||||
|
||||
|
||||
Delete a schedule
|
||||
|
||||
```
|
||||
ark schedule delete NAME
|
||||
```
|
||||
|
||||
### Options inherited from parent commands
|
||||
|
||||
```
|
||||
--alsologtostderr log to standard error as well as files
|
||||
--kubeconfig string Path to the kubeconfig file to use to talk to the Kubernetes apiserver. If unset, try the environment variable KUBECONFIG, as well as in-cluster configuration
|
||||
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
|
||||
--log_dir string If non-empty, write log files in this directory
|
||||
--logtostderr log to standard error instead of files
|
||||
--stderrthreshold severity logs at or above this threshold go to stderr (default 2)
|
||||
-v, --v Level log level for V logs
|
||||
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
|
||||
```
|
||||
|
||||
### SEE ALSO
|
||||
* [ark schedule](ark_schedule.md) - Work with schedules
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
## ark schedule get
|
||||
|
||||
Get schedules
|
||||
|
||||
### Synopsis
|
||||
|
||||
|
||||
Get schedules
|
||||
|
||||
```
|
||||
ark schedule get
|
||||
```
|
||||
|
||||
### Options
|
||||
|
||||
```
|
||||
--label-columns stringArray a comma-separated list of labels to be displayed as columns
|
||||
-o, --output string Output display format. For create commands, display the object but do not send it to the server. Valid formats are 'table', 'json', and 'yaml'. (default "table")
|
||||
-l, --selector string only show items matching this label selector
|
||||
--show-labels show labels in the last column
|
||||
```
|
||||
|
||||
### Options inherited from parent commands
|
||||
|
||||
```
|
||||
--alsologtostderr log to standard error as well as files
|
||||
--kubeconfig string Path to the kubeconfig file to use to talk to the Kubernetes apiserver. If unset, try the environment variable KUBECONFIG, as well as in-cluster configuration
|
||||
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
|
||||
--log_dir string If non-empty, write log files in this directory
|
||||
--logtostderr log to standard error instead of files
|
||||
--stderrthreshold severity logs at or above this threshold go to stderr (default 2)
|
||||
-v, --v Level log level for V logs
|
||||
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
|
||||
```
|
||||
|
||||
### SEE ALSO
|
||||
* [ark schedule](ark_schedule.md) - Work with schedules
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
## ark server
|
||||
|
||||
Run the ark server
|
||||
|
||||
### Synopsis
|
||||
|
||||
|
||||
Run the ark server
|
||||
|
||||
```
|
||||
ark server
|
||||
```
|
||||
|
||||
### Options
|
||||
|
||||
```
|
||||
--kubeconfig string Path to the kubeconfig file to use to talk to the Kubernetes apiserver. If unset, try the environment variable KUBECONFIG, as well as in-cluster configuration
|
||||
```
|
||||
|
||||
### Options inherited from parent commands
|
||||
|
||||
```
|
||||
--alsologtostderr log to standard error as well as files
|
||||
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
|
||||
--log_dir string If non-empty, write log files in this directory
|
||||
--logtostderr log to standard error instead of files
|
||||
--stderrthreshold severity logs at or above this threshold go to stderr (default 2)
|
||||
-v, --v Level log level for V logs
|
||||
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
|
||||
```
|
||||
|
||||
### SEE ALSO
|
||||
* [ark](ark.md) - Back up and restore Kubernetes cluster resources.
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
## ark version
|
||||
|
||||
Print the ark version and associated image
|
||||
|
||||
### Synopsis
|
||||
|
||||
|
||||
Print the ark version and associated image
|
||||
|
||||
```
|
||||
ark version
|
||||
```
|
||||
|
||||
### Options inherited from parent commands
|
||||
|
||||
```
|
||||
--alsologtostderr log to standard error as well as files
|
||||
--kubeconfig string Path to the kubeconfig file to use to talk to the Kubernetes apiserver. If unset, try the environment variable KUBECONFIG, as well as in-cluster configuration
|
||||
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
|
||||
--log_dir string If non-empty, write log files in this directory
|
||||
--logtostderr log to standard error instead of files
|
||||
--stderrthreshold severity logs at or above this threshold go to stderr (default 2)
|
||||
-v, --v Level log level for V logs
|
||||
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
|
||||
```
|
||||
|
||||
### SEE ALSO
|
||||
* [ark](ark.md) - Back up and restore Kubernetes cluster resources.
|
||||
|
||||
@@ -0,0 +1,358 @@
|
||||
# Cloud Provider Specifics
|
||||
|
||||
While the [Quickstart][0] uses a local storage service to quickly set up Heptio Ark as a demonstration, this document details additional configurations that are required when integrating with the cloud providers below:
|
||||
|
||||
* [Setup][12]
|
||||
* [AWS][1]
|
||||
* [GCP][2]
|
||||
* [Azure][3]
|
||||
* [Run][13]
|
||||
* [Ark server][9]
|
||||
* [Basic example (no PVs)][10]
|
||||
* [Snapshot example (with PVs)][11]
|
||||
|
||||
|
||||
## Setup
|
||||
### AWS
|
||||
|
||||
#### IAM user creation
|
||||
|
||||
To integrate Heptio Ark with AWS, you should follow the instructions below to create an Ark-specific [IAM user][14].
|
||||
|
||||
1. If you do not have the AWS CLI locally installed, follow the [user guide][5] to set it up.
|
||||
|
||||
2. Create an IAM user:
|
||||
|
||||
```
|
||||
aws iam create-user --user-name heptio-ark
|
||||
```
|
||||
|
||||
3. Attach a policy to give `heptio-ark` the necessary permissions:
|
||||
|
||||
```
|
||||
aws iam attach-user-policy \
|
||||
--policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess \
|
||||
--user-name heptio-ark
|
||||
aws iam attach-user-policy \
|
||||
--policy-arn arn:aws:iam::aws:policy/AmazonEC2FullAccess \
|
||||
--user-name heptio-ark
|
||||
```
|
||||
|
||||
4. Create an access key for the user:
|
||||
|
||||
```
|
||||
aws iam create-access-key --user-name heptio-ark
|
||||
```
|
||||
|
||||
The result should look like:
|
||||
|
||||
```
|
||||
{
|
||||
"AccessKey": {
|
||||
"UserName": "heptio-ark",
|
||||
"Status": "Active",
|
||||
"CreateDate": "2017-07-31T22:24:41.576Z",
|
||||
"SecretAccessKey": <AWS_SECRET_ACCESS_KEY>,
|
||||
"AccessKeyId": <AWS_ACCESS_KEY_ID>
|
||||
}
|
||||
}
|
||||
```
|
||||
5. Using the output from the previous command, create an Ark-specific credentials file (`credentials-ark`) in your local directory that looks like the following:
|
||||
|
||||
```
|
||||
[default]
|
||||
aws_access_key_id=<AWS_ACCESS_KEY_ID>
|
||||
aws_secret_access_key=<AWS_SECRET_ACCESS_KEY>
|
||||
```
|
||||
|
||||
|
||||
#### Credentials and configuration
|
||||
|
||||
In the Ark root directory, run the following to first set up namespaces, RBAC, and other scaffolding:
|
||||
```
|
||||
kubectl apply -f examples/common/00-prereqs.yaml
|
||||
```
|
||||
|
||||
Create a Secret, running this command in the local directory of the credentials file you just created:
|
||||
|
||||
```
|
||||
kubectl create secret generic cloud-credentials \
|
||||
--namespace heptio-ark \
|
||||
--from-file cloud=credentials-ark
|
||||
```
|
||||
|
||||
Now that you have your IAM user credentials stored in a Secret, you need to replace some placeholder values in the template files. Specifically, you need to change the following:
|
||||
|
||||
* In file `examples/aws/00-ark-config.yaml`:
|
||||
|
||||
* Replace `<YOUR_BUCKET>`, `<YOUR_REGION>`, and `<YOUR_AVAILABILITY_ZONE>`. See the [Config definition][6] for details.
|
||||
|
||||
|
||||
* In file `examples/common/10-deployment.yaml`:
|
||||
|
||||
* Make sure that `spec.template.spec.containers[*].env.name` is "AWS_SHARED_CREDENTIALS_FILE".
|
||||
|
||||
|
||||
* (Optional) If you are running the Nginx example, in file `examples/nginx-app/with-pv.yaml`:
|
||||
|
||||
* Replace `<YOUR_STORAGE_CLASS_NAME>` with `gp2`. This is AWS's default `StorageClass` name.
|
||||
|
||||
|
||||
### GCP
|
||||
|
||||
#### Service account creation
|
||||
|
||||
To integrate Heptio Ark with GCP, you should follow the instructions below to create an Ark-specific [Service Account][15].
|
||||
|
||||
1. If you do not have the gcloud CLI locally installed, follow the [user guide][16] to set it up.
|
||||
|
||||
2. View your current config settings:
|
||||
|
||||
```
|
||||
gcloud config list
|
||||
```
|
||||
|
||||
Store the `project` value from the results in the environment variable `$PROJECT_ID`.
|
||||
|
||||
2. Create a service account:
|
||||
|
||||
```
|
||||
gcloud iam service-accounts create heptio-ark \
|
||||
--display-name "Heptio Ark service account"
|
||||
```
|
||||
Then list all accounts and find the `heptio-ark` account you just created:
|
||||
```
|
||||
gcloud iam service-accounts list
|
||||
```
|
||||
Set the `$SERVICE_ACCOUNT_EMAIL` variable to match its `email` value.
|
||||
|
||||
3. Attach policies to give `heptio-ark` the necessary permissions to function (replacing placeholders appropriately):
|
||||
|
||||
```
|
||||
gcloud projects add-iam-policy-binding $PROJECT_ID \
|
||||
--member serviceAccount:$SERVICE_ACCOUNT_EMAIL \
|
||||
--role roles/compute.storageAdmin
|
||||
gcloud projects add-iam-policy-binding $PROJECT_ID \
|
||||
--member serviceAccount:$SERVICE_ACCOUNT_EMAIL \
|
||||
--role roles/storage.admin
|
||||
```
|
||||
|
||||
4. Create a service account key, specifying an output file (`credentials-ark`) in your local directory:
|
||||
|
||||
```
|
||||
gcloud iam service-accounts keys create credentials-ark \
|
||||
--iam-account $SERVICE_ACCOUNT_EMAIL
|
||||
```
|
||||
|
||||
#### Credentials and configuration
|
||||
|
||||
In the Ark root directory, run the following to first set up namespaces, RBAC, and other scaffolding:
|
||||
```
|
||||
kubectl apply -f examples/common/00-prereqs.yaml
|
||||
```
|
||||
|
||||
Create a Secret, running this command in the local directory of the credentials file you just created:
|
||||
|
||||
```
|
||||
kubectl create secret generic cloud-credentials \
|
||||
--namespace heptio-ark \
|
||||
--from-file cloud=credentials-ark
|
||||
```
|
||||
|
||||
Now that you have your Google Cloud credentials stored in a Secret, you need to replace some placeholder values in the template files. Specifically, you need to change the following:
|
||||
|
||||
* In file `examples/gcp/00-ark-config.yaml`:
|
||||
|
||||
* Replace `<YOUR_BUCKET>`, `<YOUR_PROJECT>` and `<YOUR_ZONE>`. See the [Config definition][7] for details.
|
||||
|
||||
|
||||
* In file `examples/common/10-deployment.yaml`:
|
||||
|
||||
* Change `spec.template.spec.containers[*].env.name` to "GOOGLE_APPLICATION_CREDENTIALS".
|
||||
|
||||
|
||||
* (Optional) If you are running the Nginx example, in file `examples/nginx-app/with-pv.yaml`:
|
||||
|
||||
* Replace `<YOUR_STORAGE_CLASS_NAME>` with `standard`. This is GCP's default `StorageClass` name.
|
||||
|
||||
### Azure
|
||||
|
||||
#### Service principal creation
|
||||
To integrate Heptio Ark with Azure, you should follow the instructions below to create an Ark-specific [service principal][17].
|
||||
|
||||
1. If you do not have the `az` Azure CLI 2.0 locally installed, follow the [user guide][18] to set it up. Once done, run:
|
||||
|
||||
```
|
||||
az login
|
||||
```
|
||||
|
||||
2. There are seven environment variables that need to be set for Heptio Ark to work properly. The following steps detail how to acquire these, in the process of setting up the necessary RBAC.
|
||||
|
||||
3. List your account:
|
||||
|
||||
```
|
||||
az account list
|
||||
```
|
||||
Save the relevant response values into environment variables: `id` corresponds to `$AZURE_SUBSCRIPTION_ID` and `tenantId` corresponds to `$AZURE_TENANT_ID`.
|
||||
|
||||
4. Assuming that you already have a running Kubernetes cluster on Azure, you should have a corresponding resource group as well. List your current groups to find it:
|
||||
|
||||
```
|
||||
az group list
|
||||
```
|
||||
Get your cluster's group `name` from the response, and use it to set `$AZURE_RESOURCE_GROUP`. (Also note the `location`--this is later used in the Azure-specific portion of the Ark Config).
|
||||
|
||||
5. Create a service principal with the "Contributor" role:
|
||||
|
||||
```
|
||||
az ad sp create-for-rbac --role="Contributor" --name="heptio-ark"
|
||||
```
|
||||
From the response, save `appId` into `$AZURE_CLIENT_ID` and `password` into `$AZURE_CLIENT_SECRET`.
|
||||
|
||||
6. Login into the `heptio-ark` service principal account:
|
||||
|
||||
```
|
||||
az login --service-principal \
|
||||
--username http://heptio-ark-test \
|
||||
--password $AZURE_CLIENT_SECRET \
|
||||
--tenant $AZURE_TENANT_ID
|
||||
```
|
||||
|
||||
7. Specify a *globally-unique* storage account id and save it in `$AZURE_STORAGE_ACCOUNT_ID`. Then create the storage account, specifying the optional `--location` flag if you do not have defaults from `az configure`:
|
||||
|
||||
```
|
||||
az storage account create \
|
||||
--name $AZURE_STORAGE_ACCOUNT_ID \
|
||||
--resource-group $AZURE_RESOURCE_GROUP \
|
||||
--sku Standard_GRS
|
||||
```
|
||||
You will encounter an error message if the storage account ID is not unique; change it accordingly.
|
||||
|
||||
8. Get the keys for your storage account:
|
||||
|
||||
```
|
||||
az storage account keys list \
|
||||
--account-name $AZURE_STORAGE_ACCOUNT_ID \
|
||||
--resource-group $AZURE_RESOURCE_GROUP
|
||||
```
|
||||
Set `$AZURE_STORAGE_KEY` to any one of the `value`s returned.
|
||||
|
||||
#### Credentials and configuration
|
||||
|
||||
In the Ark root directory, run the following to first set up namespaces, RBAC, and other scaffolding:
|
||||
```
|
||||
kubectl apply -f examples/common/00-prereqs.yaml
|
||||
```
|
||||
|
||||
Now you need to create a Secret that contains all the seven environment variables you just set. The command looks like the following:
|
||||
```
|
||||
kubectl create secret generic cloud-credentials \
|
||||
--namespace heptio-ark \
|
||||
--from-literal AZURE_SUBSCRIPTION_ID=${AZURE_SUBSCRIPTION_ID} \
|
||||
--from-literal AZURE_TENANT_ID=${AZURE_TENANT_ID} \
|
||||
--from-literal AZURE_RESOURCE_GROUP=${AZURE_RESOURCE_GROUP} \
|
||||
--from-literal AZURE_CLIENT_ID=${AZURE_CLIENT_ID} \
|
||||
--from-literal AZURE_CLIENT_SECRET=${AZURE_CLIENT_SECRET} \
|
||||
--from-literal AZURE_STORAGE_ACCOUNT_ID=${AZURE_STORAGE_ACCOUNT_ID} \
|
||||
--from-literal AZURE_STORAGE_KEY=${AZURE_STORAGE_KEY}
|
||||
```
|
||||
|
||||
Now that you have your Azure credentials stored in a Secret, you need to replace some placeholder values in the template files. Specifically, you need to change the following:
|
||||
|
||||
* In file `examples/azure/10-ark-config.yaml`:
|
||||
|
||||
* Replace `<YOUR_BUCKET>`, `<YOUR_LOCATION>`, and `<YOUR_TIMEOUT>`. See the [Config definition][8] for details.
|
||||
|
||||
|
||||
## Run
|
||||
|
||||
### Ark server
|
||||
|
||||
Make sure that you have run `kubectl apply -f examples/common/00-prereqs.yaml` first (this command is incorporated in the previous setup instructions because it creates the necessary namespaces).
|
||||
|
||||
* **AWS and GCP**
|
||||
|
||||
Start the Ark server itself, using the Config from the appropriate cloud-provider-specific directory:
|
||||
```
|
||||
kubectl apply -f examples/common/10-deployment.yaml
|
||||
kubectl apply -f examples/<CLOUD-PROVIDER>/
|
||||
```
|
||||
* **Azure**
|
||||
|
||||
Because Azure loads its credentials differently (from environment variables rather than a file), you need to instead run:
|
||||
```
|
||||
kubectl apply -f examples/azure/
|
||||
```
|
||||
|
||||
### Basic example (No PVs)
|
||||
|
||||
Start the sample nginx app:
|
||||
```
|
||||
kubectl apply -f examples/nginx-app/base.yaml
|
||||
```
|
||||
Now create a backup with PV snapshotting:
|
||||
```
|
||||
ark backup create nginx-backup --selector app=nginx
|
||||
```
|
||||
Simulate a disaster:
|
||||
```
|
||||
kubectl delete namespaces nginx-example
|
||||
```
|
||||
Now restore your lost resources:
|
||||
```
|
||||
ark restore create nginx-backup
|
||||
```
|
||||
|
||||
### Snapshot example (With PVs)
|
||||
|
||||
> NOTE: For Azure, your Kubernetes cluster needs to be version 1.7.2+ in order to support PV snapshotting of its managed disks.
|
||||
|
||||
Label a node so that all nginx pods end up on the same machine (avoiding PV binding issues):
|
||||
```
|
||||
nginx_node_name=$(kubectl get nodes -o jsonpath='{.items[0].metadata.name}')
|
||||
kubectl label nodes $nginx_node_name app=nginx
|
||||
```
|
||||
|
||||
Start the sample nginx app:
|
||||
```
|
||||
kubectl apply -f examples/nginx-app/with-pv.yaml
|
||||
```
|
||||
|
||||
Because Kubernetes does not automatically transfer labels from PVCs to dynamically generated PVs, you need to do so manually:
|
||||
```
|
||||
nginx_pv_name=$(kubectl get pv -o jsonpath='{.items[?(@.spec.claimRef.name=="nginx-logs")].metadata.name}')
|
||||
kubectl label pv $nginx_pv_name app=nginx
|
||||
```
|
||||
Now create a backup with PV snapshotting:
|
||||
```
|
||||
ark backup create nginx-backup --selector app=nginx --snapshot-volumes
|
||||
```
|
||||
Simulate a disaster:
|
||||
```
|
||||
kubectl delete namespaces nginx-example
|
||||
kubectl delete pv $nginx_pv_name
|
||||
```
|
||||
Now restore your lost resources:
|
||||
```
|
||||
ark restore create nginx-backup
|
||||
```
|
||||
|
||||
[0]: /README.md#quickstart
|
||||
[1]: #aws
|
||||
[2]: #gcp
|
||||
[3]: #azure
|
||||
[4]: /examples/aws
|
||||
[5]: http://docs.aws.amazon.com/cli/latest/userguide/installing.html
|
||||
[6]: config-definition.md#aws
|
||||
[7]: config-definition.md#gcp
|
||||
[8]: config-definition.md#azure
|
||||
[9]: #ark-server
|
||||
[10]: #basic-example-no-pvs
|
||||
[11]: #snapshot-example-with-pvs
|
||||
[12]: #setup
|
||||
[13]: #run
|
||||
[14]: http://docs.aws.amazon.com/IAM/latest/UserGuide/introduction.html
|
||||
[15]: https://cloud.google.com/compute/docs/access/service-accounts
|
||||
[16]: https://cloud.google.com/compute/docs/gcloud-compute
|
||||
[17]: https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-application-objects
|
||||
[18]: https://docs.microsoft.com/en-us/azure/storage/storage-azure-cli
|
||||
@@ -0,0 +1,66 @@
|
||||
# Concepts
|
||||
|
||||
* [Overview][0]
|
||||
* [Operation types][1]
|
||||
* [1. Backups][2]
|
||||
* [2. Schedules][3]
|
||||
* [3. Restores][4]
|
||||
* [Expired backup deletion][5]
|
||||
* [Cloud storage sync][6]
|
||||
|
||||
## Overview
|
||||
|
||||
Heptio Ark provides customizable degrees of recovery for all Kubernetes API objects (Pods, Deployments, Jobs, Custom Resource Definitions, etc.), as well as for persistent volumes. This recovery can be cluster-wide, or fine-tuned according to object type, namespace, or labels.
|
||||
|
||||
Ark is ideal for the disaster recovery use case, as well as for snapshotting your application state, prior to performing system operations on your cluster (e.g. upgrades).
|
||||
|
||||
## Operation types
|
||||
|
||||
This section gives a quick overview of the Ark operation types.
|
||||
|
||||
### 1. Backups
|
||||
The *backup* operation (1) uploads a tarball of copied Kubernetes resources into cloud object storage and (2) uses the cloud provider API to make disk snapshots of persistent volumes, if specified. [Annotations][8] are cleared for PVs but kept for all other object types.
|
||||
|
||||
Some things to be aware of:
|
||||
* *Cluster backups are not strictly atomic.* If API objects are being created or edited at the time of backup, they may or not be included in the backup. In practice, backups happen very quickly and so the odds of capturing inconsistent information are low, but still possible.
|
||||
|
||||
* *A backup usually takes no more than a few seconds.* The snapshotting process for persistent volumes is asynchronous, so the runtime of the `ark backup` command isn't dependent on disk size.
|
||||
|
||||
These ad-hoc backups are saved with the `<BACKUP NAME>` specified during creation.
|
||||
|
||||
|
||||
### 2. Schedules
|
||||
The *schedule* operation allows you to back up your data at recurring intervals. The first backup is performed when the schedule is first created, and subsequent backups happen at the schedule's specified interval. These intervals are specified by a Cron expression.
|
||||
|
||||
A Schedule acts as a wrapper for Backups; when triggered, it creates them behind the scenes.
|
||||
|
||||
Scheduled backups are saved with the name `<SCHEDULE NAME>-<TIMESTAMP>`, where `<TIMESTAMP>` is formatted as *YYYYMMDDhhmmss*.
|
||||
|
||||
### 3. Restores
|
||||
The *restore* operation allows you to restore all of the objects and persistent volumes from a previously created Backup. Heptio Ark supports multiple namespace remapping--for example, in a single restore, objects in namespace "abc" can be recreated under namespace "def", and the ones in "123" under "456".
|
||||
|
||||
Kubernetes API objects that have been restored can be identified with a label that looks like `ark-restore=<BACKUP NAME>-<TIMESTAMP>`, where `<TIMESTAMP>` is formatted as *YYYYMMDDhhmmss*.
|
||||
|
||||
You can also run the Ark server in *restore-only* mode, which disables backup, schedule, and garbage collection functionality during disaster recovery.
|
||||
|
||||
## Expired backup deletion
|
||||
|
||||
When first creating a backup, you can specify a TTL. If Ark sees that an existing Backup resource has expired, it removes both:
|
||||
* The Backup resource itself
|
||||
* The actual backup file from cloud object storage
|
||||
|
||||
## Cloud storage sync
|
||||
|
||||
Heptio Ark treats object storage as the source of truth. It continuously checks to see that the correct Backup resources are always present. If there is a properly formatted backup file in the storage bucket, but no corresponding Backup resources in the Kubernetes API, Ark synchronizes the information from object storage to Kubernetes.
|
||||
|
||||
This allows *restore* functionality to work in a cluster migration scenario, where the original Backup objects do not exist in the new cluster. See the [use case guide][7] for details.
|
||||
|
||||
[0]: #overview
|
||||
[1]: #operation-types
|
||||
[2]: #1-backups
|
||||
[3]: #2-schedules
|
||||
[4]: #3-restores
|
||||
[5]: #expired-backup-deletion
|
||||
[6]: #cloud-storage-sync
|
||||
[7]: use-cases.md#cluster-migration
|
||||
[8]: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
|
||||
@@ -0,0 +1,97 @@
|
||||
# Ark Config definition
|
||||
|
||||
* [Overview][10]
|
||||
* [Example][11]
|
||||
* [Parameter Reference][8]
|
||||
* [Main config][9]
|
||||
* [AWS][0]
|
||||
* [GCP][1]
|
||||
* [Azure][2]
|
||||
|
||||
## Overview
|
||||
|
||||
Heptio Ark defines its own Config object (a custom resource) for specifying Ark backup and cloud provider settings. When the Ark server is first deployed, it waits until you create a Config--specifically one named `default`--in the `heptio-ark` namespace.
|
||||
|
||||
> *NOTE*: There is an underlying assumption that you're running the Ark server as a Kubernetes deployment. If the `default` Config is modified, the server shuts down gracefully. Once the kubelet restarts the Ark server pod, the server then uses the updated Config values.
|
||||
|
||||
## Example
|
||||
|
||||
A sample YAML `Config` looks like the following:
|
||||
```
|
||||
apiVersion: ark.heptio.com/v1
|
||||
kind: Config
|
||||
metadata:
|
||||
namespace: heptio-ark
|
||||
name: default
|
||||
persistentVolumeProvider:
|
||||
aws:
|
||||
region: minio
|
||||
availabilityZone: minio
|
||||
s3ForcePathStyle: true
|
||||
s3Url: http://minio:9000
|
||||
backupStorageProvider:
|
||||
bucket: ark
|
||||
aws:
|
||||
region: minio
|
||||
availabilityZone: minio
|
||||
s3ForcePathStyle: true
|
||||
s3Url: http://minio:9000
|
||||
backupSyncPeriod: 60m
|
||||
gcSyncPeriod: 60m
|
||||
scheduleSyncPeriod: 1m
|
||||
restoreOnlyMode: false
|
||||
```
|
||||
|
||||
## Parameter Reference
|
||||
|
||||
The configurable parameters are as follows:
|
||||
|
||||
### Main config parameters
|
||||
|
||||
| Key | Type | Default | Meaning |
|
||||
| --- | --- | --- | --- |
|
||||
| `persistentVolumeProvider` | CloudProviderConfig<br><br>(Supported key values are `aws`, `gcp`, and `azure`, but only one can be present. See the corresponding [AWS][0], [GCP][1], and [Azure][2]-specific configs.) | Required Field | The specification for whichever cloud provider the cluster is using for persistent volumes (to be snapshotted).<br><br> *NOTE*: For Azure, your Kubernetes cluster needs to be version 1.7.2+ in order to support PV snapshotting of its managed disks. |
|
||||
| `backupStorageProvider`/(inline) | CloudProviderConfig<br><br>(Supported key values are `aws`, `gcp`, and `azure`, but only one can be present. See the corresponding [AWS][0], [GCP][1], and [Azure][2]-specific configs.) | Required Field | The specification for whichever cloud provider will be used to actually store the backups. |
|
||||
| `backupStorageProvider/bucket` | String | Required Field | The storage bucket where backups are to be uploaded. |
|
||||
| `backupSyncPeriod` | metav1.Duration | 60m0s | How frequently Ark queries the object storage to make sure that the appropriate Backup resources have been created for existing backup files. |
|
||||
| `gcSyncPeriod` | metav1.Duration | 60m0s | How frequently Ark queries the object storage to delete backup files that have passed their TTL. |
|
||||
| `scheduleSyncPeriod` | metav1.Duration | 1m0s | How frequently Ark checks its Schedule resource objects to see if a backup needs to be initiated. |
|
||||
| `resourcePriorities` | []string | `[namespaces, persistentvolumes, persistentvolumeclaims, secrets, configmaps]` | An ordered list that describes the order in which Kubernetes resource objects should be restored (also specified with the `<RESOURCE>.<GROUP>` format.<br><br>If a resource is not in this list, it is restored after all other prioritized resources. |
|
||||
| `restoreOnlyMode` | bool | `false` | When RestoreOnly mode is on, functionality for backups, schedules, and expired backup deletion is *turned off*. Restores are made from existing backup files in object storage. |
|
||||
|
||||
### AWS
|
||||
|
||||
**(Or other S3-compatible storage)**
|
||||
|
||||
| Key | Type | Default | Meaning |
|
||||
| --- | --- | --- | --- |
|
||||
| `region` | string | Required Field | *Example*: "us-east-1"<br><br>See [AWS documentation][3] for the full list. |
|
||||
| `availabilityZone` | string | Required Field | *Example*: "us-east-1a"<br><br>See [AWS documentation][4] for details. |
|
||||
| `disableSSL` | bool | `false` | Set this to `true` if you are using Minio (or another local, S3-compatible storage service) and your deployment is not secured. |
|
||||
| `s3ForcePathStyle` | bool | `false` | Set this to `true` if you are using a local storage service like Minio. |
|
||||
| `s3Url` | string | Required field for non-AWS-hosted storage| *Example*: http://minio:9000<br><br>You can specify the AWS S3 URL here for explicitness, but Ark can already generate it from `region`, `availabilityZone`, and `bucket`. This field is primarily for local sotrage services like Minio.|
|
||||
|
||||
### GCP
|
||||
| Key | Type | Default | Meaning |
|
||||
| --- | --- | --- | --- |
|
||||
| `project` | string | Required Field | *Example*: "project-example-3jsn23"<br><br> See the [Project ID documentation][5] for details. |
|
||||
| `zone` | string | Required Field | *Example*: "us-central1-a"<br><br>See [GCP documentation][6] for the full list. |
|
||||
|
||||
### Azure
|
||||
| Key | Type | Default | Meaning |
|
||||
| --- | --- | --- | --- |
|
||||
| `location` | string | Required Field | *Example*: "Canada East"<br><br>See [the list of available locations][7] (note that this particular page refers to them as "Regions"). |
|
||||
| `apiTimeout` | metav1.Duration | 1m0s | How long to wait for an API Azure request to complete before timeout. |
|
||||
|
||||
[0]: #aws
|
||||
[1]: #gcp
|
||||
[2]: #azure
|
||||
[3]: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-available-regions
|
||||
[4]: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-availability-zones
|
||||
[5]: https://cloud.google.com/resource-manager/docs/creating-managing-projects#identifying_projects
|
||||
[6]: https://cloud.google.com/compute/docs/regions-zones/regions-zones
|
||||
[7]: https://azure.microsoft.com/en-us/regions/
|
||||
[8]: #parameter-reference
|
||||
[9]: #main-config-parameters
|
||||
[10]: #overview
|
||||
[11]: #example
|
||||
@@ -0,0 +1,51 @@
|
||||
# Debugging Restores
|
||||
|
||||
* [Example][0]
|
||||
* [Structure][1]
|
||||
|
||||
## Example
|
||||
|
||||
When Heptio Ark finishes a Restore, its status changes to "Completed" regardless of whether or not there are issues during the process. The number of warnings and errors are indicated in the output columns from `ark restore get`:
|
||||
|
||||
```
|
||||
NAME BACKUP STATUS WARNINGS ERRORS CREATED SELECTOR
|
||||
backup-test-20170726180512 backup-test Completed 155 76 2017-07-26 11:41:14 -0400 EDT <none>
|
||||
backup-test-20170726180513 backup-test Completed 121 14 2017-07-26 11:48:24 -0400 EDT <none>
|
||||
backup-test-2-20170726180514 backup-test-2 Completed 0 0 2017-07-26 13:31:21 -0400 EDT <none>
|
||||
backup-test-2-20170726180515 backup-test-2 Completed 0 1 2017-07-26 13:32:59 -0400 EDT <none>
|
||||
```
|
||||
|
||||
To delve into the warnings and errors into more detail, you can use the `-o` option:
|
||||
```
|
||||
kubectl restore get backup-test-20170726180512 -o yaml
|
||||
```
|
||||
The output YAML has a `status` field which may look like the following:
|
||||
```
|
||||
status:
|
||||
errors:
|
||||
ark: null
|
||||
cluster: null
|
||||
namespaces: null
|
||||
phase: Completed
|
||||
validationErrors: null
|
||||
warnings:
|
||||
ark: null
|
||||
cluster: null
|
||||
namespaces:
|
||||
cm1:
|
||||
- secrets "default-token-t0slk" already exists
|
||||
```
|
||||
|
||||
## Structure
|
||||
The `status` field in a Restore's YAML has subfields for `errors` and `warnings`. `errors` appear for incomplete or partial restores. `warnings` appear for non-blocking issues (e.g. the restore looks "normal" and all resources referenced in the backup exist in some form, although some of them may have been pre-existing).
|
||||
|
||||
Both `errors` and `warnings` are structured in the same way:
|
||||
|
||||
* `ark`: A list of system-related issues encountered by the Ark server (e.g. couldn't read directory).
|
||||
|
||||
* `cluster`: A list of issues related to the restore of cluster-scoped resources.
|
||||
|
||||
* `namespaces`: A map of namespaces to the list of issues related to the restore of their respective resources.
|
||||
|
||||
[0]: #example
|
||||
[1]: #structure
|
||||
@@ -0,0 +1,39 @@
|
||||
/*
|
||||
Copyright 2017 Heptio Inc.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"log"
|
||||
"os"
|
||||
|
||||
"github.com/spf13/cobra/doc"
|
||||
"github.com/heptio/ark/pkg/cmd/ark"
|
||||
)
|
||||
|
||||
func main() {
|
||||
cmdName := os.Args[1]
|
||||
outputDir := os.Args[2]
|
||||
|
||||
cmd := ark.NewCommand(cmdName)
|
||||
// Remove auto-generated timestamps
|
||||
cmd.DisableAutoGenTag = true
|
||||
|
||||
err := doc.GenMarkdownTree(cmd, outputDir)
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
Some of these diagrams (for instance backup-process.png), have been created on [draw.io](https://www.draw.io), using the "Include a copy of my diagram" option. If you want to make changes to these diagrams, try importing them into draw.io, and you should have access to the original shapes/text that went into the originals.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 33 KiB |
@@ -0,0 +1,87 @@
|
||||
# Output file format
|
||||
|
||||
A backup is a gzip-compressed tar file whose name matches the Backup API resource's `metadata.name` (what is specified during `ark backup create <NAME>`).
|
||||
|
||||
In cloud object storage, *each backup file is stored in its own subdirectory* beneath the bucket specified in the Ark server configuration. This subdirectory includes an additional file called `ark-backup.json`. The JSON file explicitly lists all info about your associated Backup resource--including any default values used--so that you have a complete historical record of its configuration. It also specifies `status.version`, which corresponds to the output file format.
|
||||
|
||||
All together, the directory structure in your cloud storage may look like:
|
||||
|
||||
```
|
||||
rootBucket/
|
||||
backup1234/
|
||||
ark-backup.json
|
||||
backup1234.tar.gz
|
||||
```
|
||||
|
||||
## `ark-backup.json`
|
||||
An example of this file looks like the following:
|
||||
```
|
||||
{
|
||||
"kind": "Backup",
|
||||
"apiVersion": "ark.heptio.com/v1",
|
||||
"metadata": {
|
||||
"name": "test-backup",
|
||||
"namespace": "heptio-ark",
|
||||
"selfLink": "/apis/ark.heptio.com/v1/namespaces/heptio-ark/backups/testtest",
|
||||
"uid": "a12345cb-75f5-11e7-b4c2-abcdef123456",
|
||||
"resourceVersion": "337075",
|
||||
"creationTimestamp": "2017-07-31T13:39:15Z"
|
||||
},
|
||||
"spec": {
|
||||
"includedNamespaces": [
|
||||
"*"
|
||||
],
|
||||
"excludedNamespaces": null,
|
||||
"includedResources": [
|
||||
"*"
|
||||
],
|
||||
"excludedResources": null,
|
||||
"labelSelector": null,
|
||||
"snapshotVolumes": true,
|
||||
"ttl": "24h0m0s"
|
||||
},
|
||||
"status": {
|
||||
"version": 1,
|
||||
"expiration": "2017-08-01T13:39:15Z",
|
||||
"phase": "Completed",
|
||||
"volumeBackups": {
|
||||
"pvc-e1e2d345-7583-11e7-b4c2-abcdef123456": {
|
||||
"snapshotID": "snap-04b1a8e11dfb33ab0",
|
||||
"type": "gp2",
|
||||
"iops": 100
|
||||
}
|
||||
},
|
||||
"validationErrors": null
|
||||
}
|
||||
}
|
||||
```
|
||||
Note that this file includes detailed info about your volume snapshots in the `status.volumeBackups` field, which can be helpful if you want to manually check them in your cloud provider GUI.
|
||||
|
||||
## file format version: 1
|
||||
|
||||
When unzipped, a typical backup directory (e.g. `backup1234.tar.gz`) looks like the following:
|
||||
|
||||
```
|
||||
cluster/
|
||||
persistentvolumes/
|
||||
pv01.json
|
||||
...
|
||||
namespaces/
|
||||
namespace1/
|
||||
configmaps/
|
||||
myconfigmap.json
|
||||
...
|
||||
pods
|
||||
mypod.json
|
||||
...
|
||||
jobs
|
||||
awesome-job.json
|
||||
...
|
||||
deployments
|
||||
cool-deployment.json
|
||||
...
|
||||
...
|
||||
namespace2/
|
||||
...
|
||||
...
|
||||
```
|
||||
@@ -0,0 +1,54 @@
|
||||
# Use Cases
|
||||
|
||||
This doc provides sample Ark commands for the following common scenarios:
|
||||
* [Disaster recovery][0]
|
||||
* [Cluster migration][1]
|
||||
|
||||
## Disaster recovery
|
||||
|
||||
*Using Schedules and Restore-Only Mode*
|
||||
|
||||
If you periodically back up your cluster's resources, you are able to return to a previous state in case of some unexpected mishap, such as a service outage. Doing so with Heptio Ark looks like the following:
|
||||
|
||||
1. After you first run the Ark server on your cluster, set up a daily backup (replacing `<SCHEDULE NAME>` in the command as desired):
|
||||
|
||||
```
|
||||
ark schedule create <SCHEDULE NAME> --schedule "0 7 * * *"
|
||||
```
|
||||
This creates a Backup object with the name `<SCHEDULE NAME>-<TIMESTAMP>`.
|
||||
|
||||
2. A disaster happens and you need to recreate your resources.
|
||||
|
||||
3. Update the [Ark server Config][3], setting `restoreOnlyMode` to `true`. This prevents Backup objects from being created or deleted during your Restore process.
|
||||
|
||||
4. Create a restore with your most recent Ark Backup:
|
||||
```
|
||||
ark restore create <SCHEDULE NAME>-<TIMESTAMP>
|
||||
```
|
||||
|
||||
## Cluster migration
|
||||
|
||||
*Using Backups and Restores*
|
||||
|
||||
Heptio Ark can help you port your resources from one cluster to another, as long as you point each Ark Config to the same cloud object storage. In this scenario, we are also assuming that your clusters are hosted by the same cloud provider. **Note that Heptio Ark does not support the migration of persistent volumes across cloud providers.**
|
||||
|
||||
1. *(Cluster 1)* Assuming you haven't already been checkpointing your data with the Ark `schedule` operation, you need to first back up your entire cluster (replacing `<BACKUP-NAME>` as desired):
|
||||
|
||||
```
|
||||
ark backup create <BACKUP-NAME> --snapshot-volumes
|
||||
```
|
||||
The default TTL is 24 hours; you can use the `--ttl` flag to change this as necessary.
|
||||
|
||||
2. *(Cluster 2)* Make sure that the `persistentVolumeProvider` and `backupStorageProvider` fields in the Ark Config match the ones from *Cluster 1*, so that your new Ark server instance is pointing to the same bucket.
|
||||
|
||||
3. *(Cluster 2)* Make sure that the Ark Backup object has been created. Ark resources are [synced][2] with the backup files available in cloud storage.
|
||||
|
||||
4. *(Cluster 2)* Once you have confirmed that the right Backup (`<BACKUP-NAME>`) is now present, you can restore everything with:
|
||||
```
|
||||
ark restore create <BACKUP-NAME> --restore-volumes
|
||||
```
|
||||
|
||||
[0]: #disaster-recovery
|
||||
[1]: #cluster-migration
|
||||
[2]: concepts.md#cloud-storage-sync
|
||||
[3]: config-definition.md#main-config-parameters
|
||||
Reference in New Issue
Block a user