have restic share main Ark bucket

Signed-off-by: Steve Kriss <steve@heptio.com>
This commit is contained in:
Steve Kriss
2018-09-19 12:51:30 -06:00
parent 42b54586cd
commit e46e89cb61
11 changed files with 130 additions and 107 deletions

View File

@@ -23,15 +23,9 @@ cross-volume-type data migrations. Stay tuned as this evolves!
### Prerequisites
- A working install of Ark version 0.9.0 or later. See [Set up Ark][2]
- A working install of Ark version 0.10.0 or later. See [Set up Ark][2]
- A local clone of [the latest release tag of the Ark repository][3]
#### Additional steps if upgrading from version 0.9 alpha
- Manually delete all of the repositories/data from your existing restic bucket
- Delete all Ark backups from your cluster using `ark backup delete`
- Delete all secrets named `ark-restic-credentials` across all namespaces in your cluster
### Instructions
1. Download an updated Ark client from the [latest release][3], and move it to a location in your PATH.
@@ -49,20 +43,6 @@ cross-volume-type data migrations. Stay tuned as this evolves!
- GCP: `kubectl apply -f examples/gcp/20-restic-daemonset.yaml`
- Minio: `kubectl apply -f examples/minio/30-restic-daemonset.yaml`
1. Create a new bucket for restic to store its data in, and give the `heptio-ark` IAM user access to it, similarly to
the main Ark bucket you've already set up. Note that this must be a different bucket than the main Ark bucket.
We plan to remove this limitation in a future release.
1. Uncomment `resticLocation` in your Ark config and set the value appropriately, then apply:
- AWS: `kubectl apply -f examples/aws/00-ark-config.yaml`
- Azure: `kubectl apply -f examples/azure/10-ark-config.yaml`
- GCP: `kubectl apply -f examples/gcp/00-ark-config.yaml`
- Minio: `kubectl apply -f examples/minio/10-ark-config.yaml`
Note that `resticLocation` may either be just a bucket name, e.g. `my-restic-bucket`, or a bucket name plus a prefix under
which you'd like the restic data to be stored, e.g. `my-restic-bucket/ark-repos`.
You're now ready to use Ark with restic.
## Back up
@@ -139,8 +119,6 @@ You're now ready to use Ark with restic.
## Limitations
- You cannot use the main Ark bucket for storing restic backups. We plan to address this issue
in a future release.
- `hostPath` volumes are not supported. [Local persistent volumes][4] are supported.
- Those of you familiar with [restic][1] may know that it encrypts all of its data. We've decided to use a static,
common encryption key for all restic repositories created by Ark. **This means that anyone who has access to your
@@ -264,4 +242,4 @@ on to running other init containers/the main containers.
[2]: cloud-common.md
[3]: https://github.com/heptio/ark/releases/
[4]: https://kubernetes.io/docs/concepts/storage/volumes/#local
[5]: http://restic.readthedocs.io/en/latest/100_references.html#terminology
[5]: http://restic.readthedocs.io/en/latest/100_references.html#terminology

View File

@@ -26,6 +26,25 @@ Prior to v0.10, Ark stored data in an object storage bucket using the following
...
```
Ark also stored restic data, if applicable, in a separate object storage bucket, structured as:
```
<your-ark-restic-bucket>/[<your-optional-prefix>/]
namespace-1/
data/
index/
keys/
snapshots/
config
namespace-2/
data/
index/
keys/
snapshots/
config
...
```
As of v0.10, we've reorganized this layout to provide a cleaner and more extensible directory structure. The new layout looks like:
```
@@ -48,6 +67,20 @@ As of v0.10, we've reorganized this layout to provide a cleaner and more extensi
restore-of-backup-2-logs.gz
restore-of-backup-2-results.gz
...
restic/
namespace-1/
data/
index/
keys/
snapshots/
config
namespace-2/
data/
index/
keys/
snapshots/
config
...
...
```
@@ -104,7 +137,22 @@ rclone copy ${RCLONE_REMOTE_NAME}:${ARK_TEMP_MIGRATION_BUCKET} ${RCLONE_REMOTE_N
# contains an exact copy of the temporary bucket's contents:
rclone check ${RCLONE_REMOTE_NAME}:${ARK_BUCKET}/backups ${RCLONE_REMOTE_NAME}:${ARK_TEMP_MIGRATION_BUCKET}
# 9. Once you've confirmed that Ark v0.10 works with your revised Ark
# 9. OPTIONAL: If you have restic data to migrate:
# a. Copy the contents of your Ark restic location into your
# Ark bucket, under the 'restic/' directory/prefix:
ARK_RESTIC_LOCATION=<your-ark-restic-bucket[/optional-prefix]>
rclone copy ${RCLONE_REMOTE_NAME}:${ARK_RESTIC_LOCATION} ${RCLONE_REMOTE_NAME}:${ARK_BUCKET}/restic
# b. Check that the 'restic/' directory in your Ark bucket now
# contains an exact copy of your restic location:
rclone check ${RCLONE_REMOTE_NAME}:${ARK_BUCKET}/restic ${RCLONE_REMOTE_NAME}:${ARK_RESTIC_LOCATION}
# c. Delete your ResticRepository custom resources to allow Ark
# to find them in the new location:
kubectl -n heptio-ark delete resticrepositories --all
# 10. Once you've confirmed that Ark v0.10 works with your revised Ark
# bucket, you can delete the temporary migration bucket.
```