diff --git a/docs/aws-config.md b/docs/aws-config.md index 35f85d033..add8b5a87 100644 --- a/docs/aws-config.md +++ b/docs/aws-config.md @@ -141,7 +141,11 @@ Specify the following values in the example files: * In `examples/aws/00-ark-config.yaml`: - * Replace `` and `` (for S3, region is optional and will be queried from the AWS S3 API if not provided). See the [Config definition][6] for details. + * Replace ``. See the [Config definition][6] for details. + +* In `examples/aws/05-ark-backupstoragelocation.yaml`: + + * Replace `` and `` (for S3 backup storage, region is optional and will be queried from the AWS S3 API if not provided). See the [BackupStorageLocation definition][21] for details. * (Optional) If you run the nginx example, in file `examples/nginx-app/with-pv.yaml`: @@ -273,4 +277,5 @@ It can be set up for Ark by creating a role that will have required permissions, [5]: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html [6]: config-definition.md#aws [14]: http://docs.aws.amazon.com/IAM/latest/UserGuide/introduction.html -[20]: faq.md \ No newline at end of file +[20]: faq.md +[21]: backupstoragelocation-definition.md#aws \ No newline at end of file diff --git a/docs/azure-config.md b/docs/azure-config.md index c92fedc8b..b02d29229 100644 --- a/docs/azure-config.md +++ b/docs/azure-config.md @@ -135,9 +135,13 @@ Now that you have your Azure credentials stored in a Secret, you need to replace * In file `examples/azure/10-ark-config.yaml`: - * Replace `` and ``. See the [Config definition][8] for details. + * Replace ``. See the [Config definition][8] for details. -Here is an example of a completed file. +* In file `examples/azure/05-ark-backupstoragelocation.yaml`: + + * Replace ``. See the [BackupStorageLocation definition][21] for details. + +Here is an example of a completed config file. ```yaml apiVersion: ark.heptio.com/v1 @@ -149,9 +153,6 @@ persistentVolumeProvider: name: azure config: apiTimeout: 15m -backupStorageProvider: - name: azure - bucket: ark backupSyncPeriod: 30m gcSyncPeriod: 30m scheduleSyncPeriod: 1m @@ -166,9 +167,10 @@ In the root of your Ark directory, run: kubectl apply -f examples/azure/ ``` - [0]: namespace.md - [8]: config-definition.md#azure - [17]: https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-application-objects - [18]: https://docs.microsoft.com/en-us/cli/azure/install-azure-cli - [19]: https://docs.microsoft.com/en-us/azure/architecture/best-practices/naming-conventions#storage - [20]: faq.md +[0]: namespace.md +[8]: config-definition.md#azure +[17]: https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-application-objects +[18]: https://docs.microsoft.com/en-us/cli/azure/install-azure-cli +[19]: https://docs.microsoft.com/en-us/azure/architecture/best-practices/naming-conventions#storage +[20]: faq.md +[21]: backupstoragelocation-definition.md#azure \ No newline at end of file diff --git a/docs/backupstoragelocation-definition.md b/docs/backupstoragelocation-definition.md new file mode 100644 index 000000000..f8529b0fc --- /dev/null +++ b/docs/backupstoragelocation-definition.md @@ -0,0 +1,65 @@ +# Ark Backup Storage Locations + +## Backup Storage Location + +Ark can store backups in a number of locations. These are represented in the cluster via the `BackupStorageLocation` CRD. + +Ark must have at least one `BackupStorageLocation`. By default, this is expected to be named `default`, however the name can be changed by specifying `--default-backup-storage-location` on `ark server`. Backups that do not explicitly specify a storage location will be saved to this `BackupStorageLocation`. + +> *NOTE*: `BackupStorageLocation` takes the place of the `Config.backupStorageProvider` key as of v0.10.0 + +A sample YAML `BackupStorageLocation` looks like the following: + +```yaml +apiVersion: ark.heptio.com/v1 +kind: BackupStorageLocation +metadata: + name: default + namespace: heptio-ark +spec: + provider: aws + objectStorage: + bucket: myBucket + config: + region: us-west-2 +``` + +### Parameter Reference + +The configurable parameters are as follows: + +#### Main config parameters + +| Key | Type | Default | Meaning | +| --- | --- | --- | --- | +| `provider` | String (Ark natively supports `aws`, `gcp`, and `azure`. Other providers may be available via external plugins.)| Required Field | The name for whichever cloud provider will be used to actually store the backups. | +| `objectStorage` | ObjectStorageLocation | Specification of the object storage for the given provider. | +| `objectStorage/bucket` | String | Required Field | The storage bucket where backups are to be uploaded. | +| `objectStorage/prefix` | String | Optional Field | The directory inside a storage bucket where backups are to be uploaded. | +| `objectStorage/config` | map[string]string

(See the corresponding [AWS][0], [GCP][1], and [Azure][2]-specific configs or your provider's documentation.) | None (Optional) | Configuration keys/values to be passed to the cloud provider for backup storage. | + +#### AWS + +**(Or other S3-compatible storage)** + +##### objectStorage/config + +| Key | Type | Default | Meaning | +| --- | --- | --- | --- | +| `region` | string | Empty | *Example*: "us-east-1"

See [AWS documentation][3] for the full list.

Queried from the AWS S3 API if not provided. | +| `s3ForcePathStyle` | bool | `false` | Set this to `true` if you are using a local storage service like Minio. | +| `s3Url` | string | Required field for non-AWS-hosted storage| *Example*: http://minio:9000

You can specify the AWS S3 URL here for explicitness, but Ark can already generate it from `region`, and `bucket`. This field is primarily for local storage services like Minio.| +| `kmsKeyId` | string | Empty | *Example*: "502b409c-4da1-419f-a16e-eif453b3i49f" or "alias/``"

Specify an [AWS KMS key][10] id or alias to enable encryption of the backups stored in S3. Only works with AWS S3 and may require explicitly granting key usage rights.| + +#### GCP + +No parameters required. + +#### Azure + +No parameters required. + +[0]: #aws +[1]: #gcp +[2]: #azure +[3]: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-available-regions \ No newline at end of file diff --git a/docs/config-definition.md b/docs/config-definition.md index ca689e906..4b4bf0942 100644 --- a/docs/config-definition.md +++ b/docs/config-definition.md @@ -31,11 +31,6 @@ persistentVolumeProvider: name: aws config: region: us-west-2 -backupStorageProvider: - name: aws - bucket: ark - config: - region: us-west-2 ``` ### Parameter Reference @@ -49,24 +44,9 @@ The configurable parameters are as follows: | `persistentVolumeProvider` | CloudProviderConfig | None (Optional) | The specification for whichever cloud provider the cluster is using for persistent volumes (to be snapshotted), if any.

If not specified, Backups and Restores requesting PV snapshots & restores, respectively, are considered invalid.

*NOTE*: For Azure, your Kubernetes cluster needs to be version 1.7.2+ in order to support PV snapshotting of its managed disks. | | `persistentVolumeProvider/name` | String

(Ark natively supports `aws`, `gcp`, and `azure`. Other providers may be available via external plugins.) | None (Optional) | The name of the cloud provider the cluster is using for persistent volumes, if any. | | `persistentVolumeProvider/config` | map[string]string

(See the corresponding [AWS][0], [GCP][1], and [Azure][2]-specific configs or your provider's documentation.) | None (Optional) | Configuration keys/values to be passed to the cloud provider for persistent volumes. | -| `backupStorageProvider` | CloudProviderConfig | Required Field | The specification for whichever cloud provider will be used to actually store the backups. | -| `backupStorageProvider/name` | String

(Ark natively supports `aws`, `gcp`, and `azure`. Other providers may be available via external plugins.) | Required Field | The name of the cloud provider that will be used to actually store the backups. | -| `backupStorageProvider/bucket` | String | Required Field | The storage bucket where backups are to be uploaded. | -| `backupStorageProvider/config` | map[string]string

(See the corresponding [AWS][0], [GCP][1], and [Azure][2]-specific configs or your provider's documentation.) | None (Optional) | Configuration keys/values to be passed to the cloud provider for backup storage. | #### AWS -**(Or other S3-compatible storage)** - -##### backupStorageProvider/config - -| Key | Type | Default | Meaning | -| --- | --- | --- | --- | -| `region` | string | Empty | *Example*: "us-east-1"

See [AWS documentation][3] for the full list.

Queried from the AWS S3 API if not provided. | -| `s3ForcePathStyle` | bool | `false` | Set this to `true` if you are using a local storage service like Minio. | -| `s3Url` | string | Required field for non-AWS-hosted storage| *Example*: http://minio:9000

You can specify the AWS S3 URL here for explicitness, but Ark can already generate it from `region`, and `bucket`. This field is primarily for local storage services like Minio.| -| `kmsKeyId` | string | Empty | *Example*: "502b409c-4da1-419f-a16e-eif453b3i49f" or "alias/``"

Specify an [AWS KMS key][10] id or alias to enable encryption of the backups stored in S3. Only works with AWS S3 and may require explicitly granting key usage rights.| - ##### persistentVolumeProvider/config (AWS Only) | Key | Type | Default | Meaning | @@ -75,20 +55,12 @@ The configurable parameters are as follows: #### GCP -#### backupStorageProvider/config - -No parameters required. - ##### persistentVolumeProvider/config No parameters required. #### Azure -##### backupStorageProvider/config - -No parameters required. - ##### persistentVolumeProvider/config | Key | Type | Default | Meaning | diff --git a/docs/gcp-config.md b/docs/gcp-config.md index 9561010fa..2943fbb56 100644 --- a/docs/gcp-config.md +++ b/docs/gcp-config.md @@ -112,9 +112,9 @@ _Note: If you use a custom namespace, replace `heptio-ark` with the name of the Specify the following values in the example files: -* In file `examples/gcp/00-ark-config.yaml`: +* In file `examples/gcp/05-ark-backupstoragelocation.yaml`: - * Replace ``. See the [Config definition][7] for details. + * Replace ``. See the [BackupStorageLocation definition][7] for details. * (Optional) If you run the nginx example, in file `examples/nginx-app/with-pv.yaml`: @@ -130,7 +130,7 @@ In the root of your Ark directory, run: ``` [0]: namespace.md - [7]: config-definition.md#gcp + [7]: backupstoragelocation-definition.md#gcp [15]: https://cloud.google.com/compute/docs/access/service-accounts [16]: https://cloud.google.com/sdk/docs/ [20]: faq.md diff --git a/docs/ibm-config.md b/docs/ibm-config.md index 218727a3a..b27f2c13f 100644 --- a/docs/ibm-config.md +++ b/docs/ibm-config.md @@ -53,9 +53,9 @@ kubectl create secret generic cloud-credentials \ Specify the following values in the example files: -* In `examples/ibm/00-ark-config.yaml`: +* In `examples/ibm/05-ark-backupstoragelocation.yaml`: - * Replace ``, `` and ``. See the [Config definition][6] for details. + * Replace ``, `` and ``. See the [BackupStorageLocation definition][6] for details. @@ -78,5 +78,5 @@ In the root of your Ark directory, run: [3]: https://console.bluemix.net/docs/services/cloud-object-storage/iam/service-credentials.html#service-credentials [4]: https://www.ibm.com/support/knowledgecenter/SSBS6K_2.1.0/kc_welcome_containers.html [5]: https://console.bluemix.net/docs/containers/container_index.html#container_index - [6]: config-definition.md#aws + [6]: backupstoragelocation-definition.md#aws [14]: http://docs.aws.amazon.com/IAM/latest/UserGuide/introduction.html diff --git a/examples/aws/00-ark-config.yaml b/examples/aws/00-ark-config.yaml index 716c865fc..2068deec2 100644 --- a/examples/aws/00-ark-config.yaml +++ b/examples/aws/00-ark-config.yaml @@ -20,16 +20,5 @@ metadata: name: default persistentVolumeProvider: name: aws - config: - region: -backupStorageProvider: - name: aws - bucket: - # Uncomment the below line to enable restic integration. - # The format for resticLocation is [/], - # e.g. "my-restic-bucket" or "my-restic-bucket/repos". - # This MUST be a different bucket than the main Ark bucket - # specified just above. - # resticLocation: config: region: \ No newline at end of file diff --git a/examples/ibm/00-ark-config.yaml b/examples/aws/05-ark-backupstoragelocation.yaml similarity index 58% rename from examples/ibm/00-ark-config.yaml rename to examples/aws/05-ark-backupstoragelocation.yaml index b37b33e71..04789c478 100644 --- a/examples/ibm/00-ark-config.yaml +++ b/examples/aws/05-ark-backupstoragelocation.yaml @@ -14,21 +14,19 @@ --- apiVersion: ark.heptio.com/v1 -kind: Config +kind: BackupStorageLocation metadata: - namespace: heptio-ark name: default -backupStorageProvider: - name: aws - bucket: - # Uncomment the below line to enable restic integration. - # The format for resticLocation is [/], - # e.g. "my-restic-bucket" or "my-restic-bucket/repos". - # This MUST be a different bucket than the main Ark bucket - # specified just above. - # resticLocation: - config: - region: - s3ForcePathStyle: "true" - s3Url: ---- + namespace: heptio-ark +spec: + provider: aws + objectStorage: + bucket: + config: + region: + # Uncomment the below line to enable restic integration. + # The format for resticLocation is [/], + # e.g. "my-restic-bucket" or "my-restic-bucket/repos". + # This MUST be a different bucket than the main Ark bucket + # specified just above. + # restic-location: diff --git a/examples/azure/05-ark-backupstoragelocation.yaml b/examples/azure/05-ark-backupstoragelocation.yaml new file mode 100644 index 000000000..76c178206 --- /dev/null +++ b/examples/azure/05-ark-backupstoragelocation.yaml @@ -0,0 +1,30 @@ +# Copyright 2018 the Heptio Ark contributors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +--- +apiVersion: ark.heptio.com/v1 +kind: BackupStorageLocation +metadata: + name: default + namespace: heptio-ark +spec: + provider: azure + objectStorage: + bucket: + # Uncomment the below line to enable restic integration. + # The format for resticLocation is [/], + # e.g. "my-restic-bucket" or "my-restic-bucket/repos". + # This MUST be a different bucket than the main Ark bucket + # specified just above. + # restic-location: \ No newline at end of file diff --git a/examples/azure/10-ark-config.yaml b/examples/azure/10-ark-config.yaml index 4a2208ee8..9acb3d1c3 100644 --- a/examples/azure/10-ark-config.yaml +++ b/examples/azure/10-ark-config.yaml @@ -21,13 +21,4 @@ metadata: persistentVolumeProvider: name: azure config: - apiTimeout: -backupStorageProvider: - name: azure - bucket: - # Uncomment the below line to enable restic integration. - # The format for resticLocation is [/], - # e.g. "my-restic-bucket" or "my-restic-bucket/repos". - # This MUST be a different bucket than the main Ark bucket - # specified just above. - # resticLocation: \ No newline at end of file + apiTimeout: \ No newline at end of file diff --git a/examples/gcp/00-ark-config.yaml b/examples/gcp/00-ark-config.yaml index f686a3c19..9ebc5622d 100644 --- a/examples/gcp/00-ark-config.yaml +++ b/examples/gcp/00-ark-config.yaml @@ -19,14 +19,4 @@ metadata: namespace: heptio-ark name: default persistentVolumeProvider: - name: gcp -backupStorageProvider: - name: gcp - bucket: - # Uncomment the below line to enable restic integration. - # The format for resticLocation is [/], - # e.g. "my-restic-bucket" or "my-restic-bucket/repos". - # This MUST be a different bucket than the main Ark bucket - # specified just above. - # resticLocation: - + name: gcp \ No newline at end of file diff --git a/examples/gcp/05-ark-backupstoragelocation.yaml b/examples/gcp/05-ark-backupstoragelocation.yaml new file mode 100644 index 000000000..346aff838 --- /dev/null +++ b/examples/gcp/05-ark-backupstoragelocation.yaml @@ -0,0 +1,30 @@ +# Copyright 2018 the Heptio Ark contributors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +--- +apiVersion: ark.heptio.com/v1 +kind: BackupStorageLocation +metadata: + name: default + namespace: heptio-ark +spec: + provider: gcp + objectStorage: + bucket: + # Uncomment the below line to enable restic integration. + # The format for resticLocation is [/], + # e.g. "my-restic-bucket" or "my-restic-bucket/repos". + # This MUST be a different bucket than the main Ark bucket + # specified just above. + # restic-location: \ No newline at end of file diff --git a/examples/ibm/05-ark-backupstoragelocation.yaml b/examples/ibm/05-ark-backupstoragelocation.yaml new file mode 100644 index 000000000..2e65b45ad --- /dev/null +++ b/examples/ibm/05-ark-backupstoragelocation.yaml @@ -0,0 +1,34 @@ +# Copyright 2018 the Heptio Ark contributors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +--- +apiVersion: ark.heptio.com/v1 +kind: BackupStorageLocation +metadata: + name: default + namespace: heptio-ark +spec: + provider: aws + objectStorage: + bucket: + config: + s3ForcePathStyle: "true" + s3Url: + region: + # Uncomment the below line to enable restic integration. + # The format for resticLocation is [/], + # e.g. "my-restic-bucket" or "my-restic-bucket/repos". + # This MUST be a different bucket than the main Ark bucket + # specified just above. + # restic-location: