diff --git a/docs/build-from-scratch.md b/docs/build-from-scratch.md index fa5e6aee8..7e81697ed 100644 --- a/docs/build-from-scratch.md +++ b/docs/build-from-scratch.md @@ -44,6 +44,7 @@ These include: * Listers * Shared informers * Documentation +* Protobuf/gRPC types If you make any of the following changes, you will need to run `make update` to regenerate automatically generated files: @@ -51,6 +52,10 @@ automatically generated files: * Add/edit/remove commands or subcommands * Add new API types +If you make the following change, you will need to run [generate-proto.sh][13] to regenerate +automatically generated files (note that this requires the [proto compiler][14] to be installed): +* Add/edit/remove protobuf message or service definitions + ### Cross compiling By default, `make` will build an `ark` binary that runs on your host operating system and @@ -109,3 +114,5 @@ If you need to add or update the vendored dependencies, please see [Vendoring de [10]: #4-vendoring-dependencies [11]: vendoring-dependencies.md [12]: #3-test +[13]: ../hack/generate-proto.sh +[14]: https://grpc.io/docs/quickstart/go.html#install-protocol-buffers-v3 \ No newline at end of file diff --git a/docs/config-definition.md b/docs/config-definition.md index e30e7ecaa..ed22bc87b 100644 --- a/docs/config-definition.md +++ b/docs/config-definition.md @@ -24,11 +24,13 @@ metadata: namespace: heptio-ark name: default persistentVolumeProvider: - aws: + name: aws + config: region: us-west-2 backupStorageProvider: + name: aws bucket: ark - aws: + config: region: us-west-2 backupSyncPeriod: 60m gcSyncPeriod: 60m @@ -44,9 +46,13 @@ The configurable parameters are as follows: | Key | Type | Default | Meaning | | --- | --- | --- | --- | -| `persistentVolumeProvider` | CloudProviderConfig

(Supported key values are `aws`, `gcp`, and `azure`, but only one can be present. See the corresponding [AWS][0], [GCP][1], and [Azure][2]-specific configs.) | None (Optional) | The specification for whichever cloud provider the cluster is using for persistent volumes (to be snapshotted), if any.

If not specified, Backups and Restores requesting PV snapshots & restores, respectively, are considered invalid.

*NOTE*: For Azure, your Kubernetes cluster needs to be version 1.7.2+ in order to support PV snapshotting of its managed disks. | -| `backupStorageProvider`/(inline) | CloudProviderConfig

(Supported key values are `aws`, `gcp`, and `azure`, but only one can be present. See the corresponding [AWS][0], [GCP][1], and [Azure][2]-specific configs.) | Required Field | The specification for whichever cloud provider will be used to actually store the backups. | +| `persistentVolumeProvider` | CloudProviderConfig | None (Optional) | The specification for whichever cloud provider the cluster is using for persistent volumes (to be snapshotted), if any.

If not specified, Backups and Restores requesting PV snapshots & restores, respectively, are considered invalid.

*NOTE*: For Azure, your Kubernetes cluster needs to be version 1.7.2+ in order to support PV snapshotting of its managed disks. | +| `persistentVolumeProvider/name` | String

(Ark natively supports `aws`, `gcp`, and `azure`. Other providers may be available via external plugins.) | None (Optional) | The name of the cloud provider the cluster is using for persistent volumes, if any. | +| `persistentVolumeProvider/config` | map[string]string

(See the corresponding [AWS][0], [GCP][1], and [Azure][2]-specific configs or your provider's documentation.) | None (Optional) | Configuration keys/values to be passed to the cloud provider for persistent volumes. | +| `backupStorageProvider` | CloudProviderConfig | Required Field | The specification for whichever cloud provider will be used to actually store the backups. | +| `backupStorageProvider/name` | String

(Ark natively supports `aws`, `gcp`, and `azure`. Other providers may be available via external plugins.) | Required Field | The name of the cloud provider that will be used to actually store the backups. | | `backupStorageProvider/bucket` | String | Required Field | The storage bucket where backups are to be uploaded. | +| `backupStorageProvider/config` | map[string]string

(See the corresponding [AWS][0], [GCP][1], and [Azure][2]-specific configs or your provider's documentation.) | None (Optional) | Configuration keys/values to be passed to the cloud provider for backup storage. | | `backupSyncPeriod` | metav1.Duration | 60m0s | How frequently Ark queries the object storage to make sure that the appropriate Backup resources have been created for existing backup files. | | `gcSyncPeriod` | metav1.Duration | 60m0s | How frequently Ark queries the object storage to delete backup files that have passed their TTL. | | `scheduleSyncPeriod` | metav1.Duration | 1m0s | How frequently Ark checks its Schedule resource objects to see if a backup needs to be initiated. | @@ -57,17 +63,16 @@ The configurable parameters are as follows: **(Or other S3-compatible storage)** -#### backupStorageProvider +#### backupStorageProvider/config | Key | Type | Default | Meaning | | --- | --- | --- | --- | | `region` | string | Required Field | *Example*: "us-east-1"

See [AWS documentation][3] for the full list. | -| `disableSSL` | bool | `false` | Set this to `true` if you are using Minio (or another local, S3-compatible storage service) and your deployment is not secured. | | `s3ForcePathStyle` | bool | `false` | Set this to `true` if you are using a local storage service like Minio. | | `s3Url` | string | Required field for non-AWS-hosted storage| *Example*: http://minio:9000

You can specify the AWS S3 URL here for explicitness, but Ark can already generate it from `region`, and `bucket`. This field is primarily for local storage services like Minio.| | `kmsKeyId` | string | Empty | *Example*: "502b409c-4da1-419f-a16e-eif453b3i49f" or "alias/``"

Specify an [AWS KMS key][10] id or alias to enable encryption of the backups stored in S3. Only works with AWS S3 and may require explicitly granting key usage rights.| -#### persistentVolumeProvider (AWS Only) +#### persistentVolumeProvider/config (AWS Only) | Key | Type | Default | Meaning | | --- | --- | --- | --- | @@ -75,11 +80,11 @@ The configurable parameters are as follows: ### GCP -#### backupStorageProvider +#### backupStorageProvider/config -No parameters required; specify an empty object per [example file][11]. +No parameters required. -#### persistentVolumeProvider +#### persistentVolumeProvider/config | Key | Type | Default | Meaning | | --- | --- | --- | --- | @@ -87,11 +92,11 @@ No parameters required; specify an empty object per [example file][11]. ### Azure -#### backupStorageProvider +#### backupStorageProvider/config -No parameters required; specify an empty object per [example file][12]. +No parameters required. -#### persistentVolumeProvider +#### persistentVolumeProvider/config | Key | Type | Default | Meaning | | --- | --- | --- | --- | diff --git a/examples/aws/00-ark-config.yaml b/examples/aws/00-ark-config.yaml index 6b668fb8e..3780e2a01 100644 --- a/examples/aws/00-ark-config.yaml +++ b/examples/aws/00-ark-config.yaml @@ -19,11 +19,13 @@ metadata: namespace: heptio-ark name: default persistentVolumeProvider: - aws: + name: aws + config: region: backupStorageProvider: + name: aws bucket: - aws: + config: region: backupSyncPeriod: 30m gcSyncPeriod: 30m diff --git a/examples/azure/10-ark-config.yaml b/examples/azure/10-ark-config.yaml index 4edcdfe37..75c8fe221 100644 --- a/examples/azure/10-ark-config.yaml +++ b/examples/azure/10-ark-config.yaml @@ -19,12 +19,13 @@ metadata: namespace: heptio-ark name: default persistentVolumeProvider: - azure: + name: azure + config: location: apiTimeout: backupStorageProvider: + name: azure bucket: - azure: {} backupSyncPeriod: 30m gcSyncPeriod: 30m scheduleSyncPeriod: 1m diff --git a/examples/gcp/00-ark-config.yaml b/examples/gcp/00-ark-config.yaml index 6616e9656..2ee4cc119 100644 --- a/examples/gcp/00-ark-config.yaml +++ b/examples/gcp/00-ark-config.yaml @@ -19,11 +19,12 @@ metadata: namespace: heptio-ark name: default persistentVolumeProvider: - gcp: + name: gcp + config: project: backupStorageProvider: + name: gcp bucket: - gcp: {} backupSyncPeriod: 30m gcSyncPeriod: 30m scheduleSyncPeriod: 1m diff --git a/examples/minio/10-ark-config.yaml b/examples/minio/10-ark-config.yaml index 66a6dce55..4b3451177 100644 --- a/examples/minio/10-ark-config.yaml +++ b/examples/minio/10-ark-config.yaml @@ -19,8 +19,9 @@ metadata: namespace: heptio-ark name: default backupStorageProvider: + name: aws bucket: ark - aws: + config: region: minio s3ForcePathStyle: true s3Url: http://minio:9000