store backups & restores in backups/, restores/ subdirs in obj storage

Signed-off-by: Steve Kriss <steve@heptio.com>

docs/storage-layout-reorg-v0.10.md

@@ -0,0 +1,112 @@
+# Object Storage Layout Changes in v0.10
+
+## Overview
+
+Ark v0.10 includes breaking changes to where data is stored in your object storage bucket. You'll need to run a [one-time migration procedure](#upgrading-to-v0.10)
+if you're upgrading from prior versions of Ark.
+
+## Details
+
+Prior to v0.10, Ark stored data in an object storage bucket using the following structure:
+
+```
+<your-bucket>/
+    backup-1/
+        ark-backup.json
+        backup-1.tar.gz
+        backup-1-logs.gz
+        restore-of-backup-1-logs.gz
+        restore-of-backup-1-results.gz
+    backup-2/
+        ark-backup.json
+        backup-2.tar.gz
+        backup-2-logs.gz
+        restore-of-backup-2-logs.gz
+        restore-of-backup-2-results.gz
+    ...
+```
+
+As of v0.10, we've reorganized this layout to provide a cleaner and more extensible directory structure. The new layout looks like:
+
+```
+<your-bucket>[/<your-prefix>]/
+    backups/
+        backup-1/
+            ark-backup.json
+            backup-1.tar.gz
+            backup-1-logs.gz
+        backup-2/
+            ark-backup.json
+            backup-2.tar.gz
+            backup-2-logs.gz
+        ...
+    restores/
+        restore-of-backup-1/
+            restore-of-backup-1-logs.gz
+            restore-of-backup-1-results.gz
+        restore-of-backup-2/
+            restore-of-backup-2-logs.gz
+            restore-of-backup-2-results.gz
+        ...
+    ...
+```
+
+## Upgrading to v0.10
+
+Before upgrading to v0.10, you'll need to run a one-time upgrade script to rearrange the contents of your existing Ark bucket(s) to be compatible with
+the new layout.
+
+Please note that the following scripts **will not** migrate existing restore logs/results into the new `restores/` subdirectory. This means that they
+will not be accessible using `ark restore describe` or `ark restore logs`. They *will* remain in the relevant backup's subdirectory so they are manually
+accessible, and will eventually be garbage-collected along with the backup. We've taken this approach in order to keep the migration scripts simple 
+and less error-prone.
+
+### rclone-Based Script
+
+This script uses [rclone][1], which you can download and install following the instructions [here][2]. 
+Please read through the script carefully before starting and execute it step-by-step.
+
+```bash
+ARK_BUCKET=<your-ark-bucket>
+ARK_TEMP_MIGRATION_BUCKET=<a-temp-bucket-for-migration>
+
+# 1. This is an interactive step that configures rclone to be 
+#    able to access your storage provider. Follow the instructions,
+#    and keep track of the "remote name" for the next step:
+rclone config
+
+# 2. Store the name of the rclone remote that you just set up
+#    in Step #1:
+RCLONE_REMOTE_NAME=<your-remote-name>
+
+# 3. Create a temporary bucket to be used as a backup of your 
+#    current Ark bucket's contents:
+rclone mkdir ${RCLONE_REMOTE_NAME}:${ARK_TEMP_MIGRATION_BUCKET}
+
+# 4. Do a full copy of the contents of your Ark bucket into the 
+#    temporary bucket:
+rclone copy ${RCLONE_REMOTE_NAME}:${ARK_BUCKET} ${RCLONE_REMOTE_NAME}:${ARK_TEMP_MIGRATION_BUCKET}
+
+# 5. Verify that the temporary bucket contains an exact copy of 
+#    your Ark bucket's contents. You should see a short block
+#    of output stating "0 differences found":
+rclone check ${RCLONE_REMOTE_NAME}:${ARK_BUCKET} ${RCLONE_REMOTE_NAME}:${ARK_TEMP_MIGRATION_BUCKET}
+
+# 6. Delete your Ark bucket's contents (this command does not 
+#    delete the bucket itself, only the contents):
+rclone delete ${RCLONE_REMOTE_NAME}:${ARK_BUCKET}
+
+# 7. Copy the contents of the temporary bucket into your Ark bucket, 
+#    under the 'backups/' directory/prefix:
+rclone copy ${RCLONE_REMOTE_NAME}:${ARK_TEMP_MIGRATION_BUCKET} ${RCLONE_REMOTE_NAME}:${ARK_BUCKET}/backups
+
+# 8. Verify that the 'backups/' directory in your Ark bucket now
+#    contains an exact copy of the temporary bucket's contents:
+rclone check ${RCLONE_REMOTE_NAME}:${ARK_BUCKET}/backups ${RCLONE_REMOTE_NAME}:${ARK_TEMP_MIGRATION_BUCKET}
+
+# 9. Once you've confirmed that Ark v0.10 works with your revised Ark
+#    bucket, you can delete the temporary migration bucket.
+```
+
+[1]: https://rclone.org/
+[2]: https://rclone.org/downloads/