mirror of
https://github.com/vmware-tanzu/velero.git
synced 2026-08-15 11:46:06 +00:00
Add document describing manual test cases (#3601)
* Add document describing manual test cases This introduces a new document, `TESTING.md`, which describes manual tests that are currently run as part of a Velero release and test cases that we will want to introduce for future releases. Signed-off-by: Bridget McErlean <bmcerlean@vmware.com> * Move testing requirements doc to website Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
This commit is contained in:
Bridget McErlean
@@ -24,6 +24,8 @@ Some tips for doing reviews:
|
||||
|
||||
## Creating a release
|
||||
Maintainers are expected to create releases for the project. We have parts of the process automated, and full [instructions](release-instructions).
|
||||
We are working towards automating more the Velero testing, but there is still a need for manual testing as part of the release process.
|
||||
The manual test cases for release testing are documented [here](./manual-testing).
|
||||
|
||||
## Community support
|
||||
Maintainers are expected to participate in the community support rotation. We have guidelines for how we handle the [support](support-process).
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
---
|
||||
title: "Manual Testing Requirements for Velero"
|
||||
layout: docs
|
||||
---
|
||||
|
||||
Although we have automated unit and end-to-end tests, there is still a need for Velero to undergo manual tests during a release.
|
||||
This document outlines the manual test operations that Velero needs to correctly perform in order to be considered ready for release.
|
||||
|
||||
## Current test cases
|
||||
|
||||
The following are test cases that are currently performed as part of a Velero release.
|
||||
|
||||
### Install
|
||||
|
||||
- Verify that Velero CRDs are compatible with the earliest and latest versions of Kubernetes that we support:
|
||||
- Kubernetes v1.12
|
||||
- Kubernetes v1.20
|
||||
|
||||
### Upgrade
|
||||
|
||||
- Verify that Velero upgrade instructions work
|
||||
|
||||
### Basic functionality
|
||||
|
||||
The "Backup and Restore" test cases below describe general backup and restore functionality that needs to run successfully on all the following providers that we maintain plugins for:
|
||||
- AWS
|
||||
- GCP
|
||||
- Microsoft Azure
|
||||
- VMware vSphere
|
||||
|
||||
#### Backup and Restore
|
||||
|
||||
- Verify that a backup and restore using Volume Snapshots can be performed
|
||||
- Verify that a backup and restore using Restic can be performed
|
||||
- Verify that a backup of a cluster workload can be restored in a new cluster
|
||||
- Verify that an installation using the latest version can be used to restore from backups created with the last 3 versions.
|
||||
- e.g. Install Velero 1.6 and use it to restore backups from Velero v1.3, v1.4, v1.5.
|
||||
|
||||
### Working with Multiple Providers
|
||||
|
||||
The following are test cases that exercise Velero behaviour when interacting with multiple providers:
|
||||
|
||||
- Verify that a backup and restore to multiple BackupStorageLocations using the same provider with unique credentials can be performed
|
||||
- Verify that a backup and restore to multiple BackupStorageLocations using different providers with unique credentials can be performed
|
||||
- Verify that a backup and restore that includes volume snapshots using different providers for the snapshots and object storage can be performed
|
||||
- e.g. perform a backup and restore using AWS for the VolumeSnapshotLocation and Azure Blob Storage as the BackupStorageLocation
|
||||
|
||||
## Future test cases
|
||||
|
||||
The following are test cases that are not currently performed as part of a Velero release but cases that we will want to cover with future releases.
|
||||
|
||||
### Schedules
|
||||
|
||||
- Verify that schedules create a backup upon creation and create Backup resources at the correct frequency
|
||||
|
||||
### Resource management
|
||||
|
||||
- Verify that deleted backups are successfully removed from object storage
|
||||
- Verify that backups that have been removed from object storage can still be deleted with `velero delete backup`
|
||||
- Verify that Volume Snapshots associated with a deleted backup are removed
|
||||
- Verify that backups that exceed their TTL are deleted
|
||||
- Verify that existing backups in object storage are synced to Velero
|
||||
|
||||
### Restic repository test cases
|
||||
|
||||
- Verify that restic repository maintenance is performed as the specified interval
|
||||
|
||||
### Backup Hooks
|
||||
|
||||
- Verify that a pre backup hook provided via pod annotation is performed during backup
|
||||
- Verify that a pre backup hook provided via Backup spec is performed during backup
|
||||
- Verify that a post backup hook provided via pod annotation is performed during backup
|
||||
- Verify that a post backup hook provided via Backup spec is performed during backup
|
||||
|
||||
### Restore Hooks
|
||||
|
||||
- Verify that an InitContainer restore hook provided via pod annotation is performed during restore
|
||||
- Verify that an InitContainer restore hook provided via Restore spec is performed during restore
|
||||
- Verify that an InitContainer restore hook provided via Restore spec is performed during restore that includes restoring restic volumes
|
||||
- Verify that an Exec restore hook provided via pod annotation is performed during restore
|
||||
- Verify that an Exec restore hook provided via Restore spec is performed during restore
|
||||
|
||||
|
||||
#### Resource filtering
|
||||
|
||||
- Verify that backups and restores correctly apply the following resource filters:
|
||||
- `--include-namespaces`
|
||||
- `--include-resources`
|
||||
- `--include-cluster-resources`
|
||||
- `--exclude-namespaces`
|
||||
- `--exclude-resources`
|
||||
- `velero.io/exclude-from-backup=true` label
|
||||
@@ -24,6 +24,8 @@ Some tips for doing reviews:
|
||||
|
||||
## Creating a release
|
||||
Maintainers are expected to create releases for the project. We have parts of the process automated, and full [instructions](release-instructions).
|
||||
We are working towards automating more the Velero testing, but there is still a need for manual testing as part of the release process.
|
||||
The manual test cases for release testing are documented [here](./manual-testing).
|
||||
|
||||
## Community support
|
||||
Maintainers are expected to participate in the community support rotation. We have guidelines for how we handle the [support](support-process).
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
---
|
||||
title: "Manual Testing Requirements for Velero"
|
||||
layout: docs
|
||||
---
|
||||
|
||||
Although we have automated unit and end-to-end tests, there is still a need for Velero to undergo manual tests during a release.
|
||||
This document outlines the manual test operations that Velero needs to correctly perform in order to be considered ready for release.
|
||||
|
||||
## Current test cases
|
||||
|
||||
The following are test cases that are currently performed as part of a Velero release.
|
||||
|
||||
### Install
|
||||
|
||||
- Verify that Velero CRDs are compatible with the earliest and latest versions of Kubernetes that we support:
|
||||
- Kubernetes v1.12
|
||||
- Kubernetes v1.20
|
||||
|
||||
### Upgrade
|
||||
|
||||
- Verify that Velero upgrade instructions work
|
||||
|
||||
### Basic functionality
|
||||
|
||||
The "Backup and Restore" test cases below describe general backup and restore functionality that needs to run successfully on all the following providers that we maintain plugins for:
|
||||
- AWS
|
||||
- GCP
|
||||
- Microsoft Azure
|
||||
- VMware vSphere
|
||||
|
||||
#### Backup and Restore
|
||||
|
||||
- Verify that a backup and restore using Volume Snapshots can be performed
|
||||
- Verify that a backup and restore using Restic can be performed
|
||||
- Verify that a backup of a cluster workload can be restored in a new cluster
|
||||
- Verify that an installation using the latest version can be used to restore from backups created with the last 3 versions.
|
||||
- e.g. Install Velero 1.6 and use it to restore backups from Velero v1.3, v1.4, v1.5.
|
||||
|
||||
### Working with Multiple Providers
|
||||
|
||||
The following are test cases that exercise Velero behaviour when interacting with multiple providers:
|
||||
|
||||
- Verify that a backup and restore to multiple BackupStorageLocations using the same provider with unique credentials can be performed
|
||||
- Verify that a backup and restore to multiple BackupStorageLocations using different providers with unique credentials can be performed
|
||||
- Verify that a backup and restore that includes volume snapshots using different providers for the snapshots and object storage can be performed
|
||||
- e.g. perform a backup and restore using AWS for the VolumeSnapshotLocation and Azure Blob Storage as the BackupStorageLocation
|
||||
|
||||
## Future test cases
|
||||
|
||||
The following are test cases that are not currently performed as part of a Velero release but cases that we will want to cover with future releases.
|
||||
|
||||
### Schedules
|
||||
|
||||
- Verify that schedules create a backup upon creation and create Backup resources at the correct frequency
|
||||
|
||||
### Resource management
|
||||
|
||||
- Verify that deleted backups are successfully removed from object storage
|
||||
- Verify that backups that have been removed from object storage can still be deleted with `velero delete backup`
|
||||
- Verify that Volume Snapshots associated with a deleted backup are removed
|
||||
- Verify that backups that exceed their TTL are deleted
|
||||
- Verify that existing backups in object storage are synced to Velero
|
||||
|
||||
### Restic repository test cases
|
||||
|
||||
- Verify that restic repository maintenance is performed as the specified interval
|
||||
|
||||
### Backup Hooks
|
||||
|
||||
- Verify that a pre backup hook provided via pod annotation is performed during backup
|
||||
- Verify that a pre backup hook provided via Backup spec is performed during backup
|
||||
- Verify that a post backup hook provided via pod annotation is performed during backup
|
||||
- Verify that a post backup hook provided via Backup spec is performed during backup
|
||||
|
||||
### Restore Hooks
|
||||
|
||||
- Verify that an InitContainer restore hook provided via pod annotation is performed during restore
|
||||
- Verify that an InitContainer restore hook provided via Restore spec is performed during restore
|
||||
- Verify that an InitContainer restore hook provided via Restore spec is performed during restore that includes restoring restic volumes
|
||||
- Verify that an Exec restore hook provided via pod annotation is performed during restore
|
||||
- Verify that an Exec restore hook provided via Restore spec is performed during restore
|
||||
|
||||
|
||||
#### Resource filtering
|
||||
|
||||
- Verify that backups and restores correctly apply the following resource filters:
|
||||
- `--include-namespaces`
|
||||
- `--include-resources`
|
||||
- `--include-cluster-resources`
|
||||
- `--exclude-namespaces`
|
||||
- `--exclude-resources`
|
||||
- `velero.io/exclude-from-backup=true` label
|
||||
Reference in New Issue
Block a user