diff --git a/changelogs/unreleased/7610-reasonerjt b/changelogs/unreleased/7610-reasonerjt new file mode 100644 index 000000000..be5ac3949 --- /dev/null +++ b/changelogs/unreleased/7610-reasonerjt @@ -0,0 +1 @@ +Add design to introduce restore volume info \ No newline at end of file diff --git a/design/pv_restore_info.md b/design/pv_restore_info.md new file mode 100644 index 000000000..d2e49655c --- /dev/null +++ b/design/pv_restore_info.md @@ -0,0 +1,143 @@ +# Volume information for restore design + +## Background +Velero has different ways to handle data in the volumes during restore. The users want to have more clarity in terms of how +the volumes are handled in restore process via either Velero CLI or other downstream product which consumes Velero. + +## Goals +- Create new metadata to store the information of the restored volume, which will have the same life-cycle as the restore CR. +- Consume the metadata in velero CLI to enable it display more details for volumes in the output of `velero restore describe --details` + +## Non Goals +- Provide finer grained control of the volume restore process. The focus of the design is to enable displaying more details. +- Persist additional metadata like podvolume, datadownloads etc to the restore folder in backup-location. + +## Design + +### Structure of the restore volume info +The restore volume info will be stored in a file named like `${restore_name}-vol-info.json`. The content of the file will +be a list of volume info objects, each of which will map to a volume that is restored, and will contain the information +like name of the restored PV/PVC, restore method and related objects to provide details depending on the way it's restored, +it will look like this: +``` +[ + { + "pvcName": "nginx-logs-2", + "pvcNamespace": "nginx-app-restore", + "pvName": "pvc-e320d75b-a788-41a3-b6ba-267a553efa5e", + "restoreMethod": "PodVolumeRestore", + "snapshotDataMoved": false, + "pvrInfo": { + "snapshotHandle": "81973157c3a945a5229285c931b02c68", + "uploaderType": "kopia", + "volumeName": "nginx-logs", + "podName": "nginx-deployment-79b56c644b-mjdhp", + "podNamespace": "nginx-app-restore" + } + }, + { + "pvcName": "nginx-logs-1", + "pvcNamespace": "nginx-app-restore", + "pvName": "pvc-98c151f4-df47-4980-ba6d-470842f652cc", + "restoreMethod": "CSISnapshot", + "snapshotDataMoved": false, + "csiSnapshotInfo": { + "snapshotHandle": "snap-01a3b21a5e9f85528", + "size": 2147483648, + "driver": "ebs.csi.aws.com", + "vscName": "velero-velero-nginx-logs-1-jxmbg-hx9x5" + } + } +...... +] +``` +Each field will have the same meaning as the corresponding field in the backup volume info. It will not have the fields +that were introduced to help with the backup process, like `pvInfo`, `dataupload` etc. + +### How the restore volume info is generated +Two steps are involved in generating the restore volume info, the first is "collection", which is to gather the information +for restoration of the volumes, the second is "generation", which is to iterate through the data collected in the first step +and generate the volume info list as is described above. + +Unlike backup, the CR objects created during the restore process will not be persisted to the backup storage location. +Therefore, to gather the information needed to generate volume information, we either need to collect the CRs in the middle +of the restore process, or retrieve the objects based on the `resouce-list.json` of the restore via API server. +The information to be collected are: +- **PV/PVC mapping relationship:** It will be collected via the `restore-resource-list.json`, b/c at the time the json is ready, all +PVCs and PVs are already created. +- **Native snapshot information:** It will be collected in the restore workflow when each snapshot is restored. +- **podvolumerestore CRs:** It will be collected in the restore workflow after each pvr is created. +- **volumesnapshot CRs for CSI snapshot:** It will be collected in the step of collecting PVC info, by reading the `dataSource` +field in the spec of the PVC. +- **datadownload CRs** It will be collected in the phase of collecting PVC info, by querying the API-server to list the datadownload +CRs labeled with the restore name. + +After the collection step, the generation step is relatively straight-forward, as we have all the information needed in +the data structures. + +The whole collection and generation steps will be done with the "best-effort" manner, i.e. if there are any failures we +will only log the error in restore log, rather than failing the whole restore process, we will not put these errors or warnings +into the `result.json`, b/c it won't impact the restored resources. + +Depending on the number of the restored PVCs the "collection" step may involve many API calls, but it's considered acceptable +b/c at that time the resources are already created, so the actual RTO is not impacted. By using the client of controller runtime +we can make the collection step more efficient by using the cache of the API server. We may consider to make improvements if +we observe performance issues, like using multiple go-routines in the collection. + +### Implementation +Because the restore volume info shares the same data structures with the backup volume info, we will refactor the code in +package `internal/volume` to make the sub-components in backup volume info shared by both backup and restore volume info. + +We'll introduce a struct called `RestoreVolumeInfoTracker` which encapsulates the logic of collecting and generating the restore volume info: +``` +// RestoreVolumeInfoTracker is used to track the volume information during restore. +// It is used to generate the RestoreVolumeInfo array. +type RestoreVolumeInfoTracker struct { + *sync.Mutex + restore *velerov1api.Restore + log logrus.FieldLogger + client kbclient.Client + pvPvc *pvcPvMap + + // map of PV name to the NativeSnapshotInfo from which the PV is restored + pvNativeSnapshotMap map[string]NativeSnapshotInfo + // map of PV name to the CSISnapshot object from which the PV is restored + pvCSISnapshotMap map[string]snapshotv1api.VolumeSnapshot + datadownloadList *velerov2alpha1.DataDownloadList + pvrs []*velerov1api.PodVolumeRestore +} +``` +The `RestoreVolumeInfoTracker` will be created when the restore request is initialized, and it will be passed to the `restoreContext` +and carried over the whole restore process. + +The `client` in this struct is to be used to query the resources in the restored namespace, and the current client in restore +reconciler only watches the resources in the namespace where velero is installed. Therefore, we need to introduce the +`CrClient` which has the same life-cycle of velero server to the restore reconciler, because this is the client that watches all the +resources on the cluster. + +In addition to that, we will make small changes in the restore workflow to collect the information needed. We'll make the +changes un-intrusive and make sure not to change the logic of the restore to avoid break change or regression. +We'll also introduce routine changes in the package `pkg/persistence` to persist the restore volume info to the backup storage location. + +Last but not least, the `velero restore describe --details` will be updated to display the volume info in the output. + +## Alternatives Considered +There used to be suggestion that to provide more details about volume, we can query the `backup-vol-info.json` with the resource +identifier in `restore-resource-list.json`. This will not work when there're resource modifiers involved in the restore process, +which may change the metadata of PVC/PV. In addition, we may add more detailed restore-specific information about the volumes that is not available +in the `backup-vol-info.json`. Therefore, the `restore-vol-info.json` is a better approach. + +## Security Considerations +There should be no security impact introduced by this design. + +## Compatibility +The restore volume info will be consumed by Velero CLI and downstream products for displaying details. So the functionality +of backup and restore will not be impacted for restores created by older versions of Velero which do not have the restore volume info +metadata. The client should properly handle the case when the restore volume info does not exist. + +The data structures referenced by volume info is shared between both restore and backup and it's not versioned, so in the future +we must make sure there will only be incremental changes to the metadata, such that no break change will be introduced to the client. + +## Open Issues +https://github.com/vmware-tanzu/velero/issues/7546 +https://github.com/vmware-tanzu/velero/issues/6478