Michael Steven FruchtmanandGitHub
64d8014b87
* Correcting Openshift on IBM Documentation Error I have to admit to some significant error and embarrassment regarding the documentation update about Openshift on IBM Cloud pull request https://github.com/vmware-tanzu/velero/pull/8069. I will correct my error before it gets any further. Just exposing /var/data/kubelet/pods is incorrect and host path /var/lib/kubelet/pods should remain unchanged. The errors with the defaults during csi snapshot data movement were: data path backup failed: Failed to run kopia backup: unable to get local block device entry: resolveSymlink: lstat /var/data/: no such file or directory I suspected this was the same as RancherOS and Nutanix. It is not. The original tested changes changed both /var/lib/kubelet/{pods,plugins} to /var/data/kubelet/{pods,plugins}. The published changes only result in the error: ``` status: completionTimestamp: '2024-08-02T17:12:29Z' message: >- data path backup failed: Failed to run kopia backup: unable to get local block device entry: resolveSymlink: lstat /var/data/kubelet/plugins: no such file or directory node: 10.240.0.5 phase: Failed progress: {} startTimestamp: '2024-08-02T17:12:11Z' ``` After making continued modifications to the daemonset the correct configuration was: ``` volumeMounts: - name: host-pods mountPath: /host_pods mountPropagation: HostToContainer - name: host-plugins mountPath: /var/data/kubelet/plugins mountPropagation: HostToContainer ``` ``` volumes: - name: host-pods hostPath: path: /var/lib/kubelet/pods type: '' - name: host-plugins hostPath: path: /var/data/kubelet/plugins type: '' ``` Only the changes to the plugin path were required. The plugin path changes were required to both the mount path and the host path. Regardless of whether /var/lib/kubelet/pods or /var/data/kubelet/pods host path, backups and restore succeeded provided the plugin path was modified. ``` volumeMounts: - name: host-pods mountPath: /host_pods mountPropagation: HostToContainer - name: host-plugins mountPath: /var/data/kubelet/plugins mountPropagation: HostToContainer ``` ``` volumes: - name: host-pods hostPath: path: /var/data/kubelet/pods type: '' - name: host-plugins hostPath: path: /var/data/kubelet/plugins type: '' ``` After getting on-host access was able to confirm. Pods are at /var/lib/kubelet/pods. ``` ls /var/lib/kubelet/pods 07c0be63-335d-4cfb-b39f-816bc2fb32cd 51f31b3e-4710-4ef0-8626-5f1a78a624b2 a4802fd3-3b62-45a4-8f21-974880b6f92a cccb35c9-b4f9-4ca9-a697-736ae64f09ad 0a5d4366-7fa1-4525-9e45-a43a362b8542 558b0643-0661-4d4a-b03e-aac60c6ad710 a4b106fb-5b7b-48e5-828a-ea7b41ba0e59 ce1290e1-4330-4df6-8166-14784bcce930 ``` On host the volumes are in /var/data/kubelet/plugins. ``` ls /var/data/kubelet/plugins/kubernetes.io/csi/openshift-storage.cephfs.csi.ceph.com/231e04896c4f528efb95d23a3c153db9fc4a7206b7320f74443f30de7228dba5/globalmount/velero/backups/backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c/ backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-csi-volumesnapshotclasses.json.gz backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-resource-list.json.gz backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-csi-volumesnapshotcontents.json.gz backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-results.gz backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-csi-volumesnapshots.json.gz backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-volumesnapshots.json.gz backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-itemoperations.json.gz backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c.tar.gz backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-logs.gz velero-backup.json backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-podvolumebackups.json.gz ``` With the volume config changed to expose /var/data/kubelet/plugins for the plugin hostPath, the DataUploads and DataDownloads succeed for both Filesystem and Block mode PVCs. ``` status: completionTimestamp: '2024-08-02T17:23:33Z' node: 10.240.0.5 path: >- /host_pods/7fcb9d56-7885-437c-acd3-67db6b1ee8ae/volumeDevices/kubernetes.io~csi/pvc-47b91f56-db8c-44bf-9ecc-737170561b4b phase: Completed progress: bytesDone: 5368709120 totalBytes: 5368709120 snapshotID: 8faae36b3592fee4efbfad024f26033e startTimestamp: '2024-08-02T17:21:22Z' ``` ``` status: completionTimestamp: '2024-08-02T18:42:19Z' node: 10.240.0.5 phase: Completed progress: bytesDone: 5368709120 totalBytes: 5368709120 startTimestamp: '2024-08-02T18:41:00Z' ``` My apologies for the error. Signed-off-by: Michael Fruchtman <msfrucht@us.ibm.com> * Add context to plugins mountPath Signed-off-by: Michael Fruchtman <msfrucht@us.ibm.com> --------- Signed-off-by: Michael Fruchtman <msfrucht@us.ibm.com>
Overview
Velero (formerly Heptio Ark) gives you tools to back up and restore your Kubernetes cluster resources and persistent volumes. You can run Velero with a public cloud platform or on-premises.
Velero lets you:
- Take backups of your cluster and restore in case of loss.
- Migrate cluster resources to other clusters.
- Replicate your production cluster to development and testing clusters.
Velero consists of:
- A server that runs on your cluster
- A command-line client that runs locally
Documentation
The documentation provides a getting started guide and information about building from source, architecture, extending Velero and more.
Please use the version selector at the top of the site to ensure you are using the appropriate documentation for your version of Velero.
Troubleshooting
If you encounter issues, review the troubleshooting docs, file an issue, or talk to us on the #velero channel on the Kubernetes Slack server.
Contributing
If you are ready to jump in and test, add code, or help with documentation, follow the instructions on our Start contributing documentation for guidance on how to setup Velero for development.
Changelog
See the list of releases to find out about feature changes.
Velero compatibility matrix
The following is a list of the supported Kubernetes versions for each Velero version.
| Velero version | Expected Kubernetes version compatibility | Tested on Kubernetes version |
|---|---|---|
| 1.14 | 1.18-latest | 1.27.9, 1.28.9, and 1.29.4 |
| 1.13 | 1.18-latest | 1.26.5, 1.27.3, 1.27.8, and 1.28.3 |
| 1.12 | 1.18-latest | 1.25.7, 1.26.5, 1.26.7, and 1.27.3 |
| 1.11 | 1.18-latest | 1.23.10, 1.24.9, 1.25.5, and 1.26.1 |
| 1.10 | 1.18-latest | 1.22.5, 1.23.8, 1.24.6 and 1.25.1 |
Velero supports IPv4, IPv6, and dual stack environments. Support for this was tested against Velero v1.8.
The Velero maintainers are continuously working to expand testing coverage, but are not able to test every combination of Velero and supported Kubernetes versions for each Velero release. The table above is meant to track the current testing coverage and the expected supported Kubernetes versions for each Velero version.
If you are interested in using a different version of Kubernetes with a given Velero version, we'd recommend that you perform testing before installing or upgrading your environment. For full information around capabilities within a release, also see the Velero release notes or Kubernetes release notes. See the Velero support page for information about supported versions of Velero.
For each release, Velero maintainers run the test to ensure the upgrade path from n-2 minor release. For example, before the release of v1.10.x, the test will verify that the backup created by v1.9.x and v1.8.x can be restored using the build to be tagged as v1.10.x.