Commit Graph

4914 Commits

Author SHA1 Message Date
Lyndon-Li
a80c9359bf bump up kopia
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-29 13:10:08 +08:00
lyndon-li
981f30cb25 Merge pull request #8151 from reasonerjt/update-stale-exempt
Issues with "backlog" label should never stale
2024-08-26 17:42:45 +08:00
Daniel Jiang
3f1853c961 Issues with "backlog" label should never stale
Signed-off-by: Daniel Jiang <daniel.jiang@broadcom.com>
2024-08-26 15:07:40 +08:00
Tiger Kaovilai
f5671c728c Scrub namespace terminating status and deletion timestamp on restore. Descriptive restore error on terminating namespace. (#7424)
revert utils_test.go



address c7b189dd60 (r1494194484)



Update pkg/util/kube/utils.go

Signed-off-by: Tiger Kaovilai <passawit.kaovilai@gmail.com>
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2024-08-23 16:45:10 +05:30
lyndon-li
de96d4c84b Merge pull request #8139 from blackpiglet/7579_fix
Add resource modifier for velero restore describe CLI
2024-08-23 13:21:03 +08:00
Xun Jiang
c2cd6b7176 Add resource modifier for velero restore describe CLI
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-08-22 00:01:56 +08:00
Shubham Pampattiwar
934b3ea6a9 Merge pull request #8131 from Lyndon-Li/backup-repo-config-doc
Add doc for backup repo config
2024-08-21 06:22:59 -07:00
Lyndon-Li
37e0ab12cc backup repo config doc
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-21 16:15:40 +08:00
Lyndon-Li
4120d43b78 Merge branch 'main' into backup-repo-config-doc 2024-08-21 10:22:22 +08:00
lyndon-li
f63b714483 Merge pull request #8115 from Lyndon-Li/data-mover-ms-smoking-test
Data mover micro service smoke testing
2024-08-21 10:12:18 +08:00
Xun Jiang/Bruce Jiang
ec6090bd01 Merge pull request #8129 from vmware-tanzu/e2e_modification
Modify E2E and perf test report generated directory
2024-08-21 07:22:46 +08:00
Shubham Pampattiwar
6e65c73cc6 Merge pull request #8119 from shubham-pampattiwar/backup-pvc-config-docs
Add docs for backup pvc config support
2024-08-20 10:29:39 -07:00
Lyndon-Li
bdff60178a add doc for backup repo config
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-20 14:28:12 +08:00
Xun Jiang
af62dd4b3e Modify E2E and perf test result output directory.
Add LongTime label to more E2E cases.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-08-20 10:58:32 +08:00
Shubham Pampattiwar
86963bf229 Merge pull request #8097 from Lyndon-Li/issue-fix-8032
Issue 8032: make node agent configMap name configurable
2024-08-19 14:49:43 -07:00
Shubham Pampattiwar
d4e7d1472e add docs for backup pvc config support
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

add changelog

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

add section to csi dm doc and minor fixes

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

configMap name is configurable

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2024-08-19 14:39:22 -07:00
Matthieu MOREL
a6c543384b Use native cache from actions/setup-go (#7768)
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-08-19 14:45:59 -04:00
Daniel Jiang
8e1dc8e997 Merge pull request #8108 from shubham-pampattiwar/fix-docker-file
Minor fixes to Dockerfile and docs
2024-08-19 14:01:31 +08:00
Lyndon-Li
8cf1749ae0 issue 8032: make node agent configMap name configurable
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-19 10:33:17 +08:00
Shubham Pampattiwar
a9463cebe4 Merge pull request #8117 from blackpiglet/remove_code_generator
Remove code-generator from hack/update-3generated-crd.code.sh
2024-08-15 09:02:24 -07:00
Xun Jiang
c0402075fb Remove code-generator from hack/update-3generated-crd.code.sh
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-08-15 17:27:30 +08:00
Lyndon-Li
0ed1a7fc86 data mover ms smoke testing
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-15 15:06:31 +08:00
Lyndon-Li
d25a908b78 Merge branch 'main' into data-mover-ms-smoking-test 2024-08-15 11:12:44 +08:00
lyndon-li
8fde4a017d Merge pull request #8054 from sseago/iba-plugins
Iba plugins
2024-08-15 10:21:25 +08:00
lyndon-li
4e781d4009 Merge pull request #8109 from shubham-pampattiwar/backup-pvc-config-support
Add support for backup PVC configuration
2024-08-15 10:20:12 +08:00
Lyndon-Li
ed0ef67c16 data mover ms smoke testing
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-14 17:29:04 +08:00
Shubham Pampattiwar
6c3988e462 Merge pull request #8114 from blackpiglet/6190_remove_client
Delete the pkg/generated directory.
2024-08-13 21:32:05 -07:00
Xun Jiang
4ffc6d17b2 Delete the pkg/generated directory.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-08-14 10:35:23 +08:00
Shubham Pampattiwar
8eac3606d9 Add support for backup PVC configuration
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

add changelog file

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

make update

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

pass backupPVCConfig to exposer as part of csi params

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2024-08-13 17:06:06 -07:00
Lyndon-Li
3c0948c9be Merge branch 'main' into data-mover-ms-smoking-test 2024-08-14 00:15:56 +08:00
lyndon-li
07c03a8919 Merge pull request #8085 from Lyndon-Li/data-mover-ms-node-agent-resume
Data mover micro service node agent resume
2024-08-14 00:14:47 +08:00
Shubham Pampattiwar
b62b38f566 Merge pull request #8093 from Lyndon-Li/backkup-repo-config
Backup repo config
2024-08-12 13:44:00 -07:00
Shubham Pampattiwar
3aabfc3414 Minor fixes to Dockerfile and docs
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2024-08-12 13:28:19 -07:00
Daniel Jiang
260a4995c2 Merge pull request #8096 from Lyndon-Li/issue-fix-8072
Issue 8072: restic deprecation - warning messages
2024-08-12 14:13:25 +08:00
Lyndon-Li
04db3ba767 data mover ms node agent resume
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-12 10:45:59 +08:00
Scott Seago
1228b41851 Internal ItemBlockAction plugins
This PR implements the internal ItemBlockAction plugins needed for pod, PVC, and SA.

Signed-off-by: Scott Seago <sseago@redhat.com>
2024-08-09 12:24:55 -04:00
lyndon-li
cc32375b76 Merge pull request #8098 from reasonerjt/restore-status-doc
Add information about restore status to the doc
2024-08-09 18:24:54 +08:00
Daniel Jiang
8ca7cae662 Add information about restore status to the doc
fixes #6237

Signed-off-by: Daniel Jiang <daniel.jiang@broadcom.com>
2024-08-09 16:23:30 +08:00
Lyndon-Li
4dea3a48e8 data mover ms smoking test
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-09 15:15:21 +08:00
Lyndon-Li
fefb4b858c issue 8072: restic deprecation - warning messages
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-09 14:40:42 +08:00
Lyndon-Li
2c7047a304 Merge branch 'main' into data-mover-ms-node-agent-resume
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-07 17:23:15 +08:00
lyndon-li
60f5ad5cf4 Merge branch 'main' into data-mover-ms-node-agent-resume
Signed-off-by: lyndon-li <98304688+Lyndon-Li@users.noreply.github.com>
2024-08-07 17:13:35 +08:00
Lyndon-Li
3b06d915ca Merge branch 'main' into data-mover-ms-node-agent-resume 2024-08-07 17:07:38 +08:00
lyndon-li
dd3d05bbac Merge pull request #8074 from Lyndon-Li/data-mover-ms-new-controller-1
Data mover micro service new controller
2024-08-07 17:00:27 +08:00
Lyndon-Li
82d9fe4d4d backup repo config
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-07 15:34:57 +08:00
lyndon-li
26459488ed Merge pull request #8089 from kaovilai/fix-csi-snapshot-datamovementv1.14doc
Fix v1.14 site header for csi-snapshot-data-movement
2024-08-07 10:33:26 +08:00
Wenkai Yin(尹文开)
b7f2e15c6e Merge pull request #8086 from reasonerjt/fix-7812
Patch dbr's status when error happens
2024-08-07 09:43:15 +08:00
Tiger Kaovilai
be4aabccd9 Fix v1.14 site header for csi-snapshot-data-movement
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2024-08-06 19:11:25 -04:00
Michael Steven Fruchtman
64d8014b87 Correcting OpenShift on IBM Documentation Error (#8077)
* Correcting Openshift on IBM Documentation Error

I have to admit to some significant error and embarrassment regarding the documentation update
about Openshift on IBM Cloud pull request https://github.com/vmware-tanzu/velero/pull/8069.

I will correct my error before it gets any further.

Just exposing /var/data/kubelet/pods is incorrect and host path /var/lib/kubelet/pods should remain unchanged.

The errors with the defaults during csi snapshot data movement were:

   data path backup failed: Failed to run kopia backup: unable to get local
    block device entry: resolveSymlink: lstat /var/data/: no such
    file or directory

I suspected this was the same as RancherOS and Nutanix. It is not.

The original tested changes changed both /var/lib/kubelet/{pods,plugins} to
/var/data/kubelet/{pods,plugins}.

The published changes only result in the error:

```
status:
  completionTimestamp: '2024-08-02T17:12:29Z'
  message: >-
    data path backup failed: Failed to run kopia backup: unable to get local
    block device entry: resolveSymlink: lstat /var/data/kubelet/plugins: no such
    file or directory
  node: 10.240.0.5
  phase: Failed
  progress: {}
  startTimestamp: '2024-08-02T17:12:11Z'
```

After making continued modifications to the daemonset the correct configuration was:

```
volumeMounts:
- name: host-pods
  mountPath: /host_pods
  mountPropagation: HostToContainer
- name: host-plugins
  mountPath: /var/data/kubelet/plugins
  mountPropagation: HostToContainer
```

```
volumes:
- name: host-pods
  hostPath:
	path: /var/lib/kubelet/pods
	type: ''
- name: host-plugins
  hostPath:
	path: /var/data/kubelet/plugins
	type: ''
```

Only the changes to the plugin path were required.
The plugin path changes were required to both the mount path and the host path.

Regardless of whether /var/lib/kubelet/pods or /var/data/kubelet/pods host path, backups and restore
succeeded provided the plugin path was modified.

```
volumeMounts:
- name: host-pods
  mountPath: /host_pods
  mountPropagation: HostToContainer
- name: host-plugins
  mountPath: /var/data/kubelet/plugins
  mountPropagation: HostToContainer
```

```
volumes:
- name: host-pods
  hostPath:
	path: /var/data/kubelet/pods
	type: ''
- name: host-plugins
  hostPath:
	path: /var/data/kubelet/plugins
	type: ''
```

After getting on-host access was able to confirm. Pods are at /var/lib/kubelet/pods.

```
ls /var/lib/kubelet/pods
07c0be63-335d-4cfb-b39f-816bc2fb32cd
51f31b3e-4710-4ef0-8626-5f1a78a624b2
a4802fd3-3b62-45a4-8f21-974880b6f92a
cccb35c9-b4f9-4ca9-a697-736ae64f09ad
0a5d4366-7fa1-4525-9e45-a43a362b8542
558b0643-0661-4d4a-b03e-aac60c6ad710
a4b106fb-5b7b-48e5-828a-ea7b41ba0e59
ce1290e1-4330-4df6-8166-14784bcce930
```

On host the volumes are in /var/data/kubelet/plugins.

```
ls /var/data/kubelet/plugins/kubernetes.io/csi/openshift-storage.cephfs.csi.ceph.com/231e04896c4f528efb95d23a3c153db9fc4a7206b7320f74443f30de7228dba5/globalmount/velero/backups/backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c/
backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-csi-volumesnapshotclasses.json.gz
backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-resource-list.json.gz
backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-csi-volumesnapshotcontents.json.gz
backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-results.gz
backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-csi-volumesnapshots.json.gz
backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-volumesnapshots.json.gz
backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-itemoperations.json.gz
backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c.tar.gz
backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-logs.gz
velero-backup.json
backup-resources-41d84d16-47a7-4ea8-a9cb-6348d01bcb2c-podvolumebackups.json.gz
```

With the volume config changed to expose /var/data/kubelet/plugins for the plugin hostPath, the DataUploads and DataDownloads
succeed for both Filesystem and Block mode PVCs.

```
status:
  completionTimestamp: '2024-08-02T17:23:33Z'
  node: 10.240.0.5
  path: >-
    /host_pods/7fcb9d56-7885-437c-acd3-67db6b1ee8ae/volumeDevices/kubernetes.io~csi/pvc-47b91f56-db8c-44bf-9ecc-737170561b4b
  phase: Completed
  progress:
    bytesDone: 5368709120
    totalBytes: 5368709120
  snapshotID: 8faae36b3592fee4efbfad024f26033e
  startTimestamp: '2024-08-02T17:21:22Z'
```

```
status:
  completionTimestamp: '2024-08-02T18:42:19Z'
  node: 10.240.0.5
  phase: Completed
  progress:
    bytesDone: 5368709120
    totalBytes: 5368709120
  startTimestamp: '2024-08-02T18:41:00Z'
```

My apologies for the error.

Signed-off-by: Michael Fruchtman <msfrucht@us.ibm.com>

* Add context to plugins mountPath

Signed-off-by: Michael Fruchtman <msfrucht@us.ibm.com>

---------

Signed-off-by: Michael Fruchtman <msfrucht@us.ibm.com>
2024-08-06 09:05:45 -04:00
Daniel Jiang
5c88c897a5 Patch dbr's status when error happens
This commit makes sure the dbr's status is "Processed" when an error
happens before the actual deletion is started

fixes #7812

Signed-off-by: Daniel Jiang <daniel.jiang@broadcom.com>
2024-08-06 18:37:34 +08:00