Commit Graph

527 Commits

Author SHA1 Message Date
Xun Jiang/Bruce Jiang
f666667e5b Merge pull request #6547 from reasonerjt/del-moved-snapshot
Delete moved snapshots when the backup is deleted
2023-07-27 15:10:33 +08:00
Daniel Jiang
b0a343cd48 Delete moved snapshots when the backup is deleted
This commit introduces a deleteItemAction which writes a temporary configmap to
record the snapshot info so that the controller can trigger repo manager
to remove the snapshot

This process is a bit chatty and we should consider to refactor the code
so it's easier to connect to the repo directly in the DIA

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2023-07-27 13:27:33 +08:00
Xun Jiang/Bruce Jiang
6997e4a694 Merge pull request #6476 from reasonerjt/del-inprog-dbr
Delete the expired deletebackuprequests that are stuck in "InProgress"
2023-07-26 16:13:58 +08:00
lyndon
9c8275eda7 Merge pull request #6533 from Lyndon-Li/data-mover-wrong-bsl-after-sync
Data mover wrong bsl after sync
2023-07-25 14:20:10 +08:00
Lyndon-Li
74bf03b272 data mover wrong bsl after sync
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-07-25 12:38:09 +08:00
Shubham Pampattiwar
eb35f127e2 Merge pull request #6524 from yanggangtony/fix-log-scheduler 2023-07-23 18:44:47 -07:00
Xun Jiang/Bruce Jiang
605eab1eb8 Merge pull request #6531 from yanggangtony/delete-io-util
delete io/ioutil package as it is deprecated
2023-07-24 09:42:55 +08:00
Efim Smykov
8396163e77 add missing CompletionTimestamp and metrics when restore moved into terminal phase (#6397)
Signed-off-by: Nutrymaco <efim7299@icloud.com>
2023-07-21 17:22:01 -04:00
yanggang
bcc69f33f9 delete io/ioutil package as it is deprecated
Signed-off-by: yanggang <gang.yang@daocloud.io>
2023-07-21 15:18:31 +08:00
qiuming
32262babc4 Merge pull request #6479 from allenxu404/i2697
add restore finalizer to clean up external resources
2023-07-20 19:07:02 +08:00
Daniel Jiang
2548b20db9 Track the skipped PV in a backup and print the summary in backup log (#6496)
Partially address the requirements in #5834

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2023-07-20 16:13:48 +08:00
yanggang
c5af315d19 fix scheduleReconciler log info
Signed-off-by: yanggang <gang.yang@daocloud.io>
2023-07-20 13:46:08 +08:00
allenxu404
0bb509ccdd add restore finalizer to clean up external resources
Signed-off-by: allenxu404 <qix2@vmware.com>
2023-07-19 18:56:35 +08:00
Anshul Ahuja
c8f970a4f1 Merge branch 'main' into resourcemodifier
Signed-off-by: Anshul Ahuja <anshul.ahu@gmail.com>
2023-07-19 11:11:40 +05:30
lyndon
b5bd55fc56 fix issue 6490 (#6491)
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-07-14 14:05:51 -04:00
allenxu404
084fd66586 add data upload/download metrics
Signed-off-by: allenxu404 <qix2@vmware.com>
2023-07-14 10:47:53 +08:00
Anshul Ahuja
9fe7a1d136 lint
Signed-off-by: Anshul Ahuja <anshulahuja@microsoft.com>
2023-07-13 13:59:37 +05:30
Anshul Ahuja
c0ca69dc87 Improve test coverage
Signed-off-by: Anshul Ahuja <anshulahuja@microsoft.com>
2023-07-13 13:52:49 +05:30
Anshul Ahuja
ce7d2bfc87 add more logs
Signed-off-by: Anshul Ahuja <anshulahuja@microsoft.com>
2023-07-11 11:16:44 +05:30
Ming
e0bfd676cc Merge branch 'main' of https://github.com/qiuming-best/velero into mark-crs-failed 2023-07-11 02:42:46 +00:00
Lyndon-Li
9f5162ece3 add wait timeout for expose prepare
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-07-10 17:32:10 +08:00
Ming Qiu
480fe445b1 Mark dataupload datadownload status failed when velero pod restart
Signed-off-by: Ming Qiu <mqiu@vmware.com>
2023-07-10 07:14:24 +00:00
Anshul Ahuja
8db88bd392 Merge branch 'main' into resourcemodifier 2023-07-10 11:33:49 +05:30
Daniel Jiang
c500e8dc34 Delete the expired deletebackuprequests that are stuck in "InProgress"
state

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2023-07-10 10:15:19 +08:00
Lyndon-Li
8a7aa2051c add node name data mover CR
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-07-06 14:06:28 +08:00
Anshul Ahuja
4e6d31dc38 linting
Signed-off-by: Anshul Ahuja <anshulahuja@microsoft.com>
2023-07-06 11:29:33 +05:30
Anshul Ahuja
ed4437ad22 initial draft
Signed-off-by: Anshul Ahuja <anshulahuja@microsoft.com>
2023-07-05 13:51:32 +05:30
Xun Jiang/Bruce Jiang
e54a8af0ad Merge pull request #6433 from blackpiglet/6010_fix
Modify DownloadRequest controller logic
2023-07-05 10:48:13 +08:00
Ming Qiu
2f667f5191 Add data download controller UT
Signed-off-by: Ming Qiu <mqiu@vmware.com>
2023-07-04 08:47:49 +00:00
Xun Jiang
40b2ee1323 Modify DownloadRequest controller logic
1. Avoid patch DownloadRequest when it's deleted.
2. Add periodic enqueue resource for reconcile.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-07-03 17:11:48 +08:00
Ming
22a99c34b9 Fix data path concurrent
Signed-off-by: Ming <mqiu@vmware.com>
2023-06-30 18:18:31 +00:00
Ming
1bfcee776c Add data download controller
Signed-off-by: Ming <mqiu@vmware.com>
2023-06-29 13:28:19 +00:00
Lyndon-Li
b1316dae23 data mover smoking test 02
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-06-26 04:47:40 +00:00
Ming
e2bb5b3fe8 Add UT for pkg/discovery
Signed-off-by: Ming <mqiu@vmware.com>
2023-06-15 02:45:34 +00:00
lyndon
8cd55d1826 Merge pull request #6383 from Lyndon-Li/data-mover-backup-smoke-testing
Data mover backup smoke testing
2023-06-13 11:13:14 +08:00
Lyndon-Li
f781e255c6 data mover smoke testing
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-06-12 15:30:06 +08:00
Wenkai Yin(尹文开)
89c10ddcc0 Update metrics when backup failed with validation error (#6318)
Update metrics when backup failed with validation error

Fixes #6265

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2023-06-09 11:19:26 +08:00
Xun Jiang/Bruce Jiang
9743a7ce56 Fix #6118: Do not persist VolumeSnapshot and VolumeSnapshotContent for snapshot DataMover case. (#6366)
1. Because VolumeSnapshot and VolumeSnapshotContent CRs are not kept after backup completed,
   don't persist them in the backup metadata.
2. Add some builder methods needed by CSI plugin.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-06-08 17:05:17 +08:00
Xun Jiang/Bruce Jiang
4227a824cd Merge pull request #6327 from blackpiglet/6099_fix
Remove Wait VolumeSnapshot to ReadyToUse logic.
2023-06-07 14:56:26 +08:00
Ming
b3e99a7eb2 Add Dataupload controller
Signed-off-by: Ming <mqiu@vmware.com>
2023-06-06 05:46:38 +00:00
Xun Jiang
7491ab1ec5 Remove Wait VolumeSnapshot to ReadyToUse logic.
Due to the logic moving to plugin, and the plugin cannot read the
Velero server's resourceTimeout setting, add the resourceTimeout
in the backup annotation to pass to plugin.
Remove VolumeSnapshotContent reset code from Velero server.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-06-02 14:43:43 +08:00
Lyndon-Li
6c16020a3e fix pvr issue for generic data path
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-05-23 15:20:13 +08:00
Lyndon-Li
9ab85892a7 add shared generic data path 02
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-05-17 15:04:04 +08:00
Lyndon-Li
5f008d18fa Merge branch 'main' into data-mover-generic-data-path 2023-05-17 10:15:52 +08:00
lyndon
5b75f35262 Add data mover CRD under v2alpha1 (#6176)
* add data mover CRD under v1alpha1

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* data mover CRDs to v2alpha1

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* data mover crd changes

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

---------

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-05-16 15:09:54 -04:00
Lyndon-Li
623da51494 add shared generic data path
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-05-12 12:01:59 +08:00
qiuming
4db1a781fc Merge pull request #6208 from blackpiglet/linter_errcheck
Enable errcheck linter and resolve found issues.
2023-05-05 14:43:55 +08:00
Anshul Ahuja
d7e0f64c89 Remove restorestoragelocation (#6211)
Signed-off-by: Anshul Ahuja <anshulahuja@microsoft.com>
Co-authored-by: Anshul Ahuja <anshulahuja@microsoft.com>
2023-04-28 10:58:08 -04:00
Xun Jiang
f051ecaee9 Enable errcheck linter and resolve found issues.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-04-28 09:33:42 +08:00
Xun Jiang/Bruce Jiang
a8a17d725a Merge pull request #6194 from blackpiglet/linter_more
Enable more linters, and remove mal-functioned milestoned issue action.
2023-04-26 14:12:47 +08:00