Commit Graph

92 Commits

Author SHA1 Message Date
Lyndon-Li
82d9fe4d4d backup repo config
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-08-07 15:34:57 +08:00
Lyndon-Li
7b7727e808 issue 7620: backup repo config
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-07-31 16:41:27 +08:00
Matthieu MOREL
35c90f1672 testifylint: enable error-nil rule (#7670)
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-07-16 12:23:16 -04:00
Lyndon-Li
20676c1ae7 new data path for data mover ms
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-07-01 19:07:00 +08:00
Lyndon-Li
2d0bca5e29 avoid unnecessary repo connect for maintenance
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-05-30 19:13:52 +08:00
Daniel Jiang
bed10c7fe6 Fix the problems found by codespell
Signed-off-by: Daniel Jiang <daniel.jiang@broadcom.com>
2024-05-24 13:32:54 +08:00
lyndon-li
65a831ed67 Merge pull request #7762 from kaovilai/waitBackupRepoErrsVerbose
Surface errors when waiting for backupRepository
2024-05-17 10:03:19 +08:00
Tiger Kaovilai
2c6853b6e8 Surface errors when waiting for backupRepository
Make errors such as those found in https://github.com/vmware-tanzu/velero/issues/6928#issuecomment-1759369183

Makes errors easier to understand than "timed out waiting for the condition"

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2024-05-09 17:20:33 -04:00
Ming Qiu
e91d9b906c Maintenance job should not be launched if the repo already has a running one
Signed-off-by: Ming Qiu <ming.qiu@broadcom.com>
2024-05-07 06:12:05 +00:00
Lyndon-Li
776efc4460 issue 7712: don't append nil error for BatchForget of restic path
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-04-22 11:10:45 +08:00
Lyndon-Li
18976c0a62 kopia: index compaction during maintenance
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-03-29 15:24:47 +08:00
Wenkai Yin(尹文开)
8d10b68eda Improve the concurrency for PVBs in different pods
Improve the concurrency for PVBs in different pods

Fixes #6676

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2024-03-29 09:58:50 +08:00
Xun Jiang/Bruce Jiang
d640cc16ab Merge pull request #7573 from mmorel-35/golangci-lint-exclude-rules
golangci-lint: use exclude-rules instead of skip-files and skip-dirs
2024-03-28 16:39:40 +08:00
qiuming
e80bdcf2e2 Merge pull request #7451 from qiuming-best/maintenance-job
Add repository maintenance job
2024-03-28 14:47:15 +08:00
Ming Qiu
8d63c76c92 Add maintenance job
Signed-off-by: Ming Qiu <mqiu@vmware.com>
2024-03-28 03:22:06 +00:00
Matthieu MOREL
ef04ef6361 golangci-lint: use exclude-rules instead of skip-files and skip-dirs
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-03-27 20:17:34 +00:00
Matthieu MOREL
3c704ba1b1 linter(testifylint): use Len or Empty for arrays testing (#7555)
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-03-27 14:16:58 -04:00
lyndon-li
a2c87fc8b2 Merge pull request #7438 from Lyndon-Li/batch-delete-snapshot
Issue 7281: batch delete snapshot
2024-03-27 13:31:07 +08:00
Lyndon-Li
d538fc87ad batch delete snapshot
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-03-27 11:21:51 +08:00
Lyndon-Li
5d48e36b55 open kopia with no index change
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-03-25 18:14:43 +08:00
Wenkai Yin(尹文开)
79e9e31d8d Merge pull request #7489 from ywk253100/240229_lib
Bump up the versions of several Kubernetes-related libs
2024-03-12 16:12:56 +08:00
Wenkai Yin(尹文开)
8752c3a820 Bump up the versions of severel Kubernetes-related libs
Bump up the versions of severel Kubernetes-related libs

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2024-03-05 13:09:38 +08:00
Lyndon-Li
24c4eb075f issue 7211: support concatenate objects
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-02-21 17:34:04 +08:00
Lyndon-Li
57879357fc Merge branch 'main' into batch-delete-snapshot 2024-02-18 14:54:01 +08:00
Lyndon-Li
32d92ca964 batch delete snapshot
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2024-02-18 14:49:21 +08:00
Wenkai Yin(尹文开)
9649619a6f Put credential related config into getStorageCredentials function
Put credential related config into getStorageCredentials function

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2024-02-05 16:09:15 +08:00
Daniel Jiang
30728c248c Respect the config in BSL when IRSA is configured
This commit makes sure when kopia connects to the repository the
crendentials file specified in BSL.spec.config has the higher priority over
Pod Environment credentials when IRSA is configured.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2024-02-01 16:33:10 +08:00
Xun Jiang
f5c159ce56 Resolve linter issues.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-11-29 11:15:43 +08:00
qiuming
38e1ae0405 Merge pull request #7034 from ywk253100/231030_cred
Read information from the credential specified by BSL
2023-11-01 09:41:25 +08:00
Wenkai Yin(尹文开)
49a85e1636 Read information from the credential specified by BSL
Read information from the credential specified by BSL

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2023-10-30 17:28:10 +08:00
Tiger Kaovilai
d5f238c83c kopia/repository/config/aws.go: Set session.Options profile from config
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-10-24 14:05:47 -04:00
Daniel Jiang
b71d2b3898 Bump up aws sdk to aws-sdk-go-v2
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2023-10-24 17:01:26 +08:00
Lyndon-Li
3d841dd8f1 udmrepo use region specified in BSL when s3URL is empty
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-10-20 19:58:54 +08:00
Scott Seago
09be1f7995 issue #6807: Retry failed create when using generateName
When creating resources with generateName, apimachinery
does not guarantee uniqueness when it appends the random
suffix to the generateName stub, so if it fails with
already exists error, we need to retry.

Signed-off-by: Scott Seago <sseago@redhat.com>
2023-10-09 17:38:37 -04:00
Wenkai Yin(尹文开)
61a6c1ba2a Create the backup repository only when it doesn't exist
When preparing a backup repository, Velero tries to connect to it, if fails then create it. The repository status always records the error reported by creation but the real reason maybe caused by the connect operation. This is confuseing and hard to debug

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2023-09-28 14:53:59 +08:00
Wenkai Yin(尹文开)
b598150cd1 Support setting CA cert for BSL
Support setting CA cert for BSL

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2023-09-19 11:28:05 +08:00
Wenkai Yin(尹文开)
3a291e368a Make Kopia support Azure AD
This commit introduces our own Azure storage provider by wrapping Kopia's implementation rather than contributing to upstream based on the following considerations:
1. Velero needs the capability to interact with the repository concurrently while Kopia doesn't, this will increase the complexity of Kopia if we contribute to upstream
2. The configuration items provided by Velero and Kopia are conflict, e.g. Velero supports customizing storage account URI which is a full path while Kopia supports customizing storage account domain which is part of the URI. We need to consider the backward compatibility and upgrade case if we contribute to upstream which needs extra efforts
3. Contribute to upstream is a longer cycle when we need to introduce new changes. With this commit, we no longer depends on upstream for the Azure storage provider part and is easy for us to maintain

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2023-09-19 11:28:04 +08:00
lyndon
5af664d361 bump kopia to v0.14 (#6833)
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-09-18 21:05:21 +08:00
Yang Gang (成都)
ec11a5a4cc code clean for repository (#6768)
Signed-off-by: yanggang <gang.yang@daocloud.io>
2023-09-12 14:43:28 +08:00
Luis Davim
28c5dc9fda fix: load AWS config and assume role
Signed-off-by: Luis Davim <dluis@vmware.com>
2023-08-09 13:26:30 +01:00
qiuming
f78dd073bf Merge pull request #6558 from kaovilai/s3profilefix
GetS3Credentials pass profile from config to NewSharedCredentials
2023-08-07 16:38:25 +08:00
lyndon
bb96c2155c fix issue 6575 (#6587)
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-08-02 10:27:05 -04:00
Tiger Kaovilai
289aae1581 GetS3Credentials pass profile from config to NewSharedCredentials
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-07-28 11:26:40 -04:00
Daniel Jiang
b0a343cd48 Delete moved snapshots when the backup is deleted
This commit introduces a deleteItemAction which writes a temporary configmap to
record the snapshot info so that the controller can trigger repo manager
to remove the snapshot

This process is a bit chatty and we should consider to refactor the code
so it's easier to connect to the repo directly in the DIA

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2023-07-27 13:27:33 +08:00
Tiger Kaovilai
05722876b9 Moving kopia logging to remove kopia from indirect dependency in velero plugins
when running `go mod why -m github.com/kopia/kopia` in velero-plugins prior to this change you will see following

```
❯ go mod why -m github.com/kopia/kopia
github.com/konveyor/openshift-velero-plugin/velero-plugins
github.com/vmware-tanzu/velero/pkg/plugin/framework
github.com/vmware-tanzu/velero/pkg/util/logging
github.com/kopia/kopia/repo/logging
```

after
```
❯ go mod why -m github.com/kopia/kopia
(main module does not need module github.com/kopia/kopia)
```

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-07-12 10:44:33 -04:00
lyndon
7deae4cbf5 Merge pull request #6459 from blackpiglet/6001_fix
Add exit code log and possible memory shortage warning log for Restic command failure
2023-07-07 09:24:16 +08:00
Xun Jiang
d7f1ea4fbd Add exit code log and possible memory shortage warning log for Restic command failure.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-07-05 21:35:52 +08:00
Lyndon-Li
65cb25a74c fix concurrent repo ensure problem
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-06-30 14:48:51 +08:00
Lyndon-Li
38d5003c6b add ut for pkg/repository
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-06-29 07:40:59 +00:00
Lyndon-Li
45d7cc9783 fix issue 6255
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-06-02 16:11:23 +08:00