Commit Graph

232 Commits

Author SHA1 Message Date
lyndon
100d6b4430 Merge pull request #5319 from Lyndon-Li/issue-fix-4874
Issue fix 4874 and 4752
2022-09-14 11:04:56 +08:00
Lyndon-Li
fbb2606102 issue fix 4874 update
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-09-13 11:15:52 +08:00
Shubham Pampattiwar
be40d7eb19 Merge pull request #4864 from sseago/vsl-creds-main
Add credentials to volume snapshot locations.
2022-09-12 09:56:49 -04:00
lyndon
30a70cbd0d Merge branch 'vmware-tanzu:main' into issue-fix-4874 2022-09-09 09:20:38 +08:00
Scott Seago
596114b427 Add credentials to volume snapshot locations.
This is analogous to the BSL creds work that was done previously, but for VSLs instead.

Signed-off-by: Scott Seago <sseago@redhat.com>
2022-09-08 08:59:17 -04:00
Lyndon-Li
8496b43e37 refactor repo ensurer
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-09-08 16:10:51 +08:00
Lyndon-Li
a12024887f fix issue 4874
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-09-07 21:44:45 +08:00
lyndon
0282e65221 Kopia Pod Volume Backup/Restore (#5259)
* kopia pvbr

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-09-05 10:29:30 +08:00
Xun Jiang/Bruce Jiang
6fea973c57 Migrate backup sync controller from code-generator to kubebuilder (#5218)
* Migrate backup sync controller from code-generator to kubebuilder

1. use kubebuilder's reconcile logic to replace controller's old logic.
2. use ginkgo and gomega to replace testing.
3. modify BSL reconciler registration method.

Signed-off-by: Xun Jiang <blackpiglet@gmail.com>

* Add Option for PeriodicalEnqueueSource.

Signed-off-by: Xun Jiang <blackpiglet@gmail.com>

Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
Co-authored-by: Xun Jiang <blackpiglet@gmail.com>
2022-08-30 16:55:39 +08:00
Xun Jiang/Bruce Jiang
4bc3a3a784 Merge pull request #5248 from allenxu404/gc-freq
equip gc controller with configurable frequency
2022-08-30 11:12:12 +08:00
allenxu404
a8ba4875f0 equip gc controller with configurable frequency
Signed-off-by: allenxu404 <qix2@vmware.com>
2022-08-30 10:13:06 +08:00
Xun Jiang/Bruce Jiang
eaf9fab711 Merge branch 'main' into update_enabled_runtime_controllers 2022-08-29 20:06:31 +08:00
Wenkai Yin(尹文开)
4a5647a891 Move the code in pkg/restic/common.go to the proper package
Move the code in pkg/restic/common.go to the proper package

Fixes #5243

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-08-29 10:04:58 +08:00
Xun Jiang/Bruce Jiang
e8494418d4 Merge pull request #5215 from allenxu404/issue5023
Refactor GC controller with kubebuilder
2022-08-23 15:59:03 +08:00
Xun Jiang
71e5027bfb Controller refactor code modifications.
1. Add some refactored controllers initiation code into enabledRuntimeControllers.
2. Add reconciler struct initiation function for DownloadRequest and ServerStatusRequest controllers.

Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-08-23 10:32:19 +08:00
allenxu404
0378020c8d refactor gc controller with kubebuilder
Signed-off-by: allenxu404 <qix2@vmware.com>
2022-08-22 12:16:56 +08:00
Wenkai Yin(尹文开)
047c7531fa Refactor the restic repo related code for Kopia integration
Refactor the restic repo related code for Kopia integration

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-08-17 11:49:53 +08:00
Daniel Jiang
4e25f59dc1 Add parameter "uploader-type" to velero server (#5212)
This commit adds the parameter "uploader-type" to velero server, add exposes the
setting via "velero install" in CLI.

fixes #5062

Signed-off-by: Daniel Jiang <jiangd@vmware.com>

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-08-16 13:50:28 +08:00
Xun Jiang/Bruce Jiang
a5f4f8f9fc Merge branch 'main' into 5048-CSI-snapshot-timeout-configurable 2022-08-08 17:18:44 +08:00
Xun Jiang
f8d9cfdb84 Make CSI snapshot creation timeout configurable for backup and schedule.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-08-08 17:07:26 +08:00
lyndon
088eb9b83c repo credentials (#5167)
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2022-08-04 15:20:02 +08:00
Daniel Jiang
f562a7ce2b Update the CRD for kopia integration
This commit adds additional fields to podvolumebackup
and podvolumerestore.  The resticrepository will be renamed to
backuprepository

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-07-22 00:10:24 +08:00
Wenkai Yin(尹文开)
6fa4d7d606 Mark in-progress CRs as failed when starting the server
Mark in-progress CRs as failed when starting the server

Fixes #4953

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-06-07 17:02:17 +08:00
Wenkai Yin(尹文开)
44199db79d Enhance the map flag to support parsing input value contains entry delimiters
Enhance the map flag to support parsing input value contains entry delimiters

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-05-17 13:56:41 +08:00
qiuming
d48e1d9db7 Merge pull request #4897 from ywk253100/220609_gc
Make garbage collection for expired backups configurable
2022-05-11 10:46:03 +08:00
Wenkai Yin(尹文开)
6a551e546e Make garbage collection for expired backups configurable
Make garbage collection for expired backups configurable

Fixes #4875

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-05-11 10:33:59 +08:00
Ming
ef7e5a8388 change func name and repository description
Signed-off-by: Ming <mqiu@vmware.com>
2022-05-09 13:57:02 +08:00
Ming
752b28166f Merge branch 'main' of https://github.com/qiuming-best/velero into restic-repo-tmp 2022-05-06 14:43:00 +08:00
Xun Jiang
6421f8ad02 Make waiting VolumeSnapshot to ready process parallel.
Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-05-05 14:32:55 +08:00
Daniel Jiang
89e90d9be4 Delete orphan CSI snapshots in backup sync controller
This commit makes backup sync controller delete the volumesnapshot and
volumesnapshotcontent created by the backup which is cleaned up as orphan

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-05-05 10:54:34 +08:00
Ming
06d3d731ed Refactor restic repository
Signed-off-by: Ming <mqiu@vmware.com>
2022-04-29 10:26:00 +08:00
Xun Jiang/Bruce Jiang
001229a8b3 Merge pull request #4855 from reasonerjt/bak-delete-refact
Refactor backup deletion controller based on kubebuilder
2022-04-28 10:48:11 +08:00
Wenkai Yin(尹文开)
d42e353610 Merge pull request #4866 from reasonerjt/clusterclass-restore-order
Add ClusterClasses to the restore priority list
2022-04-28 09:04:41 +08:00
Daniel Jiang
74fd158a1c Refactor backup deletion controller based on kubebuilder
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-04-27 17:00:54 +08:00
Daniel Jiang
f89691c5c1 Add ClusterClasses to the restore priority list
Make sure ClusterClasses are stored before Clusters.
Fixes #4767

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-04-27 15:53:55 +08:00
Wenkai Yin(尹文开)
9e786d681b Merge pull request #4832 from reasonerjt/fix-rm-csi-bak-tmp
Make the vsc created by backup sync controller deletable
2022-04-20 17:19:09 +08:00
Daniel Jiang
4f9e445142 Make the vsc created by backup sync controller deletable
Fixes #4760

This commit make changes in 2 parts:
1) When a volumesnapshotcontent is persisted during backup, velero will reset its
   `Source` field to remove the VolumeHandle, so that the
   csi-snapshotter will not try to call `CreateSnapshot` when its synced
   to another cluster with a backup.
2) Make sure the referenced volumesnapshotclasses are persisted and
   synced with the backup, so that when the volumesnapshotcontent is
   deleted the storage snapshot is also removed.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-04-19 15:06:59 +08:00
Xun Jiang/Bruce Jiang
22189e8e19 Merge pull request #4436 from codegold79/3454-convert-pod-volume-backup-controller-to-kubebuilder-v3-framework
Convert Pod Volume Backup resource/controller to the Kubebuilder V3 framework
2022-04-15 15:07:09 +08:00
Wenkai Yin(尹文开)
016ac129c6 Refactor schedule controller with kubebuilder
Refactor schedule controller with kubebuilder

fixes #4671

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-04-07 17:32:52 +08:00
Daniel Jiang
04f59ce003 Bump up to v1 API for CSI snapshot
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-03-31 22:55:03 +08:00
F. Gold
2dd85c9c42 Convert PodVolumeBackup to Kubebuilder framework
Signed-off-by: F. Gold <fgold@vmware.com>
2022-01-26 20:09:24 -08:00
Xun Jiang
7ab4bfc632 Revert "Migrate backup sync controller from code-generator to kubebuilder (#4423)"
This reverts commit 5aaeb3ebbe.
2021-12-17 09:40:24 +08:00
Xun Jiang/Bruce Jiang
5aaeb3ebbe Migrate backup sync controller from code-generator to kubebuilder (#4423)
* Migrate backup sync controller from code-generator to kubebuilder

1. use kubebuilder's reconcile logic to replace controller's old logic.
2. use ginkgo and gomega to replace testing.

Signed-off-by: Xun Jiang <jxun@vmware.com>

* Fix: modify code according to comments

1. Remove DefaultBackupLocation
2. Remove unneccessary comment line
3. Add syncPeriod default value setting logic
4. Modify ListBackupStorageLocations function's context parameter
5. Add RequeueAfter parameter in Reconcile function return value

Signed-off-by: Xun Jiang <jxun@vmware.com>

* Reconcile function use context passed from parameter

1. Use context passed from parameter, instead of using Reconciler struct's context.
2. Delete Reconciler struct's context member.
3. Modify test case accordingly.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2021-12-14 20:07:20 -05:00
Xun Jiang
2c240c2830 Use logrusr instead of zap.
logrusr is a open source convertor, which can convert logrus logger into logr.
By using logrusr, velero can use exsiting formatted logrus logger, other than introducing zap as a new logger.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2021-11-17 18:10:37 +08:00
jxun
4a1943f6c9 Fix: add init log option for velero controller-runtime manager.
fix for issue #3737
add log option for velero controller-runtime manager to log return error in reconcile loop.

Signed-off-by: Xun Jiang jxun@vmware.com
Signed-off-by: jxun <jxun@jxun-a01.vmware.com>
2021-11-15 14:07:27 +08:00
F. Gold
9f06a1b451 Limit kubebuilder controllers to work only for objects in own namespaces
Signed-off-by: F. Gold <fgold@vmware.com>
2021-10-18 17:31:28 -07:00
Dharma Bellamkonda
dc1f17944e Page list requests by default (#3823)
Signed-off-by: Dharma Bellamkonda <bellamko@adobe.com>
2021-08-17 18:49:41 -07:00
Bridget McErlean
9ffffda11e Use Credential from BSL for restic commands (#3489)
* Use Credential from BSL for restic commands

This change introduces support for restic to make use of per-BSL
credentials. It makes use of the `credentials.FileStore` introduced in
PR #3442 to write the BSL credentials to disk. To support per-BSL
credentials for restic, the environment for the restic commands needs to
be modified for each provider to ensure that the credentials are
provided via the correct provider specific environment variables.
This change introduces a new function `restic.CmdEnv` to check the BSL
provider and create the correct mapping of environment variables for
each provider.

Previously, AWS and GCP could rely on the environment variables in the
Velero deployments to obtain the credentials file, but now these
environment variables need to be set with the path to the serialized
credentials file if a credential is set on the BSL.

For Azure, the credentials file in the environment was loaded and parsed
to set the environment variables for restic. Now, we check if the BSL
has a credential, and if it does, load and parse that file instead.

This change also introduces a few other small improvements. Now that we
are fetching the BSL to check for the `Credential` field, we can use the
BSL directly to get the `CACert` which means that we can remove the
`GetCACert` function. Also, now that we have a way to serialize secrets
to disk, we can use the `credentials.FileStore` to get a temp file for
the restic repo password and remove the `restic.TempCredentialsFile`
function.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Add documentation for per-BSL credentials

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Address review feedback

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Address review comments

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-03-11 13:10:51 -05:00
Bridget McErlean
b9a8c0b254 Pass configured BSL credential to plugin via config (#3442)
* Load credentials and pass to ObjectStorage plugins

Update NewObjectBackupStore to take a CredentialsGetter which can be
used to get the credentials for a BackupStorageLocation if it has been
configured with a Credential. If the BSL has a credential, use that
SecretKeySelector to fetch the secret, write the contents to a temp file
and then pass that file through to the plugin via the config map using
the key `credentialsFile`. This relies on the plugin being able to use
this new config field.

This does not yet handle VolumeSnapshotLocations or ResticRepositories.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Address code reviews

Add godocs and comments.
Improve formatting and test names.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Address code reviews

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-03-04 13:43:15 -08:00
Pranav Gaikwad
c46fe71b12 Restore progress reporting (#3125)
* restore progress reporting

Signed-off-by: Pranav Gaikwad <pgaikwad@redhat.com>

* add restore statistics to describe restore

Signed-off-by: Pranav Gaikwad <pgaikwad@redhat.com>

* address feedback, include namespaces in the count

Signed-off-by: Pranav Gaikwad <pgaikwad@redhat.com>
2021-03-04 16:21:44 -05:00