Commit Graph

245 Commits

Author SHA1 Message Date
danfengliu
a71237cc64 Merge pull request #5124 from danfengliu/add-migration-e2e-test
Add migration E2E test
2022-08-10 13:38:32 +08:00
danfengl
fb445b3c0d Add migration E2E test
Signed-off-by: danfengl <danfengl@vmware.com>
2022-08-09 08:31:39 +00:00
Xun Jiang/Bruce Jiang
cea5e7f218 Merge pull request #5174 from blackpiglet/reduce-crd-size-new
Reduce CRD size.
2022-08-05 18:46:29 +08:00
Wenkai Yin(尹文开)
092fc01e8d Splic pkg/restic package (#5143)
This commit splits the pkg/restic package into several packages to support Kopia integration works

Fixes #5055

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-08-05 17:15:38 +08:00
Xun Jiang
e8da5df57a Reduce CRD size.
1. Make the Restore hook.InitConatianer server side field pruing disable.
2. Remove restore patch in update-generate-crd-code.sh.
3. Modify related testcases.
4. Add Container fields validation in Restore Init hook.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-08-02 19:45:01 +08:00
Scott Seago
79be75e183 When spec.RestoreStatus is empty, don't restore status
Signed-off-by: Scott Seago <sseago@redhat.com>
2022-06-15 13:02:31 -04:00
Rafael Leal
e0e3016efa Cleanup resetStatus
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-23 17:53:32 -03:00
Rafael Leal
131c8a920f Remove break line
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-19 17:36:21 -03:00
Rafael Leal
bcef5e1d5e Refactor to simplify
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-17 10:39:14 -03:00
Rafael Leal
d85ed612cb Cleanup
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-12 12:01:00 -03:00
Rafael Leal
206709b978 Cleanup
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-12 12:01:00 -03:00
Rafael Leal
04aa7a849f Fixes tests hopefully
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-12 11:59:05 -03:00
Rafael Leal
0b9d6ae73d Add restore status mechanism
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-12 11:59:05 -03:00
Rafael Leal
7f1f881c28 Add resource status
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-12 11:58:47 -03:00
Rafael Leal
6a721403fb Fix DeepEqual when status is updated
Signed-off-by: Rafael Leal <rafaelealdias@gmail.com>
2022-05-12 10:47:24 -03:00
Shubham Pampattiwar
bfdb68a35a add multiple label selector support to backup API
Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

remove backup CLI bits

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

labelselectors spec option for velero restore

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

add changelog file

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

update spec name to OrLabelSelectors

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

minor fixes

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

add validations for labelSelector and orLabelSelectors

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

update crds.gp after fixing conflicts

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

fix CI and add unit tests

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

updated OrLabelSelector spec description and added validation failure unit tests

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

add comments and change log level

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

update site docs

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

wrap pager client calls in a function

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

resolve confilcts and update crds.go

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

rebase and update crds.go

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

combine listing items for a given label

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>
2022-05-11 09:57:47 -04:00
Shubham Pampattiwar
e3106f3e10 add existingResourcePolicy to Restore API
add updateall policy option

fix updating labels

dump updateAll policy option

remove updateAll policy refs

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

add changelog file

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

update docs

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

patch labels for sa if policy is update

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

fix existingResourcePolicy for serviceaccounts

modularize changes and add unit tests

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

fix conflicts and update crds.go

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

change log level from info to error

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

update crds.go

Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>
2022-05-09 13:36:24 -04:00
Scott Seago
da916846b1 continue rather than return for non-matching restore action label
When iterating over applicable restore actions, if a non-matching label
selector is found, velero should continue to the next action rather than
returning from the restoreItem func, which ends up preventing the item's
restore entirely.

Signed-off-by: Scott Seago <sseago@redhat.com>
2022-05-05 08:37:48 -04:00
Scott Seago
de17605c62 Merge pull request #4769 from jibutech/4754-fix
Skip povdolumerestore creation when restore excludes pv/pvc
2022-04-19 11:54:30 -04:00
I538157
74db20993b fix: delete empty backups
Signed-off-by: Yuval Manor <yuvalman958@gmail.com>
2022-04-12 13:13:56 +03:00
half-life666
e24c8845c4 Skip podvolumerestore creation when restore excludes pv/pvc
Signed-off-by: half-life666 <half-life@jibudata.com>
2022-03-31 09:27:33 +08:00
Daniel Jiang
2939914113 Ensure the restore hook applied to new namespace based on the mapping
fixes #4720

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2022-03-28 10:24:47 +08:00
Hoang, Phuong
0171a91366 Correct copyright comment and string compare
Signed-off-by: Hoang, Phuong <phuong.n.hoang@dell.com>
2022-03-21 12:01:26 -04:00
Hoang, Phuong
b8b5427388 Fix format.
Signed-off-by: Hoang, Phuong <phuong.n.hoang@dell.com>
2022-03-15 19:54:24 -04:00
Hoang, Phuong
e9d5f7121c Add unit tests, change copyright years and revert unrelated changes.
Signed-off-by: Hoang, Phuong <phuong.n.hoang@dell.com>
2022-03-15 19:51:53 -04:00
Hoang, Phuong
0741360050 Adding PriorityClass to backup podAction and restore podAction.
Signed-off-by: Hoang, Phuong <phuong.n.hoang@dell.com>
2022-03-15 19:05:13 -04:00
MatthieuFin
f6bdfd09c5 Merge remote-tracking branch 'upstream/main'
Signed-off-by: MatthieuFin <matthieu2717@gmail.com>
2022-02-23 15:03:15 +01:00
Wenkai Yin(尹文开)
fcad46ccdf Check whether the volume is provisioned by CSI driver or not by the annotation as well
Check whether the volume is provisioned by CSI driver or not by the annotation as well

Fixes #4496

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-01-12 19:05:33 +08:00
Wenkai Yin(尹文开)
62779bbcc6 Check the existence of the expected service when ignoring the NodePort already allocated error
Check the existence of the expected service when ignoring the NodePort already allocated error

Fixes 2308

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-01-12 15:06:42 +08:00
Daniel Jiang
7c2dc143d5 Merge pull request #4391 from ywk253100/211123_anno
Keep the annotation "pv.kubernetes.io/provisioned-by" when restoring PVs
2021-12-21 11:08:25 +08:00
Wenkai Yin(尹文开)
648f3ac228 Keep the annotation "pv.kubernetes.io/provisioned-by" when restoring PVs
More details please refer to https://github.com/vmware-tanzu/velero/issues/3470#issuecomment-976279606

Fixes #3470

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2021-12-21 09:52:56 +08:00
Wenkai Yin(尹文开)
c7cd95a374 Ignore the provided port is already allocated error when restoring the LoadBalancer service (#4462)
Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2021-12-21 05:42:27 +08:00
Box-Cube
69f6c8d0cd Fix statefulsets volumeClaimTemplates storageClassName after use Changing PV/PVC Storage Classes (#4375)
* fix statefulsets volumeClaimTemplates storageClassName after use Changing PV/PVC Storage Classes

Signed-off-by: Box-Cube <64300761+Box-Cube@users.noreply.github.com>

* Fix (vmware-tanzu#4373)

Signed-off-by: Box-Cube <64300761+Box-Cube@users.noreply.github.com>

* Fix StatefulSet volumeClaimTemplates storageClassName(vmware-tanzu#4373)

Signed-off-by: Box-Cube <64300761+Box-Cube@users.noreply.github.com>

* Fix StatefulSet volumeClaimTemplates storageClassName(vmware-tanzu#4373)

Signed-off-by: Box-Cube <64300761+Box-Cube@users.noreply.github.com>

* Fix StatefulSet volumeClaimTemplates storageClassName(vmware-tanzu#4373)

Signed-off-by: Box-Cube <64300761+Box-Cube@users.noreply.github.com>

* Change the isStorageClassExist logic

Signed-off-by: Box-Cube <64300761+Box-Cube@users.noreply.github.com>

* Fix StatefulSet volumeClaimTemplates storageClassName(vmware-tanzu#4373)

Signed-off-by: Box-Cube <64300761+Box-Cube@users.noreply.github.com>
2021-12-16 14:50:56 +08:00
David L. Smith-Uchida
5677e04bb1 Consolidated code for resolving actions and plugins into ActionResolver (#4410)
* Consolidated code for resolving actions and plugins into ActionResolver.  Added BackupWithResolvers and
RestoreWithResolvers.  Introduces ItemSnapshooterResolver to bring ItemSnapshotter plugins into backup and
restore.  ItemSnapshotters are not used yet.

Added action_resolver_test

Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>

* Addressed review comments

Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
2021-12-10 12:53:47 -05:00
Daniel Jiang
2a7d4cec6e Add restoreactionitem plugin to handle admission webhook configurations
This commit adds a restore action item plugin to reset invalid value
of "sideEffects" in resource of mutatingwebhookconfiguration and
validating webhookconfiguration.

To fix the problem the "sideEffects" is illegal for resource migrated
from v1beta1.

fixes #3516

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-12-01 00:48:06 +08:00
Bruce
71d482360f Use factory.Namespace() to replace hardcoded velero namespace (#4346)
* Use factory.Namespace() to replace hardcoded velero namespace

Signed-off-by: half-life666 <half-life@jibudata.com>

* Add change log for pr 4346

Signed-off-by: half-life666 <half-life@jibudata.com>
2021-11-15 20:36:29 -05:00
Wenkai Yin(尹文开)
7c4e03e9f9 Ignore the provided port is already allocated error when restoring the NodePort service (#4336)
Fixes #2308

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2021-11-15 20:25:04 +08:00
Scott Seago
983489073f PV remapClaimRefNS was being skipped when there was no snapshot (#3708)
Signed-off-by: Scott Seago <sseago@redhat.com>
2021-11-09 20:30:16 -05:00
Wenkai Yin(尹文开)
305dfa0d3c Empty the "ClusterIPs" along with "ClusterIP" when "ClusterIP" isn't "None" (#4101)
More details please refer to #4098

Fixes #4098

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2021-09-03 10:33:50 -04:00
Daniel Jiang
240b4e666f Merge pull request #4026 from sseago/service-action-unnamed-nodeport
Distinguish between different unnamed node ports when preserving
2021-09-03 20:36:58 +08:00
Daniel Jiang
746cd616fd remove IsUnstructuredCRDReady (#4085)
This commit removes `IsUnstructuredCRDReady` since
kubernetes/kubernetes#87675 is fixed.
Is uses `Is1CRDReady` to check the readiness of CRD.

After v1.7 we may consider merge the funcx `IsV1Beta1CRDReady` and
`IsV1CRDReady`

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-09-01 13:38:17 +08:00
MatthieuFin
08e4138c16 Fix lint issue and test failed
Signed-off-by: MatthieuFin <matthieu2717@gmail.com>
2021-08-31 17:43:09 +02:00
MatthieuFin
effa09a42f Add full support for setting securityContext for restic restore container
Signed-off-by: MatthieuFin <matthieu2717@gmail.com>
2021-08-31 17:03:25 +02:00
Bridget McErlean
368098b727 Use LabelSelector in Action ResourceSelector
Instead of converting the unstructured item to check for the presence of
the `kube-aggregator.kubernetes.io/automanaged` label, use this label in
the `AppliesTo` to enable the restore logic to select the item. This
means that any item that matches the selector will have restore skipped.

Also add a new test case to the restore action test to check that label
selectors are applied correctly.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-08-11 17:33:31 -04:00
Bridget McErlean
984176f156 Skip restore of APIServices managed by Kubernetes
It was discovered during Velero 1.6.3 upgrade testing that Velero was
restoring `APIService` objects for APIs that are no longer being served
by Kubernetes 1.22. If these items were restored, it would break the
behaviour of discovery within the cluster.

This change introduces a new RestoreItemAction plugin that skips the
restore of any `APIService` object which is managed by Kubernetes such
as those for built-in APIs or CRDs. The `APIService`s for these will be
created when the Kubernetes API server starts or when new CRDs are
registered. These objects are identified by looking for the
`kube-aggregator.kubernetes.io/automanaged` label.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-08-10 18:22:00 -04:00
Scott Seago
8d714d38ea Distinguish between different unnamed node ports when preserving
Signed-off-by: Scott Seago <sseago@redhat.com>
2021-08-09 16:35:07 -04:00
Daniel Jiang
bc2484e9ab Merge pull request #3955 from alrs/fix-restore-err
pkg/restore: fix dropped error
2021-07-25 22:35:41 +08:00
Bridget McErlean
04345af17a Change restic image log messages to info level
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-07-15 15:29:49 -04:00
Lars Lehtonen
6eca2201d4 pkg/restore: fix dropped error
Signed-off-by: Lars Lehtonen <lars.lehtonen@gmail.com>
2021-07-14 18:22:02 -07:00
Bridget McErlean
198ea57407 Allow registry to be configured at build time
This adds a new `buildinfo` variable `ImageRegistry` that can set at
build time like the `Version` variable. This allows us to customise the
Velero binary to use different registries.

If the variable is set, this variable wille be used when creating the
URIs for both the main `velero` and `velero-restic-restore-helper`
images. If it is not set, default to using Dockerhub (`velero/velero`,
`velero/velero-restic-restore-helper`).

There are numerous ways in which the Velero binary can be built so all
of them have been updated to add the new link time flag to set the
variable:
* `make local` (used for local developer builds to build for the local
  OS and ARCH)
* `make build` (used by developers and also VMware internal builds to
  build a specific OS and ARCH)
* Goreleaser config (used when creating OSS release binaries)
* Dockerfile (used to build the Velero binary used within the image)

All of these workflows are currently triggered from our Makefile where
the variable `REGISTRY` is already available with the default value of
`velero` and used to build the image tag. Where the new `ImageRegistry`
build variable is needed, we pass through this Makefile variable to
those tasks so it can be used accordingly.

The GitHub action and the `./hack/docker-push.sh` script used to push
container images has not been modified. This will continue to use the
default registry specified in the Makefile and will not explicitly pass
it in.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-07-14 15:42:53 -04:00