Commit Graph
446 Commits
Author SHA1 Message Date
ming qiuandMing 58325050ec Adjust structure of e2e test codes
Put every test moduels into seperate packages and all velero install parameters integrated into one struct

Signed-off-by: Ming <mqiu@vmware.com>
2021-11-22 15:57:58 +08:00
Wenkai Yin(尹文开)andGitHub 474fd61283 Merge pull request #4376 from reasonerjt/pv-zone-gcp
Add the key for GKE zone for AZ collection
2021-11-22 10:14:04 +08:00
Daniel Jiang 748cf86aa7 Add the key for GKE zone for AZ collection
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-11-20 15:58:31 +08:00
danfengl e09837cc4c Fix snapshot e2e test issue of jsonpath
Signed-off-by: danfengl <danfengl@vmware.com>
2021-11-18 08:54:57 +00:00
Wenkai Yin(尹文开)andGitHub 11abff4e8d Merge pull request #4341 from blackpiglet/3737-add-logger-for-crd-manager
Fix: add init log option for velero controller-runtime manager.
2021-11-18 13:56:14 +08:00
fc29294552 fix: modify generated from schedule's backup name timestamp to UTC ti… (#4353)
* fix: modify generated from schedule's backup name timestamp to UTC timezone

fix #4279
When backup is created from schedule, and the backup name is not specified, a containing-timestamp generated name will be used. Due to velero client not set timezone to UTC, a local timezone will be used for the generated name.

Signed-off-by: Xun Jiang <jxun@vmware.com>

* fix: modify generated from schedule's backup name timestamp to UTC timezone

fix #4279
When backup is created from schedule, and the backup name is not specified, a containing-timestamp generated name will be used. Due to velero client not set timezone to UTC, a local timezone will be used for the generated name.

Signed-off-by: Xun Jiang <jxun@vmware.com>

* fix: modify generated from schedule's backup name timestamp to UTC timezone

fix #4279
When backup is created from schedule, and the backup name is not specified, a containing-timestamp generated name will be used. Due to velero client not set timezone to UTC, a local timezone will be used for the generated name.

Signed-off-by: Xun Jiang <jxun@vmware.com>

* modify changelog description

Reword the changelog description according to comments.

Signed-off-by: Xun Jiang <jxun@vmware.com>

Co-authored-by: jxun <jxun@jxun-a01.vmware.com>
Co-authored-by: Xun Jiang <jxun@vmware.com>
2021-11-17 09:26:49 -05:00
David L. Smith-UchidaandGitHub 5150ce4891 ItemSnapshotter plugin APIs. Addresses #3753 (#4077)
Added ItemSnapshotter.proto
Added item_snapshotter Go interface
Added framework components for item_snapshotter
Updated plugins doc with ItemSnapshotter info
Added SnapshotPhase to item_snapshotter.go
ProgressOutputOutput now includes a phase as well as an error string for problems that occured

Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
2021-11-16 16:13:31 -05:00
Scott SeagoandGitHub 3aa204a30d Merge pull request #4350 from reasonerjt/read-pv-az-new
Read Availability zone from nodeAffinity requirements
2021-11-16 08:43:27 -05:00
Xun Jiang 8eee35a62e Fix: add init log option for velero controller-runtime manager.
fix for issue #3737
add log option for velero controller-runtime manager to log return error in reconcile loop.

Signed-off-by: Xun Jiang jxun@vmware.com
Signed-off-by: Xun Jiang <jxun@vmware.com>
2021-11-16 15:42:00 +08:00
Xun Jiang 7b89950031 Fix: add init log option for velero controller-runtime manager.
fix for issue #3737
add log option for velero controller-runtime manager to log return error in reconcile loop.

Signed-off-by: Xun Jiang jxun@vmware.com
Signed-off-by: Xun Jiang <jxun@vmware.com>
2021-11-16 15:37:43 +08:00
Daniel Jiang 5878a52843 Read Availability zone from nodeAffinity requirements
Velero to read the AZ info from `NodeAffinity` of a PV when it's taking
the snapshot.

Fixes #4332

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-11-16 09:44:12 +08:00
BruceandGitHub 71d482360f Use factory.Namespace() to replace hardcoded velero namespace (#4346)
* Use factory.Namespace() to replace hardcoded velero namespace

Signed-off-by: half-life666 <half-life@jibudata.com>

* Add change log for pr 4346

Signed-off-by: half-life666 <half-life@jibudata.com>
2021-11-15 20:36:29 -05:00
Wenkai Yin(尹文开)andGitHub 7c4e03e9f9 Ignore the provided port is already allocated error when restoring the NodePort service (#4336)
Fixes #2308

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2021-11-15 20:25:04 +08:00
Daniel JiangandGitHub 130602d723 Return the error if velero failed to detect S3 region for restic repo (#4343)
The error should be returned explicitly, because when the default URL is
used S3 will return a 301 and the response can't be handled by restic.

Fixes #4178

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-11-15 20:20:27 +08:00
Scott SeagoandGitHub 983489073f PV remapClaimRefNS was being skipped when there was no snapshot (#3708)
Signed-off-by: Scott Seago <sseago@redhat.com>
2021-11-09 20:30:16 -05:00
Bridget McErleanandGitHub e4019f26c1 Only set BSL credential field if provided (#4322)
Previously, the BSL credential field would always be set when using the
`create` command, even if no credential details were provided. This
would result in an empty `SecretKeySelector` in the BSL which would
cause operations using this BSL to fail as Velero would attempt to fetch
a `Secret` with an empty name from the K8s API server.

With this change, the `Credential` field is only set if credential
details have been specified. This change also includes some refactoring
to allow the change to be tested.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-11-09 17:04:58 -08:00
Wenkai Yin(尹文开)andGitHub 6801ddc9ac Merge pull request #4306 from alaypatel07/fix-paging
fix buggy pager func
2021-11-10 07:53:58 +08:00
Alay Patel 569fc1dc5b add 4306 changelog
Signed-off-by: Alay Patel <alay1431@gmail.com>
2021-11-09 10:34:26 -05:00
Scott SeagoandGitHub 550efddd88 Verify group before treating resource as cohabitating (#4126)
Signed-off-by: Scott Seago <sseago@redhat.com>
2021-11-03 18:11:32 -04:00
Wenkai Yin(尹文开)andGitHub 9f0ea22c60 Fix CVE-2020-29652 and CVE-2020-26160 (#4274)
Bump up restic to v0.12.1 to fix CVE-2020-26160.
Bump up module "github.com/vmware-tanzu/crash-diagnostics" to v0.3.7 to fix CVE-2020-29652.
The "github.com/vmware-tanzu/crash-diagnostics" updates client-go to v0.22.2 which introduces several break changes, this commit updates the related codes as well

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2021-11-03 13:30:26 -07:00
Wenkai Yin(尹文开)andGitHub 163e96b62d Don't create a backup immediately after creating a schedule (#4281)
Don't create a backup immediately after creating a schedule
Fixes #1980

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2021-10-26 16:31:58 -07:00
Daniel Jiang d08c4bae4d Refine tag-release.sh
This commit makes several changes to `tag-release.sh` according to the
change in release process:
1. It will support a "ON_RELEASE_BRANCH" param passed via env variable.
   When it's set to "TRUE". The release will be created on the commit of
   branch like `release-xxx`.  This enables us to create release branch
   before GA and tag RC release.
2. It removes the code to push a new branch to upstream.  This is
   because we decided to create branch manually.  For patch releases, we
   will not push the change to release branch, instead, we will make
   sure the release branch has all commits cherrypicked BEFORE we run
   this script to tag the release.

After the change the script will focus on only tag the release, not
making other code change to release branches.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-09-22 21:20:03 +08:00
danfengliuandGitHub 8827b4f1d9 Fix plugins incompatible issue in upgrade test (#4141)
In upgrade test, both original and to-be-upgrading velero installation should use the compatible plugins, but currently, plugin value is determined by provider.

Signed-off-by: danfengl <danfengl@vmware.com>
2021-09-21 19:39:35 -07:00
Daniel JiangandGitHub b9eb6fdb33 Merge pull request #4107 from zubron/release-1.7-rc.1
Generate docs and changelog for v1.7
2021-09-09 08:53:25 +08:00
danfengl 4db866a0b2 Add upgrade test into E2E tests
Signed-off-by: danfengl <danfengl@vmware.com>
2021-09-08 13:16:53 +00:00
Bridget McErlean 5e50a50e84 Generate 1.7 changelog
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-09-07 23:03:12 -04:00
codegold79andGitHub fbd6bcf504 Validate namespace in velero backup create command (#4057)
* Add namespace validation in the client

Signed-off-by: F. Gold <fgold@vmware.com>

* Add namespace validation in the backup controller

Signed-off-by: F. Gold <fgold@vmware.com>

* Add changelog for PR 4057

Signed-off-by: F. Gold <fgold@vmware.com>

* Update Copyright notice

Signed-off-by: F. Gold <fgold@vmware.com>

* Update include_excludes_test.go to follow Go standards and be easier to read

Signed-off-by: F. Gold <fgold@vmware.com>

* Add unit tests for namespace validation functions

Signed-off-by: F. Gold <fgold@vmware.com>

* Make changes per review comments

- use one set of namespace validation logic instead of writing two
- remove duplicate namespace validation functions and tests
- add namespace validation tests in includes_excludes_test.go

Signed-off-by: F. Gold <fgold@vmware.com>

* Return all ns validation err msgs as error list

Signed-off-by: F. Gold <fgold@vmware.com>

* Make error message more clear

Signed-off-by: F. Gold <fgold@vmware.com>
2021-09-03 11:03:35 -04:00
Wenkai Yin(尹文开)andGitHub 305dfa0d3c Empty the "ClusterIPs" along with "ClusterIP" when "ClusterIP" isn't "None" (#4101)
More details please refer to #4098

Fixes #4098

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2021-09-03 10:33:50 -04:00
Daniel JiangandGitHub 240b4e666f Merge pull request #4026 from sseago/service-action-unnamed-nodeport
Distinguish between different unnamed node ports when preserving
2021-09-03 20:36:58 +08:00
Bridget McErleanandGitHub edeec848d3 Skip restic backup/restore of DownwardAPI volumes (#4076)
Velero was including DownwardAPI volumes when backing up with restic.
When restoring these volumes, it triggered a known issue with restic (as
seen in #3863). Like projected volumes, these volumes should be skipped
as their contents are populated by the Kubernetes API server.

With this change, we are now skipping the restic backup of volumes with
a DownwardAPI source. We are also skipping the restore of any volume
that had a DownwardAPI source as there will exist backups that were
taken prior to this fix being introduced. This will allow these backups
to be restored succesfully.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-09-01 13:51:44 +08:00
Daniel JiangandGitHub 7c75cd6cf8 Implement velero debug (#4022)
This PR added a subcommand `velero debug`, which leverages `crashd` to
collect logs and specs of velero server components and bundle them in a
tarball.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-08-30 18:37:18 -07:00
kahirokunn cf42054c8c add changelog
Signed-off-by: kahirokunn <okinakahiro@gmail.com>
2021-08-24 17:13:11 +09:00
Wenkai Yin(尹文开)andGitHub 23e0e5d18f Change the base image to distroless (#4055)
Change the base image to distroless

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2021-08-20 16:01:06 +08:00
Dharma BellamkondaandGitHub dc1f17944e Page list requests by default (#3823)
Signed-off-by: Dharma Bellamkonda <bellamko@adobe.com>
2021-08-17 18:49:41 -07:00
Wenkai Yin(尹文开)andGitHub 31a8e5080c Wait the namespace deletion completed before removing the CRDs (#4007)
Wait the namespace deletion completed before removing the CRDs when uninstalling the velero

Fixes #3974

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2021-08-17 08:55:22 -07:00
Bridget McErlean 984176f156 Skip restore of APIServices managed by Kubernetes
It was discovered during Velero 1.6.3 upgrade testing that Velero was
restoring `APIService` objects for APIs that are no longer being served
by Kubernetes 1.22. If these items were restored, it would break the
behaviour of discovery within the cluster.

This change introduces a new RestoreItemAction plugin that skips the
restore of any `APIService` object which is managed by Kubernetes such
as those for built-in APIs or CRDs. The `APIService`s for these will be
created when the Kubernetes API server starts or when new CRDs are
registered. These objects are identified by looking for the
`kube-aggregator.kubernetes.io/automanaged` label.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-08-10 18:22:00 -04:00
Scott Seago 8d714d38ea Distinguish between different unnamed node ports when preserving
Signed-off-by: Scott Seago <sseago@redhat.com>
2021-08-09 16:35:07 -04:00
Bridget McErleanandGitHub d98c65f69e Use appropriate CRD API during readiness check (#4015)
* Use appropriate CRD API during readiness check

The readiness check for the Velero CRDs was still using the v1beta1 API.
This would cause the readiness check to fail on 1.22 clusters as the
v1beta1 API is no longer available. Previously, this error would be
ignored and the installation would proceed, however with #4002, we are
no longer ignoring errors from this check.

This change modifies the CRD readiness check to check the CRDs using the
same API version that was used when submitting the CRDs to the cluster.
It also introduces a new CRD builder using the V1 API for testing.

This change also fixes a bug that was identified in the polling code
where if the CRDs were not ready on the first polling iteration, they
would be added again to the list of CRDs to check resulting in
duplicates. This would cause the length check to fail on all subsequent
polls and the timeout would always be reached.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>

* Remove duplicate V1 CRD builder and update comment

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-08-05 15:17:14 -04:00
Marcel HauptandGitHub 7cb614789c Restic Backup: Add check if Pod is successful or failed for emptyDir.… (#3993)
* Restic Backup: Add check if Pod is successful or failed for emptyDir. Fix #3812
* Add changelog

Signed-off-by: mahaupt <marcel.haupt@gepardec.com>
2021-08-04 12:53:06 +08:00
Daniel JiangandGitHub 6ae9a6eb37 Bump up Go to 1.16 (#3990)
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-07-29 23:56:08 -07:00
Wenkai Yin(尹文开)andGitHub d8141eabce Select the velero deployment with both label and container name (#3996)
Select the velero deployment with both label and container name

Fixes #3961

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2021-07-29 12:01:48 -04:00
Scott Seago 2c46b47404 enable e2e tests to choose crd apiVersion
Signed-off-by: Scott Seago <sseago@redhat.com>
2021-07-28 17:40:30 -04:00
JenTing HsiaoandBridget McErlean ee920ea9bc Add changelog
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
2021-07-27 17:55:34 -04:00
Jai Subash DevmaneandGitHub 36b1aaa99d Fixing multipleNamespaceTest bug - Missing expect statement in test (#3983)
* Fixing multipleNamespaceTest bug - No expect statement in test
* adding changelog

Signed-off-by: Jai Subash Devmane <jdevmane@redhat.com>
2021-07-27 15:32:36 +08:00
Wenkai Yin(尹文开)andGitHub 70526806f1 Merge pull request #3919 from jaidevmane/installVeleroFix
Fix -install-velero flag for e2e tests
2021-07-20 18:57:12 +08:00
Scott SeagoandGitHub ff916b74e9 Merge pull request #3857 from zubron/use-region-in-bsl-for-restic-repo-identifier
Use region in BSL for restic repo identifier
2021-07-13 10:56:20 -04:00
JenTing Hsiao 25a72f90db Add changelog
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
2021-07-06 17:04:00 +08:00
Jai Subash Devmane db375f34c1 Fix -install-velero flag for e2e tests
Signed-off-by: Jai Subash Devmane <jdevmane@redhat.com>
2021-07-02 17:21:36 -04:00
Wenkai Yin(尹文开) bb05ed390b Enable the E2E test on Github Action
1. Run the E2E test with kind(provision various versions of k8s cluster) and MinIO on Github Action
2. Bug fix: the variable "stdoutBuf" is assigned to both "installPluginCmd.Stdout" and "installPluginCmd.Stderr", this causes 'if !strings.Contains(stderrBuf.String(), "Duplicate value")' takes no effect as the "stderrBuf.String()" is always empty
3. Print the stdout and stderr for easy debugging

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2021-06-30 21:45:26 +08:00
Scott SeagoandGitHub 962a957f1b regression introduced in 1.6 restore progress: fix CR restore (#3845)
Signed-off-by: Scott Seago <sseago@redhat.com>
2021-06-19 00:03:11 -07:00