300 Commits

Author SHA1 Message Date
danfengl
c9ba808bf1 Fix wrong usage of global velerocfg var, further PR
Signed-off-by: danfengl <danfengl@vmware.com>
2024-02-07 02:32:20 +00:00
danfengl
b0956322b9 Fix wrong usage of global varaible VeleroCfg
Signed-off-by: danfengl <danfengl@vmware.com>
2024-02-07 02:32:20 +00:00
Tiger Kaovilai
b1d95cf2aa Set GOBIN so Makefile don't modify $PATH on go install Fix realPath resolving when cloud credentials is prefixed by ~ for home dir Use ~/.docker/config.json if REGISTRY_CREDENTIAL_FILE not defined and skip step if does not exists since it is optional
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Set `GOBIN` so Makefile don't modify $PATH on `go install`
Fix realPath resolving when cloud credentials is prefixed by `~` for home dir
Use `~/.docker/config.json` if REGISTRY_CREDENTIAL_FILE not defined and skip step if does not exists since it is optional

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Add kind testdata storageclass

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Add kind testdata storageclass

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

log `Start to install Azure VolumeSnapshotClass ...` only on azure when csi is enabled

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Add BSL_CONFIG example and notes

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Makefile: Set `GOBIN` for `_output/...`

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

README spacing

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

StandbyClusterObjectStoreProvider typo

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Specify velero namespace during get/delete command

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Use object stores rather than cloudProvider for bucket queries

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Remove debug print

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

simplify NS get changes, add velero NS to `DeleteBackupResource`

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Skip file system backups on kind which uses hostPath volumes

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Add StorageClass change test to PR kind e2e

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Add more tests to pr

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Add NS mapping to PR e2e

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Add `SKIP_KIND` to some jobs containing volumes

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Remove kind from kibishii tests

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Label volume resource policies as restic, skip restic/snapshot tests, add more tests

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

TTLTest is a snapshot test

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Remove non working tests

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Resolve https://github.com/vmware-tanzu/velero/pull/7353#issuecomment-1925660077

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

address https://github.com/vmware-tanzu/velero/pull/7353/files#r1477218762

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Address https://github.com/vmware-tanzu/velero/pull/7353#issuecomment-1923414840

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2024-02-04 22:17:37 -05:00
danfengl
72438b7319 Support IRSA for data mover pipeline
Signed-off-by: danfengl <danfengl@vmware.com>
2024-02-02 02:04:26 +00:00
danfengl
df585053e7 Add a new EKS pipeline with IRSA as credential
Signed-off-by: danfengl <danfengl@vmware.com>
2024-01-26 02:17:21 +00:00
Guillaume Delacour
373b24e2c1 Upgrade AWS SDK
Signed-off-by: Guillaume Delacour <delacoug@amazon.com>
2024-01-16 23:35:33 +01:00
danfengl
a9c820c9d6 Fix nightly issue of missing param WithoutDisableInformerCacheParam during Velero installation
Signed-off-by: danfengl <danfengl@vmware.com>
2024-01-10 02:57:44 +00:00
Xun Jiang/Bruce Jiang
e65ef28948 Merge pull request #7272 from danfengliu/bumpup-plugins-matrix-for-1.13
Bumpup E2E test plugins matirx for v1.13
2024-01-09 10:38:36 +08:00
danfengl
1b22a49d22 Bumpup E2E test plugins matirx for v1.13
Signed-off-by: danfengl <danfengl@vmware.com>
2024-01-09 02:26:53 +00:00
danfengl
fdca488209 Add param disable informer cache for velero installation
Signed-off-by: danfengl <danfengl@vmware.com>
2024-01-05 07:22:50 +00:00
qiuming
f6dfa8e7b2 Merge pull request #7176 from danfengliu/fix-issue-of-hiiting-snapshot-limit
Add sleep to avoid snapshot limitation issue
2024-01-02 17:09:43 +08:00
danfengl
b25578d6e1 Add sleep to avoid snapshot limitation issue and skip retain PV on vSphere pipeline
1. Add sleep to avoid snapshot limitation issue https://docs.aws.amazon.com/AWSEC2/latest/APIReference/errors-overview.html#:~:text=SnapshotCreationPerVolumeRateExceeded;
2. Move InstallVelero variable out of struct of Veleroconfig as a global one since it's not for controlling any individual case;
3. Unskip migration test case on AWS pipeline, because we added a new EKS pipeline and deleted TKG AWS pipline in internal E2E test, so this restriction for TKG AWS pipline is no long existed;
4. Skip retainPV test on vSphere pipeline due to PV longtime bounding issue;
5. Fix failing get snapshot by CSI from EC2 issue, snapshot by CSI has no label of backup name.

Signed-off-by: danfengl <danfengl@vmware.com>
2024-01-02 05:53:03 +00:00
danfengliu
49e3e545be Merge pull request #7048 from danfengliu/add-readme-for-e2e-test
Update E2E README file to latest
2023-12-06 16:53:13 +08:00
yanggang
9ccb5a14bb Fix test code wrong code for VeleroInstall
Signed-off-by: yanggang <gang.yang@daocloud.io>
2023-11-27 11:13:52 +00:00
qiuming
ccd3f220ad Merge pull request #7090 from qiuming-best/perf-test-0
Enhance perf test
2023-11-27 16:10:26 +08:00
Ming
507157f812 Add perf test namespace mapping when restore
Signed-off-by: Ming <mqiu@vmware.com>
2023-11-27 02:11:13 +00:00
danfengl
4590579105 Update E2E README file to latest
Signed-off-by: danfengl <danfengl@vmware.com>
2023-11-25 12:37:21 +00:00
danfengl
55a465a941 Add E2E test for taking CSI snapshot to PV with retain reclaim policy
Signed-off-by: danfengl <danfengl@vmware.com>
2023-11-21 07:11:22 +00:00
Xun Jiang
56b5e982d9 Remove dependency of generated client part 3
Replace generated discovery client with client-go client.
Remove generated client from PVR action.
Remove generated client from pkg/cmd directory.
Delete velero generate client from client factory.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2023-11-06 11:34:39 +08:00
Daniel Jiang
b71d2b3898 Bump up aws sdk to aws-sdk-go-v2
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2023-10-24 17:01:26 +08:00
danfengl
d2fc9fa1a9 Fix fail to get backup repo due to missing api group name issue
Signed-off-by: danfengl <danfengl@vmware.com>
2023-10-20 01:50:24 +00:00
Yang Gang
e5e99c75a0 Fix dep package describe and ci words spell. (#6924)
Signed-off-by: yanggang <gang.yang@daocloud.io>
2023-10-09 12:12:14 +05:30
Guang Jiong Lou
5dd7c5cd46 add label selector in Resource Modifiers (#6704)
* add label selector in resource modifier

Signed-off-by: lou <alex1988@outlook.com>

* add ut

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

---------

Signed-off-by: lou <alex1988@outlook.com>
2023-08-31 10:36:59 +05:30
danfengl
15be42f47b monitor velero logs and fix E2E issues
1. Capture Velero pod log and K8S cluster event;
2. Fix wrong path of storageclass yaml file issue caused by pert test;
3. Fix change storageclass test issue that no sc named 'default' in EKS cluster;
4. Support AWS credential as config format;
5. Support more E2E script input parameters like standy cluster plugins and provider.

Signed-off-by: danfengl <danfengl@vmware.com>
2023-08-28 05:53:32 +00:00
danfengl
3a00945b44 Add default plugin setting for Velero v1.12 in E2E test
Signed-off-by: danfengl <danfengl@vmware.com>
2023-08-18 09:05:46 +00:00
danfengl
172166749e Replace pod with deployment in E2E test
Signed-off-by: danfengl <danfengl@vmware.com>
2023-08-18 01:57:43 +00:00
Ming
3f05a7dc1d Fix resource modifiers case
Signed-off-by: Ming <mqiu@vmware.com>
2023-08-16 03:34:59 +00:00
Anshul Ahuja
e753a08f85 E2E test for Resource Modifier & Bug fixes (#6483)
* Add resource modifier e2e test and bug fixes

Signed-off-by: Anshul Ahuja <anshulahuja@microsoft.com>
2023-08-16 11:09:59 +08:00
Ming
f890033ee8 Merge branch 'main' of https://github.com/qiuming-best/velero into perf-test 2023-08-15 07:57:01 +00:00
Ming Qiu
3b45830012 Add performance E2E test
Signed-off-by: Ming Qiu <mqiu@vmware.com>
2023-08-15 01:46:36 +00:00
danfengl
c637057dab The minimum worker node count of wowrkload cluster should be 2
Signed-off-by: danfengl <danfengl@vmware.com>
2023-08-06 08:35:38 +00:00
Xun Jiang/Bruce Jiang
605eab1eb8 Merge pull request #6531 from yanggangtony/delete-io-util
delete io/ioutil package as it is deprecated
2023-07-24 09:42:55 +08:00
Mateus Oliveira
913b610196 fix: Typos and add more spell checking rules to CI (#6415)
* fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

* fixup! fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

* fixup! fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

* fixup! fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

* fixup! fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

* fixup! fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

* fixup! fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

* fixup! fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

---------

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>
2023-07-21 17:25:24 -04:00
yanggang
bcc69f33f9 delete io/ioutil package as it is deprecated
Signed-off-by: yanggang <gang.yang@daocloud.io>
2023-07-21 15:18:31 +08:00
danfengl
35697a9509 Support API Version V1 for CR volumesnapshotclass in E2E test
1. In K8S v1.27 API Version v1beta1 for CR volumesnapshotclass is deprcated, so E2E test should adapt both API versions to cover all K8S versio;
2. Support getting additional plugin from input;
3. Velero version and plugin map should not deprated version older than v1.10, because upgrade test will use them.

Signed-off-by: danfengl <danfengl@vmware.com>
2023-07-20 08:57:15 +00:00
danfengl
bc8742566b Install plugin for datamover pipeline
Signed-off-by: danfengl <danfengl@vmware.com>
2023-07-10 02:02:50 +00:00
danfengliu
d7181fba55 Merge pull request #6323 from blackpiglet/psa_violation_fix_release_main
[main]Make the E2E testing pods obey the restricted pod security standard.
2023-06-08 15:16:11 +08:00
danfengl
028e784eb6 Prepare E2E test of data movement feature
1. Bumpup velero version to the latest 2 versions in upgrade script;
2. Bumpup velero verioin to the latest 1 vesion in migration script;
3. Bring B/R with restic test back in vSphere pipeline since vSphere plugin issue fix was included
in v1.5;
4. Disable nodeport test in AWS pipeline since AWS k8s version bumpup;
5. Prepare for data mover test, allow object store provider diffrent from cloud provider.

Signed-off-by: danfengl <danfengl@vmware.com>
2023-06-05 01:49:08 +00:00
Ming
2f6899e5a7 Optimize Velero install
Signed-off-by: Ming <mqiu@vmware.com>
2023-06-01 07:09:38 +00:00
Xun Jiang
754f02c40d Make the E2E testing pods obey the restricted pod security standard.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-05-30 15:36:44 +08:00
danfengl
3222df9ae6 Debug restic
Signed-off-by: danfengl <danfengl@vmware.com>
2023-05-25 10:31:17 +00:00
Xun Jiang
98baaa9e2f Add file system group setting for deployment and pod used in E2E test.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-05-19 11:30:44 +08:00
danfengliu
72142a9f0f Merge pull request #6278 from qiuming-best/e2e-context
Adjust E2E context
2023-05-18 09:50:09 +08:00
Xun Jiang/Bruce Jiang
7417e5b5f7 Make the E2E testing pods obey the restricted pod security standard. (#6279)
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
Co-authored-by: Xun Jiang <blackpiglet@gmail.com>
2023-05-17 10:25:38 -04:00
Ming
54d6cffb45 Adjust E2E context
Signed-off-by: Ming <mqiu@vmware.com>
2023-05-17 14:05:15 +00:00
qiuming
7ed286d886 Merge pull request #6274 from qiuming-best/e2e-time-optimize
Adjust Sleep time in E2E
2023-05-17 15:59:03 +08:00
lyndon
5b75f35262 Add data mover CRD under v2alpha1 (#6176)
* add data mover CRD under v1alpha1

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* data mover CRDs to v2alpha1

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

* data mover crd changes

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>

---------

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2023-05-16 15:09:54 -04:00
Ming
c12a0ac731 Adjust Sleep time in E2E
Signed-off-by: Ming <mqiu@vmware.com>
2023-05-16 03:33:51 +00:00
Ming
a16c17b1e3 Adjust E2E test codes
Signed-off-by: Ming <mqiu@vmware.com>
2023-05-15 03:28:56 +00:00
qiuming
a88163f308 Merge pull request #6198 from danfengliu/fix-context-withconcel-issue
Fix context issues for several E2E tests
2023-05-05 11:20:34 +08:00