Commit Graph

55 Commits

Author SHA1 Message Date
Priyansh Choudhary
ff8a070dd3 Refactor Snapshot Checkpoint in E2E Tests (#9083)
Some checks failed
Run the E2E test on kind / build (push) Failing after 8m19s
Run the E2E test on kind / setup-test-matrix (push) Successful in 4s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / Build (push) Failing after 35s
Close stale issues and PRs / stale (push) Successful in 21s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 1m54s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-aws, main) (push) Failing after 1m25s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-gcp, main) (push) Failing after 1m33s
Trivy Nightly Scan / Trivy nightly scan (velero-plugin-for-microsoft-azure, main) (push) Failing after 1m26s
* Refactor backup volume info retrieval and snapshot checkpoint building in e2e tests
Signed-off-by: Priyansh Choudhary <im1706@gmail.com>

log backup volume info retrieval and snapshot checkpoint building
Signed-off-by: Priyansh Choudhary <im1706@gmail.com>

Add error handling for volume info retrieval in backup tests
Signed-off-by: Priyansh Choudhary <im1706@gmail.com>

Add error handling for volume info retrieval in backup tests
Signed-off-by: Priyansh Choudhary <im1706@gmail.com>

* Update snapshot checkpoint building to use DefaultKibishiiWorkerCounts
Signed-off-by: Priyansh Choudhary <im1706@gmail.com>

---------

Signed-off-by: Priyansh Choudhary <im1706@gmail.com>
2025-07-21 20:22:55 +05:30
Xun Jiang
a18fe55585 Support Windows E2E.
* Please notice only Kibishii workload support Windows test,
because the other work loads use busybox image, and not support Windows.
* Refactor CreateFileToPod to support Windows.
* Add skip logic for migration test if the version is under 1.16.
* Add main in semver check.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2025-05-20 15:58:15 +08:00
Xun Jiang
14e1055a9a Support using image registry proxy in more cases.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2025-05-14 08:47:00 +08:00
Xun Jiang
e5354e123b Modify the StorageClass install and delete code.
* Only install and uninstall SC and VSC once for default cluster.
* Install and uninstall SC and VSC for standby cluster on migration case.
* Refactor the StorageClass and VolumeSnapshotClass YAMLs.
* Prettify the e2e_suite_test.go

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-11-19 11:10:50 +08:00
Xun Jiang
bcb60ed783 Modify other cases to support VKS environment.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-11-12 23:25:50 +08:00
Xun Jiang
b02fc1da96 E2E supports VKS data mover environment.
* Add new flag HAS_VSPHERE_PLUGIN for E2E test.
* Modify the E2E README for the new parameter.
* Add the VolumeSnapshotClass for VKS.
* Modify the plugin install logic.
* Modify the cases to support data mover case in VKS.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-11-12 23:25:28 +08:00
Xun Jiang
1ba78b83bf Skip uninstall and resource cleanup when fail-fast is enabled.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2024-09-23 20:24:14 +08:00
Xun Jiang
7a3b947961 Bump Ginkgo to v2.
Signed-off-by: Xun Jiang <blackpigletbruce@gmail.com>
2024-07-17 15:31:23 +08:00
danfengl
61c4d7b148 Add checkpoint for FS backup deletion test
As per PR #7281, if repository count is more than 1, then snapshots deletion is achieved with a fast way, then we should have more than 1 FS backup repository per backup.

Signed-off-by: danfengl <danfengl@vmware.com>
2024-05-07 07:52:46 +00:00
danfengl
82fc557bd1 Add checkpoint of VSC for data movement migration test
1. In data movement scenario, volumesnapshotcontent by Velero backup will be deleted instead of retained in CSI scenaito, so add
a checkpoint for data movement scenario to verify no volumesnapshotcontent left after Velero backup;

2. Fix global context varaible issue, context varaible is not effective due to it's initialized right after the very beginning of
all tests instead of beginning of each test, so if someone script a new E2E test and did not overwrite it in the test body, then it
will fail the test if it was triggerd one hour later;

3. Due to CSI plugin is deprecated, it breaked down migration tests, because v1.13 still needs to install CSI plugin for the test.

Signed-off-by: danfengl <danfengl@vmware.com>
2024-04-28 09:47:08 +00:00
eveneast
d0350960b6 Fix some comments
Signed-off-by: eveneast <qcqs@foxmail.com>
2024-04-15 14:48:20 +08:00
Matthieu MOREL
ef04ef6361 golangci-lint: use exclude-rules instead of skip-files and skip-dirs
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-03-27 20:17:34 +00:00
danfengl
7c50c3cb8c using zfs for vanilla cluster test
Signed-off-by: danfengl <danfengl@vmware.com>
2024-02-27 06:39:33 +00:00
danfengl
b0956322b9 Fix wrong usage of global varaible VeleroCfg
Signed-off-by: danfengl <danfengl@vmware.com>
2024-02-07 02:32:20 +00:00
Tiger Kaovilai
b1d95cf2aa Set GOBIN so Makefile don't modify $PATH on go install Fix realPath resolving when cloud credentials is prefixed by ~ for home dir Use ~/.docker/config.json if REGISTRY_CREDENTIAL_FILE not defined and skip step if does not exists since it is optional
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Set `GOBIN` so Makefile don't modify $PATH on `go install`
Fix realPath resolving when cloud credentials is prefixed by `~` for home dir
Use `~/.docker/config.json` if REGISTRY_CREDENTIAL_FILE not defined and skip step if does not exists since it is optional

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Add kind testdata storageclass

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Add kind testdata storageclass

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

log `Start to install Azure VolumeSnapshotClass ...` only on azure when csi is enabled

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Add BSL_CONFIG example and notes

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Makefile: Set `GOBIN` for `_output/...`

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

README spacing

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

StandbyClusterObjectStoreProvider typo

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Specify velero namespace during get/delete command

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Use object stores rather than cloudProvider for bucket queries

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Remove debug print

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

simplify NS get changes, add velero NS to `DeleteBackupResource`

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Skip file system backups on kind which uses hostPath volumes

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Add StorageClass change test to PR kind e2e

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Add more tests to pr

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Add NS mapping to PR e2e

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Add `SKIP_KIND` to some jobs containing volumes

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Remove kind from kibishii tests

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Label volume resource policies as restic, skip restic/snapshot tests, add more tests

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

TTLTest is a snapshot test

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Remove non working tests

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Resolve https://github.com/vmware-tanzu/velero/pull/7353#issuecomment-1925660077

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

address https://github.com/vmware-tanzu/velero/pull/7353/files#r1477218762

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Address https://github.com/vmware-tanzu/velero/pull/7353#issuecomment-1923414840

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2024-02-04 22:17:37 -05:00
danfengl
b25578d6e1 Add sleep to avoid snapshot limitation issue and skip retain PV on vSphere pipeline
1. Add sleep to avoid snapshot limitation issue https://docs.aws.amazon.com/AWSEC2/latest/APIReference/errors-overview.html#:~:text=SnapshotCreationPerVolumeRateExceeded;
2. Move InstallVelero variable out of struct of Veleroconfig as a global one since it's not for controlling any individual case;
3. Unskip migration test case on AWS pipeline, because we added a new EKS pipeline and deleted TKG AWS pipline in internal E2E test, so this restriction for TKG AWS pipline is no long existed;
4. Skip retainPV test on vSphere pipeline due to PV longtime bounding issue;
5. Fix failing get snapshot by CSI from EC2 issue, snapshot by CSI has no label of backup name.

Signed-off-by: danfengl <danfengl@vmware.com>
2024-01-02 05:53:03 +00:00
Yang Gang
e5e99c75a0 Fix dep package describe and ci words spell. (#6924)
Signed-off-by: yanggang <gang.yang@daocloud.io>
2023-10-09 12:12:14 +05:30
danfengl
15be42f47b monitor velero logs and fix E2E issues
1. Capture Velero pod log and K8S cluster event;
2. Fix wrong path of storageclass yaml file issue caused by pert test;
3. Fix change storageclass test issue that no sc named 'default' in EKS cluster;
4. Support AWS credential as config format;
5. Support more E2E script input parameters like standy cluster plugins and provider.

Signed-off-by: danfengl <danfengl@vmware.com>
2023-08-28 05:53:32 +00:00
Ming
f890033ee8 Merge branch 'main' of https://github.com/qiuming-best/velero into perf-test 2023-08-15 07:57:01 +00:00
Ming Qiu
3b45830012 Add performance E2E test
Signed-off-by: Ming Qiu <mqiu@vmware.com>
2023-08-15 01:46:36 +00:00
danfengl
c637057dab The minimum worker node count of wowrkload cluster should be 2
Signed-off-by: danfengl <danfengl@vmware.com>
2023-08-06 08:35:38 +00:00
Mateus Oliveira
913b610196 fix: Typos and add more spell checking rules to CI (#6415)
* fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

* fixup! fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

* fixup! fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

* fixup! fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

* fixup! fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

* fixup! fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

* fixup! fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

* fixup! fix: Typos and add more spell checking rules to CI

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>

---------

Signed-off-by: Mateus Oliveira <msouzaol@redhat.com>
2023-07-21 17:25:24 -04:00
danfengl
bc8742566b Install plugin for datamover pipeline
Signed-off-by: danfengl <danfengl@vmware.com>
2023-07-10 02:02:50 +00:00
Ming
2f6899e5a7 Optimize Velero install
Signed-off-by: Ming <mqiu@vmware.com>
2023-06-01 07:09:38 +00:00
Ming
c12a0ac731 Adjust Sleep time in E2E
Signed-off-by: Ming <mqiu@vmware.com>
2023-05-16 03:33:51 +00:00
Ming
a16c17b1e3 Adjust E2E test codes
Signed-off-by: Ming <mqiu@vmware.com>
2023-05-15 03:28:56 +00:00
danfengl
5f5db2eaca Fix context issues for several E2E tests
1. Fix context issues produced by previous PR, increase timeout or add case scpoed global timeout param to make  backup/restore command timeout configurable.
2. Add global param for storage class name using by test cases;
3. Fix param DefaultVolumesToFsBackup usage issue: set DefaultVolumesToFsBackup to false in backup CLI  in case it was set to true in install CLI.
4. Make namespace names of each namespace mapping test unique from being interfered by each other.

Signed-off-by: danfengl <danfengl@vmware.com>
2023-05-05 02:09:21 +00:00
Tiger Kaovilai
d6848ffb16 Self contain context in each high level function rather than passing around in structs
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-04-16 14:22:04 -04:00
Tiger Kaovilai
3f4b258dee defer ctxCancel
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2023-04-16 13:33:43 -04:00
Xun Jiang
fc692c49e6 Update Golang version to v1.19 for branch main.
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2023-04-04 12:00:19 +08:00
danfengl
112775f924 Add changing PVC selected node E2E test
Signed-off-by: danfengl <danfengl@vmware.com>
2023-03-29 01:36:45 +00:00
danfengl
beed887eeb Add snapshotvolumes default behavior E2E test
Signed-off-by: danfengl <danfengl@vmware.com>
2023-02-19 09:05:20 +00:00
Fish-pro
843c70959f Clean up unnecessary convert (#5819)
Signed-off-by: Fish-pro <zechun.chen@daocloud.io>
2023-02-06 14:50:31 +08:00
danfengliu
6eccaa4cf5 Add pause/unpause schedule e2e test (#5609)
Signed-off-by: danfengl <danfengl@vmware.com>
2022-11-22 17:38:41 +08:00
danfengl
78dae45c52 Update upgrade test for 1.10
Signed-off-by: danfengl <danfengl@vmware.com>
2022-11-10 03:18:01 +00:00
danfengl
cf32cabddd fix-ctx-issue
1. Fix issue of kubectl client and server mismatch version in GitAction E2E job, refer to https://github.com/elastic/cloud-on-k8s/issues/4737;
2. Adapt to the changing of keyword for involing Kpoia as fs backupper, new installtion breaked upgrade and migration tests;
3. Accept multi-labels of Ginkgo focus as input of E2E make command;
4. Distinguish workload namespace from each tests;
5. Fix issues of not using Velero util to perform Velero commands;
6. Add snapshot test case for NamespaceMapping E2E test;
7. Collect debug bundle after catching error of Velero backup or restore command;
Signed-off-by: danfengl <danfengl@vmware.com>
2022-10-16 13:18:39 +00:00
danfengliu
5631c7c9df Merge pull request #5331 from danfengliu/add-opt-in-opt-out-pv-to-e2e-test
Add backup opt-in/opt-out E2E test
2022-09-26 21:43:59 +08:00
danfengl
09240a269b Add Kopia support for Velero installation in E2E test
Signed-off-by: danfengl <danfengl@vmware.com>
2022-09-24 04:27:24 +00:00
danfengl
081b70d0eb Add backup opt-in/opt-out E2E test
Signed-off-by: danfengl <danfengl@vmware.com>
2022-09-23 09:29:32 +00:00
danfengl
80430542df Add schedule backup timing E2E test
Signed-off-by: danfengl <danfengl@vmware.com>
2022-09-21 02:32:26 +00:00
danfengl
6e8061266c Clean backups after each test and fix exlude label test issue
1. Clean backups after each test to avoid exceeding limitation of storage capability during E2E test;
2. Fix exlude label test issue that namespace should not be included and excluded at the same time no matter by which way to config.

Signed-off-by: danfengl <danfengl@vmware.com>
2022-08-19 01:24:06 +00:00
danfengl
c8544ea212 Add namespace- mappping E2E test
Signed-off-by: danfengl <danfengl@vmware.com>
2022-08-11 13:24:51 +00:00
danfengl
fb445b3c0d Add migration E2E test
Signed-off-by: danfengl <danfengl@vmware.com>
2022-08-09 08:31:39 +00:00
danfengl
463202951d Enhance checkpoint of bsl deletion
Signed-off-by: danfengl <danfengl@vmware.com>
2022-06-13 08:32:31 +00:00
Ming
56f72797db Fix nightly running failure
Signed-off-by: Ming <mqiu@vmware.com>
2022-06-06 06:47:01 +00:00
danfengl
18621a0e13 Add setting TTL in backup E2E test
Signed-off-by: danfengl <danfengl@vmware.com>
2022-05-13 06:52:47 +00:00
danfengl
33bf59a910 Add snapshot checkpoint for CSI E2E test and fix CSI snapshot name issue
1. Add checkpoint in snapshot E2E test to verify snapshot CR should be created and snapshot should be created in cloud side after backup completion;
2. Fix snapshot name issue that CSI snapshot name in cloud side is not the same with other non-CSI cloud snapshots;

Signed-off-by: danfengl <danfengl@vmware.com>
2022-05-07 08:49:34 +00:00
danfengl
7b35f94f84 Add debug switcher for E2E test
Signed-off-by: danfengl <danfengl@vmware.com>
2022-04-06 06:50:47 +00:00
danfengl
658699636d Support CSI plugin in E2E test
Signed-off-by: danfengl <danfengl@vmware.com>
2022-04-01 11:26:26 +00:00
danfengl
2c5c9246e6 Add CSI plugin in E2E test
Signed-off-by: danfengl <danfengl@vmware.com>
2022-03-30 13:09:26 +00:00