* Use appropriate CRD API during readiness check (#4015)
* Use appropriate CRD API during readiness check
The readiness check for the Velero CRDs was still using the v1beta1 API.
This would cause the readiness check to fail on 1.22 clusters as the
v1beta1 API is no longer available. Previously, this error would be
ignored and the installation would proceed, however with #4002, we are
no longer ignoring errors from this check.
This change modifies the CRD readiness check to check the CRDs using the
same API version that was used when submitting the CRDs to the cluster.
It also introduces a new CRD builder using the V1 API for testing.
This change also fixes a bug that was identified in the polling code
where if the CRDs were not ready on the first polling iteration, they
would be added again to the list of CRDs to check resulting in
duplicates. This would cause the length check to fail on all subsequent
polls and the timeout would always be reached.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Remove duplicate V1 CRD builder and update comment
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Merge pull request #4012 from jenting/add-k8s-1.22-ci-test
Add Kubernetes v1.22 CI test
* Update changelog for v1.6.3
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
Co-authored-by: Scott Seago <sseago@redhat.com>
* Merge pull request #3941 from sseago/e2e-crdversion
enable e2e tests to choose crd apiVersion
* Updated uninstall to remove both v1beta1 and v1 CRDs if present (#3997)
* Add changelog for v1.6.3
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
Co-authored-by: Wenkai Yin(尹文开) <yinw@vmware.com>
Co-authored-by: David L. Smith-Uchida <dsmithuchida@vmware.com>
In #3863, it was discovered that volumes from projected sources were
being backed up by restic when they should have been skipped. Restoring
these volumes triggers a known bug in restic.
In #3866, we started skipping volumes from a projected source, however
there will exist backups that were taken before this fix was introduced.
This change modifies the restore logic to skip the restore of any volume
that came from a projected source, allowing backups taken before #3866
to be restored successfully.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Use Credential from BSL for restic commands
This change introduces support for restic to make use of per-BSL
credentials. It makes use of the `credentials.FileStore` introduced in
PR #3442 to write the BSL credentials to disk. To support per-BSL
credentials for restic, the environment for the restic commands needs to
be modified for each provider to ensure that the credentials are
provided via the correct provider specific environment variables.
This change introduces a new function `restic.CmdEnv` to check the BSL
provider and create the correct mapping of environment variables for
each provider.
Previously, AWS and GCP could rely on the environment variables in the
Velero deployments to obtain the credentials file, but now these
environment variables need to be set with the path to the serialized
credentials file if a credential is set on the BSL.
For Azure, the credentials file in the environment was loaded and parsed
to set the environment variables for restic. Now, we check if the BSL
has a credential, and if it does, load and parse that file instead.
This change also introduces a few other small improvements. Now that we
are fetching the BSL to check for the `Credential` field, we can use the
BSL directly to get the `CACert` which means that we can remove the
`GetCACert` function. Also, now that we have a way to serialize secrets
to disk, we can use the `credentials.FileStore` to get a temp file for
the restic repo password and remove the `restic.TempCredentialsFile`
function.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Add documentation for per-BSL credentials
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Address review feedback
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Address review comments
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
We are no longer adding the Credentials field to the VSL so this reverts
part the change that added it (#3409).
The original PR also added the `snapshot-location set` command. This
command only included options for setting the credential but is part of
the work for #2426. Due to this, the command has been left in place
(with the credentials option removed) but has been hidden.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Load credentials and pass to ObjectStorage plugins
Update NewObjectBackupStore to take a CredentialsGetter which can be
used to get the credentials for a BackupStorageLocation if it has been
configured with a Credential. If the BSL has a credential, use that
SecretKeySelector to fetch the secret, write the contents to a temp file
and then pass that file through to the plugin via the config map using
the key `credentialsFile`. This relies on the plugin being able to use
this new config field.
This does not yet handle VolumeSnapshotLocations or ResticRepositories.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Address code reviews
Add godocs and comments.
Improve formatting and test names.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Address code reviews
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Add uninstall cmd
- init fn to uninstall velero
- abstract dynamic client creation to a separate fn
- creates a separate client per unstructured resource
- add delete client for CRDs
- export appendUnstructured
- add uninstall command to main cmd
- export `podTemplateOption`
- uninstall resources in the reverse order of installation
- fallback to `velero` if no ns is provided during uninstall
- skip deletion if the resource doesn't exist
- handle resource not found error
- match log formatting with cli install logs
- add Delete fn to fake client
- fix import order
- add changelog
- add comment doc for CreateClient fn
Signed-off-by: Suraj Banakar <suraj@infracloud.io>
* Re-use uninstall code from test suite
- move helper functions out of test suite
- this is to prevent cyclic imports
- move uninstall helpers to uninstall cmd
- call them from test suite
- revert export of variables/fns from install code
- because not required anymore
Signed-off-by: Suraj Banakar <suraj@infracloud.io>
* Revert `PodTemplateOption` -> `podTemplateOption`
Signed-off-by: Suraj Banakar <suraj@infracloud.io>
* Use uninstall helper under VeleroUninstall
- as a wrapper
- fix import related errors in test suite
Signed-off-by: Suraj Banakar <suraj@infracloud.io>
* Use pod namespace from backup when matching PVBs
In #3051, we introduced an additional check to ensure that a PVB matched
a particular pod by checking both the name and the namespace of the pod.
This caused an issue when using a namespace mapping on restore. In the
case where a namespace mapping is being used, the check for whether a
PVB matches a particular pod will fail as the PVB was created for the
original pod namespace and is not aware of the new namespace mapping
being used. This resulted in PVRs not being created for pods that were
being restored into new namespaces. The restic init containers were
being created to wait on the volume restore, however this would cause
the restored pods to block indefinitely as they would be waiting for a
volume restore that was not scheduled.
To fix this, we use the original namespace of the pod from the backup to
match the PVB to the pod being restored, not the new namespace where
the pod is being restored into.
Fixes#3467.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Explain why the namespace mapping can't be used
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Restore API group version by priority
Signed-off-by: F. Gold <fgold@vmware.com>
* Add changelog
Signed-off-by: F. Gold <fgold@vmware.com>
* Correct spelling
Signed-off-by: F. Gold <fgold@vmware.com>
* Refactor userResourceGroupVersionPriorities(...) to accept config map, adjust unit test
Signed-off-by: F. Gold <fgold@vmware.com>
* Move some unit tests into e2e
Signed-off-by: F. Gold <fgold@vmware.com>
* Add three e2e tests using Testify Suites
Summary of changes
Makefile - add testify e2e test target
go.sum - changed with go mod tidy
pkg/install/install.go - increased polling timeout
test/e2e/restore_priority_group_test.go - deleted
test/e2e/restore_test.go - deleted
test/e2e/velero_utils.go - made restic optional in velero install
test/e2e_testify/Makefile - makefile for testify e2e tests
test/e2e_testify/README.md - example command for running tests
test/e2e_testify/common_test.go - helper functions
test/e2e_testify/e2e_suite_test.go - prepare for tests and run
test/e2e_testify/restore_priority_apigv_test.go - test cases
Signed-off-by: F. Gold <fgold@vmware.com>
* Make changes per @nrb code review
Signed-off-by: F. Gold <fgold@vmware.com>
* Wait for pods in e2e tests
Signed-off-by: F. Gold <fgold@vmware.com>
* Remove testify suites e2e scaffolding moved to PR #3354
Signed-off-by: F. Gold <fgold@vmware.com>
* Make changes per @brito-rafa and Velero maintainers code reviews
- Made changes suggested by @brito-rafa in GitHub.
- We had a code review meeting with @carlisia, @dsu-igeek, @zubron, and @nrb
- and changes were made based on their suggetions:
- pull in logic from 'meetsAPIGVResotreReqs()' to restore.go.
- add TODO to remove APIGroupVersionFeatureFlag check
- have feature flag and backup version format checks in separate `if` statements.
- rename variables to be sourceGVs, targetGVs, and userGVs.
Signed-off-by: F. Gold <fgold@vmware.com>
* Convert Testify Suites e2e tests to existing Ginkgo framework
Signed-off-by: F. Gold <fgold@vmware.com>
* Made changes per @zubron PR review
Signed-off-by: F. Gold <fgold@vmware.com>
* Run go mod tidy after resolving go.sum merge conflict
Signed-off-by: F. Gold <fgold@vmware.com>
* Add feature documentation to velero.io site
Signed-off-by: F. Gold <fgold@vmware.com>
* Add config map e2e test; rename e2e test file and name
Signed-off-by: F. Gold <fgold@vmware.com>
* Update go.{mod,sum} files
Signed-off-by: F. Gold <fgold@vmware.com>
* Move CRDs and CRs to testdata folder
Signed-off-by: F. Gold <fgold@vmware.com>
* Fix typos in cert-manager to pass codespell CICD check
Signed-off-by: F. Gold <fgold@vmware.com>
* Make changes per @nrb code review round 2
- make checkAndReadDir function private
- add info level messages when priorties 1-3 API group versions can not be used
Signed-off-by: F. Gold <fgold@vmware.com>
* Make user config map rules less strict
Signed-off-by: F. Gold <fgold@vmware.com>
* Update e2e test image version in example
Signed-off-by: F. Gold <fgold@vmware.com>
* Update case A music-system controller code
Signed-off-by: F. Gold <fgold@vmware.com>
* Documentation updates
Signed-off-by: F. Gold <fgold@vmware.com>
* Update migration case documentation
Signed-off-by: F. Gold <fgold@vmware.com>
* Use label to select Velero deployment in plugin cmd
Signed-off-by: F. Gold <fgold@vmware.com>
* Move veleroLabel constant closer to usage
Signed-off-by: F. Gold <fgold@vmware.com>
* Add changelog
Signed-off-by: F. Gold <fgold@vmware.com>
* Remove year from copyright in new file
Signed-off-by: F. Gold <fgold@vmware.com>
* Export and use install.Labels() function
Signed-off-by: F. Gold <fgold@vmware.com>
* Add colors to describe command
* Add colors to describe backups/restore/schedules commands
* Make name in the output bold
* Disable colors via `--colorized` flag or if velero isn't in TTY
Co-authored-by: Clay Kauzlaric <ckauzlaric@vmware.com>
Signed-off-by: Clay Kauzlaric <ckauzlaric@vmware.com>
Signed-off-by: Mikael Manukyan <mmanukyan@vmware.com>
* Add changelog
* and run make update
Co-authored-by: Mikael Manukyan <mmanukyan@vmware.com>
Signed-off-by: Mikael Manukyan <mmanukyan@vmware.com>
Signed-off-by: Clay Kauzlaric <ckauzlaric@vmware.com>
* Add colorized to the client config file
Co-authored-by: Mikael Manukyan <mmanukyan@vmware.com>
Signed-off-by: Clay Kauzlaric <ckauzlaric@vmware.com>
Co-authored-by: Mikael Manukyan <mmanukyan@vmware.com>
* allow client config to use string values
* the command `velero client config set colorized=false` writes a string
value of "false" into the config. This change allows that string to be
accepted and converted into a boolean when used in program.
Signed-off-by: Clay Kauzlaric <ckauzlaric@vmware.com>
* Add docs about colored CLI output
Co-authored-by: Mikael Manukyan <mmanukyan@vmware.com>
Signed-off-by: Clay Kauzlaric <ckauzlaric@vmware.com>
* Update site/content/docs/main/customize-installation.md
Co-authored-by: JenTing Hsiao <jenting.hsiao@suse.com>
Signed-off-by: Clay Kauzlaric <ckauzlaric@vmware.com>
* docs: remove comma
* as per @carlisia 's suggestion
Signed-off-by: Clay Kauzlaric <ckauzlaric@vmware.com>
Co-authored-by: Clay Kauzlaric <ckauzlaric@vmware.com>
Co-authored-by: Clay Kauzlaric <clay.kauzlaric@gmail.com>
Co-authored-by: JenTing Hsiao <jenting.hsiao@suse.com>
* added useOwnerReferencesInBackup to crd velerio.io_schedules
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* added UseOwnerReferencesInBackup property to schedule.go
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* deepcopy schedule configured for reference the property UseOwnerReferencesInBackup
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* added UseOwnerReferencesInBackup property verification to modify OwnerReferences from backup
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* created changelog
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* removed deepcopy schedule configured for reference the property UseOwnerReferencesInBackup
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* running make update
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* running make update
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* updated the year at the top of the schedule.go file for 2020
Signed-off-by: matheusjuvelino <matheus.juvelino@outlook.com>
* -> Preserve nodePort support when restoring via "--preserve-nodeports" flag
Signed-off-by: Yusuf Güngör <yusuf.gungor@hepsiburada.com>
* -> Added changelog.
Signed-off-by: Yusuf Güngör <yusuf.gungor@hepsiburada.com>
* -> Unit test added.
-> Using boolptr.IsSetToTrue for bool ptr check.
Signed-off-by: Yusuf Güngör <yusuf.gungor@hepsiburada.com>
* -> Unit test added.
-> Using boolptr.IsSetToTrue for bool ptr check.
Signed-off-by: Yusuf Güngör <yusuf.gungor@hepsiburada.com>
* -> Other restore errors log level changed from info to error.
-> Documentation updated about Velero nodePort restore logic and preservation of them.
Signed-off-by: Yusuf Güngör <yusuf.gungor@hepsiburada.com>
Co-authored-by: Yusuf Güngör <yusuf.gungor@hepsiburada.com>
* Add default field to BSL CRD
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Add a new flag `--default` under `velero backup-location create`
add a new flag `--default` under `velero backup-location create`
to specify this new location to be the new default BSL.
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Add a new default field under `velero backup-location get`
add a new default field under `velero backup-location get` to indicate
which BSL is the default one.
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Add a new sub-command and flag under `velero backup-location`
Add a new sub-command called `velero backup-location set` sub-command
and a new flag `velero backup-cation set --default` to configure which
BSL is the default one.
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Add new flag to get the default backup-location
Add a new flag `--default` under `velero backup-location get`
to displays the current default BSL.
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Configures default BSL in BSL controller
When upgrade the BSL CRDs, none of the BSL has been labeled as default.
Sets the BSL default field to true if the BSL name matches to the default BSL setting.
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Configures the default BSL in BSL controller for velero upgrade
When upgrade the BSL CRDs, none of the BSL be marked as the default.
Sets the BSL `.spec.default: true` if the BSL name matches against the
`velero server --default-backup-storage-location`.
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Add unit test to test default BSL behavior
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Update check which one is the default BSL in backup/backup_sync/restore controller
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Add changelog
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Update docs locations.md and upgrade-to-1.6.md
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* 🐛 BSLs with validation disabled should be validated at least once
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
* review comments
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
* feat: add delete sub-command for backup-location
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Change to use kubebuilder/runtimecontroller API
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* fix get BSL by label doesn't work
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Update changelog
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Ordering by alphabet
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Better example format for help message
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Capital the comments
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
* Don't fail backup if downloading tarball fails
Previously, we would always attempt to download the tarball for a backup
for processing DeleteItemAction plugins, even if there weren't any.
This caused an issue for some users in the case where the backup tarball
had been deleted from object storage as the backup deletion would fail.
Now, we only attempt to download the tarball in the case where there are
DeleteItemAction plugins. If downloading that tarball fails, we log
the error, skip the processing of the DeleteItemAction plugins and
proceed with the rest of the deletion.
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Skip file removal in closeAndRemoveFile if nil
Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
* Basic end-to-end tests, generate data/backup/remove/restore/verify
Uses distributed data generator
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Moved backup/restore into velero_utils, started using a name for the restore
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* remove checked in binary and update test/e2e Makefile
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
* Ran make update
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Save
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
* Ran make update
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Basic end-to-end test, generate data/backup/remove/restore/verify
Uses distributed data generator
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Changed tests/e2e Makefile to just use go get to install ginkgo in the GOPATH/bin
Updated to ginkgo 1.14.2
Put cobra back to v0.0.7
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* Added CLOUD_PLATFORM env variable to Makefile, updated README, removed ginkgo from .gitignore
Signed-off-by: Dave Smith-Uchida <dsmithuchida@vmware.com>
* choose velero CLI binary based on local env
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
Co-authored-by: Ashish Amarnath <ashisham@vmware.com>
