Commit Graph

33 Commits

Author SHA1 Message Date
Xun Jiang/Bruce Jiang
aa3bd251dd Merge branch 'main' into 9097 2025-12-18 14:18:04 +08:00
Tiger Kaovilai
61bf2ef777 feat: Enhance BackupStorageLocation with Secret-based CA certificate support
- Introduced `CACertRef` field in `ObjectStorageLocation` to reference a Secret containing the CA certificate, replacing the deprecated `CACert` field.
- Implemented validation logic to ensure mutual exclusivity between `CACert` and `CACertRef`.
- Updated BSL controller and repository provider to handle the new certificate resolution logic.
- Enhanced CLI to support automatic certificate discovery from BSL configurations.
- Added unit and integration tests to validate new functionality and ensure backward compatibility.
- Documented migration strategy for users transitioning from inline certificates to Secret-based management.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2025-12-12 21:07:37 +07:00
Shubham Pampattiwar
20af2c20c5 Address PR review comments: sanitize errors and add SAS token scrubbing
This commit addresses three review comments on PR #9321:

1. Keep sanitization in controller (response to @ywk253100)
   - Maintaining centralized error handling for easier extension
   - Azure-specific patterns detected and others passed through unchanged

2. Sanitize unavailableErrors array (@priyansh17)
   - Now using sanitizeStorageError() for both unavailableErrors array
     and location.Status.Message for consistency

3. Add SAS token scrubbing (@anshulahuja98)
   - Scrubs Azure SAS token parameters to prevent credential leakage
   - Redacts: sig, se, st, sp, spr, sv, sr, sip, srt, ss
   - Example: ?sig=secret becomes ?sig=***REDACTED***

Added comprehensive test coverage for SAS token scrubbing with 4 new
test cases covering various scenarios.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2025-12-02 11:37:50 -08:00
Shubham Pampattiwar
a5d32f29da Sanitize Azure HTTP responses in BSL status messages
Azure storage errors include verbose HTTP response details and XML
in error messages, making the BSL status.message field cluttered
and hard to read. This change adds sanitization to extract only
the error code and meaningful message.

Before:
  BackupStorageLocation "test" is unavailable: rpc error: code = Unknown
  desc = GET https://...
  RESPONSE 404: 404 The specified container does not exist.
  ERROR CODE: ContainerNotFound
  <?xml version="1.0"...>

After:
  BackupStorageLocation "test" is unavailable: rpc error: code = Unknown
  desc = ContainerNotFound: The specified container does not exist.

AWS and GCP error messages are preserved as-is since they don't
contain verbose HTTP responses.

Fixes #8368

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2025-12-02 11:37:50 -08:00
Daniel Jiang
a550910f36 Add Gauge metric for BSL availability
The label of the gauge is the name of BSL

Signed-off-by: Daniel Jiang <daniel.jiang@broadcom.com>
2025-07-03 17:36:19 +08:00
Tiger Kaovilai
a3cee616dc Upgrade go.mod k8s.io/ go.mod to v0.31.3 and set klog.SetLogger() for client-go (#8450)
Some checks failed
Run the E2E test on kind / build (push) Failing after 5m44s
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
build-image / Build (push) Failing after 10s
Main CI / Build (push) Failing after 31s
Close stale issues and PRs / stale (push) Successful in 7s
Trivy Nightly Scan / Trivy nightly scan (velero, main) (push) Failing after 59s
Trivy Nightly Scan / Trivy nightly scan (velero-restore-helper, main) (push) Failing after 45s
Also bumped to support upgraded k8s.io/ deps.
- controller-gen to v0.16.5
- sigs.k8s.io/controller-runtime v0.19.2

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2025-02-17 15:05:10 -05:00
Wenkai Yin(尹文开)
0a4e417aab Bump up version of client-go and controller-runtime
Bump up version of client-go to v0.30.5
Bump up version of controller-runtime to v0.18.5

Fixes #8274

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2024-10-08 18:53:12 +08:00
Tiger Kaovilai
c8aa37d852 Remove additional param, use pkg/constant
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2024-09-11 17:13:37 -04:00
Tiger Kaovilai
5c4c66bee9 Add controller name to periodical_enqueue_source
The code changes are related to the `NewPeriodicalEnqueueSource` function in the `kube/periodical_enqueue_source.go` file. This function is used to create a new instance of the `PeriodicalEnqueueSource` struct, which is responsible for periodically enqueueing objects into a work queue.

The changes involve adding two new parameters to this function: `controllerName string` and modifying the existing `logger` parameter to include additional fields.

Here's what changed:

1. A new `controllerName` parameter was added to the `NewPeriodicalEnqueueSource` function.

These changes are to adding more context or metadata to the logging output, possibly for debugging or monitoring purposes.

The other files (`restore_operations_controller.go`, `schedule_controller.go`, and their respective test files) were modified to use this updated `NewPeriodicalEnqueueSource` function with the new `controllerName` parameter.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
2024-09-09 12:07:07 -04:00
Wenkai Yin(尹文开)
dc6eeafe98 Pass Velero server command args to the plugins
Pass Velero server command args to the plugins

Fixes #7806

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2024-09-04 13:43:27 +08:00
Wenkai Yin(尹文开)
8752c3a820 Bump up the versions of severel Kubernetes-related libs
Bump up the versions of severel Kubernetes-related libs

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2024-03-05 13:09:38 +08:00
Ming Qiu
7d2be128ae Move velero server side default backup location setting logic to server startup
Signed-off-by: Ming Qiu <mqiu@vmware.com>
2023-12-19 05:43:29 +00:00
qiuming
ea04a86eb2 Merge pull request #6771 from qiuming-best/bsl-fix
Fix default BSL setting not work
2023-12-05 19:09:50 +08:00
Ming Qiu
c6cba300fb Fix default BSL setting not work
Signed-off-by: Ming Qiu <mqiu@vmware.com>
2023-12-01 02:06:35 +00:00
Yang Gang (成都)
25898305ef delete unused shcema package and parms. (#6716)
Signed-off-by: yanggang <gang.yang@daocloud.io>
2023-09-04 14:50:10 +08:00
Xun Jiang/Bruce Jiang
7d7e3fff0d Refoctor backup controller with controller-runtime. (#5969)
Signed-off-by: Ming <mqiu@vmware.com>
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
Co-authored-by: Ming <mqiu@vmware.com>
2023-03-10 08:59:40 +08:00
Wenkai Yin(尹文开)
4b9dbfa416 Support pause/unpause schedules
Support pause/unpause schedule

Fixes #2363

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-09-15 10:42:48 +08:00
Xun Jiang/Bruce Jiang
6fea973c57 Migrate backup sync controller from code-generator to kubebuilder (#5218)
* Migrate backup sync controller from code-generator to kubebuilder

1. use kubebuilder's reconcile logic to replace controller's old logic.
2. use ginkgo and gomega to replace testing.
3. modify BSL reconciler registration method.

Signed-off-by: Xun Jiang <blackpiglet@gmail.com>

* Add Option for PeriodicalEnqueueSource.

Signed-off-by: Xun Jiang <blackpiglet@gmail.com>

Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
Co-authored-by: Xun Jiang <blackpiglet@gmail.com>
2022-08-30 16:55:39 +08:00
Wenkai Yin(尹文开)
f550f8e3cd Fix bsl validation bug
Fix bsl validation bug: the BSL is validated continually and doesn't respect the validation period configured

Fixes #5056

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-07-06 14:20:03 +08:00
Wenkai Yin(尹文开)
34087fe5f4 Disable status as sub resource in CRDs
When enabling the status as sub resource in CRD, the status will be ignored when creating the CR with status, this will cause issues when syncing backups/pvbs

Fixes #4950

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-06-09 07:52:08 +08:00
Xun Jiang
989a1e3ebb Refactor BSL controller with periodical enqueue source.
Add filter functions for PeriodicalEnqueueSource.
Move BSL's valication frequency check test case to PeriodicalEnqueueSource's test.

Signed-off-by: Xun Jiang <jxun@vmware.com>
2022-05-11 15:31:47 +08:00
Wenkai Yin(尹文开)
bcc9aa8d69 Update BSL CR as long as there is any error
1. Mark the BSL as "Unavailable" when gets any error
2. Add a new field "Message" to the BSL status to record the error message

Fixes #4485
Fixes #4405

Signed-off-by: Wenkai Yin(尹文开) <yinw@vmware.com>
2022-03-07 10:25:18 +08:00
Carlisia Thompson
11bfe82342 Convert DownloadRequest resource/controller to kubebuilder (#3004)
* Migrate DownloadRequest types to kubebuilder

Signed-off-by: Carlisia <carlisia@vmware.com>

* Migrate controller to kubebuilder

Signed-off-by: Carlisia <carlisia@vmware.com>

* Migrate download request cli to kubebuilder

Signed-off-by: Carlisia <carlisia@vmware.com>

* Format w make update

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove download file

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove kubebuilder from backup/restore apis

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix test description

Signed-off-by: Carlisia <carlisia@vmware.com>

* Import cleanups

Signed-off-by: Carlisia <carlisia@vmware.com>

* Refactor for controller runtime version update

Signed-off-by: Carlisia <carlisia@vmware.com>

* Remove year from the copyright

Signed-off-by: Carlisia <carlisia@vmware.com>

* Check for expiration regardless of phase

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix typos and godoc

Signed-off-by: Carlisia <carlisia@vmware.com>

* Fix test setup and fix a test case

Signed-off-by: Carlisia <carlisia@vmware.com>
2021-03-01 13:28:46 -05:00
Bridget McErlean
38c08e087b Replace NewObjectBackupStore with interface (#3329)
In preparation for modifying the instantiation of `BackupStores` to be
able to load credentials, change the function `NewObjectBackupStore` to
be an interface that is passed in to all controllers.

Previously, the function to get a new backup store was configurable but
for many controllers was fixed to use `NewObjectBackupStore`. This
change introduces an interface for getting the backup store and wraps
the functionality from `NewObjectBackupStore` in a type which implements
this interface. This will allow more flexibility when introducing
credentials for a specific backup store as it will allow us to create a
new `ObjectBackupStoreGetter` type which can be configured to add
credentials config when creating the ObjectBackupStore without needing
to change the API used by the controllers.

Signed-off-by: Bridget McErlean <bmcerlean@vmware.com>
2021-02-08 13:04:08 -05:00
Ron Green
8bb3615339 feat(gomod): bump versions
now versions are working and there are code changes that need to happen

- release candidate versions are aligned and working
- replaces fields are removed and not required anymore

controller runtime has been changed during the 'make' command

Signed-off-by: Ron Green <11993626+georgettica@users.noreply.github.com>
2021-01-26 13:06:27 +02:00
Carlisia
bd10b7660c Improvements to BSL logic
Signed-off-by: Carlisia <carlisia@vmware.com>
2020-12-09 13:25:01 -08:00
JenTing Hsiao
9dd158d13d feat: support configure BSL CR to indicate which one is the default (#3092)
* Add default field to BSL CRD

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Add a new flag `--default` under `velero backup-location create`

add a new flag `--default` under `velero backup-location create`
to specify this new location to be the new default BSL.

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Add a new default field under `velero backup-location get`

add a new default field under `velero backup-location get` to indicate
which BSL is the default one.

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Add a new sub-command and flag under `velero backup-location`

Add a new sub-command called `velero backup-location set` sub-command
and a new flag `velero backup-cation set --default` to configure which
BSL is the default one.

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Add new flag to get the default backup-location

Add a new flag `--default` under `velero backup-location get`
to displays the current default BSL.

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Configures default BSL in BSL controller

When upgrade the BSL CRDs, none of the BSL has been labeled as default.
Sets the BSL default field to true if the BSL name matches to the default BSL setting.

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Configures the default BSL in BSL controller for velero upgrade

When upgrade the BSL CRDs, none of the BSL be marked as the default.
Sets the BSL `.spec.default: true` if the BSL name matches against the
`velero server --default-backup-storage-location`.

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Add unit test to test default BSL behavior

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Update check which one is the default BSL in backup/backup_sync/restore controller

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Add changelog

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>

* Update docs locations.md and upgrade-to-1.6.md

Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
2020-12-08 16:38:29 -05:00
Ashish Amarnath
b321838c72 🏃‍♂️ reducing verbosity of another log message (#3109)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-11-24 08:12:51 -08:00
Ashish Amarnath
9b20e8d2e6 🏃‍♂️ Turn down logging verbosity (#3091)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-11-19 14:03:29 -05:00
Mateusz Gozdek
dbc83af77b Fix various typos found by codespell (#3057)
By running the following command:

codespell -S .git,*.png,*.jpg,*.woff,*.ttf,*.gif,*.ico -L \
iam,aks,ist,bridget,ue

Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
2020-11-10 11:48:35 -05:00
Ashish Amarnath
1be97a2b04 🏃‍♂ Improve log message clarity (#3047)
Signed-off-by: Ashish Amarnath <ashisham@vmware.com>
2020-11-02 13:39:32 -08:00
Antony S Bett
35d25c81ec Fix BSL controller to avoid invoking init() on all BSLs regardless of ValidationFrequency (#2992)
Signed-off-by: Bett, Antony <antony.bett@dell.com>
2020-10-13 12:10:32 -07:00
Carlisia Thompson
e69fac153b Centralize + rename controller names and list (#2936)
* Centralize + rename controller names and list

Signed-off-by: Carlisia <carlisia@vmware.com>

* Rename file

Signed-off-by: Carlisia <carlisia@vmware.com>

* Reset restic-repo name

Signed-off-by: Carlisia <carlisia@vmware.com>

* Reset gc controller name

Signed-off-by: Carlisia <carlisia@vmware.com>
2020-10-06 13:58:56 -04:00