Commit Graph
221 Commits
Author SHA1 Message Date
872f903091 Add configurable tolerations for PodVolumeBackup and data mover pods (#9575)
Run the E2E test on kind / setup-test-matrix (push) Failing after 3s
Scorecard supply-chain security / Scorecard analysis (push) Skipped
e2e-test-kind.yaml / extract (push) Failing after 6s
Run the E2E test on kind / get-go-version (push) Failing after 7s
Run the E2E test on kind / build (push) Skipped
Run the E2E test on kind / run-e2e-test (push) Skipped
push.yml / extract (push) Failing after 6s
Main CI / get-go-version (push) Failing after 7s
Main CI / Build (push) Skipped
* Remove toleration whitelist for PodVolumeBackup and data mover pods

Instead of filtering tolerations through a hardcoded allowlist
(ThirdPartyTolerations), inherit all tolerations from the node-agent
daemonset for PodVolumeBackup/Restore and DataUpload/Download pods,
and from the Velero deployment for maintenance jobs.

This enables backups and restores on nodes with custom NoExecute taints,
which was previously impossible since only two specific toleration keys
were whitelisted.

Fixes #9476

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>

* Fix codespell: replace 'whitelist' with 'allowlist' in changelog

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>

* Implement deduplication of tolerations and add unit tests for the new function

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

* Merge node-agent-configmap tolerations with third-party allowlist

Add a `tolerations` field to the node-agent-configmap so operators can
declare hosting-pod tolerations explicitly, per blackpiglet's review
feedback that tolerations shouldn't be read from the DaemonSet alone.
These are merged with (and deduplicated against) DaemonSet tolerations
matching the existing third-party allowlist
(kubernetes.azure.com/scalesetpriority, CriticalAddonsOnly), restoring
that allowlist per the follow-up suggestion to keep inheriting it
alongside the new config option.

The toleration dedup helper is moved from pkg/exposer to
pkg/util/kube (exported as DeduplicateTolerations) so it can be
shared with pkg/nodeagent without an import cycle.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

* Fix testifylint finding in TestGetTolerations

golangci-lint v2.12.0 (pinned in pr-linter-check.yml) flagged the
shared assert.Equal after the if/else as require-error: use require
for the error assertion so each branch is self-contained, matching
the pattern used elsewhere in this file.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

* Document toleration merge priority in GetTolerations

Per blackpiglet's review feedback: clarify that configured tolerations
take priority over allowlisted daemonset tolerations because they're
appended first and DeduplicateTolerations keeps only the first
occurrence of each exact (Key, Operator, Value, Effect) combination.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

---------

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Happy <yesreply@happy.engineering>
2026-09-14 18:05:01 -04:00
Yonghui Li 8e604b17b2 add ID to repo snapshot
Signed-off-by: Yonghui Li <lyonghui@vmware.com>
2026-09-08 18:15:34 +08:00
lyndon-liandGitHub ef9f3ed883 fix repo connection contest of the two repositories with the same storage type (#10344)
- fix repo connection contest between two BSL
- add UT for repo connection contest

Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-24 09:21:06 +00:00
lyndon-liandGitHub 61c9b5b84f credentialFile in Config of BSL should be used internally (#10254)
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-17 14:59:27 +08:00
Lyndon-Li c8127e243b object reader throughput improvement
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-11 14:07:59 +08:00
Lyndon-Li 92bcf5a3b3 add UT for prefetch
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-11 13:36:26 +08:00
Lyndon-Li c27343fc4e fix UT errors
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-11 13:07:27 +08:00
Lyndon-Li b74824f9c0 extend object reader for prefetch
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-11 13:00:44 +08:00
Lyndon-Li a41cb11902 add prefetch options for repo interface
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-08-11 12:56:23 +08:00
Xun Jiang/Bruce JiangandGitHub e354e7aec4 Merge pull request #10138 from Jay2006sawant/fix/block-uploader-and-batchforget-errors
fix: return errors correctly in block restore validation and BatchForget
2026-08-11 11:25:06 +08:00
513e93ff4b fix: correct typos in log messages and status strings (#10192)
- Fix 'dataudownload' typo in DataDownload warning log message
  (data_download_controller.go:696)
- Fix 'datadownlad' misspelled structured log field key to 'datadownload'
  (data_download_controller.go:700) - this caused the log field to be
  unqueryable by the correct key name
- Fix 'retrieveable' -> 'retrievable' in BackupRepository maintenance
  status messages (maintenance.go:354, 417)
- Update corresponding test assertion to match corrected string
  (maintenance_test.go:792)

Signed-off-by: shellyco-code <shellyco-code@users.noreply.github.com>
Co-authored-by: shellyco-code <shellyco-code@users.noreply.github.com>
2026-08-10 19:40:02 +00:00
Jay2006sawant 46f5adb7a3 fix(provider): return immediately when BatchForget flush fails
Signed-off-by: Jay2006sawant <jay242902@gmail.com>
2026-08-03 11:26:00 +05:30
Jay2006sawant 8ec4b22496 fix: return errors correctly in block restore validation and BatchForget
Signed-off-by: Jay2006sawant <jay242902@gmail.com>
2026-08-03 09:44:00 +05:30
Lyndon-Li 525fb9eaa2 Merge branch 'main' into optimize-sub-object-description 2026-07-28 15:14:15 +08:00
e5654fa7ed Fix flaky TestKopiaObjectWriterEx_ConcurrentAsyncErrors (#10030)
The test assumed all ten Write calls succeed before any async goroutine
stores its error, but with a mock that fails instantly a goroutine can
poison the writer mid-loop, making a later Write correctly fail fast —
a timing-dependent test failure.

Rewrite the test to assert the real-world contract instead of one
schedule: a failed async block write either fails a subsequent Write
fast or surfaces at Result, and is never lost. Add a separate
deterministic case pinning the late-error schedule, holding async
writes until all writes are queued so Result alone must report the
error.

Verified with -race -count=100.

Fixes #10029

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 15:42:36 -04:00
Xun Jiang/Bruce JiangandGitHub 36647b0b2f Merge pull request #9974 from blackpiglet/jxun/fips-140
Disable fips140 enforcement because Kopia doesn't support it.
2026-07-22 17:28:58 +08:00
Lyndon-Li 2b2aa061a8 optimize subobject description
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-07-20 14:17:34 +08:00
Shubham PampattiwarandGitHub ae06d40c69 Validate user-provided labels and annotations in maintenance job (#9982)
Run the E2E test on kind / get-go-version (push) Failing after 58s
Run the E2E test on kind / build (push) Has been skipped
Run the E2E test on kind / setup-test-matrix (push) Successful in 3s
Run the E2E test on kind / run-e2e-test (push) Has been skipped
Main CI / get-go-version (push) Successful in 13s
Main CI / Build (push) Failing after 26s
* Validate user-provided labels and annotations in maintenance job

User-provided labels and annotations from maintenance JobConfigs
are now validated before being applied to the maintenance Job pod
template. Invalid label keys, label values, and annotation keys
are skipped with a warning log. This prevents the Kubernetes API
from rejecting the entire Job when a user provides labels or
annotations that violate naming rules.

Additionally, user-provided labels can no longer overwrite the
internal RepositoryNameLabel used for job tracking.

Fixes velero-io/velero#9981

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

* Add tests for label and annotation validation in maintenance job

Add test cases to TestBuildJob covering:
- Invalid label key is skipped
- Invalid label value is skipped
- Label value exceeding 63 characters is skipped
- User-provided label cannot overwrite RepositoryNameLabel
- Invalid annotation key is skipped

Also fix a latent test issue where param.BackupRepo was not reset
between test cases, and add the missing assertion for
expectedPodAnnotation which was defined but never checked.

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

* Fix gofmt formatting in maintenance test file

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

* Add changelog for PR #9982

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>

---------

Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
2026-07-10 13:00:43 -04:00
Xun Jiang a9545d785f Disable fips140 enforcement because Kopia doesn't support it.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2026-07-08 17:40:43 +08:00
Lyndon-Li 8d23c7e813 block uploader backup implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-30 17:03:39 +08:00
Lyndon-Li df21463629 block uploader backup implementation
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-30 16:21:59 +08:00
Xun Jiang/Bruce JiangandGitHub 02cb85ce90 Merge pull request #9890 from blackpiglet/jxun/main/bump_deps
[main] Bump some dependancies to fix CVEs and replace some inactive maintained packages.
2026-06-11 10:08:22 +08:00
lyndon-liandGitHub 4c1950980a Merge pull request #9887 from Lyndon-Li/incremental-object-aware-write-at
Incremental aware object writer - writeat
2026-06-10 18:01:42 +08:00
Xun Jiang 49b670a791 Replace github.com/pkg/errors by github.com/cockroachdb/errors
Change errors.Cause to errors.Is, because github.com/cockroachdb/errors
New() function create a error with error stack with depth 1, but
github.com/pkg/errors's New() function create error with no depth.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2026-06-10 15:55:57 +08:00
Lyndon-Li 09f842afaf refactor object writer interface
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-10 10:00:30 +08:00
Lyndon-Li f474e313fa incremental object aware write at
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-05 17:02:41 +08:00
Lyndon-Li 8b5db5954e Merge branch 'main' into incremental-object-aware-write-at 2026-06-05 13:31:56 +08:00
Lyndon-Li 09bfc69d63 add totalSize to repo snapshot
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-03 13:23:08 +08:00
Lyndon-Li d435b0509e add velero-pins to repo snapshot
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-02 17:40:20 +08:00
Lyndon-Li 30960d1edd incremental aware object writer - writeat
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-06-02 13:10:32 +08:00
Lyndon-Li 44eaea8faf incremental aware object writer - write
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-05-28 14:30:11 +08:00
Lyndon-Li 596e774582 add incremental aware object writer
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-05-22 16:36:41 +08:00
Lyndon-Li 219975bee0 Merge branch 'main' into incremental-aware-object-writer 2026-05-22 15:43:45 +08:00
Lyndon-Li d58139536b Merge branch 'main' into kopia-repo-snapshot-operations 2026-05-22 14:57:39 +08:00
Lyndon-Li 205ca71588 kopia repo snapshot operations
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-05-22 06:26:40 +00:00
Lyndon-Li 343ed95a5e metadata operation for Kopia repo
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-05-21 14:22:36 +08:00
Lyndon-Li 3103318c9b Merge branch 'main' into metadata-operator-for-kopia-repo 2026-05-21 13:52:41 +08:00
Lyndon-Li 32969856af Merge branch 'main' into kopia-repo-snapshot-operations 2026-05-21 13:27:40 +08:00
Lyndon-Li 6257282117 add listsnapshot method
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-05-12 09:51:47 +08:00
Xun Jiang 5afe5df122 Bump Velero dependencies to latest version.
* Fix UT failures caused by client-go version bump.
* Some modifications to enhance the UT stability.
* Fix UT errors: non-constant format string in call to ...
* Fix linter issues.

Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2026-05-08 17:38:42 +08:00
Lyndon-Li 6a67f4a8a4 fix UT error
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-05-08 16:41:39 +08:00
Lyndon-Li 6ca73a00b6 fix UT error
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-05-08 16:39:24 +08:00
Lyndon-Li 4f34ae17a3 add incremental aware object writer
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-04-28 18:11:53 +08:00
Lyndon-Li 44ab9a6a1a add metadata operations for kopia repo
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-04-28 15:58:28 +08:00
Lyndon-Li 4befbc0afe add repo snapshot operations
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-04-28 15:47:27 +08:00
Lyndon-Li 455f3ba305 unified repo interface extension for block data mover
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-04-16 18:02:30 +08:00
Lyndon-Li dca3d3001f remove restic for repo
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-04-08 11:11:15 +08:00
Lyndon-Li 235e579581 remove restic for repo
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-04-07 07:35:25 +00:00
Lyndon-Li ef7b468fb9 issue 9626: let go for uninitialized repo under readonly mode
Signed-off-by: Lyndon-Li <lyonghui@vmware.com>
2026-04-01 13:09:29 +08:00
Xun Jiang ffea850522 Add ephemeral storage limit and request support for data mover and maintenance job.
Signed-off-by: Xun Jiang <xun.jiang@broadcom.com>
2026-03-05 14:22:53 +08:00