Compare commits

...
Author SHA1 Message Date
Chris Lu 8e8edd7706 not empty only if there are actual files in the bucket 2026-02-23 00:12:04 -08:00
Chris LuandGitHub 57ab99d13e fix: generate topology uuid uniformly in single-master mode (#8405)
* fix: ensure topology uuid is generated in single master setups

* ensureTopologyId adds a Hashicorp-aware implementation

* simplify
2026-02-22 23:45:48 -08:00
Chris LuandGitHub 998c8d2702 Worker maintenance tasks now use non-default grpcPort if configured (#8407)
Fixes #8401

When creating balance/vacuum tasks, the worker maintenance scheduler was
accidentally discarding the custom grpcPort defined on the DataNodeInfo
by using just its HTTP Address string, which defaults to +10000
during grpc dialing.

By using pb.NewServerAddressFromDataNode, the grpcPort suffix is correctly
encoded in the server address string, preventing connection refused errors
for users running volume servers with custom gRPC ports.
2026-02-22 22:40:14 -08:00
Chris LuandGitHub cd6832249b Fix volume.fsck crashing on EC volumes and add multi-volume vacuum support (#8406)
* helm: refine openshift-values.yaml to remove hardcoded UIDs

Remove hardcoded runAsUser, runAsGroup, and fsGroup from the
openshift-values.yaml example. This allows OpenShift's admission
controller to automatically assign a valid UID from the namespace's
allocated range, avoiding "forbidden" errors when UID 1000 is
outside the permissible range.

Updates #8381, #8390.

* helm: fix volume.logs and add consistent security context comments

* Update README.md

* fix volume.fsck crashing on EC volumes and add multi-volume vacuum support

* address comments
2026-02-22 22:07:15 -08:00
Chris Lu b5f3094619 fix format of internal node URLs in master UI templates 2026-02-22 13:47:29 -08:00
Chris Lu 07f284c391 fix links 2026-02-22 13:40:50 -08:00
Chris Lu 7b08cf74ed consistent template generation 2026-02-22 13:34:06 -08:00
Sheya BernsteinandGitHub d8b8f0dffd fix(helm): add missing app.kubernetes.io/instance label to volume service (#8403) 2026-02-22 07:20:38 -08:00
Chris LuandGitHub 8e25c55bfb S3: Truncate timestamps to milliseconds for CopyObjectResult and CopyPartResult (#8398)
* S3: Truncate timestamps to milliseconds for CopyObjectResult and CopyPartResult

Fixes #8394

* S3: Address nitpick comments in copy handlers

- Synchronize Mtime and LastModified by capturing time once\n- Optimize copyChunksForRange loop\n- Use built-in min/max\n- Remove dead previewLen code
2026-02-20 21:01:31 -08:00
Chris Lu e4b70c2521 go fix 2026-02-20 18:42:00 -08:00
Chris Lu f7c27cc81f go fmt 2026-02-20 18:40:47 -08:00
Chris Lu 66680c58b7 consistent time 2026-02-20 18:40:27 -08:00
Chris LuandGitHub 2a1ae896e4 helm: refine openshift-values.yaml for assigned UID ranges (#8396)
* helm: refine openshift-values.yaml to remove hardcoded UIDs

Remove hardcoded runAsUser, runAsGroup, and fsGroup from the
openshift-values.yaml example. This allows OpenShift's admission
controller to automatically assign a valid UID from the namespace's
allocated range, avoiding "forbidden" errors when UID 1000 is
outside the permissible range.

Updates #8381, #8390.

* helm: fix volume.logs and add consistent security context comments

* Update README.md
2026-02-20 12:05:57 -08:00
Chris LuandGitHub bd0b1fe9d5 S3 IAM: Added ListPolicyVersions and GetPolicyVersion support (#8395)
* test(s3/iam): add managed policy CRUD lifecycle integration coverage

* s3/iam: add ListPolicyVersions and GetPolicyVersion support

* test(s3/iam): cover ListPolicyVersions and GetPolicyVersion
2026-02-20 11:04:18 -08:00
Richard Chen ZhengandGitHub 964a8f5fde Allow user to define access and secret key via values (#8389)
* Allow user to define admin access and secret key via values

* Add comments to values.yaml

* Add support for read for consistency

* Simplify templating

* Add checksum to s3 config

* Update comments

* Revert "Add checksum to s3 config"

This reverts commit d21a7038a8.
2026-02-20 00:37:54 -08:00
Chris LuandGitHub 40cc0e04a6 docker: fix entrypoint chown guard; helm: add openshift-values.yaml (#8390)
* Enforce IAM for s3tables bucket creation

* Prefer IAM path when policies exist

* Ensure IAM enforcement honors default allow

* address comments

* Reused the precomputed principal when setting tableBucketMetadata.OwnerAccountID, avoiding the redundant getAccountID call.

* get identity

* fix

* dedup

* fix

* comments

* fix tests

* update iam config

* go fmt

* fix ports

* fix flags

* mini clean shutdown

* Revert "update iam config"

This reverts commit ca48fdbb0a.

Revert "mini clean shutdown"

This reverts commit 9e17f6baff.

Revert "fix flags"

This reverts commit e9e7b29d2f.

Revert "go fmt"

This reverts commit bd3241960b.

* test/s3tables: share single weed mini per test package via TestMain

Previously each top-level test function in the catalog and s3tables
package started and stopped its own weed mini instance. This caused
failures when a prior instance wasn't cleanly stopped before the next
one started (port conflicts, leaked global state).

Changes:
- catalog/iceberg_catalog_test.go: introduce TestMain that starts one
  shared TestEnvironment (external weed binary) before all tests and
  tears it down after. All individual test functions now use sharedEnv.
  Added randomSuffix() for unique resource names across tests.
- catalog/pyiceberg_test.go: updated to use sharedEnv instead of
  per-test environments.
- catalog/pyiceberg_test_helpers.go -> pyiceberg_test_helpers_test.go:
  renamed to a _test.go file so it can access TestEnvironment which is
  defined in a test file.
- table-buckets/setup.go: add package-level sharedCluster variable.
- table-buckets/s3tables_integration_test.go: introduce TestMain that
  starts one shared TestCluster before all tests. TestS3TablesIntegration
  now uses sharedCluster. Extract startMiniClusterInDir (no *testing.T)
  for TestMain use. TestS3TablesCreateBucketIAMPolicy keeps its own
  cluster (different IAM config). Remove miniClusterMutex (no longer
  needed). Fix Stop() to not panic when t is nil."

* delete

* parse

* default allow should work with anonymous

* fix port

* iceberg route

The failures are from Iceberg REST using the default bucket warehouse when no prefix is provided. Your tests create random buckets, so /v1/namespaces was looking in warehouse and failing. I updated the tests to use the prefixed Iceberg routes (/v1/{bucket}/...) via a small helper.

* test(s3tables): fix port conflicts and IAM ARN matching in integration tests

- Pass -master.dir explicitly to prevent filer store directory collision
  between shared cluster and per-test clusters running in the same process
- Pass -volume.port.public and -volume.publicUrl to prevent the global
  publicPort flag (mutated from 0 → concrete port by first cluster) from
  being reused by a second cluster, causing 'address already in use'
- Remove the flag-reset loop in Stop() that reset global flag values while
  other goroutines were reading them (race → panic)
- Fix IAM policy Resource ARN in TestS3TablesCreateBucketIAMPolicy to use
  wildcards (arn:aws:s3tables:*:*:bucket/<name>) because the handler
  generates ARNs with its own DefaultRegion (us-east-1) and principal name
  ('admin'), not the test constants testRegion/testAccountID

* docker: fix entrypoint chown guard; helm: add openshift-values.yaml

Fix a regression in entrypoint.sh where the DATA_UID/DATA_GID
ownership comparison was dropped, causing chown -R /data to run
unconditionally on every container start even when ownership was
already correct. Restore the guard so the recursive chown is
skipped when the seaweed user already owns /data — making startup
faster on subsequent runs and a no-op on OpenShift/PVC deployments
where fsGroup has already set correct ownership.

Add k8s/charts/seaweedfs/openshift-values.yaml: an example Helm
overrides file for deploying SeaweedFS on OpenShift (or any cluster
enforcing the Kubernetes restricted Pod Security Standard). Replaces
hostPath volumes with PVCs, sets runAsUser/fsGroup to 1000
(the seaweed user baked into the image), drops all capabilities,
disables privilege escalation, and enables RuntimeDefault seccomp —
satisfying OpenShift's default restricted SCC without needing a
custom SCC or root access.

Fixes #8381"
2026-02-20 00:35:42 -08:00
Michał SzynkiewiczandGitHub 2f837c4780 Fix error on deleting non-empty bucket (#8376)
* Move check for non-empty bucket deletion out of `WithFilerClient` call

* Added proper checking if a bucket has "user" objects
2026-02-19 22:56:50 -08:00
Chris LuandGitHub 36c469e34e Enforce IAM for S3 Tables bucket creation (#8388)
* Enforce IAM for s3tables bucket creation

* Prefer IAM path when policies exist

* Ensure IAM enforcement honors default allow

* address comments

* Reused the precomputed principal when setting tableBucketMetadata.OwnerAccountID, avoiding the redundant getAccountID call.

* get identity

* fix

* dedup

* fix

* comments

* fix tests

* update iam config

* go fmt

* fix ports

* fix flags

* mini clean shutdown

* Revert "update iam config"

This reverts commit ca48fdbb0a.

Revert "mini clean shutdown"

This reverts commit 9e17f6baff.

Revert "fix flags"

This reverts commit e9e7b29d2f.

Revert "go fmt"

This reverts commit bd3241960b.

* test/s3tables: share single weed mini per test package via TestMain

Previously each top-level test function in the catalog and s3tables
package started and stopped its own weed mini instance. This caused
failures when a prior instance wasn't cleanly stopped before the next
one started (port conflicts, leaked global state).

Changes:
- catalog/iceberg_catalog_test.go: introduce TestMain that starts one
  shared TestEnvironment (external weed binary) before all tests and
  tears it down after. All individual test functions now use sharedEnv.
  Added randomSuffix() for unique resource names across tests.
- catalog/pyiceberg_test.go: updated to use sharedEnv instead of
  per-test environments.
- catalog/pyiceberg_test_helpers.go -> pyiceberg_test_helpers_test.go:
  renamed to a _test.go file so it can access TestEnvironment which is
  defined in a test file.
- table-buckets/setup.go: add package-level sharedCluster variable.
- table-buckets/s3tables_integration_test.go: introduce TestMain that
  starts one shared TestCluster before all tests. TestS3TablesIntegration
  now uses sharedCluster. Extract startMiniClusterInDir (no *testing.T)
  for TestMain use. TestS3TablesCreateBucketIAMPolicy keeps its own
  cluster (different IAM config). Remove miniClusterMutex (no longer
  needed). Fix Stop() to not panic when t is nil."

* delete

* parse

* default allow should work with anonymous

* fix port

* iceberg route

The failures are from Iceberg REST using the default bucket warehouse when no prefix is provided. Your tests create random buckets, so /v1/namespaces was looking in warehouse and failing. I updated the tests to use the prefixed Iceberg routes (/v1/{bucket}/...) via a small helper.

* test(s3tables): fix port conflicts and IAM ARN matching in integration tests

- Pass -master.dir explicitly to prevent filer store directory collision
  between shared cluster and per-test clusters running in the same process
- Pass -volume.port.public and -volume.publicUrl to prevent the global
  publicPort flag (mutated from 0 → concrete port by first cluster) from
  being reused by a second cluster, causing 'address already in use'
- Remove the flag-reset loop in Stop() that reset global flag values while
  other goroutines were reading them (race → panic)
- Fix IAM policy Resource ARN in TestS3TablesCreateBucketIAMPolicy to use
  wildcards (arn:aws:s3tables:*:*:bucket/<name>) because the handler
  generates ARNs with its own DefaultRegion (us-east-1) and principal name
  ('admin'), not the test constants testRegion/testAccountID
2026-02-19 22:52:05 -08:00
Chris LuandGitHub a2005cb2a6 fix: resolve gRPC DNS resolution issues in Kubernetes #8384 (#8387)
* fix: resolve gRPC DNS resolution issues in Kubernetes #8384

- Replace direct `grpc.NewClient` calls with `pb.GrpcDial` for consistent connection establishment
- Fix async DNS resolution behavior in K8s with `ndots:5`
- Ensure high-level components use established helper for reliable networking

* refactor: refine gRPC DNS fix and add documentation

- Use instance's grpcDialOption in BrokerClient.ConfigureTopic
- Add detailed comments to GrpcDial explaining Kubernetes DNS resolution rationale

* fix: ensure proper context propagation in broker_client gRPC calls

- Pass the provided `ctx` to `pb.GrpcDial` in `ConfigureTopic` and `GetUnflushedMessages`
- Ensures that timeouts and cancellations are correctly honored during connection establishment

* docs: refine gRPC resolver documentation and cleanup dead code

- Enhanced documentation for `GrpcDial` with explicit warnings about global state mutation when using `resolver.SetDefaultScheme("passthrough")`.
- Recommended `passthrough:///` prefix as the primary migration path for `grpc.NewClient`.
- Removed dead commented-out code for `grpc.WithBlock()` and `grpc.WithTimeout()`.
2026-02-19 15:46:02 -08:00
Chris LuandGitHub e9c45144cf Implement managed policy storage (#8385)
* Persist managed IAM policies

* Add IAM list/get policy integration test

* Faster marker lookup and cleanup

* Handle delete conflict and improve listing

* Add delete-in-use policy integration test

* Stabilize policy ID and guard path prefix

* Tighten CreatePolicy guard and reload

* Add ListPolicyNames to credential store
2026-02-19 14:21:19 -08:00
Chris LuandGitHub 5ecee9e64d s3: fix signature mismatch with non-standard ports and capitalized host (#8386)
* s3: fix signature mismatch with non-standard ports and capitalized host

- ensure host header extraction is case-insensitive in SignedHeaders
- prioritize non-standard ports in X-Forwarded-Host over default ports in X-Forwarded-Port
- add regression tests for both scenarios

fixes https://github.com/seaweedfs/seaweedfs/issues/8382

* simplify
2026-02-19 14:17:31 -08:00
Konstantin LebedevandGitHub 01b3125815 [shell]: volume balance capacity by min volume density (#8026)
volume balance by min volume density and active volumes
2026-02-19 13:30:59 -08:00
Chris LuandGitHub 7b8df39cf7 s3api: add AttachUserPolicy/DetachUserPolicy/ListAttachedUserPolicies (#8379)
* iam: add XML responses for managed user policy APIs

* s3api: implement attach/detach/list attached user policies

* s3api: add embedded IAM tests for managed user policies

* iam: update CredentialStore interface and Manager for managed policies

Updated the `CredentialStore` interface to include `AttachUserPolicy`,
`DetachUserPolicy`, and `ListAttachedUserPolicies` methods.
The `CredentialManager` was updated to delegate these calls to the store.
Added common error variables for policy management.

* iam: implement managed policy methods in MemoryStore

Implemented `AttachUserPolicy`, `DetachUserPolicy`, and
`ListAttachedUserPolicies` in the MemoryStore.
Also ensured deep copying of identities includes PolicyNames.

* iam: implement managed policy methods in PostgresStore

Modified Postgres schema to include `policy_names` JSONB column in `users`.
Implemented `AttachUserPolicy`, `DetachUserPolicy`, and `ListAttachedUserPolicies`.
Updated user CRUD operations to handle policy names persistence.

* iam: implement managed policy methods in remaining stores

Implemented user policy management in:
- `FilerEtcStore` (partial implementation)
- `IamGrpcStore` (delegated via GetUser/UpdateUser)
- `PropagatingCredentialStore` (to broadcast updates)
Ensures cluster-wide consistency for policy attachments.

* s3api: refactor EmbeddedIamApi to use managed policy APIs

- Refactored `AttachUserPolicy`, `DetachUserPolicy`, and `ListAttachedUserPolicies`
  to use `e.credentialManager` directly.
- Fixed a critical error suppression bug in `ExecuteAction` that always
  returned success even on failure.
- Implemented robust error matching using string comparison fallbacks.
- Improved consistency by reloading configuration after policy changes.

* s3api: update and refine IAM integration tests

- Updated tests to use a real `MemoryStore`-backed `CredentialManager`.
- Refined test configuration synchronization using `sync.Once` and
  manual deep-copying to prevent state corruption.
- Improved `extractEmbeddedIamErrorCodeAndMessage` to handle more XML
  formats robustly.
- Adjusted test expectations to match current AWS IAM behavior.

* fix compilation

* visibility

* ensure 10 policies

* reload

* add integration tests

* Guard raft command registration

* Allow IAM actions in policy tests

* Validate gRPC policy attachments

* Revert Validate gRPC policy attachments

* Tighten gRPC policy attach/detach

* Improve IAM managed policy handling

* Improve managed policy filters
2026-02-19 12:26:27 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
6787dccace build(deps): bump filippo.io/edwards25519 from 1.1.0 to 1.1.1 (#8383)
Bumps [filippo.io/edwards25519](https://github.com/FiloSottile/edwards25519) from 1.1.0 to 1.1.1.
- [Commits](https://github.com/FiloSottile/edwards25519/compare/v1.1.0...v1.1.1)

---
updated-dependencies:
- dependency-name: filippo.io/edwards25519
  dependency-version: 1.1.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-19 10:51:27 -08:00
Chris LuandGitHub d1fecdface Fix IAM defaults and S3Tables IAM regression (#8374)
* Fix IAM defaults and s3tables identities

* Refine S3Tables identity tests

* Clarify identity tests
2026-02-18 18:20:03 -08:00
Chris LuandGitHub 38e14a867b fix: cancel volume server requests on client disconnect during S3 downloads (#8373)
* fix: cancel volume server requests on client disconnect during S3 downloads

- Use http.NewRequestWithContext in ReadUrlAsStream so in-flight volume
  server requests are properly aborted when the client disconnects and
  the request context is canceled
- Distinguish context-canceled errors (client disconnect, expected) from
  real server errors in streamFromVolumeServers; log at V(3) instead of
  ERROR to reduce noise from client-side disconnects (e.g. Nginx upstream
  timeout, browser cancel, curl --max-time)

Fixes: streamFromVolumeServers: streamFn failed...context canceled"

* fixup: separate Canceled/DeadlineExceeded log severity in streamFromVolumeServers

- context.Canceled → V(3) Infof "client disconnected" (expected, no noise)
- context.DeadlineExceeded → Warningf "server-side deadline exceeded" (unexpected, needs attention)
- all other errors → Errorf (unchanged)"
2026-02-18 17:14:54 -08:00
Chris Lu eda4a000cc Revert "Fix IAM defaults and s3tables identities"
This reverts commit bf71fe0039.
2026-02-18 16:23:13 -08:00
Chris Lu bf71fe0039 Fix IAM defaults and s3tables identities 2026-02-18 16:21:48 -08:00
53048ffffb Add md5 checksum validation support on PutObject and UploadPart (#8367)
* Add md5 checksum validation support on PutObject and UploadPart

Per the S3 specification, when a client sends a Content-MD5 header, the server must compare it against the MD5 of the received body and return BadDigest (HTTP 400) if they don't match.

SeaweedFS was silently accepting objects with incorrect Content-MD5 headers, which breaks data integrity verification for clients that rely on this feature (e.g. boto3). The error infrastructure (ErrBadDigest, ErrMsgBadDigest) already existed from PR #7306 but was never wired to an actual check.

This commit adds MD5 verification in putToFiler after the body is streamed and the MD5 is computed, and adds Content-MD5 header validation to PutObjectPartHandler (matching PutObjectHandler). Orphaned chunks are cleaned up on mismatch.

Refs: https://github.com/seaweedfs/seaweedfs/discussions/3908

* handle SSE, add uploadpart test

* s3 integration test: fix typo and add multipart upload checksum test

* s3api: move validateContentMd5 after GetBucketAndObject in PutObjectPartHandler

* s3api: move validateContentMd5 after GetBucketAndObject in PutObjectHandler

* s3api: fix MD5 validation for SSE uploads and logging in putToFiler

* add SSE test with checksum validation - mostly ai-generated

* Update s3_integration_test.go

* Address S3 integration test feedback: fix typos, rename variables, add verification steps, and clean up comments.

---------

Co-authored-by: Chris Lu <chris.lu@gmail.com>
2026-02-18 15:40:08 -08:00
6a3a97333f Add support for TLS in gRPC communication between worker and volume server (#8370)
* Add support for TLS in gRPC communication between worker and volume server

* address comments

* worker: capture shared grpc.DialOption in BalanceTask registration closure

* worker: capture shared grpc.DialOption in ErasureCodingTask registration closure

* worker: capture shared grpc.DialOption in VacuumTask registration closure

* worker: use grpc.worker security configuration section for tasks

* plugin/worker: fix compilation errors by passing grpc.DialOption to task constructors

* plugin/worker: prevent double-counting in EC skip counters

---------

Co-authored-by: Chris Lu <chris.lu@gmail.com>
2026-02-18 15:39:53 -08:00
Chris LuGitHubCopilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
8ec9ff4a12 Refactor plugin system and migrate worker runtime (#8369)
* admin: add plugin runtime UI page and route wiring

* pb: add plugin gRPC contract and generated bindings

* admin/plugin: implement worker registry, runtime, monitoring, and config store

* admin/dash: wire plugin runtime and expose plugin workflow APIs

* command: add flags to enable plugin runtime

* admin: rename remaining plugin v2 wording to plugin

* admin/plugin: add detectable job type registry helper

* admin/plugin: add scheduled detection and dispatch orchestration

* admin/plugin: prefetch job type descriptors when workers connect

* admin/plugin: add known job type discovery API and UI

* admin/plugin: refresh design doc to match current implementation

* admin/plugin: enforce per-worker scheduler concurrency limits

* admin/plugin: use descriptor runtime defaults for scheduler policy

* admin/ui: auto-load first known plugin job type on page open

* admin/plugin: bootstrap persisted config from descriptor defaults

* admin/plugin: dedupe scheduled proposals by dedupe key

* admin/ui: add job type and state filters for plugin monitoring

* admin/ui: add per-job-type plugin activity summary

* admin/plugin: split descriptor read API from schema refresh

* admin/ui: keep plugin summary metrics global while tables are filtered

* admin/plugin: retry executor reservation before timing out

* admin/plugin: expose scheduler states for monitoring

* admin/ui: show per-job-type scheduler states in plugin monitor

* pb/plugin: rename protobuf package to plugin

* admin/plugin: rename pluginRuntime wiring to plugin

* admin/plugin: remove runtime naming from plugin APIs and UI

* admin/plugin: rename runtime files to plugin naming

* admin/plugin: persist jobs and activities for monitor recovery

* admin/plugin: lease one detector worker per job type

* admin/ui: show worker load from plugin heartbeats

* admin/plugin: skip stale workers for detector and executor picks

* plugin/worker: add plugin worker command and stream runtime scaffold

* plugin/worker: implement vacuum detect and execute handlers

* admin/plugin: document external vacuum plugin worker starter

* command: update plugin.worker help to reflect implemented flow

* command/admin: drop legacy Plugin V2 label

* plugin/worker: validate vacuum job type and respect min interval

* plugin/worker: test no-op detect when min interval not elapsed

* command/admin: document plugin.worker external process

* plugin/worker: advertise configured concurrency in hello

* command/plugin.worker: add jobType handler selection

* command/plugin.worker: test handler selection by job type

* command/plugin.worker: persist worker id in workingDir

* admin/plugin: document plugin.worker jobType and workingDir flags

* plugin/worker: support cancel request for in-flight work

* plugin/worker: test cancel request acknowledgements

* command/plugin.worker: document workingDir and jobType behavior

* plugin/worker: emit executor activity events for monitor

* plugin/worker: test executor activity builder

* admin/plugin: send last successful run in detection request

* admin/plugin: send cancel request when detect or execute context ends

* admin/plugin: document worker cancel request responsibility

* admin/handlers: expose plugin scheduler states API in no-auth mode

* admin/handlers: test plugin scheduler states route registration

* admin/plugin: keep worker id on worker-generated activity records

* admin/plugin: test worker id propagation in monitor activities

* admin/dash: always initialize plugin service

* command/admin: remove plugin enable flags and default to enabled

* admin/dash: drop pluginEnabled constructor parameter

* admin/plugin UI: stop checking plugin enabled state

* admin/plugin: remove docs for plugin enable flags

* admin/dash: remove unused plugin enabled check method

* admin/dash: fallback to in-memory plugin init when dataDir fails

* admin/plugin API: expose worker gRPC port in status

* command/plugin.worker: resolve admin gRPC port via plugin status

* split plugin UI into overview/configuration/monitoring pages

* Update layout_templ.go

* add volume_balance plugin worker handler

* wire plugin.worker CLI for volume_balance job type

* add erasure_coding plugin worker handler

* wire plugin.worker CLI for erasure_coding job type

* support multi-job handlers in plugin worker runtime

* allow plugin.worker jobType as comma-separated list

* admin/plugin UI: rename to Workers and simplify config view

* plugin worker: queue detection requests instead of capacity reject

* Update plugin_worker.go

* plugin volume_balance: remove force_move/timeout from worker config UI

* plugin erasure_coding: enforce local working dir and cleanup

* admin/plugin UI: rename admin settings to job scheduling

* admin/plugin UI: persist and robustly render detection results

* admin/plugin: record and return detection trace metadata

* admin/plugin UI: show detection process and decision trace

* plugin: surface detector decision trace as activities

* mini: start a plugin worker by default

* admin/plugin UI: split monitoring into detection and execution tabs

* plugin worker: emit detection decision trace for EC and balance

* admin workers UI: split monitoring into detection and execution pages

* plugin scheduler: skip proposals for active assigned/running jobs

* admin workers UI: add job queue tab

* plugin worker: add dummy stress detector and executor job type

* admin workers UI: reorder tabs to detection queue execution

* admin workers UI: regenerate plugin template

* plugin defaults: include dummy stress and add stress tests

* plugin dummy stress: rotate detection selections across runs

* plugin scheduler: remove cross-run proposal dedupe

* plugin queue: track pending scheduled jobs

* plugin scheduler: wait for executor capacity before dispatch

* plugin scheduler: skip detection when waiting backlog is high

* plugin: add disk-backed job detail API and persistence

* admin ui: show plugin job detail modal from job id links

* plugin: generate unique job ids instead of reusing proposal ids

* plugin worker: emit heartbeats on work state changes

* plugin registry: round-robin tied executor and detector picks

* add temporary EC overnight stress runner

* plugin job details: persist and render EC execution plans

* ec volume details: color data and parity shard badges

* shard labels: keep parity ids numeric and color-only distinction

* admin: remove legacy maintenance UI routes and templates

* admin: remove dead maintenance endpoint helpers

* Update layout_templ.go

* remove dummy_stress worker and command support

* refactor plugin UI to job-type top tabs and sub-tabs

* migrate weed worker command to plugin runtime

* remove plugin.worker command and keep worker runtime with metrics

* update helm worker args for jobType and execution flags

* set plugin scheduling defaults to global 16 and per-worker 4

* stress: fix RPC context reuse and remove redundant variables in ec_stress_runner

* admin/plugin: fix lifecycle races, safe channel operations, and terminal state constants

* admin/dash: randomize job IDs and fix priority zero-value overwrite in plugin API

* admin/handlers: implement buffered rendering to prevent response corruption

* admin/plugin: implement debounced persistence flusher and optimize BuildJobDetail memory lookups

* admin/plugin: fix priority overwrite and implement bounded wait in scheduler reserve

* admin/plugin: implement atomic file writes and fix run record side effects

* admin/plugin: use P prefix for parity shard labels in execution plans

* admin/plugin: enable parallel execution for cancellation tests

* admin: refactor time.Time fields to pointers for better JSON omitempty support

* admin/plugin: implement pointer-safe time assignments and comparisons in plugin core

* admin/plugin: fix time assignment and sorting logic in plugin monitor after pointer refactor

* admin/plugin: update scheduler activity tracking to use time pointers

* admin/plugin: fix time-based run history trimming after pointer refactor

* admin/dash: fix JobSpec struct literal in plugin API after pointer refactor

* admin/view: add D/P prefixes to EC shard badges for UI consistency

* admin/plugin: use lifecycle-aware context for schema prefetching

* Update ec_volume_details_templ.go

* admin/stress: fix proposal sorting and log volume cleanup errors

* stress: refine ec stress runner with math/rand and collection name

- Added Collection field to VolumeEcShardsDeleteRequest for correct filename construction.
- Replaced crypto/rand with seeded math/rand PRNG for bulk payloads.
- Added documentation for EcMinAge zero-value behavior.
- Added logging for ignored errors in volume/shard deletion.

* admin: return internal server error for plugin store failures

Changed error status code from 400 Bad Request to 500 Internal Server Error for failures in GetPluginJobDetail to correctly reflect server-side errors.

* admin: implement safe channel sends and graceful shutdown sync

- Added sync.WaitGroup to Plugin struct to manage background goroutines.
- Implemented safeSendCh helper using recover() to prevent panics on closed channels.
- Ensured Shutdown() waits for all background operations to complete.

* admin: robustify plugin monitor with nil-safe time and record init

- Standardized nil-safe assignment for *time.Time pointers (CreatedAt, UpdatedAt, CompletedAt).
- Ensured persistJobDetailSnapshot initializes new records correctly if they don't exist on disk.
- Fixed debounced persistence to trigger immediate write on job completion.

* admin: improve scheduler shutdown behavior and logic guards

- Replaced brittle error string matching with explicit r.shutdownCh selection for shutdown detection.
- Removed redundant nil guard in buildScheduledJobSpec.
- Standardized WaitGroup usage for schedulerLoop.

* admin: implement deep copy for job parameters and atomic write fixes

- Implemented deepCopyGenericValue and used it in cloneTrackedJob to prevent shared state.
- Ensured atomicWriteFile creates parent directories before writing.

* admin: remove unreachable branch in shard classification

Removed an unreachable 'totalShards <= 0' check in classifyShardID as dataShards and parityShards are already guarded.

* admin: secure UI links and use canonical shard constants

- Added rel="noopener noreferrer" to external links for security.
- Replaced magic number 14 with erasure_coding.TotalShardsCount.
- Used renderEcShardBadge for missing shard list consistency.

* admin: stabilize plugin tests and fix regressions

- Composed a robust plugin_monitor_test.go to handle asynchronous persistence.
- Updated all time.Time literals to use timeToPtr helper.
- Added explicit Shutdown() calls in tests to synchronize with debounced writes.
- Fixed syntax errors and orphaned struct literals in tests.

* Potential fix for code scanning alert no. 278: Slice memory allocation with excessive size value

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for code scanning alert no. 283: Uncontrolled data used in path expression

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* admin: finalize refinements for error handling, scheduler, and race fixes

- Standardized HTTP 500 status codes for store failures in plugin_api.go.
- Tracked scheduled detection goroutines with sync.WaitGroup for safe shutdown.
- Fixed race condition in safeSendDetectionComplete by extracting channel under lock.
- Implemented deep copy for JobActivity details.
- Used defaultDirPerm constant in atomicWriteFile.

* test(ec): migrate admin dockertest to plugin APIs

* admin/plugin_api: fix RunPluginJobTypeAPI to return 500 for server-side detection/filter errors

* admin/plugin_api: fix ExecutePluginJobAPI to return 500 for job execution failures

* admin/plugin_api: limit parseProtoJSONBody request body to 1MB to prevent unbounded memory usage

* admin/plugin: consolidate regex to package-level validJobTypePattern; add char validation to sanitizeJobID

* admin/plugin: fix racy Shutdown channel close with sync.Once

* admin/plugin: track sendLoop and recv goroutines in WorkerStream with r.wg

* admin/plugin: document writeProtoFiles atomicity — .pb is source of truth, .json is human-readable only

* admin/plugin: extract activityLess helper to deduplicate nil-safe OccurredAt sort comparators

* test/ec: check http.NewRequest errors to prevent nil req panics

* test/ec: replace deprecated ioutil/math/rand, fix stale step comment 5.1→3.1

* plugin(ec): raise default detection and scheduling throughput limits

* topology: include empty disks in volume list and EC capacity fallback

* topology: remove hard 10-task cap for detection planning

* Update ec_volume_details_templ.go

* adjust default

* fix tests

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-02-18 13:42:41 -08:00
github-pawoandGitHub 5463038760 Remove trailing spaces (line 53) in seaweedfs-dev-compose.yml (#8365) 2026-02-18 07:32:07 -08:00
github-pawoandGitHub 828cbabb55 Add Admin UI to Docker Compose files (#8364) 2026-02-18 01:05:54 -08:00
Chris LuandGitHub 5919f519fd fix: allow overriding Enterprise image name using Helm #8361 (#8363)
* fix: allow overriding Enterprise image name using Helm #8361

* refactor: flatten image name construction logic for better readability
2026-02-17 13:49:16 -08:00
Chris Lu 63f641a6c9 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2026-02-16 17:01:26 -08:00
Chris Lu 3c3a78d08e 4.13 2026-02-16 17:01:19 -08:00
Chris LuandGitHub 3300874cb5 filer: add default log purging to master maintenance scripts (#8359)
* filer: add default log purging to master maintenance scripts

* filer: fix default maintenance scripts to include full set of tasks

* filer: refactor maintenance scripts to avoid duplication
2026-02-16 16:58:15 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>Chris Lu
bddd7960c1 build(deps): bump org.apache.avro:avro from 1.11.4 to 1.11.5 in /test/java/spark (#8358)
build(deps): bump org.apache.avro:avro in /test/java/spark

Bumps org.apache.avro:avro from 1.11.4 to 1.11.5.

---
updated-dependencies:
- dependency-name: org.apache.avro:avro
  dependency-version: 1.11.5
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2026-02-16 15:28:55 -08:00
Lisandro PinandGitHub a9d12a0792 Implement full scrubbing for EC volumes (#8318)
Implement full scrubbing for EC volumes.
2026-02-16 15:09:01 -08:00
Chris Lu 564fc56698 Update docker-compose.yml 2026-02-16 15:01:09 -08:00
Lisandro PinandGitHub 11fdb68281 Fix superblock write error checks on volume compaction. (#8352) 2026-02-16 14:44:37 -08:00
Chris Lu 35ad7d08a5 remove debug 2026-02-16 14:03:02 -08:00
Chris LuandGitHub 0d8588e3ae S3: Implement IAM defaults and STS signing key fallback (#8348)
* S3: Implement IAM defaults and STS signing key fallback logic

* S3: Refactor startup order to init SSE-S3 key manager before IAM

* S3: Derive STS signing key from KEK using HKDF for security isolation

* S3: Document STS signing key fallback in security.toml

* fix(s3api): refine anonymous access logic and secure-by-default behavior

- Initialize anonymous identity by default in `NewIdentityAccessManagement` to prevent nil pointer exceptions.
- Ensure `ReplaceS3ApiConfiguration` preserves the anonymous identity if not present in the new configuration.
- Update `NewIdentityAccessManagement` signature to accept `filerClient`.
- In legacy mode (no policy engine), anonymous defaults to Deny (no actions), preserving secure-by-default behavior.
- Use specific `LookupAnonymous` method instead of generic map lookup.
- Update tests to accommodate signature changes and verify improved anonymous handling.

* feat(s3api): make IAM configuration optional

- Start S3 API server without a configuration file if `EnableIam` option is set.
- Default to `Allow` effect for policy engine when no configuration is provided (Zero-Config mode).
- Handle empty configuration path gracefully in `loadIAMManagerFromConfig`.
- Add integration test `iam_optional_test.go` to verify empty config behavior.

* fix(iamapi): fix signature mismatch in NewIdentityAccessManagementWithStore

* fix(iamapi): properly initialize FilerClient instead of passing nil

* fix(iamapi): properly initialize filer client for IAM management

- Instead of passing `nil`, construct a `wdclient.FilerClient` using the provided `Filers` addresses.
- Ensure `NewIdentityAccessManagementWithStore` receives a valid `filerClient` to avoid potential nil pointer dereferences or limited functionality.

* clean: remove dead code in s3api_server.go

* refactor(s3api): improve IAM initialization, safety and anonymous access security

* fix(s3api): ensure IAM config loads from filer after client init

* fix(s3): resolve test failures in integration, CORS, and tagging tests

- Fix CORS tests by providing explicit anonymous permissions config
- Fix S3 integration tests by setting admin credentials in init
- Align tagging test credentials in CI with IAM defaults
- Added goroutine to retry IAM config load in iamapi server

* fix(s3): allow anonymous access to health targets and S3 Tables when identities are present

* fix(ci): use /healthz for Caddy health check in awscli tests

* iam, s3api: expose DefaultAllow from IAM and Policy Engine

This allows checking the global "Open by Default" configuration from
other components like S3 Tables.

* s3api/s3tables: support DefaultAllow in permission logic and handler

Updated CheckPermissionWithContext to respect the DefaultAllow flag
in PolicyContext. This enables "Open by Default" behavior for
unauthenticated access in zero-config environments. Added a targeted
unit test to verify the logic.

* s3api/s3tables: propagate DefaultAllow through handlers

Propagated the DefaultAllow flag to individual handlers for
namespaces, buckets, tables, policies, and tagging. This ensures
consistent "Open by Default" behavior across all S3 Tables API
endpoints.

* s3api: wire up DefaultAllow for S3 Tables API initialization

Updated registerS3TablesRoutes to query the global IAM configuration
and set the DefaultAllow flag on the S3 Tables API server. This
completes the end-to-end propagation required for anonymous access in
zero-config environments. Added a SetDefaultAllow method to
S3TablesApiServer to facilitate this.

* s3api: fix tests by adding DefaultAllow to mock IAM integrations

The IAMIntegration interface was updated to include DefaultAllow(),
breaking several mock implementations in tests. This commit fixes
the build errors by adding the missing method to the mocks.

* env

* ensure ports

* env

* env

* fix default allow

* add one more test using non-anonymous user

* debug

* add more debug

* less logs
2026-02-16 13:59:13 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
cc58272219 build(deps): bump github.com/klauspost/compress from 1.18.3 to 1.18.4 (#8353)
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress) from 1.18.3 to 1.18.4.
- [Release notes](https://github.com/klauspost/compress/releases)
- [Commits](https://github.com/klauspost/compress/compare/v1.18.3...v1.18.4)

---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
  dependency-version: 1.18.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-16 10:58:13 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
5be4ee9892 build(deps): bump github.com/redis/go-redis/v9 from 9.17.2 to 9.18.0 (#8356)
Bumps [github.com/redis/go-redis/v9](https://github.com/redis/go-redis) from 9.17.2 to 9.18.0.
- [Release notes](https://github.com/redis/go-redis/releases)
- [Changelog](https://github.com/redis/go-redis/blob/master/RELEASE-NOTES.md)
- [Commits](https://github.com/redis/go-redis/compare/v9.17.2...v9.18.0)

---
updated-dependencies:
- dependency-name: github.com/redis/go-redis/v9
  dependency-version: 9.18.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-16 10:57:54 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
22e74221cb build(deps): bump github.com/getsentry/sentry-go from 0.40.0 to 0.42.0 (#8357)
Bumps [github.com/getsentry/sentry-go](https://github.com/getsentry/sentry-go) from 0.40.0 to 0.42.0.
- [Release notes](https://github.com/getsentry/sentry-go/releases)
- [Changelog](https://github.com/getsentry/sentry-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/getsentry/sentry-go/compare/v0.40.0...v0.42.0)

---
updated-dependencies:
- dependency-name: github.com/getsentry/sentry-go
  dependency-version: 0.42.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-16 10:57:43 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
cc80641be1 build(deps): bump github.com/mattn/go-sqlite3 from 1.14.33 to 1.14.34 (#8355)
Bumps [github.com/mattn/go-sqlite3](https://github.com/mattn/go-sqlite3) from 1.14.33 to 1.14.34.
- [Release notes](https://github.com/mattn/go-sqlite3/releases)
- [Commits](https://github.com/mattn/go-sqlite3/compare/v1.14.33...v1.14.34)

---
updated-dependencies:
- dependency-name: github.com/mattn/go-sqlite3
  dependency-version: 1.14.34
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-16 10:57:31 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
927c906379 build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob from 1.6.3 to 1.6.4 (#8354)
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob

Bumps [github.com/Azure/azure-sdk-for-go/sdk/storage/azblob](https://github.com/Azure/azure-sdk-for-go) from 1.6.3 to 1.6.4.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/storage/azblob/v1.6.3...sdk/storage/azblob/v1.6.4)

---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
  dependency-version: 1.6.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-16 10:57:17 -08:00
Lisandro PinandGitHub 0721e3c1e9 Rework volume compaction (a.k.a vacuuming) logic to cleanly support new parameters. (#8337)
We'll leverage on this to support a "ignore broken needles" option, necessary
to properly recover damaged volumes, as described in
https://github.com/seaweedfs/seaweedfs/issues/7442#issuecomment-3897784283 .
2026-02-16 02:15:14 -08:00
Chris LuandGitHub 703d5e27b3 Fix S3 ListObjectsV2 recursion issue (#8347)
* Fix S3 ListObjectsV2 recursion issue (#8346)

Removed aggressive Limit=1 optimization in doListFilerEntries that caused missed directory entries when prefix ended with a delimiter. Added regression tests to verify deep directory traversal.

* Address PR comments: condense test comments
2026-02-15 10:52:10 -08:00
Chris Lu e863767ac7 cleanup(iam): final removal of temporary debug logging from STS and S3 API 2026-02-14 22:15:06 -08:00
Chris Lu e29a7f1741 cleanup(iam): remove temporary debug logging from STS and S3 API (redo) 2026-02-14 22:14:33 -08:00
Chris LuandGitHub cf8e383e1e STS: Fallback to Caller Identity when RoleArn is missing in AssumeRole (#8345)
* s3api: make RoleArn optional in AssumeRole

* s3api: address PR feedback for optional RoleArn

* iam: add configurable default role for AssumeRole

* S3 STS: Use caller identity when RoleArn is missing

- Fallback to PrincipalArn/Context in AssumeRole if RoleArn is empty

- Handle User ARNs in prepareSTSCredentials

- Fix PrincipalArn generation for env var credentials

* Test: Add unit test for AssumeRole caller identity fallback

* fix(s3api): propagate admin permissions to assumed role session when using caller identity fallback

* STS: Fix is_admin propagation and optimize IAM policy evaluation for assumed roles

- Restore is_admin propagation via JWT req_ctx
- Optimize IsActionAllowed to skip role lookups for admin sessions
- Ensure session policies are still applied for downscoping
- Remove debug logging
- Fix syntax errors in cleanup

* fix(iam): resolve STS policy bypass for admin sessions

- Fixed IsActionAllowed in iam_manager.go to correctly identify and validate internal STS tokens, ensuring session policies are enforced.
- Refactored VerifyActionPermission in auth_credentials.go to properly handle session tokens and avoid legacy authorization short-circuits.
- Added debug logging for better tracing of policy evaluation and session validation.
2026-02-14 22:00:59 -08:00
f49f6c6876 FUSE mount: fix failed git clone (#8344)
tests: reset MemoryStore to avoid test pollution; fix port reservation to prevent duplicate ports in mini

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-02-14 00:28:20 -08:00
7799915e50 Fix IAM identity loss on S3 restart migration (#8343)
* Fix IAM reload after legacy config migration

Handle legacy identity.json metadata events by reloading from the credential manager instead of parsing event content, and watch the correct /etc/iam multi-file directories so identity changes are applied.

Add regression tests for legacy deletion and /etc/iam/identities change events.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix auth_credentials_subscribe_test helper to not pollute global memory store

The SaveConfiguration call was affecting other tests. Use local credential manager and ReplaceS3ApiConfiguration instead.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix IAM event watching: subscribe to IAM directories and improve directory matching

- Add /etc/iam and its subdirectories (identities, policies, service_accounts) to directoriesToWatch
- Fix directory matching to avoid false positives from sibling directories
  - Use exact match or prefix with trailing slash instead of plain HasPrefix
  - Prevents matching hypothetical /etc/iam/identities_backup directory

This ensures IAM config change events are actually delivered to the handler.

* fix tests

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-02-13 22:49:27 -08:00
Chris LuandGitHub c090604143 Add UpdateAccessKey support to IAM API (#8342)
* Add UpdateAccessKey support to IAM API

* simplify
2026-02-13 21:11:07 -08:00
Chris LuandGitHub f44e25b422 fix(iam): ensure access key status is persisted and defaulted to Active (#8341)
* Fix master leader election startup issue

Fixes #error-log-leader-not-selected-yet

* not useful test

* fix(iam): ensure access key status is persisted and defaulted to Active

* make pb

* update tests

* using constants
2026-02-13 20:28:41 -08:00
Lisandro PinandGitHub fbe7dd32c2 Implement full scrubbing for regular volumes (#8254)
Implement full scrubbing for regular volumes.
2026-02-13 15:47:29 -08:00
Lisandro PinandGitHub 1ebc9dd530 Have local EC volume scrubbing check needle integrity whenever possible. (#8334)
If local EC scrubbing hits needles whose chunk location reside entirely
in local shards, we can fully reconstruct them, and check CRCs for
data integrity.
2026-02-13 15:43:17 -08:00
Chris LuandGitHub b08bb8237c Fix master leader election startup issue (#8340)
* Fix master leader election startup issue

Fixes #error-log-leader-not-selected-yet

* Fix master leader election startup issue

This change improves server address comparison using the 'Equals' method and handles recursion in topology leader lookup, resolving the 'leader not selected yet' error during master startup.

* Merge user improvements: use MaybeLeader for non-blocking checks

* not useful test

* Address code review: optimize Equals, fix deadlock in IsLeader, safe access in Leader
2026-02-13 15:39:39 -08:00
Chris Lu f1bf60d288 faster 2026-02-13 14:09:30 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
35b6e895cc build(deps): bump org.apache.avro:avro from 1.11.4 to 1.11.5 in /test/kafka/kafka-client-loadtest/tools (#8339)
build(deps): bump org.apache.avro:avro

Bumps org.apache.avro:avro from 1.11.4 to 1.11.5.

---
updated-dependencies:
- dependency-name: org.apache.avro:avro
  dependency-version: 1.11.5
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-13 13:58:42 -08:00
Chris LuandGitHub 49a64f50f1 Add session policy support to IAM (#8338)
* Add session policy support to IAM

- Implement policy evaluation for session tokens in policy_engine.go
- Add session_policy field to session claims for tracking applied policies
- Update STS service to include session policies in token generation
- Add IAM integration tests for session policy validation
- Update IAM manager to support policy attachment to sessions
- Extend S3 API STS endpoint to handle session policy restrictions

* fix: optimize session policy evaluation and add documentation

* sts: add NormalizeSessionPolicy helper for inline session policies

* sts: support inline session policies for AssumeRoleWithWebIdentity and credential-based flows

* s3api: parse and normalize Policy parameter for STS HTTP handlers

* tests: add session policy unit tests and integration tests for inline policy downscoping

* tests: add s3tables STS inline policy integration

* iam: handle user principals and validate tokens

* sts: enforce inline session policy size limit

* tests: harden s3tables STS integration config

* iam: clarify principal policy resolution errors

* tests: improve STS integration endpoint selection
2026-02-13 13:58:22 -08:00
Chris LuandGitHub beeb375a88 Add volume server integration test suite and CI workflow (#8322)
* docs(volume_server): add integration test development plan

* test(volume_server): add integration harness and profile matrix

* test(volume_server/http): add admin and options integration coverage

* test(volume_server/grpc): add state and status integration coverage

* test(volume_server): auto-build weed binary and harden cluster startup

* test(volume_server/http): add upload read range head delete coverage

* test(volume_server/grpc): expand admin lifecycle and state coverage

* docs(volume_server): update progress tracker for implemented tests

* test(volume_server/http): cover if-none-match and invalid-range branches

* test(volume_server/grpc): add batch delete integration coverage

* docs(volume_server): log latest HTTP and gRPC test coverage

* ci(volume_server): run volume server integration tests in github actions

* test(volume_server/grpc): add needle status configure ping and leave coverage

* docs(volume_server): record additional grpc coverage progress

* test(volume_server/grpc): add vacuum integration coverage

* docs(volume_server): record vacuum test coverage progress

* test(volume_server/grpc): add read and write needle blob error-path coverage

* docs(volume_server): record data rw grpc coverage progress

* test(volume_server/http): add jwt auth integration coverage

* test(volume_server/grpc): add sync copy and stream error-path coverage

* docs(volume_server): record jwt and sync/copy test coverage

* test(volume_server/grpc): add scrub and query integration coverage

* test(volume_server/grpc): add volume tail sender and receiver coverage

* docs(volume_server): record scrub query and tail test progress

* test(volume_server/grpc): add readonly writable and collection lifecycle coverage

* test(volume_server/http): add public-port cors and method parity coverage

* test(volume_server/grpc): add blob meta and read-all success path coverage

* test(volume_server/grpc): expand scrub and query variation coverage

* test(volume_server/grpc): add tiering and remote fetch error-path coverage

* test(volume_server/http): add unchanged write and delete edge-case coverage

* test(volume_server/grpc): add ping unknown and unreachable target coverage

* test(volume_server/grpc): add volume delete only-empty variation coverage

* test(volume_server/http): add jwt fid-mismatch auth coverage

* test(volume_server/grpc): add scrub ec auto-select empty coverage

* test(volume_server/grpc): stabilize ping timestamp assertion

* docs(volume_server): update integration coverage progress log

* test(volume_server/grpc): add tier remote backend and config variation coverage

* docs(volume_server): record tier remote variation progress

* test(volume_server/grpc): add incremental copy and receive-file protocol coverage

* test(volume_server/http): add read path shape and if-modified-since coverage

* test(volume_server/grpc): add copy-file compaction and receive-file success coverage

* test(volume_server/http): add passthrough headers and static asset coverage

* test(volume_server/grpc): add ping filer unreachable coverage

* docs(volume_server): record copy receive and http variant progress

* test(volume_server/grpc): add erasure coding maintenance and missing-path coverage

* docs(volume_server): record initial erasure coding rpc coverage

* test(volume_server/http): add multi-range multipart response coverage

* docs(volume_server): record multi-range http coverage progress

* test(volume_server/grpc): add query empty-stripe no-match coverage

* docs(volume_server): record query no-match stream behavior coverage

* test(volume_server/http): add upload throttling timeout and replicate bypass coverage

* docs(volume_server): record upload throttling coverage progress

* test(volume_server/http): add download throttling timeout coverage

* docs(volume_server): record download throttling coverage progress

* test(volume_server/http): add jwt wrong-cookie fid mismatch coverage

* docs(volume_server): record jwt wrong-cookie mismatch coverage

* test(volume_server/http): add jwt expired-token rejection coverage

* docs(volume_server): record jwt expired-token coverage

* test(volume_server/http): add jwt query and cookie transport coverage

* docs(volume_server): record jwt token transport coverage

* test(volume_server/http): add jwt token-source precedence coverage

* docs(volume_server): record jwt token-source precedence coverage

* test(volume_server/http): add jwt header-over-cookie precedence coverage

* docs(volume_server): record jwt header cookie precedence coverage

* test(volume_server/http): add jwt query-over-cookie precedence coverage

* docs(volume_server): record jwt query cookie precedence coverage

* test(volume_server/grpc): add setstate version mismatch and nil-state coverage

* docs(volume_server): record setstate validation coverage

* test(volume_server/grpc): add readonly persist-true lifecycle coverage

* docs(volume_server): record readonly persist variation coverage

* test(volume_server/http): add options origin cors header coverage

* docs(volume_server): record options origin cors coverage

* test(volume_server/http): add trace unsupported-method parity coverage

* docs(volume_server): record trace method parity coverage

* test(volume_server/grpc): add batch delete cookie-check variation coverage

* docs(volume_server): record batch delete cookie-check coverage

* test(volume_server/grpc): add admin lifecycle missing and maintenance variants

* docs(volume_server): record admin lifecycle edge-case coverage

* test(volume_server/grpc): add mixed batch delete status matrix coverage

* docs(volume_server): record mixed batch delete matrix coverage

* test(volume_server/http): add jwt-profile ui access gating coverage

* docs(volume_server): record jwt ui-gating http coverage

* test(volume_server/http): add propfind unsupported-method parity coverage

* docs(volume_server): record propfind method parity coverage

* test(volume_server/grpc): add volume configure success and rollback-path coverage

* docs(volume_server): record volume configure branch coverage

* test(volume_server/grpc): add volume needle status missing-path coverage

* docs(volume_server): record volume needle status error-path coverage

* test(volume_server/http): add readDeleted query behavior coverage

* docs(volume_server): record readDeleted http behavior coverage

* test(volume_server/http): add delete ts override parity coverage

* docs(volume_server): record delete ts parity coverage

* test(volume_server/grpc): add invalid blob/meta offset coverage

* docs(volume_server): record invalid blob/meta offset coverage

* test(volume_server/grpc): add read-all mixed volume abort coverage

* docs(volume_server): record read-all mixed-volume abort coverage

* test(volume_server/http): assert head response body parity

* docs(volume_server): record head body parity assertion

* test(volume_server/grpc): assert status state and memory payload completeness

* docs(volume_server): record volume server status payload coverage

* test(volume_server/grpc): add batch delete chunk-manifest rejection coverage

* docs(volume_server): record batch delete chunk-manifest coverage

* test(volume_server/grpc): add query cookie-mismatch eof parity coverage

* docs(volume_server): record query cookie-mismatch parity coverage

* test(volume_server/grpc): add ping master success target coverage

* docs(volume_server): record ping master success coverage

* test(volume_server/http): add head if-none-match conditional parity

* docs(volume_server): record head if-none-match parity coverage

* test(volume_server/http): add head if-modified-since parity coverage

* docs(volume_server): record head if-modified-since parity coverage

* test(volume_server/http): add connect unsupported-method parity coverage

* docs(volume_server): record connect method parity coverage

* test(volume_server/http): assert options allow-headers cors parity

* docs(volume_server): record options allow-headers coverage

* test(volume_server/framework): add dual volume cluster integration harness

* test(volume_server/http): add missing-local read mode proxy redirect local coverage

* docs(volume_server): record read mode missing-local matrix coverage

* test(volume_server/http): add download over-limit replica proxy fallback coverage

* docs(volume_server): record download replica fallback coverage

* test(volume_server/http): add missing-local readDeleted proxy redirect parity coverage

* docs(volume_server): record missing-local readDeleted mode coverage

* test(volume_server/framework): add single-volume cluster with filer harness

* test(volume_server/grpc): add ping filer success target coverage

* docs(volume_server): record ping filer success coverage

* test(volume_server/http): add proxied-loop guard download timeout coverage

* docs(volume_server): record proxied-loop download coverage

* test(volume_server/http): add disabled upload and download limit coverage

* docs(volume_server): record disabled throttling path coverage

* test(volume_server/grpc): add idempotent volume server leave coverage

* docs(volume_server): record leave idempotence coverage

* test(volume_server/http): add redirect collection query preservation coverage

* docs(volume_server): record redirect collection query coverage

* test(volume_server/http): assert admin server headers on status and health

* docs(volume_server): record admin server header coverage

* test(volume_server/http): assert healthz request-id echo parity

* docs(volume_server): record healthz request-id parity coverage

* test(volume_server/http): add over-limit invalid-vid download branch coverage

* docs(volume_server): record over-limit invalid-vid branch coverage

* test(volume_server/http): add public-port static asset coverage

* docs(volume_server): record public static endpoint coverage

* test(volume_server/http): add public head method parity coverage

* docs(volume_server): record public head parity coverage

* test(volume_server/http): add throttling wait-then-proceed path coverage

* docs(volume_server): record throttling wait-then-proceed coverage

* test(volume_server/http): add read cookie-mismatch not-found coverage

* docs(volume_server): record read cookie-mismatch coverage

* test(volume_server/http): add throttling timeout-recovery coverage

* docs(volume_server): record throttling timeout-recovery coverage

* test(volume_server/grpc): add ec generate mount info unmount lifecycle coverage

* docs(volume_server): record ec positive lifecycle coverage

* test(volume_server/grpc): add ec shard read and blob delete lifecycle coverage

* docs(volume_server): record ec shard read/blob delete lifecycle coverage

* test(volume_server/grpc): add ec rebuild and to-volume error branch coverage

* docs(volume_server): record ec rebuild and to-volume branch coverage

* test(volume_server/grpc): add ec shards-to-volume success roundtrip coverage

* docs(volume_server): record ec shards-to-volume success coverage

* test(volume_server/grpc): add ec receive and copy-file missing-source coverage

* docs(volume_server): record ec receive and copy-file coverage

* test(volume_server/grpc): add ec last-shard delete cleanup coverage

* docs(volume_server): record ec last-shard delete cleanup coverage

* test(volume_server/grpc): add volume copy success path coverage

* docs(volume_server): record volume copy success coverage

* test(volume_server/grpc): add volume copy overwrite-destination coverage

* docs(volume_server): record volume copy overwrite coverage

* test(volume_server/http): add write error-path variant coverage

* docs(volume_server): record http write error-path coverage

* test(volume_server/http): add conditional header precedence coverage

* docs(volume_server): record conditional header precedence coverage

* test(volume_server/http): add oversized combined range guard coverage

* docs(volume_server): record oversized range guard coverage

* test(volume_server/http): add image resize and crop read coverage

* docs(volume_server): record image transform coverage

* test(volume_server/http): add chunk-manifest expansion and bypass coverage

* docs(volume_server): record chunk-manifest read coverage

* test(volume_server/http): add compressed read encoding matrix coverage

* docs(volume_server): record compressed read matrix coverage

* test(volume_server/grpc): add tail receiver source replication coverage

* docs(volume_server): record tail receiver replication coverage

* test(volume_server/grpc): add tail sender large-needle chunking coverage

* docs(volume_server): record tail sender chunking coverage

* test(volume_server/grpc): add ec-backed volume needle status coverage

* docs(volume_server): record ec-backed needle status coverage

* test(volume_server/grpc): add ec shard copy from peer success coverage

* docs(volume_server): record ec shard copy success coverage

* test(volume_server/http): add chunk-manifest delete child cleanup coverage

* docs(volume_server): record chunk-manifest delete cleanup coverage

* test(volume_server/http): add chunk-manifest delete failure-path coverage

* docs(volume_server): record chunk-manifest delete failure coverage

* test(volume_server/grpc): add ec shard copy source-unavailable coverage

* docs(volume_server): record ec shard copy source-unavailable coverage

* parallel
2026-02-13 00:40:56 -08:00
Chris LuandGitHub c433fee36a s3api: fix AccessDenied by correctly propagating principal ARN in vended tokens (#8330)
* s3api: fix AccessDenied by correctly propagating principal ARN in vended tokens

* s3api: update TestLoadS3ApiConfiguration to match standardized ARN format

* s3api: address PR review comments (nil-safety and cleanup)

* s3api: address second round of PR review comments (cleanups and naming conventions)

* s3api: address third round of PR review comments (unify default account ID and duplicate log)

* s3api: address fourth round of PR review comments (define defaultAccountID as constant)
2026-02-12 23:11:41 -08:00
Chris LuandGitHub 1e4f30c56f pb: fix IPv6 double brackets in ServerAddress formatting (#8329)
* pb: fix IPv6 double brackets in ServerAddress formatting

* pb: refactor IPv6 tests into table-driven test

* util: add JoinHostPortStr and use it in pb to avoid unsafe port parsing
2026-02-12 18:11:03 -08:00
796f23f68a Fix STS InvalidAccessKeyId and request body consumption issues (#8328)
* Fix STS InvalidAccessKeyId and request body consumption in Lakekeeper integration test

* Remove debug prints

* Add Lakekeeper integration tests to CI

* Fix connection refused in CI by binding to 0.0.0.0

* Add timeout to docker run in Lakekeeper integration test

* Update weed/s3api/auth_credentials.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-12 17:37:07 -08:00
951eeefb76 fix(s3): lifecycle TTL rules inherit replication and volumeGrowthCount from filer config (#8321)
* fix(s3): lifecycle TTL rules inherit replication from parent path and filer config

PutBucketLifecycleConfiguration wrote filer.conf entries with empty replication,
so effective replication could differ from operator default. Now we resolve
replication from parent path rule (MatchStorageRule) then filer global config;
only Replication is set on the rule (no DataCenter/Rack/DataNode for S3).

* add volumeGrowthCount

* review

---------

Co-authored-by: Dmitiy Gushchin <dag@fivegen.ru>
2026-02-12 16:46:05 -08:00
Chris LuandGitHub 25ea48227f Fix STS temporary credentials to use ASIA prefix instead of AKIA (#8326)
Temporary credentials from STS AssumeRole were using "AKIA" prefix
(permanent IAM user credentials) instead of "ASIA" prefix (temporary
security credentials). This violates AWS conventions and may cause
compatibility issues with AWS SDKs that validate credential types.

Changes:
- Rename generateAccessKeyId to generateTemporaryAccessKeyId for clarity
- Update function to use ASIA prefix for temporary credentials
- Add unit tests to verify ASIA prefix format (weed/iam/sts/credential_prefix_test.go)
- Add integration test to verify ASIA prefix in S3 API (test/s3/iam/s3_sts_credential_prefix_test.go)
- Ensure AWS-compatible credential format (ASIA + 16 hex chars)

The credentials are already deterministic (SHA256-based from session ID)
and the SessionToken is correctly set to the JWT token, so this is just
a prefix fix to follow AWS standards.

Fixes #8312
2026-02-12 14:47:20 -08:00
Chris LuandGitHub 0082c47e04 Test: Add RisingWave DML verification test (#8317)
* Test: Verify RisingWave DML operations (INSERT, UPDATE, DELETE) support

* Test: Refine RisingWave DML test (remove sleeps, use polling)
2026-02-12 14:24:04 -08:00
abd681b54b Fix service name in the worker deployment (seaweedfs#8314) (#8315)
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2026-02-12 14:22:42 -08:00
Chris LuandGitHub 4e1065e485 Fix: preserve request body for STS signature verification (#8324)
* Fix: preserve request body for STS signature verification

- Save and restore request body in UnifiedPostHandler after ParseForm()
- This allows STS handler to verify signatures correctly
- Fixes 'invalid AWS signature: 53' error (ErrContentSHA256Mismatch)
- ParseForm() consumes the body, so we need to restore it for downstream handlers

* Improve error handling in UnifiedPostHandler

- Add http.MaxBytesReader to limit body size to 10 MiB (iamRequestBodyLimit)
- Add proper error handling for io.ReadAll failures
- Log errors when body reading fails
- Prevents DoS attacks from oversized request bodies
- Addresses code review feedback
2026-02-12 13:28:12 -08:00
Chris LuandGitHub c1a9263e37 Fix STS AssumeRole with POST body param (#8320)
* Fix STS AssumeRole with POST body param and add integration test

* Add STS integration test to CI workflow

* Address code review feedback: fix HPP vulnerability and style issues

* Refactor: address code review feedback

- Fix HTTP Parameter Pollution vulnerability in UnifiedPostHandler
- Refactor permission check logic for better readability
- Extract test helpers to testutil/docker.go to reduce duplication
- Clean up imports and simplify context setting

* Add SigV4-style test variant for AssumeRole POST body routing

- Added ActionInBodyWithSigV4Style test case to validate real-world scenario
- Test confirms routing works correctly for AWS SigV4-signed requests
- Addresses code review feedback about testing with SigV4 signatures

* Fix: always set identity in context when non-nil

- Ensure UnifiedPostHandler always calls SetIdentityInContext when identity is non-nil
- Only call SetIdentityNameInContext when identity.Name is non-empty
- This ensures downstream handlers (embeddedIam.DoActions) always have access to identity
- Addresses potential issue where empty identity.Name would skip context setting
2026-02-12 12:04:07 -08:00
Chris LuandGitHub 6bd6bba594 Fix inconsistent admin argument in worker pods (#8316)
* Fix inconsistent admin argument in worker pods

* Use seaweedfs.componentName for admin service naming
2026-02-12 09:50:53 -08:00
Chris LuandGitHub b8ef48c8f1 Add RisingWave catalog tests (#8308)
* Add RisingWave catalog tests for S3 tables

* Add RisingWave catalog integration tests to CI workflow

* Refactor RisingWave catalog tests based on PR feedback

* Address PR feedback: optimize checks, cleanup logs

* fix tests

* consistent
2026-02-11 22:00:06 -08:00
Chris LuandGitHub 75faf826d4 Fix LevelDB panic on lazy reload (#8269) (#8307)
* fix LevelDB panic on lazy reload

Implemented a thread-safe reload mechanism using double-checked
locking and a retry loop in Get, Put, and Delete. Added a concurrency
test to verify the fix and prevent regressions.

Fixes #8269

* refactor: use helper for leveldb fix and remove deprecated ioutil

* fix: prevent deadlock by using getFromDb helper

Extracted DB lookup to internal helper to avoid recursive RLock in Put/Delete methods.
Updated Get to use the helper as well.

* fix: resolve syntax error and commit deadlock prevention

Fixed a duplicate function declaration syntax error.
Verified that getFromDb helper correctly prevents recursive RLock scenarios.

* refactor: remove redundant timeout checks

Removed nested `if m.ldbTimeout > 0` checks in Get, Put, and Delete
methods as suggested in PR review.
2026-02-11 14:17:21 -08:00
Lisandro PinandGitHub 221bd237c4 Fix file stat collection metric bug for the cluster.status command. (#8302)
When the `--files` flag is present, `cluster.status` will scrape file metrics
from volume servers to provide detailed stats on those. The progress indicator
was not being updated properly though, so the command would complete before
it read 100%.
2026-02-11 13:34:20 -08:00
Chris LuandGitHub a3136c523f Fix volume.fsck 401 Unauthorized by adding JWT to HTTP delete requests (#8306)
* Fix volume.fsck 401 Unauthorized by adding JWT to HTTP delete requests

* Additionally, for performance, consider fetching the jwt.filer_signing.key once before any loops that call httpDelete, rather than inside httpDelete itself, to avoid repeated configuration lookups.
2026-02-11 13:32:56 -08:00
Chris Lu ac242d04ee one time manual run 2026-02-11 13:20:52 -08:00
Chris Lu 21543134c8 fix manual build process 2026-02-11 13:20:41 -08:00
8b5d31e5eb s3api/policy_engine: use forwarded client IP for aws:SourceIp (#8304)
* s3api: honor forwarded source IP for policy conditions

Prefer X-Forwarded-For/X-Real-Ip before RemoteAddr when populating aws:SourceIp in policy condition evaluation. Also avoid noisy parsing behavior for unix socket markers and add coverage for precedence/fallback paths.\n\nFixes #8301.

* s3api: simplify remote addr parsing

* s3api: guard aws:SourceIp against DNS hosts

* s3api: simplify remote addr fallback

* s3api: simplify remote addr parsing

* Update weed/s3api/policy_engine/engine.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fix TestExtractConditionValuesFromRequestSourceIPPrecedence using trusted private IP

* Refactor extractSourceIP to use R-to-L XFF parsing and net.IP.IsPrivate

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-11 12:47:03 -08:00
Chris Lu 7151181d54 fix flaky tests 2026-02-11 12:23:35 -08:00
Lisandro PinandGitHub e657e7d827 Implement local scrubbing for EC volumes. (#8283) 2026-02-11 11:04:08 -08:00
Lisandro PinandGitHub 2a73219397 Add weed shell command volumeServer.state to query/update volume server state settings. (#8271)
Add weed shell command `volumeServer.state` to query/update volume server states.
2026-02-11 11:02:37 -08:00
Chris LuandGitHub 7fcbffed7f filer.sync: support manifest chunks (#8299)
* filer.sync support manifest chunks

* filersink: address manifest sync review feedback
2026-02-10 20:18:35 -08:00
Chris LuandGitHub be0379f6fd Fix filer.sync retry on stale chunk (#8298)
* Fix filer.sync stale chunk uploads

* Tweak filersink stale logging
2026-02-10 19:06:35 -08:00
Chris LuandGitHub b57429ef2e Switch empty-folder cleanup to bucket policy (#8292)
* Fix Spark _temporary cleanup and add issue #8285 regression test

* Generalize empty folder cleanup for Spark temp artifacts

* Revert synchronous folder pruning and add cleanup diagnostics

* Add actionable empty-folder cleanup diagnostics

* Fix Spark temp marker cleanup in async folder cleaner

* Fix Spark temp cleanup with implicit directory markers

* Keep explicit directory markers non-implicit

* logging

* more logs

* Switch empty-folder cleanup to bucket policy

* Seaweed-X-Amz-Allow-Empty-Folders

* less logs

* go vet

* less logs

* refactoring
2026-02-10 18:38:38 -08:00
Chris LuandGitHub 5c365e7090 s3api: return 400 for invalid namespace query in REST table routes (#8296)
* s3api: reject invalid namespace query in REST table routes

* s3api: expand namespace validation REST tests
2026-02-10 17:57:08 -08:00
Chris LuandGitHub 822dbed552 s3api: fix ListObjectsV2 NextContinuationToken duplication for nested prefix (#8294)
* s3api: fix duplicate ListObjectsV2 continuation token for nested prefix

* s3api: include prefix in common-prefix continuation token
2026-02-10 14:17:41 -08:00
Chris LuandGitHub 2d97685390 ci: fix container_release_unified manual dispatch and workflow parsing (#8293)
ci: fix unified release workflow dispatch matrix filtering
2026-02-10 13:15:52 -08:00
Chris LuandGitHub 1b2f719d7c admin: fix file browser items-per-page selector (#8291)
* admin: fix file browser page size selector

Fix file browser pagination page-size selectors to use explicit select IDs instead of this.value in templ-generated handlers, which could resolve to undefined and produce limit=undefined in requests.

Add a focused template render regression test to prevent this from recurring.

Fixes #8284

* revert file browser template regression test
2026-02-10 12:56:34 -08:00
Chris LuandGitHub b73bd08470 ci: move manual container builds to unified release workflow (#8290)
* ci: move manual dev container build into unified release workflow

* ci: make unified manual container build release-tag based
2026-02-10 12:47:39 -08:00
Chris LuandGitHub 17f85361e9 Remove unsupported iceberg rest signing-region from tests and docs (#8289) 2026-02-10 12:23:31 -08:00
Chris LuandGitHub b261c89675 Fix RocksDB container build compatibility and add manual rocksdb dispatch (#8288)
* Fix RocksDB build compatibility and add manual rocksdb trigger

* Upgrade RocksDB defaults and keep grocksdb v1.10.7

* Add manual latest-image trigger inputs for ref and variant

* Allow manual latest build to set image tag and source ref

* Fix manual variant selection using setup job matrix output
2026-02-10 11:48:42 -08:00
Chris LuGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
0385acba02 s3tables: fix shared table-location bucket mapping collisions (#8286)
* s3tables: prevent shared table-location bucket mapping overwrite

* Update weed/s3api/bucket_paths.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-02-10 11:28:29 -08:00
Chris LuandGitHub d6825ffce2 Iceberg: implement stage-create finalize flow (phase 1) (#8279)
* iceberg: implement stage-create and create-on-commit finalize

* iceberg: add create validation error typing and stage-create integration test

* tests: merge stage-create integration check into catalog suite

* tests: cover stage-create finalize lifecycle in catalog integration

* iceberg: persist and cleanup stage-create markers

* iceberg: add stage-create rollout flag and marker pruning

* docs: add stage-create support design and rollout plan

* docs: drop stage-create design draft from PR

* iceberg: use conservative 72h stage-marker retention

* iceberg: address review comments on create-on-commit and tests

* iceberg: keep stage-create metadata out of table location

* refactor(iceberg): split iceberg.go into focused files
2026-02-10 09:46:09 -08:00
Chris LuandGitHub d88f6ed0af Iceberg commit reliability: preserve statistics updates and return 409 conflicts (#8277)
* iceberg: harden table commit updates and conflict handling

* iceberg: refine commit retry and statistics patching

* iceberg: cleanup metadata on non-conflict commit errors
2026-02-09 23:00:03 -08:00
Chris LuandGitHub 5ae3be44d1 iceberg: persist namespace properties for create/get (#8276)
* iceberg: persist namespace properties via s3tables metadata

* iceberg: simplify namespace properties normalization

* s3tables: broaden namespace properties round-trip test

* adjust logs

* adjust logs
2026-02-09 22:20:45 -08:00
Chris LuGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
1c62808c0e iceberg: wire pagination for list namespaces/tables REST APIs (#8275)
* s3api/iceberg: wire list pagination tokens and page size

* fmt

* Update weed/s3api/iceberg/iceberg.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-02-09 21:46:55 -08:00
Chris Lu db76eb26e7 compile 2026-02-09 21:06:07 -08:00
Chris Lu 4ccc7668ce admin: resolve merge conflicts 2026-02-09 20:56:20 -08:00
Chris LuandGitHub aef2de3109 s3tables: support multi-level namespaces in parser/admin paths (#8273)
* s3tables: support multi-level namespace normalization

* admin: handle namespace parsing errors centrally

* admin: clean namespace validation duplication
2026-02-09 20:20:05 -08:00
Chris Lu be26ce74ce s3tables: support multi-level namespace normalization 2026-02-09 19:42:31 -08:00
Chris Lu 0b80f055c2 Merge branch 'fix/8270-leader-not-elected' 2026-02-09 18:15:59 -08:00
Chris Lu af8273386d 4.12 2026-02-09 18:15:19 -08:00
Chris LuandGitHub ba8e2aaae9 Fix master leader election when grpc ports change (#8272)
* Fix master leader detection when grpc ports change

* Canonicalize self peer entry to avoid raft self-alias panic

* Normalize and deduplicate master peer addresses
2026-02-09 18:13:02 -08:00
Chris Lu 15d0a46679 Normalize and deduplicate master peer addresses 2026-02-09 18:10:54 -08:00
Chris Lu ae27e17e6f Canonicalize self peer entry to avoid raft self-alias panic 2026-02-09 18:07:01 -08:00
Chris Lu 02dac23119 Fix master leader detection when grpc ports change 2026-02-09 18:00:09 -08:00
Lisandro PinandGitHub f400fb44a0 Update cluster.status to resolve file details on EC volumes. (#8268)
Also parallelizes queries for file metrics collections when the `--files`
flag is specified, and improves the command's output for readability:

```
> cluster.status --files
collecting file stats: 100%

cluster:
	id:       topo
	status:   LOCKED
	nodes:    10
	topology: 1 DC, 10 disks on 1 rack

volumes:
	total:    3 volumes, 1 collection
	max size: 32 GB
	regular:  1/80 volume on 3 replicas, 3 writable (100%), 0 read-only (0%)
	EC:       2 EC volumes on 28 shards (14 shards/volume)

storage:
	total:           269 MB (522 MB raw, 193.95%)
	regular volumes: 91 MB (272 MB raw, 300%)
	EC volumes:      178 MB (250 MB raw, 140%)

files:
	total:   363 files, 300 readable (82.64%), 63 deleted (17.35%), avg 522 kB per file
	regular: 168 files, 105 readable (62.5%), 63 deleted (37.5%), avg 540 kB per file
	EC:      195 files, 195 readable (100%), 0 deleted (0%), avg 506 kB per file
```
2026-02-09 17:52:43 -08:00
Chris LuandGitHub 30812b85f3 fix ec.encode skipping volumes when one replica is on a full disk (#8227)
* fix ec.encode skipping volumes when one replica is on a full disk

This fixes issue #8218. Previously, ec.encode would skip a volume if ANY
of its replicas resided on a disk with low free volume count. Now it
accepts the volume if AT LEAST ONE replica is on a healthy disk.

* refine noFreeDisk counter logic in ec.encode

Ensure noFreeDisk is decremented if a volume initially marked as bad
is later found to have a healthy replica. This ensures accurate
summary statistics.

* defer noFreeDisk counting and refine logging in ec.encode

Updated logging to be replica-scoped and deferred noFreeDisk counting to
the final pass over vidMap. This ensures that the counter only reflects
volumes that are definitively excluded because all replicas are on full
disks.

* filter replicas by free space during ec.encode

Updated doEcEncode to filter out replicas on disks with
FreeVolumeCount < 2 before selecting the best replica for encoding.
This ensures that EC shards are not generated on healthy source
replicas that happen to be on disks with low free space.
2026-02-09 14:23:11 -08:00
Chris LuandGitHub 6a61037333 fix issue #8230: volume.fsck deletion logic to respect purgeAbsent flag (#8266)
* fix issue #8230: volume.fsck deletion logic to respect purgeAbsent flag

This commit fixes two issues in volume.fsck:
1. Missing chunks in existing volumes are now deleted if -reallyDeleteFilerEntries is set.
2. Missing volumes are now properly handled when a -volumeId filter is specified, allowing deletion of filer entries for those volumes.

* address PR feedback for issue #8230

- Ensure volume filter is applied before reporting missing volumes
- Fix potential nil-pointer dereferences in httpDelete method
- Use proper error checking throughout httpDelete

* address second round PR feedback for issue #8230

- Use fmt.Fprintf(c.writer, ...) instead of fmt.Printf
- Add missing newline in "deleting path" log message
2026-02-09 13:23:17 -08:00
Chris LuandGitHub 839028b2e0 Fix EC rebuild shard detection (#8265)
Fix EC rebuild shard counting
2026-02-09 12:34:38 -08:00
Lisandro PinandGitHub 1a5679a5eb Implement a VolumeEcStatus() RPC for volume servers. (#8006)
Just like `VolumeStatus()`, this call allows inspecting details for
a given EC volume - including number of files and their total size.
2026-02-09 11:52:08 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
818a1ff8b1 build(deps): bump golang.org/x/crypto from 0.47.0 to 0.48.0 (#8258)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.47.0 to 0.48.0.
- [Commits](https://github.com/golang/crypto/compare/v0.47.0...v0.48.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.48.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-09 11:49:23 -08:00
Chris LuandGitHub 59b02e0cba s3api: fix multipart Complete ETag matching and lower empty-upload log noise (#8264)
s3api: fix multipart part etag validation and reduce empty upload warning noise
2026-02-09 11:45:57 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
d9987669cb build(deps): bump github.com/jhump/protoreflect from 1.17.0 to 1.18.0 (#8261)
Bumps [github.com/jhump/protoreflect](https://github.com/jhump/protoreflect) from 1.17.0 to 1.18.0.
- [Release notes](https://github.com/jhump/protoreflect/releases)
- [Commits](https://github.com/jhump/protoreflect/compare/v1.17.0...v1.18.0)

---
updated-dependencies:
- dependency-name: github.com/jhump/protoreflect
  dependency-version: 1.18.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-09 11:32:55 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
0c16663014 build(deps): bump golang.org/x/sys from 0.40.0 to 0.41.0 (#8260)
Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.40.0 to 0.41.0.
- [Commits](https://github.com/golang/sys/compare/v0.40.0...v0.41.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-version: 0.41.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-09 11:31:01 -08:00
Chris LuandGitHub ccf35459be Explicitly disable signing for public buckets. (#8263) 2026-02-09 11:28:07 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
88b1768814 build(deps): bump go.etcd.io/etcd/client/v3 from 3.6.6 to 3.6.7 (#8257)
Bumps [go.etcd.io/etcd/client/v3](https://github.com/etcd-io/etcd) from 3.6.6 to 3.6.7.
- [Release notes](https://github.com/etcd-io/etcd/releases)
- [Commits](https://github.com/etcd-io/etcd/compare/v3.6.6...v3.6.7)

---
updated-dependencies:
- dependency-name: go.etcd.io/etcd/client/v3
  dependency-version: 3.6.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-09 11:23:12 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
62063de1ca build(deps): bump github.com/linxGnu/grocksdb from 1.10.3 to 1.10.7 (#8259)
Bumps [github.com/linxGnu/grocksdb](https://github.com/linxGnu/grocksdb) from 1.10.3 to 1.10.7.
- [Release notes](https://github.com/linxGnu/grocksdb/releases)
- [Commits](https://github.com/linxGnu/grocksdb/compare/v1.10.3...v1.10.7)

---
updated-dependencies:
- dependency-name: github.com/linxGnu/grocksdb
  dependency-version: 1.10.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-09 11:00:34 -08:00
Lisandro PinandGitHub 63b846b73b Parallelize operations for the volume.scrub and ec.scrub commands (#8247)
Parallelize operations for the `volume.scrub` and `ec.scrub` commands.
2026-02-09 09:07:06 -08:00
Chris LuandGitHub c9428c2c0f Modify AI review comments checklist in PR template
Updated AI code review checklist to include potential additional reviews.
2026-02-09 08:58:35 -08:00
Chris LuandGitHub cb9e21cdc5 Normalize hashicorp raft peer ids (#8253)
* Normalize raft voter ids

* 4.11

* Update raft_hashicorp.go
2026-02-09 07:46:34 -08:00
Chris Lu 2ed5a8f65c add tests 2026-02-09 01:37:56 -08:00
Chris Lu 5a279c4d2f fmt 2026-02-08 21:19:00 -08:00
Chris Lu 0c89185291 4.10 2026-02-08 21:16:58 -08:00
Chris LuGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
458c12fb99 test: add Spark S3 integration regression for issue #8234 (#8249)
* test: add Spark S3 regression integration test for issue 8234

* Update test/s3/spark/setup_test.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-02-08 21:13:31 -08:00
Chris LuandGitHub 5a0204310c Add Iceberg admin UI (#8246)
* Add Iceberg table details view

* Enhance Iceberg catalog browsing UI

* Fix Iceberg UI security and logic issues

- Fix selectSchema() and partitionFieldsFromFullMetadata() to always search for matching IDs instead of checking != 0
- Fix snapshotsFromFullMetadata() to defensive-copy before sorting to prevent mutating caller's slice
- Fix XSS vulnerabilities in s3tables.js: replace innerHTML with textContent/createElement for user-controlled data
- Fix deleteIcebergTable() to redirect to namespace tables list on details page instead of reloading
- Fix data-bs-target in iceberg_namespaces.templ: remove templ.SafeURL for CSS selector
- Add catalogName to delete modal data attributes for proper redirect
- Remove unused hidden inputs from create table form (icebergTableBucketArn, icebergTableNamespace)

* Regenerate templ files for Iceberg UI updates

* Support complex Iceberg type objects in schema

Change Type field from string to json.RawMessage in both IcebergSchemaFieldInfo
and internal icebergSchemaField to properly handle Iceberg spec's complex type
objects (e.g. {"type": "struct", "fields": [...]}). Currently test data
only shows primitive string types, but this change makes the implementation
defensively robust for future complex types by preserving the exact JSON
representation. Add typeToString() helper and update schema extraction
functions to marshal string types as JSON. Update template to convert
json.RawMessage to string for display.

* Regenerate templ files for Type field changes

* templ

* Fix additional Iceberg UI issues from code review

- Fix lazy-load flag that was set before async operation completed, preventing retries
  on error; now sets loaded flag only after successful load and throws error to caller
  for proper error handling and UI updates
- Add zero-time guards for CreatedAt and ModifiedAt fields in table details to avoid
  displaying Go zero-time values; render dash when time is zero
- Add URL path escaping for all catalog/namespace/table names in URLs to prevent
  malformed URLs when names contain special characters like /, ?, or #
- Remove redundant innerHTML clear in loadIcebergNamespaceTables that cleared twice
  before appending the table list
- Fix selectSnapshotForMetrics to remove != 0 guard for consistency with selectSchema
  fix; now always searches for CurrentSnapshotID without zero-value gate
- Enhance typeToString() helper to display '(complex)' for non-primitive JSON types

* Regenerate templ files for Phase 3 updates

* Fix template generation to use correct file paths

Run templ generate from repo root instead of weed/admin directory to ensure
generated _templ.go files have correct absolute paths in error messages
(e.g., 'weed/admin/view/app/iceberg_table_details.templ' instead of
'app/iceberg_table_details.templ'). This ensures both 'make admin-generate'
at repo root and 'make generate' in weed/admin directory produce identical
output with consistent file path references.

* Regenerate template files with correct path references

* Validate S3 Tables names in UI

- Add client-side validation for table bucket and namespace names to surface
  errors for invalid characters (dots/underscores) before submission
- Use HTML validity messages with reportValidity for immediate feedback
- Update namespace helper text to reflect actual constraints (single-level,
  lowercase letters, numbers, and underscores)

* Regenerate templ files for namespace helper text

* Fix Iceberg catalog REST link and actions

* Disallow S3 object access on table buckets

* Validate Iceberg layout for table bucket objects

* Fix REST API link to /v1/config

* merge iceberg page with table bucket page

* Allowed Trino/Iceberg stats files in metadata validation

* fixes

  - Backend/data handling:
      - Normalized Iceberg type display and fallback handling in weed/admin/dash/s3tables_management.go.
      - Fixed snapshot fallback pointer semantics in weed/admin/dash/s3tables_management.go.
      - Added CSRF token generation/propagation/validation for namespace create/delete in:
          - weed/admin/dash/csrf.go
          - weed/admin/dash/auth_middleware.go
          - weed/admin/dash/middleware.go
          - weed/admin/dash/s3tables_management.go
          - weed/admin/view/layout/layout.templ
          - weed/admin/static/js/s3tables.js
  - UI/template fixes:
      - Zero-time guards for CreatedAt fields in:
          - weed/admin/view/app/iceberg_namespaces.templ
          - weed/admin/view/app/iceberg_tables.templ
      - Fixed invalid templ-in-script interpolation and host/port rendering in:
          - weed/admin/view/app/iceberg_catalog.templ
          - weed/admin/view/app/s3tables_buckets.templ
      - Added data-catalog-name consistency on Iceberg delete action in weed/admin/view/app/iceberg_tables.templ.
      - Updated retry wording in weed/admin/static/js/s3tables.js.
      - Regenerated all affected _templ.go files.
  - S3 API/comment follow-ups:
      - Reused cached table-bucket validator in weed/s3api/bucket_paths.go.
      - Added validation-failure debug logging in weed/s3api/s3api_object_handlers_tagging.go.
      - Added multipart path-validation design comment in weed/s3api/s3api_object_handlers_multipart.go.
  - Build tooling:
      - Fixed templ generate working directory issues in weed/admin/Makefile (watch + pattern rule).

* populate data

* test/s3tables: harden populate service checks

* admin: skip table buckets in object-store bucket list

* admin sidebar: move object store to top-level links

* admin iceberg catalog: guard zero times and escape links

* admin forms: add csrf/error handling and client-side name validation

* admin s3tables: fix namespace delete modal redeclaration

* admin: replace native confirm dialogs with modal helpers

* admin modal-alerts: remove noisy confirm usage console log

* reduce logs

* test/s3tables: use partitioned tables in trino and spark populate

* admin file browser: normalize filer ServerAddress for HTTP parsing
2026-02-08 20:06:32 -08:00
Chris Lu bbcb8b7590 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2026-02-08 20:05:24 -08:00
Chris LuandGitHub be6b5db65a s3: fix health check endpoints returning 404 for HEAD requests #8243 (#8248)
* Fix disk errors handling in vacuum compaction

When a disk reports IO errors during vacuum compaction (e.g., 'read /mnt/d1/weed/oc_xyz.dat: input/output error'), the vacuum task should signal the error to the master so it can:
1. Drop the faulty volume replica
2. Rebuild the replica from healthy copies

Changes:
- Add checkReadWriteError() calls in vacuum read paths (ReadNeedleBlob, ReadData, ScanVolumeFile) to flag EIO errors in volume.lastIoError
- Preserve error wrapping using %w format instead of %v so EIO propagates correctly
- The existing heartbeat logic will detect lastIoError and remove the bad volume

Fixes issue #8237

* error

* s3: fix health check endpoints returning 404 for HEAD requests #8243
2026-02-08 19:08:10 -08:00
Chris LuandGitHub 403592bb9f Add Spark Iceberg catalog integration tests and CI support (#8242)
* Add Spark Iceberg catalog integration tests and CI support

Implement comprehensive integration tests for Spark with SeaweedFS Iceberg REST catalog:
- Basic CRUD operations (Create, Read, Update, Delete) on Iceberg tables
- Namespace (database) management
- Data insertion, querying, and deletion
- Time travel capabilities via snapshot versioning
- Compatible with SeaweedFS S3 and Iceberg REST endpoints

Tests mirror the structure of existing Trino integration tests but use Spark's
Python SQL API and PySpark for testing.

Add GitHub Actions CI job for spark-iceberg-catalog-tests in s3-tables-tests.yml
to automatically run Spark integration tests on pull requests.

* fmt

* Fix Spark integration tests - code review feedback

* go mod tidy

* Add go mod tidy step to integration test jobs

Add 'go mod tidy' step before test runs for all integration test jobs:
- s3-tables-tests
- iceberg-catalog-tests
- trino-iceberg-catalog-tests
- spark-iceberg-catalog-tests

This ensures dependencies are clean before running tests.

* Fix remaining Spark operations test issues

Address final code review comments:

Setup & Initialization:
- Add waitForSparkReady() helper function that polls Spark readiness
  with backoff instead of hardcoded 10-second sleep
- Extract setupSparkTestEnv() helper to reduce boilerplate duplication
  between TestSparkCatalogBasicOperations and TestSparkTimeTravel
- Both tests now use helpers for consistent, reliable setup

Assertions & Validation:
- Make setup-critical operations (namespace, table creation, initial
  insert) use t.Fatalf instead of t.Errorf to fail fast
- Validate setupSQL output in TestSparkTimeTravel and fail if not
  'Setup complete'
- Add validation after second INSERT in TestSparkTimeTravel:
  verify row count increased to 2 before time travel test
- Add context to error messages with namespace and tableName params

Code Quality:
- Remove code duplication between test functions
- All critical paths now properly validated
- Consistent error handling throughout

* Fix go vet errors in S3 Tables tests

Fixes:
1. setup_test.go (Spark):
   - Add missing import: github.com/testcontainers/testcontainers-go/wait
   - Use wait.ForLog instead of undefined testcontainers.NewLogStrategy
   - Remove unused strings import

2. trino_catalog_test.go:
   - Use net.JoinHostPort instead of fmt.Sprintf for address formatting
   - Properly handles IPv6 addresses by wrapping them in brackets

* Use weed mini for simpler SeaweedFS startup

Replace complex multi-process startup (master, volume, filer, s3)
with single 'weed mini' command that starts all services together.

Benefits:
- Simpler, more reliable startup
- Single weed mini process vs 4 separate processes
- Automatic coordination between components
- Better port management with no manual coordination

Changes:
- Remove separate master, volume, filer process startup
- Use weed mini with -master.port, -filer.port, -s3.port flags
- Keep Iceberg REST as separate service (still needed)
- Increase timeout to 15s for port readiness (weed mini startup)
- Remove volumePort and filerProcess fields from TestEnvironment
- Simplify cleanup to only handle two processes (mini, iceberg rest)

* Clean up dead code and temp directory leaks

Fixes:

1. Remove dead s3Process field and cleanup:
   - weed mini bundles S3 gateway, no separate process needed
   - Removed s3Process field from TestEnvironment
   - Removed unnecessary s3Process cleanup code

2. Fix temp config directory leak:
   - Add sparkConfigDir field to TestEnvironment
   - Store returned configDir in writeSparkConfig
   - Clean up sparkConfigDir in Cleanup() with os.RemoveAll
   - Prevents accumulation of temp directories in test runs

3. Simplify Cleanup:
   - Now handles only necessary processes (weed mini, iceberg rest)
   - Removes both seaweedfsDataDir and sparkConfigDir
   - Cleaner shutdown sequence

* Use weed mini's built-in Iceberg REST and fix python binary

Changes:
- Add -s3.port.iceberg flag to weed mini for built-in Iceberg REST Catalog
- Remove separate 'weed server' process for Iceberg REST
- Remove icebergRestProcess field from TestEnvironment
- Simplify Cleanup() to only manage weed mini + Spark
- Add port readiness check for iceberg REST from weed mini
- Set Spark container Cmd to '/bin/sh -c sleep 3600' to keep it running
- Change python to python3 in container.Exec calls

This simplifies to truly one all-in-one weed mini process (master, filer, s3,
iceberg-rest) plus just the Spark container.

* go fmt

* clean up

* bind on a non-loopback IP for container access, aligned Iceberg metadata saves/locations with table locations, and reworked Spark time travel to use TIMESTAMP AS OF   with safe timestamp extraction.

* shared mini start

* Fixed internal directory creation under /buckets so .objects paths can auto-create without failing bucket-name validation, which restores table bucket object writes

* fix path

  Updated table bucket objects to write under `/buckets/<bucket>` and saved Iceberg metadata there, adjusting Spark time-travel timestamp to committed_at +1s. Rebuilt the weed binary (`go
  install ./weed`) and confirmed passing tests for Spark and Trino with focused test commands.

* Updated table bucket creation to stop creating /buckets/.objects and switched Trino REST warehouse to s3://<bucket> to match Iceberg layout.

* Stabilize S3Tables integration tests

* Fix timestamp extraction and remove dead code in bucketDir

* Use table bucket as warehouse in s3tables tests

* Update trino_blog_operations_test.go

* adds the CASCADE option to handle any remaining table metadata/files in the schema directory

* skip namespace not empty
2026-02-08 10:03:53 -08:00
Andrii BrataninandGitHub aba42419be Fix tip message in maintenance_workers.templ (#8245) 2026-02-08 03:10:56 -08:00
Chris Lu 3bb9493a5b Enhance Iceberg catalog browsing UI 2026-02-08 00:00:02 -08:00
Chris Lu d9e3fb2b8e Add Iceberg table details view 2026-02-07 23:59:49 -08:00
Chris LuandGitHub 330ba7d9dc Fix disk errors handling in vacuum compaction (#8244)
When a disk reports IO errors during vacuum compaction (e.g., 'read /mnt/d1/weed/oc_xyz.dat: input/output error'), the vacuum task should signal the error to the master so it can:
1. Drop the faulty volume replica
2. Rebuild the replica from healthy copies

Changes:
- Add checkReadWriteError() calls in vacuum read paths (ReadNeedleBlob, ReadData, ScanVolumeFile) to flag EIO errors in volume.lastIoError
- Preserve error wrapping using %w format instead of %v so EIO propagates correctly
- The existing heartbeat logic will detect lastIoError and remove the bad volume

Fixes issue #8237
2026-02-07 21:33:02 -08:00
Chris LuGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
e6ee293c17 Add table operations test (#8241)
* Add Trino blog operations test

* Update test/s3tables/catalog_trino/trino_blog_operations_test.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* feat: add table bucket path helpers and filer operations

- Add table object root and table location mapping directories
- Implement ensureDirectory, upsertFile, deleteEntryIfExists helpers
- Support table location bucket mapping for S3 access

* feat: manage table bucket object roots on creation/deletion

- Create .objects directory for table buckets on creation
- Clean up table object bucket paths on deletion
- Enable S3 operations on table bucket object roots

* feat: add table location mapping for Iceberg REST

- Track table location bucket mappings when tables are created/updated/deleted
- Enable location-based routing for S3 operations on table data

* feat: route S3 operations to table bucket object roots

- Route table-s3 bucket names to mapped table paths
- Route table buckets to object root directories
- Support table location bucket mapping lookup

* feat: emit table-s3 locations from Iceberg REST

- Generate unique table-s3 bucket names with UUID suffix
- Store table metadata under table bucket paths
- Return table-s3 locations for Trino compatibility

* fix: handle missing directories in S3 list operations

- Propagate ErrNotFound from ListEntries for non-existent directories
- Treat missing directories as empty results for list operations
- Fixes Trino non-empty location checks on table creation

* test: improve Trino CSV parsing for single-value results

- Sanitize Trino output to skip jline warnings
- Handle single-value CSV results without header rows
- Strip quotes from numeric values in tests

* refactor: use bucket path helpers throughout S3 API

- Replace direct bucket path operations with helper functions
- Leverage centralized table bucket routing logic
- Improve maintainability with consistent path resolution

* fix: add table bucket cache and improve filer error handling

- Cache table bucket lookups to reduce filer overhead on repeated checks
- Use filer_pb.CreateEntry and filer_pb.UpdateEntry helpers to check resp.Error
- Fix delete order in handler_bucket_get_list_delete: delete table object before directory
- Make location mapping errors best-effort: log and continue, don't fail API
- Update table location mappings to delete stale prior bucket mappings on update
- Add 1-second sleep before timestamp time travel query to ensure timestamps are in past
- Fix CSV parsing: examine all lines, not skip first; handle single-value rows

* fix: properly handle stale metadata location mapping cleanup

- Capture oldMetadataLocation before mutation in handleUpdateTable
- Update updateTableLocationMapping to accept both old and new locations
- Use passed-in oldMetadataLocation to detect location changes
- Delete stale mapping only when location actually changes
- Pass empty string for oldLocation in handleCreateTable (new tables have no prior mapping)
- Improve logging to show old -> new location transitions

* refactor: cleanup imports and cache design

- Remove unused 'sync' import from bucket_paths.go
- Use filer_pb.UpdateEntry helper in setExtendedAttribute and deleteExtendedAttribute for consistent error handling
- Add dedicated tableBucketCache map[string]bool to BucketRegistry instead of mixing concerns with metadataCache
- Improve cache separation: table buckets cache is now separate from bucket metadata cache

* fix: improve cache invalidation and add transient error handling

Cache invalidation (critical fix):
- Add tableLocationCache to BucketRegistry for location mapping lookups
- Clear tableBucketCache and tableLocationCache in RemoveBucketMetadata
- Prevents stale cache entries when buckets are deleted/recreated

Transient error handling:
- Only cache table bucket lookups when conclusive (found or ErrNotFound)
- Skip caching on transient errors (network, permission, etc)
- Prevents marking real table buckets as non-table due to transient failures

Performance optimization:
- Cache tableLocationDir results to avoid repeated filer RPCs on hot paths
- tableLocationDir now checks cache before making expensive filer lookups
- Cache stores empty string for 'not found' to avoid redundant lookups

Code clarity:
- Add comment to deleteDirectory explaining DeleteEntry response lacks Error field

* go fmt

* fix: mirror transient error handling in tableLocationDir and optimize bucketDir

Transient error handling:
- tableLocationDir now only caches definitive results
- Mirrors isTableBucket behavior to prevent treating transient errors as permanent misses
- Improves reliability on flaky systems or during recovery

Performance optimization:
- bucketDir avoids redundant isTableBucket call via bucketRoot
- Directly use s3a.option.BucketsPath for regular buckets
- Saves one cache lookup for every non-table bucket operation

* fix: revert bucketDir optimization to preserve bucketRoot logic

The optimization to directly use BucketsPath bypassed bucketRoot's logic
and caused issues with S3 list operations on delimiter+prefix cases.

Revert to using path.Join(s3a.bucketRoot(bucket), bucket) which properly
handles all bucket types and ensures consistent path resolution across
the codebase.

The slight performance cost of an extra cache lookup is worth the correctness
and consistency benefits.

* feat: move table buckets under /buckets

Add a table-bucket marker attribute, reuse bucket metadata cache for table bucket detection, and update list/validation/UI/test paths to treat table buckets as /buckets entries.

* Fix S3 Tables code review issues

- handler_bucket_create.go: Fix bucket existence check to properly validate
  entryResp.Entry before setting s3BucketExists flag (nil Entry should not
  indicate existing bucket)
- bucket_paths.go: Add clarifying comment to bucketRoot() explaining unified
  buckets root path for all bucket types
- file_browser_data.go: Optimize by extracting table bucket check early to
  avoid redundant WithFilerClient call

* Fix list prefix delimiter handling

* Handle list errors conservatively

* Fix Trino FOR TIMESTAMP query - use past timestamp

Iceberg requires the timestamp to be strictly in the past.
Use current_timestamp - interval '1' second instead of current_timestamp.

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-02-07 13:27:47 -08:00
Chris LuandGitHub c284e51d20 fix: multipart upload ETag calculation (#8238)
* fix multipart etag

* address comments

* clean up

* clean up

* optimization

* address comments

* unquoted etag

* dedup

* upgrade

* clean

* etag

* return quoted tag

* quoted etag

* debug

* s3api: unify ETag retrieval and quoting across handlers

Refactor newListEntry to take *S3ApiServer and use getObjectETag,
and update setResponseHeaders to use the same logic. This ensures
consistent ETags are returned for both listing and direct access.

* s3api: implement ListObjects deduplication for versioned buckets

Handle duplicate entries between the main path and the .versions
directory by prioritizing the latest version when bucket versioning
is enabled.

* s3api: cleanup stale main file entries during versioned uploads

Add explicit deletion of pre-existing "main" files when creating new
versions in versioned buckets. This prevents stale entries from
appearing in bucket listings and ensures consistency.

* s3api: fix cleanup code placement in versioned uploads

Correct the placement of rm calls in completeMultipartUpload and
putVersionedObject to ensure stale main files are properly deleted
during versioned uploads.

* s3api: improve getObjectETag fallback for empty ExtETagKey

Ensure that when ExtETagKey exists but contains an empty value,
the function falls through to MD5/chunk-based calculation instead
of returning an empty string.

* s3api: fix test files for new newListEntry signature

Update test files to use the new newListEntry signature where the
first parameter is *S3ApiServer. Created mockS3ApiServer to properly
test owner display name lookup functionality.

* s3api: use filer.ETag for consistent Md5 handling in getEtagFromEntry

Change getEtagFromEntry fallback to use filer.ETag(entry) instead of
filer.ETagChunks to ensure legacy entries with Attributes.Md5 are
handled consistently with the rest of the codebase.

* s3api: optimize list logic and fix conditional header logging

- Hoist bucket versioning check out of per-entry callback to avoid
  repeated getVersioningState calls
- Extract appendOrDedup helper function to eliminate duplicate
  dedup/append logic across multiple code paths
- Change If-Match mismatch logging from glog.Errorf to glog.V(3).Infof
  and remove DEBUG prefix for consistency

* s3api: fix test mock to properly initialize IAM accounts

Fixed nil pointer dereference in TestNewListEntryOwnerDisplayName by
directly initializing the IdentityAccessManagement.accounts map in the
test setup. This ensures newListEntry can properly look up account
display names without panicking.

* cleanup

* s3api: remove premature main file cleanup in versioned uploads

Removed incorrect cleanup logic that was deleting main files during
versioned uploads. This was causing test failures because it deleted
objects that should have been preserved as null versions when
versioning was first enabled. The deduplication logic in listing is
sufficient to handle duplicate entries without deleting files during
upload.

* s3api: add empty-value guard to getEtagFromEntry

Added the same empty-value guard used in getObjectETag to prevent
returning quoted empty strings. When ExtETagKey exists but is empty,
the function now falls through to filer.ETag calculation instead of
returning "".

* s3api: fix listing of directory key objects with matching prefix

Revert prefix handling logic to use strings.TrimPrefix instead of
checking HasPrefix with empty string result. This ensures that when a
directory key object exactly matches the prefix (e.g. prefix="dir/",
object="dir/"), it is correctly handled as a regular entry instead of
being skipped or incorrectly processed as a common prefix. Also fixed
missing variable definition.

* s3api: refactor list inline dedup to use appendOrDedup helper

Refactored the inline deduplication logic in listFilerEntries to use the
shared appendOrDedup helper function. This ensures consistent behavior
and reduces code duplication.

* test: fix port allocation race in s3tables integration test

Updated startMiniCluster to find all required ports simultaneously using
findAvailablePorts instead of sequentially. This prevents race conditions
where the OS reallocates a port that was just released, causing multiple
services (e.g. Filer and Volume) to be assigned the same port and fail
to start.
2026-02-06 21:54:43 -08:00
Feng ShaoandGitHub 963398ac8c use ReadFull (#40) (#8240)
* use ReadFull

* fix error checking
2026-02-06 20:51:47 -08:00
Chris LuandGitHub 2163570d16 test: add CRUD tests for S3 Tables Catalog Trino integration (#8236)
* test: add comprehensive CRUD tests for S3 Tables Catalog Trino integration

- Add TestNamespaceCRUD: Tests complete Create-Read-Update-Delete lifecycle for namespaces
- Add TestNamespaceListingPagination: Tests listing multiple namespaces with verification
- Add TestNamespaceErrorHandling: Tests error handling for edge cases (IF EXISTS, IF NOT EXISTS)
- Add TestSchemaIntegrationWithCatalog: Tests integration between Trino SQL and Iceberg REST Catalog

All tests pass successfully and use Trino SQL interface for practical integration testing.
Tests properly skip when Docker is unavailable.
Use randomized namespace names to avoid conflicts in parallel execution.

The tests provide comprehensive coverage of namespace/schema CRUD operations which form the
foundation of the Iceberg catalog integration with Trino.

* test: Address code review feedback for S3 Tables Catalog Trino CRUD tests

- Extract common test setup into setupTrinoTest() helper function
- Replace all fmt.Printf calls with idiomatic t.Logf
- Change namespace deletion verification from t.Logf to t.Errorf for proper test failures
- Enhance TestNamespaceErrorHandling with persistence verification test
- Remove unnecessary fmt import
- Improve test documentation with clarifying comments

* test: Fix schema naming and remove verbose output logging

- Fix TestNamespaceListingPagination schema name generation: use fmt.Sprintf instead of string(rune())
- Remove verbose logging of SHOW SCHEMAS output to reduce noise in test logs
- Keep high-level operation logging while removing detailed result output
2026-02-06 14:30:40 -08:00
Chris Lu 217d977579 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2026-02-06 13:14:30 -08:00
Chris LuandGitHub a3b83f8808 test: add Trino Iceberg catalog integration test (#8228)
* test: add Trino Iceberg catalog integration test

- Create test/s3/catalog_trino/trino_catalog_test.go with TestTrinoIcebergCatalog
- Tests integration between Trino SQL engine and SeaweedFS Iceberg REST catalog
- Starts weed mini with all services and Trino in Docker container
- Validates Iceberg catalog schema creation and listing operations
- Uses native S3 filesystem support in Trino with path-style access
- Add workflow job to s3-tables-tests.yml for CI execution

* fix: preserve AWS environment credentials when replacing S3 configuration

When S3 configuration is loaded from filer/db, it replaces the identities list
and inadvertently removes AWS_ACCESS_KEY_ID credentials that were added from
environment variables. This caused auth to remain disabled even though valid
credentials were present.

Fix by preserving environment-based identities when replacing the configuration
and re-adding them after the replacement. This ensures environment credentials
persist across configuration reloads and properly enable authentication.

* fix: use correct ServerAddress format with gRPC port encoding

The admin server couldn't connect to master because the master address
was missing the gRPC port information. Use pb.NewServerAddress() which
properly encodes both HTTP and gRPC ports in the address string.

Changes:
- weed/command/mini.go: Use pb.NewServerAddress for master address in admin
- test/s3/policy/policy_test.go: Store and use gRPC ports for master/filer addresses

This fix applies to:
1. Admin server connection to master (mini.go)
2. Test shell commands that need master/filer addresses (policy_test.go)

* move

* move

* fix: always include gRPC port in server address encoding

The NewServerAddress() function was omitting the gRPC port from the address
string when it matched the port+10000 convention. However, gRPC port allocation
doesn't always follow this convention - when the calculated port is busy, an
alternative port is allocated.

This caused a bug where:
1. Master's gRPC port was allocated as 50661 (sequential, not port+10000)
2. Address was encoded as '192.168.1.66:50660' (gRPC port omitted)
3. Admin client called ToGrpcAddress() which assumed port+10000 offset
4. Admin tried to connect to 60660 but master was on 50661 → connection failed

Fix: Always include explicit gRPC port in address format (host:httpPort.grpcPort)
unless gRPC port is 0. This makes addresses unambiguous and works regardless of
the port allocation strategy used.

Impacts: All server-to-server gRPC connections now use properly formatted addresses.

* test: fix Iceberg REST API readiness check

The Iceberg REST API endpoints require authentication. When checked without
credentials, the API returns 403 Forbidden (not 401 Unauthorized).  The
readiness check now accepts both auth error codes (401/403) as indicators
that the service is up and ready, it just needs credentials.

This fixes the 'Iceberg REST API did not become ready' test failure.

* Fix AWS SigV4 signature verification for base64-encoded payload hashes

   AWS SigV4 canonical requests must use hex-encoded SHA256 hashes,
   but the X-Amz-Content-Sha256 header may be transmitted as base64.

   Changes:
   - Added normalizePayloadHash() function to convert base64 to hex
   - Call normalizePayloadHash() in extractV4AuthInfoFromHeader()
   - Added encoding/base64 import

   Fixes 403 Forbidden errors on POST requests to Iceberg REST API
   when clients send base64-encoded content hashes in the header.

   Impacted services: Iceberg REST API, S3Tables

* Fix AWS SigV4 signature verification for base64-encoded payload hashes

   AWS SigV4 canonical requests must use hex-encoded SHA256 hashes,
   but the X-Amz-Content-Sha256 header may be transmitted as base64.

   Changes:
   - Added normalizePayloadHash() function to convert base64 to hex
   - Call normalizePayloadHash() in extractV4AuthInfoFromHeader()
   - Added encoding/base64 import
   - Removed unused fmt import

   Fixes 403 Forbidden errors on POST requests to Iceberg REST API
   when clients send base64-encoded content hashes in the header.

   Impacted services: Iceberg REST API, S3Tables

* pass sigv4

* s3api: fix identity preservation and logging levels

- Ensure environment-based identities are preserved during config replacement
- Update accessKeyIdent and nameToIdentity maps correctly
- Downgrade informational logs to V(2) to reduce noise

* test: fix trino integration test and s3 policy test

- Pin Trino image version to 479
- Fix port binding to 0.0.0.0 for Docker connectivity
- Fix S3 policy test hang by correctly assigning MiniClusterCtx
- Improve port finding robustness in policy tests

* ci: pre-pull trino image to avoid timeouts

- Pull trinodb/trino:479 after Docker setup
- Ensure image is ready before integration tests start

* iceberg: remove unused checkAuth and improve logging

- Remove unused checkAuth method
- Downgrade informational logs to V(2)
- Ensure loggingMiddleware uses a status writer for accurate reported codes
- Narrow catch-all route to avoid interfering with other subsystems

* iceberg: fix build failure by removing unused s3api import

* Update iceberg.go

* use warehouse

* Update trino_catalog_test.go
2026-02-06 13:12:25 -08:00
Chris LuGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
a04e8dd00b Support Linux file/dir ACL in weed mount (#8233)
* Support Linux file/dir ACL in weed mount #8229

* Update weed/command/mount_std.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-02-06 11:33:36 -08:00
Lisandro PinandGitHub 2cda4289f4 Add a version token on RPCs to read/update volume server states. (#8191)
* Add a version token on `GetState()`/`SetState()` RPCs for volume server states.

* Make state version a property ov `VolumeServerState` instead of an in-memory counter.

Also extend state atomicity to reads, instead of just writes.
2026-02-06 10:58:43 -08:00
NikitaandGitHub c44716f9af helm: add a trafficDistribution field to an s3 service (#8232)
helm: add trafficDistribution field to s3 service

Signed-off-by: nbykov0 <166552198+nbykov0@users.noreply.github.com>
2026-02-06 10:47:39 -08:00
Chris Lu 833bcde9f3 test: add Trino Iceberg catalog integration test
- Create test/s3/catalog_trino/trino_catalog_test.go with TestTrinoIcebergCatalog
- Tests integration between Trino SQL engine and SeaweedFS Iceberg REST catalog
- Starts weed mini with all services and Trino in Docker container
- Validates Iceberg catalog schema creation and listing operations
- Uses native S3 filesystem support in Trino with path-style access
- Add workflow job to s3-tables-tests.yml for CI execution
2026-02-05 16:10:31 -08:00
Lisandro PinandGitHub 9d751a7b61 Contrib/volume scrub local (#8226) 2026-02-05 14:44:12 -08:00
Chris LuandGitHub 3306abae10 shell: add minCacheAge flag to remote.uncache command (#8225)
* add minCacheAge flag to remote.uncache command #8221

* address code review feedback: add nil check and improve test isolation

* address code review feedback: use consistent timestamp in FileFilter
2026-02-05 12:57:27 -08:00
Lisandro PinandGitHub f84b70c362 Implement index (fast) scrubbing for regular/EC volumes. (#8207)
Implement index (fast) scrubbing for regular/EC volumes via `ScrubVolume()`/`ScrubEcVolume()`.

Also rearranges existing index test files for reuse across unit tests for different modules.
2026-02-05 11:27:03 -08:00
Chris LuandGitHub 82d9d8687b Fix concurrent map access in EC shards info (#8222)
* fix concurrent map access in EC shards info #8219

* refactor: simplify Disk.ToDiskInfo to use ecShards snapshot and avoid redundant locking

* refactor: improve GetEcShards with pre-allocation and defer
2026-02-05 10:24:18 -08:00
Chris Lu e39a4c2041 fix flaky test 2026-02-04 23:16:31 -08:00
Chris LuandGitHub c2bfd7b524 fix: honor SSE-C chunk offsets in decryption for large chunked uploads (#8216)
* fix: honor SSE-C chunk offsets in decryption for large chunked uploads

Fixes issue #8215 where SSE-C decryption for large objects could corrupt
data by ignoring per-chunk PartOffset values.

Changes:
- Add TestSSECLargeObjectChunkReassembly unit test to verify correct
  decryption of 19MB object split into 8MB chunks using PartOffset
- Update decryptSSECChunkView and createMultipartSSECDecryptedReaderDirect
  to extract PartOffset from SSE-C metadata and pass to
  CreateSSECDecryptedReaderWithOffset for offset-aware decryption
- Fix createCTRStreamWithOffset to use calculateIVWithOffset for proper
  block-aligned counter advancement, matching SSE-KMS/S3 behavior
- Update comments to clarify SSE-C IV handling uses per-chunk offsets
  (unlike base IV approach used by KMS/S3)

All tests pass: go test ./weed/s3api ✓

* fix: close chunkReader on error paths in createMultipartSSECDecryptedReader

Address resource leak issue reported in PR #8216: ensure chunkReader is
properly closed before returning on all error paths, including:
- DeserializeSSECMetadata failures
- IV decoding errors
- Invalid PartOffset values
- SSE-C reader creation failures
- Missing per-chunk metadata

This prevents leaking network connections and file handles during
SSE-C multipart decryption error scenarios.

* docs: clarify SSE-C IV handling in decryptSSECChunkView comment

Replace misleading warning 'Do NOT call calculateIVWithOffset' with
accurate explanation that:
- CreateSSECDecryptedReaderWithOffset internally uses calculateIVWithOffset
  to advance the CTR counter to reach PartOffset
- calculateIVWithOffset is applied only to the per-part IV, NOT to derive
  a global base IV for all parts
- This differs fundamentally from SSE-KMS/SSE-S3 which use base IV +
  calculateIVWithOffset(ChunkOffset)

This clarifies the IV advancement mechanism while contrasting it with
the base IV approach used by other encryption schemes.
2026-02-04 22:57:41 -08:00
Chris LuGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
19c18d827a admin: fix capacity leak in maintenance system by preserving Task IDs (#8214)
* admin: fix capacity leak in maintenance system by preserving Task IDs

Preserve the original TaskID generated during detection and sync task
states (Assign/Complete/Retry) with ActiveTopology. This ensures that
capacity reserved during task assignment is properly released when a
task completes or fails, preventing 'need 9, have 0' capacity exhaustion.

Fixes https://github.com/seaweedfs/seaweedfs/issues/8202

* Update weed/admin/maintenance/maintenance_queue.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update weed/admin/maintenance/maintenance_queue.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* test: rename ActiveTopologySync to TaskIDPreservation

Rename the test case to more accurately reflect its scope, as suggested
by the code review bot.

* Add TestMaintenanceQueue_ActiveTopologySync to verify task state synchronization and capacity management

* Implement task assignment rollback and add verification test

* Enhance ActiveTopology.CompleteTask to support pending tasks

* Populate storage impact in MaintenanceIntegration.SyncTask

* Release capacity in RemoveStaleWorkers when worker becomes unavailable

* Release capacity in MaintenanceManager.CancelTask when pending task is cancelled

* Sync reloaded tasks with ActiveTopology in LoadTasksFromPersistence

* Add verification tests for consistent capacity management lifecycle

* Add TestMaintenanceQueue_RetryCapacitySync to verify capacity tracking during retries

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-02-04 20:39:34 -08:00
Lisandro PinandGitHub 2ecbae3611 Add volume.scrub and ec.scrub shell commands to scrub regular & EC volumes on demand. (#8188)
* Implement RPC skeleton for regular/EC volumes scrubbing.

See https://github.com/seaweedfs/seaweedfs/issues/8018 for details.

* Add `volume.scrub` and `ec.scrub` shell commands to scrub regular & EC volumes on demand.

F.ex:

```
> ec.scrub --full
Scrubbing 10.200.17.13:9005 (1/10)...
Scrubbing 10.200.17.13:9001 (2/10)...
Scrubbing 10.200.17.13:9008 (3/10)...
Scrubbing 10.200.17.13:9009 (4/10)...
Scrubbing 10.200.17.13:9004 (5/10)...
Scrubbing 10.200.17.13:9010 (6/10)...
Scrubbing 10.200.17.13:9007 (7/10)...
Scrubbing 10.200.17.13:9002 (8/10)...
Scrubbing 10.200.17.13:9003 (9/10)...
Scrubbing 10.200.17.13:9006 (10/10)...
Scrubbed 20 EC files and 20 volumes on 10 nodes

Got scrub failures on 1 EC volumes and 2 EC shards :(
Affected volumes: 10.200.17.13:9005:1
Details:
	[10.200.17.13:9005] expected 551041 bytes for needle 6, got 551072
	[10.200.17.13:9005] needles in volume file (1) don't match index entries (173) for volume 1
```
2026-02-04 17:08:31 -08:00
Chris LuandGitHub 7831257ed5 s3: allow single Statement object in policy document (#8212)
* s3: allow single Statement object in policy document

Fixes #8201

* s3: add unit test for single Statement object in policy

* s3: improve error message for malformed PolicyDocument.Statement

* s3: simplify error message for malformed PolicyDocument.Statement
2026-02-04 16:23:20 -08:00
Chris LuandGitHub c9c46db77e s3api: fix ListObjectVersions inconsistency with delimiters (#8210)
* s3api: fix ListObjectVersions inconsistency with delimiters (fixes #8206)

Prioritize handling of .versions and .uploads directories before
delimiter processing in collectVersions. This ensures .versions
directories are processed as version containers instead of being
incorrectly rolled up into CommonPrefixes when a delimiter is used.

* s3api: refactor processDirectory to remove redundant special directory checks

These checks are now handled in the main collectVersions loop.
2026-02-04 14:33:46 -08:00
Chris Lu 27d5f44415 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2026-02-04 12:44:54 -08:00
Chris Lu 000e2bd4a9 logging and debugging 2026-02-04 12:44:52 -08:00
ef3b5f7efa helm/add iceberg rest catalog ingress for s3 (#8205)
* helm: add Iceberg REST catalog support to S3 service

* helm: add Iceberg REST catalog support to S3 service

* add ingress for iceberg catalog endpoint

* helm: conditionally render ingressClassName in s3-iceberg-ingress.yaml

* helm: refactor s3-iceberg-ingress.yaml to use named template for paths

* helm: remove unused $serviceName variable in s3-iceberg-ingress.yaml

---------

Co-authored-by: yalin.sahin <yalin.sahin@tradition.ch>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
2026-02-04 12:00:59 -08:00
Chris LuandGitHub bd4e7ff14e command: fix s3 panic in filer command (#8208)
command: fix s3 panic in filer command due to uninitialized options

This fixes a nil pointer dereference panic when starting the S3 server
via the `weed filer` command by correctly initializing `iamReadOnly`
and `portIceberg` flags.

Relates to #8200
2026-02-04 10:37:20 -08:00
Chris LuandGitHub 72a8f598f2 Fix Maintenance Task Sorting and Refactor Log Persistence (#8199)
* fix float stepping

* do not auto refresh

* only logs when non 200 status

* fix maintenance task sorting and cleanup redundant handler logic

* Refactor log retrieval to persist to disk and fix slowness

- Move log retrieval to disk-based persistence in GetMaintenanceTaskDetail
- Implement background log fetching on task completion in worker_grpc_server.go
- Implement async background refresh for in-progress tasks
- Completely remove blocking gRPC calls from the UI path to fix 10s timeouts
- Cleanup debug logs and performance profiling code

* Ensure consistent deterministic sorting in config_persistence cleanup

* Replace magic numbers with constants and remove debug logs

- Added descriptive constants for truncation limits and timeouts in admin_server.go and worker_grpc_server.go
- Replaced magic numbers with these constants throughout the codebase
- Verified removal of stdout debug printing
- Ensured consistent truncation logic during log persistence

* Address code review feedback on history truncation and logging logic

- Fix AssignmentHistory double-serialization by copying task in GetMaintenanceTaskDetail
- Fix handleTaskCompletion logging logic (mutually exclusive success/failure logs)
- Remove unused Timeout field from LogRequestContext and sync select timeouts with constants
- Ensure AssignmentHistory is only provided in the top-level field for better JSON structure

* Implement goroutine leak protection and request deduplication

- Add request deduplication in RequestTaskLogs to prevent multiple concurrent fetches for the same task
- Implement safe cleanup in timeout handlers to avoid race conditions in pendingLogRequests map
- Add a 10s cooldown for background log refreshes in GetMaintenanceTaskDetail to prevent spamming
- Ensure all persistent log-fetching goroutines are bounded and efficiently managed

* Fix potential nil pointer panics in maintenance handlers

- Add nil checks for adminServer in ShowTaskDetail, ShowMaintenanceWorkers, and UpdateTaskConfig
- Update getMaintenanceQueueData to return a descriptive error instead of nil when adminServer is uninitialized
- Ensure internal helper methods consistently check for adminServer initialization before use

* Strictly enforce disk-only log reading

- Remove background log fetching from GetMaintenanceTaskDetail to prevent timeouts and network calls during page view
- Remove unused lastLogFetch tracking fields to clean up dead code
- Ensure logs are only updated upon task completion via handleTaskCompletion

* Refactor GetWorkerLogs to read from disk

- Update /api/maintenance/workers/:id/logs endpoint to use configPersistence.LoadTaskExecutionLogs
- Remove synchronous gRPC call RequestTaskLogs to prevent timeouts and bad gateway errors
- Ensure consistent log retrieval behavior across the application (disk-only)

* Fix timestamp parsing in log viewer

- Update task_detail.templ JS to handle both ISO 8601 strings and Unix timestamps
- Fix "Invalid time value" error when displaying logs fetched from disk
- Regenerate templates

* master: fallback to HDD if SSD volumes are full in Assign

* worker: improve EC detection logging and fix skip counters

* worker: add Sync method to TaskLogger interface

* worker: implement Sync and ensure logs are flushed before task completion

* admin: improve task log retrieval with retries and better timeouts

* admin: robust timestamp parsing in task detail view
2026-02-04 08:48:55 -08:00
Chris Lu 2ff1cd9fc9 format 2026-02-03 18:39:01 -08:00
Chris Lu 5a5cc38692 4.09 2026-02-03 17:56:25 -08:00
Chris LuandGitHub f66a23b472 Fix: filer not yet available in s3.configure (#8198)
* Fix: Initialize filer CredentialManager with filer address

* The fix involves checking for directory existence before creation.

* adjust error message

* Fix: Implement FilerAddressSetter in PropagatingCredentialStore

* Refactor: Reorder credential manager initialization in filer server

* refactor
2026-02-03 17:43:58 -08:00
Chris LuandGitHub b244bb58aa s3tables: redesign Iceberg REST Catalog using iceberg-go and automate integration tests (#8197)
* full integration with iceberg-go

* Table Commit Operations (handleUpdateTable)

* s3tables: fix Iceberg v2 compliance and namespace properties

This commit ensures SeaweedFS Iceberg REST Catalog is compliant with
Iceberg Format Version 2 by:
- Using iceberg-go's table.NewMetadataWithUUID for strict v2 compliance.
- Explicitly initializing namespace properties to empty maps.
- Removing omitempty from required Iceberg response fields.
- Fixing CommitTableRequest unmarshaling using table.Requirements and table.Updates.

* s3tables: automate Iceberg integration tests

- Added Makefile for local test execution and cluster management.
- Added docker-compose for PyIceberg compatibility kit.
- Added Go integration test harness for PyIceberg.
- Updated GitHub CI to run Iceberg catalog tests automatically.

* s3tables: update PyIceberg test suite for compatibility

- Updated test_rest_catalog.py to use latest PyIceberg transaction APIs.
- Updated Dockerfile to include pyarrow and pandas dependencies.
- Improved namespace and table handling in integration tests.

* s3tables: address review feedback on Iceberg Catalog

- Implemented robust metadata version parsing and incrementing.
- Ensured table metadata changes are persisted during commit (handleUpdateTable).
- Standardized namespace property initialization for consistency.
- Fixed unused variable and incorrect struct field build errors.

* s3tables: finalize Iceberg REST Catalog and optimize tests

- Implemented robust metadata versioning and persistence.
- Standardized namespace property initialization.
- Optimized integration tests using pre-built Docker image.
- Added strict property persistence validation to test suite.
- Fixed build errors from previous partial updates.

* Address PR review: fix Table UUID stability, implement S3Tables UpdateTable, and support full metadata persistence individually

* fix: Iceberg catalog stable UUIDs, metadata persistence, and file writing

- Ensure table UUIDs are stable (do not regenerate on load).
- Persist full table metadata (Iceberg JSON) in s3tables extended attributes.
- Add `MetadataVersion` to explicitly track version numbers, replacing regex parsing.
- Implement `saveMetadataFile` to persist metadata JSON files to the Filer on commit.
- Update `CreateTable` and `UpdateTable` handlers to use the new logic.

* test: bind weed mini to 0.0.0.0 in integration tests to fix Docker connectivity

* Iceberg: fix metadata handling in REST catalog

- Add nil guard in createTable
- Fix updateTable to correctly load existing metadata from storage
- Ensure full metadata persistence on updates
- Populate loadTable result with parsed metadata

* S3Tables: add auth checks and fix response fields in UpdateTable

- Add CheckPermissionWithContext to UpdateTable handler
- Include TableARN and MetadataLocation in UpdateTable response
- Use ErrCodeConflict (409) for version token mismatches

* Tests: improve Iceberg catalog test infrastructure and cleanup

- Makefile: use PID file for precise process killing
- test_rest_catalog.py: remove unused variables and fix f-strings

* Iceberg: fix variable shadowing in UpdateTable

- Rename inner loop variable `req` to `requirement` to avoid shadowing outer request variable

* S3Tables: simplify MetadataVersion initialization

- Use `max(req.MetadataVersion, 1)` instead of anonymous function

* Tests: remove unicode characters from S3 tables integration test logs

- Remove unicode checkmarks from test output for cleaner logs

* Iceberg: improve metadata persistence robustness

- Fix MetadataLocation in LoadTableResult to fallback to generated location
- Improve saveMetadataFile to ensure directory hierarchy existence and robust error handling
2026-02-03 15:30:04 -08:00
47fc9e771f helm: add Iceberg REST catalog support to S3 service (#8193)
* helm: add Iceberg REST catalog support to S3 service

* helm: add Iceberg REST catalog support to S3 service

---------

Co-authored-by: yalin.sahin <yalin.sahin@tradition.ch>
2026-02-03 13:44:52 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
f01876e051 build(deps): bump bytes from 1.10.1 to 1.11.1 in /seaweedfs-rdma-sidecar/rdma-engine (#8195)
build(deps): bump bytes in /seaweedfs-rdma-sidecar/rdma-engine

Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.10.1 to 1.11.1.
- [Release notes](https://github.com/tokio-rs/bytes/releases)
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/bytes/compare/v1.10.1...v1.11.1)

---
updated-dependencies:
- dependency-name: bytes
  dependency-version: 1.11.1
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-03 13:40:42 -08:00
Chris LuandGitHub 1274cf038c s3: enforce authentication and JSON error format for Iceberg REST Catalog (#8192)
* s3: enforce authentication and JSON error format for Iceberg REST Catalog

* s3/iceberg: align error exception types with OpenAPI spec examples

* s3api: refactor AuthenticateRequest to return identity object

* s3/iceberg: propagate full identity object to request context

* s3/iceberg: differentiate NotAuthorizedException and ForbiddenException

* s3/iceberg: reject requests if authenticator is nil to prevent auth bypass

* s3/iceberg: refactor Auth middleware to build context incrementally and use switch for error mapping

* s3api: update misleading comment for authRequestWithAuthType

* s3api: return ErrAccessDenied if IAM is not configured to prevent auth bypass

* s3/iceberg: optimize context update in Auth middleware

* s3api: export CanDo for external authorization use

* s3/iceberg: enforce identity-based authorization in all API handlers

* s3api: fix compilation errors by updating internal CanDo references

* s3/iceberg: robust identity validation and consistent action usage in handlers

* s3api: complete CanDo rename across tests and policy engine integration

* s3api: fix integration tests by allowing admin access when auth is disabled and explicit gRPC ports

* duckdb

* create test bucket
2026-02-03 11:55:12 -08:00
Chris Lu 746df25164 fix formatting 2026-02-03 00:27:20 -08:00
Chris Lu 7169fe585d Delete verify_gc_empty_test.go 2026-02-03 00:20:38 -08:00
Chris Lu 1c1f80358f templ 2026-02-03 00:10:43 -08:00
Chris LuandGitHub 2bb21ea276 feat: Add Iceberg REST Catalog server and admin UI (#8175)
* feat: Add Iceberg REST Catalog server

Implement Iceberg REST Catalog API on a separate port (default 8181)
that exposes S3 Tables metadata through the Apache Iceberg REST protocol.

- Add new weed/s3api/iceberg package with REST handlers
- Implement /v1/config endpoint returning catalog configuration
- Implement namespace endpoints (list/create/get/head/delete)
- Implement table endpoints (list/create/load/head/delete/update)
- Add -port.iceberg flag to S3 standalone server (s3.go)
- Add -s3.port.iceberg flag to combined server mode (server.go)
- Add -s3.port.iceberg flag to mini cluster mode (mini.go)
- Support prefix-based routing for multiple catalogs

The Iceberg REST server reuses S3 Tables metadata storage under
/table-buckets and enables DuckDB, Spark, and other Iceberg clients
to connect to SeaweedFS as a catalog.

* feat: Add Iceberg Catalog pages to admin UI

Add admin UI pages to browse Iceberg catalogs, namespaces, and tables.

- Add Iceberg Catalog menu item under Object Store navigation
- Create iceberg_catalog.templ showing catalog overview with REST info
- Create iceberg_namespaces.templ listing namespaces in a catalog
- Create iceberg_tables.templ listing tables in a namespace
- Add handlers and routes in admin_handlers.go
- Add Iceberg data provider methods in s3tables_management.go
- Add Iceberg data types in types.go

The Iceberg Catalog pages provide visibility into the same S3 Tables
data through an Iceberg-centric lens, including REST endpoint examples
for DuckDB and PyIceberg.

* test: Add Iceberg catalog integration tests and reorg s3tables tests

- Reorganize existing s3tables tests to test/s3tables/table-buckets/
- Add new test/s3tables/catalog/ for Iceberg REST catalog tests
- Add TestIcebergConfig to verify /v1/config endpoint
- Add TestIcebergNamespaces to verify namespace listing
- Add TestDuckDBIntegration for DuckDB connectivity (requires Docker)
- Update CI workflow to use new test paths

* fix: Generate proper random UUIDs for Iceberg tables

Address code review feedback:
- Replace placeholder UUID with crypto/rand-based UUID v4 generation
- Add detailed TODO comments for handleUpdateTable stub explaining
  the required atomic metadata swap implementation

* fix: Serve Iceberg on localhost listener when binding to different interface

Address code review feedback: properly serve the localhost listener
when the Iceberg server is bound to a non-localhost interface.

* ci: Add Iceberg catalog integration tests to CI

Add new job to run Iceberg catalog tests in CI, along with:
- Iceberg package build verification
- Iceberg unit tests
- Iceberg go vet checks
- Iceberg format checks

* fix: Address code review feedback for Iceberg implementation

- fix: Replace hardcoded account ID with s3_constants.AccountAdminId in buildTableBucketARN()
- fix: Improve UUID generation error handling with deterministic fallback (timestamp + PID + counter)
- fix: Update handleUpdateTable to return HTTP 501 Not Implemented instead of fake success
- fix: Better error handling in handleNamespaceExists to distinguish 404 from 500 errors
- fix: Use relative URL in template instead of hardcoded localhost:8181
- fix: Add HTTP timeout to test's waitForService function to avoid hangs
- fix: Use dynamic ephemeral ports in integration tests to avoid flaky parallel failures
- fix: Add Iceberg port to final port configuration logging in mini.go

* fix: Address critical issues in Iceberg implementation

- fix: Cache table UUIDs to ensure persistence across LoadTable calls
  The UUID now remains stable for the lifetime of the server session.
  TODO: For production, UUIDs should be persisted in S3 Tables metadata.

- fix: Remove redundant URL-encoded namespace parsing
  mux router already decodes %1F to \x1F before passing to handlers.
  Redundant ReplaceAll call could cause bugs with literal %1F in namespace.

* fix: Improve test robustness and reduce code duplication

- fix: Make DuckDB test more robust by failing on unexpected errors
  Instead of silently logging errors, now explicitly check for expected
  conditions (extension not available) and skip the test appropriately.

- fix: Extract username helper method to reduce duplication
  Created getUsername() helper in AdminHandlers to avoid duplicating
  the username retrieval logic across Iceberg page handlers.

* fix: Add mutex protection to table UUID cache

Protects concurrent access to the tableUUIDs map with sync.RWMutex.
Uses read-lock for fast path when UUID already cached, and write-lock
for generating new UUIDs. Includes double-check pattern to handle race
condition between read-unlock and write-lock.

* style: fix go fmt errors

* feat(iceberg): persist table UUID in S3 Tables metadata

* feat(admin): configure Iceberg port in Admin UI and commands

* refactor: address review comments (flags, tests, handlers)

- command/mini: fix tracking of explicit s3.port.iceberg flag
- command/admin: add explicit -iceberg.port flag
- admin/handlers: reuse getUsername helper
- tests: use 127.0.0.1 for ephemeral ports and os.Stat for file size check

* test: check error from FileStat in verify_gc_empty_test
2026-02-02 23:12:13 -08:00
Chris Lu 330bd92ddc 4.08 2026-02-02 20:44:13 -08:00
Chris Lu ba8816e2e1 4.08 2026-02-02 20:36:03 -08:00
Chris Lu 3b9e367c1a templ 2026-02-02 20:34:37 -08:00
Lisandro PinandGitHub ff5a8f0579 Implement RPC skeleton for regular/EC volumes scrubbing. (#8187)
* Implement RPC skeleton for regular/EC volumes scrubbing.

See https://github.com/seaweedfs/seaweedfs/issues/8018 for details.

* Minor proto improvements for `ScrubVolume()`, `ScrubEcVolume()`:

  - Add fields for scrubbing details in `ScrubVolumeResponse` and `ScrubEcVolumeResponse`,
    instead of reporting these through RPC errors.
  - Return a list of broken shards when scrubbing EC volumes, via `EcShardInfo'.
2026-02-02 17:55:04 -08:00
Lisandro PinandGitHub 345ac950b6 Add volume server RPCs to read and update state flags. (#8186)
* Boostrap persistent state for volume servers.

This PR implements logic load/save persistent state information for storages
associated with volume servers, and reporting state changes back to masters
via heartbeat messages.

More work ensues!

See https://github.com/seaweedfs/seaweedfs/issues/7977 for details.

* Add volume server RPCs to read and update state flags.
2026-02-02 16:22:17 -08:00
Lisandro PinandGitHub 9638d37fe2 Block RPC write operations on volume servers when maintenance mode is enabled (#8115)
* Boostrap persistent state for volume servers.

This PR implements logic load/save persistent state information for storages
associated with volume servers, and reporting state changes back to masters
via heartbeat messages.

More work ensues!

See https://github.com/seaweedfs/seaweedfs/issues/7977 for details.

* Block RPC operations writing to volume servers when maintenance mode is on.
2026-02-02 13:21:02 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
fca1216f6d build(deps): bump github.com/klauspost/compress from 1.18.2 to 1.18.3 (#8181)
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress) from 1.18.2 to 1.18.3.
- [Release notes](https://github.com/klauspost/compress/releases)
- [Commits](https://github.com/klauspost/compress/compare/v1.18.2...v1.18.3)

---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
  dependency-version: 1.18.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-02 11:14:00 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
97e33b3dbd build(deps): bump cloud.google.com/go/storage from 1.59.1 to 1.59.2 (#8182)
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) from 1.59.1 to 1.59.2.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/storage/v1.59.1...storage/v1.59.2)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
  dependency-version: 1.59.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-02 11:13:50 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
c50aaa9fbd build(deps): bump github.com/lib/pq from 1.10.9 to 1.11.1 (#8183)
Bumps [github.com/lib/pq](https://github.com/lib/pq) from 1.10.9 to 1.11.1.
- [Release notes](https://github.com/lib/pq/releases)
- [Changelog](https://github.com/lib/pq/blob/master/CHANGELOG.md)
- [Commits](https://github.com/lib/pq/compare/v1.10.9...v1.11.1)

---
updated-dependencies:
- dependency-name: github.com/lib/pq
  dependency-version: 1.11.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-02 11:13:37 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
a992417058 build(deps): bump github.com/golang-jwt/jwt/v5 from 5.3.0 to 5.3.1 (#8184)
Bumps [github.com/golang-jwt/jwt/v5](https://github.com/golang-jwt/jwt) from 5.3.0 to 5.3.1.
- [Release notes](https://github.com/golang-jwt/jwt/releases)
- [Commits](https://github.com/golang-jwt/jwt/compare/v5.3.0...v5.3.1)

---
updated-dependencies:
- dependency-name: github.com/golang-jwt/jwt/v5
  dependency-version: 5.3.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-02 11:13:28 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
723fca35d4 build(deps): bump github.com/shirou/gopsutil/v4 from 4.25.12 to 4.26.1 (#8185)
Bumps [github.com/shirou/gopsutil/v4](https://github.com/shirou/gopsutil) from 4.25.12 to 4.26.1.
- [Release notes](https://github.com/shirou/gopsutil/releases)
- [Commits](https://github.com/shirou/gopsutil/compare/v4.25.12...v4.26.1)

---
updated-dependencies:
- dependency-name: github.com/shirou/gopsutil/v4
  dependency-version: 4.26.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-02 11:13:16 -08:00
Chris LuandGitHub f23e09f58b fix: skip exhausted blocks before creating an interval (#8180)
* fix: skip exhausted blocks before creating an interval

* refactor: optimize interval creation and fix logic duplication

* docs: add docstring for LocateData

* refactor: extract moveToNextBlock helper to deduplicate logic

* fix: use int64 for block index comparison to prevent overflow

* test: add unit test for LocateData boundary crossing (issue #8179)

* fix: skip exhausted blocks to prevent negative interval size and panics (issue #8179)

* refactor: apply review suggestions for test maintainability and code style
2026-02-02 11:12:31 -08:00
Chris LuGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
621834d96a s3tables: add Iceberg file layout validation for table buckets (#8176)
* s3tables: add Iceberg file layout validation for table buckets

This PR adds file layout validation for table buckets to enforce Apache
Iceberg table structure. Files uploaded to table buckets must conform
to the expected Iceberg layout:

- metadata/ directory: contains metadata files (*.json, *.avro)
  - v*.metadata.json (table metadata)
  - snap-*.avro (snapshot manifests)
  - *-m*.avro (manifest files)
  - version-hint.text

- data/ directory: contains data files (*.parquet, *.orc, *.avro)
  - Supports partition paths (e.g., year=2024/month=01/)
  - Supports bucket subdirectories

The validator exports functions for use by the S3 API:
- IsTableBucketPath: checks if a path is under /table-buckets/
- GetTableInfoFromPath: extracts bucket/namespace/table from path
- ValidateTableBucketUpload: validates file layout for table bucket uploads
- ValidateTableBucketUploadWithClient: validates with filer client access

Invalid uploads receive InvalidIcebergLayout error response.

* Address review comments: regex performance, error handling, stricter patterns

* Fix validateMetadataFile and validateDataFile to handle subdirectories and directory creation

* Fix error handling, metadata validation, reduce code duplication

* Fix empty remainingPath handling for directory paths

* Refactor: unify validateMetadataFile and validateDataFile

* Refactor: extract UUID pattern constant

* fix: allow Iceberg partition and directory paths without trailing slashes

Modified validateFile to correctly handle directory paths that do not end with a trailing slash.
This ensures that paths like 'data/year=2024' are validated as directories if they match
partition or subdirectory patterns, rather than being incorrectly rejected as invalid files.
Added comprehensive test cases for various directory and partition path combinations.

* refactor: use standard path package and idiomatic returns

Simplified directory and filename extraction in validateFile by using the standard
path package (aliased as pathpkg). This improves readability and avoids manual string
manipulation. Also updated GetTableInfoFromPath to use naked returns for named
return values, aligning with Go conventions for short functions.

* feat: enforce strict Iceberg top-level directories and metadata restrictions

Implemented strict validation for Iceberg layout:
- Bare top-level keys like 'metadata' and 'data' are now rejected; they must have
  a trailing slash or a subpath.
- Subdirectories under 'metadata/' are now prohibited to enforce the flat structure
  required by Iceberg.
- Updated the test suite with negative test cases and ensured proper formatting.

* feat: allow table root directory markers in ValidateTableBucketUpload

Modified ValidateTableBucketUpload to short-circuit and return nil when the
relative path within a table is empty. This occurs when a trailing slash is
used on the table directory (e.g., /table-buckets/mybucket/myns/mytable/).
Added a test case 'table dir with slash' to verify this behavior.

* test: add regression cases for metadata subdirs and table markers

Enforced a strictly flat structure for the metadata directory by removing the
"directory without trailing slash" fallback in validateFile for metadata.
Added regression test cases:
- metadata/nested (must fail)
- /table-buckets/.../mytable/ (must pass)
Verified all tests pass.

* feat: reject double slashes in Iceberg table paths

Modified validateDirectoryPath to return an error when encountering empty path
segments, effectively rejecting double slashes like 'data//file.parquet'.
Updated validateFile to use manual path splitting instead of the 'path' package
for intermediate directories to ensure redundant slashes are not auto-cleaned
before validation. Added regression tests for various double slash scenarios.

* refactor: separate isMetadata logic in validateDirectoryPath

Following reviewer feedback, refactored validateDirectoryPath to explicitly
separate the handling of metadata and data paths. This improves readability
and clarifies the function's intent while maintaining the strict validation rules
and double-slash rejection previously implemented.

* feat: validate bucket, namespace, and table path segments

Updated ValidateTableBucketUpload to ensure that bucket, namespace, and table
segments in the path are non-empty. This prevents invalid paths like
'/table-buckets//myns/mytable/...' from being accepted during upload.
Added regression tests for various empty segment scenarios.

* Update weed/s3api/s3tables/iceberg_layout.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* feat: block double-slash bypass in table relative paths

Added a guard in ValidateTableBucketUpload to reject tableRelativePath if it
starts with a '/' or contains '//'. This ensures that paths like
'/table-buckets/b/ns/t//data/file.parquet' are properly rejected and cannot
bypass the layout validation. Added regression tests to verify.

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-02-02 10:05:04 -08:00
Chris LuGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2ee6e4f391 mount: refresh and evict hot dir cache (#8174)
* mount: refresh and evict hot dir cache

* mount: guard dir update window and extend TTL

* mount: reuse timestamp for cache mark

* Apply suggestion from @gemini-code-assist[bot]

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* mount: make dir cache tuning configurable

* mount: dedupe dir update notices

* mount: restore invalidate-all cache helper

* mount: keep hot dir tuning constants

* mount: centralize cache state reset

* mount: mark refresh completion time

* mount: allow disabling idle eviction

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-01-31 13:46:37 -08:00
Chris LuandGitHub fe6f8d737d mount: invalidate meta cache on follow retry (#8173)
* mount: invalidate meta cache on follow retry

* mount: clear cache under mount root on retry
2026-01-31 11:18:26 -08:00
Chris LuandGitHub 79722bcf30 Add s3tables shell and admin UI (#8172)
* Add shared s3tables manager

* Add s3tables shell commands

* Add s3tables admin API

* Add s3tables admin UI

* Fix admin s3tables namespace create

* Rename table buckets menu

* Centralize s3tables tag validation

* Reuse s3tables manager in admin

* Extract s3tables list limit

* Add s3tables bucket ARN helper

* Remove write middleware from s3tables APIs

* Fix bucket link and policy hint

* Fix table tag parsing and nav link

* Disable namespace table link on invalid ARN

* Improve s3tables error decode

* Return flag parse errors for s3tables tag

* Accept query params for namespace create

* Bind namespace create form data

* Read s3tables JS data from DOM

* s3tables: allow empty region ARN

* shell: pass s3tables account id

* shell: require account for table buckets

* shell: use bucket name for namespaces

* shell: use bucket name for tables

* shell: use bucket name for tags

* admin: add table buckets links in file browser

* s3api: reuse s3tables tag validation

* admin: harden s3tables UI handlers

* fix admin list table buckets

* allow admin s3tables access

* validate s3tables bucket tags

* log s3tables bucket metadata errors

* rollback table bucket on owner failure

* show s3tables bucket owner

* add s3tables iam conditions

* Add s3tables user permissions UI

* Authorize s3tables using identity actions

* Add s3tables permissions to user modal

* Disambiguate bucket scope in user permissions

* Block table bucket names that match S3 buckets

* Pretty-print IAM identity JSON

* Include tags in s3tables permission context

* admin: refactor S3 Tables inline JavaScript into a separate file

* s3tables: extend IAM policy condition operators support

* shell: use LookupEntry wrapper for s3tables bucket conflict check

* admin: handle buildBucketPermissions validation in create/update flows
2026-01-30 22:57:05 -08:00
Chris LuandGitHub b2b0a38e71 s3api: allow empty region and account id in s3tables ARN (#8171)
* s3api: allow empty region and account id in s3tables ARN

* s3api: refactor S3 Tables ARN regex into a constant
2026-01-30 13:15:39 -08:00
Chris LuandGitHub 6a9e7360df s3api: fix S3 Tables auth to allow auto-hashing of body (#8170)
* s3api: allow auto-hashing of request body for s3tables

* s3api: add unit test for s3tables body hashing
2026-01-30 12:02:18 -08:00
f1e27b8f30 s3: change s3 tables to use RESTful API (#8169)
* s3: refactor s3 tables to use RESTful API

* test/s3tables: guard empty namespaces

* s3api: document tag parsing and validate get-table

* s3api: limit S3Tables REST body size

* Update weed/s3api/s3api_tables.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/s3tables/handler.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* s3api: accept encoded table bucket ARNs

* s3api: validate namespaces and close body

* s3api: match encoded table bucket ARNs

* s3api: scope table bucket ARN routes

* s3api: dedupe table bucket request builders

* test/s3tables: allow list tables without namespace

* s3api: validate table params and tag ARN

* s3api: tighten tag handling and get-table params

* s3api: loosen tag ARN route matching

* Fix S3 Tables REST routing and tests

* Adjust S3 Tables request parsing

* Gate S3 Tables target routing

* Avoid double decoding namespaces

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-30 10:37:34 -08:00
Lisandro PinandGitHub 9e15823855 Have masters update DataNode details based on state heartbeats from volume servers. (#8017) 2026-01-29 21:51:46 -08:00
Chris LuandGitHub 6940b7d06e fix Filer startup failure due to JWT on / path #8149 (#8167)
* fix Filer startup failure due to JWT on / path #8149

- Comment out JWT keys in security.toml.example
- Revert Dockerfile.local change that enabled security by default
- Exempt GET/HEAD on / from JWT check for health checks

* refactor: simplify JWT bypass condition as per PR feedback
2026-01-29 21:45:15 -08:00
Chris LuGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
23c25379ca iam: add ECDSA support for OIDC token validation (#8166)
* iam: add ECDSA support for OIDC token validation

Fixes seaweedfs/seaweedfs#8148

* iam: refactor OIDC ECDSA tests and add failure cases

- Refactored TestOIDCProviderJWTValidationECDSA to use t.Run
- Added sub-tests for expired token, wrong key, invalid issuer, and invalid audience

* Update weed/iam/oidc/oidc_provider_test.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* iam: improve error type assertions for OIDC invalid signature tests

- Updated both RSA and ECDSA tests to specifically check for ErrProviderInvalidToken

* iam: pad EC coordinates in OIDC tests to comply with RFC 7518

- Coordinates are now zero-padded to the full field size (e.g., 32 bytes for P-256)
- Ensures interoperability with strict OIDC providers

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-01-29 20:03:43 -08:00
Chris Lu c7c2d8d606 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2026-01-29 20:03:26 -08:00
Chris Lu 88c27615c4 /table-buckets 2026-01-29 20:03:17 -08:00
Chris LuandGitHub 49c66bbb2e shell: allow spaces in arguments via quoting (#8157) (#8165)
* shell: allow spaces in arguments via quoting (#8157)

- updated argument splitting regex to handle quoted segments
- added robust quote stripping to remove matching quotes from flags
- added unit tests for regex splitting and flag parsing

* shell: use robust state machine parser for command line arguments

- replaced regex-based splitter with splitCommandLine state machine
- added escape character support in splitCommandLine and stripQuotes
- updated unit tests to include escaped quotes and single-quote literals
- addressed feedback regarding escaped quotes handling (#8157)

* shell: detect unbalanced quotes in stripQuotes

- modified stripQuotes to return the original string if quotes are unbalanced
- added test cases for unbalanced quotes in shell_liner_test.go

* shell: refactor shared parsing logic into parseShellInput helper

- unified splitting and unquoting logic into a single state machine
- splitCommandLine now returns unquoted tokens directly
- simplified processEachCmd by removing redundant unquoting loop
- improved maintainability by eliminating code duplication

* shell: detect trailing backslash in stripQuotes

- updated parseShellInput to include escaped state in unbalanced flag
- stripQuotes now returns original string if it ends with an unescaped backslash
- added test case for trailing backslash in shell_liner_test.go
2026-01-29 19:06:17 -08:00
Chris Lu 94e0b902f9 shell: update fs.verify and volume.fsck for new BFS signature
Updated dependent commands to match the refactored
doTraverseBfsAndSaving signature and use context for channel sends.
2026-01-29 14:42:10 -08:00
Chris Lu 3b05efbdbc shell: fix potential deadlock in fs.meta.save BFS traversal
Refactored doTraverseBfsAndSaving to use context cancellation.
If the saving process fails, the traversal is stopped immediately
to prevent workers from blocking on the output channel.
2026-01-29 14:42:10 -08:00
Chris Lu b91427c30f filer: preserve existing TTL during CreateEntry/UpdateEntry gRPC calls
This ensures that metadata load correctly restores entries with their
original TTL instead of overwriting with the default from filer.conf.
Fixes #8159.
2026-01-29 14:42:09 -08:00
Chris LuandGitHub 550a4ff761 Fix inconsistent TTL reporting in volume.list #8158 (#8164)
* fix inconsistent TTL reporting in volume.list #8158

* simplify volume.list output using vi.String()
2026-01-29 14:16:42 -08:00
Chris LuandGitHub 8b61fd77b5 s3api: ensure MD5 is calculated or reused during CopyObject (#8163)
* s3api: ensure MD5 is calculated or reused during CopyObject

Fixes #8155
- Capture and reuse source MD5 for direct copies
- Calculate MD5 for small inline objects during copy

* s3api: refactor encryption logic and safe MD5 copying

- Extract duplicated bucket default encryption logic into helper
- Use safe append copy for MD5 slice to avoid shared modifications

* refactor

* avoids unnecessary MD5 recalculations for small files
2026-01-29 12:53:38 -08:00
4d513a2b3d feat(gcs): add application default credentials fallback support (#8161)
* feat(gcs): add application default credentials fallback support

* refactor

* Update weed/remote_storage/gcs/gcs_storage_client.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Chris Lu <chris.lu@gmail.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-29 09:57:49 -08:00
Chris LuandGitHub c52d3d1229 fix: correct chunk size in encrypted uploads (fixes #8151) (#8154)
* fix: correct chunk size in encrypted uploads

When uploading with encryption enabled (-encryptVolumeData flag),
the chunk metadata was incorrectly storing the encrypted data size
instead of the original uncompressed size.

This caused data corruption when reading files back because:
1. Encrypted data is larger than original data
2. After decryption and decompression, the actual data is smaller
3. Size validation in readEncryptedUrl would fail
4. Files appeared truncated with null bytes

The fix ensures uploadResult.Size uses clearDataLen (the original
uncompressed size) in both cipher and non-cipher code paths,
matching the expected behavior.

Fixes: #8151

* test: add comprehensive tests for encrypted upload size fix

Add unit tests to verify that encrypted uploads correctly store the
original uncompressed data size, not the encrypted size.

Tests cover:
- Cipher key generation and encryption/decryption roundtrip
- Compression + encryption roundtrip with size preservation
- Different data sizes
- Compression detection and effectiveness
- Size behavior demonstration showing the bug and fix

All tests verify that metadata stores the original size (19 bytes in
the key example) not the encrypted size (75 bytes), which is critical
for preventing data corruption on read operations.

Issue: #8151

* remove emojis

* Update upload_content_test.go

* Delete upload_content_test.go
2026-01-28 20:46:03 -08:00
Chris Lu 9e575822a3 Merge master into s3tables-by-claude (resolve conflicts, keep s3tables changes) 2026-01-28 19:47:42 -08:00
Chris Lu d399113e0c test: fix duplicate subtest names in permissions_test.go
Rename duplicate 'combined * and ?' test cases to include singular/plural suffix
for clarity and to support targeted test runs.
2026-01-28 19:42:19 -08:00
Chris Lu c106532b79 fix: prevent MiniClusterCtx race conditions in command shutdown
Capture global MiniClusterCtx into local variables before goroutine/select
evaluation to prevent nil dereference/data race when context is reset to nil
after nil check. Applied to filer, master, volume, and s3 commands.
2026-01-28 19:42:16 -08:00
Chris Lu a4217dff5f s3tables: enhance DeleteTable authorization with policy checking
Fetch and evaluate table policies in DeleteTable handler to support policy-based
delegation. Aligns authorization behavior with GetTable and ListTables handlers
instead of only checking ownership.
2026-01-28 19:42:12 -08:00
Chris Lu 745a7e40a6 s3tables: improve bucket policy error handling in DeleteTableBucket
Explicitly handle ErrAttributeNotFound vs other errors when fetching bucket policy.
Return errors for non-expected failures to prevent masking filer issues and
ensure correct authorization decisions.
2026-01-28 19:42:08 -08:00
Chris Lu d5ce6a4cda s3tables: refactor bucket name validation into single function
Combine length, character, and reserved pattern validation into validateBucketName()
which returns descriptive error messages. Keep isValidBucketName() for backward
compatibility. This simplifies handler validation and provides better error reporting.
2026-01-28 19:42:01 -08:00
Chris Lu fe66d00ab0 Merge branch 'master' into s3tables-by-claude 2026-01-28 19:40:46 -08:00
Chris LuGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2f155ee5ee feat: Add S3 Tables support for Iceberg tabular data (#8147)
* s3tables: extract utility and filer operations to separate modules

- Move ARN parsing, path helpers, and metadata structures to utils.go
- Extract all extended attribute and filer operations to filer_ops.go
- Reduces code duplication and improves modularity
- Improves code organization and maintainability

* s3tables: split table bucket operations into focused modules

- Create bucket_create.go for CreateTableBucket operation
- Create bucket_get_list_delete.go for Get, List, Delete operations
- Related operations grouped for better maintainability
- Each file has a single, clear responsibility
- Improves code clarity and makes it easier to test

* s3tables: simplify handler by removing duplicate utilities

- Reduce handler.go from 370 to 195 lines (47% reduction)
- Remove duplicate ARN parsing and path helper functions
- Remove filer operation methods moved to filer_ops.go
- Remove metadata structure definitions moved to utils.go
- Keep handler focused on request routing and response formatting
- Maintains all functionality with improved code organization

* s3tables: complete s3tables package implementation

- namespace.go: namespace CRUD operations (310 lines)
- table.go: table CRUD operations with Iceberg schema support (409 lines)
- policy.go: resource policies and tagging operations (419 lines)
- types.go: request/response types and error definitions (290 lines)
- All handlers updated to use standalone utilities from utils.go
- All files follow single responsibility principle

* s3api: add S3 Tables integration layer

- Create s3api_tables.go to integrate S3 Tables with S3 API server
- Implement S3 Tables route matcher for X-Amz-Target header
- Register S3 Tables routes with API router
- Provide gRPC filer client interface for S3 Tables handlers
- All S3 Tables operations accessible via S3 API endpoint

* s3api: register S3 Tables routes in API server

- Add S3 Tables route registration in s3api_server.go registerRouter method
- Enable S3 Tables API operations to be routed through S3 API server
- Routes handled by s3api_tables.go integration layer
- Minimal changes to existing S3 API structure

* test: add S3 Tables test infrastructure

- Create setup.go with TestCluster and S3TablesClient definitions
- Create client.go with HTTP client methods for all operations
- Test utilities and client methods organized for reusability
- Foundation for S3 Tables integration tests

* test: add S3 Tables integration tests

- Comprehensive integration tests for all 23 S3 Tables operations
- Test cluster setup based on existing S3 integration tests
- Tests cover:
  * Table bucket lifecycle (create, get, list, delete)
  * Namespace operations
  * Table CRUD with Iceberg schema
  * Table bucket and table policies
  * Resource tagging operations
- Ready for CI/CD pipeline integration

* ci: add S3 Tables integration tests to GitHub Actions

- Create new workflow for S3 Tables integration testing
- Add build verification job for s3tables package and s3api integration
- Add format checking for S3 Tables code
- Add go vet checks for code quality
- Workflow runs on all pull requests
- Includes test output logging and artifact upload on failure

* s3tables: add handler_ prefix to operation handler files

- Rename bucket_create.go → handler_bucket_create.go
- Rename bucket_get_list_delete.go → handler_bucket_get_list_delete.go
- Rename namespace.go → handler_namespace.go
- Rename table.go → handler_table.go
- Rename policy.go → handler_policy.go

Improves file organization by clearly identifying handler implementations.
No code changes, refactoring only.

* s3tables test: refactor to eliminate duplicate definitions

- Move all client methods to client.go
- Remove duplicate types/constants from s3tables_integration_test.go
- Keep setup.go for test infrastructure
- Keep integration test logic in s3tables_integration_test.go
- Clean up unused imports
- Test compiles successfully

* Delete client_methods.go

* s3tables: add bucket name validation and fix error handling

- Add isValidBucketName validation function for [a-z0-9_-] characters
- Validate bucket name characters match ARN parsing regex
- Fix error handling in WithFilerClient closure - properly check for lookup errors
- Add error handling for json.Marshal calls (metadata and tags)
- Improve error messages and logging

* s3tables: add error handling for json.Marshal calls

- Add error handling in handler_namespace.go (metadata marshaling)
- Add error handling in handler_table.go (metadata and tags marshaling)
- Add error handling in handler_policy.go (tag marshaling in TagResource and UntagResource)
- Return proper errors with context instead of silently ignoring failures

* s3tables: replace custom splitPath with stdlib functions

- Remove custom splitPath implementation (23 lines)
- Use filepath.Dir and filepath.Base from stdlib
- More robust and handles edge cases correctly
- Reduces code duplication

* s3tables: improve error handling specificity in ListTableBuckets

- Specifically check for 'not found' errors instead of catching all errors
- Return empty list only when directory doesn't exist
- Propagate other errors (network, permission) with context
- Prevents masking real errors

* s3api_tables: optimize action validation with map lookup

- Replace O(n) slice iteration with O(1) map lookup
- Move s3TablesActionsMap to package level
- Avoid recreating the map on every function call
- Improves performance for request validation

* s3tables: implement permission checking and authorization

- Add permissions.go with permission definitions and checks
- Define permissions for all 21 S3 Tables operations
- Add permission checking helper functions
- Add getPrincipalFromRequest to extract caller identity
- Implement access control in CreateTableBucket, GetTableBucket, DeleteTableBucket
- Return 403 Forbidden for unauthorized operations
- Only bucket owner can perform operations (extensible for future policies)
- Add AuthError type for authorization failures

* workflow: fix s3 tables tests path and working directory

The workflow was failing because it was running inside 'weed' directory,
but the tests are at the repository root. Removed working-directory
default and updated relative paths to weed source.

* workflow: remove emojis from echo statements

* test: format s3tables client.go

* workflow: fix go install path to ./weed

* ci: fail s3 tables tests if any command in pipeline fails

* s3tables: use path.Join for path construction and align namespace paths

* s3tables: improve integration test stability and error reporting

* s3tables: propagate request context to filer operations

* s3tables: clean up unused code and improve error response formatting

* Refine S3 Tables implementation to address code review feedback

- Standardize namespace representation to []string
- Improve listing logic with pagination and StartFromFileName
- Enhance error handling with sentinel errors and robust checks
- Add JSON encoding error logging
- Fix CI workflow to use gofmt -l
- Standardize timestamps in directory creation
- Validate single-level namespaces

* s3tables: further refinements to filer operations and utilities

- Add multi-segment namespace support to ARN parsing
- Refactor permission checking to use map lookup
- Wrap lookup errors with ErrNotFound in filer operations
- Standardize splitPath to use path package

* test: improve S3 Tables client error handling and cleanup

- Add detailed error reporting when decoding failure responses
- Remove orphaned comments and unused sections

* command: implement graceful shutdown for mini cluster

- Introduce MiniClusterCtx to coordinate shutdown across mini services
- Update Master, Volume, Filer, S3, and WebDAV servers to respect context cancellation
- Ensure all resources are cleaned up properly during test teardown
- Integrate MiniClusterCtx in s3tables integration tests

* s3tables: fix pagination and enhance error handling in list/delete operations

- Fix InclusiveStartFrom logic to ensure exclusive start on continued pages
- Prevent duplicates in bucket, namespace, and table listings
- Fail fast on listing errors during bucket and namespace deletion
- Stop swallowing errors in handleListTables and return proper HTTP error responses

* s3tables: align ARN formatting and optimize resource handling

- Update generateTableARN to match AWS S3 Tables specification
- Move defer r.Body.Close() to follow standard Go patterns
- Remove unused generateNamespaceARN helper

* command: fix stale error variable logging in filer serving goroutines

- Use local 'err' variable instead of stale 'e' from outer scope
- Applied to both TLS and non-TLS paths for local listener

* s3tables: implement granular authorization and refine error responses

- Remove mandatory ACTION_ADMIN at the router level
- Enforce granular permissions in bucket and namespace handlers
- Prioritize AccountID in ExtractPrincipalFromContext for ARN matching
- Distinguish between 404 (NoSuchBucket) and 500 (InternalError) in metadata lookups
- Clean up unused imports in s3api_tables.go

* test: refactor S3 Tables client for DRYness and multi-segment namespaces

- Implement doRequestAndDecode to eliminate HTTP boilerplate
- Update client API to accept []string for namespaces to support hierarchy
- Standardize error response decoding across all client methods

* test: update integration tests to match refactored S3 Tables client

- Pass namespaces as []string to support hierarchical structures
- Adapt test calls to new client API signatures

* s3tables: normalize filer errors and use standard helpers

- Migrate from custom ErrNotFound to filer_pb.ErrNotFound
- Use filer_pb.LookupEntry for automatic error normalization
- Normalize entryExists and attribute lookups

* s3tables: harden namespace validation and correct ARN parsing

- Prohibit path traversal (".", "..") and "/" in namespaces
- Restrict namespace characters to [a-z0-9_] for consistency
- Switch to url.PathUnescape for correct decoding of ARN path components
- Align ARN parsing regex with single-segment namespace validation

* s3tables: improve robustness, security, and error propagation in handlers

- Implement strict table name validation (prevention of path traversal and character enforcement)
- Add nil checks for entry.Entry in all listing loops to prevent panics
- Propagate backend errors instead of swallowing them or assuming 404
- Correctly map filer_pb.ErrNotFound to appropriate S3 error codes
- Standardize existence checks across bucket, namespace, and table handlers

* test: add miniClusterMutex to prevent race conditions

- Introduce sync.Mutex to protect global state (os.Args, os.Chdir)
- Ensure serialized initialization of the mini cluster runner
- Fix intermittent race conditions during parallel test execution

* s3tables: improve error handling and permission logic

- Update handleGetNamespace to distinguish between 404 and 500 errors
- Refactor CanManagePolicy to use CheckPermission for consistent enforcement
- Ensure empty identities are correctly handled in policy management checks

* s3tables: optimize regex usage and improve version token uniqueness

- Pre-compile regex patterns as package-level variables to avoid re-compilation overhead on every call
- Add a random component to version token generation to reduce collision probability under high concurrency

* s3tables: harden auth and error handling

- Add authorization checks to all S3 Tables handlers (policy, table ops) to enforce security
- Improve error handling to distinguish between NotFound (404) and InternalError (500)
- Fix directory FileMode usage in filer_ops
- Improve test randomness for version tokens
- Update permissions comments to acknowledge IAM gaps

* S3 Tables: fix gRPC stream loop handling for list operations

- Correctly handle io.EOF to terminate loops gracefully.
- Propagate other errors to prevent silent failures.
- Ensure all list results are processed effectively.

* S3 Tables: validate ARN namespace to prevent path traversal

- Enforce validation on decoded namespace in parseTableFromARN.
- Ensures path components are safe after URL unescaping.

* S3 Tables: secure API router with IAM authentication

- Wrap S3 Tables handler with authenticateS3Tables.
- Use AuthSignatureOnly to enforce valid credentials while delegating granular authorization to handlers.
- Prevent anonymous access to all S3 Tables endpoints.

* S3 Tables: fix gRPC stream loop handling in namespace handlers

- Correctly handle io.EOF in handleListNamespaces and handleDeleteNamespace.
- Propagate other errors to prevent silent failures or accidental data loss.
- Added necessary io import.

* S3 Tables: use os.ModeDir constant in filer_ops.go

- Replace magic number 1<<31 with os.ModeDir for better readability.
- Added necessary os import.

* s3tables: improve principal extraction using identity context

* s3tables: remove duplicate comment in permissions.go

* s3tables test: improve error reporting on decoding failure

* s3tables: implement validateTableName helper

* s3tables: add table name validation and 404 propagation to policy handlers

* s3tables: add table name validation and cleanup duplicated logic in table handlers

* s3tables: ensure root tables directory exists before bucket creation

* s3tables: implement token-based pagination for table buckets listing

* s3tables: implement token-based pagination for namespace listing

* s3tables: refine permission helpers to align with operation names

* s3tables: return 404 in handleDeleteNamespace if namespace not found

* s3tables: fix cross-namespace pagination in listTablesInAllNamespaces

* s3tables test: expose pagination parameters in client list methods

* s3tables test: update integration tests for new client API

* s3tables: use crypto/rand for secure version token generation

Replaced math/rand with crypto/rand to ensure version tokens are
cryptographically secure and unpredictable for optimistic concurrency control.

* s3tables: improve account ID handling and define missing error codes

Updated getPrincipalFromRequest to prioritize X-Amz-Account-ID header and
added getAccountID helper. Defined ErrVersionTokenMismatch and ErrCodeConflict
for better optimistic concurrency support.

* s3tables: update bucket handlers for multi-account support

Ensured bucket ownership is correctly attributed to the authenticated
account ID and updated ARNs to use the request-derived account ID. Added
standard S3 existence checks for bucket deletion.

* s3tables: update namespace handlers for multi-account support

Updated namespace creation to use authenticated account ID for ownership
and unified permission checks across all namespace operations to use the
correct account principal.

* s3tables: implement optimistic concurrency for table deletion

Added VersionToken validation to handleDeleteTable. Refactored table
listing to use request context for accurate ARN generation and fixed
cross-namespace pagination issues.

* s3tables: improve resource resolution and error mapping for policies and tagging

Refactored resolveResourcePath to return resource type, enabling accurate
NoSuchBucket vs NoSuchTable error codes. Added existence checks before
deleting policies.

* s3tables: enhance test robustness and resilience

Updated random string generation to use crypto/rand in s3tables tests.
Increased resilience of IAM distributed tests by adding "connection refused"
to retryable errors.

* s3tables: remove legacy principal fallback header

Removed the fallback to X-Amz-Principal in getPrincipalFromRequest as
S3 Tables is a new feature and does not require legacy header support.

* s3tables: remove unused ExtractPrincipalFromContext function

Removed the unused ExtractPrincipalFromContext utility and its
accompanying iam/utils import to keep the new s3tables codebase clean.

* s3tables: allow hyphens in namespace and table names

Relaxed regex validation in utils.go to support hyphens in S3 Tables
namespaces and table names, improving consistency with S3 bucket naming
and allowing derived names from services like S3 Storage Lens.

* s3tables: add isAuthError helper to handler.go

* s3tables: refactor permission checks to use resource owner in bucket handlers

* s3tables: refactor permission checks to use resource owner in namespace handlers

* s3tables: refactor permission checks to use resource owner in table handlers

* s3tables: refactor permission checks to use resource owner in policy and tagging handlers

* ownerAccountID

* s3tables: implement strict AWS-aligned name validation for buckets, namespaces, and tables

* s3tables: enforce strict resource ownership and implement result filtering for buckets

* s3tables: enforce strict resource ownership and implement result filtering for namespaces

* s3tables: enforce strict resource ownership and implement result filtering for tables

* s3tables: align getPrincipalFromRequest with account ID for IAM compatibility

* s3tables: fix inconsistent permission check in handleCreateTableBucket

* s3tables: improve pagination robustness and error handling in table listing handlers

* s3tables: refactor handleDeleteTableBucket to use strongly typed AuthError

* s3tables: align ARN regex patterns with S3 standards and refactor to constants

* s3tables: standardize access denied errors using ErrAccessDenied constant

* go fmt

* s3tables: fix double-write issue in handleListTables

Remove premature HTTP error writes from within WithFilerClient closure
to prevent duplicate status code responses. Error handling is now
consistently performed at the top level using isAuthError.

* s3tables: update bucket name validation message

Remove "underscores" from error message to accurately reflect that
bucket names only allow lowercase letters, numbers, and hyphens.

* s3tables: add table policy test coverage

Add comprehensive test coverage for table policy operations:
- Added PutTablePolicy, GetTablePolicy, DeleteTablePolicy methods to test client
- Implemented testTablePolicy lifecycle test validating Put/Get/Delete operations
- Verified error handling for missing policies

* follow aws spec

* s3tables: add request body size limiting

Add request body size limiting (10MB) to readRequestBody method:
- Define maxRequestBodySize constant to prevent unbounded reads
- Use io.LimitReader to enforce size limit
- Add explicit error handling for oversized requests
- Prevents potential DoS attacks via large request bodies

* S3 Tables API now properly enforces resource policies

addressing the critical security gap where policies were created but never evaluated.

* s3tables: Add upper bound validation for MaxTables parameter

MaxTables is user-controlled and influences gRPC ListEntries limits via
uint32(maxTables*2). Without an upper bound, very large values can overflow
uint32 or cause excessively large directory scans. Cap MaxTables to 1000 and
return InvalidRequest for out-of-range values, consistent with S3 MaxKeys
handling.

* s3tables: Add upper bound validation for MaxBuckets parameter

MaxBuckets is user-controlled and used in uint32(maxBuckets*2) for ListEntries.
Very large values can overflow uint32 or trigger overly expensive scans. Cap
MaxBuckets to 1000 and reject out-of-range values, consistent with MaxTables
handling and S3 MaxKeys validation elsewhere in the codebase.

* s3tables: Validate bucket name in parseBucketNameFromARN()

Enforce the same bucket name validation rules (length, characters, reserved
prefixes/suffixes) when extracting from ARN. This prevents accepting ARNs
that the system would never create and ensures consistency with
CreateTableBucket validation.

* s3tables: Fix parseTableFromARN() namespace and table name validation

- Remove dead URL unescape for namespace (regex [a-z0-9_]+ cannot contain
  percent-escapes)
- Add URL decoding and validation of extracted table name via
  validateTableName() to prevent callers from bypassing request validation
  done in other paths

* s3tables: Rename tableMetadataInternal.Schema to Metadata

The field name 'Schema' was confusing given it holds a *TableMetadata struct
and serializes as 'metadata' in JSON. Rename to 'Metadata' for clarity and
consistency with the JSON tag and intended meaning.

* s3tables: Improve bucket name validation error message

Replace misleading character-only error message with generic 'invalid bucket
name'. The isValidBucketName() function checks multiple constraints beyond
character set (length, reserved prefixes/suffixes, start/end rules), so a
specific character message is inaccurate.

* s3tables: Separate permission checks for tagging and untagging

- Add CanTagResource() to check TagResource permission
- Add CanUntagResource() to check UntagResource permission
- Update CanManageTags() to check both operations (OR logic)

This prevents UntagResource from incorrectly checking 'ManageTags' permission
and ensures each operation validates the correct permission when per-operation
permissions are enforced.

* s3tables: Consolidate getPrincipalFromRequest and getAccountID into single method

Both methods had identical implementations - they return the account ID from
request header or fall back to handler's default. Remove the duplicate
getPrincipalFromRequest and use getAccountID throughout, with updated comment
explaining its dual role as both caller identity and principal for permission
checks.

* s3tables: Fetch bucket policy in handleListTagsForResource for permission evaluation

Update handleListTagsForResource to fetch and pass bucket policy to
CheckPermission, matching the behavior of handleTagResource/handleUntagResource.
This enables bucket-policy-based permission grants to be evaluated for
ListTagsForResource, not just ownership-based checks.

* s3tables: Extract resource owner and bucket extraction into helper method

Create extractResourceOwnerAndBucket() helper to consolidate the repeated pattern
of unmarshaling metadata and extracting bucket name from resource path. This
pattern was duplicated in handleTagResource, handleListTagsForResource, and
handleUntagResource. Update all three handlers to use the helper.

Also update remaining uses of getPrincipalFromRequest() (in handler_bucket_create,
handler_bucket_get_list_delete, handler_namespace) to use getAccountID() after
consolidating the two identical methods.

* s3tables: Add log message when cluster shutdown times out

The timeout path (2 second wait for graceful shutdown) was silent. Add a
warning log message when it occurs to help diagnose flaky test issues and
indicate when the mini cluster didn't shut down cleanly.

* s3tables: Use policy_engine wildcard matcher for complete IAM compatibility

Replace the custom suffix-only wildcard implementation in matchesActionPattern
and matchesPrincipal with the policy_engine.MatchesWildcard function from
PR #8052. This enables full wildcard support including:

- Middle wildcards: s3tables:Get*Table matches GetTable
- Question mark wildcards: Get? matches any single character
- Combined patterns: s3tables:*Table* matches any action containing 'Table'

Benefits:
- Code reuse: eliminates duplicate wildcard logic
- Complete IAM compatibility: supports all AWS wildcard patterns
- Performance: uses efficient O(n) backtracking algorithm
- Consistency: same wildcard behavior across S3 API and S3 Tables

Add comprehensive unit tests covering exact matches, suffix wildcards,
middle wildcards, question marks, and combined patterns for both action
and principal matching.

* go fmt

* s3tables: Fix vet error - remove undefined c.t reference in Stop()

The TestCluster.Stop() method doesn't have access to testing.T object.
Remove the log statement and keep the timeout handling comment for clarity.
The original intent (warning about shutdown timeout) is still captured in
the code comment explaining potential issues.

* clean up

* s3tables: Add t field to TestCluster for logging

Add *testing.T field to TestCluster struct and initialize it in
startMiniCluster. This allows Stop() to properly log warnings when
cluster shutdown times out. Includes the t field in the test cluster
initialization and restores the logging statement in Stop().

* s3tables: Fix bucket policy error handling in permission checks

Replace error-swallowing pattern where all errors from getExtendedAttribute
were ignored for bucket policy reads. Now properly distinguish between:

- ErrAttributeNotFound: Policy not found is expected; continue with empty policy
- Other errors: Return internal server error and stop processing

Applied fix to all bucket policy reads in:
- handleDeleteTableBucketPolicy (line 220)
- handleTagResource (line 313)
- handleUntagResource (line 405)
- handleListTagsForResource (line 488)
- And additional occurrences in closures

This prevents silent failures and ensures policy-related errors are surfaced
to callers rather than being silently ignored.

* s3tables: Pre-validate namespace to return 400 instead of 500

Move validateNamespace call outside of filerClient.WithFilerClient closure
so that validation errors return HTTP 400 (InvalidRequest) instead of 500
(InternalError).

Before: Validation error inside closure → treated as internal error → 500
After: Validation error before closure → handled as bad request → 400

This provides correct error semantics: namespace validation is an input
validation issue, not a server error.

* Update weed/s3api/s3tables/handler.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* s3tables: Normalize action names to include service prefix

Add automatic normalization of operations to full IAM-style action names
(e.g., 's3tables:CreateTableBucket') in CheckPermission(). This ensures
policy statements using prefixed actions (s3tables:*) correctly match
operations evaluated by permission helpers.

Also fixes incorrect r.Context() passed to GetIdentityNameFromContext
which expects *http.Request. Now passes r directly.

* s3tables: Use policy framework for table creation authorization

Replace strict ownership check in CreateTable with policy-based authorization.
Now checks both namespace and bucket policies for CreateTable permission,
allowing delegation via resource policies while still respecting owner bypass.

Authorization logic:
- Namespace policy grants CreateTable → allowed
- Bucket policy grants CreateTable → allowed
- Otherwise → denied (even if same owner)

This enables cross-principal table creation via policies while maintaining
security through explicit allow/deny semantics.

* s3tables: Use policy framework for GetTable authorization

Replace strict ownership check with policy-based authorization in GetTable.
Now checks both table and bucket policies for GetTable permission, allowing
authorized non-owners to read table metadata.

Authorization logic:
- Table policy grants GetTable → allowed
- Bucket policy grants GetTable → allowed
- Otherwise → 404 NotFound (no access disclosed)

Maintains security through policy evaluation while enabling read delegation.

* s3tables: Generate ARNs using resource owner account ID

Change ARN generation to use resource OwnerAccountID instead of caller
identity (h.getAccountID(r)). This ensures ARNs are stable and consistent
regardless of which principal accesses the resource.

Updated generateTableBucketARN and generateTableARN function signatures
to accept ownerAccountID parameter. All call sites updated to pass the
resource owner's account ID from metadata.

This prevents ARN inconsistency issues when multiple principals have
access to the same resource via policies.

* s3tables: Fix remaining policy error handling in namespace and bucket handlers

Replace silent error swallowing (err == nil) with proper error distinction
for bucket policy reads. Now properly checks ErrAttributeNotFound and
propagates other errors as internal server errors.

Fixed 5 locations:
- handleCreateNamespace (policy fetch)
- handleDeleteNamespace (policy fetch)
- handleListNamespaces (policy fetch)
- handleGetNamespace (policy fetch)
- handleGetTableBucket (policy fetch)

This prevents masking of filer issues when policies cannot be read due
to I/O errors or other transient failures.

* ci: Pin GitHub Actions to commit SHAs for s3-tables-tests

Update all action refs to use pinned commit SHAs instead of floating tags:
- actions/checkout: @v6 → @8e8c483 (v4)
- actions/setup-go: @v6 → @0c52d54 (v5)
- actions/upload-artifact: @v6 → @65d8626 (v4)

Pinned SHAs improve reproducibility and reduce supply chain risk by
preventing accidental or malicious changes in action releases. Aligns
with repository conventions used in other workflows (e.g., go.yml).

* s3tables: Add resource ARN validation to policy evaluation

Implement resource-specific policy validation to prevent over-broad
permission grants. Add matchesResource and matchesResourcePattern functions
to validate statement Resource fields against specific resource ARNs.

Add new CheckPermissionWithResource function that includes resource ARN
validation, while keeping CheckPermission unchanged for backward compatibility.

This enables policies to grant access to specific resources only:
- statements with Resource: "arn:aws:s3tables:...:bucket/specific-bucket/*"
  will only match when accessing that specific bucket
- statements without Resource field match all resources (implicit *)
- resource patterns support wildcards (* for any sequence, ? for single char)

For future use: Handlers can call CheckPermissionWithResource with the
target resource ARN to enforce resource-level access control.

* Revert "ci: Pin GitHub Actions to commit SHAs for s3-tables-tests"

This reverts commit 01da26fbcb.

* s3tables: Remove duplicate bucket extraction logic in helper

Move bucket name extraction outside the if/else block in
extractResourceOwnerAndBucket since the logic is identical for both
ResourceTypeTable and ResourceTypeBucket cases. This reduces code
duplication and improves maintainability.

The extraction pattern (parts[1] from /tables/{bucket}/...) works for
both resource types, so it's now performed once before the type-specific
metadata unmarshaling.

* go fmt

* s3tables: Fix ownership consistency across handlers

Address three related ownership consistency issues:

1. CreateNamespace now sets OwnerAccountID to bucketMetadata.OwnerAccountID
   instead of request principal. This prevents namespaces created by
   delegated callers (via bucket policy) from becoming unmanageable, since
   ListNamespaces filters by bucket owner.

2. CreateTable now:
   - Fetches bucket metadata to use correct owner for bucket policy evaluation
   - Uses namespaceMetadata.OwnerAccountID for namespace policy checks
   - Uses bucketMetadata.OwnerAccountID for bucket policy checks
   - Sets table OwnerAccountID to namespaceMetadata.OwnerAccountID (inherited)

3. GetTable now:
   - Fetches bucket metadata to use correct owner for bucket policy evaluation
   - Uses metadata.OwnerAccountID for table policy checks
   - Uses bucketMetadata.OwnerAccountID for bucket policy checks

This ensures:
- Bucket owner retains implicit "owner always allowed" behavior even when
  evaluating bucket policies
- Ownership hierarchy is consistent (namespace owned by bucket, table owned by namespace)
- Cross-principal delegation via policies doesn't break ownership chains

* s3tables: Fix ListTables authorization and policy parsing

Make ListTables authorization consistent with GetTable/CreateTable:

1. ListTables authorization now evaluates policies instead of owner-only checks:
   - For namespace listing: checks namespace policy AND bucket policy
   - For bucket-wide listing: checks bucket policy
   - Uses CanListTables permission framework

2. Remove owner-only filter in listTablesWithClient that prevented policy-based
   sharing of tables. Authorization is now enforced at the handler level, so all
   tables in the namespace/bucket are returned to authorized callers (who have
   access either via ownership or policy).

3. Add flexible PolicyDocument.UnmarshalJSON to support both single-object and
   array forms of Statement field:
   - Handles: {"Statement": {...}}
   - Handles: {"Statement": [{...}, {...}]}
   - Improves AWS IAM compatibility

This ensures cross-account table listing works when delegated via bucket/namespace
policies, consistent with the authorization model for other operations.

* go fmt

* s3tables: Separate table name pattern constant for clarity

Define a separate tableNamePatternStr constant for the table name component in
the ARN regex, even though it currently has the same value as
tableNamespacePatternStr. This improves code clarity and maintainability, making
it easier to modify if the naming rules for tables and namespaces diverge in the
future.

* refactor

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-01-28 19:39:48 -08:00
Chris Lu 549b65785d refactor 2026-01-28 18:55:16 -08:00
Chris Lu 590e7efbef s3tables: Separate table name pattern constant for clarity
Define a separate tableNamePatternStr constant for the table name component in
the ARN regex, even though it currently has the same value as
tableNamespacePatternStr. This improves code clarity and maintainability, making
it easier to modify if the naming rules for tables and namespaces diverge in the
future.
2026-01-28 18:40:02 -08:00
Chris Lu 78c00e313a go fmt 2026-01-28 18:34:32 -08:00
Chris Lu f5d26b803b s3tables: Fix ListTables authorization and policy parsing
Make ListTables authorization consistent with GetTable/CreateTable:

1. ListTables authorization now evaluates policies instead of owner-only checks:
   - For namespace listing: checks namespace policy AND bucket policy
   - For bucket-wide listing: checks bucket policy
   - Uses CanListTables permission framework

2. Remove owner-only filter in listTablesWithClient that prevented policy-based
   sharing of tables. Authorization is now enforced at the handler level, so all
   tables in the namespace/bucket are returned to authorized callers (who have
   access either via ownership or policy).

3. Add flexible PolicyDocument.UnmarshalJSON to support both single-object and
   array forms of Statement field:
   - Handles: {"Statement": {...}}
   - Handles: {"Statement": [{...}, {...}]}
   - Improves AWS IAM compatibility

This ensures cross-account table listing works when delegated via bucket/namespace
policies, consistent with the authorization model for other operations.
2026-01-28 18:27:37 -08:00
Chris Lu 25b0f86bda s3tables: Fix ownership consistency across handlers
Address three related ownership consistency issues:

1. CreateNamespace now sets OwnerAccountID to bucketMetadata.OwnerAccountID
   instead of request principal. This prevents namespaces created by
   delegated callers (via bucket policy) from becoming unmanageable, since
   ListNamespaces filters by bucket owner.

2. CreateTable now:
   - Fetches bucket metadata to use correct owner for bucket policy evaluation
   - Uses namespaceMetadata.OwnerAccountID for namespace policy checks
   - Uses bucketMetadata.OwnerAccountID for bucket policy checks
   - Sets table OwnerAccountID to namespaceMetadata.OwnerAccountID (inherited)

3. GetTable now:
   - Fetches bucket metadata to use correct owner for bucket policy evaluation
   - Uses metadata.OwnerAccountID for table policy checks
   - Uses bucketMetadata.OwnerAccountID for bucket policy checks

This ensures:
- Bucket owner retains implicit "owner always allowed" behavior even when
  evaluating bucket policies
- Ownership hierarchy is consistent (namespace owned by bucket, table owned by namespace)
- Cross-principal delegation via policies doesn't break ownership chains
2026-01-28 18:03:47 -08:00
Chris Lu b049e883e1 go fmt 2026-01-28 17:51:02 -08:00
Chris Lu c99e8d4152 s3tables: Remove duplicate bucket extraction logic in helper
Move bucket name extraction outside the if/else block in
extractResourceOwnerAndBucket since the logic is identical for both
ResourceTypeTable and ResourceTypeBucket cases. This reduces code
duplication and improves maintainability.

The extraction pattern (parts[1] from /tables/{bucket}/...) works for
both resource types, so it's now performed once before the type-specific
metadata unmarshaling.
2026-01-28 17:47:14 -08:00
Chris Lu 3dcaee56aa Revert "ci: Pin GitHub Actions to commit SHAs for s3-tables-tests"
This reverts commit 01da26fbcb.
2026-01-28 17:43:11 -08:00
Chris Lu 21584e4ac8 s3tables: Add resource ARN validation to policy evaluation
Implement resource-specific policy validation to prevent over-broad
permission grants. Add matchesResource and matchesResourcePattern functions
to validate statement Resource fields against specific resource ARNs.

Add new CheckPermissionWithResource function that includes resource ARN
validation, while keeping CheckPermission unchanged for backward compatibility.

This enables policies to grant access to specific resources only:
- statements with Resource: "arn:aws:s3tables:...:bucket/specific-bucket/*"
  will only match when accessing that specific bucket
- statements without Resource field match all resources (implicit *)
- resource patterns support wildcards (* for any sequence, ? for single char)

For future use: Handlers can call CheckPermissionWithResource with the
target resource ARN to enforce resource-level access control.
2026-01-28 17:41:22 -08:00
Chris Lu 01da26fbcb ci: Pin GitHub Actions to commit SHAs for s3-tables-tests
Update all action refs to use pinned commit SHAs instead of floating tags:
- actions/checkout: @v6 → @8e8c483 (v4)
- actions/setup-go: @v6 → @0c52d54 (v5)
- actions/upload-artifact: @v6 → @65d8626 (v4)

Pinned SHAs improve reproducibility and reduce supply chain risk by
preventing accidental or malicious changes in action releases. Aligns
with repository conventions used in other workflows (e.g., go.yml).
2026-01-28 17:40:40 -08:00
Chris Lu 2c45b69775 s3tables: Fix remaining policy error handling in namespace and bucket handlers
Replace silent error swallowing (err == nil) with proper error distinction
for bucket policy reads. Now properly checks ErrAttributeNotFound and
propagates other errors as internal server errors.

Fixed 5 locations:
- handleCreateNamespace (policy fetch)
- handleDeleteNamespace (policy fetch)
- handleListNamespaces (policy fetch)
- handleGetNamespace (policy fetch)
- handleGetTableBucket (policy fetch)

This prevents masking of filer issues when policies cannot be read due
to I/O errors or other transient failures.
2026-01-28 17:39:44 -08:00
Chris Lu b7bba7e7dc s3tables: Generate ARNs using resource owner account ID
Change ARN generation to use resource OwnerAccountID instead of caller
identity (h.getAccountID(r)). This ensures ARNs are stable and consistent
regardless of which principal accesses the resource.

Updated generateTableBucketARN and generateTableARN function signatures
to accept ownerAccountID parameter. All call sites updated to pass the
resource owner's account ID from metadata.

This prevents ARN inconsistency issues when multiple principals have
access to the same resource via policies.
2026-01-28 17:38:22 -08:00
Chris Lu e7b2869aa9 s3tables: Use policy framework for GetTable authorization
Replace strict ownership check with policy-based authorization in GetTable.
Now checks both table and bucket policies for GetTable permission, allowing
authorized non-owners to read table metadata.

Authorization logic:
- Table policy grants GetTable → allowed
- Bucket policy grants GetTable → allowed
- Otherwise → 404 NotFound (no access disclosed)

Maintains security through policy evaluation while enabling read delegation.
2026-01-28 17:37:12 -08:00
Chris Lu bea0f8eda0 s3tables: Use policy framework for table creation authorization
Replace strict ownership check in CreateTable with policy-based authorization.
Now checks both namespace and bucket policies for CreateTable permission,
allowing delegation via resource policies while still respecting owner bypass.

Authorization logic:
- Namespace policy grants CreateTable → allowed
- Bucket policy grants CreateTable → allowed
- Otherwise → denied (even if same owner)

This enables cross-principal table creation via policies while maintaining
security through explicit allow/deny semantics.
2026-01-28 17:36:53 -08:00
Chris Lu cf5043a9f9 s3tables: Normalize action names to include service prefix
Add automatic normalization of operations to full IAM-style action names
(e.g., 's3tables:CreateTableBucket') in CheckPermission(). This ensures
policy statements using prefixed actions (s3tables:*) correctly match
operations evaluated by permission helpers.

Also fixes incorrect r.Context() passed to GetIdentityNameFromContext
which expects *http.Request. Now passes r directly.
2026-01-28 17:36:16 -08:00
5c8de5e282 fix: close volumes and EC shards in tests for Windows compatibility (#8152)
* fix: close volumes and EC shards in tests to prevent Windows cleanup failures

On Windows, t.TempDir() cleanup fails when test files are still open
because Windows enforces mandatory file locking. Add defer v.Close(),
defer store.Close(), and EC volume cleanup to ensure all file handles
are released before temp directory removal.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract closeEcVolumes helper to reduce duplication

Address code review feedback by extracting the repeated EC volume
cleanup loop into a closeEcVolumes() helper function.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 17:21:14 -08:00
Chris LuGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
ee468749bd Update weed/s3api/s3tables/handler.go
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-01-28 17:12:18 -08:00
Chris Lu 08bd1e2563 s3tables: Pre-validate namespace to return 400 instead of 500
Move validateNamespace call outside of filerClient.WithFilerClient closure
so that validation errors return HTTP 400 (InvalidRequest) instead of 500
(InternalError).

Before: Validation error inside closure → treated as internal error → 500
After: Validation error before closure → handled as bad request → 400

This provides correct error semantics: namespace validation is an input
validation issue, not a server error.
2026-01-28 17:03:04 -08:00
Chris Lu 8eee6b2a0e s3tables: Fix bucket policy error handling in permission checks
Replace error-swallowing pattern where all errors from getExtendedAttribute
were ignored for bucket policy reads. Now properly distinguish between:

- ErrAttributeNotFound: Policy not found is expected; continue with empty policy
- Other errors: Return internal server error and stop processing

Applied fix to all bucket policy reads in:
- handleDeleteTableBucketPolicy (line 220)
- handleTagResource (line 313)
- handleUntagResource (line 405)
- handleListTagsForResource (line 488)
- And additional occurrences in closures

This prevents silent failures and ensures policy-related errors are surfaced
to callers rather than being silently ignored.
2026-01-28 17:02:59 -08:00
Chris Lu fe856928c4 s3tables: Add t field to TestCluster for logging
Add *testing.T field to TestCluster struct and initialize it in
startMiniCluster. This allows Stop() to properly log warnings when
cluster shutdown times out. Includes the t field in the test cluster
initialization and restores the logging statement in Stop().
2026-01-28 17:02:53 -08:00
Chris Lu 6658a655f6 clean up 2026-01-28 17:00:42 -08:00
Chris Lu c5eadadf5a s3tables: Fix vet error - remove undefined c.t reference in Stop()
The TestCluster.Stop() method doesn't have access to testing.T object.
Remove the log statement and keep the timeout handling comment for clarity.
The original intent (warning about shutdown timeout) is still captured in
the code comment explaining potential issues.
2026-01-28 16:44:16 -08:00
Chris Lu 1e18c01a78 go fmt 2026-01-28 16:42:46 -08:00
Chris Lu 3e8d2a0a71 s3tables: Use policy_engine wildcard matcher for complete IAM compatibility
Replace the custom suffix-only wildcard implementation in matchesActionPattern
and matchesPrincipal with the policy_engine.MatchesWildcard function from
PR #8052. This enables full wildcard support including:

- Middle wildcards: s3tables:Get*Table matches GetTable
- Question mark wildcards: Get? matches any single character
- Combined patterns: s3tables:*Table* matches any action containing 'Table'

Benefits:
- Code reuse: eliminates duplicate wildcard logic
- Complete IAM compatibility: supports all AWS wildcard patterns
- Performance: uses efficient O(n) backtracking algorithm
- Consistency: same wildcard behavior across S3 API and S3 Tables

Add comprehensive unit tests covering exact matches, suffix wildcards,
middle wildcards, question marks, and combined patterns for both action
and principal matching.
2026-01-28 16:37:31 -08:00
Chris Lu dbf6465b0e s3tables: Add log message when cluster shutdown times out
The timeout path (2 second wait for graceful shutdown) was silent. Add a
warning log message when it occurs to help diagnose flaky test issues and
indicate when the mini cluster didn't shut down cleanly.
2026-01-28 16:24:15 -08:00
Chris Lu a27f6527ab s3tables: Extract resource owner and bucket extraction into helper method
Create extractResourceOwnerAndBucket() helper to consolidate the repeated pattern
of unmarshaling metadata and extracting bucket name from resource path. This
pattern was duplicated in handleTagResource, handleListTagsForResource, and
handleUntagResource. Update all three handlers to use the helper.

Also update remaining uses of getPrincipalFromRequest() (in handler_bucket_create,
handler_bucket_get_list_delete, handler_namespace) to use getAccountID() after
consolidating the two identical methods.
2026-01-28 16:24:07 -08:00
Chris Lu 0b41ade726 s3tables: Fetch bucket policy in handleListTagsForResource for permission evaluation
Update handleListTagsForResource to fetch and pass bucket policy to
CheckPermission, matching the behavior of handleTagResource/handleUntagResource.
This enables bucket-policy-based permission grants to be evaluated for
ListTagsForResource, not just ownership-based checks.
2026-01-28 16:23:12 -08:00
Chris Lu 41e799b4e0 s3tables: Consolidate getPrincipalFromRequest and getAccountID into single method
Both methods had identical implementations - they return the account ID from
request header or fall back to handler's default. Remove the duplicate
getPrincipalFromRequest and use getAccountID throughout, with updated comment
explaining its dual role as both caller identity and principal for permission
checks.
2026-01-28 16:23:01 -08:00
Chris Lu ee3d779a5d s3tables: Separate permission checks for tagging and untagging
- Add CanTagResource() to check TagResource permission
- Add CanUntagResource() to check UntagResource permission
- Update CanManageTags() to check both operations (OR logic)

This prevents UntagResource from incorrectly checking 'ManageTags' permission
and ensures each operation validates the correct permission when per-operation
permissions are enforced.
2026-01-28 16:21:38 -08:00
Chris Lu 169ee629fa s3tables: Improve bucket name validation error message
Replace misleading character-only error message with generic 'invalid bucket
name'. The isValidBucketName() function checks multiple constraints beyond
character set (length, reserved prefixes/suffixes, start/end rules), so a
specific character message is inaccurate.
2026-01-28 16:21:15 -08:00
Chris Lu fb8390c6a7 s3tables: Rename tableMetadataInternal.Schema to Metadata
The field name 'Schema' was confusing given it holds a *TableMetadata struct
and serializes as 'metadata' in JSON. Rename to 'Metadata' for clarity and
consistency with the JSON tag and intended meaning.
2026-01-28 16:21:06 -08:00
Chris Lu 191a858e72 s3tables: Fix parseTableFromARN() namespace and table name validation
- Remove dead URL unescape for namespace (regex [a-z0-9_]+ cannot contain
  percent-escapes)
- Add URL decoding and validation of extracted table name via
  validateTableName() to prevent callers from bypassing request validation
  done in other paths
2026-01-28 16:20:58 -08:00
Chris Lu fb4fb8b082 s3tables: Validate bucket name in parseBucketNameFromARN()
Enforce the same bucket name validation rules (length, characters, reserved
prefixes/suffixes) when extracting from ARN. This prevents accepting ARNs
that the system would never create and ensures consistency with
CreateTableBucket validation.
2026-01-28 16:20:49 -08:00
Chris Lu b1d7f3d6e8 s3tables: Add upper bound validation for MaxBuckets parameter
MaxBuckets is user-controlled and used in uint32(maxBuckets*2) for ListEntries.
Very large values can overflow uint32 or trigger overly expensive scans. Cap
MaxBuckets to 1000 and reject out-of-range values, consistent with MaxTables
handling and S3 MaxKeys validation elsewhere in the codebase.
2026-01-28 16:20:36 -08:00
Chris Lu e0da63fd0a s3tables: Add upper bound validation for MaxTables parameter
MaxTables is user-controlled and influences gRPC ListEntries limits via
uint32(maxTables*2). Without an upper bound, very large values can overflow
uint32 or cause excessively large directory scans. Cap MaxTables to 1000 and
return InvalidRequest for out-of-range values, consistent with S3 MaxKeys
handling.
2026-01-28 16:20:32 -08:00
Chris Lu 2d556ac2a5 S3 Tables API now properly enforces resource policies
addressing the critical security gap where policies were created but never evaluated.
2026-01-28 16:15:34 -08:00
Chris Lu e862888d2d s3tables: add request body size limiting
Add request body size limiting (10MB) to readRequestBody method:
- Define maxRequestBodySize constant to prevent unbounded reads
- Use io.LimitReader to enforce size limit
- Add explicit error handling for oversized requests
- Prevents potential DoS attacks via large request bodies
2026-01-28 14:54:45 -08:00
Chris Lu b142689232 follow aws spec 2026-01-28 14:52:05 -08:00
Chris Lu 473e699368 s3tables: add table policy test coverage
Add comprehensive test coverage for table policy operations:
- Added PutTablePolicy, GetTablePolicy, DeleteTablePolicy methods to test client
- Implemented testTablePolicy lifecycle test validating Put/Get/Delete operations
- Verified error handling for missing policies
2026-01-28 14:44:28 -08:00
Chris Lu 0115e60919 s3tables: update bucket name validation message
Remove "underscores" from error message to accurately reflect that
bucket names only allow lowercase letters, numbers, and hyphens.
2026-01-28 14:41:15 -08:00
Chris Lu a6c3e96f7b s3tables: fix double-write issue in handleListTables
Remove premature HTTP error writes from within WithFilerClient closure
to prevent duplicate status code responses. Error handling is now
consistently performed at the top level using isAuthError.
2026-01-28 14:41:14 -08:00
Chris Lu dffe038efa go fmt 2026-01-28 14:34:07 -08:00
Chris Lu 4d4af0589b s3tables: standardize access denied errors using ErrAccessDenied constant 2026-01-28 14:33:01 -08:00
Chris Lu d98e104dc5 s3tables: align ARN regex patterns with S3 standards and refactor to constants 2026-01-28 14:28:12 -08:00
Chris Lu f5d71008d7 s3tables: refactor handleDeleteTableBucket to use strongly typed AuthError 2026-01-28 14:28:12 -08:00
Chris Lu 0d65daad4c s3tables: improve pagination robustness and error handling in table listing handlers 2026-01-28 14:04:09 -08:00
Chris Lu 612eae9ae8 s3tables: fix inconsistent permission check in handleCreateTableBucket 2026-01-28 14:04:08 -08:00
Chris Lu a689c1e052 s3tables: align getPrincipalFromRequest with account ID for IAM compatibility 2026-01-28 14:04:08 -08:00
Chris Lu d4ebafbacd s3tables: enforce strict resource ownership and implement result filtering for tables 2026-01-28 13:59:28 -08:00
Chris Lu 43aebc10da s3tables: enforce strict resource ownership and implement result filtering for namespaces 2026-01-28 13:59:24 -08:00
Chris Lu 9d54f4d160 s3tables: enforce strict resource ownership and implement result filtering for buckets 2026-01-28 13:59:21 -08:00
Chris Lu c8cfbaa069 s3tables: implement strict AWS-aligned name validation for buckets, namespaces, and tables 2026-01-28 13:59:16 -08:00
Chris Lu 1697ec862f ownerAccountID 2026-01-28 13:54:49 -08:00
Chris Lu 1fdd9c3372 s3tables: refactor permission checks to use resource owner in policy and tagging handlers 2026-01-28 13:50:24 -08:00
Chris Lu 78a007d42b s3tables: refactor permission checks to use resource owner in table handlers 2026-01-28 13:50:19 -08:00
Chris Lu ef0bae45e3 s3tables: refactor permission checks to use resource owner in namespace handlers 2026-01-28 13:50:16 -08:00
Chris Lu 32fade010a s3tables: refactor permission checks to use resource owner in bucket handlers 2026-01-28 13:50:13 -08:00
Chris Lu 2f4cee9538 s3tables: add isAuthError helper to handler.go 2026-01-28 13:50:10 -08:00
Chris Lu 090d473822 s3tables: allow hyphens in namespace and table names
Relaxed regex validation in utils.go to support hyphens in S3 Tables
namespaces and table names, improving consistency with S3 bucket naming
and allowing derived names from services like S3 Storage Lens.
2026-01-28 13:38:41 -08:00
Chris Lu d6f6bf4ce7 s3tables: remove unused ExtractPrincipalFromContext function
Removed the unused ExtractPrincipalFromContext utility and its
accompanying iam/utils import to keep the new s3tables codebase clean.
2026-01-28 13:31:53 -08:00
Chris Lu e4e4dea140 s3tables: remove legacy principal fallback header
Removed the fallback to X-Amz-Principal in getPrincipalFromRequest as
S3 Tables is a new feature and does not require legacy header support.
2026-01-28 13:28:42 -08:00
Chris Lu dfdace9a13 s3tables: enhance test robustness and resilience
Updated random string generation to use crypto/rand in s3tables tests.
Increased resilience of IAM distributed tests by adding "connection refused"
to retryable errors.
2026-01-28 13:25:32 -08:00
Chris Lu 2b2ff008cd s3tables: improve resource resolution and error mapping for policies and tagging
Refactored resolveResourcePath to return resource type, enabling accurate
NoSuchBucket vs NoSuchTable error codes. Added existence checks before
deleting policies.
2026-01-28 13:25:32 -08:00
Chris Lu f17ec59d46 s3tables: implement optimistic concurrency for table deletion
Added VersionToken validation to handleDeleteTable. Refactored table
listing to use request context for accurate ARN generation and fixed
cross-namespace pagination issues.
2026-01-28 13:25:32 -08:00
Chris Lu 12c1190a5c s3tables: update namespace handlers for multi-account support
Updated namespace creation to use authenticated account ID for ownership
and unified permission checks across all namespace operations to use the
correct account principal.
2026-01-28 13:25:27 -08:00
Chris Lu 922b0c3171 s3tables: update bucket handlers for multi-account support
Ensured bucket ownership is correctly attributed to the authenticated
account ID and updated ARNs to use the request-derived account ID. Added
standard S3 existence checks for bucket deletion.
2026-01-28 13:25:27 -08:00
Chris Lu 31867b6f75 s3tables: improve account ID handling and define missing error codes
Updated getPrincipalFromRequest to prioritize X-Amz-Account-ID header and
added getAccountID helper. Defined ErrVersionTokenMismatch and ErrCodeConflict
for better optimistic concurrency support.
2026-01-28 13:25:22 -08:00
Chris Lu e381b81b47 s3tables: use crypto/rand for secure version token generation
Replaced math/rand with crypto/rand to ensure version tokens are
cryptographically secure and unpredictable for optimistic concurrency control.
2026-01-28 13:25:19 -08:00
51ef39fc76 Update Helm hook annotations for post-install and upgrade (#8150)
* Update Helm hook annotations for post-install and upgrade

I believe it makes sense to allow this job to run also after installation. Assuming weed shell is idempotent, and assuming someone wants to add a new bucket after the initial installation, it makes sense to trigger the job again.

* Add check for existing buckets before creation

* Enhances S3 bucket existence check

Improves the reliability of checking for existing S3 buckets in the post-install hook.

The previous `grep -w` command could lead to imprecise matches. This update extracts only the bucket name and performs an exact, whole-line match to ensure accurate detection of existing buckets. This prevents potential issues with redundant creation attempts or false negatives.

* Currently Bucket Creation is ignored if filer.s3.enabled is disabled

This commit enables bucket creation on both scenarios,i.e. if any of filer.s3.enabled or s3.enabled are used.

---------

Co-authored-by: Emanuele <emanuele.leopardi@tset.com>
2026-01-28 13:08:20 -08:00
Chris Lu 89b85bfd5e s3tables test: update integration tests for new client API 2026-01-28 12:46:21 -08:00
Chris Lu 783fe25eac s3tables test: expose pagination parameters in client list methods 2026-01-28 12:46:21 -08:00
Chris Lu 5c43f1e6a4 s3tables: fix cross-namespace pagination in listTablesInAllNamespaces 2026-01-28 12:46:21 -08:00
Chris Lu 5eed1874a9 s3tables: return 404 in handleDeleteNamespace if namespace not found 2026-01-28 12:46:20 -08:00
Chris Lu ae7743304c s3tables: refine permission helpers to align with operation names 2026-01-28 12:46:20 -08:00
Chris Lu babf1b06ac s3tables: implement token-based pagination for namespace listing 2026-01-28 12:30:33 -08:00
Chris Lu 6ff683a627 s3tables: implement token-based pagination for table buckets listing 2026-01-28 12:30:33 -08:00
Chris Lu 1fb3aefa95 s3tables: ensure root tables directory exists before bucket creation 2026-01-28 12:30:32 -08:00
Chris Lu 47ef8c3cce s3tables: add table name validation and cleanup duplicated logic in table handlers 2026-01-28 12:30:32 -08:00
Chris Lu f83cef1259 s3tables: add table name validation and 404 propagation to policy handlers 2026-01-28 12:30:31 -08:00
Chris Lu ab6351e1c9 s3tables: implement validateTableName helper 2026-01-28 12:30:31 -08:00
Chris Lu 9f1dd57939 s3tables test: improve error reporting on decoding failure 2026-01-28 12:30:30 -08:00
Chris Lu 1f70d82170 s3tables: remove duplicate comment in permissions.go 2026-01-28 12:30:30 -08:00
Chris Lu 6d01e42cef s3tables: improve principal extraction using identity context 2026-01-28 12:30:29 -08:00
Chris Lu 5cea00ff07 S3 Tables: use os.ModeDir constant in filer_ops.go
- Replace magic number 1<<31 with os.ModeDir for better readability.
- Added necessary os import.
2026-01-28 12:13:35 -08:00
Chris Lu d8c7c16aad S3 Tables: fix gRPC stream loop handling in namespace handlers
- Correctly handle io.EOF in handleListNamespaces and handleDeleteNamespace.
- Propagate other errors to prevent silent failures or accidental data loss.
- Added necessary io import.
2026-01-28 12:13:29 -08:00
Chris Lu b7c1eba0a1 S3 Tables: secure API router with IAM authentication
- Wrap S3 Tables handler with authenticateS3Tables.
- Use AuthSignatureOnly to enforce valid credentials while delegating granular authorization to handlers.
- Prevent anonymous access to all S3 Tables endpoints.
2026-01-28 12:09:15 -08:00
Chris Lu ae19621230 S3 Tables: validate ARN namespace to prevent path traversal
- Enforce validation on decoded namespace in parseTableFromARN.
- Ensures path components are safe after URL unescaping.
2026-01-28 12:09:09 -08:00
Chris Lu f13e250fc3 S3 Tables: fix gRPC stream loop handling for list operations
- Correctly handle io.EOF to terminate loops gracefully.
- Propagate other errors to prevent silent failures.
- Ensure all list results are processed effectively.
2026-01-28 12:09:04 -08:00
Chris Lu dc4c62e742 s3tables: harden auth and error handling
- Add authorization checks to all S3 Tables handlers (policy, table ops) to enforce security
- Improve error handling to distinguish between NotFound (404) and InternalError (500)
- Fix directory FileMode usage in filer_ops
- Improve test randomness for version tokens
- Update permissions comments to acknowledge IAM gaps
2026-01-28 11:49:57 -08:00
Chris Lu a3af5eb77a s3tables: optimize regex usage and improve version token uniqueness
- Pre-compile regex patterns as package-level variables to avoid re-compilation overhead on every call
- Add a random component to version token generation to reduce collision probability under high concurrency
2026-01-28 11:44:56 -08:00
Chris Lu 1c0d37e15a s3tables: improve error handling and permission logic
- Update handleGetNamespace to distinguish between 404 and 500 errors
- Refactor CanManagePolicy to use CheckPermission for consistent enforcement
- Ensure empty identities are correctly handled in policy management checks
2026-01-28 11:39:28 -08:00
Chris Lu 6fc170c645 test: add miniClusterMutex to prevent race conditions
- Introduce sync.Mutex to protect global state (os.Args, os.Chdir)
- Ensure serialized initialization of the mini cluster runner
- Fix intermittent race conditions during parallel test execution
2026-01-28 11:39:22 -08:00
Chris Lu 62a1178a0b s3tables: improve robustness, security, and error propagation in handlers
- Implement strict table name validation (prevention of path traversal and character enforcement)
- Add nil checks for entry.Entry in all listing loops to prevent panics
- Propagate backend errors instead of swallowing them or assuming 404
- Correctly map filer_pb.ErrNotFound to appropriate S3 error codes
- Standardize existence checks across bucket, namespace, and table handlers
2026-01-28 11:37:02 -08:00
Chris Lu da15ee3e49 s3tables: harden namespace validation and correct ARN parsing
- Prohibit path traversal (".", "..") and "/" in namespaces
- Restrict namespace characters to [a-z0-9_] for consistency
- Switch to url.PathUnescape for correct decoding of ARN path components
- Align ARN parsing regex with single-segment namespace validation
2026-01-28 11:36:56 -08:00
Chris Lu b4d8350936 s3tables: normalize filer errors and use standard helpers
- Migrate from custom ErrNotFound to filer_pb.ErrNotFound
- Use filer_pb.LookupEntry for automatic error normalization
- Normalize entryExists and attribute lookups
2026-01-28 11:36:49 -08:00
Chris Lu 1d1634c2a2 test: update integration tests to match refactored S3 Tables client
- Pass namespaces as []string to support hierarchical structures
- Adapt test calls to new client API signatures
2026-01-28 11:31:49 -08:00
Chris Lu 44f580c24e test: refactor S3 Tables client for DRYness and multi-segment namespaces
- Implement doRequestAndDecode to eliminate HTTP boilerplate
- Update client API to accept []string for namespaces to support hierarchy
- Standardize error response decoding across all client methods
2026-01-28 11:31:44 -08:00
Chris Lu 04514071a7 s3tables: implement granular authorization and refine error responses
- Remove mandatory ACTION_ADMIN at the router level
- Enforce granular permissions in bucket and namespace handlers
- Prioritize AccountID in ExtractPrincipalFromContext for ARN matching
- Distinguish between 404 (NoSuchBucket) and 500 (InternalError) in metadata lookups
- Clean up unused imports in s3api_tables.go
2026-01-28 11:31:38 -08:00
Chris Lu 580c2b4ad4 command: fix stale error variable logging in filer serving goroutines
- Use local 'err' variable instead of stale 'e' from outer scope
- Applied to both TLS and non-TLS paths for local listener
2026-01-28 11:27:18 -08:00
Chris Lu c62a332c7f s3tables: align ARN formatting and optimize resource handling
- Update generateTableARN to match AWS S3 Tables specification
- Move defer r.Body.Close() to follow standard Go patterns
- Remove unused generateNamespaceARN helper
2026-01-28 10:36:38 -08:00
Chris Lu 2c551dad5d s3tables: fix pagination and enhance error handling in list/delete operations
- Fix InclusiveStartFrom logic to ensure exclusive start on continued pages
- Prevent duplicates in bucket, namespace, and table listings
- Fail fast on listing errors during bucket and namespace deletion
- Stop swallowing errors in handleListTables and return proper HTTP error responses
2026-01-28 10:36:28 -08:00
Chris Lu 01c17478ae command: implement graceful shutdown for mini cluster
- Introduce MiniClusterCtx to coordinate shutdown across mini services
- Update Master, Volume, Filer, S3, and WebDAV servers to respect context cancellation
- Ensure all resources are cleaned up properly during test teardown
- Integrate MiniClusterCtx in s3tables integration tests
2026-01-28 10:36:19 -08:00
Chris Lu 07002cf54c test: improve S3 Tables client error handling and cleanup
- Add detailed error reporting when decoding failure responses
- Remove orphaned comments and unused sections
2026-01-28 10:36:11 -08:00
Chris Lu 3aace37cf6 s3tables: further refinements to filer operations and utilities
- Add multi-segment namespace support to ARN parsing
- Refactor permission checking to use map lookup
- Wrap lookup errors with ErrNotFound in filer operations
- Standardize splitPath to use path package
2026-01-28 10:36:03 -08:00
Chris Lu 33da87452b Refine S3 Tables implementation to address code review feedback
- Standardize namespace representation to []string
- Improve listing logic with pagination and StartFromFileName
- Enhance error handling with sentinel errors and robust checks
- Add JSON encoding error logging
- Fix CI workflow to use gofmt -l
- Standardize timestamps in directory creation
- Validate single-level namespaces
2026-01-28 10:04:27 -08:00
Chris Lu 08ee4e37d8 s3tables: clean up unused code and improve error response formatting 2026-01-28 09:38:10 -08:00
Chris Lu b30631c3b5 s3tables: propagate request context to filer operations 2026-01-28 09:38:01 -08:00
Chris Lu 6cdd34da77 s3tables: improve integration test stability and error reporting 2026-01-28 09:37:58 -08:00
Chris Lu b01504649d s3tables: use path.Join for path construction and align namespace paths 2026-01-28 09:37:54 -08:00
Chris Lu 24c78d524c ci: fail s3 tables tests if any command in pipeline fails 2026-01-28 09:37:51 -08:00
Chris Lu 05c184b610 workflow: fix go install path to ./weed 2026-01-28 01:35:24 -08:00
Chris Lu 33c1a8251a test: format s3tables client.go 2026-01-28 01:30:07 -08:00
Chris Lu 96a6e4c551 workflow: remove emojis from echo statements 2026-01-28 01:29:42 -08:00
Chris Lu f4e472d396 workflow: fix s3 tables tests path and working directory
The workflow was failing because it was running inside 'weed' directory,
but the tests are at the repository root. Removed working-directory
default and updated relative paths to weed source.
2026-01-28 01:28:47 -08:00
Chris Lu fb0e12e985 s3tables: implement permission checking and authorization
- Add permissions.go with permission definitions and checks
- Define permissions for all 21 S3 Tables operations
- Add permission checking helper functions
- Add getPrincipalFromRequest to extract caller identity
- Implement access control in CreateTableBucket, GetTableBucket, DeleteTableBucket
- Return 403 Forbidden for unauthorized operations
- Only bucket owner can perform operations (extensible for future policies)
- Add AuthError type for authorization failures
2026-01-28 01:18:11 -08:00
Chris Lu 1b9c8b8614 s3api_tables: optimize action validation with map lookup
- Replace O(n) slice iteration with O(1) map lookup
- Move s3TablesActionsMap to package level
- Avoid recreating the map on every function call
- Improves performance for request validation
2026-01-28 01:14:31 -08:00
Chris Lu 450407fda1 s3tables: improve error handling specificity in ListTableBuckets
- Specifically check for 'not found' errors instead of catching all errors
- Return empty list only when directory doesn't exist
- Propagate other errors (network, permission) with context
- Prevents masking real errors
2026-01-28 01:14:14 -08:00
Chris Lu b09d4d5d69 s3tables: replace custom splitPath with stdlib functions
- Remove custom splitPath implementation (23 lines)
- Use filepath.Dir and filepath.Base from stdlib
- More robust and handles edge cases correctly
- Reduces code duplication
2026-01-28 01:13:56 -08:00
Chris Lu ef3873b616 s3tables: add error handling for json.Marshal calls
- Add error handling in handler_namespace.go (metadata marshaling)
- Add error handling in handler_table.go (metadata and tags marshaling)
- Add error handling in handler_policy.go (tag marshaling in TagResource and UntagResource)
- Return proper errors with context instead of silently ignoring failures
2026-01-28 01:13:42 -08:00
Chris Lu dd7003a328 s3tables: add bucket name validation and fix error handling
- Add isValidBucketName validation function for [a-z0-9_-] characters
- Validate bucket name characters match ARN parsing regex
- Fix error handling in WithFilerClient closure - properly check for lookup errors
- Add error handling for json.Marshal calls (metadata and tags)
- Improve error messages and logging
2026-01-28 01:12:57 -08:00
Chris Lu 22a9fbf062 Delete client_methods.go 2026-01-28 01:07:53 -08:00
Chris Lu 06f90028f7 s3tables test: refactor to eliminate duplicate definitions
- Move all client methods to client.go
- Remove duplicate types/constants from s3tables_integration_test.go
- Keep setup.go for test infrastructure
- Keep integration test logic in s3tables_integration_test.go
- Clean up unused imports
- Test compiles successfully
2026-01-28 01:07:11 -08:00
Chris Lu 3b1920cf43 s3tables: add handler_ prefix to operation handler files
- Rename bucket_create.go → handler_bucket_create.go
- Rename bucket_get_list_delete.go → handler_bucket_get_list_delete.go
- Rename namespace.go → handler_namespace.go
- Rename table.go → handler_table.go
- Rename policy.go → handler_policy.go

Improves file organization by clearly identifying handler implementations.
No code changes, refactoring only.
2026-01-28 01:00:00 -08:00
Chris Lu 0dcb175514 ci: add S3 Tables integration tests to GitHub Actions
- Create new workflow for S3 Tables integration testing
- Add build verification job for s3tables package and s3api integration
- Add format checking for S3 Tables code
- Add go vet checks for code quality
- Workflow runs on all pull requests
- Includes test output logging and artifact upload on failure
2026-01-28 00:57:53 -08:00
Chris Lu ddcdefb6b4 test: add S3 Tables integration tests
- Comprehensive integration tests for all 23 S3 Tables operations
- Test cluster setup based on existing S3 integration tests
- Tests cover:
  * Table bucket lifecycle (create, get, list, delete)
  * Namespace operations
  * Table CRUD with Iceberg schema
  * Table bucket and table policies
  * Resource tagging operations
- Ready for CI/CD pipeline integration
2026-01-28 00:55:50 -08:00
Chris Lu d66e194284 test: add S3 Tables test infrastructure
- Create setup.go with TestCluster and S3TablesClient definitions
- Create client.go with HTTP client methods for all operations
- Test utilities and client methods organized for reusability
- Foundation for S3 Tables integration tests
2026-01-28 00:55:45 -08:00
Chris Lu 6a12438351 s3api: register S3 Tables routes in API server
- Add S3 Tables route registration in s3api_server.go registerRouter method
- Enable S3 Tables API operations to be routed through S3 API server
- Routes handled by s3api_tables.go integration layer
- Minimal changes to existing S3 API structure
2026-01-28 00:55:39 -08:00
Chris Lu b1b922d757 s3api: add S3 Tables integration layer
- Create s3api_tables.go to integrate S3 Tables with S3 API server
- Implement S3 Tables route matcher for X-Amz-Target header
- Register S3 Tables routes with API router
- Provide gRPC filer client interface for S3 Tables handlers
- All S3 Tables operations accessible via S3 API endpoint
2026-01-28 00:55:34 -08:00
Chris Lu 0be47f9efd s3tables: complete s3tables package implementation
- namespace.go: namespace CRUD operations (310 lines)
- table.go: table CRUD operations with Iceberg schema support (409 lines)
- policy.go: resource policies and tagging operations (419 lines)
- types.go: request/response types and error definitions (290 lines)
- All handlers updated to use standalone utilities from utils.go
- All files follow single responsibility principle
2026-01-28 00:55:28 -08:00
Chris Lu 62fd4bd017 s3tables: simplify handler by removing duplicate utilities
- Reduce handler.go from 370 to 195 lines (47% reduction)
- Remove duplicate ARN parsing and path helper functions
- Remove filer operation methods moved to filer_ops.go
- Remove metadata structure definitions moved to utils.go
- Keep handler focused on request routing and response formatting
- Maintains all functionality with improved code organization
2026-01-28 00:55:23 -08:00
Chris Lu 09bb90e8dc s3tables: split table bucket operations into focused modules
- Create bucket_create.go for CreateTableBucket operation
- Create bucket_get_list_delete.go for Get, List, Delete operations
- Related operations grouped for better maintainability
- Each file has a single, clear responsibility
- Improves code clarity and makes it easier to test
2026-01-28 00:55:17 -08:00
Chris Lu d730f81321 s3tables: extract utility and filer operations to separate modules
- Move ARN parsing, path helpers, and metadata structures to utils.go
- Extract all extended attribute and filer operations to filer_ops.go
- Reduces code duplication and improves modularity
- Improves code organization and maintainability
2026-01-28 00:55:11 -08:00
Chris LuGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
6542d1e0aa Enable weed fuse on FreeBSD (#8146)
* Enable weed fuse on FreeBSD

* Update weed/command/fuse_notsupported.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update weed/command/fuse_std.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-01-27 21:37:23 -08:00
Chris LuandGitHub e86e65e5ab fix #8081: build latest container is missing latest_large_disk (#8145)
* fix #8081: build latest container is missing latest_large_disk

* fix: simplify QEMU setup condition in container_latest.yml matrix
2026-01-27 19:05:29 -08:00
Chris LuandGitHub 8e9c12e2c7 mount: apply UID/GID mapping in lookupEntry for cache misses (#8144)
* mount: apply UID/GID mapping in lookupEntry for cache misses

This fixes issue #8134 where rsync would fail with "Operation not permitted"
during chgrp. The issue was that entries fetched directly from the filer
(on cache miss) were not being mapped to local UIDs/GIDs.

* mount: add nil check for entry.Attributes in lookupEntry
2026-01-27 18:41:37 -08:00
Chris LuandGitHub 4f5f1f6be7 refactor(helm): Unified Naming Truncation and Bug Fixes (#8143)
* refactor(helm): add componentName helper for truncation

* fix(helm): unify ingress backend naming with truncation

* fix(helm): unify statefulset/deployment naming with truncation

* fix(helm): add missing labels to services for servicemonitor discovery

* chore(helm): secure secrets and add upgrade notes

* fix(helm): truncate context instead of suffix in componentName

* revert(docs): remove upgrade notes per feedback

* fix(helm): use componentName for COSI serviceAccountName

* helm: update master -ip to use component name for correct truncation

* helm: refactor masterServers helper to use truncated component names

* helm: update volume -ip to use component name and cleanup redundant printf

* helm: refine helpers with robustness check and updated docs
2026-01-27 17:45:16 -08:00
20952aa514 Fix jwt error in admin UI (#8140)
* add jwt token in weed admin headers requests

* add jwt token to header for download

* :s/upload/download

* filer_signing.read despite of filer_signing key

* finalize filer_browser_handlers.go

* admin: add JWT authorization to file browser handlers

* security: fix typos in JWT read validation descriptions

* Move security.toml to example and secure keys

* security: address PR feedback on JWT enforcement and example keys

* security: refactor JWT logic and improve example keys readability

* Update docker/Dockerfile.local

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Chris Lu <chris.lu@gmail.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-27 17:27:02 -08:00
Chris LuandGitHub c9c91ba568 Refactor Helm chart to use dynamic names for resources (#8142)
* Refactor Helm chart to use dynamic names for resources

* ensure name length
2026-01-27 12:52:06 -08:00
Alasdair MacmillanandGitHub 41d079a316 Fix Javascript merge issue and UI worker detail display bug (#8135)
* Fix previous merge issues in Javascript

Signed-off-by: Alasdair Macmillan <aimmac23@gmail.com>

* Fix issue where worker detail doesn't display without tasks

---------

Signed-off-by: Alasdair Macmillan <aimmac23@gmail.com>
2026-01-27 10:56:28 -08:00
Andrei KvapilGitHubClaudegemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Chris Lugemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
b1d63d0943 feat(shell): add Object Lock management commands (#8141)
* feat(shell): add s3.bucket.lock command for Object Lock management

Add new weed shell command to view and enable S3 Object Lock on existing
buckets. This allows administrators to enable Object Lock without
recreating buckets, which is useful when buckets already contain data.

The command:
- Shows current Object Lock and Versioning status
- Enables Object Lock with -enable flag (irreversible, per AWS S3 spec)
- Automatically enables Versioning if not already enabled (required for Object Lock)

Usage:
  s3.bucket.lock -name <bucket>          # view status
  s3.bucket.lock -name <bucket> -enable  # enable Object Lock

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>

* feat(shell): add -withLock flag to s3.bucket.create command

Add support for creating buckets with Object Lock enabled directly from
weed shell. The flag automatically enables versioning as required by
Object Lock.

Usage:
  s3.bucket.create -name mybucket -withLock

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>

* Apply suggestion from @gemini-code-assist[bot]

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-01-27 10:50:16 -08:00
Chris Lu 92800c31a2 adjust logs and errors 2026-01-27 07:45:24 -08:00
Chris LuandGitHub 551a31e156 Implement IAM propagation to S3 servers (#8130)
* Implement IAM propagation to S3 servers

- Add PropagatingCredentialStore to propagate IAM changes to S3 servers via gRPC
- Add Policy management RPCs to S3 proto and S3ApiServer
- Update CredentialManager to use PropagatingCredentialStore when MasterClient is available
- Wire FilerServer to enable propagation

* Implement parallel IAM propagation and fix S3 cluster registration

- Parallelized IAM change propagation with 10s timeout.
- Refined context usage in PropagatingCredentialStore.
- Added S3Type support to cluster node management.
- Enabled S3 servers to register with gRPC address to the master.
- Ensured IAM configuration reload after policy updates via gRPC.

* Optimize IAM propagation with direct in-memory cache updates

* Secure IAM propagation: Use metadata to skip persistence only on propagation

* pb: refactor IAM and S3 services for unidirectional IAM propagation

- Move SeaweedS3IamCache service from iam.proto to s3.proto.
- Remove legacy IAM management RPCs and empty SeaweedS3 service from s3.proto.
- Enforce that S3 servers only use the synchronization interface.

* pb: regenerate Go code for IAM and S3 services

Updated generated code following the proto refactoring of IAM synchronization services.

* s3api: implement read-only mode for Embedded IAM API

- Add readOnly flag to EmbeddedIamApi to reject write operations via HTTP.
- Enable read-only mode by default in S3ApiServer.
- Handle AccessDenied error in writeIamErrorResponse.
- Embed SeaweedS3IamCacheServer in S3ApiServer.

* credential: refactor PropagatingCredentialStore for unidirectional IAM flow

- Update to use s3_pb.SeaweedS3IamCacheClient for propagation to S3 servers.
- Propagate full Identity object via PutIdentity for consistency.
- Remove redundant propagation of specific user/account/policy management RPCs.
- Add timeout context for propagation calls.

* s3api: implement SeaweedS3IamCacheServer for unidirectional sync

- Update S3ApiServer to implement the cache synchronization gRPC interface.
- Methods (PutIdentity, RemoveIdentity, etc.) now perform direct in-memory cache updates.
- Register SeaweedS3IamCacheServer in command/s3.go.
- Remove registration for the legacy and now empty SeaweedS3 service.

* s3api: update tests for read-only IAM and propagation

- Added TestEmbeddedIamReadOnly to verify rejection of write operations in read-only mode.
- Update test setup to pass readOnly=false to NewEmbeddedIamApi in routing tests.
- Updated EmbeddedIamApiForTest helper with read-only checks matching production behavior.

* s3api: add back temporary debug logs for IAM updates

Log IAM updates received via:
- gRPC propagation (PutIdentity, PutPolicy, etc.)
- Metadata configuration reloads (LoadS3ApiConfigurationFromCredentialManager)
- Core identity management (UpsertIdentity, RemoveIdentity)

* IAM: finalize propagation fix with reduced logging and clarified architecture

* Allow configuring IAM read-only mode for S3 server integration tests

* s3api: add defensive validation to UpsertIdentity

* s3api: fix log message to reference correct IAM read-only flag

* test/s3/iam: ensure WaitForS3Service checks for IAM write permissions

* test: enable writable IAM in Makefile for integration tests

* IAM: add GetPolicy/ListPolicies RPCs to s3.proto

* S3: add GetBucketPolicy and ListBucketPolicies helpers

* S3: support storing generic IAM policies in IdentityAccessManagement

* S3: implement IAM policy RPCs using IdentityAccessManagement

* IAM: fix stale user identity on rename propagation
2026-01-26 22:59:43 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
0a6b289025 chore(deps-dev): bump org.assertj:assertj-core from 3.24.2 to 3.27.7 in /other/java/s3copier (#8129)
chore(deps-dev): bump org.assertj:assertj-core in /other/java/s3copier

Bumps [org.assertj:assertj-core](https://github.com/assertj/assertj) from 3.24.2 to 3.27.7.
- [Release notes](https://github.com/assertj/assertj/releases)
- [Commits](https://github.com/assertj/assertj/compare/assertj-build-3.24.2...assertj-build-3.27.7)

---
updated-dependencies:
- dependency-name: org.assertj:assertj-core
  dependency-version: 3.27.7
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-26 14:45:01 -08:00
Chris LuandGitHub a72e269a2e s3api: fix volume assignment failure in copy operations (#8128)
Recent changes in the S3 unified copy strategy were constructing source
and destination paths without the necessary BucketsPath prefix (typically
/buckets). When these paths reached the Filer for volume assignment, it
failed to resolve the correct collection and storage rules, defaulting to
a disk type with no available capacity.

This fix ensures all relevant paths in S3 copy handlers include the
correct BucketsPath prefix for proper collection resolution.

Fixes replication issue with Harbor upload via S3 API.
2026-01-26 14:44:30 -08:00
Chris LuandGitHub 43229b05ce Explicit IAM gRPC APIs for S3 Server (#8126)
* Update IAM and S3 protobuf definitions for explicit IAM gRPC APIs

* Refactor s3api: Extract generic ExecuteAction method for IAM operations

* Implement explicit IAM gRPC APIs in S3 server

* iam: remove deprecated GetConfiguration and PutConfiguration RPCs

* iamapi: refactor handlers to use CredentialManager directly

* s3api: refactor embedded IAM to use CredentialManager directly

* server: remove deprecated configuration gRPC handlers

* credential/grpc: refactor configuration calls to return error

* shell: update s3.configure to list users instead of full config

* s3api: fix CreateServiceAccount gRPC handler to map required fields

* s3api: fix UpdateServiceAccount gRPC handler to map fields and safe status

* s3api: enforce UserName in embedded IAM ListAccessKeys

* test: fix test_config.json structure to match proto definition

* Revert "credential/grpc: refactor configuration calls to return error"

This reverts commit cde707dd8b.

* Revert "server: remove deprecated configuration gRPC handlers"

This reverts commit 7307e205a0.

* Revert "s3api: enforce UserName in embedded IAM ListAccessKeys"

This reverts commit adf727ba52.

* Revert "s3api: fix UpdateServiceAccount gRPC handler to map fields and safe status"

This reverts commit 6a4be3314d.

* Revert "s3api: fix CreateServiceAccount gRPC handler to map required fields"

This reverts commit 9bb4425f07.

* Revert "shell: update s3.configure to list users instead of full config"

This reverts commit f3304ead53.

* Revert "s3api: refactor embedded IAM to use CredentialManager directly"

This reverts commit 9012f27af8.

* Revert "iamapi: refactor handlers to use CredentialManager directly"

This reverts commit 3a14821223.

* Revert "iam: remove deprecated GetConfiguration and PutConfiguration RPCs"

This reverts commit e16e08aa00.

* s3api: address IAM code review comments (error handling, logging, gRPC response mapping)

* s3api: add robustness to startup by retrying KEK and IAM config loading from Filer

* s3api: address IAM gRPC code review comments (safety, validation, status logic)

* fix return
2026-01-26 13:38:15 -08:00
Chris Lu c5b53397c6 templ 2026-01-26 11:38:18 -08:00
Chris Lu 5ba0db7af4 Merge branch 'origin/master' into master
Resolved merge conflicts in:
- weed/admin/static/js/modal-alerts.js: Adopted incoming improvements and HTML support.
- weed/admin/view/app/collection_details.templ: Switched to showAlert info type.
- weed/admin/view/app/file_browser.templ: Used descriptive delete message.
- weed/admin/view/app/maintenance_workers.templ: Used encoding and headers in pauseWorker.
- weed/admin/view/app/object_store_users.templ: Restored accidentally deleted delete functions and used encodeURIComponent.
- weed/admin/view/app/policies.templ: Standardized on showAlert and descriptive confirmations.

Regenerated all templ files.
2026-01-26 11:37:12 -08:00
Chris LuandGitHub 5a7c74feac migrate IAM policies to multi-file storage (#8114)
* Add IAM gRPC service definition

- Add GetConfiguration/PutConfiguration for config management
- Add CreateUser/GetUser/UpdateUser/DeleteUser/ListUsers for user management
- Add CreateAccessKey/DeleteAccessKey/GetUserByAccessKey for access key management
- Methods mirror existing IAM HTTP API functionality

* Add IAM gRPC handlers on filer server

- Implement IamGrpcServer with CredentialManager integration
- Handle configuration get/put operations
- Handle user CRUD operations
- Handle access key create/delete operations
- All methods delegate to CredentialManager for actual storage

* Wire IAM gRPC service to filer server

- Add CredentialManager field to FilerOption and FilerServer
- Import credential store implementations in filer command
- Initialize CredentialManager from credential.toml if available
- Register IAM gRPC service on filer gRPC server
- Enable credential management via gRPC alongside existing filer services

* Regenerate IAM protobuf with gRPC service methods

* fix: compilation error in DeleteUser

* fix: address code review comments for IAM migration

* feat: migrate policies to multi-file layout and fix identity duplicated content

* refactor: remove configuration.json and migrate Service Accounts to multi-file layout

* refactor: standardize Service Accounts as distinct store entities and fix Admin Server persistence

* config: set ServiceAccountsDirectory to /etc/iam/service_accounts

* Fix Chrome dialog auto-dismiss with Bootstrap modals

- Add modal-alerts.js library with Bootstrap modal replacements
- Replace all 15 confirm() calls with showConfirm/showDeleteConfirm
- Auto-override window.alert() for all alert() calls
- Fixes Chrome 132+ aggressively blocking native dialogs

* Upgrade Bootstrap from 5.3.2 to 5.3.8

* Fix syntax error in object_store_users.templ - remove duplicate closing braces

* create policy

* display errors

* migrate to multi-file policies

* address PR feedback: use showDeleteConfirm and showErrorMessage in policies.templ, refine migration check

* Update policies_templ.go

* add service account to iam grpc

* iam: fix potential path traversal in policy names by validating name pattern

* iam: add GetServiceAccountByAccessKey to CredentialStore interface

* iam: implement service account support for PostgresStore

Includes full CRUD operations and efficient lookup by access key.

* iam: implement GetServiceAccountByAccessKey for filer_etc, grpc, and memory stores

Provides efficient lookup of service accounts by access key where possible,
with linear scan fallbacks for file-based stores.

* iam: remove filer_multiple support

Deleted its implementation and references in imports, scaffold config,
and core interface constants. Redundant with filer_etc.

* clear comment

* dash: robustify service account construction

- Guard against nil sa.Credential when constructing responses
- Fix Expiration logic to only set if > 0, avoiding Unix epoch 1970
- Ensure consistency across Get, Create, and Update handlers

* credential/filer_etc: improve error propagation in configuration handlers

- Return error from loadServiceAccountsFromMultiFile to callers
- Ensure listEntries errors in SaveConfiguration (cleanup logic) are
  propagated unless they are "not found" failures.
- Fixes potential silent failures during IAM configuration sync.

* credential/filer_etc: add existence check to CreateServiceAccount

Ensures consistency with other stores by preventing accidental overwrite
of existing service accounts during creation.

* credential/memory: improve store robustness and Reset logic

- Enforce ID immutability in UpdateServiceAccount to prevent orphans
- Update Reset() to also clear the policies map, ensuring full state
  cleanup for tests.

* dash: improve service account robustness and policy docs

- Wrap parent user lookup errors to preserve context
- Strictly validate Status field in UpdateServiceAccount
- Add deprecation comments to legacy policy management methods

* credential/filer_etc: protect against path traversal in service accounts

Implemented ID validation (alphanumeric, underscores, hyphens) and applied
it to Get, Save, and Delete operations to ensure no directory traversal
via saId.json filenames.

* credential/postgres: improve robustness and cleanup comments

- Removed brainstorming comments in GetServiceAccountByAccessKey
- Added missing rows.Err() check during iteration
- Properly propagate Scan and Unmarshal errors instead of swallowing them

* admin: unify UI alerts and confirmations using Bootstrap modals

- Updated modal-alerts.js with improved automated alert type detection
- Replaced native alert() and confirm() with showAlert(), showConfirm(),
  and showDeleteConfirm() across various Templ components
- Improved UX for delete operations by providing better context and styling
- Ensured consistent error reporting across IAM and Maintenance views

* admin: additional UI consistency fixes for alerts and confirmations

- Replaced native alert() and confirm() with Bootstrap modals in:
  - EC volumes (repair flow)
  - Collection details (repair flow)
  - File browser (properties and delete)
  - Maintenance config schema (save and reset)
- Improved delete confirmation in file browser with item context
- Ensured consistent success/error/info styling for all feedbacks

* make

* iam: add GetServiceAccountByAccessKey RPC and update GetConfiguration

* iam: implement GetServiceAccountByAccessKey on server and client

* iam: centralize policy and service account validation

* iam: optimize MemoryStore service account lookups with indexing

* iam: fix postgres service_accounts table and optimize lookups

* admin: refactor modal alerts and clean up dashboard logic

* admin: fix EC shards table layout mismatch

* admin: URL-encode IAM path parameters for safety

* admin: implement pauseWorker logic in maintenance view

* iam: add rows.Err() check to postgres ListServiceAccounts

* iam: standardize ErrServiceAccountNotFound across credential stores

* iam: map ErrServiceAccountNotFound to codes.NotFound in DeleteServiceAccount

* iam: refine service account store logic, errors and schema

* iam: add validation to GetServiceAccountByAccessKey

* admin: refine modal titles and ensure URL safety

* admin: address bot review comments for alerts and async usage

* iam: fix syntax error by restoring missing function declaration

* [FilerEtcStore] improve error handling in CreateServiceAccount

Refine error handling to provide clearer messages when checking for
existing service accounts.

* [PostgresStore] add nil guards and validation to service account methods

Ensure input parameters are not nil and required IDs are present
to prevent runtime panics and ensure data integrity.

* [JS] add shared IAM utility script

Consolidate common IAM operations like deleteUser and deleteAccessKey
into a shared utility script for better maintainability.

* [View] include shared IAM utilities in layout

Include iam-utils.js in the main layout to make IAM functions
available across all administrative pages.

* [View] refactor IAM logic and restore async in EC Shards view

Remove redundant local IAM functions and ensure that delete
confirmation callbacks are properly marked as async.

* [View] consolidate IAM logic in Object Store Users view

Remove redundant local definitions of deleteUser and deleteAccessKey,
relying on the shared utilities instead.

* [View] update generated templ files for UI consistency

* credential/postgres: remove redundant name column from service_accounts table

The id is already used as the unique identifier and was being copied to the name column.
This removes the name column from the schema and updates the INSERT/UPDATE queries.

* credential/filer_etc: improve logging for policy migration failures

Added Errorf log if AtomicRenameEntry fails during migration to ensure visibility of common failure points.

* credential: allow uppercase characters in service account ID username

Updated ServiceAccountIdPattern to allow [A-Za-z0-9_-]+ for the username component,
matching the actual service account creation logic which uses the parent user name directly.

* Update object_store_users_templ.go

* admin: fix ec_shards pagination to handle numeric page arguments

Updated goToPage in cluster_ec_shards.templ to accept either an Event
or a numeric page argument. This prevents errors when goToPage(1)
is called directly. Corrected both the .templ source and generated Go code.

* credential/filer_etc: improve service account storage robustness

Added nil guard to saveServiceAccount, updated GetServiceAccount
to return ErrServiceAccountNotFound for empty data, and improved
deleteServiceAccount to handle response-level Filer errors.
2026-01-26 11:28:23 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
a29806d752 chore(deps): bump github.com/jackc/pgx/v5 from 5.7.6 to 5.8.0 (#8118)
Bumps [github.com/jackc/pgx/v5](https://github.com/jackc/pgx) from 5.7.6 to 5.8.0.
- [Changelog](https://github.com/jackc/pgx/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jackc/pgx/compare/v5.7.6...v5.8.0)

---
updated-dependencies:
- dependency-name: github.com/jackc/pgx/v5
  dependency-version: 5.8.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-26 11:17:00 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
9ca43c452b chore(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore from 1.20.0 to 1.21.0 (#8119)
chore(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore

Bumps [github.com/Azure/azure-sdk-for-go/sdk/azcore](https://github.com/Azure/azure-sdk-for-go) from 1.20.0 to 1.21.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.20.0...sdk/azcore/v1.21.0)

---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azcore
  dependency-version: 1.21.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-26 11:16:52 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
06920f79f9 chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.125.1 to 3.125.3 (#8120)
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3

Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk) from 3.125.1 to 3.125.3.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases)
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.125.1...v3.125.3)

---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
  dependency-version: 3.125.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-26 11:16:44 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
b502411884 chore(deps): bump actions/checkout from 4 to 6 (#8121)
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 6.
- [Release notes](https://github.com/actions/checkout/releases)
- [Commits](https://github.com/actions/checkout/compare/v4...v6)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-26 11:16:37 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
16dc90e3bd chore(deps): bump actions/setup-go from 5 to 6 (#8124)
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5 to 6.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-26 11:16:28 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
6714973ffe chore(deps): bump actions/upload-artifact from 4 to 6 (#8125)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4 to 6.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v4...v6)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-26 11:16:19 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1e5a1871e5 chore(deps): bump modernc.org/sqlite from 1.44.2 to 1.44.3 (#8122)
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.44.2 to 1.44.3.
- [Changelog](https://gitlab.com/cznic/sqlite/blob/master/CHANGELOG.md)
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.44.2...v1.44.3)

---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-version: 1.44.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-26 11:16:10 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
db6b4ab918 chore(deps): bump github.com/a-h/templ from 0.3.943 to 0.3.977 (#8123)
Bumps [github.com/a-h/templ](https://github.com/a-h/templ) from 0.3.943 to 0.3.977.
- [Release notes](https://github.com/a-h/templ/releases)
- [Commits](https://github.com/a-h/templ/compare/v0.3.943...v0.3.977)

---
updated-dependencies:
- dependency-name: github.com/a-h/templ
  dependency-version: 0.3.977
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-26 11:16:02 -08:00
Chris Lu 759a6cd345 Merge branch 'fix/chrome-dialog-modal-approach' 2026-01-25 23:21:23 -08:00
Chris Lu 7e3bb4016e Fix syntax error in object_store_users.templ - remove duplicate closing braces 2026-01-25 23:18:40 -08:00
Chris Lu 1e09950ea7 Upgrade Bootstrap from 5.3.2 to 5.3.8 2026-01-25 23:12:27 -08:00
Chris Lu 74c7b10bc7 Fix Chrome dialog auto-dismiss with Bootstrap modals
- Add modal-alerts.js library with Bootstrap modal replacements
- Replace all 15 confirm() calls with showConfirm/showDeleteConfirm
- Auto-override window.alert() for all alert() calls
- Fixes Chrome 132+ aggressively blocking native dialogs
2026-01-25 23:09:14 -08:00
Chris Lu 31a1c87db0 fix compilation 2026-01-25 22:07:12 -08:00
Chris Lu 533650579d credential/filer_etc: improve migration merge logic 2026-01-25 13:53:48 -08:00
Chris Lu 74b5c57dcf credential/filer_etc: migrate to multi-file identity storage 2026-01-25 13:48:46 -08:00
Chris LuandGitHub 6bf088cec9 IAM Policy Management via gRPC (#8109)
* Add IAM gRPC service definition

- Add GetConfiguration/PutConfiguration for config management
- Add CreateUser/GetUser/UpdateUser/DeleteUser/ListUsers for user management
- Add CreateAccessKey/DeleteAccessKey/GetUserByAccessKey for access key management
- Methods mirror existing IAM HTTP API functionality

* Add IAM gRPC handlers on filer server

- Implement IamGrpcServer with CredentialManager integration
- Handle configuration get/put operations
- Handle user CRUD operations
- Handle access key create/delete operations
- All methods delegate to CredentialManager for actual storage

* Wire IAM gRPC service to filer server

- Add CredentialManager field to FilerOption and FilerServer
- Import credential store implementations in filer command
- Initialize CredentialManager from credential.toml if available
- Register IAM gRPC service on filer gRPC server
- Enable credential management via gRPC alongside existing filer services

* Regenerate IAM protobuf with gRPC service methods

* iam_pb: add Policy Management to protobuf definitions

* credential: implement PolicyManager in credential stores

* filer: implement IAM Policy Management RPCs

* shell: add s3.policy command

* test: add integration test for s3.policy

* test: fix compilation errors in policy_test

* pb

* fmt

* test

* weed shell: add -policies flag to s3.configure

This allows linking/unlinking IAM policies to/from identities
directly from the s3.configure command.

* test: verify s3.configure policy linking and fix port allocation

- Added test case for linking policies to users via s3.configure
- Implemented findAvailablePortPair to ensure HTTP and gRPC ports
  are both available, avoiding conflicts with randomized port assignments.
- Updated assertion to match jsonpb output (policyNames)

* credential: add StoreTypeGrpc constant

* credential: add IAM gRPC store boilerplate

* credential: implement identity methods in gRPC store

* credential: implement policy methods in gRPC store

* admin: use gRPC credential store for AdminServer

This ensures that all IAM and policy changes made through the Admin UI
are persisted via the Filer's IAM gRPC service instead of direct file manipulation.

* shell: s3.configure use granular IAM gRPC APIs instead of full config patching

* shell: s3.configure use granular IAM gRPC APIs

* shell: replace deprecated ioutil with os in s3.policy

* filer: use gRPC FailedPrecondition for unconfigured credential manager

* test: improve s3.policy integration tests and fix error checks

* ci: add s3 policy shell integration tests to github workflow

* filer: fix LoadCredentialConfiguration error handling

* credential/grpc: propagate unmarshal errors in GetPolicies

* filer/grpc: improve error handling and validation

* shell: use gRPC status codes in s3.configure

* credential: document PutPolicy as create-or-replace

* credential/postgres: reuse CreatePolicy in PutPolicy to deduplicate logic

* shell: add timeout context and strictly enforce flags in s3.policy

* iam: standardize policy content field naming in gRPC and proto

* shell: extract slice helper functions in s3.configure

* filer: map credential store errors to gRPC status codes

* filer: add input validation for UpdateUser and CreateAccessKey

* iam: improve validation in policy and config handlers

* filer: ensure IAM service registration by defaulting credential manager

* credential: add GetStoreName method to manager

* test: verify policy deletion in integration test
2026-01-25 13:39:30 -08:00
Lisandro PinandGitHub 59d40f7186 Return volume server state flags via VolumeServerStatus() RPCs. (#8016) 2026-01-24 21:45:23 -08:00
Chris Lu 5a0c761976 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2026-01-24 13:43:28 -08:00
Chris Lu b40551f960 helm: seaweedfs admin should not allow setting multiple admin servers 2026-01-24 13:43:08 -08:00
Chris LuandGitHub 8814c2a07d iam: support ForAnyValue and ForAllValues condition set operators (#8105)
* iam: support ForAnyValue and ForAllValues condition set operators

This implementation adds support for AWS-style IAM condition set operators
`ForAnyValue:` and `ForAllValues:`. These are essential for trust policies
that evaluate collection-based claims like `oidc:roles` or groups.

- Updated EvaluateStringCondition to handle set operators.
- Added set operator support to numeric, date, and boolean conditions.
- ForAnyValue matches if any request value matches any condition value (default).
- ForAllValues matches if every request value matches at least one condition value.

* iam: add test suite for condition set operators

* iam: ensure ForAllValues is vacuously true for all condition types

Aligned Numeric, Date, and Boolean conditions with AWS IAM behavior
where ForAllValues returns true when the request context values are empty.

* iam: add Date vacuously true test case for ForAllValues

* iam: expand policy variables in case-insensitive string conditions

Added expandPolicyVariables support to evaluateStringConditionIgnoreCase
to ensure consistency with case-sensitive counterparts.

* iam: fix negation issues in string set operators

Refactored EvaluateStringCondition and evaluateStringConditionIgnoreCase
to evaluate operators (including negation) per context value before
aggregating. This ensures StringNotEquals and StringNotLike work
correctly with ForAllValues and ForAnyValue.

* iam: add []string support for Date and Boolean context values

Ensures consistency with Numeric conditions by allowing context values
to be provided as slices of strings, which is common in JSON/OIDC claims.

* iam: simplify redundant type check in policy engine

The `evaluateStringConditionIgnoreCase` function had a redundant type
check for `string` in the `default` block of a type switch that
already handled the `string` case.

* iam: remove outdated "currently fails" comment in negation tests

* iam: add StringLikeIgnoreCase condition support

* iam: explicitly handle empty context sets for ForAnyValue

AWS IAM treats empty request sets as "no match" for ForAnyValue.
Added an explicit check and comment to make this behavior clear.

* iam: refactor EvaluateStringCondition to expand policy variables once

Avoid redundant calls to expandPolicyVariables by expanding them once
per condition value instead of inside awsIAMMatch or in the exact
matching branch.

* iam: fix StringLike case sensitivity to match AWS IAM specs

StringLike and StringNotLike condition operators are case-sensitive in
AWS IAM. Changed the implementation to use filepath.Match for
case-sensitive wildcard matching instead of the case-insensitive
awsIAMMatch.

* iam: integrate StringLike case-sensitivity test into suite

Integrated the case-sensitivity verification into condition_set_test.go
and updated the consistency test to use StringLikeIgnoreCase to maintain
its case-insensitive matching verification.

* iam: fix NumericNotEquals logic to follow "not equal to any" semantics

Updated evaluateNumericCondition to correctly handle NumericNotEquals by
ensuring a context value matches only if it is not equal to ANY of the
provided expected values. Also added support for []string expected
values.

* iam: fix DateNotEquals logic and integrate tests

Updated evaluateDateCondition to correctly handle DateNotEquals logic.
Integrated the new test cases for NumericNotEquals and DateNotEquals into
condition_set_test.go.

* iam: fix validation error in integrated NotEquals tests

Added missing Resource field to IAM policy statements in
condition_set_test.go to satisfy validation requirements.

* iam: add set operator support for IP and Null conditions

Implemented ForAllValues and ForAnyValue support for IpAddress,
NotIpAddress, and Null condition operators. Also added test coverage for
ForAnyValue with an empty context to ensure correct behavior.

* iam: refine IP condition evaluation to handle multiple policy value types

Updated evaluateIPCondition to correctly handle string, []string, and
[]interface{} values for IP address conditions in policy documents.
Added IpAddress:SingleStringValue test case to verify consistency.

* iam: refine Null and case-insensitive string conditions

- Reverted evaluateNullCondition to standard AWS behavior (no set operators).
- Refactored evaluateStringConditionIgnoreCase to use idiomatic helpers
  (strings.EqualFold and AwsWildcardMatch).
- Cleaned up tests in condition_set_test.go.

* iam: normalize policy value handling across condition evaluators

- Implemented normalizeRanges helper for consistent IP range extraction.
- Expanded type switches in IP, Bool, and String condition evaluators to
  support string, []string, and []interface{} policy values.
- Fixed ForAnyValue bool matching to support string slices.
- Added targeted tests for []string policy values in condition_set_test.go.

* iam: refactor IP condition to support arbitrary context keys

Refactored evaluateIPCondition to iterate through all keys in the
condition block instead of hardcoding aws:SourceIp. This ensures
consistency with other condition types and allows custom context keys.
Added IpAddress:CustomContextKey test case to verify the change.
2026-01-24 13:34:49 -08:00
Chris Lu d3f79d4c38 Update detection.go 2026-01-23 21:38:51 -08:00
Chris LuandGitHub 6394e2f6a5 Fix IAM OIDC role mapping and OIDC claims in trust policy (#8104)
* Fix IAM OIDC role mapping and OIDC claims in trust policy

* Address PR review: Add config safety checks and refactor tests
2026-01-23 21:35:26 -08:00
Chris Lu 57a16b0b87 Improve error handling in GetObjectStoreUsers per PR review 2026-01-23 20:34:39 -08:00
Chris Lu e559b8df37 Refactor Admin UI to use unified IAM storage and add Shutdown hook 2026-01-23 20:29:21 -08:00
Chris LuandGitHub 81009c1a81 Refactor IAM Storage: Multi-File Backend & Unified Interface (#8102)
Refactor IAM Shutdown to use sync.Once for thread safety
2026-01-23 20:27:22 -08:00
Chris LuandGitHub f6318edbc9 Refactor Admin UI to use unified IAM storage and add MultipleFileStore (#8101)
* Refactor Admin UI to use unified IAM storage and add MultipleFileStore

* Address PR feedback: fix renames, error handling, and sync logic in FilerMultipleStore

* Address refined PR feedback: safe rename order, rollback logic, and structural sync refinement

* Optimize LoadConfiguration: use streaming callback for memory efficiency

* Refactor UpdateUser: log rollback failures during rename

* Implement PolicyManager for FilerMultipleStore

* include the filer_multiple backend configuration

* Implement cross-S3 synchronization and proper shutdown for all IAM backends

* Extract Admin UI refactoring to a separate PR
2026-01-23 20:12:59 -08:00
Chris LuandGitHub 535be3096b Add AWS IAM integration tests and refactor admin authorization (#8098)
* Add AWS IAM integration tests and refactor admin authorization
- Added AWS IAM management integration tests (User, AccessKey, Policy)
- Updated test framework to support IAM client creation with JWT/OIDC
- Refactored s3api authorization to be policy-driven for IAM actions
- Removed hardcoded role name checks for admin privileges
- Added new tests to GitHub Actions basic test matrix

* test(s3/iam): add UpdateUser and UpdateAccessKey tests and fix nil pointer dereference

* feat(s3api): add DeletePolicy and update tests with cleanup logic

* test(s3/iam): use t.Cleanup for managed policy deletion in CreatePolicy test
2026-01-23 16:41:51 -08:00
Chris Lu 25a4691135 Update store_ec_recovery_test.go 2026-01-23 16:38:36 -08:00
Chris LuandGitHub d664ca5ed3 fix: IAM authentication with AWS Signature V4 and environment credentials (#8099)
* fix: IAM authentication with AWS Signature V4 and environment credentials

Three key fixes for authenticated IAM requests to work:

1. Fix request body consumption before signature verification
   - iamMatcher was calling r.ParseForm() which consumed POST body
   - This broke AWS Signature V4 verification on subsequent reads
   - Now only check query string in matcher, preserving body for verification
   - File: weed/s3api/s3api_server.go

2. Preserve environment variable credentials across config reloads
   - After IAM mutations, config reload overwrote env var credentials
   - Extract env var loading into loadEnvironmentVariableCredentials()
   - Call after every config reload to persist credentials
   - File: weed/s3api/auth_credentials.go

3. Add authenticated IAM tests and test infrastructure
   - New TestIAMAuthenticated suite with AWS SDK + Signature V4
   - Dynamic port allocation for independent test execution
   - Flag reset to prevent state leakage between tests
   - CI workflow to run S3 and IAM tests separately
   - Files: test/s3/example/*, .github/workflows/s3-example-integration-tests.yml

All tests pass:
- TestIAMCreateUser (unauthenticated)
- TestIAMAuthenticated (with AWS Signature V4)
- S3 integration tests

* fmt

* chore: rename test/s3/example to test/s3/normal

* simplify: CI runs all integration tests in single job

* Update s3-example-integration-tests.yml

* ci: run each test group separately to avoid raft registry conflicts
2026-01-23 16:27:42 -08:00
Chris Lu afbe52f262 cleanup 2026-01-23 12:46:10 -08:00
Chris Lu 67b35757d7 Delete reproduce_issue_s3.sh 2026-01-23 12:35:23 -08:00
Chris LuandGitHub b203ed4124 Fix imbalance detection disk type grouping and volume grow errors (#8097)
* Fix imbalance detection disk type grouping and volume grow errors

This PR addresses two issues:

1. Imbalance Detection: Previously, balance detection did not verify disk types, leading to false positives when comparing heterogenous nodes (e.g. SSD vs HDD). Logic is now updated to group volumes by DiskType before calculating imbalance.
2. Volume Grow Errors: Fixed a variable scope issue in master_grpc_server_volume.go and added a pre-check for available space to prevent 'only 0 volumes left' error logs when a disk type is full or abandoned.

Included units tests for the detection logic.

* Refactor balance detection loop into detectForDiskType

* Fix potential panic in volume grow logic by checking replica placement parse error
2026-01-23 12:25:11 -08:00
Chris LuandGitHub b0b7bd0273 Add check for AI code review comments in PR template
Updated the pull request template to include a new check for addressing AI code review comments.
2026-01-23 12:19:00 -08:00
Lisandro PinandGitHub 7e81c0bf0d Clarfiy errors upon needle CRC mismatches. (#8096) 2026-01-23 10:48:29 -08:00
Chris Lu a5474c30d1 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2026-01-22 22:18:49 -08:00
Chris LuandGitHub e717a63665 Fix EC shard recovery with improved diagnostics (#8091)
* storage: fix EC shard recovery with improved diagnostics and logging

- Fix buffer size mismatch in ReconstructData call
- Add detailed logging of available and missing shards
- Improve error messages when recovery is impossible
- Add unit tests for EC recovery shard counting logic

* test: refine EC recovery unit tests

- Remove redundant tests that only validate setup
- Use standard strings.Contains instead of custom recursive helper

* adjust tests and minor improvement
2026-01-22 20:34:19 -08:00
Chris Lu c8909cd031 Create reproduce_issue_s3.sh 2026-01-22 18:19:02 -08:00
Chris LuandGitHub bc1113208d fix: S3 listing NextMarker missing intermediate directory component (#8089)
* fix: S3 listing NextMarker missing intermediate directory component

When listing with nested prefixes like "character/member/", the NextMarker
was incorrectly constructed as "character/res024/" instead of
"character/member/res024/", causing continuation requests to fail.

Root cause: The code at line 331 was constructing NextMarker as:
  nextMarker = requestDir + "/" + nextMarker

This worked when nextMarker already contained the full relative path,
but failed when it was just the entry name from the innermost recursion.

Fix: Include the prefix component when constructing NextMarker:
  if prefix != "" {
      nextMarker = requestDir + "/" + prefix + "/" + nextMarker
  }

This ensures the full path is always constructed correctly for both:
- CommonPrefix entries (directories)
- Regular entries (files)

Also includes fix for cursor.prefixEndsOnDelimiter state leak that was
causing sibling directories to be incorrectly listed.

* test: add regression tests for NextMarker construction

Add comprehensive unit tests to verify NextMarker is correctly constructed
with nested prefixes. Tests cover:
- Regular entries with nested prefix (character/member/res024)
- CommonPrefix entries (directories)
- Edge cases (no requestDir, no prefix, deeply nested)

These tests ensure the fix prevents regression of the bug where
NextMarker was missing intermediate directory components.
2026-01-22 16:56:35 -08:00
Chris LuandGitHub 066410dbd0 Fix S3 Gateway Read Failover #8076 (#8087)
* fix s3 read failover #8076

- Implement cache invalidation in vidMapClient
- Add retry logic in shared PrepareStreamContentWithThrottler
- Update S3 Gateway to use FilerClient directly for invalidation support
- Remove obsolete simpleMasterClient struct

* improve observability for chunk re-lookup failures

Added a warning log when volume location re-lookup fails after cache invalidation in PrepareStreamContentWithThrottler.

* address code review feedback

- Prevent infinite retry loops by comparing old/new URLs before retry
- Update fileId2Url map after successful re-lookup for subsequent references
- Add comprehensive test coverage for failover logic
- Add tests for InvalidateCache method

* Fix: prevent data duplication in stream retry and improve VidMap robustness

* Cleanup: remove redundant check in InvalidateCache
2026-01-22 14:07:24 -08:00
Chris Lu 2e9a7e13e2 cast i to int64 first, ensuring the calculation happens in 64-bit space
fix https://github.com/seaweedfs/seaweedfs/issues/8086
2026-01-22 14:05:45 -08:00
751 changed files with 88819 additions and 18621 deletions
+1
View File
@@ -12,6 +12,7 @@
# Checks
- [ ] I have added unit tests if possible.
- [ ] I will add related wiki document changes and link to this PR after merging.
- [ ] All AI code review comments have been addressed. No more comments to fix if reviewed again. Reviewer may request additional gemini and copilot reviews.
# Checks for AI generated PRs
- [ ] I have reviewed every line of code.
-1
View File
@@ -3,7 +3,6 @@ name: "docker: build dev containers"
on:
push:
branches: [ master ]
workflow_dispatch: {}
permissions:
contents: read
+91 -31
View File
@@ -4,29 +4,58 @@ on:
push:
tags:
- '*'
workflow_dispatch: {}
workflow_dispatch:
inputs:
source_ref:
description: 'Git ref to build (branch, tag, or commit SHA)'
required: true
default: 'master'
image_tag:
description: 'Docker tag to publish (without variant suffix)'
required: true
default: 'latest'
variant:
description: 'Variant to build manually'
required: true
type: choice
default: all
options:
- all
- standard
- large_disk
permissions:
contents: read
jobs:
setup:
runs-on: ubuntu-latest
outputs:
variants: ${{ steps.set-variants.outputs.variants }}
steps:
- name: Select variants for this run
id: set-variants
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ] && [ "${{ github.event.inputs.variant }}" != "all" ]; then
variants="[\"${{ github.event.inputs.variant }}\"]"
else
variants='["standard","large_disk"]'
fi
echo "variants=$variants" >> "$GITHUB_OUTPUT"
build:
needs: [setup]
runs-on: ubuntu-latest
strategy:
matrix:
platform: [amd64, arm64, arm, 386]
include:
- platform: amd64
qemu: false
- platform: arm64
qemu: true
- platform: arm
qemu: true
- platform: 386
qemu: true
variant: ${{ fromJSON(needs.setup.outputs.variants) }}
steps:
- name: Checkout
uses: actions/checkout@v6
with:
ref: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.source_ref || github.ref }}
- name: Free Disk Space
run: |
echo "Available disk space before cleanup:"
@@ -43,6 +72,18 @@ jobs:
[ -d /go/pkg ] && rm -rf /go/pkg || true
echo "Available disk space after cleanup:"
df -h
- name: Configure variant
id: config
run: |
if [ "${{ matrix.variant }}" == "large_disk" ]; then
echo "tag_suffix=_large_disk" >> $GITHUB_OUTPUT
echo "build_args=TAGS=5BytesOffset" >> $GITHUB_OUTPUT
else
echo "tag_suffix=" >> $GITHUB_OUTPUT
echo "build_args=" >> $GITHUB_OUTPUT
fi
- name: Docker meta
id: docker_meta
uses: docker/metadata-action@v5
@@ -50,13 +91,13 @@ jobs:
images: |
chrislusf/seaweedfs
ghcr.io/chrislusf/seaweedfs
tags: type=raw,value=latest
tags: type=raw,value=${{ github.event_name == 'workflow_dispatch' && github.event.inputs.image_tag || 'latest' }},suffix=${{ steps.config.outputs.tag_suffix }}
labels: |
org.opencontainers.image.title=seaweedfs
org.opencontainers.image.description=SeaweedFS is a distributed storage system for blobs, objects, files, and data lake, to store and serve billions of files fast!
org.opencontainers.image.vendor=Chris Lu
- name: Set up QEMU
if: matrix.qemu
if: matrix.platform != 'amd64'
uses: docker/setup-qemu-action@v3
- name: Create BuildKit config
run: |
@@ -82,7 +123,7 @@ jobs:
registry: ghcr.io
username: ${{ secrets.GHCR_USERNAME }}
password: ${{ secrets.GHCR_TOKEN }}
- name: Build ${{ matrix.platform }}
- name: Build ${{ matrix.platform }} ${{ matrix.variant }}
uses: docker/build-push-action@v6
env:
DOCKER_BUILDKIT: 1
@@ -92,13 +133,14 @@ jobs:
file: ./docker/Dockerfile.go_build
platforms: linux/${{ matrix.platform }}
# Push to GHCR only during build to avoid Docker Hub rate limits
tags: ghcr.io/chrislusf/seaweedfs:latest-${{ matrix.platform }}
tags: ghcr.io/chrislusf/seaweedfs:${{ github.event_name == 'workflow_dispatch' && github.event.inputs.image_tag || 'latest' }}${{ steps.config.outputs.tag_suffix }}-${{ matrix.platform }}
labels: ${{ steps.docker_meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
cache-from: type=gha,scope=${{ matrix.variant }}-${{ matrix.platform }}
cache-to: type=gha,mode=max,scope=${{ matrix.variant }}-${{ matrix.platform }}
build-args: |
BUILDKIT_INLINE_CACHE=1
BRANCH=${{ github.sha }}
BRANCH=${{ github.event_name == 'workflow_dispatch' && github.event.inputs.source_ref || github.sha }}
${{ steps.config.outputs.build_args }}
- name: Clean up build artifacts
if: always()
run: |
@@ -109,11 +151,26 @@ jobs:
create-manifest:
runs-on: ubuntu-latest
needs: [build]
needs: [setup, build]
if: github.event_name != 'pull_request'
strategy:
matrix:
variant: ${{ fromJSON(needs.setup.outputs.variants) }}
steps:
- name: Checkout
uses: actions/checkout@v6
with:
ref: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.source_ref || github.ref }}
- name: Configure variant
id: config
run: |
if [ "${{ matrix.variant }}" == "large_disk" ]; then
echo "tag_suffix=_large_disk" >> $GITHUB_OUTPUT
else
echo "tag_suffix=" >> $GITHUB_OUTPUT
fi
- name: Docker meta
id: docker_meta
uses: docker/metadata-action@v5
@@ -121,7 +178,7 @@ jobs:
images: |
chrislusf/seaweedfs
ghcr.io/chrislusf/seaweedfs
tags: type=raw,value=latest
tags: type=raw,value=${{ github.event_name == 'workflow_dispatch' && github.event.inputs.image_tag || 'latest' }},suffix=${{ steps.config.outputs.tag_suffix }}
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
@@ -142,13 +199,16 @@ jobs:
crane version
- name: Create and push manifest
run: |
SUFFIX="${{ steps.config.outputs.tag_suffix }}"
BASE_TAG="${{ github.event_name == 'workflow_dispatch' && github.event.inputs.image_tag || 'latest' }}"
# Create manifest on GHCR first (no rate limits)
echo "Creating GHCR manifest (no rate limits)..."
docker buildx imagetools create -t ghcr.io/chrislusf/seaweedfs:latest \
ghcr.io/chrislusf/seaweedfs:latest-amd64 \
ghcr.io/chrislusf/seaweedfs:latest-arm64 \
ghcr.io/chrislusf/seaweedfs:latest-arm \
ghcr.io/chrislusf/seaweedfs:latest-386
docker buildx imagetools create -t ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX} \
ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-amd64 \
ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-arm64 \
ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-arm \
ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-386
# Copy the complete multi-arch image from GHCR to Docker Hub
# This only requires one pull from GHCR (no rate limit) and one push to Docker Hub
@@ -184,16 +244,16 @@ jobs:
# Use crane or skopeo to copy, fallback to docker if not available
if command -v crane &> /dev/null; then
echo "Using crane to copy..."
retry_with_backoff crane copy ghcr.io/chrislusf/seaweedfs:latest chrislusf/seaweedfs:latest
retry_with_backoff crane copy ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX} chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}
elif command -v skopeo &> /dev/null; then
echo "Using skopeo to copy..."
retry_with_backoff skopeo copy --all docker://ghcr.io/chrislusf/seaweedfs:latest docker://chrislusf/seaweedfs:latest
retry_with_backoff skopeo copy --all docker://ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX} docker://chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}
else
echo "Using docker buildx imagetools (pulling 4 images from Docker Hub)..."
# Fallback: create manifest directly on Docker Hub (pulls from Docker Hub - rate limited)
retry_with_backoff docker buildx imagetools create -t chrislusf/seaweedfs:latest \
ghcr.io/chrislusf/seaweedfs:latest-amd64 \
ghcr.io/chrislusf/seaweedfs:latest-arm64 \
ghcr.io/chrislusf/seaweedfs:latest-arm \
ghcr.io/chrislusf/seaweedfs:latest-386
retry_with_backoff docker buildx imagetools create -t chrislusf/seaweedfs:${BASE_TAG}${SUFFIX} \
ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-amd64 \
ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-arm64 \
ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-arm \
ghcr.io/chrislusf/seaweedfs:${BASE_TAG}${SUFFIX}-386
fi
-152
View File
@@ -1,152 +0,0 @@
name: "docker: build release containers for normal volume"
# DISABLED: Merged into container_release_unified.yml
on:
workflow_dispatch:
inputs:
force_run:
description: 'This workflow is disabled. Use container_release_unified.yml instead'
required: true
default: 'disabled'
permissions:
contents: read
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
platform: [amd64, arm64, arm, 386]
include:
- platform: amd64
qemu: false
- platform: arm64
qemu: true
- platform: arm
qemu: true
- platform: 386
qemu: true
steps:
- name: Checkout
uses: actions/checkout@v6
- name: Free Disk Space
run: |
echo "Available disk space before cleanup:"
df -h
# Remove pre-installed tools
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL
# Clean package managers
sudo apt-get clean
sudo rm -rf /var/lib/apt/lists/*
# Clean Docker aggressively
sudo docker system prune -af --volumes
# Clean Go cache if it exists
[ -d ~/.cache/go-build ] && rm -rf ~/.cache/go-build || true
[ -d /go/pkg ] && rm -rf /go/pkg || true
echo "Available disk space after cleanup:"
df -h
- name: Docker meta
id: docker_meta
uses: docker/metadata-action@v5
with:
images: chrislusf/seaweedfs
tags: type=ref,event=tag
flavor: latest=false
- name: Set up QEMU
if: matrix.qemu
uses: docker/setup-qemu-action@v3
- name: Create BuildKit config
run: |
cat > /tmp/buildkitd.toml <<EOF
[registry."docker.io"]
mirrors = ["https://mirror.gcr.io"]
EOF
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
buildkitd-config: /tmp/buildkitd.toml
- name: Login to Docker Hub
if: github.event_name != 'pull_request'
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Build ${{ matrix.platform }}
uses: docker/build-push-action@v6
env:
DOCKER_BUILDKIT: 1
with:
context: ./docker
push: ${{ github.event_name != 'pull_request' }}
file: ./docker/Dockerfile.go_build
platforms: linux/${{ matrix.platform }}
tags: ${{ steps.docker_meta.outputs.tags }}-${{ matrix.platform }}
labels: ${{ steps.docker_meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
build-args: |
BUILDKIT_INLINE_CACHE=1
BRANCH=${{ github.sha }}
- name: Clean up build artifacts
if: always()
run: |
# Clean up Docker build cache and temporary files
sudo docker system prune -f
# Remove Go build cache
sudo rm -rf /tmp/go-build*
create-manifest:
runs-on: ubuntu-latest
needs: [build]
if: github.event_name != 'pull_request'
steps:
- name: Checkout
uses: actions/checkout@v6
- name: Docker meta
id: docker_meta
uses: docker/metadata-action@v5
with:
images: chrislusf/seaweedfs
tags: type=ref,event=tag
flavor: latest=false
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Create and push manifest
run: |
# Function to retry command with exponential backoff
retry_with_backoff() {
local max_attempts=5
local timeout=1
local attempt=1
local exit_code=0
while [ $attempt -le $max_attempts ]; do
if "$@"; then
return 0
else
exit_code=$?
fi
if [ $attempt -lt $max_attempts ]; then
echo "Attempt $attempt failed. Retrying in ${timeout}s..." >&2
sleep $timeout
timeout=$((timeout * 2))
fi
attempt=$((attempt + 1))
done
echo "Command failed after $max_attempts attempts" >&2
return $exit_code
}
# Create manifest with retry
retry_with_backoff docker buildx imagetools create -t ${{ steps.docker_meta.outputs.tags }} \
${{ steps.docker_meta.outputs.tags }}-amd64 \
${{ steps.docker_meta.outputs.tags }}-arm64 \
${{ steps.docker_meta.outputs.tags }}-arm \
${{ steps.docker_meta.outputs.tags }}-386
-153
View File
@@ -1,153 +0,0 @@
name: "docker: build release containers for large volume"
# DISABLED: Merged into container_release_unified.yml
on:
workflow_dispatch:
inputs:
force_run:
description: 'This workflow is disabled. Use container_release_unified.yml instead'
required: true
default: 'disabled'
permissions:
contents: read
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
platform: [amd64, arm64, arm, 386]
include:
- platform: amd64
qemu: false
- platform: arm64
qemu: true
- platform: arm
qemu: true
- platform: 386
qemu: true
steps:
- name: Checkout
uses: actions/checkout@v6
- name: Free Disk Space
run: |
echo "Available disk space before cleanup:"
df -h
# Remove pre-installed tools
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL
# Clean package managers
sudo apt-get clean
sudo rm -rf /var/lib/apt/lists/*
# Clean Docker aggressively
sudo docker system prune -af --volumes
# Clean Go cache if it exists
[ -d ~/.cache/go-build ] && rm -rf ~/.cache/go-build || true
[ -d /go/pkg ] && rm -rf /go/pkg || true
echo "Available disk space after cleanup:"
df -h
- name: Docker meta
id: docker_meta
uses: docker/metadata-action@v5
with:
images: chrislusf/seaweedfs
tags: type=ref,event=tag,suffix=_large_disk
flavor: latest=false
- name: Set up QEMU
if: matrix.qemu
uses: docker/setup-qemu-action@v3
- name: Create BuildKit config
run: |
cat > /tmp/buildkitd.toml <<EOF
[registry."docker.io"]
mirrors = ["https://mirror.gcr.io"]
EOF
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
buildkitd-config: /tmp/buildkitd.toml
- name: Login to Docker Hub
if: github.event_name != 'pull_request'
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Build ${{ matrix.platform }}
uses: docker/build-push-action@v6
env:
DOCKER_BUILDKIT: 1
with:
context: ./docker
push: ${{ github.event_name != 'pull_request' }}
file: ./docker/Dockerfile.go_build
build-args: |
TAGS=5BytesOffset
BUILDKIT_INLINE_CACHE=1
BRANCH=${{ github.sha }}
platforms: linux/${{ matrix.platform }}
tags: ${{ steps.docker_meta.outputs.tags }}-${{ matrix.platform }}
labels: ${{ steps.docker_meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Clean up build artifacts
if: always()
run: |
# Clean up Docker build cache and temporary files
sudo docker system prune -f
# Remove Go build cache
sudo rm -rf /tmp/go-build*
create-manifest:
runs-on: ubuntu-latest
needs: [build]
if: github.event_name != 'pull_request'
steps:
- name: Checkout
uses: actions/checkout@v6
- name: Docker meta
id: docker_meta
uses: docker/metadata-action@v5
with:
images: chrislusf/seaweedfs
tags: type=ref,event=tag,suffix=_large_disk
flavor: latest=false
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Create and push manifest
run: |
# Function to retry command with exponential backoff
retry_with_backoff() {
local max_attempts=5
local timeout=1
local attempt=1
local exit_code=0
while [ $attempt -le $max_attempts ]; do
if "$@"; then
return 0
else
exit_code=$?
fi
if [ $attempt -lt $max_attempts ]; then
echo "Attempt $attempt failed. Retrying in ${timeout}s..." >&2
sleep $timeout
timeout=$((timeout * 2))
fi
attempt=$((attempt + 1))
done
echo "Command failed after $max_attempts attempts" >&2
return $exit_code
}
# Create manifest with retry
retry_with_backoff docker buildx imagetools create -t ${{ steps.docker_meta.outputs.tags }} \
${{ steps.docker_meta.outputs.tags }}-amd64 \
${{ steps.docker_meta.outputs.tags }}-arm64 \
${{ steps.docker_meta.outputs.tags }}-arm \
${{ steps.docker_meta.outputs.tags }}-386
-73
View File
@@ -1,73 +0,0 @@
name: "docker: build release containers for rocksdb"
# DISABLED: Merged into container_release_unified.yml
on:
workflow_dispatch:
inputs:
force_run:
description: 'This workflow is disabled. Use container_release_unified.yml instead'
required: true
default: 'disabled'
permissions:
contents: read
jobs:
build-large-release-container_rocksdb:
runs-on: [ubuntu-latest]
steps:
-
name: Checkout
uses: actions/checkout@v6
-
name: Free Disk Space
run: |
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc
sudo docker system prune -af
-
name: Docker meta
id: docker_meta
uses: docker/metadata-action@v5
with:
images: chrislusf/seaweedfs
tags: type=ref,event=tag,suffix=_large_disk_rocksdb
flavor: latest=false
labels: |
org.opencontainers.image.title=seaweedfs
org.opencontainers.image.description=SeaweedFS is a distributed storage system for blobs, objects, files, and data lake, to store and serve billions of files fast!
org.opencontainers.image.vendor=Chris Lu
-
name: Create BuildKit config
run: |
cat > /tmp/buildkitd.toml <<EOF
[registry."docker.io"]
mirrors = ["https://mirror.gcr.io"]
EOF
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
buildkitd-config: /tmp/buildkitd.toml
-
name: Login to Docker Hub
if: github.event_name != 'pull_request'
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
-
name: Build
uses: docker/build-push-action@v6
with:
context: ./docker
push: ${{ github.event_name != 'pull_request' }}
file: ./docker/Dockerfile.rocksdb_large
build-args: |
BRANCH=${{ github.sha }}
platforms: linux/amd64
tags: ${{ steps.docker_meta.outputs.tags }}
labels: ${{ steps.docker_meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
-72
View File
@@ -1,72 +0,0 @@
name: "docker: build release containers for all tags"
# DISABLED: Merged into container_release_unified.yml
on:
workflow_dispatch:
inputs:
force_run:
description: 'This workflow is disabled. Use container_release_unified.yml instead'
required: true
default: 'disabled'
permissions:
contents: read
jobs:
build-default-release-container:
runs-on: [ubuntu-latest]
steps:
-
name: Checkout
uses: actions/checkout@v6
-
name: Free Disk Space
run: |
echo "Before cleanup:"
df -h
sudo rm -rf /usr/share/dotnet
sudo rm -rf /usr/local/lib/android
sudo rm -rf /opt/ghc
sudo rm -rf /opt/hostedtoolcache/CodeQL
sudo docker system prune -af
echo "After cleanup:"
df -h
-
name: Docker meta
id: docker_meta
uses: docker/metadata-action@v5
with:
images: |
chrislusf/seaweedfs
tags: |
type=ref,event=tag,suffix=_full
flavor: |
latest=false
labels: |
org.opencontainers.image.title=seaweedfs
org.opencontainers.image.description=SeaweedFS is a distributed storage system for blobs, objects, files, and data lake, to store and serve billions of files fast!
org.opencontainers.image.vendor=Chris Lu
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Login to Docker Hub
if: github.event_name != 'pull_request'
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
-
name: Build
uses: docker/build-push-action@v6
with:
context: ./docker
push: ${{ github.event_name != 'pull_request' }}
file: ./docker/Dockerfile.go_build
build-args: TAGS=elastic,gocdk,rclone,sqlite,tarantool,tikv,ydb
platforms: linux/amd64
tags: ${{ steps.docker_meta.outputs.tags }}
labels: ${{ steps.docker_meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
-62
View File
@@ -1,62 +0,0 @@
name: "docker: build release containers for all tags and large volume"
# DISABLED: Merged into container_release_unified.yml
on:
workflow_dispatch:
inputs:
force_run:
description: 'This workflow is disabled. Use container_release_unified.yml instead'
required: true
default: 'disabled'
permissions:
contents: read
jobs:
build-default-release-container:
runs-on: [ubuntu-latest]
steps:
-
name: Checkout
uses: actions/checkout@v6
-
name: Free Disk Space
run: |
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc
sudo docker system prune -af
-
name: Docker meta
id: docker_meta
uses: docker/metadata-action@v5
with:
images: chrislusf/seaweedfs
tags: type=ref,event=tag,suffix=_large_disk_full
flavor: latest=false
labels: |
org.opencontainers.image.title=seaweedfs
org.opencontainers.image.description=SeaweedFS is a distributed storage system for blobs, objects, files, and data lake, to store and serve billions of files fast!
org.opencontainers.image.vendor=Chris Lu
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Login to Docker Hub
if: github.event_name != 'pull_request'
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
-
name: Build
uses: docker/build-push-action@v6
with:
context: ./docker
push: ${{ github.event_name != 'pull_request' }}
file: ./docker/Dockerfile.go_build
build-args: TAGS=5BytesOffset,elastic,gocdk,rclone,sqlite,tarantool,tikv,ydb
platforms: linux/amd64
tags: ${{ steps.docker_meta.outputs.tags }}
labels: ${{ steps.docker_meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
+45 -13
View File
@@ -4,11 +4,35 @@ on:
push:
tags:
- '*'
workflow_dispatch: {}
workflow_dispatch:
inputs:
variant:
description: 'Variant to build manually'
required: true
type: choice
default: all
options:
- all
- normal
- large_disk
- full
- large_disk_full
- rocksdb
release_tag:
description: 'Release tag to publish (e.g. 3.93)'
required: true
default: ''
rocksdb_version:
description: 'RocksDB git tag to use when variant=rocksdb'
required: false
default: 'v10.10.1'
permissions:
contents: read
env:
RELEASE_TAG: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.release_tag || github.ref_name }}
# Limit concurrent builds to avoid rate limits
concurrency:
group: release-${{ github.ref }}
@@ -59,9 +83,11 @@ jobs:
steps:
- name: Checkout
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
uses: actions/checkout@v6
- name: Free Disk Space
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
run: |
echo "Available disk space before cleanup:"
df -h
@@ -75,13 +101,14 @@ jobs:
df -h
- name: Docker meta
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
id: docker_meta
uses: docker/metadata-action@v5
with:
images: |
chrislusf/seaweedfs
ghcr.io/chrislusf/seaweedfs
tags: type=ref,event=tag,suffix=${{ matrix.tag_suffix }}
tags: type=raw,value=${{ env.RELEASE_TAG }}${{ matrix.tag_suffix }}
flavor: latest=false
labels: |
org.opencontainers.image.title=seaweedfs
@@ -89,10 +116,11 @@ jobs:
org.opencontainers.image.vendor=Chris Lu
- name: Set up QEMU
if: contains(matrix.platforms, 'arm')
if: (github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant) && contains(matrix.platforms, 'arm')
uses: docker/setup-qemu-action@v3
- name: Create BuildKit config
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
run: |
cat > /tmp/buildkitd.toml <<EOF
[registry."docker.io"]
@@ -100,19 +128,20 @@ jobs:
EOF
- name: Set up Docker Buildx
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
uses: docker/setup-buildx-action@v3
with:
buildkitd-config: /tmp/buildkitd.toml
- name: Login to Docker Hub
if: github.event_name != 'pull_request'
if: (github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant) && github.event_name != 'pull_request'
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to GHCR
if: github.event_name != 'pull_request'
if: (github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant) && github.event_name != 'pull_request'
uses: docker/login-action@v3
with:
registry: ghcr.io
@@ -120,6 +149,7 @@ jobs:
password: ${{ secrets.GHCR_TOKEN }}
- name: Build and push ${{ matrix.variant }}
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
uses: docker/build-push-action@v6
env:
DOCKER_BUILDKIT: 1
@@ -130,7 +160,7 @@ jobs:
platforms: ${{ matrix.platforms }}
# Push to GHCR to avoid Docker Hub rate limits on pulls
tags: |
ghcr.io/chrislusf/seaweedfs:${{ github.ref_name }}${{ matrix.tag_suffix }}
ghcr.io/chrislusf/seaweedfs:${{ env.RELEASE_TAG }}${{ matrix.tag_suffix }}
labels: ${{ steps.docker_meta.outputs.labels }}
cache-from: type=gha,scope=${{ matrix.variant }}
cache-to: type=gha,mode=max,scope=${{ matrix.variant }}
@@ -138,9 +168,10 @@ jobs:
${{ matrix.build_args }}
BUILDKIT_INLINE_CACHE=1
BRANCH=${{ github.sha }}
${{ matrix.variant == 'rocksdb' && format('ROCKSDB_VERSION={0}', github.event.inputs.rocksdb_version || 'v10.10.1') || '' }}
- name: Clean up build artifacts
if: always()
if: always() && (github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant)
run: |
sudo docker system prune -f
sudo rm -rf /tmp/go-build*
@@ -166,12 +197,14 @@ jobs:
steps:
- name: Login to Docker Hub
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to GHCR
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
uses: docker/login-action@v3
with:
registry: ghcr.io
@@ -179,6 +212,7 @@ jobs:
password: ${{ secrets.GHCR_TOKEN }}
- name: Install crane
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
run: |
cd $(mktemp -d)
curl -sL "https://github.com/google/go-containerregistry/releases/latest/download/go-containerregistry_Linux_x86_64.tar.gz" | tar xz
@@ -186,6 +220,7 @@ jobs:
crane version
- name: Copy ${{ matrix.variant }} from GHCR to Docker Hub
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
run: |
# Function to retry with exponential backoff
retry_with_backoff() {
@@ -218,14 +253,15 @@ jobs:
# This is much more efficient than pulling/pushing individual arch images
echo "Copying ${{ matrix.variant }} from GHCR to Docker Hub..."
retry_with_backoff crane copy \
ghcr.io/chrislusf/seaweedfs:${{ github.ref_name }}${{ matrix.tag_suffix }} \
chrislusf/seaweedfs:${{ github.ref_name }}${{ matrix.tag_suffix }}
ghcr.io/chrislusf/seaweedfs:${{ env.RELEASE_TAG }}${{ matrix.tag_suffix }} \
chrislusf/seaweedfs:${{ env.RELEASE_TAG }}${{ matrix.tag_suffix }}
echo "✓ Successfully copied ${{ matrix.variant }} to Docker Hub"
helm-release:
runs-on: ubuntu-latest
needs: [copy-to-dockerhub]
if: github.event_name == 'push' || github.event_name == 'workflow_dispatch'
permissions:
contents: write
pages: write
@@ -239,7 +275,3 @@ jobs:
target_dir: helm
branch: gh-pages
helm_version: "3.18.4"
@@ -4,9 +4,9 @@ on:
workflow_dispatch:
inputs:
rocksdb_version:
description: 'RocksDB git tag or branch to build (e.g. v10.5.1)'
description: 'RocksDB git tag or branch to build (e.g. v10.10.1)'
required: true
default: 'v10.5.1'
default: 'v10.10.1'
seaweedfs_ref:
description: 'SeaweedFS git tag, branch, or commit to build'
required: true
+1 -1
View File
@@ -24,7 +24,7 @@ jobs:
timeout-minutes: 30
steps:
- name: Set up Go 1.x
uses: actions/setup-go@4dc6199c7b1a012772edbd06daecab0f50c9053c # v2
uses: actions/setup-go@a5f9b05d2d216f63e13859e0d847461041025775 # v2
with:
go-version: ^1.13
id: go
+3 -3
View File
@@ -23,10 +23,10 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Set up Go
uses: actions/setup-go@v5
uses: actions/setup-go@v6
with:
go-version: '1.24'
@@ -42,7 +42,7 @@ jobs:
- name: Upload test logs on failure
if: failure()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v6
with:
name: ec-test-logs
path: test/erasure_coding/admin_dockertest/tmp/logs/
+25 -7
View File
@@ -15,24 +15,20 @@ permissions:
jobs:
build:
name: Build
vet:
name: Go Vet
runs-on: ubuntu-latest
steps:
- name: Set up Go 1.x
uses: actions/setup-go@4dc6199c7b1a012772edbd06daecab0f50c9053c # v2
uses: actions/setup-go@a5f9b05d2d216f63e13859e0d847461041025775 # v2
with:
go-version: ^1.13
id: go
- name: Check out code into the Go module directory
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v2
- name: Get dependencies
run: |
cd weed; go get -v -t -d ./...
- name: Go Vet (excluding protobuf lock copying)
run: |
cd weed
@@ -42,8 +38,30 @@ jobs:
# Fail only if there are actual vet errors (not counting the filtered lock warnings)
if grep -q "vet:" vet-output.txt; then exit 1; fi
build:
name: Build
runs-on: ubuntu-latest
steps:
- name: Set up Go 1.x
uses: actions/setup-go@a5f9b05d2d216f63e13859e0d847461041025775 # v2
with:
go-version: ^1.13
id: go
- name: Check out code into the Go module directory
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v2
- name: Build
run: cd weed; go build -tags "elastic gocdk sqlite ydb tarantool tikv rclone" -v .
test:
name: Test
runs-on: ubuntu-latest
steps:
- name: Set up Go 1.x
uses: actions/setup-go@a5f9b05d2d216f63e13859e0d847461041025775 # v2
with:
go-version: ^1.13
id: go
- name: Check out code into the Go module directory
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v2
- name: Test
run: cd weed; go test -tags "elastic gocdk sqlite ydb tarantool tikv rclone" -v ./...
+1 -1
View File
@@ -27,7 +27,7 @@ jobs:
- uses: actions/setup-python@v6
with:
python-version: '3.9'
python-version: '3.10'
check-latest: true
- name: Set up chart-testing
+25
View File
@@ -0,0 +1,25 @@
name: "helm: manual release"
on:
workflow_dispatch:
permissions:
contents: write
pages: write
jobs:
helm-release:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Publish Helm charts
uses: stefanprodan/helm-gh-pages@v1.7.0
with:
token: ${{ secrets.GITHUB_TOKEN }}
charts_dir: k8s/charts
target_dir: helm
branch: gh-pages
helm_version: "3.18.4"
+1 -1
View File
@@ -111,7 +111,7 @@ jobs:
# Wait for S3 API
for i in {1..30}; do
if curl -s http://localhost:8333/ > /dev/null 2>&1; then
if curl -s http://localhost:8333/healthz > /dev/null 2>&1; then
echo "✓ S3 API is ready"
break
fi
@@ -0,0 +1,64 @@
name: "S3 Authenticated Integration Tests"
on:
pull_request:
concurrency:
group: ${{ github.head_ref }}/s3-integration-tests
cancel-in-progress: true
permissions:
contents: read
jobs:
tests:
name: S3 Integration Tests
runs-on: ubuntu-22.04
timeout-minutes: 30
steps:
- name: Check out code
uses: actions/checkout@v6
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
id: go
- name: Build SeaweedFS
run: |
cd weed
go build -o weed -buildvcs=false
- name: Run S3 Integration Tests
timeout-minutes: 15
working-directory: test/s3/normal
run: |
set -x
echo "=== Running S3 Integration Tests ==="
go test -v -timeout=60s -run TestS3Integration ./...
- name: Run S3 DeleteBucketNotEmpty Tests
timeout-minutes: 15
working-directory: test/s3/normal
run: |
set -x
echo "=== Running S3 DeleteBucketNotEmpty Tests ==="
go test -v -timeout=60s -run TestS3DeleteBucketNotEmpty ./...
- name: Run IAM Integration Tests
timeout-minutes: 15
working-directory: test/s3/normal
run: |
set -x
echo "=== Running IAM Integration Tests ==="
go test -v -timeout=60s -run TestIAMOperations ./...
- name: Upload test logs on failure
if: failure()
uses: actions/upload-artifact@v6
with:
name: integration-test-logs
path: test/s3/normal/*.log
retention-days: 3
+2
View File
@@ -461,6 +461,8 @@ jobs:
export S3_ENDPOINT="http://localhost:8006"
export S3_ACCESS_KEY="0555b35654ad1656d804"
export S3_SECRET_KEY="h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q=="
export AWS_ACCESS_KEY_ID="$S3_ACCESS_KEY"
export AWS_SECRET_ACCESS_KEY="$S3_SECRET_KEY"
# Run the specific test that is equivalent to AWS S3 tagging behavior
make test-with-server || {
+1 -1
View File
@@ -117,7 +117,7 @@ jobs:
"basic")
echo "Running basic IAM functionality tests..."
make clean setup start-services wait-for-services
go test -v -timeout 15m -run "TestS3IAMAuthentication|TestS3IAMBasicWorkflow|TestS3IAMTokenValidation" ./...
go test -v -timeout 15m -run "TestS3IAMAuthentication|TestS3IAMBasicWorkflow|TestS3IAMTokenValidation|TestIAM" ./...
;;
"advanced")
echo "Running advanced IAM feature tests..."
+1 -1
View File
@@ -97,7 +97,7 @@ jobs:
# Verify service accessibility
echo "=== Verifying Service Accessibility ==="
curl -f http://localhost:8080/realms/master
curl -s http://localhost:8333
curl -s http://localhost:8333/healthz
echo "✅ SeaweedFS S3 API is responding (IAM-protected endpoint)"
# Run Keycloak-specific tests
+35
View File
@@ -9,6 +9,7 @@ on:
- 'weed/s3api/policy/**'
- 'weed/iam/**'
- 'test/s3/iam/**'
- 'test/s3/policy/**'
- '.github/workflows/s3-policy-tests.yml'
push:
branches: [ master, main ]
@@ -19,6 +20,7 @@ on:
- 'weed/s3api/policy/**'
- 'weed/iam/**'
- 'test/s3/iam/**'
- 'test/s3/policy/**'
- '.github/workflows/s3-policy-tests.yml'
concurrency:
@@ -389,3 +391,36 @@ jobs:
name: trusted-proxy-test-logs
path: /tmp/weed_proxy_test.log
retention-days: 3
# S3 Policy Shell Integration Tests
s3-policy-shell-tests:
name: S3 Policy Shell Integration Tests
runs-on: ubuntu-22.04
timeout-minutes: 15
steps:
- name: Check out code
uses: actions/checkout@v6
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
id: go
- name: Install SeaweedFS
run: |
go install -buildvcs=false
- name: Run S3 Policy Shell Tests
timeout-minutes: 10
working-directory: test/s3/policy
run: |
set -x
echo "=== Running S3 Policy Shell Tests ==="
# Set WEED_BINARY to use the installed version (though test uses 'weed' command)
export WEED_BINARY=$(which weed)
export PATH=$PATH:$(dirname $WEED_BINARY)
go test -v -timeout 10m ./...
+80
View File
@@ -0,0 +1,80 @@
name: "S3 Spark Integration Tests"
on:
pull_request:
paths:
- 'weed/s3api/**'
- 'test/s3/spark/**'
- 'test/s3tables/testutil/**'
- '.github/workflows/s3-spark-tests.yml'
workflow_dispatch:
concurrency:
group: ${{ github.head_ref }}/s3-spark-tests
cancel-in-progress: true
permissions:
contents: read
jobs:
s3-spark-issue-repro-tests:
name: S3 Spark Issue Reproduction Tests
runs-on: ubuntu-22.04
timeout-minutes: 45
steps:
- name: Check out code
uses: actions/checkout@v6
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
id: go
- name: Set up Docker
uses: docker/setup-buildx-action@v3
- name: Install SeaweedFS
run: |
go install -buildvcs=false ./weed
- name: Pre-pull Spark image
run: docker pull apache/spark:3.5.8
- name: Run S3 Spark integration tests
working-directory: test/s3/spark
timeout-minutes: 35
run: |
set -x
set -o pipefail
echo "=== System Information ==="
uname -a
free -h
df -h
echo "=== Starting S3 Spark Integration Tests ==="
go test -v -timeout 30m . 2>&1 | tee test-output.log || {
echo "S3 Spark integration tests failed"
exit 1
}
- name: Show test output on failure
if: failure()
working-directory: test/s3/spark
run: |
echo "=== Test Output ==="
if [ -f test-output.log ]; then
tail -200 test-output.log
fi
echo "=== Process information ==="
ps aux | grep -E "(weed|test|docker|spark)" || true
- name: Upload test logs on failure
if: failure()
uses: actions/upload-artifact@v6
with:
name: s3-spark-test-logs
path: test/s3/spark/test-output.log
retention-days: 3
+625
View File
@@ -0,0 +1,625 @@
name: "S3 Tables Integration Tests"
on:
pull_request:
concurrency:
group: ${{ github.head_ref }}/s3-tables-tests
cancel-in-progress: true
permissions:
contents: read
jobs:
s3-tables-tests:
name: S3 Tables Integration Tests
runs-on: ubuntu-22.04
timeout-minutes: 30
steps:
- name: Check out code
uses: actions/checkout@v6
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
id: go
- name: Run go mod tidy
run: go mod tidy
- name: Install SeaweedFS
run: |
go install -buildvcs=false ./weed
- name: Run S3 Tables Integration Tests
timeout-minutes: 25
working-directory: test/s3tables/table-buckets
run: |
set -x
set -o pipefail
echo "=== System Information ==="
uname -a
free -h
df -h
echo "=== Starting S3 Tables Tests ==="
# Run S3 Tables integration tests
go test -v -timeout 20m . 2>&1 | tee test-output.log || {
echo "S3 Tables integration tests failed"
exit 1
}
- name: Show test output on failure
if: failure()
working-directory: test/s3tables/table-buckets
run: |
echo "=== Test Output ==="
if [ -f test-output.log ]; then
tail -200 test-output.log
fi
echo "=== Process information ==="
ps aux | grep -E "(weed|test)" || true
- name: Upload test logs on failure
if: failure()
uses: actions/upload-artifact@v6
with:
name: s3-tables-test-logs
path: test/s3tables/table-buckets/test-output.log
retention-days: 3
iceberg-catalog-tests:
name: Iceberg Catalog Integration Tests
runs-on: ubuntu-22.04
timeout-minutes: 30
steps:
- name: Check out code
uses: actions/checkout@v6
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
id: go
- name: Run go mod tidy
run: go mod tidy
- name: Run Iceberg Catalog Integration Tests
timeout-minutes: 25
working-directory: test/s3tables/catalog
run: |
set -x
set -o pipefail
echo "=== System Information ==="
uname -a
free -h
df -h
echo "=== Starting Iceberg Catalog Tests ==="
# Run Iceberg catalog integration tests using Makefile (handles build)
make test 2>&1 | tee test-output.log || {
echo "Iceberg catalog integration tests failed"
exit 1
}
- name: Show test output on failure
if: failure()
working-directory: test/s3tables/catalog
run: |
echo "=== Test Output ==="
if [ -f test-output.log ]; then
tail -200 test-output.log
fi
echo "=== Process information ==="
ps aux | grep -E "(weed|test|docker)" || true
- name: Upload test logs on failure
if: failure()
uses: actions/upload-artifact@v6
with:
name: iceberg-catalog-test-logs
path: test/s3tables/catalog/test-output.log
retention-days: 3
trino-iceberg-catalog-tests:
name: Trino Iceberg Catalog Integration Tests
runs-on: ubuntu-22.04
timeout-minutes: 30
steps:
- name: Check out code
uses: actions/checkout@v6
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
id: go
- name: Set up Docker
uses: docker/setup-buildx-action@v3
- name: Pre-pull Trino image
run: docker pull trinodb/trino:479
- name: Run go mod tidy
run: go mod tidy
- name: Install SeaweedFS
run: |
go install -buildvcs=false ./weed
- name: Run Trino Iceberg Catalog Integration Tests
timeout-minutes: 25
working-directory: test/s3tables/catalog_trino
run: |
set -x
set -o pipefail
echo "=== System Information ==="
uname -a
free -h
df -h
echo "=== Starting Trino Iceberg Catalog Tests ==="
# Run Trino + Iceberg catalog integration tests
go test -v -timeout 20m . 2>&1 | tee test-output.log || {
echo "Trino Iceberg catalog integration tests failed"
exit 1
}
- name: Show test output on failure
if: failure()
working-directory: test/s3tables/catalog_trino
run: |
echo "=== Test Output ==="
if [ -f test-output.log ]; then
tail -200 test-output.log
fi
echo "=== Process information ==="
ps aux | grep -E "(weed|test|docker)" || true
- name: Upload test logs on failure
if: failure()
uses: actions/upload-artifact@v6
with:
name: trino-iceberg-catalog-test-logs
path: test/s3tables/catalog_trino/test-output.log
retention-days: 3
spark-iceberg-catalog-tests:
name: Spark Iceberg Catalog Integration Tests
runs-on: ubuntu-22.04
timeout-minutes: 30
steps:
- name: Check out code
uses: actions/checkout@v6
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
id: go
- name: Set up Docker
uses: docker/setup-buildx-action@v3
- name: Pre-pull Spark image
run: docker pull apache/spark:3.5.1
- name: Run go mod tidy
run: go mod tidy
- name: Install SeaweedFS
run: |
go install -buildvcs=false ./weed
- name: Run Spark Iceberg Catalog Integration Tests
timeout-minutes: 25
working-directory: test/s3tables/catalog_spark
run: |
set -x
set -o pipefail
echo "=== System Information ==="
uname -a
free -h
df -h
echo "=== Starting Spark Iceberg Catalog Tests ==="
# Run Spark + Iceberg catalog integration tests
go test -v -timeout 20m . 2>&1 | tee test-output.log || {
echo "Spark Iceberg catalog integration tests failed"
exit 1
}
- name: Show test output on failure
if: failure()
working-directory: test/s3tables/catalog_spark
run: |
echo "=== Test Output ==="
if [ -f test-output.log ]; then
tail -200 test-output.log
fi
echo "=== Process information ==="
ps aux | grep -E "(weed|test|docker)" || true
- name: Upload test logs on failure
if: failure()
uses: actions/upload-artifact@v6
with:
name: spark-iceberg-catalog-test-logs
path: test/s3tables/catalog_spark/test-output.log
retention-days: 3
risingwave-catalog-tests:
name: RisingWave Catalog Integration Tests
runs-on: ubuntu-22.04
timeout-minutes: 30
steps:
- name: Check out code
uses: actions/checkout@v6
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
id: go
- name: Set up Docker
uses: docker/setup-buildx-action@v3
- name: Pre-pull RisingWave image
run: |
docker pull risingwavelabs/risingwave:v2.5.0
docker pull postgres:16-alpine
- name: Run go mod tidy
run: go mod tidy
- name: Install SeaweedFS
run: |
go install -buildvcs=false ./weed
- name: Run RisingWave Catalog Integration Tests
timeout-minutes: 25
working-directory: test/s3tables/catalog_risingwave
run: |
set -x
set -o pipefail
echo "=== System Information ==="
uname -a
free -h
df -h
echo "=== Starting RisingWave Catalog Tests ==="
# Run RisingWave catalog integration tests
go test -v -timeout 20m . 2>&1 | tee test-output.log || {
echo "RisingWave catalog integration tests failed"
exit 1
}
- name: Show test output on failure
if: failure()
working-directory: test/s3tables/catalog_risingwave
run: |
echo "=== Test Output ==="
if [ -f test-output.log ]; then
tail -200 test-output.log
fi
echo "=== Process information ==="
ps aux | grep -E "(weed|test|docker)" || true
- name: Upload test logs on failure
if: failure()
uses: actions/upload-artifact@v6
with:
name: risingwave-catalog-test-logs
path: test/s3tables/catalog_risingwave/test-output.log
retention-days: 3
sts-integration-tests:
name: STS Integration Tests
runs-on: ubuntu-22.04
timeout-minutes: 30
steps:
- name: Check out code
uses: actions/checkout@v6
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
id: go
- name: Set up Docker
uses: docker/setup-buildx-action@v3
- name: Pre-pull Python image
run: docker pull python:3
- name: Run go mod tidy
run: go mod tidy
- name: Install SeaweedFS
run: |
go install -buildvcs=false ./weed
- name: Run STS Integration Tests
timeout-minutes: 25
working-directory: test/s3tables/sts_integration
run: |
set -x
set -o pipefail
echo "=== System Information ==="
uname -a
free -h
df -h
echo "=== Starting STS Integration Tests ==="
# Run STS integration tests
go test -v -timeout 20m . 2>&1 | tee test-output.log || {
echo "STS integration tests failed"
exit 1
}
- name: Show test output on failure
if: failure()
working-directory: test/s3tables/sts_integration
run: |
echo "=== Test Output ==="
if [ -f test-output.log ]; then
tail -200 test-output.log
fi
echo "=== Process information ==="
ps aux | grep -E "(weed|test|docker)" || true
- name: Upload test logs on failure
if: failure()
uses: actions/upload-artifact@v6
with:
name: sts-integration-test-logs
path: test/s3tables/sts_integration/test-output.log
retention-days: 3
lakekeeper-integration-tests:
name: Lakekeeper Integration Tests
runs-on: ubuntu-22.04
timeout-minutes: 30
steps:
- name: Check out code
uses: actions/checkout@v6
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
id: go
- name: Set up Docker
uses: docker/setup-buildx-action@v3
- name: Pre-pull Python image
run: docker pull python:3
- name: Pre-pull LocalStack image (if needed)
run: docker pull localstack/localstack:latest || true
- name: Run go mod tidy
run: go mod tidy
- name: Install SeaweedFS
run: |
go install -buildvcs=false ./weed
- name: Run Lakekeeper Integration Tests
timeout-minutes: 25
working-directory: test/s3tables/lakekeeper
run: |
set -x
set -o pipefail
echo "=== System Information ==="
uname -a
free -h
df -h
echo "=== Starting Lakekeeper Integration Tests ==="
# Run Lakekeeper integration tests
go test -v -timeout 20m . 2>&1 | tee test-output.log || {
echo "Lakekeeper integration tests failed"
exit 1
}
- name: Show test output on failure
if: failure()
working-directory: test/s3tables/lakekeeper
run: |
echo "=== Test Output ==="
if [ -f test-output.log ]; then
tail -200 test-output.log
fi
echo "=== Process information ==="
ps aux | grep -E "(weed|test|docker)" || true
- name: Upload test logs on failure
if: failure()
uses: actions/upload-artifact@v6
with:
name: lakekeeper-integration-test-logs
path: test/s3tables/lakekeeper/test-output.log
retention-days: 3
s3-tables-build-verification:
name: S3 Tables Build Verification
runs-on: ubuntu-22.04
timeout-minutes: 15
steps:
- name: Check out code
uses: actions/checkout@v6
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
id: go
- name: Verify S3 Tables Package Builds
run: |
set -x
echo "=== Building S3 Tables package ==="
go build ./weed/s3api/s3tables || {
echo "S3 Tables package build failed"
exit 1
}
echo "S3 Tables package built successfully"
- name: Verify S3 API Integration Builds
run: |
set -x
echo "=== Building S3 API with S3 Tables integration ==="
go build ./weed/s3api || {
echo "S3 API build with S3 Tables failed"
exit 1
}
echo "S3 API with S3 Tables integration built successfully"
- name: Run Go Tests for S3 Tables Package
run: |
set -x
echo "=== Running Go unit tests for S3 Tables ==="
go test -v -race -timeout 5m ./weed/s3api/s3tables/... || {
echo "S3 Tables unit tests failed"
exit 1
}
echo "S3 Tables unit tests passed"
- name: Verify Iceberg Package Builds
run: |
set -x
echo "=== Building Iceberg package ==="
go build ./weed/s3api/iceberg || {
echo "Iceberg package build failed"
exit 1
}
echo "Iceberg package built successfully"
- name: Run Go Tests for Iceberg Package
run: |
set -x
echo "=== Running Go unit tests for Iceberg ==="
go test -v -race -timeout 5m ./weed/s3api/iceberg/... || {
echo "Iceberg unit tests failed"
exit 1
}
echo "Iceberg unit tests passed"
s3-tables-fmt-check:
name: S3 Tables Format Check
runs-on: ubuntu-22.04
timeout-minutes: 10
steps:
- name: Check out code
uses: actions/checkout@v6
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
id: go
- name: Check Go Format
run: |
set -x
echo "=== Checking S3 Tables Go format ==="
unformatted=$(gofmt -l ./weed/s3api/s3tables)
if [ -n "$unformatted" ]; then
echo "Go format check failed - files need formatting"
echo "$unformatted"
exit 1
fi
echo "All S3 Tables files are properly formatted"
- name: Check S3 Tables Test Format
run: |
set -x
echo "=== Checking S3 Tables test format ==="
unformatted=$(gofmt -l ./test/s3tables)
if [ -n "$unformatted" ]; then
echo "Go format check failed for tests"
echo "$unformatted"
exit 1
fi
echo "All S3 Tables test files are properly formatted"
- name: Check Iceberg Format
run: |
set -x
echo "=== Checking Iceberg Go format ==="
unformatted=$(gofmt -l ./weed/s3api/iceberg)
if [ -n "$unformatted" ]; then
echo "Go format check failed for Iceberg - files need formatting"
echo "$unformatted"
exit 1
fi
echo "All Iceberg files are properly formatted"
s3-tables-vet:
name: S3 Tables Go Vet Check
runs-on: ubuntu-22.04
timeout-minutes: 10
steps:
- name: Check out code
uses: actions/checkout@v6
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
id: go
- name: Run Go Vet
run: |
set -x
echo "=== Running go vet on S3 Tables package ==="
go vet ./weed/s3api/s3tables/... || {
echo "go vet check failed"
exit 1
}
echo "go vet checks passed"
- name: Run Go Vet on Tests
run: |
set -x
echo "=== Running go vet on S3 Tables tests ==="
go vet ./test/s3tables/... || {
echo "go vet check failed for tests"
exit 1
}
echo "go vet checks passed for tests"
- name: Run Go Vet on Iceberg
run: |
set -x
echo "=== Running go vet on Iceberg package ==="
go vet ./weed/s3api/iceberg/... || {
echo "go vet check failed for Iceberg"
exit 1
}
echo "go vet checks passed for Iceberg"
@@ -35,7 +35,7 @@ jobs:
set -e
mkdir -p /tmp/data
./weed -v=3 server -s3 -dir=/tmp/data -s3.config=../docker/compose/s3.json -master.peers=none > weed.log 2>&1 &
until curl -s http://localhost:8333/ > /dev/null; do sleep 1; done
until curl -s http://localhost:8333/healthz > /dev/null; do sleep 1; done
- name: Setup Caddy
run: |
@@ -54,7 +54,7 @@ jobs:
- name: Start Caddy
run: |
./caddy start
until curl -fsS --insecure https://localhost:8443 > /dev/null; do sleep 1; done
until curl -fsS --insecure https://localhost:8443/healthz > /dev/null; do sleep 1; done
- name: Create Bucket
run: |
@@ -0,0 +1,122 @@
name: "Volume Server Integration Tests"
on:
pull_request:
branches: [ master ]
paths:
- 'test/volume_server/**'
- 'weed/server/**'
- 'weed/storage/**'
- 'weed/pb/volume_server.proto'
- 'weed/pb/volume_server_pb/**'
- '.github/workflows/volume-server-integration-tests.yml'
push:
branches: [ master, main ]
paths:
- 'test/volume_server/**'
- 'weed/server/**'
- 'weed/storage/**'
- 'weed/pb/volume_server.proto'
- 'weed/pb/volume_server_pb/**'
- '.github/workflows/volume-server-integration-tests.yml'
concurrency:
group: ${{ github.head_ref || github.ref }}/volume-server-integration-tests
cancel-in-progress: true
permissions:
contents: read
env:
GO_VERSION: '1.24'
TEST_TIMEOUT: '30m'
jobs:
volume-server-integration-tests:
name: Volume Server Integration Tests (${{ matrix.test-type }} - Shard ${{ matrix.shard }})
runs-on: ubuntu-22.04
timeout-minutes: 45
strategy:
fail-fast: false
matrix:
test-type: [grpc, http]
shard: [1, 2, 3]
steps:
- name: Checkout code
uses: actions/checkout@v6
- name: Set up Go ${{ env.GO_VERSION }}
uses: actions/setup-go@v6
with:
go-version: ${{ env.GO_VERSION }}
- name: Build SeaweedFS binary
run: |
cd weed
go build -o weed .
chmod +x weed
./weed version
- name: Run volume server integration tests
env:
WEED_BINARY: ${{ github.workspace }}/weed/weed
run: |
if [ "${{ matrix.test-type }}" == "grpc" ]; then
if [ "${{ matrix.shard }}" == "1" ]; then
TEST_PATTERN="^Test[A-H]"
elif [ "${{ matrix.shard }}" == "2" ]; then
TEST_PATTERN="^Test[I-S]"
else
TEST_PATTERN="^Test[T-Z]"
fi
else
if [ "${{ matrix.shard }}" == "1" ]; then
TEST_PATTERN="^Test[A-G]"
elif [ "${{ matrix.shard }}" == "2" ]; then
TEST_PATTERN="^Test[H-R]"
else
TEST_PATTERN="^Test[S-Z]"
fi
fi
echo "Running volume server integration tests for ${{ matrix.test-type }} (Shard ${{ matrix.shard }}, pattern: ${TEST_PATTERN})..."
go test -v -count=1 -timeout=${{ env.TEST_TIMEOUT }} ./test/volume_server/${{ matrix.test-type }}/... -run "${TEST_PATTERN}"
- name: Collect logs on failure
if: failure()
run: |
mkdir -p /tmp/volume-server-it-logs
find /tmp -maxdepth 1 -type d -name "seaweedfs_volume_server_it_*" -print -exec cp -r {} /tmp/volume-server-it-logs/ \; || true
- name: Archive logs on failure
if: failure()
uses: actions/upload-artifact@v6
with:
name: volume-server-integration-test-logs
path: /tmp/volume-server-it-logs/
if-no-files-found: warn
retention-days: 7
- name: Test summary
if: always()
run: |
if [ "${{ matrix.test-type }}" == "grpc" ]; then
if [ "${{ matrix.shard }}" == "1" ]; then
TEST_PATTERN="^Test[A-H]"
elif [ "${{ matrix.shard }}" == "2" ]; then
TEST_PATTERN="^Test[I-S]"
else
TEST_PATTERN="^Test[T-Z]"
fi
else
if [ "${{ matrix.shard }}" == "1" ]; then
TEST_PATTERN="^Test[A-G]"
elif [ "${{ matrix.shard }}" == "2" ]; then
TEST_PATTERN="^Test[H-R]"
else
TEST_PATTERN="^Test[S-Z]"
fi
fi
echo "## Volume Server Integration Test Summary (${{ matrix.test-type }} - Shard ${{ matrix.shard }})" >> "$GITHUB_STEP_SUMMARY"
echo "- Suite: test/volume_server/${{ matrix.test-type }} (Pattern: ${TEST_PATTERN})" >> "$GITHUB_STEP_SUMMARY"
echo "- Command: go test -v -count=1 -timeout=${{ env.TEST_TIMEOUT }} ./test/volume_server/${{ matrix.test-type }}/... -run \"${TEST_PATTERN}\"" >> "$GITHUB_STEP_SUMMARY"
+1
View File
@@ -141,3 +141,4 @@ test/s3/iam/.test_env
/test/erasure_coding/admin_dockertest/tmp
/test/erasure_coding/admin_dockertest/task_logs
weed_bin
.aider*
-242
View File
@@ -1,242 +0,0 @@
# Bucket Policy Engine Integration - Complete
## Summary
Successfully integrated the `policy_engine` package to evaluate bucket policies for **all requests** (both anonymous and authenticated). This provides comprehensive AWS S3-compatible bucket policy support.
## What Changed
### 1. **New File: `s3api_bucket_policy_engine.go`**
Created a wrapper around `policy_engine.PolicyEngine` to:
- Load bucket policies from filer entries
- Sync policies from the bucket config cache
- Evaluate policies for any request (bucket, object, action, principal)
- Return structured results (allowed, evaluated, error)
### 2. **Modified: `s3api_server.go`**
- Added `policyEngine *BucketPolicyEngine` field to `S3ApiServer` struct
- Initialized the policy engine in `NewS3ApiServerWithStore()`
- Linked `IdentityAccessManagement` back to `S3ApiServer` for policy evaluation
### 3. **Modified: `auth_credentials.go`**
- Added `s3ApiServer *S3ApiServer` field to `IdentityAccessManagement` struct
- Added `buildPrincipalARN()` helper to convert identities to AWS ARN format
- **Integrated bucket policy evaluation into the authentication flow:**
- Policies are now checked **before** IAM/identity-based permissions
- Explicit `Deny` in bucket policy blocks access immediately
- Explicit `Allow` in bucket policy grants access and **bypasses IAM checks** (enables cross-account access)
- If no policy exists, falls through to normal IAM checks
- Policy evaluation errors result in access denial (fail-close security)
### 4. **Modified: `s3api_bucket_config.go`**
- Added policy engine sync when bucket configs are loaded
- Ensures policies are loaded into the engine for evaluation
### 5. **Modified: `auth_credentials_subscribe.go`**
- Added policy engine sync when bucket metadata changes
- Keeps the policy engine up-to-date via event-driven updates
## How It Works
### Anonymous Requests
```
1. Request comes in (no credentials)
2. Check ACL-based public access → if public, allow
3. Check bucket policy for anonymous ("*") access → if allowed, allow
4. Otherwise, deny
```
### Authenticated Requests (NEW!)
```
1. Request comes in (with credentials)
2. Authenticate user → get Identity
3. Build principal ARN (e.g., "arn:aws:iam::123456:user/bob")
4. Check bucket policy:
- If DENY → reject immediately
- If ALLOW → grant access immediately (bypasses IAM checks)
- If no policy or no matching statements → continue to step 5
5. Check IAM/identity-based permissions (only if not already allowed by bucket policy)
6. Allow or deny based on identity permissions
```
## Policy Evaluation Flow
```
┌─────────────────────────────────────────────────────────┐
│ Request (GET /bucket/file) │
└───────────────────────────┬─────────────────────────────┘
│
┌───────────▼──────────┐
│ Authenticate User │
│ (or Anonymous) │
└───────────┬──────────┘
│
┌───────────▼──────────────────────────────┐
│ Build Principal ARN │
│ - Anonymous: "*" │
│ - User: "arn:aws:iam::123456:user/bob" │
└───────────┬──────────────────────────────┘
│
┌───────────▼──────────────────────────────┐
│ Evaluate Bucket Policy (PolicyEngine) │
│ - Action: "s3:GetObject" │
│ - Resource: "arn:aws:s3:::bucket/file" │
│ - Principal: (from above) │
└───────────┬──────────────────────────────┘
│
┌─────────────┼─────────────┐
│ │ │
DENY │ ALLOW │ NO POLICY
│ │ │
▼ ▼ ▼
Reject Request Grant Access Continue
│
┌───────────────────┘
│
┌────────────▼─────────────┐
│ IAM/Identity Check │
│ (identity.canDo) │
└────────────┬─────────────┘
│
┌─────────┴─────────┐
│ │
ALLOW │ DENY │
▼ ▼
Grant Access Reject Request
```
## Example Policies That Now Work
### 1. **Public Read Access** (Anonymous)
```json
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::mybucket/*"
}]
}
```
- Anonymous users can read all objects
- Authenticated users are also evaluated against this policy. If they don't match an explicit `Allow` for this action, they will fall back to their own IAM permissions
### 2. **Grant Access to Specific User** (Authenticated)
```json
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::123456789012:user/bob"},
"Action": ["s3:GetObject", "s3:PutObject"],
"Resource": "arn:aws:s3:::mybucket/shared/*"
}]
}
```
- User "bob" can read/write objects in `/shared/` prefix
- Other users cannot (unless granted by their IAM policies)
### 3. **Deny Access to Specific Path** (Both)
```json
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": "arn:aws:s3:::mybucket/confidential/*"
}]
}
```
- **No one** can access `/confidential/` objects
- Denies override all other allows (AWS policy evaluation rules)
## Performance Characteristics
### Policy Loading
- **Cold start**: Policy loaded from filer → parsed → compiled → cached
- **Warm path**: Policy retrieved from `BucketConfigCache` (already parsed)
- **Updates**: Event-driven sync via metadata subscription (real-time)
### Policy Evaluation
- **Compiled policies**: Pre-compiled regex patterns and matchers
- **Pattern cache**: Regex patterns cached with LRU eviction (max 1000)
- **Fast path**: Common patterns (`*`, exact matches) optimized
- **Case sensitivity**: Actions case-insensitive, resources case-sensitive (AWS-compatible)
### Overhead
- **Anonymous requests**: Minimal (policy already checked, now using compiled engine)
- **Authenticated requests**: ~1-2ms added for policy evaluation (compiled patterns)
- **No policy**: Near-zero overhead (quick indeterminate check)
## Testing
All tests pass:
```bash
✅ TestBucketPolicyValidationBasics
✅ TestPrincipalMatchesAnonymous
✅ TestActionToS3Action
✅ TestResourceMatching
✅ TestMatchesPatternRegexEscaping (security tests)
✅ TestActionMatchingCaseInsensitive
✅ TestResourceMatchingCaseSensitive
✅ All policy_engine package tests (30+ tests)
```
## Security Improvements
1. **Regex Metacharacter Escaping**: Patterns like `*.json` properly match only files ending in `.json` (not `filexjson`)
2. **Case-Insensitive Actions**: S3 actions matched case-insensitively per AWS spec
3. **Case-Sensitive Resources**: Resource paths matched case-sensitively for security
4. **Pattern Cache Size Limit**: Prevents DoS attacks via unbounded cache growth
5. **Principal Validation**: Supports `[]string` for manually constructed policies
## AWS Compatibility
The implementation follows AWS S3 bucket policy evaluation rules:
1. **Explicit Deny** always wins (checked first)
2. **Explicit Allow** grants access (checked second)
3. **Default Deny** if no matching statements (implicit)
4. Bucket policies work alongside IAM policies (both are evaluated)
## Files Changed
```
Modified:
weed/s3api/auth_credentials.go (+47 lines)
weed/s3api/auth_credentials_subscribe.go (+8 lines)
weed/s3api/s3api_bucket_config.go (+8 lines)
weed/s3api/s3api_server.go (+5 lines)
New:
weed/s3api/s3api_bucket_policy_engine.go (115 lines)
```
## Migration Notes
- **Backward Compatible**: Existing setups without bucket policies work unchanged
- **No Breaking Changes**: All existing ACL and IAM-based authorization still works
- **Additive Feature**: Bucket policies are an additional layer of authorization
- **Performance**: Minimal impact on existing workloads
## Future Enhancements
Potential improvements (not implemented yet):
- [ ] Condition support (IP address, time-based, etc.) - already in policy_engine
- [ ] Cross-account policies (different AWS accounts)
- [ ] Policy validation API endpoint
- [ ] Policy simulation/testing tool
- [ ] Metrics for policy evaluations (allow/deny counts)
## Conclusion
Bucket policies now work for **all requests** in SeaweedFS S3 API:
- ✅ Anonymous requests (public access)
- ✅ Authenticated requests (user-specific policies)
- ✅ High performance (compiled policies, caching)
- ✅ AWS-compatible (follows AWS evaluation rules)
- ✅ Secure (proper escaping, case sensitivity)
The integration is complete, tested, and ready for use!
-413
View File
@@ -1,413 +0,0 @@
# SeaweedFS Task Distribution System Design
## Overview
This document describes the design of a distributed task management system for SeaweedFS that handles Erasure Coding (EC) and vacuum operations through a scalable admin server and worker process architecture.
## System Architecture
### High-Level Components
```
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ Master │◄──►│ Admin Server │◄──►│ Workers │
│ │ │ │ │ │
│ - Volume Info │ │ - Task Discovery │ │ - Task Exec │
│ - Shard Status │ │ - Task Assign │ │ - Progress │
│ - Heartbeats │ │ - Progress Track │ │ - Error Report │
└─────────────────┘ └──────────────────┘ └─────────────────┘
│ │ │
│ │ │
▼ ▼ ▼
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ Volume Servers │ │ Volume Monitor │ │ Task Execution │
│ │ │ │ │ │
│ - Store Volumes │ │ - Health Check │ │ - EC Convert │
│ - EC Shards │ │ - Usage Stats │ │ - Vacuum Clean │
│ - Report Status │ │ - State Sync │ │ - Status Report │
└─────────────────┘ └──────────────────┘ └─────────────────┘
```
## 1. Admin Server Design
### 1.1 Core Responsibilities
- **Task Discovery**: Scan volumes to identify EC and vacuum candidates
- **Worker Management**: Track available workers and their capabilities
- **Task Assignment**: Match tasks to optimal workers
- **Progress Tracking**: Monitor in-progress tasks for capacity planning
- **State Reconciliation**: Sync with master server for volume state updates
### 1.2 Task Discovery Engine
```go
type TaskDiscoveryEngine struct {
masterClient MasterClient
volumeScanner VolumeScanner
taskDetectors map[TaskType]TaskDetector
scanInterval time.Duration
}
type VolumeCandidate struct {
VolumeID uint32
Server string
Collection string
TaskType TaskType
Priority TaskPriority
Reason string
DetectedAt time.Time
Parameters map[string]interface{}
}
```
**EC Detection Logic**:
- Find volumes >= 95% full and idle for > 1 hour
- Exclude volumes already in EC format
- Exclude volumes with ongoing operations
- Prioritize by collection and age
**Vacuum Detection Logic**:
- Find volumes with garbage ratio > 30%
- Exclude read-only volumes
- Exclude volumes with recent vacuum operations
- Prioritize by garbage percentage
### 1.3 Worker Registry & Management
```go
type WorkerRegistry struct {
workers map[string]*Worker
capabilities map[TaskType][]*Worker
lastHeartbeat map[string]time.Time
taskAssignment map[string]*Task
mutex sync.RWMutex
}
type Worker struct {
ID string
Address string
Capabilities []TaskType
MaxConcurrent int
CurrentLoad int
Status WorkerStatus
LastSeen time.Time
Performance WorkerMetrics
}
```
### 1.4 Task Assignment Algorithm
```go
type TaskScheduler struct {
registry *WorkerRegistry
taskQueue *PriorityQueue
inProgressTasks map[string]*InProgressTask
volumeReservations map[uint32]*VolumeReservation
}
// Worker Selection Criteria:
// 1. Has required capability (EC or Vacuum)
// 2. Available capacity (CurrentLoad < MaxConcurrent)
// 3. Best performance history for task type
// 4. Lowest current load
// 5. Geographically close to volume server (optional)
```
## 2. Worker Process Design
### 2.1 Worker Architecture
```go
type MaintenanceWorker struct {
id string
config *WorkerConfig
adminClient AdminClient
taskExecutors map[TaskType]TaskExecutor
currentTasks map[string]*RunningTask
registry *TaskRegistry
heartbeatTicker *time.Ticker
requestTicker *time.Ticker
}
```
### 2.2 Task Execution Framework
```go
type TaskExecutor interface {
Execute(ctx context.Context, task *Task) error
EstimateTime(task *Task) time.Duration
ValidateResources(task *Task) error
GetProgress() float64
Cancel() error
}
type ErasureCodingExecutor struct {
volumeClient VolumeServerClient
progress float64
cancelled bool
}
type VacuumExecutor struct {
volumeClient VolumeServerClient
progress float64
cancelled bool
}
```
### 2.3 Worker Capabilities & Registration
```go
type WorkerCapabilities struct {
SupportedTasks []TaskType
MaxConcurrent int
ResourceLimits ResourceLimits
PreferredServers []string // Affinity for specific volume servers
}
type ResourceLimits struct {
MaxMemoryMB int64
MaxDiskSpaceMB int64
MaxNetworkMbps int64
MaxCPUPercent float64
}
```
## 3. Task Lifecycle Management
### 3.1 Task States
```go
type TaskState string
const (
TaskStatePending TaskState = "pending"
TaskStateAssigned TaskState = "assigned"
TaskStateInProgress TaskState = "in_progress"
TaskStateCompleted TaskState = "completed"
TaskStateFailed TaskState = "failed"
TaskStateCancelled TaskState = "cancelled"
TaskStateStuck TaskState = "stuck" // Taking too long
TaskStateDuplicate TaskState = "duplicate" // Detected duplicate
)
```
### 3.2 Progress Tracking & Monitoring
```go
type InProgressTask struct {
Task *Task
WorkerID string
StartedAt time.Time
LastUpdate time.Time
Progress float64
EstimatedEnd time.Time
VolumeReserved bool // Reserved for capacity planning
}
type TaskMonitor struct {
inProgressTasks map[string]*InProgressTask
timeoutChecker *time.Ticker
stuckDetector *time.Ticker
duplicateChecker *time.Ticker
}
```
## 4. Volume Capacity Reconciliation
### 4.1 Volume State Tracking
```go
type VolumeStateManager struct {
masterClient MasterClient
inProgressTasks map[uint32]*InProgressTask // VolumeID -> Task
committedChanges map[uint32]*VolumeChange // Changes not yet in master
reconcileInterval time.Duration
}
type VolumeChange struct {
VolumeID uint32
ChangeType ChangeType // "ec_encoding", "vacuum_completed"
OldCapacity int64
NewCapacity int64
TaskID string
CompletedAt time.Time
ReportedToMaster bool
}
```
### 4.2 Shard Assignment Integration
When the master needs to assign shards, it must consider:
1. **Current volume state** from its own records
2. **In-progress capacity changes** from admin server
3. **Committed but unreported changes** from admin server
```go
type CapacityOracle struct {
adminServer AdminServerClient
masterState *MasterVolumeState
updateFreq time.Duration
}
func (o *CapacityOracle) GetAdjustedCapacity(volumeID uint32) int64 {
baseCapacity := o.masterState.GetCapacity(volumeID)
// Adjust for in-progress tasks
if task := o.adminServer.GetInProgressTask(volumeID); task != nil {
switch task.Type {
case TaskTypeErasureCoding:
// EC reduces effective capacity
return baseCapacity / 2 // Simplified
case TaskTypeVacuum:
// Vacuum may increase available space
return baseCapacity + int64(float64(baseCapacity) * 0.3)
}
}
// Adjust for completed but unreported changes
if change := o.adminServer.GetPendingChange(volumeID); change != nil {
return change.NewCapacity
}
return baseCapacity
}
```
## 5. Error Handling & Recovery
### 5.1 Worker Failure Scenarios
```go
type FailureHandler struct {
taskRescheduler *TaskRescheduler
workerMonitor *WorkerMonitor
alertManager *AlertManager
}
// Failure Scenarios:
// 1. Worker becomes unresponsive (heartbeat timeout)
// 2. Task execution fails (reported by worker)
// 3. Task gets stuck (progress timeout)
// 4. Duplicate task detection
// 5. Resource exhaustion
```
### 5.2 Recovery Strategies
**Worker Timeout Recovery**:
- Mark worker as inactive after 3 missed heartbeats
- Reschedule all assigned tasks to other workers
- Cleanup any partial state
**Task Stuck Recovery**:
- Detect tasks with no progress for > 2x estimated time
- Cancel stuck task and mark volume for cleanup
- Reschedule if retry count < max_retries
**Duplicate Task Prevention**:
```go
type DuplicateDetector struct {
activeFingerprints map[string]bool // VolumeID+TaskType
recentCompleted *LRUCache // Recently completed tasks
}
func (d *DuplicateDetector) IsTaskDuplicate(task *Task) bool {
fingerprint := fmt.Sprintf("%d-%s", task.VolumeID, task.Type)
return d.activeFingerprints[fingerprint] ||
d.recentCompleted.Contains(fingerprint)
}
```
## 6. Simulation & Testing Framework
### 6.1 Failure Simulation
```go
type TaskSimulator struct {
scenarios map[string]SimulationScenario
}
type SimulationScenario struct {
Name string
WorkerCount int
VolumeCount int
FailurePatterns []FailurePattern
Duration time.Duration
}
type FailurePattern struct {
Type FailureType // "worker_timeout", "task_stuck", "duplicate"
Probability float64 // 0.0 to 1.0
Timing TimingSpec // When during task execution
Duration time.Duration
}
```
### 6.2 Test Scenarios
**Scenario 1: Worker Timeout During EC**
- Start EC task on 30GB volume
- Kill worker at 50% progress
- Verify task reassignment
- Verify no duplicate EC operations
**Scenario 2: Stuck Vacuum Task**
- Start vacuum on high-garbage volume
- Simulate worker hanging at 75% progress
- Verify timeout detection and cleanup
- Verify volume state consistency
**Scenario 3: Duplicate Task Prevention**
- Submit same EC task from multiple sources
- Verify only one task executes
- Verify proper conflict resolution
**Scenario 4: Master-Admin State Divergence**
- Create in-progress EC task
- Simulate master restart
- Verify state reconciliation
- Verify shard assignment accounts for in-progress work
## 7. Performance & Scalability
### 7.1 Metrics & Monitoring
```go
type SystemMetrics struct {
TasksPerSecond float64
WorkerUtilization float64
AverageTaskTime time.Duration
FailureRate float64
QueueDepth int
VolumeStatesSync bool
}
```
### 7.2 Scalability Considerations
- **Horizontal Worker Scaling**: Add workers without admin server changes
- **Admin Server HA**: Master-slave admin servers for fault tolerance
- **Task Partitioning**: Partition tasks by collection or datacenter
- **Batch Operations**: Group similar tasks for efficiency
## 8. Implementation Plan
### Phase 1: Core Infrastructure
1. Admin server basic framework
2. Worker registration and heartbeat
3. Simple task assignment
4. Basic progress tracking
### Phase 2: Advanced Features
1. Volume state reconciliation
2. Sophisticated worker selection
3. Failure detection and recovery
4. Duplicate prevention
### Phase 3: Optimization & Monitoring
1. Performance metrics
2. Load balancing algorithms
3. Capacity planning integration
4. Comprehensive monitoring
This design provides a robust, scalable foundation for distributed task management in SeaweedFS while maintaining consistency with the existing architecture patterns.
+1 -1
View File
@@ -40,7 +40,7 @@ test: admin-generate
# Admin component targets
admin-generate:
@echo "Generating admin component templates..."
@cd $(ADMIN_DIR) && $(MAKE) generate
@cd $(ADMIN_DIR) && templ generate ./view
admin-build: admin-generate
@echo "Building admin component..."
-145
View File
@@ -1,145 +0,0 @@
# SQL Query Engine Feature, Dev, and Test Plan
This document outlines the plan for adding SQL querying support to SeaweedFS, focusing on reading and analyzing data from Message Queue (MQ) topics.
## Feature Plan
**1. Goal**
To provide a SQL querying interface for SeaweedFS, enabling analytics on existing MQ topics. This enables:
- Basic querying with SELECT, WHERE, aggregations on MQ topics
- Schema discovery and metadata operations (SHOW DATABASES, SHOW TABLES, DESCRIBE)
- In-place analytics on Parquet-stored messages without data movement
**2. Key Features**
* **Schema Discovery and Metadata:**
* `SHOW DATABASES` - List all MQ namespaces
* `SHOW TABLES` - List all topics in a namespace
* `DESCRIBE table_name` - Show topic schema details
* Automatic schema detection from existing Parquet data
* **Basic Query Engine:**
* `SELECT` support with `WHERE`, `LIMIT`, `OFFSET`
* Aggregation functions: `COUNT()`, `SUM()`, `AVG()`, `MIN()`, `MAX()`
* Temporal queries with timestamp-based filtering
* **User Interfaces:**
* New CLI command `weed sql` with interactive shell mode
* Optional: Web UI for query execution and result visualization
* **Output Formats:**
* JSON (default), CSV, Parquet for result sets
* Streaming results for large queries
* Pagination support for result navigation
## Development Plan
**3. Data Source Integration**
* **MQ Topic Connector (Primary):**
* Build on existing `weed/mq/logstore/read_parquet_to_log.go`
* Implement efficient Parquet scanning with predicate pushdown
* Support schema evolution and backward compatibility
* Handle partition-based parallelism for scalable queries
* **Schema Registry Integration:**
* Extend `weed/mq/schema/schema.go` for SQL metadata operations
* Read existing topic schemas for query planning
* Handle schema evolution during query execution
**4. API & CLI Integration**
* **CLI Command:**
* New `weed sql` command with interactive shell mode (similar to `weed shell`)
* Support for script execution and result formatting
* Connection management for remote SeaweedFS clusters
* **gRPC API:**
* Add SQL service to existing MQ broker gRPC interface
* Enable efficient query execution with streaming results
## Example Usage Scenarios
**Scenario 1: Schema Discovery and Metadata**
```sql
-- List all namespaces (databases)
SHOW DATABASES;
-- List topics in a namespace
USE my_namespace;
SHOW TABLES;
-- View topic structure and discovered schema
DESCRIBE user_events;
```
**Scenario 2: Data Querying**
```sql
-- Basic filtering and projection
SELECT user_id, event_type, timestamp
FROM user_events
WHERE timestamp > 1640995200000
LIMIT 100;
-- Aggregation queries
SELECT COUNT(*) as event_count
FROM user_events
WHERE timestamp >= 1640995200000;
-- More aggregation examples
SELECT MAX(timestamp), MIN(timestamp)
FROM user_events;
```
**Scenario 3: Analytics & Monitoring**
```sql
-- Basic analytics
SELECT COUNT(*) as total_events
FROM user_events
WHERE timestamp >= 1640995200000;
-- Simple monitoring
SELECT AVG(response_time) as avg_response
FROM api_logs
WHERE timestamp >= 1640995200000;
## Architecture Overview
```
SQL Query Flow:
1. Parse SQL 2. Plan & Optimize 3. Execute Query
┌─────────────┐ ┌──────────────┐ ┌─────────────────┐ ┌──────────────┐
│ Client │ │ SQL Parser │ │ Query Planner │ │ Execution │
│ (CLI) │──→ │ PostgreSQL │──→ │ & Optimizer │──→ │ Engine │
│ │ │ (Custom) │ │ │ │ │
└─────────────┘ └──────────────┘ └─────────────────┘ └──────────────┘
│ │
│ Schema Lookup │ Data Access
▼ ▼
┌─────────────────────────────────────────────────────────────┐
│ Schema Catalog │
│ • Namespace → Database mapping │
│ • Topic → Table mapping │
│ • Schema version management │
└─────────────────────────────────────────────────────────────┘
▲
│ Metadata
│
┌─────────────────────────────────────────────────────────────────────────────┐
│ MQ Storage Layer │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ▲ │
│ │ Topic A │ │ Topic B │ │ Topic C │ │ ... │ │ │
│ │ (Parquet) │ │ (Parquet) │ │ (Parquet) │ │ (Parquet) │ │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │
└──────────────────────────────────────────────────────────────────────────│──┘
│
Data Access
```
## Success Metrics
* **Feature Completeness:** Support for all specified SELECT operations and metadata commands
* **Performance:**
* **Simple SELECT queries**: < 100ms latency for single-table queries with up to 3 WHERE predicates on ≤ 100K records
* **Complex queries**: < 1s latency for queries involving aggregations (COUNT, SUM, MAX, MIN) on ≤ 1M records
* **Time-range queries**: < 500ms for timestamp-based filtering on ≤ 500K records within 24-hour windows
* **Scalability:** Handle topics with millions of messages efficiently
+1 -1
View File
@@ -3,7 +3,7 @@ FROM golang:1.24 AS builder
RUN apt-get update
RUN apt-get install -y build-essential libsnappy-dev zlib1g-dev libbz2-dev libgflags-dev liblz4-dev libzstd-dev
ARG ROCKSDB_VERSION=v10.5.1
ARG ROCKSDB_VERSION=v10.10.1
ENV ROCKSDB_VERSION=${ROCKSDB_VERSION}
# build RocksDB
+1 -1
View File
@@ -3,7 +3,7 @@ FROM golang:1.24 AS builder
RUN apt-get update
RUN apt-get install -y build-essential libsnappy-dev zlib1g-dev libbz2-dev libgflags-dev liblz4-dev libzstd-dev
ARG ROCKSDB_VERSION=v10.5.1
ARG ROCKSDB_VERSION=v10.10.1
ENV ROCKSDB_VERSION=${ROCKSDB_VERSION}
# build RocksDB
+6 -2
View File
@@ -20,13 +20,17 @@ if [ "$(id -u)" = "0" ]; then
DATA_UID=$(stat -c '%u' /data 2>/dev/null)
DATA_GID=$(stat -c '%g' /data 2>/dev/null)
# Only run chown -R if ownership doesn't match (much faster for subsequent starts)
# Only run chown -R if ownership doesn't already match (avoids expensive
# recursive chown on subsequent starts, and is a no-op on OpenShift when
# fsGroup has already set correct ownership on the PVC).
if [ "$DATA_UID" != "$SEAWEED_UID" ] || [ "$DATA_GID" != "$SEAWEED_GID" ]; then
echo "Fixing /data ownership for seaweed user (uid=$SEAWEED_UID, gid=$SEAWEED_GID)"
if ! chown -R seaweed:seaweed /data; then
echo "Warning: Failed to change ownership of /data. This may cause permission errors." >&2
echo "If /data is read-only or has mount issues, the application may fail to start." >&2
fi
fi
# Use su-exec to drop privileges and run as seaweed user
exec su-exec seaweed "$0" "$@"
+9
View File
@@ -42,3 +42,12 @@ services:
- master
- volume
- filer
admin:
image: chrislusf/seaweedfs:dev # use a remote dev image
ports:
- 23646:23646
command: 'admin -master=master:9333'
depends_on:
- master
- volume
- filer
+34
View File
@@ -0,0 +1,34 @@
# Put this file to one of the location, with descending priority
# ./security.toml
# $HOME/.seaweedfs/security.toml
# /etc/seaweedfs/security.toml
# this file is read by master, volume server, filer, and worker
# comma separated origins allowed to make requests to the filer and s3 gateway.
# enter in this format: https://domain.com, or http://localhost:port
[cors.allowed_origins]
values = "*"
# this jwt signing key is read by master and volume server, and it is used for write operations:
# - the Master server generates the JWT, which can be used to write a certain file on a volume server
# - the Volume server validates the JWT on writing
# the jwt defaults to expire after 10 seconds.
# [jwt.signing]
# key = "V1JJVEVTRUNSRVRFWEFNUExFMTIzNDU2Nzg5MDEy" # Example: WRITESECRETEXAMPLE123456789012
# this jwt signing key is read by master and volume server, and it is used for read operations:
# - the Master server generates the JWT, which can be used to read a certain file on a volume server
# - the Volume server validates the JWT on reading
# [jwt.signing.read]
# key = "UkVBRFNFQ1JFVUVYQU1QTEUxMjM0NTY3ODkwMTI=" # Example: READSECRETEXAMPLE123456789012
# If this JWT key is configured, Filer only accepts writes over HTTP if they are signed with this JWT:
# - f.e. the S3 API Shim generates the JWT
# - the Filer server validates the JWT on writing
# the jwt defaults to expire after 10 seconds.
# [jwt.filer_signing]
# key = "RklMRVJXUklURVNFQ1JFVEVYQU1QTEUxMjM0NTY3OA==" # Example: FILERWRITESECRETEXAMPLE12345678
# If this JWT key is configured, Filer only accepts reads over HTTP if they are signed with this JWT:
# - f.e. the S3 API Shim generates the JWT
# - the Filer server validates the JWT on reading
# the jwt defaults to expire after 10 seconds.
# [jwt.filer_signing.read]
# key = "RklMRVJSRUFEU0VDUkVURVhBTVBMRTEyMzQ1Njc4OQ==" # Example: FILERREADSECRETEXAMPLE123456789
+81 -36
View File
@@ -5,7 +5,7 @@ go 1.24.9
require (
cloud.google.com/go v0.123.0 // indirect
cloud.google.com/go/pubsub v1.50.1
cloud.google.com/go/storage v1.59.1
cloud.google.com/go/storage v1.59.2
github.com/Shopify/sarama v1.38.1
github.com/aws/aws-sdk-go v1.55.8
github.com/beorn7/perks v1.0.1 // indirect
@@ -39,17 +39,17 @@ require (
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/go-uuid v1.0.3 // indirect
github.com/jackc/pgx/v5 v5.7.6
github.com/jackc/pgx/v5 v5.8.0
github.com/jcmturner/gofork v1.7.6 // indirect
github.com/jcmturner/gokrb5/v8 v8.4.4 // indirect
github.com/jinzhu/copier v0.4.0
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/json-iterator/go v1.1.12
github.com/karlseguin/ccache/v2 v2.0.8
github.com/klauspost/compress v1.18.2
github.com/klauspost/compress v1.18.4
github.com/klauspost/reedsolomon v1.13.0
github.com/kurin/blazer v0.5.3
github.com/linxGnu/grocksdb v1.10.3
github.com/linxGnu/grocksdb v1.10.7
github.com/mailru/easyjson v0.9.1 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
@@ -86,20 +86,20 @@ require (
github.com/xdg-go/scram v1.1.2 // indirect
github.com/xdg-go/stringprep v1.0.4 // indirect
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 // indirect
go.etcd.io/etcd/client/v3 v3.6.6
go.etcd.io/etcd/client/v3 v3.6.7
go.mongodb.org/mongo-driver v1.17.6
go.opencensus.io v0.24.0 // indirect
gocloud.dev v0.44.0
gocloud.dev/pubsub/natspubsub v0.44.0
gocloud.dev/pubsub/rabbitpubsub v0.44.0
golang.org/x/crypto v0.46.0
golang.org/x/crypto v0.48.0
golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546
golang.org/x/image v0.35.0
golang.org/x/net v0.48.0
golang.org/x/net v0.49.0
golang.org/x/oauth2 v0.34.0
golang.org/x/sys v0.39.0
golang.org/x/text v0.33.0 // indirect
golang.org/x/tools v0.40.0 // indirect
golang.org/x/sys v0.41.0
golang.org/x/text v0.34.0 // indirect
golang.org/x/tools v0.41.0 // indirect
golang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da // indirect
google.golang.org/api v0.258.0
google.golang.org/genproto v0.0.0-20250922171735-9219d122eba9 // indirect
@@ -109,7 +109,7 @@ require (
modernc.org/b v1.0.0 // indirect
modernc.org/mathutil v1.7.1
modernc.org/memory v1.11.0 // indirect
modernc.org/sqlite v1.44.2
modernc.org/sqlite v1.44.3
modernc.org/strutil v1.2.1
)
@@ -118,8 +118,9 @@ require (
github.com/Azure/azure-sdk-for-go/sdk/keyvault/azkeys v0.10.0
github.com/Jille/raft-grpc-transport v1.6.1
github.com/ThreeDotsLabs/watermill v1.5.1
github.com/a-h/templ v0.3.943
github.com/a-h/templ v0.3.977
github.com/apache/cassandra-gocql-driver/v2 v2.0.0
github.com/apache/iceberg-go v0.4.0
github.com/apple/foundationdb/bindings/go v0.0.0-20250911184653-27f7192f47c3
github.com/arangodb/go-driver v1.6.9
github.com/armon/go-metrics v0.4.1
@@ -129,19 +130,19 @@ require (
github.com/aws/aws-sdk-go-v2/service/s3 v1.95.0
github.com/cognusion/imaging v1.0.2
github.com/fluent/fluent-logger-golang v1.10.1
github.com/getsentry/sentry-go v0.40.0
github.com/getsentry/sentry-go v0.42.0
github.com/gin-contrib/sessions v1.0.4
github.com/gin-gonic/gin v1.11.0
github.com/go-ldap/ldap/v3 v3.4.12
github.com/golang-jwt/jwt/v5 v5.3.0
github.com/golang-jwt/jwt/v5 v5.3.1
github.com/google/flatbuffers/go v0.0.0-20230108230133-3b8644d32c50
github.com/hashicorp/raft v1.7.3
github.com/hashicorp/raft-boltdb/v2 v2.3.1
github.com/hashicorp/vault/api v1.22.0
github.com/jhump/protoreflect v1.17.0
github.com/lib/pq v1.10.9
github.com/jhump/protoreflect v1.18.0
github.com/lib/pq v1.11.1
github.com/linkedin/goavro/v2 v2.14.1
github.com/mattn/go-sqlite3 v1.14.33
github.com/mattn/go-sqlite3 v1.14.34
github.com/minio/crc64nvme v1.1.1
github.com/orcaman/concurrent-map/v2 v2.0.1
github.com/parquet-go/parquet-go v0.26.4
@@ -149,15 +150,16 @@ require (
github.com/rabbitmq/amqp091-go v1.10.0
github.com/rclone/rclone v1.72.1
github.com/rdleal/intervalst v1.5.0
github.com/redis/go-redis/v9 v9.17.2
github.com/redis/go-redis/v9 v9.18.0
github.com/schollz/progressbar/v3 v3.19.0
github.com/seaweedfs/go-fuse/v2 v2.9.1
github.com/shirou/gopsutil/v4 v4.25.12
github.com/shirou/gopsutil/v4 v4.26.1
github.com/tarantool/go-tarantool/v2 v2.4.1
github.com/testcontainers/testcontainers-go v0.39.0
github.com/tikv/client-go/v2 v2.0.7
github.com/xeipuuv/gojsonschema v1.2.0
github.com/ydb-platform/ydb-go-sdk-auth-environ v0.5.1
github.com/ydb-platform/ydb-go-sdk/v3 v3.125.1
github.com/ydb-platform/ydb-go-sdk/v3 v3.125.3
go.etcd.io/etcd/client/pkg/v3 v3.6.7
go.uber.org/atomic v1.11.0
golang.org/x/sync v0.19.0
@@ -168,36 +170,57 @@ require (
require github.com/k0kubun/colorstring v0.0.0-20150214042306-9440f1994b88 // indirect
require (
atomicgo.dev/cursor v0.2.0 // indirect
atomicgo.dev/keyboard v0.2.9 // indirect
atomicgo.dev/schedule v0.1.0 // indirect
cloud.google.com/go/longrunning v0.7.0 // indirect
cloud.google.com/go/pubsub/v2 v2.2.1 // indirect
dario.cat/mergo v1.0.2 // indirect
github.com/Azure/azure-sdk-for-go/sdk/keyvault/internal v0.7.1 // indirect
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect
github.com/Azure/go-autorest v14.2.0+incompatible // indirect
github.com/Azure/go-autorest/autorest/to v0.4.1 // indirect
github.com/a1ex3/zstd-seekable-format-go/pkg v0.10.0 // indirect
github.com/anchore/go-lzo v0.1.0 // indirect
github.com/antlr4-go/antlr/v4 v4.13.1 // indirect
github.com/apache/arrow-go/v18 v18.4.1 // indirect
github.com/apache/thrift v0.22.0 // indirect
github.com/aws/aws-sdk-go-v2/service/signin v1.0.5 // indirect
github.com/bahlo/generic-list-go v0.2.0 // indirect
github.com/bazelbuild/rules_go v0.46.0 // indirect
github.com/biogo/store v0.0.0-20201120204734-aad293a2328f // indirect
github.com/blevesearch/snowballstem v0.9.0 // indirect
github.com/boombuler/barcode v1.1.0 // indirect
github.com/bufbuild/protocompile v0.14.1 // indirect
github.com/buger/jsonparser v1.1.1 // indirect
github.com/cenkalti/backoff/v5 v5.0.3 // indirect
github.com/clipperhouse/stringish v0.1.1 // indirect
github.com/clipperhouse/uax29/v2 v2.3.0 // indirect
github.com/cockroachdb/apd/v3 v3.1.0 // indirect
github.com/cockroachdb/apd/v3 v3.2.1 // indirect
github.com/cockroachdb/errors v1.11.3 // indirect
github.com/cockroachdb/logtags v0.0.0-20241215232642-bb51bb14a506 // indirect
github.com/cockroachdb/redact v1.1.5 // indirect
github.com/cockroachdb/version v0.0.0-20250314144055-3860cd14adf2 // indirect
github.com/containerd/console v1.0.5 // indirect
github.com/containerd/errdefs v1.0.0 // indirect
github.com/containerd/errdefs/pkg v0.3.0 // indirect
github.com/containerd/log v0.1.0 // indirect
github.com/containerd/platforms v1.0.0-rc.1 // indirect
github.com/cpuguy83/dockercfg v0.3.2 // indirect
github.com/dave/dst v0.27.2 // indirect
github.com/diskfs/go-diskfs v1.7.0 // indirect
github.com/distribution/reference v0.6.0 // indirect
github.com/docker/docker v28.5.0+incompatible // indirect
github.com/docker/go-connections v0.6.0 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/go-asn1-ber/asn1-ber v1.5.8-0.20250403174932-29230038a667 // indirect
github.com/go-git/go-billy/v5 v5.6.2 // indirect
github.com/goccy/go-yaml v1.18.0 // indirect
github.com/golang/geo v0.0.0-20210211234256-740aa86cb551 // indirect
github.com/google/go-cmp v0.7.0 // indirect
github.com/gookit/color v1.5.4 // indirect
github.com/gopherjs/gopherjs v1.17.2 // indirect
github.com/grpc-ecosystem/grpc-gateway v1.16.0 // indirect
github.com/hamba/avro/v2 v2.30.0 // indirect
github.com/hashicorp/go-rootcerts v1.0.2 // indirect
github.com/hashicorp/go-secure-stdlib/parseutil v0.2.0 // indirect
github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 // indirect
@@ -207,16 +230,33 @@ require (
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/jaegertracing/jaeger v1.47.0 // indirect
github.com/jhump/protoreflect/v2 v2.0.0-beta.1 // indirect
github.com/jtolds/gls v4.20.0+incompatible // indirect
github.com/klauspost/asmfmt v1.3.2 // indirect
github.com/kr/pretty v0.3.1 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/lithammer/fuzzysearch v1.1.8 // indirect
github.com/lithammer/shortuuid/v3 v3.0.7 // indirect
github.com/magiconair/properties v1.8.10 // indirect
github.com/minio/asm2plan9s v0.0.0-20200509001527-cdd76441f9d8 // indirect
github.com/minio/c2goasm v0.0.0-20190812172519-36a3d3bbc4f3 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/go-archive v0.1.0 // indirect
github.com/moby/patternmatcher v0.6.0 // indirect
github.com/moby/sys/sequential v0.6.0 // indirect
github.com/moby/sys/user v0.4.0 // indirect
github.com/moby/sys/userns v0.1.0 // indirect
github.com/moby/term v0.5.2 // indirect
github.com/morikuni/aec v1.0.0 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.1 // indirect
github.com/openzipkin/zipkin-go v0.4.3 // indirect
github.com/parquet-go/bitpack v1.0.0 // indirect
github.com/parquet-go/jsonlite v1.0.0 // indirect
github.com/petermattis/goid v0.0.0-20180202154549-b0b1615b78e5 // indirect
github.com/petermattis/goid v0.0.0-20260113132338-7c7de50cc741 // indirect
github.com/pierrre/geohash v1.0.0 // indirect
github.com/pquerna/otp v1.5.0 // indirect
github.com/pterm/pterm v0.12.81 // indirect
github.com/quic-go/qpack v0.6.0 // indirect
github.com/quic-go/quic-go v0.57.0 // indirect
github.com/rogpeppe/go-internal v1.14.1 // indirect
@@ -224,12 +264,16 @@ require (
github.com/sasha-s/go-deadlock v0.3.1 // indirect
github.com/smarty/assertions v1.15.0 // indirect
github.com/stretchr/objx v0.5.2 // indirect
github.com/substrait-io/substrait v0.69.0 // indirect
github.com/substrait-io/substrait-go/v4 v4.4.0 // indirect
github.com/substrait-io/substrait-protobuf/go v0.71.0 // indirect
github.com/twpayne/go-geom v1.4.1 // indirect
github.com/twpayne/go-kml v1.5.2 // indirect
github.com/ulikunitz/xz v0.5.15 // indirect
github.com/wk8/go-ordered-map/v2 v2.1.8 // indirect
github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f // indirect
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb // indirect
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
github.com/zeebo/xxh3 v1.0.2 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.37.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.37.0 // indirect
@@ -237,7 +281,8 @@ require (
go.opentelemetry.io/proto/otlp v1.7.0 // indirect
go.yaml.in/yaml/v2 v2.4.3 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/mod v0.31.0 // indirect
golang.org/x/mod v0.32.0 // indirect
golang.org/x/telemetry v0.0.0-20260109210033-bd525da824e2 // indirect
gonum.org/v1/gonum v0.16.0 // indirect
)
@@ -248,13 +293,13 @@ require (
cloud.google.com/go/compute/metadata v0.9.0 // indirect
cloud.google.com/go/iam v1.5.3 // indirect
cloud.google.com/go/monitoring v1.24.2 // indirect
filippo.io/edwards25519 v1.1.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.20.0
filippo.io/edwards25519 v1.1.1 // indirect
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.21.0
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.13.1
github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2 // indirect
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.3
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.4
github.com/Azure/azure-sdk-for-go/sdk/storage/azfile v1.5.3 // indirect
github.com/Azure/go-ntlmssp v0.0.2-0.20251110135918-10b7b7e7cd26 // indirect
github.com/Azure/go-ntlmssp v0.1.0 // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v1.6.0 // indirect
github.com/Files-com/files-sdk-go/v3 v3.2.264 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.30.0 // indirect
@@ -291,7 +336,7 @@ require (
github.com/aws/aws-sdk-go-v2/service/sso v1.30.9 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.13 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.41.6 // indirect
github.com/aws/smithy-go v1.24.0 // indirect
github.com/aws/smithy-go v1.24.0
github.com/boltdb/bolt v1.3.1 // indirect
github.com/bradenaw/juniper v0.15.3 // indirect
github.com/bradfitz/iter v0.0.0-20191230175014-e8f45d346db8 // indirect
@@ -349,7 +394,7 @@ require (
github.com/gorilla/securecookie v1.1.2 // indirect
github.com/gorilla/sessions v1.4.0 // indirect
github.com/grpc-ecosystem/go-grpc-middleware v1.4.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.1 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.3 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-hclog v1.6.3 // indirect
github.com/hashicorp/go-immutable-radix v1.3.1 // indirect
@@ -432,7 +477,7 @@ require (
github.com/tklauser/go-sysconf v0.3.16 // indirect
github.com/tklauser/numcpus v0.11.0 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/twmb/murmur3 v1.1.3 // indirect
github.com/twmb/murmur3 v1.1.8 // indirect
github.com/ugorji/go/codec v1.3.0 // indirect
github.com/unknwon/goconfig v1.0.0 // indirect
github.com/vmihailenco/msgpack/v5 v5.4.1 // indirect
@@ -447,7 +492,7 @@ require (
github.com/zeebo/blake3 v0.2.4 // indirect
github.com/zeebo/errs v1.4.0 // indirect
go.etcd.io/bbolt v1.4.3 // indirect
go.etcd.io/etcd/api/v3 v3.6.6 // indirect
go.etcd.io/etcd/api/v3 v3.6.7 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/detectors/gcp v1.38.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.63.0 // indirect
@@ -458,11 +503,11 @@ require (
go.opentelemetry.io/otel/sdk/metric v1.38.0 // indirect
go.opentelemetry.io/otel/trace v1.38.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.27.0 // indirect
go.uber.org/zap v1.27.1 // indirect
golang.org/x/arch v0.20.0 // indirect
golang.org/x/term v0.38.0 // indirect
golang.org/x/term v0.40.0 // indirect
golang.org/x/time v0.14.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20251111163417-95abcf5c77ba // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20251124214823-79d6a2a48846 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20251213004720-97cd9d5aeac2 // indirect
gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect
gopkg.in/validator.v2 v2.0.1 // indirect
+375 -70
View File
@@ -1,3 +1,11 @@
atomicgo.dev/assert v0.0.2 h1:FiKeMiZSgRrZsPo9qn/7vmr7mCsh5SZyXY4YGYiYwrg=
atomicgo.dev/assert v0.0.2/go.mod h1:ut4NcI3QDdJtlmAxQULOmA13Gz6e2DWbSAS8RUOmNYQ=
atomicgo.dev/cursor v0.2.0 h1:H6XN5alUJ52FZZUkI7AlJbUc1aW38GWZalpYRPpoPOw=
atomicgo.dev/cursor v0.2.0/go.mod h1:Lr4ZJB3U7DfPPOkbH7/6TOtJ4vFGHlgj1nc+n900IpU=
atomicgo.dev/keyboard v0.2.9 h1:tOsIid3nlPLZ3lwgG8KZMp/SFmr7P0ssEN5JUsm78K8=
atomicgo.dev/keyboard v0.2.9/go.mod h1:BC4w9g00XkxH/f1HXhW2sXmJFOCWbKn9xrOunSFtExQ=
atomicgo.dev/schedule v0.1.0 h1:nTthAbhZS5YZmgYbb2+DH8uQIZcTlIrd4eYr3UQxEjs=
atomicgo.dev/schedule v0.1.0/go.mod h1:xeUa3oAkiuHYh8bKiQBRojqAMq3PXXbJujjb0hw8pEU=
cel.dev/expr v0.24.0 h1:56OvJKSH3hDGL0ml5uSxZmz3/3Pq4tJ+fb1unVLAFcY=
cel.dev/expr v0.24.0/go.mod h1:hLPLo1W4QUmuYdA72RBX06QTs6MXw941piREPl3Yfiw=
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
@@ -477,8 +485,8 @@ cloud.google.com/go/storage v1.22.1/go.mod h1:S8N1cAStu7BOeFfE8KAQzmyyLkK8p/vmRq
cloud.google.com/go/storage v1.23.0/go.mod h1:vOEEDNFnciUMhBeT6hsJIn3ieU5cFRmzeLgDvXzfIXc=
cloud.google.com/go/storage v1.27.0/go.mod h1:x9DOL8TK/ygDUMieqwfhdpQryTeEkhGKMi80i/iqR2s=
cloud.google.com/go/storage v1.28.1/go.mod h1:Qnisd4CqDdo6BGs2AD5LLnEsmSQ80wQ5ogcBBKhU86Y=
cloud.google.com/go/storage v1.59.1 h1:DXAZLcTimtiXdGqDSnebROVPd9QvRsFVVlptz02Wk58=
cloud.google.com/go/storage v1.59.1/go.mod h1:cMWbtM+anpC74gn6qjLh+exqYcfmB9Hqe5z6adx+CLI=
cloud.google.com/go/storage v1.59.2 h1:gmOAuG1opU8YvycMNpP+DvHfT9BfzzK5Cy+arP+Nocw=
cloud.google.com/go/storage v1.59.2/go.mod h1:cMWbtM+anpC74gn6qjLh+exqYcfmB9Hqe5z6adx+CLI=
cloud.google.com/go/storagetransfer v1.5.0/go.mod h1:dxNzUopWy7RQevYFHewchb29POFv3/AaBgnhqzqiK0w=
cloud.google.com/go/storagetransfer v1.6.0/go.mod h1:y77xm4CQV/ZhFZH75PLEXY0ROiS7Gh6pSKrM8dJyg6I=
cloud.google.com/go/storagetransfer v1.7.0/go.mod h1:8Giuj1QNb1kfLAiWM1bN6dHzfdlDAVC9rv9abHot2W4=
@@ -536,13 +544,19 @@ cloud.google.com/go/workflows v1.7.0/go.mod h1:JhSrZuVZWuiDfKEFxU0/F1PQjmpnpcoIS
cloud.google.com/go/workflows v1.8.0/go.mod h1:ysGhmEajwZxGn1OhGOGKsTXc5PyxOc0vfKf5Af+to4M=
cloud.google.com/go/workflows v1.9.0/go.mod h1:ZGkj1aFIOd9c8Gerkjjq7OW7I5+l6cSvT3ujaO/WwSA=
cloud.google.com/go/workflows v1.10.0/go.mod h1:fZ8LmRmZQWacon9UCX1r/g/DfAXx5VcPALq2CxzdePw=
dario.cat/mergo v1.0.2 h1:85+piFYR1tMbRrLcDwR18y4UKJ3aH1Tbzi24VRW1TK8=
dario.cat/mergo v1.0.2/go.mod h1:E/hbnu0NxMFBjpMIE34DRGLWqDy0g5FuKDhCb31ngxA=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
filippo.io/edwards25519 v1.1.0 h1:FNf4tywRC1HmFuKW5xopWpigGjJKiJSV0Cqo0cJWDaA=
filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4=
filippo.io/edwards25519 v1.1.1 h1:YpjwWWlNmGIDyXOn8zLzqiD+9TyIlPhGFG96P39uBpw=
filippo.io/edwards25519 v1.1.1/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4=
gioui.org v0.0.0-20210308172011-57750fc8a0a6/go.mod h1:RSH6KIUZ0p2xy5zHDxgAM4zumjgTw83q2ge/PI+yyw8=
git.sr.ht/~sbinet/gg v0.3.1/go.mod h1:KGYtlADtqsqANL9ueOFkWymvzUvLMQllU5Ixo+8v3pc=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.20.0 h1:JXg2dwJUmPB9JmtVmdEB16APJ7jurfbY5jnfXpJoRMc=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.20.0/go.mod h1:YD5h/ldMsG0XiIw7PdyNhLxaM317eFh5yNLccNfGdyw=
github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6 h1:He8afgbRMd7mFxO99hRNu+6tazq8nFF9lIwo9JFroBk=
github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6/go.mod h1:8o94RPi1/7XTJvwPpRSzSUedZrtlirdB3r9Z20bi2f8=
github.com/AlecAivazis/survey/v2 v2.3.7 h1:6I/u8FvytdGsgonrYsVn2t8t4QiRnh6QSTqkkhIiSjQ=
github.com/AlecAivazis/survey/v2 v2.3.7/go.mod h1:xUTIdE4KCOIjsBAE1JYsUPoCqYdZ1reCfTwbto0Fduo=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.21.0 h1:fou+2+WFTib47nS+nz/ozhEBnvU96bKHy6LjRsY4E28=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.21.0/go.mod h1:t76Ruy8AHvUAC8GfMWJMa0ElSbuIcO03NLpynfbgsPA=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.13.1 h1:Hk5QBxZQC1jb2Fwj6mpzme37xbCDdNTxU7O9eb5+LB4=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.13.1/go.mod h1:IYus9qsFobWIc2YVwe/WPjcnyCkPKtnHAqUYeebc8z0=
github.com/Azure/azure-sdk-for-go/sdk/azidentity/cache v0.3.2 h1:yz1bePFlP5Vws5+8ez6T3HWXPmwOK7Yvq8QxDBD3SKY=
@@ -555,13 +569,19 @@ github.com/Azure/azure-sdk-for-go/sdk/keyvault/internal v0.7.1 h1:FbH3BbSb4bvGlu
github.com/Azure/azure-sdk-for-go/sdk/keyvault/internal v0.7.1/go.mod h1:9V2j0jn9jDEkCkv8w/bKTNppX/d0FVA1ud77xCIP4KA=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/storage/armstorage v1.8.1 h1:/Zt+cDPnpC3OVDm/JKLOs7M2DKmLRIIp3XIx9pHHiig=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/storage/armstorage v1.8.1/go.mod h1:Ng3urmn6dYe8gnbCMoHHVl5APYz2txho3koEkV2o2HA=
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.3 h1:ZJJNFaQ86GVKQ9ehwqyAFE6pIfyicpuJ8IkVaPBc6/4=
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.3/go.mod h1:URuDvhmATVKqHBH9/0nOiNKk0+YcwfQ3WkK5PqHKxc8=
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.4 h1:jWQK1GI+LeGGUKBADtcH2rRqPxYB1Ljwms5gFA2LqrM=
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.4/go.mod h1:8mwH4klAm9DUgR2EEHyEEAQlRDvLPyg5fQry3y+cDew=
github.com/Azure/azure-sdk-for-go/sdk/storage/azfile v1.5.3 h1:sxgSqOB9CDToiaVFpxuvb5wGgGqWa3lCShcm5o0n3bE=
github.com/Azure/azure-sdk-for-go/sdk/storage/azfile v1.5.3/go.mod h1:XdED8i399lEVblYHTZM8eXaP07gv4Z58IL6ueMlVlrg=
github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78/go.mod h1:LmzpDX56iTiv29bbRTIsUNlaFfuhWRQBWjQdVyAevI8=
github.com/Azure/go-ntlmssp v0.0.2-0.20251110135918-10b7b7e7cd26 h1:gy/jrlpp8EfSyA73a51fofoSfhp5rPNQAUvDr4Dm91c=
github.com/Azure/go-ntlmssp v0.0.2-0.20251110135918-10b7b7e7cd26/go.mod h1:NYqdhxd/8aAct/s4qSYZEerdPuH1liG2/X9DiVTbhpk=
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c h1:udKWzYgxTojEKWjV8V+WSxDXJ4NFATAsZjh8iIbsQIg=
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E=
github.com/Azure/go-autorest v14.2.0+incompatible h1:V5VMDjClD3GiElqLWO7mz2MxNAK/vTfRHdAubSIPRgs=
github.com/Azure/go-autorest v14.2.0+incompatible/go.mod h1:r+4oMnoxhatjLLJ6zxSWATqVooLgysK6ZNox3g/xq24=
github.com/Azure/go-autorest/autorest/to v0.4.1 h1:CxNHBqdzTr7rLtdrtb5CMjJcDut+WNGCVv7OmS5+lTc=
github.com/Azure/go-autorest/autorest/to v0.4.1/go.mod h1:EtaofgU4zmtvn1zT2ARsjRFdq9vXx0YWtmElwL+GZ9M=
github.com/Azure/go-ntlmssp v0.1.0 h1:DjFo6YtWzNqNvQdrwEyr/e4nhU3vRiwenz5QX7sFz+A=
github.com/Azure/go-ntlmssp v0.1.0/go.mod h1:NYqdhxd/8aAct/s4qSYZEerdPuH1liG2/X9DiVTbhpk=
github.com/AzureAD/microsoft-authentication-extensions-for-go/cache v0.1.1 h1:WJTmL004Abzc5wDB5VtZG2PJk5ndYDgVacGqfirKxjM=
github.com/AzureAD/microsoft-authentication-extensions-for-go/cache v0.1.1/go.mod h1:tCcJZ0uHAmvjsVYzEFivsRTN00oz5BEsRgQHu5JZ9WE=
github.com/AzureAD/microsoft-authentication-library-for-go v1.6.0 h1:XRzhVemXdgvJqCH0sFfrBUTnUJSBrBf7++ypk+twtRs=
@@ -574,6 +594,8 @@ github.com/DATA-DOG/go-sqlmock v1.3.2 h1:2L2f5t3kKnCLxnClDD/PrDfExFFa1wjESgxHG/B
github.com/DATA-DOG/go-sqlmock v1.3.2/go.mod h1:f/Ixk793poVmq4qj/V1dPUg2JEAKC73Q5eFN3EC/SaM=
github.com/DataDog/datadog-go v3.2.0+incompatible/go.mod h1:LButxg5PwREeZtORoXG3tL4fMGNddJ+vMq1mwgfaqoQ=
github.com/DataDog/zstd v1.5.2/go.mod h1:g4AWEaM3yOg3HYfnJ3YIawPnVdXJh9QME85blwSAmyw=
github.com/DefangLabs/secret-detector v0.0.0-20250403165618-22662109213e h1:rd4bOvKmDIx0WeTv9Qz+hghsgyjikFiPrseXHlKepO0=
github.com/DefangLabs/secret-detector v0.0.0-20250403165618-22662109213e/go.mod h1:blbwPQh4DTlCZEfk1BLU4oMIhLda2U+A840Uag9DsZw=
github.com/Files-com/files-sdk-go/v3 v3.2.264 h1:lMHTplAYI9FtmCo/QOcpRxmPA5REVAct1r2riQmDQKw=
github.com/Files-com/files-sdk-go/v3 v3.2.264/go.mod h1:wGqkOzRu/ClJibvDgcfuJNAqI2nLhe8g91tPlDKRCdE=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.30.0 h1:sBEjpZlNHzK1voKq9695PJSX2o5NEXl7/OL3coiIY0c=
@@ -589,11 +611,20 @@ github.com/IBM/go-sdk-core/v5 v5.21.0/go.mod h1:Q3BYO6iDA2zweQPDGbNTtqft5tDcEpm6
github.com/Jille/raft-grpc-transport v1.6.1 h1:gN3sjapb+fVbiebS7AfQQgbV2ecTOI7ur7NPPC7Mhoc=
github.com/Jille/raft-grpc-transport v1.6.1/go.mod h1:HbOjEdu/yzCJ/mjTF6wEOJNbAUpHfU2UOA2hVD4CNFg=
github.com/JohnCGriffin/overflow v0.0.0-20211019200055-46fa312c352c/go.mod h1:X0CRv0ky0k6m906ixxpzmDRLvX58TFUKS2eePweuyxk=
github.com/MarvinJWendt/testza v0.1.0/go.mod h1:7AxNvlfeHP7Z/hDQ5JtE3OKYT3XFUeLCDE2DQninSqs=
github.com/MarvinJWendt/testza v0.2.1/go.mod h1:God7bhG8n6uQxwdScay+gjm9/LnO4D3kkcZX4hv9Rp8=
github.com/MarvinJWendt/testza v0.2.8/go.mod h1:nwIcjmr0Zz+Rcwfh3/4UhBp7ePKVhuBExvZqnKYWlII=
github.com/MarvinJWendt/testza v0.2.10/go.mod h1:pd+VWsoGUiFtq+hRKSU1Bktnn+DMCSrDrXDpX2bG66k=
github.com/MarvinJWendt/testza v0.2.12/go.mod h1:JOIegYyV7rX+7VZ9r77L/eH6CfJHHzXjB69adAhzZkI=
github.com/MarvinJWendt/testza v0.3.0/go.mod h1:eFcL4I0idjtIx8P9C6KkAuLgATNKpX4/2oUqKc6bF2c=
github.com/MarvinJWendt/testza v0.4.2/go.mod h1:mSdhXiKH8sg/gQehJ63bINcCKp7RtYewEjXsvsVUPbE=
github.com/MarvinJWendt/testza v0.5.2 h1:53KDo64C1z/h/d/stCYCPY69bt/OSwjq5KpFNwi+zB4=
github.com/MarvinJWendt/testza v0.5.2/go.mod h1:xu53QFE5sCdjtMCKk8YMQ2MnymimEctc4n3EjyIYvEY=
github.com/Masterminds/goutils v1.1.0/go.mod h1:8cTjp+g8YejhMuvIA5y2vz3BpJxksy863GQaJW2MFNU=
github.com/Masterminds/semver v1.5.0 h1:H65muMkzWKEuNDnfl9d70GUjFniHKHRbFPGBuZ3QEww=
github.com/Masterminds/semver v1.5.0/go.mod h1:MB6lktGJrhw8PrUyiEoblNEGEQ+RzHPF078ddwwvV3Y=
github.com/Masterminds/semver/v3 v3.2.0 h1:3MEsd0SM6jqZojhjLWWeBY+Kcjy9i6MQAeY7YgDP83g=
github.com/Masterminds/semver/v3 v3.2.0/go.mod h1:qvl/7zhW3nngYb5+80sSMF+FG2BjYrf8m9wsX0PNOMQ=
github.com/Masterminds/semver/v3 v3.2.1 h1:RN9w6+7QoMeJVGyfmbcgs28Br8cvmnucEXnY0rYXWg0=
github.com/Masterminds/semver/v3 v3.2.1/go.mod h1:qvl/7zhW3nngYb5+80sSMF+FG2BjYrf8m9wsX0PNOMQ=
github.com/Masterminds/sprig v2.22.0+incompatible/go.mod h1:y6hNFY5UBTIWBxnzTeuNhlNS5hqE0NB0E6fgfo2Br3o=
github.com/Max-Sum/base32768 v0.0.0-20230304063302-18e6ce5945fd h1:nzE1YQBdx1bq9IlZinHa+HVffy+NmVRoKr+wHN8fpLE=
github.com/Max-Sum/base32768 v0.0.0-20230304063302-18e6ce5945fd/go.mod h1:C8yoIfvESpM3GD07OCHU7fqI7lhwyZ2Td1rbNbTAhnc=
@@ -628,14 +659,16 @@ github.com/ThreeDotsLabs/watermill v1.5.1 h1:t5xMivyf9tpmU3iozPqyrCZXHvoV1XQDfih
github.com/ThreeDotsLabs/watermill v1.5.1/go.mod h1:Uop10dA3VeJWsSvis9qO3vbVY892LARrKAdki6WtXS4=
github.com/TomiHiltunen/geohash-golang v0.0.0-20150112065804-b3e4e625abfb h1:wumPkzt4zaxO4rHPBrjDK8iZMR41C1qs7njNqlacwQg=
github.com/TomiHiltunen/geohash-golang v0.0.0-20150112065804-b3e4e625abfb/go.mod h1:QiYsIBRQEO+Z4Rz7GoI+dsHVneZNONvhczuA+llOZNM=
github.com/a-h/templ v0.3.943 h1:o+mT/4yqhZ33F3ootBiHwaY4HM5EVaOJfIshvd5UNTY=
github.com/a-h/templ v0.3.943/go.mod h1:oCZcnKRf5jjsGpf2yELzQfodLphd2mwecwG4Crk5HBo=
github.com/a-h/templ v0.3.977 h1:kiKAPXTZE2Iaf8JbtM21r54A8bCNsncrfnokZZSrSDg=
github.com/a-h/templ v0.3.977/go.mod h1:oCZcnKRf5jjsGpf2yELzQfodLphd2mwecwG4Crk5HBo=
github.com/a1ex3/zstd-seekable-format-go/pkg v0.10.0 h1:iLDOF0rdGTrol/q8OfPIIs5kLD8XvA2q75o6Uq/tgak=
github.com/a1ex3/zstd-seekable-format-go/pkg v0.10.0/go.mod h1:DrEWcQJjz7t5iF2duaiyhg4jyoF0kxOD6LtECNGkZ/Q=
github.com/aalpar/deheap v0.0.0-20210914013432-0cc84d79dec3 h1:hhdWprfSpFbN7lz3W1gM40vOgvSh1WCSMxYD6gGB4Hs=
github.com/aalpar/deheap v0.0.0-20210914013432-0cc84d79dec3/go.mod h1:XaUnRxSCYgL3kkgX0QHIV0D+znljPIDImxlv2kbGv0Y=
github.com/abbot/go-http-auth v0.4.0 h1:QjmvZ5gSC7jm3Zg54DqWE/T5m1t2AfDu6QlXJT0EVT0=
github.com/abbot/go-http-auth v0.4.0/go.mod h1:Cz6ARTIzApMJDzh5bRMSUou6UMSp0IEXg9km/ci7TJM=
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d h1:licZJFw2RwpHMqeKTCYkitsPqHNxTmd4SNR5r94FGM8=
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d/go.mod h1:asat636LX7Bqt5lYEZ27JNDcqxfjdBQuJ/MM4CN/Lzo=
github.com/ajstarks/deck v0.0.0-20200831202436-30c9fc6549a9/go.mod h1:JynElWSGnm/4RlzPXRlREEwqTHAN3T56Bv2ITsFT3gY=
github.com/ajstarks/deck/generate v0.0.0-20210309230005-c3f852c02e19/go.mod h1:T13YZdzov6OU0A1+RfKZiZN9ca6VeKdBdyDV+BY97Tk=
github.com/ajstarks/svgo v0.0.0-20180226025133-644b8db467af/go.mod h1:K08gAheRH3/J6wwsYMMT4xOr94bZjxIelGM0+d/wbFw=
@@ -655,10 +688,20 @@ github.com/andybalholm/brotli v1.2.0/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUS
github.com/andybalholm/cascadia v1.3.3 h1:AG2YHrzJIm4BZ19iwJ/DAua6Btl3IwJX+VI4kktS1LM=
github.com/andybalholm/cascadia v1.3.3/go.mod h1:xNd9bqTn98Ln4DwST8/nG+H0yuB8Hmgu1YHNnWw0GeA=
github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=
github.com/antlr4-go/antlr/v4 v4.13.1 h1:SqQKkuVZ+zWkMMNkjy5FZe5mr5WURWnlpmOuzYWrPrQ=
github.com/antlr4-go/antlr/v4 v4.13.1/go.mod h1:GKmUxMtwp6ZgGwZSva4eWPC5mS6vUAmOABFgjdkM7Nw=
github.com/apache/arrow-go/v18 v18.4.1 h1:q/jVkBWCJOB9reDgaIZIdruLQUb1kbkvOnOFezVH1C4=
github.com/apache/arrow-go/v18 v18.4.1/go.mod h1:tLyFubsAl17bvFdUAy24bsSvA/6ww95Iqi67fTpGu3E=
github.com/apache/arrow/go/v10 v10.0.1/go.mod h1:YvhnlEePVnBS4+0z3fhPfUy7W1Ikj0Ih0vcRo/gZ1M0=
github.com/apache/cassandra-gocql-driver/v2 v2.0.0 h1:Omnzb1Z/P90Dr2TbVNu54ICQL7TKVIIsJO231w484HU=
github.com/apache/cassandra-gocql-driver/v2 v2.0.0/go.mod h1:QH/asJjB3mHvY6Dot6ZKMMpTcOrWJ8i9GhsvG1g0PK4=
github.com/apache/iceberg-go v0.4.0 h1:O3RwMyoc4gZpu9b1aBDJBiH14riga68BWypZVaSHZE0=
github.com/apache/iceberg-go v0.4.0/go.mod h1:TInpnxHnMdPrsmW/klToy9mM8gs6/6FsI+O2kwtlHMQ=
github.com/apache/thrift v0.16.0/go.mod h1:PHK3hniurgQaNMZYaCLEqXKsYK8upmhPbmdP2FXSqgU=
github.com/apache/thrift v0.22.0 h1:r7mTJdj51TMDe6RtcmNdQxgn9XcyfGDOzegMDRg47uc=
github.com/apache/thrift v0.22.0/go.mod h1:1e7J/O1Ae6ZQMTYdy9xa3w9k+XHWPfRvdPyJeynQ+/g=
github.com/apparentlymart/go-textseg/v15 v15.0.0 h1:uYvfpb3DyLSCGWnctWKGj857c6ew1u1fNQOlOtuGxQY=
github.com/apparentlymart/go-textseg/v15 v15.0.0/go.mod h1:K8XmNZdhEBkdlyDdvbmmsvpAG721bKi0joRfFdHIWJ4=
github.com/apple/foundationdb/bindings/go v0.0.0-20250911184653-27f7192f47c3 h1:WZaTKNHCfcw7fWSR6/RKnCldVzvYZC+Y20Su4lffEIg=
github.com/apple/foundationdb/bindings/go v0.0.0-20250911184653-27f7192f47c3/go.mod h1:OMVSB21p9+xQUIqlGizHPZfjK+SHws1ht+ZytVDoz9U=
github.com/appscode/go-querystring v0.0.0-20170504095604-0126cfb3f1dc h1:LoL75er+LKDHDUfU5tRvFwxH0LjPpZN8OoG8Ll+liGU=
@@ -669,6 +712,7 @@ github.com/arangodb/go-velocypack v0.0.0-20200318135517-5af53c29c67e h1:Xg+hGrY2
github.com/arangodb/go-velocypack v0.0.0-20200318135517-5af53c29c67e/go.mod h1:mq7Shfa/CaixoDxiyAAc5jZ6CVBAyPaNQCGS7mkj4Ho=
github.com/armon/go-metrics v0.4.1 h1:hR91U9KYmb6bLBYLQjyM+3j+rcd/UhE+G78SFnF8gJA=
github.com/armon/go-metrics v0.4.1/go.mod h1:E6amYzXo6aW1tqzoZGT755KkbgrJsSdpwZ+3JqfkOG4=
github.com/atomicgo/cursor v0.0.1/go.mod h1:cBON2QmmrysudxNBFthvMtN32r3jxVRIvzkUiF/RuIk=
github.com/aws/aws-sdk-go v1.55.8 h1:JRmEUbU52aJQZ2AjX4q4Wu7t4uZjOu71uyNmaWlUkJQ=
github.com/aws/aws-sdk-go v1.55.8/go.mod h1:ZkViS9AqA6otK+JBBNH2++sx1sgxrPKcSzPPvQkUtXk=
github.com/aws/aws-sdk-go-v2 v1.41.1 h1:ABlyEARCDLN034NhxlRUSZr4l71mh+T5KAeGh6cerhU=
@@ -749,6 +793,8 @@ github.com/buengese/sgzip v0.1.1 h1:ry+T8l1mlmiWEsDrH/YHZnCVWD2S3im1KLsyO+8ZmTU=
github.com/buengese/sgzip v0.1.1/go.mod h1:i5ZiXGF3fhV7gL1xaRRL1nDnmpNj0X061FQzOS8VMas=
github.com/bufbuild/protocompile v0.14.1 h1:iA73zAf/fyljNjQKwYzUHD6AD4R8KMasmwa/FBatYVw=
github.com/bufbuild/protocompile v0.14.1/go.mod h1:ppVdAIhbr2H8asPk6k4pY7t9zB1OU5DoEw9xY/FUi1c=
github.com/buger/goterm v1.0.4 h1:Z9YvGmOih81P0FbVtEYTFF6YsSgxSUKEhf/f9bTMXbY=
github.com/buger/goterm v1.0.4/go.mod h1:HiFWV3xnkolgrBV3mY8m0X0Pumt4zg4QhbdOzQtB8tE=
github.com/buger/jsonparser v1.1.1 h1:2PnMjfWD7wBILjqQbt530v576A/cAbQvEW9gGIpYMUs=
github.com/buger/jsonparser v1.1.1/go.mod h1:6RYKKt7H4d4+iWqouImQ9R2FZql3VbhNgx27UK13J/0=
github.com/bwesterb/go-ristretto v1.2.0/go.mod h1:fUIoIZaG73pV5biE2Blr2xEzDoMj7NFEuV9ekS419A0=
@@ -813,8 +859,8 @@ github.com/cncf/xds/go v0.0.0-20230105202645-06c439db220b/go.mod h1:eXthEFrGJvWH
github.com/cncf/xds/go v0.0.0-20230310173818-32f1caf87195/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
github.com/cncf/xds/go v0.0.0-20251022180443-0feb69152e9f h1:Y8xYupdHxryycyPlc9Y+bSQAYZnetRJ70VMVKm5CKI0=
github.com/cncf/xds/go v0.0.0-20251022180443-0feb69152e9f/go.mod h1:HlzOvOjVBOfTGSRXRyY0OiCS/3J1akRGQQpRO/7zyF4=
github.com/cockroachdb/apd/v3 v3.1.0 h1:MK3Ow7LH0W8zkd5GMKA1PvS9qG3bWFI95WaVNfyZJ/w=
github.com/cockroachdb/apd/v3 v3.1.0/go.mod h1:6qgPBMXjATAdD/VefbRP9NoSLKjbB4LCoA7gN4LpHs4=
github.com/cockroachdb/apd/v3 v3.2.1 h1:U+8j7t0axsIgvQUqthuNm82HIrYXodOV2iWLWtEaIwg=
github.com/cockroachdb/apd/v3 v3.2.1/go.mod h1:klXJcjp+FffLTHlhIG69tezTDvdP065naDsHzKhYSqc=
github.com/cockroachdb/errors v1.11.3 h1:5bA+k2Y6r+oz/6Z/RFlNeVCesGARKuC6YymtcDrbC/I=
github.com/cockroachdb/errors v1.11.3/go.mod h1:m4UIW4CDjx+R5cybPsNrRbreomiFqt8o1h1wUVazSd8=
github.com/cockroachdb/logtags v0.0.0-20241215232642-bb51bb14a506 h1:ASDL+UJcILMqgNeV5jiqR4j+sTuvQNHdf2chuKj1M5k=
@@ -827,12 +873,39 @@ github.com/cognusion/imaging v1.0.2 h1:BQwBV8V8eF3+dwffp8Udl9xF1JKh5Z0z5JkJwAi98
github.com/cognusion/imaging v1.0.2/go.mod h1:mj7FvH7cT2dlFogQOSUQRtotBxJ4gFQ2ySMSmBm5dSk=
github.com/colinmarc/hdfs/v2 v2.4.0 h1:v6R8oBx/Wu9fHpdPoJJjpGSUxo8NhHIwrwsfhFvU9W0=
github.com/colinmarc/hdfs/v2 v2.4.0/go.mod h1:0NAO+/3knbMx6+5pCv+Hcbaz4xn/Zzbn9+WIib2rKVI=
github.com/compose-spec/compose-go/v2 v2.6.0 h1:/+oBD2ixSENOeN/TlJqWZmUak0xM8A7J08w/z661Wd4=
github.com/compose-spec/compose-go/v2 v2.6.0/go.mod h1:vPlkN0i+0LjLf9rv52lodNMUTJF5YHVfHVGLLIP67NA=
github.com/containerd/console v1.0.3/go.mod h1:7LqA/THxQ86k76b8c/EMSiaJ3h1eZkMkXar0TQ1gf3U=
github.com/containerd/console v1.0.5 h1:R0ymNeydRqH2DmakFNdmjR2k0t7UPuiOV/N/27/qqsc=
github.com/containerd/console v1.0.5/go.mod h1:YynlIjWYF8myEu6sdkwKIvGQq+cOckRm6So2avqoYAk=
github.com/containerd/containerd/api v1.8.0 h1:hVTNJKR8fMc/2Tiw60ZRijntNMd1U+JVMyTRdsD2bS0=
github.com/containerd/containerd/api v1.8.0/go.mod h1:dFv4lt6S20wTu/hMcP4350RL87qPWLVa/OHOwmmdnYc=
github.com/containerd/containerd/v2 v2.0.5 h1:2vg/TjUXnaohAxiHnthQg8K06L9I4gdYEMcOLiMc8BQ=
github.com/containerd/containerd/v2 v2.0.5/go.mod h1:Qqo0UN43i2fX1FLkrSTCg6zcHNfjN7gEnx3NPRZI+N0=
github.com/containerd/continuity v0.0.0-20190827140505-75bee3e2ccb6/go.mod h1:GL3xCUCBDV3CZiTSEKksMWbLE66hEyuu9qyDOOqM47Y=
github.com/containerd/continuity v0.4.5 h1:ZRoN1sXq9u7V6QoHMcVWGhOwDFqZ4B9i5H6un1Wh0x4=
github.com/containerd/continuity v0.4.5/go.mod h1:/lNJvtJKUQStBzpVQ1+rasXO1LAWtUQssk28EZvJ3nE=
github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI=
github.com/containerd/errdefs v1.0.0/go.mod h1:+YBYIdtsnF4Iw6nWZhJcqGSg/dwvV7tyJ/kCkyJ2k+M=
github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151Xdx3ZPPE=
github.com/containerd/errdefs/pkg v0.3.0/go.mod h1:NJw6s9HwNuRhnjJhM7pylWwMyAkmCQvQ4GpJHEqRLVk=
github.com/containerd/log v0.1.0 h1:TCJt7ioM2cr/tfR8GPbGf9/VRAX8D2B4PjzCpfX540I=
github.com/containerd/log v0.1.0/go.mod h1:VRRf09a7mHDIRezVKTRCrOq78v577GXq3bSa3EhrzVo=
github.com/containerd/platforms v1.0.0-rc.1 h1:83KIq4yy1erSRgOVHNk1HYdPvzdJ5CnsWaRoJX4C41E=
github.com/containerd/platforms v1.0.0-rc.1/go.mod h1:J71L7B+aiM5SdIEqmd9wp6THLVRzJGXfNuWCZCllLA4=
github.com/containerd/ttrpc v1.2.7 h1:qIrroQvuOL9HQ1X6KHe2ohc7p+HP/0VE6XPU7elJRqQ=
github.com/containerd/ttrpc v1.2.7/go.mod h1:YCXHsb32f+Sq5/72xHubdiJRQY9inL4a4ZQrAbN1q9o=
github.com/containerd/typeurl/v2 v2.2.3 h1:yNA/94zxWdvYACdYO8zofhrTVuQY73fFU1y++dYSw40=
github.com/containerd/typeurl/v2 v2.2.3/go.mod h1:95ljDnPfD3bAbDJRugOiShd/DlAAsxGtUBhJxIn7SCk=
github.com/coreos/go-semver v0.3.1 h1:yi21YpKnrx1gt5R+la8n5WgS0kCrsPp33dmEyHReZr4=
github.com/coreos/go-semver v0.3.1/go.mod h1:irMmmIw/7yzSRPWryHsK7EYSg09caPQL03VsM8rvUec=
github.com/coreos/go-systemd/v22 v22.6.0 h1:aGVa/v8B7hpb0TKl0MWoAavPDmHvobFe5R5zn0bCJWo=
github.com/coreos/go-systemd/v22 v22.6.0/go.mod h1:iG+pp635Fo7ZmV/j14KUcmEyWF+0X7Lua8rrTWzYgWU=
github.com/cpuguy83/dockercfg v0.3.2 h1:DlJTyZGBDlXqUZ2Dk2Q3xHs/FtnooJJVaad2S9GKorA=
github.com/cpuguy83/dockercfg v0.3.2/go.mod h1:sugsbF4//dDlL/i+S+rtpIWp+5h0BHJHfjj5/jFyUJc=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/creack/pty v1.1.18 h1:n56/Zwd5o6whRC5PMGretI4IdRLlmBXYNjScPaBgsbY=
github.com/creack/pty v1.1.18/go.mod h1:MOBLtS5ELjhRRrroQr9kyvTxUAFNvYEK993ew/Vr4O4=
github.com/creasty/defaults v1.8.0 h1:z27FJxCAa0JKt3utc0sCImAEb+spPucmKoOdLHvHYKk=
github.com/creasty/defaults v1.8.0/go.mod h1:iGzKe6pbEHnpMPtfDXZEr0NVxWnPTjb1bbDy08fPzYM=
github.com/cronokirby/saferith v0.33.0 h1:TgoQlfsD4LIwx71+ChfRcIpjkw+RPOapDEVxa+LhwLo=
@@ -858,12 +931,36 @@ github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f h1:lO4WD4F/r
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f/go.mod h1:cuUVRXasLTGF7a8hSLbxyZXjz+1KgoB3wDUb6vlszIc=
github.com/diskfs/go-diskfs v1.7.0 h1:vonWmt5CMowXwUc79jWyGrf2DIMeoOjkLlMnQYGVOs8=
github.com/diskfs/go-diskfs v1.7.0/go.mod h1:LhQyXqOugWFRahYUSw47NyZJPezFzB9UELwhpszLP/k=
github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
github.com/djherbis/times v1.6.0 h1:w2ctJ92J8fBvWPxugmXIv7Nz7Q3iDMKNx9v5ocVH20c=
github.com/djherbis/times v1.6.0/go.mod h1:gOHeRAz2h+VJNZ5Gmc/o7iD9k4wW7NMVqieYCY99oc0=
github.com/dnaeon/go-vcr v1.2.0 h1:zHCHvJYTMh1N7xnV7zf1m1GPBF9Ad0Jk/whtQ1663qI=
github.com/dnaeon/go-vcr v1.2.0/go.mod h1:R4UdLID7HZT3taECzJs4YgbbH6PIGXB6W/sc5OLb6RQ=
github.com/docker/buildx v0.22.0 h1:pGTcGZa+kxpYUlM/6ACsp1hXhkEDulz++RNXPdE8Afk=
github.com/docker/buildx v0.22.0/go.mod h1:ThbnUe4kNiStlq6cLXruElyEdSTdPL3k/QerNUmPvHE=
github.com/docker/cli v28.0.4+incompatible h1:pBJSJeNd9QeIWPjRcV91RVJihd/TXB77q1ef64XEu4A=
github.com/docker/cli v28.0.4+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/cli-docs-tool v0.9.0 h1:CVwQbE+ZziwlPqrJ7LRyUF6GvCA+6gj7MTCsayaK9t0=
github.com/docker/cli-docs-tool v0.9.0/go.mod h1:ClrwlNW+UioiRyH9GiAOe1o3J/TsY3Tr1ipoypjAUtc=
github.com/docker/compose/v2 v2.35.0 h1:bU23OeFrbGyHYrKijMSEwkOeDg2TLhAGntU2F3hwX1o=
github.com/docker/compose/v2 v2.35.0/go.mod h1:S5ejUILn9KTYC6noX3IxznWu3/sb3FxdZqIYbq4seAk=
github.com/docker/distribution v2.8.3+incompatible h1:AtKxIZ36LoNK51+Z6RpzLpddBirtxJnzDrHLEKxTAYk=
github.com/docker/distribution v2.8.3+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker v28.5.0+incompatible h1:ZdSQoRUE9XxhFI/B8YLvhnEFMmYN9Pp8Egd2qcaFk1E=
github.com/docker/docker v28.5.0+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker-credential-helpers v0.8.2 h1:bX3YxiGzFP5sOXWc3bTPEXdEaZSeVMrFgOr3T+zrFAo=
github.com/docker/docker-credential-helpers v0.8.2/go.mod h1:P3ci7E3lwkZg6XiHdRKft1KckHiO9a2rNtyFbZ/ry9M=
github.com/docker/go v1.5.1-1.0.20160303222718-d30aec9fd63c h1:lzqkGL9b3znc+ZUgi7FlLnqjQhcXxkNM/quxIjBVMD0=
github.com/docker/go v1.5.1-1.0.20160303222718-d30aec9fd63c/go.mod h1:CADgU4DSXK5QUlFslkQu2yW2TKzFZcXq/leZfM0UH5Q=
github.com/docker/go-connections v0.4.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5XhDvyHbTtUxmeec=
github.com/docker/go-connections v0.6.0 h1:LlMG9azAe1TqfR7sO+NJttz1gy6KO7VJBh+pMmjSD94=
github.com/docker/go-connections v0.6.0/go.mod h1:AahvXYshr6JgfUJGdDCs2b5EZG/vmaMAntpSFH5BFKE=
github.com/docker/go-metrics v0.0.1 h1:AgB/0SvBxihN0X8OR4SjsblXkbMvalQ8cjmtKQ2rQV8=
github.com/docker/go-metrics v0.0.1/go.mod h1:cG1hvH2utMXtqgqqYE9plW6lDxS3/5ayHzueweSI3Vw=
github.com/docker/go-units v0.4.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE=
github.com/dropbox/dropbox-sdk-go-unofficial/v6 v6.0.5 h1:FT+t0UEDykcor4y3dMVKXIiWJETBpRgERYTGlmMd7HU=
github.com/dropbox/dropbox-sdk-go-unofficial/v6 v6.0.5/go.mod h1:rSS3kM9XMzSQ6pw91Qgd6yB5jdt70N4OdtrAf74As5M=
@@ -880,6 +977,8 @@ github.com/eapache/queue v1.1.0 h1:YOEu7KNc61ntiQlcEeUIoDTJ2o8mQznoNvUhiigpIqc=
github.com/eapache/queue v1.1.0/go.mod h1:6eCeP0CKFpHLu8blIFXhExK/dRa7WDZfr6jVFPTqq+I=
github.com/ebitengine/purego v0.9.1 h1:a/k2f2HQU3Pi399RPW1MOaZyhKJL9w/xFpKAg4q1s0A=
github.com/ebitengine/purego v0.9.1/go.mod h1:iIjxzd6CiRiOG0UyXP+V1+jWqUXVjPKLAI0mRfJZTmQ=
github.com/eiannone/keyboard v0.0.0-20220611211555-0d226195f203 h1:XBBHcIb256gUJtLmY22n99HaZTz+r2Z51xUPi01m3wg=
github.com/eiannone/keyboard v0.0.0-20220611211555-0d226195f203/go.mod h1:E1jcSv8FaEny+OP/5k9UxZVw9YFWGj7eI4KR/iOBqCg=
github.com/elastic/gosigar v0.14.3 h1:xwkKwPia+hSfg9GqrCUKYdId102m9qTJIIr7egmK/uo=
github.com/elastic/gosigar v0.14.3/go.mod h1:iXRIGg2tLnu7LBdpqzyQfGDEidKCfWcCMS0WKyPWoMs=
github.com/elliotwutingfeng/asciiset v0.0.0-20230602022725-51bbb787efab h1:h1UgjJdAAhj+uPL68n7XASS6bU+07ZX1WJvVS2eyoeY=
@@ -888,6 +987,8 @@ github.com/emersion/go-message v0.18.2 h1:rl55SQdjd9oJcIoQNhubD2Acs1E6IzlZISRTK7
github.com/emersion/go-message v0.18.2/go.mod h1:XpJyL70LwRvq2a8rVbHXikPgKj8+aI0kGdHlg16ibYA=
github.com/emersion/go-vcard v0.0.0-20241024213814-c9703dde27ff h1:4N8wnS3f1hNHSmFD5zgFkWCyA4L1kCDkImPAtK7D6tg=
github.com/emersion/go-vcard v0.0.0-20241024213814-c9703dde27ff/go.mod h1:HMJKR5wlh/ziNp+sHEDV2ltblO4JD2+IdDOWtGcQBTM=
github.com/emicklei/go-restful/v3 v3.11.0 h1:rAQeMHw1c7zTmncogyy8VvRZwtkmkZ4FxERmMY4rD+g=
github.com/emicklei/go-restful/v3 v3.11.0/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
@@ -938,16 +1039,22 @@ github.com/fortytw2/leaktest v1.3.0 h1:u8491cBMTQ8ft8aeV+adlcytMZylmA5nnwwkRZjI8
github.com/fortytw2/leaktest v1.3.0/go.mod h1:jDsjWgpAGjm2CA7WthBh/CdZYEPF31XHquHwclZch5g=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/fsnotify/fsevents v0.2.0 h1:BRlvlqjvNTfogHfeBOFvSC9N0Ddy+wzQCQukyoD7o/c=
github.com/fsnotify/fsevents v0.2.0/go.mod h1:B3eEk39i4hz8y1zaWS/wPrAP4O6wkIl7HQwKBr1qH/w=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/fsnotify/fsnotify v1.6.0/go.mod h1:sl3t1tCWJFWoRz9R8WJCbQihKKwmorjAbSClcnxKAGw=
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
github.com/fvbommel/sortorder v1.1.0 h1:fUmoe+HLsBTctBDoaBwpQo5N+nrCp8g/BjKb/6ZQmYw=
github.com/fvbommel/sortorder v1.1.0/go.mod h1:uk88iVf1ovNn1iLfgUVU2F9o5eO30ui720w+kxuqRs0=
github.com/fxamacker/cbor/v2 v2.7.0 h1:iM5WgngdRBanHcxugY4JySA0nk1wZorNOpTgCMedv5E=
github.com/fxamacker/cbor/v2 v2.7.0/go.mod h1:pxXPTn3joSm21Gbwsv0w9OSA2y1HFR9qXEeXQVeNoDQ=
github.com/gabriel-vasile/mimetype v1.4.11 h1:AQvxbp830wPhHTqc1u7nzoLT+ZFxGY7emj5DR5DYFik=
github.com/gabriel-vasile/mimetype v1.4.11/go.mod h1:d+9Oxyo1wTzWdyVUPMmXFvp4F9tea18J8ufA774AB3s=
github.com/geoffgarside/ber v1.2.0 h1:/loowoRcs/MWLYmGX9QtIAbA+V/FrnVLsMMPhwiRm64=
github.com/geoffgarside/ber v1.2.0/go.mod h1:jVPKeCbj6MvQZhwLYsGwaGI52oUorHoHKNecGT85ZCc=
github.com/getsentry/sentry-go v0.40.0 h1:VTJMN9zbTvqDqPwheRVLcp0qcUcM+8eFivvGocAaSbo=
github.com/getsentry/sentry-go v0.40.0/go.mod h1:eRXCoh3uvmjQLY6qu63BjUZnaBu5L5WhMV1RwYO8W5s=
github.com/getsentry/sentry-go v0.42.0 h1:eeFMACuZTbUQf90RE8dE4tXeSe4CZyfvR1MBL7RLEt8=
github.com/getsentry/sentry-go v0.42.0/go.mod h1:eRXCoh3uvmjQLY6qu63BjUZnaBu5L5WhMV1RwYO8W5s=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/gin-contrib/sessions v1.0.4 h1:ha6CNdpYiTOK/hTp05miJLbpTSNfOnFg5Jm2kbcqy8U=
github.com/gin-contrib/sessions v1.0.4/go.mod h1:ccmkrb2z6iU2osiAHZG3x3J4suJK+OU27oqzlWOqQgs=
@@ -995,8 +1102,14 @@ github.com/go-ole/go-ole v1.3.0 h1:Dt6ye7+vXGIKZ7Xtk4s6/xVdGDQynvom7xCFEdWr6uE=
github.com/go-ole/go-ole v1.3.0/go.mod h1:5LS6F96DhAwUc7C+1HLexzMXY1xGRSryjyPPKW6zv78=
github.com/go-openapi/errors v0.22.4 h1:oi2K9mHTOb5DPW2Zjdzs/NIvwi2N3fARKaTJLdNabaM=
github.com/go-openapi/errors v0.22.4/go.mod h1:z9S8ASTUqx7+CP1Q8dD8ewGH/1JWFFLX/2PmAYNQLgk=
github.com/go-openapi/jsonpointer v0.19.6 h1:eCs3fxoIi3Wh6vtgmLTOjdhSpiqphQ+DaPn38N2ZdrE=
github.com/go-openapi/jsonpointer v0.19.6/go.mod h1:osyAmYz/mB/C3I+WsTTSgw1ONzaLJoLCyoi6/zppojs=
github.com/go-openapi/jsonreference v0.20.2 h1:3sVjiK66+uXK/6oQ8xgcRKcFgQ5KXa2KvnJRumpMGbE=
github.com/go-openapi/jsonreference v0.20.2/go.mod h1:Bl1zwGIM8/wsvqjsOQLJ/SH+En5Ap4rVB5KVcIDZG2k=
github.com/go-openapi/strfmt v0.25.0 h1:7R0RX7mbKLa9EYCTHRcCuIPcaqlyQiWNPTXwClK0saQ=
github.com/go-openapi/strfmt v0.25.0/go.mod h1:nNXct7OzbwrMY9+5tLX4I21pzcmE6ccMGXl3jFdPfn8=
github.com/go-openapi/swag v0.22.4 h1:QLMzNJnMGPRNDCbySlcj1x01tzU8/9LTTL9hZZZogBU=
github.com/go-openapi/swag v0.22.4/go.mod h1:UzaqsxGiab7freDnrUUra0MwWfN/q7tE4j+VcZ0yl14=
github.com/go-openapi/testify/v2 v2.0.2 h1:X999g3jeLcoY8qctY/c/Z8iBHTbwLz7R2WXd6Ub6wls=
github.com/go-openapi/testify/v2 v2.0.2/go.mod h1:HCPmvFFnheKK2BuwSA0TbbdxJ3I16pjwMkYkP4Ywn54=
github.com/go-pdf/fpdf v0.5.0/go.mod h1:HzcnA+A23uwogo0tp9yU+l3V+KXhiESpt1PMayhOh5M=
@@ -1046,8 +1159,8 @@ github.com/golang-jwt/jwt/v4 v4.4.1/go.mod h1:m21LjoU+eqJr34lmDMbreY2eSTRJ1cv77w
github.com/golang-jwt/jwt/v4 v4.4.3/go.mod h1:m21LjoU+eqJr34lmDMbreY2eSTRJ1cv77w39/MY0Ch0=
github.com/golang-jwt/jwt/v4 v4.5.2 h1:YtQM7lnr8iZ+j5q71MGKkNw9Mn7AjHM68uc9g5fXeUI=
github.com/golang-jwt/jwt/v4 v4.5.2/go.mod h1:m21LjoU+eqJr34lmDMbreY2eSTRJ1cv77w39/MY0Ch0=
github.com/golang-jwt/jwt/v5 v5.3.0 h1:pv4AsKCKKZuqlgs5sUmn4x8UlGa0kEVt/puTpKx9vvo=
github.com/golang-jwt/jwt/v5 v5.3.0/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0/go.mod h1:E/TSTwGwJL78qG/PmXZO1EjYhfJinVAhrmmHX6Z8B9k=
github.com/golang/geo v0.0.0-20210211234256-740aa86cb551 h1:gtexQ/VGyN+VVFRXSFiguSNcXmS6rkKT+X7FdIrTtfo=
github.com/golang/geo v0.0.0-20210211234256-740aa86cb551/go.mod h1:QZ0nwyI2jOfgRAoBvP+ab5aRr7c9x7lhGEJrKvBwjWI=
@@ -1057,8 +1170,8 @@ github.com/golang/glog v1.1.0/go.mod h1:pfYeQZ3JWZoXTV5sFc986z3HTpwQs9At6P4ImfuP
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 h1:f+oWsMOmNPc8JmEHVZIycC7hBoQxHH9pNKQORJNozsQ=
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8/go.mod h1:wcDNUvekVysuuOpQKo3191zZyTpiI6se1N1ULghS0sw=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y=
@@ -1104,6 +1217,8 @@ github.com/google/btree v1.1.3/go.mod h1:qOPhT0dTNdNzV6Z/lhRX0YXUafgPLFUh+gZMl76
github.com/google/flatbuffers v2.0.8+incompatible/go.mod h1:1AeVuKshWv4vARoZatz6mlQ0JxURH0Kv5+zNeJKJCa8=
github.com/google/flatbuffers/go v0.0.0-20230108230133-3b8644d32c50 h1:T0YCYlZLzmdsd0bsozI4ecxk03KYOiszof14y7ekQFw=
github.com/google/flatbuffers/go v0.0.0-20230108230133-3b8644d32c50/go.mod h1:qmRCJW6OqZkfBt584Cmq1im0f4367CLrdABrq5lMOWo=
github.com/google/gnostic-models v0.6.8 h1:yo/ABAfM5IMRsS1VnXjTBvUb61tFIHozhlYvRgGre9I=
github.com/google/gnostic-models v0.6.8/go.mod h1:5n7qKqH0f5wFt+aWF8CW6pZLLNOfYuF5OpfBSENuI8U=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
@@ -1157,6 +1272,8 @@ github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/s2a-go v0.1.9 h1:LGD7gtMgezd8a/Xak7mEWL0PjoTQFvpRudN895yqKW0=
github.com/google/s2a-go v0.1.9/go.mod h1:YA0Ei2ZQL3acow2O62kdp9UlnvMmU7kA6Eutn0dXayM=
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 h1:El6M4kTTCOh6aBiKaUGG7oYTSPP8MxqL4YI3kZKwcP4=
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510/go.mod h1:pupxD2MaaD3pAXIBCelhxNneeOaAeabZDe5s4K6zSpQ=
github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.2.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
@@ -1186,6 +1303,10 @@ github.com/googleapis/gax-go/v2 v2.15.0 h1:SyjDc1mGgZU5LncH8gimWo9lW1DtIfPibOG81
github.com/googleapis/gax-go/v2 v2.15.0/go.mod h1:zVVkkxAQHa1RQpg9z2AUCMnKhi0Qld9rcmyfL1OZhoc=
github.com/googleapis/go-type-adapters v1.0.0/go.mod h1:zHW75FOG2aur7gAO2B+MLby+cLsWGBF62rFAi7WjWO4=
github.com/googleapis/google-cloud-go-testing v0.0.0-20200911160855-bcd43fbb19e8/go.mod h1:dvDLG8qkwmyD9a/MJJN3XJcT3xFxOKAvTZGvuZmac9g=
github.com/gookit/color v1.4.2/go.mod h1:fqRyamkC1W8uxl+lxCQxOT09l/vYfZ+QeiX3rKQHCoQ=
github.com/gookit/color v1.5.0/go.mod h1:43aQb+Zerm/BWh2GnrgOQm7ffz7tvQXEKV6BFMl7wAo=
github.com/gookit/color v1.5.4 h1:FZmqs7XOyGgCAxmWyPslpiok1k05wmY3SJTytgvYFs0=
github.com/gookit/color v1.5.4/go.mod h1:pZJOeOS8DM43rXbp4AZo1n9zCU2qjpcRko0b6/QJi9w=
github.com/gopherjs/gopherjs v1.17.2 h1:fQnZVsXk8uxXIStYb0N4bGk7jeyTalG/wsZjQ25dO0g=
github.com/gopherjs/gopherjs v1.17.2/go.mod h1:pRRIvn/QzFLrKfvEz3qUuEhtE/zLCWfreZ6J5gM2i+k=
github.com/gorilla/context v1.1.2 h1:WRkNAv2uoa03QNIc1A6u4O7DAGMUVoopZhkiXWA2V1o=
@@ -1200,14 +1321,18 @@ github.com/gorilla/securecookie v1.1.2/go.mod h1:NfCASbcHqRSY+3a8tlWJwsQap2VX5pw
github.com/gorilla/sessions v1.2.1/go.mod h1:dk2InVEVJ0sfLlnXv9EAgkf6ecYs/i80K/zI+bUmuGM=
github.com/gorilla/sessions v1.4.0 h1:kpIYOp/oi6MG/p5PgxApU8srsSw9tuFbt46Lt7auzqQ=
github.com/gorilla/sessions v1.4.0/go.mod h1:FLWm50oby91+hl7p/wRxDth9bWSuk0qVL2emc7lT5ik=
github.com/gorilla/websocket v1.5.0 h1:PPwGk2jz7EePpoHN/+ClbZu8SPxiqlu12wZP/3sWmnc=
github.com/gorilla/websocket v1.5.0/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/grpc-ecosystem/go-grpc-middleware v1.4.0 h1:UH//fgunKIs4JdUbpDl1VZCDaL56wXCB/5+wF6uHfaI=
github.com/grpc-ecosystem/go-grpc-middleware v1.4.0/go.mod h1:g5qyo/la0ALbONm6Vbp88Yd8NsDy6rZz+RcrMPxvld8=
github.com/grpc-ecosystem/grpc-gateway v1.16.0 h1:gmcG1KaJ57LophUzW0Hy8NmPhnMZb4M0+kPpLofRdBo=
github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.7.0/go.mod h1:hgWBS7lorOAVIJEQMi4ZsPv9hVvWI6+ch50m39Pf2Ks=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.11.3/go.mod h1:o//XUCC/F+yRGJoPO/VU0GSB0f8Nhgmxx0VIRUvaC0w=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.1 h1:X5VWvz21y3gzm9Nw/kaUeku/1+uBhcekkmy4IkffJww=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.1/go.mod h1:Zanoh4+gvIgluNqcfMVTJueD4wSS5hT7zTt4Mrutd90=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.3 h1:NmZ1PKzSTQbuGHw9DGPFomqkkLWMC+vZCkfs+FHv1Vg=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.3/go.mod h1:zQrxl1YP88HQlA6i9c63DSVPFklWpGX4OWAc9bFuaH4=
github.com/hamba/avro/v2 v2.30.0 h1:OaIdh0+dZIJ331FO/+YYBwZZRdGVyyHuRSyHsjZLJoA=
github.com/hamba/avro/v2 v2.30.0/go.mod h1:X6gDhYv6DQVAT56VqOKuW+PLnQrEQqGB9l1nhlMdAdQ=
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I=
github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
@@ -1245,6 +1370,8 @@ github.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/b
github.com/hashicorp/go-uuid v1.0.2/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
github.com/hashicorp/go-uuid v1.0.3 h1:2gKiV6YVmrJ1i2CKKa9obLvRieoRGviZFL26PcT/Co8=
github.com/hashicorp/go-uuid v1.0.3/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
github.com/hashicorp/go-version v1.7.0 h1:5tqGy27NaOTB8yJKUZELlFAS/LTKJkrmONwQKeRZfjY=
github.com/hashicorp/go-version v1.7.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v0.5.4/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4=
@@ -1275,14 +1402,20 @@ github.com/iancoleman/strcase v0.2.0/go.mod h1:iwCmte+B7n89clKwxIoIXy/HfoL7AsD47
github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/imdario/mergo v0.3.9/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
github.com/imdario/mergo v0.3.16 h1:wwQJbIsHYGMUyLSPrEq1CT16AhnhNJQ51+4fdHUnCl4=
github.com/imdario/mergo v0.3.16/go.mod h1:WBLT9ZmE3lPoWsEzCh9LPo3TiwVN+ZKEjmz+hD27ysY=
github.com/in-toto/in-toto-golang v0.5.0 h1:hb8bgwr0M2hGdDsLjkJ3ZqJ8JFLL/tgYdAxF/XEFBbY=
github.com/in-toto/in-toto-golang v0.5.0/go.mod h1:/Rq0IZHLV7Ku5gielPT4wPHJfH1GdHMCq8+WPxw8/BE=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/inhies/go-bytesize v0.0.0-20220417184213-4913239db9cf h1:FtEj8sfIcaaBfAKrE1Cwb61YDtYq9JxChK1c7AKce7s=
github.com/inhies/go-bytesize v0.0.0-20220417184213-4913239db9cf/go.mod h1:yrqSXGoD/4EKfF26AOGzscPOgTTJcyAwM2rpixWT+t4=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.7.6 h1:rWQc5FwZSPX58r1OQmkuaNicxdmExaEz5A2DO2hUuTk=
github.com/jackc/pgx/v5 v5.7.6/go.mod h1:aruU7o91Tc2q2cFp5h4uP3f6ztExVpyVv88Xl/8Vl8M=
github.com/jackc/pgx/v5 v5.8.0 h1:TYPDoleBBme0xGSAX3/+NujXXtpZn9HBONkQC7IEZSo=
github.com/jackc/pgx/v5 v5.8.0/go.mod h1:QVeDInX2m9VyzvNeiCJVjCkNFqzsNb43204HshNSZKw=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/jaegertracing/jaeger v1.47.0 h1:XXxTMO+GxX930gxKWsg90rFr6RswkCRIW0AgWFnTYsg=
@@ -1299,10 +1432,14 @@ github.com/jcmturner/gokrb5/v8 v8.4.4 h1:x1Sv4HaTpepFkXbt2IkL29DXRf8sOfZXo8eRKh6
github.com/jcmturner/gokrb5/v8 v8.4.4/go.mod h1:1btQEpgT6k+unzCwX1KdWMEwPPkkgBtP+F6aCACiMrs=
github.com/jcmturner/rpc/v2 v2.0.3 h1:7FXXj8Ti1IaVFpSAziCZWNzbNuZmnvw/i6CqLNdWfZY=
github.com/jcmturner/rpc/v2 v2.0.3/go.mod h1:VUJYCIDm3PVOEHw8sgt091/20OJjskO/YJki3ELg/Hc=
github.com/jhump/protoreflect v1.17.0 h1:qOEr613fac2lOuTgWN4tPAtLL7fUSbuJL5X5XumQh94=
github.com/jhump/protoreflect v1.17.0/go.mod h1:h9+vUUL38jiBzck8ck+6G/aeMX8Z4QUY/NiJPwPNi+8=
github.com/jhump/protoreflect v1.18.0 h1:TOz0MSR/0JOZ5kECB/0ufGnC2jdsgZ123Rd/k4Z5/2w=
github.com/jhump/protoreflect v1.18.0/go.mod h1:ezWcltJIVF4zYdIFM+D/sHV4Oh5LNU08ORzCGfwvTz8=
github.com/jhump/protoreflect/v2 v2.0.0-beta.1 h1:Dw1rslK/VotaUGYsv53XVWITr+5RCPXfvvlGrM/+B6w=
github.com/jhump/protoreflect/v2 v2.0.0-beta.1/go.mod h1:D9LBEowZyv8/iSu97FU2zmXG3JxVTmNw21mu63niFzU=
github.com/jinzhu/copier v0.4.0 h1:w3ciUoD19shMCRargcpm0cm91ytaBhDvuRpz1ODO/U8=
github.com/jinzhu/copier v0.4.0/go.mod h1:DfbEm0FYsaqBcKcFuvmOZb218JkPGtvSHsKg8S8hyyg=
github.com/jinzhu/inflection v1.0.0 h1:K317FqzuhWc8YvSVlFMCCUb36O/S9MCKRDI7QkRKD/E=
github.com/jinzhu/inflection v1.0.0/go.mod h1:h+uFLlag+Qp1Va5pdKtLDYj+kHp5pxUVkryuEj+Srlc=
github.com/jlaffaye/ftp v0.2.1-0.20240918233326-1b970516f5d3 h1:ZxO6Qr2GOXPdcW80Mcn3nemvilMPvpWqxrNfK2ZnNNs=
github.com/jlaffaye/ftp v0.2.1-0.20240918233326-1b970516f5d3/go.mod h1:dvLUr/8Fs9a2OBrEnCC5duphbkz/k/mSy5OkXg3PAgI=
github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg=
@@ -1341,16 +1478,20 @@ github.com/karlseguin/ccache/v2 v2.0.8 h1:lT38cE//uyf6KcFok0rlgXtGFBWxkI6h/qg4tb
github.com/karlseguin/ccache/v2 v2.0.8/go.mod h1:2BDThcfQMf/c0jnZowt16eW405XIqZPavt+HoYEtcxQ=
github.com/karlseguin/expect v1.0.2-0.20190806010014-778a5f0c6003 h1:vJ0Snvo+SLMY72r5J4sEfkuE7AFbixEP2qRbEcum/wA=
github.com/karlseguin/expect v1.0.2-0.20190806010014-778a5f0c6003/go.mod h1:zNBxMY8P21owkeogJELCLeHIt+voOSduHYTFUbwRAV8=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8=
github.com/keybase/go-keychain v0.0.1 h1:way+bWYa6lDppZoZcgMbYsvC7GxljxrskdNInRtuthU=
github.com/keybase/go-keychain v0.0.1/go.mod h1:PdEILRW3i9D8JcdM+FmY6RwkHGnhHxXwkPPMeUgOK1k=
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/asmfmt v1.3.2 h1:4Ri7ox3EwapiOjCki+hw14RyKk201CN4rzyCJRFLpK4=
github.com/klauspost/asmfmt v1.3.2/go.mod h1:AG8TuvYojzulgDAMCnYn50l/5QV3Bs/tp6j0HLHbNSE=
github.com/klauspost/compress v1.15.9/go.mod h1:PhcZ0MbTNciWF3rruxRgKxI5NkcHHrHUDtV4Yw2GlzU=
github.com/klauspost/compress v1.18.2 h1:iiPHWW0YrcFgpBYhsA6D1+fqHssJscY/Tm/y2Uqnapk=
github.com/klauspost/compress v1.18.2/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4=
github.com/klauspost/compress v1.18.4 h1:RPhnKRAQ4Fh8zU2FY/6ZFDwTVTxgJ/EMydqSTzE9a2c=
github.com/klauspost/compress v1.18.4/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4=
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/klauspost/cpuid/v2 v2.0.10/go.mod h1:g2LTdtYhdyuGPqyWyv7qRAmj1WBqxuObKfj5c0PQa7c=
github.com/klauspost/cpuid/v2 v2.0.12/go.mod h1:g2LTdtYhdyuGPqyWyv7qRAmj1WBqxuObKfj5c0PQa7c=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/klauspost/reedsolomon v1.13.0 h1:E0Cmgf2kMuhZTj6eefnvpKC4/Q4jhCi9YIjcZjK4arc=
@@ -1385,12 +1526,14 @@ github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
github.com/lib/pq v0.0.0-20180327071824-d34b9ff171c2/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo=
github.com/lib/pq v1.8.0/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
github.com/lib/pq v1.11.1 h1:wuChtj2hfsGmmx3nf1m7xC2XpK6OtelS2shMY+bGMtI=
github.com/lib/pq v1.11.1/go.mod h1:/p+8NSbOcwzAEI7wiMXFlgydTwcgTr3OSKMsD2BitpA=
github.com/linkedin/goavro/v2 v2.14.1 h1:/8VjDpd38PRsy02JS0jflAu7JZPfJcGTwqWgMkFS2iI=
github.com/linkedin/goavro/v2 v2.14.1/go.mod h1:KXx+erlq+RPlGSPmLF7xGo6SAbh8sCQ53x064+ioxhk=
github.com/linxGnu/grocksdb v1.10.3 h1:0laII9AQ6kFxo5SjhdTfSh9EgF20piD6TMHK6YuDm+4=
github.com/linxGnu/grocksdb v1.10.3/go.mod h1:OLQKZwiKwaJiAVCsOzWKvwiLwfZ5Vz8Md5TYR7t7pM8=
github.com/linxGnu/grocksdb v1.10.7 h1:fCi4qvZWo04VgFwGWmO8HQJgUVounJBy+C2TMVPU/ho=
github.com/linxGnu/grocksdb v1.10.7/go.mod h1:OLQKZwiKwaJiAVCsOzWKvwiLwfZ5Vz8Md5TYR7t7pM8=
github.com/lithammer/fuzzysearch v1.1.8 h1:/HIuJnjHuXS8bKaiTMeeDlW2/AyIWk2brx1V8LFgLN4=
github.com/lithammer/fuzzysearch v1.1.8/go.mod h1:IdqeyBClc3FFqSzYq/MXESsS4S0FsZ5ajtkr5xPLts4=
github.com/lithammer/shortuuid/v3 v3.0.7 h1:trX0KTHy4Pbwo/6ia8fscyHoGA+mf1jWbPJVuvyJQQ8=
github.com/lithammer/shortuuid/v3 v3.0.7/go.mod h1:vMk8ke37EmiewwolSO1NLW8vP4ZaKlRuDIi8tWWmAts=
github.com/lpar/date v1.0.0 h1:bq/zVqFTUmsxvd/CylidY4Udqpr9BOFrParoP6p0x/I=
@@ -1400,6 +1543,8 @@ github.com/lufia/plan9stats v0.0.0-20251013123823-9fd1530e3ec3/go.mod h1:autxFIv
github.com/lyft/protoc-gen-star v0.6.0/go.mod h1:TGAoBVkt8w7MPG72TrKIu85MIdXwDuzJYeZuUPFPNwA=
github.com/lyft/protoc-gen-star v0.6.1/go.mod h1:TGAoBVkt8w7MPG72TrKIu85MIdXwDuzJYeZuUPFPNwA=
github.com/lyft/protoc-gen-star/v2 v2.0.1/go.mod h1:RcCdONR2ScXaYnQC5tUzxzlpA3WVYF7/opLeUgcQs/o=
github.com/magiconair/properties v1.8.10 h1:s31yESBquKXCV9a/ScB3ESkOjUYYv+X0rg8SYxI99mE=
github.com/magiconair/properties v1.8.10/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
github.com/mailru/easyjson v0.9.1 h1:LbtsOm5WAswyWbvTEOqhypdPeZzHavpZx96/n553mR8=
github.com/mailru/easyjson v0.9.1/go.mod h1:1+xMtQp2MRNVL/V1bOzuP3aP8VNwRW55fQUto+XFtTU=
@@ -1413,13 +1558,22 @@ github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-runewidth v0.0.3/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
github.com/mattn/go-runewidth v0.0.13/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/mattn/go-runewidth v0.0.19 h1:v++JhqYnZuu5jSKrk9RbgF5v4CGUjqRfBm05byFGLdw=
github.com/mattn/go-runewidth v0.0.19/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
github.com/mattn/go-shellwords v1.0.12 h1:M2zGm7EW6UQJvDeQxo4T51eKPurbeFbe8WtebGE2xrk=
github.com/mattn/go-shellwords v1.0.12/go.mod h1:EZzvwXDESEeg03EKmM+RmDnNOPKG4lLtQsUlTZDWQ8Y=
github.com/mattn/go-sqlite3 v1.14.14/go.mod h1:NyWgC/yNuGj7Q9rpYnZvas74GogHl5/Z4A/KQRfk6bU=
github.com/mattn/go-sqlite3 v1.14.33 h1:A5blZ5ulQo2AtayQ9/limgHEkFreKj1Dv226a1K73s0=
github.com/mattn/go-sqlite3 v1.14.33/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
github.com/mattn/go-sqlite3 v1.14.34 h1:3NtcvcUnFBPsuRcno8pUtupspG/GM+9nZ88zgJcp6Zk=
github.com/mattn/go-sqlite3 v1.14.34/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b h1:j7+1HpAFS1zy5+Q4qx1fWh90gTKwiN4QCGoY9TWyyO4=
github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE=
github.com/miekg/pkcs11 v1.1.1 h1:Ugu9pdy6vAYku5DEpVWVFPYnzV+bxB+iRdbuFSu7TvU=
github.com/miekg/pkcs11 v1.1.1/go.mod h1:XsNlhZGX73bx86s2hdc/FuaLm2CPZJemRLMA+WTFxgs=
github.com/minio/asm2plan9s v0.0.0-20200509001527-cdd76441f9d8 h1:AMFGa4R4MiIpspGNG7Z948v4n35fFGB3RR3G/ry4FWs=
github.com/minio/asm2plan9s v0.0.0-20200509001527-cdd76441f9d8/go.mod h1:mC1jAcsrzbxHt8iiaC+zU4b1ylILSosueou12R++wfY=
github.com/minio/c2goasm v0.0.0-20190812172519-36a3d3bbc4f3 h1:+n/aFZefKZp7spd8DFdX7uMikMLXX4oubIzJF4kv/wI=
github.com/minio/c2goasm v0.0.0-20190812172519-36a3d3bbc4f3/go.mod h1:RagcQ7I8IeTMnF8JTXieKnO4Z6JCsikNEzj0DwauVzE=
github.com/minio/crc64nvme v1.1.1 h1:8dwx/Pz49suywbO+auHCBpCtlW1OfpcLN7wYgVR6wAI=
github.com/minio/crc64nvme v1.1.1/go.mod h1:eVfm2fAzLlxMdUGc0EEBGSMmPwmXD5XiNRpnu9J3bvg=
@@ -1430,13 +1584,43 @@ github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db/go.mod h1:l0
github.com/mitchellh/copystructure v1.0.0/go.mod h1:SNtv71yrdKgLRyLFxmLdkAbkKEFWgYaq1OVrnRcwhnw=
github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=
github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
github.com/mitchellh/hashstructure/v2 v2.0.2 h1:vGKWl0YJqUNxE8d+h8f6NJLcCJrgbhC4NcD46KavDd4=
github.com/mitchellh/hashstructure/v2 v2.0.2/go.mod h1:MG3aRVU/N29oo/V/IhBX8GR/zz4kQkprJgF2EVszyDE=
github.com/mitchellh/mapstructure v1.5.1-0.20220423185008-bf980b35cac4 h1:BpfhmLKZf+SjVanKKhCgf3bg+511DmU9eDQTen7LLbY=
github.com/mitchellh/mapstructure v1.5.1-0.20220423185008-bf980b35cac4/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
github.com/mitchellh/reflectwalk v1.0.0/go.mod h1:mSTlrgnPZtwu0c4WaC2kGObEpuNDbx0jmZXqmk4esnw=
github.com/mmcloughlin/geohash v0.9.0 h1:FihR004p/aE1Sju6gcVq5OLDqGcMnpBY+8moBqIsVOs=
github.com/mmcloughlin/geohash v0.9.0/go.mod h1:oNZxQo5yWJh0eMQEP/8hwQuVx9Z9tjwFUqcTB1SmG0c=
github.com/moby/buildkit v0.20.1 h1:sT0ZXhhNo5rVbMcYfgttma3TdUHfO5JjFA0UAL8p9fY=
github.com/moby/buildkit v0.20.1/go.mod h1:Rq9nB/fJImdk6QeM0niKtOHJqwKeYMrK847hTTDVuA4=
github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0=
github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
github.com/moby/go-archive v0.1.0 h1:Kk/5rdW/g+H8NHdJW2gsXyZ7UnzvJNOy6VKJqueWdcQ=
github.com/moby/go-archive v0.1.0/go.mod h1:G9B+YoujNohJmrIYFBpSd54GTUB4lt9S+xVQvsJyFuo=
github.com/moby/locker v1.0.1 h1:fOXqR41zeveg4fFODix+1Ch4mj/gT0NE1XJbp/epuBg=
github.com/moby/locker v1.0.1/go.mod h1:S7SDdo5zpBK84bzzVlKr2V0hz+7x9hWbYC/kq7oQppc=
github.com/moby/patternmatcher v0.6.0 h1:GmP9lR19aU5GqSSFko+5pRqHi+Ohk1O69aFiKkVGiPk=
github.com/moby/patternmatcher v0.6.0/go.mod h1:hDPoyOpDY7OrrMDLaYoY3hf52gNCR/YOUYxkhApJIxc=
github.com/moby/spdystream v0.4.0 h1:Vy79D6mHeJJjiPdFEL2yku1kl0chZpJfZcPpb16BRl8=
github.com/moby/spdystream v0.4.0/go.mod h1:xBAYlnt/ay+11ShkdFKNAG7LsyK/tmNBVvVOwrfMgdI=
github.com/moby/sys/atomicwriter v0.1.0 h1:kw5D/EqkBwsBFi0ss9v1VG3wIkVhzGvLklJ+w3A14Sw=
github.com/moby/sys/atomicwriter v0.1.0/go.mod h1:Ul8oqv2ZMNHOceF643P6FKPXeCmYtlQMvpizfsSoaWs=
github.com/moby/sys/capability v0.4.0 h1:4D4mI6KlNtWMCM1Z/K0i7RV1FkX+DBDHKVJpCndZoHk=
github.com/moby/sys/capability v0.4.0/go.mod h1:4g9IK291rVkms3LKCDOoYlnV8xKwoDTpIrNEE35Wq0I=
github.com/moby/sys/mountinfo v0.7.2 h1:1shs6aH5s4o5H2zQLn796ADW1wMrIwHsyJ2v9KouLrg=
github.com/moby/sys/mountinfo v0.7.2/go.mod h1:1YOa8w8Ih7uW0wALDUgT1dTTSBrZ+HiBLGws92L2RU4=
github.com/moby/sys/sequential v0.6.0 h1:qrx7XFUd/5DxtqcoH1h438hF5TmOvzC/lspjy7zgvCU=
github.com/moby/sys/sequential v0.6.0/go.mod h1:uyv8EUTrca5PnDsdMGXhZe6CCe8U/UiTWd+lL+7b/Ko=
github.com/moby/sys/signal v0.7.1 h1:PrQxdvxcGijdo6UXXo/lU/TvHUWyPhj7UOpSo8tuvk0=
github.com/moby/sys/signal v0.7.1/go.mod h1:Se1VGehYokAkrSQwL4tDzHvETwUZlnY7S5XtQ50mQp8=
github.com/moby/sys/symlink v0.3.0 h1:GZX89mEZ9u53f97npBy4Rc3vJKj7JBDj/PN2I22GrNU=
github.com/moby/sys/symlink v0.3.0/go.mod h1:3eNdhduHmYPcgsJtZXW1W4XUJdZGBIkttZ8xKqPUJq0=
github.com/moby/sys/user v0.4.0 h1:jhcMKit7SA80hivmFJcbB1vqmw//wU61Zdui2eQXuMs=
github.com/moby/sys/user v0.4.0/go.mod h1:bG+tYYYJgaMtRKgEmuueC0hJEAZWwtIbZTB+85uoHjs=
github.com/moby/sys/userns v0.1.0 h1:tVLXkFOxVu9A64/yh59slHVv9ahO9UIev4JZusOLG/g=
github.com/moby/sys/userns v0.1.0/go.mod h1:IHUYgu/kao6N8YZlp9Cf444ySSvCmDlmzUcYfDHOl28=
github.com/moby/term v0.5.2 h1:6qk3FJAFDs6i/q3W/pQ97SX192qKfZgGjCQqfCJkgzQ=
github.com/moby/term v0.5.2/go.mod h1:d3djjFCrjnB+fl8NJux+EJzu0msscUP+f8it8hPkFLc=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
@@ -1446,10 +1630,14 @@ github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9G
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/montanaflynn/stats v0.7.1 h1:etflOAAHORrCC44V+aR6Ftzort912ZU+YLiSTuV8eaE=
github.com/montanaflynn/stats v0.7.1/go.mod h1:etXPPgVO6n31NxCd9KQUMvCM+ve0ruNzt6R8Bnaayow=
github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f h1:y5//uYreIhSUg3J1GEMiLbxo1LJaP8RfCpH6pymGZus=
github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f/go.mod h1:ZdcZmHo+o7JKHSa8/e818NopupXU1YMK5fe1lsApnBw=
github.com/nats-io/jwt/v2 v2.5.0 h1:WQQ40AAlqqfx+f6ku+i0pOVm+ASirD4fUh+oQsiE9Ak=
github.com/nats-io/jwt/v2 v2.5.0/go.mod h1:24BeQtRwxRV8ruvC4CojXlx/WQ/VjuwlYiH+vu/+ibI=
github.com/nats-io/nats-server/v2 v2.9.23 h1:6Wj6H6QpP9FMlpCyWUaNu2yeZ/qGj+mdRkZ1wbikExU=
@@ -1480,7 +1668,11 @@ github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1Cpa
github.com/onsi/gomega v1.37.0 h1:CdEG8g0S133B4OswTDC/5XPSzE1OeP29QOioj2PID2Y=
github.com/onsi/gomega v1.37.0/go.mod h1:8D9+Txp43QWKhM24yyOBEdpkzN8FvJyAwecBgsU4KU0=
github.com/opencontainers/go-digest v1.0.0-rc1/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.0.1/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0=
github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040=
github.com/opencontainers/image-spec v1.1.1/go.mod h1:qpqAh3Dmcf36wStyyWU+kCeDgrGnAve2nCC8+7h8Q0M=
github.com/opencontainers/runc v1.0.0-rc9/go.mod h1:qT5XzbpPznkRYVz/mWwUaVBUv2rmF59PVA73FjuZG0U=
github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o=
github.com/opentracing/opentracing-go v1.2.0 h1:uEJPy/1a5RIPAJ0Ov+OIO8OxWu77jEv+1B0VhjKrZUs=
@@ -1504,14 +1696,17 @@ github.com/pascaldekloe/goe v0.1.0 h1:cBOtyMzM9HTpWjXfbbunk26uA6nG3a8n06Wieeh0Mw
github.com/pascaldekloe/goe v0.1.0/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=
github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc=
github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ=
github.com/pelletier/go-toml v1.9.5 h1:4yBQzkHv+7BHq2PQUZF3Mx0IYxG7LsP222s7Agd3ve8=
github.com/pelletier/go-toml v1.9.5/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c=
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/pengsrc/go-shared v0.2.1-0.20190131101655-1999055a4a14 h1:XeOYlK9W1uCmhjJSsY78Mcuh7MVkNjTzmHx1yBzizSU=
github.com/pengsrc/go-shared v0.2.1-0.20190131101655-1999055a4a14/go.mod h1:jVblp62SafmidSkvWrXyxAme3gaTfEtWwRPGz5cpvHg=
github.com/peterh/liner v1.2.2 h1:aJ4AOodmL+JxOZZEL2u9iJf8omNRpqHc/EbrK+3mAXw=
github.com/peterh/liner v1.2.2/go.mod h1:xFwJyiKIXJZUKItq5dGHZSTBRAuG/CpeNpWLyiNRNwI=
github.com/petermattis/goid v0.0.0-20180202154549-b0b1615b78e5 h1:q2e307iGHPdTGp0hoxKjt1H5pDo6utceo3dQVK3I5XQ=
github.com/petermattis/goid v0.0.0-20180202154549-b0b1615b78e5/go.mod h1:jvVRKCrJTQWu0XVbaOlby/2lO20uSCHEMzzplHXte1o=
github.com/petermattis/goid v0.0.0-20260113132338-7c7de50cc741 h1:KPpdlQLZcHfTMQRi6bFQ7ogNO0ltFT4PmtwTLW4W+14=
github.com/petermattis/goid v0.0.0-20260113132338-7c7de50cc741/go.mod h1:pxMtw7cyUw6B2bRH0ZBANSPg+AoSud1I1iyJHI69jH4=
github.com/philhofer/fwd v1.1.2/go.mod h1:qkPdfjR2SIEbspLqpe1tO4n5yICnr2DY7mqEx2tUTP0=
github.com/philhofer/fwd v1.2.0 h1:e6DnBTl7vGY+Gz322/ASL4Gyp1FspeMvx1RNDoToZuM=
github.com/philhofer/fwd v1.2.0/go.mod h1:RqIHx9QI14HlwKwm98g9Re5prTQ6LdeRQn+gXJFxsJM=
@@ -1592,12 +1787,25 @@ github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4O
github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA=
github.com/prometheus/procfs v0.19.2 h1:zUMhqEW66Ex7OXIiDkll3tl9a1ZdilUOd/F6ZXw4Vws=
github.com/prometheus/procfs v0.19.2/go.mod h1:M0aotyiemPhBCM0z5w87kL22CxfcH05ZpYlu+b4J7mw=
github.com/pterm/pterm v0.12.27/go.mod h1:PhQ89w4i95rhgE+xedAoqous6K9X+r6aSOI2eFF7DZI=
github.com/pterm/pterm v0.12.29/go.mod h1:WI3qxgvoQFFGKGjGnJR849gU0TsEOvKn5Q8LlY1U7lg=
github.com/pterm/pterm v0.12.30/go.mod h1:MOqLIyMOgmTDz9yorcYbcw+HsgoZo3BQfg2wtl3HEFE=
github.com/pterm/pterm v0.12.31/go.mod h1:32ZAWZVXD7ZfG0s8qqHXePte42kdz8ECtRyEejaWgXU=
github.com/pterm/pterm v0.12.33/go.mod h1:x+h2uL+n7CP/rel9+bImHD5lF3nM9vJj80k9ybiiTTE=
github.com/pterm/pterm v0.12.36/go.mod h1:NjiL09hFhT/vWjQHSj1athJpx6H8cjpHXNAK5bUw8T8=
github.com/pterm/pterm v0.12.40/go.mod h1:ffwPLwlbXxP+rxT0GsgDTzS3y3rmpAO1NMjUkGTYf8s=
github.com/pterm/pterm v0.12.81 h1:ju+j5I2++FO1jBKMmscgh5h5DPFDFMB7epEjSoKehKA=
github.com/pterm/pterm v0.12.81/go.mod h1:TyuyrPjnxfwP+ccJdBTeWHtd/e0ybQHkOS/TakajZCw=
github.com/putdotio/go-putio/putio v0.0.0-20200123120452-16d982cac2b8 h1:Y258uzXU/potCYnQd1r6wlAnoMB68BiCkCcCnKx1SH8=
github.com/putdotio/go-putio/putio v0.0.0-20200123120452-16d982cac2b8/go.mod h1:bSJjRokAHHOhA+XFxplld8w2R/dXLH7Z3BZ532vhFwU=
github.com/puzpuzpuz/xsync/v3 v3.5.1 h1:GJYJZwO6IdxN/IKbneznS6yPkVC+c3zyY/j19c++5Fg=
github.com/puzpuzpuz/xsync/v3 v3.5.1/go.mod h1:VjzYrABPabuM4KyBh1Ftq6u8nhwY5tBPKP9jpmh0nnA=
github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8=
github.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII=
github.com/quic-go/quic-go v0.57.0 h1:AsSSrrMs4qI/hLrKlTH/TGQeTMY0ib1pAOX7vA3AdqE=
github.com/quic-go/quic-go v0.57.0/go.mod h1:ly4QBAjHA2VhdnxhojRsCUOeJwKYg+taDlos92xb1+s=
github.com/r3labs/sse v0.0.0-20210224172625-26fe804710bc h1:zAsgcP8MhzAbhMnB1QQ2O7ZhWYVGYSR2iVcjzQuPV+o=
github.com/r3labs/sse v0.0.0-20210224172625-26fe804710bc/go.mod h1:S8xSOnV3CgpNrWd0GQ/OoQfMtlg2uPRSuTzcSGrzwK8=
github.com/rabbitmq/amqp091-go v1.10.0 h1:STpn5XsHlHGcecLmMFCtg7mqq0RnD+zFr4uzukfVhBw=
github.com/rabbitmq/amqp091-go v1.10.0/go.mod h1:Hy4jKW5kQART1u+JkDTF9YYOQUHXqMuhrgxOEeS7G4o=
github.com/rclone/rclone v1.72.1 h1:Cc/NshKd3/TP3CC0cx9Jg9nTLG8YQ8yLYMTm6Z/LdHk=
@@ -1606,8 +1814,8 @@ github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 h1:N/ElC8H3+5X
github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4=
github.com/rdleal/intervalst v1.5.0 h1:SEB9bCFz5IqD1yhfH1Wv8IBnY/JQxDplwkxHjT6hamU=
github.com/rdleal/intervalst v1.5.0/go.mod h1:xO89Z6BC+LQDH+IPQQw/OESt5UADgFD41tYMUINGpxQ=
github.com/redis/go-redis/v9 v9.17.2 h1:P2EGsA4qVIM3Pp+aPocCJ7DguDHhqrXNhVcEp4ViluI=
github.com/redis/go-redis/v9 v9.17.2/go.mod h1:u410H11HMLoB+TP67dz8rL9s6QW2j76l0//kSOd3370=
github.com/redis/go-redis/v9 v9.18.0 h1:pMkxYPkEbMPwRdenAzUNyFNrDgHx9U+DrBabWNfSRQs=
github.com/redis/go-redis/v9 v9.18.0/go.mod h1:k3ufPphLU5YXwNTUcCRXGxUoF1fqxnhFQmscfkCoDA0=
github.com/redis/rueidis v1.0.69 h1:WlUefRhuDekji5LsD387ys3UCJtSFeBVf0e5yI0B8b4=
github.com/redis/rueidis v1.0.69/go.mod h1:Lkhr2QTgcoYBhxARU7kJRO8SyVlgUuEkcJO1Y8MCluA=
github.com/redis/rueidis/rueidiscompat v1.0.69 h1:IWVYY9lXdjNO3do2VpJT7aDFi8zbCUuQxZB6E2Grahs=
@@ -1622,6 +1830,7 @@ github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/rfjakob/eme v1.1.2 h1:SxziR8msSOElPayZNFfQw4Tjx/Sbaeeh3eRvrHVMUs4=
github.com/rfjakob/eme v1.1.2/go.mod h1:cVvpasglm/G3ngEfcfT/Wt0GwhkuO32pf/poW6Nyk1k=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ=
@@ -1654,12 +1863,18 @@ github.com/seaweedfs/goexif v1.0.3 h1:ve/OjI7dxPW8X9YQsv3JuVMaxEyF9Rvfd04ouL+Bz3
github.com/seaweedfs/goexif v1.0.3/go.mod h1:Oni780Z236sXpIQzk1XoJlTwqrJ02smEin9zQeff7Fk=
github.com/seaweedfs/raft v1.1.6 h1:e83Xn0boscPnuSiBllUPeWRVS6JKrqJPYBozgFyBiC0=
github.com/seaweedfs/raft v1.1.6/go.mod h1:9cYlEBA+djJbnf/5tWsCybtbL7ICYpi+Uxcg3MxjuNs=
github.com/secure-systems-lab/go-securesystemslib v0.4.0 h1:b23VGrQhTA8cN2CbBw7/FulN9fTtqYUdS5+Oxzt+DUE=
github.com/secure-systems-lab/go-securesystemslib v0.4.0/go.mod h1:FGBZgq2tXWICsxWQW1msNf49F0Pf2Op5Htayx335Qbs=
github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo=
github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
github.com/sergi/go-diff v1.2.0 h1:XU+rvMAioB0UC3q1MFrIQy4Vo5/4VsRDQQXHsEya6xQ=
github.com/sergi/go-diff v1.2.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
github.com/shirou/gopsutil/v4 v4.25.12 h1:e7PvW/0RmJ8p8vPGJH4jvNkOyLmbkXgXW4m6ZPic6CY=
github.com/shirou/gopsutil/v4 v4.25.12/go.mod h1:EivAfP5x2EhLp2ovdpKSozecVXn1TmuG7SMzs/Wh4PU=
github.com/serialx/hashring v0.0.0-20200727003509-22c0c7ab6b1b h1:h+3JX2VoWTFuyQEo87pStk/a99dzIO1mM9KxIyLPGTU=
github.com/serialx/hashring v0.0.0-20200727003509-22c0c7ab6b1b/go.mod h1:/yeG0My1xr/u+HZrFQ1tOQQQQrOawfyMUH13ai5brBc=
github.com/shibumi/go-pathspec v1.3.0 h1:QUyMZhFo0Md5B8zV8x2tesohbb5kfbpTi9rBnKh5dkI=
github.com/shibumi/go-pathspec v1.3.0/go.mod h1:Xutfslp817l2I1cZvgcfeMQJG5QnU2lh5tVaaMCl3jE=
github.com/shirou/gopsutil/v4 v4.26.1 h1:TOkEyriIXk2HX9d4isZJtbjXbEjf5qyKPAzbzY0JWSo=
github.com/shirou/gopsutil/v4 v4.26.1/go.mod h1:medLI9/UNAb0dOI9Q3/7yWSqKkj00u+1tgY8nvv41pc=
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q=
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
@@ -1726,6 +1941,12 @@ github.com/stvp/tempredis v0.0.0-20181119212430-b82af8480203 h1:QVqDTf3h2WHt08Yu
github.com/stvp/tempredis v0.0.0-20181119212430-b82af8480203/go.mod h1:oqN97ltKNihBbwlX8dLpwxCl3+HnXKV/R0e+sRLd9C8=
github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
github.com/substrait-io/substrait v0.69.0 h1:qfwUe1qKa3PsCclMpubQOF6nqIqS14geUuvzJ1P7gsM=
github.com/substrait-io/substrait v0.69.0/go.mod h1:MPFNw6sToJgpD5Z2rj0rQrdP/Oq8HG7Z2t3CAEHtkHw=
github.com/substrait-io/substrait-go/v4 v4.4.0 h1:mFArMNFxlOLyTuhPcaPzZCwYh6kUopTExTy7XOqtYBM=
github.com/substrait-io/substrait-go/v4 v4.4.0/go.mod h1:GzpaFqO5VRtMkEjATgRxGK5p82OmEtCmszAVYxE+iWc=
github.com/substrait-io/substrait-protobuf/go v0.71.0 h1:vkYGEEPJ8lWSwaJvX7Y+hEmwmrz5/qeDmGI43JpKJZE=
github.com/substrait-io/substrait-protobuf/go v0.71.0/go.mod h1:hn+Szm1NmZZc91FwWK9EXD/lmuGBSRTJ5IvHhlG1YnQ=
github.com/syndtr/goleveldb v1.0.1-0.20190318030020-c3a204f8e965 h1:1oFLiOyVl+W7bnBzGhf7BbIv9loSFQcieWWYIjLqcAw=
github.com/syndtr/goleveldb v1.0.1-0.20190318030020-c3a204f8e965/go.mod h1:9OrXJhf154huy1nPWmuSrkgjPUtUNhA+Zmy+6AESzuA=
github.com/t3rm1n4l/go-mega v0.0.0-20251031123324-a804aaa87491 h1:rrGZv6xYk37hx0tW2sYfgbO0PqStbHqz6Bq6oc9Hurg=
@@ -1735,8 +1956,14 @@ github.com/tarantool/go-iproto v1.1.0 h1:HULVOIHsiehI+FnHfM7wMDntuzUddO09DKqu2Wn
github.com/tarantool/go-iproto v1.1.0/go.mod h1:LNCtdyZxojUed8SbOiYHoc3v9NvaZTB7p96hUySMlIo=
github.com/tarantool/go-tarantool/v2 v2.4.1 h1:Bk9mh+gMPVmHTSefHvVBpEkf6P2UZA/8xa5kqgyQtyo=
github.com/tarantool/go-tarantool/v2 v2.4.1/go.mod h1:MTbhdjFc3Jl63Lgi/UJr5D+QbT+QegqOzsNJGmaw7VM=
github.com/testcontainers/testcontainers-go v0.39.0 h1:uCUJ5tA+fcxbFAB0uP3pIK3EJ2IjjDUHFSZ1H1UxAts=
github.com/testcontainers/testcontainers-go v0.39.0/go.mod h1:qmHpkG7H5uPf/EvOORKvS6EuDkBUPE3zpVGaH9NL7f8=
github.com/testcontainers/testcontainers-go/modules/compose v0.39.0 h1:N9Kn9UOIq24o3Y01SFDYF5y3hpq4dNBzDS4pynHb/OQ=
github.com/testcontainers/testcontainers-go/modules/compose v0.39.0/go.mod h1:7OreVKOBlnD0EmOYXMSBbJkHM8JZQINr34vB0X/NJRs=
github.com/the42/cartconvert v0.0.0-20131203171324-aae784c392b8 h1:I4DY8wLxJXCrMYzDM6lKCGc3IQwJX0PlTLsd3nQqI3c=
github.com/the42/cartconvert v0.0.0-20131203171324-aae784c392b8/go.mod h1:fWO/msnJVhHqN1yX6OBoxSyfj7TEj1hHiL8bJSQsK30=
github.com/theupdateframework/notary v0.7.0 h1:QyagRZ7wlSpjT5N2qQAh/pN+DVqgekv4DzbAiAiEL3c=
github.com/theupdateframework/notary v0.7.0/go.mod h1:c9DRxcmhHmVLDay4/2fUYdISnHqbFDGRSlXPO0AhYWw=
github.com/tiancaiamao/gp v0.0.0-20221230034425-4025bc8a4d4a h1:J/YdBZ46WKpXsxsW93SG+q0F8KI+yFrcIDT4c/RNoc4=
github.com/tiancaiamao/gp v0.0.0-20221230034425-4025bc8a4d4a/go.mod h1:h4xBhSNtOeEosLJ4P7JyKXX7Cabg7AVkWCK5gV2vOrM=
github.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=
@@ -1750,6 +1977,8 @@ github.com/tikv/client-go/v2 v2.0.7 h1:nNTx/AR6n8Ew5VtHanFPG8NkFLLXbaNs5/K43DDma
github.com/tikv/client-go/v2 v2.0.7/go.mod h1:9JNUWtHN8cx8eynHZ9xzdPi5YY6aiN1ILQyhfPUBcMo=
github.com/tikv/pd/client v0.0.0-20230329114254-1948c247c2b1 h1:bzlSSzw+6qTwPs8pMcPI1bt27TAOhSdAEwdPCz6eBlg=
github.com/tikv/pd/client v0.0.0-20230329114254-1948c247c2b1/go.mod h1:3cTcfo8GRA2H/uSttqA3LvMfMSHVBJaXk3IgkFXFVxo=
github.com/tilt-dev/fsnotify v1.4.8-0.20220602155310-fff9c274a375 h1:QB54BJwA6x8QU9nHY3xJSZR2kX9bgpZekRKGkLTmEXA=
github.com/tilt-dev/fsnotify v1.4.8-0.20220602155310-fff9c274a375/go.mod h1:xRroudyp5iVtxKqZCrA6n2TLFRBf8bmnjr1UD4x+z7g=
github.com/tinylib/msgp v1.1.8/go.mod h1:qkpG+2ldGg4xRFmx+jfTvZPxfGFhi64BcnL9vkCm/Tw=
github.com/tinylib/msgp v1.5.0 h1:GWnqAE54wmnlFazjq2+vgr736Akg58iiHImh+kPY2pc=
github.com/tinylib/msgp v1.5.0/go.mod h1:cvjFkb4RiC8qSBOPMGPSzSAx47nAsfhLVTCZZNuHv5o=
@@ -1757,13 +1986,25 @@ github.com/tklauser/go-sysconf v0.3.16 h1:frioLaCQSsF5Cy1jgRBrzr6t502KIIwQ0MArYI
github.com/tklauser/go-sysconf v0.3.16/go.mod h1:/qNL9xxDhc7tx3HSRsLWNnuzbVfh3e7gh/BmM179nYI=
github.com/tklauser/numcpus v0.11.0 h1:nSTwhKH5e1dMNsCdVBukSZrURJRoHbSEQjdEbY+9RXw=
github.com/tklauser/numcpus v0.11.0/go.mod h1:z+LwcLq54uWZTX0u/bGobaV34u6V7KNlTZejzM6/3MQ=
github.com/tmthrgd/go-hex v0.0.0-20190904060850-447a3041c3bc h1:9lRDQMhESg+zvGYmW5DyG0UqvY96Bu5QYsTLvCHdrgo=
github.com/tmthrgd/go-hex v0.0.0-20190904060850-447a3041c3bc/go.mod h1:bciPuU6GHm1iF1pBvUfxfsH0Wmnc2VbpgvbI9ZWuIRs=
github.com/tonistiigi/dchapes-mode v0.0.0-20241001053921-ca0759fec205 h1:eUk79E1w8yMtXeHSzjKorxuC8qJOnyXQnLaJehxpJaI=
github.com/tonistiigi/dchapes-mode v0.0.0-20241001053921-ca0759fec205/go.mod h1:3Iuxbr0P7D3zUzBMAZB+ois3h/et0shEz0qApgHYGpY=
github.com/tonistiigi/fsutil v0.0.0-20250113203817-b14e27f4135a h1:EfGw4G0x/8qXWgtcZ6KVaPS+wpWOQMaypczzP8ojkMY=
github.com/tonistiigi/fsutil v0.0.0-20250113203817-b14e27f4135a/go.mod h1:Dl/9oEjK7IqnjAm21Okx/XIxUCFJzvh+XdVHUlBwXTw=
github.com/tonistiigi/go-csvvalue v0.0.0-20240710180619-ddb21b71c0b4 h1:7I5c2Ig/5FgqkYOh/N87NzoyI9U15qUPXhDD8uCupv8=
github.com/tonistiigi/go-csvvalue v0.0.0-20240710180619-ddb21b71c0b4/go.mod h1:278M4p8WsNh3n4a1eqiFcV2FGk7wE5fwUpUom9mK9lE=
github.com/tonistiigi/units v0.0.0-20180711220420-6950e57a87ea h1:SXhTLE6pb6eld/v/cCndK0AMpt1wiVFb/YYmqB3/QG0=
github.com/tonistiigi/units v0.0.0-20180711220420-6950e57a87ea/go.mod h1:WPnis/6cRcDZSUvVmezrxJPkiO87ThFYsoUiMwWNDJk=
github.com/tonistiigi/vt100 v0.0.0-20240514184818-90bafcd6abab h1:H6aJ0yKQ0gF49Qb2z5hI1UHxSQt4JMyxebFR15KnApw=
github.com/tonistiigi/vt100 v0.0.0-20240514184818-90bafcd6abab/go.mod h1:ulncasL3N9uLrVann0m+CDlJKWsIAP34MPcOJF6VRvc=
github.com/tsuna/gohbase v0.0.0-20201125011725-348991136365 h1:6iRwZdrFUzbcVYZwa8dXTIILGIxmmhjyUPJEcwzPGaU=
github.com/tsuna/gohbase v0.0.0-20201125011725-348991136365/go.mod h1:zj0GJHGvyf1ed3Jm/Tb4830c/ZKDq+YoLsCt2rGQuT0=
github.com/tv42/httpunix v0.0.0-20150427012821-b75d8614f926/go.mod h1:9ESjWnEqriFuLhtthL60Sar/7RFoluCcXsuvEwTV5KM=
github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
github.com/twmb/murmur3 v1.1.3 h1:D83U0XYKcHRYwYIpBKf3Pks91Z0Byda/9SJ8B6EMRcA=
github.com/twmb/murmur3 v1.1.3/go.mod h1:Qq/R7NUyOfr65zD+6Q5IHKsJLwP7exErjN6lyyq3OSQ=
github.com/twmb/murmur3 v1.1.8 h1:8Yt9taO/WN3l08xErzjeschgZU2QSrwm1kclYq+0aRg=
github.com/twmb/murmur3 v1.1.8/go.mod h1:Qq/R7NUyOfr65zD+6Q5IHKsJLwP7exErjN6lyyq3OSQ=
github.com/twpayne/go-geom v1.4.1 h1:LeivFqaGBRfyg0XJJ9pkudcptwhSSrYN9KZUW6HcgdA=
github.com/twpayne/go-geom v1.4.1/go.mod h1:k/zktXdL+qnA6OgKsdEGUTA17jbQ2ZPTUa3CCySuGpE=
github.com/twpayne/go-kml v1.5.2 h1:rFMw2/EwgkVssGS2MT6YfWSPZz6BgcJkLxQ53jnE8rQ=
@@ -1778,6 +2019,22 @@ github.com/ulikunitz/xz v0.5.15 h1:9DNdB5s+SgV3bQ2ApL10xRc35ck0DuIX/isZvIk+ubY=
github.com/ulikunitz/xz v0.5.15/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/unknwon/goconfig v1.0.0 h1:rS7O+CmUdli1T+oDm7fYj1MwqNWtEJfNj+FqcUHML8U=
github.com/unknwon/goconfig v1.0.0/go.mod h1:qu2ZQ/wcC/if2u32263HTVC39PeOQRSmidQk3DuDFQ8=
github.com/uptrace/bun v1.2.15 h1:Ut68XRBLDgp9qG9QBMa9ELWaZOmzHNdczHQdrOZbEFE=
github.com/uptrace/bun v1.2.15/go.mod h1:Eghz7NonZMiTX/Z6oKYytJ0oaMEJ/eq3kEV4vSqG038=
github.com/uptrace/bun/dialect/mssqldialect v1.2.15 h1:QbXtaIlBwx8z0PctUzAQrg4uxRRAKUhkOV4WJvkNo74=
github.com/uptrace/bun/dialect/mssqldialect v1.2.15/go.mod h1:PJxf6utV3uwiBww37CQVD5jvarUKkJHNqSWDO1GkmN4=
github.com/uptrace/bun/dialect/mysqldialect v1.2.15 h1:z/Seg0ljdqoATl0RGPBLHkod1bT0RofL5nNvqdt+UcM=
github.com/uptrace/bun/dialect/mysqldialect v1.2.15/go.mod h1:VUi7mXAL3ttEphcdDta+dXeB7wyI/uvQiE6G8S8ipSQ=
github.com/uptrace/bun/dialect/oracledialect v1.2.15 h1:M7V0BZXlBWirFg5TM/7KZzvB37bHsjslBufOTb/cQqU=
github.com/uptrace/bun/dialect/oracledialect v1.2.15/go.mod h1:M79iy3e2GOkQ4ItatKh5Nz3hKrrV14hnCJVhHbpk538=
github.com/uptrace/bun/dialect/pgdialect v1.2.15 h1:er+/3giAIqpfrXJw+KP9B7ujyQIi5XkPnFmgjAVL6bA=
github.com/uptrace/bun/dialect/pgdialect v1.2.15/go.mod h1:QSiz6Qpy9wlGFsfpf7UMSL6mXAL1jDJhFwuOVacCnOQ=
github.com/uptrace/bun/dialect/sqlitedialect v1.2.15 h1:7upGMVjFRB1oI78GQw6ruNLblYn5CR+kxqcbbeBBils=
github.com/uptrace/bun/dialect/sqlitedialect v1.2.15/go.mod h1:c7YIDaPNS2CU2uI1p7umFuFWkuKbDcPDDvp+DLHZnkI=
github.com/uptrace/bun/driver/sqliteshim v1.2.15 h1:M/rZJSjOPV4OmfTVnDPtL+wJmdMTqDUn8cuk5ycfABA=
github.com/uptrace/bun/driver/sqliteshim v1.2.15/go.mod h1:YqwxFyvM992XOCpGJtXyKPkgkb+aZpIIMzGbpaw1hIk=
github.com/uptrace/bun/extra/bundebug v1.2.15 h1:IY2Z/pVyVg0ApWnQ/pEnwe6BWxlDDATCz7IFZghutCs=
github.com/uptrace/bun/extra/bundebug v1.2.15/go.mod h1:JuE+BT7NjTZ9UKr74eC8s9yZ9dnQCeufDwFRTC8w3Xo=
github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
github.com/viant/assertly v0.9.0 h1:uB3jO+qmWQcrSCHQRxA2kk88eXAdaklUUDxxCU5wBHQ=
@@ -1794,6 +2051,8 @@ github.com/wk8/go-ordered-map/v2 v2.1.8 h1:5h/BUHu93oj4gIdvHHHGsScSTMijfx5PeYkE/
github.com/wk8/go-ordered-map/v2 v2.1.8/go.mod h1:5nJHM5DyteebpVlHnWMV0rPz6Zp7+xBAnxjb1X5vnTw=
github.com/wsxiaoys/terminal v0.0.0-20160513160801-0940f3fc43a0 h1:3UeQBvD0TFrlVjOeLOBz+CPAI8dnbqNSVwUwRrkp7vQ=
github.com/wsxiaoys/terminal v0.0.0-20160513160801-0940f3fc43a0/go.mod h1:IXCdmsXIht47RaVFLEdVnh1t+pgYtTAhQGj73kz+2DM=
github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM=
github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg=
github.com/xanzy/ssh-agent v0.3.3 h1:+/15pJfg/RsTxqYcX6fHqOXZwwMP+2VyYWJeWM2qQFM=
github.com/xanzy/ssh-agent v0.3.3/go.mod h1:6dzNDKs0J9rVPHPhaGCukekBHKqfl+L3KghI1Bc68Uw=
github.com/xdg-go/pbkdf2 v1.0.0 h1:Su7DPu48wXMwC3bs7MCNG+z4FhcyEuz5dlvchbq0B0c=
@@ -1802,12 +2061,18 @@ github.com/xdg-go/scram v1.1.2 h1:FHX5I5B4i4hKRVRBCFRxq1iQRej7WO3hhBuJf+UUySY=
github.com/xdg-go/scram v1.1.2/go.mod h1:RT/sEzTbU5y00aCK8UOx6R7YryM0iF1N2MOmC3kKLN4=
github.com/xdg-go/stringprep v1.0.4 h1:XLI/Ng3O1Atzq0oBs3TWm+5ZVgkq2aqdlvP9JtoZ6c8=
github.com/xdg-go/stringprep v1.0.4/go.mod h1:mPGuuIYwz7CmR2bT9j4GbQqutWS1zV24gijq1dTyGkM=
github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f h1:J9EGpcZtP0E/raorCMxlFGSTBrsSlaDGf3jU/qvAE2c=
github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f/go.mod h1:N2zxlSyiKSe5eX1tZViRH5QA0qijqEDrYZiPEAiq3wU=
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb h1:zGWFAtiMcyryUHoUjUJX0/lt1H2+i2Ka2n+D3DImSNo=
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb/go.mod h1:N2zxlSyiKSe5eX1tZViRH5QA0qijqEDrYZiPEAiq3wU=
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 h1:EzJWgHovont7NscjpAxXsDA8S8BMYve8Y5+7cuRE7R0=
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415/go.mod h1:GwrjFmJcFw6At/Gs6z4yjiIwzuJ1/+UwLxMQDVQXShQ=
github.com/xeipuuv/gojsonschema v1.2.0 h1:LhYJRs+L4fBtjZUfuSZIKGeVu0QRy8e5Xi7D17UxZ74=
github.com/xeipuuv/gojsonschema v1.2.0/go.mod h1:anYRn/JVcOK2ZgGU+IjEV4nwlhoK5sQluxsYJ78Id3Y=
github.com/xhit/go-str2duration/v2 v2.1.0 h1:lxklc02Drh6ynqX+DdPyp5pCKLUQpRT8bp8Ydu2Bstc=
github.com/xhit/go-str2duration/v2 v2.1.0/go.mod h1:ohY8p+0f07DiV6Em5LKB0s2YpLtXVyJfNt1+BlmyAsU=
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778/go.mod h1:2MuV+tbUrU1zIOPMxZ5EncGwgmMJsa+9ucAQZXxsObs=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM=
github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU=
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
github.com/yandex-cloud/go-genproto v0.0.0-20211115083454-9ca41db5ed9e h1:9LPdmD1vqadsDQUva6t2O9MbnyvoOgo8nFNPaOIH5U8=
@@ -1820,8 +2085,8 @@ github.com/ydb-platform/ydb-go-sdk-auth-environ v0.5.1 h1:XaRxeVrOyl3y6v9CiYMWaF
github.com/ydb-platform/ydb-go-sdk-auth-environ v0.5.1/go.mod h1:9YzkhlIymWaJGX6KMU3vh5sOf3UKbCXkG/ZdjaI3zNM=
github.com/ydb-platform/ydb-go-sdk/v3 v3.44.0/go.mod h1:oSLwnuilwIpaF5bJJMAofnGgzPJusoI3zWMNb8I+GnM=
github.com/ydb-platform/ydb-go-sdk/v3 v3.47.3/go.mod h1:bWnOIcUHd7+Sl7DN+yhyY1H/I61z53GczvwJgXMgvj0=
github.com/ydb-platform/ydb-go-sdk/v3 v3.125.1 h1:YaqzRVbcncabB34YNjOl5ADomYUFva+6l74svIIIJUo=
github.com/ydb-platform/ydb-go-sdk/v3 v3.125.1/go.mod h1:stS1mQYjbJvwwYaYzKyFY9eMiuVXWWXQA6T+SpOLg9c=
github.com/ydb-platform/ydb-go-sdk/v3 v3.125.3 h1:hTwpF+PdbuR7vcixN+4AC6yu4asaUIAljQCxL51eDII=
github.com/ydb-platform/ydb-go-sdk/v3 v3.125.3/go.mod h1:stS1mQYjbJvwwYaYzKyFY9eMiuVXWWXQA6T+SpOLg9c=
github.com/ydb-platform/ydb-go-yc v0.12.1 h1:qw3Fa+T81+Kpu5Io2vYHJOwcrYrVjgJlT6t/0dOXJrA=
github.com/ydb-platform/ydb-go-yc v0.12.1/go.mod h1:t/ZA4ECdgPWjAb4jyDe8AzQZB5dhpGbi3iCahFaNwBY=
github.com/ydb-platform/ydb-go-yc-metadata v0.6.1 h1:9E5q8Nsy2RiJMZDNVy0A3KUrIMBPakJ2VgloeWbcI84=
@@ -1839,6 +2104,8 @@ github.com/yunify/qingstor-sdk-go/v3 v3.2.0 h1:9sB2WZMgjwSUNZhrgvaNGazVltoFUUfuS
github.com/yunify/qingstor-sdk-go/v3 v3.2.0/go.mod h1:KciFNuMu6F4WLk9nGwwK69sCGKLCdd9f97ac/wfumS4=
github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo0=
github.com/yusufpapurcu/wmi v1.2.4/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0=
github.com/zclconf/go-cty v1.16.0 h1:xPKEhst+BW5D0wxebMZkxgapvOE/dw7bFTlgSc9nD6w=
github.com/zclconf/go-cty v1.16.0/go.mod h1:VvMs5i0vgZdhYawQNq5kePSpLAoz8u1xvZgrPIxfnZE=
github.com/zeebo/assert v1.3.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0=
github.com/zeebo/assert v1.3.1 h1:vukIABvugfNMZMQO1ABsyQDJDTVQbn+LWSMy1ol1h6A=
github.com/zeebo/assert v1.3.1/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0=
@@ -1854,12 +2121,12 @@ go.einride.tech/aip v0.73.0 h1:bPo4oqBo2ZQeBKo4ZzLb1kxYXTY1ysJhpvQyfuGzvps=
go.einride.tech/aip v0.73.0/go.mod h1:Mj7rFbmXEgw0dq1dqJ7JGMvYCZZVxmGOR3S4ZcV5LvQ=
go.etcd.io/bbolt v1.4.3 h1:dEadXpI6G79deX5prL3QRNP6JB8UxVkqo4UPnHaNXJo=
go.etcd.io/bbolt v1.4.3/go.mod h1:tKQlpPaYCVFctUIgFKFnAlvbmB3tpy1vkTnDWohtc0E=
go.etcd.io/etcd/api/v3 v3.6.6 h1:mcaMp3+7JawWv69p6QShYWS8cIWUOl32bFLb6qf8pOQ=
go.etcd.io/etcd/api/v3 v3.6.6/go.mod h1:f/om26iXl2wSkcTA1zGQv8reJRSLVdoEBsi4JdfMrx4=
go.etcd.io/etcd/api/v3 v3.6.7 h1:7BNJ2gQmc3DNM+9cRkv7KkGQDayElg8x3X+tFDYS+E0=
go.etcd.io/etcd/api/v3 v3.6.7/go.mod h1:xJ81TLj9hxrYYEDmXTeKURMeY3qEDN24hqe+q7KhbnI=
go.etcd.io/etcd/client/pkg/v3 v3.6.7 h1:vvzgyozz46q+TyeGBuFzVuI53/yd133CHceNb/AhBVs=
go.etcd.io/etcd/client/pkg/v3 v3.6.7/go.mod h1:2IVulJ3FZ/czIGl9T4lMF1uxzrhRahLqe+hSgy+Kh7Q=
go.etcd.io/etcd/client/v3 v3.6.6 h1:G5z1wMf5B9SNexoxOHUGBaULurOZPIgGPsW6CN492ec=
go.etcd.io/etcd/client/v3 v3.6.6/go.mod h1:36Qv6baQ07znPR3+n7t+Rk5VHEzVYPvFfGmfF4wBHV8=
go.etcd.io/etcd/client/v3 v3.6.7 h1:9WqA5RpIBtdMxAy1ukXLAdtg2pAxNqW5NUoO2wQrE6U=
go.etcd.io/etcd/client/v3 v3.6.7/go.mod h1:2XfROY56AXnUqGsvl+6k29wrwsSbEh1lAouQB1vHpeE=
go.mongodb.org/mongo-driver v1.17.6 h1:87JUG1wZfWsr6rIz3ZmpH90rL5tea7O3IHuSwHUpsss=
go.mongodb.org/mongo-driver v1.17.6/go.mod h1:Hy04i7O2kC4RS06ZrhPRqj/u4DTYkFDAAccj+rVKqgQ=
go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
@@ -1877,14 +2144,22 @@ go.opentelemetry.io/contrib/detectors/gcp v1.38.0 h1:ZoYbqX7OaA/TAikspPl3ozPI6iY
go.opentelemetry.io/contrib/detectors/gcp v1.38.0/go.mod h1:SU+iU7nu5ud4oCb3LQOhIZ3nRLj6FNVrKgtflbaf2ts=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.63.0 h1:YH4g8lQroajqUwWbq/tr2QX1JFmEXaDLgG+ew9bLMWo=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.63.0/go.mod h1:fvPi2qXDqFs8M4B4fmJhE92TyQs9Ydjlg3RvfUp+NbQ=
go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.56.0 h1:4BZHA+B1wXEQoGNHxW8mURaLhcdGwvRnmhGbm+odRbc=
go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.56.0/go.mod h1:3qi2EEwMgB4xnKgPLqsDP3j9qxnHDZeHsnAxfjQqTko=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0 h1:RbKq8BG0FI8OiXhBfcRtqqHcZcka+gU3cskNuf05R18=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0/go.mod h1:h06DGIukJOevXaj/xrNjhi/2098RZzcLTbc0jDAUbsg=
go.opentelemetry.io/otel v1.38.0 h1:RkfdswUDRimDg0m2Az18RKOsnI8UDzppJAtj01/Ymk8=
go.opentelemetry.io/otel v1.38.0/go.mod h1:zcmtmQ1+YmQM9wrNsTGV/q/uyusom3P8RxwExxkZhjM=
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.37.0 h1:zG8GlgXCJQd5BU98C0hZnBbElszTmUgCNCfYneaDL0A=
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.37.0/go.mod h1:hOfBCz8kv/wuq73Mx2H2QnWokh/kHZxkh6SNF2bdKtw=
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.31.0 h1:ZsXq73BERAiNuuFXYqP4MR5hBrjXfMGSO+Cx7qoOZiM=
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.31.0/go.mod h1:hg1zaDMpyZJuUzjFxFsRYBoccE86tM9Uf4IqNMUxvrY=
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.37.0 h1:Ahq7pZmv87yiyn3jeFz/LekZmPLLdKejuO3NcK9MssM=
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.37.0/go.mod h1:MJTqhM0im3mRLw1i8uGHnCvUEeS7VwRyxlLC78PA18M=
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.37.0 h1:EtFWSnwW9hGObjkIdmlnWSydO+Qs8OwzfzXLUPg4xOc=
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.37.0/go.mod h1:QjUEoiGCPkvFZ/MjK6ZZfNOS6mfVEVKYE99dFhuN2LI=
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.31.0 h1:lUsI2TYsQw2r1IASwoROaCnjdj2cvC2+Jbxvk6nHnWU=
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.31.0/go.mod h1:2HpZxxQurfGxJlJDblybejHB6RX6pmExPNe517hREw4=
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.38.0 h1:wm/Q0GAAykXv83wzcKzGGqAnnfLFyFe7RslekZuv+VI=
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.38.0/go.mod h1:ra3Pa40+oKjvYh+ZD3EdxFZZB0xdMfuileHAm4nNN7w=
go.opentelemetry.io/otel/exporters/zipkin v1.36.0 h1:s0n95ya5tOG03exJ5JySOdJFtwGo4ZQ+KeY7Zro4CLI=
@@ -1919,8 +2194,8 @@ go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/zap v1.18.1/go.mod h1:xg/QME4nWcxGxrpdeYfq7UvYrLh66cuVKdrbD1XF/NI=
go.uber.org/zap v1.19.0/go.mod h1:xg/QME4nWcxGxrpdeYfq7UvYrLh66cuVKdrbD1XF/NI=
go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8=
go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
go.uber.org/zap v1.27.1 h1:08RqriUEv8+ArZRYSTXy1LeBScaMpVSTBhCeaZYfMYc=
go.uber.org/zap v1.27.1/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0=
go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8=
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
@@ -1953,8 +2228,8 @@ golang.org/x/crypto v0.14.0/go.mod h1:MVFd36DqK4CsrnJYDkBA3VC4m2GkXAM0PvzMCn4JQf
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
golang.org/x/crypto v0.46.0 h1:cKRW/pmt1pKAfetfu+RCEvjvZkA9RimPbh7bhFjGVBU=
golang.org/x/crypto v0.46.0/go.mod h1:Evb/oLKmMraqjZ2iQTwDwvCtJkczlDuTmdJXoZVzqU0=
golang.org/x/crypto v0.48.0 h1:/VRzVqiRSggnhY7gNRxPauEQ5Drw9haKdM0jqfcCFts=
golang.org/x/crypto v0.48.0/go.mod h1:r0kV5h3qnFPlQnBSrULhlsRfryS2pmewsg+XfMgkVos=
golang.org/x/exp v0.0.0-20180321215751-8460e604b9de/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20180807140117-3d87b88a115f/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
@@ -2020,8 +2295,8 @@ golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.13.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.15.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.31.0 h1:HaW9xtz0+kOcWKwli0ZXy79Ix+UW/vOfmWI5QVd2tgI=
golang.org/x/mod v0.31.0/go.mod h1:43JraMp9cGx1Rx3AqioxrbrhNsLl2l/iNAvuBkrezpg=
golang.org/x/mod v0.32.0 h1:9F4d3PHLljb6x//jOyokMv3eX+YDeepZSEo3mFJy93c=
golang.org/x/mod v0.32.0/go.mod h1:SgipZ/3h2Ci89DlEtEXWUk/HteuRin+HHhN+WbNhguU=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -2090,8 +2365,8 @@ golang.org/x/net v0.16.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE=
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
golang.org/x/net v0.48.0 h1:zyQRTTrjc33Lhh0fBgT/H3oZq9WuvRR5gPC70xpDiQU=
golang.org/x/net v0.48.0/go.mod h1:+ndRgGjkh8FGtu1w1FGbEC31if4VrNVMuKTgcAAnQRY=
golang.org/x/net v0.49.0 h1:eeHFmOGUTtaaPSGNmjBKpbng9MulQsJURQUAfUwY++o=
golang.org/x/net v0.49.0/go.mod h1:/ysNB2EvaqvesRkuLAyjI1ycPZlQHM3q01F02UY/MV8=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -2212,6 +2487,7 @@ golang.org/x/sys v0.0.0-20210823070655-63515b42dcdf/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20210908233432-aa78b53d3365/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211007075335-d3039528d8ac/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211013075003-97ac67df715c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211019181941-9d821ace8654/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211025201205-69cdffdb9359/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211117180635-dee7805ff2e1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
@@ -2221,6 +2497,7 @@ golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220128215802-99c3d69c2c27/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220209214540-3681064d5158/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220227234510-4e6760a101f9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220319134239-a9b59b0215f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220328115105-d36c6a25d886/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220408201424-a24fb2fb8a0f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220412211240-33da011f77ad/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
@@ -2248,10 +2525,14 @@ golang.org/x/sys v0.13.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.39.0 h1:CvCKL8MeisomCi6qNZ+wbb0DN9E5AATixKsvNtMoMFk=
golang.org/x/sys v0.39.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/sys v0.41.0 h1:Ivj+2Cp/ylzLiEU89QhWblYnOE9zerudt9Ftecq2C6k=
golang.org/x/sys v0.41.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2/go.mod h1:TeRTkGYfJXctD9OcfyVLyj2J3IxLnKwHJR8f4D8a3YE=
golang.org/x/telemetry v0.0.0-20260109210033-bd525da824e2 h1:O1cMQHRfwNpDfDJerqRoE2oD+AFlyid87D40L/OkkJo=
golang.org/x/telemetry v0.0.0-20260109210033-bd525da824e2/go.mod h1:b7fPSJ0pKZ3ccUh8gnTONJxhn3c/PS6tyzQvyqw4iA8=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210220032956-6a3ed077a48d/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc=
golang.org/x/term v0.3.0/go.mod h1:q750SLmJuPmVoN1blW3UFBPREJfb1KmY3vwxfr+nFDA=
@@ -2264,8 +2545,8 @@ golang.org/x/term v0.13.0/go.mod h1:LTmsnFJwVN6bCy1rVCoS+qHT1HhALEFxKncY3WNNh4U=
golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk=
golang.org/x/term v0.20.0/go.mod h1:8UkIAJTvZgivsXaD6/pH6U9ecQzZ45awqEOzuCvwpFY=
golang.org/x/term v0.27.0/go.mod h1:iMsnZpn0cago0GOrHO2+Y7u7JPn5AylBrcoWkElMTSM=
golang.org/x/term v0.38.0 h1:PQ5pkm/rLO6HnxFR7N2lJHOZX6Kez5Y1gDSJla6jo7Q=
golang.org/x/term v0.38.0/go.mod h1:bSEAKrOT1W+VSu9TSCMtoGEOUcKxOKgl3LE5QEF/xVg=
golang.org/x/term v0.40.0 h1:36e4zGLqU4yhjlmxEaagx2KuYbJq3EwY8K943ZsHcvg=
golang.org/x/term v0.40.0/go.mod h1:w2P8uVp06p2iyKKuvXIm7N/y0UCRt3UfJTfZ7oOpglM=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -2286,8 +2567,8 @@ golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
golang.org/x/text v0.33.0 h1:B3njUFyqtHDUI5jMn1YIr5B0IE2U0qck04r6d4KPAxE=
golang.org/x/text v0.33.0/go.mod h1:LuMebE6+rBincTi9+xWTY8TztLzKHc/9C1uBCG27+q8=
golang.org/x/text v0.34.0 h1:oL/Qq0Kdaqxa1KbNeMKwQq0reLCCaFtqu2eNuSeNHbk=
golang.org/x/text v0.34.0/go.mod h1:homfLqTYRFyVYemLBFl5GgL/DWEiH5wcsQ5gSh1yziA=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
@@ -2366,8 +2647,8 @@ golang.org/x/tools v0.7.0/go.mod h1:4pg6aUX35JBAogB10C9AtvVL+qowtN4pT3CGSQex14s=
golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58=
golang.org/x/tools v0.14.0/go.mod h1:uYBEerGOWcJyEORxN+Ek8+TT266gXkNlHdJBwexUsBg=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
golang.org/x/tools v0.40.0 h1:yLkxfA+Qnul4cs9QA3KnlFu0lVmd8JJfoq+E41uSutA=
golang.org/x/tools v0.40.0/go.mod h1:Ik/tzLRlbscWpqqMRjyWYDisX8bG13FrdXp3o4Sr9lc=
golang.org/x/tools v0.41.0 h1:a9b8iMweWG+S0OBnlU36rzLp20z1Rp10w+IY2czHTQc=
golang.org/x/tools v0.41.0/go.mod h1:XSY6eDqxVNiYgezAVqqCeihT4j1U2CCsqvH3WhQpnlg=
golang.org/x/tools/godoc v0.1.0-deprecated h1:o+aZ1BOj6Hsx/GBdJO/s815sqftjSnrZZwyYTHODvtk=
golang.org/x/tools/godoc v0.1.0-deprecated/go.mod h1:qM63CriJ961IHWmnWa9CjZnBndniPt4a3CK0PVB9bIg=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@@ -2582,8 +2863,8 @@ google.golang.org/genproto v0.0.0-20230222225845-10f96fb3dbec/go.mod h1:3Dl5ZL0q
google.golang.org/genproto v0.0.0-20230306155012-7f2fa6fef1f4/go.mod h1:NWraEVixdDnqcqQ30jipen1STv2r/n24Wb7twVTGR4s=
google.golang.org/genproto v0.0.0-20250922171735-9219d122eba9 h1:LvZVVaPE0JSqL+ZWb6ErZfnEOKIqqFWUJE2D0fObSmc=
google.golang.org/genproto v0.0.0-20250922171735-9219d122eba9/go.mod h1:QFOrLhdAe2PsTp3vQY4quuLKTi9j3XG3r6JPPaw7MSc=
google.golang.org/genproto/googleapis/api v0.0.0-20251111163417-95abcf5c77ba h1:B14OtaXuMaCQsl2deSvNkyPKIzq3BjfxQp8d00QyWx4=
google.golang.org/genproto/googleapis/api v0.0.0-20251111163417-95abcf5c77ba/go.mod h1:G5IanEx8/PgI9w6CFcYQf7jMtHQhZruvfM1i3qOqk5U=
google.golang.org/genproto/googleapis/api v0.0.0-20251124214823-79d6a2a48846 h1:ZdyUkS9po3H7G0tuh955QVyyotWvOD4W0aEapeGeUYk=
google.golang.org/genproto/googleapis/api v0.0.0-20251124214823-79d6a2a48846/go.mod h1:Fk4kyraUvqD7i5H6S43sj2W98fbZa75lpZz/eUyhfO0=
google.golang.org/genproto/googleapis/rpc v0.0.0-20251213004720-97cd9d5aeac2 h1:2I6GHUeJ/4shcDpoUlLs/2WPnhg7yJwvXtqcMJt9liA=
google.golang.org/genproto/googleapis/rpc v0.0.0-20251213004720-97cd9d5aeac2/go.mod h1:7i2o+ce6H/6BluujYR+kqX3GKH+dChPTQU19wjRPiGk=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
@@ -2652,6 +2933,8 @@ google.golang.org/protobuf v1.30.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqw
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
gopkg.in/cenkalti/backoff.v1 v1.1.0 h1:Arh75ttbsvlpVA7WtVpH4u9h6Zl46xuptxqLxPiSo4Y=
gopkg.in/cenkalti/backoff.v1 v1.1.0/go.mod h1:J6Vskwqd+OMVJl8C33mmtxTBs2gyzfv7UDAkHu8BrjI=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
@@ -2661,6 +2944,8 @@ gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc=
gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw=
gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=
gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
gopkg.in/mgo.v2 v2.0.0-20190816093944-a6b53ec6cb22/go.mod h1:yeKp02qBN3iKW1OzL3MGk2IdtZzaj7SFntXj72NppTA=
gopkg.in/natefinch/lumberjack.v2 v2.0.0/go.mod h1:l0ndWWf7gzL7RNwBG7wST/UCcT4T24xpD6X8LsfU/+k=
gopkg.in/natefinch/lumberjack.v2 v2.2.1 h1:bBRl1b0OH9s/DuPhuXpNl+VtCaJXFZ5/uEFST95x9zc=
@@ -2686,6 +2971,8 @@ gopkg.in/yaml.v3 v3.0.0/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gotest.tools/v3 v3.0.2/go.mod h1:3SzNCllyD9/Y+b5r9JIKQ474KzkZyqLqEfYqMsX94Bk=
gotest.tools/v3 v3.5.2 h1:7koQfIKdy+I8UTetycgUqXWSDwpgv193Ka+qRsmBY8Q=
gotest.tools/v3 v3.5.2/go.mod h1:LtdLGcnqToBH83WByAAi/wiwSFCArdFIUV/xxN4pcjA=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
@@ -2694,6 +2981,18 @@ honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt
honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
honnef.co/go/tools v0.1.3/go.mod h1:NgwopIslSNH47DimFoV78dnkksY2EFtX0ajyb3K/las=
k8s.io/api v0.31.2 h1:3wLBbL5Uom/8Zy98GRPXpJ254nEFpl+hwndmk9RwmL0=
k8s.io/api v0.31.2/go.mod h1:bWmGvrGPssSK1ljmLzd3pwCQ9MgoTsRCuK35u6SygUk=
k8s.io/apimachinery v0.31.2 h1:i4vUt2hPK56W6mlT7Ry+AO8eEsyxMD1U44NR22CLTYw=
k8s.io/apimachinery v0.31.2/go.mod h1:rsPdaZJfTfLsNJSQzNHQvYoTmxhoOEofxtOsF3rtsMo=
k8s.io/client-go v0.31.2 h1:Y2F4dxU5d3AQj+ybwSMqQnpZH9F30//1ObxOKlTI9yc=
k8s.io/client-go v0.31.2/go.mod h1:NPa74jSVR/+eez2dFsEIHNa+3o09vtNaWwWwb1qSxSs=
k8s.io/klog/v2 v2.130.1 h1:n9Xl7H1Xvksem4KFG4PYbdQCQxqc/tTUyrgXaOhHSzk=
k8s.io/klog/v2 v2.130.1/go.mod h1:3Jpz1GvMt720eyJH1ckRHK1EDfpxISzJ7I9OYgaDtPE=
k8s.io/kube-openapi v0.0.0-20240228011516-70dd3763d340 h1:BZqlfIlq5YbRMFko6/PM7FjZpUb45WallggurYhKGag=
k8s.io/kube-openapi v0.0.0-20240228011516-70dd3763d340/go.mod h1:yD4MZYeKMBwQKVht279WycxKyM84kkAx2DPrTXaeb98=
k8s.io/utils v0.0.0-20240711033017-18e509b52bc8 h1:pUdcCO1Lk/tbT5ztQWOBi5HBgbBP1J8+AsQnQCKsi8A=
k8s.io/utils v0.0.0-20240711033017-18e509b52bc8/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0=
lukechampine.com/uint128 v1.1.1/go.mod h1:c4eWIwlEGaxC/+H1VguhU4PHXNWDCDMUlWdIWl2j1gk=
lukechampine.com/uint128 v1.2.0/go.mod h1:c4eWIwlEGaxC/+H1VguhU4PHXNWDCDMUlWdIWl2j1gk=
modernc.org/b v1.0.0 h1:vpvqeyp17ddcQWF29Czawql4lDdABCDRbXRAS4+aF2o=
@@ -2748,8 +3047,8 @@ modernc.org/opt v0.1.4/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.18.1/go.mod h1:6ho+Gow7oX5V+OiOQ6Tr4xeqbx13UZ6t+Fw9IRUG4d4=
modernc.org/sqlite v1.44.2 h1:EdYqXeBpKFJjg8QYnw6E71MpANkoxyuYi+g68ugOL8g=
modernc.org/sqlite v1.44.2/go.mod h1:CzbrU2lSB1DKUusvwGz7rqEKIq+NUd8GWuBBZDs9/nA=
modernc.org/sqlite v1.44.3 h1:+39JvV/HWMcYslAwRxHb8067w+2zowvFOUrOWIy9PjY=
modernc.org/sqlite v1.44.3/go.mod h1:CzbrU2lSB1DKUusvwGz7rqEKIq+NUd8GWuBBZDs9/nA=
modernc.org/strutil v1.1.0/go.mod h1:lstksw84oURvj9y3tn8lGvRxyRC1S2+g5uuIzNfIOBs=
modernc.org/strutil v1.1.1/go.mod h1:DE+MQQ/hjKBZS2zNInV5hhcipt5rLPWkmpbGeW5mmdw=
modernc.org/strutil v1.1.3/go.mod h1:MEHNA7PdEnEwLvspRMtWTNnp2nnyvMfkimT1NKNAGbw=
@@ -2766,6 +3065,10 @@ rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8
rsc.io/pdf v0.1.1/go.mod h1:n8OzWcQ6Sp37PL01nO98y4iUCRdTGarVfzxY20ICaU4=
rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd h1:EDPBXCAspyGV4jQlpZSudPeMmr1bNJefnuqLsRAsHZo=
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd/go.mod h1:B8JuhiUyNFVKdsE8h686QcCxMaH6HrOAZj4vswFpcB0=
sigs.k8s.io/structured-merge-diff/v4 v4.4.1 h1:150L+0vs/8DA78h1u02ooW1/fFq/Lwr+sGiqlzvrtq4=
sigs.k8s.io/structured-merge-diff/v4 v4.4.1/go.mod h1:N8hJocpFajUSSeSJ9bOZ77VzejKZaXsTtZo4/u7Io08=
sigs.k8s.io/yaml v1.6.0 h1:G8fkbMSAFqgEFgh4b1wmtzDnioxFCUgTZhlbj5P9QYs=
sigs.k8s.io/yaml v1.6.0/go.mod h1:796bPqUfzR/0jLAl6XjHl3Ck7MiyVv8dbTdyT3/pMf4=
storj.io/common v0.0.0-20251107171817-6221ae45072c h1:UDXSrdeLJe3QFouavSW10fYdpclK0YNu3KvQHzqq2+k=
@@ -2780,3 +3083,5 @@ storj.io/picobuf v0.0.4 h1:qswHDla+YZ2TovGtMnU4astjvrADSIz84FXRn0qgP6o=
storj.io/picobuf v0.0.4/go.mod h1:hSMxmZc58MS/2qSLy1I0idovlO7+6K47wIGUyRZa6mg=
storj.io/uplink v1.13.1 h1:C8RdW/upALoCyuF16Lod9XGCXEdbJAS+ABQy9JO/0pA=
storj.io/uplink v1.13.1/go.mod h1:x0MQr4UfFsQBwgVWZAtEsLpuwAn6dg7G0Mpne1r516E=
tags.cncf.io/container-device-interface v1.0.1 h1:KqQDr4vIlxwfYh0Ed/uJGVgX+CHAkahrgabg6Q8GYxc=
tags.cncf.io/container-device-interface v1.0.1/go.mod h1:JojJIOeW3hNbcnOH2q0NrWNha/JuHoDZcmYxAZwb2i0=
+2 -2
View File
@@ -1,6 +1,6 @@
apiVersion: v1
description: SeaweedFS
name: seaweedfs
appVersion: "4.07"
appVersion: "4.13"
# Dev note: Trigger a helm chart release by `git tag -a helm-<version>`
version: 4.0.407
version: 4.0.413
+34 -15
View File
@@ -1,6 +1,5 @@
# SEAWEEDFS - helm chart (2.x+)
## Getting Started
### Add the helm repo
@@ -24,7 +23,7 @@ helm install --values=values.yaml seaweedfs seaweedfs/seaweedfs
* master/filer/volume are stateful sets with anti-affinity on the hostname,
so your deployment will be spread/HA.
* chart is using memsql(mysql) as the filer backend to enable HA (multiple filer instances) and backup/HA memsql can provide.
* mysql user/password are created in a k8s secret (secret-seaweedfs-db.yaml) and injected to the filer with ENV.
* mysql user/password are created in a k8s secret (default: `<release>-seaweedfs-db-secret`) and injected to the filer with ENV.
* cert config exists and can be enabled, but not been tested, requires cert-manager to be installed.
## Prerequisites
@@ -35,7 +34,9 @@ leveldb is the default database, this supports multiple filer replicas that will
When the [limitations](https://github.com/seaweedfs/seaweedfs/wiki/Filer-Store-Replication#limitation) apply, or for a large number of filer replicas, an external datastore is recommended.
Such as MySQL-compatible database, as specified in the `values.yaml` at `filer.extraEnvironmentVars`.
This database should be pre-configured and initialized by running:
This database should be pre-configured and initialized. If using the default `db-init-config`, the configmap name is now dynamic (e.g., `<release>-seaweedfs-db-init-config`). You can override this name via `filer.dbInitConfigName`.
To initialize manually:
```sql
CREATE TABLE IF NOT EXISTS `filemeta` (
`dirhash` BIGINT NOT NULL COMMENT 'first 64 bits of MD5 hash value of directory field',
@@ -211,8 +212,9 @@ To enable workers, add the following to your values.yaml:
worker:
enabled: true
replicas: 2 # Scale based on workload
capabilities: "vacuum,balance,erasure_coding" # Tasks this worker can handle
maxConcurrent: 3 # Maximum concurrent tasks per worker
jobType: "vacuum,volume_balance,erasure_coding" # Job types this worker can handle
maxDetect: 1 # Maximum concurrent detection requests
maxExecute: 4 # Maximum concurrent execution jobs per worker
# Working directory for task execution
# Default: "/tmp/seaweedfs-worker"
@@ -247,14 +249,14 @@ worker:
memory: "2Gi"
```
### Worker Capabilities
### Worker Job Types
Workers can be configured with different capabilities:
Workers can be configured with different job types:
- **vacuum**: Reclaim deleted file space
- **balance**: Balance volumes across volume servers
- **volume_balance**: Balance volumes across volume servers
- **erasure_coding**: Handle erasure coding operations
You can configure workers with all capabilities or create specialized worker pools with specific capabilities.
You can configure workers with all job types or create specialized worker pools with specific job types.
### Worker Deployment Strategy
@@ -263,11 +265,11 @@ For production deployments, consider:
1. **Multiple Workers**: Deploy 2+ worker replicas for high availability
2. **Resource Allocation**: Workers need sufficient CPU/memory for maintenance tasks
3. **Storage**: Workers need temporary storage for vacuum and balance operations (size depends on volume size)
4. **Specialized Workers**: Create separate worker deployments for different capabilities if needed
4. **Specialized Workers**: Create separate worker deployments for different job types if needed
Example specialized worker configuration:
For specialized worker pools, deploy separate Helm releases with different capabilities:
For specialized worker pools, deploy separate Helm releases with different job types:
**values-worker-vacuum.yaml** (for vacuum operations):
```yaml
@@ -286,8 +288,8 @@ admin:
worker:
enabled: true
replicas: 2
capabilities: "vacuum"
maxConcurrent: 2
jobType: "vacuum"
maxExecute: 2
# REQUIRED: Point to the admin service of your main SeaweedFS release
# Replace <namespace> with the namespace where your main seaweedfs is deployed
# Example: If deploying in namespace "production":
@@ -312,8 +314,8 @@ admin:
worker:
enabled: true
replicas: 1
capabilities: "balance"
maxConcurrent: 1
jobType: "volume_balance"
maxExecute: 1
# REQUIRED: Point to the admin service of your main SeaweedFS release
# Replace <namespace> with the namespace where your main seaweedfs is deployed
# Example: If deploying in namespace "production":
@@ -322,6 +324,7 @@ worker:
```
Deploy the specialized workers as separate releases:
### Specialized Worker Deployment
```bash
# Deploy vacuum workers
helm install seaweedfs-worker-vacuum seaweedfs/seaweedfs -f values-worker-vacuum.yaml
@@ -330,6 +333,22 @@ helm install seaweedfs-worker-vacuum seaweedfs/seaweedfs -f values-worker-vacuum
helm install seaweedfs-worker-balance seaweedfs/seaweedfs -f values-worker-balance.yaml
```
## OpenShift Support
SeaweedFS can be deployed on OpenShift or any cluster enforcing the Kubernetes "restricted" Pod Security Standard. By default, OpenShift blocks containers that run as root or use `hostPath` volumes.
To deploy on OpenShift, use the provided `openshift-values.yaml` which overrides the default configuration to:
1. Use `PersistentVolumeClaims` instead of `hostPath`.
2. Enable `runAsNonRoot` and omit hardcoded UIDs to allow OpenShift to assign valid UIDs automatically.
3. Apply appropriate `seccompProfile` and drop capabilities.
Usage:
```bash
helm install seaweedfs seaweedfs/seaweedfs \
-n seaweedfs --create-namespace \
-f openshift-values.yaml
```
## Enterprise
For enterprise users, please visit [seaweedfs.com](https://seaweedfs.com) for the SeaweedFS Enterprise Edition,
+116
View File
@@ -0,0 +1,116 @@
# openshift-values.yaml
#
# Example overrides for deploying SeaweedFS on OpenShift (or any cluster
# enforcing the Kubernetes "restricted" Pod Security Standard).
#
# OpenShift's default "restricted" SCC blocks containers that:
# - Run as UID 0 (root)
# - Request privilege escalation
# - Use hostPath volumes
# - Omit a seccompProfile
#
# These overrides satisfy all four requirements by:
# 1. Replacing hostPath volumes with PersistentVolumeClaims (or emptyDir for logs)
# 2. Enabling runAsNonRoot: true. By omitting runAsUser, OpenShift will
# automatically assign a valid UID from the namespace's allocated range.
# 3. Dropping all Linux capabilities and setting allowPrivilegeEscalation: false
# 4. Enabling RuntimeDefault seccompProfile
#
# Usage:
# helm install seaweedfs seaweedfs/seaweedfs \
# -n seaweedfs --create-namespace \
# -f openshift-values.yaml
#
# Adjust storageClass and sizes to match your cluster's available StorageClasses.
# On OpenShift you can discover them with: oc get storageclass
master:
data:
type: "persistentVolumeClaim"
size: "10Gi"
storageClass: "" # leave empty to use the cluster default StorageClass
logs:
type: "emptyDir" # avoids hostPath; use persistentVolumeClaim if you need log persistence
podSecurityContext:
enabled: true
# On OpenShift, we omit runAsUser/runAsGroup/fsGroup to let the admission
# controller assign them automatically based on the namespace's SCC.
runAsNonRoot: true
containerSecurityContext:
enabled: true
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
volume:
dataDirs:
- name: data1
type: "persistentVolumeClaim"
size: "100Gi"
storageClass: "" # leave empty to use the cluster default StorageClass
maxVolumes: 0
logs:
type: "emptyDir"
podSecurityContext:
enabled: true
# On OpenShift, we omit runAsUser/runAsGroup/fsGroup to let the admission
# controller assign them automatically based on the namespace's SCC.
runAsNonRoot: true
containerSecurityContext:
enabled: true
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
filer:
data:
type: "persistentVolumeClaim"
size: "25Gi"
storageClass: "" # leave empty to use the cluster default StorageClass
logs:
type: "emptyDir"
podSecurityContext:
enabled: true
# On OpenShift, we omit runAsUser/runAsGroup/fsGroup to let the admission
# controller assign them automatically based on the namespace's SCC.
runAsNonRoot: true
containerSecurityContext:
enabled: true
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
# S3 gateway (if enabled)
s3:
podSecurityContext:
enabled: true
# On OpenShift, we omit runAsUser/runAsGroup/fsGroup to let the admission
# controller assign them automatically based on the namespace's SCC.
runAsNonRoot: true
containerSecurityContext:
enabled: true
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
@@ -8,7 +8,7 @@ apiVersion: extensions/v1beta1
{{- end }}
kind: Ingress
metadata:
name: ingress-{{ template "seaweedfs.name" . }}-admin
name: ingress-{{ include "seaweedfs.fullname" . }}-admin
namespace: {{ .Release.Namespace }}
annotations:
{{- if and (not (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion)) .Values.admin.ingress.className }}
@@ -42,11 +42,11 @@ spec:
backend:
{{- if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion }}
service:
name: {{ template "seaweedfs.name" . }}-admin
name: {{ include "seaweedfs.componentName" (list . "admin") }}
port:
number: {{ .Values.admin.port }}
{{- else }}
serviceName: {{ template "seaweedfs.name" . }}-admin
serviceName: {{ include "seaweedfs.componentName" (list . "admin") }}
servicePort: {{ .Values.admin.port }}
{{- end }}
{{- end }}
@@ -3,7 +3,7 @@ apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: {{ template "seaweedfs.name" . }}-admin-secret
name: {{ include "seaweedfs.fullname" . }}-admin-secret
namespace: {{ .Release.Namespace }}
annotations:
"helm.sh/resource-policy": keep
@@ -2,7 +2,7 @@
apiVersion: v1
kind: Service
metadata:
name: {{ template "seaweedfs.name" . }}-admin
name: {{ include "seaweedfs.componentName" (list . "admin") }}
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -3,7 +3,7 @@
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: {{ template "seaweedfs.name" . }}-admin
name: {{ include "seaweedfs.fullname" . }}-admin
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -26,6 +26,7 @@ spec:
selector:
matchLabels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: admin
{{- end }}
{{- end }}
@@ -1,11 +1,14 @@
{{- if .Values.admin.enabled }}
{{- if gt (.Values.admin.replicas | int) 1 }}
{{- fail "admin.replicas must be 0 or 1" -}}
{{- end }}
{{- if and (not .Values.admin.masters) (not .Values.global.masterServer) (not .Values.master.enabled) }}
{{- fail "admin.masters or global.masterServer must be set if master.enabled is false" -}}
{{- end }}
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: {{ template "seaweedfs.name" . }}-admin
name: {{ include "seaweedfs.componentName" (list . "admin") }}
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -18,7 +21,7 @@ metadata:
{{- toYaml .Values.admin.annotations | nindent 4 }}
{{- end }}
spec:
serviceName: {{ template "seaweedfs.name" . }}-admin
serviceName: {{ include "seaweedfs.componentName" (list . "admin") }}
podManagementPolicy: {{ .Values.admin.podManagementPolicy }}
replicas: {{ .Values.admin.replicas }}
selector:
@@ -87,7 +90,7 @@ spec:
{{- if and .Values.admin.secret.existingSecret (not .Values.admin.secret.pwKey) -}}
{{- fail "admin.secret.pwKey must be set when admin.secret.existingSecret is provided" -}}
{{- end -}}
{{- $adminSecretName := .Values.admin.secret.existingSecret | default (printf "%s-admin-secret" (include "seaweedfs.name" .)) }}
{{- $adminSecretName := .Values.admin.secret.existingSecret | default (printf "%s-admin-secret" (include "seaweedfs.fullname" .)) }}
env:
{{- if $adminAuthEnabled }}
- name: SEAWEEDFS_ADMIN_USER
@@ -114,7 +117,7 @@ spec:
fieldRef:
fieldPath: metadata.namespace
- name: SEAWEEDFS_FULLNAME
value: "{{ template "seaweedfs.name" . }}"
value: "{{ include "seaweedfs.fullname" . }}"
{{- if .Values.admin.extraEnvironmentVars }}
{{- range $key, $value := .Values.admin.extraEnvironmentVars }}
- name: {{ $key }}
@@ -281,25 +284,25 @@ spec:
{{- if .Values.global.enableSecurity }}
- name: security-config
configMap:
name: {{ template "seaweedfs.name" . }}-security-config
name: {{ include "seaweedfs.fullname" . }}-security-config
- name: ca-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-ca-cert
secretName: {{ include "seaweedfs.fullname" . }}-ca-cert
- name: master-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-master-cert
secretName: {{ include "seaweedfs.fullname" . }}-master-cert
- name: volume-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-volume-cert
secretName: {{ include "seaweedfs.fullname" . }}-volume-cert
- name: filer-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-filer-cert
secretName: {{ include "seaweedfs.fullname" . }}-filer-cert
- name: client-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-client-cert
secretName: {{ include "seaweedfs.fullname" . }}-client-cert
- name: admin-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-admin-cert
secretName: {{ include "seaweedfs.fullname" . }}-admin-cert
{{- end }}
{{ tpl .Values.admin.extraVolumes . | indent 8 | trim }}
{{- if .Values.admin.nodeSelector }}
@@ -2,7 +2,7 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ template "seaweedfs.name" . }}-all-in-one
name: {{ include "seaweedfs.fullname" . }}-all-in-one
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -98,7 +98,7 @@ spec:
fieldRef:
fieldPath: metadata.namespace
- name: SEAWEEDFS_FULLNAME
value: "{{ template "seaweedfs.name" . }}"
value: "{{ include "seaweedfs.fullname" . }}"
{{- if .Values.allInOne.extraEnvironmentVars }}
{{- range $key, $value := .Values.allInOne.extraEnvironmentVars }}
{{- if and (ne $key $clusterMasterKey) (ne $key $clusterFilerKey) }}
@@ -421,7 +421,7 @@ spec:
type: DirectoryOrCreate
{{- else if eq .Values.allInOne.data.type "persistentVolumeClaim" }}
persistentVolumeClaim:
claimName: {{ template "seaweedfs.name" . }}-all-in-one-data
claimName: {{ include "seaweedfs.fullname" . }}-all-in-one-data
{{- else if eq .Values.allInOne.data.type "existingClaim" }}
persistentVolumeClaim:
claimName: {{ .Values.allInOne.data.claimName }}
@@ -432,47 +432,47 @@ spec:
- name: config-s3-users
secret:
defaultMode: 420
secretName: {{ default (printf "%s-s3-secret" (include "seaweedfs.name" .)) (or .Values.allInOne.s3.existingConfigSecret .Values.s3.existingConfigSecret .Values.filer.s3.existingConfigSecret) }}
secretName: {{ default (printf "%s-s3-secret" (include "seaweedfs.fullname" .)) (or .Values.allInOne.s3.existingConfigSecret .Values.s3.existingConfigSecret .Values.filer.s3.existingConfigSecret) }}
{{- end }}
{{- if .Values.allInOne.sftp.enabled }}
- name: config-ssh
secret:
defaultMode: 420
secretName: {{ default (printf "%s-sftp-ssh-secret" (include "seaweedfs.name" .)) (or .Values.allInOne.sftp.existingSshConfigSecret .Values.sftp.existingSshConfigSecret) }}
secretName: {{ default (printf "%s-sftp-ssh-secret" (include "seaweedfs.fullname" .)) (or .Values.allInOne.sftp.existingSshConfigSecret .Values.sftp.existingSshConfigSecret) }}
{{- if or .Values.allInOne.sftp.enableAuth .Values.sftp.enableAuth }}
- name: config-users
secret:
defaultMode: 420
secretName: {{ default (printf "%s-sftp-secret" (include "seaweedfs.name" .)) (or .Values.allInOne.sftp.existingConfigSecret .Values.sftp.existingConfigSecret) }}
secretName: {{ default (printf "%s-sftp-secret" (include "seaweedfs.fullname" .)) (or .Values.allInOne.sftp.existingConfigSecret .Values.sftp.existingConfigSecret) }}
{{- end }}
{{- end }}
{{- if .Values.filer.notificationConfig }}
- name: notification-config
configMap:
name: {{ template "seaweedfs.name" . }}-notification-config
name: {{ include "seaweedfs.fullname" . }}-notification-config
{{- end }}
- name: master-config
configMap:
name: {{ template "seaweedfs.name" . }}-master-config
name: {{ include "seaweedfs.fullname" . }}-master-config
{{- if .Values.global.enableSecurity }}
- name: security-config
configMap:
name: {{ template "seaweedfs.name" . }}-security-config
name: {{ include "seaweedfs.fullname" . }}-security-config
- name: ca-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-ca-cert
secretName: {{ include "seaweedfs.fullname" . }}-ca-cert
- name: master-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-master-cert
secretName: {{ include "seaweedfs.fullname" . }}-master-cert
- name: volume-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-volume-cert
secretName: {{ include "seaweedfs.fullname" . }}-volume-cert
- name: filer-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-filer-cert
secretName: {{ include "seaweedfs.fullname" . }}-filer-cert
- name: client-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-client-cert
secretName: {{ include "seaweedfs.fullname" . }}-client-cert
{{- end }}
{{ tpl .Values.allInOne.extraVolumes . | nindent 8 }}
{{- if .Values.allInOne.nodeSelector }}
@@ -3,7 +3,7 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ template "seaweedfs.name" . }}-all-in-one-data
name: {{ include "seaweedfs.fullname" . }}-all-in-one-data
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -2,7 +2,7 @@
apiVersion: v1
kind: Service
metadata:
name: {{ template "seaweedfs.name" . }}-all-in-one
name: {{ printf "%s-all-in-one" (include "seaweedfs.fullname" .) | trunc 63 | trimSuffix "-" }}
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -81,5 +81,6 @@ spec:
selector:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: seaweedfs-all-in-one
{{- end }}
@@ -3,7 +3,7 @@
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: {{ template "seaweedfs.name" . }}-all-in-one
name: {{ include "seaweedfs.fullname" . }}-all-in-one
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -24,6 +24,7 @@ spec:
selector:
matchLabels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: seaweedfs-all-in-one
{{- end }}
{{- end }}
@@ -2,7 +2,7 @@
apiVersion: cert-manager.io/v1{{ if .Values.global.certificates.alphacrds }}alpha1{{ end }}
kind: Certificate
metadata:
name: {{ template "seaweedfs.name" . }}-admin-cert
name: {{ include "seaweedfs.fullname" . }}-admin-cert
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -15,19 +15,19 @@ metadata:
{{- toYaml .Values.admin.annotations | nindent 4 }}
{{- end }}
spec:
secretName: {{ template "seaweedfs.name" . }}-admin-cert
secretName: {{ include "seaweedfs.fullname" . }}-admin-cert
issuerRef:
name: {{ template "seaweedfs.name" . }}-ca-issuer
name: {{ include "seaweedfs.fullname" . }}-ca-issuer
kind: Issuer
commonName: {{ .Values.certificates.commonName }}
subject:
organizations:
- "SeaweedFS CA"
dnsNames:
- '*.{{ template "seaweedfs.name" . }}-admin'
- '*.{{ template "seaweedfs.name" . }}-admin.{{ .Release.Namespace }}'
- '*.{{ template "seaweedfs.name" . }}-admin.{{ .Release.Namespace }}.svc'
- '*.{{ template "seaweedfs.name" . }}-admin.{{ .Release.Namespace }}.svc.cluster.local'
- '*.{{ include "seaweedfs.fullname" . }}-admin'
- '*.{{ include "seaweedfs.fullname" . }}-admin.{{ .Release.Namespace }}'
- '*.{{ include "seaweedfs.fullname" . }}-admin.{{ .Release.Namespace }}.svc'
- '*.{{ include "seaweedfs.fullname" . }}-admin.{{ .Release.Namespace }}.svc.cluster.local'
{{- if .Values.certificates.ipAddresses }}
ipAddresses:
{{- range .Values.certificates.ipAddresses }}
@@ -2,7 +2,7 @@
apiVersion: cert-manager.io/v1{{ if .Values.global.certificates.alphacrds }}alpha1{{ end }}
kind: Certificate
metadata:
name: {{ template "seaweedfs.name" . }}-ca-cert
name: {{ include "seaweedfs.fullname" . }}-ca-cert
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -10,8 +10,8 @@ metadata:
app.kubernetes.io/managed-by: {{ .Release.Service }}
app.kubernetes.io/instance: {{ .Release.Name }}
spec:
secretName: {{ template "seaweedfs.name" . }}-ca-cert
commonName: "{{ template "seaweedfs.name" . }}-root-ca"
secretName: {{ include "seaweedfs.fullname" . }}-ca-cert
commonName: "{{ include "seaweedfs.fullname" . }}-root-ca"
isCA: true
{{- if .Values.certificates.ca.duration }}
duration: {{ .Values.certificates.ca.duration }}
@@ -20,6 +20,6 @@ spec:
renewBefore: {{ .Values.certificates.ca.renewBefore }}
{{- end }}
issuerRef:
name: {{ template "seaweedfs.name" . }}-issuer
name: {{ include "seaweedfs.fullname" . }}-issuer
kind: Issuer
{{- end }}
@@ -2,7 +2,7 @@
apiVersion: cert-manager.io/v1{{ if .Values.global.certificates.alphacrds }}alpha1{{ end }}
kind: Issuer
metadata:
name: {{ template "seaweedfs.name" . }}-ca-issuer
name: {{ include "seaweedfs.fullname" . }}-ca-issuer
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -11,5 +11,5 @@ metadata:
app.kubernetes.io/instance: {{ .Release.Name }}
spec:
ca:
secretName: {{ template "seaweedfs.name" . }}-ca-cert
secretName: {{ include "seaweedfs.fullname" . }}-ca-cert
{{- end }}
@@ -2,7 +2,7 @@
apiVersion: cert-manager.io/v1{{ if .Values.global.certificates.alphacrds }}alpha1{{ end }}
kind: Issuer
metadata:
name: {{ template "seaweedfs.name" . }}-issuer
name: {{ include "seaweedfs.fullname" . }}-issuer
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
@@ -2,7 +2,7 @@
apiVersion: cert-manager.io/v1{{ if .Values.global.certificates.alphacrds }}alpha1{{ end }}
kind: Certificate
metadata:
name: {{ template "seaweedfs.name" . }}-client-cert
name: {{ include "seaweedfs.fullname" . }}-client-cert
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -10,9 +10,9 @@ metadata:
app.kubernetes.io/managed-by: {{ .Release.Service }}
app.kubernetes.io/instance: {{ .Release.Name }}
spec:
secretName: {{ template "seaweedfs.name" . }}-client-cert
secretName: {{ include "seaweedfs.fullname" . }}-client-cert
issuerRef:
name: {{ template "seaweedfs.name" . }}-ca-issuer
name: {{ include "seaweedfs.fullname" . }}-ca-issuer
kind: Issuer
commonName: {{ .Values.certificates.commonName }}
subject:
@@ -22,10 +22,10 @@ spec:
- '*.{{ .Release.Namespace }}'
- '*.{{ .Release.Namespace }}.svc'
- '*.{{ .Release.Namespace }}.svc.cluster.local'
- '*.{{ template "seaweedfs.name" . }}-master'
- '*.{{ template "seaweedfs.name" . }}-master.{{ .Release.Namespace }}'
- '*.{{ template "seaweedfs.name" . }}-master.{{ .Release.Namespace }}.svc'
- '*.{{ template "seaweedfs.name" . }}-master.{{ .Release.Namespace }}.svc.cluster.local'
- '*.{{ include "seaweedfs.fullname" . }}-master'
- '*.{{ include "seaweedfs.fullname" . }}-master.{{ .Release.Namespace }}'
- '*.{{ include "seaweedfs.fullname" . }}-master.{{ .Release.Namespace }}.svc'
- '*.{{ include "seaweedfs.fullname" . }}-master.{{ .Release.Namespace }}.svc.cluster.local'
{{- if .Values.certificates.ipAddresses }}
ipAddresses:
{{- range .Values.certificates.ipAddresses }}
@@ -2,7 +2,7 @@
apiVersion: cert-manager.io/v1{{ if .Values.global.certificates.alphacrds }}alpha1{{ end }}
kind: Certificate
metadata:
name: {{ template "seaweedfs.name" . }}-filer-cert
name: {{ include "seaweedfs.fullname" . }}-filer-cert
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -15,9 +15,9 @@ metadata:
{{- toYaml .Values.filer.annotations | nindent 4 }}
{{- end }}
spec:
secretName: {{ template "seaweedfs.name" . }}-filer-cert
secretName: {{ include "seaweedfs.fullname" . }}-filer-cert
issuerRef:
name: {{ template "seaweedfs.name" . }}-ca-issuer
name: {{ include "seaweedfs.fullname" . }}-ca-issuer
kind: Issuer
commonName: {{ .Values.certificates.commonName }}
subject:
@@ -27,10 +27,10 @@ spec:
- '*.{{ .Release.Namespace }}'
- '*.{{ .Release.Namespace }}.svc'
- '*.{{ .Release.Namespace }}.svc.cluster.local'
- '*.{{ template "seaweedfs.name" . }}-master'
- '*.{{ template "seaweedfs.name" . }}-master.{{ .Release.Namespace }}'
- '*.{{ template "seaweedfs.name" . }}-master.{{ .Release.Namespace }}.svc'
- '*.{{ template "seaweedfs.name" . }}-master.{{ .Release.Namespace }}.svc.cluster.local'
- '*.{{ include "seaweedfs.fullname" . }}-master'
- '*.{{ include "seaweedfs.fullname" . }}-master.{{ .Release.Namespace }}'
- '*.{{ include "seaweedfs.fullname" . }}-master.{{ .Release.Namespace }}.svc'
- '*.{{ include "seaweedfs.fullname" . }}-master.{{ .Release.Namespace }}.svc.cluster.local'
{{- if .Values.certificates.ipAddresses }}
ipAddresses:
{{- range .Values.certificates.ipAddresses }}
@@ -2,7 +2,7 @@
apiVersion: cert-manager.io/v1{{ if .Values.global.certificates.alphacrds }}alpha1{{ end }}
kind: Certificate
metadata:
name: {{ template "seaweedfs.name" . }}-master-cert
name: {{ include "seaweedfs.fullname" . }}-master-cert
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -15,9 +15,9 @@ metadata:
{{- toYaml .Values.master.annotations | nindent 4 }}
{{- end }}
spec:
secretName: {{ template "seaweedfs.name" . }}-master-cert
secretName: {{ include "seaweedfs.fullname" . }}-master-cert
issuerRef:
name: {{ template "seaweedfs.name" . }}-ca-issuer
name: {{ include "seaweedfs.fullname" . }}-ca-issuer
kind: Issuer
commonName: {{ .Values.certificates.commonName }}
subject:
@@ -27,10 +27,10 @@ spec:
- '*.{{ .Release.Namespace }}'
- '*.{{ .Release.Namespace }}.svc'
- '*.{{ .Release.Namespace }}.svc.cluster.local'
- '*.{{ template "seaweedfs.name" . }}-master'
- '*.{{ template "seaweedfs.name" . }}-master.{{ .Release.Namespace }}'
- '*.{{ template "seaweedfs.name" . }}-master.{{ .Release.Namespace }}.svc'
- '*.{{ template "seaweedfs.name" . }}-master.{{ .Release.Namespace }}.svc.cluster.local'
- '*.{{ include "seaweedfs.fullname" . }}-master'
- '*.{{ include "seaweedfs.fullname" . }}-master.{{ .Release.Namespace }}'
- '*.{{ include "seaweedfs.fullname" . }}-master.{{ .Release.Namespace }}.svc'
- '*.{{ include "seaweedfs.fullname" . }}-master.{{ .Release.Namespace }}.svc.cluster.local'
{{- if .Values.certificates.ipAddresses }}
ipAddresses:
{{- range .Values.certificates.ipAddresses }}
@@ -2,7 +2,7 @@
apiVersion: cert-manager.io/v1{{ if .Values.global.certificates.alphacrds }}alpha1{{ end }}
kind: Certificate
metadata:
name: {{ template "seaweedfs.name" . }}-volume-cert
name: {{ include "seaweedfs.fullname" . }}-volume-cert
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -15,9 +15,9 @@ metadata:
{{- toYaml .Values.volume.annotations | nindent 4 }}
{{- end }}
spec:
secretName: {{ template "seaweedfs.name" . }}-volume-cert
secretName: {{ include "seaweedfs.fullname" . }}-volume-cert
issuerRef:
name: {{ template "seaweedfs.name" . }}-ca-issuer
name: {{ include "seaweedfs.fullname" . }}-ca-issuer
kind: Issuer
commonName: {{ .Values.certificates.commonName }}
subject:
@@ -27,10 +27,10 @@ spec:
- '*.{{ .Release.Namespace }}'
- '*.{{ .Release.Namespace }}.svc'
- '*.{{ .Release.Namespace }}.svc.cluster.local'
- '*.{{ template "seaweedfs.name" . }}-master'
- '*.{{ template "seaweedfs.name" . }}-master.{{ .Release.Namespace }}'
- '*.{{ template "seaweedfs.name" . }}-master.{{ .Release.Namespace }}.svc'
- '*.{{ template "seaweedfs.name" . }}-master.{{ .Release.Namespace }}.svc.cluster.local'
- '*.{{ include "seaweedfs.fullname" . }}-master'
- '*.{{ include "seaweedfs.fullname" . }}-master.{{ .Release.Namespace }}'
- '*.{{ include "seaweedfs.fullname" . }}-master.{{ .Release.Namespace }}.svc'
- '*.{{ include "seaweedfs.fullname" . }}-master.{{ .Release.Namespace }}.svc.cluster.local'
{{- if .Values.certificates.ipAddresses }}
ipAddresses:
{{- range .Values.certificates.ipAddresses }}
@@ -2,7 +2,7 @@
apiVersion: cert-manager.io/v1{{ if .Values.global.certificates.alphacrds }}alpha1{{ end }}
kind: Certificate
metadata:
name: {{ template "seaweedfs.name" . }}-worker-cert
name: {{ include "seaweedfs.fullname" . }}-worker-cert
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -15,19 +15,19 @@ metadata:
{{- toYaml .Values.worker.annotations | nindent 4 }}
{{- end }}
spec:
secretName: {{ template "seaweedfs.name" . }}-worker-cert
secretName: {{ include "seaweedfs.fullname" . }}-worker-cert
issuerRef:
name: {{ template "seaweedfs.name" . }}-ca-issuer
name: {{ include "seaweedfs.fullname" . }}-ca-issuer
kind: Issuer
commonName: {{ .Values.certificates.commonName }}
subject:
organizations:
- "SeaweedFS CA"
dnsNames:
- '*.{{ template "seaweedfs.name" . }}-worker'
- '*.{{ template "seaweedfs.name" . }}-worker.{{ .Release.Namespace }}'
- '*.{{ template "seaweedfs.name" . }}-worker.{{ .Release.Namespace }}.svc'
- '*.{{ template "seaweedfs.name" . }}-worker.{{ .Release.Namespace }}.svc.cluster.local'
- '*.{{ include "seaweedfs.fullname" . }}-worker'
- '*.{{ include "seaweedfs.fullname" . }}-worker.{{ .Release.Namespace }}'
- '*.{{ include "seaweedfs.fullname" . }}-worker.{{ .Release.Namespace }}.svc'
- '*.{{ include "seaweedfs.fullname" . }}-worker.{{ .Release.Namespace }}.svc.cluster.local'
{{- if .Values.certificates.ipAddresses }}
ipAddresses:
{{- range .Values.certificates.ipAddresses }}
@@ -3,7 +3,7 @@
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: {{ .Values.global.serviceAccountName }}-objectstorage-provisioner
name: {{ include "seaweedfs.fullname" . }}-objectstorage-provisioner
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
@@ -52,7 +52,7 @@ rules:
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: {{ .Values.global.serviceAccountName }}-objectstorage-provisioner
name: {{ include "seaweedfs.fullname" . }}-objectstorage-provisioner
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
@@ -64,6 +64,6 @@ subjects:
namespace: {{ .Release.Namespace }}
roleRef:
kind: ClusterRole
name: {{ .Values.global.serviceAccountName }}-objectstorage-provisioner
name: {{ include "seaweedfs.fullname" . }}-objectstorage-provisioner
apiGroup: rbac.authorization.k8s.io
{{- end }}
@@ -2,7 +2,7 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ template "seaweedfs.name" . }}-objectstorage-provisioner
name: {{ include "seaweedfs.fullname" . }}-objectstorage-provisioner
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -57,7 +57,7 @@ spec:
priorityClassName: {{ .Values.cosi.priorityClassName | quote }}
{{- end }}
enableServiceLinks: false
serviceAccountName: {{ .Values.global.serviceAccountName }}-objectstorage-provisioner
serviceAccountName: {{ include "seaweedfs.componentName" (list . "objectstorage-provisioner") }}
{{- if .Values.cosi.initContainers }}
initContainers:
{{ tpl .Values.cosi.initContainers . | nindent 8 | trim }}
@@ -78,16 +78,16 @@ spec:
{{- else if .Values.s3.ingress.enabled }}
value: "{{ printf "https://%s" .Values.s3.ingress.host }}"
{{- else if .Values.s3.enabled }}
value: "{{ printf "https://%s-s3.%s.svc" (include "seaweedfs.name" .) .Release.Namespace }}"
value: "{{ printf "https://%s.%s.svc" (include "seaweedfs.componentName" (list . "s3")) .Release.Namespace }}"
{{- else }}
value: "{{ printf "https://%s-filer.%s.svc" (include "seaweedfs.name" .) .Release.Namespace }}"
value: "{{ printf "https://%s.%s.svc" (include "seaweedfs.componentName" (list . "filer")) .Release.Namespace }}"
{{- end }}
{{- with .Values.cosi.region }}
- name: REGION
value: "{{ . }}"
{{- end }}
- name: SEAWEEDFS_FILER
value: "{{ template "seaweedfs.name" . }}-filer:{{ .Values.filer.grpcPort }}"
value: "{{ include "seaweedfs.componentName" (list . "filer") }}:{{ .Values.filer.grpcPort }}"
{{- if .Values.global.enableSecurity }}
- name: WEED_GRPC_CLIENT_KEY
value: /usr/local/share/ca-certificates/client/tls.key
@@ -185,28 +185,28 @@ spec:
{{- if .Values.cosi.existingConfigSecret }}
secretName: {{ .Values.cosi.existingConfigSecret }}
{{- else }}
secretName: seaweedfs-s3-secret
secretName: {{ include "seaweedfs.fullname" . }}-s3-secret
{{- end }}
{{- end }}
{{- if .Values.global.enableSecurity }}
- name: security-config
configMap:
name: {{ template "seaweedfs.name" . }}-security-config
name: {{ include "seaweedfs.fullname" . }}-security-config
- name: ca-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-ca-cert
secretName: {{ include "seaweedfs.fullname" . }}-ca-cert
- name: master-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-master-cert
secretName: {{ include "seaweedfs.fullname" . }}-master-cert
- name: volume-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-volume-cert
secretName: {{ include "seaweedfs.fullname" . }}-volume-cert
- name: filer-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-filer-cert
secretName: {{ include "seaweedfs.fullname" . }}-filer-cert
- name: client-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-client-cert
secretName: {{ include "seaweedfs.fullname" . }}-client-cert
{{- end }}
{{ tpl .Values.cosi.extraVolumes . | indent 8 | trim }}
{{- if .Values.cosi.nodeSelector }}
@@ -2,7 +2,7 @@
{{- $filerEnabled := or .Values.filer.enabled .Values.allInOne.enabled }}
{{- if and $filerEnabled .Values.filer.ingress.enabled }}
{{- /* Determine service name based on deployment mode */}}
{{- $serviceName := ternary (printf "%s-all-in-one" (include "seaweedfs.name" .)) (printf "%s-filer" (include "seaweedfs.name" .)) .Values.allInOne.enabled }}
{{- $serviceName := ternary (include "seaweedfs.componentName" (list . "all-in-one")) (include "seaweedfs.componentName" (list . "filer")) .Values.allInOne.enabled }}
{{- if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion }}
apiVersion: networking.k8s.io/v1
{{- else if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion }}
@@ -12,7 +12,7 @@ apiVersion: extensions/v1beta1
{{- end }}
kind: Ingress
metadata:
name: ingress-{{ template "seaweedfs.name" . }}-filer
name: ingress-{{ include "seaweedfs.fullname" . }}-filer
namespace: {{ .Release.Namespace }}
{{- with .Values.filer.ingress.annotations }}
annotations:
@@ -2,7 +2,7 @@
apiVersion: v1
kind: Service
metadata:
name: {{ template "seaweedfs.name" . }}-filer-client
name: {{ printf "%s-filer-client" (include "seaweedfs.fullname" .) | trunc 63 | trimSuffix "-" }}
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -36,5 +36,6 @@ spec:
{{- end }}
selector:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: filer
{{- end }}
@@ -4,7 +4,7 @@ kind: Service
metadata:
annotations:
service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
name: {{ template "seaweedfs.name" . }}-filer
name: {{ printf "%s-filer" (include "seaweedfs.fullname" .) | trunc 63 | trimSuffix "-" }}
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -48,5 +48,6 @@ spec:
{{- end }}
selector:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: filer
{{- end }}
@@ -4,7 +4,7 @@
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: {{ template "seaweedfs.name" . }}-filer
name: {{ include "seaweedfs.fullname" . }}-filer
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -27,6 +27,7 @@ spec:
selector:
matchLabels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: filer
{{- end }}
{{- end }}
@@ -2,7 +2,7 @@
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: {{ template "seaweedfs.name" . }}-filer
name: {{ include "seaweedfs.componentName" (list . "filer") }}
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -15,7 +15,7 @@ metadata:
{{- toYaml .Values.filer.annotations | nindent 4 }}
{{- end }}
spec:
serviceName: {{ template "seaweedfs.name" . }}-filer
serviceName: {{ include "seaweedfs.componentName" (list . "filer") }}
podManagementPolicy: {{ .Values.filer.podManagementPolicy }}
replicas: {{ .Values.filer.replicas }}
{{- if (gt (int .Values.filer.updatePartition) 0) }}
@@ -70,7 +70,7 @@ spec:
{{ tpl .Values.filer.tolerations . | nindent 8 | trim }}
{{- end }}
{{- include "seaweedfs.imagePullSecrets" . | nindent 6 }}
serviceAccountName: {{ .Values.filer.serviceAccountName | default .Values.global.serviceAccountName | quote }} # for deleting statefulset pods after migration
serviceAccountName: {{ .Values.filer.serviceAccountName | default (include "seaweedfs.serviceAccountName" .) | quote }} # for deleting statefulset pods after migration
terminationGracePeriodSeconds: 60
{{- if .Values.filer.priorityClassName }}
priorityClassName: {{ .Values.filer.priorityClassName | quote }}
@@ -103,17 +103,17 @@ spec:
- name: WEED_MYSQL_USERNAME
valueFrom:
secretKeyRef:
name: secret-seaweedfs-db
name: {{ include "seaweedfs.fullname" . }}-db-secret
key: user
optional: true
- name: WEED_MYSQL_PASSWORD
valueFrom:
secretKeyRef:
name: secret-seaweedfs-db
name: {{ include "seaweedfs.fullname" . }}-db-secret
key: password
optional: true
- name: SEAWEEDFS_FULLNAME
value: "{{ template "seaweedfs.name" . }}"
value: "{{ include "seaweedfs.fullname" . }}"
{{- if .Values.filer.extraEnvironmentVars }}
{{- range $key, $value := .Values.filer.extraEnvironmentVars }}
- name: {{ $key }}
@@ -359,7 +359,7 @@ spec:
{{- end }}
- name: db-schema-config-volume
configMap:
name: seaweedfs-db-init-config
name: {{ default (printf "%s-db-init-config" (include "seaweedfs.fullname" .)) .Values.filer.dbInitConfigName }}
{{- if and .Values.filer.s3.enabled .Values.filer.s3.enableAuth }}
- name: config-users
secret:
@@ -367,33 +367,33 @@ spec:
{{- if .Values.filer.s3.existingConfigSecret }}
secretName: {{ .Values.filer.s3.existingConfigSecret }}
{{- else }}
secretName: seaweedfs-s3-secret
secretName: {{ include "seaweedfs.fullname" . }}-s3-secret
{{- end }}
{{- end }}
{{- if .Values.filer.notificationConfig }}
- name: notification-config
configMap:
name: {{ template "seaweedfs.name" . }}-notification-config
name: {{ include "seaweedfs.fullname" . }}-notification-config
{{- end }}
{{- if .Values.global.enableSecurity }}
- name: security-config
configMap:
name: {{ template "seaweedfs.name" . }}-security-config
name: {{ include "seaweedfs.fullname" . }}-security-config
- name: ca-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-ca-cert
secretName: {{ include "seaweedfs.fullname" . }}-ca-cert
- name: master-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-master-cert
secretName: {{ include "seaweedfs.fullname" . }}-master-cert
- name: volume-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-volume-cert
secretName: {{ include "seaweedfs.fullname" . }}-volume-cert
- name: filer-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-filer-cert
secretName: {{ include "seaweedfs.fullname" . }}-filer-cert
- name: client-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-client-cert
secretName: {{ include "seaweedfs.fullname" . }}-client-cert
{{- end }}
{{ tpl .Values.filer.extraVolumes . | indent 8 | trim }}
{{- if .Values.filer.nodeSelector }}
@@ -2,7 +2,7 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "seaweedfs.name" . }}-master-config
name: {{ include "seaweedfs.fullname" . }}-master-config
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -9,7 +9,7 @@ apiVersion: extensions/v1beta1
{{- end }}
kind: Ingress
metadata:
name: ingress-{{ template "seaweedfs.name" . }}-master
name: ingress-{{ include "seaweedfs.fullname" . }}-master
namespace: {{ .Release.Namespace }}
{{- with .Values.master.ingress.annotations }}
annotations:
@@ -33,15 +33,15 @@ spec:
backend:
{{- if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion }}
service:
name: {{ template "seaweedfs.name" . }}-master
name: {{ include "seaweedfs.componentName" (list . "master") }}
port:
number: {{ .Values.master.port }}
#name:
{{- else }}
serviceName: {{ template "seaweedfs.name" . }}-master
serviceName: {{ include "seaweedfs.componentName" (list . "master") }}
servicePort: {{ .Values.master.port }}
{{- end }}
{{- if .Values.filer.ingress.host }}
{{- if .Values.master.ingress.host }}
host: {{ .Values.master.ingress.host }}
{{- end }}
{{- end }}
@@ -2,13 +2,14 @@
apiVersion: v1
kind: Service
metadata:
name: {{ template "seaweedfs.name" . }}-master
name: {{ printf "%s-master" (include "seaweedfs.fullname" .) | trunc 63 | trimSuffix "-" }}
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
app.kubernetes.io/component: master
helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
app.kubernetes.io/instance: {{ .Release.Name }}
annotations:
service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
{{- if .Values.master.annotations }}
@@ -34,5 +35,6 @@ spec:
{{- end }}
selector:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: master
{{- end }}
@@ -4,7 +4,7 @@
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: {{ template "seaweedfs.name" . }}-master
name: {{ include "seaweedfs.fullname" . }}-master
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -27,6 +27,7 @@ spec:
selector:
matchLabels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: master
{{- end }}
{{- end }}
@@ -2,7 +2,7 @@
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: {{ template "seaweedfs.name" . }}-master
name: {{ include "seaweedfs.componentName" (list . "master") }}
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -15,7 +15,7 @@ metadata:
{{- toYaml .Values.master.annotations | nindent 4 }}
{{- end }}
spec:
serviceName: {{ template "seaweedfs.name" . }}-master
serviceName: {{ include "seaweedfs.componentName" (list . "master") }}
podManagementPolicy: {{ .Values.master.podManagementPolicy }}
replicas: {{ .Values.master.replicas }}
{{- if (gt (int .Values.master.updatePartition) 0) }}
@@ -70,7 +70,7 @@ spec:
{{- end }}
enableServiceLinks: false
{{- if .Values.global.createClusterRole }}
serviceAccountName: {{ .Values.master.serviceAccountName | default .Values.global.serviceAccountName | quote }} # for deleting statefulset pods after migration
serviceAccountName: {{ .Values.master.serviceAccountName | default (include "seaweedfs.serviceAccountName" .) | quote }} # for deleting statefulset pods after migration
{{- end }}
{{- if .Values.master.initContainers }}
initContainers:
@@ -97,7 +97,7 @@ spec:
fieldRef:
fieldPath: metadata.namespace
- name: SEAWEEDFS_FULLNAME
value: "{{ template "seaweedfs.name" . }}"
value: "{{ include "seaweedfs.fullname" . }}"
{{- if .Values.master.extraEnvironmentVars }}
{{- range $key, $value := .Values.master.extraEnvironmentVars }}
- name: {{ $key }}
@@ -183,7 +183,7 @@ spec:
{{- if .Values.master.garbageThreshold }}
-garbageThreshold={{ .Values.master.garbageThreshold }} \
{{- end }}
-ip=${POD_NAME}.${SEAWEEDFS_FULLNAME}-master.{{ .Release.Namespace }} \
-ip=${POD_NAME}.{{ include "seaweedfs.componentName" (list . "master") }}.{{ .Release.Namespace }} \
-peers={{ include "seaweedfs.masterServers" . }} \
{{- range .Values.master.extraArgs }}
{{ . }} \
@@ -297,26 +297,26 @@ spec:
{{- end }}
- name: master-config
configMap:
name: {{ template "seaweedfs.name" . }}-master-config
name: {{ include "seaweedfs.fullname" . }}-master-config
{{- if .Values.global.enableSecurity }}
- name: security-config
configMap:
name: {{ template "seaweedfs.name" . }}-security-config
name: {{ include "seaweedfs.fullname" . }}-security-config
- name: ca-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-ca-cert
secretName: {{ include "seaweedfs.fullname" . }}-ca-cert
- name: master-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-master-cert
secretName: {{ include "seaweedfs.fullname" . }}-master-cert
- name: volume-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-volume-cert
secretName: {{ include "seaweedfs.fullname" . }}-volume-cert
- name: filer-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-filer-cert
secretName: {{ include "seaweedfs.fullname" . }}-filer-cert
- name: client-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-client-cert
secretName: {{ include "seaweedfs.fullname" . }}-client-cert
{{- end }}
{{ tpl .Values.master.extraVolumes . | indent 8 | trim }}
{{- if .Values.master.nodeSelector }}
@@ -2,7 +2,7 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ template "seaweedfs.name" . }}-s3
name: {{ include "seaweedfs.fullname" . }}-s3
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -89,7 +89,7 @@ spec:
fieldRef:
fieldPath: metadata.namespace
- name: SEAWEEDFS_FULLNAME
value: "{{ template "seaweedfs.name" . }}"
value: "{{ include "seaweedfs.fullname" . }}"
{{- if .Values.s3.extraEnvironmentVars }}
{{- range $key, $value := .Values.s3.extraEnvironmentVars }}
- name: {{ $key }}
@@ -149,7 +149,10 @@ spec:
{{- if .Values.s3.auditLogConfig }}
-auditLogConfig=/etc/sw/s3_auditLogConfig.json \
{{- end }}
-filer={{ template "seaweedfs.name" . }}-filer-client.{{ .Release.Namespace }}:{{ .Values.filer.port }} \
-filer={{ include "seaweedfs.componentName" (list . "filer-client") }}.{{ .Release.Namespace }}:{{ .Values.filer.port }} \
{{- if .Values.s3.icebergPort }}
-port.iceberg={{ .Values.s3.icebergPort }} \
{{- end }}
{{- range .Values.s3.extraArgs }}
{{ . }} \
{{- end }}
@@ -192,6 +195,10 @@ spec:
- containerPort: {{ .Values.s3.httpsPort }}
name: swfs-s3-tls
{{- end }}
{{- if .Values.s3.icebergPort }}
- containerPort: {{ .Values.s3.icebergPort }}
name: swfs-iceberg
{{- end }}
{{- if .Values.s3.metricsPort }}
- containerPort: {{ .Values.s3.metricsPort }}
name: metrics
@@ -238,7 +245,7 @@ spec:
{{- if .Values.s3.existingConfigSecret }}
secretName: {{ .Values.s3.existingConfigSecret }}
{{- else }}
secretName: seaweedfs-s3-secret
secretName: {{ include "seaweedfs.fullname" . }}-s3-secret
{{- end }}
{{- end }}
{{- if eq .Values.s3.logs.type "hostPath" }}
@@ -254,22 +261,22 @@ spec:
{{- if .Values.global.enableSecurity }}
- name: security-config
configMap:
name: {{ template "seaweedfs.name" . }}-security-config
name: {{ include "seaweedfs.fullname" . }}-security-config
- name: ca-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-ca-cert
secretName: {{ include "seaweedfs.fullname" . }}-ca-cert
- name: master-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-master-cert
secretName: {{ include "seaweedfs.fullname" . }}-master-cert
- name: volume-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-volume-cert
secretName: {{ include "seaweedfs.fullname" . }}-volume-cert
- name: filer-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-filer-cert
secretName: {{ include "seaweedfs.fullname" . }}-filer-cert
- name: client-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-client-cert
secretName: {{ include "seaweedfs.fullname" . }}-client-cert
{{- end }}
{{ tpl .Values.s3.extraVolumes . | indent 8 | trim }}
{{- if .Values.s3.nodeSelector }}
@@ -0,0 +1,61 @@
{{- define "seaweedfs.s3.iceberg.ingress.paths" -}}
paths:
- path: {{ .Values.s3.icebergIngress.path | quote }}
pathType: {{ .Values.s3.icebergIngress.pathType | quote }}
backend:
{{- if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion }}
service:
name: {{ include "seaweedfs.componentName" (list . "s3") }}
port:
number: {{ .Values.s3.icebergPort }}
{{- else }}
serviceName: {{ include "seaweedfs.componentName" (list . "s3") }}
servicePort: {{ .Values.s3.icebergPort }}
{{- end }}
{{- end -}}
{{- if and .Values.s3.enabled .Values.s3.icebergPort .Values.s3.icebergIngress.enabled }}
{{- $hosts := list }}
{{- if kindIs "slice" .Values.s3.icebergIngress.host }}
{{- $hosts = .Values.s3.icebergIngress.host }}
{{- else if .Values.s3.icebergIngress.host }}
{{- $hosts = list .Values.s3.icebergIngress.host }}
{{- end }}
{{- if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion }}
apiVersion: networking.k8s.io/v1
{{- else if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion }}
apiVersion: networking.k8s.io/v1beta1
{{- else }}
apiVersion: extensions/v1beta1
{{- end }}
kind: Ingress
metadata:
name: ingress-{{ include "seaweedfs.fullname" . }}-s3-iceberg
namespace: {{ .Release.Namespace }}
{{- with .Values.s3.icebergIngress.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: s3-iceberg
spec:
{{- if .Values.s3.icebergIngress.className }}
ingressClassName: {{ .Values.s3.icebergIngress.className | quote }}
{{- end }}
tls:
{{ .Values.s3.icebergIngress.tls | default list | toYaml | nindent 6}}
rules:
{{- if $hosts }}
{{- range $host := $hosts }}
- host: {{ $host | quote }}
http:
{{- include "seaweedfs.s3.iceberg.ingress.paths" $ | nindent 6 }}
{{- end }}
{{- else }}
- http:
{{- include "seaweedfs.s3.iceberg.ingress.paths" . | nindent 4 }}
{{- end }}
{{- end }}
@@ -2,7 +2,7 @@
{{- $s3Enabled := or .Values.s3.enabled (and .Values.filer.s3.enabled (not .Values.allInOne.enabled)) (and .Values.allInOne.enabled .Values.allInOne.s3.enabled) }}
{{- if and $s3Enabled .Values.s3.ingress.enabled }}
{{- /* Determine service name based on deployment mode */}}
{{- $serviceName := ternary (printf "%s-all-in-one" (include "seaweedfs.name" .)) (printf "%s-s3" (include "seaweedfs.name" .)) .Values.allInOne.enabled }}
{{- $serviceName := ternary (include "seaweedfs.componentName" (list . "all-in-one")) (include "seaweedfs.componentName" (list . "s3")) .Values.allInOne.enabled }}
{{- $s3Port := .Values.allInOne.s3.port | default .Values.s3.port }}
{{- /* Build hosts list - support both legacy .host (string) and new .hosts (array) for backwards compatibility */}}
{{- $hosts := list }}
@@ -20,7 +20,7 @@ apiVersion: extensions/v1beta1
{{- end }}
kind: Ingress
metadata:
name: ingress-{{ template "seaweedfs.name" . }}-s3
name: ingress-{{ include "seaweedfs.fullname" . }}-s3
namespace: {{ .Release.Namespace }}
{{- with .Values.s3.ingress.annotations }}
annotations:
@@ -1,13 +1,35 @@
{{- if or (and (or .Values.s3.enabled .Values.allInOne.enabled) .Values.s3.enableAuth (not .Values.s3.existingConfigSecret)) (and .Values.filer.s3.enabled .Values.filer.s3.enableAuth (not .Values.filer.s3.existingConfigSecret)) }}
{{- $access_key_admin := include "getOrGeneratePassword" (dict "namespace" .Release.Namespace "secretName" "seaweedfs-s3-secret" "key" "admin_access_key_id" "length" 20) -}}
{{- $secret_key_admin := include "getOrGeneratePassword" (dict "namespace" .Release.Namespace "secretName" "seaweedfs-s3-secret" "key" "admin_secret_access_key" "length" 40) -}}
{{- $access_key_read := include "getOrGeneratePassword" (dict "namespace" .Release.Namespace "secretName" "seaweedfs-s3-secret" "key" "read_access_key_id" "length" 20) -}}
{{- $secret_key_read := include "getOrGeneratePassword" (dict "namespace" .Release.Namespace "secretName" "seaweedfs-s3-secret" "key" "read_secret_access_key" "length" 40) -}}
{{- $secretName := printf "%s-s3-secret" (include "seaweedfs.fullname" .) }}
{{- $legacySecretName := "seaweedfs-s3-secret" }}
{{- $lookupName := $secretName }}
{{- if .Values.s3.reuseLegacySecret }}
{{- $lookupName = default $legacySecretName .Values.s3.legacySecretName }}
{{- end }}
{{- $existingSecret := lookup "v1" "Secret" .Release.Namespace $lookupName }}
{{- $reuse := false }}
{{- if and .Values.s3.reuseLegacySecret $existingSecret }}
{{- $reuse = true }}
{{- end }}
{{- $creds := .Values.s3.credentials | default dict -}}
{{- $adminCreds := $creds.admin | default dict -}}
{{- $access_key_admin := $adminCreds.accessKey -}}
{{- $secret_key_admin := $adminCreds.secretKey -}}
{{- if not (and $access_key_admin $secret_key_admin) -}}
{{- $access_key_admin = include "getOrGeneratePassword" (dict "namespace" .Release.Namespace "secretName" $secretName "key" "admin_access_key_id" "length" 20 "existingSecret" (ternary $existingSecret nil $reuse)) -}}
{{- $secret_key_admin = include "getOrGeneratePassword" (dict "namespace" .Release.Namespace "secretName" $secretName "key" "admin_secret_access_key" "length" 40 "existingSecret" (ternary $existingSecret nil $reuse)) -}}
{{- end -}}
{{- $readCreds := $creds.read | default dict -}}
{{- $access_key_read := $readCreds.accessKey -}}
{{- $secret_key_read := $readCreds.secretKey -}}
{{- if not (and $access_key_read $secret_key_read) -}}
{{- $access_key_read = include "getOrGeneratePassword" (dict "namespace" .Release.Namespace "secretName" $secretName "key" "read_access_key_id" "length" 20 "existingSecret" (ternary $existingSecret nil $reuse)) -}}
{{- $secret_key_read = include "getOrGeneratePassword" (dict "namespace" .Release.Namespace "secretName" $secretName "key" "read_secret_access_key" "length" 40 "existingSecret" (ternary $existingSecret nil $reuse)) -}}
{{- end -}}
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: seaweedfs-s3-secret
name: {{ $secretName }}
namespace: {{ .Release.Namespace }}
annotations:
"helm.sh/resource-policy": keep
@@ -32,4 +54,4 @@ stringData:
s3_auditLogConfig.json: |
{{ toJson .Values.s3.auditLogConfig | nindent 4 }}
{{- end }}
{{- end }}
{{- end }}
@@ -2,24 +2,34 @@
apiVersion: v1
kind: Service
metadata:
name: {{ template "seaweedfs.name" . }}-s3
name: {{ printf "%s-s3" (include "seaweedfs.fullname" .) | trunc 63 | trimSuffix "-" }}
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
app.kubernetes.io/component: s3
helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- if .Values.s3.annotations }}
annotations:
{{- toYaml .Values.s3.annotations | nindent 4 }}
{{- end }}
spec:
internalTrafficPolicy: {{ .Values.s3.internalTrafficPolicy | default "Cluster" }}
{{- if and (semverCompare ">=1.31-0" .Capabilities.KubeVersion.GitVersion) (or .Values.s3.trafficDistribution .Values.filer.s3.trafficDistribution) }}
trafficDistribution: {{ include "seaweedfs.trafficDistribution" . }}
{{- end }}
ports:
- name: "swfs-s3"
port: {{ if .Values.s3.enabled }}{{ .Values.s3.port }}{{ else }}{{ .Values.filer.s3.port }}{{ end }}
targetPort: {{ if .Values.s3.enabled }}{{ .Values.s3.port }}{{ else }}{{ .Values.filer.s3.port }}{{ end }}
protocol: TCP
{{- if and .Values.s3.enabled .Values.s3.icebergPort }}
- name: "swfs-iceberg"
port: {{ .Values.s3.icebergPort }}
targetPort: {{ .Values.s3.icebergPort }}
protocol: TCP
{{- end }}
{{- if and .Values.s3.enabled .Values.s3.httpsPort }}
- name: "swfs-s3-tls"
port: {{ .Values.s3.httpsPort }}
@@ -34,5 +44,6 @@ spec:
{{- end }}
selector:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: {{ if .Values.s3.enabled }}s3{{ else }}filer{{ end }}
{{- end }}
@@ -4,7 +4,7 @@
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: {{ template "seaweedfs.name" . }}-s3
name: {{ include "seaweedfs.fullname" . }}-s3
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -27,6 +27,7 @@ spec:
selector:
matchLabels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: s3
{{- end }}
{{- end }}
@@ -2,7 +2,7 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ template "seaweedfs.name" . }}-sftp
name: {{ include "seaweedfs.fullname" . }}-sftp
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -89,7 +89,7 @@ spec:
fieldRef:
fieldPath: metadata.namespace
- name: SEAWEEDFS_FULLNAME
value: "{{ template "seaweedfs.name" . }}"
value: "{{ include "seaweedfs.fullname" . }}"
{{- if .Values.sftp.extraEnvironmentVars }}
{{- range $key, $value := .Values.sftp.extraEnvironmentVars }}
- name: {{ $key }}
@@ -171,7 +171,7 @@ spec:
-key.file=/usr/local/share/ca-certificates/client/tls.key \
{{- end }}
-userStoreFile=/etc/sw/seaweedfs_sftp_config \
-filer={{ template "seaweedfs.name" . }}-filer-client.{{ .Release.Namespace }}:{{ .Values.filer.port }}
-filer={{ include "seaweedfs.componentName" (list . "filer-client") }}.{{ .Release.Namespace }}:{{ .Values.filer.port }}
volumeMounts:
{{- if or (eq .Values.sftp.logs.type "hostPath") (eq .Values.sftp.logs.type "emptyDir") }}
- name: logs
@@ -252,7 +252,7 @@ spec:
{{- if .Values.sftp.existingConfigSecret }}
secretName: {{ .Values.sftp.existingConfigSecret }}
{{- else }}
secretName: seaweedfs-sftp-secret
secretName: {{ include "seaweedfs.fullname" . }}-sftp-secret
{{- end }}
{{- end }}
- name: config-ssh
@@ -261,7 +261,7 @@ spec:
{{- if .Values.sftp.existingSshConfigSecret }}
secretName: {{ .Values.sftp.existingSshConfigSecret }}
{{- else }}
secretName: seaweedfs-sftp-ssh-secret
secretName: {{ include "seaweedfs.fullname" . }}-sftp-ssh-secret
{{- end }}
{{- if eq .Values.sftp.logs.type "hostPath" }}
- name: logs
@@ -276,22 +276,22 @@ spec:
{{- if .Values.global.enableSecurity }}
- name: security-config
configMap:
name: {{ template "seaweedfs.name" . }}-security-config
name: {{ include "seaweedfs.fullname" . }}-security-config
- name: ca-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-ca-cert
secretName: {{ include "seaweedfs.fullname" . }}-ca-cert
- name: master-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-master-cert
secretName: {{ include "seaweedfs.fullname" . }}-master-cert
- name: volume-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-volume-cert
secretName: {{ include "seaweedfs.fullname" . }}-volume-cert
- name: filer-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-filer-cert
secretName: {{ include "seaweedfs.fullname" . }}-filer-cert
- name: client-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-client-cert
secretName: {{ include "seaweedfs.fullname" . }}-client-cert
{{- end }}
{{ tpl .Values.sftp.extraVolumes . | indent 8 | trim }}
{{- if .Values.sftp.nodeSelector }}
@@ -1,12 +1,13 @@
{{- if or .Values.sftp.enabled .Values.allInOne.enabled }}
{{- $admin_pwd := include "getOrGeneratePassword" (dict "namespace" .Release.Namespace "secretName" "seaweedfs-sftp-secret" "key" "admin_password" 20) -}}
{{- $read_user_pwd := include "getOrGeneratePassword" (dict "namespace" .Release.Namespace "secretName" "seaweedfs-sftp-secret" "key" "readonly_password" 20) -}}
{{- $public_user_pwd := include "getOrGeneratePassword" (dict "namespace" .Release.Namespace "secretName" "seaweedfs-sftp-secret" "key" "public_user_password" 20) -}}
{{- $secretName := printf "%s-sftp-secret" (include "seaweedfs.fullname" .) }}
{{- $admin_pwd := include "getOrGeneratePassword" (dict "namespace" .Release.Namespace "secretName" $secretName "key" "admin_password" 20) -}}
{{- $read_user_pwd := include "getOrGeneratePassword" (dict "namespace" .Release.Namespace "secretName" $secretName "key" "readonly_password" 20) -}}
{{- $public_user_pwd := include "getOrGeneratePassword" (dict "namespace" .Release.Namespace "secretName" $secretName "key" "public_user_password" 20) -}}
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: seaweedfs-sftp-secret
name: {{ $secretName }}
namespace: {{ .Release.Namespace }}
annotations:
"helm.sh/resource-policy": keep
@@ -2,10 +2,11 @@
apiVersion: v1
kind: Service
metadata:
name: {{ template "seaweedfs.name" . }}-sftp
name: {{ printf "%s-sftp" (include "seaweedfs.fullname" .) | trunc 63 | trimSuffix "-" }}
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: sftp
helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
@@ -28,5 +29,6 @@ spec:
{{- end }}
selector:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: sftp
{{- end }}
@@ -4,7 +4,7 @@
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: {{ template "seaweedfs.name" . }}-sftp
name: {{ include "seaweedfs.fullname" . }}-sftp
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -27,6 +27,7 @@ spec:
selector:
matchLabels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: sftp
{{- end }}
{{- end }}
@@ -17,6 +17,22 @@ be used as a full name.
{{- end -}}
{{- end -}}
{{/*
Create a truncated component name.
Usage: {{ include "seaweedfs.componentName" (list . "component-suffix") }}
*/}}
{{- define "seaweedfs.componentName" -}}
{{- $context := index . 0 -}}
{{- $suffix := index . 1 -}}
{{- if gt (len $suffix) 61 -}}
{{- fail (printf "Suffix '%s' is too long for componentName helper. Max length is 61." $suffix) -}}
{{- end -}}
{{- $fullname := include "seaweedfs.fullname" $context -}}
{{- $maxLen := sub 62 (len $suffix) | int -}}
{{- $truncatedFullname := trunc $maxLen $fullname | trimSuffix "-" -}}
{{- printf "%s-%s" $truncatedFullname $suffix -}}
{{- end -}}
{{/*
Create chart name and version as used by the chart label.
*/}}
@@ -119,7 +135,9 @@ Inject extra environment vars in the format key:value, if populated
{{- $repositoryName := default .Values.image.repository .Values.global.repository | toString -}}
{{- $name := .Values.global.imageName | toString -}}
{{- $tag := default .Chart.AppVersion .Values.image.tag | toString -}}
{{- if $repositoryName -}}
{{- if .Values.image.repository -}}
{{- $name = $repositoryName -}}
{{- else if $repositoryName -}}
{{- $name = printf "%s/%s" (trimSuffix "/" $repositoryName) (base $name) -}}
{{- end -}}
{{- if $registryName -}}
@@ -244,7 +262,7 @@ or generate a new random password if it doesn't exist.
{{- $key := $params.key -}}
{{- $length := default 16 $params.length -}}
{{- $existingSecret := lookup "v1" "Secret" $namespace $secretName -}}
{{- $existingSecret := default (lookup "v1" "Secret" $namespace $secretName) $params.existingSecret -}}
{{- if and $existingSecret (index $existingSecret.data $key) -}}
{{- index $existingSecret.data $key | b64dec -}}
{{- else -}}
@@ -261,7 +279,7 @@ If allInOne is enabled, point to the all-in-one service; otherwise, point to the
{{- if .Values.allInOne.enabled -}}
{{- $serviceNameSuffix = "-all-in-one" -}}
{{- end -}}
{{- printf "%s%s.%s:%d" (include "seaweedfs.name" .) $serviceNameSuffix .Release.Namespace (int .Values.master.port) -}}
{{- printf "%s.%s:%d" (printf "%s%s" (include "seaweedfs.fullname" .) $serviceNameSuffix | trunc 63 | trimSuffix "-") .Release.Namespace (int .Values.master.port) -}}
{{- end -}}
{{/*
@@ -273,19 +291,19 @@ If allInOne is enabled, point to the all-in-one service; otherwise, point to the
{{- if .Values.allInOne.enabled -}}
{{- $serviceNameSuffix = "-all-in-one" -}}
{{- end -}}
{{- printf "%s%s.%s:%d" (include "seaweedfs.name" .) $serviceNameSuffix .Release.Namespace (int .Values.filer.port) -}}
{{- printf "%s.%s:%d" (printf "%s%s" (include "seaweedfs.fullname" .) $serviceNameSuffix | trunc 63 | trimSuffix "-") .Release.Namespace (int .Values.filer.port) -}}
{{- end -}}
{{/*
Generate comma-separated list of master server addresses.
Usage: {{ include "seaweedfs.masterServers" . }}
Output example: ${SEAWEEDFS_FULLNAME}-master-0.${SEAWEEDFS_FULLNAME}-master.namespace:9333,${SEAWEEDFS_FULLNAME}-master-1...
Output example: my-release-master-0.my-release-master.namespace:9333,my-release-master-1...
*/}}
{{- define "seaweedfs.masterServers" -}}
{{- $fullname := include "seaweedfs.name" . -}}
{{- $masterName := include "seaweedfs.componentName" (list . "master") -}}
{{- range $index := until (.Values.master.replicas | int) -}}
{{- if $index }},{{ end -}}
${SEAWEEDFS_FULLNAME}-master-{{ $index }}.${SEAWEEDFS_FULLNAME}-master.{{ $.Release.Namespace }}:{{ $.Values.master.port }}
{{ $masterName }}-{{ $index }}.{{ $masterName }}.{{ $.Release.Namespace }}:{{ $.Values.master.port }}
{{- end -}}
{{- end -}}
@@ -300,3 +318,19 @@ Usage: {{ include "seaweedfs.masterServerArg" . }}
{{- include "seaweedfs.masterServers" . -}}
{{- end -}}
{{- end -}}
{{/*
Create the name of the service account to use
*/}}
{{- define "seaweedfs.serviceAccountName" -}}
{{- .Values.global.serviceAccountName | default "seaweedfs" -}}
{{- end -}}
{{/* Generate a compatible trafficDistribution value due to "PreferClose" fast deprecation in k8s v1.35 */}}
{{- define "seaweedfs.trafficDistribution" -}}
{{- if .Values.s3.trafficDistribution -}}
{{- and (eq .Values.s3.trafficDistribution "PreferClose") (semverCompare ">=1.35-0" .Capabilities.KubeVersion.GitVersion) | ternary "PreferSameZone" .Values.s3.trafficDistribution -}}
{{- else if .Values.filer.s3.trafficDistribution -}}
{{- and (eq .Values.filer.s3.trafficDistribution "PreferClose") (semverCompare ">=1.35-0" .Capabilities.KubeVersion.GitVersion) | ternary "PreferSameZone" .Values.filer.s3.trafficDistribution -}}
{{- end -}}
{{- end -}}
@@ -4,7 +4,7 @@
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: {{ .Values.global.serviceAccountName }}-rw-cr
name: {{ include "seaweedfs.fullname" . }}-rw-cr
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
@@ -18,7 +18,7 @@ rules:
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: system:serviceaccount:{{ .Values.global.serviceAccountName }}:default
name: {{ include "seaweedfs.fullname" . }}-rw-crb
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
@@ -26,10 +26,10 @@ metadata:
app.kubernetes.io/instance: {{ .Release.Name }}
subjects:
- kind: ServiceAccount
name: {{ .Values.global.serviceAccountName }}
name: {{ include "seaweedfs.serviceAccountName" . }}
namespace: {{ .Release.Namespace }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: {{ .Values.global.serviceAccountName }}-rw-cr
name: {{ include "seaweedfs.fullname" . }}-rw-cr
{{- end }}
@@ -2,7 +2,7 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "seaweedfs.name" . }}-notification-config
name: {{ include "seaweedfs.fullname" . }}-notification-config
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -15,14 +15,18 @@
{{- $existingConfigSecret = or .Values.allInOne.s3.existingConfigSecret .Values.s3.existingConfigSecret .Values.filer.s3.existingConfigSecret }}
{{- end }}
{{- else if .Values.master.enabled }}
{{- /* Check standalone filer.s3 mode */}}
{{- if .Values.filer.s3.enabled }}
{{- /* Check if embedded (in filer) or standalone S3 gateway is enabled */}}
{{- if or .Values.filer.s3.enabled .Values.s3.enabled }}
{{- $s3Enabled = true }}
{{- if .Values.filer.s3.createBuckets }}
{{- if .Values.s3.createBuckets }}
{{- $createBuckets = .Values.s3.createBuckets }}
{{- $enableAuth = .Values.s3.enableAuth }}
{{- $existingConfigSecret = .Values.s3.existingConfigSecret }}
{{- else if .Values.filer.s3.createBuckets }}
{{- $createBuckets = .Values.filer.s3.createBuckets }}
{{- $enableAuth = .Values.filer.s3.enableAuth }}
{{- $existingConfigSecret = .Values.filer.s3.existingConfigSecret }}
{{- end }}
{{- $enableAuth = .Values.filer.s3.enableAuth }}
{{- $existingConfigSecret = .Values.filer.s3.existingConfigSecret }}
{{- end }}
{{- end }}
@@ -36,7 +40,7 @@ metadata:
app.kubernetes.io/managed-by: {{ .Release.Service | quote }}
app.kubernetes.io/instance: {{ .Release.Name | quote }}
annotations:
"helm.sh/hook": post-install
"helm.sh/hook": post-install,post-upgrade
"helm.sh/hook-weight": "-5"
"helm.sh/hook-delete-policy": hook-succeeded
spec:
@@ -74,7 +78,7 @@ spec:
fieldRef:
fieldPath: metadata.namespace
- name: SEAWEEDFS_FULLNAME
value: "{{ template "seaweedfs.name" . }}"
value: "{{ include "seaweedfs.fullname" . }}"
command:
- "/bin/sh"
- "-ec"
@@ -105,9 +109,12 @@ spec:
wait_for_service "http://$WEED_CLUSTER_SW_FILER{{ .Values.filer.readinessProbe.httpGet.path }}"
{{- end }}
{{- range $createBuckets }}
/bin/echo \
"s3.bucket.create --name {{ .name }}" |\
/usr/bin/weed shell
if /bin/echo "s3.bucket.list" | /usr/bin/weed shell | awk '{print $1}' | grep -Fxq "{{ .name }}"; then
echo "Bucket '{{ .name }}' already exists, skipping creation."
else
echo "Creating bucket '{{ .name }}'..."
/bin/echo "s3.bucket.create --name {{ .name }}" | /usr/bin/weed shell
fi
{{- end }}
{{- range $createBuckets }}
{{- if .anonymousRead }}
@@ -145,7 +152,7 @@ spec:
{{- if $existingConfigSecret }}
secretName: {{ $existingConfigSecret }}
{{- else }}
secretName: {{ template "seaweedfs.name" . }}-s3-secret
secretName: {{ include "seaweedfs.fullname" . }}-s3-secret
{{- end }}
{{- end }}
{{- end }}
@@ -7,7 +7,7 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ printf "%s" $dashboardName | lower | replace "_" "-" }}
name: {{ include "seaweedfs.fullname" $ }}-{{ printf "%s" $dashboardName | lower | replace "_" "-" }}
namespace: {{ $.Release.Namespace }}
labels:
grafana_dashboard: "1"
@@ -3,7 +3,7 @@ apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: secret-seaweedfs-db
name: {{ include "seaweedfs.fullname" . }}-db-secret
namespace: {{ .Release.Namespace }}
annotations:
"helm.sh/resource-policy": keep
@@ -15,7 +15,6 @@ metadata:
app.kubernetes.io/instance: {{ .Release.Name }}
stringData:
user: "YourSWUser"
password: "HardCodedPassword"
# better to random generate and create in DB
# password: {{ randAlphaNum 10 | sha256sum | b64enc | trunc 32 }}
# auto-generated password
password: {{ randAlphaNum 10 | sha256sum | b64enc | trunc 32 }}
{{- end }}
@@ -2,7 +2,7 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "seaweedfs.name" . }}-security-config
name: {{ include "seaweedfs.fullname" . }}-security-config
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -10,7 +10,13 @@ metadata:
app.kubernetes.io/managed-by: {{ .Release.Service }}
app.kubernetes.io/instance: {{ .Release.Name }}
data:
{{- $existing := (lookup "v1" "ConfigMap" .Release.Namespace (printf "%s-security-config" (include "seaweedfs.name" .))) }}
{{- $fullname := include "seaweedfs.fullname" . }}
{{- $securityConfigName := printf "%s-security-config" $fullname }}
{{- $existing := lookup "v1" "ConfigMap" .Release.Namespace $securityConfigName }}
{{- if not $existing }}
{{- $legacyName := printf "%s-%s" (include "seaweedfs.name" .) "security-config" }}
{{- $existing = lookup "v1" "ConfigMap" .Release.Namespace $legacyName }}
{{- end }}
{{- $securityConfig := fromToml (dig "data" "security.toml" "" $existing) }}
security.toml: |-
# this file is read by master, volume server, and filer
@@ -42,7 +48,7 @@ data:
{{- if .Values.global.securityConfig.jwtSigning.filerRead }}
# If this JWT key is configured, Filer only accepts reads over HTTP if they are signed with this JWT:
# - f.e. the S3 API Shim generates the JWT
# - the Filer server validates the JWT on writing
# - the Filer server validates the JWT on reading
# the jwt defaults to expire after 10 seconds.
[jwt.filer_signing.read]
key = "{{ dig "jwt" "filer_signing" "read" "key" (randAlphaNum 10 | b64enc) $securityConfig }}"
@@ -1,7 +1,7 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ .Values.global.serviceAccountName }}
name: {{ include "seaweedfs.serviceAccountName" . }}
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -8,7 +8,7 @@ apiVersion: extensions/v1beta1
{{- end }}
kind: Ingress
metadata:
name: ingress-{{ template "seaweedfs.name" . }}-volume
name: ingress-{{ include "seaweedfs.fullname" . }}-volume
namespace: {{ .Release.Namespace }}
annotations:
{{- if and (not (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion)) .Values.volume.ingress.className }}
@@ -42,11 +42,11 @@ spec:
backend:
{{- if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion }}
service:
name: {{ template "seaweedfs.name" . }}-volume
name: {{ include "seaweedfs.componentName" (list . "volume") }}
port:
number: {{ .Values.volume.port }}
{{- else }}
serviceName: {{ template "seaweedfs.name" . }}-volume
serviceName: {{ include "seaweedfs.componentName" (list . "volume") }}
servicePort: {{ .Values.volume.port }}
{{- end }}
{{- end }}
@@ -1,4 +1,4 @@
{{- $seaweedfsName := include "seaweedfs.name" $ }}
{{- $seaweedfsName := include "seaweedfs.fullname" $ }}
{{- $volumes := deepCopy .Values.volumes | mergeOverwrite (dict "" .Values.volume) }}
@@ -8,10 +8,11 @@
apiVersion: v1
kind: Service
metadata:
name: {{ template "seaweedfs.name" $ }}-{{ $volumeName }}
name: {{ printf "%s-%s" (include "seaweedfs.fullname" $) $volumeName | trunc 63 | trimSuffix "-" }}
namespace: {{ $.Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" $ }}
app.kubernetes.io/instance: {{ $.Release.Name }}
app.kubernetes.io/component: {{ $volumeName }}
helm.sh/chart: {{ $.Chart.Name }}-{{ $.Chart.Version | replace "+" "_" }}
app.kubernetes.io/managed-by: {{ $.Release.Service }}
@@ -39,6 +40,7 @@ spec:
{{- end }}
selector:
app.kubernetes.io/name: {{ template "seaweedfs.name" $ }}
app.kubernetes.io/instance: {{ $.Release.Name }}
app.kubernetes.io/component: {{ $volumeName }}
{{- end }}
{{- end }}
@@ -10,7 +10,7 @@
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: {{ template "seaweedfs.name" $ }}-{{ $volumeName }}
name: {{ include "seaweedfs.fullname" $ }}-{{ $volumeName }}
namespace: {{ $.Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" $ }}
@@ -33,6 +33,7 @@ spec:
selector:
matchLabels:
app.kubernetes.io/name: {{ template "seaweedfs.name" $ }}
app.kubernetes.io/instance: {{ $.Release.Name }}
app.kubernetes.io/component: {{ $volumeName }}
{{- end }}
{{- end }}
@@ -8,7 +8,7 @@
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: {{ template "seaweedfs.name" $ }}-{{ $volumeName }}
name: {{ include "seaweedfs.componentName" (list $ $volumeName) }}
namespace: {{ $.Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" $ }}
@@ -21,7 +21,7 @@ metadata:
{{- toYaml $volume.annotations | nindent 4 }}
{{- end }}
spec:
serviceName: {{ template "seaweedfs.name" $ }}-{{ $volumeName }}
serviceName: {{ include "seaweedfs.componentName" (list $ $volumeName) }}
replicas: {{ $volume.replicas }}
podManagementPolicy: {{ $volume.podManagementPolicy }}
selector:
@@ -70,7 +70,7 @@ spec:
{{- end }}
enableServiceLinks: false
{{- if $.Values.global.createClusterRole }}
serviceAccountName: {{ $volume.serviceAccountName | default $.Values.global.serviceAccountName | quote }} # for deleting statefulset pods after migration
serviceAccountName: {{ $volume.serviceAccountName | default (include "seaweedfs.serviceAccountName" $) | quote }} # for deleting statefulset pods after migration
{{- end }}
{{- $initContainers_exists := include "volume.initContainers_exists" $ -}}
{{- if $initContainers_exists }}
@@ -117,7 +117,7 @@ spec:
fieldRef:
fieldPath: status.hostIP
- name: SEAWEEDFS_FULLNAME
value: "{{ template "seaweedfs.name" $ }}"
value: "{{ include "seaweedfs.fullname" $ }}"
{{- if $volume.extraEnvironmentVars }}
{{- range $key, $value := $volume.extraEnvironmentVars }}
- name: {{ $key }}
@@ -197,7 +197,7 @@ spec:
-fileSizeLimitMB={{ $volume.fileSizeLimitMB }} \
{{- end }}
-minFreeSpacePercent={{ $volume.minFreeSpacePercent }} \
-ip=${POD_NAME}.${SEAWEEDFS_FULLNAME}-{{ $volumeName }}.{{ $.Release.Namespace }} \
-ip=${POD_NAME}.{{ include "seaweedfs.componentName" (list $ $volumeName) }}.{{ $.Release.Namespace }} \
-compactionMBps={{ $volume.compactionMBps }} \
-master={{ include "seaweedfs.masterServerArg" $ }} \
{{- range $volume.extraArgs }}
@@ -343,22 +343,22 @@ spec:
{{- if $.Values.global.enableSecurity }}
- name: security-config
configMap:
name: {{ template "seaweedfs.name" $ }}-security-config
name: {{ include "seaweedfs.fullname" $ }}-security-config
- name: ca-cert
secret:
secretName: {{ template "seaweedfs.name" $ }}-ca-cert
secretName: {{ include "seaweedfs.fullname" $ }}-ca-cert
- name: master-cert
secret:
secretName: {{ template "seaweedfs.name" $ }}-master-cert
secretName: {{ include "seaweedfs.fullname" $ }}-master-cert
- name: volume-cert
secret:
secretName: {{ template "seaweedfs.name" $ }}-volume-cert
secretName: {{ include "seaweedfs.fullname" $ }}-volume-cert
- name: filer-cert
secret:
secretName: {{ template "seaweedfs.name" $ }}-filer-cert
secretName: {{ include "seaweedfs.fullname" $ }}-filer-cert
- name: client-cert
secret:
secretName: {{ template "seaweedfs.name" $ }}-client-cert
secretName: {{ include "seaweedfs.fullname" $ }}-client-cert
{{- end }}
{{- if $volume.extraVolumes }}
{{ tpl $volume.extraVolumes $ | indent 8 | trim }}
@@ -5,7 +5,7 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ template "seaweedfs.name" . }}-worker
name: {{ include "seaweedfs.fullname" . }}-worker
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -92,7 +92,7 @@ spec:
fieldRef:
fieldPath: metadata.namespace
- name: SEAWEEDFS_FULLNAME
value: "{{ template "seaweedfs.name" . }}"
value: "{{ include "seaweedfs.fullname" . }}"
{{- if .Values.worker.extraEnvironmentVars }}
{{- range $key, $value := .Values.worker.extraEnvironmentVars }}
- name: {{ $key }}
@@ -134,13 +134,17 @@ spec:
{{- if .Values.worker.adminServer }}
-admin={{ .Values.worker.adminServer }} \
{{- else }}
-admin={{ template "seaweedfs.name" . }}-admin.{{ .Release.Namespace }}:{{ .Values.admin.port }}{{ if .Values.admin.grpcPort }}.{{ .Values.admin.grpcPort }}{{ end }} \
-admin={{ template "seaweedfs.fullname" . }}-admin.{{ .Release.Namespace }}:{{ .Values.admin.port }}{{ if .Values.admin.grpcPort }}.{{ .Values.admin.grpcPort }}{{ end }} \
{{- end }}
-capabilities={{ .Values.worker.capabilities }} \
-maxConcurrent={{ .Values.worker.maxConcurrent }} \
-workingDir={{ .Values.worker.workingDir }}{{- if or .Values.worker.metricsPort .Values.worker.extraArgs }} \{{ end }}
-jobType={{ .Values.worker.jobType }} \
-maxDetect={{ .Values.worker.maxDetect }} \
-maxExecute={{ .Values.worker.maxExecute }} \
-workingDir={{ .Values.worker.workingDir }}{{- if or .Values.worker.metricsPort .Values.worker.metricsIp .Values.worker.extraArgs }} \{{ end }}
{{- if .Values.worker.metricsPort }}
-metricsPort={{ .Values.worker.metricsPort }}{{- if .Values.worker.extraArgs }} \{{ end }}
-metricsPort={{ .Values.worker.metricsPort }}{{- if or .Values.worker.metricsIp .Values.worker.extraArgs }} \{{ end }}
{{- end }}
{{- if .Values.worker.metricsIp }}
-metricsIp={{ .Values.worker.metricsIp }}{{- if .Values.worker.extraArgs }} \{{ end }}
{{- end }}
{{- range $index, $arg := .Values.worker.extraArgs }}
{{ $arg }}{{- if lt $index (sub (len $.Values.worker.extraArgs) 1) }} \{{ end }}
@@ -260,25 +264,25 @@ spec:
{{- if .Values.global.enableSecurity }}
- name: security-config
configMap:
name: {{ template "seaweedfs.name" . }}-security-config
name: {{ include "seaweedfs.fullname" . }}-security-config
- name: ca-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-ca-cert
secretName: {{ include "seaweedfs.fullname" . }}-ca-cert
- name: master-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-master-cert
secretName: {{ include "seaweedfs.fullname" . }}-master-cert
- name: volume-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-volume-cert
secretName: {{ include "seaweedfs.fullname" . }}-volume-cert
- name: filer-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-filer-cert
secretName: {{ include "seaweedfs.fullname" . }}-filer-cert
- name: client-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-client-cert
secretName: {{ include "seaweedfs.fullname" . }}-client-cert
- name: worker-cert
secret:
secretName: {{ template "seaweedfs.name" . }}-worker-cert
secretName: {{ include "seaweedfs.fullname" . }}-worker-cert
{{- end }}
{{ tpl .Values.worker.extraVolumes . | indent 8 | trim }}
{{- if .Values.worker.nodeSelector }}
@@ -2,7 +2,7 @@
apiVersion: v1
kind: Service
metadata:
name: {{ template "seaweedfs.name" . }}-worker
name: {{ printf "%s-worker" (include "seaweedfs.fullname" .) | trunc 63 | trimSuffix "-" }}
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -4,7 +4,7 @@
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: {{ template "seaweedfs.name" . }}-worker
name: {{ include "seaweedfs.fullname" . }}-worker
namespace: {{ .Release.Namespace }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
@@ -27,6 +27,7 @@ spec:
selector:
matchLabels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/component: worker
{{- end }}
{{- end }}
+47 -7
View File
@@ -890,8 +890,12 @@ filer:
# set to the name of an existing kubernetes Secret with the s3 json config file
# should have a secret key called seaweedfs_s3_config with an inline json configure
existingConfigSecret: null
# To provide explicit credentials for the S3 gateway, set them under
# the top-level s3.credentials key (not filer.s3.credentials).
# The s3-secret.yaml template only reads from .Values.s3.credentials.
# See: s3.credentials.admin.accessKey, s3.credentials.read.accessKey
auditLogConfig: {}
# You may specify buckets to be created during the install process.
# You may specify buckets to be created during the install or upgrade process.
# Buckets may be exposed publicly by setting `anonymousRead` to `true`
# createBuckets:
# - name: bucket-a
@@ -909,13 +913,33 @@ s3:
# add additional https port
httpsPort: 0
metricsPort: 9327
# Iceberg catalog REST port (Apache Iceberg REST Catalog API)
# Set to a port number to enable, or 0/null to disable
icebergPort: null
loggingOverrideLevel: null
# enable user & permission to s3 (need to inject to all services)
enableAuth: false
# set to the name of an existing kubernetes Secret with the s3 json config file
# should have a secret key called seaweedfs_s3_config with an inline json config
existingConfigSecret: null
# Optionally provide explicit credentials for the S3 gateway.
# When set, these are used in the generated s3 secret instead of
# auto-generating random credentials.
# credentials:
# admin:
# accessKey: ""
# secretKey: ""
# read:
# accessKey: ""
# secretKey: ""
auditLogConfig: {}
# You may specify buckets to be created during the install or upgrade process.
# Buckets may be exposed publicly by setting `anonymousRead` to `true`
# createBuckets:
# - name: bucket-a
# anonymousRead: true
# - name: bucket-b
# anonymousRead: false
# Suffix of the host name, {bucket}.{domainName}
domainName: ""
@@ -1043,6 +1067,15 @@ s3:
annotations: {}
tls: []
icebergIngress:
enabled: false
className: ""
host: "seaweedfs-iceberg.cluster.local"
path: "/"
pathType: Prefix
annotations: {}
tls: []
sftp:
enabled: false
imageOverride: null
@@ -1251,17 +1284,20 @@ worker:
replicas: 1
loggingOverrideLevel: null
metricsPort: 9327
metricsIp: "" # If empty, defaults to 0.0.0.0
# Admin server to connect to
adminServer: ""
# Worker capabilities - comma-separated list
# Available: vacuum, balance, erasure_coding
# Default: "vacuum,balance,erasure_coding" (all capabilities)
capabilities: "vacuum,balance,erasure_coding"
# Worker job types - comma-separated list
# Available: vacuum, volume_balance, erasure_coding
jobType: "vacuum,volume_balance,erasure_coding"
# Maximum number of concurrent tasks
maxConcurrent: 3
# Maximum number of concurrent detection requests
maxDetect: 1
# Maximum number of concurrent execution jobs
maxExecute: 4
# Working directory for task execution
workingDir: "/tmp/seaweedfs-worker"
@@ -1399,6 +1435,10 @@ allInOne:
# Set to the name of an existing kubernetes Secret with the s3 json config file
# should have a secret key called seaweedfs_s3_config with an inline json config
existingConfigSecret: null
# To provide explicit credentials for the S3 gateway, set them under
# the top-level s3.credentials key (not allInOne.s3.credentials).
# The s3-secret.yaml template only reads from .Values.s3.credentials.
# See: s3.credentials.admin.accessKey, s3.credentials.read.accessKey
auditLogConfig: null # S3 audit log configuration (null inherits from s3.auditLogConfig)
# You may specify buckets to be created during the install process.
# Buckets may be exposed publicly by setting `anonymousRead` to `true`
+1 -1
View File
@@ -68,7 +68,7 @@
<dependency>
<groupId>org.assertj</groupId>
<artifactId>assertj-core</artifactId>
<version>3.24.2</version>
<version>3.27.7</version>
<scope>test</scope>
</dependency>
</dependencies>
+2 -2
View File
@@ -211,9 +211,9 @@ checksum = "1fd0f2584146f6f2ef48085050886acf353beff7305ebd1ae69500e27c67f64b"
[[package]]
name = "bytes"
version = "1.10.1"
version = "1.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d71b6127be86fdcfddb610f7182ac57211d4b18a3e9c82eb2d17662f2227ad6a"
checksum = "1e748733b7cbc798e1434b6ac524f0c1ff2ab456fe201501e6497c8417a4fc33"
[[package]]
name = "cast"
@@ -41,7 +41,7 @@ chrono = { version = "0.4", features = ["serde"] }
# Memory management and utilities
memmap2 = "0.9"
bytes = "1.0"
bytes = "1.11"
parking_lot = "0.12" # Fast mutexes
# IPC and networking

Some files were not shown because too many files have changed in this diff Show More