mirrors/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-05-23 18:21:28 +00:00
Code Issues Packages Projects Releases Wiki Activity
11,815 Commits 155 Branches 341 Tags
8797e73523762c1edca48bf6effd8024f76eefb6
Commit Graph

11815 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
chrislu
8797e73523 cachedLookupEcShardLocations updated for generation-specific caching 2025-08-10 14:51:55 -07:00
chrislu
f00dc46607 VolumeEcShardRead to read from correct (vid, generation) EcVolume 2025-08-10 14:44:27 -07:00
chrislu
1b41544f97 detecting ec volumes 2025-08-10 14:41:06 -07:00
chrislu
e9033136aa todo 2025-08-10 14:24:59 -07:00
chrislu
56d1ed77c1 copy file 2025-08-10 14:24:30 -07:00
chrislu
9e2e600b6d VolumeEcShardsGenerate updated for generation-specific file creation 2025-08-10 14:14:37 -07:00
chrislu
99f132729c MountEcShards/UnmountEcShards updated for generation support 2025-08-10 14:09:07 -07:00
chrislu
8c31d5e331 EcVolume creation properly refactored 2025-08-10 13:45:08 -07:00
chrislu
ef5f9f629a Generation file layout 2025-08-10 13:32:20 -07:00
chrislu
50cc17e8fa ActivateEcGeneration RPC implemented 2025-08-10 13:11:47 -07:00
chrislu
870b2ffe07 LookupEcVolume handle generation parameter 2025-08-10 13:06:37 -07:00
chrislu
e4f266d927 Active generation tracking implemented 2025-08-10 13:03:52 -07:00
chrislu
5650b6f505 Add master_pb.ActivateEcGenerationRequest/Response RPC definition 2025-08-10 12:53:10 -07:00
chrislu
09d740db47 Add generation field to volume_server_pb.VolumeEcShardReadRequest 2025-08-10 12:51:27 -07:00
chrislu
8c54b72af3 Add generation field to volume_server_pb VolumeEcShards{Copy,Generate,Rebuild,ToVolume}Request 2025-08-10 12:51:01 -07:00
chrislu
312fee4ddf Add generation field to volume_server_pb VolumeEcShards{Mount,Unmount}Request 2025-08-10 12:49:43 -07:00
chrislu
03ad8f2490 Add generation field to master_pb.VolumeEcShardInformationMessage 2025-08-10 12:47:47 -07:00
chrislu
fc666e2e48 collect ec volume deleted bytes 2025-08-10 12:37:18 -07:00
chrislu
3a9ec6f30c Update worker.go 2025-08-10 02:16:23 -07:00
chrislu
8603a3a1e9 ec vacuum task need to register itself 2025-08-10 02:16:15 -07:00
chrislu
bea9ca3616 generate ec data and deleted data 2025-08-10 02:15:55 -07:00
chrislu
32e7ba2f9d adding ec vacuum 2025-08-10 00:58:26 -07:00
chrislu
05a0cc156b Self-Contained Design 
To prove the system is truly self-contained:
To add a new task:
Create a task package (e.g., worker/tasks/compression/)
Import it: _ "github.com/.../worker/tasks/compression"
That's it! No other changes needed.
To remove a task:
Delete the task package directory
Remove the import line
That's it! No other changes needed.
 2025-08-10 00:15:26 -07:00
chrislu
96d6d27607 remove ❌ Vacuum - Completely removed ❌ Balance - Completely removed 2025-08-10 00:00:46 -07:00
chrislu
c7f291e4e4 simplify logging 2025-08-09 23:49:45 -07:00
Chris Lu
25bbf4c3d4 Admin UI: Fetch task logs (#7114) 
* show task details

* loading tasks

* task UI works

* generic rendering

* rendering the export link

* removing placementConflicts from task parameters

* remove TaskSourceLocation

* remove "Server ID" column

* rendering balance task source

* sources and targets

* fix ec task generation

* move info

* render timeline

* simplified worker id

* simplify

* read task logs from worker

* isValidTaskID

* address comments

* Update weed/worker/tasks/balance/execution.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/worker/tasks/erasure_coding/ec_task.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/worker/tasks/task_log_handler.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix shard ids

* plan distributing shard id

* rendering planned shards in task details

* remove Conflicts

* worker logs correctly

* pass in dc and rack

* task logging

* Update weed/admin/maintenance/maintenance_queue.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* display log details

* logs have fields now

* sort field keys

* fix link

* fix collection filtering

* avoid hard coded ec shard counts

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-08-09 21:47:29 -07:00
Chris Lu
3ac2a2e22d fix tikv listing due to expired entries (#7115) 
* fix tikv listing due to expired entries

When there are many entries with empty fileName values (which can happen after TTL cleanup), the continue statements prevent the loop counter from incrementing, creating an infinite loop.

* address comments

* Update weed/filer/tikv/tikv_store.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* address comments

Update weed/filer/tikv/tikv_store.go

Co-Authored-By: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-08-08 12:38:55 -07:00
Devin Lauderdale
fae416586b Move helm templates into folders (#7113) 
* refactor: move helm templates into respective service folders

* fix: update template path reference in filer-statefulset for s3-secret
 2025-08-08 10:36:01 -07:00
Devin Lauderdale
92cebe12f0 chore: remove default replica count for all-in-one deployment (#7111) 2025-08-07 21:18:17 -07:00
Chris Lu
535985adb6 Shell: add verbose ec encoding mode (#7105) 
* add verbose ec encoding mode

* address comments
 2025-08-07 00:12:05 -07:00
Chris Lu
cde2d65c16 ec candidate selection needs to adjust same rack count compare (#7106) 
ec needs to adjust same rack count compare
 2025-08-07 00:09:51 -07:00
Chris Lu
b4d9618efc volume server UI: fix ec volume ui (#7104) 
* fix ec volume ui

* Update weed/storage/erasure_coding/ec_volume.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-08-07 00:07:03 -07:00
chrislu
dd4880d55a fix for baidu cloud storage 2025-08-06 20:53:05 -07:00
Chris Lu
4af182f880 Context cancellation during reading range reading large files (#7093) 
* context cancellation during reading range reading large files

* address comments

* cancellation for fuse read

* fix cancellation

* pass in context for each function to avoid racing condition

* Update reader_at_test.go

* remove dead code

* Update weed/filer/reader_at.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/filer/filechunk_group.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/filer/filechunk_group.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* address comments

* Update weed/mount/weedfs_file_read.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/mount/weedfs_file_lseek.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/mount/weedfs_file_read.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/filer/reader_at.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/mount/weedfs_file_lseek.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* test cancellation

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-08-06 10:09:26 -07:00
Chris Lu
e446234e9c remove spoof-able request header (#7103) 
* remove spoof-able request header

https://github.com/seaweedfs/seaweedfs/issues/7094#issuecomment-3158320497

* Update weed/security/guard.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-08-06 10:08:30 -07:00
Chris Lu
0703308270 remote address parsing should handle special cases (#7101) 
* remote address parsing should handle special cases

* handling ipv6

* simplify

* Update weed/security/guard.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/security/guard.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* x-real-ip

* Update guard.go

* fixes

 Hostname Whitelisting: Fully restored - supports localhost, example.com, etc.
 IP Whitelisting: Still works - supports exact IPs and CIDR ranges
 Header Support: Consistent handling of X-Forwarded-For, X-Real-IP

* simplify

* Update weed/security/guard.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update weed/security/guard.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update guard.go

* adjust function signature

* Update weed/security/guard.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* indention

* skip empty host

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-08-06 01:03:00 -07:00
Chris Lu
c6d9756933 fix signature hashing for iam (#7100) 
* fix signature hashing for iam

* add tests

* address comments

* Update weed/s3api/auto_signature_v4_test.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* indention

* fix test

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-08-05 22:54:54 -07:00
dependabot[bot]
b01b5e0f34 chore(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.29.18 to 1.30.2 (#7099) 
chore(deps): bump github.com/aws/aws-sdk-go-v2/config

Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.29.18 to 1.30.2.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.29.18...v1.30.2)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-version: 1.30.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-05 22:06:58 -07:00
dependabot[bot]
315e089d41 chore(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.71 to 1.18.2 (#7084) 
chore(deps): bump github.com/aws/aws-sdk-go-v2/credentials

Bumps [github.com/aws/aws-sdk-go-v2/credentials](https://github.com/aws/aws-sdk-go-v2) from 1.17.71 to 1.18.2.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/config/v1.18.2/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/credentials/v1.17.71...config/v1.18.2)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/credentials
  dependency-version: 1.18.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-05 21:22:07 -07:00
dependabot[bot]
ec56e7e8d9 chore(deps): bump github.com/getsentry/sentry-go from 0.34.1 to 0.35.0 (#7098) 
Bumps [github.com/getsentry/sentry-go](https://github.com/getsentry/sentry-go) from 0.34.1 to 0.35.0.
- [Release notes](https://github.com/getsentry/sentry-go/releases)
- [Changelog](https://github.com/getsentry/sentry-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/getsentry/sentry-go/compare/v0.34.1...v0.35.0)

---
updated-dependencies:
- dependency-name: github.com/getsentry/sentry-go
  dependency-version: 0.35.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-05 14:31:32 -07:00
dependabot[bot]
bc19d144b5 chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.84.1 to 1.85.1 (#7083) 
chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3

Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.84.1 to 1.85.1.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.84.1...service/s3/v1.85.1)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/s3
  dependency-version: 1.85.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-05 14:31:19 -07:00
dependabot[bot]
69fb524e36 chore(deps): bump google.golang.org/api from 0.243.0 to 0.244.0 (#7096) 
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.243.0 to 0.244.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.243.0...v0.244.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-version: 0.244.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-05 14:31:08 -07:00
chrislu
2d11e9bc1a Pin helm-gh-pages action to stable version v1.7.0 
- Use stable release instead of @master to avoid potential regressions
- This should help resolve the 'gzip: invalid magic' error
 2025-08-05 08:42:05 -07:00
chrislu
4f38b7c3b9 Revert "Fix helm-gh-pages action configuration" 
This reverts commit 26ef76b7b9.
 2025-08-05 08:41:18 -07:00
chrislu
26ef76b7b9 Fix helm-gh-pages action configuration 
- Add missing commit_username and commit_email parameters
- This should resolve the 'gzip: invalid magic' error in the Helm GitHub Pages action
 2025-08-05 08:39:53 -07:00
Chris Lu
a834327755 context cancellation during reading range reading large files (#7092) 
* context cancellation during reading range reading large files

* address comments
 2025-08-04 23:22:11 -07:00
dependabot[bot]
375dfe18a4 chore(deps): bump modernc.org/sqlite from 1.38.1 to 1.38.2 (#7091) 
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.38.1 to 1.38.2.
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.38.1...v1.38.2)

---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-version: 1.38.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-04 21:30:55 -07:00
dependabot[bot]
ca9da29504 chore(deps): bump github.com/prometheus/client_golang from 1.22.0 to 1.23.0 (#7090) 
chore(deps): bump github.com/prometheus/client_golang

Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.22.0 to 1.23.0.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.22.0...v1.23.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-version: 1.23.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-04 21:30:48 -07:00
dependabot[bot]
b376eccd22 chore(deps): bump github.com/golang-jwt/jwt/v5 from 5.2.3 to 5.3.0 (#7089) 
Bumps [github.com/golang-jwt/jwt/v5](https://github.com/golang-jwt/jwt) from 5.2.3 to 5.3.0.
- [Release notes](https://github.com/golang-jwt/jwt/releases)
- [Changelog](https://github.com/golang-jwt/jwt/blob/main/VERSION_HISTORY.md)
- [Commits](https://github.com/golang-jwt/jwt/compare/v5.2.3...v5.3.0)

---
updated-dependencies:
- dependency-name: github.com/golang-jwt/jwt/v5
  dependency-version: 5.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-04 21:30:39 -07:00
dependabot[bot]
b1ce55291f chore(deps): bump docker/metadata-action from 5.7.0 to 5.8.0 (#7082) 2025-08-04 19:15:32 -07:00