11806 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
chrislu	50cc17e8fa	ActivateEcGeneration RPC implemented	2025-08-10 13:11:47 -07:00
chrislu	870b2ffe07	LookupEcVolume handle generation parameter	2025-08-10 13:06:37 -07:00
chrislu	e4f266d927	Active generation tracking implemented	2025-08-10 13:03:52 -07:00
chrislu	5650b6f505	Add master_pb.ActivateEcGenerationRequest/Response RPC definition	2025-08-10 12:53:10 -07:00
chrislu	09d740db47	Add generation field to volume_server_pb.VolumeEcShardReadRequest	2025-08-10 12:51:27 -07:00
chrislu	8c54b72af3	Add generation field to volume_server_pb VolumeEcShards{Copy,Generate,Rebuild,ToVolume}Request	2025-08-10 12:51:01 -07:00
chrislu	312fee4ddf	Add generation field to volume_server_pb VolumeEcShards{Mount,Unmount}Request	2025-08-10 12:49:43 -07:00
chrislu	03ad8f2490	Add generation field to master_pb.VolumeEcShardInformationMessage	2025-08-10 12:47:47 -07:00
chrislu	fc666e2e48	collect ec volume deleted bytes	2025-08-10 12:37:18 -07:00
chrislu	3a9ec6f30c	Update worker.go	2025-08-10 02:16:23 -07:00
chrislu	8603a3a1e9	ec vacuum task need to register itself	2025-08-10 02:16:15 -07:00
chrislu	bea9ca3616	generate ec data and deleted data	2025-08-10 02:15:55 -07:00
chrislu	32e7ba2f9d	adding ec vacuum	2025-08-10 00:58:26 -07:00
chrislu	05a0cc156b	Self-Contained Design ... To prove the system is truly self-contained: To add a new task: Create a task package (e.g., worker/tasks/compression/) Import it: _ "github.com/.../worker/tasks/compression" That's it! No other changes needed. To remove a task: Delete the task package directory Remove the import line That's it! No other changes needed.	2025-08-10 00:15:26 -07:00
chrislu	96d6d27607	remove ❌ Vacuum - Completely removed ❌ Balance - Completely removed	2025-08-10 00:00:46 -07:00
chrislu	c7f291e4e4	simplify logging	2025-08-09 23:49:45 -07:00
5 people	25bbf4c3d4	Admin UI: Fetch task logs (#7114) ... * show task details * loading tasks * task UI works * generic rendering * rendering the export link * removing placementConflicts from task parameters * remove TaskSourceLocation * remove "Server ID" column * rendering balance task source * sources and targets * fix ec task generation * move info * render timeline * simplified worker id * simplify * read task logs from worker * isValidTaskID * address comments * Update weed/worker/tasks/balance/execution.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/worker/tasks/erasure_coding/ec_task.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/worker/tasks/task_log_handler.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix shard ids * plan distributing shard id * rendering planned shards in task details * remove Conflicts * worker logs correctly * pass in dc and rack * task logging * Update weed/admin/maintenance/maintenance_queue.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * display log details * logs have fields now * sort field keys * fix link * fix collection filtering * avoid hard coded ec shard counts --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>	2025-08-09 21:47:29 -07:00
3 people	3ac2a2e22d	fix tikv listing due to expired entries (#7115) ... * fix tikv listing due to expired entries When there are many entries with empty fileName values (which can happen after TTL cleanup), the continue statements prevent the loop counter from incrementing, creating an infinite loop. * address comments * Update weed/filer/tikv/tikv_store.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * address comments Update weed/filer/tikv/tikv_store.go Co-Authored-By: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-08-08 12:38:55 -07:00
Devin LauderdaleandGitHub	fae416586b	Move helm templates into folders (#7113) ... * refactor: move helm templates into respective service folders * fix: update template path reference in filer-statefulset for s3-secret	2025-08-08 10:36:01 -07:00
Devin LauderdaleandGitHub	92cebe12f0	chore: remove default replica count for all-in-one deployment (#7111)	2025-08-07 21:18:17 -07:00
Chris LuandGitHub	535985adb6	Shell: add verbose ec encoding mode (#7105) ... * add verbose ec encoding mode * address comments	2025-08-07 00:12:05 -07:00
Chris LuandGitHub	cde2d65c16	ec candidate selection needs to adjust same rack count compare (#7106) ... ec needs to adjust same rack count compare	2025-08-07 00:09:51 -07:00
3 people	b4d9618efc	volume server UI: fix ec volume ui (#7104) ... * fix ec volume ui * Update weed/storage/erasure_coding/ec_volume.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-08-07 00:07:03 -07:00
chrislu	dd4880d55a	fix for baidu cloud storage	2025-08-06 20:53:05 -07:00
3 people	4af182f880	Context cancellation during reading range reading large files (#7093) ... * context cancellation during reading range reading large files * address comments * cancellation for fuse read * fix cancellation * pass in context for each function to avoid racing condition * Update reader_at_test.go * remove dead code * Update weed/filer/reader_at.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/filer/filechunk_group.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/filer/filechunk_group.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * address comments * Update weed/mount/weedfs_file_read.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/mount/weedfs_file_lseek.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/mount/weedfs_file_read.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/filer/reader_at.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/mount/weedfs_file_lseek.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * test cancellation --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-08-06 10:09:26 -07:00
3 people	e446234e9c	remove spoof-able request header (#7103) ... * remove spoof-able request header https://github.com/seaweedfs/seaweedfs/issues/7094#issuecomment-3158320497 * Update weed/security/guard.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-08-06 10:08:30 -07:00
8 people	0703308270	remote address parsing should handle special cases (#7101) ... * remote address parsing should handle special cases * handling ipv6 * simplify * Update weed/security/guard.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/security/guard.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * x-real-ip * Update guard.go * fixes Hostname Whitelisting: Fully restored - supports localhost, example.com, etc. IP Whitelisting: Still works - supports exact IPs and CIDR ranges Header Support: Consistent handling of X-Forwarded-For, X-Real-IP * simplify * Update weed/security/guard.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update weed/security/guard.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update guard.go * adjust function signature * Update weed/security/guard.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * indention * skip empty host --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>	2025-08-06 01:03:00 -07:00
4 people	c6d9756933	fix signature hashing for iam (#7100) ... * fix signature hashing for iam * add tests * address comments * Update weed/s3api/auto_signature_v4_test.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * indention * fix test --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>	2025-08-05 22:54:54 -07:00
3 people	b01b5e0f34	chore(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.29.18 to 1.30.2 (#7099) ... chore(deps): bump github.com/aws/aws-sdk-go-v2/config Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.29.18 to 1.30.2. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.29.18...v1.30.2) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/config dependency-version: 1.30.2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2025-08-05 22:06:58 -07:00
3 people	315e089d41	chore(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.71 to 1.18.2 (#7084) ... chore(deps): bump github.com/aws/aws-sdk-go-v2/credentials Bumps [github.com/aws/aws-sdk-go-v2/credentials](https://github.com/aws/aws-sdk-go-v2) from 1.17.71 to 1.18.2. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/config/v1.18.2/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/credentials/v1.17.71...config/v1.18.2) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/credentials dependency-version: 1.18.2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2025-08-05 21:22:07 -07:00
3 people	ec56e7e8d9	chore(deps): bump github.com/getsentry/sentry-go from 0.34.1 to 0.35.0 (#7098) ... Bumps [github.com/getsentry/sentry-go](https://github.com/getsentry/sentry-go) from 0.34.1 to 0.35.0. - [Release notes](https://github.com/getsentry/sentry-go/releases) - [Changelog](https://github.com/getsentry/sentry-go/blob/master/CHANGELOG.md) - [Commits](https://github.com/getsentry/sentry-go/compare/v0.34.1...v0.35.0) --- updated-dependencies: - dependency-name: github.com/getsentry/sentry-go dependency-version: 0.35.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2025-08-05 14:31:32 -07:00
3 people	bc19d144b5	chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.84.1 to 1.85.1 (#7083) ... chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.84.1 to 1.85.1. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.84.1...service/s3/v1.85.1) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/service/s3 dependency-version: 1.85.1 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2025-08-05 14:31:19 -07:00
3 people	69fb524e36	chore(deps): bump google.golang.org/api from 0.243.0 to 0.244.0 (#7096) ... Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.243.0 to 0.244.0. - [Release notes](https://github.com/googleapis/google-api-go-client/releases) - [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.243.0...v0.244.0) --- updated-dependencies: - dependency-name: google.golang.org/api dependency-version: 0.244.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2025-08-05 14:31:08 -07:00
chrislu	2d11e9bc1a	Pin helm-gh-pages action to stable version v1.7.0 ... - Use stable release instead of @master to avoid potential regressions - This should help resolve the 'gzip: invalid magic' error	2025-08-05 08:42:05 -07:00
chrislu	4f38b7c3b9	Revert "Fix helm-gh-pages action configuration" ... This reverts commit 26ef76b7b9.	2025-08-05 08:41:18 -07:00
chrislu	26ef76b7b9	Fix helm-gh-pages action configuration ... - Add missing commit_username and commit_email parameters - This should resolve the 'gzip: invalid magic' error in the Helm GitHub Pages action	2025-08-05 08:39:53 -07:00
Chris LuandGitHub	a834327755	context cancellation during reading range reading large files (#7092) ... * context cancellation during reading range reading large files * address comments	2025-08-04 23:22:11 -07:00
3 people	375dfe18a4	chore(deps): bump modernc.org/sqlite from 1.38.1 to 1.38.2 (#7091) ... Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.38.1 to 1.38.2. - [Commits](https://gitlab.com/cznic/sqlite/compare/v1.38.1...v1.38.2) --- updated-dependencies: - dependency-name: modernc.org/sqlite dependency-version: 1.38.2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2025-08-04 21:30:55 -07:00
3 people	ca9da29504	chore(deps): bump github.com/prometheus/client_golang from 1.22.0 to 1.23.0 (#7090) ... chore(deps): bump github.com/prometheus/client_golang Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.22.0 to 1.23.0. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md) - [Commits](https://github.com/prometheus/client_golang/compare/v1.22.0...v1.23.0) --- updated-dependencies: - dependency-name: github.com/prometheus/client_golang dependency-version: 1.23.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2025-08-04 21:30:48 -07:00
3 people	b376eccd22	chore(deps): bump github.com/golang-jwt/jwt/v5 from 5.2.3 to 5.3.0 (#7089) ... Bumps [github.com/golang-jwt/jwt/v5](https://github.com/golang-jwt/jwt) from 5.2.3 to 5.3.0. - [Release notes](https://github.com/golang-jwt/jwt/releases) - [Changelog](https://github.com/golang-jwt/jwt/blob/main/VERSION_HISTORY.md) - [Commits](https://github.com/golang-jwt/jwt/compare/v5.2.3...v5.3.0) --- updated-dependencies: - dependency-name: github.com/golang-jwt/jwt/v5 dependency-version: 5.3.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2025-08-04 21:30:39 -07:00
dependabot[bot]andGitHub	b1ce55291f	chore(deps): bump docker/metadata-action from 5.7.0 to 5.8.0 (#7082)	2025-08-04 19:15:32 -07:00
dependabot[bot]andGitHub	deb771817f	chore(deps): bump docker/login-action from 3.4.0 to 3.5.0 (#7085)	2025-08-04 19:15:07 -07:00
dependabot[bot]andGitHub	16a16d79ce	chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.113.2 to 3.113.4 (#7086)	2025-08-04 19:14:35 -07:00
dependabot[bot]andGitHub	20694a84f0	chore(deps): bump github.com/aws/aws-sdk-go from 1.55.7 to 1.55.8 (#7087)	2025-08-04 19:14:24 -07:00
dependabot[bot]andGitHub	fd568cd796	chore(deps): bump cloud.google.com/go/pubsub from 1.49.0 to 1.50.0 (#7088)	2025-08-04 19:14:03 -07:00
5 people	72176601c1	S3: Fix iam payload hash (#7081) ... * fix iam payload hash * streaming hash * Update weed/s3api/auto_signature_v4_test.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update weed/s3api/auto_signature_v4_test.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * address comments --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>	2025-08-04 09:10:01 -07:00
Chris LuandGitHub	365d03ff32	mount ec shards correctly (#7079)	2025-08-03 23:10:28 -07:00
4 people	513ac58504	Filer: fix filer range read (#7078) ... * fix filer range read Only return true if we're reading the ENTIRE chunk from the beginning. // This prevents bandwidth amplification when range requests happen to align // with chunk boundaries but don't actually want the full chunk. * Update weed/filer/filechunks.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>	2025-08-03 14:26:15 -07:00
Chris LuandGitHub	4fb7bbb215	Filer Store: postgres backend support pgbouncer (#7077) ... support pgbouncer	2025-08-03 11:56:04 -07:00
3 people	d49b44f2a4	Postgres (CockroachDB) with full certificate verification (#7076) ... * Postgres (CockroachDB) with full certificate verification * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * remove duplicated comments --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-08-03 09:43:33 -07:00