mirrors/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-09-08 00:57:16 +00:00
Code Issues Packages Projects Releases Wiki Activity
11,887 Commits 177 Branches 358 Tags
c8c758e63991bed67e04721ee1e40472afc3c4e7
Commit Graph
11887 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
chrislu c8c758e639 fix hanging task detail page 2025-08-11 23:55:24 -07:00
chrislu 56db0a081a distribute .ecx files 2025-08-11 23:47:42 -07:00
chrislu 0d6ea416a7 fix retry link 2025-08-11 23:37:17 -07:00
chrislu 4f9f9ec971 fix routing 2025-08-11 22:36:48 -07:00
chrislu 3bbe4481f2 remove unused code 2025-08-11 21:54:21 -07:00
chrislu cac6a51cbf implement todos 2025-08-11 21:48:45 -07:00
chrislu 7780a7d652 ec vacuum workflow is correct now 2025-08-11 21:41:18 -07:00
chrislu 83b6a94bfe change to correct ec vacuum workflow 2025-08-11 21:14:36 -07:00
chrislu 0f1c734626 fix mounting ec volumes 2025-08-11 20:34:02 -07:00
chrislu 2a61cda2cd adding retry button 2025-08-11 18:28:02 -07:00
chrislu 5c93557314 detect shard sources 2025-08-11 17:36:27 -07:00
chrislu 7cf5ddf8ac more self contained tasks 2025-08-11 17:16:16 -07:00
chrislu 7622d14580 remove unused config 2025-08-11 17:08:48 -07:00
chrislu bddb2cd5cc keep only ec config 2025-08-11 17:02:15 -07:00
chrislu 96a2749355 adding EcVacuumTaskConfig 2025-08-11 16:59:55 -07:00
chrislu f9e7ca4308 Merge branch 'master' into add-ec-vacuum 2025-08-11 16:45:45 -07:00
chrislu 7889e78f4d use one http client 2025-08-11 16:45:13 -07:00
Chris LuandGitHub af3300e063 filer: server side copying (#7121) 
* copy

* address comments

* remove unused functions, reuse http clients

* address hardlink, checking existing directory

* destination is directory

* check for the key's existence in the map first before accessing its members

* address comments

* deep copy remote entry

* address comments

* copying chunks in parallel

* handle manifest chunks

* address comments

* errgroup

* there could be large chunks

* address comments

* address comments
 2025-08-11 16:40:46 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 		7e86045e22 chore(deps): bump golang.org/x/net from 0.42.0 to 0.43.0 (#7126) 
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.42.0 to 0.43.0.
- [Commits](https://github.com/golang/net/compare/v0.42.0...v0.43.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-version: 0.43.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-11 15:21:46 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 		529c68a798 chore(deps): bump google.golang.org/api from 0.244.0 to 0.246.0 (#7127) 
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.244.0 to 0.246.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.244.0...v0.246.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-version: 0.246.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-11 15:06:25 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 		e3b15073a0 chore(deps): bump actions/checkout from 4 to 5 (#7125) 
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 5.
- [Release notes](https://github.com/actions/checkout/releases)
- [Commits](https://github.com/actions/checkout/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-11 15:06:14 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 		5f62b11889 chore(deps): bump golang.org/x/sys from 0.34.0 to 0.35.0 (#7124) 
Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.34.0 to 0.35.0.
- [Commits](https://github.com/golang/sys/compare/v0.34.0...v0.35.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-version: 0.35.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-11 15:06:01 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 		524c1916b6 chore(deps): bump github.com/redis/go-redis/v9 from 9.11.0 to 9.12.0 (#7123) 
Bumps [github.com/redis/go-redis/v9](https://github.com/redis/go-redis) from 9.11.0 to 9.12.0.
- [Release notes](https://github.com/redis/go-redis/releases)
- [Changelog](https://github.com/redis/go-redis/blob/master/RELEASE-NOTES.md)
- [Commits](https://github.com/redis/go-redis/compare/v9.11.0...v9.12.0)

---
updated-dependencies:
- dependency-name: github.com/redis/go-redis/v9
  dependency-version: 9.12.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-11 15:05:53 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 		412fd6ee86 chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.113.4 to 3.113.5 (#7122) 
chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3

Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk) from 3.113.4 to 3.113.5.
- [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases)
- [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.113.4...v3.113.5)

---
updated-dependencies:
- dependency-name: github.com/ydb-platform/ydb-go-sdk/v3
  dependency-version: 3.113.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-11 15:05:44 -07:00
Chris LuandGitHub 605b3333c1 fix sftp start with filer (#7120) 
* fix sftp start with filer

* add bindIp
 2025-08-11 10:56:40 -07:00
Chris LuandGitHub 9265e81fe9 S3 API: unsigned streaming (no cred) but chunks contain signatures (#7118) 
* This handles the case where we have unsigned streaming (no cred) but chunks contain signatures

* Update chunked_reader_v4.go

* address comments
 2025-08-11 10:31:01 -07:00
chrislu a265a07922 fix ttl objects 
fix https://github.com/seaweedfs/seaweedfs/discussions/7107#discussioncomment-14069018
 2025-08-11 02:20:02 -07:00
chrislu f03f4383af fix missing grpc dial option 2025-08-11 02:19:22 -07:00
chrislu c57ccd554a worker_pb.TaskParams_VacuumParams 2025-08-11 02:11:39 -07:00
chrislu 9df006b49d fix min volume age 2025-08-11 01:54:10 -07:00
chrislu 424bb7fe11 showing criteria 2025-08-11 01:34:33 -07:00
chrislu 97d58e77c6 getting ec volume deletions 2025-08-11 01:03:21 -07:00
chrislu 553a229fd3 reduce lock scope 2025-08-11 00:21:04 -07:00
chrislu 57d025910d cancel context 2025-08-11 00:20:52 -07:00
chrislu 213dd9b470 testing 2025-08-11 00:20:37 -07:00
chrislu a31fc1ac01 Bypass compaction revision check to handle volumes compacted after task creation 2025-08-10 20:39:48 -07:00
chrislu 91d641e685 avoid dead lock 2025-08-10 20:23:18 -07:00
chrislu 740983d69e Delete ec_test_files.json 2025-08-10 20:15:36 -07:00
chrislu 0e649f710a collect deletion for ec shards 2025-08-10 20:08:45 -07:00
chrislu 772ee0f967 address comments 2025-08-10 19:58:52 -07:00
chrislu 0c6980182c ec shards with generation 2025-08-10 19:40:35 -07:00
chrislu 47ea1ac228 unmount 2025-08-10 19:39:24 -07:00
chrislu 6446893e3c metrics 2025-08-10 19:39:11 -07:00
chrislu 06c012ea60 Update maintenance_scanner.go 2025-08-10 19:37:54 -07:00
chrislu 5a6954be1b sort 2025-08-10 18:42:27 -07:00
chrislu 0f1ca16457 more accurate estimation 2025-08-10 18:42:22 -07:00
chrislu 72f0a47563 CRITICAL: Check ALL task states for volume conflicts 
Fix major scheduling bug where only active tasks were checked for conflicts.

Changes:
- Check PENDING tasks: Prevent scheduling if task is queued for same volume
- Check ASSIGNED/ACTIVE tasks: Prevent scheduling if task is running on same volume
- Check RECENT tasks: Prevent immediate re-scheduling on same volume after completion

This prevents dangerous scenarios like:
❌ Scheduling vacuum while another vacuum is pending on same volume
❌ Scheduling balance while erasure coding is queued for same volume
❌ Immediately re-scheduling failed tasks without cooldown period

Critical safety improvement ensuring comprehensive volume-level task isolation.
 2025-08-10 18:04:31 -07:00
chrislu 751cfac7d7 Implement volume-aware task conflict checking 
MAJOR IMPROVEMENT: Tasks now conflict by volume ID, not globally by task type

Changes:
- PRIMARY RULE: Tasks on the same volume ID always conflict (prevents race conditions)
- SECONDARY RULE: Minimal global task type conflicts (currently none)
- Add isDiskAvailableForVolume() for volume-specific availability checking
- Add GetAvailableDisksForVolume() and GetDisksWithEffectiveCapacityForVolume()
- Remove overly restrictive global task type conflicts
- Update planning functions to focus on capacity, not conflicts

Benefits:
✅ Multiple vacuum tasks can run on different volumes simultaneously
✅ Balance and erasure coding can run on different volumes
✅ Still prevents dangerous concurrent operations on same volume
✅ Much more efficient resource utilization
✅ Maintains data integrity and prevents race conditions

This addresses the user feedback that task conflicts should be volume-specific,
not global task type restrictions.
 2025-08-10 18:02:42 -07:00
chrislu 5c1e6e904d CRITICAL: Restore task conflict definitions to prevent data integrity issues 
- Restore conflicts between vacuum, balance, erasure_coding, and ec_vacuum tasks
- Prevent dangerous concurrent operations on same volumes/resources
- Add comprehensive task conflict matrix to avoid race conditions
- This addresses a serious safety regression where all conflicts were removed

Critical conflicts restored:
- vacuum ↔ balance, erasure_coding, ec_vacuum
- balance ↔ vacuum, erasure_coding, ec_vacuum
- erasure_coding ↔ vacuum, balance, ec_vacuum
- ec_vacuum ↔ vacuum, balance, erasure_coding
- replication ↔ vacuum, balance (destructive ops)
 2025-08-10 17:56:13 -07:00
chrislu 04e3a68b61 Replace deprecated ioutil with os package functions 
- Replace ioutil.WriteFile with os.WriteFile
- Replace ioutil.ReadFile with os.ReadFile
- Remove deprecated io/ioutil import
- Aligns with Go 1.16+ best practices and modern Go standards
 2025-08-10 17:53:48 -07:00