mirrors/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-09-08 09:07:18 +00:00
Code Issues Packages Projects Releases Wiki Activity
11,095 Commits 177 Branches 359 Tags
e2d30cc3bc05b225fed1a111b4a403b9dbff5aba
Commit Graph
7151 Commits
Author SHA1 Message Date
wyangandGitHub a7973ed7d1 fix deadlock hang when broadcast to clients (#6184) 
fix deadlock when broadcast to clients

when master thransfer leader, the old master will disconnect with all
filers and volumeServers, if the cluster is a big , the broadcast
messages may be more big than the max of the channel len 100, then if the
KeepConnect was not listen on the channel in disconnect, it will
deadlock. and the whole cluster will not serve!
 2024-11-03 23:20:48 -08:00
Konstantin LebedevandGitHub 5bddf0c085 [shell] volume.balance collect volume servers by dc rack node (#6191) 
* chore: balance by rack

* fix: rm check lock

* fix: selected racks

* fix: selected nodes

* fix: containts

* fix: one collectVolumeServersByDcRackNode

* fix: revert lock and add lock

* fix: panic test

* revert noLock
 2024-11-03 11:08:45 -08:00
hanandGitHub 65fb8fad99 Update filer_multipart.go (#6188) 
fix the #6177 bug
 2024-11-01 00:38:35 -07:00
BruceGitHubChris Lu
 		0060a2cf9c Fix 6181/6182 (#6183) 
* set larger buf size for LogBuffer

* jump to next day when no more entry found

* Update weed/filer/filer_notify_read.go

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2024-10-31 08:40:05 -07:00
wyangandGitHub c29c912bdc fix format (#6185) 
unitest weed/shell fail
 2024-10-31 08:30:35 -07:00
chrislu ad6240f066 fix parameter 2024-10-30 20:30:51 -07:00
chrislu 8802843511 pass in bytes buffer to avoid possible race condition 
related to https://github.com/seaweedfs/seaweedfs/issues/6180
 2024-10-30 09:31:22 -07:00
chrislu 3feb66d0a1 adding more debug message 
related to https://github.com/seaweedfs/seaweedfs/issues/6180
 2024-10-30 09:16:40 -07:00
chrislu 228946369c 3.79 2024-10-28 19:44:39 -07:00
chrislu 9873b033d1 backward compatible vif loading 2024-10-28 19:44:30 -07:00
chrislu 9105c6bdd1 fix format 2024-10-28 11:29:08 -07:00
chrislu ebbb35d533 3.78 2024-10-27 17:44:22 -07:00
chrislu 089d4316ef ensure 2 volume space since actual need 1.4x volume size empty space 2024-10-24 22:44:53 -07:00
chrislu 6e388e29c9 correcting free volume count, factor it during ec encoding to ensure enough disk space available 
fix https://github.com/seaweedfs/seaweedfs/issues/6163
 2024-10-24 22:42:38 -07:00
chrislu 8d6189bcc5 adjust output format 2024-10-24 21:41:39 -07:00
chrislu 2f3d820f52 rename proto field 
This should not have any impact.
 2024-10-24 21:36:56 -07:00
chrislu ae5bd0667a rename proto field from DestroyTime to expire_at_sec 
For TTL volume converted into EC volume, this change may leave the volumes staying.
 2024-10-24 21:35:11 -07:00
chrislu 1c2b10303a remove X-Forwarded-For 
fix https://github.com/seaweedfs/seaweedfs/issues/6136
 2024-10-23 17:52:45 -07:00
chrislu 73921ce4f6 adjust help message 2024-10-22 08:51:02 -07:00
Guang Jiong LouGitHubChris Lu
 		9369a88c5c stop renaming worm files (#6154) 
* stop renaming worm file

Signed-off-by: lou <alex1988@outlook.com>

* update after review

Signed-off-by: lou <alex1988@outlook.com>

* Update weed/server/filer_server_handlers_write.go

---------

Signed-off-by: lou <alex1988@outlook.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2024-10-22 08:46:18 -07:00
chrislu d38456dc06 do not delete collection metrics when unmount a volume 
fix https://github.com/seaweedfs/seaweedfs/issues/6139
 2024-10-22 00:44:52 -07:00
chrislu 6c7824c813 consistent sorted tags elastic,gocdk,rclone,sqlite,tikv,ydb 2024-10-21 22:53:29 -07:00
chrislu 0df4a4298b adjust comment 2024-10-21 22:36:25 -07:00
Guang Jiong LouandGitHub 54b91ba2b2 fix permission check in put request (#6142) 2024-10-18 07:00:45 -07:00
dsdGitHub邓书东Chris Lu
 		1e13b6879c fix(volume): to avoid duplicate write a same needle (#6138) 
fix WriteNeedleBlob to avoid duplicate write a same needle

Co-authored-by: 邓书东 <shudong_deng@hhnb2024010108.intsig.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2024-10-18 01:20:50 -07:00
chrislu f11567816e Revert "fix: volume data integrity check (#6118)" 
This reverts commit e12b9ab746.
 2024-10-17 18:52:30 -07:00
wusongGitHubwang wusong
 		e12b9ab746 fix: volume data integrity check (#6118) 
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
 2024-10-14 12:17:41 -07:00
steve.weiandGitHub cfbe45c765 feat: add in-flight metric for s3/file/volume-server (#6120) 2024-10-14 12:10:05 -07:00
chrislu 804e765397 remove unused 2024-10-13 23:12:10 -07:00
chrislu 6845e25318 set proxied request content length 2024-10-10 13:55:32 -07:00
chrislu 35fd1e1c9a optimize memory usage for large number of volumes 
1. unwrap the map to avoid extra map object creation
2. fix ec shard counting in UpdateEcShards
 2024-10-10 10:00:30 -07:00
chrislu b28b1a3402 3.77 2024-10-10 01:28:06 -07:00
chrislu 07df5c53de avoid nil 
I1010 03:10:25.220345 mount_std.go:281 This is SeaweedFS version 30GB 3.76 82ed61c655 linux amd64
^[OR^[OR2024/10/10 03:12:56 Unimplemented opcode POLL
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x3983b41]
goroutine 112 [running]:
github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents.func2.mergeProceesors.1(0xc000513540)
/github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:30 +0x41
github.com/seaweedfs/seaweedfs/weed/pb.WithFilerClientFollowMetadata.makeSubscribeMetadataFunc.func1({0x514a560, 0xc0009b06d0})
/github/workspace/weed/pb/filer_pb_tail.go:87 +0x293
github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).WithFilerClient.func1.1(0xc000d9e008)
/github/workspace/weed/mount/wfs_filer_client.go:27 +0x68
github.com/seaweedfs/seaweedfs/weed/pb.WithGrpcClient(0xa9?, 0x9d1f90fb?, 0xc00098fbe0, {0xc000e1c078, 0x15}, 0x0?, {0xc000a76bd0?, 0x1?, 0x1?
})
/github/workspace/weed/pb/grpc_client_server.go:155 +0x33a
github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).WithFilerClient.func1()
/github/workspace/weed/mount/wfs_filer_client.go:25 +0x14c
github.com/seaweedfs/seaweedfs/weed/util.Retry({0x489a10c, 0xa}, 0xc00098fcf8)
/github/workspace/weed/util/retry.go:16 +0xa7
github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).WithFilerClient(0x0?, 0x30?, 0xc00056a568?)
/github/workspace/weed/mount/wfs_filer_client.go:18 +0x65
github.com/seaweedfs/seaweedfs/weed/pb.WithFilerClientFollowMetadata({0x50f5898, 0xc000e2c180}, 0xc00101e000, 0xc000d22030)
/github/workspace/weed/pb/filer_pb_tail.go:51 +0x87
github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents.func2()
/github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:115 +0xcb
github.com/seaweedfs/seaweedfs/weed/util.RetryUntil({0x48b8b35, 0x11}, 0xc00098fef0, 0x4af0048)
/github/workspace/weed/util/retry.go:64 +0x4e
github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents(0xc000d1d980, 0x9d1f90fb, {0x50f5898, 0xc000e2c180}, {0x7ffdf6f98dca,
0x4}, 0x17fcf8b7c7ecf8cf, {0xc0000ac208, 0x1, 0x1})
/github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:113 +0x39b
created by github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).StartBackgroundTasks in goroutine 1
/github/workspace/weed/mount/weedfs.go:150 +0x146
 2024-10-09 22:46:00 -07:00
chrislu 82ed61c655 3.76 2024-10-06 19:37:03 -07:00
chrislu c0e36231ad use only one metadata follow process 2024-10-06 12:55:19 -07:00
Konstantin LebedevGitHubChris Lu
 		44b275879b [s3] add {Get,Put,Delete}BucketTagging and PublicAccessBlock Handlers (#6088) 
* add {Get,Put,Delete}BucketTagging Handlers

* s3 add skip bucket PublicAccessBlock handlers

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2024-10-04 10:59:14 -07:00
chrislu a3a8f6217e fix 2024-10-03 09:03:17 -07:00
chrislu 117c3aebda 3.75 2024-10-03 08:33:35 -07:00
Konstantin LebedevandGitHub a3556acd98 [s3] add skip bucket encryption handlers (#6091) 
s3 add skip bucket encryption handlers
 2024-10-03 08:10:43 -07:00
Er2andGitHub 5644bc8f01 s3api: Fix signature v4 with reverse proxy at sub-path (#6092) 2024-10-03 08:08:19 -07:00
chrislu ff47956530 add helper functions 2024-10-03 00:13:46 -07:00
chrislu 11e53dd8ef skip error if conf file is not created 2024-10-01 22:46:48 -07:00
chrislu f747767aa1 support load balancer in front of s3 2024-10-01 12:57:45 -07:00
chrislu 7dd480acad handle missing leading zero padding in replica 2024-10-01 10:16:21 -07:00
chrislu e3fa87bcc1 3.74 2024-09-30 11:29:01 -07:00
chrislu cdd7fa81ab fix help message 2024-09-29 21:53:07 -07:00
chrislu 20929f2a57 adjust resource heavy for volume.fix.replication 2024-09-29 11:32:18 -07:00
chrislu 6564ceda91 skip resource heavy commands from running on master nodes 2024-09-29 10:51:17 -07:00
chrislu ec30a504ba refactor 2024-09-29 10:38:22 -07:00
chrislu 9cd263b2ce refactor 2024-09-29 10:35:53 -07:00