Commit Graph
1715 Commits
Author SHA1 Message Date
25535e9c36 Delete volume is empty (#4561)
* use onlyEmpty for deleteVolume
https://github.com/seaweedfs/seaweedfs/issues/4559

* fix IsEmpty

* fix test

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-06-12 10:42:44 -07:00
柏杰andGitHub 68feffa4e6 avoid overwriting non-nil uploadReaderToChunks.uploadErr with nil value (#4560) 2023-06-10 23:15:27 -07:00
Konstantin LebedevandGitHub 5ee04d20fa Healthz check for deadlocks (#4558) 2023-06-09 09:42:48 -07:00
chrislu 0bffb3770f support TTL when creating folders
related to https://github.com/seaweedfs/seaweedfs/discussions/4549
2023-06-05 23:29:25 -07:00
26f15d0079 Fix no more writable volumes by delay judgment (#4548)
* fix nomore writables volumes while disk free space is sufficient by time delay

* reset

---------

Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2023-06-05 10:17:21 -07:00
5aec6da8a3 fix deadlock for filer upload (#4527)
Signed-off-by: wang wusong <wangwusong@virtaitech.com>
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2023-06-03 13:38:27 -07:00
wusongandGitHub e7f4418855 add return for error handle (#4502) 2023-05-24 07:45:06 -07:00
chrislu d999f1f0e2 update raft version
fix #4460
2023-05-09 22:54:23 -07:00
chrislu 4511edc871 update raft 2023-05-07 13:33:44 -07:00
e9fda774f4 [Filer] post add param:saveInside (#4434)
* fix:mount deadlock

* feat: filer http upload to metadata

* feat: /etc save inside

---------

Co-authored-by: zemul <zhouzemiao@ihuman.com>
2023-04-29 08:31:05 -07:00
4867aa03ab Fix: http rename move dir to subdir (#4432)
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2023-04-26 20:54:36 -07:00
zemulandGitHub f352616b7d Volume range read use sync.pool (#4422) 2023-04-24 07:00:48 -07:00
a408b46d95 compilation fail (#4414)
Signed-off-by: Wusong Wang <wangwusong@virtaitech.com>
Co-authored-by: Wusong Wang <wangwusong@virtaitech.com>
2023-04-18 00:36:55 -07:00
chrislu 0a22eea55d collect ec shard from multiple locations
fix https://github.com/seaweedfs/seaweedfs/issues/4365
2023-04-17 22:56:21 -07:00
chrislu ae9388723f adjust error message 2023-04-17 22:01:39 -07:00
Konstantin LebedevandGitHub 130bc3e668 s3 fix get fake dir object key (#4390) 2023-04-11 07:36:22 -07:00
wusongandGitHub 2b82d0f871 style: clerical error (#4361) 2023-03-31 07:36:21 -07:00
Konstantin LebedevandGitHub de4545c28b add validate config for raft (#4332) 2023-03-21 06:36:13 -07:00
chrislu adb90bd252 avoid lower casing the command
fix https://github.com/seaweedfs/seaweedfs/pull/4321
2023-03-19 21:20:46 -07:00
chrislu 784daea1fa fix volume not found if marked as read only
fix https://github.com/seaweedfs/seaweedfs/issues/4088
2023-03-19 18:30:13 -07:00
LHHDZandGitHub d695119073 The filer does not set defaultReplication to the defaultReplication o… (#4315)
The filer does not set defaultReplication to the defaultReplication of the master (it is not necessary, because if the filer is not set, the configuration of the master will be used when calling Assign), otherwise, when the defaultReplication of the master is modified and all master nodes are restarted, the defaultReplication will not take effect (because the filer Keep the previous defaultReplication setting from the master)

Signed-off-by: changlin.shi <changlin.shi@ly.com>
2023-03-15 22:10:39 -07:00
Rohit ChormaleandGitHub 59706c89fb #4270 set http status code to 409 if dir already exists (#4287) 2023-03-08 07:45:08 -08:00
0bf56298d5 fix chunk.ModifiedTsNs (#4264)
* fix

* fix mtime s > ns

---------

Co-authored-by: zemul <zhouzemiao@ihuman.com>
2023-03-02 08:24:36 -08:00
chrislu 8241c9a829 adjust ui pagination size
fix https://github.com/seaweedfs/seaweedfs/issues/4258
2023-02-28 08:31:26 -08:00
1976ca9160 add -disk to filer command (#4247)
* add -disk to filer command

* add diskType to filer.grpc

* use filer.disk when filerWebDavOptions.disk is empty

* add filer.disk to weed server command.

---------

Co-authored-by: 三千院羽 <3000y@MacBook-Pro.lan>
2023-02-25 09:48:59 -08:00
Zachary WaltersandGitHub ef2f741823 Updated the deprecated ioutil dependency (#4239) 2023-02-21 19:47:33 -08:00
NeoandGitHub d5f77706a8 volume:fix return if Replicate write error (#4188) 2023-02-06 22:34:56 -08:00
chrislu c47f75eb5b fix null content type caused exception in hdfs client
fix https://github.com/seaweedfs/seaweedfs/issues/4170
2023-02-01 02:38:05 -08:00
aandGitHub 0d085ec175 make directory lister trigger if mime type is httpd/unix-directory (#4169) 2023-01-31 06:54:40 -08:00
chrislu 81fdf3651b grpc connection to filer add sw-client-id header 2023-01-20 01:48:12 -08:00
chrislu bfe5d910c6 use one readerCache for the whole file 2023-01-16 22:43:02 -08:00
Guo LeiandGitHub 83e29faa55 use time.NewTicker instead of time.Tick. (#4119) 2023-01-09 06:47:08 -08:00
monchickeyandGitHub 3e2c9ea73d Add image cropping. (#4117) 2023-01-06 09:28:07 -08:00
chrislu 7bdae5172e batch delete EC needles
fix https://github.com/seaweedfs/seaweedfs/issues/4107
2023-01-03 22:05:26 -08:00
Chris LuandGitHub d4566d4aaa more solid weed mount (#4089)
* compare chunks by timestamp

* fix slab clearing error

* fix test compilation

* move oldest chunk to sealed, instead of by fullness

* lock on fh.entryViewCache

* remove verbose logs

* revert slat clearing

* less logs

* less logs

* track write and read by timestamp

* remove useless logic

* add entry lock on file handle release

* use mem chunk only, swap file chunk has problems

* comment out code that maybe used later

* add debug mode to compare data read and write

* more efficient readResolvedChunks with linked list

* small optimization

* fix test compilation

* minor fix on writer

* add SeparateGarbageChunks

* group chunks into sections

* turn off debug mode

* fix tests

* fix tests

* tmp enable swap file chunk

* Revert "tmp enable swap file chunk"

This reverts commit 985137ec47.

* simple refactoring

* simple refactoring

* do not re-use swap file chunk. Sealed chunks should not be re-used.

* comment out debugging facilities

* either mem chunk or swap file chunk is fine now

* remove orderedMutex  as *semaphore.Weighted

not found impactful

* optimize size calculation for changing large files

* optimize performance to avoid going through the long list of chunks

* still problems with swap file chunk

* rename

* tiny optimization

* swap file chunk save only successfully read data

* fix

* enable both mem and swap file chunk

* resolve chunks with range

* rename

* fix chunk interval list

* also change file handle chunk group when adding chunks

* pick in-active chunk with time-decayed counter

* fix compilation

* avoid nil with empty fh.entry

* refactoring

* rename

* rename

* refactor visible intervals to *list.List

* refactor chunkViews to *list.List

* add IntervalList for generic interval list

* change visible interval to use IntervalList in generics

* cahnge chunkViews to *IntervalList[*ChunkView]

* use NewFileChunkSection to create

* rename variables

* refactor

* fix renaming leftover

* renaming

* renaming

* add insert interval

* interval list adds lock

* incrementally add chunks to readers

Fixes:
1. set start and stop offset for the value object
2. clone the value object
3. use pointer instead of copy-by-value when passing to interval.Value
4. use insert interval since adding chunk could be out of order

* fix tests compilation

* fix tests compilation
2023-01-02 23:20:45 -08:00
Guo LeiandGitHub d8cfa1552b support enable/disable vacuum (#4087)
* stop vacuum

* suspend/resume vacuum

* remove unused code

* rename

* rename param
2022-12-28 01:36:44 -08:00
lfhyandGitHub 038bab3726 add -filer.path to webdav command (#4061) 2022-12-17 20:07:02 -08:00
chrislu fc6b9e6e0c volume server logs add url info
debug https://github.com/seaweedfs/seaweedfs/issues/3964
2022-12-11 13:15:38 -08:00
chrislu 38c63c429b Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-12-10 21:49:09 -08:00
chrislu e1ca6308cb add chunk etag when downloading from remote storage
fix https://github.com/seaweedfs/seaweedfs/issues/3987
2022-12-10 21:49:07 -08:00
549354e324 Fix hardlink counting (#4042)
Signed-off-by: wusong <wangwusong@virtaitech.com>

Signed-off-by: wusong <wangwusong@virtaitech.com>
Co-authored-by: wusong <wangwusong@virtaitech.com>
2022-12-08 10:50:57 -08:00
Jiffs MaverickandGitHub 5f59d43c88 [filer] Fix uri escape during renaming (#4039) 2022-12-07 07:47:40 -08:00
famosssandGitHub 15284e2bee fix: Lost Content-Encoding metadata (#4010) 2022-11-23 22:52:34 -08:00
chrislu 70a4c98b00 refactor filer_pb.Entry and filer.Entry to use GetChunks()
for later locking on reading chunks
2022-11-15 06:33:36 -08:00
5b905fb2b7 Lazy loading (#3958)
* types packages is imported more than onece

* lazy-loading

* fix bugs

* fix bugs

* fix unit tests

* fix test error

* rename function

* unload ldb after initial startup

* Don't load ldb when starting volume server if ldbtimeout is set.

* remove uncessary unloadldb

* Update weed/command/server.go

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>

* Update weed/command/volume.go

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>

Co-authored-by: guol-fnst <goul-fnst@fujitsu.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2022-11-14 00:19:27 -08:00
chrislu aae82dca6f file name escape uri
fix https://github.com/seaweedfs/seaweedfs/issues/3969
2022-11-13 23:01:56 -08:00
chrislu ea2637734a refactor filer proto chunk variable from mtime to modified_ts_ns 2022-10-28 12:53:19 -07:00
51d462f204 ADHOC: volume fsck using append at ns (#3906)
* ADHOC: volume fsck using append at ns

* nit

* nit

Co-authored-by: root <root@HQ-10MSTD3EY.roblox.local>
2022-10-24 22:09:38 -07:00
liubaojiangandGitHub 25471d579a add md5 header when UploadData to replication in ReplicatedWrite (#3881) 2022-10-19 01:34:14 -07:00
chrislu de286fe662 shell: volume.move handles volume moved to cloud tier
fix https://github.com/seaweedfs/seaweedfs/issues/3803
2022-10-16 17:52:22 -07:00