Commit Graph
472 Commits
Author SHA1 Message Date
SmsS4andGitHub 17e91d2917 Use filerGroup for s3 buckets collection prefix (#4465)
* Use filerGroup for s3 buckets collection prefix

* Fix templates

* Remove flags

* Remove s3CollectionPrefix
2023-05-16 09:39:43 -07:00
Konstantin LebedevandGitHub d75a7b7f62 allow deleting only older empty dir without recursion (#4430) 2023-04-25 08:31:14 -07:00
5614ad0000 fix s3test test_bucket_listv2_delimiter_prefix_ends_with_delimiter (#4399)
* fix s3test test_bucket_listv2_delimiter_prefix_ends_with_delimiter

* fix list with delimiter and start token

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-04-12 16:53:49 -07:00
Konstantin LebedevandGitHub 88b2bbadd1 try fix s3test test_bucket_listv2_delimiter_prefix (#4396) 2023-04-12 08:47:20 -07:00
095cdb0c7f fix test prefix is a directory (#4393)
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-04-12 00:08:13 -07:00
44ad07276c s3 fix get list of dir object key with slash suffix (#4391)
* s3 fix get list of dir object key with slash suffix
https://github.com/seaweedfs/seaweedfs/issues/3086

* list only entry dir eq prefix

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-04-11 23:52:35 -07:00
Konstantin LebedevandGitHub 130bc3e668 s3 fix get fake dir object key (#4390) 2023-04-11 07:36:22 -07:00
chrislu 6bfb91b539 update entry Attributes.Mtime = time.Now().Unix()
fix https://github.com/seaweedfs/seaweedfs/issues/4337
2023-03-25 22:02:57 -07:00
chrislu 5db9fcccd4 refactoring 2023-03-21 23:01:49 -07:00
0bf56298d5 fix chunk.ModifiedTsNs (#4264)
* fix

* fix mtime s > ns

---------

Co-authored-by: zemul <zhouzemiao@ihuman.com>
2023-03-02 08:24:36 -08:00
LHHDZandGitHub db5515eada Fix s3api_object_list_handlers returning contents less than the specified limit when more data actually exists (#4240)
Fix when the stored data is actually enough but s3api_object_list_handlers returns less than the specified limit

Signed-off-by: changlin.shi <changlin.shi@ly.com>
2023-02-21 23:08:52 -08:00
chrislu 79d68a81fd text change 2023-02-07 15:00:41 -08:00
chrislu 999a126a3f typo 2023-02-07 14:59:47 -08:00
chrislu 1c2364936a detect invalid bucket name
fix https://github.com/seaweedfs/seaweedfs/issues/4143
2023-02-07 14:57:29 -08:00
chrislu 5de93fe442 refactoring 2023-02-07 14:45:20 -08:00
chrislu 81fdf3651b grpc connection to filer add sw-client-id header 2023-01-20 01:48:12 -08:00
Guo LeiandGitHub 5f993a5f20 Fix ide warning (#4082)
fix warning message from IDE
"redundant type from array, slice, or map composite literal"
2022-12-26 00:03:47 -08:00
LHHDZandGitHub f84c13dabd fix response not allowed (#4057) 2022-12-13 05:48:50 -08:00
LHHDZandGitHub bb289d7d69 sort tags by key (#4047)
Signed-off-by: changlin.shi <changlin.shi@ly.com>

Signed-off-by: changlin.shi <changlin.shi@ly.com>
2022-12-11 22:49:57 -08:00
Konstantin LebedevandGitHub 2b910d1cf8 avoid recursive deleting newly created empty directories (#4016) 2022-11-25 08:45:47 -08:00
chrislu 70a4c98b00 refactor filer_pb.Entry and filer.Entry to use GetChunks()
for later locking on reading chunks
2022-11-15 06:33:36 -08:00
LHHDZandGitHub 39f0246deb fix 'NoSuchUpload' when upload part (#3968) 2022-11-11 04:53:53 -08:00
famosssandGitHub df0662038c feat:add a uuid after uploadid (#3963) 2022-11-10 07:17:29 -08:00
pavelzhurovandGitHub abe4a61659 Bug fix: empty key in DeleteMultipleObjects request caused bucket delete (#3939) 2022-11-02 05:51:13 -07:00
chrislu 8b9957d461 add back "/" prefix if it is missing in object
fix https://github.com/seaweedfs/seaweedfs/issues/3737
2022-10-29 17:54:30 -07:00
chrislu ea2637734a refactor filer proto chunk variable from mtime to modified_ts_ns 2022-10-28 12:53:19 -07:00
famosssandGitHub 34132b2c9f fix: isTruncated returns problematic (#3907) 2022-10-24 21:15:37 -07:00
famosssandGitHub 874fd197b5 feat: simplify a bit (#3905) 2022-10-24 17:58:26 -07:00
famosssandGitHub 25e012d30b fix: set user metadata key to lowercase (#3894)
* fix: set user metadata key to lowercase

* feat: simplify a bit
2022-10-24 17:29:52 -07:00
Guo LeiandGitHub 97edb40275 Fix errinfo (#3893)
* types packages is imported more than onece

* Fix error response when format of --expires is wrong.
It MUST be in RFC 1123 date format.
2022-10-24 09:25:48 -07:00
LHHDZandGitHub a5b867af69 don't skip empty value at AssembleEntryWithAcp (#3855)
* add acl helper functionalities

Signed-off-by: changlin.shi <changlin.shi@ly.com>

* add tests

Signed-off-by: changlin.shi <changlin.shi@ly.com>

* remove 0 when create map

Signed-off-by: changlin.shi <changlin.shi@ly.com>

* delete when empty at `AssembleEntryWithAcp`

`PutBucketAcl/PutObjectAcl` allow request with empty grants, `AssembleEntryWithAcp`  shouldn't skip empty value

Signed-off-by: changlin.shi <changlin.shi@ly.com>

Signed-off-by: changlin.shi <changlin.shi@ly.com>
2022-10-13 22:50:44 -07:00
LHHDZandGitHub bf5e45b66a add acl helper functionalities (#3831) 2022-10-12 20:15:16 -07:00
LHHDZandGitHub d21e2f523d split ExtAcpKey to ExtAmzOwnerKey and ExtAmzAclKey to avoid unn… (#3824)
split `ExtAcpKey` to `ExtAmzOwnerKey` and `ExtAmzAclKey` to avoid unnecessary `json.Unmarshal()` call

Signed-off-by: changlin.shi <changlin.shi@ly.com>

Signed-off-by: changlin.shi <changlin.shi@ly.com>
2022-10-11 20:14:14 -07:00
LHHDZandGitHub 3550692afc change s3_account.go package to avoid cycle dependency (#3813) 2022-10-10 06:44:29 -07:00
Konstantin LebedevandGitHub ec46a34f33 avoid increasing the coordinates of metric labels (#3785) 2022-10-04 01:20:01 -07:00
LHHDZandGitHub e9584d9661 add ownership rest apis (#3765) 2022-10-01 19:18:00 -07:00
LHHDZandGitHub 3de1e19780 s3: sync bucket info from filer (#3759) 2022-09-29 12:29:01 -07:00
LHHDZandGitHub aacdcc4cad s3: add account (#3753)
associate `Account` and `Identity` by accountId
2022-09-28 12:45:18 -07:00
chrislu 31922b2bf2 s3 to watch specific directories 2022-09-20 09:30:05 -07:00
Ryan RussellandGitHub 6f7ef8711a docs(s3api): readability improvements (#3696)
Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-09-15 03:13:21 -07:00
Ryan RussellandGitHub ad3a3c8782 refactor(s3api_object_handlers): deleteMultipleObjectsLimmit -> `de… (#3695)
refactor(s3api_object_handlers): `deleteMultipleObjectsLimmit` -> `deleteMultipleObjectsLimit`

Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-09-15 03:12:48 -07:00
ImFantuanandGitHub b64411bda8 fix 0 size (#3683) 2022-09-15 01:27:02 -07:00
famosssandGitHub bb880d6253 IsTruncated returned by the ListMultipartUploads method is set to false by default (#3682)
* simplify a bit

* fix: IsTruncated returned by the ListMultipartUploads method is set to false by default
2022-09-15 00:27:15 -07:00
chrislu 21c0587900 go fmt 2022-09-14 23:06:44 -07:00
Ryan RussellandGitHub d54eb9966f refactor: Directory readability (#3665) 2022-09-14 10:11:31 -07:00
Konstantin LebedevandGitHub 8c3040db81 avoid DATA RACE on S3Options.localFilerSocket (#3571)
* avoid DATA RACE on S3Options.localFilerSocket
https://github.com/seaweedfs/seaweedfs/issues/3552

* copy localSocket
2022-09-01 10:33:23 -07:00
chrislu a54f30b6c6 s3: close response body with copying object 2022-08-31 00:09:23 -07:00
Konstantin LebedevandGitHub f7aeb06544 s3: report metadata if the directory is explicitly created (#3498)
* replace mkdir to mkFile

* ContentLength must be zero

* revert mkDir

* Seaweedfs-Is-Directory-Key return metadata
2022-08-24 00:15:44 -07:00
askeipxandGitHub 2e78a522ab remove old raft servers if they don't answer to pings for too long (#3398)
* remove old raft servers if they don't answer to pings for too long

add ping durations as options

rename ping fields

fix some todos

get masters through masterclient

raft remove server from leader

use raft servers to ping them

CheckMastersAlive for hashicorp raft only

* prepare blocking ping

* pass waitForReady as param

* pass waitForReady through all functions

* waitForReady works

* refactor

* remove unneeded params

* rollback unneeded changes

* fix
2022-08-23 23:18:21 -07:00
chrislu 9fce75607d s3: report http.StatusOK if the directory is explicitly created
fix https://github.com/seaweedfs/seaweedfs/issues/3457
2022-08-23 01:16:46 -07:00