Commit Graph

579 Commits

Author SHA1 Message Date
chrislu bd4891a117 change version directory 2025-06-03 22:46:10 -07:00
Bruce Zou 62aaaa18f3 unlimit the list entries in completeMultipartUpload (#6822) 2025-05-27 03:41:27 -07:00
royatwp 2786bea839 Unable to upload empty files to seaweed via. multipart when length is unknown (#6729) 2025-05-26 23:25:01 -07:00
Aleksey Kosov 165af32d6b added context to filer_client method calls (#6808)
Co-authored-by: akosov <a.kosov@kryptonite.ru>
2025-05-22 09:46:49 -07:00
Taehyung Lim 0fbdedecc7 [s3] refactor duplicate put-object code paths (#6727) 2025-05-01 13:08:03 -07:00
Taehyung Lim ea941462f9 [s3] use chunkedReader in PutObjectPartHandler to handle aws-chunked encoding (#6725) 2025-04-23 15:45:46 -07:00
Konstantin Lebedev 8c1cb203fa [s3] fix checksum algorithm CRC64NVMe (#6722) 2025-04-23 07:31:03 -07:00
zuzuviewer 396a602f86 * Fix s3 auth failed with X-Forwarded-Host and X-Forwarded-Port (#6698) 2025-04-08 21:26:19 -07:00
dependabot[bot] 216c52e377 chore(deps): bump gocloud.dev from 0.40.0 to 0.41.0 (#6679)
* chore(deps): bump gocloud.dev from 0.40.0 to 0.41.0

Bumps [gocloud.dev](https://github.com/google/go-cloud) from 0.40.0 to 0.41.0.
- [Release notes](https://github.com/google/go-cloud/releases)
- [Commits](https://github.com/google/go-cloud/compare/v0.40.0...v0.41.0)

---
updated-dependencies:
- dependency-name: gocloud.dev
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix error

* fix printing errors

* Update go.mod

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
2025-03-31 21:42:54 -07:00
Tom Crasset be2b389b81 add more logging for s3 signature (#6652) 2025-03-21 07:37:15 -07:00
zuzuviewer db759a1ad1 Fix s3 auth failed with X-Forwarded-Host and X-Forwarded-Port (#6650) 2025-03-20 20:47:07 -07:00
chrislu cb33ee006e skip headers when signing a request
fix https://github.com/seaweedfs/seaweedfs/issues/6576#issuecomment-2724577279
2025-03-18 08:35:53 -07:00
SmoothDenis e60aa71890 fix: restore deletion audit of individual objects (#6644) 2025-03-18 05:01:54 -07:00
NinaLua 271505717e chore: make function comment match function name (#6607) 2025-03-07 00:45:47 -08:00
Tom Crasset 9604db2c93 implement s3 streaming-unsigned-payload-trailer (#6539)
* implement s3 streaming-unsigned-payload-trailer

* chore: remove print
2025-02-12 12:29:13 -08:00
Tom Crasset 9ca2b0b763 omit http and https ports when using X-Forwarded-Port (#6527) 2025-02-07 10:55:09 -08:00
Tom Crasset a7b964af96 add s3 signature tests and prepare implementation of STREAMING-UNSIGNED-PAYLOAD-TRAILER (#6525)
* add tests for s3 signature

* add test for newSignV4ChunkedReader.Read()

* add glog import
2025-02-07 10:54:31 -08:00
zouyixiong 8eab76c5db fix: record and delete bucket metrics after inactive (#6523)
Co-authored-by: XYZ <XYZ>
2025-02-07 10:26:39 -08:00
Tom Crasset a250699225 use X-Forwarded-Host and X-Forwarded-Port to determine correct host for AWS signature (#6514) 2025-02-06 07:54:50 -08:00
chrislu 7fd42f5d06 Revert "Add content length to filer's requests (#6489)"
This reverts commit 9f42ce3f01.
2025-02-03 09:24:14 -08:00
chrislu 42efade0dc adjust fix
fix https://github.com/seaweedfs/seaweedfs/issues/6497
2025-02-01 14:11:57 -08:00
Tom Crasset 551baa44b2 verify if bucket exists in GetBucketLocationHandler (#6485) 2025-01-29 08:55:08 -08:00
Hadi Zamani 9f42ce3f01 Add content length to filer's requests (#6489) 2025-01-29 08:53:39 -08:00
Hadi Zamani a2330f624b Add metrics for uploaded and deleted s3 objects (#6475) 2025-01-25 21:55:06 -08:00
Tom Crasset c5f21b2b01 fix S3 per-user-directory Policy (#6443)
* fix S3 per-user-directory Policy

* Delete docker/config.json

* add tests

* remove logs

* undo modifications of weed/shell/command_volume_balance.go

* remove modifications of docker-compose

* fix failing test

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2025-01-17 01:03:17 -08:00
Hadi Zamani c7ae969c06 Add bucket's traffic metrics (#6444)
* Add bucket's traffic metrics

* Add bucket traffic to dashboards

* Fix bucket metrics help messages

* Fix variable names
2025-01-16 08:23:35 -08:00
Tom Crasset aa299462f2 improve iam error handling (#6446)
* improve iam error handling

* Delete docker/test.py
2025-01-16 08:23:19 -08:00
ftong2020 2304d2b472 Bugfix s3 audit missing requester for PUT requests (#6434)
fix s3 audit missing requster for PUT
2025-01-15 11:58:43 -08:00
chrislu d7dfb11be9 fix compilation 2024-12-19 19:26:28 -08:00
chrislu ec155022e7 "golang.org/x/exp/slices" => "slices" and go fmt 2024-12-19 19:25:06 -08:00
Warren Hodgkinson a1a76ccb8c Fix for DeleteMultipleObjectsHandler wrongly deleting parent folders (#6380)
What problem are we solving?
Fix: #6379

How are we solving the problem?
We check for the AllowEmptyFolders option prior to cascade
deleting parent folders in S3 DeleteMultipleObjectsHandler.

How is the PR tested?
We ran SeaweedFS in a Kubernetes Cluster with a joint Filer
and S3 server in one container, with leveldb2 as the filer storage,
and AllowEmptyFolders set to true.

When using the Distribution Registry as the S3 client, it calls the
DeleteMultipleObjectsHandler as part of the artifact upload process
(uploads to a temp location, then performs a copy and delete).
Without this fix, the deletion cascade deleted parent folder until
the entire contents of the bucket were gone.

With this fix, the existing content of the bucket remained, and the
newly uploaded content was added.

Checks
[ ] I have added unit tests if possible.
[ ] I will add related wiki document changes and link to this PR after merging.

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2024-12-19 18:00:08 -08:00
zouyixiong 9987a65e8a fix: record and delete bucket metrics after inactive (#6349) 2024-12-12 20:34:02 -08:00
steve.wei 88fa64a01a change comment for expect header (#6289)
* remove case of expect

* Set the default value of the Expect header for compatibility.
2024-11-26 08:35:41 -08:00
chrislu 6b2612690b consistent error message if auth is not setup
fix https://github.com/seaweedfs/seaweedfs/issues/6262
2024-11-20 23:11:18 -08:00
han 65fb8fad99 Update filer_multipart.go (#6188)
fix the #6177 bug
2024-11-01 00:38:35 -07:00
chrislu 1c2b10303a remove X-Forwarded-For
fix https://github.com/seaweedfs/seaweedfs/issues/6136
2024-10-23 17:52:45 -07:00
chrislu 0df4a4298b adjust comment 2024-10-21 22:36:25 -07:00
steve.wei cfbe45c765 feat: add in-flight metric for s3/file/volume-server (#6120) 2024-10-14 12:10:05 -07:00
chrislu 6845e25318 set proxied request content length 2024-10-10 13:55:32 -07:00
Konstantin Lebedev 44b275879b [s3] add {Get,Put,Delete}BucketTagging and PublicAccessBlock Handlers (#6088)
* add {Get,Put,Delete}BucketTagging Handlers

* s3 add skip bucket PublicAccessBlock handlers

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2024-10-04 10:59:14 -07:00
chrislu a3a8f6217e fix 2024-10-03 09:03:17 -07:00
Konstantin Lebedev a3556acd98 [s3] add skip bucket encryption handlers (#6091)
s3 add skip bucket encryption handlers
2024-10-03 08:10:43 -07:00
Er2 5644bc8f01 s3api: Fix signature v4 with reverse proxy at sub-path (#6092) 2024-10-03 08:08:19 -07:00
chrislu f747767aa1 support load balancer in front of s3 2024-10-01 12:57:45 -07:00
steve.wei ae8c6d95a6 fix: Prevent potential metadata change events from being lost. (#6066) 2024-09-26 08:31:57 -07:00
chrislu 50e22864cf refactor 2024-09-17 23:10:41 -07:00
chrislu 43d9f68219 more consistent server header for s3 apis 2024-09-09 09:07:58 -07:00
Konstantin Lebedev 16984ef07a [s3] reducing the number of metrics for 403 (#5961)
reducing the number of metrics
2024-09-04 11:06:54 -07:00
chrislu 6ce27affa4 also use /healthz for most consistent health check 2024-08-21 10:55:39 -07:00
Oleg Salionov 4cfc7d3b10 Dont try lazy decode content in proxyToFiler if no accept-encoding provided (#5907) 2024-08-18 21:51:17 -07:00