Commit Graph
21 Commits
Author SHA1 Message Date
Chris LuandGitHub c00aa90990 fix(s3/audit): populate requester for GET/HEAD/IAM operations (#9581)
Authentication records the identity with r.WithContext, which returns a
request copy. Handlers that log their own audit entry (PUT, DELETE,
tagging) see it, but GET/HEAD object and IAM operations rely on track()'s
fallback entry, which is built from the original request the auth copy
never reached - so requester came out empty.

Install a mutable identity holder on the request before authentication
and have SetIdentityNameInContext record into it. The holder is shared by
pointer across every request copy, so the fallback entry recovers the
authenticated requester. The per-request context value still takes
precedence, so nothing changes for handlers that see the auth copy.
2026-05-20 10:13:33 -07:00
Chris LuandGitHub 3f1eaf9724 fix(s3/audit): emit audit log for successful GET/HEAD (#9467)
* fix(s3/audit): emit audit log for successful GET/HEAD

Successful GET/HEAD object requests never produced a fluent audit entry
because those handlers write the response directly (streaming for GET,
WriteHeader for HEAD) and never reach a PostLog call site. The wiki
advertises GET as an audited verb, so the asymmetry surprises operators
who rely on the log for read-access auditing.

Move the safety net into the track() middleware: tag each request with
an audit-tracking flag, let PostLog/PostAccessLog (delete path) mark it,
and emit a single fallback entry after the handler returns when nothing
fired. The recorder's status flows into the fallback so the audit row
still reflects 200/206 vs 404 etc. No double logging for handlers that
already emit (write helpers, error paths, bulk delete).

Refs #9463

* fix(s3/audit): defensive nil checks on audit-tracking helpers

Address PR review: guard against nil request and nil *atomic.Bool stored
under the audit-tracking key. The conditions are unreachable today (the
key is private and we only ever store new(atomic.Bool)), but the checks
are free and keep the helpers safe if a future caller misbehaves.

* test(s3/audit): track() audit fallback coverage + stale comment cleanup (#9469)

test(s3/audit): cover track() fallback wiring + cleanup

Adds two unit tests in weed/s3api/stats_test.go that exercise the
audit-tracking flag set up by track(): one verifies the fallback path
fires when a handler writes the response directly (the GET/HEAD object
regression in #9463), the other verifies the flag is set when a handler
emits PostLog itself so the fallback is skipped.

To make the wiring observable without standing up fluent, PostLog now
marks the audit flag before short-circuiting on a nil Logger; production
behavior is unchanged (no logger, no posting) but the flag stays
consistent.

Also drops two stale comments in s3api_object_handlers.go that still
referenced proxyToFiler — that helper was removed when GET/HEAD started
streaming from volume servers directly.

Stacks on #9467.
2026-05-13 09:24:59 -07:00
Roman ShishkinandGitHub 83c1bfbacd Populate bucket_traffic_received_bytes_total metric (#7249) 2025-09-17 19:04:51 -07:00
Chris LuandGitHub 33b9017b48 fix listing objects (#7008)
* fix listing objects

* add more list testing

* address comments

* fix next marker

* fix isTruncated in listing

* fix tests

* address tests

* Update s3api_object_handlers_multipart.go

* fixes

* store json into bucket content, for tagging and cors

* switch bucket metadata from json to proto

* fix

* Update s3api_bucket_config.go

* fix test issue

* fix test_bucket_listv2_delimiter_prefix

* Update cors.go

* skip special characters

* passing listing

* fix test_bucket_list_delimiter_prefix

* ok. fix the xsd generated go code now

* fix cors tests

* fix test

* fix test_bucket_list_unordered and test_bucket_listv2_unordered

do not accept the allow-unordered and delimiter parameter combination

* fix test_bucket_list_objects_anonymous and test_bucket_listv2_objects_anonymous

The tests test_bucket_list_objects_anonymous and test_bucket_listv2_objects_anonymous were failing because they try to set bucket ACL to public-read, but SeaweedFS only supported private ACL.

Updated PutBucketAclHandler to use the existing ExtractAcl function which already supports all standard S3 canned ACLs
Replaced the hardcoded check for only private ACL with proper ACL parsing that handles public-read, public-read-write, authenticated-read, bucket-owner-read, bucket-owner-full-control, etc.
Added unit tests to verify all standard canned ACLs are accepted

* fix list unordered

The test is expecting the error code to be InvalidArgument instead of InvalidRequest

* allow anonymous listing( and head, get)

* fix test_bucket_list_maxkeys_invalid

Invalid values: max-keys=blah → Returns ErrInvalidMaxKeys (HTTP 400)

* updating IsPublicRead when parsing acl

* more logs

* CORS Test Fix

* fix test_bucket_list_return_data

* default to private

* fix test_bucket_list_delimiter_not_skip_special

* default no acl

* add debug logging

* more logs

* use basic http client

remove logs also

* fixes

* debug

* Update stats.go

* debugging

* fix anonymous test expectation

anonymous user can read, as configured in s3 json.
2025-07-22 01:07:15 -07:00
chrislu bd4891a117 change version directory 2025-06-03 22:46:10 -07:00
8eab76c5db fix: record and delete bucket metrics after inactive (#6523)
Co-authored-by: XYZ <XYZ>
2025-02-07 10:26:39 -08:00
Hadi ZamaniandGitHub c7ae969c06 Add bucket's traffic metrics (#6444)
* Add bucket's traffic metrics

* Add bucket traffic to dashboards

* Fix bucket metrics help messages

* Fix variable names
2025-01-16 08:23:35 -08:00
zouyixiongandGitHub 9987a65e8a fix: record and delete bucket metrics after inactive (#6349) 2024-12-12 20:34:02 -08:00
steve.weiandGitHub cfbe45c765 feat: add in-flight metric for s3/file/volume-server (#6120) 2024-10-14 12:10:05 -07:00
chrislu 43d9f68219 more consistent server header for s3 apis 2024-09-09 09:07:58 -07:00
Konstantin LebedevandGitHub 16984ef07a [s3] reducing the number of metrics for 403 (#5961)
reducing the number of metrics
2024-09-04 11:06:54 -07:00
Konstantin LebedevandGitHub a7fc723ae0 chore: add status code for request_total metrics (#5188) 2024-01-10 10:05:27 -08:00
SmsS4andGitHub f61490966f Add time to first byte metric for s3 (#4768)
* Add time to first byte metric for s3

* Change second to millisecond
2023-08-21 00:42:39 -07:00
Konstantin LebedevandGitHub ec46a34f33 avoid increasing the coordinates of metric labels (#3785) 2022-10-04 01:20:01 -07:00
chrislu 26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
zzq09494 62f74f5e3c add bucket label to s3 prometheus metrics 2022-06-16 13:21:25 +08:00
Chris Lu 59d1435d65 s3: avoid possible attacks by version 2021-11-08 17:47:56 -08:00
Konstantin Lebedev 68463e92c1 add status code in S3RequestCounter 2020-10-01 00:59:39 +05:00
Chris Lu 9a3b564508 adjust printout 2020-09-20 16:01:56 -07:00
Chris Lu 62563a895a refactoring 2020-09-20 16:00:01 -07:00
Chris Lu 23e9ede068 s3: collect metrics 2020-09-18 00:09:04 -07:00