Fixes#2180Fixes#2181
Migrate the gateway from Fiber v2 to Fiber v3.3.0 and update the affected server, middleware, handler, controller, and test code for the new APIs.
Replace the deprecated Fiber filesystem middleware used by the WebUI with the Fiber v3 static middleware, serving the embedded WebUI assets from an fs.Sub filesystem.
Fix the request header limit handling regression by adding a temporary handler for Fiber v3/fasthttp small-buffer errors so oversized request headers return the expected regulated S3 error response.
Fix the debuglogger panic by reworking the boxed key/value formatter used for debug request and response dumps. The formatter now handles long header keys and values without producing invalid wrap widths, negative padding, or out-of-range string slices.
The PATCH /:bucket/create admin route was missing
middlewares.ApplyDefaultCORS, while every other admin PATCH route
applies it. The OPTIONS preflight handler already sets CORS headers,
so browsers pass preflight but block the actual response for lacking
Access-Control-Allow-Origin. This caused the WebUI bucket-creation
flow to fail with ERR_FAILED even though the server returned 201.
Fixes#2105. Introduced in #1739 when the endpoint was added.
Co-Authored-By: Claude <noreply@anthropic.com>
Some SDKs/clients make the assumption that us-east-1 will always
work for bucket listings even if individual buckets are in
different regions. This is how AWS works with allowing full
bucket listing from the default us-east-1 region as well as
any other valid region.
Fixes#1879
Closes#1731
Implements the admin `CreateBucket` (`PATCH /:bucket/create`) endpoint and CLI command, which create a new bucket with the provided owner access key ID. The endpoint internally calls the S3 `CreateBucket` API, storing the new owner information in the request context under the `bucket-owner` key. This value is then retrieved by the S3 API layer and the backends.
The endpoint uses the custom `x-vgw-owner` HTTP header to pass the bucket owner access key ID.
The admin CLI command mirrors `aws s3api create-bucket` and supports all flags implemented by the gateway (for example, `--create-bucket-configuration`, `--acl`, `--object-ownership`, etc.).
There is some desire to have a web dashboard for the gateway. So
that we dont have to proxy all requests through the webserver
and expose credentials over the wire, the better approach would
be to enable CORS headers to allow browser requests directly to
the s3/admin service.
The default for these headers is off, so that they are only
enabled for instances that specfically want to support this
workload.
Closes#1549Fixes#1593Fixes#1521Fixes#1427Fixes#1311Fixes#1301Fixes#1040
This PR primarily focuses on checksum calculation within the gateway, but it also includes several related fixes and improvements.
It introduces a middleware responsible for handling and calculating checksums for the `x-amz-checksum-*` headers and `Content-MD5`. The middleware is applied only to actions that expect a request body or checksum headers. It also enforces validation for actions that require a non-empty request body, returning an error if the body is missing. Similarly, it returns an error for actions where at least one checksum header (`Content-MD5` or `x-amz-checksum-*`) is required but none is provided.
The implementation is based on [https://gist.github.com/niksis02/eec3198f03e561a0998d67af75c648d7](the reference table), tested directly against S3:
It also fixes the error case where the `x-amz-sdk-checksum-algorithm` header is present but no corresponding `x-amz-checksum-*` or `x-amz-trailer` header is included.
Additionally, the PR improves validation for the `x-amz-content-sha256` header. For actions that require this header, an error is now returned when it’s missing. For actions that don’t require it, the middleware no longer enforces its presence. Following the common S3 pattern, the header remains mandatory for admin routes.
Finally, the `x-amz-content-sha256` header is now optional for anonymous requests, as it is not required in that case.
Closes#1525
* Adds validation for the `Content-MD5` header.
* If the header value is invalid, the gateway now returns an `InvalidDigest` error.
* If the value is valid but does not match the payload, it returns a `BadDigest` error.
* Adds integration test cases for `PutBucketCors` with `Content-MD5`.
The debuglogger should be a top level module since we expect
all modules within the project to make use of this. If its
hidden in s3api, then contributors are less likely to make
use of this outside of s3api.
Adjusts the admin apis to the new advanced routing changes.
Enables debug logging for the separate admin server(when a separate server is run for the admin apis).
Adds the quiet mode for the separate admin server.
niksis02
Dylan Dellett-Wion
Ben McClelland
jonaustin09