On small screens the sidebar now collapses out of view by default,
replaced by a visible toggle button that slides it back in. Without
this, the sidebar occupied the full screen width on phones and tablets,
leaving no room for page content.
Co-authored-by: Ben McClelland <ben.mcclelland@versity.com>
New cli options added:
webui-gateways - override auto-detected S3 gateway URLs for WebUI
webui-admin-gateways - override auto-detected admin gateway URLs
for WebUI
These also accept env vars VGW_WEBUI_GATEWAYS and
VGW_WEBUI_ADMIN_GATEWAYS for the options.
When setting these, this will override the url auto-detection for
the webui service urls dropdown options. By default, the gateway
auto-detects URLs based on the configured port settings. Use these
options to specify custom URLs when the auto-detected values are
incorrect (e.g., when running behind a reverse proxy or load
balancer). Multiple URLs can be specified with repeated options
or a comma-separated list with the environment variables.
for example:
--webui-gateways https://s3.example.com \
--webui-gateways http://192.168.1.100:7070
or
VGW_WEBUI_GATEWAYS=https://s3.example.com,http://192.168.1.100:7070
The gateway will validate the provided URLs with warnings for any
invalid URL specified. The gateway will terminate if these options
are set but contain no valid URLs.
Also added sorting to the auto-detected URLs so that localhost
URLs will be last in the list, since these will not likely work
on remote systems. The specified lists when provided are left
in the order they are specified to allow admins to determine
dropdown list ordering.
Fixes#1851
Fixes#1869
Generally, when object ownership is not explicitly specified during bucket creation, it defaults to `BucketOwnerEnforced`. With `BucketOwnerEnforced`, ACLs are disabled and any attempt to set one results in an `InvalidBucketAclWithObjectOwnership` error.
However, there is an edge case. When the `private` canned ACL is used during bucket creation—which is effectively the default ACL for all buckets—`BucketOwnerEnforced` is still permitted. Moreover, if no explicit object ownership is specified together with the `private` canned ACL, the ownership defaults to `BucketOwnerPreferred`.
This fix also resolves the issue with rclone bucket creation, since rclone sends `x-amz-acl: private` by default:
```
rclone mkdir vgw:test
```
This allows specifying the following options more than once:
port, admin-port, webui
or using a comma-separated list for the env vars:
e.g., VGW_PORT=:7070,:8080,localhost:9090
This will also expand multiple interfaces from hostnames, for example
"localhost" in this case would resolve to both IPv4 and IPv6 interfaces:
localhost has address 127.0.0.1
localhost has IPv6 address ::1
This updates the banner to reflect all of the listening interfaces/ports,
and starts the service listener on all requested interfaces/ports.
Fixes#1761
Fixes#1849
If no `Content-Type` is provided during object upload, S3 defaults it to `application/octet-stream`. This behavior was missing in the gateway, causing backends to persist an empty `Content-Type`, which Fiber then overrides with its default `text/plain`. The behavior has now been corrected for the object upload operations: `PutObject`, `CreateMultipartUpload`, and `CopyObject`.
Closes#1815
Implements posix actions concurrency limiter. Since posix actions perform filesystem-heavy syscalls, a semaphore-based limiter is introduced to cap the maximum number of concurrent posix actions. When the limit is reached, additional action calls block until a slot becomes available.
For internal posix calls, the `no_acquire_slot` context key is used to prevent acquiring the limiter multiple times within a single action (e.g., PutObject internally calling PutObjectLegalHold).
The posix concurrency limit can be configured via the gateway posix subcommand flag (--concurrency) or the environment variable `VGW_POSIX_CONCURRENCY`. The default value is `5000`.
This is part of the thread exhaustion issue (#1815).
This PR introduces:
* A **maximum Fiber HTTP connections limit**
* A middleware that enforces a **hard limit on in-flight HTTP requests**
When the in-flight request limit is reached, the middleware returns an **S3-compatible `503 SlowDown`** error.
The same mechanism is implemented for the **admin server** (both max connections and max in-flight requests).
All limits are configurable via **CLI flags** and **environment variables**, for both the `s3api` server and the `admin` server.
---
| Setting | CLI Flag | Alias | Environment Variable | Default |
| --------------- | ------------------- | ----- | --------------------- | ------- |
| Max Connections | `--max-connections` | `-mc` | `VGW_MAX_CONNECTIONS` | 250000 |
| Max Requests | `--max-requests` | `-mr` | `VGW_MAX_REQUESTS` | 100000 |
---
| Setting | CLI Flag | Alias | Environment Variable | Default |
| --------------- | ------------------------- | ------ | --------------------------- | ------- |
| Max Connections | `--admin-max-connections` | `-amc` | `VGW_ADMIN_MAX_CONNECTIONS` | 250000 |
| Max Requests | `--admin-max-requests` | `-amr` | `VGW_ADMIN_MAX_REQUESTS` | 100000 |
The system.yml file was giving this warning:
Context access might be invalid: SAFE_RUN_SET
The warning occurs because this was trying to access env.SAFE_RUN_SET
in a with: key of an action, but GitHub Actions has restrictions on where
context variables can be accessed.
The env context isn't always guaranteed to be available in the with: key
of actions, especially when it depends on runtime values set in previous
steps.
The recommended fix is to change from $GITHUB_ENV to $GITHUB_OUTPUT
for this case.
Fixes#1852Fixes#1821
Fiber used to return the `text/plain` default `Content-Type` for error responses, because it wasn't explicitly set. Now for all error responses the `application/xml` content type is set.
* Update client.go to support anonymous S3 access
* Update s3.go to make access and secret parameters optional
* Update example.conf for more clear S3 access and secret usage
Fixes#1836
The latest version of the go sdk has deprecate the s3 manager in
favor of the new transfermanager. This updates to the new
transfermanager functionality for the two places we were using
the manager, iam s3 and integration tests.
This also removes the pinning of nats nkeys since they have fixed
the tags in their repo now. And general cleanup of the go.mod.
Fixes#1835
If-Match in DeleteObject is a precondition header that compares the client-provided ETag with the server-side ETag before deleting the object. Previously, the comparison failed when the client sent an unquoted ETag, because server ETags are stored with quotes. The implementation now trims quotes from both the input ETag and the server ETag before comparison to avoid mismatches. Both quoted and unquoted ETags are valid according to S3.
CopyObject was failing with NoSuchKey when source keys contained special
characters like {} or spaces. The X-Amz-Copy-Source header is URL-encoded
by clients, but ParseCopySource wasn't decoding before filesystem access.
Added url.QueryUnescape() to properly decode bucket and object names,
fixing copy operations for keys with special characters.
Fixing this also uncovered an errors with azure blob url encoding with
similar special character handling. Added this fix in for the integration
tests to pass.
Fixes#1832Fixes#1637
