signedAdminRequest takes the signing region and a body, and always names
root in X-Vgw-Owner (admin create-bucket requires it, the other admin
routes ignore it), so the create-bucket test reuses it instead of its own
copy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CreateBucket was sent with an empty body, so on a gateway with any region
other than us-east-1 both the bucket dialog and the explorer failed with
IllegalLocationConstraintException. Send a CreateBucketConfiguration
naming the signing region, which the gateway requires to be its own, for
the S3 and the admin create-bucket call alike.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The admin server never ran SetDefaultValues, so the request context had no
region and CreateBucket validated the LocationConstraint of an admin
create-bucket against the us-east-1 fallback, while the signature had to
use the gateway region. On a gateway outside us-east-1 the standalone admin
port therefore rejected the correct LocationConstraint and accepted a
missing one, the opposite of the same route on the s3 port.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GitHub has no FreeBSD runners, so the new `functional-freebsd.yml` workflow runs the `runtests.sh` inside a FreeBSD 15.1 VM. The VM is booted under KVM on `ubuntu-latest` by `vmactions/freebsd-vm`, the most widely used action for this.
The `prepare` step installs `bash`, `git` and `gmake`, downloads the latest stable Go from `go.dev` to match `setup-go`'s `stable`, and creates a `tester` user that owns the synced workspace. `git` is required because the synced `.git` directory makes `go build` stamp VCS info. The `run` step then runs `gmake testbin`, `runtests.sh` and the coverage report as `tester`, matching the unprivileged Linux job.
FreeBSD enables ASLR by default (`kern.elf64.aslr.enable=1`), and the race-enabled `versitygw` built by `testbin` refuses to start under it with "This sanitizer is not compatible with enabled ASLR", so `prepare` turns ASLR off with `sysctl`.
The nested VM freezes for up to ~13 seconds at a time under the parallel suite, longer than the few seconds of margin some object-lock tests gave their retain until dates, so a date could expire in transit (the gateway then rejects it with `InvalidArgument`) or a COMPLIANCE lock could expire mid-test. A new `retryOnPastRetainDate` helper resends a request whose seconds-out date expired in transit, `complianceTestRetention` grows from 10s to 30s, and the tests that locked for twice that window, or extended by a whole one, now stay within a single window, which keeps the added suite time to about 10 seconds per run.
Fixes#2509
Concurrent `CompleteMultipartUpload` calls for the same upload ID could return a 500 `InternalError` on FreeBSD, logging `get object tagging: xattr.get ...inprogress: no such file or directory`. The posix backend claims an upload by renaming `<uploadId>` to `<uploadId>.<etag>.inprogress` and relied on only one concurrent `rename` succeeding. FreeBSD's `kern_renameat` looks up the source and the target in separate unlocked `namei` calls and returns success without doing anything when both resolve to the same vnode. A call that found `<uploadId>` before a sibling renamed it, and the `.inprogress` directory after, therefore claimed the upload as well. Both callers assembled the object, and the first to finish removed the `.inprogress` directory while the second still had to read its attributes. Linux holds the parent directory lock across both lookups, so the race never showed there.
The claim rename now runs under the object's publish lock (`lockObjectPublish`), released right after the rename. Claiming renames of the same upload can no longer overlap: within a process in every `--object-lock-mode`, and across processes with `flock` or `fcntl`. Assembling the parts still happens outside the lock.
Follow-up to #2514, which added PosixOpts.ExcludeDirs (--exclude-dirs,
VGW_EXCLUDE_DIRS) so the listings can skip filesystem control
directories such as ZFS's .zfs.
The walks matched skipped names against every directory entry, files
included. In WalkVersions, returning fs.SkipDir for a file skips the
rest of its parent directory, so a regular file named like an excluded
directory (for example an object "d/.zfs") made ListObjectVersions drop
every sibling that sorted after it. Walk hid the file itself.
Previously only .sgwtmp was skipped, which clients can't create, but
excluded names are chosen by the operator and any client can PUT such a
key. Walk and WalkVersions now skip only directories, so a file with an
excluded name is listed like any other object.
Excluded names were also not reserved: requests could still read and
write objects under an excluded directory that the listings never show.
On ZFS, .zfs is reachable by name even with snapdir=hidden, so GetObject
could read snapshot contents. That bypasses bucket policies scoped to a
key prefix and exposes data that was deleted or overwritten since the
snapshot was taken. On other filesystems, objects written there were
orphans that no listing reports. validateObjectName now rejects, with
ReservedObjectKey, any key that has an excluded name as a directory at
any depth, ignoring case and after resolving "..", "." and repeated
slashes. Every object operation already calls it. A directory-object
key (trailing slash) counts its last element as a directory; a plain
key with an excluded last element is still a valid file. The new
ErrExcludedObjectKey keeps the ReservedObjectKey code with a description
that fits excluded directories, leaving the .sgwtmp error unchanged.
Nothing changes when ExcludeDirs is empty.
posix.New now also rejects "." and ".." as excluded names, since they
can never match a directory entry and would silently do nothing.
Follow-up to #2486
The AWS SDKs and CLI send `ObjectIdentifier.LastModifiedTime` as an HTTP date, but `s3response.DeleteObjects` decoded it into the SDK's `time.Time` field, which accepts only RFC 3339. Any `DeleteObjects` request that carried a `LastModifiedTime` was therefore rejected as a whole with `InvalidRequest`, and the RFC 3339 form the gateway did accept is one that S3 rejects.
`DeleteObjects` now has its own `UnmarshalXML`. It embeds `types.ObjectIdentifier`, decodes `LastModifiedTime` as a string and parses it with `smithytime.ParseHTTPDate`, falling back to `time.RFC1123Z` for numeric zone offsets. This accepts IMF-fixdate, RFC 850, ANSIC and `+hhmm` offsets and rejects RFC 3339, which matches S3 directory buckets, the only bucket type where S3 evaluates this field.
Errors are reported as S3 reports them. An unparseable date returns `InvalidRequest` with "Invalid value for date attribute: <value>" through the new `s3err.GetInvalidDateAttributeErr`, which echoes the raw value of the first bad date in the body. An empty, whitespace-only or self-closing `LastModifiedTime` returns `MalformedXML`, which takes precedence over an invalid date in any object. The `DeleteObjects` controller now passes S3 errors from the decoder through instead of turning every decode failure into `InvalidRequest`.
fiber `v3.6.0` (gofiber/fiber#4695) normalizes every request path before routing as RFC 3986 Section 6.2.2 describes: it decodes the escapes of unreserved characters and removes `.` and `..` segments, and `ctx.Path()`, `ctx.Params()` and every `ctx.Path(override)` return the normalized path with no way to opt out. `DecodeURL` sets the decoded path back through `ctx.Path()`, so keys were decoded twice (`100%2541` was stored as `100A`, `q%7euser` as `q~user`) and dot segments rerouted requests: `PUT /bucket/.` became `CreateBucket` and `/bucket/../../etc/passwd` named the bucket `etc`.
The new `httpctx.Ctx`, created with `fiber.NewWithCustomCtx`, keeps a path set with `Path()` as it is and has fiber route a copy in which every `%` and the dots of a `.` or `..` segment are `_`. The copy keeps the length of the path, since fiber matches routes on it but takes the route parameters from `Path()` at the matched offsets, so `Params("bucket")` holds the real bucket. The S3, admin and website servers and the RDMA ops tracker use it.
S3 stores keys holding `.` and `..` segments as they are (`a/../b`, `x/.` and `..` are objects of their own) and only rejects, before authentication, a request path whose decoded `..` segments climb above the root, separating the segments at `\` as well as at `/` and skipping the empty ones. `DecodeURL` now rejects such a path with `InvalidURI` and keeps the dot segments in the key otherwise.
`Backend.StoresDotSegmentKeys()` reports whether a backend stores such keys as their own objects: `azure` and `s3proxy` do, while `posix` and the `BackendUnsupported` default don't, since a filesystem resolves the segments to another object or to a path outside the bucket. `IsObjectNameValid` takes the backend and rejects keys holding dot segments for those backends in the request path, the `DeleteObjects` keys, the `PostObject` key and the copy sources, so `posix` now also rejects `a/../b` and `a/./b`, which it stored as `b` and `a/b`. The unused `path.go` is removed.
Let captureHTTPClient answer with a configurable header and body, and
use it for the GetObjectAttributes tests instead of a separate stub
client that repeated the same client setup.
Fixes#2525
`CompleteMultipartUpload` returned the stored part's ETag in `InvalidPart` errors instead of the ETag the client sent: posix on an ETag mismatch, and azure on a zero-byte part mismatch, where it returned the expected block ID. Azure also returned an empty ETag for a part that was never uploaded. All three now return the client's ETag. Like S3, `GetInvalidPartErr` removes every `"` from it, inner quotes included, and keeps surrounding whitespace.
`SetResponseHeaders` turns off header name normalization and leaves it off, so response header names stay lowercase. Before it runs, the `RequestIDs` middleware stored `X-Amz-Request-Id` and `X-Amz-Id-2`. Afterwards, the `EnsureRequestIDs` call in `ProcessController` stored `x-amz-request-id` and `x-amz-id-2`. Every response with a headers map, success responses included, therefore sent both pairs. `EnsureRequestIDs`, `RequestID` and `HostID` now use `SetCanonical` and `PeekCanonical` with the lowercase names, so the normalization setting doesn't affect them, and each header is sent once in lowercase as S3 does.
The previous revision hard-coded .zfs next to .sgwtmp in the
ListObjects, ListObjectsV2 and ListObjectVersions walks. That hides a
directory named .zfs on filesystems that don't have ZFS snapshots, and
it gives an operator no way to exclude a different control directory.
Add PosixOpts.ExcludeDirs and build the effective skip list once in New
from the gateway's own metadata directory plus the configured entries,
so all three listing operations read the same list. The new
--exclude-dirs flag (VGW_EXCLUDE_DIRS) exposes it, so a dataset mounted
with snapdir=visible is handled with --exclude-dirs .zfs.
.zfs isn't in the default list: on a filesystem without ZFS snapshots a
directory with that name is ordinary bucket content. An operator who
needs the skip sets --exclude-dirs .zfs.
The test was skipped for the gateway with a link to #925, which is fixed
here. The gateway already returns InvalidArgument for an unknown
attribute, so the test passes, and the skip would point to a closed
issue once #925 is closed.
S3Proxy.GetObjectAttributes dropped VersionId and DeleteMarker from the
upstream response, so the gateway never sent x-amz-version-id or
x-amz-delete-marker for objects behind an s3 backend. Copy them into the
response, and add tests for how the upstream response is mapped,
including the parts count.
ListObjects and ListObjectsV2 ignored the encoding-type query parameter.
Any value was accepted, and with encoding-type=url the response carried
neither an EncodingType element nor encoded key names.
S3 accepts only "url" and answers anything else with InvalidArgument
("Invalid Encoding Method specified in Request", argument encoding-type).
With "url" it sets EncodingType and encodes the key names in the
listing as a query component, "a+ b.txt" becoming "a%2B+b.txt", with
"/" left as is: Key, Prefix, Delimiter and CommonPrefixes in both
operations, Marker and NextMarker in ListObjects, and StartAfter in
ListObjectsV2. The continuation tokens are opaque and are not encoded.
The encoding is applied to a copy of the backend result in the
controllers, so it covers every backend.
Fixes#1985Fixes#1984
Behind a reverse proxy the request log and the S3, admin, website and WebUI
access logs record the proxy's address instead of the client's. Add a
client-ip-header option that takes the logged address from X-Forwarded-For or
X-Real-Ip. It is off by default, so an existing deployment logs exactly what it
logged before.
The header is only honoured when the connecting peer is listed in the new
trusted-proxies list of IPs and CIDRs. Without that, the value is whatever the
client sent: nginx's standard
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for appends to a
client-supplied header, leaving the client's own address leftmost.
X-Forwarded-For is walked right to left, skipping the listed proxies, so the
first unlisted address is the client and a forged leftmost entry is never
reached; the two-proxy case from #2243 works once both proxies are listed.
Every header line is walked (a proxy may add its own line instead of
appending), ip:port entries are accepted, and a request from an unlisted peer
keeps the socket address. Setting the header with an empty trusted-proxies list
fails startup rather than silently trusting every peer.
X-Real-Ip reads the last header line, so a proxy that adds a line rather than
replacing the client's is the one that speaks. An address carrying an IPv6 zone
is rejected: the zone is free text that the space-delimited access log would
write verbatim, shifting every field after RemoteIP. When every hop in
X-Forwarded-For is a trusted proxy, the leftmost hop is logged, as nginx's
real_ip_recursive does, since the socket peer is then the nearest proxy. The
trusted-proxies list accepts the entry "unix" to trust the peer of a listener
bound to a unix socket path, which fasthttp reports as 0.0.0.0.
The resolver, middleware, log tag and parsers live in internal/clientip, shared
by s3api, iamapi, website and webui; s3api/utils re-exports them for the S3
callers, so the standalone IAM service no longer pulls in the S3 side. On this
branch go list -deps ./iamapi drops from 375 to 299 packages, against 298 on
main.
ctx.IP() is unchanged and still returns the socket peer address, because it
feeds aws:SourceIp in IAM policy conditions and SourceIP in policy evaluation;
only the address written to the logs and S3 event records comes from the
header.
Fixes#2243.
Fixes#2511
`CreateMultipartUpload` in the posix backend creates the upload directory under `.sgwtmp/multipart` with `os.MkdirAll` and returned any failure wrapped as `create upload temp dir`, so a full filesystem quota or a full disk came back as a 500 `InternalError` (which SDKs also retry) instead of the error the other write paths return. `PutObject`, `UploadPart`, `UploadPartCopy` and `CompleteMultipartUpload` already map these errors.
`syscall.EDQUOT` from the directory creation now returns `QuotaExceeded` (403) and `syscall.ENOSPC` returns `InsufficientStorage` (507), the same mapping the rest of the posix write paths use.
The posix backend stages multipart uploads and temporary files inside the bucket directory under `.sgwtmp`, but nothing stopped an object key from resolving there. A client with access to the bucket could read the parts of other clients' in-progress uploads with `GetObject`/`HeadObject`, copy them out with `CopyObject`, overwrite or delete them with `PutObject`/`DeleteObject` so the upload failed on completion with `InvalidPart`, and store data that never shows up in listings. Spellings such as `./.sgwtmp/...`, `x/../.sgwtmp/...`, a leading `/` and, on case-insensitive file systems, `.SGWTMP/...` reached the same files.
Add the non-AWS `ReservedObjectKey` error (400) and return it whenever an object key resolves to `.sgwtmp` or a path inside it. The key is resolved the same way `ObjectPath` resolves it, and its first path element is compared case-insensitively. The check runs after the bucket existence check in `PutObject`, `GetObject`, `HeadObject`, `GetObjectAttributes`, `DeleteObject`, `DeleteObjects` (reported per key), `CopyObject` (source and destination), the `UploadPartCopy` source, `CreateMultipartUpload`, `CompleteMultipartUpload`, object tagging, legal hold and retention. The exported `ValidateObjectKey` covers the ScoutFS `GetObject` and `RestoreObject` entry points. `UploadPart`, `ListParts` and `AbortMultipartUpload` only hash the key and stay unchanged, so uploads created with a reserved key before this fix can still be aborted.
`ListObjects` and `ListObjectsV2` also listed the staged parts when the prefix named a directory inside `.sgwtmp`, because the prefix fast path in `quickWalk` compared only the last element of the prefix directory against the skipped directories. `inSkipDir` now checks every element, ignoring case.
The controller now passes the requested attributes to the backend in
ObjectAttributes, so posix and azure only do the extra HeadObject for the
parts count when ObjectParts was asked for. This also lets the s3 backend
reach the upstream service, which rejected the request before because the
field was missing.
The s3 backend now copies TotalPartsCount into PartsCount, leaves out
ObjectParts when the upstream response has none, and sets the part number
markers, which were dropped by inverted strconv.Atoi error checks.
Ben McClellandandGitHub
Christian SchmittandClaude Opus 5.5
niksis02
LuisFigueroaG
dependabot[bot]andGitHub
Tung Lam
Javier GarciaandBen McClelland
Luke McCrone