Bucket policy `Principal` named callers by access key id. Under the standalone IAM service it now names them by AWS-style ARN, as real S3 does: a user ARN, a role ARN (covering every session of that role), an assumed-role ARN (covering one session), the account root ARN or bare account id, or `*`. Every other IAM backend has no ARNs to name anything by and keeps access-key principals unchanged, gated on a new `auth.PrincipalResolver` capability interface that only the standalone client implements.
`auth.Account` carries `Arn` and `RoleArn`, filled at authentication time, so a session can be matched against both its own ARN and its role's. Principals are validated at PutBucketPolicy time through a new `/private/resolve-principals` endpoint, which rejects anything that does not name a live identity with `MalformedPolicy: Invalid principal in policy`.
An `Allow` naming the account root ARN or bare account id delegates to the account's own IAM rather than granting on its own, while a `Deny` naming it denies every principal in the account outright. Denial messages now name the caller by ARN wherever one exists.
Also fixes `aws:PrincipalArn` for assumed-role sessions, which reported the session ARN where AWS reports the role's, and stops an unreachable IAM service being reported as a malformed policy.
AWS SigV4 signatures are attacker-controlled inputs compared against
server-computed HMAC values. Ordinary string comparison exits at the first
differing byte, which can expose the length of the matching prefix through
response timing and, in principle, enable signature forgery for a fixed request
after many probes.
Use the shared sigv4auth.SecureCompare helper for browser POST-policy signatures
and streaming chunk and trailer signatures. The helper preserves the existing
accept/reject behavior, including rejecting malformed or different-length
signatures, while using crypto/subtle.ConstantTimeCompare for equal-length
values.
The access-log and admin-access-log options now accept stdout, stderr, or - for
stdout in addition to file paths. Standard stream destinations are kept open
during shutdown and SIGHUP handling, while file destinations continue to support
normal reopen behavior for log rotation. CLI help, embedded config comments, and
the example configuration describe the new destination values.
Fixes#2245
The multipart upload temp directory was created with a hard-coded
0755 mode instead of the configured --dir-perms value, unlike every
other directory creation path in this backend. This caused
inconsistent permissions below bucket directories.
Fixes#2266
* rdma: add the hipobj-rc-v2 control routes to the vgwrdma gateway
Mount the three control routes (prepare, ready, cancel) on the
S3 port behind the standard SigV4 middleware. The routes own
authentication-adjacent policy the C server cannot see: the
middleware wrapper yields to the handler on success, READY and
CANCEL re-read the account through the IAM cache bypass so
mid-flow deletions and credential rotations take effect
immediately, and every object access re-authorizes against the
decoded bucket and key.
The READY handler implements the session ownership contract:
the completion-reference finalizer installs only after the
transfer claim succeeds, the PUT path hands the reference to
the put view exactly at the borrow point, and the FINAL reply
carries the stored object's metadata. Backend I/O runs under a
context merged with the RC service context so shutdown unblocks
in-flight handlers, with a bounded pool for the fresh IAM
lookups.
vgwrdma starts the session server alongside the gateway when an
RDMA interface is configured, tears it down on exit, and shuts
the IAM service down on any startup failure. embedgw learns the
readonly flag for the object access checks the routes share.
Signed-off-by: Jihyeon Gim <potatogim@potatogim.net>
* rdma: add the missing stub handlers for non-Linux builds
The non-Linux rcroutes stub exposed only Register while the vgwrdma
gateway registers the prepare/ready/cancel handlers directly, so
cross-compiling cmd/vgwrdma failed with undefined methods. Add the
three stub handlers answering 501 Not Implemented and let Register
reuse them, matching the Linux Handler API surface.
* auth: drop the duplicated GetUserAccountFresh definition
The rebase onto main (which already carries GetUserAccountFresh from
the iam-cache-fresh change) kept both copies of the method, breaking
the build with a redeclaration error. Remove the second copy so the
method is defined once.
* rdma: address the review findings on the control route wiring
Drop the unused Handler.Register from both build variants: the
gateway mounts the three control routes through s3api.WithRoute so
the SigV4 verifier wrapper (rcAuth) runs in front of each handler,
and nothing else calls Register.
Clear iamOwned only when RunVersityGW returns nil. It shuts the IAM
service down itself at the end of its shutdown sequence, but its
early failure paths return before reaching that point, so the
deferred shutdown must keep covering those errors.
Remove the unused rcserver.SessionInfo parameter from sizeOf; the
transferred byte count comes from the READY response alone.
* rdma: keep transient IAM failures retryable in the fresh revalidation
The fresh account revalidation turned every GetUserAccountFresh
error into 403, which reports transient backend failures (LDAP
timeouts, network errors) as a revoked account and leaves the
client no room to retry. Only a confirmed missing account
(auth.ErrNoSuchUser) means that; answer anything else with 503 so
clients can retry the request.
* rdma: make the IAM shutdown exactly-once and keep gateway errors visible
The gateway and RunVersityGW share the IAM service, and which side
shut it down could not be told from the return value: runtime
failures return after RunVersityGW already shut the service down,
while early setup failures return before any shutdown happens. The
iamOwned flag therefore either shut the service down twice or leaked
it depending on the error, and the error itself was dropped.
Wrap the service so Shutdown runs exactly once no matter which side
calls it, keep the deferred shutdown for every early failure path,
and return the gateway error again. The wrapper re-exposes the
optional interfaces (fresh account reads, signing keys, policy
evaluation, fixed bucket ownership) so feature detection through the
IAM service keeps working.
* rdma: reuse the SigV4 account for RC control requests
READY and CANCEL are independently authenticated SigV4 requests.
Use the account resolved by the normal SigV4 path instead of
bypassing the IAM cache a second time. This aligns RC revocation
latency with other signed S3 requests and removes the extra
backend IAM lookup, its concurrency cap, and the RC-specific IAM
error mapping. The session owner check and the READY target and
operation authorization are unchanged.
* rdma: reword the READY reauthorization comment
The comment implied a revocation inside the session window always
takes effect at READY, but the account used here is the one SigV4
resolved, which may be a cached entry. State what the check does
without claiming account-cache freshness.
* rdma: preserve IAM cache behavior and standalone region
---------
Signed-off-by: Jihyeon Gim <potatogim@potatogim.net>
Co-authored-by: Ben McClelland <ben.mcclelland@versity.com>
The `UnsignedStreamingPayloadTrailer_invalid_chunk_size` integration test failed intermittently in CI with `write: connection reset by peer` instead of the expected `InvalidChunkSizeError`, and needed a rerun to pass. Its tenth case sends a 36KB `aws-chunked` payload that the chunk reader rejects roughly 8KB in, so the gateway answered and closed the connection while the client was still writing the remaining 28KB.
fasthttp streams request bodies (`StreamRequestBody`) and never drains what a handler leaves behind: after the handler returns it only calls `releaseRequestStream`, and with `DisableKeepalive` it breaks out of the serve loop and closes the socket with unread bytes still queued, so the kernel answers the client's in-flight writes with an RST. Go's `net/http` transport prefers a request body write error over an already received response, so on the losing side of that race the client never sees the S3 error XML at all. The same exposure applies to every early rejection, not just chunk framing: signature failures, missing buckets and policy denials are all decided before the payload is read.
The new `DrainRequestBody` middleware reads and discards whatever is left of the body once the handler chain is done with it, so the client can finish its write and read the real error. It is registered right after the panic recovery middleware and before every route, so it wraps all of them. The drain is capped at 256KB, matching net/http's `maxPostHandlerReadBytes`, so a rejected multi-gigabyte upload is not streamed through the gateway just to be thrown away, and it is bounded by a one second idle deadline and a five second total deadline so a client that stops sending cannot pin a worker. Bodies with more than 256KB still unread are deliberately left alone, and those clients can still see a reset.
Only `Content-Length` framed bodies are drained. fasthttp's `requestStream` reports EOF idempotently for those, but for a chunked body it goes back to the socket for another chunk header past the terminating chunk, so re-reading one the handler had already finished would block until the deadline and hold a successful response back with it.
This also fixes a second problem found while testing the first. With `--keep-alive`, nothing in fasthttp sets `connectionClose` when a streamed body is left unread, and the same `bufio.Reader` is reused for the next request on that connection, so leftover upload bytes were parsed and served as a separate HTTP request: a `GET` placed inside a `PUT` body was routed, answered and written to the access log. The middleware now sets `Connection: close` whenever the drain does not reach EOF, so a connection that may still hold body bytes is never reused. `--keep-alive` is off by default and is not set by any shipped deployment artifact.
`CreateAccessKey`, `UpdateAccessKey`, `DeleteAccessKey` and `ListAccessKeys` required an explicit `UserName`. Real IAM treats it as optional and resolves it from the access key signing the request, so an IAM user managing its own keys had to name itself. They now infer it, matching AWS: only an entirely absent parameter is inferred, while a present-but-empty one stays a `ValidationError`, and the inferred scope is strictly the caller's own user — another user's key id returns `NoSuchEntity` rather than being touched.
A caller with no IAM user of its own gets IAM's own `Must specify userName when calling with non-User credentials` `ValidationError`, shared with `GetUser` as `iamerr.MustSpecifyUserName`. That covers assumed-role sessions and also the gateway's root credential, which is configured rather than stored as an IAM user and so owns no access keys the API could manage — real IAM manages the root account's own keys here, which has no equivalent in this gateway.
The policy middleware resolves the same four actions through `callerOrNamedUserResource`, so the resource-level check targets the caller's own user ARN when UserName is omitted instead of falling back to no resource at all, which would have denied every request authorized by an own-ARN-scoped grant.
Role last-used tracking was missing entirely - `GetRole` returned a `RoleLastUsed` element that nothing ever wrote, rendering the zero time instead of the empty element AWS returns for an unused role - and access key last-used only ever saw the `IAM`/`STS` control plane, so a credential used exclusively against the S3 gateway reported as never used. Roles now record a use whenever a request authenticates with one of their session credentials, through a new `Storer.RecordRoleUsage` mirroring `RecordAccessKeyUsage`, gated on the session's role still being the one it was minted against so a session outliving its role can't attribute its use to a same-named replacement. `LastUsedDate` became a `*time.Time` so an unused role renders as an empty element.
Both records now cover the S3 data plane as well: the gateway sends its configured region and `s3` on evaluate-policy and the IAM service records the caller there, so `GetAccessKeyLastUsed's` `ServiceName` is now iam, sts or s3. That call was chosen over derive-signing-key, which runs before signature verification and takes its region and service from the caller's own `Authorization` header - recording there would let anyone who knows an access key id refresh and poison another identity's audit record. Requests denied by a bucket policy or made against a public bucket are not recorded, since neither reaches identity-policy evaluation. To keep per-request recording affordable, an update is skipped while the stored record has the same service and region and is under a minute old; a change of either is written through immediately.
Assuming a role is not a use, a request denied by an identity policy is, and both successful and denied S3 requests update the record. Also moves the `OIDC-dependent` tests into the `s3-iam-session` group so runoidctests.sh runs a single group.
objectsAccessErrors recorded the first resource-policy Deny and returned immediately, leaving every later key in the same action subset with a nil result — which VerifyObjectsAccess caller reads as "authorized" and sends straight to the backend. An explicit bucket-policy Deny therefore let the keys after it skip authorization entirely and be deleted, including keys that same bucket policy explicitly denied.
The loop now continues rather than returning, so every denied key is settled with its own error. The identity-policy round trip the early return was saving is still skipped, but only when the bucket policy denied every key in the batch, since no identity-policy answer could change any result then. Both loops that follow skip keys already holding an error, so a resource-level explicit deny is never overwritten by an identity-policy result nor flattened to the generic AccessDenied message.
Fail closed on unexpected advisory-lock errors instead of silently reducing
cross-process exclusion to a local mutex. Make local publish-slot waits honor
request cancellation. Move version snapshots under the publish lock so
concurrent versioned PUTs preserve publication order. Add regression coverage
for canceled lock waiters.
Also add an option to disable flock files and only rely on in process locking.
Bind the RC session server archive through cgo: Init opens the
verbs device with the resource limits, and the RCSvc wrapper
carries the admission gate handlers use around every session
call (TryEnter/Leave), a service-lifetime context that Close
cancels so in-flight handler I/O unblocks during shutdown, and
an idempotent Close that marks every session for reaping, waits
for admitted calls to drain, and destroys the server.
The rest of the surface maps the C ABI one-to-one: prepare,
ready (with the transfer outcome returned atomically in the
reply), staging borrow/finish, the put-view handoff, session
introspection, and cancel. Non-linux or non-cgo builds compile
against a stub so the package is portable.
Nothing imports the package yet; the gateway integration that
links and exercises it follows.
Signed-off-by: Jihyeon Gim <potatogim@potatogim.net>
Wrap the v2 session core with the server-side C ABI the gateway
binds to: prepare/ready/cancel session calls, staging leases for
GET side-loading, the put-view handoff for PUT commits, session
introspection, and the server lifecycle.
The ABI owns the parts that must be shared across sessions: the
verbs device handle, global and per-principal resource limits,
session accounting with consume-once handles keyed by epoch and
nonce, the completion reference that pins a session from the
READY claim until its finalizer, and the reaper that tears
transport objects down once every reference drains. A background
thread expires sessions past their prepare or execute deadlines
so abandoned sessions cannot pin the limits, and teardown
failures keep the affected verbs objects and device alive rather
than freeing memory the NIC may still reference.
A peer-busy READY rolls the claim back and re-arms the QP
through RESET so the client can retry the same session. The
vgwrdma target now links the archive built from these sources.
Signed-off-by: Jihyeon Gim <potatogim@potatogim.net>
With the posix backend running --chuid/--chgid against the standalone IAM service, CreateBucket failed for every bucket name and left a half-created directory behind. Bucket ownership is fixed to the gateway's root account there, and that account was constructed from the root credentials alone, so its UserID/GroupID stayed at zero and the gateway tried to chown each new bucket to uid/gid 0 - something a process that is not root can never do. Root-account object writes failed the same way, because the identity the S3 request path uses for root also comes from the root credentials and never from the IAM backend. On top of that, the failed chown returned before the acl xattr was written, so the leftover directory made every later request for that name fail with "get bucket acl: no such key" until it was removed by hand.
The standalone IAM client now reports the root account with UserID, GroupID and ProjectID taken from --iam-standalone-default-uid, -gid and -project-id, returning a copy so the stored root account keeps the credentials it is compared against. ResolveDerivedKey copies that same identity onto root when the IAM backend fixes bucket ownership to the root access key, which keeps root's own writes consistent with the buckets root owns. CreateBucket now removes the bucket directory, its sidecar attributes and its versioning directory on any failure after the mkdir, so a failed create leaves nothing behind and the name stays retryable. A chown EPERM reports the target uid/gid, the flags that asked for it and the process euid/egid instead of a bare "operation not permitted", and the posix backend warns at startup when chuid/chgid are set on an unprivileged gateway.
The built-in IAM backends do not fix bucket ownership, so root and every other account reach the storage backend exactly as before.
S3 explicitly documents this action as owner-only ("To retrieve the versioning state of a bucket, you must be the bucket owner." — https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketVersioning.html), and the handler enforced that with an extra auth.IsAdminOrOwner check on top of VerifyAccess. Real S3 behaves differently: verified against AWS that a bucket policy explicitly denying s3:GetBucketVersioning denies the bucket owner itself, and that an Allow grants the action to a principal that doesn't own the bucket. It goes through ordinary bucket policy/ACL evaluation like any other bucket subresource read, which is what the write side, PutBucketVersioning, already did here. Removes the extra check along with auth.IsAdminOrOwner, which had no other call site.
Port the hipObject v2 reliable-connection session core into
cuwrapper/rc: the session table and state machine, the wire
codec for the hipobj-rc-v2 headers, request parsing, the
injectable clock and randomness sources, the transport layer
(QP/CQ lifecycle, RTR/RTS transitions, staging registration),
the data phase (RDMA write with immediate for GET, receive with
immediate for PUT), the RDMA token codec, and the dynamically
loaded ibverbs shim (ibv-core.h plus the dlopen host binding).
The sources are a port of the upstream hipObject v2 core, kept
close to the original so the two trees can be diffed during
review. Nothing links against them yet; a Makefile rule builds
the objects into rdma/librcserver.a for the ABI layer that
follows.
Signed-off-by: Jihyeon Gim <potatogim@potatogim.net>
The cached GetUserAccount keeps serving a stale entry for the
cache TTL after the backing IAM service changes, which delays
credential revocation by up to the configured expiry.
Add GetUserAccountFresh to the IAM cache, which reads directly
from the underlying service and refreshes the cached entry with
the result. Callers that need revocations to take effect
immediately can use it instead of the cached path.
Signed-off-by: Jihyeon Gim <potatogim@potatogim.net>
Ben McClellandandGitHub
niksis02
Luke McCrone
Andrei BesleagaandBen McClelland
dependabot[bot]andGitHub