Once a bucket had any policy, `verifyResourceAccess` evaluated the policy alone and ignored the bucket ACL, so a non-root bucket owner kept only what the policy explicitly allowed. A Deny-only policy (e.g. Deny `*` `s3:GetObject` on one prefix) therefore locked the owner out of every operation, including `DeleteBucketPolicy`, and only root/admin could recover. ACL grants also authorized every bucket operation through the controller's `AclPermission`, so a non-owner with a `WRITE` grant could install a bucket policy that locked out the owner and every other grantee.
The policy and the ACL are now combined per action the way AWS does it: an explicit policy Deny wins, and otherwise the action is allowed by either a policy Allow or an ACL grant. The bucket owner keeps everything a policy doesn't explicitly deny and, like an AWS account root user, can always call `GetBucketPolicy`, `PutBucketPolicy` and `DeleteBucketPolicy` even past a Deny. The protection covers only those three APIs; `GetBucketPolicyStatus` and every other action stay deniable, matching AWS.
ACL grants to non-owners now follow AWS's ACL-to-action mapping through the new `aclActionPermissions` table: `READ` grants `s3:ListBucket`, `s3:ListBucketVersions`, `s3:ListBucketMultipartUploads`, `s3:GetObject`, `s3:GetObjectVersion` and `s3:GetObjectAttributes`; `WRITE` grants `s3:PutObject` and `s3:DeleteObject`; `READ_ACP` and `WRITE_ACP` grant reading and writing the bucket and object ACLs. Bucket configuration, the bucket policy, tagging, object lock settings and `DeleteBucket` are the owner's alone unless a policy grants them. `verifyACL` is replaced by `aclAllows`, which derives the required permission from the action, so `AccessOptions.AclPermission` now only feeds the readonly gate. The anonymous path uses the same table in place of `publicACLAllowedActions`, which leaves its behavior unchanged.
`GetBucketPolicyStatus` also reported a Deny-only `Principal: "*"` policy as public; only Allow statements now count toward `IsPublic`.
CopyObject ignored the destination's `If-Match` and `If-None-Match` headers and always overwrote or created the destination, so a stale `If-Match`, `If-None-Match: *` on an existing key, `If-Match` on a missing key and `If-Match` over a delete marker all returned 200. The controller now parses both headers and passes them to the backend through new `IfMatch` and `IfNoneMatch` fields on `s3response.CopyObjectInput`, which match the SDK's `s3.CopyObjectInput`.
As in S3, the destination is evaluated before the copy source is looked up, so a failing destination precondition wins over a missing source, a failing `x-amz-copy-source-*` precondition and an invalid copy onto itself. Posix checks the destination with `checkPutPreconditions` right after the destination bucket check, then forwards the headers to `PutObject`, which repeats the check under the object publish lock. The unversioned in-place self copy does not go through `PutObject`, so it now takes `lockObjectPublish` and repeats the check itself.
`checkPutPreconditions` now treats a current delete marker as a missing key: `If-Match` returns `NoSuchKey` and `If-None-Match: *` creates a new version. Before, the hidden version's etag was evaluated, which also affected `PutObject` and `CompleteMultipartUpload`.
`EvaluateObjectPutPreconditions` now rejects unsupported headers in S3's order: both headers present, then `If-None-Match` other than `*`, then `If-Match: *`, each with `NotImplemented`. `If-Match: *` was previously compared as a literal etag. The new `NmpAdditionalMessageIfMatch` message is added, and a stray backslash is removed from the `If-None-Match` message.
Azure evaluates the destination preconditions before copying, and s3proxy forwards the headers. `s3:if-match` and `s3:if-none-match` are now populated in the policy condition context for CopyObject, as S3 does, while UploadPartCopy stays excluded.
The bucket default retention was never stored on objects: protection was computed at delete time from the live bucket rule (`CreatedAt` + period), so changing or removing the rule unlocked existing objects, `COMPLIANCE` ones included, and a rule set N days ago stopped protecting even new objects. The posix and azure backends now store the rule's retention, `{Mode, now + Days/Years}` truncated to milliseconds, as the object's own retention whenever a write sets no Object Lock parameters: `PutObject` (which also covers POST, RDMA and `CopyObject`), `CreateMultipartUpload` (the rule and start time are fixed at creation, as S3 does) and the unversioned in-place self-copy, which applied no lock settings before. As on S3, any explicit `x-amz-object-lock-legal-hold`, `ON` or `OFF`, means the default retention is not applied. The bucket-rule fallback in `checkObject` and `BucketLockConfig.CreatedAt` are removed, so objects written before this change lose bucket-derived protection, and an expired retention no longer skips the legal hold check.
`PutObjectLockConfiguration` now rejects a `DefaultRetention` with a mode but no `Days`/`Years` with `InvalidRequest` "DefaultRetention must specify a fixed retention (Days or Years) or a DefaultEventHold.", and a `<Rule>` without `DefaultRetention` with `MalformedXML`. Azure `CopyObject` with the `COPY` metadata directive no longer copies the source's `Objectretention`/`Objectlegalhold` metadata to the destination.
Browser POST uploads now validate and apply the `x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date` and `x-amz-object-lock-legal-hold` form fields. They require `s3:PutObjectRetention`/`s3:PutObjectLegalHold`, for anonymous uploads too, and return `ErrMissingObjectLockConfigurationNoSpaces` on buckets without Object Lock after authorization. A denied POST names the actions in S3's order: retention, legal hold, tagging, then `s3:PutObject`.
`NoSuchObjectLockConfiguration` now returns 404 without a trailing period. `x-amz-object-lock-retain-until-date` on HEAD and GET has millisecond precision and omits zero milliseconds, `GetObjectRetention` returns `<Retention>` with a `.000Z` date, and posix and azure `GetObject` now return the Object Lock headers.
Fixes#2464
`versitygw iam` never set `IAMConfig.SigHup`, so the reload loop in `RunIAMAPI` never ran and the IAM service kept serving the certificates it started with. It is now connected to `gwcli.SigHup`.
The signal forwarder used blocking sends into the one-slot `SigHup` and `SigDone` channels. In modes that never read `SigHup` (`iam` until this fix, `admin`, `utils`), the second SIGHUP left the forwarder stuck, every later SIGINT/SIGTERM was lost, and the process could only be killed. `forwardSignals` now sends without blocking, and a notification that is already pending stands in for the new one.
The private listener loaded its client CA pool (`--private-client-ca`) only once. `netutil.TLSOptions.ClientCAs` is replaced by `GetClientCAs`, which `GetConfigForClient` calls on every handshake and which is backed by the new `netutil.CAPoolStorage`, so a reloaded bundle applies to the next connection. When no pool is loaded the listener refuses the client rather than falling back to the system roots. On SIGHUP the IAM service now also reloads the client CA pool and the certificate of the WebUI it hosts, in addition to its public and private certificates.
The gateway's standalone IAM client loaded its client cert/key and server CA (`--iam-standalone-client-cert[-key]`, `--iam-standalone-server-ca`) only once. With the new `auth.CertReloader` interface, the gateway's SIGHUP handler reloads them through `IAMServiceStandalone.ReloadCerts`. It builds a new `http.Client` from the files, swaps it in atomically and closes the old client's idle connections. Requests already in flight finish on the old client, and a failed load keeps the current one. The transports now set `IdleConnTimeout`, so a connection left on a replaced client cannot stay open indefinitely. The IAM wrapper in `vgwrdma` re-exports `CertReloader` so the capability is not lost there.
Object routes match subresources with `MatchQueryArgs`, and a request whose subresource has no route falls through to the catch-all `PutObject`, `GetObject` and `DeleteObject` handlers. As a result `PutObjectAnnotation` overwrote the object with the annotation payload or created a missing key, `DeleteObjectAnnotation` deleted the object or permanently deleted the version named by `versionId`, `RenameObject` replaced the destination with an empty object, and `PUT`/`DELETE ?torrent` overwrote or deleted the object.
`PUT ?annotation`, `GET ?annotation&annotationName`, `GET ?annotation` and `DELETE ?annotation` now return `NotImplemented` and follow the existing not-implemented route pattern: `BucketObjectNameValidator`, `AuthorizePublicBucketAccess`, signature verification and `ParseAcl`. Each route has its own metrics action, and the new `s3:PutObjectAnnotation`, `s3:GetObjectAnnotation`, `s3:ListObjectAnnotations` and `s3:DeleteObjectAnnotation` policy actions are also accepted in bucket policies.
`PUT ?renameObject` now returns `NotImplemented`. `?torrent` returns `MethodNotAllowed` with the new `TORRENT` resource type on every method, with `Allow: GET` on all methods except `GET`. Both are answered before authentication, as AWS does, and there is no `s3:RenameObject` policy action.
Fixes#2445
The S3 auth middlewares (header, presigned and POST) return 403 `InvalidAccessKeyId` only when the IAM lookup returns exactly `auth.ErrNoSuchUser`, since they compare with `==`. Three IAM backends returned a different error for an unknown access key, and the client received that error unchanged.
The root-only `IAMServiceSingle` returned the admin API's `ErrAdminUserNotFound`, so clients got 404 `XAdminUserNotFound`, which tools such as pgBackRest and rclone read as missing data rather than bad credentials. It now returns `ErrNoSuchUser`. The `ErrAdminUserNotFound` check in `resolveAccountsByLookup` is removed because no backend returns that error anymore.
In the Vault backend, `KvV2Read` of a missing key fails with a 404, which was passed through and rendered as 500 `InternalError`. The same error broke the admin `create-bucket`, `change-bucket-owner` and `update-user` commands for unknown users, which returned 500 instead of `XAdminUserNotFound`. `GetUserAccount` now maps a 404 to `ErrNoSuchUser` on both the initial read and the retry after re-authentication, the same way `ListUserAccounts` already handles 404.
FreeIPA reports a missing object as `NotFound` (code `4001`), both from `user_show` for an unknown user and from `vault_retrieve_internal` for a user who has no S3 vault. Both cases rendered as 500. `rpcInternal` now returns `errRpcNotFound`, which wraps `errRpc`, for code `4001`, and `GetUserAccount` maps it to `ErrNoSuchUser` in both places. A user without the vault has no S3 secret, so it is treated as a nonexistent account, and the PKCS#1 v1.5 to OAEP retry is skipped in that case.
Fixes#1535Fixes#1542Fixes#1740Fixes#1776
S3 requires `Content-MD5`, an `x-amz-checksum-*` header or an `x-amz-checksum-*` trailer on every `PutObject` that carries Object Lock parameters, and rejects the upload otherwise with `InvalidRequest` "Content-MD5 OR x-amz-checksum- HTTP header is required for Put Object requests with Object Lock parameters". The parameters are an `x-amz-object-lock-legal-hold` header of either status, a retention mode and date, or the bucket's default retention rule, which gives even a plain upload lock parameters. A payload hash, `UNSIGNED-PAYLOAD` and signed chunks don't count as an integrity check. The gateway accepted all of these uploads.
`auth.VerifyWriteObjectLock` reads the bucket's lock configuration, rejects lock headers on a bucket without Object Lock with "Bucket is missing ObjectLockConfiguration", legal hold `OFF` included, and reports whether the write carries lock parameters. `PutObject` calls it when the request has lock headers or no integrity check, so an upload with a checksum and no lock headers pays no extra lookup. `CopyObject` and `CreateMultipartUpload` call it only for lock headers, since S3 requires no integrity check for them.
S3 applies the same rule to `UploadPart`, with "Put Part requests" in the message, when the multipart upload carries lock parameters: its own lock headers, or a default retention rule in force at `CreateMultipartUpload`, which still counts if the rule changes later. posix and azure now mark such an upload when it's created, with the `mp-object-lock` attribute and the `Mpobjectlock` sgwtmp metadata respectively, and `UploadPart` rejects a part of it that has no integrity check. The `UploadPart` controller now passes `Content-MD5` to the backend.
An anonymous `PutObject` carrying lock parameters, including a plain one into a bucket with a default retention rule, is refused with `InvalidArgument` "Put Object requests with Object Lock parameters require AWS Signature Version 4". This comes after the missing configuration and integrity checks, and before the upload itself is authorized, so a private bucket with a default retention rule refuses it too. Anonymous parts of an upload with lock parameters only need an integrity check, and browser-based `POST` uploads are unaffected, as on S3.
Anonymous uploads were authorized for `s3:PutObject` alone, so a public bucket granting only that let anonymous callers tag objects. `AuthorizePublicBucketAccess` now authorizes every action a request carries in one `VerifyPublicAccess` call. A `PutObject` with a non-empty `x-amz-tagging` or a browser-based `POST` with a non-empty tag set needs `s3:PutObjectTagging` as well as `s3:PutObject`. The shared `auth.ObjectUploadActions` derives these actions for both anonymous and signed uploads. Each action is granted on its own, by the bucket policy or, failing that, the bucket ACL, whose WRITE grant covers `s3:PutObject` only, and an explicit public Deny of any one of them denies the request.
Anonymous requests naming an object version were authorized for the plain action. A non-empty `versionId` now makes the public access check authorize the version action instead: `s3:GetObjectVersion` rather than `s3:GetObject` for `GetObject` and `HeadObject`, `s3:DeleteObjectVersion` for `DeleteObject`, `s3:GetObjectVersionAttributes` for `GetObjectAttributes`, and the version tagging actions for the tagging APIs. A public ACL READ grant covers `s3:GetObjectVersion`, but a WRITE grant never covers `s3:DeleteObjectVersion`, which S3 grants to the bucket owner alone.
The "x-amz-object-lock-retain-until-date and x-amz-object-lock-mode must both be supplied" errors no longer end with a period, and the anonymous `CopyObject` and `CreateMultipartUpload` rejections have two spaces before "Please authenticate.", both matching S3.
Some requests need more than one S3 action authorized on the same object. `PutObject`, `CopyObject` and `CreateMultipartUpload` add `s3:PutObjectTagging`, `s3:PutObjectLegalHold` or `s3:PutObjectRetention` depending on the headers sent, and a browser-based `POST` with a `tagging` field adds `s3:PutObjectTagging`. The gateway allowed such a request only when a single policy source allowed every action, because `verifyBucketPolicy` and `aggregateActionDecisions` each reduced their side to one verdict before `objectsAccessErrors` combined them. S3 authorizes each action separately. As a result, a user granted `s3:PutObject` by the bucket policy and `s3:PutObjectTagging` by their identity policy was refused a tagged `PutObject` that S3 allows, and the denial message wrongly said no identity-based policy allows `s3:PutObjectTagging`. Only gateways backed by the standalone IAM service (`--iam-standalone-endpoint`) were affected.
`verifyResourceAccess` now returns one decision per action for each key. It computes them with `decisionFor` and parses the bucket policy once per request instead of once per key. `identityPolicyDecisions` now returns the `EvaluatePolicy` result without reducing it. `objectsAccessErrors` combines the two sources action by action. An explicit deny from the bucket, identity or session policy still wins and is reported as before. Otherwise an action is allowed when either source allows it, and, when a session policy is present, the session policy must also allow it. A request with any action not allowed is denied, and the denial names the first such action in list order, as S3 does. Each request still fetches the bucket policy once and makes one `EvaluatePolicy` round trip, and `VerifyObjectsAccess` still makes one round trip per action group.
In posix a key and the same key with a trailing slash, such as `foo` and `foo/`, map to one path and share one set of attributes: a file there is the object of `foo`, a directory object is the object of `foo/`. Several operations used the entry at that path without checking that it belongs to the requested key. They now check it through the new `isLiveObject`, `statLiveObject` and `objVersionAttrPath` helpers.
`DeleteObject` of `foo` with a version id of `foo/` treated the directory as the current version of `foo` and removed it. That left the noncurrent versions of `foo/` unlisted and undeletable, so `DeleteBucket` failed with `BucketNotEmpty`. A delete of `foo` without a version id failed with an internal error while trying to version the directory as a file. Both now succeed without touching `foo/`, as for any object that doesn't exist.
`CompleteMultipartUpload` of `foo` cleared the `delete-marker` attribute of `foo/` and then failed with an internal error when linking the object onto the directory, which turned the delete marker back into a live version. It now returns `ExistingObjectIsDirectory` before any attribute is changed, both before the parts are assembled and again under the object publish lock. The idempotent completion path also no longer reports a missing upload as completed just because `foo/` exists.
The object tagging, legal hold and retention APIs read and wrote the attributes of the other key. For example, `PutObjectLegalHold` on `foo` could turn off the legal hold of `foo/`, and the object lock check could block a delete because of the other key's retention. When the requested key has no object, they now return `NoSuchKey` for the current version and `NoSuchVersion` for a specific version. They also resolve the `null` version id to a current null version instead of looking for it in the versioning directory. The object lock check treats `NoSuchVersion` like `NoSuchKey`, since a version that doesn't exist has nothing to protect. Conditional writes no longer evaluate `If-Match` and `If-None-Match` against the other key's ETag. `CreateMultipartUpload` now stores the upload's tagging and object lock settings directly on the upload directory, and `PutObject` of a directory object sets its tagging after the directory gets its ETag.
* fix: omit ObjectLockConfiguration Rule when there is no default retention
ParseBucketLockConfigurationOutput always set Rule, so a bucket with object lock enabled and no default retention answered GET ?object-lock with an empty <Rule></Rule>. AWS S3 omits the element, and the AWS SDK v2 clients that read the rule from it follow the empty element with a malformed request.
Fixes#2397
* test: cover GetObjectLockConfiguration with no default retention in the integration suite and drop the stale non-nil Rule expectation from the controller unit test
---------
Co-authored-by: Tung Lam <lamphamabtung96@gmail.com>
`293d9f50` migrated `auth/iam_ipa.go`'s KRA session-key wrap from `rsa.EncryptPKCS1v15` to `rsa.EncryptOAEP` to silence a Go 1.26 deprecation warning. FreeIPA's KRA can be configured to unwrap session keys with either PKCS#1 v1.5 (its default) or OAEP, and the REST API has no way to query which one a given deployment uses — the same constraint FreeIPA's own client (`ipaclient/plugins/vault.py`) works around by trying one padding and falling back to the other. Hardcoding OAEP with no fallback breaks `GetUserAccount` against any KRA using the default PKCS#1 v1.5 configuration, which includes every deployment that worked before that change.
This restores compatibility by trying PKCS#1 v1.5 first, matching the pre-`293d9f50` behavior, and falling back to OAEP on failure so FIPS-mode KRAs (which reject PKCS#1 v1.5) keep working too.
Separately, `293d9f50` also changed `iamapi/internal/iamutil/webidentity.go`'s OIDC JWKS parsing to build EC public keys via `ecdsa.ParseUncompressedPublicKey` instead of setting `ecdsa.PublicKey`'s `X`/`Y` fields directly — a real improvement, since it validates the point is on the curve, which the old code never did. But it writes the JWK's `x`/`y` coordinates into a fixed-size buffer via `big.Int.FillBytes` without checking their length first, so an oversized `x` or `y` in a JWKS response panics instead of returning an error. That JWKS is fetched from the OIDC issuer configured on a role's trust policy, so a malformed or compromised response can crash request handling for `AssumeRoleWithWebIdentity`. This adds a bounds check before the `FillBytes` calls, plus `TestJwkPublicKeyEC` covering both the valid round-trip and the oversized-coordinate case, since `jwk.publicKey()`'s EC branch had no prior test coverage.
Closes#2294
Support AWS's conditional-write condition keys in both S3 bucket policies and IAM identity policies, so a policy can require that an upload or delete carry an `If-Match`/`If-None-Match` precondition.
`requestConditionContext` now populates both keys from the request headers, which covers both policy types at once: the same map is evaluated in-process for bucket policies and shipped to the IAM service for identity policies. Only the three requests whose preconditions the gateway actually enforces contribute - PutObject, CompleteMultipartUpload and DeleteObject. Copies, form uploads, DeleteObjects batches, upload parts and the sub-resource writes all ignore the headers, and a policy must never grant on a precondition that won't be checked; reads are excluded for the same reason, since GET and HEAD take these headers as ordinary HTTP cache preconditions. The value is the ETag with its surrounding quotes stripped, matching what the gateway enforces against.
Bucket policies validate condition keys against a fixed catalogue at `PutBucketPolicy` time, so both keys are added there with the action sets AWS accepts: `s3:if-match` on `s3:PutObject` and `s3:DeleteObject`, `s3:if-none-match` on `s3:PutObject` alone. Identity policies validate only the operator vocabulary, matching AWS, so they need no change.
Go 1.26 marks PKCS#1 v1.5 RSA encryption and direct ECDSA public-key coordinate access
as deprecated. The IPA code path uses rsa.EncryptPKCS1v15 to wrap an AES session key for
vault retrieval, and the OIDC JWKS parser rebuilt EC public keys by setting ecdsa.PublicKey.X
and Y directly.
This change replaces the deprecated RSA wrap with rsa.EncryptOAEP using SHA-256, which is the
standard safe replacement for PKCS#1 v1.5 encryption and preserves the same protocol semantics
for the IPA vault exchange. For EC JWKs, it reconstructs the raw uncompressed public point and
parses it through ecdsa.ParseUncompressedPublicKey, which is the supported Go API for EC public
keys and avoids mutating deprecated fields while preserving the exact mathematical key value.
These changes do not alter the wire protocol or trust decisions; they only migrate to the
supported stdlib APIs for equivalent behavior.
Bucket policy `Principal` named callers by access key id. Under the standalone IAM service it now names them by AWS-style ARN, as real S3 does: a user ARN, a role ARN (covering every session of that role), an assumed-role ARN (covering one session), the account root ARN or bare account id, or `*`. Every other IAM backend has no ARNs to name anything by and keeps access-key principals unchanged, gated on a new `auth.PrincipalResolver` capability interface that only the standalone client implements.
`auth.Account` carries `Arn` and `RoleArn`, filled at authentication time, so a session can be matched against both its own ARN and its role's. Principals are validated at PutBucketPolicy time through a new `/private/resolve-principals` endpoint, which rejects anything that does not name a live identity with `MalformedPolicy: Invalid principal in policy`.
An `Allow` naming the account root ARN or bare account id delegates to the account's own IAM rather than granting on its own, while a `Deny` naming it denies every principal in the account outright. Denial messages now name the caller by ARN wherever one exists.
Also fixes `aws:PrincipalArn` for assumed-role sessions, which reported the session ARN where AWS reports the role's, and stops an unreachable IAM service being reported as a malformed policy.
Role last-used tracking was missing entirely - `GetRole` returned a `RoleLastUsed` element that nothing ever wrote, rendering the zero time instead of the empty element AWS returns for an unused role - and access key last-used only ever saw the `IAM`/`STS` control plane, so a credential used exclusively against the S3 gateway reported as never used. Roles now record a use whenever a request authenticates with one of their session credentials, through a new `Storer.RecordRoleUsage` mirroring `RecordAccessKeyUsage`, gated on the session's role still being the one it was minted against so a session outliving its role can't attribute its use to a same-named replacement. `LastUsedDate` became a `*time.Time` so an unused role renders as an empty element.
Both records now cover the S3 data plane as well: the gateway sends its configured region and `s3` on evaluate-policy and the IAM service records the caller there, so `GetAccessKeyLastUsed's` `ServiceName` is now iam, sts or s3. That call was chosen over derive-signing-key, which runs before signature verification and takes its region and service from the caller's own `Authorization` header - recording there would let anyone who knows an access key id refresh and poison another identity's audit record. Requests denied by a bucket policy or made against a public bucket are not recorded, since neither reaches identity-policy evaluation. To keep per-request recording affordable, an update is skipped while the stored record has the same service and region and is under a minute old; a change of either is written through immediately.
Assuming a role is not a use, a request denied by an identity policy is, and both successful and denied S3 requests update the record. Also moves the `OIDC-dependent` tests into the `s3-iam-session` group so runoidctests.sh runs a single group.
objectsAccessErrors recorded the first resource-policy Deny and returned immediately, leaving every later key in the same action subset with a nil result — which VerifyObjectsAccess caller reads as "authorized" and sends straight to the backend. An explicit bucket-policy Deny therefore let the keys after it skip authorization entirely and be deleted, including keys that same bucket policy explicitly denied.
The loop now continues rather than returning, so every denied key is settled with its own error. The identity-policy round trip the early return was saving is still skipped, but only when the bucket policy denied every key in the batch, since no identity-policy answer could change any result then. Both loops that follow skip keys already holding an error, so a resource-level explicit deny is never overwritten by an identity-policy result nor flattened to the generic AccessDenied message.
With the posix backend running --chuid/--chgid against the standalone IAM service, CreateBucket failed for every bucket name and left a half-created directory behind. Bucket ownership is fixed to the gateway's root account there, and that account was constructed from the root credentials alone, so its UserID/GroupID stayed at zero and the gateway tried to chown each new bucket to uid/gid 0 - something a process that is not root can never do. Root-account object writes failed the same way, because the identity the S3 request path uses for root also comes from the root credentials and never from the IAM backend. On top of that, the failed chown returned before the acl xattr was written, so the leftover directory made every later request for that name fail with "get bucket acl: no such key" until it was removed by hand.
The standalone IAM client now reports the root account with UserID, GroupID and ProjectID taken from --iam-standalone-default-uid, -gid and -project-id, returning a copy so the stored root account keeps the credentials it is compared against. ResolveDerivedKey copies that same identity onto root when the IAM backend fixes bucket ownership to the root access key, which keeps root's own writes consistent with the buckets root owns. CreateBucket now removes the bucket directory, its sidecar attributes and its versioning directory on any failure after the mkdir, so a failed create leaves nothing behind and the name stays retryable. A chown EPERM reports the target uid/gid, the flags that asked for it and the process euid/egid instead of a bare "operation not permitted", and the posix backend warns at startup when chuid/chgid are set on an unprivileged gateway.
The built-in IAM backends do not fix bucket ownership, so root and every other account reach the storage backend exactly as before.
S3 explicitly documents this action as owner-only ("To retrieve the versioning state of a bucket, you must be the bucket owner." — https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketVersioning.html), and the handler enforced that with an extra auth.IsAdminOrOwner check on top of VerifyAccess. Real S3 behaves differently: verified against AWS that a bucket policy explicitly denying s3:GetBucketVersioning denies the bucket owner itself, and that an Allow grants the action to a principal that doesn't own the bucket. It goes through ordinary bucket policy/ACL evaluation like any other bucket subresource read, which is what the write side, PutBucketVersioning, already did here. Removes the extra check along with auth.IsAdminOrOwner, which had no other call site.
The cached GetUserAccount keeps serving a stale entry for the
cache TTL after the backing IAM service changes, which delays
credential revocation by up to the configured expiry.
Add GetUserAccountFresh to the IAM cache, which reads directly
from the underlying service and refreshes the cached entry with
the result. Callers that need revocations to take effect
immediately can use it instead of the cached path.
Signed-off-by: Jihyeon Gim <potatogim@potatogim.net>
Gates bucket listing behind an identity policy, lets browsers reach the standalone IAM API, and turns the WebUI into a dashboard for S3, IAM, or both.
**Bucket listing.** `ListBuckets` is now gated by the new `s3:ListAllMyBuckets` action, evaluated against `arn:aws:s3:::*`. The request names no bucket, so only identity policies apply — there is no resource policy to combine with, which is the same shape `CreateBucket` already had, so both now share one identity-only evaluation path. Root and admin bypass it, and backends with no identity-policy layer keep listing as before since their listing is already narrowed to the caller's own buckets. The action is IAM-only and is deliberately absent from the bucket-policy action list.
**Fixed bucket ownership.** The standalone IAM client has no per-user ownership to express — accounts are all plain users, cannot be enumerated, and access is decided by policy rather than ACL — so it now implements `auth.FixedBucketOwner` and every bucket is owned by root. Bucket creation stops resolving an owner, `ListBuckets` returns every bucket to every caller (what they may then do with one stays a per-request policy decision), and the admin `ChangeBucketOwner` reports method-not-supported. Other IAM backends are untouched.
**IAM service CORS.** `--cors-allow-origin` now applies to the `iam` command: it answers preflights and stamps the CORS headers, mirroring back the requested method and headers rather than enumerating the SigV4 header set. Without it no browser can reach the IAM API at all, so setting `--webui` without it falls back to `*` with a warning. The chart gets `iamServer.corsAllowOrigin`.
**WebUI.** New IAM pages for users, roles and OIDC providers, signing IAM/STS query-form requests directly from the browser. Navigation is capability-gated rather than role-gated: on sign-in the session probes the S3, admin and IAM endpoints independently and each page shows only what those credentials actually reach, so one build serves an IAM-only dashboard, an S3-only dashboard, and a combined one. The login page takes an optional IAM endpoint, seeded from the new `--webui-iam-gateways` (chart: `webui.iamGateways`) — never auto-detected, since the IAM service is a separate process. The WebUI can also be hosted by `versitygw iam` itself, for deployments with no S3 gateway behind it.
**The admin API is ignored once an IAM endpoint is in play.** The IAM service is then the user directory and bucket ownership is fixed, which leaves the admin API no job: the session is given no admin endpoint at all, its login field is hidden, `users.html` redirects to its IAM counterpart, and every admin-only surface stays off screen. Dashboard and Buckets remain available to any S3 session in such a deployment, running on the S3 and IAM APIs alone and surfacing each denial per action instead of redirecting.
Also fixes two WebUI bugs: embedded assets went out with a zero modification time and no `Cache-Control`, so browsers treated them as fresh for centuries and an upgraded gateway served new HTML against stale JS — they now revalidate against an ETag; and the login page's advanced-options section clipped its last field, since it animated to a height named in the stylesheet rather than the one it measures now.
**Usage**
IAM-only dashboard, served by the IAM service:
versitygw iam --port :7076 --webui :8080 --cors-allow-origin http://localhost:8080/
IAM + S3, dashboard served by the IAM service — point it at the gateway with `--webui-gateways`, and let the gateway accept the dashboard's origin:
versitygw iam --port :7076 --webui :8080 --webui-gateways http://localhost:7070/ --cors-allow-origin http://localhost:8080/
versitygw --port :7070 --cors-allow-origin http://localhost:8080/ posix /data
IAM + S3, dashboard served by the S3 gateway — point it at the IAM service with `--webui-iam-gateways`, and let the IAM service accept the dashboard's origin:
versitygw --port :7070 --webui :8080 --webui-iam-gateways http://localhost:7076/ posix /data
versitygw iam --port :7076 --cors-allow-origin http://localhost:8080/
The S3 gateway and the standalone IAM service exchange authorization decisions over the private endpoints, where a version skew is silently unsafe in both directions: an older service drops a request field it does not know (a `Condition` block, say) and evaluates fail-open, while an older gateway ignores a response field it does not know and misses a deny the service intended. Neither side could previously detect either case.
Both peers now declare a protocol version on every exchange via the `X-Vgw-Private-Protocol` header — the gateway on each request, the service on each response, error responses included — and each refuses a peer it cannot serve safely. The service rejects a gateway below `MinClientProtocol` with a `ProtocolMismatch` code; the gateway rejects a service older than the `ProtocolVersion` it speaks, and rejects a response carrying no version at all, since no build of this protocol omits the header and something else answering on that address should not be interpreted as an IAM decision. `ParseProtocolVersion` is shared by both sides and deliberately strict: an unreadable value is a mismatch, never an assumed default.
A new root-signed `/private/version` endpoint reports the protocol version, the minimum client the service will serve, and the build tag (`WithPrivateServerVersion`). It is exempt from the service's own client-version check so it can still answer a gateway the service refuses — which is how that gateway learns why. Being authenticated like every other private endpoint, it also lets the gateway's startup probe verify its own credential and its mTLS transport in the same round trip.
The gateway probes it once in `NewIAMServiceStandalone` rather than discovering a skew as an opaque per-request 500. An incompatible service is fatal after a 30s window, since a gateway that cannot authorize a single request is more useful refusing to start with the reason in its log; an unreachable one is only a warning, because the two processes legitimately start in parallel and every request checks the version regardless. Only conditions that can resolve on their own are retried — a rejected credential is reported immediately.
Fixes#1327Fixes#1567Closes#2264
Wires the S3 gateway up to the standalone IAM service so identity policies, not just bucket policies and ACLs, are enforced on the S3 data plane. The gateway authenticates SigV4 requests by calling new private derive-signing-key and resolve-identity endpoints on the IAM service instead of holding secrets itself, and evaluates identity policy through the same PolicyEvaluator path added to auth.VerifyAccess, combined with the bucket policy using explicit-deny-wins precedence. The private endpoints are served over their own mTLS listener (new iamapi/private package, genmtlscerts.sh to generate test material, and client-cert support in internal/netutil), separate from the public IAM API. As part of this the vendored aws/signer/v4 package is deleted and replaced by a pure-Go SigV4 implementation in internal/sigv4auth, which now reads canonical request data directly off the fiber.Ctx instead of reconstructing an http.Request, and is shared by both the S3 request-signing verification and the new private-endpoint signing.
DeleteObjects moves from an all-or-nothing authorization check to true partial success: VerifyObjectsAccess evaluates every object in a batch independently against both the identity policy and any object lock, so a denial or a locked object only removes that key from the batch instead of failing the whole request. It also batches the identity-policy round trip and the bucket-policy fetch once per request rather than once per object, and separates plain deletes from versioned ones since a versioned delete needs s3:DeleteObjectVersion rather than s3:DeleteObject. Object lock handling got a few correctness fixes alongside this: a bypass is now modeled as BypassNone/BypassRequested/BypassOverwrite rather than a single bool, because root's blanket ability to override a GOVERNANCE retention should only apply when the client actually asked to bypass it (DeleteObject/DeleteObjects/PutObjectRetention), not when the gateway is silently replacing a locked object via an overwrite, which needs the permission from everyone including root. Retention changes are now correctly classified as an extension (allowed under plain s3:PutObjectRetention) versus a weakening (date or mode change, which needs the bypass permission), and a COMPLIANCE lock can never be weakened by anyone regardless of permissions, matching AWS. Separately, VerifyObjectCopyAccess had a readonly-mode gap: it returned early for root/admin before ever calling VerifyAccess, so the readonly check inside VerifyAccess never ran for them on CopyObject; access checks are now ordered so the readonly gate always applies before any root/admin bypass, for copy as well as every other write path.
Bucket policies also gained Condition block support, via a new shared internal/condition package moved out of the IAM policy package since both bucket and identity policies share the same evaluation semantics. It implements the full AWS operator set — String{Equals,NotEquals,EqualsIgnoreCase,NotEqualsIgnoreCase,Like,NotLike}, Numeric{Equals,NotEquals,LessThan,LessThanEquals,GreaterThan,GreaterThanEquals}, Date{Equals,NotEquals,LessThan,LessThanEquals,GreaterThan,GreaterThanEquals}, Bool, BinaryEquals, Arn{Equals,Like,NotEquals,NotLike}, IpAddress/NotIpAddress, and Null — along with the ForAllValues/ForAnyValue set qualifiers and the IfExists modifier. A new requestConditionContext builds the per-request keys a bucket policy's Condition block can reference — aws:SourceIp, aws:SecureTransport, aws:CurrentTime, aws:EpochTime, aws:UserAgent, aws:Referer, s3:prefix, s3:delimiter, s3:max-keys, s3:x-amz-acl, s3:VersionId — following AWS's own per-action rules for which keys a given S3 operation actually populates. Identity-derived keys such as aws:PrincipalArn and aws:username are deliberately left unwired here, since the gateway has no way to know them; the standalone IAM service fills those in itself when it evaluates an identity policy.
Also added new integration test suites for S3-side IAM: s3_iam_access_control.go and s3_iam_session_access_control.go cover identity-policy enforcement and session-credential requests against real S3 operations, alongside expanded OIDC/web-identity coverage and a new runoidctests.sh runner wired into the OIDC GitHub Actions workflow.
Closes#1640
Add a standalone AWS IAM Query API implementation for managing IAM users through standard AWS SDKs and the AWS CLI.
Server usage
Start the IAM server with internal file-backed storage:
mkdir -p /tmp/versitygw-iam
./versitygw --port 127.0.0.1:7070 --access user --secret pass iam --dir /tmp/versitygw-iam
Start the IAM server with Vault KV v2 storage using AppRole:
VGW_IAM_VAULT_ROLE_SECRET=<role-secret> ./versitygw --port 127.0.0.1:7070 --access user --secret pass iam --vault-endpoint-url http://127.0.0.1:8200 --vault-auth-method approle --vault-role-id <role-id> --vault-mount-path kv --vault-secret-storage-path iam
Vault authentication also supports root tokens, separate authentication and secret-storage namespaces, custom mount paths, server certificate validation, and mutual TLS client certificates.
Configure the AWS CLI credentials used by the IAM server:
export AWS_ACCESS_KEY_ID=user
export AWS_SECRET_ACCESS_KEY=pass
export AWS_DEFAULT_REGION=us-east-1
Implemented IAM actions
CreateUser creates an IAM user with an AWS-compatible ARN, generated AIDA user ID, creation timestamp, optional path, and tags. It validates usernames, paths, tag limits, reserved tag prefixes, duplicate tag keys, and existing users.
aws --endpoint-url http://127.0.0.1:7070 iam create-user --user-name bob
aws --endpoint-url http://127.0.0.1:7070 iam create-user --user-name bob --path /engineering/ --tags Key=team,Value=storage
GetUser returns a stored user or the root identity when requested without a username through the IAM Query API.
aws --endpoint-url http://127.0.0.1:7070 iam get-user --user-name bob
ListUsers returns users in deterministic username order and supports path filtering, marker-based pagination, and MaxItems limits.
aws --endpoint-url http://127.0.0.1:7070 iam list-users
aws --endpoint-url http://127.0.0.1:7070 iam list-users --path-prefix /engineering/ --max-items 100
UpdateUser updates the username and/or path, recalculates the user ARN, and rejects conflicts with existing users.
aws --endpoint-url http://127.0.0.1:7070 iam update-user --user-name bob --new-user-name robert --new-path /platform/
DeleteUser permanently removes an IAM user and returns AWS-compatible errors for missing users.
aws --endpoint-url http://127.0.0.1:7070 iam delete-user --user-name robert
IAM protocol and authentication
- Support the AWS IAM Query protocol version 2010-05-08 over GET and POST form requests.
- Return AWS-compatible XML responses, error documents, status codes, request IDs, user metadata, and pagination fields.
- Authenticate root credentials with AWS Signature Version 4 for the IAM service in us-east-1.
- Support both Authorization-header and query-string SigV4 authentication.
- Validate credential scope, signed headers, timestamps, clock skew, content length, signatures, and unsupported signature or session-token modes.
- Add IAM-specific validation and error mapping for malformed requests, invalid actions, duplicate entities, missing users, throttling, and internal failures.
Storage implementations
- Add an internal JSON-backed store using iam.json and iam.json.backup with atomic temporary-file replacement, concurrent access protection, stable ordering, pagination, and persistence across restarts.
- Add a Vault KV v2 store with one secret per user, CAS-based duplicate protection, permanent deletion, AppRole reauthentication, namespace support, configurable authentication and KV mounts, root-token authentication, and TLS/mTLS configuration.
- Introduce a common Storer interface and require exactly one storage backend to be configured.
Server and embedding support
- Register the new `versitygw iam` command with environment-variable and CLI configuration for both storage backends.
- Add `embedgw.RunIAMAPI` and `IAMConfig` for embedding the IAM service in Go applications.
Gateway-level internal packages
- Add `internal/iamstore` as a reusable generic file-backed IAM persistence engine and migrate the existing gateway internal IAM service to it.
- Add `internal/sigv4auth` for shared SigV4 header and presigned-query parsing, canonical request generation, signature verification, and structured authentication errors.
- Refactor the S3 authentication paths to use the shared SigV4 implementation while preserving S3-specific error responses.
- Add `internal/httpctx` for shared Fiber context keys and AWS-style request ID handling.
- Add `internal/routekit` for shared query, form, and header route matchers.
- Add `internal/netutil` for reusable certificate storage, hostname-aware listeners, multi-address serving, TLS listeners, and UNIX socket handling.
- Update the custom SigV4 signer to honor an explicitly supplied signed-header list so unrelated headers do not alter IAM signatures.
Testing and CI
- Add AWS IAM SDK-based integration coverage for all supported user actions, header authentication, query authentication, validation, errors, filtering, and pagination.
- Split standalone IAM tests into `versitygw test iam` and retain existing gateway IAM tests under `versitygw test gw-iam`.
- Add unit coverage for controllers, authentication, routing, storage, embedding, listeners, request matching, persistence, and signing behavior.
- Add `runiamtests.sh` to exercise internal storage over HTTP and HTTPS plus Vault storage through AppRole.
- Add a dedicated IAM functional-test workflow with a Vault service and merged runtime coverage reporting.
- Include the IAM test runner in shellcheck and add the AWS IAM SDK dependency.
LDAP-backed IAM lookups built a search filter using the request
access key before signature verification, allowing filter
metacharacters to alter query semantics. This made account
lookup behavior observable and could be abused as a
pre-authentication oracle in affected LDAP deployments.
This change escapes access key values with ldap.EscapeFilter
when constructing LDAP search filters and adds a regression
test that verifies metacharacters are encoded rather than
interpreted as filter operators.
Also add DN escaping along with this fix too.
Reported by GitHub user KrisKennawayDD.
Fixes#2187
Enforce maximum default retention periods when parsing PutObjectLockConfiguration requests. Reject Days values greater than 36500 and Years values greater than 100 with InvalidArgument errors.
Fixes#2180Fixes#2181
Migrate the gateway from Fiber v2 to Fiber v3.3.0 and update the affected server, middleware, handler, controller, and test code for the new APIs.
Replace the deprecated Fiber filesystem middleware used by the WebUI with the Fiber v3 static middleware, serving the embedded WebUI assets from an fs.Sub filesystem.
Fix the request header limit handling regression by adding a temporary handler for Fiber v3/fasthttp small-buffer errors so oversized request headers return the expected regulated S3 error response.
Fix the debuglogger panic by reworking the boxed key/value formatter used for debug request and response dumps. The formatter now handles long header keys and values without producing invalid wrap widths, negative padding, or out-of-range string slices.
Replace PutBucketWebsite, GetBucketWebsite, DeleteBucketWebsite
NotImplemented test stubs with comprehensive integration tests covering:
- non-existing bucket errors
- validation (empty suffix, suffix with slash, invalid protocol, mutual
exclusion of RedirectAllRequestsTo and IndexDocument)
- successful put/get round-trips for both index+error and redirect-all configs
- delete idempotency and verification
Signed-off-by: Marc Singer <marc@singer.gg>
Add error document serving, routing rules, and integration tests
Implement Features 1 and 2 of S3 static website hosting:
- WebsiteErrorDocument controller wrapper intercepts 4xx errors on
website-enabled buckets and serves the configured error document or
evaluates post-request routing rules (error code match redirects)
- ResolveWebsiteIndex middleware now caches parsed WebsiteConfiguration
in context, handles RedirectAllRequestsTo, evaluates pre-request
routing rules (key prefix match redirects), and rewrites directory
keys for index document
- MatchPreRequestRule and MatchPostRequestRule methods on
WebsiteConfiguration for routing rule evaluation
- 14 unit tests for routing rule matching
- 7 integration tests covering error document, routing rules,
redirect-all, and index document behavior
Signed-off-by: Marc Singer <marc@singer.gg>
Add separate website hosting endpoint with virtual-host routing
Signed-off-by: Marc Singer <marc@singer.gg>
Support catch-all mode for website endpoint when --website-domain is omitted
Signed-off-by: Marc Singer <marc@singer.gg>
Object key validation allowed internal parent-directory segments such as `public/../private.txt`. Bucket policy and auth checks evaluated the raw key, so a policy allowing bucket/public/* could match the request while posix backend later resolved the key with `filepath.Join` and accessed `bucket/private.txt`.
Add backend-specific object key normalization to close that mismatch. The Backend interface now exposes `NormalizeObjectKey` so authorization can evaluate resources using the same key shape a backend will use for storage access.
Backends that do not collapse object paths, including Azure and the S3 proxy, inherit `BackendUnsupported.NormalizeObjectKey`. That implementation returns the input key unchanged, avoiding unnecessary normalization and keeping policy evaluation unpolluted for object stores where ../ is part of the key name.
posix/scoutfs normalize keys with filepath.Join so policy resources and request keys are compared after internal dot segments are collapsed.
Bucket policy evaluation now normalizes both the incoming object key and object resource patterns from the policy before matching. Object lock governance bypass policy checks use the same backend normalizer as well, so retention and legal hold authorization cannot diverge from backend path resolution.
Distinguish public bucket policy no-match from explicit deny during anonymous access checks. This preserves ACL fallback only for requests that are not allowed by policy, while ensuring a matching Deny statement short-circuits authorization and returns AccessDenied even when a public ACL would otherwise grant access.
Fixes#2123Fixes#2120Fixes#2116Fixes#2111Fixes#2108Fixes#2086Fixes#2085Fixes#2083Fixes#2081Fixes#2080Fixes#2073Fixes#2072Fixes#2071Fixes#2069Fixes#2044Fixes#2043Fixes#2042Fixes#2041Fixes#2040Fixes#2039Fixes#2036Fixes#2035Fixes#2034Fixes#2028Fixes#2020Fixes#1842Fixes#1810Fixes#1780Fixes#1775Fixes#1736Fixes#1705Fixes#1663Fixes#1645Fixes#1583Fixes#1526Fixes#1514Fixes#1493Fixes#1487Fixes#959Fixes#779Closes#823Closes#85
Refactor global S3 error handling around structured error types and centralized XML response generation.
All S3 errors now share the common APIError base for the fields every error has: Code, HTTP status code, and Message. Non-traditional errors that need AWS-compatible XML fields now have dedicated typed errors in the s3err package. Each typed error implements the shared S3Error behavior so controllers and middleware can handle errors consistently while still emitting error-specific XML fields.
Add a dedicated InvalidArgumentError type because InvalidArgument is used widely across request validation, auth, copy source handling, object lock validation, multipart validation, and header parsing. The new InvalidArgument path uses explicit InvalidArgErrorCode constants with predefined descriptions and ArgumentName values, keeping call sites readable while preserving the correct InvalidArgument XML shape and optional ArgumentValue.
New structured errors added in s3err:
- `AccessForbiddenError`: Method, ResourceType
- `BadDigestError`: CalculatedDigest, ExpectedDigest
- `BucketError`: BucketName
- `ContentSHA256MismatchError`: ClientComputedContentSHA256, S3ComputedContentSHA256
- `EntityTooLargeError`: ProposedSize, MaxSizeAllowed
- `EntityTooSmallError`: ProposedSize, MinSizeAllowed
- `ExpiredPresignedURLError`: ServerTime, XAmzExpires, Expires
- `InvalidAccessKeyIdError`: AWSAccessKeyId
- `InvalidArgumentError`: Description, ArgumentName, ArgumentValue
- `InvalidChunkSizeError`: Chunk, BadChunkSize
- `InvalidDigestError`: ContentMD5
- `InvalidLocationConstraintError`: LocationConstraint
- `InvalidPartError`: UploadId, PartNumber, ETag
- `InvalidRangeError`: RangeRequested, ActualObjectSize
- `InvalidTagError`: TagKey, TagValue
- `KeyTooLongError`: Size, MaxSizeAllowed
- `MetadataTooLargeError`: Size, MaxSizeAllowed
- `MethodNotAllowedError`: Method, ResourceType, AllowedMethods
- `NoSuchUploadError`: UploadId
- `NoSuchVersionError`: Key, VersionId
- `NotImplementedError`: Header, AdditionalMessage
- `PreconditionFailedError`: Condition
- `RequestTimeTooSkewedError`: RequestTime, ServerTime, MaxAllowedSkewMilliseconds
- `SignatureDoesNotMatchError`: AWSAccessKeyId, StringToSign, SignatureProvided, StringToSignBytes, CanonicalRequest, CanonicalRequestBytes
Fix CompleteMultipartUpload validation in the Azure backend so missing or empty `ETag` values return the appropriate S3 error instead of allowing a gateway panic.
Fix presigned authentication expiration validation to compare server time in `UTC`, matching the `UTC` timestamp used by presigned URL signing.
Add request ID and host ID support across S3 requests. Each request now receives AWS S3-like identifiers, returned in response headers as `x-amz-request-id` and `x-amz-id-2` and included in all XML error responses as RequestId and HostId. The generated ID structure is designed to resemble AWS S3 request IDs and host IDs.
The request signature calculation/validation for streaming uploads was previously delayed until the request body was fully read, both for Authorization header authentication and presigned URLs.
Now, the signature is validated immediately in the authorization middlewares without reading the request body, since the signature calculation itself does not depend on the request body. Instead, only the `x-amz-content-sha256` SHA-256 hash calculation is delayed.
Fixes#1986
When a client includes tagging, legal hold, or retention headers in a PutObject, CopyObject or CreateMultipartUpload request, the corresponding bucket policy permissions must be verified in addition to s3:PutObject:
`X-Amz-Tagging` - `s3:PutObjectTagging`
`X-Amz-Object-Lock-Legal-Hold` - `s3:PutObjectLegalHold`
`X-Amz-Object-Lock-Mode` - `s3:PutObjectRetention`
Previously, only s3:PutObject was checked, allowing users to set tagging, legal hold, and retention without having the required permissions. Now each action permission is check, if user tries to add them.
For CopyObject these permissions are checked on destination object.
Closes#1648Fixes#1980Fixes#1981
This PR implements browser-based POST object uploads for S3-compatible form uploads. It adds support for handling `multipart/form-data` object uploads submitted from browsers, including streaming multipart parsing so file content is not buffered in memory, POST policy decoding and evaluation, SigV4-based form authorization, and integration with the existing `PutObject` backend flow. The implementation covers the full browser POST upload path, including validation of required form fields, credential scope and request date checks, signature verification, metadata extraction from `x-amz-meta-*` fields, checksum field parsing, object tagging conversion from XML into the query-string format expected by `PutObject`, and browser-compatible success handling through `success_action_status` and `success_action_redirect`. It also wires the new flow into the router and metrics layer and adds POST-specific error handling and debug logging across policy parsing, multipart parsing, and POST authorization. AWS S3 also accepts the `redirect` form field alongside `success_action_redirect`, but since AWS has marked `redirect` as deprecated and is planning to remove it, this gateway intentionally does not support it.
This is a fixup of the codebase using:
go run golang.org/x/tools/go/analysis/passes/modernize/cmd/modernize@latest -fix ./...
This has no bahvior changes, and only updates safe changes for
modern go features.
Closes#1847
This PR introduces a global optional gateway CLI flag `--disable-acl` (`VGW_DISABLE_ACL`) to disable ACL handling. When this flag is enabled, the gateway ignores all ACL-related headers, particularly in `CreateBucket`, `PutObject`, `CopyObject`, and `CreateMultipartUpload`.
`GetBucketAcl` behavior is unchanged simply returning the bucket ACL config.
There's no change in object ACL actions(`PutObjectACL`, `GetObjectACL`). They return a`NotImplemented` error as before.
A new custom error is added for PutBucketAcl calls when ACLs are disabled at the gateway level. Its HTTP status code and error code match AWS S3’s behavior, with only a slightly different error message.
In the access-control checker, ACL evaluation is fully bypassed. If ACLs are disabled only the bucket owner gets access to the bucket and all grantee checks are ignored.
The PR also includes minor refactoring of the S3 API server and router. The growing list of parameters passed to the router’s Init method has been consolidated into fields within the router struct, initialized during router construction. Parameters not needed by the S3 server are no longer stored in the server configuration and are instead forwarded directly to the router.
Fixes#1870Fixes#1863
A validation has been added to **PutBucketCors** for `CORSRule.AllowedOrigins`. The `AllowedOrigins` list can no longer be empty—otherwise a **MalformedXML** error is returned. Additionally, each origin is now validated to ensure it does not contain more than one wildcard.
A similar validation has been added for `AllowedMethods`. The list must not be empty, or a **MalformedXML** error is returned. Previously, empty method values (e.g., `[]string{""}`) were incorrectly treated as valid. This has been fixed, and an **UnsupportedCORSMethod** error is now returned.
Fixes#1869
Generally, when object ownership is not explicitly specified during bucket creation, it defaults to `BucketOwnerEnforced`. With `BucketOwnerEnforced`, ACLs are disabled and any attempt to set one results in an `InvalidBucketAclWithObjectOwnership` error.
However, there is an edge case. When the `private` canned ACL is used during bucket creation—which is effectively the default ACL for all buckets—`BucketOwnerEnforced` is still permitted. Moreover, if no explicit object ownership is specified together with the `private` canned ACL, the ownership defaults to `BucketOwnerPreferred`.
This fix also resolves the issue with rclone bucket creation, since rclone sends `x-amz-acl: private` by default:
```
rclone mkdir vgw:test
```
The latest version of the go sdk has deprecate the s3 manager in
favor of the new transfermanager. This updates to the new
transfermanager functionality for the two places we were using
the manager, iam s3 and integration tests.
This also removes the pinning of nats nkeys since they have fixed
the tags in their repo now. And general cleanup of the go.mod.
Fixes#1765Fixes#1771
This PR addresses two issues:
1. CreateBucket was previously allowed when the gateway was running in read-only mode. It is now correctly blocked.
2. Write operations were permitted on public buckets in read-only mode because the public access checks in `auth.VerifyAccess` were evaluated before the read-only check. The read-only check now takes precedence, and all write operations on public buckets are blocked.
Fixes#1766Fixes#1750
This PR focuses on two bug fixes:
First, it blocks access to delete `DeleteMarkers` for the following operations by returning a `MethodNotAllowed` error: `PutObjectTagging`, `GetObjectTagging`, `DeleteObjectTagging`, `PutObjectLegalHold`, `GetObjectLegalHold`, `PutObjectRetention`, and `GetObjectRetention`.
Second, it removes the access check that previously prevented deleting a delete marker locked by a bucket default retention rule. A delete marker should always be allowed to be deleted.
Fixes#1741
An object delete request without a `versionId` results in the creation of a new delete marker in versioning-enabled buckets. Even if the latest object version is locked, a new delete marker must still be created.
This implementation skips the object lock check for delete requests in versioning-enabled buckets when the `versionId` is missing, allowing the delete marker to be created as expected.
Additionally, it introduces a flag in the `createObjVersion` method in POSIX to remove unnecessary xattr attributes from an object after creating a new object version. A delete marker must not carry object-specific attributes such as tagging, legal hold, or retention. Currently, the cleanup is limited to legal hold and retention attributes, but this list will be expanded after fixing issue #1751.
Closes#1731
Implements the admin `CreateBucket` (`PATCH /:bucket/create`) endpoint and CLI command, which create a new bucket with the provided owner access key ID. The endpoint internally calls the S3 `CreateBucket` API, storing the new owner information in the request context under the `bucket-owner` key. This value is then retrieved by the S3 API layer and the backends.
The endpoint uses the custom `x-vgw-owner` HTTP header to pass the bucket owner access key ID.
The admin CLI command mirrors `aws s3api create-bucket` and supports all flags implemented by the gateway (for example, `--create-bucket-configuration`, `--acl`, `--object-ownership`, etc.).
niksis02
Ben McClellandandGitHub
Andrei BesleagaandBen McClelland
Jihyeon Gim
Marc Singerandniksis02
