Add IAMAssumeRoleWithWebIdentity_github_oidc_live, the only web-identity test that exercises AssumeRoleWithWebIdentity against a real external OIDC provider end-to-end: GitHub Actions' own issuer, with real discovery-document fetch, JWKS fetch, RS256 signature verification, claims mapping, and session credential issuance. Every other web-identity test in the suite uses a fake token that never reaches real signature verification. The test registers a throwaway OIDC provider and trust role scoped to this repo (via a distinct test audience and repo-scoped sub condition), fetches a real ID token from GitHub's runtime endpoint, assumes the role, and confirms the issued session credentials work with a follow-up GetCallerIdentity call. It cleans up the role and provider unconditionally and skips itself when run outside a GitHub Actions job with id-token: write permission (e.g. local runs or fork PRs, where GitHub downgrades OIDC permissions to read-only).
Add functional-iam-oidc.yml to run this test in CI on push to main and on same-repo pull_request runs, isolated from the full iam suite since it's the only test needing id-token: write.
Add a SKIP counter and skipF() alongside the existing runF/passF/failF, and report it in the final RAN/PASS/FAIL summary, so a test opting out via skipF() (as this one does when OIDC env vars aren't present) is visible instead of silently absent from the count.
Implements the `AssumeRoleWithWebIdentity` and `GetCallerIdentity` STS actions, letting callers exchange an external OIDC token for temporary credentials scoped to an IAM role. Token handling covers JWT claim parsing, issuer/audience resolution (including `azp` override semantics), JWKS fetching and caching with `singleflight`-deduplicated refresh, and rate-limited forced refresh on unrecognized `kid` values. OIDC provider thumbprint fetching now performs a real TLS handshake verified against the system trust store and the provider hostname (previously `InsecureSkipVerify`), since the observed certificate is persisted as a long-lived trust anchor rather than used once and discarded; all discovery-document and JWKS fetches go through an SSRF-safe HTTP client with bounded redirects and response size.
Adds policy `Condition` block evaluation, supporting `String`, `Numeric`, `Date`, `Bool`, `BinaryEquals`, and `IpAddress` operators along with their `IfExists`/`Not` variants and `ForAllValues`/`ForAnyValues` set qualifiers, plus policy variable substitution (e.g. `${aws:username}`) in supported operators. Adds identity-based inline policy evaluation and a new IAM authorization middleware that authorizes each request against action, resource, and condition context together, applying the session-policy-intersects-role-policy semantics for assumed-role sessions.
Adds a new debug logger `--log-level` flag (`silent`/`debug`/`unsafe`), along with a tree-based XML masker that redacts secrets and tokens at the property level in logged request/response bodies instead of skipping the whole body. The old `--debug/VGW_DEBUG` flag is kept as a deprecated alias for `--log-level=debug`, printing a console warning that points users at `--log-level` for finer-grained control.
Fixes a Vault storage bug where CAS (check-and-set) writes always read the current document version as 0 because `kvVersion` asserted metadata as `float64` while the Vault client actually returns `json.Number`, causing every write past the first to be rejected as a concurrent modification. Also adds a constant-time `SecureCompare` for signature/token comparisons in sigv4 auth.
Adds an integration test suite (`iam_access_control.go`) covering IAM access control across user, role, and session identities.
Add support for `CreateOpenIDConnectProvider`, `GetOpenIDConnectProvider`, `ListOpenIDConnectProviders`, `DeleteOpenIDConnectProvider`, `AddClientIDToOpenIDConnectProvider`, `RemoveClientIDFromOpenIDConnectProvider`, and `UpdateOpenIDConnectProviderThumbprint` on both the internal and Vault storage backends, rounding out the standalone IAM service with the same OIDC identity provider management AWS IAM exposes.
CreateOpenIDConnectProvider validates the issuer URL, enforces the client ID and per-provider client ID list limits, and accepts an optional ThumbprintList. When the caller omits ThumbprintList, the provider auto-fetches the thumbprint by opening an outbound TLS connection to the issuer URL and hashing its top-level CA certificate, matching real AWS behavior. This auto-fetch is configurable: it can be turned off with the `--disable-oidc-thumbprint-autofetch` CLI flag (or the `VGW_IAM_DISABLE_OIDC_THUMBPRINT_AUTOFETCH` environment variable) for restricted or air-gapped deployments where the IAM server shouldn't make outbound connections, in which case an omitted ThumbprintList is rejected instead. AddClientIDToOpenIDConnectProvider and RemoveClientIDFromOpenIDConnectProvider manage a provider's client ID list, and UpdateOpenIDConnectProviderThumbprint replaces its thumbprint list, all with the same length and format validation applied at creation time.
Provider ARNs are derived from the issuer URL, and GetOpenIDConnectProvider and DeleteOpenIDConnectProvider resolve providers by ARN, returning NoSuchEntity when a provider doesn't exist. ListOpenIDConnectProviders returns the full set of stored providers. These actions are wired into the IAM API router and given their own XML response types under iamapi/types, with a new iamapi/internal/iamutil package handling URL validation, thumbprint fetching and normalization, and ARN construction shared across the controller methods.
Add support for the `PutRolePolicy`, `GetRolePolicy`, `DeleteRolePolicy`, and `ListRolePolicies` actions in the IAM-compatible gateway service, extending role management with the same inline-policy lifecycle already available for IAM users. `PutRolePolicy` validates the policy name and document, parses the document for AWS-compatible syntax and semantic errors (missing actions/resources, malformed ARNs, disallowed principals, duplicate statement IDs, and so on), and rejects documents once the role's aggregate inline-policy size would exceed `MaxInlinePolicyBytesPerRole` (10240 bytes, distinct from the 2048-byte quota enforced for users). Putting a policy under an existing name overwrites its document in place. `GetRolePolicy` and `DeleteRolePolicy` look up or remove a named inline policy from a role, returning a `NoSuchEntity` error when the role or the policy is not found. `ListRolePolicies` returns a role's inline policy names in sorted order with marker-based pagination.
These actions are implemented for both the internal file-backed store and the Vault-backed store, wired into the IAM API router, and given their own XML response types under `iamapi/types`. A new `NoSuchEntityRolePolicy` error was added to `iamapi/iamerr` to mirror the existing user-policy error.
Adds `CreateRole`, `GetRole`, `ListRoles`, `DeleteRole`, and `UpdateAssumeRolePolicy` to the standalone IAM service, following the same controller/storage patterns established for users. Both the internal filesystem/S3-backed store and the Vault-backed store implement the new `Storer` methods, with role-specific indexing and lookup helpers mirroring the existing user ones.
Role creation requires a trust policy, passed as `AssumeRolePolicyDocument`. A trust policy is a distinct kind of IAM policy document that governs who (or what) is allowed to assume a role, rather than what actions the role itself is permitted to perform. Its grammar is effectively the inverse of an identity policy: `Principal` is required, `Action`/`NotAction` values must carry the `sts:` prefix, and `Resource`/`NotResource` are forbidden. This is implemented in `iamapi/policy/trust.go` as a new validation path alongside the existing identity-policy validation, and is reused by `UpdateAssumeRolePolicy` when replacing a role's trust policy.
Also fixes user name uniqueness enforcement to be case-insensitive, matching AWS IAM behavior, and applies the same case-insensitive handling to role names. The internal store now maintains lowercase name indexes for both users and roles, and the Vault store resolves the canonical stored key via a case-insensitive list-and-compare fallback since Vault's KV paths are case-sensitive.
Add support for AWS-compatible inline identity-based policies on IAM
users, implementing the `PutUserPolicy`, `GetUserPolicy`, `DeleteUserPolicy`, and `ListUserPolicies` actions on both the internal and Vault storage
backends.
- iamapi/policy is a new package that parses and validates policy documents against IAM's parameter-level constraints (max length, allowed charset) and policy grammar (Version, Effect, mutually exclusive Action/NotAction and Resource/NotResource, vendor-prefixed actions, ARN-shaped resources, no Principal/NotPrincipal, unique Sids).
- `PutUserPolicy` creates or replaces a named inline policy on a user, enforcing a 2048-byte aggregate quota across all of a user's inline policies (MaxInlinePolicyBytesPerUser), matching the AWS IAM quota.
- `GetUserPolicy` returns a policy's document RFC 3986 percent-encoded, matching how real IAM encodes the PolicyDocument response element.
- `DeleteUserPolicy` removes a named inline policy from a user.
- `ListUserPolicies` returns a paginated, sorted list of a user's inline policy names, honoring Marker/MaxItems like the other IAM list APIs.
- `DeleteUser` is now rejected with a DeleteConflict error if the user still has inline policies attached, mirroring the existing access-key delete-conflict behavior.
Add `CreateAccessKey`, `UpdateAccessKey`, `DeleteAccessKey`, `ListAccessKeys`, and `GetAccessKeyLastUsed` actions for managing user access keys and retrieving their latest usage details.
Generate AWS-style access key IDs and secrets, validate key identifiers and statuses, enforce per-user key quotas, and prevent deleting users that still own access keys. Persist access keys across internal and Vault storage backends with ownership indexing, pagination, and IAM-compatible errors and XML responses.
Closes#1640
Add a standalone AWS IAM Query API implementation for managing IAM users through standard AWS SDKs and the AWS CLI.
Server usage
Start the IAM server with internal file-backed storage:
mkdir -p /tmp/versitygw-iam
./versitygw --port 127.0.0.1:7070 --access user --secret pass iam --dir /tmp/versitygw-iam
Start the IAM server with Vault KV v2 storage using AppRole:
VGW_IAM_VAULT_ROLE_SECRET=<role-secret> ./versitygw --port 127.0.0.1:7070 --access user --secret pass iam --vault-endpoint-url http://127.0.0.1:8200 --vault-auth-method approle --vault-role-id <role-id> --vault-mount-path kv --vault-secret-storage-path iam
Vault authentication also supports root tokens, separate authentication and secret-storage namespaces, custom mount paths, server certificate validation, and mutual TLS client certificates.
Configure the AWS CLI credentials used by the IAM server:
export AWS_ACCESS_KEY_ID=user
export AWS_SECRET_ACCESS_KEY=pass
export AWS_DEFAULT_REGION=us-east-1
Implemented IAM actions
CreateUser creates an IAM user with an AWS-compatible ARN, generated AIDA user ID, creation timestamp, optional path, and tags. It validates usernames, paths, tag limits, reserved tag prefixes, duplicate tag keys, and existing users.
aws --endpoint-url http://127.0.0.1:7070 iam create-user --user-name bob
aws --endpoint-url http://127.0.0.1:7070 iam create-user --user-name bob --path /engineering/ --tags Key=team,Value=storage
GetUser returns a stored user or the root identity when requested without a username through the IAM Query API.
aws --endpoint-url http://127.0.0.1:7070 iam get-user --user-name bob
ListUsers returns users in deterministic username order and supports path filtering, marker-based pagination, and MaxItems limits.
aws --endpoint-url http://127.0.0.1:7070 iam list-users
aws --endpoint-url http://127.0.0.1:7070 iam list-users --path-prefix /engineering/ --max-items 100
UpdateUser updates the username and/or path, recalculates the user ARN, and rejects conflicts with existing users.
aws --endpoint-url http://127.0.0.1:7070 iam update-user --user-name bob --new-user-name robert --new-path /platform/
DeleteUser permanently removes an IAM user and returns AWS-compatible errors for missing users.
aws --endpoint-url http://127.0.0.1:7070 iam delete-user --user-name robert
IAM protocol and authentication
- Support the AWS IAM Query protocol version 2010-05-08 over GET and POST form requests.
- Return AWS-compatible XML responses, error documents, status codes, request IDs, user metadata, and pagination fields.
- Authenticate root credentials with AWS Signature Version 4 for the IAM service in us-east-1.
- Support both Authorization-header and query-string SigV4 authentication.
- Validate credential scope, signed headers, timestamps, clock skew, content length, signatures, and unsupported signature or session-token modes.
- Add IAM-specific validation and error mapping for malformed requests, invalid actions, duplicate entities, missing users, throttling, and internal failures.
Storage implementations
- Add an internal JSON-backed store using iam.json and iam.json.backup with atomic temporary-file replacement, concurrent access protection, stable ordering, pagination, and persistence across restarts.
- Add a Vault KV v2 store with one secret per user, CAS-based duplicate protection, permanent deletion, AppRole reauthentication, namespace support, configurable authentication and KV mounts, root-token authentication, and TLS/mTLS configuration.
- Introduce a common Storer interface and require exactly one storage backend to be configured.
Server and embedding support
- Register the new `versitygw iam` command with environment-variable and CLI configuration for both storage backends.
- Add `embedgw.RunIAMAPI` and `IAMConfig` for embedding the IAM service in Go applications.
Gateway-level internal packages
- Add `internal/iamstore` as a reusable generic file-backed IAM persistence engine and migrate the existing gateway internal IAM service to it.
- Add `internal/sigv4auth` for shared SigV4 header and presigned-query parsing, canonical request generation, signature verification, and structured authentication errors.
- Refactor the S3 authentication paths to use the shared SigV4 implementation while preserving S3-specific error responses.
- Add `internal/httpctx` for shared Fiber context keys and AWS-style request ID handling.
- Add `internal/routekit` for shared query, form, and header route matchers.
- Add `internal/netutil` for reusable certificate storage, hostname-aware listeners, multi-address serving, TLS listeners, and UNIX socket handling.
- Update the custom SigV4 signer to honor an explicitly supplied signed-header list so unrelated headers do not alter IAM signatures.
Testing and CI
- Add AWS IAM SDK-based integration coverage for all supported user actions, header authentication, query authentication, validation, errors, filtering, and pagination.
- Split standalone IAM tests into `versitygw test iam` and retain existing gateway IAM tests under `versitygw test gw-iam`.
- Add unit coverage for controllers, authentication, routing, storage, embedding, listeners, request matching, persistence, and signing behavior.
- Add `runiamtests.sh` to exercise internal storage over HTTP and HTTPS plus Vault storage through AppRole.
- Add a dedicated IAM functional-test workflow with a Vault service and merged runtime coverage reporting.
- Include the IAM test runner in shellcheck and add the AWS IAM SDK dependency.
When uploads hit ENOSPC or EDQUOT, the server was returning the
correct S3 error but could close the connection while unread
request bytes were still in flight, which caused TCP resets and
surfaced as broken pipe/connection reset errors in SDKs instead
of a clean Insufficient Storage response. This change drains the
remaining upload body before returning the error so the response
can be delivered and the connection can close gracefully,
preserving correct client-visible behavior under disk-full and
quota-exceeded conditions.
Fixes#2209
Expose S3 option injection in embed gateway config so
integrators can attach request middleware and other
server behaviors without needing to re-implement
RunVersityGW().
This change adds Windows functional test execution in CI and updates
backend handling so windows filesystem error/path semantics map correctly
to expected S3 outcomes.
The only meta supported on windows right now is sidecar, so the tests
in windows mode also skip sidecar skips.
Future work is to address the skips and/or more clearly document
the unsupported/incompatible behavior on windows.
The windows support will still remain best effort, but these tests
should at least flag when future changes introduce incompatible
behavior on windows.
Fixes#2187
Enforce maximum default retention periods when parsing PutObjectLockConfiguration requests. Reject Days values greater than 36500 and Years values greater than 100 with InvalidArgument errors.
Fixes#2185
Add asterisk handling for `If-Match` and `If-None-Match` read preconditions across `GetObject`, `HeadObject`, `CopyObject`, and `UploadPartCopy`.
`If-Match`: `*` now matches any `ETag`, while `If-None-Match`: `*` returns `304 Not Modified`.
Fixes#2179
Treat a present but empty `Content-MD5` header as an invalid digest instead of handling it as if the header were absent. This makes PUT operations return `InvalidDigest` for empty `Content-MD5` values while preserving existing behavior for missing headers.
Fixes#2180Fixes#2181
Migrate the gateway from Fiber v2 to Fiber v3.3.0 and update the affected server, middleware, handler, controller, and test code for the new APIs.
Replace the deprecated Fiber filesystem middleware used by the WebUI with the Fiber v3 static middleware, serving the embedded WebUI assets from an fs.Sub filesystem.
Fix the request header limit handling regression by adding a temporary handler for Fiber v3/fasthttp small-buffer errors so oversized request headers return the expected regulated S3 error response.
Fix the debuglogger panic by reworking the boxed key/value formatter used for debug request and response dumps. The formatter now handles long header keys and values without producing invalid wrap widths, negative padding, or out-of-range string slices.
Integrate x-amz-website-redirect-location across object metadata flows so uploads, copies, multipart creation, HEAD, and GET preserve and return redirect locations, and website hosting applies object-level redirects from the stored value.
Sis NikoyanandGitHub
Ben McClellandandGitHub
dependabot[bot]andGitHub
Luke McCrone
Mart-Kuc
Peter Dahlberg