mirror of
https://github.com/versity/versitygw.git
synced 2026-10-22 06:12:39 +00:00
main
26
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Tung Lam
|
e63998fb31 |
fix: log the client IP from a trusted proxy header
Behind a reverse proxy the request log and the S3, admin, website and WebUI access logs record the proxy's address instead of the client's. Add a client-ip-header option that takes the logged address from X-Forwarded-For or X-Real-Ip. It is off by default, so an existing deployment logs exactly what it logged before. The header is only honoured when the connecting peer is listed in the new trusted-proxies list of IPs and CIDRs. Without that, the value is whatever the client sent: nginx's standard proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for appends to a client-supplied header, leaving the client's own address leftmost. X-Forwarded-For is walked right to left, skipping the listed proxies, so the first unlisted address is the client and a forged leftmost entry is never reached; the two-proxy case from #2243 works once both proxies are listed. Every header line is walked (a proxy may add its own line instead of appending), ip:port entries are accepted, and a request from an unlisted peer keeps the socket address. Setting the header with an empty trusted-proxies list fails startup rather than silently trusting every peer. X-Real-Ip reads the last header line, so a proxy that adds a line rather than replacing the client's is the one that speaks. An address carrying an IPv6 zone is rejected: the zone is free text that the space-delimited access log would write verbatim, shifting every field after RemoteIP. When every hop in X-Forwarded-For is a trusted proxy, the leftmost hop is logged, as nginx's real_ip_recursive does, since the socket peer is then the nearest proxy. The trusted-proxies list accepts the entry "unix" to trust the peer of a listener bound to a unix socket path, which fasthttp reports as 0.0.0.0. The resolver, middleware, log tag and parsers live in internal/clientip, shared by s3api, iamapi, website and webui; s3api/utils re-exports them for the S3 callers, so the standalone IAM service no longer pulls in the S3 side. On this branch go list -deps ./iamapi drops from 375 to 299 packages, against 298 on main. ctx.IP() is unchanged and still returns the socket peer address, because it feeds aws:SourceIp in IAM policy conditions and SourceIP in policy evaluation; only the address written to the logs and S3 event records comes from the header. Fixes #2243. |
||
  Ben McClellandandGitHub
|
485b4f22d7 |
Merge pull request #2466 from versity/ben/reject-sse-c
feat: validate SSE-C headers and reject SSE-C in posix and azure |
||
|
Ben McClelland
|
c354e1a2d3 |
feat: validate SSE-C headers and reject SSE-C in posix and azure
Add request plumbing for server-side encryption with customer-provided keys (SSE-C) and stop silently ignoring it on backends that cannot honor it. Previously SSE-C headers were dropped before reaching the backend, so posix stored the data unencrypted and returned success, and later reads without the key succeeded. Clients believed their data was encrypted when it was not. Partial or malformed SSE-C header sets were also accepted instead of failing with 400 as S3 does. Rejecting SSE-C explicitly is the safe behavior until posix implements it, and azure cannot faithfully map SSE-C onto customer-provided keys. This addresses part of #2193: invalid/partial SSE-C headers now fail with 400. Requiring SSE-C headers on reads of SSE-C objects depends on posix SSE-C support and remains open. The changes also reject unsupported SSE-S3, SSE-KMS, and DSSE-KMS requests instead of silently accepting them. And enforcea TLS for SSE-C by default, with --disable-sse-c-tls-enforcement as the opt-out in case the deployment handles TLS in front of the gateway. |
||
 niksis02
|
2ebe5be70a |
fix: reload standalone IAM and private-channel mTLS certs on SIGHUP
Fixes #2464 `versitygw iam` never set `IAMConfig.SigHup`, so the reload loop in `RunIAMAPI` never ran and the IAM service kept serving the certificates it started with. It is now connected to `gwcli.SigHup`. The signal forwarder used blocking sends into the one-slot `SigHup` and `SigDone` channels. In modes that never read `SigHup` (`iam` until this fix, `admin`, `utils`), the second SIGHUP left the forwarder stuck, every later SIGINT/SIGTERM was lost, and the process could only be killed. `forwardSignals` now sends without blocking, and a notification that is already pending stands in for the new one. The private listener loaded its client CA pool (`--private-client-ca`) only once. `netutil.TLSOptions.ClientCAs` is replaced by `GetClientCAs`, which `GetConfigForClient` calls on every handshake and which is backed by the new `netutil.CAPoolStorage`, so a reloaded bundle applies to the next connection. When no pool is loaded the listener refuses the client rather than falling back to the system roots. On SIGHUP the IAM service now also reloads the client CA pool and the certificate of the WebUI it hosts, in addition to its public and private certificates. The gateway's standalone IAM client loaded its client cert/key and server CA (`--iam-standalone-client-cert[-key]`, `--iam-standalone-server-ca`) only once. With the new `auth.CertReloader` interface, the gateway's SIGHUP handler reloads them through `IAMServiceStandalone.ReloadCerts`. It builds a new `http.Client` from the files, swaps it in atomically and closes the old client's idle connections. Requests already in flight finish on the old client, and a failed load keeps the current one. The transports now set `IdleConnTimeout`, so a connection left on a replaced client cannot stay open indefinitely. The IAM wrapper in `vgwrdma` re-exports `CertReloader` so the capability is not lost there. |
||
|
niksis02
|
966a6a8650 |
feat: admin API path prefix, Helm WebUI prefixes and separate IAM storage
Closes #2322 The admin API could not be served under a path prefix. Admin requests are SigV4-signed over the full path, so a reverse proxy or Gateway API route cannot strip a prefix before forwarding. That made it impossible to serve the WebUI and the admin API under one hostname. The new `--admin-path-prefix` (`VGW_ADMIN_PATH_PREFIX`) option mounts the admin routes under a single-segment prefix such as `/admin`, on `--admin-port` or, when that is unset, on the S3 port. Admin clients include the prefix in their endpoint URL, which the admin CLI and the WebUI already support. The prefix is limited to unreserved characters, because clients and the gateway encode other characters differently when signing. When the admin API shares the S3 port, the prefix must also differ from `--webui-s3-prefix`, otherwise the WebUI mount answers the admin requests. Auto-detected WebUI admin gateway URLs and the startup banner now include the prefix. The admin routes served on the S3 port now reuse `S3AdminRouter` instead of a duplicated route list. This also stops the standalone admin server from recording bucket creation under the `ActionAdminListBuckets` action. The Helm chart now exposes `webui.pathPrefix`, `webui.s3Prefix` and `admin.pathPrefix`. The WebUI gateway lists are also passed when only `webui.s3Prefix` is set, since the WebUI hosted on the S3 port does not require `webui.enabled`. Internal IAM data can now live on its own volume. `iam.dir` sets the IAM directory, and `iam.persistence` creates or references a dedicated PVC when `iam.enabled` is true and `iam.type` is `internal`. Otherwise IAM data stays in the `iam` subdirectory of the backend data volume, so existing releases are unaffected. The chart now rejects overlapping storage directories. `iam.dir` must be an absolute path outside `/mnt/data`, `gateway.backend.sidecarDir` and `gateway.backend.versioningDir`, and those three directories must not overlap each other. Deleting a bucket removes `<dir>/<bucket>` from the sidecar and versioning directories, so with nested directories, deleting a suitably named bucket could wipe IAM accounts, object versions or the entire backend data directory. |
||
|
niksis02
|
1c1272c8a5 |
feat: add per-provider OIDC discovery URL override
`AssumeRoleWithWebIdentity` always fetched a provider's discovery document from `<provider url>/.well-known/openid-configuration`, so an identity provider that issues tokens naming a public issuer while serving its metadata and keys on a cluster-internal path could not be used: reaching it meant relaxing the endpoint checks for every registered provider. `--oidc-discovery-url` moves that one fetch to an operator-named endpoint, which is how keys can be looked up over an optimized private path while the tokens themselves stay verifiable from the public internet against the issuer alone, as the JWT spec requires. The flag takes `<provider url>=<discovery url>` pairs, can be repeated once per provider, and is also read from `VGW_IAM_OIDC_DISCOVERY_URLS` as a comma-separated list; the Helm chart exposes the same list as `iamServer.oidc.discoveryUrls`. The discovery URL is fetched exactly as written, so it must carry the `/.well-known/openid-configuration` path when the provider serves it there. A malformed pair is rejected at startup rather than at the first assume-role call. Only the fetch moves. The provider URL is still what a token's `iss` claim is matched against, the fetched document's own `issuer` field must still equal it, and the key set still comes from the `jwks_uri` that document publishes. A configured discovery endpoint is named by the operator at startup rather than by a request, so it and the `jwks_uri` it publishes waive the private-address check for that provider's fetch chain only, without `--oidc-allow-private-endpoints` and its far broader effect on every other provider. Transport rules are unchanged: a plaintext discovery URL still requires `--oidc-allow-insecure-transport`. Thumbprint auto-fetch follows the override and pins the discovery endpoint's certificate chain, since that is the host every later fetch is verified against. |
||
 Jihyeon Gim
|
64a6ed0c37 |
rdma: synchronize publication acceptance with the drain boundary
Service the overflow list during normal operation: the worker publishes and clears it after every channel job, so a burst that exceeds the channel buffer drains as soon as the sink recovers instead of accumulating until shutdown. Make dispatch and the drain sweep share one critical section. An append either lands before the sweep and is drained, or runs after the worker exited and publishes inline; the check-then-send window that could strand a record between the two is closed. Request publications use the same boundary with drop semantics: once the drain finished, no owner can guarantee the sinks are still open, so the record is dropped rather than published. Never close the metrics datapoint channel. The forwarder exits through the canceled context after draining the buffer, the closed flag turns late producers into drops, and no send can race a closure. Implement the new Manager method in the test mock. |
||
|
Jihyeon Gim
|
55c82f5e4e |
rdma: size publications to session capacity and preserve committed PUT facts
Size the publication queue from the configured session limit instead of a fixed depth with an overflow semaphore: each session publishes exactly one terminal record, so the queue can never fill and native callbacks hand off without waiting. Remove the semaphore fallback. Admit verification failures under the shutdown barrier so an authentication publication cannot land after the sinks close. Give the metrics manager a lifetime independent of the gateway context so shutdown drain updates are counted, and pass the captured bucket explicitly so RC datapoints appear in bucket-filtered metrics. Track reservations by claim generation: release and publication validate the generation, so a stale claim cannot consume a newer owner's record. Install the reservation cleanup defer immediately after acquisition so a panic during authorization cannot orphan it. Preserve committed PUT facts independent of the native finalizer: when the backend object exists, record the commit, keep the committed byte count on the error publication, and still emit the object-created event. |
||
|
Jihyeon Gim
|
49b1c6f6bc |
rdma: publish RC transfer outcomes into the gateway operational services
Two-phase RC transfers were invisible to the access log, metrics, and bucket notifications: every outcome record on the S3 surface is driven by the fiber request context, and the RC wire requests carry the transfer session rather than the object. Publish one record per session by tracking the operational context from PREPARE through completion. A tracker table registers each successfully prepared session with its account, region, object, and start time; whichever path confirms the final outcome first (READY completion, READY failure, or the reaper teardown callback) claims the entry and publishes exactly once. Sessions that end before PREPARE succeeds publish a request record directly. The record is emitted through a synthetic fiber context carrying the object path and the captured locals, so the existing logger, metrics manager, and event sender produce the same schema as the S3 surface without any interface change. Bucket notifications fire for completed PUTs. The gateway creates the operational services inside RunVersityGW, after the RC routes exist. Add an OnServicesReady callback to the embedded gateway config and wire it in vgwrdma to hand the services to the RC routes and install the teardown callback. |
||
|
Ben McClellandandGitHub
|
8637b65793 |
Merge pull request #2373 from versity/sis/oidc-local-relaxations
feat: add OIDC endpoint relaxations for private/isolated networks |
||
|
niksis02
|
658c37907d |
feat: add OIDC endpoint relaxations for private/isolated networks
Closes #2364 `AssumeRoleWithWebIdentity` only ever trusted an `OIDC` provider reachable over verified `https`, at a publicly routable address, on the implicit `:443`. That posture is right for an internet-facing IdP but rejects every address an internal one can have, so a `SPIFFE/SPIRE` OIDC discovery provider in the same cluster — or as a sidecar in the same pod — could never be registered, let alone verified against, and no setting could express "this private address is the IdP". Two opt-in flags on `versitygw iam`, both off by default: `--oidc-allow-private-endpoints` Permit a provider `Url` resolving to a loopback/private/link-local address, and an explicit port. Transport is unchanged: still `https`, still fully verified (a self-signed in-cluster cert is trusted the way AWS documents, through `ThumbprintList`). `--oidc-allow-insecure-transport` Additionally permit plaintext `http` provider URLs, discovery/JWKS endpoints and redirects, and drop TLS verification (`thumbprint` pinning included) for `https` ones. Both apply uniformly to the thumbprint auto-fetch at `CreateOpenIDConnectProvider` time and to the discovery-document plus `JWKS` fetch at `AssumeRoleWithWebIdentity` time. Neither weakens anything past the endpoint: signature verification, issuer matching, audience and trust policy evaluation are untouched, and the DNS-resolve-once/dial-the-resolved-IP shape stays in place so a rebind still cannot redirect a connection. An `http` provider keeps its scheme in its stored `Url`, `ARN` and `iss` matching, rather than being stripped like an `https` one — otherwise `"http://host"` and `"https://host"` would collapse onto a single ARN and storage key and each could satisfy the other's trust policy. It also stores an empty `ThumbprintList` rather than failing: a plaintext provider presents no certificate to thumbprint. Helm: `iamServer.oidc.{allowPrivateEndpoints,allowInsecureTransport}`, alongside `disableThumbprintAutofetch` moved into the same block (the flat `iamServer.disableOidcThumbprintAutofetch` stays honored). Chart `0.4.1 -> 0.4.2`. The WebUI's create-provider form no longer rejects `http` URLs and ports client-side; it cannot see the service's settings, so those two rules are left to the server, whose error surfaces as a toast like any other. |
||
|
Jihyeon Gim
|
6eb574dc1a |
rdma: expose live RC sessions on the admin server
Add a SessionsSnapshot view over the new C ABI entry point and serve it from the admin server as GET /rc-sessions. The admin server gains a WithAdminRoute option so an embedding binary can register extra admin routes that run with the same signature verification and admin checks as the built-in endpoints; vgwrdma registers the snapshot there when the RC feature is enabled. The route replies with the usual XML error surface so unsigned or non-admin requests get a 403 rather than a generic 500. Stub builds return a not supported error, keeping the build matrix unchanged. |
||
|
Ben McClelland
|
5f9041ff5f |
feat: support admin/access logs on standard streams
The access-log and admin-access-log options now accept stdout, stderr, or - for stdout in addition to file paths. Standard stream destinations are kept open during shutdown and SIGHUP handling, while file destinations continue to support normal reopen behavior for log rotation. CLI help, embedded config comments, and the example configuration describe the new destination values. Fixes #2245 |
||
|
|
535cc9d521 |
feat: add the hipobj-rc-v2 control routes to the vgwrdma gateway
* rdma: add the hipobj-rc-v2 control routes to the vgwrdma gateway Mount the three control routes (prepare, ready, cancel) on the S3 port behind the standard SigV4 middleware. The routes own authentication-adjacent policy the C server cannot see: the middleware wrapper yields to the handler on success, READY and CANCEL re-read the account through the IAM cache bypass so mid-flow deletions and credential rotations take effect immediately, and every object access re-authorizes against the decoded bucket and key. The READY handler implements the session ownership contract: the completion-reference finalizer installs only after the transfer claim succeeds, the PUT path hands the reference to the put view exactly at the borrow point, and the FINAL reply carries the stored object's metadata. Backend I/O runs under a context merged with the RC service context so shutdown unblocks in-flight handlers, with a bounded pool for the fresh IAM lookups. vgwrdma starts the session server alongside the gateway when an RDMA interface is configured, tears it down on exit, and shuts the IAM service down on any startup failure. embedgw learns the readonly flag for the object access checks the routes share. Signed-off-by: Jihyeon Gim <potatogim@potatogim.net> * rdma: add the missing stub handlers for non-Linux builds The non-Linux rcroutes stub exposed only Register while the vgwrdma gateway registers the prepare/ready/cancel handlers directly, so cross-compiling cmd/vgwrdma failed with undefined methods. Add the three stub handlers answering 501 Not Implemented and let Register reuse them, matching the Linux Handler API surface. * auth: drop the duplicated GetUserAccountFresh definition The rebase onto main (which already carries GetUserAccountFresh from the iam-cache-fresh change) kept both copies of the method, breaking the build with a redeclaration error. Remove the second copy so the method is defined once. * rdma: address the review findings on the control route wiring Drop the unused Handler.Register from both build variants: the gateway mounts the three control routes through s3api.WithRoute so the SigV4 verifier wrapper (rcAuth) runs in front of each handler, and nothing else calls Register. Clear iamOwned only when RunVersityGW returns nil. It shuts the IAM service down itself at the end of its shutdown sequence, but its early failure paths return before reaching that point, so the deferred shutdown must keep covering those errors. Remove the unused rcserver.SessionInfo parameter from sizeOf; the transferred byte count comes from the READY response alone. * rdma: keep transient IAM failures retryable in the fresh revalidation The fresh account revalidation turned every GetUserAccountFresh error into 403, which reports transient backend failures (LDAP timeouts, network errors) as a revoked account and leaves the client no room to retry. Only a confirmed missing account (auth.ErrNoSuchUser) means that; answer anything else with 503 so clients can retry the request. * rdma: make the IAM shutdown exactly-once and keep gateway errors visible The gateway and RunVersityGW share the IAM service, and which side shut it down could not be told from the return value: runtime failures return after RunVersityGW already shut the service down, while early setup failures return before any shutdown happens. The iamOwned flag therefore either shut the service down twice or leaked it depending on the error, and the error itself was dropped. Wrap the service so Shutdown runs exactly once no matter which side calls it, keep the deferred shutdown for every early failure path, and return the gateway error again. The wrapper re-exposes the optional interfaces (fresh account reads, signing keys, policy evaluation, fixed bucket ownership) so feature detection through the IAM service keeps working. * rdma: reuse the SigV4 account for RC control requests READY and CANCEL are independently authenticated SigV4 requests. Use the account resolved by the normal SigV4 path instead of bypassing the IAM cache a second time. This aligns RC revocation latency with other signed S3 requests and removes the extra backend IAM lookup, its concurrency cap, and the RC-specific IAM error mapping. The session owner check and the READY target and operation authorization are unchanged. * rdma: reword the READY reauthorization comment The comment implied a revocation inside the session window always takes effect at READY, but the account used here is the one SigV4 resolved, which may be a cached entry. State what the check does without claiming account-cache freshness. * rdma: preserve IAM cache behavior and standalone region --------- Signed-off-by: Jihyeon Gim <potatogim@potatogim.net> Co-authored-by: Ben McClelland <ben.mcclelland@versity.com> |
||
|
niksis02
|
afbee5be01 |
fix: role last-used tracking, and record S3 requests in last-used metadata
Role last-used tracking was missing entirely - `GetRole` returned a `RoleLastUsed` element that nothing ever wrote, rendering the zero time instead of the empty element AWS returns for an unused role - and access key last-used only ever saw the `IAM`/`STS` control plane, so a credential used exclusively against the S3 gateway reported as never used. Roles now record a use whenever a request authenticates with one of their session credentials, through a new `Storer.RecordRoleUsage` mirroring `RecordAccessKeyUsage`, gated on the session's role still being the one it was minted against so a session outliving its role can't attribute its use to a same-named replacement. `LastUsedDate` became a `*time.Time` so an unused role renders as an empty element. Both records now cover the S3 data plane as well: the gateway sends its configured region and `s3` on evaluate-policy and the IAM service records the caller there, so `GetAccessKeyLastUsed's` `ServiceName` is now iam, sts or s3. That call was chosen over derive-signing-key, which runs before signature verification and takes its region and service from the caller's own `Authorization` header - recording there would let anyone who knows an access key id refresh and poison another identity's audit record. Requests denied by a bucket policy or made against a public bucket are not recorded, since neither reaches identity-policy evaluation. To keep per-request recording affordable, an update is skipped while the stored record has the same service and region and is under a minute old; a change of either is written through immediately. Assuming a role is not a use, a request denied by an identity policy is, and both successful and denied S3 requests update the record. Also moves the `OIDC-dependent` tests into the `s3-iam-session` group so runoidctests.sh runs a single group. |
||
|
Ben McClelland
|
a8c5810bee |
feat: allow supplying an external IAM service implementation to embedgw
Add Config.IAMService (auth.IAMService) so embedders that already have their own IAM implementation can inject it directly instead of relying on one of the gateway's built-in backends (local dir, LDAP, Vault, S3-backed, FreeIPA, or the standalone IAM service). When set, it takes priority over all other IAM backend configuration and RunVersityGW skips calling auth.New entirely. This avoids forcing embedders to duplicate their own account store as one of the built-in IAM backends just to satisfy the gateway's IAM interface. |
||
|
niksis02
|
abb3b27149 |
feat: add standalone IAM support in WebGUI
Gates bucket listing behind an identity policy, lets browsers reach the standalone IAM API, and turns the WebUI into a dashboard for S3, IAM, or both.
**Bucket listing.** `ListBuckets` is now gated by the new `s3:ListAllMyBuckets` action, evaluated against `arn:aws:s3:::*`. The request names no bucket, so only identity policies apply — there is no resource policy to combine with, which is the same shape `CreateBucket` already had, so both now share one identity-only evaluation path. Root and admin bypass it, and backends with no identity-policy layer keep listing as before since their listing is already narrowed to the caller's own buckets. The action is IAM-only and is deliberately absent from the bucket-policy action list.
**Fixed bucket ownership.** The standalone IAM client has no per-user ownership to express — accounts are all plain users, cannot be enumerated, and access is decided by policy rather than ACL — so it now implements `auth.FixedBucketOwner` and every bucket is owned by root. Bucket creation stops resolving an owner, `ListBuckets` returns every bucket to every caller (what they may then do with one stays a per-request policy decision), and the admin `ChangeBucketOwner` reports method-not-supported. Other IAM backends are untouched.
**IAM service CORS.** `--cors-allow-origin` now applies to the `iam` command: it answers preflights and stamps the CORS headers, mirroring back the requested method and headers rather than enumerating the SigV4 header set. Without it no browser can reach the IAM API at all, so setting `--webui` without it falls back to `*` with a warning. The chart gets `iamServer.corsAllowOrigin`.
**WebUI.** New IAM pages for users, roles and OIDC providers, signing IAM/STS query-form requests directly from the browser. Navigation is capability-gated rather than role-gated: on sign-in the session probes the S3, admin and IAM endpoints independently and each page shows only what those credentials actually reach, so one build serves an IAM-only dashboard, an S3-only dashboard, and a combined one. The login page takes an optional IAM endpoint, seeded from the new `--webui-iam-gateways` (chart: `webui.iamGateways`) — never auto-detected, since the IAM service is a separate process. The WebUI can also be hosted by `versitygw iam` itself, for deployments with no S3 gateway behind it.
**The admin API is ignored once an IAM endpoint is in play.** The IAM service is then the user directory and bucket ownership is fixed, which leaves the admin API no job: the session is given no admin endpoint at all, its login field is hidden, `users.html` redirects to its IAM counterpart, and every admin-only surface stays off screen. Dashboard and Buckets remain available to any S3 session in such a deployment, running on the S3 and IAM APIs alone and surfacing each denial per action instead of redirecting.
Also fixes two WebUI bugs: embedded assets went out with a zero modification time and no `Cache-Control`, so browsers treated them as fresh for centuries and an upgraded gateway served new HTML against stale JS — they now revalidate against an ETag; and the login page's advanced-options section clipped its last field, since it animated to a height named in the stylesheet rather than the one it measures now.
**Usage**
IAM-only dashboard, served by the IAM service:
versitygw iam --port :7076 --webui :8080 --cors-allow-origin http://localhost:8080/
IAM + S3, dashboard served by the IAM service — point it at the gateway with `--webui-gateways`, and let the gateway accept the dashboard's origin:
versitygw iam --port :7076 --webui :8080 --webui-gateways http://localhost:7070/ --cors-allow-origin http://localhost:8080/
versitygw --port :7070 --cors-allow-origin http://localhost:8080/ posix /data
IAM + S3, dashboard served by the S3 gateway — point it at the IAM service with `--webui-iam-gateways`, and let the IAM service accept the dashboard's origin:
versitygw --port :7070 --webui :8080 --webui-iam-gateways http://localhost:7076/ posix /data
versitygw iam --port :7076 --cors-allow-origin http://localhost:8080/
|
||
|
niksis02
|
a4d4519ffe |
feat: version the private IAM protocol between gateway and standalone service
The S3 gateway and the standalone IAM service exchange authorization decisions over the private endpoints, where a version skew is silently unsafe in both directions: an older service drops a request field it does not know (a `Condition` block, say) and evaluates fail-open, while an older gateway ignores a response field it does not know and misses a deny the service intended. Neither side could previously detect either case. Both peers now declare a protocol version on every exchange via the `X-Vgw-Private-Protocol` header — the gateway on each request, the service on each response, error responses included — and each refuses a peer it cannot serve safely. The service rejects a gateway below `MinClientProtocol` with a `ProtocolMismatch` code; the gateway rejects a service older than the `ProtocolVersion` it speaks, and rejects a response carrying no version at all, since no build of this protocol omits the header and something else answering on that address should not be interpreted as an IAM decision. `ParseProtocolVersion` is shared by both sides and deliberately strict: an unreadable value is a mismatch, never an assumed default. A new root-signed `/private/version` endpoint reports the protocol version, the minimum client the service will serve, and the build tag (`WithPrivateServerVersion`). It is exempt from the service's own client-version check so it can still answer a gateway the service refuses — which is how that gateway learns why. Being authenticated like every other private endpoint, it also lets the gateway's startup probe verify its own credential and its mTLS transport in the same round trip. The gateway probes it once in `NewIAMServiceStandalone` rather than discovering a skew as an opaque per-request 500. An incompatible service is fatal after a 30s window, since a gateway that cannot authorize a single request is more useful refusing to start with the reason in its log; an unreachable one is only a warning, because the two processes legitimately start in parallel and every request checks the version regardless. Only conditions that can resolve on their own are retried — a rejected credential is reported immediately. |
||
|
niksis02
|
2147a0c304 |
feat: integrate standalone IAM service with S3 gateway for identity-based policy enforcement
Fixes #1327 Fixes #1567 Closes #2264 Wires the S3 gateway up to the standalone IAM service so identity policies, not just bucket policies and ACLs, are enforced on the S3 data plane. The gateway authenticates SigV4 requests by calling new private derive-signing-key and resolve-identity endpoints on the IAM service instead of holding secrets itself, and evaluates identity policy through the same PolicyEvaluator path added to auth.VerifyAccess, combined with the bucket policy using explicit-deny-wins precedence. The private endpoints are served over their own mTLS listener (new iamapi/private package, genmtlscerts.sh to generate test material, and client-cert support in internal/netutil), separate from the public IAM API. As part of this the vendored aws/signer/v4 package is deleted and replaced by a pure-Go SigV4 implementation in internal/sigv4auth, which now reads canonical request data directly off the fiber.Ctx instead of reconstructing an http.Request, and is shared by both the S3 request-signing verification and the new private-endpoint signing. DeleteObjects moves from an all-or-nothing authorization check to true partial success: VerifyObjectsAccess evaluates every object in a batch independently against both the identity policy and any object lock, so a denial or a locked object only removes that key from the batch instead of failing the whole request. It also batches the identity-policy round trip and the bucket-policy fetch once per request rather than once per object, and separates plain deletes from versioned ones since a versioned delete needs s3:DeleteObjectVersion rather than s3:DeleteObject. Object lock handling got a few correctness fixes alongside this: a bypass is now modeled as BypassNone/BypassRequested/BypassOverwrite rather than a single bool, because root's blanket ability to override a GOVERNANCE retention should only apply when the client actually asked to bypass it (DeleteObject/DeleteObjects/PutObjectRetention), not when the gateway is silently replacing a locked object via an overwrite, which needs the permission from everyone including root. Retention changes are now correctly classified as an extension (allowed under plain s3:PutObjectRetention) versus a weakening (date or mode change, which needs the bypass permission), and a COMPLIANCE lock can never be weakened by anyone regardless of permissions, matching AWS. Separately, VerifyObjectCopyAccess had a readonly-mode gap: it returned early for root/admin before ever calling VerifyAccess, so the readonly check inside VerifyAccess never ran for them on CopyObject; access checks are now ordered so the readonly gate always applies before any root/admin bypass, for copy as well as every other write path. Bucket policies also gained Condition block support, via a new shared internal/condition package moved out of the IAM policy package since both bucket and identity policies share the same evaluation semantics. It implements the full AWS operator set — String{Equals,NotEquals,EqualsIgnoreCase,NotEqualsIgnoreCase,Like,NotLike}, Numeric{Equals,NotEquals,LessThan,LessThanEquals,GreaterThan,GreaterThanEquals}, Date{Equals,NotEquals,LessThan,LessThanEquals,GreaterThan,GreaterThanEquals}, Bool, BinaryEquals, Arn{Equals,Like,NotEquals,NotLike}, IpAddress/NotIpAddress, and Null — along with the ForAllValues/ForAnyValue set qualifiers and the IfExists modifier. A new requestConditionContext builds the per-request keys a bucket policy's Condition block can reference — aws:SourceIp, aws:SecureTransport, aws:CurrentTime, aws:EpochTime, aws:UserAgent, aws:Referer, s3:prefix, s3:delimiter, s3:max-keys, s3:x-amz-acl, s3:VersionId — following AWS's own per-action rules for which keys a given S3 operation actually populates. Identity-derived keys such as aws:PrincipalArn and aws:username are deliberately left unwired here, since the gateway has no way to know them; the standalone IAM service fills those in itself when it evaluates an identity policy. Also added new integration test suites for S3-side IAM: s3_iam_access_control.go and s3_iam_session_access_control.go cover identity-policy enforcement and session-credential requests against real S3 operations, alongside expanded OIDC/web-identity coverage and a new runoidctests.sh runner wired into the OIDC GitHub Actions workflow. |
||
|
niksis02
|
ab2b816633 |
feat: add STS web identity federation, IAM policy Condition support, and access control enforcement
Implements the `AssumeRoleWithWebIdentity` and `GetCallerIdentity` STS actions, letting callers exchange an external OIDC token for temporary credentials scoped to an IAM role. Token handling covers JWT claim parsing, issuer/audience resolution (including `azp` override semantics), JWKS fetching and caching with `singleflight`-deduplicated refresh, and rate-limited forced refresh on unrecognized `kid` values. OIDC provider thumbprint fetching now performs a real TLS handshake verified against the system trust store and the provider hostname (previously `InsecureSkipVerify`), since the observed certificate is persisted as a long-lived trust anchor rather than used once and discarded; all discovery-document and JWKS fetches go through an SSRF-safe HTTP client with bounded redirects and response size.
Adds policy `Condition` block evaluation, supporting `String`, `Numeric`, `Date`, `Bool`, `BinaryEquals`, and `IpAddress` operators along with their `IfExists`/`Not` variants and `ForAllValues`/`ForAnyValues` set qualifiers, plus policy variable substitution (e.g. `${aws:username}`) in supported operators. Adds identity-based inline policy evaluation and a new IAM authorization middleware that authorizes each request against action, resource, and condition context together, applying the session-policy-intersects-role-policy semantics for assumed-role sessions.
Adds a new debug logger `--log-level` flag (`silent`/`debug`/`unsafe`), along with a tree-based XML masker that redacts secrets and tokens at the property level in logged request/response bodies instead of skipping the whole body. The old `--debug/VGW_DEBUG` flag is kept as a deprecated alias for `--log-level=debug`, printing a console warning that points users at `--log-level` for finer-grained control.
Fixes a Vault storage bug where CAS (check-and-set) writes always read the current document version as 0 because `kvVersion` asserted metadata as `float64` while the Vault client actually returns `json.Number`, causing every write past the first to be rejected as a concurrent modification. Also adds a constant-time `SecureCompare` for signature/token comparisons in sigv4 auth.
Adds an integration test suite (`iam_access_control.go`) covering IAM access control across user, role, and session identities.
|
||
|
niksis02
|
ec579ca841 |
feat: add IAM OIDC provider CRUD
Add support for `CreateOpenIDConnectProvider`, `GetOpenIDConnectProvider`, `ListOpenIDConnectProviders`, `DeleteOpenIDConnectProvider`, `AddClientIDToOpenIDConnectProvider`, `RemoveClientIDFromOpenIDConnectProvider`, and `UpdateOpenIDConnectProviderThumbprint` on both the internal and Vault storage backends, rounding out the standalone IAM service with the same OIDC identity provider management AWS IAM exposes. CreateOpenIDConnectProvider validates the issuer URL, enforces the client ID and per-provider client ID list limits, and accepts an optional ThumbprintList. When the caller omits ThumbprintList, the provider auto-fetches the thumbprint by opening an outbound TLS connection to the issuer URL and hashing its top-level CA certificate, matching real AWS behavior. This auto-fetch is configurable: it can be turned off with the `--disable-oidc-thumbprint-autofetch` CLI flag (or the `VGW_IAM_DISABLE_OIDC_THUMBPRINT_AUTOFETCH` environment variable) for restricted or air-gapped deployments where the IAM server shouldn't make outbound connections, in which case an omitted ThumbprintList is rejected instead. AddClientIDToOpenIDConnectProvider and RemoveClientIDFromOpenIDConnectProvider manage a provider's client ID list, and UpdateOpenIDConnectProviderThumbprint replaces its thumbprint list, all with the same length and format validation applied at creation time. Provider ARNs are derived from the issuer URL, and GetOpenIDConnectProvider and DeleteOpenIDConnectProvider resolve providers by ARN, returning NoSuchEntity when a provider doesn't exist. ListOpenIDConnectProviders returns the full set of stored providers. These actions are wired into the IAM API router and given their own XML response types under iamapi/types, with a new iamapi/internal/iamutil package handling URL validation, thumbprint fetching and normalization, and ARN construction shared across the controller methods. |
||
|
niksis02
|
e012a6fd01 |
feat: add AWS-compatible standalone IAM service
Closes #1640 Add a standalone AWS IAM Query API implementation for managing IAM users through standard AWS SDKs and the AWS CLI. Server usage Start the IAM server with internal file-backed storage: mkdir -p /tmp/versitygw-iam ./versitygw --port 127.0.0.1:7070 --access user --secret pass iam --dir /tmp/versitygw-iam Start the IAM server with Vault KV v2 storage using AppRole: VGW_IAM_VAULT_ROLE_SECRET=<role-secret> ./versitygw --port 127.0.0.1:7070 --access user --secret pass iam --vault-endpoint-url http://127.0.0.1:8200 --vault-auth-method approle --vault-role-id <role-id> --vault-mount-path kv --vault-secret-storage-path iam Vault authentication also supports root tokens, separate authentication and secret-storage namespaces, custom mount paths, server certificate validation, and mutual TLS client certificates. Configure the AWS CLI credentials used by the IAM server: export AWS_ACCESS_KEY_ID=user export AWS_SECRET_ACCESS_KEY=pass export AWS_DEFAULT_REGION=us-east-1 Implemented IAM actions CreateUser creates an IAM user with an AWS-compatible ARN, generated AIDA user ID, creation timestamp, optional path, and tags. It validates usernames, paths, tag limits, reserved tag prefixes, duplicate tag keys, and existing users. aws --endpoint-url http://127.0.0.1:7070 iam create-user --user-name bob aws --endpoint-url http://127.0.0.1:7070 iam create-user --user-name bob --path /engineering/ --tags Key=team,Value=storage GetUser returns a stored user or the root identity when requested without a username through the IAM Query API. aws --endpoint-url http://127.0.0.1:7070 iam get-user --user-name bob ListUsers returns users in deterministic username order and supports path filtering, marker-based pagination, and MaxItems limits. aws --endpoint-url http://127.0.0.1:7070 iam list-users aws --endpoint-url http://127.0.0.1:7070 iam list-users --path-prefix /engineering/ --max-items 100 UpdateUser updates the username and/or path, recalculates the user ARN, and rejects conflicts with existing users. aws --endpoint-url http://127.0.0.1:7070 iam update-user --user-name bob --new-user-name robert --new-path /platform/ DeleteUser permanently removes an IAM user and returns AWS-compatible errors for missing users. aws --endpoint-url http://127.0.0.1:7070 iam delete-user --user-name robert IAM protocol and authentication - Support the AWS IAM Query protocol version 2010-05-08 over GET and POST form requests. - Return AWS-compatible XML responses, error documents, status codes, request IDs, user metadata, and pagination fields. - Authenticate root credentials with AWS Signature Version 4 for the IAM service in us-east-1. - Support both Authorization-header and query-string SigV4 authentication. - Validate credential scope, signed headers, timestamps, clock skew, content length, signatures, and unsupported signature or session-token modes. - Add IAM-specific validation and error mapping for malformed requests, invalid actions, duplicate entities, missing users, throttling, and internal failures. Storage implementations - Add an internal JSON-backed store using iam.json and iam.json.backup with atomic temporary-file replacement, concurrent access protection, stable ordering, pagination, and persistence across restarts. - Add a Vault KV v2 store with one secret per user, CAS-based duplicate protection, permanent deletion, AppRole reauthentication, namespace support, configurable authentication and KV mounts, root-token authentication, and TLS/mTLS configuration. - Introduce a common Storer interface and require exactly one storage backend to be configured. Server and embedding support - Register the new `versitygw iam` command with environment-variable and CLI configuration for both storage backends. - Add `embedgw.RunIAMAPI` and `IAMConfig` for embedding the IAM service in Go applications. Gateway-level internal packages - Add `internal/iamstore` as a reusable generic file-backed IAM persistence engine and migrate the existing gateway internal IAM service to it. - Add `internal/sigv4auth` for shared SigV4 header and presigned-query parsing, canonical request generation, signature verification, and structured authentication errors. - Refactor the S3 authentication paths to use the shared SigV4 implementation while preserving S3-specific error responses. - Add `internal/httpctx` for shared Fiber context keys and AWS-style request ID handling. - Add `internal/routekit` for shared query, form, and header route matchers. - Add `internal/netutil` for reusable certificate storage, hostname-aware listeners, multi-address serving, TLS listeners, and UNIX socket handling. - Update the custom SigV4 signer to honor an explicitly supplied signed-header list so unrelated headers do not alter IAM signatures. Testing and CI - Add AWS IAM SDK-based integration coverage for all supported user actions, header authentication, query authentication, validation, errors, filtering, and pagination. - Split standalone IAM tests into `versitygw test iam` and retain existing gateway IAM tests under `versitygw test gw-iam`. - Add unit coverage for controllers, authentication, routing, storage, embedding, listeners, request matching, persistence, and signing behavior. - Add `runiamtests.sh` to exercise internal storage over HTTP and HTTPS plus Vault storage through AppRole. - Add a dedicated IAM functional-test workflow with a Vault service and merged runtime coverage reporting. - Include the IAM test runner in shellcheck and add the AWS IAM SDK dependency. |
||
|
Ben McClelland
|
6f1dfe84e5 |
feat: add options extensions to embed config
Expose S3 option injection in embed gateway config so integrators can attach request middleware and other server behaviors without needing to re-implement RunVersityGW(). |
||
|
niksis02
|
4d391cabc8 |
feat: migrate Fiber to v3.3.0
Fixes #2180 Fixes #2181 Migrate the gateway from Fiber v2 to Fiber v3.3.0 and update the affected server, middleware, handler, controller, and test code for the new APIs. Replace the deprecated Fiber filesystem middleware used by the WebUI with the Fiber v3 static middleware, serving the embedded WebUI assets from an fs.Sub filesystem. Fix the request header limit handling regression by adding a temporary handler for Fiber v3/fasthttp small-buffer errors so oversized request headers return the expected regulated S3 error response. Fix the debuglogger panic by reworking the boxed key/value formatter used for debug request and response dumps. The formatter now handles long header keys and values without producing invalid wrap widths, negative padding, or out-of-range string slices. |
||
|
niksis02
|
1625c5963e |
feat: improve static website hosting support
Enhances the static website hosting implementation with more complete S3-compatible behavior across request handling, backend storage, validation, CORS, and errors. Adds dedicated website endpoint handling for GET, HEAD, and OPTIONS requests, including index document resolution, error document serving, redirect-all support, pre-fetch and post-error routing rules, query string preservation in redirects, public access checks before object reads, and method-not-allowed responses. Improves error handling for website responses by returning S3-compatible HTML error bodies with request IDs, host IDs, x-amz-error-code, x-amz-error-message, and specialized error fields. This also fixes website-related validation errors to return more accurate S3-style error codes and messages, including invalid redirect protocols, invalid HTTP redirect/error codes, conflicting routing rule replacements, routing rule limits, and oversized website configuration requests. Adds website CORS support for GET, HEAD, and OPTIONS preflight requests, including bucket CORS lookup through website host bucket resolution, allowed origin/method/header validation, exposed header handling, ETag exposure, Vary headers, max-age handling, and CORS access-denied responses. Adds debug logging around website configuration parsing, validation failures, CORS checks, backend lookup failures, and internal website error paths to make failures easier to diagnose. Adds compressed website configuration storage so larger configs fit backend metadata limits, including gzip storage for POSIX extended attributes and base64-encoded compressed metadata for Azure. Also adds Azure PutBucketWebsite, GetBucketWebsite, and DeleteBucketWebsite support. Adds and expands test coverage for website config validation, S3-compatible HTML error bodies, website routing behavior, public access enforcement, HEAD behavior, CORS handling, PutBucketWebsite limits, and end-to-end website hosting through a Docker-based dnsmasq test setup and CI workflow. |
||
|
Ben McClelland
|
20939bd7b4 |
feat: extract gateway runtime into embeddable package
Move the runGateway implementation from cmd/versitygw/main.go into a new embedgw package, exposing RunVersityGW(ctx, Backend, *Config) and a Config struct. This allows external applications to embed and run the VersityGW S3 gateway as a library. |