mirror of
https://github.com/versity/versitygw.git
synced 2026-09-24 00:44:23 +00:00
v1.8.0
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Jihyeon Gim
|
726d65dbc9 |
rdma: gate the RC data plane behind a dedicated flag
Add --rdma-rc-enable (VGW_RDMA_RC_ENABLE, default false) so the RC control routes and data plane start without implying the cuObject v1 backend. The global CLI hook resolves the mode first: gateway commands require either --rdma-ip or --rdma-rc-enable, and neither path implies the other, so a v2-only deployment boots without a v1 address. The v1 port, retry, pool, and DCI validations also ran for every mode, so stale v1 environment values blocked v2-only startup with unrelated errors. Those validations moved behind the v1 check as a cgo-free helper in internal/rdmamode, exercised alongside the mode matrix, and the CQ-depth limit keeps its 32-bit boundary check there. The RC data plane builds its IAM service, starts the session server, and mounts the three control routes behind SigV4. Startup and shutdown own the backend chain through idempotent guards: the gateway wraps the input backend in a once guard and defers a rollback closure that follows the chain as it grows; the completed v1 chain gets its own once owner, and the RC service is closed first through a backend wrapper installed right after a successful session-server init. Startup failures close exactly what was built, the RunVersityGW lifecycle consumes the same guards instead of closing again, and the RC sessions drain before the backend chain shuts down. |
||
|
Jihyeon Gim
|
50fc9fe941 |
rdma: authenticate RC routes through the terminal error path
The RC auth adapter returned signature-verification errors to Fiber, so the production S3 error handler collapsed them into a generic 500 response; authentication failures lost their real status and code the same way route failures did before the terminal serializer. The adapter now sends verification errors through the shared serializer as well. |
||
 niksis02
|
c84c5f645a |
feat: accept principal ARNs in bucket policies under standalone IAM
Bucket policy `Principal` named callers by access key id. Under the standalone IAM service it now names them by AWS-style ARN, as real S3 does: a user ARN, a role ARN (covering every session of that role), an assumed-role ARN (covering one session), the account root ARN or bare account id, or `*`. Every other IAM backend has no ARNs to name anything by and keeps access-key principals unchanged, gated on a new `auth.PrincipalResolver` capability interface that only the standalone client implements. `auth.Account` carries `Arn` and `RoleArn`, filled at authentication time, so a session can be matched against both its own ARN and its role's. Principals are validated at PutBucketPolicy time through a new `/private/resolve-principals` endpoint, which rejects anything that does not name a live identity with `MalformedPolicy: Invalid principal in policy`. An `Allow` naming the account root ARN or bare account id delegates to the account's own IAM rather than granting on its own, while a `Deny` naming it denies every principal in the account outright. Denial messages now name the caller by ARN wherever one exists. Also fixes `aws:PrincipalArn` for assumed-role sessions, which reported the session ARN where AWS reports the role's, and stops an unreachable IAM service being reported as a malformed policy. |
||
 Ben McClelland
|
5f9041ff5f |
feat: support admin/access logs on standard streams
The access-log and admin-access-log options now accept stdout, stderr, or - for stdout in addition to file paths. Standard stream destinations are kept open during shutdown and SIGHUP handling, while file destinations continue to support normal reopen behavior for log rotation. CLI help, embedded config comments, and the example configuration describe the new destination values. Fixes #2245 |
||
|
535cc9d521 |
feat: add the hipobj-rc-v2 control routes to the vgwrdma gateway
* rdma: add the hipobj-rc-v2 control routes to the vgwrdma gateway Mount the three control routes (prepare, ready, cancel) on the S3 port behind the standard SigV4 middleware. The routes own authentication-adjacent policy the C server cannot see: the middleware wrapper yields to the handler on success, READY and CANCEL re-read the account through the IAM cache bypass so mid-flow deletions and credential rotations take effect immediately, and every object access re-authorizes against the decoded bucket and key. The READY handler implements the session ownership contract: the completion-reference finalizer installs only after the transfer claim succeeds, the PUT path hands the reference to the put view exactly at the borrow point, and the FINAL reply carries the stored object's metadata. Backend I/O runs under a context merged with the RC service context so shutdown unblocks in-flight handlers, with a bounded pool for the fresh IAM lookups. vgwrdma starts the session server alongside the gateway when an RDMA interface is configured, tears it down on exit, and shuts the IAM service down on any startup failure. embedgw learns the readonly flag for the object access checks the routes share. Signed-off-by: Jihyeon Gim <potatogim@potatogim.net> * rdma: add the missing stub handlers for non-Linux builds The non-Linux rcroutes stub exposed only Register while the vgwrdma gateway registers the prepare/ready/cancel handlers directly, so cross-compiling cmd/vgwrdma failed with undefined methods. Add the three stub handlers answering 501 Not Implemented and let Register reuse them, matching the Linux Handler API surface. * auth: drop the duplicated GetUserAccountFresh definition The rebase onto main (which already carries GetUserAccountFresh from the iam-cache-fresh change) kept both copies of the method, breaking the build with a redeclaration error. Remove the second copy so the method is defined once. * rdma: address the review findings on the control route wiring Drop the unused Handler.Register from both build variants: the gateway mounts the three control routes through s3api.WithRoute so the SigV4 verifier wrapper (rcAuth) runs in front of each handler, and nothing else calls Register. Clear iamOwned only when RunVersityGW returns nil. It shuts the IAM service down itself at the end of its shutdown sequence, but its early failure paths return before reaching that point, so the deferred shutdown must keep covering those errors. Remove the unused rcserver.SessionInfo parameter from sizeOf; the transferred byte count comes from the READY response alone. * rdma: keep transient IAM failures retryable in the fresh revalidation The fresh account revalidation turned every GetUserAccountFresh error into 403, which reports transient backend failures (LDAP timeouts, network errors) as a revoked account and leaves the client no room to retry. Only a confirmed missing account (auth.ErrNoSuchUser) means that; answer anything else with 503 so clients can retry the request. * rdma: make the IAM shutdown exactly-once and keep gateway errors visible The gateway and RunVersityGW share the IAM service, and which side shut it down could not be told from the return value: runtime failures return after RunVersityGW already shut the service down, while early setup failures return before any shutdown happens. The iamOwned flag therefore either shut the service down twice or leaked it depending on the error, and the error itself was dropped. Wrap the service so Shutdown runs exactly once no matter which side calls it, keep the deferred shutdown for every early failure path, and return the gateway error again. The wrapper re-exposes the optional interfaces (fresh account reads, signing keys, policy evaluation, fixed bucket ownership) so feature detection through the IAM service keeps working. * rdma: reuse the SigV4 account for RC control requests READY and CANCEL are independently authenticated SigV4 requests. Use the account resolved by the normal SigV4 path instead of bypassing the IAM cache a second time. This aligns RC revocation latency with other signed S3 requests and removes the extra backend IAM lookup, its concurrency cap, and the RC-specific IAM error mapping. The session owner check and the READY target and operation authorization are unchanged. * rdma: reword the READY reauthorization comment The comment implied a revocation inside the session window always takes effect at READY, but the account used here is the one SigV4 resolved, which may be a cached entry. State what the check does without claiming account-cache freshness. * rdma: preserve IAM cache behavior and standalone region --------- Signed-off-by: Jihyeon Gim <potatogim@potatogim.net> Co-authored-by: Ben McClelland <ben.mcclelland@versity.com> |
||
|
niksis02
|
2147a0c304 |
feat: integrate standalone IAM service with S3 gateway for identity-based policy enforcement
Fixes #1327 Fixes #1567 Closes #2264 Wires the S3 gateway up to the standalone IAM service so identity policies, not just bucket policies and ACLs, are enforced on the S3 data plane. The gateway authenticates SigV4 requests by calling new private derive-signing-key and resolve-identity endpoints on the IAM service instead of holding secrets itself, and evaluates identity policy through the same PolicyEvaluator path added to auth.VerifyAccess, combined with the bucket policy using explicit-deny-wins precedence. The private endpoints are served over their own mTLS listener (new iamapi/private package, genmtlscerts.sh to generate test material, and client-cert support in internal/netutil), separate from the public IAM API. As part of this the vendored aws/signer/v4 package is deleted and replaced by a pure-Go SigV4 implementation in internal/sigv4auth, which now reads canonical request data directly off the fiber.Ctx instead of reconstructing an http.Request, and is shared by both the S3 request-signing verification and the new private-endpoint signing. DeleteObjects moves from an all-or-nothing authorization check to true partial success: VerifyObjectsAccess evaluates every object in a batch independently against both the identity policy and any object lock, so a denial or a locked object only removes that key from the batch instead of failing the whole request. It also batches the identity-policy round trip and the bucket-policy fetch once per request rather than once per object, and separates plain deletes from versioned ones since a versioned delete needs s3:DeleteObjectVersion rather than s3:DeleteObject. Object lock handling got a few correctness fixes alongside this: a bypass is now modeled as BypassNone/BypassRequested/BypassOverwrite rather than a single bool, because root's blanket ability to override a GOVERNANCE retention should only apply when the client actually asked to bypass it (DeleteObject/DeleteObjects/PutObjectRetention), not when the gateway is silently replacing a locked object via an overwrite, which needs the permission from everyone including root. Retention changes are now correctly classified as an extension (allowed under plain s3:PutObjectRetention) versus a weakening (date or mode change, which needs the bypass permission), and a COMPLIANCE lock can never be weakened by anyone regardless of permissions, matching AWS. Separately, VerifyObjectCopyAccess had a readonly-mode gap: it returned early for root/admin before ever calling VerifyAccess, so the readonly check inside VerifyAccess never ran for them on CopyObject; access checks are now ordered so the readonly gate always applies before any root/admin bypass, for copy as well as every other write path. Bucket policies also gained Condition block support, via a new shared internal/condition package moved out of the IAM policy package since both bucket and identity policies share the same evaluation semantics. It implements the full AWS operator set — String{Equals,NotEquals,EqualsIgnoreCase,NotEqualsIgnoreCase,Like,NotLike}, Numeric{Equals,NotEquals,LessThan,LessThanEquals,GreaterThan,GreaterThanEquals}, Date{Equals,NotEquals,LessThan,LessThanEquals,GreaterThan,GreaterThanEquals}, Bool, BinaryEquals, Arn{Equals,Like,NotEquals,NotLike}, IpAddress/NotIpAddress, and Null — along with the ForAllValues/ForAnyValue set qualifiers and the IfExists modifier. A new requestConditionContext builds the per-request keys a bucket policy's Condition block can reference — aws:SourceIp, aws:SecureTransport, aws:CurrentTime, aws:EpochTime, aws:UserAgent, aws:Referer, s3:prefix, s3:delimiter, s3:max-keys, s3:x-amz-acl, s3:VersionId — following AWS's own per-action rules for which keys a given S3 operation actually populates. Identity-derived keys such as aws:PrincipalArn and aws:username are deliberately left unwired here, since the gateway has no way to know them; the standalone IAM service fills those in itself when it evaluates an identity policy. Also added new integration test suites for S3-side IAM: s3_iam_access_control.go and s3_iam_session_access_control.go cover identity-policy enforcement and session-credential requests against real S3 operations, alongside expanded OIDC/web-identity coverage and a new runoidctests.sh runner wired into the OIDC GitHub Actions workflow. |
||
|
niksis02
|
ab2b816633 |
feat: add STS web identity federation, IAM policy Condition support, and access control enforcement
Implements the `AssumeRoleWithWebIdentity` and `GetCallerIdentity` STS actions, letting callers exchange an external OIDC token for temporary credentials scoped to an IAM role. Token handling covers JWT claim parsing, issuer/audience resolution (including `azp` override semantics), JWKS fetching and caching with `singleflight`-deduplicated refresh, and rate-limited forced refresh on unrecognized `kid` values. OIDC provider thumbprint fetching now performs a real TLS handshake verified against the system trust store and the provider hostname (previously `InsecureSkipVerify`), since the observed certificate is persisted as a long-lived trust anchor rather than used once and discarded; all discovery-document and JWKS fetches go through an SSRF-safe HTTP client with bounded redirects and response size.
Adds policy `Condition` block evaluation, supporting `String`, `Numeric`, `Date`, `Bool`, `BinaryEquals`, and `IpAddress` operators along with their `IfExists`/`Not` variants and `ForAllValues`/`ForAnyValues` set qualifiers, plus policy variable substitution (e.g. `${aws:username}`) in supported operators. Adds identity-based inline policy evaluation and a new IAM authorization middleware that authorizes each request against action, resource, and condition context together, applying the session-policy-intersects-role-policy semantics for assumed-role sessions.
Adds a new debug logger `--log-level` flag (`silent`/`debug`/`unsafe`), along with a tree-based XML masker that redacts secrets and tokens at the property level in logged request/response bodies instead of skipping the whole body. The old `--debug/VGW_DEBUG` flag is kept as a deprecated alias for `--log-level=debug`, printing a console warning that points users at `--log-level` for finer-grained control.
Fixes a Vault storage bug where CAS (check-and-set) writes always read the current document version as 0 because `kvVersion` asserted metadata as `float64` while the Vault client actually returns `json.Number`, causing every write past the first to be rejected as a concurrent modification. Also adds a constant-time `SecureCompare` for signature/token comparisons in sigv4 auth.
Adds an integration test suite (`iam_access_control.go`) covering IAM access control across user, role, and session identities.
|
||
|
Ben McClelland
|
62c6787a17 | feat: add new vgwrdma service for cuObject compatible server |