mirror of
https://github.com/versity/versitygw.git
synced 2026-07-08 09:06:59 +00:00
b590ac9eca
Closes #1640 Add a standalone AWS IAM Query API implementation for managing IAM users through standard AWS SDKs and the AWS CLI. Server usage Start the IAM server with internal file-backed storage: mkdir -p /tmp/versitygw-iam ./versitygw --port 127.0.0.1:7070 --access user --secret pass iam --dir /tmp/versitygw-iam Start the IAM server with Vault KV v2 storage using AppRole: VGW_IAM_VAULT_ROLE_SECRET=<role-secret> ./versitygw --port 127.0.0.1:7070 --access user --secret pass iam --vault-endpoint-url http://127.0.0.1:8200 --vault-auth-method approle --vault-role-id <role-id> --vault-mount-path kv --vault-secret-storage-path iam Vault authentication also supports root tokens, separate authentication and secret-storage namespaces, custom mount paths, server certificate validation, and mutual TLS client certificates. Configure the AWS CLI credentials used by the IAM server: export AWS_ACCESS_KEY_ID=user export AWS_SECRET_ACCESS_KEY=pass export AWS_DEFAULT_REGION=us-east-1 Implemented IAM actions CreateUser creates an IAM user with an AWS-compatible ARN, generated AIDA user ID, creation timestamp, optional path, and tags. It validates usernames, paths, tag limits, reserved tag prefixes, duplicate tag keys, and existing users. aws --endpoint-url http://127.0.0.1:7070 iam create-user --user-name bob aws --endpoint-url http://127.0.0.1:7070 iam create-user --user-name bob --path /engineering/ --tags Key=team,Value=storage GetUser returns a stored user or the root identity when requested without a username through the IAM Query API. aws --endpoint-url http://127.0.0.1:7070 iam get-user --user-name bob ListUsers returns users in deterministic username order and supports path filtering, marker-based pagination, and MaxItems limits. aws --endpoint-url http://127.0.0.1:7070 iam list-users aws --endpoint-url http://127.0.0.1:7070 iam list-users --path-prefix /engineering/ --max-items 100 UpdateUser updates the username and/or path, recalculates the user ARN, and rejects conflicts with existing users. aws --endpoint-url http://127.0.0.1:7070 iam update-user --user-name bob --new-user-name robert --new-path /platform/ DeleteUser permanently removes an IAM user and returns AWS-compatible errors for missing users. aws --endpoint-url http://127.0.0.1:7070 iam delete-user --user-name robert IAM protocol and authentication - Support the AWS IAM Query protocol version 2010-05-08 over GET and POST form requests. - Return AWS-compatible XML responses, error documents, status codes, request IDs, user metadata, and pagination fields. - Authenticate root credentials with AWS Signature Version 4 for the IAM service in us-east-1. - Support both Authorization-header and query-string SigV4 authentication. - Validate credential scope, signed headers, timestamps, clock skew, content length, signatures, and unsupported signature or session-token modes. - Add IAM-specific validation and error mapping for malformed requests, invalid actions, duplicate entities, missing users, throttling, and internal failures. Storage implementations - Add an internal JSON-backed store using iam.json and iam.json.backup with atomic temporary-file replacement, concurrent access protection, stable ordering, pagination, and persistence across restarts. - Add a Vault KV v2 store with one secret per user, CAS-based duplicate protection, permanent deletion, AppRole reauthentication, namespace support, configurable authentication and KV mounts, root-token authentication, and TLS/mTLS configuration. - Introduce a common Storer interface and require exactly one storage backend to be configured. Server and embedding support - Register the new `versitygw iam` command with environment-variable and CLI configuration for both storage backends. - Add `embedgw.RunIAMAPI` and `IAMConfig` for embedding the IAM service in Go applications. Gateway-level internal packages - Add `internal/iamstore` as a reusable generic file-backed IAM persistence engine and migrate the existing gateway internal IAM service to it. - Add `internal/sigv4auth` for shared SigV4 header and presigned-query parsing, canonical request generation, signature verification, and structured authentication errors. - Refactor the S3 authentication paths to use the shared SigV4 implementation while preserving S3-specific error responses. - Add `internal/httpctx` for shared Fiber context keys and AWS-style request ID handling. - Add `internal/routekit` for shared query, form, and header route matchers. - Add `internal/netutil` for reusable certificate storage, hostname-aware listeners, multi-address serving, TLS listeners, and UNIX socket handling. - Update the custom SigV4 signer to honor an explicitly supplied signed-header list so unrelated headers do not alter IAM signatures. Testing and CI - Add AWS IAM SDK-based integration coverage for all supported user actions, header authentication, query authentication, validation, errors, filtering, and pagination. - Split standalone IAM tests into `versitygw test iam` and retain existing gateway IAM tests under `versitygw test gw-iam`. - Add unit coverage for controllers, authentication, routing, storage, embedding, listeners, request matching, persistence, and signing behavior. - Add `runiamtests.sh` to exercise internal storage over HTTP and HTTPS plus Vault storage through AppRole. - Add a dedicated IAM functional-test workflow with a Vault service and merged runtime coverage reporting. - Include the IAM test runner in shellcheck and add the AWS IAM SDK dependency.
183 lines
5.6 KiB
Go
183 lines
5.6 KiB
Go
// Copyright 2023 Versity Software
|
|
// This file is licensed under the Apache License, Version 2.0
|
|
// (the "License"); you may not use this file except in compliance
|
|
// with the License. You may obtain a copy of the License at
|
|
//
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
//
|
|
// Unless required by applicable law or agreed to in writing,
|
|
// software distributed under the License is distributed on an
|
|
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
// KIND, either express or implied. See the License for the
|
|
// specific language governing permissions and limitations
|
|
// under the License.
|
|
|
|
package utils
|
|
|
|
import (
|
|
"crypto/hmac"
|
|
"crypto/sha256"
|
|
"encoding/hex"
|
|
"errors"
|
|
"time"
|
|
|
|
"github.com/gofiber/fiber/v3"
|
|
"github.com/versity/versitygw/internal/sigv4auth"
|
|
"github.com/versity/versitygw/s3err"
|
|
)
|
|
|
|
const (
|
|
iso8601Format = sigv4auth.ISO8601Format
|
|
yyyymmdd = sigv4auth.YYYYMMDD
|
|
)
|
|
|
|
func HexBytes(s string) string {
|
|
return sigv4auth.HexBytes(s)
|
|
}
|
|
|
|
const (
|
|
service = sigv4auth.ServiceS3
|
|
)
|
|
|
|
// CheckValidSignature validates the ctx v4 auth signature
|
|
func CheckValidSignature(ctx fiber.Ctx, auth AuthData, secret, checksum string, tdate time.Time, contentLen int64) (string, error) {
|
|
result, err := sigv4auth.CheckSignature(ctx, auth, secret, checksum, tdate, contentLen, sigv4auth.CheckOptions{
|
|
Service: service,
|
|
DisableURIPathEscaping: true,
|
|
})
|
|
if err != nil {
|
|
return "", mapSigV4Error(err)
|
|
}
|
|
|
|
return result.CanonicalString, nil
|
|
}
|
|
|
|
// AuthData is the parsed authorization data from the header.
|
|
type AuthData = sigv4auth.AuthData
|
|
|
|
// ParseAuthorization returns the parsed fields for the aws v4 auth header
|
|
// example authorization string from aws docs:
|
|
// Authorization: AWS4-HMAC-SHA256
|
|
// Credential=AKIAIOSFODNN7EXAMPLE/20130524/us-east-1/s3/aws4_request,
|
|
// SignedHeaders=host;range;x-amz-date,
|
|
// Signature=fe5f80f77d5fa3beca038a248ff027d0445342fe2855ddc963176630326f1024
|
|
func ParseAuthorization(authorization string) (AuthData, error) {
|
|
authData, err := sigv4auth.ParseAuthorization(authorization, service)
|
|
if err != nil {
|
|
return AuthData{}, mapSigV4Error(err)
|
|
}
|
|
return authData, nil
|
|
}
|
|
|
|
type CredentialsScope = sigv4auth.CredentialsScope
|
|
|
|
type CredsError interface {
|
|
MalformedCredential(string) s3err.S3Error
|
|
IncorrectService(string, string) s3err.S3Error
|
|
IncorrectTerminal(string, string) s3err.S3Error
|
|
InvalidDateFormat(string, string) s3err.S3Error
|
|
}
|
|
|
|
func ParseCredentials(input string, errHandler CredsError) (*CredentialsScope, error) {
|
|
creds, err := sigv4auth.ParseCredentials(input, service)
|
|
if err != nil {
|
|
return nil, mapCredentialsError(input, err, errHandler)
|
|
}
|
|
return creds, nil
|
|
}
|
|
|
|
func SignPostPolicy(base64Policy, yyyymmdd, region, secretKey string) (string, error) {
|
|
signingKey := deriveSigningKey(secretKey, yyyymmdd, region)
|
|
sig := hmacSHA256(signingKey, []byte(base64Policy))
|
|
return hex.EncodeToString(sig), nil
|
|
}
|
|
|
|
func deriveSigningKey(secretKey, yyyymmdd, region string) []byte {
|
|
kDate := hmacSHA256([]byte("AWS4"+secretKey), []byte(yyyymmdd))
|
|
kRegion := hmacSHA256(kDate, []byte(region))
|
|
kService := hmacSHA256(kRegion, []byte(service))
|
|
kSigning := hmacSHA256(kService, []byte("aws4_request"))
|
|
return kSigning
|
|
}
|
|
|
|
func hmacSHA256(key, data []byte) []byte {
|
|
h := hmac.New(sha256.New, key)
|
|
h.Write(data)
|
|
return h.Sum(nil)
|
|
}
|
|
|
|
func mapSigV4Error(err error) error {
|
|
var parseErr *sigv4auth.ParseError
|
|
if errors.As(err, &parseErr) {
|
|
return mapAuthParseError(parseErr)
|
|
}
|
|
|
|
var headersErr *sigv4auth.HeadersNotSignedError
|
|
if errors.As(err, &headersErr) {
|
|
return s3err.GetHeadersNotSignedErr(headersErr.Headers)
|
|
}
|
|
|
|
var sigErr *sigv4auth.SignatureMismatchError
|
|
if errors.As(err, &sigErr) {
|
|
return s3err.GetSignatureDoesNotMatchErr(
|
|
sigErr.AccessKeyID,
|
|
sigErr.StringToSign,
|
|
sigErr.SignatureProvided,
|
|
sigErr.StringToSignBytes,
|
|
sigErr.CanonicalRequest,
|
|
sigErr.CanonicalRequestBytes,
|
|
)
|
|
}
|
|
|
|
return err
|
|
}
|
|
|
|
func mapAuthParseError(err *sigv4auth.ParseError) error {
|
|
switch err.Kind {
|
|
case sigv4auth.ErrInvalidAuthorizationHeader:
|
|
return s3err.GetInvalidArgumentErr(s3err.InvalidArgAuthHeader, err.Input)
|
|
case sigv4auth.ErrUnsupportedAuthorizationVersion:
|
|
return s3err.GetAPIError(s3err.ErrUnsupportedAuthorizationMechanism)
|
|
case sigv4auth.ErrInvalidAuthorizationType:
|
|
return s3err.GetInvalidArgumentErr(s3err.InvalidArgAuthorizationType, err.Value)
|
|
case sigv4auth.ErrMissingComponents:
|
|
return s3err.MalformedAuth.MissingComponents()
|
|
case sigv4auth.ErrMissingCredential:
|
|
return s3err.MalformedAuth.MissingCredential()
|
|
case sigv4auth.ErrMissingSignedHeaders:
|
|
return s3err.MalformedAuth.MissingSignedHeaders()
|
|
case sigv4auth.ErrMissingSignature:
|
|
return s3err.MalformedAuth.MissingSignature()
|
|
case sigv4auth.ErrMalformedComponent:
|
|
return s3err.MalformedAuth.MalformedComponent(err.Value)
|
|
default:
|
|
return mapCredentialsParseError(err, s3err.MalformedAuth)
|
|
}
|
|
}
|
|
|
|
func mapCredentialsError(input string, err error, errHandler CredsError) error {
|
|
var parseErr *sigv4auth.ParseError
|
|
if !errors.As(err, &parseErr) {
|
|
return err
|
|
}
|
|
if parseErr.Input == "" {
|
|
parseErr.Input = input
|
|
}
|
|
return mapCredentialsParseError(parseErr, errHandler)
|
|
}
|
|
|
|
func mapCredentialsParseError(err *sigv4auth.ParseError, errHandler CredsError) error {
|
|
switch err.Kind {
|
|
case sigv4auth.ErrMalformedCredential:
|
|
return errHandler.MalformedCredential(err.Input)
|
|
case sigv4auth.ErrIncorrectService:
|
|
return errHandler.IncorrectService(err.Input, err.Actual)
|
|
case sigv4auth.ErrIncorrectTerminal:
|
|
return errHandler.IncorrectTerminal(err.Input, err.Actual)
|
|
case sigv4auth.ErrInvalidDateFormat:
|
|
return errHandler.InvalidDateFormat(err.Input, err.Value)
|
|
default:
|
|
return err
|
|
}
|
|
}
|