mirror of
https://github.com/versity/versitygw.git
synced 2026-08-17 20:56:21 +00:00
Closes #1640 Add a standalone AWS IAM Query API implementation for managing IAM users through standard AWS SDKs and the AWS CLI. Server usage Start the IAM server with internal file-backed storage: mkdir -p /tmp/versitygw-iam ./versitygw --port 127.0.0.1:7070 --access user --secret pass iam --dir /tmp/versitygw-iam Start the IAM server with Vault KV v2 storage using AppRole: VGW_IAM_VAULT_ROLE_SECRET=<role-secret> ./versitygw --port 127.0.0.1:7070 --access user --secret pass iam --vault-endpoint-url http://127.0.0.1:8200 --vault-auth-method approle --vault-role-id <role-id> --vault-mount-path kv --vault-secret-storage-path iam Vault authentication also supports root tokens, separate authentication and secret-storage namespaces, custom mount paths, server certificate validation, and mutual TLS client certificates. Configure the AWS CLI credentials used by the IAM server: export AWS_ACCESS_KEY_ID=user export AWS_SECRET_ACCESS_KEY=pass export AWS_DEFAULT_REGION=us-east-1 Implemented IAM actions CreateUser creates an IAM user with an AWS-compatible ARN, generated AIDA user ID, creation timestamp, optional path, and tags. It validates usernames, paths, tag limits, reserved tag prefixes, duplicate tag keys, and existing users. aws --endpoint-url http://127.0.0.1:7070 iam create-user --user-name bob aws --endpoint-url http://127.0.0.1:7070 iam create-user --user-name bob --path /engineering/ --tags Key=team,Value=storage GetUser returns a stored user or the root identity when requested without a username through the IAM Query API. aws --endpoint-url http://127.0.0.1:7070 iam get-user --user-name bob ListUsers returns users in deterministic username order and supports path filtering, marker-based pagination, and MaxItems limits. aws --endpoint-url http://127.0.0.1:7070 iam list-users aws --endpoint-url http://127.0.0.1:7070 iam list-users --path-prefix /engineering/ --max-items 100 UpdateUser updates the username and/or path, recalculates the user ARN, and rejects conflicts with existing users. aws --endpoint-url http://127.0.0.1:7070 iam update-user --user-name bob --new-user-name robert --new-path /platform/ DeleteUser permanently removes an IAM user and returns AWS-compatible errors for missing users. aws --endpoint-url http://127.0.0.1:7070 iam delete-user --user-name robert IAM protocol and authentication - Support the AWS IAM Query protocol version 2010-05-08 over GET and POST form requests. - Return AWS-compatible XML responses, error documents, status codes, request IDs, user metadata, and pagination fields. - Authenticate root credentials with AWS Signature Version 4 for the IAM service in us-east-1. - Support both Authorization-header and query-string SigV4 authentication. - Validate credential scope, signed headers, timestamps, clock skew, content length, signatures, and unsupported signature or session-token modes. - Add IAM-specific validation and error mapping for malformed requests, invalid actions, duplicate entities, missing users, throttling, and internal failures. Storage implementations - Add an internal JSON-backed store using iam.json and iam.json.backup with atomic temporary-file replacement, concurrent access protection, stable ordering, pagination, and persistence across restarts. - Add a Vault KV v2 store with one secret per user, CAS-based duplicate protection, permanent deletion, AppRole reauthentication, namespace support, configurable authentication and KV mounts, root-token authentication, and TLS/mTLS configuration. - Introduce a common Storer interface and require exactly one storage backend to be configured. Server and embedding support - Register the new `versitygw iam` command with environment-variable and CLI configuration for both storage backends. - Add `embedgw.RunIAMAPI` and `IAMConfig` for embedding the IAM service in Go applications. Gateway-level internal packages - Add `internal/iamstore` as a reusable generic file-backed IAM persistence engine and migrate the existing gateway internal IAM service to it. - Add `internal/sigv4auth` for shared SigV4 header and presigned-query parsing, canonical request generation, signature verification, and structured authentication errors. - Refactor the S3 authentication paths to use the shared SigV4 implementation while preserving S3-specific error responses. - Add `internal/httpctx` for shared Fiber context keys and AWS-style request ID handling. - Add `internal/routekit` for shared query, form, and header route matchers. - Add `internal/netutil` for reusable certificate storage, hostname-aware listeners, multi-address serving, TLS listeners, and UNIX socket handling. - Update the custom SigV4 signer to honor an explicitly supplied signed-header list so unrelated headers do not alter IAM signatures. Testing and CI - Add AWS IAM SDK-based integration coverage for all supported user actions, header authentication, query authentication, validation, errors, filtering, and pagination. - Split standalone IAM tests into `versitygw test iam` and retain existing gateway IAM tests under `versitygw test gw-iam`. - Add unit coverage for controllers, authentication, routing, storage, embedding, listeners, request matching, persistence, and signing behavior. - Add `runiamtests.sh` to exercise internal storage over HTTP and HTTPS plus Vault storage through AppRole. - Add a dedicated IAM functional-test workflow with a Vault service and merged runtime coverage reporting. - Include the IAM test runner in shellcheck and add the AWS IAM SDK dependency.
341 lines
11 KiB
Go
341 lines
11 KiB
Go
// Copyright 2026 Versity Software
|
|
// This file is licensed under the Apache License, Version 2.0
|
|
// (the "License"); you may not use this file except in compliance
|
|
// with the License. You may obtain a copy of the License at
|
|
//
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
//
|
|
// Unless required by applicable law or agreed to in writing,
|
|
// software distributed under the License is distributed on an
|
|
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
// KIND, either express or implied. See the License for the
|
|
// specific language governing permissions and limitations
|
|
// under the License.
|
|
|
|
package integration
|
|
|
|
import (
|
|
"bytes"
|
|
"context"
|
|
"crypto/sha256"
|
|
"encoding/hex"
|
|
"fmt"
|
|
"net/http"
|
|
"strings"
|
|
|
|
"github.com/aws/aws-sdk-go-v2/aws"
|
|
vgwv4 "github.com/versity/versitygw/aws/signer/v4"
|
|
"github.com/versity/versitygw/iamapi/iamerr"
|
|
"github.com/versity/versitygw/internal/sigv4auth"
|
|
)
|
|
|
|
func IAMQueryAuth_success(s *S3Conf) error {
|
|
testName := "IAMQueryAuth_success"
|
|
return iamQueryAuthHandler(s, iamAuthConfig(testName), func(req *http.Request) error {
|
|
return checkIAMQueryAuthRequest(s, req, nil)
|
|
})
|
|
}
|
|
|
|
func IAMQueryAuth_security_token_not_supported(s *S3Conf) error {
|
|
testName := "IAMQueryAuth_security_token_not_supported"
|
|
return iamQueryAuthHandler(s, iamAuthConfig(testName), func(req *http.Request) error {
|
|
setIAMQueryParameter(req, sigv4auth.QuerySecurityToken, "my_token")
|
|
|
|
return checkIAMQueryAuthRequest(s, req, iamerr.GetAPIError(iamerr.ErrInvalidClientTokenID))
|
|
})
|
|
}
|
|
|
|
func IAMQueryAuth_unsupported_algorithm(s *S3Conf) error {
|
|
testName := "IAMQueryAuth_unsupported_algorithm"
|
|
return iamQueryAuthHandler(s, iamAuthConfig(testName), func(req *http.Request) error {
|
|
const algorithm = "AWS4-SHA256"
|
|
setIAMQueryParameter(req, sigv4auth.QueryAlgorithm, algorithm)
|
|
|
|
return checkIAMQueryAuthRequest(s, req, iamerr.GetAPIError(iamerr.ErrUnsupportedQueryAlgorithm))
|
|
})
|
|
}
|
|
|
|
func IAMQueryAuth_ECDSA_not_supported(s *S3Conf) error {
|
|
testName := "IAMQueryAuth_ECDSA_not_supported"
|
|
return iamQueryAuthHandler(s, iamAuthConfig(testName), func(req *http.Request) error {
|
|
setIAMQueryParameter(req, sigv4auth.QueryAlgorithm, sigv4auth.AlgorithmECDSAP256SHA256)
|
|
|
|
return checkIAMQueryAuthRequest(s, req, iamerr.GetAPIError(iamerr.ErrUnsupportedQueryAlgorithm))
|
|
})
|
|
}
|
|
|
|
func IAMQueryAuth_missing_query_parameters(s *S3Conf) error {
|
|
testName := "IAMQueryAuth_missing_query_parameters"
|
|
return iamQueryAuthHandler(s, iamAuthConfig(testName), func(req *http.Request) error {
|
|
testCases := []struct {
|
|
name string
|
|
parameter string
|
|
expected iamerr.APIError
|
|
}{
|
|
{
|
|
name: "missing_algorithm",
|
|
parameter: sigv4auth.QueryAlgorithm,
|
|
expected: iamerr.GetAPIError(iamerr.ErrMissingAuthenticationToken),
|
|
},
|
|
{
|
|
name: "missing_credential",
|
|
parameter: sigv4auth.QueryCredential,
|
|
expected: iamerr.IncompleteSignatureMissingQueryParameter(sigv4auth.QueryCredential),
|
|
},
|
|
{
|
|
name: "missing_date",
|
|
parameter: sigv4auth.QueryDate,
|
|
expected: iamerr.IncompleteSignatureMissingQueryParameter(sigv4auth.QueryDate),
|
|
},
|
|
{
|
|
name: "missing_signed_headers",
|
|
parameter: sigv4auth.QuerySignedHeaders,
|
|
expected: iamerr.IncompleteSignatureMissingQueryParameter(sigv4auth.QuerySignedHeaders),
|
|
},
|
|
{
|
|
name: "missing_signature",
|
|
parameter: sigv4auth.QuerySignature,
|
|
expected: iamerr.IncompleteSignatureMissingQueryParameter(sigv4auth.QuerySignature),
|
|
},
|
|
}
|
|
|
|
for _, testCase := range testCases {
|
|
testReq := req.Clone(req.Context())
|
|
deleteIAMQueryParameter(testReq, testCase.parameter)
|
|
if err := checkIAMQueryAuthRequest(s, testReq, testCase.expected); err != nil {
|
|
return fmt.Errorf("%s: %w", testCase.name, err)
|
|
}
|
|
}
|
|
|
|
return nil
|
|
})
|
|
}
|
|
|
|
func IAMQueryAuth_malformed_credential(s *S3Conf) error {
|
|
testName := "IAMQueryAuth_malformed_credential"
|
|
return iamQueryAuthHandler(s, iamAuthConfig(testName), func(req *http.Request) error {
|
|
const credential = "access/hello/world"
|
|
setIAMQueryParameter(req, sigv4auth.QueryCredential, credential)
|
|
|
|
return checkIAMQueryAuthRequest(s, req, iamerr.IncompleteSignatureMalformedCredential(credential))
|
|
})
|
|
}
|
|
|
|
func IAMQueryAuth_credentials_invalid_terminal(s *S3Conf) error {
|
|
testName := "IAMQueryAuth_credentials_invalid_terminal"
|
|
return iamQueryAuthHandler(s, iamAuthConfig(testName), func(req *http.Request) error {
|
|
if err := changeIAMQueryCredential(req, "aws_request", credTerminator); err != nil {
|
|
return err
|
|
}
|
|
|
|
return checkIAMQueryAuthRequest(s, req, iamerr.GetAPIError(iamerr.ErrInvalidTerminal))
|
|
})
|
|
}
|
|
|
|
func IAMQueryAuth_credentials_incorrect_service(s *S3Conf) error {
|
|
testName := "IAMQueryAuth_credentials_incorrect_service"
|
|
return iamQueryAuthHandler(s, iamAuthConfig(testName), func(req *http.Request) error {
|
|
if err := changeIAMQueryCredential(req, "ec2", credService); err != nil {
|
|
return err
|
|
}
|
|
|
|
return checkIAMQueryAuthRequest(s, req, iamerr.GetAPIError(iamerr.ErrIncorrectService))
|
|
})
|
|
}
|
|
|
|
func IAMQueryAuth_credentials_incorrect_region(s *S3Conf) error {
|
|
testName := "IAMQueryAuth_credentials_incorrect_region"
|
|
cfg := iamAuthConfig(testName)
|
|
cfg.region = "us-west-1"
|
|
return iamQueryAuthHandler(s, cfg, func(req *http.Request) error {
|
|
return checkIAMQueryAuthRequest(s, req, iamerr.GetAPIError(iamerr.ErrInvalidRegion))
|
|
})
|
|
}
|
|
|
|
func IAMQueryAuth_credentials_invalid_date(s *S3Conf) error {
|
|
testName := "IAMQueryAuth_credentials_invalid_date"
|
|
return iamQueryAuthHandler(s, iamAuthConfig(testName), func(req *http.Request) error {
|
|
if err := changeIAMQueryCredential(req, "3223423234", credDate); err != nil {
|
|
return err
|
|
}
|
|
|
|
return checkIAMQueryAuthRequest(s, req, iamerr.GetAPIError(iamerr.ErrInvalidCredentialDate))
|
|
})
|
|
}
|
|
|
|
func IAMQueryAuth_non_existing_access_key(s *S3Conf) error {
|
|
testName := "IAMQueryAuth_non_existing_access_key"
|
|
cfg := iamAuthConfig(testName)
|
|
cfg.access = "a_rarely_existing_access_key_id_a7s86df78as6df89790a8sd7f"
|
|
return iamQueryAuthHandler(s, cfg, func(req *http.Request) error {
|
|
return checkIAMQueryAuthRequest(s, req, iamerr.GetAPIError(iamerr.ErrInvalidClientTokenID))
|
|
})
|
|
}
|
|
|
|
func IAMQueryAuth_invalid_date(s *S3Conf) error {
|
|
testName := "IAMQueryAuth_invalid_date"
|
|
return iamQueryAuthHandler(s, iamAuthConfig(testName), func(req *http.Request) error {
|
|
const invalidDate = "03032006"
|
|
setIAMQueryParameter(req, sigv4auth.QueryDate, invalidDate)
|
|
|
|
return checkIAMQueryAuthRequest(s, req, iamerr.IncompleteSignatureInvalidXAmzDate(invalidDate))
|
|
})
|
|
}
|
|
|
|
func IAMQueryAuth_date_mismatch(s *S3Conf) error {
|
|
testName := "IAMQueryAuth_date_mismatch"
|
|
return iamQueryAuthHandler(s, iamAuthConfig(testName), func(req *http.Request) error {
|
|
if err := changeIAMQueryCredential(req, "20000101", credDate); err != nil {
|
|
return err
|
|
}
|
|
|
|
return checkIAMQueryAuthRequest(s, req, iamerr.GetAPIError(iamerr.ErrInvalidCredentialDate))
|
|
})
|
|
}
|
|
|
|
func IAMQueryAuth_unsigned_query_parameter(s *S3Conf) error {
|
|
testName := "IAMQueryAuth_unsigned_query_parameter"
|
|
return iamQueryAuthHandler(s, iamAuthConfig(testName), func(req *http.Request) error {
|
|
setIAMQueryParameter(req, "ExtraParam", "value")
|
|
|
|
return checkIAMQueryAuthRequest(s, req, iamerr.GetAPIError(iamerr.ErrSignatureDoesNotMatch))
|
|
})
|
|
}
|
|
|
|
func IAMQueryAuth_incorrect_secret_key(s *S3Conf) error {
|
|
testName := "IAMQueryAuth_incorrect_secret_key"
|
|
cfg := iamAuthConfig(testName)
|
|
cfg.secret = s.awsSecret + "a"
|
|
return iamQueryAuthHandler(s, cfg, func(req *http.Request) error {
|
|
return checkIAMQueryAuthRequest(s, req, iamerr.GetAPIError(iamerr.ErrSignatureDoesNotMatch))
|
|
})
|
|
}
|
|
|
|
func IAMQueryAuth_invalid_sha256_payload_hash_ignored(s *S3Conf) error {
|
|
testName := "IAMQueryAuth_invalid_sha256_payload_hash_ignored"
|
|
return iamQueryAuthHandler(s, iamAuthConfig(testName), func(req *http.Request) error {
|
|
req.Header.Set("X-Amz-Content-Sha256", "invalid_sha256")
|
|
|
|
return checkIAMQueryAuthRequest(s, req, nil)
|
|
})
|
|
}
|
|
|
|
func IAMQueryAuth_with_expect_header(s *S3Conf) error {
|
|
testName := "IAMQueryAuth_with_expect_header"
|
|
return iamQueryAuthHandler(s, iamAuthConfig(testName), func(req *http.Request) error {
|
|
req.Header.Set("Expect", "100-continue")
|
|
|
|
return checkIAMQueryAuthRequest(s, req, nil)
|
|
})
|
|
}
|
|
|
|
func iamQueryAuthHandler(s *S3Conf, cfg *authConfig, handler func(req *http.Request) error) error {
|
|
runF(cfg.testName)
|
|
|
|
access, secret, region := s.awsID, s.awsSecret, s.awsRegion
|
|
if cfg.access != "" {
|
|
access = cfg.access
|
|
}
|
|
if cfg.secret != "" {
|
|
secret = cfg.secret
|
|
}
|
|
if cfg.region != "" {
|
|
region = cfg.region
|
|
}
|
|
|
|
req, err := createIAMQuerySignedRequest(s.endpoint, cfg, access, secret, region)
|
|
if err == nil {
|
|
err = handler(req)
|
|
}
|
|
if err != nil {
|
|
failF("%v: %v", cfg.testName, err)
|
|
return fmt.Errorf("%v: %w", cfg.testName, err)
|
|
}
|
|
|
|
passF(cfg.testName)
|
|
return nil
|
|
}
|
|
|
|
func createIAMQuerySignedRequest(endpoint string, cfg *authConfig, access, secret, region string) (*http.Request, error) {
|
|
target := strings.TrimRight(endpoint, "/") + "/" + strings.TrimLeft(cfg.path, "/")
|
|
req, err := http.NewRequest(cfg.method, target, bytes.NewReader(cfg.body))
|
|
if err != nil {
|
|
return nil, fmt.Errorf("create IAM query auth request: %w", err)
|
|
}
|
|
|
|
for key, value := range cfg.headers {
|
|
req.Header.Set(key, value)
|
|
}
|
|
|
|
payloadHash := cfg.overrideSha256
|
|
if payloadHash == "" {
|
|
hash := sha256.Sum256(cfg.body)
|
|
payloadHash = hex.EncodeToString(hash[:])
|
|
}
|
|
|
|
signer := vgwv4.NewSigner()
|
|
signedURL, signedHeaders, _, err := signer.PresignHTTP(
|
|
context.Background(),
|
|
aws.Credentials{AccessKeyID: access, SecretAccessKey: secret},
|
|
req,
|
|
payloadHash,
|
|
cfg.service,
|
|
region,
|
|
cfg.date,
|
|
nil,
|
|
)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("sign IAM query auth request: %w", err)
|
|
}
|
|
|
|
signedReq, err := http.NewRequest(cfg.method, signedURL, bytes.NewReader(cfg.body))
|
|
if err != nil {
|
|
return nil, fmt.Errorf("create signed IAM query auth request: %w", err)
|
|
}
|
|
for key, value := range cfg.headers {
|
|
signedReq.Header.Set(key, value)
|
|
}
|
|
for key, values := range signedHeaders {
|
|
signedReq.Header[key] = append([]string(nil), values...)
|
|
}
|
|
|
|
return signedReq, nil
|
|
}
|
|
|
|
func checkIAMQueryAuthRequest(s *S3Conf, req *http.Request, expected iamerr.APIError) error {
|
|
if expected != nil {
|
|
return checkIAMAuthRequest(s, req, expected)
|
|
}
|
|
|
|
resp, err := s.httpClient.Do(req)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
return checkIAMSuccess(resp)
|
|
}
|
|
|
|
func setIAMQueryParameter(req *http.Request, parameter, value string) {
|
|
query := req.URL.Query()
|
|
query.Set(parameter, value)
|
|
req.URL.RawQuery = query.Encode()
|
|
}
|
|
|
|
func deleteIAMQueryParameter(req *http.Request, parameter string) {
|
|
query := req.URL.Query()
|
|
query.Del(parameter)
|
|
req.URL.RawQuery = query.Encode()
|
|
}
|
|
|
|
func changeIAMQueryCredential(req *http.Request, value string, index int) error {
|
|
query := req.URL.Query()
|
|
credential := query.Get(sigv4auth.QueryCredential)
|
|
parts := strings.Split(credential, "/")
|
|
if len(parts) != 5 {
|
|
return fmt.Errorf("unexpected generated IAM query credential %q", credential)
|
|
}
|
|
parts[index] = value
|
|
query.Set(sigv4auth.QueryCredential, strings.Join(parts, "/"))
|
|
req.URL.RawQuery = query.Encode()
|
|
return nil
|
|
}
|