mirror of
https://github.com/versity/versitygw.git
synced 2026-09-28 18:55:54 +00:00
feat: add AWS-compatible standalone IAM service
Closes #1640 Add a standalone AWS IAM Query API implementation for managing IAM users through standard AWS SDKs and the AWS CLI. Server usage Start the IAM server with internal file-backed storage: mkdir -p /tmp/versitygw-iam ./versitygw --port 127.0.0.1:7070 --access user --secret pass iam --dir /tmp/versitygw-iam Start the IAM server with Vault KV v2 storage using AppRole: VGW_IAM_VAULT_ROLE_SECRET=<role-secret> ./versitygw --port 127.0.0.1:7070 --access user --secret pass iam --vault-endpoint-url http://127.0.0.1:8200 --vault-auth-method approle --vault-role-id <role-id> --vault-mount-path kv --vault-secret-storage-path iam Vault authentication also supports root tokens, separate authentication and secret-storage namespaces, custom mount paths, server certificate validation, and mutual TLS client certificates. Configure the AWS CLI credentials used by the IAM server: export AWS_ACCESS_KEY_ID=user export AWS_SECRET_ACCESS_KEY=pass export AWS_DEFAULT_REGION=us-east-1 Implemented IAM actions CreateUser creates an IAM user with an AWS-compatible ARN, generated AIDA user ID, creation timestamp, optional path, and tags. It validates usernames, paths, tag limits, reserved tag prefixes, duplicate tag keys, and existing users. aws --endpoint-url http://127.0.0.1:7070 iam create-user --user-name bob aws --endpoint-url http://127.0.0.1:7070 iam create-user --user-name bob --path /engineering/ --tags Key=team,Value=storage GetUser returns a stored user or the root identity when requested without a username through the IAM Query API. aws --endpoint-url http://127.0.0.1:7070 iam get-user --user-name bob ListUsers returns users in deterministic username order and supports path filtering, marker-based pagination, and MaxItems limits. aws --endpoint-url http://127.0.0.1:7070 iam list-users aws --endpoint-url http://127.0.0.1:7070 iam list-users --path-prefix /engineering/ --max-items 100 UpdateUser updates the username and/or path, recalculates the user ARN, and rejects conflicts with existing users. aws --endpoint-url http://127.0.0.1:7070 iam update-user --user-name bob --new-user-name robert --new-path /platform/ DeleteUser permanently removes an IAM user and returns AWS-compatible errors for missing users. aws --endpoint-url http://127.0.0.1:7070 iam delete-user --user-name robert IAM protocol and authentication - Support the AWS IAM Query protocol version 2010-05-08 over GET and POST form requests. - Return AWS-compatible XML responses, error documents, status codes, request IDs, user metadata, and pagination fields. - Authenticate root credentials with AWS Signature Version 4 for the IAM service in us-east-1. - Support both Authorization-header and query-string SigV4 authentication. - Validate credential scope, signed headers, timestamps, clock skew, content length, signatures, and unsupported signature or session-token modes. - Add IAM-specific validation and error mapping for malformed requests, invalid actions, duplicate entities, missing users, throttling, and internal failures. Storage implementations - Add an internal JSON-backed store using iam.json and iam.json.backup with atomic temporary-file replacement, concurrent access protection, stable ordering, pagination, and persistence across restarts. - Add a Vault KV v2 store with one secret per user, CAS-based duplicate protection, permanent deletion, AppRole reauthentication, namespace support, configurable authentication and KV mounts, root-token authentication, and TLS/mTLS configuration. - Introduce a common Storer interface and require exactly one storage backend to be configured. Server and embedding support - Register the new `versitygw iam` command with environment-variable and CLI configuration for both storage backends. - Add `embedgw.RunIAMAPI` and `IAMConfig` for embedding the IAM service in Go applications. Gateway-level internal packages - Add `internal/iamstore` as a reusable generic file-backed IAM persistence engine and migrate the existing gateway internal IAM service to it. - Add `internal/sigv4auth` for shared SigV4 header and presigned-query parsing, canonical request generation, signature verification, and structured authentication errors. - Refactor the S3 authentication paths to use the shared SigV4 implementation while preserving S3-specific error responses. - Add `internal/httpctx` for shared Fiber context keys and AWS-style request ID handling. - Add `internal/routekit` for shared query, form, and header route matchers. - Add `internal/netutil` for reusable certificate storage, hostname-aware listeners, multi-address serving, TLS listeners, and UNIX socket handling. - Update the custom SigV4 signer to honor an explicitly supplied signed-header list so unrelated headers do not alter IAM signatures. Testing and CI - Add AWS IAM SDK-based integration coverage for all supported user actions, header authentication, query authentication, validation, errors, filtering, and pagination. - Split standalone IAM tests into `versitygw test iam` and retain existing gateway IAM tests under `versitygw test gw-iam`. - Add unit coverage for controllers, authentication, routing, storage, embedding, listeners, request matching, persistence, and signing behavior. - Add `runiamtests.sh` to exercise internal storage over HTTP and HTTPS plus Vault storage through AppRole. - Add a dedicated IAM functional-test workflow with a Vault service and merged runtime coverage reporting. - Include the IAM test runner in shellcheck and add the AWS IAM SDK dependency.
This commit is contained in:
niksis02
@@ -947,7 +947,7 @@ func TestFullFlow(ts *TestState) {
|
||||
if ts.conf.versioningEnabled {
|
||||
TestVersioning(ts)
|
||||
}
|
||||
TestIAM(ts)
|
||||
TestGatewayIAM(ts)
|
||||
TestServer(ts)
|
||||
}
|
||||
|
||||
@@ -1070,7 +1070,7 @@ func TestScoutfs(ts *TestState) {
|
||||
ts.Run(DeleteObject_directory_not_empty)
|
||||
}
|
||||
|
||||
func TestIAM(ts *TestState) {
|
||||
func TestGatewayIAM(ts *TestState) {
|
||||
ts.Run(IAM_user_access_denied)
|
||||
ts.Run(IAM_userplus_access_denied)
|
||||
ts.Run(IAM_userplus_CreateBucket)
|
||||
@@ -1082,6 +1082,115 @@ func TestIAM(ts *TestState) {
|
||||
ts.Run(IAM_CreateBucket_success)
|
||||
}
|
||||
|
||||
func TestIAMAuth(ts *TestState) {
|
||||
ts.Run(IAMAuth_invalid_auth_header)
|
||||
ts.Run(IAMAuth_unsupported_signature_version)
|
||||
ts.Run(IAMAuth_malformed_component)
|
||||
ts.Run(IAMAuth_missing_authorization_component)
|
||||
ts.Run(IAMAuth_malformed_credential)
|
||||
ts.Run(IAMAuth_credentials_invalid_terminal)
|
||||
ts.Run(IAMAuth_credentials_incorrect_service)
|
||||
ts.Run(IAMAuth_credentials_incorrect_region)
|
||||
ts.Run(IAMAuth_credentials_invalid_date)
|
||||
ts.Run(IAMAuth_credentials_future_date)
|
||||
ts.Run(IAMAuth_credentials_past_date)
|
||||
ts.Run(IAMAuth_credentials_non_existing_access_key)
|
||||
ts.Run(IAMAuth_missing_date_header)
|
||||
ts.Run(IAMAuth_invalid_date_header)
|
||||
ts.Run(IAMAuth_date_mismatch)
|
||||
ts.Run(IAMAuth_invalid_sha256_payload_hash_ignored)
|
||||
ts.Run(IAMAuth_unsigned_required_header)
|
||||
ts.Run(IAMAuth_unsigned_non_required_header)
|
||||
ts.Run(IAMAuth_signature_error_incorrect_secret_key)
|
||||
ts.Run(IAMAuth_sigv2_not_supported)
|
||||
ts.Run(IAMAuth_with_expect_header)
|
||||
}
|
||||
|
||||
func TestIAMQueryAuth(ts *TestState) {
|
||||
ts.Run(IAMQueryAuth_success)
|
||||
ts.Run(IAMQueryAuth_security_token_not_supported)
|
||||
ts.Run(IAMQueryAuth_unsupported_algorithm)
|
||||
ts.Run(IAMQueryAuth_ECDSA_not_supported)
|
||||
ts.Run(IAMQueryAuth_missing_query_parameters)
|
||||
ts.Run(IAMQueryAuth_malformed_credential)
|
||||
ts.Run(IAMQueryAuth_credentials_invalid_terminal)
|
||||
ts.Run(IAMQueryAuth_credentials_incorrect_service)
|
||||
ts.Run(IAMQueryAuth_credentials_incorrect_region)
|
||||
ts.Run(IAMQueryAuth_credentials_invalid_date)
|
||||
ts.Run(IAMQueryAuth_non_existing_access_key)
|
||||
ts.Run(IAMQueryAuth_invalid_date)
|
||||
ts.Run(IAMQueryAuth_date_mismatch)
|
||||
ts.Run(IAMQueryAuth_unsigned_query_parameter)
|
||||
ts.Run(IAMQueryAuth_incorrect_secret_key)
|
||||
ts.Run(IAMQueryAuth_invalid_sha256_payload_hash_ignored)
|
||||
ts.Run(IAMQueryAuth_with_expect_header)
|
||||
}
|
||||
|
||||
func TestIAMCreateUser(ts *TestState) {
|
||||
ts.Run(IAMCreateUser_user_already_exists)
|
||||
ts.Run(IAMCreateUser_invalid_user_name)
|
||||
ts.Run(IAMCreateUser_long_user_name)
|
||||
ts.Run(IAMCreateUser_missing_user_name)
|
||||
ts.Run(IAMCreateUser_invalid_tag_key)
|
||||
ts.Run(IAMCreateUser_invalid_tag_value)
|
||||
ts.Run(IAMCreateUser_long_tag_key)
|
||||
ts.Run(IAMCreateUser_long_tag_value)
|
||||
ts.Run(IAMCreateUser_duplicate_tag_keys)
|
||||
ts.Run(IAMCreateUser_success)
|
||||
ts.Run(IAMCreateUser_default_path)
|
||||
ts.Run(IAMCreateUser_invalid_path)
|
||||
ts.Run(IAMCreateUser_long_path)
|
||||
}
|
||||
|
||||
func TestIAMGetUser(ts *TestState) {
|
||||
ts.Run(IAMGetUser_long_user_name)
|
||||
ts.Run(IAMGetUser_invalid_user_name)
|
||||
ts.Run(IAMGetUser_non_existing_user)
|
||||
ts.Run(IAMGetUser_success)
|
||||
ts.Run(IAMGetUser_root_user)
|
||||
}
|
||||
|
||||
func TestIAMListUsers(ts *TestState) {
|
||||
ts.Run(IAMListUsers_invalid_path_prefix)
|
||||
ts.Run(IAMListUsers_long_path_prefix)
|
||||
ts.Run(IAMListUsers_invalid_max_items)
|
||||
ts.Run(IAMListUsers_invalid_max_items_format)
|
||||
ts.Run(IAMListUsers_empty_result)
|
||||
ts.Run(IAMListUsers_success)
|
||||
ts.Run(IAMListUsers_path_prefix)
|
||||
ts.Run(IAMListUsers_pagination)
|
||||
ts.Run(IAMListUsers_path_prefix_pagination)
|
||||
}
|
||||
|
||||
func TestIAMDeleteUser(ts *TestState) {
|
||||
ts.Run(IAMDeleteUser_invalid_user_name)
|
||||
ts.Run(IAMDeleteUser_long_user_name)
|
||||
ts.Run(IAMDeleteUser_non_existing_user)
|
||||
ts.Run(IAMDeleteUser_success)
|
||||
}
|
||||
|
||||
func TestIAMUpdateUser(ts *TestState) {
|
||||
ts.Run(IAMUpdateUser_invalid_user_name)
|
||||
ts.Run(IAMUpdateUser_long_user_name)
|
||||
ts.Run(IAMUpdateUser_invalid_new_user_name)
|
||||
ts.Run(IAMUpdateUser_long_new_user_name)
|
||||
ts.Run(IAMUpdateUser_non_existing_user)
|
||||
ts.Run(IAMUpdateUser_invalid_new_path)
|
||||
ts.Run(IAMUpdateUser_long_new_path)
|
||||
ts.Run(IAMUpdateUser_new_user_name_already_exists)
|
||||
ts.Run(IAMUpdateUser_success)
|
||||
}
|
||||
|
||||
func TestIAM(ts *TestState) {
|
||||
TestIAMAuth(ts)
|
||||
TestIAMQueryAuth(ts)
|
||||
TestIAMCreateUser(ts)
|
||||
TestIAMGetUser(ts)
|
||||
TestIAMListUsers(ts)
|
||||
TestIAMDeleteUser(ts)
|
||||
TestIAMUpdateUser(ts)
|
||||
}
|
||||
|
||||
func TestAccessControl(ts *TestState) {
|
||||
ts.Run(AccessControl_default_ACL_user_access_denied)
|
||||
ts.Run(AccessControl_default_ACL_userplus_access_denied)
|
||||
@@ -1396,6 +1505,84 @@ func GetIntTests() IntTests {
|
||||
"Authentication_signature_error_incorrect_secret_key": Authentication_signature_error_incorrect_secret_key,
|
||||
"Authentication_sigv2_not_supported": Authentication_sigv2_not_supported,
|
||||
"Authentication_with_expect_header": Authentication_with_expect_header,
|
||||
"IAMAuth_invalid_auth_header": IAMAuth_invalid_auth_header,
|
||||
"IAMAuth_unsupported_signature_version": IAMAuth_unsupported_signature_version,
|
||||
"IAMAuth_malformed_component": IAMAuth_malformed_component,
|
||||
"IAMAuth_missing_authorization_component": IAMAuth_missing_authorization_component,
|
||||
"IAMAuth_malformed_credential": IAMAuth_malformed_credential,
|
||||
"IAMAuth_credentials_invalid_terminal": IAMAuth_credentials_invalid_terminal,
|
||||
"IAMAuth_credentials_incorrect_service": IAMAuth_credentials_incorrect_service,
|
||||
"IAMAuth_credentials_incorrect_region": IAMAuth_credentials_incorrect_region,
|
||||
"IAMAuth_credentials_invalid_date": IAMAuth_credentials_invalid_date,
|
||||
"IAMAuth_credentials_future_date": IAMAuth_credentials_future_date,
|
||||
"IAMAuth_credentials_past_date": IAMAuth_credentials_past_date,
|
||||
"IAMAuth_credentials_non_existing_access_key": IAMAuth_credentials_non_existing_access_key,
|
||||
"IAMAuth_missing_date_header": IAMAuth_missing_date_header,
|
||||
"IAMAuth_invalid_date_header": IAMAuth_invalid_date_header,
|
||||
"IAMAuth_date_mismatch": IAMAuth_date_mismatch,
|
||||
"IAMAuth_invalid_sha256_payload_hash_ignored": IAMAuth_invalid_sha256_payload_hash_ignored,
|
||||
"IAMAuth_unsigned_required_header": IAMAuth_unsigned_required_header,
|
||||
"IAMAuth_unsigned_non_required_header": IAMAuth_unsigned_non_required_header,
|
||||
"IAMAuth_signature_error_incorrect_secret_key": IAMAuth_signature_error_incorrect_secret_key,
|
||||
"IAMAuth_sigv2_not_supported": IAMAuth_sigv2_not_supported,
|
||||
"IAMAuth_with_expect_header": IAMAuth_with_expect_header,
|
||||
"IAMQueryAuth_success": IAMQueryAuth_success,
|
||||
"IAMQueryAuth_security_token_not_supported": IAMQueryAuth_security_token_not_supported,
|
||||
"IAMQueryAuth_unsupported_algorithm": IAMQueryAuth_unsupported_algorithm,
|
||||
"IAMQueryAuth_ECDSA_not_supported": IAMQueryAuth_ECDSA_not_supported,
|
||||
"IAMQueryAuth_missing_query_parameters": IAMQueryAuth_missing_query_parameters,
|
||||
"IAMQueryAuth_malformed_credential": IAMQueryAuth_malformed_credential,
|
||||
"IAMQueryAuth_credentials_invalid_terminal": IAMQueryAuth_credentials_invalid_terminal,
|
||||
"IAMQueryAuth_credentials_incorrect_service": IAMQueryAuth_credentials_incorrect_service,
|
||||
"IAMQueryAuth_credentials_incorrect_region": IAMQueryAuth_credentials_incorrect_region,
|
||||
"IAMQueryAuth_credentials_invalid_date": IAMQueryAuth_credentials_invalid_date,
|
||||
"IAMQueryAuth_non_existing_access_key": IAMQueryAuth_non_existing_access_key,
|
||||
"IAMQueryAuth_invalid_date": IAMQueryAuth_invalid_date,
|
||||
"IAMQueryAuth_date_mismatch": IAMQueryAuth_date_mismatch,
|
||||
"IAMQueryAuth_unsigned_query_parameter": IAMQueryAuth_unsigned_query_parameter,
|
||||
"IAMQueryAuth_incorrect_secret_key": IAMQueryAuth_incorrect_secret_key,
|
||||
"IAMQueryAuth_invalid_sha256_payload_hash_ignored": IAMQueryAuth_invalid_sha256_payload_hash_ignored,
|
||||
"IAMQueryAuth_with_expect_header": IAMQueryAuth_with_expect_header,
|
||||
"IAMCreateUser_user_already_exists": IAMCreateUser_user_already_exists,
|
||||
"IAMCreateUser_invalid_user_name": IAMCreateUser_invalid_user_name,
|
||||
"IAMCreateUser_long_user_name": IAMCreateUser_long_user_name,
|
||||
"IAMCreateUser_missing_user_name": IAMCreateUser_missing_user_name,
|
||||
"IAMCreateUser_invalid_tag_key": IAMCreateUser_invalid_tag_key,
|
||||
"IAMCreateUser_invalid_tag_value": IAMCreateUser_invalid_tag_value,
|
||||
"IAMCreateUser_long_tag_key": IAMCreateUser_long_tag_key,
|
||||
"IAMCreateUser_long_tag_value": IAMCreateUser_long_tag_value,
|
||||
"IAMCreateUser_duplicate_tag_keys": IAMCreateUser_duplicate_tag_keys,
|
||||
"IAMCreateUser_success": IAMCreateUser_success,
|
||||
"IAMCreateUser_default_path": IAMCreateUser_default_path,
|
||||
"IAMCreateUser_invalid_path": IAMCreateUser_invalid_path,
|
||||
"IAMCreateUser_long_path": IAMCreateUser_long_path,
|
||||
"IAMGetUser_long_user_name": IAMGetUser_long_user_name,
|
||||
"IAMGetUser_invalid_user_name": IAMGetUser_invalid_user_name,
|
||||
"IAMGetUser_non_existing_user": IAMGetUser_non_existing_user,
|
||||
"IAMGetUser_success": IAMGetUser_success,
|
||||
"IAMGetUser_root_user": IAMGetUser_root_user,
|
||||
"IAMListUsers_invalid_path_prefix": IAMListUsers_invalid_path_prefix,
|
||||
"IAMListUsers_long_path_prefix": IAMListUsers_long_path_prefix,
|
||||
"IAMListUsers_invalid_max_items": IAMListUsers_invalid_max_items,
|
||||
"IAMListUsers_invalid_max_items_format": IAMListUsers_invalid_max_items_format,
|
||||
"IAMListUsers_empty_result": IAMListUsers_empty_result,
|
||||
"IAMListUsers_success": IAMListUsers_success,
|
||||
"IAMListUsers_path_prefix": IAMListUsers_path_prefix,
|
||||
"IAMListUsers_pagination": IAMListUsers_pagination,
|
||||
"IAMListUsers_path_prefix_pagination": IAMListUsers_path_prefix_pagination,
|
||||
"IAMDeleteUser_invalid_user_name": IAMDeleteUser_invalid_user_name,
|
||||
"IAMDeleteUser_long_user_name": IAMDeleteUser_long_user_name,
|
||||
"IAMDeleteUser_non_existing_user": IAMDeleteUser_non_existing_user,
|
||||
"IAMDeleteUser_success": IAMDeleteUser_success,
|
||||
"IAMUpdateUser_invalid_user_name": IAMUpdateUser_invalid_user_name,
|
||||
"IAMUpdateUser_long_user_name": IAMUpdateUser_long_user_name,
|
||||
"IAMUpdateUser_invalid_new_user_name": IAMUpdateUser_invalid_new_user_name,
|
||||
"IAMUpdateUser_long_new_user_name": IAMUpdateUser_long_new_user_name,
|
||||
"IAMUpdateUser_non_existing_user": IAMUpdateUser_non_existing_user,
|
||||
"IAMUpdateUser_invalid_new_path": IAMUpdateUser_invalid_new_path,
|
||||
"IAMUpdateUser_long_new_path": IAMUpdateUser_long_new_path,
|
||||
"IAMUpdateUser_new_user_name_already_exists": IAMUpdateUser_new_user_name_already_exists,
|
||||
"IAMUpdateUser_success": IAMUpdateUser_success,
|
||||
"PresignedAuth_security_token_not_supported": PresignedAuth_security_token_not_supported,
|
||||
"PresignedAuth_unsupported_algorithm": PresignedAuth_unsupported_algorithm,
|
||||
"PresignedAuth_ECDSA_not_supported": PresignedAuth_ECDSA_not_supported,
|
||||
|
||||
Reference in New Issue
Block a user