mirror of
https://github.com/versity/versitygw.git
synced 2026-08-17 12:46:23 +00:00
Closes #1640 Add a standalone AWS IAM Query API implementation for managing IAM users through standard AWS SDKs and the AWS CLI. Server usage Start the IAM server with internal file-backed storage: mkdir -p /tmp/versitygw-iam ./versitygw --port 127.0.0.1:7070 --access user --secret pass iam --dir /tmp/versitygw-iam Start the IAM server with Vault KV v2 storage using AppRole: VGW_IAM_VAULT_ROLE_SECRET=<role-secret> ./versitygw --port 127.0.0.1:7070 --access user --secret pass iam --vault-endpoint-url http://127.0.0.1:8200 --vault-auth-method approle --vault-role-id <role-id> --vault-mount-path kv --vault-secret-storage-path iam Vault authentication also supports root tokens, separate authentication and secret-storage namespaces, custom mount paths, server certificate validation, and mutual TLS client certificates. Configure the AWS CLI credentials used by the IAM server: export AWS_ACCESS_KEY_ID=user export AWS_SECRET_ACCESS_KEY=pass export AWS_DEFAULT_REGION=us-east-1 Implemented IAM actions CreateUser creates an IAM user with an AWS-compatible ARN, generated AIDA user ID, creation timestamp, optional path, and tags. It validates usernames, paths, tag limits, reserved tag prefixes, duplicate tag keys, and existing users. aws --endpoint-url http://127.0.0.1:7070 iam create-user --user-name bob aws --endpoint-url http://127.0.0.1:7070 iam create-user --user-name bob --path /engineering/ --tags Key=team,Value=storage GetUser returns a stored user or the root identity when requested without a username through the IAM Query API. aws --endpoint-url http://127.0.0.1:7070 iam get-user --user-name bob ListUsers returns users in deterministic username order and supports path filtering, marker-based pagination, and MaxItems limits. aws --endpoint-url http://127.0.0.1:7070 iam list-users aws --endpoint-url http://127.0.0.1:7070 iam list-users --path-prefix /engineering/ --max-items 100 UpdateUser updates the username and/or path, recalculates the user ARN, and rejects conflicts with existing users. aws --endpoint-url http://127.0.0.1:7070 iam update-user --user-name bob --new-user-name robert --new-path /platform/ DeleteUser permanently removes an IAM user and returns AWS-compatible errors for missing users. aws --endpoint-url http://127.0.0.1:7070 iam delete-user --user-name robert IAM protocol and authentication - Support the AWS IAM Query protocol version 2010-05-08 over GET and POST form requests. - Return AWS-compatible XML responses, error documents, status codes, request IDs, user metadata, and pagination fields. - Authenticate root credentials with AWS Signature Version 4 for the IAM service in us-east-1. - Support both Authorization-header and query-string SigV4 authentication. - Validate credential scope, signed headers, timestamps, clock skew, content length, signatures, and unsupported signature or session-token modes. - Add IAM-specific validation and error mapping for malformed requests, invalid actions, duplicate entities, missing users, throttling, and internal failures. Storage implementations - Add an internal JSON-backed store using iam.json and iam.json.backup with atomic temporary-file replacement, concurrent access protection, stable ordering, pagination, and persistence across restarts. - Add a Vault KV v2 store with one secret per user, CAS-based duplicate protection, permanent deletion, AppRole reauthentication, namespace support, configurable authentication and KV mounts, root-token authentication, and TLS/mTLS configuration. - Introduce a common Storer interface and require exactly one storage backend to be configured. Server and embedding support - Register the new `versitygw iam` command with environment-variable and CLI configuration for both storage backends. - Add `embedgw.RunIAMAPI` and `IAMConfig` for embedding the IAM service in Go applications. Gateway-level internal packages - Add `internal/iamstore` as a reusable generic file-backed IAM persistence engine and migrate the existing gateway internal IAM service to it. - Add `internal/sigv4auth` for shared SigV4 header and presigned-query parsing, canonical request generation, signature verification, and structured authentication errors. - Refactor the S3 authentication paths to use the shared SigV4 implementation while preserving S3-specific error responses. - Add `internal/httpctx` for shared Fiber context keys and AWS-style request ID handling. - Add `internal/routekit` for shared query, form, and header route matchers. - Add `internal/netutil` for reusable certificate storage, hostname-aware listeners, multi-address serving, TLS listeners, and UNIX socket handling. - Update the custom SigV4 signer to honor an explicitly supplied signed-header list so unrelated headers do not alter IAM signatures. Testing and CI - Add AWS IAM SDK-based integration coverage for all supported user actions, header authentication, query authentication, validation, errors, filtering, and pagination. - Split standalone IAM tests into `versitygw test iam` and retain existing gateway IAM tests under `versitygw test gw-iam`. - Add unit coverage for controllers, authentication, routing, storage, embedding, listeners, request matching, persistence, and signing behavior. - Add `runiamtests.sh` to exercise internal storage over HTTP and HTTPS plus Vault storage through AppRole. - Add a dedicated IAM functional-test workflow with a Vault service and merged runtime coverage reporting. - Include the IAM test runner in shellcheck and add the AWS IAM SDK dependency.
175 lines
8.1 KiB
PowerShell
175 lines
8.1 KiB
PowerShell
#!/usr/bin/env pwsh
|
|
# Copyright 2026 Versity Software
|
|
# This file is licensed under the Apache License, Version 2.0
|
|
# (the "License"); you may not use this file except in compliance
|
|
# with the License. You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing,
|
|
# software distributed under the License is distributed on an
|
|
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
# KIND, either express or implied. See the License for the
|
|
# specific language governing permissions and limitations
|
|
# under the License.
|
|
|
|
# PowerShell equivalent of runtests.sh for Windows
|
|
|
|
$ErrorActionPreference = "Stop"
|
|
|
|
# Temp directories
|
|
$tmpGw = Join-Path $env:TEMP "gw"
|
|
$tmpCovdata = Join-Path $env:TEMP "covdata"
|
|
$tmpHttpsCovdata = Join-Path $env:TEMP "https.covdata"
|
|
$tmpVersioningCovdata = Join-Path $env:TEMP "versioning.covdata"
|
|
$tmpVersioningHttpsCovdata = Join-Path $env:TEMP "versioning.https.covdata"
|
|
$tmpNoaclCovdata = Join-Path $env:TEMP "noacl.covdata"
|
|
$tmpVersioningDir = Join-Path $env:TEMP "versioningdir"
|
|
$tmpSidecar = Join-Path $env:TEMP "sidecar"
|
|
|
|
foreach ($dir in @($tmpGw, $tmpCovdata, $tmpHttpsCovdata, $tmpVersioningCovdata,
|
|
$tmpVersioningHttpsCovdata, $tmpNoaclCovdata, $tmpVersioningDir,
|
|
$tmpSidecar)) {
|
|
if (Test-Path $dir) { Remove-Item -Recurse -Force $dir }
|
|
New-Item -ItemType Directory -Path $dir | Out-Null
|
|
}
|
|
|
|
# Setup TLS certificate and key
|
|
Write-Host "Generating TLS certificate and key in the cert.pem and key.pem files"
|
|
openssl genpkey -algorithm RSA -out key.pem -pkeyopt rsa_keygen_bits:2048
|
|
if ($LASTEXITCODE -ne 0) { throw "Failed to generate private key" }
|
|
openssl req -new -x509 -key key.pem -out cert.pem -days 365 `
|
|
-subj "/C=US/ST=California/L=San Francisco/O=Versity/OU=Software/CN=versity.com"
|
|
if ($LASTEXITCODE -ne 0) { throw "Failed to generate certificate" }
|
|
|
|
function Start-Gateway {
|
|
param(
|
|
[string] $CoverDir,
|
|
[string[]] $GwArgs
|
|
)
|
|
$env:GOCOVERDIR = $CoverDir
|
|
$proc = Start-Process -FilePath ".\versitygw.exe" -ArgumentList $GwArgs -PassThru -NoNewWindow
|
|
Remove-Item Env:\GOCOVERDIR -ErrorAction SilentlyContinue
|
|
return $proc
|
|
}
|
|
|
|
function Invoke-GwTest {
|
|
param(
|
|
[string] $Description,
|
|
[string[]] $TestArgs,
|
|
[System.Diagnostics.Process] $GatewayProc
|
|
)
|
|
& .\versitygw.exe test @TestArgs
|
|
if ($LASTEXITCODE -ne 0) {
|
|
Write-Host "$Description failed"
|
|
Stop-Process -Id $GatewayProc.Id -Force -ErrorAction SilentlyContinue
|
|
exit 1
|
|
}
|
|
}
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 1. HTTP (port 7070)
|
|
# ---------------------------------------------------------------------------
|
|
Write-Host "Running the sdk test over http"
|
|
$gwProc = Start-Gateway -CoverDir $tmpCovdata `
|
|
-GwArgs @("-a", "user", "-s", "pass", "--iam-dir", $tmpGw, "posix", "--sidecar", $tmpSidecar, $tmpGw)
|
|
Start-Sleep -Seconds 1
|
|
|
|
if ($gwProc.HasExited) {
|
|
Write-Host "server no longer running"
|
|
exit 1
|
|
}
|
|
|
|
Invoke-GwTest -Description "full flow tests" -GatewayProc $gwProc `
|
|
-TestArgs @("-a", "user", "-s", "pass", "-e", "http://127.0.0.1:7070", "full-flow", "--parallel", "--windows-test-mode")
|
|
Invoke-GwTest -Description "posix tests" -GatewayProc $gwProc `
|
|
-TestArgs @("-a", "user", "-s", "pass", "-e", "http://127.0.0.1:7070", "posix", "--windows-test-mode")
|
|
Invoke-GwTest -Description "iam tests" -GatewayProc $gwProc `
|
|
-TestArgs @("-a", "user", "-s", "pass", "-e", "http://127.0.0.1:7070", "gw-iam")
|
|
|
|
Stop-Process -Id $gwProc.Id -Force -ErrorAction SilentlyContinue
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 2. HTTPS (port 7071)
|
|
# ---------------------------------------------------------------------------
|
|
Write-Host "Running the sdk test over https"
|
|
$gwHttpsProc = Start-Gateway -CoverDir $tmpHttpsCovdata `
|
|
-GwArgs @("--cert", "$PWD\cert.pem", "--key", "$PWD\key.pem",
|
|
"-p", ":7071", "-a", "user", "-s", "pass", "--iam-dir", $tmpGw, "posix", "--sidecar", $tmpSidecar, $tmpGw)
|
|
Start-Sleep -Seconds 1
|
|
|
|
if ($gwHttpsProc.HasExited) {
|
|
Write-Host "https server no longer running"
|
|
exit 1
|
|
}
|
|
|
|
Invoke-GwTest -Description "https full flow tests" -GatewayProc $gwHttpsProc `
|
|
-TestArgs @("--allow-insecure", "-a", "user", "-s", "pass", "-e", "https://127.0.0.1:7071", "full-flow", "--parallel", "--windows-test-mode")
|
|
Invoke-GwTest -Description "https posix tests" -GatewayProc $gwHttpsProc `
|
|
-TestArgs @("--allow-insecure", "-a", "user", "-s", "pass", "-e", "https://127.0.0.1:7071", "posix", "--windows-test-mode")
|
|
Invoke-GwTest -Description "https iam tests" -GatewayProc $gwHttpsProc `
|
|
-TestArgs @("--allow-insecure", "-a", "user", "-s", "pass", "-e", "https://127.0.0.1:7071", "gw-iam")
|
|
|
|
Stop-Process -Id $gwHttpsProc.Id -Force -ErrorAction SilentlyContinue
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 3. Versioning HTTP (port 7072)
|
|
# ---------------------------------------------------------------------------
|
|
Write-Host "Running the sdk test over http against the versioning-enabled gateway"
|
|
$gwVsProc = Start-Gateway -CoverDir $tmpVersioningCovdata `
|
|
-GwArgs @("-p", ":7072", "-a", "user", "-s", "pass", "--iam-dir", $tmpGw,
|
|
"posix", "--sidecar", $tmpSidecar, "--versioning-dir", $tmpVersioningDir, $tmpGw)
|
|
Start-Sleep -Seconds 1
|
|
|
|
if ($gwVsProc.HasExited) {
|
|
Write-Host "versioning-enabled server no longer running"
|
|
exit 1
|
|
}
|
|
|
|
Invoke-GwTest -Description "versioning-enabled full-flow tests" -GatewayProc $gwVsProc `
|
|
-TestArgs @("-a", "user", "-s", "pass", "-e", "http://127.0.0.1:7072", "full-flow", "-vs", "--parallel", "--windows-test-mode")
|
|
Invoke-GwTest -Description "versioning-enabled posix tests" -GatewayProc $gwVsProc `
|
|
-TestArgs @("-a", "user", "-s", "pass", "-e", "http://127.0.0.1:7072", "posix", "-vs", "--windows-test-mode")
|
|
|
|
Stop-Process -Id $gwVsProc.Id -Force -ErrorAction SilentlyContinue
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 4. Versioning HTTPS (port 7073)
|
|
# ---------------------------------------------------------------------------
|
|
Write-Host "Running the sdk test over https against the versioning-enabled gateway"
|
|
$gwVsHttpsProc = Start-Gateway -CoverDir $tmpVersioningHttpsCovdata `
|
|
-GwArgs @("--cert", "$PWD\cert.pem", "--key", "$PWD\key.pem",
|
|
"-p", ":7073", "-a", "user", "-s", "pass", "--iam-dir", $tmpGw,
|
|
"posix", "--sidecar", $tmpSidecar, "--versioning-dir", $tmpVersioningDir, $tmpGw)
|
|
Start-Sleep -Seconds 1
|
|
|
|
if ($gwVsHttpsProc.HasExited) {
|
|
Write-Host "versioning-enabled https server no longer running"
|
|
exit 1
|
|
}
|
|
|
|
Invoke-GwTest -Description "versioning-enabled https full-flow tests" -GatewayProc $gwVsHttpsProc `
|
|
-TestArgs @("--allow-insecure", "-a", "user", "-s", "pass", "-e", "https://127.0.0.1:7073", "full-flow", "-vs", "--parallel", "--windows-test-mode")
|
|
Invoke-GwTest -Description "versioning-enabled https posix tests" -GatewayProc $gwVsHttpsProc `
|
|
-TestArgs @("--allow-insecure", "-a", "user", "-s", "pass", "-e", "https://127.0.0.1:7073", "posix", "-vs", "--windows-test-mode")
|
|
|
|
Stop-Process -Id $gwVsHttpsProc.Id -Force -ErrorAction SilentlyContinue
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 5. No ACL (port 7074)
|
|
# ---------------------------------------------------------------------------
|
|
Write-Host "Running No ACL integration tests"
|
|
$gwNoAclProc = Start-Gateway -CoverDir $tmpNoaclCovdata `
|
|
-GwArgs @("-p", ":7074", "-a", "user", "-s", "pass", "-noacl", "--iam-dir", $tmpGw, "posix", "--sidecar", $tmpSidecar, $tmpGw)
|
|
Start-Sleep -Seconds 1
|
|
|
|
if ($gwNoAclProc.HasExited) {
|
|
Write-Host "noacl server no longer running"
|
|
exit 1
|
|
}
|
|
|
|
Invoke-GwTest -Description "No ACL integration tests" -GatewayProc $gwNoAclProc `
|
|
-TestArgs @("--allow-insecure", "-a", "user", "-s", "pass", "-e", "http://127.0.0.1:7074", "noacl")
|
|
|
|
Stop-Process -Id $gwNoAclProc.Id -Force -ErrorAction SilentlyContinue
|