From 895d49b55b2e3a66a6b0b552343150e8341cd3cf Mon Sep 17 00:00:00 2001 From: Khris Richardson Date: Wed, 30 Sep 2026 05:45:47 -0700 Subject: [PATCH] s3/iam: manage roles through the IAM API, with an opt-in persistent role store (#11522) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * s3/iam: manage roles through the IAM API, with an opt-in persistent role store Roles could only come from the IAM config file: the S3 server pinned the role store to memory and the embedded IAM API had no role actions, so a role could not be created, retrusted or revoked without editing the file and restarting every gateway. Role store - Read the `roleStore` key (the IAMConfig field already existed). With an IAM config file the default stays memory; with none it is the filer, as for OIDC providers, so zero-config clusters keep runtime-created roles. - Roles from the IAM config file never go into a persistent role store, which outlives the file and may be shared by S3 servers with different files. They are served from memory beneath the store, as OIDC providers are: a stored role of the same name takes precedence, and deleting it restores the file's. A config-file role cannot be changed or deleted through the API (UnmodifiableEntity), and removing one from the file removes it at the next start. An in-memory store holds them as records, as before. They have no creation time, so CreateDate is omitted rather than reporting when this server started. SetRoleStore installs a store the same way, so a store set after startup keeps the config-file roles, as SetOIDCProviderStore does for providers. - Watch /etc/iam/roles and drop the cached role definitions on change. The cached filer store otherwise serves a peer's stale role for up to its 5m TTL, which keeps a revoked trust policy in force on the other gateways. - Role stores wrap ErrRoleNotFound for a missing role; the filer store used to report any failed lookup as "role not found". CreateRole proceeds only on a confirmed absence, so an unreadable store cannot let it write over an existing role. IAM actions - CreateRole, GetRole, ListRoles, DeleteRole, UpdateAssumeRolePolicy, AttachRolePolicy, DetachRolePolicy, ListAttachedRolePolicies. The reads are allowed in read-only mode. - A role defined in the config file is reloaded from it at every start, so changing or deleting it through the API is refused (UnmodifiableEntity) rather than silently reverted. - DeleteRole with policies attached is refused (DeleteConflict), as on AWS. - Role names follow AWS's rules ([\w+=,.@-]{1,64}); a role is stored as .json in the filer, so this also keeps a name from leaving the role store's directory. At most 10 managed policies per role (AWS's default quota; MaxManagedPoliciesPerUser is 10 too), LimitExceeded beyond. - DeletePolicy is refused (DeleteConflict) while a role attaches the policy, as it already is for users and groups: roles attach policies by name, so a policy created later under the deleted one's name would otherwise take effect on the role. - Role paths other than "/" and role tags are not stored, so they are refused rather than dropped. Role IDs and sessions - Roles get a unique RoleId when first stored (random, AWS AROA form), kept across updates; a config-file role gets a stable ID derived from its name, since it is created again at every start. - Sessions issued through AssumeRoleWithWebIdentity, AssumeRoleWithCredentials and AssumeRole carry the role's ID (claim "rid"), and a request under a role whose current ID differs is denied. Resolving a session's policies by role name let a session outlive its role: once a role was deleted, a role later created under the same name — with a different trust policy and different policies — revived every unexpired session of the old one with the new role's permissions. Sessions issued before this change carry no ID and are unaffected until they expire. Integration test (test/s3/iam, run with `make start-services`): TestWebIdentityWithProviderAndRoleManagedThroughIAMAPI configures an OIDC provider, a managed policy and a role entirely through the IAM API against a JWKS served by the test, then checks the trusted subject gets credentials scoped to the attached policy; another subject, a token signed by another key, an unsigned token and a token for another audience are refused; and UpdateAssumeRolePolicy moves the trust at once. Co-Authored-By: Claude Opus 5.5 (1M context) * s3/iam: bind every role session to its role, and change roles atomically Review follow-ups. Session binding - The role-ID check ran only when a session carried no policy names, and AssumeRole embeds the role's attached policies, so those sessions kept their permissions after the role was deleted or recreated. The check now runs for every session carrying a role ID, before policy selection. - A named role that cannot be resolved at issuance gets no session, instead of one with no role ID (which nothing binds). - A config-file role's ID is derived from its name and trust policy, not the name alone: a different role put in the file under the same name gets a new ID, while an unchanged role keeps its sessions across restarts. Role writes - RoleStore gains UpdateRole, a read-modify-write that lands only if the role is unchanged since the read, and otherwise re-reads and retries. The filer store uses the filer's write conditions (IF_NOT_EXISTS for a new role, IF_ENTRY_EQUAL otherwise). CreateRole, UpdateAssumeRolePolicy and Attach/DetachRolePolicy all go through it, so two gateways no longer overwrite each other's changes, a change racing a delete no longer writes the role back, and of two concurrent creates one gets EntityAlreadyExists. - The filer store's ListRoles pages past 1,000 entries and fails on a broken stream instead of returning what arrived, so DeletePolicy's attachment check sees every role. ListRoles skips a role deleted between listing and reading it. - CreateRole validates first; a failed write is ServiceFailure, not InvalidInput. Any Tags.* parameter is refused, not only the first key. - ExecuteAction's skipPersist covers the S3ApiConfiguration only; the comment now says so. Role and OIDC provider actions write their own stores. Each fix has a test that fails without it. Against a real filer with two gateways, concurrent AttachRolePolicy calls lost 1-4 of 8 attachments per run before this change and none after. Co-Authored-By: Claude Opus 5.5 (1M context) * s3/iam: one role snapshot per decision; DeleteRole is atomic; watch a custom role store path Review follow-ups. - Authorization evaluates the policies of the role definition the session's binding was checked against, instead of reading the role again: a role replaced in between cannot lend a session its policies. - AssumeRole and AssumeRoleWithLDAPIdentity issue the session from the definition whose trust admits the caller (IAMManager.ResolveRoleForPrincipal), and take its ID, duration cap and embedded policies from that same definition. A role replaced after the caller's trust check by one that does not trust the caller now yields AccessDenied, not a session bound to the replacement. - A RoleUpdate that returns nil deletes the role, on the same condition as a write: the filer store deletes with ObjectTransaction on IF_ENTRY_EQUAL, routed and locked like the conditional CreateEntry. DeleteRole decides against the role it deletes, so a policy attached meanwhile on another server is a DeleteConflict, and a delete never removes a role written after its check. - S3 servers watch the role store's configured basePath, not only /etc/iam/roles, so a custom path also drops peers' cached roles on change. Each has a test that fails without it. Live against a real filer: DeleteRole refuses while a policy is attached and removes the entry once detached; all test/s3/iam CI stages pass. Co-Authored-By: Claude Opus 5.5 (1M context) * s3/iam: state which roles DeletePolicy's attachment check can see RolesAttachingPolicy sees the stored roles and this server's config-file roles. A role defined only in another server's IAM config file is invisible to it, so a config-file role that attaches a managed policy is protected only on the servers whose file defines it. The doc comment now says so and how to avoid it: keep such roles in every server's file, or attach only config-file policies to config-file roles. Co-Authored-By: Claude Opus 5.5 (1M context) * iam: note that a role store set after startup is not watched for peer changes S3 servers build their metadata watch list once, at startup, from the role store installed then. SetRoleStore's doc now says that a filer-backed store installed later with a different basePath is not watched, so peers' changes to it reach this server's cached roles only when the cache expires. Co-Authored-By: Claude Opus 5.5 (1M context) --------- Co-authored-by: Claude Opus 5.5 (1M context) --- test/s3/iam/s3_sts_web_identity_api_test.go | 258 +++++++++++ .../integration/cached_role_store_generic.go | 11 + weed/iam/integration/iam_manager.go | 273 +++++++++++- weed/iam/integration/iam_manager_trust.go | 24 +- .../role_id_session_binding_test.go | 165 +++++++ weed/iam/integration/role_store.go | 416 +++++++++++++++-- weed/iam/integration/role_store_filer_test.go | 286 ++++++++++++ .../integration/role_store_notfound_test.go | 14 + .../integration/role_store_persist_test.go | 196 ++++++++ weed/iam/responses.go | 80 ++++ weed/iam/sts/session_claims.go | 15 +- weed/iam/sts/sts_service.go | 22 +- weed/s3api/auth_credentials_subscribe.go | 39 ++ weed/s3api/iam_defaults_test.go | 34 ++ weed/s3api/s3api_embedded_iam.go | 32 +- weed/s3api/s3api_embedded_iam_role.go | 419 ++++++++++++++++++ weed/s3api/s3api_iam_role_test.go | 340 ++++++++++++++ weed/s3api/s3api_server.go | 46 +- weed/s3api/s3api_sts.go | 63 ++- weed/s3api/s3api_sts_assume_role_test.go | 52 ++- 20 files changed, 2700 insertions(+), 85 deletions(-) create mode 100644 test/s3/iam/s3_sts_web_identity_api_test.go create mode 100644 weed/iam/integration/role_id_session_binding_test.go create mode 100644 weed/iam/integration/role_store_filer_test.go create mode 100644 weed/iam/integration/role_store_notfound_test.go create mode 100644 weed/iam/integration/role_store_persist_test.go create mode 100644 weed/s3api/s3api_embedded_iam_role.go create mode 100644 weed/s3api/s3api_iam_role_test.go diff --git a/test/s3/iam/s3_sts_web_identity_api_test.go b/test/s3/iam/s3_sts_web_identity_api_test.go new file mode 100644 index 000000000..87e3719d6 --- /dev/null +++ b/test/s3/iam/s3_sts_web_identity_api_test.go @@ -0,0 +1,258 @@ +package iam + +import ( + "crypto/rand" + "crypto/rsa" + "encoding/base64" + "encoding/json" + "encoding/xml" + "io" + "math/big" + "net/http" + "net/http/httptest" + "net/url" + "strconv" + "testing" + "time" + + "github.com/aws/aws-sdk-go/aws" + "github.com/aws/aws-sdk-go/aws/credentials" + "github.com/aws/aws-sdk-go/aws/session" + "github.com/aws/aws-sdk-go/service/iam" + "github.com/aws/aws-sdk-go/service/s3" + "github.com/golang-jwt/jwt/v5" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +const ( + webIdentityAudience = "web-identity-api-test" + webIdentitySubject = "spiffe://example.org/ns/app/sa/app" +) + +// webIdentityIssuer is an OIDC issuer served by the test: discovery document +// and JWKS for one RSA key, so the server under test fetches real keys. +type webIdentityIssuer struct { + server *httptest.Server + key *rsa.PrivateKey +} + +func newWebIdentityIssuer(t *testing.T) *webIdentityIssuer { + t.Helper() + key, err := rsa.GenerateKey(rand.Reader, 2048) + require.NoError(t, err) + issuer := &webIdentityIssuer{key: key} + mux := http.NewServeMux() + mux.HandleFunc("/.well-known/openid-configuration", func(w http.ResponseWriter, _ *http.Request) { + _ = json.NewEncoder(w).Encode(map[string]any{ + "issuer": issuer.server.URL, + "jwks_uri": issuer.server.URL + "/keys", + "id_token_signing_alg_values_supported": []string{"RS256"}, + }) + }) + mux.HandleFunc("/keys", func(w http.ResponseWriter, _ *http.Request) { + b64 := func(b []byte) string { return base64.RawURLEncoding.EncodeToString(b) } + _ = json.NewEncoder(w).Encode(map[string]any{"keys": []any{map[string]any{ + "kty": "RSA", "kid": "k1", "use": "sig", "alg": "RS256", + "n": b64(key.PublicKey.N.Bytes()), "e": b64(big.NewInt(int64(key.PublicKey.E)).Bytes()), + }}}) + }) + issuer.server = httptest.NewServer(mux) + t.Cleanup(issuer.server.Close) + return issuer +} + +func (i *webIdentityIssuer) claims(sub string) jwt.MapClaims { + now := time.Now() + return jwt.MapClaims{"iss": i.server.URL, "sub": sub, "aud": webIdentityAudience, + "iat": now.Unix(), "exp": now.Add(10 * time.Minute).Unix()} +} + +func (i *webIdentityIssuer) token(t *testing.T, sub string, key *rsa.PrivateKey) string { + t.Helper() + tok := jwt.NewWithClaims(jwt.SigningMethodRS256, i.claims(sub)) + tok.Header["kid"] = "k1" + signed, err := tok.SignedString(key) + require.NoError(t, err) + return signed +} + +func (i *webIdentityIssuer) tokenForAudience(t *testing.T, sub, aud string) string { + t.Helper() + claims := i.claims(sub) + claims["aud"] = aud + tok := jwt.NewWithClaims(jwt.SigningMethodRS256, claims) + tok.Header["kid"] = "k1" + signed, err := tok.SignedString(i.key) + require.NoError(t, err) + return signed +} + +func (i *webIdentityIssuer) unsignedToken(t *testing.T, sub string) string { + t.Helper() + signed, err := jwt.NewWithClaims(jwt.SigningMethodNone, i.claims(sub)).SignedString(jwt.UnsafeAllowNoneSignatureType) + require.NoError(t, err) + return signed +} + +func trustPolicyFor(issuerURL, sub string) string { + doc, _ := json.Marshal(map[string]any{"Version": "2012-10-17", "Statement": []any{map[string]any{ + "Effect": "Allow", "Principal": map[string]any{"Federated": issuerURL}, + "Action": []string{"sts:AssumeRoleWithWebIdentity"}, + "Condition": map[string]any{"StringEquals": map[string]any{"oidc:sub": sub}}, + }}}) + return string(doc) +} + +// assumeWithWebIdentity returns session credentials, or nil and the error body. +func assumeWithWebIdentity(t *testing.T, roleArn, token string) (*credentials.Credentials, string) { + t.Helper() + resp, err := callSTSAPI(t, url.Values{ + "Action": {"AssumeRoleWithWebIdentity"}, "Version": {"2011-06-15"}, + "RoleArn": {roleArn}, "RoleSessionName": {"web-identity-api"}, "WebIdentityToken": {token}, + }) + require.NoError(t, err) + defer resp.Body.Close() + body, err := io.ReadAll(resp.Body) + require.NoError(t, err) + if resp.StatusCode != http.StatusOK { + return nil, string(body) + } + var out AssumeRoleWithWebIdentityTestResponse + require.NoError(t, xml.Unmarshal(body, &out), "body: %s", body) + c := out.Result.Credentials + return credentials.NewStaticCredentials(c.AccessKeyId, c.SecretAccessKey, c.SessionToken), "" +} + +func s3ClientWith(t *testing.T, creds *credentials.Credentials) *s3.S3 { + t.Helper() + sess, err := session.NewSession(&aws.Config{ + Region: aws.String(TestRegion), Endpoint: aws.String(TestS3Endpoint), + Credentials: creds, S3ForcePathStyle: aws.Bool(true), DisableSSL: aws.Bool(true), + }) + require.NoError(t, err) + return s3.New(sess) +} + +// TestWebIdentityWithProviderAndRoleManagedThroughIAMAPI configures STS +// federation entirely at runtime — an OIDC provider, a managed policy and a role +// created through the IAM API, with no static configuration — and checks that +// the role admits exactly the subject its trust policy names, with exactly the +// permissions of its attached policy. +func TestWebIdentityWithProviderAndRoleManagedThroughIAMAPI(t *testing.T) { + if testing.Short() { + t.Skip("Skipping integration test in short mode") + } + if !isSTSEndpointRunning(t) { + t.Fatal("SeaweedFS STS endpoint is not running at", TestSTSEndpoint, "- please run 'make setup-all-tests' first") + } + + framework := NewS3IAMTestFramework(t) + defer framework.Cleanup() + admin, err := framework.CreateIAMClientWithJWT("admin-user", "TestAdminRole") + require.NoError(t, err) + adminS3, err := framework.CreateS3ClientWithJWT("admin-user", "TestAdminRole") + require.NoError(t, err) + + issuer := newWebIdentityIssuer(t) + bucket := framework.GenerateUniqueBucketName("web-identity") + require.NoError(t, framework.CreateBucketWithCleanup(adminS3, bucket)) + + provider, err := admin.CreateOpenIDConnectProvider(&iam.CreateOpenIDConnectProviderInput{ + Url: aws.String(issuer.server.URL), + ClientIDList: []*string{aws.String(webIdentityAudience)}, + // Required by this SDK version; pinning only applies to a TLS issuer, + // and the test issuer is plain HTTP. + ThumbprintList: []*string{aws.String("0000000000000000000000000000000000000000")}, + }) + require.NoError(t, err) + defer admin.DeleteOpenIDConnectProvider(&iam.DeleteOpenIDConnectProviderInput{OpenIDConnectProviderArn: provider.OpenIDConnectProviderArn}) + + policyDoc, _ := json.Marshal(map[string]any{"Version": "2012-10-17", "Statement": []any{map[string]any{ + "Effect": "Allow", "Action": []string{"s3:*"}, + "Resource": []string{"arn:aws:s3:::" + bucket, "arn:aws:s3:::" + bucket + "/*"}, + }}}) + policy, err := admin.CreatePolicy(&iam.CreatePolicyInput{ + PolicyName: aws.String(bucket + "-rw"), PolicyDocument: aws.String(string(policyDoc)), + }) + require.NoError(t, err) + defer admin.DeletePolicy(&iam.DeletePolicyInput{PolicyArn: policy.Policy.Arn}) + + // Role names are at most 64 characters; the bucket name is longer. + roleName := "web-identity-" + strconv.FormatInt(time.Now().UnixNano(), 36) + role, err := admin.CreateRole(&iam.CreateRoleInput{ + RoleName: aws.String(roleName), + AssumeRolePolicyDocument: aws.String(trustPolicyFor(issuer.server.URL, webIdentitySubject)), + }) + require.NoError(t, err) + defer admin.DeleteRole(&iam.DeleteRoleInput{RoleName: aws.String(roleName)}) + _, err = admin.AttachRolePolicy(&iam.AttachRolePolicyInput{RoleName: aws.String(roleName), PolicyArn: policy.Policy.Arn}) + require.NoError(t, err) + defer admin.DetachRolePolicy(&iam.DetachRolePolicyInput{RoleName: aws.String(roleName), PolicyArn: policy.Policy.Arn}) + roleArn := aws.StringValue(role.Role.Arn) + + t.Run("the trusted subject gets credentials scoped to the attached policy", func(t *testing.T) { + creds, failure := assumeWithWebIdentity(t, roleArn, issuer.token(t, webIdentitySubject, issuer.key)) + require.NotNil(t, creds, "AssumeRoleWithWebIdentity refused the trusted subject: %s", failure) + client := s3ClientWith(t, creds) + _, err := client.PutObject(&s3.PutObjectInput{Bucket: aws.String(bucket), Key: aws.String("federated.txt")}) + assert.NoError(t, err, "the session cannot write the bucket its policy grants") + _, err = client.CreateBucket(&s3.CreateBucketInput{Bucket: aws.String(bucket + "-other")}) + assert.Error(t, err, "the session created a bucket its policy does not grant") + }) + + refusals := map[string]string{ + "another subject": issuer.token(t, "spiffe://example.org/ns/other/sa/other", issuer.key), + "a token signed by another key": func() string { + other, err := rsa.GenerateKey(rand.Reader, 2048) + require.NoError(t, err) + return issuer.token(t, webIdentitySubject, other) + }(), + "an unsigned token": issuer.unsignedToken(t, webIdentitySubject), + // The provider accepts only its registered client IDs as audiences. + "a token issued for another audience": issuer.tokenForAudience(t, webIdentitySubject, "some-other-service"), + } + for name, token := range refusals { + t.Run("refuses "+name, func(t *testing.T) { + creds, _ := assumeWithWebIdentity(t, roleArn, token) + assert.Nil(t, creds, "AssumeRoleWithWebIdentity issued credentials for %s", name) + }) + } + + t.Run("an updated trust policy takes effect", func(t *testing.T) { + moved := "spiffe://example.org/ns/app/sa/moved" + _, err := admin.UpdateAssumeRolePolicy(&iam.UpdateAssumeRolePolicyInput{ + RoleName: aws.String(roleName), PolicyDocument: aws.String(trustPolicyFor(issuer.server.URL, moved)), + }) + require.NoError(t, err) + creds, _ := assumeWithWebIdentity(t, roleArn, issuer.token(t, webIdentitySubject, issuer.key)) + assert.Nil(t, creds, "the subject removed from the trust policy still assumes the role") + creds, failure := assumeWithWebIdentity(t, roleArn, issuer.token(t, moved, issuer.key)) + assert.NotNil(t, creds, "the subject added to the trust policy was refused: %s", failure) + }) + + t.Run("a session does not survive its role being deleted and created again", func(t *testing.T) { + trust := trustPolicyFor(issuer.server.URL, webIdentitySubject) + _, err := admin.UpdateAssumeRolePolicy(&iam.UpdateAssumeRolePolicyInput{RoleName: aws.String(roleName), PolicyDocument: aws.String(trust)}) + require.NoError(t, err) + before, failure := assumeWithWebIdentity(t, roleArn, issuer.token(t, webIdentitySubject, issuer.key)) + require.NotNil(t, before, "precondition: %s", failure) + + _, err = admin.DetachRolePolicy(&iam.DetachRolePolicyInput{RoleName: aws.String(roleName), PolicyArn: policy.Policy.Arn}) + require.NoError(t, err) + _, err = admin.DeleteRole(&iam.DeleteRoleInput{RoleName: aws.String(roleName)}) + require.NoError(t, err) + recreated, err := admin.CreateRole(&iam.CreateRoleInput{RoleName: aws.String(roleName), AssumeRolePolicyDocument: aws.String(trust)}) + require.NoError(t, err) + _, err = admin.AttachRolePolicy(&iam.AttachRolePolicyInput{RoleName: aws.String(roleName), PolicyArn: policy.Policy.Arn}) + require.NoError(t, err) + assert.NotEqual(t, aws.StringValue(role.Role.RoleId), aws.StringValue(recreated.Role.RoleId), "the recreated role reuses the deleted role's ID") + + _, err = s3ClientWith(t, before).PutObject(&s3.PutObjectInput{Bucket: aws.String(bucket), Key: aws.String("revived.txt")}) + assert.Error(t, err, "a session of the deleted role works again under the new role of the same name") + after, failure := assumeWithWebIdentity(t, roleArn, issuer.token(t, webIdentitySubject, issuer.key)) + require.NotNil(t, after, "the new role refuses its trusted subject: %s", failure) + _, err = s3ClientWith(t, after).PutObject(&s3.PutObjectInput{Bucket: aws.String(bucket), Key: aws.String("fresh.txt")}) + assert.NoError(t, err, "a session of the new role cannot use its policy") + }) +} diff --git a/weed/iam/integration/cached_role_store_generic.go b/weed/iam/integration/cached_role_store_generic.go index 4165958d4..23f64fe3c 100644 --- a/weed/iam/integration/cached_role_store_generic.go +++ b/weed/iam/integration/cached_role_store_generic.go @@ -116,6 +116,14 @@ func (c *GenericCachedRoleStore) DeleteRole(ctx context.Context, filerAddress st return c.Delete(ctx, filerAddress, roleName) } +// UpdateRole implements RoleStore interface. The update reads the filer, not +// the cache, and the cache is dropped afterwards whatever the outcome: a +// refused write may mean the cached role is stale. +func (c *GenericCachedRoleStore) UpdateRole(ctx context.Context, filerAddress string, roleName string, update RoleUpdate) error { + defer c.ClearCache() + return c.adapter.store.UpdateRole(ctx, filerAddress, roleName, update) +} + // genericCopyRoleDefinition creates a deep copy of a RoleDefinition for the generic cache func genericCopyRoleDefinition(role *RoleDefinition) *RoleDefinition { if role == nil { @@ -127,6 +135,9 @@ func genericCopyRoleDefinition(role *RoleDefinition) *RoleDefinition { RoleArn: role.RoleArn, Description: role.Description, MaxSessionDuration: role.MaxSessionDuration, + Source: role.Source, + CreatedAt: role.CreatedAt, + RoleId: role.RoleId, } // Deep copy trust policy if it exists diff --git a/weed/iam/integration/iam_manager.go b/weed/iam/integration/iam_manager.go index 45290c5e4..3254025d5 100644 --- a/weed/iam/integration/iam_manager.go +++ b/weed/iam/integration/iam_manager.go @@ -2,7 +2,10 @@ package integration import ( "context" + "crypto/rand" + "crypto/sha256" "encoding/base64" + "encoding/hex" "encoding/json" "errors" "fmt" @@ -36,6 +39,9 @@ type IAMManager struct { // file, by ARN. With an in-memory store they are also written to the // store; a persistent store never holds them (see installOIDCProviderStore). staticOIDCProviders map[string]*OIDCProviderRecord + // staticRoles are the roles of this server's IAM config file, by name, once + // LoadStaticRoles has run (see installRoleStore). + staticRoles map[string]*RoleDefinition // oidcRetryMu guards the background refresh retry and which store is // current: cancelOIDCLoad stops the retry, oidcRetryGen names the one // running (0 when none) so at most one runs, and oidcRetryAgain records a @@ -138,6 +144,20 @@ func (m *IAMManager) GetOIDCProviderStore() OIDCProviderStore { return m.oidcProviderStore } +// GetRoleStore returns the configured role store. +func (m *IAMManager) GetRoleStore() RoleStore { + return m.roleStore +} + +// SetRoleStore replaces the role store. An S3 server builds the list of +// directories it watches for peers' changes once, at startup, from the store +// installed then (RoleStoreDirectory): a filer-backed store installed later +// with a different basePath is not watched, so peers' changes to it reach this +// server's cached roles only when the cache expires. +func (m *IAMManager) SetRoleStore(store RoleStore) { + m.installRoleStore(context.Background(), store) +} + // GetOIDCProvider returns the record for the given ARN, or an error if the // store is not configured or the record is missing. func (m *IAMManager) GetOIDCProvider(ctx context.Context, arn string) (*OIDCProviderRecord, error) { @@ -470,6 +490,88 @@ type RoleDefinition struct { // set it must satisfy AWS bounds: 3600 ≤ MaxSessionDuration ≤ 43200. // Honoured by AssumeRole, AssumeRoleWithWebIdentity, AssumeRoleWithCredentials. MaxSessionDuration int64 `json:"maxSessionDuration,omitempty"` + + // Source records where the role came from. RoleSourceStaticConfig marks a + // role loaded from the IAM config file; empty means it was created at + // runtime. Only static-config roles are pruned when they leave the file. + Source string `json:"source,omitempty"` + + // CreatedAt is when the role was created through the IAM API. Zero for + // roles loaded from the config file. + CreatedAt time.Time `json:"createdAt,omitempty"` + + // RoleId uniquely identifies this role, as AWS's RoleId does. A role + // deleted and created again under the same name gets a new ID, and a + // session is honoured only while the role it was issued for still has + // the ID the session carries — so a session outlives neither the role's + // deletion nor a later role that reuses its name. + RoleId string `json:"roleId,omitempty"` +} + +// NewRoleID returns a fresh, random role ID in AWS's AROA form. +func NewRoleID() string { + const alphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZ234567" + b := make([]byte, 17) + if _, err := rand.Read(b); err != nil { + panic(fmt.Sprintf("read random role id: %v", err)) + } + for i := range b { + b[i] = alphabet[int(b[i])%len(alphabet)] + } + return "AROA" + string(b) +} + +// StaticRoleID is the ID of a role defined in the IAM config file. Such a +// role is created again at every start, so its ID is derived rather than +// random, to keep sessions valid across restarts. It is derived from the name +// and the trust policy together: a role removed from the file and replaced by +// a different one under the same name trusts different principals, and must +// not inherit the old role's sessions. Restoring the same role restores its ID. +func StaticRoleID(role *RoleDefinition) string { + h := sha256.New() + h.Write([]byte("static-role:" + role.RoleName + "\x00")) + if role.TrustPolicy != nil { + trust, err := json.Marshal(role.TrustPolicy) + if err != nil { + // A trust policy that cannot be encoded cannot be matched on + // either; give the role an ID no session can carry. + return NewRoleID() + } + h.Write(trust) + } + return "AROA" + strings.ToUpper(hex.EncodeToString(h.Sum(nil)))[:17] +} + +// RoleSourceStaticConfig is the Source of a role loaded from the IAM config +// file. +const RoleSourceStaticConfig = "static-config" + +// checkSessionRoleBinding refuses a session issued for a role that has since +// been deleted, or replaced by a role reusing its name: the role's current ID +// must be the one the session carries. It runs for every session carrying a +// role ID, whatever policies the session embeds — the policies a session +// embeds are the ones its role had, and outlive the role otherwise. A session +// issued before role IDs were recorded carries none and is not bound. It +// returns the role it checked, nil for an unbound session. +func (m *IAMManager) checkSessionRoleBinding(ctx context.Context, sessionInfo *sts.SessionInfo) (*RoleDefinition, error) { + if sessionInfo == nil || sessionInfo.RoleId == "" { + return nil, nil + } + roleName := utils.ExtractRoleNameFromArn(sessionInfo.RoleArn) + if roleName == "" { + return nil, nil + } + role, err := m.roleStore.GetRole(ctx, m.getFilerAddress(), roleName) + if errors.Is(err, ErrRoleNotFound) { + return nil, fmt.Errorf("session was issued for role %s, which no longer exists", roleName) + } + if err != nil { + return nil, fmt.Errorf("resolve role %s for session: %w", roleName, err) + } + if role.RoleId != sessionInfo.RoleId { + return nil, fmt.Errorf("session was issued for an earlier role named %s", roleName) + } + return role, nil } // ActionRequest represents a request to perform an action @@ -1008,14 +1110,51 @@ func (m *IAMManager) CreateRole(ctx context.Context, filerAddress string, roleNa if !m.initialized { return fmt.Errorf("IAM manager not initialized") } + if err := PrepareRoleDefinition(roleName, roleDef); err != nil { + return err + } + if roleDef.RoleId == "" { + roleDef.RoleId = NewRoleID() + } + // Store role definition + return m.roleStore.StoreRole(ctx, "", roleName, roleDef) +} + +// UpdateRole changes a role atomically in the role store (see +// RoleStore.UpdateRole). update receives the role's current definition, nil +// when it does not exist, and its result is validated like CreateRole's; a +// nil result deletes the role. +// The IAM API's role actions use it, so a change made on one S3 server is +// neither lost to a concurrent change on another nor written over a delete. +func (m *IAMManager) UpdateRole(ctx context.Context, roleName string, update RoleUpdate) error { + if !m.initialized { + return fmt.Errorf("IAM manager not initialized") + } + return m.roleStore.UpdateRole(ctx, "", roleName, func(current *RoleDefinition) (*RoleDefinition, error) { + next, err := update(current) + if err != nil || next == nil { + return next, err + } + if err := PrepareRoleDefinition(roleName, next); err != nil { + return nil, err + } + return next, nil + }) +} + +// PrepareRoleDefinition applies CreateRole's defaults and validation to a role +// about to be stored or loaded. +func PrepareRoleDefinition(roleName string, roleDef *RoleDefinition) error { if roleName == "" { return fmt.Errorf("role name cannot be empty") } - if roleDef == nil { return fmt.Errorf("role definition cannot be nil") } + if roleDef.RoleName == "" { + roleDef.RoleName = roleName + } // Set role ARN if not provided if roleDef.RoleArn == "" { @@ -1035,12 +1174,122 @@ func (m *IAMManager) CreateRole(ctx context.Context, filerAddress string, roleNa return fmt.Errorf("MaxSessionDuration must be between 3600 and 43200 seconds, got %d", roleDef.MaxSessionDuration) } } + return nil +} - // Store role definition - return m.roleStore.StoreRole(ctx, "", roleName, roleDef) +// LoadStaticRoles installs the roles of the IAM config file. +// +// An in-memory store holds them as records, as it always has. A persistent +// store never does: it outlives this process and may be shared by S3 servers +// with different config files, so a record written from one file would +// outlive its removal from that file and be honoured by every server. Those +// roles are served from memory instead, ahead of the store, and cannot be +// changed or deleted through the store (ErrRoleStatic). A role stored under +// the same name takes precedence. They report no creation time. +func (m *IAMManager) LoadStaticRoles(ctx context.Context, roles []*RoleDefinition) { + defs := make(map[string]*RoleDefinition, len(roles)) + for _, role := range roles { + if role == nil { + continue + } + role.Source = RoleSourceStaticConfig + if role.RoleId == "" { + role.RoleId = StaticRoleID(role) + } + if err := PrepareRoleDefinition(role.RoleName, role); err != nil { + glog.Warningf("Failed to load role %s: %v", role.RoleName, err) + continue + } + defs[role.RoleName] = role + } + m.staticRoles = defs + m.installRoleStore(ctx, m.roleStore) +} + +// installRoleStore makes store the role store, with the config-file roles +// installed in it as LoadStaticRoles describes, so a store set after startup +// behaves like the one set at startup. +func (m *IAMManager) installRoleStore(ctx context.Context, store RoleStore) { + if overlay, ok := store.(*staticRoleOverlay); ok { + store = overlay.inner + } + if store == nil || m.staticRoles == nil { + m.roleStore = store + return + } + if _, inMemory := store.(*MemoryRoleStore); inMemory { + for name, role := range m.staticRoles { + if err := store.StoreRole(ctx, "", name, role); err != nil { + glog.Warningf("Failed to create role %s: %v", name, err) + } + } + m.roleStore = store + return + } + m.roleStore = &staticRoleOverlay{static: m.staticRoles, inner: store} } // GetRole retrieves a role definition by name. +// ListRoles returns every stored role definition. +func (m *IAMManager) ListRoles(ctx context.Context) ([]*RoleDefinition, error) { + if !m.initialized { + return nil, fmt.Errorf("IAM manager not initialized") + } + names, err := m.roleStore.ListRoles(ctx, m.getFilerAddress()) + if err != nil { + return nil, fmt.Errorf("list roles: %w", err) + } + roles := make([]*RoleDefinition, 0, len(names)) + for _, name := range names { + role, err := m.roleStore.GetRole(ctx, m.getFilerAddress(), name) + if errors.Is(err, ErrRoleNotFound) { + continue // deleted between list and read + } + if err != nil { + return nil, fmt.Errorf("get role %s: %w", name, err) + } + roles = append(roles, role) + } + return roles, nil +} + +// DeleteRole removes a role definition. +func (m *IAMManager) DeleteRole(ctx context.Context, roleName string) error { + if !m.initialized { + return fmt.Errorf("IAM manager not initialized") + } + if roleName == "" { + return fmt.Errorf("role name cannot be empty") + } + return m.roleStore.DeleteRole(ctx, m.getFilerAddress(), roleName) +} + +// InvalidateRoleCache drops any cached role definitions, so a change written +// to the store by a peer is seen on the next lookup rather than after the +// cache TTL. +func (m *IAMManager) InvalidateRoleCache() { + if cached, ok := m.roleStore.(interface{ ClearCache() }); ok { + cached.ClearCache() + } +} + +// RoleStoreDirectory is the filer directory the role store keeps roles in, +// its configured basePath; empty when the store is not filer-backed. S3 +// servers watch it to drop cached roles when a peer changes one. +func (m *IAMManager) RoleStoreDirectory() string { + store := m.roleStore + if overlay, ok := store.(*staticRoleOverlay); ok { + store = overlay.inner + } + if cached, ok := store.(*GenericCachedRoleStore); ok { + store = cached.adapter.store + } + if filerStore, ok := store.(*FilerRoleStore); ok { + return filerStore.basePath + } + return "" +} + func (m *IAMManager) GetRole(ctx context.Context, roleName string) (*RoleDefinition, error) { if !m.initialized { return nil, fmt.Errorf("IAM manager not initialized") @@ -1151,6 +1400,7 @@ func (m *IAMManager) AssumeRoleWithWebIdentity(ctx context.Context, request *sts // the global MaxSessionLength and the source-token-expiry cap on top of // this; per-role takes precedence whenever it is the tightest bound. request.DurationSeconds = capDurationByRole(request.DurationSeconds, roleDef.MaxSessionDuration, m.defaultTokenDurationSeconds(), m.maxSessionLengthSeconds()) + request.RoleId = roleDef.RoleId // Use STS service to assume the role return m.stsService.AssumeRoleWithWebIdentity(ctx, request) @@ -1287,6 +1537,7 @@ func (m *IAMManager) AssumeRoleWithCredentials(ctx context.Context, request *sts request.DurationSeconds = capDurationByRole(request.DurationSeconds, roleDef.MaxSessionDuration, m.defaultTokenDurationSeconds(), m.maxSessionLengthSeconds()) // Use STS service to assume the role + request.RoleId = roleDef.RoleId return m.stsService.AssumeRoleWithCredentials(ctx, request) } @@ -1300,6 +1551,10 @@ func (m *IAMManager) IsActionAllowed(ctx context.Context, request *ActionRequest // We always try to validate with the internal STS service first if it's a SeaweedFS token. // This ensures that session policies embedded in the token are correctly extracted and enforced. var sessionInfo *sts.SessionInfo + // boundRole is the role a session carrying a role ID was checked against; + // its policies are the ones evaluated, so the check and the evaluation + // see one definition even if the role is replaced in between. + var boundRole *RoleDefinition if request.SessionToken != "" { // Parse unverified to check issuer parsed, _, err := new(jwt.Parser).ParseUnverified(request.SessionToken, jwt.MapClaims{}) @@ -1332,6 +1587,9 @@ func (m *IAMManager) IsActionAllowed(ctx context.Context, request *ActionRequest return false, fmt.Errorf("session has been revoked") } } + if boundRole, err = m.checkSessionRoleBinding(ctx, sessionInfo); err != nil { + return false, err + } } } @@ -1428,9 +1686,12 @@ func (m *IAMManager) IsActionAllowed(ctx context.Context, request *ActionRequest policies = user.GetPolicyNames() } else { // Get role definition - roleDef, err := m.roleStore.GetRole(ctx, m.getFilerAddress(), roleName) - if err != nil { - return false, fmt.Errorf("role not found: %s", roleName) + roleDef := boundRole + if roleDef == nil || roleDef.RoleName != roleName { + roleDef, err = m.roleStore.GetRole(ctx, m.getFilerAddress(), roleName) + if err != nil { + return false, fmt.Errorf("role not found: %s", roleName) + } } hasManagedSubject = true diff --git a/weed/iam/integration/iam_manager_trust.go b/weed/iam/integration/iam_manager_trust.go index 462ec14a7..896ef0a1f 100644 --- a/weed/iam/integration/iam_manager_trust.go +++ b/weed/iam/integration/iam_manager_trust.go @@ -2,16 +2,30 @@ package integration import ( "context" + "errors" "fmt" "github.com/seaweedfs/seaweedfs/weed/iam/policy" "github.com/seaweedfs/seaweedfs/weed/iam/utils" ) +// ErrTrustPolicyDenied is wrapped when a role's trust policy does not admit +// the principal. +var ErrTrustPolicyDenied = errors.New("trust policy denies access to principal") + // ValidateTrustPolicyForPrincipal validates if a principal is allowed to assume a role func (m *IAMManager) ValidateTrustPolicyForPrincipal(ctx context.Context, roleArn, principalArn string) error { + _, err := m.ResolveRoleForPrincipal(ctx, roleArn, principalArn) + return err +} + +// ResolveRoleForPrincipal returns the role roleArn names if its trust policy +// admits principalArn. Issuing a session from the definition returned binds +// the session to the role whose trust was evaluated, not to a role that +// replaced it under the same name in between. +func (m *IAMManager) ResolveRoleForPrincipal(ctx context.Context, roleArn, principalArn string) (*RoleDefinition, error) { if !m.initialized { - return fmt.Errorf("IAM manager not initialized") + return nil, fmt.Errorf("IAM manager not initialized") } // Extract role name from ARN @@ -20,11 +34,11 @@ func (m *IAMManager) ValidateTrustPolicyForPrincipal(ctx context.Context, roleAr // Get role definition roleDef, err := m.roleStore.GetRole(ctx, m.getFilerAddress(), roleName) if err != nil { - return fmt.Errorf("failed to get role %s: %w", roleName, err) + return nil, fmt.Errorf("failed to get role %s: %w", roleName, err) } if roleDef.TrustPolicy == nil { - return fmt.Errorf("role has no trust policy") + return nil, fmt.Errorf("%w: role has no trust policy", ErrTrustPolicyDenied) } // Create evaluation context with RequestContext populated so that @@ -43,8 +57,8 @@ func (m *IAMManager) ValidateTrustPolicyForPrincipal(ctx context.Context, roleAr // Evaluate the trust policy if !m.evaluateTrustPolicy(roleDef.TrustPolicy, evalCtx) { - return fmt.Errorf("trust policy denies access to principal: %s", principalArn) + return nil, fmt.Errorf("%w: %s", ErrTrustPolicyDenied, principalArn) } - return nil + return roleDef, nil } diff --git a/weed/iam/integration/role_id_session_binding_test.go b/weed/iam/integration/role_id_session_binding_test.go new file mode 100644 index 000000000..8ff209d58 --- /dev/null +++ b/weed/iam/integration/role_id_session_binding_test.go @@ -0,0 +1,165 @@ +package integration + +import ( + "context" + "testing" + + "github.com/seaweedfs/seaweedfs/weed/iam/policy" + "github.com/seaweedfs/seaweedfs/weed/iam/sts" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +// A session is bound to the role it was issued for, not to the role's name: +// deleting the role ends it, and a role created later under the same name — +// by whoever may create roles — must not inherit it. +func TestSessionIsBoundToTheRoleNotItsName(t *testing.T) { + ctx := context.Background() + m := setupIntegratedIAMSystem(t) + + assume := func() (token, principal string) { + resp, err := m.AssumeRoleWithWebIdentity(ctx, &sts.AssumeRoleWithWebIdentityRequest{ + RoleArn: "arn:aws:iam::role/S3ReadOnlyRole", + WebIdentityToken: createTestJWT(t, "https://test-issuer.com", "test-user-123", "test-signing-key"), + RoleSessionName: "binding-test", + }) + require.NoError(t, err) + return resp.Credentials.SessionToken, resp.AssumedRoleUser.Arn + } + allowed := func(token, principal string) bool { + ok, _ := m.IsActionAllowed(ctx, &ActionRequest{ + Principal: principal, + Action: "s3:GetObject", + Resource: "arn:aws:s3:::test-bucket/file.txt", + SessionToken: token, + }) + return ok + } + + original, err := m.GetRole(ctx, "S3ReadOnlyRole") + require.NoError(t, err) + require.NotEmpty(t, original.RoleId, "a created role has an ID") + + token, principal := assume() + require.True(t, allowed(token, principal), "precondition: the session works") + + require.NoError(t, m.DeleteRole(ctx, "S3ReadOnlyRole")) + assert.False(t, allowed(token, principal), "the session outlived its role's deletion") + + recreated := *original + recreated.RoleId = "" + require.NoError(t, m.CreateRole(ctx, "", "S3ReadOnlyRole", &recreated)) + assert.NotEqual(t, original.RoleId, recreated.RoleId, "a role created again under the same name got the same ID") + assert.False(t, allowed(token, principal), "a session of the deleted role works again under a new role of the same name") + + fresh, freshPrincipal := assume() + assert.True(t, allowed(fresh, freshPrincipal), "a session issued for the new role is refused") +} + +// A session carries its role's attached policies, and S3 evaluates those +// rather than looking the role up; the binding must hold on that path too. +func TestASessionCarryingItsPoliciesIsStillBoundToItsRole(t *testing.T) { + ctx := context.Background() + m := setupIntegratedIAMSystem(t) + + original, err := m.GetRole(ctx, "S3ReadOnlyRole") + require.NoError(t, err) + require.NotEmpty(t, original.AttachedPolicies, "precondition: the role attaches a policy") + resp, err := m.AssumeRoleWithWebIdentity(ctx, &sts.AssumeRoleWithWebIdentityRequest{ + RoleArn: "arn:aws:iam::role/S3ReadOnlyRole", + WebIdentityToken: createTestJWT(t, "https://test-issuer.com", "test-user-123", "test-signing-key"), + RoleSessionName: "binding-test", + }) + require.NoError(t, err) + allowed := func() bool { + ok, _ := m.IsActionAllowed(ctx, &ActionRequest{ + Principal: resp.AssumedRoleUser.Arn, + Action: "s3:GetObject", + Resource: "arn:aws:s3:::test-bucket/file.txt", + SessionToken: resp.Credentials.SessionToken, + PolicyNames: original.AttachedPolicies, + }) + return ok + } + require.True(t, allowed(), "precondition: the session works") + + require.NoError(t, m.DeleteRole(ctx, "S3ReadOnlyRole")) + assert.False(t, allowed(), "the session's embedded policies outlived its role's deletion") + + recreated := *original + recreated.RoleId = "" + require.NoError(t, m.CreateRole(ctx, "", "S3ReadOnlyRole", &recreated)) + assert.False(t, allowed(), "the session's embedded policies work again under a new role of the same name") +} + +func TestStaticRoleIDIsStableAndRuntimeIDsAreUnique(t *testing.T) { + trusting := func(principal string) *policy.PolicyDocument { + return &policy.PolicyDocument{Version: "2012-10-17", Statement: []policy.Statement{{ + Effect: "Allow", Action: []string{"sts:AssumeRoleWithWebIdentity"}, + Principal: map[string]interface{}{"Federated": principal}, + }}} + } + app := &RoleDefinition{RoleName: "app", TrustPolicy: trusting("https://a.example")} + restarted := &RoleDefinition{RoleName: "app", TrustPolicy: trusting("https://a.example")} + replaced := &RoleDefinition{RoleName: "app", TrustPolicy: trusting("https://b.example")} + + assert.Equal(t, StaticRoleID(app), StaticRoleID(restarted), "a config-file role must keep its ID across restarts") + assert.NotEqual(t, StaticRoleID(app), StaticRoleID(replaced), "a different role under the same name inherits the old one's ID") + assert.NotEqual(t, StaticRoleID(app), StaticRoleID(&RoleDefinition{RoleName: "other", TrustPolicy: trusting("https://a.example")})) + assert.Regexp(t, `^AROA[A-Z0-9]{17}$`, StaticRoleID(app)) + assert.NotEqual(t, NewRoleID(), NewRoleID()) + assert.Regexp(t, `^AROA[A-Z2-7]{17}$`, NewRoleID()) +} + +// replacedAfterFirstReadStore serves the stored role on its first read and a +// replacement of the same name afterwards: a role replaced while a request +// is being authorized. +type replacedAfterFirstReadStore struct { + RoleStore + reads int + replacement *RoleDefinition +} + +func (s *replacedAfterFirstReadStore) GetRole(ctx context.Context, addr, name string) (*RoleDefinition, error) { + s.reads++ + if s.reads > 1 && name == s.replacement.RoleName { + return copyRoleDefinition(s.replacement), nil + } + return s.RoleStore.GetRole(ctx, addr, name) +} + +// The session's binding is checked against one definition of its role, and +// that definition's policies are the ones evaluated: a replacement read in +// between must not lend the session its permissions. +func TestAuthorizationEvaluatesTheRoleTheBindingCheckSaw(t *testing.T) { + ctx := context.Background() + m := setupIntegratedIAMSystem(t) + require.NoError(t, m.CreatePolicy(ctx, "", "S3WritePolicy", &policy.PolicyDocument{ + Version: "2012-10-17", + Statement: []policy.Statement{{ + Effect: "Allow", Action: []string{"s3:PutObject"}, + Resource: []string{"arn:aws:s3:::test-bucket/*"}, + }}, + })) + resp, err := m.AssumeRoleWithWebIdentity(ctx, &sts.AssumeRoleWithWebIdentityRequest{ + RoleArn: "arn:aws:iam::role/S3ReadOnlyRole", + WebIdentityToken: createTestJWT(t, "https://test-issuer.com", "test-user-123", "test-signing-key"), + RoleSessionName: "snapshot-test", + }) + require.NoError(t, err) + + original, err := m.GetRole(ctx, "S3ReadOnlyRole") + require.NoError(t, err) + replacement := *original + replacement.RoleId = NewRoleID() + replacement.AttachedPolicies = []string{"S3WritePolicy"} + m.roleStore = &replacedAfterFirstReadStore{RoleStore: m.roleStore, replacement: &replacement} + + allowed, _ := m.IsActionAllowed(ctx, &ActionRequest{ + Principal: resp.AssumedRoleUser.Arn, + Action: "s3:PutObject", + Resource: "arn:aws:s3:::test-bucket/file.txt", + SessionToken: resp.Credentials.SessionToken, + }) + assert.False(t, allowed, "the session was authorized by the replacement role's policies") +} diff --git a/weed/iam/integration/role_store.go b/weed/iam/integration/role_store.go index c814293f0..b1841224e 100644 --- a/weed/iam/integration/role_store.go +++ b/weed/iam/integration/role_store.go @@ -3,7 +3,10 @@ package integration import ( "context" "encoding/json" + "errors" "fmt" + "io" + "regexp" "strings" "sync" "time" @@ -13,9 +16,38 @@ import ( "github.com/seaweedfs/seaweedfs/weed/iam/policy" "github.com/seaweedfs/seaweedfs/weed/pb" "github.com/seaweedfs/seaweedfs/weed/pb/filer_pb" + "github.com/seaweedfs/seaweedfs/weed/s3api/s3_constants" "google.golang.org/grpc" + "google.golang.org/grpc/codes" + "google.golang.org/grpc/status" ) +// ErrRoleNotFound is wrapped by every RoleStore's GetRole when the role does +// not exist, so callers can tell a missing role from a store that could not +// be read. +var ErrRoleNotFound = errors.New("role not found") + +// ErrRoleStatic refuses a change to a role defined in the server's IAM config +// file: change it there instead. +var ErrRoleStatic = errors.New("role is defined in the IAM config file") + +// ErrRoleExists is returned by an UpdateRole whose update refuses a role that +// already exists (CreateRole). +var ErrRoleExists = errors.New("role already exists") + +// RoleUpdate computes a role's new definition from its current one, nil when +// the role does not exist. It returns nil to delete the role, and an error to +// leave it unchanged. It may run more than once: it is called again with the +// fresh definition when another writer changed the role in between. +type RoleUpdate func(current *RoleDefinition) (*RoleDefinition, error) + +// maxRoleUpdateAttempts bounds UpdateRole's retries under contention. +const maxRoleUpdateAttempts = 10 + +// errRoleUpdateContended is returned when the role kept changing under +// UpdateRole for maxRoleUpdateAttempts reads. +var errRoleUpdateContended = errors.New("role changed concurrently; retry") + // RoleStore defines the interface for storing IAM role definitions type RoleStore interface { // StoreRole stores a role definition (filerAddress ignored for memory stores) @@ -29,6 +61,14 @@ type RoleStore interface { // DeleteRole deletes a role definition (filerAddress ignored for memory stores) DeleteRole(ctx context.Context, filerAddress string, roleName string) error + + // UpdateRole replaces a role with update's result, atomically: the write + // lands only if the role is still as update saw it, absent included, and + // update is retried against the current role otherwise. Every change the + // IAM API and the filer IAM service make to a role goes through it, so + // writers on different servers neither lose each other's changes nor bring + // back a role deleted in between. + UpdateRole(ctx context.Context, filerAddress string, roleName string, update RoleUpdate) error } // MemoryRoleStore implements RoleStore using in-memory storage @@ -72,7 +112,7 @@ func (m *MemoryRoleStore) GetRole(ctx context.Context, filerAddress string, role role, exists := m.roles[roleName] if !exists { - return nil, fmt.Errorf("role not found: %s", roleName) + return nil, fmt.Errorf("%w: %s", ErrRoleNotFound, roleName) } // Return a copy to prevent external modifications @@ -105,6 +145,26 @@ func (m *MemoryRoleStore) DeleteRole(ctx context.Context, filerAddress string, r return nil } +// UpdateRole applies update under the store's lock (filerAddress ignored for +// memory store). +func (m *MemoryRoleStore) UpdateRole(ctx context.Context, filerAddress string, roleName string, update RoleUpdate) error { + if roleName == "" { + return fmt.Errorf("role name cannot be empty") + } + m.mutex.Lock() + defer m.mutex.Unlock() + next, err := update(copyRoleDefinition(m.roles[roleName])) + if err != nil { + return err + } + if next == nil { + delete(m.roles, roleName) + return nil + } + m.roles[roleName] = copyRoleDefinition(next) + return nil +} + // copyRoleDefinition creates a deep copy of a role definition func copyRoleDefinition(original *RoleDefinition) *RoleDefinition { if original == nil { @@ -116,6 +176,9 @@ func copyRoleDefinition(original *RoleDefinition) *RoleDefinition { RoleArn: original.RoleArn, Description: original.Description, MaxSessionDuration: original.MaxSessionDuration, + Source: original.Source, + CreatedAt: original.CreatedAt, + RoleId: original.RoleId, } // Deep copy trust policy if it exists @@ -136,6 +199,10 @@ func copyRoleDefinition(original *RoleDefinition) *RoleDefinition { return copied } +// roleListPageSize is the number of entries FilerRoleStore.ListRoles asks the +// filer for per page. +const roleListPageSize = 1000 + // FilerRoleStore implements RoleStore using SeaweedFS filer type FilerRoleStore struct { grpcDialOption grpc.DialOption @@ -214,6 +281,139 @@ func (f *FilerRoleStore) StoreRole(ctx context.Context, filerAddress string, rol }) } +// UpdateRole reads the role's entry, applies update, and writes the result on +// the condition that the entry is unchanged since the read — absent, when the +// role did not exist — so the filer refuses a write racing another writer's +// change or delete, and update is applied again to what that writer left. A +// delete is made on the same condition, so it removes the role update saw and +// not one written after it. +func (f *FilerRoleStore) UpdateRole(ctx context.Context, filerAddress string, roleName string, update RoleUpdate) error { + if filerAddress == "" && f.filerAddressProvider != nil { + filerAddress = f.filerAddressProvider() + } + if filerAddress == "" { + return fmt.Errorf("filer address is required for FilerRoleStore") + } + if roleName == "" { + return fmt.Errorf("role name cannot be empty") + } + return f.withFilerClient(filerAddress, func(client filer_pb.SeaweedFilerClient) error { + for attempt := 0; attempt < maxRoleUpdateAttempts; attempt++ { + var entry *filer_pb.Entry + var current *RoleDefinition + resp, err := filer_pb.LookupEntry(ctx, client, &filer_pb.LookupDirectoryEntryRequest{ + Directory: f.basePath, + Name: f.getRoleFileName(roleName), + }) + switch { + case errors.Is(err, filer_pb.ErrNotFound): + case err != nil: + return fmt.Errorf("lookup role %s: %w", roleName, err) + case resp.Entry != nil: + entry = resp.Entry + current = &RoleDefinition{} + if err := json.Unmarshal(entry.Content, current); err != nil { + return fmt.Errorf("failed to deserialize role %s: %v", roleName, err) + } + } + + next, err := update(current) + if err != nil { + return err + } + if next == nil { + if entry == nil { + return nil + } + deleted, err := f.deleteRoleEntryIfUnchanged(ctx, client, entry) + if err != nil { + return fmt.Errorf("failed to delete role %s: %w", roleName, err) + } + if !deleted { + glog.V(3).Infof("Role %s changed before its delete; retrying", roleName) + continue + } + return nil + } + roleData, err := json.MarshalIndent(next, "", " ") + if err != nil { + return fmt.Errorf("failed to serialize role: %v", err) + } + + clause := &filer_pb.WriteCondition_Clause{Kind: filer_pb.WriteCondition_IF_NOT_EXISTS} + if entry != nil { + clause = &filer_pb.WriteCondition_Clause{Kind: filer_pb.WriteCondition_IF_ENTRY_EQUAL, ExpectedEntry: entry} + } + now := time.Now().Unix() + created, err := client.CreateEntry(ctx, &filer_pb.CreateEntryRequest{ + Directory: f.basePath, + Entry: &filer_pb.Entry{ + Name: f.getRoleFileName(roleName), + Attributes: &filer_pb.FuseAttributes{ + Mtime: now, + Crtime: now, + FileMode: uint32(0600), + }, + Content: roleData, + }, + Condition: &filer_pb.WriteCondition{Clauses: []*filer_pb.WriteCondition_Clause{clause}}, + }) + if isRoleWriteConflict(created, err) { + glog.V(3).Infof("Role %s changed during update; retrying", roleName) + continue + } + if err != nil { + return fmt.Errorf("failed to store role %s: %v", roleName, err) + } + if created.Error != "" { + return fmt.Errorf("failed to store role %s: %s", roleName, created.Error) + } + return nil + } + return fmt.Errorf("update role %s: %w", roleName, errRoleUpdateContended) + }) +} + +// deleteRoleEntryIfUnchanged deletes the role's entry if it still equals +// entry, reporting false when it changed. The delete is routed and locked as +// the conditional CreateEntry of the same path is, so the two serialize. +func (f *FilerRoleStore) deleteRoleEntryIfUnchanged(ctx context.Context, client filer_pb.SeaweedFilerClient, entry *filer_pb.Entry) (bool, error) { + fullPath := f.basePath + "/" + entry.Name + resp, err := client.ObjectTransaction(ctx, &filer_pb.ObjectTransactionRequest{ + LockKey: fullPath, + RouteKey: s3_constants.ObjectWriteRouteKeyPrefix + fullPath, + Condition: &filer_pb.WriteCondition{Clauses: []*filer_pb.WriteCondition_Clause{{ + Kind: filer_pb.WriteCondition_IF_ENTRY_EQUAL, ExpectedEntry: entry, + }}}, + Mutations: []*filer_pb.ObjectMutation{{ + Type: filer_pb.ObjectMutation_DELETE, Directory: f.basePath, Name: entry.Name, IsDeleteData: true, + }}, + }) + if err != nil { + if status.Code(err) == codes.FailedPrecondition { + return false, nil + } + return false, err + } + if resp.ErrorCode == filer_pb.FilerError_PRECONDITION_FAILED { + return false, nil + } + if resp.Error != "" { + return false, errors.New(resp.Error) + } + return true, nil +} + +// isRoleWriteConflict reports a write the filer refused because its condition +// no longer held: in the response, or as FailedPrecondition when the write +// was forwarded to the entry's owner filer. +func isRoleWriteConflict(resp *filer_pb.CreateEntryResponse, err error) bool { + if err != nil { + return status.Code(err) == codes.FailedPrecondition + } + return resp != nil && resp.ErrorCode == filer_pb.FilerError_PRECONDITION_FAILED +} + // GetRole retrieves a role definition from filer func (f *FilerRoleStore) GetRole(ctx context.Context, filerAddress string, roleName string) (*RoleDefinition, error) { // Use provider function if filerAddress is not provided @@ -235,13 +435,16 @@ func (f *FilerRoleStore) GetRole(ctx context.Context, filerAddress string, roleN } glog.V(3).Infof("Looking up role %s", roleName) - response, err := client.LookupDirectoryEntry(ctx, request) + response, err := filer_pb.LookupEntry(ctx, client, request) + if errors.Is(err, filer_pb.ErrNotFound) { + return fmt.Errorf("%w: %s", ErrRoleNotFound, roleName) + } if err != nil { - return fmt.Errorf("role not found: %v", err) + return fmt.Errorf("lookup role %s: %w", roleName, err) } if response.Entry == nil { - return fmt.Errorf("role not found") + return fmt.Errorf("%w: %s", ErrRoleNotFound, roleName) } roleData = response.Entry.Content @@ -271,46 +474,50 @@ func (f *FilerRoleStore) ListRoles(ctx context.Context, filerAddress string) ([] return nil, fmt.Errorf("filer address is required for FilerRoleStore") } + // Page through the directory, and fail on a broken stream rather than + // return what arrived: a truncated list would let DeletePolicy miss a role + // that still attaches the policy (RolesAttachingPolicy). var roleNames []string - err := f.withFilerClient(filerAddress, func(client filer_pb.SeaweedFilerClient) error { - request := &filer_pb.ListEntriesRequest{ - Directory: f.basePath, - Prefix: "", - StartFromFileName: "", - InclusiveStartFrom: false, - Limit: 1000, // Process in batches of 1000 - } - glog.V(3).Infof("Listing roles in %s", f.basePath) - stream, err := client.ListEntries(ctx, request) - if err != nil { - return fmt.Errorf("failed to list roles: %v", err) - } - + startFrom := "" for { - resp, err := stream.Recv() + stream, err := client.ListEntries(ctx, &filer_pb.ListEntriesRequest{ + Directory: f.basePath, + StartFromFileName: startFrom, + Limit: roleListPageSize, + }) if err != nil { - break // End of stream or error + return err } - - if resp.Entry == nil || resp.Entry.IsDirectory { - continue + received := 0 + for { + resp, err := stream.Recv() + if errors.Is(err, io.EOF) { + break + } + if err != nil { + return err + } + if resp.Entry == nil { + continue + } + received++ + startFrom = resp.Entry.Name + if resp.Entry.IsDirectory { + continue + } + if name, ok := strings.CutSuffix(resp.Entry.Name, ".json"); ok { + roleNames = append(roleNames, name) + } } - - // Extract role name from filename - filename := resp.Entry.Name - if strings.HasSuffix(filename, ".json") { - roleName := strings.TrimSuffix(filename, ".json") - roleNames = append(roleNames, roleName) + if received < roleListPageSize { + return nil } } - - return nil }) - if err != nil { - return nil, err + return nil, fmt.Errorf("failed to list roles: %w", err) } return roleNames, nil @@ -389,3 +596,146 @@ type CachedFilerRoleStoreConfig struct { ListTTL string `json:"listTtl,omitempty"` // e.g., "1m", "30s" MaxCacheSize int `json:"maxCacheSize,omitempty"` // Maximum number of cached roles } + +// RolesAttachingPolicy returns the names of the roles that attach the policy. +// A policy is attached to a role by name, so deleting it while attached would +// let a policy created later under the same name take effect on the role; +// callers refuse the delete instead, as AWS does (DeleteConflict). +// +// It sees the stored roles, which every server sharing the store sees, and +// this server's config-file roles. A role defined only in another server's +// config file is invisible here: servers sharing a role store may have +// different config files, but a config-file role that attaches a managed +// (API-created) policy is protected only on the servers whose file defines it. +// Keep such roles in every server's file, or attach only config-file policies +// to config-file roles. +func RolesAttachingPolicy(ctx context.Context, store RoleStore, policyName string) ([]string, error) { + names, err := store.ListRoles(ctx, "") + if err != nil { + return nil, fmt.Errorf("list roles: %w", err) + } + var attaching []string + for _, name := range names { + role, err := store.GetRole(ctx, "", name) + if errors.Is(err, ErrRoleNotFound) { + continue // deleted between list and read + } + if err != nil { + return nil, fmt.Errorf("get role %s: %w", name, err) + } + for _, attached := range role.AttachedPolicies { + if attached == policyName { + attaching = append(attaching, name) + break + } + } + } + return attaching, nil +} + +// MaxManagedPoliciesPerRole caps the managed policies attached to one role, +// as AWS's default quota does (and as MaxManagedPoliciesPerUser does for users). +const MaxManagedPoliciesPerRole = 10 + +var roleNamePattern = regexp.MustCompile(`^[\w+=,.@-]{1,64}$`) + +// ValidateRoleName checks a role name against AWS's rules. A role is stored as +// .json in the filer, so the rules also keep a name from leaving the +// role store's directory. +func ValidateRoleName(name string) error { + if !roleNamePattern.MatchString(name) { + return fmt.Errorf("invalid role name %q: must be 1-64 characters of letters, digits and +=,.@_-", name) + } + return nil +} + +// staticRoleOverlay serves the IAM config file's roles from memory beside a +// persistent store, which never holds them (see IAMManager.LoadStaticRoles). +// A role stored under the same name takes precedence, as a stored OIDC +// provider does over a config-file one; deleting it brings the config-file +// role back. A config-file role itself cannot be stored over or deleted. +type staticRoleOverlay struct { + static map[string]*RoleDefinition + inner RoleStore +} + +// storedRoleExists reports whether the store itself holds the role. +func (o *staticRoleOverlay) storedRoleExists(ctx context.Context, filerAddress, roleName string) (bool, error) { + _, err := o.inner.GetRole(ctx, filerAddress, roleName) + if errors.Is(err, ErrRoleNotFound) { + return false, nil + } + return err == nil, err +} + +func (o *staticRoleOverlay) StoreRole(ctx context.Context, filerAddress string, roleName string, role *RoleDefinition) error { + if _, ok := o.static[roleName]; ok { + stored, err := o.storedRoleExists(ctx, filerAddress, roleName) + if err != nil { + return err + } + if !stored { + return fmt.Errorf("%w: %s", ErrRoleStatic, roleName) + } + } + return o.inner.StoreRole(ctx, filerAddress, roleName, role) +} + +// UpdateRole refuses to create a stored role over a config-file role, as +// StoreRole does; a stored role of that name may be changed. +func (o *staticRoleOverlay) UpdateRole(ctx context.Context, filerAddress string, roleName string, update RoleUpdate) error { + _, static := o.static[roleName] + return o.inner.UpdateRole(ctx, filerAddress, roleName, func(current *RoleDefinition) (*RoleDefinition, error) { + if current == nil && static { + return nil, fmt.Errorf("%w: %s", ErrRoleStatic, roleName) + } + return update(current) + }) +} + +func (o *staticRoleOverlay) GetRole(ctx context.Context, filerAddress string, roleName string) (*RoleDefinition, error) { + role, err := o.inner.GetRole(ctx, filerAddress, roleName) + if errors.Is(err, ErrRoleNotFound) { + if static, ok := o.static[roleName]; ok { + return copyRoleDefinition(static), nil + } + } + return role, err +} + +func (o *staticRoleOverlay) ListRoles(ctx context.Context, filerAddress string) ([]string, error) { + names, err := o.inner.ListRoles(ctx, filerAddress) + if err != nil { + return nil, err + } + seen := make(map[string]bool, len(names)) + for _, name := range names { + seen[name] = true + } + for name := range o.static { + if !seen[name] { + names = append(names, name) + } + } + return names, nil +} + +func (o *staticRoleOverlay) DeleteRole(ctx context.Context, filerAddress string, roleName string) error { + if _, ok := o.static[roleName]; ok { + stored, err := o.storedRoleExists(ctx, filerAddress, roleName) + if err != nil { + return err + } + if !stored { + return fmt.Errorf("%w: %s", ErrRoleStatic, roleName) + } + } + return o.inner.DeleteRole(ctx, filerAddress, roleName) +} + +// ClearCache forwards cache invalidation to the store underneath. +func (o *staticRoleOverlay) ClearCache() { + if cached, ok := o.inner.(interface{ ClearCache() }); ok { + cached.ClearCache() + } +} diff --git a/weed/iam/integration/role_store_filer_test.go b/weed/iam/integration/role_store_filer_test.go new file mode 100644 index 000000000..193b52edd --- /dev/null +++ b/weed/iam/integration/role_store_filer_test.go @@ -0,0 +1,286 @@ +package integration + +import ( + "context" + "errors" + "fmt" + "net" + "slices" + "sort" + "strconv" + "sync" + "testing" + + "github.com/seaweedfs/seaweedfs/weed/pb" + "github.com/seaweedfs/seaweedfs/weed/pb/filer_pb" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "google.golang.org/grpc" + "google.golang.org/grpc/codes" + "google.golang.org/grpc/credentials/insecure" + "google.golang.org/grpc/status" + "google.golang.org/protobuf/proto" +) + +// roleStoreTestFiler is a filer holding one directory. It evaluates the write +// conditions FilerRoleStore sends as the filer does, and can run another +// writer between a role's read and its write (afterLookup) or break a listing +// stream partway (failListAfter). +type roleStoreTestFiler struct { + filer_pb.UnimplementedSeaweedFilerServer + mu sync.Mutex + entries map[string]*filer_pb.Entry + afterLookup func() + failListAfter int +} + +func (s *roleStoreTestFiler) LookupDirectoryEntry(_ context.Context, req *filer_pb.LookupDirectoryEntryRequest) (*filer_pb.LookupDirectoryEntryResponse, error) { + s.mu.Lock() + entry, found := s.entries[req.Name] + hook := s.afterLookup + s.afterLookup = nil + s.mu.Unlock() + if hook != nil { + defer hook() + } + if !found { + return nil, status.Error(codes.NotFound, filer_pb.ErrNotFound.Error()) + } + return &filer_pb.LookupDirectoryEntryResponse{Entry: proto.Clone(entry).(*filer_pb.Entry)}, nil +} + +func (s *roleStoreTestFiler) CreateEntry(_ context.Context, req *filer_pb.CreateEntryRequest) (*filer_pb.CreateEntryResponse, error) { + s.mu.Lock() + defer s.mu.Unlock() + current, exists := s.entries[req.Entry.Name] + for _, c := range req.GetCondition().GetClauses() { + var ok bool + switch c.Kind { + case filer_pb.WriteCondition_IF_NOT_EXISTS: + ok = !exists + case filer_pb.WriteCondition_IF_ENTRY_EQUAL: + ok = exists && proto.Equal(current, c.ExpectedEntry) + default: + return nil, fmt.Errorf("unexpected condition %v", c.Kind) + } + if !ok { + return &filer_pb.CreateEntryResponse{Error: "precondition failed", ErrorCode: filer_pb.FilerError_PRECONDITION_FAILED}, nil + } + } + s.entries[req.Entry.Name] = proto.Clone(req.Entry).(*filer_pb.Entry) + return &filer_pb.CreateEntryResponse{}, nil +} + +// ObjectTransaction applies the conditional delete FilerRoleStore sends. +func (s *roleStoreTestFiler) ObjectTransaction(_ context.Context, req *filer_pb.ObjectTransactionRequest) (*filer_pb.ObjectTransactionResponse, error) { + s.mu.Lock() + defer s.mu.Unlock() + if len(req.Mutations) != 1 || req.Mutations[0].Type != filer_pb.ObjectMutation_DELETE { + return nil, fmt.Errorf("unexpected mutations %v", req.Mutations) + } + name := req.Mutations[0].Name + current, exists := s.entries[name] + for _, c := range req.GetCondition().GetClauses() { + if c.Kind != filer_pb.WriteCondition_IF_ENTRY_EQUAL { + return nil, fmt.Errorf("unexpected condition %v", c.Kind) + } + if !exists || !proto.Equal(current, c.ExpectedEntry) { + return &filer_pb.ObjectTransactionResponse{Error: "precondition failed", ErrorCode: filer_pb.FilerError_PRECONDITION_FAILED}, nil + } + } + delete(s.entries, name) + return &filer_pb.ObjectTransactionResponse{}, nil +} + +func (s *roleStoreTestFiler) ListEntries(req *filer_pb.ListEntriesRequest, stream grpc.ServerStreamingServer[filer_pb.ListEntriesResponse]) error { + s.mu.Lock() + var names []string + for name := range s.entries { + if name > req.StartFromFileName { + names = append(names, name) + } + } + sort.Strings(names) + if req.Limit > 0 && len(names) > int(req.Limit) { + names = names[:req.Limit] + } + page := make([]*filer_pb.Entry, 0, len(names)) + for _, name := range names { + page = append(page, proto.Clone(s.entries[name]).(*filer_pb.Entry)) + } + failAfter := s.failListAfter + s.mu.Unlock() + for i, entry := range page { + if failAfter > 0 && i == failAfter { + return status.Error(codes.Unavailable, "filer went away") + } + if err := stream.Send(&filer_pb.ListEntriesResponse{Entry: entry}); err != nil { + return err + } + } + return nil +} + +func (s *roleStoreTestFiler) DeleteEntry(_ context.Context, req *filer_pb.DeleteEntryRequest) (*filer_pb.DeleteEntryResponse, error) { + s.mu.Lock() + defer s.mu.Unlock() + delete(s.entries, req.Name) + return &filer_pb.DeleteEntryResponse{}, nil +} + +func newTestFilerRoleStore(t *testing.T) (*FilerRoleStore, *roleStoreTestFiler) { + t.Helper() + lis, err := net.Listen("tcp", "127.0.0.1:0") + require.NoError(t, err) + filer := &roleStoreTestFiler{entries: map[string]*filer_pb.Entry{}} + server := pb.NewGrpcServer() + filer_pb.RegisterSeaweedFilerServer(server, filer) + go func() { _ = server.Serve(lis) }() + t.Cleanup(func() { + server.Stop() + _ = lis.Close() + }) + host, port, err := net.SplitHostPort(lis.Addr().String()) + require.NoError(t, err) + grpcPort, err := strconv.Atoi(port) + require.NoError(t, err) + store, err := NewFilerRoleStore(nil, func() string { return string(pb.NewServerAddress(host, 1, grpcPort)) }) + require.NoError(t, err) + store.grpcDialOption = grpc.WithTransportCredentials(insecure.NewCredentials()) + return store, filer +} + +func attachPolicy(policyName string) RoleUpdate { + return func(current *RoleDefinition) (*RoleDefinition, error) { + if current == nil { + return nil, ErrRoleNotFound + } + current.AttachedPolicies = append(current.AttachedPolicies, policyName) + return current, nil + } +} + +func createRole(roleID string) RoleUpdate { + return func(current *RoleDefinition) (*RoleDefinition, error) { + if current != nil { + return nil, ErrRoleExists + } + return &RoleDefinition{RoleName: "app", RoleId: roleID}, nil + } +} + +// Two servers changing one role: the change written second is applied to +// what the first left, not to the role as it read it. +func TestFilerRoleUpdateIsNotLostToAConcurrentChange(t *testing.T) { + ctx := context.Background() + store, filer := newTestFilerRoleStore(t) + require.NoError(t, store.UpdateRole(ctx, "", "app", createRole("AROA1"))) + + filer.afterLookup = func() { assert.NoError(t, store.UpdateRole(ctx, "", "app", attachPolicy("peer"))) } + require.NoError(t, store.UpdateRole(ctx, "", "app", attachPolicy("mine"))) + + role, err := store.GetRole(ctx, "", "app") + require.NoError(t, err) + assert.ElementsMatch(t, []string{"peer", "mine"}, role.AttachedPolicies, "one server's change was lost") +} + +// A change racing a delete must not write the role back. +func TestFilerRoleUpdateDoesNotReviveADeletedRole(t *testing.T) { + ctx := context.Background() + store, filer := newTestFilerRoleStore(t) + require.NoError(t, store.UpdateRole(ctx, "", "app", createRole("AROA1"))) + + filer.afterLookup = func() { assert.NoError(t, store.DeleteRole(ctx, "", "app")) } + err := store.UpdateRole(ctx, "", "app", attachPolicy("mine")) + assert.ErrorIs(t, err, ErrRoleNotFound) + + _, err = store.GetRole(ctx, "", "app") + assert.ErrorIs(t, err, ErrRoleNotFound, "the deleted role was written back") +} + +// Of two creates of one name, the second sees the first's role. +func TestFilerRoleCreateRefusesARoleCreatedConcurrently(t *testing.T) { + ctx := context.Background() + store, filer := newTestFilerRoleStore(t) + + filer.afterLookup = func() { assert.NoError(t, store.UpdateRole(ctx, "", "app", createRole("AROA-FIRST"))) } + err := store.UpdateRole(ctx, "", "app", createRole("AROA-SECOND")) + assert.ErrorIs(t, err, ErrRoleExists) + + role, err := store.GetRole(ctx, "", "app") + require.NoError(t, err) + assert.Equal(t, "AROA-FIRST", role.RoleId, "the second create replaced the first role") +} + +var errRoleHasPolicies = errors.New("role has attached policies") + +// deleteUnattached deletes the role only if it attaches no policy, as +// DeleteRole requires. +func deleteUnattached(current *RoleDefinition) (*RoleDefinition, error) { + if current == nil { + return nil, ErrRoleNotFound + } + if len(current.AttachedPolicies) > 0 { + return nil, errRoleHasPolicies + } + return nil, nil +} + +func TestFilerRoleUpdateCanDeleteTheRole(t *testing.T) { + ctx := context.Background() + store, _ := newTestFilerRoleStore(t) + require.NoError(t, store.UpdateRole(ctx, "", "app", createRole("AROA1"))) + + require.NoError(t, store.UpdateRole(ctx, "", "app", deleteUnattached)) + _, err := store.GetRole(ctx, "", "app") + assert.ErrorIs(t, err, ErrRoleNotFound) +} + +// A delete is decided against the role it removes: a policy attached between +// the check and the delete makes it refuse, not delete the role anyway. +func TestFilerRoleDeleteIsDecidedAgainstTheRoleItRemoves(t *testing.T) { + ctx := context.Background() + store, filer := newTestFilerRoleStore(t) + require.NoError(t, store.UpdateRole(ctx, "", "app", createRole("AROA1"))) + + filer.afterLookup = func() { assert.NoError(t, store.UpdateRole(ctx, "", "app", attachPolicy("peer"))) } + err := store.UpdateRole(ctx, "", "app", deleteUnattached) + assert.ErrorIs(t, err, errRoleHasPolicies) + + role, err := store.GetRole(ctx, "", "app") + require.NoError(t, err, "the role was deleted although a policy was attached to it") + assert.Equal(t, []string{"peer"}, role.AttachedPolicies) +} + +func TestFilerRoleListingPagesPastTheFirstThousand(t *testing.T) { + store, filer := newTestFilerRoleStore(t) + for i := range roleListPageSize + 1 { + name := fmt.Sprintf("role-%04d.json", i) + filer.entries[name] = &filer_pb.Entry{Name: name} + } + filer.entries["role-0500"] = &filer_pb.Entry{Name: "role-0500", IsDirectory: true} + + names, err := store.ListRoles(context.Background(), "") + require.NoError(t, err) + assert.Len(t, names, roleListPageSize+1) + assert.True(t, slices.Contains(names, fmt.Sprintf("role-%04d", roleListPageSize)), "the role past the first page is missing") +} + +// A listing cut short must fail: DeletePolicy decides from it whether any +// role still attaches the policy. +func TestFilerRoleListingFailsOnABrokenStream(t *testing.T) { + store, filer := newTestFilerRoleStore(t) + for i := range 5 { + name := fmt.Sprintf("role-%d.json", i) + filer.entries[name] = &filer_pb.Entry{Name: name} + } + filer.failListAfter = 3 + + _, err := store.ListRoles(context.Background(), "") + require.Error(t, err, "a partial listing was returned as complete") + + attaching, err := RolesAttachingPolicy(context.Background(), store, "read") + assert.Error(t, err) + assert.Nil(t, attaching) + assert.False(t, errors.Is(err, ErrRoleNotFound)) +} diff --git a/weed/iam/integration/role_store_notfound_test.go b/weed/iam/integration/role_store_notfound_test.go new file mode 100644 index 000000000..02bf02bed --- /dev/null +++ b/weed/iam/integration/role_store_notfound_test.go @@ -0,0 +1,14 @@ +package integration + +import ( + "context" + "errors" + "testing" +) + +func TestMemoryRoleStoreWrapsErrRoleNotFound(t *testing.T) { + _, err := NewMemoryRoleStore().GetRole(context.Background(), "", "missing") + if !errors.Is(err, ErrRoleNotFound) { + t.Fatalf("missing role error does not wrap ErrRoleNotFound: %v", err) + } +} diff --git a/weed/iam/integration/role_store_persist_test.go b/weed/iam/integration/role_store_persist_test.go new file mode 100644 index 000000000..8293dfc3e --- /dev/null +++ b/weed/iam/integration/role_store_persist_test.go @@ -0,0 +1,196 @@ +package integration + +import ( + "context" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +// persistentTestRoleStore stands in for a role store that outlives the process +// and may be shared (the filer store): anything but *MemoryRoleStore is +// treated as persistent. +type persistentTestRoleStore struct { + *MemoryRoleStore + cleared int +} + +func (s *persistentTestRoleStore) ClearCache() { s.cleared++ } + +// startRoleServer initializes a manager as an S3 server would at boot, with the +// given role store and config file roles. +func startRoleServer(t *testing.T, store RoleStore, staticRoles ...string) *IAMManager { + t.Helper() + mgr := NewIAMManager() + require.NoError(t, mgr.Initialize(persistTestConfig(), func() string { return "localhost:8888" })) + mgr.roleStore = store + var defs []*RoleDefinition + for _, name := range staticRoles { + defs = append(defs, &RoleDefinition{RoleName: name}) + } + mgr.LoadStaticRoles(context.Background(), defs) + return mgr +} + +func roleNames(t *testing.T, store RoleStore) map[string]bool { + t.Helper() + names, err := store.ListRoles(context.Background(), "") + require.NoError(t, err) + out := map[string]bool{} + for _, n := range names { + out[n] = true + } + return out +} + +// A persistent store never receives the config file's roles: it outlives the +// file and may be shared with servers whose files differ. +func TestPersistentRoleStoreNeverHoldsConfigFileRoles(t *testing.T) { + store := &persistentTestRoleStore{MemoryRoleStore: NewMemoryRoleStore()} + mgr := startRoleServer(t, store, "from-file") + + assert.Empty(t, roleNames(t, store), "a config-file role was written to the persistent store") + assert.True(t, roleNames(t, mgr.GetRoleStore())["from-file"], "the config-file role is not listed") + role, err := mgr.GetRole(context.Background(), "from-file") + require.NoError(t, err) + assert.Equal(t, RoleSourceStaticConfig, role.Source) + assert.Equal(t, StaticRoleID(&RoleDefinition{RoleName: "from-file"}), role.RoleId) +} + +// Servers sharing a store, with different config files, must not remove or +// replace each other's roles — including a zero-config server. +func TestServersSharingARoleStoreKeepEachOthersRoles(t *testing.T) { + store := &persistentTestRoleStore{MemoryRoleStore: NewMemoryRoleStore()} + configured := startRoleServer(t, store, "from-file") + require.NoError(t, configured.CreateRole(context.Background(), "", "created-at-runtime", &RoleDefinition{})) + + zeroConfig := startRoleServer(t, store) + + assert.True(t, roleNames(t, store)["created-at-runtime"], "a peer's start removed a role created at runtime") + assert.True(t, roleNames(t, configured.GetRoleStore())["from-file"], "the configured server lost its config-file role") + _, err := zeroConfig.GetRole(context.Background(), "from-file") + assert.ErrorIs(t, err, ErrRoleNotFound, "a server sees a role only its peer's config file defines") + _, err = zeroConfig.GetRole(context.Background(), "created-at-runtime") + assert.NoError(t, err, "the peer does not see the shared role created at runtime") +} + +// Removing a role from the config file removes it at the next start. +func TestRemovingARoleFromTheConfigFileRemovesIt(t *testing.T) { + store := &persistentTestRoleStore{MemoryRoleStore: NewMemoryRoleStore()} + startRoleServer(t, store, "from-file") + + restarted := startRoleServer(t, store) // the file no longer lists it + + _, err := restarted.GetRole(context.Background(), "from-file") + assert.ErrorIs(t, err, ErrRoleNotFound, "a role removed from the config file is still served") +} + +// The store refuses to replace or delete a role the config file defines; the +// file is where it changes. +func TestConfigFileRolesCannotBeReplacedOrDeletedThroughTheStore(t *testing.T) { + store := &persistentTestRoleStore{MemoryRoleStore: NewMemoryRoleStore()} + mgr := startRoleServer(t, store, "from-file") + + err := mgr.CreateRole(context.Background(), "", "from-file", &RoleDefinition{}) + assert.ErrorIs(t, err, ErrRoleStatic) + assert.ErrorIs(t, mgr.DeleteRole(context.Background(), "from-file"), ErrRoleStatic) + assert.Empty(t, roleNames(t, store), "a refused change still wrote to the store") +} + +// An in-memory store keeps its behaviour: the config file's roles are records +// in it. +func TestInMemoryRoleStoreStillHoldsConfigFileRoles(t *testing.T) { + store := NewMemoryRoleStore() + startRoleServer(t, store, "from-file") + assert.True(t, roleNames(t, store)["from-file"]) +} + +// Invalidating the role cache reaches the store beneath the config-file roles. +func TestRoleCacheInvalidationReachesTheStoreBeneathTheOverlay(t *testing.T) { + store := &persistentTestRoleStore{MemoryRoleStore: NewMemoryRoleStore()} + mgr := startRoleServer(t, store, "from-file") + mgr.InvalidateRoleCache() + assert.Equal(t, 1, store.cleared) +} + +func TestRoleCopiesKeepSource(t *testing.T) { + role := &RoleDefinition{RoleName: "r", Source: RoleSourceStaticConfig} + assert.Equal(t, RoleSourceStaticConfig, copyRoleDefinition(role).Source) + assert.Equal(t, RoleSourceStaticConfig, genericCopyRoleDefinition(role).Source) +} + +// Config-file roles report no creation time: a time taken at load would +// change with every restart. +func TestConfigFileRolesReportNoCreationTime(t *testing.T) { + for name, store := range map[string]RoleStore{ + "persistent": &persistentTestRoleStore{MemoryRoleStore: NewMemoryRoleStore()}, + "in-memory": NewMemoryRoleStore(), + } { + t.Run(name, func(t *testing.T) { + mgr := startRoleServer(t, store, "from-file") + role, err := mgr.GetRole(context.Background(), "from-file") + require.NoError(t, err) + assert.True(t, role.CreatedAt.IsZero()) + }) + } +} + +// A role stored under a config-file role's name takes precedence, as a stored +// OIDC provider does; it can be changed, and deleting it brings the +// config-file role back. +func TestStoredRoleTakesPrecedenceOverTheConfigFileOne(t *testing.T) { + ctx := context.Background() + store := &persistentTestRoleStore{MemoryRoleStore: NewMemoryRoleStore()} + configured := startRoleServer(t, store, "shared-name") + peer := startRoleServer(t, store) + require.NoError(t, peer.CreateRole(ctx, "", "shared-name", &RoleDefinition{Description: "stored"})) + + role, err := configured.GetRole(ctx, "shared-name") + require.NoError(t, err) + assert.Equal(t, "stored", role.Description, "the config-file role hides the stored one") + + role.Description = "changed" + assert.NoError(t, configured.CreateRole(ctx, "", "shared-name", role), "the stored role cannot be changed") + + require.NoError(t, configured.DeleteRole(ctx, "shared-name")) + role, err = configured.GetRole(ctx, "shared-name") + require.NoError(t, err) + assert.Equal(t, RoleSourceStaticConfig, role.Source, "deleting the stored role did not bring the config-file one back") +} + +// A store installed through SetRoleStore behaves like one installed at +// startup: the config-file roles stay visible and protected, and it serves +// the roles it holds. +func TestSetRoleStoreInstallsLikeStartup(t *testing.T) { + mgr := startRoleServer(t, &persistentTestRoleStore{MemoryRoleStore: NewMemoryRoleStore()}, "from-file") + + store := &persistentTestRoleStore{MemoryRoleStore: NewMemoryRoleStore()} + require.NoError(t, store.StoreRole(context.Background(), "", "stored", &RoleDefinition{RoleName: "stored"})) + mgr.SetRoleStore(store) + + names := roleNames(t, mgr.GetRoleStore()) + assert.True(t, names["from-file"], "the config-file role is no longer listed") + assert.True(t, names["stored"], "the new store's role is not listed") + assert.ErrorIs(t, mgr.GetRoleStore().DeleteRole(context.Background(), "", "from-file"), ErrRoleStatic) + assert.False(t, roleNames(t, store)["from-file"], "the config-file role was written into the new store") +} + +// S3 servers watch the directory a filer-backed role store keeps roles in, so +// a store configured with its own basePath must report it, through the +// cache and the config-file overlay alike. +func TestRoleStoreDirectoryIsTheConfiguredBasePath(t *testing.T) { + provider := func() string { return "localhost:8888" } + cached, err := NewGenericCachedRoleStore(map[string]interface{}{"basePath": "/custom/roles"}, provider) + require.NoError(t, err) + mgr := startRoleServer(t, cached, "from-file") + assert.Equal(t, "/custom/roles", mgr.RoleStoreDirectory()) + + uncached, err := NewFilerRoleStore(nil, provider) + require.NoError(t, err) + mgr.SetRoleStore(uncached) + assert.Equal(t, "/etc/iam/roles", mgr.RoleStoreDirectory()) + + mgr.SetRoleStore(NewMemoryRoleStore()) + assert.Empty(t, mgr.RoleStoreDirectory(), "a memory store has no directory") +} diff --git a/weed/iam/responses.go b/weed/iam/responses.go index 45813f402..0bd0660c0 100644 --- a/weed/iam/responses.go +++ b/weed/iam/responses.go @@ -532,3 +532,83 @@ type ListUserTagsResponse struct { } `xml:"ListUserTagsResult"` CommonResponse } + +// IAMRole is the Role element returned by the role actions. +type IAMRole struct { + Path string `xml:"Path"` + RoleName string `xml:"RoleName"` + RoleId string `xml:"RoleId"` + Arn string `xml:"Arn"` + CreateDate string `xml:"CreateDate,omitempty"` + AssumeRolePolicyDocument string `xml:"AssumeRolePolicyDocument,omitempty"` + Description string `xml:"Description,omitempty"` + MaxSessionDuration int64 `xml:"MaxSessionDuration,omitempty"` +} + +// IAMAttachedPolicy is one element of ListAttachedRolePolicies. +type IAMAttachedPolicy struct { + PolicyName string `xml:"PolicyName"` + PolicyArn string `xml:"PolicyArn"` +} + +// CreateRoleResponse is the response for CreateRole. +type CreateRoleResponse struct { + XMLName xml.Name `xml:"https://iam.amazonaws.com/doc/2010-05-08/ CreateRoleResponse"` + CreateRoleResult struct { + Role IAMRole `xml:"Role"` + } `xml:"CreateRoleResult"` + CommonResponse +} + +// GetRoleResponse is the response for GetRole. +type GetRoleResponse struct { + XMLName xml.Name `xml:"https://iam.amazonaws.com/doc/2010-05-08/ GetRoleResponse"` + GetRoleResult struct { + Role IAMRole `xml:"Role"` + } `xml:"GetRoleResult"` + CommonResponse +} + +// ListRolesResponse is the response for ListRoles. +type ListRolesResponse struct { + XMLName xml.Name `xml:"https://iam.amazonaws.com/doc/2010-05-08/ ListRolesResponse"` + ListRolesResult struct { + Roles []*IAMRole `xml:"Roles>member"` + IsTruncated bool `xml:"IsTruncated"` + } `xml:"ListRolesResult"` + CommonResponse +} + +// DeleteRoleResponse is the response for DeleteRole. +type DeleteRoleResponse struct { + XMLName xml.Name `xml:"https://iam.amazonaws.com/doc/2010-05-08/ DeleteRoleResponse"` + CommonResponse +} + +// UpdateAssumeRolePolicyResponse is the response for UpdateAssumeRolePolicy. +type UpdateAssumeRolePolicyResponse struct { + XMLName xml.Name `xml:"https://iam.amazonaws.com/doc/2010-05-08/ UpdateAssumeRolePolicyResponse"` + CommonResponse +} + +// AttachRolePolicyResponse is the response for AttachRolePolicy. +type AttachRolePolicyResponse struct { + XMLName xml.Name `xml:"https://iam.amazonaws.com/doc/2010-05-08/ AttachRolePolicyResponse"` + CommonResponse +} + +// DetachRolePolicyResponse is the response for DetachRolePolicy. +type DetachRolePolicyResponse struct { + XMLName xml.Name `xml:"https://iam.amazonaws.com/doc/2010-05-08/ DetachRolePolicyResponse"` + CommonResponse +} + +// ListAttachedRolePoliciesResponse is the response for ListAttachedRolePolicies. +type ListAttachedRolePoliciesResponse struct { + XMLName xml.Name `xml:"https://iam.amazonaws.com/doc/2010-05-08/ ListAttachedRolePoliciesResponse"` + ListAttachedRolePoliciesResult struct { + AttachedPolicies []*IAMAttachedPolicy `xml:"AttachedPolicies>member"` + IsTruncated bool `xml:"IsTruncated"` + } `xml:"ListAttachedRolePoliciesResult"` + CommonResponse +} diff --git a/weed/iam/sts/session_claims.go b/weed/iam/sts/session_claims.go index e1449e52d..376dcd911 100644 --- a/weed/iam/sts/session_claims.go +++ b/weed/iam/sts/session_claims.go @@ -65,9 +65,10 @@ type STSSessionClaims struct { TokenType string `json:"typ"` // token_type // Role information - RoleArn string `json:"role"` // role_arn - AssumedRole string `json:"assumed"` // assumed_role_user - Principal string `json:"principal"` // principal_arn + RoleArn string `json:"role"` // role_arn + RoleId string `json:"rid,omitempty"` // unique ID of the assumed role + AssumedRole string `json:"assumed"` // assumed_role_user + Principal string `json:"principal"` // principal_arn // Authorization data Policies []string `json:"pol,omitempty"` // policies (abbreviated) @@ -136,6 +137,7 @@ func (c *STSSessionClaims) ToSessionInfo(credGen *CredentialGenerator) *SessionI SessionId: c.SessionId, SessionName: c.SessionName, RoleArn: c.RoleArn, + RoleId: c.RoleId, AssumedRoleUser: c.AssumedRole, Principal: c.Principal, Policies: c.Policies, @@ -196,6 +198,13 @@ func (c *STSSessionClaims) WithRoleInfo(roleArn, assumedRole, principal string) } // WithPolicies sets the policies associated with this session +// WithRoleId binds the session to the assumed role's unique ID. An empty ID +// leaves the session unbound (roles that predate role IDs). +func (c *STSSessionClaims) WithRoleId(roleId string) *STSSessionClaims { + c.RoleId = roleId + return c +} + func (c *STSSessionClaims) WithPolicies(policies []string) *STSSessionClaims { c.Policies = policies return c diff --git a/weed/iam/sts/sts_service.go b/weed/iam/sts/sts_service.go index ff9cc53dd..89d09919a 100644 --- a/weed/iam/sts/sts_service.go +++ b/weed/iam/sts/sts_service.go @@ -187,6 +187,11 @@ type AssumeRoleWithWebIdentityRequest struct { // Policy is an optional session policy (optional) Policy *string `json:"Policy,omitempty"` + + // RoleId is the assumed role's unique ID, set by the IAM manager after it + // resolves the role. It is embedded in the session so the session is bound + // to this role and not to whatever role later holds the same name. + RoleId string `json:"-"` } // AssumeRoleWithCredentialsRequest represents a request to assume role with username/password @@ -211,6 +216,9 @@ type AssumeRoleWithCredentialsRequest struct { // Policy is an optional session policy (optional) Policy *string `json:"Policy,omitempty"` + + // RoleId is set by the IAM manager; see AssumeRoleWithWebIdentityRequest. + RoleId string `json:"-"` } // AssumeRoleResponse represents the response from assume role operations @@ -304,6 +312,10 @@ type SessionInfo struct { // ParentUser is the stable hashed identity (sub+iss) derived at federation time. ParentUser string `json:"parentUser,omitempty"` + + // RoleId is the unique ID of the role the session was issued for; empty + // for sessions issued before role IDs were recorded. + RoleId string `json:"roleId,omitempty"` } // NewSTSService creates a new STS service @@ -688,6 +700,7 @@ func (s *STSService) AssumeRoleWithWebIdentity(ctx context.Context, request *Ass sessionClaims := NewSTSSessionClaims(sessionId, s.Config.Issuer, expiresAt). WithSessionName(request.RoleSessionName). WithRoleInfo(effectiveRoleArn, assumedRoleUser.Arn, assumedRoleUser.Arn). + WithRoleId(request.RoleId). WithIdentityProvider(provider.Name(), externalIdentity.UserID, externalIdentity.Issuer). WithMaxDuration(sessionDuration). WithRequestContext(requestContext) @@ -760,13 +773,13 @@ func (s *STSService) AssumeRoleWithCredentials(ctx context.Context, request *Ass } // 4-7. Mint the session - return s.issueSession(request.RoleArn, request.RoleSessionName, sessionPolicy, + return s.issueSession(request.RoleArn, request.RoleId, request.RoleSessionName, sessionPolicy, request.DurationSeconds, provider.Name(), externalIdentity.UserID) } // issueSession mints temporary credentials and the self-contained JWT that // carries the whole session, shared by every assume-role entry point. -func (s *STSService) issueSession(roleArn, roleSessionName, sessionPolicy string, +func (s *STSService) issueSession(roleArn, roleId, roleSessionName, sessionPolicy string, durationSeconds *int64, providerName, subject string) (*AssumeRoleResponse, error) { sessionDuration := s.CalculateSessionDuration(durationSeconds) @@ -791,6 +804,7 @@ func (s *STSService) issueSession(roleArn, roleSessionName, sessionPolicy string sessionClaims := NewSTSSessionClaims(sessionId, s.Config.Issuer, expiresAt). WithSessionName(roleSessionName). WithRoleInfo(roleArn, assumedRoleUser.Arn, assumedRoleUser.Arn). + WithRoleId(roleId). WithIdentityProvider(providerName, subject, ""). WithMaxDuration(sessionDuration) if sessionPolicy != "" { @@ -891,7 +905,9 @@ func (s *STSService) AssumeRoleForPrincipal(ctx context.Context, request *Assume return nil, fmt.Errorf("role assumption denied: %w", err) } - return s.issueSession(request.RoleArn, request.RoleSessionName, sessionPolicy, + // Iceberg credential vending resolves no role definition, so its sessions + // carry no role ID and stay bound by name. + return s.issueSession(request.RoleArn, "", request.RoleSessionName, sessionPolicy, request.DurationSeconds, request.ProviderName, request.Principal) } diff --git a/weed/s3api/auth_credentials_subscribe.go b/weed/s3api/auth_credentials_subscribe.go index 8f2519ae6..69334a5fa 100644 --- a/weed/s3api/auth_credentials_subscribe.go +++ b/weed/s3api/auth_credentials_subscribe.go @@ -14,6 +14,9 @@ import ( const oidcProvidersDir = filer.IamConfigDirectory + "/oidc-providers" +// rolesDir is the filer role store's default base path (sts.DefaultRoleBasePath). +const rolesDir = filer.IamConfigDirectory + "/roles" + func (s3a *S3ApiServer) subscribeMetaEvents(clientName string, lastTsNs int64, prefix string, directoriesToWatch []string) { processEventFn := func(resp *filer_pb.SubscribeMetadataResponse) error { @@ -33,6 +36,7 @@ func (s3a *S3ApiServer) subscribeMetaEvents(clientName string, lastTsNs int64, p _ = s3a.onBucketMetadataChange(dir, message.OldEntry, message.NewEntry) _ = s3a.onIamConfigChange(dir, message.OldEntry, message.NewEntry) _ = s3a.onOIDCProviderChange(dir, message.OldEntry, message.NewEntry) + s3a.onRoleChange(dir) _ = s3a.onCircuitBreakerConfigChange(dir, message.OldEntry, message.NewEntry) // For moves across directories, replay a delete event for the source directory @@ -40,6 +44,7 @@ func (s3a *S3ApiServer) subscribeMetaEvents(clientName string, lastTsNs int64, p _ = s3a.onBucketMetadataChange(resp.Directory, message.OldEntry, nil) _ = s3a.onIamConfigChange(resp.Directory, message.OldEntry, nil) _ = s3a.onOIDCProviderChange(resp.Directory, message.OldEntry, nil) + s3a.onRoleChange(resp.Directory) _ = s3a.onCircuitBreakerConfigChange(resp.Directory, message.OldEntry, nil) } @@ -137,6 +142,40 @@ func (s3a *S3ApiServer) onOIDCProviderChange(dir string, oldEntry *filer_pb.Entr return nil } +// roleStoreDir is the directory the persisted role store keeps roles in: its +// configured basePath, else the default. The metadata subscription watches it. +func (s3a *S3ApiServer) roleStoreDir() string { + if s3a.iam != nil { + if provider, ok := s3a.iam.iamIntegration.(IAMManagerProvider); ok { + if mgr := provider.GetIAMManager(); mgr != nil { + if dir := mgr.RoleStoreDirectory(); dir != "" { + return dir + } + } + } + } + return rolesDir +} + +// onRoleChange drops the cached role definitions when the persisted role +// store's directory changes, so a role created, changed or deleted on a peer +// takes effect here on the next lookup instead of after the cache TTL. +func (s3a *S3ApiServer) onRoleChange(dir string) { + base := s3a.roleStoreDir() + if dir != base && !strings.HasPrefix(dir, base+"/") { + return + } + if s3a.iam == nil || s3a.iam.iamIntegration == nil { + return + } + s3iam, ok := s3a.iam.iamIntegration.(*S3IAMIntegration) + if !ok || s3iam.iamManager == nil { + return + } + s3iam.iamManager.InvalidateRoleCache() + glog.V(2).Infof("Invalidated cached roles after %s change", dir) +} + // onCircuitBreakerConfigChange handles circuit breaker config file changes (create, update, delete) func (s3a *S3ApiServer) onCircuitBreakerConfigChange(dir string, oldEntry *filer_pb.Entry, newEntry *filer_pb.Entry) error { if dir != s3_constants.CircuitBreakerConfigDir { diff --git a/weed/s3api/iam_defaults_test.go b/weed/s3api/iam_defaults_test.go index 8ce1febf9..5f9d4747a 100644 --- a/weed/s3api/iam_defaults_test.go +++ b/weed/s3api/iam_defaults_test.go @@ -306,3 +306,37 @@ func TestLoadIAMManagerFromConfig_HonorsOIDCProviderStore(t *testing.T) { }) } } + +func TestLoadIAMManagerFromConfig_HonorsRoleStoreAndMarksStaticRoles(t *testing.T) { + cases := []struct { + name string + store string + filer bool + }{ + {"absent keeps memory", ``, false}, + {"filer persists", `,"roleStore":{"storeType":"filer"}`, true}, + {"no config file persists", "no-file", true}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + configPath := "" + if tc.store != "no-file" { + configPath = filepath.Join(t.TempDir(), "iam_config.json") + configContent := `{"sts":{"providers":[]},"policy":{"storeType":"memory","defaultEffect":"Deny"},` + + `"roles":[{"roleName":"from-file","trustPolicy":{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Federated":"https://issuer.example"},"Action":["sts:AssumeRoleWithWebIdentity"]}]}}]` + + tc.store + `}` + assert.NoError(t, os.WriteFile(configPath, []byte(configContent), 0644)) + } + + manager, err := loadIAMManagerFromConfig(configPath, func() string { return "localhost:8888" }, func() string { return "role-store-signing-key" }) + assert.NoError(t, err) + _, inMemory := manager.GetRoleStore().(*integration.MemoryRoleStore) + assert.Equal(t, tc.filer, !inMemory) + if !tc.filer { + role, err := manager.GetRole(context.Background(), "from-file") + assert.NoError(t, err) + assert.Equal(t, integration.RoleSourceStaticConfig, role.Source) + } + }) + } +} diff --git a/weed/s3api/s3api_embedded_iam.go b/weed/s3api/s3api_embedded_iam.go index 9603dc05e..055754384 100644 --- a/weed/s3api/s3api_embedded_iam.go +++ b/weed/s3api/s3api_embedded_iam.go @@ -22,6 +22,7 @@ import ( "github.com/seaweedfs/seaweedfs/weed/credential" "github.com/seaweedfs/seaweedfs/weed/glog" iamlib "github.com/seaweedfs/seaweedfs/weed/iam" + "github.com/seaweedfs/seaweedfs/weed/iam/integration" "github.com/seaweedfs/seaweedfs/weed/pb/filer_pb" "github.com/seaweedfs/seaweedfs/weed/pb/iam_pb" "github.com/seaweedfs/seaweedfs/weed/s3api/policy_engine" @@ -646,6 +647,20 @@ func (e *EmbeddedIamApi) DeletePolicy(ctx context.Context, values url.Values) (* } } } + // Roles attach policies by name too; see integration.RolesAttachingPolicy, + // including why a role only in a peer's IAM config file is not seen here. + if mgr := e.oidcIAMManager(); mgr != nil && mgr.GetRoleStore() != nil { + roles, err := integration.RolesAttachingPolicy(ctx, mgr.GetRoleStore(), policyName) + if err != nil { + return resp, &iamError{Code: iam.ErrCodeServiceFailureException, Error: err} + } + if len(roles) > 0 { + return resp, &iamError{ + Code: iam.ErrCodeDeleteConflictException, + Error: fmt.Errorf("policy %s is attached to role %s", policyName, roles[0]), + } + } + } if err := e.credentialManager.DeletePolicy(ctx, policyName); err != nil { return resp, &iamError{Code: iam.ErrCodeServiceFailureException, Error: err} } @@ -2584,7 +2599,10 @@ func (e *EmbeddedIamApi) AuthIam(f http.HandlerFunc, _ Action) http.HandlerFunc } // ExecuteAction executes an IAM action with the given values. -// If skipPersist is true, the changed configuration is not saved to the persistent store. +// If skipPersist is true, the changed S3ApiConfiguration is not saved to the +// persistent store. OIDC provider and role actions do not change that +// configuration: they write to the IAM manager's own stores, whichever the +// server was configured with, and skipPersist does not apply to them. // reqID is set on the response; if empty, a new request ID is generated. func (e *EmbeddedIamApi) ExecuteAction(ctx context.Context, values url.Values, skipPersist bool, reqID string) (iamlib.RequestIDSetter, *iamError) { if reqID == "" { @@ -2600,7 +2618,8 @@ func (e *EmbeddedIamApi) ExecuteAction(ctx context.Context, values url.Values, s case "ListUsers", "ListAccessKeys", "GetUser", "GetUserPolicy", "ListUserPolicies", "ListAttachedUserPolicies", "ListPolicies", "GetPolicy", "ListPolicyVersions", "GetPolicyVersion", "ListServiceAccounts", "GetServiceAccount", "GetGroup", "ListGroups", "ListAttachedGroupPolicies", "GetGroupPolicy", "ListGroupPolicies", "ListGroupsForUser", "ListUserTags", - actionListOpenIDConnectProviders, actionGetOpenIDConnectProvider: + actionListOpenIDConnectProviders, actionGetOpenIDConnectProvider, + actionGetRole, actionListRoles, actionListAttachedRolePolicies: // Allowed read-only actions default: return nil, &iamError{Code: s3err.GetAPIError(s3err.ErrAccessDenied).Code, Error: fmt.Errorf("IAM write operations are disabled on this server")} @@ -2617,6 +2636,15 @@ func (e *EmbeddedIamApi) ExecuteAction(ctx context.Context, values url.Values, s return response, nil } + // Role actions operate on the IAM manager's role store, likewise. + if response, iamErr, ok := e.dispatchRoleAction(ctx, values); ok { + if iamErr != nil { + return nil, iamErr + } + response.SetRequestId(reqID) + return response, nil + } + s3cfg := &iam_pb.S3ApiConfiguration{} if err := e.GetS3ApiConfiguration(s3cfg); err != nil && !errors.Is(err, filer_pb.ErrNotFound) { return nil, &iamError{Code: s3err.GetAPIError(s3err.ErrInternalError).Code, Error: fmt.Errorf("failed to get s3 api configuration: %v", err)} diff --git a/weed/s3api/s3api_embedded_iam_role.go b/weed/s3api/s3api_embedded_iam_role.go new file mode 100644 index 000000000..3f4dbe84d --- /dev/null +++ b/weed/s3api/s3api_embedded_iam_role.go @@ -0,0 +1,419 @@ +package s3api + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "net/url" + "slices" + "strconv" + "strings" + "time" + + "github.com/aws/aws-sdk-go/service/iam" + iamlib "github.com/seaweedfs/seaweedfs/weed/iam" + "github.com/seaweedfs/seaweedfs/weed/iam/integration" + "github.com/seaweedfs/seaweedfs/weed/iam/policy" +) + +// Role IAM actions handled by this file. +const ( + actionCreateRole = "CreateRole" + actionGetRole = "GetRole" + actionListRoles = "ListRoles" + actionDeleteRole = "DeleteRole" + actionUpdateAssumeRolePolicy = "UpdateAssumeRolePolicy" + actionAttachRolePolicy = "AttachRolePolicy" + actionDetachRolePolicy = "DetachRolePolicy" + actionListAttachedRolePolicies = "ListAttachedRolePolicies" +) + +// isRoleAction reports whether an action belongs to the role family. +func isRoleAction(action string) bool { + switch action { + case actionCreateRole, actionGetRole, actionListRoles, actionDeleteRole, + actionUpdateAssumeRolePolicy, actionAttachRolePolicy, actionDetachRolePolicy, + actionListAttachedRolePolicies: + return true + default: + return false + } +} + +// dispatchRoleAction handles the role IAM actions. Roles live in the IAM +// manager's role store, not in S3ApiConfiguration, so like the OIDC provider +// actions they are dispatched before the configuration load. The boolean +// reports whether the action was recognised. +func (e *EmbeddedIamApi) dispatchRoleAction(ctx context.Context, values url.Values) (iamlib.RequestIDSetter, *iamError, bool) { + if !isRoleAction(values.Get("Action")) { + return nil, nil, false + } + mgr := e.oidcIAMManager() + if mgr == nil { + return nil, &iamError{Code: iam.ErrCodeServiceFailureException, Error: errors.New("role store not configured: start the S3 server with an IAM config")}, true + } + + switch values.Get("Action") { + case actionCreateRole: + resp, err := e.createRole(ctx, mgr, values) + return resp, err, true + case actionGetRole: + resp, err := e.getRole(ctx, mgr, values) + return resp, err, true + case actionListRoles: + resp, err := e.listRoles(ctx, mgr) + return resp, err, true + case actionDeleteRole: + resp, err := e.deleteRole(ctx, mgr, values) + return resp, err, true + case actionUpdateAssumeRolePolicy: + resp, err := e.updateAssumeRolePolicy(ctx, mgr, values) + return resp, err, true + case actionAttachRolePolicy: + resp, err := e.attachRolePolicy(ctx, mgr, values) + return resp, err, true + case actionDetachRolePolicy: + resp, err := e.detachRolePolicy(ctx, mgr, values) + return resp, err, true + case actionListAttachedRolePolicies: + resp, err := e.listAttachedRolePolicies(ctx, mgr, values) + return resp, err, true + } + return nil, nil, false +} + +// parseTrustPolicy decodes and validates an AssumeRolePolicyDocument. +func parseTrustPolicy(document string) (*policy.PolicyDocument, *iamError) { + if strings.TrimSpace(document) == "" { + return nil, &iamError{Code: iam.ErrCodeInvalidInputException, Error: errors.New("AssumeRolePolicyDocument is required")} + } + var doc policy.PolicyDocument + if err := json.Unmarshal([]byte(document), &doc); err != nil { + return nil, &iamError{Code: iam.ErrCodeMalformedPolicyDocumentException, Error: fmt.Errorf("parse trust policy: %w", err)} + } + if err := policy.ValidateTrustPolicyDocument(&doc); err != nil { + return nil, &iamError{Code: iam.ErrCodeMalformedPolicyDocumentException, Error: err} + } + return &doc, nil +} + +// requireRole loads the named role, mapping a missing role to NoSuchEntity. +func requireRole(ctx context.Context, mgr *integration.IAMManager, values url.Values) (*integration.RoleDefinition, *iamError) { + name := strings.TrimSpace(values.Get("RoleName")) + if name == "" { + return nil, &iamError{Code: iam.ErrCodeInvalidInputException, Error: errors.New("RoleName is required")} + } + role, err := mgr.GetRole(ctx, name) + if errors.Is(err, integration.ErrRoleNotFound) || (err == nil && role == nil) { + return nil, &iamError{Code: iam.ErrCodeNoSuchEntityException, Error: fmt.Errorf("role %s not found", name)} + } + if err != nil { + return nil, &iamError{Code: iam.ErrCodeServiceFailureException, Error: err} + } + return role, nil +} + +// requireMutableRole is requireRole for actions that change or delete the +// role. A role loaded from the IAM config file is reloaded from it at every +// start, so a change made through the API would be silently reverted; it is +// refused instead. +func requireMutableRole(ctx context.Context, mgr *integration.IAMManager, values url.Values) (*integration.RoleDefinition, *iamError) { + role, iamErr := requireRole(ctx, mgr, values) + if iamErr != nil { + return nil, iamErr + } + if role.Source == integration.RoleSourceStaticConfig { + return nil, &iamError{Code: iam.ErrCodeUnmodifiableEntityException, Error: fmt.Errorf("role %s is defined in the IAM config file; change it there", role.RoleName)} + } + return role, nil +} + +// errRoleUpdateRefused aborts a RoleStore update whose refusal the handler +// has already recorded as an iamError. +var errRoleUpdateRefused = errors.New("role update refused") + +// updateRole applies change to the named role through the role store's atomic +// update, so a concurrent change on another server is neither lost nor +// written over a delete. change sees the current role and returns an iamError +// to refuse; a missing role is NoSuchEntity and a config-file role is +// UnmodifiableEntity, as requireMutableRole reports them. +func updateRole(ctx context.Context, mgr *integration.IAMManager, name string, change func(role *integration.RoleDefinition) *iamError) *iamError { + var refused *iamError + err := mgr.UpdateRole(ctx, name, func(current *integration.RoleDefinition) (*integration.RoleDefinition, error) { + if current == nil { + refused = &iamError{Code: iam.ErrCodeNoSuchEntityException, Error: fmt.Errorf("role %s not found", name)} + return nil, errRoleUpdateRefused + } + if refused = change(current); refused != nil { + return nil, errRoleUpdateRefused + } + return current, nil + }) + return roleWriteError(err, refused) +} + +// roleWriteError maps a role store write's outcome to the IAM error to +// report: the handler's own refusal, else the store's error by kind. A write +// that failed for another reason is a service failure, which clients retry. +func roleWriteError(err error, refused *iamError) *iamError { + switch { + case err == nil: + return nil + case errors.Is(err, errRoleUpdateRefused) && refused != nil: + return refused + case errors.Is(err, integration.ErrRoleStatic): + return &iamError{Code: iam.ErrCodeUnmodifiableEntityException, Error: fmt.Errorf("%w; change it there", err)} + case errors.Is(err, integration.ErrRoleExists): + return &iamError{Code: iam.ErrCodeEntityAlreadyExistsException, Error: err} + default: + return &iamError{Code: iam.ErrCodeServiceFailureException, Error: err} + } +} + +// roleID is the role's stored ID. A role stored before IDs were recorded has +// none; it is reported with the ID StaticRoleID derives for it. +func roleID(role *integration.RoleDefinition) string { + if role.RoleId != "" { + return role.RoleId + } + return integration.StaticRoleID(role) +} + +func toIAMRole(role *integration.RoleDefinition) iamlib.IAMRole { + out := iamlib.IAMRole{ + Path: "/", + RoleName: role.RoleName, + RoleId: roleID(role), + Arn: role.RoleArn, + Description: role.Description, + MaxSessionDuration: role.MaxSessionDuration, + } + if !role.CreatedAt.IsZero() { + out.CreateDate = role.CreatedAt.UTC().Format(time.RFC3339) + } + if role.TrustPolicy != nil { + if doc, err := json.Marshal(role.TrustPolicy); err == nil { + // AWS returns the document URL-encoded. + out.AssumeRolePolicyDocument = url.PathEscape(string(doc)) + } + } + return out +} + +func (e *EmbeddedIamApi) createRole(ctx context.Context, mgr *integration.IAMManager, values url.Values) (*iamlib.CreateRoleResponse, *iamError) { + name := strings.TrimSpace(values.Get("RoleName")) + if name == "" { + return nil, &iamError{Code: iam.ErrCodeInvalidInputException, Error: errors.New("RoleName is required")} + } + if err := integration.ValidateRoleName(name); err != nil { + return nil, &iamError{Code: iam.ErrCodeInvalidInputException, Error: err} + } + if path := values.Get("Path"); path != "" && path != "/" { + return nil, &iamError{Code: iam.ErrCodeInvalidInputException, Error: fmt.Errorf("role paths are not supported: %s", path)} + } + for key := range values { + if strings.HasPrefix(key, "Tags.") { + return nil, &iamError{Code: iam.ErrCodeInvalidInputException, Error: errors.New("role tags are not supported")} + } + } + // A config-file role is served beside the store, not from it, so the + // store's create-if-absent cannot see it. + existing, err := mgr.GetRole(ctx, name) + if err == nil && existing != nil { + return nil, &iamError{Code: iam.ErrCodeEntityAlreadyExistsException, Error: fmt.Errorf("role %s already exists", name)} + } + if err != nil && !errors.Is(err, integration.ErrRoleNotFound) { + return nil, &iamError{Code: iam.ErrCodeServiceFailureException, Error: err} + } + trust, iamErr := parseTrustPolicy(values.Get("AssumeRolePolicyDocument")) + if iamErr != nil { + return nil, iamErr + } + var maxSession int64 + if raw := values.Get("MaxSessionDuration"); raw != "" { + n, err := strconv.ParseInt(raw, 10, 64) + if err != nil { + return nil, &iamError{Code: iam.ErrCodeInvalidInputException, Error: fmt.Errorf("MaxSessionDuration: %w", err)} + } + maxSession = n + } + + role := &integration.RoleDefinition{ + RoleName: name, + TrustPolicy: trust, + Description: values.Get("Description"), + MaxSessionDuration: maxSession, + CreatedAt: time.Now().UTC(), + RoleId: integration.NewRoleID(), + } + // Validation first: only a bad request is InvalidInput. The write below + // is created only if no role of this name exists by then, so of two + // concurrent creates, one fails with EntityAlreadyExists. + if err := integration.PrepareRoleDefinition(name, role); err != nil { + return nil, &iamError{Code: iam.ErrCodeInvalidInputException, Error: err} + } + err = mgr.UpdateRole(ctx, name, func(current *integration.RoleDefinition) (*integration.RoleDefinition, error) { + if current != nil { + return nil, fmt.Errorf("%w: %s", integration.ErrRoleExists, name) + } + return role, nil + }) + if iamErr := roleWriteError(err, nil); iamErr != nil { + return nil, iamErr + } + resp := &iamlib.CreateRoleResponse{} + resp.CreateRoleResult.Role = toIAMRole(role) + return resp, nil +} + +func (e *EmbeddedIamApi) getRole(ctx context.Context, mgr *integration.IAMManager, values url.Values) (*iamlib.GetRoleResponse, *iamError) { + role, iamErr := requireRole(ctx, mgr, values) + if iamErr != nil { + return nil, iamErr + } + resp := &iamlib.GetRoleResponse{} + resp.GetRoleResult.Role = toIAMRole(role) + return resp, nil +} + +func (e *EmbeddedIamApi) listRoles(ctx context.Context, mgr *integration.IAMManager) (*iamlib.ListRolesResponse, *iamError) { + roles, err := mgr.ListRoles(ctx) + if err != nil { + return nil, &iamError{Code: iam.ErrCodeServiceFailureException, Error: err} + } + resp := &iamlib.ListRolesResponse{} + resp.ListRolesResult.Roles = make([]*iamlib.IAMRole, 0, len(roles)) + for _, role := range roles { + view := toIAMRole(role) + resp.ListRolesResult.Roles = append(resp.ListRolesResult.Roles, &view) + } + return resp, nil +} + +func (e *EmbeddedIamApi) deleteRole(ctx context.Context, mgr *integration.IAMManager, values url.Values) (*iamlib.DeleteRoleResponse, *iamError) { + role, iamErr := requireMutableRole(ctx, mgr, values) + if iamErr != nil { + return nil, iamErr + } + // The delete is decided against the role as it is when deleted, in the + // store's atomic update: a policy attached meanwhile on another server is + // a DeleteConflict, as AWS reports a role with managed policies attached. + var refused *iamError + err := mgr.UpdateRole(ctx, role.RoleName, func(current *integration.RoleDefinition) (*integration.RoleDefinition, error) { + if current == nil { + refused = &iamError{Code: iam.ErrCodeNoSuchEntityException, Error: fmt.Errorf("role %s not found", role.RoleName)} + return nil, errRoleUpdateRefused + } + if len(current.AttachedPolicies) > 0 { + refused = &iamError{Code: iam.ErrCodeDeleteConflictException, Error: fmt.Errorf("role %s has attached policies; detach them first", role.RoleName)} + return nil, errRoleUpdateRefused + } + return nil, nil + }) + if iamErr := roleWriteError(err, refused); iamErr != nil { + return nil, iamErr + } + return &iamlib.DeleteRoleResponse{}, nil +} + +func (e *EmbeddedIamApi) updateAssumeRolePolicy(ctx context.Context, mgr *integration.IAMManager, values url.Values) (*iamlib.UpdateAssumeRolePolicyResponse, *iamError) { + role, iamErr := requireMutableRole(ctx, mgr, values) + if iamErr != nil { + return nil, iamErr + } + trust, iamErr := parseTrustPolicy(values.Get("PolicyDocument")) + if iamErr != nil { + return nil, iamErr + } + if iamErr := updateRole(ctx, mgr, role.RoleName, func(current *integration.RoleDefinition) *iamError { + current.TrustPolicy = trust + return nil + }); iamErr != nil { + return nil, iamErr + } + return &iamlib.UpdateAssumeRolePolicyResponse{}, nil +} + +// rolePolicyName resolves PolicyArn to the name of an existing managed policy. +func (e *EmbeddedIamApi) rolePolicyName(ctx context.Context, values url.Values) (string, *iamError) { + name, err := iamPolicyNameFromArn(values.Get("PolicyArn")) + if err != nil { + return "", &iamError{Code: iam.ErrCodeInvalidInputException, Error: err} + } + if e.credentialManager == nil { + return "", &iamError{Code: iam.ErrCodeServiceFailureException, Error: errors.New("credential manager not configured")} + } + existing, err := e.credentialManager.GetPolicy(ctx, name) + if err != nil { + return "", &iamError{Code: iam.ErrCodeServiceFailureException, Error: err} + } + if existing == nil { + return "", &iamError{Code: iam.ErrCodeNoSuchEntityException, Error: fmt.Errorf("policy %s not found", name)} + } + return name, nil +} + +func (e *EmbeddedIamApi) attachRolePolicy(ctx context.Context, mgr *integration.IAMManager, values url.Values) (*iamlib.AttachRolePolicyResponse, *iamError) { + role, iamErr := requireMutableRole(ctx, mgr, values) + if iamErr != nil { + return nil, iamErr + } + name, iamErr := e.rolePolicyName(ctx, values) + if iamErr != nil { + return nil, iamErr + } + if slices.Contains(role.AttachedPolicies, name) { + return &iamlib.AttachRolePolicyResponse{}, nil + } + if iamErr := updateRole(ctx, mgr, role.RoleName, func(current *integration.RoleDefinition) *iamError { + if slices.Contains(current.AttachedPolicies, name) { + return nil + } + if len(current.AttachedPolicies) >= integration.MaxManagedPoliciesPerRole { + return &iamError{Code: iam.ErrCodeLimitExceededException, + Error: fmt.Errorf("cannot attach more than %d managed policies to role %s", integration.MaxManagedPoliciesPerRole, current.RoleName)} + } + current.AttachedPolicies = append(current.AttachedPolicies, name) + return nil + }); iamErr != nil { + return nil, iamErr + } + return &iamlib.AttachRolePolicyResponse{}, nil +} + +func (e *EmbeddedIamApi) detachRolePolicy(ctx context.Context, mgr *integration.IAMManager, values url.Values) (*iamlib.DetachRolePolicyResponse, *iamError) { + role, iamErr := requireMutableRole(ctx, mgr, values) + if iamErr != nil { + return nil, iamErr + } + name, err := iamPolicyNameFromArn(values.Get("PolicyArn")) + if err != nil { + return nil, &iamError{Code: iam.ErrCodeInvalidInputException, Error: err} + } + if iamErr := updateRole(ctx, mgr, role.RoleName, func(current *integration.RoleDefinition) *iamError { + idx := slices.Index(current.AttachedPolicies, name) + if idx < 0 { + return &iamError{Code: iam.ErrCodeNoSuchEntityException, Error: fmt.Errorf("policy %s is not attached to role %s", name, current.RoleName)} + } + current.AttachedPolicies = slices.Delete(current.AttachedPolicies, idx, idx+1) + return nil + }); iamErr != nil { + return nil, iamErr + } + return &iamlib.DetachRolePolicyResponse{}, nil +} + +func (e *EmbeddedIamApi) listAttachedRolePolicies(ctx context.Context, mgr *integration.IAMManager, values url.Values) (*iamlib.ListAttachedRolePoliciesResponse, *iamError) { + role, iamErr := requireRole(ctx, mgr, values) + if iamErr != nil { + return nil, iamErr + } + resp := &iamlib.ListAttachedRolePoliciesResponse{} + resp.ListAttachedRolePoliciesResult.AttachedPolicies = make([]*iamlib.IAMAttachedPolicy, 0, len(role.AttachedPolicies)) + for _, name := range role.AttachedPolicies { + resp.ListAttachedRolePoliciesResult.AttachedPolicies = append(resp.ListAttachedRolePoliciesResult.AttachedPolicies, + &iamlib.IAMAttachedPolicy{PolicyName: name, PolicyArn: iamPolicyArn(name)}) + } + return resp, nil +} diff --git a/weed/s3api/s3api_iam_role_test.go b/weed/s3api/s3api_iam_role_test.go new file mode 100644 index 000000000..985c3e8a3 --- /dev/null +++ b/weed/s3api/s3api_iam_role_test.go @@ -0,0 +1,340 @@ +package s3api + +import ( + "context" + "encoding/xml" + "errors" + "fmt" + "net/url" + "strings" + "testing" + + "github.com/aws/aws-sdk-go/service/iam" + iamlib "github.com/seaweedfs/seaweedfs/weed/iam" + "github.com/seaweedfs/seaweedfs/weed/iam/integration" + "github.com/seaweedfs/seaweedfs/weed/pb/iam_pb" + "github.com/seaweedfs/seaweedfs/weed/s3api/policy_engine" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +const spiffeTrustPolicy = `{"Version":"2012-10-17","Statement":[{"Effect":"Allow",` + + `"Principal":{"Federated":"https://oidc.example"},"Action":["sts:AssumeRoleWithWebIdentity"],` + + `"Condition":{"StringEquals":{"oidc:sub":"spiffe://example.org/ns/app/sa/app"}}}]}` + +func roleAction(t *testing.T, api *EmbeddedIamApiForTest, params map[string]string) (iamlib.RequestIDSetter, *iamError) { + t.Helper() + values := url.Values{} + for k, v := range params { + values.Set(k, v) + } + return api.ExecuteAction(context.Background(), values, true, "role-test") +} + +func requireIamCode(t *testing.T, iamErr *iamError, code string) { + t.Helper() + require.NotNil(t, iamErr, "expected %s", code) + assert.Equal(t, code, iamErr.Code, "error: %v", iamErr.Error) +} + +func newRoleTestAPI(t *testing.T) (*EmbeddedIamApiForTest, *integration.IAMManager) { + t.Helper() + api, mgr := newOIDCTestAPI(t) + doc := policy_engine.PolicyDocument{Version: "2012-10-17", Statement: []policy_engine.PolicyStatement{{ + Effect: policy_engine.PolicyEffectAllow, + Action: policy_engine.NewStringOrStringSlice("s3:GetObject"), + Resource: policy_engine.NewStringOrStringSlicePtr("arn:aws:s3:::bucket/*"), + }}} + require.NoError(t, api.credentialManager.CreatePolicy(context.Background(), "read-bucket", doc)) + return api, mgr +} + +func TestRoleLifecycle(t *testing.T) { + api, mgr := newRoleTestAPI(t) + + resp, iamErr := roleAction(t, api, map[string]string{ + "Action": actionCreateRole, "RoleName": "app", "AssumeRolePolicyDocument": spiffeTrustPolicy, + "Description": "app runtime", "MaxSessionDuration": "3600", + }) + require.Nil(t, iamErr) + created := resp.(*iamlib.CreateRoleResponse).CreateRoleResult.Role + assert.Equal(t, "arn:aws:iam::role/app", created.Arn) + assert.NotEmpty(t, created.RoleId) + assert.NotEmpty(t, created.CreateDate) + decoded, err := url.PathUnescape(created.AssumeRolePolicyDocument) + require.NoError(t, err) + assert.Contains(t, decoded, "spiffe://example.org/ns/app/sa/app") + + // The role is the one STS evaluates. + stored, err := mgr.GetRole(context.Background(), "app") + require.NoError(t, err) + assert.Equal(t, "https://oidc.example", stored.TrustPolicy.Statement[0].Principal.(map[string]interface{})["Federated"]) + + _, iamErr = roleAction(t, api, map[string]string{"Action": actionAttachRolePolicy, "RoleName": "app", "PolicyArn": "arn:aws:iam:::policy/read-bucket"}) + require.Nil(t, iamErr) + resp, iamErr = roleAction(t, api, map[string]string{"Action": actionListAttachedRolePolicies, "RoleName": "app"}) + require.Nil(t, iamErr) + attached := resp.(*iamlib.ListAttachedRolePoliciesResponse).ListAttachedRolePoliciesResult.AttachedPolicies + require.Len(t, attached, 1) + assert.Equal(t, "read-bucket", attached[0].PolicyName) + + _, iamErr = roleAction(t, api, map[string]string{"Action": actionDeleteRole, "RoleName": "app"}) + requireIamCode(t, iamErr, iam.ErrCodeDeleteConflictException) + + _, iamErr = roleAction(t, api, map[string]string{"Action": actionDetachRolePolicy, "RoleName": "app", "PolicyArn": "arn:aws:iam:::policy/read-bucket"}) + require.Nil(t, iamErr) + _, iamErr = roleAction(t, api, map[string]string{"Action": actionDeleteRole, "RoleName": "app"}) + require.Nil(t, iamErr) + _, iamErr = roleAction(t, api, map[string]string{"Action": actionGetRole, "RoleName": "app"}) + requireIamCode(t, iamErr, iam.ErrCodeNoSuchEntityException) +} + +func TestUpdateAssumeRolePolicyReplacesTheTrustPolicy(t *testing.T) { + api, mgr := newRoleTestAPI(t) + _, iamErr := roleAction(t, api, map[string]string{"Action": actionCreateRole, "RoleName": "app", "AssumeRolePolicyDocument": spiffeTrustPolicy}) + require.Nil(t, iamErr) + + updated := `{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Federated":"https://other.example"},"Action":["sts:AssumeRoleWithWebIdentity"]}]}` + _, iamErr = roleAction(t, api, map[string]string{"Action": actionUpdateAssumeRolePolicy, "RoleName": "app", "PolicyDocument": updated}) + require.Nil(t, iamErr) + stored, err := mgr.GetRole(context.Background(), "app") + require.NoError(t, err) + assert.Equal(t, "https://other.example", stored.TrustPolicy.Statement[0].Principal.(map[string]interface{})["Federated"]) +} + +func TestRoleActionsRefuseWhatTheyCannotHonour(t *testing.T) { + api, _ := newRoleTestAPI(t) + _, iamErr := roleAction(t, api, map[string]string{"Action": actionCreateRole, "RoleName": "app", "AssumeRolePolicyDocument": spiffeTrustPolicy}) + require.Nil(t, iamErr) + + cases := []struct { + name string + params map[string]string + code string + }{ + {"duplicate", map[string]string{"Action": actionCreateRole, "RoleName": "app", "AssumeRolePolicyDocument": spiffeTrustPolicy}, iam.ErrCodeEntityAlreadyExistsException}, + {"no trust policy", map[string]string{"Action": actionCreateRole, "RoleName": "b"}, iam.ErrCodeInvalidInputException}, + {"malformed trust policy", map[string]string{"Action": actionCreateRole, "RoleName": "b", "AssumeRolePolicyDocument": "{"}, iam.ErrCodeMalformedPolicyDocumentException}, + {"path", map[string]string{"Action": actionCreateRole, "RoleName": "b", "Path": "/team/", "AssumeRolePolicyDocument": spiffeTrustPolicy}, iam.ErrCodeInvalidInputException}, + {"tags", map[string]string{"Action": actionCreateRole, "RoleName": "b", "Tags.member.1.Key": "k", "Tags.member.1.Value": "v", "AssumeRolePolicyDocument": spiffeTrustPolicy}, iam.ErrCodeInvalidInputException}, + {"session out of bounds", map[string]string{"Action": actionCreateRole, "RoleName": "b", "MaxSessionDuration": "60", "AssumeRolePolicyDocument": spiffeTrustPolicy}, iam.ErrCodeInvalidInputException}, + {"attach missing policy", map[string]string{"Action": actionAttachRolePolicy, "RoleName": "app", "PolicyArn": "arn:aws:iam:::policy/nope"}, iam.ErrCodeNoSuchEntityException}, + {"detach unattached", map[string]string{"Action": actionDetachRolePolicy, "RoleName": "app", "PolicyArn": "arn:aws:iam:::policy/read-bucket"}, iam.ErrCodeNoSuchEntityException}, + {"missing role", map[string]string{"Action": actionDeleteRole, "RoleName": "nope"}, iam.ErrCodeNoSuchEntityException}, + {"name leaving the role store", map[string]string{"Action": actionCreateRole, "RoleName": "../identities/admin", "AssumeRolePolicyDocument": spiffeTrustPolicy}, iam.ErrCodeInvalidInputException}, + {"name too long", map[string]string{"Action": actionCreateRole, "RoleName": strings.Repeat("a", 65), "AssumeRolePolicyDocument": spiffeTrustPolicy}, iam.ErrCodeInvalidInputException}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + _, iamErr := roleAction(t, api, tc.params) + requireIamCode(t, iamErr, tc.code) + }) + } +} + +// A role from the IAM config file is reloaded from it at every start, so an +// API change to it would be silently reverted; it is refused instead. +func TestConfigFileRolesAreUnmodifiable(t *testing.T) { + api, mgr := newRoleTestAPI(t) + trust, iamErr := parseTrustPolicy(spiffeTrustPolicy) + require.Nil(t, iamErr) + require.NoError(t, mgr.CreateRole(context.Background(), "", "from-file", &integration.RoleDefinition{ + RoleName: "from-file", TrustPolicy: trust, Source: integration.RoleSourceStaticConfig, + })) + + for _, params := range []map[string]string{ + {"Action": actionDeleteRole, "RoleName": "from-file"}, + {"Action": actionUpdateAssumeRolePolicy, "RoleName": "from-file", "PolicyDocument": spiffeTrustPolicy}, + {"Action": actionAttachRolePolicy, "RoleName": "from-file", "PolicyArn": "arn:aws:iam:::policy/read-bucket"}, + {"Action": actionDetachRolePolicy, "RoleName": "from-file", "PolicyArn": "arn:aws:iam:::policy/read-bucket"}, + } { + t.Run(params["Action"], func(t *testing.T) { + _, iamErr := roleAction(t, api, params) + requireIamCode(t, iamErr, iam.ErrCodeUnmodifiableEntityException) + }) + } + _, iamErr = roleAction(t, api, map[string]string{"Action": actionGetRole, "RoleName": "from-file"}) + assert.Nil(t, iamErr, "reading a config-file role is allowed") +} + +func TestReadOnlyAllowsRoleReadsAndDeniesMutations(t *testing.T) { + api, _ := newRoleTestAPI(t) + _, iamErr := roleAction(t, api, map[string]string{"Action": actionCreateRole, "RoleName": "app", "AssumeRolePolicyDocument": spiffeTrustPolicy}) + require.Nil(t, iamErr) + api.readOnly = true + + for _, action := range []string{actionGetRole, actionListRoles, actionListAttachedRolePolicies} { + _, iamErr := roleAction(t, api, map[string]string{"Action": action, "RoleName": "app"}) + assert.Nil(t, iamErr, "%s must be allowed in read-only mode", action) + } + for _, action := range []string{actionCreateRole, actionDeleteRole, actionUpdateAssumeRolePolicy, actionAttachRolePolicy, actionDetachRolePolicy} { + _, iamErr := roleAction(t, api, map[string]string{"Action": action, "RoleName": "app"}) + assert.NotNil(t, iamErr, "%s must be denied in read-only mode", action) + } +} + +// unreadableRoleStore fails every read the way an unreachable filer does. +type unreadableRoleStore struct{ *integration.MemoryRoleStore } + +func (unreadableRoleStore) GetRole(context.Context, string, string) (*integration.RoleDefinition, error) { + return nil, errors.New("lookup role: filer unavailable") +} + +// An unreadable store is not an absent role: CreateRole must not write over a +// role it could not see, and the reads must not report NoSuchEntity. +func TestRoleActionsTreatAnUnreadableStoreAsAFailureNotAnAbsence(t *testing.T) { + api, mgr := newRoleTestAPI(t) + store := unreadableRoleStore{integration.NewMemoryRoleStore()} + mgr.SetRoleStore(store) + + _, iamErr := roleAction(t, api, map[string]string{"Action": actionCreateRole, "RoleName": "app", "AssumeRolePolicyDocument": spiffeTrustPolicy}) + requireIamCode(t, iamErr, iam.ErrCodeServiceFailureException) + names, err := store.ListRoles(context.Background(), "") + require.NoError(t, err) + assert.Empty(t, names, "CreateRole wrote through a store it could not read") + + _, iamErr = roleAction(t, api, map[string]string{"Action": actionGetRole, "RoleName": "app"}) + requireIamCode(t, iamErr, iam.ErrCodeServiceFailureException) +} + +// A policy is attached to a role by name, so deleting it while attached would +// let a policy created later under that name take effect on the role. It is +// refused, as for users and groups. +func TestDeletePolicyAttachedToARoleIsAConflict(t *testing.T) { + api, _ := newRoleTestAPI(t) + // The test API replaces the credential store with mockConfig on the first + // action that loads the configuration, so the policy has to be declared + // there too. + api.mockConfig.Policies = append(api.mockConfig.Policies, &iam_pb.Policy{ + Name: "read-bucket", + Content: `{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["s3:GetObject"],"Resource":["arn:aws:s3:::bucket/*"]}]}`, + }) + _, iamErr := roleAction(t, api, map[string]string{"Action": actionCreateRole, "RoleName": "app", "AssumeRolePolicyDocument": spiffeTrustPolicy}) + require.Nil(t, iamErr) + _, iamErr = roleAction(t, api, map[string]string{"Action": actionAttachRolePolicy, "RoleName": "app", "PolicyArn": "arn:aws:iam:::policy/read-bucket"}) + require.Nil(t, iamErr) + + _, iamErr = roleAction(t, api, map[string]string{"Action": "DeletePolicy", "PolicyArn": "arn:aws:iam:::policy/read-bucket"}) + requireIamCode(t, iamErr, iam.ErrCodeDeleteConflictException) + + _, iamErr = roleAction(t, api, map[string]string{"Action": actionDetachRolePolicy, "RoleName": "app", "PolicyArn": "arn:aws:iam:::policy/read-bucket"}) + require.Nil(t, iamErr) + _, iamErr = roleAction(t, api, map[string]string{"Action": "DeletePolicy", "PolicyArn": "arn:aws:iam:::policy/read-bucket"}) + assert.Nil(t, iamErr, "a detached policy could not be deleted") +} + +func TestAttachRolePolicyStopsAtTheRoleQuota(t *testing.T) { + api, _ := newRoleTestAPI(t) + _, iamErr := roleAction(t, api, map[string]string{"Action": actionCreateRole, "RoleName": "app", "AssumeRolePolicyDocument": spiffeTrustPolicy}) + require.Nil(t, iamErr) + doc := policy_engine.PolicyDocument{Version: "2012-10-17", Statement: []policy_engine.PolicyStatement{{ + Effect: policy_engine.PolicyEffectAllow, Action: policy_engine.NewStringOrStringSlice("s3:GetObject"), + Resource: policy_engine.NewStringOrStringSlicePtr("arn:aws:s3:::bucket/*"), + }}} + for i := 0; i <= integration.MaxManagedPoliciesPerRole; i++ { + name := fmt.Sprintf("p%d", i) + require.NoError(t, api.credentialManager.CreatePolicy(context.Background(), name, doc)) + _, iamErr = roleAction(t, api, map[string]string{"Action": actionAttachRolePolicy, "RoleName": "app", "PolicyArn": "arn:aws:iam:::policy/" + name}) + if i < integration.MaxManagedPoliciesPerRole { + require.Nil(t, iamErr, "attach %d", i) + } + } + requireIamCode(t, iamErr, iam.ErrCodeLimitExceededException) +} + +// A role without a creation time omits CreateDate rather than sending it +// empty, which clients cannot parse. +func TestRoleWithoutACreationTimeOmitsCreateDate(t *testing.T) { + out, err := xml.Marshal(toIAMRole(&integration.RoleDefinition{RoleName: "r", RoleArn: "arn:aws:iam::role/r"})) + require.NoError(t, err) + assert.NotContains(t, string(out), "CreateDate") +} + +// Roles take no tags, so any tag parameter is refused rather than dropped. +func TestCreateRoleRefusesEveryTagParameter(t *testing.T) { + api, _ := newRoleTestAPI(t) + for _, key := range []string{"Tags.member.1.Key", "Tags.member.2.Key", "Tags.member.1.Value"} { + _, iamErr := roleAction(t, api, map[string]string{ + "Action": actionCreateRole, "RoleName": "app", "AssumeRolePolicyDocument": spiffeTrustPolicy, key: "x", + }) + requireIamCode(t, iamErr, iam.ErrCodeInvalidInputException) + } +} + +// unwritableRoleStore fails every write the way an unreachable filer does. +type unwritableRoleStore struct{ *integration.MemoryRoleStore } + +func (unwritableRoleStore) UpdateRole(context.Context, string, string, integration.RoleUpdate) error { + return errors.New("store role: filer unavailable") +} + +// A store that cannot be written is a service failure, which clients retry; +// only a bad request is InvalidInput. +func TestCreateRoleReportsAFailedWriteAsAServiceFailure(t *testing.T) { + api, mgr := newRoleTestAPI(t) + mgr.SetRoleStore(unwritableRoleStore{integration.NewMemoryRoleStore()}) + + _, iamErr := roleAction(t, api, map[string]string{"Action": actionCreateRole, "RoleName": "app", "AssumeRolePolicyDocument": spiffeTrustPolicy}) + requireIamCode(t, iamErr, iam.ErrCodeServiceFailureException) + + _, iamErr = roleAction(t, api, map[string]string{ + "Action": actionCreateRole, "RoleName": "app", "AssumeRolePolicyDocument": spiffeTrustPolicy, "MaxSessionDuration": "60", + }) + requireIamCode(t, iamErr, iam.ErrCodeInvalidInputException) +} + +// racingRoleStore reports every role absent to plain reads while the store +// holds them: the view of a server whose create raced another server's. +type racingRoleStore struct{ *integration.MemoryRoleStore } + +func (racingRoleStore) GetRole(_ context.Context, _ string, name string) (*integration.RoleDefinition, error) { + return nil, fmt.Errorf("%w: %s", integration.ErrRoleNotFound, name) +} + +// Of two creates of one name, the second is told the role exists; it does +// not replace the first role. +func TestCreateRoleRacingAnotherCreateIsEntityAlreadyExists(t *testing.T) { + api, mgr := newRoleTestAPI(t) + store := racingRoleStore{integration.NewMemoryRoleStore()} + require.NoError(t, store.StoreRole(context.Background(), "", "app", &integration.RoleDefinition{RoleName: "app", RoleId: "AROA-FIRST"})) + mgr.SetRoleStore(store) + + _, iamErr := roleAction(t, api, map[string]string{"Action": actionCreateRole, "RoleName": "app", "AssumeRolePolicyDocument": spiffeTrustPolicy}) + requireIamCode(t, iamErr, iam.ErrCodeEntityAlreadyExistsException) + role, err := store.MemoryRoleStore.GetRole(context.Background(), "", "app") + require.NoError(t, err) + assert.Equal(t, "AROA-FIRST", role.RoleId, "the second create replaced the first role") +} + +// vanishingRoleStore lists a role that is gone by the time it is read. +type vanishingRoleStore struct{ *integration.MemoryRoleStore } + +func (s vanishingRoleStore) ListRoles(ctx context.Context, filerAddress string) ([]string, error) { + names, err := s.MemoryRoleStore.ListRoles(ctx, filerAddress) + return append(names, "deleted-meanwhile"), err +} + +func TestListRolesSkipsARoleDeletedWhileListing(t *testing.T) { + api, mgr := newRoleTestAPI(t) + mgr.SetRoleStore(vanishingRoleStore{integration.NewMemoryRoleStore()}) + _, iamErr := roleAction(t, api, map[string]string{"Action": actionCreateRole, "RoleName": "app", "AssumeRolePolicyDocument": spiffeTrustPolicy}) + require.Nil(t, iamErr) + + resp, iamErr := roleAction(t, api, map[string]string{"Action": actionListRoles}) + require.Nil(t, iamErr) + roles := resp.(*iamlib.ListRolesResponse).ListRolesResult.Roles + require.Len(t, roles, 1) + assert.Equal(t, "app", roles[0].RoleName) +} + +// Peer role changes are watched in the role store's own directory. +func TestRoleChangesAreWatchedInTheRoleStoresDirectory(t *testing.T) { + api, mgr := newRoleTestAPI(t) + s3a := &S3ApiServer{iam: api.iam} + assert.Equal(t, rolesDir, s3a.roleStoreDir(), "a memory store falls back to the default directory") + + store, err := integration.NewGenericCachedRoleStore(map[string]interface{}{"basePath": "/custom/roles"}, func() string { return "localhost:8888" }) + require.NoError(t, err) + mgr.SetRoleStore(store) + assert.Equal(t, "/custom/roles", s3a.roleStoreDir()) +} diff --git a/weed/s3api/s3api_server.go b/weed/s3api/s3api_server.go index e54d2ebfa..aea2a558a 100644 --- a/weed/s3api/s3api_server.go +++ b/weed/s3api/s3api_server.go @@ -501,7 +501,7 @@ func NewS3ApiServerWithStore(router *mux.Router, option *S3ApiServerOption, expl s3ApiServer.registerRouter(router) - go s3ApiServer.subscribeMetaEvents("s3", startTsNs, filer.DirectoryEtcRoot, []string{ + watched := []string{ option.BucketsPath, filer.IamConfigDirectory, filer.IamConfigDirectory + "/identities", @@ -509,7 +509,13 @@ func NewS3ApiServerWithStore(router *mux.Router, option *S3ApiServerOption, expl filer.IamConfigDirectory + "/service_accounts", filer.IamConfigDirectory + "/groups", filer.IamConfigDirectory + "/oidc-providers", - }) + filer.IamConfigDirectory + "/roles", + } + // A role store configured with its own basePath is watched there too. + if dir := s3ApiServer.roleStoreDir(); !slices.Contains(watched, dir) { + watched = append(watched, dir) + } + go s3ApiServer.subscribeMetaEvents("s3", startTsNs, filer.DirectoryEtcRoot, watched) // Start bucket size metrics collection in background go s3ApiServer.startBucketSizeMetricsLoop(context.Background()) @@ -1171,7 +1177,10 @@ func loadIAMManagerFromConfig(configPath string, filerAddressProvider func() str // OIDCProviderStore selects where IAM-managed OIDC providers persist. // Absent, they live in memory and are lost on restart. OIDCProviderStore *integration.OIDCProviderStoreConfig `json:"oidcProviderStore"` - Policies []struct { + // RoleStore selects where roles persist. Absent, they live in memory and + // only the roles in this file exist. + RoleStore *integration.RoleStoreConfig `json:"roleStore"` + Policies []struct { Name string `json:"name"` Document *policy.PolicyDocument `json:"document"` } `json:"policies"` @@ -1217,21 +1226,28 @@ func loadIAMManagerFromConfig(configPath string, filerAddressProvider func() str } // With no IAM config file there is nothing static for a persisted - // provider to shadow or outlive, so providers created at runtime default - // to the filer, where restarts and peer S3 servers see them. A config - // file keeps the in-memory default unless it sets oidcProviderStore. + // provider or role to shadow or outlive, so those created at runtime + // default to the filer, where restarts and peer S3 servers see them. A + // config file keeps the in-memory defaults unless it sets + // oidcProviderStore / roleStore. + persistByDefault := configPath == "" && filerAddressProvider != nil oidcProviderStore := configRoot.OIDCProviderStore - if oidcProviderStore == nil && configPath == "" && filerAddressProvider != nil { + if oidcProviderStore == nil && persistByDefault { oidcProviderStore = &integration.OIDCProviderStoreConfig{StoreType: "filer"} } + roleStore := configRoot.RoleStore + if roleStore == nil { + roleStore = &integration.RoleStoreConfig{StoreType: sts.StoreTypeMemory} + if persistByDefault { + roleStore = &integration.RoleStoreConfig{StoreType: "filer"} + } + } // Create IAM configuration iamConfig := &integration.IAMConfig{ - STS: configRoot.STS, - Policy: configRoot.Policy, - Roles: &integration.RoleStoreConfig{ - StoreType: sts.StoreTypeMemory, // Use memory store for JSON config-based setup - }, + STS: configRoot.STS, + Policy: configRoot.Policy, + Roles: roleStore, OIDCProviders: oidcProviderStore, } @@ -1315,11 +1331,7 @@ func loadIAMManagerFromConfig(configPath string, filerAddressProvider func() str } // Load roles - for _, roleDef := range configRoot.Roles { - if err := iamManager.CreateRole(context.Background(), "", roleDef.RoleName, roleDef); err != nil { - glog.Warningf("Failed to create role %s: %v", roleDef.RoleName, err) - } - } + iamManager.LoadStaticRoles(context.Background(), configRoot.Roles) glog.V(1).Infof("Loaded %d providers, %d policies and %d roles from config", len(configRoot.Providers), len(configRoot.Policies), len(configRoot.Roles)) diff --git a/weed/s3api/s3api_sts.go b/weed/s3api/s3api_sts.go index 106a32e93..813d34ba2 100644 --- a/weed/s3api/s3api_sts.go +++ b/weed/s3api/s3api_sts.go @@ -400,6 +400,9 @@ func (h *STSHandlers) handleAssumeRole(w http.ResponseWriter, r *http.Request) { identity.Name, roleArn, roleSessionName) assumesSelf := roleArn == "" + // trustedPrincipal is the caller a named role's trust policy admitted; + // issuance evaluates that trust again on the definition it binds. + var trustedPrincipal string // A named role is authorized by its trust policy, which declares which // principals may assume it, so no separate identity-side sts:AssumeRole allow @@ -415,6 +418,7 @@ func (h *STSHandlers) handleAssumeRole(w http.ResponseWriter, r *http.Request) { return } callerArn := h.callerPrincipalArn(identity) + trustedPrincipal = callerArn if err := h.iam.ValidateTrustPolicyForPrincipal(r.Context(), roleArn, callerArn); err != nil { glog.V(2).Infof("AssumeRole: %s not authorized to assume %s: %v", identity.Name, roleArn, err) h.writeSTSErrorResponse(w, r, STSErrAccessDenied, @@ -461,7 +465,13 @@ func (h *STSHandlers) handleAssumeRole(w http.ResponseWriter, r *http.Request) { } // Generate common STS components - stsCreds, assumedUser, err := h.prepareSTSCredentials(r.Context(), roleArn, roleSessionName, durationSeconds, sessionPolicyJSON, modifyClaims) + stsCreds, assumedUser, err := h.prepareSTSCredentials(r.Context(), roleArn, trustedPrincipal, roleSessionName, durationSeconds, sessionPolicyJSON, modifyClaims) + if errors.Is(err, integration.ErrTrustPolicyDenied) { + // The role was replaced after its trust check by one that does not + // trust the caller. + h.writeSTSErrorResponse(w, r, STSErrAccessDenied, fmt.Errorf("trust policy denies access")) + return + } if err != nil { h.writeSTSErrorResponse(w, r, STSErrInternalError, err) return @@ -584,7 +594,8 @@ func (h *STSHandlers) handleAssumeRoleWithLDAPIdentity(w http.ResponseWriter, r // Verify that the identity is allowed to assume the role by checking the Trust Policy // The LDAP user doesn't have identity policies, so we strictly check if the Role trusts this principal. - if err := h.iam.ValidateTrustPolicyForPrincipal(r.Context(), roleArn, ldapUserIdentity.PrincipalArn); err != nil { + trustedPrincipal := ldapUserIdentity.PrincipalArn + if err := h.iam.ValidateTrustPolicyForPrincipal(r.Context(), roleArn, trustedPrincipal); err != nil { glog.V(2).Infof("AssumeRoleWithLDAPIdentity: trust policy validation failed for %s to assume %s: %v", ldapUsername, roleArn, err) h.writeSTSErrorResponse(w, r, STSErrAccessDenied, fmt.Errorf("trust policy denies access")) return @@ -602,7 +613,13 @@ func (h *STSHandlers) handleAssumeRoleWithLDAPIdentity(w http.ResponseWriter, r claims.WithIdentityProvider("ldap", identity.UserID, identity.Provider) } - stsCreds, assumedUser, err := h.prepareSTSCredentials(r.Context(), roleArn, roleSessionName, durationSeconds, sessionPolicyJSON, modifyClaims) + stsCreds, assumedUser, err := h.prepareSTSCredentials(r.Context(), roleArn, trustedPrincipal, roleSessionName, durationSeconds, sessionPolicyJSON, modifyClaims) + if errors.Is(err, integration.ErrTrustPolicyDenied) { + // The role was replaced after its trust check by one that does not + // trust the caller. + h.writeSTSErrorResponse(w, r, STSErrAccessDenied, fmt.Errorf("trust policy denies access")) + return + } if err != nil { h.writeSTSErrorResponse(w, r, STSErrInternalError, err) return @@ -831,7 +848,14 @@ func (h *STSHandlers) handleGetFederationToken(w http.ResponseWriter, r *http.Re } // prepareSTSCredentials extracts common shared logic for credential generation -func (h *STSHandlers) prepareSTSCredentials(ctx context.Context, roleArn, roleSessionName string, +// +// principalArn is the caller whose assumption of a named role was authorized +// by its trust policy. The role is resolved once, with that trust evaluated +// again on the definition resolved, and the session's role ID, duration cap +// and embedded policies all come from that one definition: a role replaced +// under the same name between the caller's trust check and here yields no +// session rather than one for a role whose trust was never checked. +func (h *STSHandlers) prepareSTSCredentials(ctx context.Context, roleArn, principalArn, roleSessionName string, durationSeconds *int64, sessionPolicy string, modifyClaims func(*sts.STSSessionClaims)) (STSCredentials, *AssumedRoleUser, error) { duration := time.Hour @@ -843,12 +867,31 @@ func (h *STSHandlers) prepareSTSCredentials(ctx context.Context, roleArn, roleSe // A named role's MaxSessionDuration bounds the resolved duration the same // way capDurationByRole does on the SDK paths; self-assumption has no role - // definition to consult. + // definition to consult. The role's ID binds the session to this role, so + // a named role that cannot be resolved here gets no session: one issued + // without the ID would be bound to no role at all. + var roleID string + var resolvedRole *integration.RoleDefinition if h.iam != nil && h.iam.iamIntegration != nil { if roleName := utils.ExtractRoleNameFromArn(roleArn); roleName != "" { if provider, ok := h.iam.iamIntegration.(IAMManagerProvider); ok { if mgr := provider.GetIAMManager(); mgr != nil { - if roleDef, roleErr := mgr.GetRole(ctx, roleName); roleErr == nil && roleDef.MaxSessionDuration > 0 { + var roleDef *integration.RoleDefinition + var roleErr error + if principalArn != "" { + roleDef, roleErr = mgr.ResolveRoleForPrincipal(ctx, roleArn, principalArn) + } else { + roleDef, roleErr = mgr.GetRole(ctx, roleName) + } + if roleErr != nil { + return STSCredentials{}, nil, fmt.Errorf("resolve role %s: %w", roleName, roleErr) + } + if roleDef == nil { + return STSCredentials{}, nil, fmt.Errorf("role %s not found", roleName) + } + resolvedRole = roleDef + roleID = roleDef.RoleId + if roleDef.MaxSessionDuration > 0 { if roleMax := time.Duration(roleDef.MaxSessionDuration) * time.Second; duration > roleMax { duration = roleMax } @@ -894,7 +937,8 @@ func (h *STSHandlers) prepareSTSCredentials(ctx context.Context, roleArn, roleSe // This ensures that subsequent requests using this token are correctly identified as the assumed role. claims := sts.NewSTSSessionClaims(sessionId, h.stsService.Config.Issuer, expiration). WithSessionName(roleSessionName). - WithRoleInfo(effectiveRoleArn, fmt.Sprintf("%s:%s", roleName, roleSessionName), assumedRoleArn) + WithRoleInfo(effectiveRoleArn, fmt.Sprintf("%s:%s", roleName, roleSessionName), assumedRoleArn). + WithRoleId(roleID) // If IAM integration is available, embed the role's attached policies into the session token. // This makes the token self-sufficient for authorization even when role lookup is unavailable. @@ -912,7 +956,10 @@ func (h *STSHandlers) prepareSTSCredentials(ctx context.Context, roleArn, roleSe } if roleNameForPolicies != "" && len(claims.Policies) == 0 { - roleDef, err := policyManager.GetRole(ctx, roleNameForPolicies) + roleDef, err := resolvedRole, error(nil) + if roleDef == nil || roleDef.RoleName != roleNameForPolicies { + roleDef, err = policyManager.GetRole(ctx, roleNameForPolicies) + } if err != nil { glog.V(2).Infof("Failed to load role %q for policy embedding: %v", roleNameForPolicies, err) } else if roleDef == nil { diff --git a/weed/s3api/s3api_sts_assume_role_test.go b/weed/s3api/s3api_sts_assume_role_test.go index d4329ec0f..af7f62718 100644 --- a/weed/s3api/s3api_sts_assume_role_test.go +++ b/weed/s3api/s3api_sts_assume_role_test.go @@ -78,7 +78,7 @@ func TestAssumeRole_CallerIdentityFallback(t *testing.T) { } } - stsCreds, assumedUser, err := stsHandlers.prepareSTSCredentials(context.Background(), fallbackRoleArn, "test-session", nil, "", modifyClaims) + stsCreds, assumedUser, err := stsHandlers.prepareSTSCredentials(context.Background(), fallbackRoleArn, "", "test-session", nil, "", modifyClaims) require.NoError(t, err) // Assertions @@ -110,7 +110,7 @@ func TestAssumeRole_CallerIdentityFallback(t *testing.T) { fallbackRoleArn := callerIdentity.PrincipalArn - stsCreds, assumedUser, err := stsHandlers.prepareSTSCredentials(context.Background(), fallbackRoleArn, "nested-session", nil, "", nil) + stsCreds, assumedUser, err := stsHandlers.prepareSTSCredentials(context.Background(), fallbackRoleArn, "", "nested-session", nil, "", nil) require.NoError(t, err) // The role name should be extracted from the assumed role ARN ("admin") @@ -127,7 +127,7 @@ func TestAssumeRole_CallerIdentityFallback(t *testing.T) { t.Run("Explicit RoleArn Provided", func(t *testing.T) { explicitRoleArn := "arn:aws:iam::111122223333:role/TargetRole" - stsCreds, assumedUser, err := stsHandlers.prepareSTSCredentials(context.Background(), explicitRoleArn, "explicit-session", nil, "", nil) + stsCreds, assumedUser, err := stsHandlers.prepareSTSCredentials(context.Background(), explicitRoleArn, "", "explicit-session", nil, "", nil) require.NoError(t, err) // Role name should be "TargetRole" @@ -143,7 +143,7 @@ func TestAssumeRole_CallerIdentityFallback(t *testing.T) { t.Run("Malformed ARN", func(t *testing.T) { malformedArn := "invalid-arn" - stsCreds, assumedUser, err := stsHandlers.prepareSTSCredentials(context.Background(), malformedArn, "bad-session", nil, "", nil) + stsCreds, assumedUser, err := stsHandlers.prepareSTSCredentials(context.Background(), malformedArn, "", "bad-session", nil, "", nil) require.NoError(t, err) // Fallback behavior: use full string as role name if extraction fails @@ -187,7 +187,7 @@ func TestAssumeRole_EmbedsRolePolicies(t *testing.T) { stsHandlers := NewSTSHandlers(manager.GetSTSService(), iam) roleArn := fmt.Sprintf("arn:aws:iam::%s:role/%s", defaultAccountID, roleName) - stsCreds, _, err := stsHandlers.prepareSTSCredentials(ctx, roleArn, "test-session", nil, "", nil) + stsCreds, _, err := stsHandlers.prepareSTSCredentials(ctx, roleArn, "", "test-session", nil, "", nil) require.NoError(t, err) sessionInfo, err := manager.GetSTSService().ValidateSessionToken(ctx, stsCreds.SessionToken) @@ -211,7 +211,7 @@ func TestAssumeRole_EmbedsRolePolicies(t *testing.T) { stsHandlers := NewSTSHandlers(manager.GetSTSService(), iam) roleArn := fmt.Sprintf("arn:aws:iam::%s:role/%s", defaultAccountID, roleName) - stsCreds, _, err := stsHandlers.prepareSTSCredentials(ctx, roleArn, "test-session", nil, "", nil) + stsCreds, _, err := stsHandlers.prepareSTSCredentials(ctx, roleArn, "", "test-session", nil, "", nil) require.NoError(t, err) sessionInfo, err := manager.GetSTSService().ValidateSessionToken(ctx, stsCreds.SessionToken) @@ -257,7 +257,7 @@ func TestPrepareSTSCredentialsHonorsConfiguredDurations(t *testing.T) { roleArn := fmt.Sprintf("arn:aws:iam::%s:role/test-role", defaultAccountID) expiresIn := func(durationSeconds *int64) time.Duration { - stsCreds, _, err := stsHandlers.prepareSTSCredentials(context.Background(), roleArn, "test-session", durationSeconds, "", nil) + stsCreds, _, err := stsHandlers.prepareSTSCredentials(context.Background(), roleArn, "", "test-session", durationSeconds, "", nil) require.NoError(t, err) exp, err := time.Parse(time.RFC3339, stsCreds.Expiration) require.NoError(t, err) @@ -291,7 +291,7 @@ func TestPrepareSTSCredentialsCapsAtRoleMaxDuration(t *testing.T) { roleArn := fmt.Sprintf("arn:aws:iam::%s:role/ShortLivedRole", defaultAccountID) expiresIn := func(durationSeconds *int64) time.Duration { - stsCreds, _, err := stsHandlers.prepareSTSCredentials(ctx, roleArn, "test-session", durationSeconds, "", nil) + stsCreds, _, err := stsHandlers.prepareSTSCredentials(ctx, roleArn, "", "test-session", durationSeconds, "", nil) require.NoError(t, err) exp, err := time.Parse(time.RFC3339, stsCreds.Expiration) require.NoError(t, err) @@ -302,3 +302,39 @@ func TestPrepareSTSCredentialsCapsAtRoleMaxDuration(t *testing.T) { assert.InDelta(t, float64(3600), expiresIn(nil).Seconds(), 60, "omitted duration resolves to the 2h default but the role caps it at 1h") assert.InDelta(t, float64(3600), expiresIn(&twoHours).Seconds(), 60, "explicit duration above the role max is capped") } + +// A session is issued from the role definition whose trust admits the +// caller. A role replaced after the caller's trust check by one that does +// not trust the caller yields no session, not one bound to the replacement. +func TestPrepareSTSCredentialsChecksTrustOnTheRoleItBinds(t *testing.T) { + ctx := context.Background() + manager := newTestSTSIntegrationManager(t) + trusting := func(principal string) *policy.PolicyDocument { + return &policy.PolicyDocument{Version: "2012-10-17", Statement: []policy.Statement{{ + Effect: "Allow", Action: []string{"sts:AssumeRole"}, + Principal: map[string]interface{}{"AWS": principal}, + }}} + } + caller := fmt.Sprintf("arn:aws:iam::%s:user/alice", defaultAccountID) + roleName := "ReplacedRole" + roleArn := fmt.Sprintf("arn:aws:iam::%s:role/%s", defaultAccountID, roleName) + stsHandlers := NewSTSHandlers(manager.GetSTSService(), &IdentityAccessManagement{iamIntegration: NewS3IAMIntegration(manager, "")}) + + // The caller's trust check passed against a role since replaced by one + // trusting someone else. + require.NoError(t, manager.CreateRole(ctx, "", roleName, &integration.RoleDefinition{ + RoleName: roleName, TrustPolicy: trusting(fmt.Sprintf("arn:aws:iam::%s:user/bob", defaultAccountID)), + })) + _, _, err := stsHandlers.prepareSTSCredentials(ctx, roleArn, caller, "s", nil, "", nil) + require.ErrorIs(t, err, integration.ErrTrustPolicyDenied, "a session was issued for a role that does not trust the caller") + + // A replacement that does trust the caller binds its own ID. + require.NoError(t, manager.CreateRole(ctx, "", roleName, &integration.RoleDefinition{RoleName: roleName, TrustPolicy: trusting(caller)})) + role, err := manager.GetRole(ctx, roleName) + require.NoError(t, err) + creds, _, err := stsHandlers.prepareSTSCredentials(ctx, roleArn, caller, "s", nil, "", nil) + require.NoError(t, err) + session, err := manager.GetSTSService().ValidateSessionToken(ctx, creds.SessionToken) + require.NoError(t, err) + assert.Equal(t, role.RoleId, session.RoleId) +}