docs(sts): add comprehensive runtime filer address documentation

- Document the complete refactoring rationale and implementation - Provide before/after code examples and usage patterns - Include migration guide for existing code - Detail production deployment strategies - Show dynamic filer selection, failover, and load balancing examples - Explain memory store compatibility and interface consistency - Demonstrate environment-agnostic configuration benefits
2026-08-19 05:36:58 +00:00 · 2025-08-24 14:12:17 -07:00
parent 8718c301ba
commit eac053f7c4
1 changed files with 356 additions and 0 deletions
@@ -0,0 +1,356 @@
+# Runtime Filer Address Implementation
+
+This document describes the implementation of runtime filer address passing for the STSService, addressing the requirement that filer addresses should be passed at call-time rather than initialization time.
+
+## Problem Statement
+
+The user identified a critical issue with the original STS implementation:
+
+> "the filer address should be passed when called, not during init time, since the filer may change."
+
+This is important because:
+1. **Filer Failover**: Filer addresses can change during runtime due to failover scenarios
+2. **Load Balancing**: Different requests may need to hit different filer instances
+3. **Environment Agnostic**: Configuration files should work across dev/staging/prod without hardcoded addresses
+4. **SeaweedFS Patterns**: Follows existing SeaweedFS patterns used throughout the codebase
+
+## Implementation Changes
+
+### 1. SessionStore Interface Refactoring
+
+**Before:**
+```go
+type SessionStore interface {
+    StoreSession(ctx context.Context, sessionId string, session *SessionInfo) error
+    GetSession(ctx context.Context, sessionId string) (*SessionInfo, error)
+    RevokeSession(ctx context.Context, sessionId string) error
+    CleanupExpiredSessions(ctx context.Context) error
+}
+```
+
+**After:**
+```go
+type SessionStore interface {
+    // filerAddress ignored for memory stores, required for filer stores
+    StoreSession(ctx context.Context, filerAddress string, sessionId string, session *SessionInfo) error
+    GetSession(ctx context.Context, filerAddress string, sessionId string) (*SessionInfo, error)
+    RevokeSession(ctx context.Context, filerAddress string, sessionId string) error
+    CleanupExpiredSessions(ctx context.Context, filerAddress string) error
+}
+```
+
+### 2. FilerSessionStore Changes
+
+**Before:**
+```go
+type FilerSessionStore struct {
+    filerGrpcAddress string  // ❌ Fixed at init time
+    grpcDialOption   grpc.DialOption
+    basePath         string
+}
+
+func NewFilerSessionStore(filerAddress string, config map[string]interface{}) (*FilerSessionStore, error) {
+    store := &FilerSessionStore{
+        filerGrpcAddress: filerAddress,  // ❌ Locked in during init
+        basePath:         DefaultSessionBasePath,
+    }
+    // ...
+}
+```
+
+**After:**
+```go
+type FilerSessionStore struct {
+    grpcDialOption grpc.DialOption  // ✅ No fixed filer address
+    basePath       string
+}
+
+func NewFilerSessionStore(config map[string]interface{}) (*FilerSessionStore, error) {
+    store := &FilerSessionStore{
+        basePath: DefaultSessionBasePath,  // ✅ Only path configuration
+    }
+    // ✅ filerAddress passed at call time
+}
+
+func (f *FilerSessionStore) StoreSession(ctx context.Context, filerAddress string, sessionId string, session *SessionInfo) error {
+    // ✅ filerAddress provided per call
+    return f.withFilerClient(filerAddress, func(client filer_pb.SeaweedFilerClient) error {
+        // ... store logic
+    })
+}
+```
+
+### 3. STS Service Method Signatures
+
+**Before:**
+```go
+func (s *STSService) AssumeRoleWithWebIdentity(ctx context.Context, request *AssumeRoleWithWebIdentityRequest) (*AssumeRoleResponse, error)
+func (s *STSService) ValidateSessionToken(ctx context.Context, sessionToken string) (*SessionInfo, error)
+func (s *STSService) RevokeSession(ctx context.Context, sessionToken string) error
+```
+
+**After:**
+```go
+func (s *STSService) AssumeRoleWithWebIdentity(ctx context.Context, filerAddress string, request *AssumeRoleWithWebIdentityRequest) (*AssumeRoleResponse, error)
+func (s *STSService) ValidateSessionToken(ctx context.Context, filerAddress string, sessionToken string) (*SessionInfo, error)
+func (s *STSService) RevokeSession(ctx context.Context, filerAddress string, sessionToken string) error
+```
+
+### 4. Configuration Cleanup
+
+**Before (iam_config_distributed.json):**
+```json
+{
+  "sts": {
+    "sessionStoreConfig": {
+      "filerAddress": "localhost:8888",  // ❌ Environment-specific
+      "basePath": "/etc/iam/sessions"
+    }
+  },
+  "policy": {
+    "storeConfig": {
+      "filerAddress": "localhost:8888",  // ❌ Environment-specific
+      "basePath": "/etc/iam/policies"
+    }
+  }
+}
+```
+
+**After (iam_config_distributed.json):**
+```json
+{
+  "sts": {
+    "sessionStoreConfig": {
+      "basePath": "/etc/iam/sessions"    // ✅ Environment-agnostic
+    }
+  },
+  "policy": {
+    "storeConfig": {
+      "basePath": "/etc/iam/policies"    // ✅ Environment-agnostic
+    }
+  }
+}
+```
+
+## Usage Examples
+
+### Caller Perspective (S3 API Server)
+
+**Before:**
+```go
+// STS service locked to specific filer during init
+stsService.Initialize(&STSConfig{
+    SessionStoreConfig: map[string]interface{}{
+        "filerAddress": "filer-1:8888",  // ❌ Fixed choice
+        "basePath": "/etc/iam/sessions",
+    },
+})
+
+// All calls go to filer-1, no failover possible
+response, err := stsService.AssumeRoleWithWebIdentity(ctx, request)
+```
+
+**After:**
+```go
+// STS service configured without specific filer
+stsService.Initialize(&STSConfig{
+    SessionStoreConfig: map[string]interface{}{
+        "basePath": "/etc/iam/sessions",  // ✅ Just the path
+    },
+})
+
+// Caller determines filer address per request
+currentFiler := s.getCurrentFilerAddress()  // ✅ Dynamic selection
+response, err := stsService.AssumeRoleWithWebIdentity(ctx, currentFiler, request)
+```
+
+### Dynamic Filer Selection
+
+```go
+type S3ApiServer struct {
+    stsService   *sts.STSService
+    filerClient  *filer.Client
+}
+
+func (s *S3ApiServer) getCurrentFilerAddress() string {
+    // ✅ Can implement any strategy:
+    // - Load balancing across multiple filers
+    // - Health checking and failover
+    // - Geographic routing
+    // - Round-robin selection
+    return s.filerClient.GetAvailableFiler()
+}
+
+func (s *S3ApiServer) handleAssumeRole(ctx context.Context, request *AssumeRoleRequest) {
+    // ✅ Filer address determined at request time
+    filerAddr := s.getCurrentFilerAddress()
+    
+    response, err := s.stsService.AssumeRoleWithWebIdentity(ctx, filerAddr, request)
+    if err != nil && isNetworkError(err) {
+        // ✅ Retry with different filer
+        filerAddr = s.getBackupFilerAddress()
+        response, err = s.stsService.AssumeRoleWithWebIdentity(ctx, filerAddr, request)
+    }
+}
+```
+
+## Memory Store Compatibility
+
+The `MemorySessionStore` accepts the `filerAddress` parameter but ignores it, maintaining interface consistency:
+
+```go
+func (m *MemorySessionStore) StoreSession(ctx context.Context, filerAddress string, sessionId string, session *SessionInfo) error {
+    // filerAddress ignored for memory store - maintains interface compatibility
+    if sessionId == "" {
+        return fmt.Errorf(ErrSessionIDCannotBeEmpty)
+    }
+    // ... in-memory storage logic
+}
+```
+
+## Benefits Achieved
+
+### 1. **Dynamic Filer Selection**
+```go
+// Load balancing
+filerAddr := loadBalancer.GetNextFiler()
+
+// Failover support
+filerAddr := failoverManager.GetHealthyFiler()
+
+// Geographic routing
+filerAddr := geoRouter.GetClosestFiler(clientIP)
+```
+
+### 2. **Environment Portability**
+```bash
+# Same config works everywhere
+dev:     STSService.method(ctx, "dev-filer:8888", ...)
+staging: STSService.method(ctx, "staging-filer:8888", ...)
+prod:    STSService.method(ctx, "prod-filer-lb:8888", ...)
+```
+
+### 3. **Operational Flexibility**
+- **Hot filer replacement**: Switch filers without restarting STS
+- **A/B testing**: Route different requests to different filers
+- **Disaster recovery**: Automatic failover to backup filers
+- **Performance optimization**: Route to least loaded filer
+
+### 4. **SeaweedFS Consistency**
+Follows the same pattern used throughout SeaweedFS codebase where filer addresses are passed to methods, not stored in structs.
+
+## Migration Guide
+
+### For Code Calling STS Methods
+
+**Before:**
+```go
+response, err := stsService.AssumeRoleWithWebIdentity(ctx, request)
+session, err := stsService.ValidateSessionToken(ctx, token)
+err := stsService.RevokeSession(ctx, token)
+```
+
+**After:**
+```go
+filerAddr := getCurrentFilerAddress()  // Implement your strategy
+response, err := stsService.AssumeRoleWithWebIdentity(ctx, filerAddr, request)
+session, err := stsService.ValidateSessionToken(ctx, filerAddr, token)
+err := stsService.RevokeSession(ctx, filerAddr, token)
+```
+
+### For Configuration Files
+
+Remove `filerAddress` from all store configurations:
+
+```bash
+# Update all iam_config*.json files
+sed -i 's|"filerAddress": ".*",||g' iam_config*.json
+```
+
+## Testing
+
+All tests have been updated to pass a test filer address:
+
+```go
+func TestAssumeRoleWithWebIdentity(t *testing.T) {
+    service := setupTestSTSService(t)
+    testFilerAddress := "localhost:8888" // Test filer address
+    
+    response, err := service.AssumeRoleWithWebIdentity(ctx, testFilerAddress, request)
+    // ... test logic
+}
+```
+
+## Production Deployment
+
+### High Availability Setup
+
+```go
+type FilerManager struct {
+    primaryFilers   []string
+    backupFilers    []string
+    healthChecker   *HealthChecker
+}
+
+func (fm *FilerManager) GetAvailableFiler() string {
+    // Check primary filers first
+    for _, filer := range fm.primaryFilers {
+        if fm.healthChecker.IsHealthy(filer) {
+            return filer
+        }
+    }
+    
+    // Fallback to backup filers
+    for _, filer := range fm.backupFilers {
+        if fm.healthChecker.IsHealthy(filer) {
+            return filer
+        }
+    }
+    
+    // Return first primary as last resort
+    return fm.primaryFilers[0]
+}
+```
+
+### Load Balanced Configuration
+
+```json
+{
+  "sts": {
+    "sessionStoreType": "filer",
+    "sessionStoreConfig": {
+      "basePath": "/etc/iam/sessions"
+    }
+  }
+}
+```
+
+```go
+// Runtime filer selection
+filerLoadBalancer := &RoundRobinBalancer{
+    Filers: []string{
+        "filer-1.prod:8888",
+        "filer-2.prod:8888", 
+        "filer-3.prod:8888",
+    },
+}
+
+response, err := stsService.AssumeRoleWithWebIdentity(
+    ctx, 
+    filerLoadBalancer.Next(),  // ✅ Dynamic selection
+    request,
+)
+```
+
+## Conclusion
+
+This refactoring successfully addresses the requirement for runtime filer address passing, enabling:
+
+- ✅ **Dynamic filer selection** per request
+- ✅ **Automatic failover** capabilities  
+- ✅ **Environment-agnostic** configurations
+- ✅ **Load balancing** support
+- ✅ **SeaweedFS pattern** compliance
+- ✅ **Operational flexibility** for production deployments
+
+The implementation maintains backward compatibility for memory stores while enabling powerful distributed deployment scenarios for filer-backed stores.