at-container-registry/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

ATCR (ATProto Container Registry) is an OCI-compliant container registry that uses the AT Protocol for manifest storage and S3 for blob storage. This creates a decentralized container registry where manifests are stored in users' Personal Data Servers (PDS) while layers are stored in S3.

Build Commands

# Build all binaries
# create go builds in the bin/ directory
go build -o bin/atcr-appview ./cmd/appview
go build -o bin/atcr-hold ./cmd/hold
go build -o bin/docker-credential-atcr ./cmd/credential-helper

# Run tests
go test ./...

# Run with race detector
go test -race ./...

# Update dependencies
go mod tidy

# Build Docker images
docker build -t atcr.io/appview:latest .
docker build -f Dockerfile.hold -t atcr.io/hold:latest .

# Or use docker-compose
docker-compose up -d

# Run locally (AppView)
export ATPROTO_DID=did:plc:your-did
export ATPROTO_ACCESS_TOKEN=your-token
./atcr-appview serve config/config.yml

# Run hold service (configure via env vars - see .env.example)
export HOLD_PUBLIC_URL=http://127.0.0.1:8080
export STORAGE_DRIVER=filesystem
export STORAGE_ROOT_DIR=/tmp/atcr-hold
export HOLD_OWNER=did:plc:your-did-here
./atcr-hold
# Check logs for OAuth URL, visit in browser to complete registration

Architecture Overview

Core Design

ATCR uses distribution/distribution as a library and extends it through middleware to route different types of content to different storage backends:

• Manifests → ATProto PDS (small JSON metadata, stored as io.atcr.manifest records)
• Blobs/Layers → S3 or user-deployed storage (large binary data)
• Authentication → ATProto OAuth with DPoP + Docker credential helpers

Three-Component Architecture

1. AppView (cmd/appview) - OCI Distribution API server

   • Resolves identities (handle/DID → PDS endpoint)
   • Routes manifests to user's PDS
   • Routes blobs to storage endpoint (default or BYOS)
   • Validates OAuth tokens via PDS
   • Issues registry JWTs

2. Hold Service (cmd/hold) - Optional BYOS component

   • Lightweight HTTP server for presigned URLs
   • Supports S3, Storj, Minio, filesystem, etc.
   • Authorization based on PDS records (hold.public, crew records)
   • Auto-registration via OAuth
   • Configured entirely via environment variables

3. Credential Helper (cmd/credential-helper) - Client-side OAuth

   • Implements Docker credential helper protocol
   • ATProto OAuth flow with DPoP
   • Token caching and refresh
   • Exchanges OAuth token for registry JWT

Request Flow

Push with Default Storage

1. Client: docker push atcr.io/alice/myapp:latest
2. HTTP Request → /v2/alice/myapp/manifests/latest
3. Registry Middleware (pkg/middleware/registry.go)
   → Resolves "alice" to DID and PDS endpoint
   → Queries alice's sailor profile for defaultHold
   → If not set, checks alice's io.atcr.hold records
   → Falls back to AppView's default_storage_endpoint
   → Stores DID/PDS/storage endpoint in context
4. Repository Middleware (pkg/middleware/repository.go)
   → Creates RoutingRepository
   → Returns ATProto ManifestStore for manifests
   → Returns ProxyBlobStore for blobs
5. Blob PUT → Resolved hold service (redirects to S3/storage)
6. Manifest PUT → alice's PDS as io.atcr.manifest record (includes holdEndpoint)

Push with BYOS (Bring Your Own Storage)

1. Client: docker push atcr.io/alice/myapp:latest
2. Registry Middleware resolves alice → did:plc:alice123
3. Hold discovery via findStorageEndpoint():
   a. Check alice's sailor profile for defaultHold
   b. If not set, check alice's io.atcr.hold records
   c. Fall back to AppView's default_storage_endpoint
4. Found: alice's profile has defaultHold = "https://alice-storage.fly.dev"
5. Routing Repository returns ProxyBlobStore(alice-storage.fly.dev)
6. ProxyBlobStore calls alice-storage.fly.dev for presigned URL
7. Storage service validates alice's DID, generates S3 presigned URL
8. Client redirected to upload blob directly to alice's S3/Storj
9. Manifest stored in alice's PDS with holdEndpoint = "https://alice-storage.fly.dev"

Pull Flow

1. Client: docker pull atcr.io/alice/myapp:latest
2. GET /v2/alice/myapp/manifests/latest
3. AppView fetches manifest from alice's PDS
4. Manifest contains holdEndpoint = "https://alice-storage.fly.dev"
5. Hold endpoint cached: (alice's DID, "myapp") → "https://alice-storage.fly.dev"
6. Client requests blobs: GET /v2/alice/myapp/blobs/sha256:abc123
7. AppView checks cache, routes to hold from manifest (not re-discovered)
8. ProxyBlobStore calls alice-storage.fly.dev for presigned download URL
9. Client redirected to download blob directly from alice's S3

Key insight: Pull uses the historical holdEndpoint from the manifest, ensuring blobs are fetched from the hold where they were originally pushed, even if alice later changes her default hold.

Name Resolution

Names follow the pattern: atcr.io/<identity>/<image>:<tag>

Where <identity> can be:

• Handle: alice.bsky.social → resolved via .well-known/atproto-did
• DID: did:plc:xyz123 → resolved via PLC directory

Resolution happens in pkg/atproto/resolver.go:

1. Handle → DID (via DNS/HTTPS)
2. DID → PDS endpoint (via DID document)

Middleware System

ATCR uses two levels of middleware:

1. Registry Middleware (pkg/middleware/registry.go)

• Wraps distribution.Namespace
• Intercepts Repository(name) calls
• Performs name resolution (alice → did:plc:xyz → pds.example.com)
• Queries PDS for io.atcr.hold records to find storage endpoint
• Stores resolved identity and storage endpoint in context

2. Repository Middleware (pkg/middleware/repository.go)

• Wraps distribution.Repository
• Returns custom Manifests() and Blobs() implementations
• Routes manifests to ATProto, blobs to S3 or BYOS

Authentication Architecture

ATProto OAuth with DPoP

ATCR implements the full ATProto OAuth specification with mandatory security features:

Required Components:

• DPoP (RFC 9449) - Cryptographic proof-of-possession for every request
• PAR (RFC 9126) - Pushed Authorization Requests for server-to-server parameter exchange
• PKCE (RFC 7636) - Proof Key for Code Exchange to prevent authorization code interception

Key Components (pkg/auth/oauth/):

1. Client (client.go) - Core OAuth client with encapsulated configuration

   • Constructor: NewClient(baseURL) - accepts base URL, derives client ID/redirect URI
   • NewClientWithKey(baseURL, dpopKey) - for token refresh with stored DPoP key
   • ClientID() - computes localhost vs production client ID dynamically
   • RedirectURI() - returns baseURL + "/auth/oauth/callback"
   • GetDefaultScopes() - returns ATCR registry scopes
   • All OAuth flows (authorization, token exchange, refresh) in one place

2. DPoP Transport (transport.go) - HTTP RoundTripper that auto-adds DPoP headers

3. Token Storage (tokenstorage.go) - Persists refresh tokens and DPoP keys for AppView

   • File-based storage in /var/lib/atcr/refresh-tokens.json (AppView)
   • Client uses ~/.atcr/oauth-token.json (credential helper)

4. Refresher (refresher.go) - Token refresh manager for AppView

   • Caches access tokens with automatic refresh
   • Per-DID locking prevents concurrent refresh races
   • Uses Client methods for consistency

5. Server (server.go) - OAuth authorization endpoints for AppView

   • GET /auth/oauth/authorize - starts OAuth flow
   • GET /auth/oauth/callback - handles OAuth callback
   • Uses Client methods for authorization and token exchange

6. Interactive Flow (flow.go) - Reusable OAuth flow for CLI tools

   • Used by credential helper and hold service registration
   • Two-phase callback setup ensures PAR metadata availability

Authentication Flow:

1. User runs: docker-credential-atcr configure
2. Helper generates ECDSA P-256 DPoP key
3. Resolve handle → DID → PDS endpoint
4. Discover OAuth server metadata from PDS
5. PAR request with DPoP header → get request_uri
6. Open browser for user authorization
7. Exchange code for token with DPoP proof
8. Save: access token, refresh token, DPoP key, DID, handle

Later (docker push):
9. Docker calls credential helper
10. Helper loads token, refreshes if needed
11. Helper calls /auth/exchange with OAuth token + handle
12. AppView validates token via PDS getSession
13. AppView ensures sailor profile exists (creates with defaultHold if first login)
14. AppView issues registry JWT with validated DID
15. Helper returns JWT to Docker

Security:

• Tokens validated against authoritative source (user's PDS)
• No trust in client-provided identity information
• DPoP binds tokens to specific client key
• 15-minute token expiry for registry JWTs

Key Components

ATProto Integration (pkg/atproto/)

resolver.go: DID and handle resolution

• ResolveIdentity(): alice → did:plc:xyz → pds.example.com
• ResolveHandle(): Uses .well-known/atproto-did
• ResolvePDS(): Parses DID document for PDS endpoint

client.go: ATProto PDS client

• PutRecord(): Store manifest as ATProto record
• GetRecord(): Retrieve manifest from PDS
• DeleteRecord(): Remove manifest
• Uses XRPC protocol (com.atproto.repo.*)

lexicon.go: ATProto record schemas

• ManifestRecord: OCI manifest stored as ATProto record (includes holdEndpoint field)
• TagRecord: Tag pointing to manifest digest
• HoldRecord: Storage hold definition (for BYOS)
• HoldCrewRecord: Hold crew membership/permissions
• SailorProfileRecord: User profile with defaultHold preference
• Collections: io.atcr.manifest, io.atcr.tag, io.atcr.hold, io.atcr.hold.crew, io.atcr.sailor.profile

profile.go: Sailor profile management

• EnsureProfile(): Creates profile with default hold on first authentication
• GetProfile(): Retrieves user's profile from PDS
• UpdateProfile(): Updates user's profile

manifest_store.go: Implements distribution.ManifestService

• Stores OCI manifests as ATProto records
• Digest-based addressing (sha256:abc123 → record key)
• Converts between OCI and ATProto formats

Storage Layer (pkg/storage/)

routing_repository.go: Routes content by type

• Manifests() → returns ATProto ManifestStore (caches instance for hold endpoint extraction)
• Blobs() → checks hold cache for pull, uses discovery for push
  • Pull: Uses cached holdEndpoint from manifest (historical reference)
  • Push: Uses discovery-based endpoint from findStorageEndpoint()
  • Always returns ProxyBlobStore (routes to hold service)
• Implements distribution.Repository interface

hold_cache.go: In-memory hold endpoint cache

• Caches (DID, repository) → holdEndpoint for pull operations
• TTL: 10 minutes (covers typical pull operations)
• Cleanup: Background goroutine runs every 5 minutes
• NOTE: Simple in-memory cache for MVP. For production: use Redis or similar
• Prevents expensive ATProto lookups on every blob request

s3_blob_store.go: S3 blob storage wrapper

• Wraps distribution's built-in S3 driver
• Inherits full distribution.BlobStore interface
• Used for default shared storage

proxy_blob_store.go: External storage proxy

• Calls user's storage service for presigned URLs
• Issues HTTP redirects for blob uploads/downloads
• Implements full distribution.BlobStore interface
• Used when user has io.atcr.hold record

Hold Service (cmd/hold/)

Lightweight standalone service for BYOS (Bring Your Own Storage):

Architecture:

• Reuses distribution's storage driver factory
• Supports all distribution drivers: S3, Storj, Minio, Azure, GCS, filesystem
• Authorization follows ATProto's public-by-default model
• Generates presigned URLs (15min expiry) or proxies uploads/downloads

Authorization Model:

Read access:

• Public hold (HOLD_PUBLIC=true): Anonymous + all authenticated users
• Private hold (HOLD_PUBLIC=false): Authenticated users only (any ATCR user)

Write access:

• Hold owner OR crew members only
• Verified via io.atcr.hold.crew records in owner's PDS

Key insight: "Private" gates anonymous access, not authenticated access. This reflects ATProto's current limitation (no private PDS records yet).

Endpoints:

• POST /get-presigned-url - Get download URL for blob
• POST /put-presigned-url - Get upload URL for blob
• GET /blobs/{digest} - Proxy download (fallback if no presigned URL support)
• PUT /blobs/{digest} - Proxy upload (fallback)
• POST /register - Manual registration endpoint
• GET /health - Health check

Configuration: Environment variables (see .env.example)

• HOLD_PUBLIC_URL - Public URL of hold service (required)
• STORAGE_DRIVER - Storage driver type (s3, filesystem)
• AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY - S3 credentials
• S3_BUCKET, S3_ENDPOINT - S3 configuration
• HOLD_PUBLIC - Allow public reads (default: false)
• HOLD_OWNER - DID for auto-registration (optional)

Deployment: Can run on Fly.io, Railway, Docker, Kubernetes, etc.

ATProto Storage Model

Manifests are stored as records with this structure:

{
  "$type": "io.atcr.manifest",
  "repository": "myapp",
  "digest": "sha256:abc123...",
  "holdEndpoint": "https://hold1.alice.com",
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "config": { "digest": "sha256:...", "size": 1234 },
  "layers": [
    { "digest": "sha256:...", "size": 5678 }
  ],
  "createdAt": "2025-09-30T..."
}

Record key = manifest digest (without algorithm prefix) Collection = io.atcr.manifest

Sailor Profile System

ATCR uses a "sailor profile" to manage user preferences for hold (storage) selection. The nautical theme reflects the architecture:

• Sailors = Registry users
• Captains = Hold owners
• Crew = Hold members with access
• Holds = Storage endpoints (BYOS)

Profile Record (io.atcr.sailor.profile):

{
  "$type": "io.atcr.sailor.profile",
  "defaultHold": "https://hold1.alice.com",
  "createdAt": "2025-10-02T...",
  "updatedAt": "2025-10-02T..."
}

Profile Management:

• Created automatically on first authentication (OAuth or Basic Auth)
• If AppView has default_storage_endpoint configured, profile gets that as defaultHold
• Users can update their profile to change default hold (future: via UI)
• Setting defaultHold to null opts out of defaults (use own holds or AppView default)

Hold Resolution Priority (in findStorageEndpoint()):

1. Profile's defaultHold - User's explicit preference
2. User's io.atcr.hold records - User's own holds
3. AppView's default_storage_endpoint - Fallback default

This ensures:

• Users can join shared holds by setting their profile's defaultHold
• Users can opt out of defaults (set defaultHold to null)
• URL structure remains atcr.io/<owner>/<image> (ownership-based, not hold-based)
• Hold choice is transparent infrastructure (like choosing an S3 region)

Key Design Decisions

1. No fork of distribution: Uses distribution as library, extends via middleware
2. Hybrid storage: Manifests in ATProto (small, federated), blobs in S3 or BYOS (cheap, scalable)
3. Content addressing: Manifests stored by digest, blobs deduplicated globally
4. ATProto-native: Manifests are first-class ATProto records, discoverable via AT Protocol
5. OCI compliant: Fully compatible with Docker/containerd/podman
6. Account-agnostic AppView: Server validates any user's token, queries their PDS for config
7. BYOS architecture: Users can deploy their own storage service, AppView just routes
8. OAuth with DPoP: Full ATProto OAuth implementation with mandatory DPoP proofs
9. Sailor profile system: User preferences for hold selection, transparent to image ownership
10. Historical hold references: Manifests store holdEndpoint for immutable blob location tracking

Configuration

AppView configuration (config/config.yml):

• S3 bucket settings under storage.s3
• ATProto middleware under middleware.repository
• Name resolver under middleware.registry
• Default storage endpoint: middleware.registry.options.default_storage_endpoint
• Auth token signing keys and expiration

Hold Service configuration (environment variables):

• Storage driver config via env vars: STORAGE_DRIVER, AWS_*, S3_*
• Authorization: Based on PDS records (hold.public, crew records)
• Server settings: HOLD_SERVER_ADDR, HOLD_PUBLIC_URL, HOLD_PUBLIC
• Auto-registration: HOLD_OWNER (optional)

Credential Helper:

• Token storage: ~/.atcr/oauth-token.json
• Contains: access token, refresh token, DPoP key (PEM), DID, handle

Environment variables:

• ATPROTO_DID: DID for authentication with PDS (AppView only)
• ATPROTO_ACCESS_TOKEN: Access token for PDS operations (AppView only)

Development Notes

General:

• Middleware is registered via init() functions in pkg/middleware/
• Import _ "atcr.io/pkg/middleware" in main.go to register middleware
• Storage drivers imported as _ "github.com/distribution/distribution/v3/registry/storage/driver/s3-aws"
• Storage service reuses distribution's driver factory for multi-backend support

OAuth implementation:

• Client (pkg/auth/oauth/client.go) encapsulates all OAuth configuration
• Uses authelia.com/client/oauth2 for PAR support
• DPoP proofs generated with github.com/AxisCommunications/go-dpop (auto-handles JWK)
• Token validation via com.atproto.server.getSession ensures no trust in client-provided identity
• All ATCR components use standardized /auth/oauth/callback path
• Client ID generation (localhost query-based vs production metadata URL) handled internally

Testing Strategy

When writing tests:

• Mock ATProto client for manifest operations
• Mock S3 driver for blob operations
• Test name resolution independently
• Integration tests require real PDS + S3

Common Tasks

Adding a new ATProto record type:

1. Define schema in pkg/atproto/lexicon.go
2. Add collection constant (e.g., MyCollection = "io.atcr.my-type")
3. Add constructor function (e.g., NewMyRecord())
4. Update client methods if needed

Modifying storage routing:

1. Edit pkg/storage/routing_repository.go
2. Update Blobs() method to change routing logic
3. Consider context values: storage.endpoint, atproto.did

Changing name resolution:

1. Modify pkg/atproto/resolver.go for DID/handle resolution
2. Update pkg/middleware/registry.go if changing routing logic
3. Remember: findStorageEndpoint() queries PDS for io.atcr.hold records

Working with OAuth client:

• Client is self-contained: pass baseURL, it handles client ID/redirect URI/scopes
• For AppView server/refresher: use NewClient(baseURL) or NewClientWithKey(baseURL, storedKey)
• For custom scopes: call client.SetScopes(customScopes) after initialization
• Standard callback path: /auth/oauth/callback (used by all ATCR components)
• Client methods are consistent across authorization, token exchange, and refresh flows

Adding BYOS support for a user:

1. User sets environment variables (storage credentials, public URL)
2. User runs hold service with HOLD_OWNER set - auto-registration via OAuth
3. Hold service creates io.atcr.hold + io.atcr.hold.crew records in PDS
4. AppView automatically queries PDS and routes blobs to user's storage
5. No AppView changes needed - fully decentralized

Supporting a new storage backend:

1. Ensure driver is registered in cmd/hold/main.go imports
2. Distribution supports: S3, Azure, GCS, Swift, filesystem, OSS
3. For custom drivers: implement storagedriver.StorageDriver interface
4. Add case to buildStorageConfig() in cmd/hold/main.go
5. Update .env.example with new driver's env vars

Important Context Values

When working with the codebase, these context values are used for routing:

• atproto.did - Resolved DID for the user (e.g., did:plc:alice123)
• atproto.pds - User's PDS endpoint (e.g., https://bsky.social)
• atproto.identity - Original identity string (handle or DID)
• storage.endpoint - Storage service URL (if user has io.atcr.registry record)
• auth.did - Authenticated DID from validated token

Documentation References

• BYOS Architecture: See docs/BYOS.md for complete BYOS documentation
• OAuth Implementation: See docs/OAUTH.md for OAuth/DPoP flow details
• ATProto Spec: https://atproto.com/specs/oauth
• OCI Distribution Spec: https://github.com/opencontainers/distribution-spec
• DPoP RFC: https://datatracker.ietf.org/doc/html/rfc9449
• PAR RFC: https://datatracker.ietf.org/doc/html/rfc9126
• PKCE RFC: https://datatracker.ietf.org/doc/html/rfc7636