Files
at-container-registry/pkg/appview/config.go
T
Evan Jarrett ab4a4ebf9d admin panel long running imrovements, billing fixes, ui cleanup
1. Multiple registry domains + per-user domain preference

The biggest feature. The appview can serve several registry domains (e.g. buoy.cr, atcr.io),
and users can now pick which one shows up in their pull/push commands.

- Lexicon/record: adds registryDomain (and documents ociClient)
to the sailor profile (lexicons/.../profile.json, pkg/atproto/lexicon.go).
- DB: new registry_domain column on users (schema.sql + migration 0027),
with GetUserByDID/Handle reads, UpdateUserRegistryDomain writer,
and Jetstream caching it on profile updates (writes unconditionally so clearing propagates).
- UI/handlers: new UpdateRegistryDomainHandler + /api/profile/registry-domain route,
a <select> in the user settings panel (only shown when >1 domain configured), and resolveRegistryURL()
which falls back to the primary domain if the user's pref is stale/removed. Tests added for all of it.

2. default_hold_did removed → first managed_holds entry is the default

Consolidates two overlapping config fields into one. ServerConfig.DefaultHoldDID is gone;
PrimaryHoldDID() now returns managed_holds[0]. managed_holds is now REQUIRED.
Updated in config, validation, server wiring, test harness, example YAML, and the deploy template.

3. Admin long-running operations → generic background-job framework

New pkg/hold/admin/jobs.go introduces a reusable startJob/jobRegistry pattern
 (a detached context.Background() job + a /admin/api/jobs/{key}/status polling endpoint).
This replaces the bespoke scan-backfill goroutine state machine, and now also wraps crew tier remap and crew import
all three previously looped synchronously on the request context and got 504'd/cancelled mid-run by the reverse proxy.
 Forms switched from POST-redirect to htmx fragments (job_progress.html, job_result.html, crew_import_results.html)
 the old crew_import_results.html page and scan_backfill_progress.html partial were deleted.
This is also captured as a new rule in CLAUDE.md.

4. Cascade-delete manifest on last-tag deletion

DeleteTagHandler now, after removing the last tag pointing to a digest, cascade-deletes the manifest itself
 (PDS + DB + hold blob purge) — but only if it's not a child of a manifest list (multi-arch parent).
 New GetTagDigest and ShouldCascadeDeleteManifest queries back it, plus cascade_delete_test.go.
 Also switches tag rkey computation to the atproto.RepositoryTagToRKey helper.

5. Billing simplification

Drops the OwnerBadge config option (hold-owner supporter badge).
The user-profile template no longer special-cases an "owner" badge value (only "Captain").
Example tiers renamed to the nautical scheme (deckhand/bosun/quartermaster).

6. Build/deploy: go generate always runs via Make

make generate is now a phony target that always runs go generate ./... (regenerating cbor_gen, icon sprites, etc.),
 and build-trixie depends on it. The deploy tooling (provision.go/update.go)
drops its own runGenerate calls since the Makefile handles it.

7. New cmd/firehose-tap tool (untracked)

A standalone CLI that subscribes to a com.atproto.sync.subscribeRepos endpoint and pretty-prints events,
with emphasis on Sync 1.1 compliance fields (per-op prev CIDs, commit prevData) and a --validate CI mode.
Fits with the recent "more sync1.1 compliant" commit.
2026-06-05 20:57:25 -05:00

463 lines
18 KiB
Go

// Package appview implements the ATCR AppView component, which serves as the main
// OCI Distribution API server. It resolves identities (handle/DID to PDS endpoint),
// routes manifests to user's PDS, routes blobs to hold services, validates OAuth tokens,
// and issues registry JWTs. This package provides Viper-based configuration with YAML
// file support, environment variable overrides, and HTTP server setup for the AppView service.
package appview
import (
"crypto/rand"
"encoding/hex"
"fmt"
"net/url"
"os"
"strings"
"time"
"github.com/distribution/distribution/v3/configuration"
"github.com/spf13/viper"
"atcr.io/pkg/billing"
"atcr.io/pkg/config"
)
// Config represents the AppView service configuration
type Config struct {
Version string `yaml:"version" comment:"Configuration format version."`
LogLevel string `yaml:"log_level" comment:"Log level: debug, info, warn, error."`
LogShipper config.LogShipperConfig `yaml:"log_shipper" comment:"Remote log shipping settings."`
Server ServerConfig `yaml:"server" comment:"HTTP server and identity settings."`
UI UIConfig `yaml:"ui" comment:"Web UI settings."`
Health HealthConfig `yaml:"health" comment:"Health check and cache settings."`
Jetstream JetstreamConfig `yaml:"jetstream" comment:"ATProto Jetstream event stream settings."`
Auth AuthConfig `yaml:"auth" comment:"JWT authentication settings."`
CredentialHelper CredentialHelperConfig `yaml:"credential_helper" comment:"Credential helper download settings."`
Legal LegalConfig `yaml:"legal" comment:"Legal page customization for self-hosted instances."`
AI AIConfig `yaml:"ai" comment:"AI-powered image advisor settings."`
Labeler LabelerRefConfig `yaml:"labeler" comment:"ATProto labeler for content moderation (DMCA takedowns)."`
Billing billing.Config `yaml:"billing" comment:"Stripe billing integration (requires -tags billing build)."`
Distribution *configuration.Configuration `yaml:"-"` // Wrapped distribution config for compatibility
}
// ServerConfig defines server settings
type ServerConfig struct {
// Listen address for the HTTP server.
Addr string `yaml:"addr" comment:"Listen address, e.g. \":5000\" or \"127.0.0.1:5000\"."`
// Public-facing URL for OAuth callbacks and JWT realm.
BaseURL string `yaml:"base_url" comment:"Public-facing URL for OAuth callbacks and JWT realm. Auto-detected if empty."`
// Allows HTTP (not HTTPS) for DID resolution.
TestMode bool `yaml:"test_mode" comment:"Allows HTTP (not HTTPS) for DID resolution and uses transition:generic OAuth scope."`
// Display name shown on OAuth authorization screens.
ClientName string `yaml:"client_name" comment:"Display name shown on OAuth authorization screens."`
// Short name used in page titles and browser tabs.
ClientShortName string `yaml:"client_short_name" comment:"Short name used in page titles and browser tabs."`
// Separate domains for OCI registry API. First entry is the primary (used for JWT service name and UI display).
RegistryDomains []string `yaml:"registry_domains" comment:"Separate domains for OCI registry API (e.g. [\"buoy.cr\"]). First is primary. Browser visits redirect to BaseURL."`
// DIDs of holds this appview manages billing for. The first entry is also
// the default blob-storage hold (used when a user has no hold selected).
ManagedHolds []string `yaml:"managed_holds" comment:"DIDs of holds this appview manages billing for (REQUIRED). The first entry is the default blob-storage hold. Tier updates are pushed to these holds."`
}
// PrimaryHoldDID returns the appview's default blob-storage hold, which is the
// first managed hold. Empty if none configured.
func (s ServerConfig) PrimaryHoldDID() string {
if len(s.ManagedHolds) > 0 {
return s.ManagedHolds[0]
}
return ""
}
// UIConfig defines web UI settings
type UIConfig struct {
// SQLite database path.
DatabasePath string `yaml:"database_path" comment:"SQLite/libSQL database for OAuth sessions, stars, pull counts, and device approvals."`
// Visual theme name (e.g. "seamark"). Empty string uses default atcr.io branding.
Theme string `yaml:"theme" comment:"Visual theme name (e.g. \"seamark\"). Empty uses default atcr.io branding."`
// libSQL sync URL for embedded replicas. Works with Turso cloud or self-hosted libsql-server.
// Leave empty for local-only SQLite mode (selfhost/dev).
LibsqlSyncURL string `yaml:"libsql_sync_url" comment:"libSQL sync URL (libsql://...). Works with Turso cloud or self-hosted libsql-server. Leave empty for local-only SQLite."`
// Auth token for libSQL sync. Required if LibsqlSyncURL is set.
LibsqlAuthToken string `yaml:"libsql_auth_token" comment:"Auth token for libSQL sync. Required if libsql_sync_url is set."`
// How often to sync with the remote libSQL server.
LibsqlSyncInterval time.Duration `yaml:"libsql_sync_interval" comment:"How often to sync with remote libSQL server. Default: 60s."`
// Source code URL displayed in the footer "Source" link.
SourceURL string `yaml:"source_url" comment:"Source code URL displayed in the footer \"Source\" link. Defaults to the upstream ATCR project."`
}
// HealthConfig defines health check and cache settings
type HealthConfig struct {
// How long to cache hold health check results.
CacheTTL time.Duration `yaml:"cache_ttl" comment:"How long to cache hold health check results."`
// How often to refresh hold health checks.
CheckInterval time.Duration `yaml:"check_interval" comment:"How often to refresh hold health checks."`
}
// JetstreamConfig defines ATProto Jetstream settings
type JetstreamConfig struct {
// Jetstream WebSocket endpoints, tried in order on failure.
URLs []string `yaml:"urls" comment:"Jetstream WebSocket endpoints, tried in order on failure."`
// Sync existing records from PDS on startup.
BackfillEnabled bool `yaml:"backfill_enabled" comment:"Sync existing records from PDS on startup."`
// How often to re-run backfill to catch missed events. Set to 0 to only backfill on startup.
BackfillInterval time.Duration `yaml:"backfill_interval" comment:"How often to re-run backfill to catch missed events. Set to 0 to only backfill on startup."`
// Relay endpoints for backfill — MUST support com.atproto.sync.listReposByCollection. Tried in order on failure.
RelayEndpoints []string `yaml:"relay_endpoints" comment:"Endpoints used for backfill. MUST support com.atproto.sync.listReposByCollection. Tried in order on failure."`
}
// AuthConfig defines authentication settings
type AuthConfig struct {
// X.509 certificate matching the JWT signing key.
CertPath string `yaml:"cert_path" comment:"X.509 certificate matching the JWT signing key (auto-generated on each boot from the JWT key in the database)."`
// TokenExpiration is the JWT expiration duration (5 minutes, not configurable)
TokenExpiration time.Duration `yaml:"-"`
// ServiceName is the service name used for JWT issuer and service fields.
// Derived from base URL hostname (e.g., "atcr.io")
ServiceName string `yaml:"-"`
}
// CredentialHelperConfig defines credential helper download settings
type CredentialHelperConfig struct {
// TangledRepo is the Tangled repository URL for downloads
TangledRepo string `yaml:"tangled_repo" comment:"Tangled repository URL for credential helper downloads."`
}
// LegalConfig defines legal page customization for self-hosted instances
type LegalConfig struct {
// Organization name for legal pages. Defaults to ClientName.
CompanyName string `yaml:"company_name" comment:"Organization name for Terms of Service and Privacy Policy. Defaults to server.client_name."`
// Governing law jurisdiction for legal terms.
Jurisdiction string `yaml:"jurisdiction" comment:"Governing law jurisdiction for legal terms."`
}
// AIConfig defines AI-powered image advisor settings
type AIConfig struct {
// Anthropic API key for the AI Image Advisor feature.
APIKey string `yaml:"api_key" comment:"Anthropic API key for AI Image Advisor. Also reads CLAUDE_API_KEY env var as fallback."`
}
// LabelerRefConfig defines the connection to an ATProto labeler service.
type LabelerRefConfig struct {
// DID of the labeler service for content moderation. The HTTP endpoint
// is resolved at runtime via the labeler's #atproto_labeler service entry
// in its DID document (plc.directory for did:plc, /.well-known/did.json
// for did:web).
DID string `yaml:"did" comment:"DID of the ATProto labeler (did:plc:... or did:web:...). Empty disables label filtering."`
}
// setDefaults registers all default values on the given Viper instance.
func setDefaults(v *viper.Viper) {
v.SetDefault("version", "0.1")
v.SetDefault("log_level", "info")
// Server defaults
v.SetDefault("server.addr", ":5000")
v.SetDefault("server.base_url", "")
v.SetDefault("server.test_mode", false)
v.SetDefault("server.client_name", "AT Container Registry")
v.SetDefault("server.client_short_name", "ATCR")
v.SetDefault("server.registry_domains", []string{})
v.SetDefault("server.managed_holds", []string{})
// UI defaults
v.SetDefault("ui.database_path", "/var/lib/atcr/ui.db")
v.SetDefault("ui.theme", "")
v.SetDefault("ui.libsql_sync_url", "")
v.SetDefault("ui.libsql_auth_token", "")
v.SetDefault("ui.libsql_sync_interval", "60s")
v.SetDefault("ui.source_url", "https://tangled.org/evan.jarrett.net/at-container-registry")
// Health defaults
v.SetDefault("health.cache_ttl", "15m")
v.SetDefault("health.check_interval", "15m")
// Jetstream defaults
v.SetDefault("jetstream.urls", []string{
"wss://jetstream2.us-west.bsky.network/subscribe",
"wss://jetstream1.us-west.bsky.network/subscribe",
"wss://jetstream2.us-east.bsky.network/subscribe",
"wss://jetstream1.us-east.bsky.network/subscribe",
})
v.SetDefault("jetstream.backfill_enabled", true)
v.SetDefault("jetstream.backfill_interval", "24h")
v.SetDefault("jetstream.relay_endpoints", []string{
"https://relay1.us-east.bsky.network",
"https://relay1.us-west.bsky.network",
})
// Auth defaults
v.SetDefault("auth.cert_path", "/var/lib/atcr/auth/private-key.crt")
// Log shipper defaults
v.SetDefault("log_shipper.batch_size", 100)
v.SetDefault("log_shipper.flush_interval", "5s")
// AI defaults
v.SetDefault("ai.api_key", "")
// Legal defaults
v.SetDefault("legal.company_name", "")
v.SetDefault("legal.jurisdiction", "")
// Labeler defaults
v.SetDefault("labeler.did", "")
// Log formatter (used by distribution config, not in Config struct)
v.SetDefault("log_formatter", "text")
}
// DefaultConfig returns a Config populated with all default values (no validation).
func DefaultConfig() *Config {
v := config.NewViper("ATCR", "")
setDefaults(v)
cfg := &Config{}
_ = v.Unmarshal(cfg, config.UnmarshalOption())
return cfg
}
// ExampleYAML returns a fully-commented YAML configuration with default values.
func ExampleYAML() ([]byte, error) {
cfg := DefaultConfig()
// Populate example billing tiers so operators see the structure
cfg.Billing.Currency = "usd"
cfg.Billing.SuccessURL = "{base_url}/settings/billing"
cfg.Billing.CancelURL = "{base_url}/settings/billing"
cfg.Billing.Tiers = []billing.BillingTierConfig{
{Name: "deckhand", Description: "Get started with basic storage", MaxWebhooks: 1},
{Name: "bosun", Description: "More storage with scan-on-push", StripePriceMonthly: "price_xxx", StripePriceYearly: "price_yyy", MaxWebhooks: 5, WebhookAllTriggers: true, SupporterBadge: true},
{Name: "quartermaster", Description: "Maximum storage for power users", StripePriceMonthly: "price_xxx", StripePriceYearly: "price_yyy", MaxWebhooks: -1, WebhookAllTriggers: true, SupporterBadge: true},
}
return config.MarshalCommentedYAML("ATCR AppView Configuration", cfg)
}
// LoadConfig builds a complete configuration using Viper layered loading:
// defaults -> YAML file -> environment variables.
// yamlPath is optional; empty string means env-only (backward compatible).
func LoadConfig(yamlPath string) (*Config, error) {
v := config.NewViper("ATCR", yamlPath)
// Set defaults
setDefaults(v)
// Unmarshal into config struct
cfg := &Config{}
if err := v.Unmarshal(cfg, config.UnmarshalOption()); err != nil {
return nil, fmt.Errorf("failed to unmarshal config: %w", err)
}
// Post-load: auto-detect base URL if not set
if cfg.Server.BaseURL == "" {
cfg.Server.BaseURL = autoDetectBaseURL(cfg.Server.Addr)
}
// Post-load: fixed values
cfg.Auth.TokenExpiration = 5 * time.Minute
cfg.Auth.ServiceName = deriveServiceName(cfg)
cfg.CredentialHelper.TangledRepo = "https://tangled.org/evan.jarrett.net/at-container-registry"
// Post-load: CompanyName defaults to ClientName
if cfg.Legal.CompanyName == "" {
cfg.Legal.CompanyName = cfg.Server.ClientName
}
// Post-load: AI API key fallback to CLAUDE_API_KEY env
if cfg.AI.APIKey == "" {
cfg.AI.APIKey = os.Getenv("CLAUDE_API_KEY")
}
// Validation
if len(cfg.Server.ManagedHolds) == 0 {
return nil, fmt.Errorf("server.managed_holds is required (at least one hold DID; env: ATCR_SERVER_MANAGED_HOLDS)")
}
if cfg.Labeler.DID != "" && !strings.HasPrefix(cfg.Labeler.DID, "did:") {
return nil, fmt.Errorf("labeler.did must be a DID (did:plc:... or did:web:...), got %q", cfg.Labeler.DID)
}
// Build distribution config (unchanged)
distConfig, err := buildDistributionConfig(cfg, v)
if err != nil {
return nil, fmt.Errorf("failed to build distribution config: %w", err)
}
cfg.Distribution = distConfig
return cfg, nil
}
// deriveServiceName extracts the JWT service name from the config.
func deriveServiceName(cfg *Config) string {
if len(cfg.Server.RegistryDomains) > 0 {
return cfg.Server.RegistryDomains[0]
}
return getServiceName(cfg.Server.BaseURL)
}
// buildDistributionConfig creates a distribution Configuration from our Config
// This maintains compatibility with the distribution library
func buildDistributionConfig(cfg *Config, v *viper.Viper) (*configuration.Configuration, error) {
distConfig := &configuration.Configuration{}
// Version
distConfig.Version = configuration.MajorMinorVersion(0, 1)
// Logging
logFormatter := v.GetString("log_formatter")
if logFormatter == "" {
logFormatter = "text"
}
distConfig.Log = configuration.Log{
Level: configuration.Loglevel(cfg.LogLevel),
Formatter: logFormatter,
Fields: map[string]any{
"service": "atcr-appview",
},
}
// HTTP server
httpSecret := os.Getenv("REGISTRY_HTTP_SECRET")
if httpSecret == "" {
// Generate a random 32-byte secret
randomBytes := make([]byte, 32)
if _, err := rand.Read(randomBytes); err != nil {
return nil, fmt.Errorf("failed to generate random secret: %w", err)
}
httpSecret = hex.EncodeToString(randomBytes)
}
distConfig.HTTP = configuration.HTTP{
Addr: cfg.Server.Addr,
Secret: httpSecret,
Headers: map[string][]string{
"X-Content-Type-Options": {"nosniff"},
},
}
// Storage (fake in-memory placeholder - all real storage is proxied)
distConfig.Storage = buildStorageConfig()
// Middleware (ATProto resolver)
distConfig.Middleware = buildMiddlewareConfig(cfg.Server.PrimaryHoldDID(), cfg.Server.BaseURL, cfg.Server.TestMode)
// Auth (use values from cfg.Auth)
// Realm always points to BaseURL where auth endpoints live
// Docker's WWW-Authenticate: realm="https://seamark.dev/auth/token",service="buoy.cr"
realm := cfg.Server.BaseURL + "/auth/token"
distConfig.Auth = configuration.Auth{
"token": configuration.Parameters{
"realm": realm,
"service": cfg.Auth.ServiceName,
"issuer": cfg.Auth.ServiceName,
"rootcertbundle": cfg.Auth.CertPath,
"expiration": int(cfg.Auth.TokenExpiration.Seconds()),
},
}
// Health checks
distConfig.Health = buildHealthConfig()
return distConfig, nil
}
// autoDetectBaseURL determines the base URL for the service from the HTTP address
func autoDetectBaseURL(httpAddr string) string {
// Auto-detect from HTTP addr
if httpAddr[0] == ':' {
// Just a port, assume localhost
// Use "127.0.0.1" per RFC 8252 (OAuth servers reject "localhost")
return fmt.Sprintf("http://127.0.0.1%s", httpAddr)
}
// Full address provided
return fmt.Sprintf("http://%s", httpAddr)
}
// buildStorageConfig creates a fake in-memory storage config
// This is required for distribution validation but is never actually used
// All storage is routed through middleware to ATProto (manifests) and hold services (blobs)
func buildStorageConfig() configuration.Storage {
storage := configuration.Storage{}
// Use in-memory storage as a placeholder
storage["inmemory"] = configuration.Parameters{}
// Disable upload purging
// NOTE: Must use map[any]any for uploadpurging (not configuration.Parameters)
// because distribution's validation code does a type assertion to map[any]any
storage["maintenance"] = configuration.Parameters{
"uploadpurging": map[any]any{
"enabled": false,
"age": 7 * 24 * time.Hour, // 168h
"interval": 24 * time.Hour, // 24h
"dryrun": false,
},
}
return storage
}
// buildMiddlewareConfig creates middleware configuration
func buildMiddlewareConfig(defaultHoldDID string, baseURL string, testMode bool) map[string][]configuration.Middleware {
return map[string][]configuration.Middleware{
"registry": {
{
Name: "atproto-resolver",
Options: configuration.Parameters{
"default_hold_did": defaultHoldDID,
"test_mode": testMode,
"base_url": baseURL,
},
},
},
}
}
// buildHealthConfig creates health check configuration
func buildHealthConfig() configuration.Health {
return configuration.Health{
StorageDriver: configuration.StorageDriver{
Enabled: true,
Interval: 10 * time.Second,
Threshold: 3,
},
}
}
// getServiceName extracts service name from base URL hostname
func getServiceName(baseURL string) string {
// Extract from base URL
parsed, err := url.Parse(baseURL)
if err == nil && parsed.Hostname() != "" {
hostname := parsed.Hostname()
// Strip localhost/127.0.0.1 and use default
if hostname == "localhost" || hostname == "127.0.0.1" {
return "atcr.io"
}
return hostname
}
// Default fallback
return "atcr.io"
}