mirror of
https://tangled.org/evan.jarrett.net/at-container-registry
synced 2026-09-19 16:54:15 +00:00
Implement did:plc support for holds with the ability to import/export CARs.
did:plc Identity Support (pkg/hold/pds/did.go, pkg/hold/config.go, pkg/hold/server.go) The big feature — holds can now use did:plc identities instead of only did:web. This adds: - LoadOrCreateDID() — resolves hold DID by priority: config DID > did.txt on disk > create new - CreatePLCIdentity() — builds a genesis operation, signs with rotation key, submits to PLC directory - EnsurePLCCurrent() — on boot, compares local signing key + URL against PLC directory and auto-updates if they've drifted (requires rotation key) - New config fields: did_method (web/plc), did, plc_directory_url, rotation_key_path - GenerateDIDDocument() now uses the stored DID instead of always deriving did:web from URL - NewHoldServer wired up to call LoadOrCreateDID instead of GenerateDIDFromURL CAR Export/Import (pkg/hold/pds/export.go, pkg/hold/pds/import.go, cmd/hold/repo.go) New CLI subcommands for repo backup/restore: - atcr-hold repo export — streams the hold's repo as a CAR file to stdout - atcr-hold repo import <file>... — reads CAR files, upserts all records in a single atomic commit. Uses a bulkImportRecords method that opens a delta session, checks each record for create vs update, commits once, and fires repo events. - openHoldPDS() helper to spin up a HoldPDS from config for offline CLI operations Admin UI Fixes (pkg/hold/admin/) - Logout changed from GET to POST — nav template now uses a <form method=POST> instead of an <a> link (prevents CSRF on logout) - Removed return_to parameter from login flow — simplified redirect logic, auth middleware now redirects to /admin/auth/login without query params Config/Deploy - config-hold.example.yaml and deploy/upcloud/configs/hold.yaml.tmpl updated with the four new did:plc config fields - go.mod / go.sum — added github.com/did-method-plc/go-didplc dependency
This commit is contained in:
Evan Jarrett
@@ -36,7 +36,7 @@ go test -run TestManifestStore ./pkg/atproto/... # specific test
|
||||
go test -race ./... # race detector
|
||||
|
||||
# Docker
|
||||
docker build -t atcr.io/appview:latest .
|
||||
docker build -f Dockerfile.appview -t atcr.io/appview:latest .
|
||||
docker build -f Dockerfile.hold -t atcr.io/hold:latest .
|
||||
docker build -f Dockerfile.scanner -t atcr.io/scanner:latest .
|
||||
docker-compose up -d
|
||||
@@ -53,6 +53,12 @@ SCANNER_HOLD_URL=ws://localhost:8080 SCANNER_SHARED_SECRET=secret ./bin/atcr-sca
|
||||
# Usage report
|
||||
go run ./cmd/usage-report --hold https://hold01.atcr.io
|
||||
go run ./cmd/usage-report --hold https://hold01.atcr.io --from-manifests
|
||||
|
||||
# Utilities
|
||||
go run ./cmd/db-migrate --help # SQLite → libsql migration
|
||||
go run ./cmd/record-query --help # Query ATProto relay by collection
|
||||
go run ./cmd/s3-test # S3 connectivity test
|
||||
go run ./cmd/healthcheck <url> # HTTP health check (for Docker)
|
||||
```
|
||||
|
||||
## Architecture Overview
|
||||
@@ -66,7 +72,7 @@ ATCR uses **distribution/distribution** as a library, extending it via middlewar
|
||||
### Four Components
|
||||
|
||||
1. **AppView** (`cmd/appview`) — OCI Distribution API server. Resolves identities, routes manifests to PDS, routes blobs to hold service, validates OAuth, issues registry JWTs. Includes web UI for browsing.
|
||||
2. **Hold Service** (`cmd/hold`) — BYOS blob storage. Embedded PDS with captain/crew records, S3-compatible storage, presigned URLs. Optional subsystems: admin UI, quotas, billing (Stripe), GC, scan dispatch.
|
||||
2. **Hold Service** (`cmd/hold`) — BYOS blob storage. Embedded PDS with captain/crew/stats/scan records (all ATProto records in CAR store), S3-compatible storage, presigned URLs. Supports did:web (default) or did:plc identity with auto-recovery. Optional subsystems: admin UI, quotas, billing (Stripe), GC, scan dispatch, Bluesky status posts.
|
||||
3. **Scanner** (`scanner/cmd/scanner`) — Vulnerability scanning. Connects to hold via WebSocket, generates SBOMs (Syft), scans vulnerabilities (Grype). Priority queue with tier-based scheduling.
|
||||
4. **Credential Helper** (`cmd/credential-helper`) — Docker credential helper implementing ATProto OAuth flow, exchanges OAuth token for registry JWT.
|
||||
|
||||
@@ -92,6 +98,20 @@ Resolution in `pkg/atproto/resolver.go`: Handle → DID (DNS/HTTPS) → PDS endp
|
||||
- **Sailors** = registry users, **Captains** = hold owners, **Crew** = hold members
|
||||
- **Holds** = storage endpoints (BYOS), **Quartermaster/Bosun/Deckhand** = crew tiers
|
||||
|
||||
### Hold Embedded PDS Records
|
||||
|
||||
The hold's embedded PDS stores all operational data as ATProto records in a CAR store (not SQLite). SQLite holds only the records index and events.
|
||||
|
||||
| Collection | Cardinality | Description |
|
||||
|---|---|---|
|
||||
| `io.atcr.hold.captain` | Singleton | Hold identity, owner DID, settings |
|
||||
| `io.atcr.hold.crew` | Per-member | Crew membership + permissions |
|
||||
| `io.atcr.hold.layer` | Per-layer | Layer metadata (digest, size, media type) |
|
||||
| `io.atcr.hold.stats` | Per-repo | Push/pull counts per owner+repository |
|
||||
| `io.atcr.hold.scan` | Per-scan | Vulnerability scan results |
|
||||
| `app.bsky.feed.post` | Status posts | Online/offline status, push notifications |
|
||||
| `sh.tangled.actor.profile` | Singleton | Hold profile (name, description, avatar) |
|
||||
|
||||
## Authentication
|
||||
|
||||
Three token types flow through the system:
|
||||
@@ -135,6 +155,13 @@ The credential helper never manages OAuth tokens directly — AppView owns the O
|
||||
| OAuth client & session refresher | `pkg/auth/oauth/client.go` |
|
||||
| OAuth P-256 key management | `pkg/auth/oauth/keys.go` |
|
||||
| Hold PDS endpoints & auth | `pkg/hold/pds/xrpc.go`, `pkg/hold/pds/auth.go` |
|
||||
| Hold DID management (did:web, did:plc, PLC recovery) | `pkg/hold/pds/did.go` |
|
||||
| Hold captain records | `pkg/hold/pds/captain.go` |
|
||||
| Hold crew management | `pkg/hold/pds/crew.go` |
|
||||
| Hold push/pull stats (ATProto records in CAR store) | `pkg/hold/pds/stats.go` |
|
||||
| Hold layer records | `pkg/hold/pds/layer.go` |
|
||||
| Hold scan records & scanner integration | `pkg/hold/pds/scan.go`, `pkg/hold/pds/scan_broadcaster.go` |
|
||||
| Hold Bluesky status posts | `pkg/hold/pds/status.go` |
|
||||
| Hold OCI upload endpoints | `pkg/hold/oci/xrpc.go` |
|
||||
| Hold config | `pkg/hold/config.go` |
|
||||
| AppView config | `pkg/appview/config.go` |
|
||||
@@ -163,6 +190,9 @@ See `config-appview.example.yaml` and `config-hold.example.yaml` for all options
|
||||
- **Context keys** (`auth.method`, `puller.did`) exist because `Repository()` receives `context.Context` from the distribution library interface — context values are the only way to pass data from HTTP middleware into the distribution middleware layer. Both are copied into `RegistryContext` inside `Repository()`.
|
||||
- **OAuth key types**: AppView uses P-256 (ES256) for OAuth, not K-256 like PDS keys
|
||||
- **Confidential vs public clients**: Production uses P-256 key at `/var/lib/atcr/oauth/client.key` (auto-generated); localhost is always public client
|
||||
- **Hold stats are ATProto records in CAR store** — `io.atcr.hold.stats` records are stored via `repomgr.PutRecord()`, not in SQLite. Lost if CAR store is lost without backup.
|
||||
- **PLC auto-update on boot** — When using did:plc, `LoadOrCreateDID()` calls `EnsurePLCCurrent()` every startup. If local signing key or URL doesn't match plc.directory, it auto-updates (requires rotation key on disk).
|
||||
- **Hold CAR store is the source of truth** — Captain, crew, layer, stats, scan records, Bluesky posts, profiles are all ATProto records in the CAR store. SQLite holds only the records index and events.
|
||||
|
||||
## Common Tasks
|
||||
|
||||
@@ -199,9 +229,16 @@ See `config-appview.example.yaml` and `config-hold.example.yaml` for all options
|
||||
- **Adding new tables**: Add to `schema.sql` only (no migration needed)
|
||||
- **Altering tables**: Create migration AND update `schema.sql` to keep them in sync
|
||||
|
||||
**Hold DID recovery/migration (did:plc):**
|
||||
1. Back up `rotation.key` and DID string (from `did.txt` or plc.directory)
|
||||
2. Set `database.did_method: plc` and `database.did: "did:plc:..."` in config
|
||||
3. Provide `rotation_key_path` — signing key auto-generates if missing
|
||||
4. On boot: `LoadOrCreateDID()` adopts the DID, `EnsurePLCCurrent()` auto-updates PLC directory if keys/URL changed
|
||||
5. Without rotation key: hold boots but logs warning about PLC mismatch
|
||||
|
||||
**Adding web UI features:**
|
||||
- Add handler in `pkg/appview/handlers/`
|
||||
- Register route in `cmd/appview/serve.go`
|
||||
- Register route in `pkg/appview/routes/routes.go`
|
||||
- Create template in `pkg/appview/templates/pages/`
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
@@ -77,30 +77,33 @@ See **[INSTALLATION.md](./INSTALLATION.md)** for detailed installation instructi
|
||||
|
||||
### Running Your Own AppView
|
||||
|
||||
**Using Docker Compose:**
|
||||
```bash
|
||||
cp .env.appview.example .env.appview
|
||||
# Edit .env.appview with your configuration
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
**Local development:**
|
||||
```bash
|
||||
# Build
|
||||
go build -o bin/atcr-appview ./cmd/appview
|
||||
go build -o bin/atcr-hold ./cmd/hold
|
||||
|
||||
# Configure
|
||||
cp .env.appview.example .env.appview
|
||||
# Edit .env.appview - set ATCR_DEFAULT_HOLD
|
||||
source .env.appview
|
||||
# Generate a config file with all defaults
|
||||
./bin/atcr-appview config init config-appview.yaml
|
||||
# Edit config-appview.yaml — set server.default_hold_did at minimum
|
||||
|
||||
# Run
|
||||
./bin/atcr-appview serve
|
||||
./bin/atcr-appview serve --config config-appview.yaml
|
||||
```
|
||||
|
||||
**Using Docker:**
|
||||
```bash
|
||||
docker build -f Dockerfile.appview -t atcr-appview:latest .
|
||||
docker run -d -p 5000:5000 \
|
||||
-v ./config-appview.yaml:/config.yaml:ro \
|
||||
-v atcr-data:/var/lib/atcr \
|
||||
atcr-appview:latest serve --config /config.yaml
|
||||
```
|
||||
|
||||
See **[deploy/README.md](./deploy/README.md)** for production deployment.
|
||||
|
||||
### Running Your Own Hold (BYOS Storage)
|
||||
|
||||
See **[docs/hold.md](./docs/hold.md)** for deploying your own storage backend.
|
||||
|
||||
## Development
|
||||
|
||||
### Building from Source
|
||||
@@ -122,23 +125,43 @@ go test -race ./...
|
||||
cmd/
|
||||
├── appview/ # Registry server + web UI
|
||||
├── hold/ # Storage service (BYOS)
|
||||
└── credential-helper/ # Docker credential helper
|
||||
├── credential-helper/ # Docker credential helper
|
||||
├── oauth-helper/ # OAuth debug tool
|
||||
├── healthcheck/ # HTTP health check (for Docker)
|
||||
├── db-migrate/ # SQLite → libsql migration
|
||||
├── usage-report/ # Hold storage usage report
|
||||
├── record-query/ # Query ATProto relay by collection
|
||||
└── s3-test/ # S3 connectivity test
|
||||
|
||||
pkg/
|
||||
├── appview/
|
||||
│ ├── db/ # SQLite database (migrations, queries, stores)
|
||||
│ ├── handlers/ # HTTP handlers (home, repo, search, auth, settings)
|
||||
│ ├── holdhealth/ # Hold service health checker
|
||||
│ ├── jetstream/ # ATProto Jetstream consumer
|
||||
│ ├── middleware/ # Auth & registry middleware
|
||||
│ ├── storage/ # Storage routing (hold cache, blob proxy, repository)
|
||||
│ ├── ogcard/ # OpenGraph image generation
|
||||
│ ├── readme/ # Repository README fetcher
|
||||
│ ├── routes/ # HTTP route registration
|
||||
│ ├── storage/ # Storage routing (blob proxy, manifest store)
|
||||
│ ├── public/ # Static assets (JS, CSS, install scripts)
|
||||
│ └── templates/ # HTML templates
|
||||
├── atproto/ # ATProto client, records, manifest/tag stores
|
||||
├── auth/
|
||||
│ ├── oauth/ # OAuth client, server, refresher, storage
|
||||
│ ├── oauth/ # OAuth client, refresher, storage
|
||||
│ ├── token/ # JWT issuer, validator, claims
|
||||
│ └── atproto/ # Session validation
|
||||
└── hold/ # Hold service (authorization, storage, multipart, S3)
|
||||
│ └── holdlocal/ # Local hold authorization
|
||||
├── config/ # Config marshaling (commented YAML)
|
||||
├── hold/
|
||||
│ ├── admin/ # Admin web UI
|
||||
│ ├── billing/ # Stripe billing integration
|
||||
│ ├── db/ # Vendored carstore (go-libsql)
|
||||
│ ├── gc/ # Garbage collection
|
||||
│ ├── oci/ # OCI upload endpoints
|
||||
│ ├── pds/ # Embedded PDS (DID, captain, crew, stats, scans)
|
||||
│ └── quota/ # Storage quotas
|
||||
├── logging/ # Structured logging + remote shipping
|
||||
└── s3/ # S3 client utilities
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
@@ -76,6 +76,7 @@ func init() {
|
||||
|
||||
rootCmd.AddCommand(serveCmd)
|
||||
rootCmd.AddCommand(configCmd)
|
||||
rootCmd.AddCommand(repoCmd)
|
||||
}
|
||||
|
||||
func main() {
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"os"
|
||||
|
||||
"atcr.io/pkg/hold"
|
||||
holddb "atcr.io/pkg/hold/db"
|
||||
"atcr.io/pkg/hold/pds"
|
||||
|
||||
"github.com/spf13/cobra"
|
||||
)
|
||||
|
||||
var repoCmd = &cobra.Command{
|
||||
Use: "repo",
|
||||
Short: "Repository management commands",
|
||||
}
|
||||
|
||||
var repoExportCmd = &cobra.Command{
|
||||
Use: "export",
|
||||
Short: "Export the hold's repo as a CAR file to stdout",
|
||||
Long: `Export the hold's ATProto repository as a CAR (Content Addressable Archive) file.
|
||||
The CAR is written to stdout, so redirect to a file:
|
||||
|
||||
atcr-hold repo export --config config.yaml > backup.car`,
|
||||
Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
cfg, err := hold.LoadConfig(repoConfigFile)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to load config: %w", err)
|
||||
}
|
||||
|
||||
ctx := context.Background()
|
||||
holdPDS, cleanup, err := openHoldPDS(ctx, cfg)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer cleanup()
|
||||
|
||||
if err := holdPDS.ExportToCAR(ctx, os.Stdout); err != nil {
|
||||
return fmt.Errorf("failed to export: %w", err)
|
||||
}
|
||||
|
||||
fmt.Fprintf(os.Stderr, "Export complete\n")
|
||||
return nil
|
||||
},
|
||||
}
|
||||
|
||||
var repoImportCmd = &cobra.Command{
|
||||
Use: "import <file> [file...]",
|
||||
Short: "Import records from one or more CAR files",
|
||||
Long: `Import ATProto records from CAR files into the hold's repo.
|
||||
Records are upserted (existing records are overwritten). Multiple files can be
|
||||
imported additively.
|
||||
|
||||
atcr-hold repo import --config config.yaml backup.car
|
||||
atcr-hold repo import --config config.yaml backup.car extra-records.car`,
|
||||
Args: cobra.MinimumNArgs(1),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
cfg, err := hold.LoadConfig(repoConfigFile)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to load config: %w", err)
|
||||
}
|
||||
|
||||
ctx := context.Background()
|
||||
holdPDS, cleanup, err := openHoldPDS(ctx, cfg)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer cleanup()
|
||||
|
||||
for _, path := range args {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to open %s: %w", path, err)
|
||||
}
|
||||
|
||||
result, err := holdPDS.ImportFromCAR(ctx, f)
|
||||
f.Close()
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to import %s: %w", path, err)
|
||||
}
|
||||
|
||||
fmt.Fprintf(os.Stderr, "Imported %d records from %s\n", result.Total, path)
|
||||
for collection, count := range result.PerCollection {
|
||||
fmt.Fprintf(os.Stderr, " %s: %d\n", collection, count)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
},
|
||||
}
|
||||
|
||||
var repoConfigFile string
|
||||
|
||||
func init() {
|
||||
repoCmd.PersistentFlags().StringVarP(&repoConfigFile, "config", "c", "", "path to YAML configuration file")
|
||||
|
||||
repoCmd.AddCommand(repoExportCmd)
|
||||
repoCmd.AddCommand(repoImportCmd)
|
||||
}
|
||||
|
||||
// openHoldPDS creates a HoldPDS from config for offline CLI operations.
|
||||
// Returns the PDS and a cleanup function that must be deferred.
|
||||
func openHoldPDS(ctx context.Context, cfg *hold.Config) (*pds.HoldPDS, func(), error) {
|
||||
holdDID, err := pds.LoadOrCreateDID(ctx, pds.DIDConfig{
|
||||
DID: cfg.Database.DID,
|
||||
DIDMethod: cfg.Database.DIDMethod,
|
||||
PublicURL: cfg.Server.PublicURL,
|
||||
DBPath: cfg.Database.Path,
|
||||
SigningKeyPath: cfg.Database.KeyPath,
|
||||
RotationKeyPath: cfg.Database.RotationKeyPath,
|
||||
PLCDirectoryURL: cfg.Database.PLCDirectoryURL,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, nil, fmt.Errorf("failed to resolve hold DID: %w", err)
|
||||
}
|
||||
slog.Info("Using hold DID", "did", holdDID)
|
||||
|
||||
// Open shared database
|
||||
dbFilePath := cfg.Database.Path + "/db.sqlite3"
|
||||
libsqlCfg := holddb.LibsqlConfig{
|
||||
SyncURL: cfg.Database.LibsqlSyncURL,
|
||||
AuthToken: cfg.Database.LibsqlAuthToken,
|
||||
SyncInterval: cfg.Database.LibsqlSyncInterval,
|
||||
}
|
||||
holdDB, err := holddb.OpenHoldDB(dbFilePath, libsqlCfg)
|
||||
if err != nil {
|
||||
return nil, nil, fmt.Errorf("failed to open hold database: %w", err)
|
||||
}
|
||||
|
||||
holdPDS, err := pds.NewHoldPDSWithDB(ctx, holdDID, cfg.Server.PublicURL, cfg.Database.Path, cfg.Database.KeyPath, false, holdDB.DB)
|
||||
if err != nil {
|
||||
holdDB.Close()
|
||||
return nil, nil, fmt.Errorf("failed to initialize PDS: %w", err)
|
||||
}
|
||||
|
||||
cleanup := func() {
|
||||
holdPDS.Close()
|
||||
holdDB.Close()
|
||||
}
|
||||
|
||||
return holdPDS, cleanup, nil
|
||||
}
|
||||
@@ -69,6 +69,14 @@ database:
|
||||
path: /var/lib/atcr-hold
|
||||
# PDS signing key path. Defaults to {database.path}/signing.key.
|
||||
key_path: ""
|
||||
# DID method: 'web' (default, derived from public_url) or 'plc' (registered with PLC directory).
|
||||
did_method: web
|
||||
# Explicit DID for this hold. If set with did_method 'plc', adopts this identity instead of creating new. Use for recovery/migration.
|
||||
did: ""
|
||||
# PLC directory URL. Only used when did_method is 'plc'. Default: https://plc.directory
|
||||
plc_directory_url: https://plc.directory
|
||||
# Rotation key path for did:plc. Controls DID identity (separate from signing key). Defaults to {database.path}/rotation.key.
|
||||
rotation_key_path: ""
|
||||
# libSQL sync URL (libsql://...). Works with Turso cloud, Bunny DB, or self-hosted libsql-server. Leave empty for local-only SQLite.
|
||||
libsql_sync_url: ""
|
||||
# Auth token for libSQL sync. Required if libsql_sync_url is set.
|
||||
|
||||
@@ -32,6 +32,10 @@ registration:
|
||||
database:
|
||||
path: "{{.BasePath}}"
|
||||
key_path: ""
|
||||
did_method: web
|
||||
did: ""
|
||||
plc_directory_url: https://plc.directory
|
||||
rotation_key_path: ""
|
||||
libsql_sync_url: ""
|
||||
libsql_auth_token: ""
|
||||
libsql_sync_interval: 1m0s
|
||||
|
||||
+187
-213
@@ -6,23 +6,21 @@
|
||||
|
||||
**AppView** is the frontend server component of ATCR. It serves as the OCI-compliant registry API endpoint and web interface that Docker clients interact with when pushing and pulling container images.
|
||||
|
||||
### What AppView Does
|
||||
|
||||
AppView is the orchestration layer that:
|
||||
|
||||
- **Serves the OCI Distribution API V2** - Compatible with Docker, containerd, podman, and all OCI clients
|
||||
- **Resolves ATProto identities** - Converts handles (`alice.bsky.social`) and DIDs (`did:plc:xyz123`) to PDS endpoints
|
||||
- **Routes manifests** - Stores container image manifests as ATProto records in users' Personal Data Servers
|
||||
- **Routes blobs** - Proxies blob (layer) operations to hold services for S3-compatible storage
|
||||
- **Provides web UI** - Browse repositories, search images, view tags, track pull counts, manage stars
|
||||
- **Manages authentication** - Validates OAuth tokens and issues registry JWTs to Docker clients
|
||||
- **Provides web UI** - Browse repositories, search images, view tags, track pull counts, manage stars, vulnerability scan results
|
||||
- **Manages authentication** - ATProto OAuth with device authorization flow, issues registry JWTs to Docker clients
|
||||
|
||||
### The ATCR Ecosystem
|
||||
|
||||
AppView is the **frontend** of a multi-component architecture:
|
||||
|
||||
1. **AppView** (this component) - Registry API + web interface
|
||||
2. **[Hold Service](https://atcr.io/r/evan.jarrett.net/atcr-hold)** - Storage backend with embedded PDS for blob storage
|
||||
2. **[Hold Service](hold.md)** - Storage backend with embedded PDS for blob storage
|
||||
3. **Credential Helper** - Client-side tool for ATProto OAuth authentication
|
||||
|
||||
**Data flow:**
|
||||
@@ -45,255 +43,231 @@ Most users can simply use **https://atcr.io** - you don't need to run your own A
|
||||
- Maintain full control over registry infrastructure
|
||||
|
||||
**Prerequisites:**
|
||||
- A running [Hold service](https://atcr.io/r/evan.jarrett.net/atcr-hold) (required for blob storage)
|
||||
- A running [Hold service](hold.md) (required for blob storage)
|
||||
- (Optional) Domain name with SSL/TLS certificates for production
|
||||
- (Optional) Access to ATProto Jetstream for real-time indexing
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Using Docker Compose
|
||||
|
||||
The fastest way to run AppView alongside a Hold service:
|
||||
### 1. Build the Docker image
|
||||
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone https://tangled.org/evan.jarrett.net/at-container-registry
|
||||
cd atcr
|
||||
docker build -t atcr-appview:latest -f Dockerfile.appview .
|
||||
```
|
||||
|
||||
# Copy and configure environment
|
||||
cp .env.appview.example .env.appview
|
||||
# Edit .env.appview - set ATCR_DEFAULT_HOLD_DID (see Configuration below)
|
||||
This produces a ~30MB scratch image with a statically-linked binary.
|
||||
|
||||
# Start services
|
||||
docker-compose up -d
|
||||
### 2. Generate a config file
|
||||
|
||||
# Verify
|
||||
```bash
|
||||
docker run --rm atcr-appview config init > config-appview.yaml
|
||||
```
|
||||
|
||||
This creates a fully-commented YAML file with all available options and their defaults. You can also generate it from a local binary:
|
||||
|
||||
```bash
|
||||
./bin/atcr-appview config init config-appview.yaml
|
||||
```
|
||||
|
||||
### 3. Set the required field
|
||||
|
||||
Edit `config-appview.yaml` and set `server.default_hold_did` to your hold service's DID:
|
||||
|
||||
```yaml
|
||||
server:
|
||||
default_hold_did: "did:web:127.0.0.1:8080" # local dev
|
||||
# default_hold_did: "did:web:hold01.example.com" # production
|
||||
```
|
||||
|
||||
This is the **only required configuration field**. To find a hold's DID, visit its `/.well-known/did.json` endpoint.
|
||||
|
||||
For production, also set your public URL:
|
||||
|
||||
```yaml
|
||||
server:
|
||||
base_url: "https://registry.example.com"
|
||||
default_hold_did: "did:web:hold01.example.com"
|
||||
```
|
||||
|
||||
### 4. Run
|
||||
|
||||
```bash
|
||||
docker run -d \
|
||||
-v ./config-appview.yaml:/config.yaml:ro \
|
||||
-v atcr-data:/var/lib/atcr \
|
||||
-p 5000:5000 \
|
||||
atcr-appview serve --config /config.yaml
|
||||
```
|
||||
|
||||
### 5. Verify
|
||||
|
||||
```bash
|
||||
curl http://localhost:5000/v2/
|
||||
# Should return: {}
|
||||
|
||||
curl http://localhost:5000/health
|
||||
# Should return: {"status":"ok"}
|
||||
```
|
||||
|
||||
### Minimal Configuration
|
||||
## Configuration
|
||||
|
||||
At minimum, you must set:
|
||||
AppView uses YAML configuration with environment variable overrides. The generated `config-appview.yaml` is the canonical reference — every field is commented inline with its purpose and default value.
|
||||
|
||||
### Config loading priority (highest wins)
|
||||
|
||||
1. Environment variables (`ATCR_` prefix)
|
||||
2. YAML config file (`--config`)
|
||||
3. Built-in defaults
|
||||
|
||||
### Environment variable convention
|
||||
|
||||
YAML paths map to env vars with `ATCR_` prefix and `_` separators:
|
||||
|
||||
```
|
||||
server.default_hold_did → ATCR_SERVER_DEFAULT_HOLD_DID
|
||||
server.base_url → ATCR_SERVER_BASE_URL
|
||||
ui.database_path → ATCR_UI_DATABASE_PATH
|
||||
jetstream.backfill_enabled → ATCR_JETSTREAM_BACKFILL_ENABLED
|
||||
```
|
||||
|
||||
### Config sections overview
|
||||
|
||||
| Section | Purpose | Notes |
|
||||
|---------|---------|-------|
|
||||
| `server` | Listen address, public URL, hold DID, OAuth key, branding | Only `default_hold_did` is required |
|
||||
| `ui` | Database path, theme, libSQL sync | All have defaults; auto-creates DB on first run |
|
||||
| `auth` | JWT signing key/cert paths | Auto-generated on first run |
|
||||
| `jetstream` | Real-time ATProto event streaming, backfill sync | Runs automatically; backfill enabled by default |
|
||||
| `health` | Hold health check interval and cache TTL | Sensible defaults (15m) |
|
||||
| `log_shipper` | Remote log shipping (Victoria, OpenSearch, Loki) | Disabled by default |
|
||||
| `legal` | Terms/privacy page customization | Optional |
|
||||
| `credential_helper` | Credential helper download source | Optional |
|
||||
|
||||
### Auto-generated files
|
||||
|
||||
On first run, AppView auto-generates these under `/var/lib/atcr/`:
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `ui.db` | SQLite database (OAuth sessions, stars, pull counts, device approvals) |
|
||||
| `auth/private-key.pem` | RSA private key for signing registry JWTs |
|
||||
| `auth/private-key.crt` | X.509 certificate for JWT verification |
|
||||
| `oauth/client.key` | P-256 private key for OAuth client authentication |
|
||||
|
||||
**Persist `/var/lib/atcr/` across restarts.** Losing the auth keys invalidates all active sessions; losing the database loses OAuth state and UI data.
|
||||
|
||||
## Deployment
|
||||
|
||||
### Docker (recommended)
|
||||
|
||||
`Dockerfile.appview` builds a minimal scratch image (~30MB) containing:
|
||||
- Static `atcr-appview` binary (CGO-enabled with embedded SQLite)
|
||||
- `healthcheck` binary for container health checks
|
||||
- CA certificates and timezone data
|
||||
|
||||
**Port:** `5000` (HTTP)
|
||||
|
||||
**Volume:** `/var/lib/atcr` (auth keys, database, OAuth keys)
|
||||
|
||||
**Health check:** `GET /health` returns `{"status":"ok"}`
|
||||
|
||||
```bash
|
||||
# Required: Default hold service for blob storage
|
||||
ATCR_DEFAULT_HOLD_DID=did:web:127.0.0.1:8080
|
||||
|
||||
# Recommended for production
|
||||
ATCR_BASE_URL=https://registry.example.com
|
||||
ATCR_HTTP_ADDR=:5000
|
||||
docker run -d \
|
||||
--name atcr-appview \
|
||||
-v ./config-appview.yaml:/config.yaml:ro \
|
||||
-v atcr-data:/var/lib/atcr \
|
||||
-p 5000:5000 \
|
||||
--health-cmd '/healthcheck http://localhost:5000/health' \
|
||||
--health-interval 30s \
|
||||
--restart unless-stopped \
|
||||
atcr-appview serve --config /config.yaml
|
||||
```
|
||||
|
||||
See **Configuration Reference** below for all options.
|
||||
### Production with reverse proxy
|
||||
|
||||
## Configuration Reference
|
||||
AppView serves HTTP on port 5000. For production, put a reverse proxy in front for HTTPS termination. The repository includes a working Caddy + Docker Compose setup at [`deploy/docker-compose.prod.yml`](../deploy/docker-compose.prod.yml) that runs AppView, Hold, and Caddy together with automatic TLS.
|
||||
|
||||
AppView is configured entirely via environment variables. Load them with:
|
||||
```bash
|
||||
source .env.appview
|
||||
./bin/atcr-appview serve
|
||||
A minimal production compose override:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
atcr-appview:
|
||||
image: atcr-appview:latest
|
||||
command: ["serve", "--config", "/config.yaml"]
|
||||
environment:
|
||||
ATCR_SERVER_BASE_URL: https://registry.example.com
|
||||
ATCR_SERVER_DEFAULT_HOLD_DID: did:web:hold.example.com
|
||||
volumes:
|
||||
- ./config-appview.yaml:/config.yaml:ro
|
||||
- atcr-appview-data:/var/lib/atcr
|
||||
healthcheck:
|
||||
test: ["CMD", "/healthcheck", "http://localhost:5000/health"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 30s
|
||||
|
||||
volumes:
|
||||
atcr-appview-data:
|
||||
```
|
||||
|
||||
Or via Docker Compose (recommended).
|
||||
### Systemd (bare metal)
|
||||
|
||||
### Server Configuration
|
||||
|
||||
#### `ATCR_HTTP_ADDR`
|
||||
- **Default:** `:5000`
|
||||
- **Description:** HTTP listen address for the registry API and web UI
|
||||
- **Example:** `:5000`, `:8080`, `0.0.0.0:5000`
|
||||
|
||||
#### `ATCR_BASE_URL`
|
||||
- **Default:** Auto-detected from `ATCR_HTTP_ADDR` (e.g., `http://127.0.0.1:5000`)
|
||||
- **Description:** Public URL for the AppView service. Used to generate OAuth redirect URIs and JWT realm claims.
|
||||
- **Development:** Auto-detection works fine (`http://127.0.0.1:5000`)
|
||||
- **Production:** Set to your public URL (e.g., `https://atcr.example.com`)
|
||||
- **Example:** `https://atcr.io`, `http://127.0.0.1:5000`
|
||||
|
||||
### Storage Configuration
|
||||
|
||||
#### `ATCR_DEFAULT_HOLD_DID` ⚠️ REQUIRED
|
||||
- **Default:** None (required)
|
||||
- **Description:** DID of the default hold service for blob storage. Used when users don't have their own hold configured in their sailor profile. AppView routes all blob operations to this hold.
|
||||
- **Format:** `did:web:hostname[:port]`
|
||||
- **Docker Compose:** `did:web:atcr-hold:8080` (internal Docker network)
|
||||
- **Local dev:** `did:web:127.0.0.1:8080`
|
||||
- **Production:** `did:web:hold01.atcr.io`
|
||||
- **Note:** This hold must be reachable from AppView. To find a hold's DID, visit `https://hold-url/.well-known/did.json`
|
||||
|
||||
### Authentication Configuration
|
||||
|
||||
#### `ATCR_AUTH_KEY_PATH`
|
||||
- **Default:** `/var/lib/atcr/auth/private-key.pem`
|
||||
- **Description:** Path to JWT signing private key (RSA). Auto-generated if missing.
|
||||
- **Note:** Keep this secure - it signs all registry JWTs issued to Docker clients
|
||||
|
||||
#### `ATCR_AUTH_CERT_PATH`
|
||||
- **Default:** `/var/lib/atcr/auth/private-key.crt`
|
||||
- **Description:** Path to JWT signing certificate. Auto-generated if missing.
|
||||
- **Note:** Paired with `ATCR_AUTH_KEY_PATH`
|
||||
|
||||
### Web UI Configuration
|
||||
|
||||
#### `ATCR_UI_DATABASE_PATH`
|
||||
- **Default:** `/var/lib/atcr/ui.db`
|
||||
- **Description:** SQLite database path for UI data (OAuth sessions, stars, pull counts, repository metadata)
|
||||
- **Note:** For multi-instance deployments, use PostgreSQL (see production docs)
|
||||
|
||||
### Logging Configuration
|
||||
|
||||
#### `ATCR_LOG_LEVEL`
|
||||
- **Default:** `info`
|
||||
- **Options:** `debug`, `info`, `warn`, `error`
|
||||
- **Description:** Log verbosity level
|
||||
- **Development:** Use `debug` for detailed troubleshooting
|
||||
- **Production:** Use `info` or `warn`
|
||||
|
||||
#### `ATCR_LOG_FORMATTER`
|
||||
- **Default:** `text`
|
||||
- **Options:** `text`, `json`
|
||||
- **Description:** Log output format
|
||||
- **Production:** Use `json` for structured logging (easier to parse with log aggregators)
|
||||
|
||||
### Hold Health Check Configuration
|
||||
|
||||
AppView periodically checks if hold services are reachable and caches results to display health indicators in the UI.
|
||||
|
||||
#### `ATCR_HEALTH_CHECK_INTERVAL`
|
||||
- **Default:** `15m`
|
||||
- **Description:** How often to check health of hold endpoints in the background
|
||||
- **Format:** Duration string (e.g., `5m`, `15m`, `30m`, `1h`)
|
||||
- **Recommendation:** 15-30 minutes for production
|
||||
|
||||
#### `ATCR_HEALTH_CACHE_TTL`
|
||||
- **Default:** `15m`
|
||||
- **Description:** How long to cache health check results before re-checking
|
||||
- **Format:** Duration string (e.g., `15m`, `30m`, `1h`)
|
||||
- **Note:** Should be >= `ATCR_HEALTH_CHECK_INTERVAL` for efficiency
|
||||
|
||||
### Jetstream Configuration (ATProto Event Streaming)
|
||||
|
||||
Jetstream provides real-time indexing of ATProto records (manifests, tags) into the AppView database for the web UI.
|
||||
|
||||
#### `JETSTREAM_URL`
|
||||
- **Default:** `wss://jetstream2.us-west.bsky.network/subscribe`
|
||||
- **Description:** Jetstream WebSocket URL for real-time ATProto events
|
||||
- **Note:** Connects to Bluesky's public Jetstream by default
|
||||
|
||||
#### `ATCR_BACKFILL_ENABLED`
|
||||
- **Default:** `false`
|
||||
- **Description:** Enable periodic sync of historical ATProto records. Set to `true` for production to ensure database completeness.
|
||||
- **Recommendation:** Enable for production AppView instances
|
||||
|
||||
#### `ATCR_RELAY_ENDPOINT`
|
||||
- **Default:** `https://relay1.us-east.bsky.network`
|
||||
- **Description:** ATProto relay endpoint for backfill sync API
|
||||
- **Note:** Used when `ATCR_BACKFILL_ENABLED=true`
|
||||
|
||||
### Legacy Configuration
|
||||
|
||||
#### `TEST_MODE`
|
||||
- **Default:** `false`
|
||||
- **Description:** Enable test mode (skips some validations). Do not use in production.
|
||||
|
||||
## Web Interface Features
|
||||
|
||||
The AppView web UI provides:
|
||||
|
||||
- **Home page** - Featured repositories and recent pushes feed
|
||||
- **Repository pages** - View tags, manifests, pull instructions, health status
|
||||
- **Search** - Find repositories by owner handle or repository name
|
||||
- **User profiles** - View a user's repositories and activity
|
||||
- **Stars** - Favorite repositories (requires OAuth login)
|
||||
- **Pull counts** - Track image pull statistics
|
||||
- **Multi-arch support** - Display platform-specific manifests (linux/amd64, linux/arm64)
|
||||
- **Health indicators** - Real-time hold service reachability status
|
||||
- **Install scripts** - Host credential helper installation scripts at `/install.sh`
|
||||
For non-Docker deployments, see the systemd service templates in [`deploy/upcloud/`](../deploy/upcloud/) which include security hardening (dedicated user, filesystem protection, private tmp).
|
||||
|
||||
## Deployment Scenarios
|
||||
|
||||
### Public Registry (like atcr.io)
|
||||
### Public Registry
|
||||
|
||||
Open to all ATProto users:
|
||||
|
||||
```bash
|
||||
# AppView config
|
||||
ATCR_BASE_URL=https://registry.example.com
|
||||
ATCR_DEFAULT_HOLD_DID=did:web:hold01.example.com
|
||||
ATCR_BACKFILL_ENABLED=true
|
||||
|
||||
# Hold config (linked hold service)
|
||||
HOLD_PUBLIC=true # Allow public pulls
|
||||
HOLD_ALLOW_ALL_CREW=true # Allow all authenticated users to push
|
||||
```yaml
|
||||
# config-appview.yaml
|
||||
server:
|
||||
base_url: "https://registry.example.com"
|
||||
default_hold_did: "did:web:hold01.example.com"
|
||||
jetstream:
|
||||
backfill_enabled: true
|
||||
```
|
||||
|
||||
The linked hold service should have `server.public: true` and `registration.allow_all_crew: true`.
|
||||
|
||||
### Private Organizational Registry
|
||||
|
||||
Restricted to crew members only:
|
||||
|
||||
```bash
|
||||
# AppView config
|
||||
ATCR_BASE_URL=https://registry.internal.example.com
|
||||
ATCR_DEFAULT_HOLD_DID=did:web:hold.internal.example.com
|
||||
|
||||
# Hold config (linked hold service)
|
||||
HOLD_PUBLIC=false # Require auth for pulls
|
||||
HOLD_ALLOW_ALL_CREW=false # Only owner + explicit crew can push
|
||||
HOLD_OWNER=did:plc:your-org-did # Organization DID
|
||||
```yaml
|
||||
# config-appview.yaml
|
||||
server:
|
||||
base_url: "https://registry.internal.example.com"
|
||||
default_hold_did: "did:web:hold.internal.example.com"
|
||||
```
|
||||
|
||||
### Development/Testing
|
||||
The linked hold service should have `server.public: false` and `registration.allow_all_crew: false`, with an explicit `registration.owner_did` set to the organization's DID.
|
||||
|
||||
Local Docker Compose setup with Minio for S3-compatible storage:
|
||||
### Local Development
|
||||
|
||||
```bash
|
||||
# Start Minio (S3-compatible storage)
|
||||
docker run -p 9000:9000 -p 9001:9001 minio/minio server /data --console-address ":9001"
|
||||
|
||||
# AppView config
|
||||
ATCR_HTTP_ADDR=:5000
|
||||
ATCR_DEFAULT_HOLD_DID=did:web:atcr-hold:8080
|
||||
ATCR_LOG_LEVEL=debug
|
||||
|
||||
# Hold config (linked hold service)
|
||||
AWS_ACCESS_KEY_ID=minioadmin
|
||||
AWS_SECRET_ACCESS_KEY=minioadmin
|
||||
S3_BUCKET=test
|
||||
S3_ENDPOINT=http://minio:9000
|
||||
HOLD_PUBLIC=true
|
||||
HOLD_ALLOW_ALL_CREW=true
|
||||
```yaml
|
||||
# config-appview.yaml
|
||||
log_level: debug
|
||||
server:
|
||||
default_hold_did: "did:web:127.0.0.1:8080"
|
||||
test_mode: true # allows HTTP for DID resolution
|
||||
```
|
||||
|
||||
## Production Deployment
|
||||
Run a hold service locally with Minio for S3-compatible storage. See [hold.md](hold.md) for hold setup.
|
||||
|
||||
For production deployments with:
|
||||
- Multiple AppView instances (load balancing)
|
||||
- PostgreSQL database (instead of SQLite)
|
||||
- SSL/TLS certificates
|
||||
- Systemd service files
|
||||
- Log rotation
|
||||
- Monitoring
|
||||
## Web Interface
|
||||
|
||||
See **[deploy/README.md](https://tangled.org/evan.jarrett.net/at-container-registry/blob/main/deploy/README.md)** for comprehensive production deployment guide.
|
||||
The AppView web UI provides:
|
||||
|
||||
### Quick Production Checklist
|
||||
|
||||
Before going to production:
|
||||
|
||||
- [ ] Set `ATCR_BASE_URL` to your public HTTPS URL
|
||||
- [ ] Set `ATCR_DEFAULT_HOLD_DID` to a production hold service
|
||||
- [ ] Enable Jetstream backfill (`ATCR_BACKFILL_ENABLED=true`)
|
||||
- [ ] Use `ATCR_LOG_FORMATTER=json` for structured logging
|
||||
- [ ] Secure JWT keys (`ATCR_AUTH_KEY_PATH`, `ATCR_AUTH_CERT_PATH`)
|
||||
- [ ] Configure SSL/TLS termination (nginx/Caddy/Cloudflare)
|
||||
- [ ] Set up database backups (if using SQLite, consider PostgreSQL)
|
||||
- [ ] Monitor hold health checks
|
||||
- [ ] Test OAuth flow end-to-end
|
||||
- [ ] Verify Docker push/pull works
|
||||
|
||||
## Configuration Files Reference
|
||||
|
||||
- **[.env.appview.example](https://tangled.org/evan.jarrett.net/at-container-registry/blob/main/.env.appview.example)** - All available environment variables with documentation
|
||||
- **[deploy/.env.prod.template](https://tangled.org/evan.jarrett.net/at-container-registry/blob/main/deploy/.env.prod.template)** - Production configuration template
|
||||
- **[deploy/README.md](https://tangled.org/evan.jarrett.net/at-container-registry/blob/main/deploy/README.md)** - Production deployment guide
|
||||
- **[Hold Service Documentation](https://atcr.io/r/evan.jarrett.net/atcr-hold)** - Storage backend setup
|
||||
- **Home page** - Featured repositories and recent pushes
|
||||
- **Repository pages** - Tags, manifests, pull instructions, health status, vulnerability scan results
|
||||
- **Search** - Find repositories by owner handle or repository name
|
||||
- **User profiles** - View a user's repositories and starred images
|
||||
- **Stars** - Favorite repositories (requires login)
|
||||
- **Pull counts** - Image pull statistics
|
||||
- **Multi-arch support** - Platform-specific manifests (linux/amd64, linux/arm64, etc.)
|
||||
- **Health indicators** - Real-time hold service reachability
|
||||
- **Device management** - Approve and revoke Docker credential helper pairings
|
||||
- **Settings** - Choose default hold, view crew memberships, storage usage
|
||||
|
||||
+140
-332
@@ -1,382 +1,190 @@
|
||||
# ATCR Hold Service
|
||||
|
||||
> The storage backend component of ATCR (ATProto Container Registry)
|
||||
Hold Service is the BYOS (Bring Your Own Storage) blob storage backend for ATCR. It stores container image layers in your own S3-compatible storage (AWS S3, Storj, Minio, UpCloud, etc.) and generates presigned URLs so clients transfer data directly to/from S3. Each hold runs an embedded ATProto PDS with its own DID, repository, and crew-based access control.
|
||||
|
||||
## Overview
|
||||
Hold Service is one component of the ATCR ecosystem:
|
||||
|
||||
**Hold Service** is the storage backend component of ATCR. It enables BYOS (Bring Your Own Storage) - users can store their own container image layers in their own S3-compatible storage (AWS S3, Storj, Minio, UpCloud, etc.). Each hold runs as a full ATProto user with an embedded PDS, exposing both standard ATProto sync endpoints and custom XRPC endpoints for OCI multipart blob uploads.
|
||||
1. **[AppView](https://atcr.io/r/evan.jarrett.net/atcr-appview)** — Registry API + web interface
|
||||
2. **Hold Service** (this component) — Storage backend with embedded PDS
|
||||
3. **Credential Helper** — Client-side tool for ATProto OAuth authentication
|
||||
|
||||
### What Hold Service Does
|
||||
|
||||
Hold Service is the storage layer that:
|
||||
|
||||
- **Bring Your Own Storage (BYOS)** - Store your own container image layers in your own S3-compatible storage (AWS S3, Storj, Minio, UpCloud, etc.)
|
||||
- **Embedded ATProto PDS** - Each hold is a full ATProto user with its own DID, repository, and identity
|
||||
- **Custom XRPC Endpoints** - OCI-compatible multipart upload endpoints (`io.atcr.hold.*`) for blob operations
|
||||
- **Presigned URL Generation** - Creates time-limited S3 URLs for direct client-to-storage transfers (~99% bandwidth reduction)
|
||||
- **Crew Management** - Controls access via captain and crew records stored in the hold's embedded PDS
|
||||
- **Standard ATProto Sync** - Exposes com.atproto.sync.* endpoints for repository synchronization and firehose
|
||||
- **S3 Storage** - Works with any S3-compatible storage (AWS S3, Storj, Minio, UpCloud, Azure, GCS via S3 gateway)
|
||||
- **Bluesky Integration** - Optional: Posts container image push notifications from the hold's identity to Bluesky
|
||||
|
||||
### The ATCR Ecosystem
|
||||
|
||||
Hold Service is the **storage backend** of a multi-component architecture:
|
||||
|
||||
1. **[AppView](https://atcr.io/r/evan.jarrett.net/atcr-appview)** - Registry API + web interface
|
||||
2. **Hold Service** (this component) - Storage backend with embedded PDS
|
||||
3. **Credential Helper** - Client-side tool for ATProto OAuth authentication
|
||||
|
||||
**Data flow:**
|
||||
```
|
||||
Docker Client → AppView (resolves identity) → User's PDS (stores manifest)
|
||||
↓
|
||||
Hold Service (generates presigned URL)
|
||||
↓
|
||||
S3/Storj/etc. (client uploads/downloads blobs directly)
|
||||
Docker Client --> AppView (resolves identity) --> User's PDS (stores manifest)
|
||||
|
|
||||
Hold Service (generates presigned URL)
|
||||
|
|
||||
S3/Storj/etc. (client uploads/downloads directly)
|
||||
```
|
||||
|
||||
Manifests (small JSON metadata) live in users' ATProto PDS, while blobs (large binary layers) live in hold services. AppView orchestrates the routing, and hold services provide presigned URLs to eliminate bandwidth bottlenecks.
|
||||
Manifests (small JSON metadata) live in users' ATProto PDS. Blobs (large binary layers) live in hold services. AppView orchestrates the routing.
|
||||
|
||||
## When to Run Your Own Hold
|
||||
|
||||
Most users can push to the default hold at **https://hold01.atcr.io** - you don't need to run your own hold.
|
||||
Most users can push to the default hold at **https://hold01.atcr.io** — you don't need to run your own.
|
||||
|
||||
**Run your own hold if you want to:**
|
||||
- Control where your container layer data is stored (own S3 bucket, Storj, etc.)
|
||||
Run your own hold if you want to:
|
||||
- Control where your container layer data is stored (own S3 bucket, geographic region)
|
||||
- Manage access for a team or organization via crew membership
|
||||
- Reduce bandwidth costs by using presigned URLs for direct S3 transfers
|
||||
- Run a shared hold for a community or project
|
||||
- Maintain data sovereignty (keep blobs in specific geographic regions)
|
||||
- Use a CDN pull zone for faster downloads
|
||||
|
||||
**Prerequisites:**
|
||||
- S3-compatible storage (AWS S3, Storj, Minio, UpCloud, etc.)
|
||||
- (Optional) Domain name with SSL/TLS certificates for production
|
||||
- ATProto DID for hold owner (get from: `https://bsky.social/xrpc/com.atproto.identity.resolveHandle?handle=yourhandle.bsky.social`)
|
||||
**Prerequisites:** S3-compatible storage with a bucket already created, and a domain with TLS for production.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Using Docker Compose
|
||||
|
||||
The fastest way to run Hold service with S3 storage:
|
||||
### 1. Generate Configuration
|
||||
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone https://tangled.org/evan.jarrett.net/at-container-registry
|
||||
cd atcr
|
||||
# Build the hold binary
|
||||
go build -o bin/atcr-hold ./cmd/hold
|
||||
|
||||
# Copy and configure environment
|
||||
cp .env.hold.example .env.hold
|
||||
# Edit .env.hold - set HOLD_PUBLIC_URL, HOLD_OWNER, S3 credentials (see Configuration below)
|
||||
|
||||
# Start hold service
|
||||
docker-compose -f docker-compose.hold.yml up -d
|
||||
|
||||
# Verify
|
||||
curl http://localhost:8080/.well-known/did.json
|
||||
# Generate a fully-commented config file with all defaults
|
||||
./bin/atcr-hold config init config-hold.yaml
|
||||
```
|
||||
|
||||
### Minimal Configuration
|
||||
|
||||
At minimum, you must set:
|
||||
Or generate config from Docker without building locally:
|
||||
|
||||
```bash
|
||||
# Required: Public URL (generates did:web identity)
|
||||
HOLD_PUBLIC_URL=https://hold.example.com
|
||||
|
||||
# Required: Your ATProto DID (for captain record)
|
||||
HOLD_OWNER=did:plc:your-did-here
|
||||
|
||||
# Required: S3 credentials and bucket
|
||||
AWS_ACCESS_KEY_ID=your-access-key
|
||||
AWS_SECRET_ACCESS_KEY=your-secret-key
|
||||
S3_BUCKET=your-bucket-name
|
||||
|
||||
# Recommended: Database directory for embedded PDS
|
||||
HOLD_DATABASE_DIR=/var/lib/atcr-hold
|
||||
docker run --rm -i $(docker build -q -f Dockerfile.hold .) config init > config-hold.yaml
|
||||
```
|
||||
|
||||
See **Configuration Reference** below for all options.
|
||||
The generated file documents every option with inline comments. Edit only what you need.
|
||||
|
||||
## Configuration Reference
|
||||
### 2. Minimal Configuration
|
||||
|
||||
Hold Service is configured entirely via environment variables. Load them with:
|
||||
```bash
|
||||
source .env.hold
|
||||
./bin/atcr-hold
|
||||
Only three things need to be set — everything else has sensible defaults:
|
||||
|
||||
```yaml
|
||||
storage:
|
||||
access_key: "YOUR_S3_ACCESS_KEY"
|
||||
secret_key: "YOUR_S3_SECRET_KEY"
|
||||
bucket: "your-bucket-name"
|
||||
endpoint: "https://gateway.storjshare.io" # omit for AWS S3
|
||||
|
||||
server:
|
||||
public_url: "https://hold.example.com"
|
||||
|
||||
registration:
|
||||
owner_did: "did:plc:your-did-here"
|
||||
```
|
||||
|
||||
Or via Docker Compose (recommended).
|
||||
- **`server.public_url`** — Your hold's public HTTPS URL. This becomes the hold's `did:web` identity.
|
||||
- **`storage.bucket`** — S3 bucket name (must already exist).
|
||||
- **`registration.owner_did`** — Your ATProto DID. Creates you as captain (admin) on first boot. Get yours from: `https://bsky.social/xrpc/com.atproto.identity.resolveHandle?handle=yourhandle.bsky.social`
|
||||
|
||||
### Server Configuration
|
||||
|
||||
#### `HOLD_PUBLIC_URL` ⚠️ REQUIRED
|
||||
- **Default:** None (required)
|
||||
- **Description:** Public URL of this hold service. Used to generate the hold's did:web identity. The hostname becomes the hold's DID.
|
||||
- **Format:** `https://hold.example.com` or `http://127.0.0.1:8080` (development)
|
||||
- **Example:** `https://hold01.atcr.io` → DID is `did:web:hold01.atcr.io`
|
||||
- **Note:** This URL must be reachable by AppView and Docker clients
|
||||
|
||||
#### `HOLD_SERVER_ADDR`
|
||||
- **Default:** `:8080`
|
||||
- **Description:** HTTP listen address for XRPC endpoints
|
||||
- **Example:** `:8080`, `:9000`, `0.0.0.0:8080`
|
||||
|
||||
#### `HOLD_PUBLIC`
|
||||
- **Default:** `false`
|
||||
- **Description:** Allow public blob reads (pulls) without authentication. Writes always require crew membership.
|
||||
- **Use cases:**
|
||||
- `true`: Public registry (anyone can pull, authenticated users can push if crew)
|
||||
- `false`: Private registry (authentication required for both push and pull)
|
||||
|
||||
### S3 Storage Configuration
|
||||
|
||||
S3 is the only supported storage backend. Presigned URLs enable direct client-to-storage transfers (~99% bandwidth reduction).
|
||||
|
||||
##### `AWS_ACCESS_KEY_ID` ⚠️ REQUIRED for S3
|
||||
- **Description:** S3 access key ID for authentication
|
||||
- **Example:** `AKIAIOSFODNN7EXAMPLE`
|
||||
|
||||
##### `AWS_SECRET_ACCESS_KEY` ⚠️ REQUIRED for S3
|
||||
- **Description:** S3 secret access key for authentication
|
||||
- **Example:** `wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY`
|
||||
|
||||
##### `AWS_REGION`
|
||||
- **Default:** `us-east-1`
|
||||
- **Description:** S3 region
|
||||
- **AWS regions:** `us-east-1`, `us-west-2`, `eu-west-1`, etc.
|
||||
- **UpCloud regions:** `us-chi1`, `us-nyc1`, `de-fra1`, `uk-lon1`, `sg-sin1`
|
||||
|
||||
##### `S3_BUCKET` ⚠️ REQUIRED for S3
|
||||
- **Description:** S3 bucket name where blobs will be stored
|
||||
- **Example:** `atcr-blobs`, `my-company-registry-blobs`
|
||||
- **Note:** Bucket must already exist
|
||||
|
||||
##### `S3_ENDPOINT`
|
||||
- **Default:** None (uses AWS S3)
|
||||
- **Description:** S3-compatible endpoint URL for non-AWS providers
|
||||
- **Storj:** `https://gateway.storjshare.io`
|
||||
- **UpCloud:** `https://[bucket-id].upcloudobjects.com`
|
||||
- **Minio:** `http://minio:9000`
|
||||
- **Note:** Leave empty for AWS S3
|
||||
|
||||
### Embedded PDS Configuration
|
||||
|
||||
#### `HOLD_DATABASE_DIR`
|
||||
- **Default:** `/var/lib/atcr-hold`
|
||||
- **Description:** Directory path for embedded PDS carstore (SQLite database). Carstore creates `db.sqlite3` inside this directory.
|
||||
- **Note:** This must be a directory path, NOT a file path. If empty, embedded PDS is disabled (not recommended - hold authorization requires PDS).
|
||||
|
||||
#### `HOLD_KEY_PATH`
|
||||
- **Default:** `{HOLD_DATABASE_DIR}/signing.key`
|
||||
- **Description:** Path to hold's signing key (secp256k1). Auto-generated on first run if missing.
|
||||
- **Note:** Keep this secure - it's used to sign ATProto commits in the hold's repository
|
||||
|
||||
### Access Control
|
||||
|
||||
#### `HOLD_OWNER`
|
||||
- **Default:** None
|
||||
- **Description:** Your ATProto DID. Used to create the captain record and add you as the first crew member with admin role.
|
||||
- **Get your DID:** `https://bsky.social/xrpc/com.atproto.identity.resolveHandle?handle=yourhandle.bsky.social`
|
||||
- **Example:** `did:plc:abc123xyz789`
|
||||
- **Note:** If set, the hold will initialize with your DID as owner on first run
|
||||
|
||||
#### `HOLD_ALLOW_ALL_CREW`
|
||||
- **Default:** `false`
|
||||
- **Description:** Allow any authenticated ATCR user to write to this hold (treat all as crew)
|
||||
- **Security model:**
|
||||
- `true`: Any authenticated user can push images (useful for shared/community holds)
|
||||
- `false`: Only hold owner and explicit crew members can push (verified via crew records in hold's PDS)
|
||||
- **Use cases:**
|
||||
- Public registry: `HOLD_PUBLIC=true, HOLD_ALLOW_ALL_CREW=true`
|
||||
- ATProto users only: `HOLD_PUBLIC=false, HOLD_ALLOW_ALL_CREW=true`
|
||||
- Private hold: `HOLD_PUBLIC=false, HOLD_ALLOW_ALL_CREW=false` (default)
|
||||
|
||||
### Bluesky Integration
|
||||
|
||||
#### `HOLD_BLUESKY_POSTS_ENABLED`
|
||||
- **Default:** `false`
|
||||
- **Description:** Create Bluesky posts when users push container images. Posts include image name, tag, size, and layer count.
|
||||
- **Note:** Posts are created from the hold's embedded PDS identity (did:web). Requires hold to be crawled by Bluesky relay.
|
||||
- **Enable relay crawl:** `./deploy/request-crawl.sh hold.example.com`
|
||||
|
||||
#### `HOLD_PROFILE_AVATAR`
|
||||
- **Default:** `https://imgs.blue/evan.jarrett.net/1TpTOdtS60GdJWBYEqtK22y688jajbQ9a5kbYRFtwuqrkBAE`
|
||||
- **Description:** URL to download avatar image for hold's Bluesky profile. Downloaded and uploaded as blob during bootstrap.
|
||||
- **Note:** Avatar is stored in hold's PDS and displayed on Bluesky profile
|
||||
|
||||
### Advanced Configuration
|
||||
|
||||
#### `TEST_MODE`
|
||||
- **Default:** `false`
|
||||
- **Description:** Enable test mode (skips some validations). Do not use in production.
|
||||
|
||||
## XRPC Endpoints
|
||||
|
||||
Hold Service exposes two types of XRPC endpoints:
|
||||
|
||||
### ATProto Sync Endpoints (Standard)
|
||||
- `GET /.well-known/did.json` - DID document (did:web resolution)
|
||||
- `GET /xrpc/com.atproto.sync.getRepo` - Download full repository as CAR file
|
||||
- `GET /xrpc/com.atproto.sync.getBlob` - Get blob or presigned download URL
|
||||
- `GET /xrpc/com.atproto.sync.subscribeRepos` - WebSocket firehose for real-time events
|
||||
- `GET /xrpc/com.atproto.sync.listRepos` - List all repositories (single-user PDS)
|
||||
- `GET /xrpc/com.atproto.repo.describeRepo` - Repository metadata
|
||||
- `GET /xrpc/com.atproto.repo.getRecord` - Get record by collection and rkey
|
||||
- `GET /xrpc/com.atproto.repo.listRecords` - List records in collection
|
||||
- `POST /xrpc/com.atproto.repo.deleteRecord` - Delete record (owner/crew admin only)
|
||||
|
||||
### OCI Multipart Upload Endpoints (Custom)
|
||||
- `POST /xrpc/io.atcr.hold.initiateUpload` - Start multipart upload session
|
||||
- `POST /xrpc/io.atcr.hold.getPartUploadUrl` - Get presigned URL for uploading a part
|
||||
- `POST /xrpc/io.atcr.hold.completeUpload` - Finalize multipart upload
|
||||
- `POST /xrpc/io.atcr.hold.abortUpload` - Cancel multipart upload
|
||||
- `POST /xrpc/io.atcr.hold.notifyManifest` - Notify hold of manifest upload (creates layer records, Bluesky posts)
|
||||
|
||||
## Authorization Model
|
||||
|
||||
Hold Service uses crew membership records in its embedded PDS for access control:
|
||||
|
||||
### Read Access (Blob Downloads)
|
||||
|
||||
**Public Hold** (`HOLD_PUBLIC=true`):
|
||||
- Anonymous users: ✅ Allowed
|
||||
- Authenticated users: ✅ Allowed
|
||||
|
||||
**Private Hold** (`HOLD_PUBLIC=false`):
|
||||
- Anonymous users: ❌ Forbidden
|
||||
- Authenticated users with crew membership: ✅ Allowed
|
||||
- Crew must have `blob:read` permission
|
||||
|
||||
### Write Access (Blob Uploads)
|
||||
|
||||
Regardless of `HOLD_PUBLIC` setting:
|
||||
- Hold owner (from captain record): ✅ Allowed
|
||||
- Crew members with `blob:write` permission: ✅ Allowed
|
||||
- Non-crew authenticated users: Depends on `HOLD_ALLOW_ALL_CREW`
|
||||
- `HOLD_ALLOW_ALL_CREW=true`: ✅ Allowed
|
||||
- `HOLD_ALLOW_ALL_CREW=false`: ❌ Forbidden
|
||||
|
||||
### Authentication Method
|
||||
|
||||
AppView uses **service tokens** from user's PDS to authenticate with hold service:
|
||||
1. AppView calls user's PDS: `com.atproto.server.getServiceAuth` with hold DID
|
||||
2. User's PDS returns a service token scoped to the hold DID
|
||||
3. AppView includes service token in XRPC requests to hold
|
||||
4. Hold validates token and checks crew membership in its embedded PDS
|
||||
|
||||
## Deployment Scenarios
|
||||
|
||||
### Personal Hold (Single User)
|
||||
|
||||
Your own storage for your images:
|
||||
### 3. Build and Run with Docker
|
||||
|
||||
```bash
|
||||
# Hold config
|
||||
HOLD_PUBLIC_URL=https://hold.alice.com
|
||||
HOLD_OWNER=did:plc:alice-did
|
||||
HOLD_PUBLIC=false # Private (only you can pull)
|
||||
HOLD_ALLOW_ALL_CREW=false # Only you can push
|
||||
HOLD_DATABASE_DIR=/var/lib/atcr-hold
|
||||
# Build the image
|
||||
docker build -f Dockerfile.hold -t atcr-hold:latest .
|
||||
|
||||
# S3 storage (using Storj)
|
||||
AWS_ACCESS_KEY_ID=your-key
|
||||
AWS_SECRET_ACCESS_KEY=your-secret
|
||||
S3_BUCKET=alice-container-registry
|
||||
S3_ENDPOINT=https://gateway.storjshare.io
|
||||
# Run it
|
||||
docker run -d \
|
||||
--name atcr-hold \
|
||||
-p 8080:8080 \
|
||||
-v $(pwd)/config-hold.yaml:/config.yaml:ro \
|
||||
-v atcr-hold-data:/var/lib/atcr-hold \
|
||||
atcr-hold:latest serve --config /config.yaml
|
||||
```
|
||||
|
||||
### Shared Hold (Team/Organization)
|
||||
- **`/var/lib/atcr-hold`** — Persistent volume for the embedded PDS (carstore database + signing keys). Back this up.
|
||||
- **Port 8080** — Default listen address. Put a reverse proxy (Caddy, nginx) in front for TLS.
|
||||
- The image is built `FROM scratch` — the binary includes SQLite statically linked.
|
||||
- Optional: `docker build --build-arg BILLING_ENABLED=true` to include Stripe billing support.
|
||||
|
||||
Shared storage for a team with crew members:
|
||||
## Configuration
|
||||
|
||||
Config loads in layers: **defaults → YAML file → environment variables**. Later layers override earlier ones.
|
||||
|
||||
All YAML fields can be overridden with environment variables using the `HOLD_` prefix and `_` path separators. For example, `server.public_url` becomes `HOLD_SERVER_PUBLIC_URL`.
|
||||
|
||||
S3 credentials also accept standard AWS environment variable names: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION`, `S3_BUCKET`, `S3_ENDPOINT`.
|
||||
|
||||
For the complete configuration reference with all options and defaults, see [`config-hold.example.yaml`](../config-hold.example.yaml) or run `atcr-hold config init`.
|
||||
|
||||
## Access Control
|
||||
|
||||
| Setting | Who can pull | Who can push |
|
||||
|---|---|---|
|
||||
| `server.public: true` | Anyone | Captain + crew with `blob:write` |
|
||||
| `server.public: false` (default) | Crew with `blob:read` | Captain + crew with `blob:write` |
|
||||
| + `registration.allow_all_crew: true` | (per above) | Any authenticated user |
|
||||
|
||||
The captain (set via `registration.owner_did`) has all permissions implicitly. `blob:write` implies `blob:read`.
|
||||
|
||||
Authentication uses ATProto service tokens: AppView requests a token from the user's PDS scoped to the hold's DID, then includes it in XRPC requests. The hold validates the token and checks crew membership.
|
||||
|
||||
See [BYOS.md](BYOS.md) for the full authorization model.
|
||||
|
||||
## Optional Subsystems
|
||||
|
||||
| Subsystem | Default | Config key | Notes |
|
||||
|---|---|---|---|
|
||||
| Admin panel | Enabled | `admin.enabled` | Web UI for crew, settings, and storage management |
|
||||
| Quotas | Disabled | `quota.tiers` | Tier-based storage limits (e.g., deckhand=5GB, bosun=50GB) |
|
||||
| Garbage collection | Disabled | `gc.enabled` | Nightly cleanup of orphaned blobs and records |
|
||||
| Vulnerability scanner | Disabled | `scanner.secret` | Requires separate scanner service; see [SBOM_SCANNING.md](SBOM_SCANNING.md) |
|
||||
| Billing (Stripe) | Disabled | Build flag + env | Build with `--build-arg BILLING_ENABLED=true`; see [BILLING.md](BILLING.md) |
|
||||
| Bluesky posts | Disabled | `registration.enable_bluesky_posts` | Posts push notifications from hold's identity |
|
||||
|
||||
## Hold Identity
|
||||
|
||||
**did:web (default)** — Derived from `server.public_url` with zero setup. `https://hold.example.com` becomes `did:web:hold.example.com`. The DID document is served at `/.well-known/did.json`. Tied to domain ownership — if you lose the domain, you lose the identity.
|
||||
|
||||
**did:plc (portable)** — Set `database.did_method: plc` in config. Registered with plc.directory. Survives domain changes. Requires a rotation key (auto-generated at `{database.path}/rotation.key`). Use `database.did` to adopt an existing DID for recovery or migration.
|
||||
|
||||
## Verification
|
||||
|
||||
After starting your hold, verify it's working:
|
||||
|
||||
```bash
|
||||
# Hold config
|
||||
HOLD_PUBLIC_URL=https://hold.acme.corp
|
||||
HOLD_OWNER=did:plc:acme-org-did
|
||||
HOLD_PUBLIC=false # Private reads (crew only)
|
||||
HOLD_ALLOW_ALL_CREW=false # Explicit crew membership required
|
||||
HOLD_DATABASE_DIR=/var/lib/atcr-hold
|
||||
# Health check — should return {"version":"..."}
|
||||
curl https://hold.example.com/xrpc/_health
|
||||
|
||||
# S3 storage
|
||||
AWS_ACCESS_KEY_ID=your-key
|
||||
AWS_SECRET_ACCESS_KEY=your-secret
|
||||
S3_BUCKET=acme-registry-blobs
|
||||
# DID document — should return valid JSON with service endpoints
|
||||
curl https://hold.example.com/.well-known/did.json
|
||||
|
||||
# Captain record — should show your owner DID
|
||||
curl "https://hold.example.com/xrpc/com.atproto.repo.listRecords?repo=HOLD_DID&collection=io.atcr.hold.captain"
|
||||
|
||||
# Crew records
|
||||
curl "https://hold.example.com/xrpc/com.atproto.repo.listRecords?repo=HOLD_DID&collection=io.atcr.hold.crew"
|
||||
```
|
||||
|
||||
Then add crew members via XRPC or hold PDS records.
|
||||
Replace `HOLD_DID` with your hold's DID (from the `/.well-known/did.json` response).
|
||||
|
||||
### Public Hold (Community Registry)
|
||||
## Docker Compose
|
||||
|
||||
Open storage allowing anyone to push and pull:
|
||||
```yaml
|
||||
services:
|
||||
atcr-hold:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile.hold
|
||||
command: ["serve", "--config", "/config.yaml"]
|
||||
volumes:
|
||||
- ./config-hold.yaml:/config.yaml:ro
|
||||
- atcr-hold-data:/var/lib/atcr-hold
|
||||
ports:
|
||||
- "8080:8080"
|
||||
healthcheck:
|
||||
test: ["CMD", "/healthcheck", "http://localhost:8080/xrpc/_health"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 30s
|
||||
|
||||
```bash
|
||||
# Hold config
|
||||
HOLD_PUBLIC_URL=https://hold.community.io
|
||||
HOLD_OWNER=did:plc:community-did
|
||||
HOLD_PUBLIC=true # Public reads (anyone can pull)
|
||||
HOLD_ALLOW_ALL_CREW=true # Any authenticated user can push
|
||||
HOLD_DATABASE_DIR=/var/lib/atcr-hold
|
||||
|
||||
# S3 storage
|
||||
AWS_ACCESS_KEY_ID=your-key
|
||||
AWS_SECRET_ACCESS_KEY=your-secret
|
||||
S3_BUCKET=community-registry-blobs
|
||||
volumes:
|
||||
atcr-hold-data:
|
||||
```
|
||||
|
||||
### Development/Testing with Minio
|
||||
For production with TLS termination, see [`deploy/docker-compose.prod.yml`](../deploy/docker-compose.prod.yml) which includes a Caddy reverse proxy.
|
||||
|
||||
For local development, use Minio as an S3-compatible storage:
|
||||
## Further Reading
|
||||
|
||||
```bash
|
||||
# Start Minio
|
||||
docker run -p 9000:9000 -p 9001:9001 minio/minio server /data --console-address ":9001"
|
||||
|
||||
# Hold config
|
||||
HOLD_PUBLIC_URL=http://127.0.0.1:8080
|
||||
HOLD_OWNER=did:plc:your-test-did
|
||||
HOLD_PUBLIC=true
|
||||
HOLD_ALLOW_ALL_CREW=true
|
||||
HOLD_DATABASE_DIR=/tmp/atcr-hold
|
||||
|
||||
# Minio S3 storage
|
||||
AWS_ACCESS_KEY_ID=minioadmin
|
||||
AWS_SECRET_ACCESS_KEY=minioadmin
|
||||
S3_BUCKET=test
|
||||
S3_ENDPOINT=http://localhost:9000
|
||||
```
|
||||
|
||||
## Production Deployment
|
||||
|
||||
For production deployments with:
|
||||
- SSL/TLS certificates
|
||||
- S3 storage with presigned URLs
|
||||
- Proper access control
|
||||
- Systemd service files
|
||||
- Monitoring
|
||||
|
||||
See **[deploy/README.md](https://tangled.org/evan.jarrett.net/at-container-registry/blob/main/deploy/README.md)** for comprehensive production deployment guide.
|
||||
|
||||
### Quick Production Checklist
|
||||
|
||||
Before going to production:
|
||||
|
||||
- [ ] Set `HOLD_PUBLIC_URL` to your public HTTPS URL
|
||||
- [ ] Set `HOLD_OWNER` to your ATProto DID
|
||||
- [ ] Set `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `S3_BUCKET`, `S3_ENDPOINT`
|
||||
- [ ] Set `HOLD_DATABASE_DIR` to persistent directory
|
||||
- [ ] Configure `HOLD_PUBLIC` and `HOLD_ALLOW_ALL_CREW` for desired access model
|
||||
- [ ] Configure SSL/TLS termination (Caddy/nginx/Cloudflare)
|
||||
- [ ] Verify DID document: `curl https://hold.example.com/.well-known/did.json`
|
||||
- [ ] Test presigned URLs: Check logs for "presigned URL" messages during push
|
||||
- [ ] Monitor crew membership: `curl https://hold.example.com/xrpc/com.atproto.repo.listRecords?repo={holdDID}&collection=io.atcr.hold.crew`
|
||||
- [ ] (Optional) Enable Bluesky posts: `HOLD_BLUESKY_POSTS_ENABLED=true`
|
||||
- [ ] (Optional) Request relay crawl: `./deploy/request-crawl.sh hold.example.com`
|
||||
|
||||
## Configuration Files Reference
|
||||
|
||||
- **[.env.hold.example](https://tangled.org/evan.jarrett.net/at-container-registry/blob/main/.env.hold.example)** - All available environment variables with documentation
|
||||
- **[deploy/.env.prod.template](https://tangled.org/evan.jarrett.net/at-container-registry/blob/main/deploy/.env.prod.template)** - Production configuration template (includes both AppView and Hold)
|
||||
- **[deploy/README.md](https://tangled.org/evan.jarrett.net/at-container-registry/blob/main/deploy/README.md)** - Production deployment guide
|
||||
- **[AppView Documentation](https://atcr.io/r/evan.jarrett.net/atcr-appview)** - Registry API server setup
|
||||
- **[BYOS Architecture](https://tangled.org/evan.jarrett.net/at-container-registry/blob/main/docs/BYOS.md)** - Bring Your Own Storage technical design
|
||||
- [`config-hold.example.yaml`](../config-hold.example.yaml) — Complete configuration reference with inline comments
|
||||
- [BYOS.md](BYOS.md) — Bring Your Own Storage architecture and authorization model
|
||||
- [HOLD_XRPC_ENDPOINTS.md](HOLD_XRPC_ENDPOINTS.md) — XRPC endpoint reference
|
||||
- [BILLING.md](BILLING.md) — Stripe billing integration
|
||||
- [QUOTAS.md](QUOTAS.md) — Quota management
|
||||
- [SBOM_SCANNING.md](SBOM_SCANNING.md) — Vulnerability scanning
|
||||
|
||||
@@ -9,6 +9,7 @@ require (
|
||||
github.com/aws/aws-sdk-go-v2/credentials v1.19.7
|
||||
github.com/aws/aws-sdk-go-v2/service/s3 v1.96.0
|
||||
github.com/bluesky-social/indigo v0.0.0-20260213003059-85cdd0d6871c
|
||||
github.com/did-method-plc/go-didplc v0.0.0-20251009212921-7b7a252b8019
|
||||
github.com/distribution/distribution/v3 v3.0.0
|
||||
github.com/distribution/reference v0.6.0
|
||||
github.com/earthboundkid/versioninfo/v2 v2.24.1
|
||||
|
||||
@@ -84,6 +84,8 @@ github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0 h1:NMZiJj8QnKe1LgsbDayM4UoHwbvw
|
||||
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0/go.mod h1:ZXNYxsqcloTdSy/rNShjYzMhyjf0LaoftYK0p+A3h40=
|
||||
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f h1:lO4WD4F/rVNCu3HqELle0jiPLLBs70cWOduZpkS1E78=
|
||||
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f/go.mod h1:cuUVRXasLTGF7a8hSLbxyZXjz+1KgoB3wDUb6vlszIc=
|
||||
github.com/did-method-plc/go-didplc v0.0.0-20251009212921-7b7a252b8019 h1:MhDee1P3Zar8u72U6RtOKvzSd7dBAU3l2hhrOLQsfB0=
|
||||
github.com/did-method-plc/go-didplc v0.0.0-20251009212921-7b7a252b8019/go.mod h1:dBm0+R8Diqo90As3Q6p2wXAdrGXJgPEWBKUnpV5SUzI=
|
||||
github.com/distribution/distribution/v3 v3.0.0 h1:q4R8wemdRQDClzoNNStftB2ZAfqOiN6UX90KJc4HjyM=
|
||||
github.com/distribution/distribution/v3 v3.0.0/go.mod h1:tRNuFoZsUdyRVegq8xGNeds4KLjwLCRin/tTo6i1DhU=
|
||||
github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
|
||||
|
||||
@@ -75,6 +75,7 @@ func setupTestDB(t *testing.T) *sql.DB {
|
||||
size INTEGER NOT NULL,
|
||||
media_type TEXT NOT NULL,
|
||||
layer_index INTEGER NOT NULL,
|
||||
annotations TEXT,
|
||||
PRIMARY KEY(manifest_id, layer_index)
|
||||
);
|
||||
|
||||
|
||||
@@ -421,7 +421,7 @@ func (ui *AdminUI) RegisterRoutes(r chi.Router) {
|
||||
r.Get("/admin/api/relay/status", ui.handleRelayStatus)
|
||||
|
||||
// Logout
|
||||
r.Get("/admin/auth/logout", ui.handleLogout)
|
||||
r.Post("/admin/auth/logout", ui.handleLogout)
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -13,7 +13,7 @@ func (ui *AdminUI) requireOwner(next http.Handler) http.Handler {
|
||||
// Get session cookie
|
||||
token, ok := getSessionCookie(r)
|
||||
if !ok {
|
||||
http.Redirect(w, r, "/admin/auth/login?return_to="+r.URL.Path, http.StatusFound)
|
||||
http.Redirect(w, r, "/admin/auth/login", http.StatusFound)
|
||||
return
|
||||
}
|
||||
|
||||
|
||||
@@ -21,23 +21,16 @@ func (ui *AdminUI) handleLogin(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
}
|
||||
|
||||
returnTo := r.URL.Query().Get("return_to")
|
||||
if returnTo == "" {
|
||||
returnTo = "/admin"
|
||||
}
|
||||
|
||||
data := struct {
|
||||
PageData
|
||||
ReturnTo string
|
||||
Error string
|
||||
Error string
|
||||
}{
|
||||
PageData: PageData{
|
||||
Title: "Login",
|
||||
ActivePage: "login",
|
||||
HoldDID: ui.pds.DID(),
|
||||
},
|
||||
ReturnTo: returnTo,
|
||||
Error: r.URL.Query().Get("error"),
|
||||
Error: r.URL.Query().Get("error"),
|
||||
}
|
||||
|
||||
ui.renderTemplate(w, "pages/login.html", data)
|
||||
|
||||
@@ -8,7 +8,9 @@
|
||||
<div class="flex items-center gap-3">
|
||||
{{template "admin-theme-toggle"}}
|
||||
<span class="text-sm opacity-80">{{.User.Handle}}</span>
|
||||
<a href="/admin/auth/logout" class="btn btn-sm btn-ghost">Logout</a>
|
||||
<form method="POST" action="/admin/auth/logout" class="inline">
|
||||
<button type="submit" class="btn btn-sm btn-ghost">Logout</button>
|
||||
</form>
|
||||
</div>
|
||||
{{end}}
|
||||
</div>
|
||||
|
||||
@@ -19,8 +19,6 @@
|
||||
{{end}}
|
||||
|
||||
<form action="/admin/auth/oauth/authorize" method="GET">
|
||||
<input type="hidden" name="return_to" value="{{.ReturnTo}}">
|
||||
|
||||
<fieldset class="fieldset mb-4">
|
||||
<label class="fieldset-label" for="handle">Handle or DID</label>
|
||||
<input type="text" id="handle" name="handle"
|
||||
|
||||
+25
-1
@@ -143,6 +143,18 @@ type DatabaseConfig struct {
|
||||
// PDS signing key path.
|
||||
KeyPath string `yaml:"key_path" comment:"PDS signing key path. Defaults to {database.path}/signing.key."`
|
||||
|
||||
// DID method for hold identity: "web" (default) or "plc".
|
||||
DIDMethod string `yaml:"did_method" comment:"DID method: 'web' (default, derived from public_url) or 'plc' (registered with PLC directory)."`
|
||||
|
||||
// Explicit DID for this hold. Used for recovery/migration with did:plc.
|
||||
DID string `yaml:"did" comment:"Explicit DID for this hold. If set with did_method 'plc', adopts this identity instead of creating new. Use for recovery/migration."`
|
||||
|
||||
// PLC directory URL. Only used when did_method is "plc".
|
||||
PLCDirectoryURL string `yaml:"plc_directory_url" comment:"PLC directory URL. Only used when did_method is 'plc'. Default: https://plc.directory"`
|
||||
|
||||
// Rotation key path for did:plc. Separate from signing key for recovery.
|
||||
RotationKeyPath string `yaml:"rotation_key_path" comment:"Rotation key path for did:plc. Controls DID identity (separate from signing key). Defaults to {database.path}/rotation.key."`
|
||||
|
||||
// libSQL sync URL for embedded replica mode.
|
||||
LibsqlSyncURL string `yaml:"libsql_sync_url" comment:"libSQL sync URL (libsql://...). Works with Turso cloud, Bunny DB, or self-hosted libsql-server. Leave empty for local-only SQLite."`
|
||||
|
||||
@@ -177,6 +189,10 @@ func setHoldDefaults(v *viper.Viper) {
|
||||
// Database defaults
|
||||
v.SetDefault("database.path", "/var/lib/atcr-hold")
|
||||
v.SetDefault("database.key_path", "")
|
||||
v.SetDefault("database.did_method", "web")
|
||||
v.SetDefault("database.did", "")
|
||||
v.SetDefault("database.plc_directory_url", "https://plc.directory")
|
||||
v.SetDefault("database.rotation_key_path", "")
|
||||
v.SetDefault("database.libsql_sync_url", "")
|
||||
v.SetDefault("database.libsql_auth_token", "")
|
||||
v.SetDefault("database.libsql_sync_interval", "60s")
|
||||
@@ -267,10 +283,18 @@ func LoadConfig(yamlPath string) (*Config, error) {
|
||||
return nil, fmt.Errorf("storage.bucket is required (env: S3_BUCKET) - S3 is the only supported storage backend")
|
||||
}
|
||||
|
||||
// Post-load: derive key path from database path if not set
|
||||
// Post-load: derive key paths from database path if not set
|
||||
if cfg.Database.KeyPath == "" && cfg.Database.Path != "" {
|
||||
cfg.Database.KeyPath = filepath.Join(cfg.Database.Path, "signing.key")
|
||||
}
|
||||
if cfg.Database.RotationKeyPath == "" && cfg.Database.Path != "" {
|
||||
cfg.Database.RotationKeyPath = filepath.Join(cfg.Database.Path, "rotation.key")
|
||||
}
|
||||
|
||||
// Validate DID method
|
||||
if cfg.Database.DIDMethod != "" && cfg.Database.DIDMethod != "web" && cfg.Database.DIDMethod != "plc" {
|
||||
return nil, fmt.Errorf("database.did_method must be 'web' or 'plc', got %q", cfg.Database.DIDMethod)
|
||||
}
|
||||
|
||||
// Store config path for subsystem config loading (e.g. billing)
|
||||
cfg.configPath = yamlPath
|
||||
|
||||
+298
-12
@@ -1,9 +1,18 @@
|
||||
package pds
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net/url"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"atcr.io/pkg/auth/oauth"
|
||||
"github.com/bluesky-social/indigo/atproto/atcrypto"
|
||||
didplc "github.com/did-method-plc/go-didplc"
|
||||
)
|
||||
|
||||
// DIDDocument represents a did:web document
|
||||
@@ -32,25 +41,21 @@ type Service struct {
|
||||
ServiceEndpoint string `json:"serviceEndpoint"`
|
||||
}
|
||||
|
||||
// GenerateDIDDocument creates a DID document for a did:web identity
|
||||
// GenerateDIDDocument creates a DID document for the hold's identity.
|
||||
// It uses the hold's stored DID (which may be did:web or did:plc).
|
||||
func (p *HoldPDS) GenerateDIDDocument(publicURL string) (*DIDDocument, error) {
|
||||
// Parse URL to extract host and port
|
||||
did := p.did
|
||||
|
||||
// Parse URL for alsoKnownAs
|
||||
u, err := url.Parse(publicURL)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to parse public URL: %w", err)
|
||||
}
|
||||
|
||||
hostname := u.Hostname()
|
||||
port := u.Port()
|
||||
|
||||
// Build host string (include non-standard ports per did:web spec)
|
||||
host := hostname
|
||||
if port != "" && port != "80" && port != "443" {
|
||||
host = fmt.Sprintf("%s:%s", hostname, port)
|
||||
host := u.Hostname()
|
||||
if port := u.Port(); port != "" && port != "80" && port != "443" {
|
||||
host = fmt.Sprintf("%s:%s", host, port)
|
||||
}
|
||||
|
||||
did := fmt.Sprintf("did:web:%s", host)
|
||||
|
||||
// Get public key in multibase format using indigo's crypto
|
||||
pubKey, err := p.signingKey.PublicKey()
|
||||
if err != nil {
|
||||
@@ -106,6 +111,287 @@ func (p *HoldPDS) MarshalDIDDocument() ([]byte, error) {
|
||||
return json.MarshalIndent(doc, "", " ")
|
||||
}
|
||||
|
||||
// DIDConfig holds parameters for DID creation/loading.
|
||||
type DIDConfig struct {
|
||||
DID string // Explicit DID for adoption/recovery (optional)
|
||||
DIDMethod string // "web" or "plc"
|
||||
PublicURL string
|
||||
DBPath string
|
||||
SigningKeyPath string
|
||||
RotationKeyPath string
|
||||
PLCDirectoryURL string
|
||||
}
|
||||
|
||||
// LoadOrCreateDID returns the hold's DID, either by deriving it from the URL (did:web)
|
||||
// or by loading/creating a did:plc identity registered with the PLC directory.
|
||||
//
|
||||
// For did:plc, the priority is: config DID > did.txt > create new.
|
||||
// When an existing DID is found (config or did.txt), EnsurePLCCurrent is called
|
||||
// to auto-update the PLC directory if the signing key or URL has changed.
|
||||
func LoadOrCreateDID(ctx context.Context, cfg DIDConfig) (string, error) {
|
||||
if cfg.DIDMethod != "plc" {
|
||||
return GenerateDIDFromURL(cfg.PublicURL), nil
|
||||
}
|
||||
|
||||
didPath := filepath.Join(cfg.DBPath, "did.txt")
|
||||
|
||||
// Priority: config DID > did.txt > create new
|
||||
var did string
|
||||
if cfg.DID != "" {
|
||||
if !strings.HasPrefix(cfg.DID, "did:plc:") {
|
||||
return "", fmt.Errorf("database.did must be a did:plc identifier, got %q", cfg.DID)
|
||||
}
|
||||
did = cfg.DID
|
||||
slog.Info("Using DID from config (adoption/recovery)", "did", did)
|
||||
} else if data, err := os.ReadFile(didPath); err == nil {
|
||||
d := strings.TrimSpace(string(data))
|
||||
if strings.HasPrefix(d, "did:plc:") {
|
||||
did = d
|
||||
slog.Info("Loaded existing did:plc identity", "did", did)
|
||||
}
|
||||
}
|
||||
|
||||
if did != "" {
|
||||
// Persist to did.txt (may be from config on first adoption)
|
||||
if err := os.MkdirAll(filepath.Dir(didPath), 0755); err != nil {
|
||||
return "", fmt.Errorf("failed to create directory for did.txt: %w", err)
|
||||
}
|
||||
if err := os.WriteFile(didPath, []byte(did+"\n"), 0600); err != nil {
|
||||
return "", fmt.Errorf("failed to write did.txt: %w", err)
|
||||
}
|
||||
|
||||
// Load signing key (generate if missing — recovery case)
|
||||
signingKey, err := oauth.GenerateOrLoadPDSKey(cfg.SigningKeyPath)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to load signing key: %w", err)
|
||||
}
|
||||
|
||||
// Try to load rotation key (optional — may be stored offline)
|
||||
rotationKey, _ := loadOptionalK256Key(cfg.RotationKeyPath)
|
||||
|
||||
if err := EnsurePLCCurrent(ctx, did, rotationKey, signingKey, cfg.PublicURL, cfg.PLCDirectoryURL); err != nil {
|
||||
return "", fmt.Errorf("failed to ensure PLC identity is current: %w", err)
|
||||
}
|
||||
|
||||
return did, nil
|
||||
}
|
||||
|
||||
// No existing DID — create new genesis operation
|
||||
slog.Info("Creating new did:plc identity")
|
||||
|
||||
// Load or generate signing key
|
||||
signingKey, err := oauth.GenerateOrLoadPDSKey(cfg.SigningKeyPath)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to load signing key: %w", err)
|
||||
}
|
||||
|
||||
// Load or generate rotation key
|
||||
rotationKey, err := oauth.GenerateOrLoadPDSKey(cfg.RotationKeyPath)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to load rotation key: %w", err)
|
||||
}
|
||||
|
||||
did, err = CreatePLCIdentity(ctx, rotationKey, signingKey, cfg.PublicURL, cfg.PLCDirectoryURL)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to create PLC identity: %w", err)
|
||||
}
|
||||
|
||||
// Persist DID
|
||||
if err := os.MkdirAll(filepath.Dir(didPath), 0755); err != nil {
|
||||
return "", fmt.Errorf("failed to create directory for did.txt: %w", err)
|
||||
}
|
||||
if err := os.WriteFile(didPath, []byte(did+"\n"), 0600); err != nil {
|
||||
return "", fmt.Errorf("failed to write did.txt: %w", err)
|
||||
}
|
||||
|
||||
slog.Info("Created did:plc identity",
|
||||
"did", did,
|
||||
"plc_directory", cfg.PLCDirectoryURL,
|
||||
)
|
||||
slog.Warn("Back up rotation.key and optionally remove it from the server. It is only needed for DID updates (URL changes, key rotation).",
|
||||
"rotation_key_path", cfg.RotationKeyPath,
|
||||
)
|
||||
|
||||
return did, nil
|
||||
}
|
||||
|
||||
// loadOptionalK256Key attempts to load a K-256 private key from disk.
|
||||
// Returns nil if the file does not exist (key stored offline).
|
||||
func loadOptionalK256Key(path string) (*atcrypto.PrivateKeyK256, error) {
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
key, err := atcrypto.ParsePrivateBytesK256(data)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to parse K-256 key from %s: %w", path, err)
|
||||
}
|
||||
return key, nil
|
||||
}
|
||||
|
||||
// EnsurePLCCurrent checks the PLC directory for the given DID and updates it
|
||||
// if the local signing key or public URL doesn't match what's registered.
|
||||
// If rotationKey is nil, mismatches are logged as warnings but not fatal.
|
||||
func EnsurePLCCurrent(ctx context.Context, did string, rotationKey, signingKey *atcrypto.PrivateKeyK256, publicURL, plcDirectoryURL string) error {
|
||||
client := &didplc.Client{DirectoryURL: plcDirectoryURL}
|
||||
|
||||
// Fetch current op log
|
||||
opLog, err := client.OpLog(ctx, did)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to fetch PLC op log for %s: %w", did, err)
|
||||
}
|
||||
if len(opLog) == 0 {
|
||||
return fmt.Errorf("empty op log for %s", did)
|
||||
}
|
||||
|
||||
lastEntry := opLog[len(opLog)-1]
|
||||
lastOp := lastEntry.Regular
|
||||
if lastOp == nil {
|
||||
// Last op is not a regular op (could be legacy or tombstone) — skip update
|
||||
slog.Warn("Last PLC operation is not a regular op, skipping auto-update", "did", did)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Compare local state vs PLC state
|
||||
sigPub, err := signingKey.PublicKey()
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to get signing public key: %w", err)
|
||||
}
|
||||
localVerificationKey := sigPub.DIDKey()
|
||||
plcVerificationKey := lastOp.VerificationMethods["atproto"]
|
||||
|
||||
localEndpoint := publicURL
|
||||
var plcEndpoint string
|
||||
if svc, ok := lastOp.Services["atproto_pds"]; ok {
|
||||
plcEndpoint = svc.Endpoint
|
||||
}
|
||||
|
||||
keyMatch := localVerificationKey == plcVerificationKey
|
||||
endpointMatch := localEndpoint == plcEndpoint
|
||||
|
||||
if keyMatch && endpointMatch {
|
||||
slog.Info("PLC identity is current", "did", did)
|
||||
return nil
|
||||
}
|
||||
|
||||
slog.Info("PLC identity needs update",
|
||||
"did", did,
|
||||
"signing_key_changed", !keyMatch,
|
||||
"endpoint_changed", !endpointMatch,
|
||||
)
|
||||
|
||||
if rotationKey == nil {
|
||||
slog.Warn("PLC document doesn't match local state but no rotation key available. Provide rotation key to auto-update PLC directory.",
|
||||
"did", did,
|
||||
"signing_key_changed", !keyMatch,
|
||||
"endpoint_changed", !endpointMatch,
|
||||
)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Build update operation
|
||||
rotPub, err := rotationKey.PublicKey()
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to get rotation public key: %w", err)
|
||||
}
|
||||
|
||||
// Extract hostname for alsoKnownAs
|
||||
u, err := url.Parse(publicURL)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to parse public URL: %w", err)
|
||||
}
|
||||
host := u.Hostname()
|
||||
if port := u.Port(); port != "" && port != "80" && port != "443" {
|
||||
host = host + ":" + port
|
||||
}
|
||||
|
||||
prevCID := lastEntry.AsOperation().CID().String()
|
||||
|
||||
op := &didplc.RegularOp{
|
||||
Type: "plc_operation",
|
||||
RotationKeys: []string{rotPub.DIDKey()},
|
||||
VerificationMethods: map[string]string{
|
||||
"atproto": localVerificationKey,
|
||||
},
|
||||
AlsoKnownAs: []string{"at://" + host},
|
||||
Services: map[string]didplc.OpService{
|
||||
"atproto_pds": {Type: "AtprotoPersonalDataServer", Endpoint: publicURL},
|
||||
"atcr_hold": {Type: "AtcrHoldService", Endpoint: publicURL},
|
||||
},
|
||||
Prev: &prevCID,
|
||||
}
|
||||
|
||||
if err := op.Sign(rotationKey); err != nil {
|
||||
return fmt.Errorf("failed to sign PLC update operation: %w", err)
|
||||
}
|
||||
|
||||
if err := client.Submit(ctx, did, op); err != nil {
|
||||
return fmt.Errorf("failed to submit PLC update: %w", err)
|
||||
}
|
||||
|
||||
slog.Info("Updated PLC identity",
|
||||
"did", did,
|
||||
"signing_key_rotated", !keyMatch,
|
||||
"endpoint_changed", !endpointMatch,
|
||||
)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// CreatePLCIdentity creates a new did:plc identity by building a genesis operation,
|
||||
// signing it with the rotation key, and submitting it to the PLC directory.
|
||||
func CreatePLCIdentity(ctx context.Context, rotationKey, signingKey *atcrypto.PrivateKeyK256, publicURL, plcDirectoryURL string) (string, error) {
|
||||
rotPub, err := rotationKey.PublicKey()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to get rotation public key: %w", err)
|
||||
}
|
||||
|
||||
sigPub, err := signingKey.PublicKey()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to get signing public key: %w", err)
|
||||
}
|
||||
|
||||
// Extract hostname for alsoKnownAs
|
||||
u, err := url.Parse(publicURL)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to parse public URL: %w", err)
|
||||
}
|
||||
host := u.Hostname()
|
||||
if port := u.Port(); port != "" && port != "80" && port != "443" {
|
||||
host = host + ":" + port
|
||||
}
|
||||
|
||||
op := &didplc.RegularOp{
|
||||
Type: "plc_operation",
|
||||
RotationKeys: []string{rotPub.DIDKey()},
|
||||
VerificationMethods: map[string]string{
|
||||
"atproto": sigPub.DIDKey(),
|
||||
},
|
||||
AlsoKnownAs: []string{"at://" + host},
|
||||
Services: map[string]didplc.OpService{
|
||||
"atproto_pds": {Type: "AtprotoPersonalDataServer", Endpoint: publicURL},
|
||||
"atcr_hold": {Type: "AtcrHoldService", Endpoint: publicURL},
|
||||
},
|
||||
Prev: nil,
|
||||
}
|
||||
|
||||
if err := op.Sign(rotationKey); err != nil {
|
||||
return "", fmt.Errorf("failed to sign PLC genesis operation: %w", err)
|
||||
}
|
||||
|
||||
did, err := op.DID()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to compute DID from genesis operation: %w", err)
|
||||
}
|
||||
|
||||
client := &didplc.Client{DirectoryURL: plcDirectoryURL}
|
||||
if err := client.Submit(ctx, did, op); err != nil {
|
||||
return "", fmt.Errorf("failed to submit genesis operation to PLC directory: %w", err)
|
||||
}
|
||||
|
||||
return did, nil
|
||||
}
|
||||
|
||||
// GenerateDIDFromURL creates a did:web identifier from a public URL
|
||||
// Example: "http://hold1.example.com:8080" -> "did:web:hold1.example.com:8080"
|
||||
// Note: Per did:web spec, non-standard ports (not 80/443) are included in the DID
|
||||
|
||||
@@ -0,0 +1,11 @@
|
||||
package pds
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io"
|
||||
)
|
||||
|
||||
// ExportToCAR streams the hold's repo as a CAR file to the writer.
|
||||
func (p *HoldPDS) ExportToCAR(ctx context.Context, w io.Writer) error {
|
||||
return p.repomgr.ReadRepo(ctx, p.uid, "", w)
|
||||
}
|
||||
@@ -0,0 +1,180 @@
|
||||
package pds
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"strings"
|
||||
|
||||
"github.com/bluesky-social/indigo/repo"
|
||||
"github.com/ipfs/go-cid"
|
||||
"go.opentelemetry.io/otel"
|
||||
)
|
||||
|
||||
// rawCBOR wraps raw bytes to satisfy cbg.CBORMarshaler.
|
||||
// Used to pass through record bytes from a CAR without decoding.
|
||||
type rawCBOR []byte
|
||||
|
||||
func (r rawCBOR) MarshalCBOR(w io.Writer) error {
|
||||
_, err := w.Write(r)
|
||||
return err
|
||||
}
|
||||
|
||||
// bulkRecord holds a single record to import.
|
||||
type bulkRecord struct {
|
||||
Collection string
|
||||
Rkey string
|
||||
Data rawCBOR
|
||||
}
|
||||
|
||||
// ImportResult summarizes a CAR import operation.
|
||||
type ImportResult struct {
|
||||
Total int
|
||||
PerCollection map[string]int
|
||||
}
|
||||
|
||||
// ImportFromCAR reads a CAR file and imports all records into the hold's repo.
|
||||
// Records are upserted (overwrite on conflict) in a single atomic commit.
|
||||
// The repo is initialized if it doesn't exist yet.
|
||||
func (p *HoldPDS) ImportFromCAR(ctx context.Context, r io.Reader) (*ImportResult, error) {
|
||||
// Ensure repo exists
|
||||
head, err := p.carstore.GetUserRepoHead(ctx, p.uid)
|
||||
if err != nil || !head.Defined() {
|
||||
if err := p.repomgr.InitNewActor(ctx, p.uid, "", p.did, "", "", ""); err != nil {
|
||||
return nil, fmt.Errorf("failed to initialize repo: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
// Parse the CAR into an in-memory repo
|
||||
sourceRepo, err := repo.ReadRepoFromCar(ctx, r)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to read CAR: %w", err)
|
||||
}
|
||||
|
||||
// Collect all records
|
||||
var records []bulkRecord
|
||||
err = sourceRepo.ForEach(ctx, "", func(k string, v cid.Cid) error {
|
||||
_, recBytes, err := sourceRepo.GetRecordBytes(ctx, k)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to get record bytes for %s: %w", k, err)
|
||||
}
|
||||
|
||||
parts := strings.SplitN(k, "/", 2)
|
||||
if len(parts) != 2 {
|
||||
return fmt.Errorf("unexpected record path format: %s", k)
|
||||
}
|
||||
|
||||
records = append(records, bulkRecord{
|
||||
Collection: parts[0],
|
||||
Rkey: parts[1],
|
||||
Data: rawCBOR(*recBytes),
|
||||
})
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to iterate CAR records: %w", err)
|
||||
}
|
||||
|
||||
if len(records) == 0 {
|
||||
return &ImportResult{PerCollection: map[string]int{}}, nil
|
||||
}
|
||||
|
||||
// Bulk upsert all records in a single commit
|
||||
if err := p.bulkImportRecords(ctx, records); err != nil {
|
||||
return nil, fmt.Errorf("failed to import records: %w", err)
|
||||
}
|
||||
|
||||
// Build result
|
||||
result := &ImportResult{
|
||||
Total: len(records),
|
||||
PerCollection: make(map[string]int),
|
||||
}
|
||||
for _, rec := range records {
|
||||
result.PerCollection[rec.Collection]++
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// bulkImportRecords writes all records in a single delta session + commit.
|
||||
// Each record is upserted: created if new, updated if exists.
|
||||
func (p *HoldPDS) bulkImportRecords(ctx context.Context, records []bulkRecord) error {
|
||||
ctx, span := otel.Tracer("repoman").Start(ctx, "BulkImportRecords")
|
||||
defer span.End()
|
||||
|
||||
unlock := p.repomgr.lockUser(ctx, p.uid)
|
||||
defer unlock()
|
||||
|
||||
rev, err := p.repomgr.cs.GetUserRepoRev(ctx, p.uid)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
ds, err := p.repomgr.cs.NewDeltaSession(ctx, p.uid, &rev)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
head := ds.BaseCid()
|
||||
r, err := repo.OpenRepo(ctx, ds, head)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
ops := make([]RepoOp, 0, len(records))
|
||||
for _, rec := range records {
|
||||
rpath := rec.Collection + "/" + rec.Rkey
|
||||
|
||||
// Check if record exists to determine create vs update
|
||||
_, _, getErr := r.GetRecordBytes(ctx, rpath)
|
||||
recordExists := getErr == nil
|
||||
|
||||
var cc cid.Cid
|
||||
var evtKind EventKind
|
||||
if recordExists {
|
||||
cc, err = r.UpdateRecord(ctx, rpath, rec.Data)
|
||||
evtKind = EvtKindUpdateRecord
|
||||
} else {
|
||||
cc, err = r.PutRecord(ctx, rpath, rec.Data)
|
||||
evtKind = EvtKindCreateRecord
|
||||
}
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to write %s: %w", rpath, err)
|
||||
}
|
||||
|
||||
ops = append(ops, RepoOp{
|
||||
Kind: evtKind,
|
||||
Collection: rec.Collection,
|
||||
Rkey: rec.Rkey,
|
||||
RecCid: &cc,
|
||||
})
|
||||
}
|
||||
|
||||
nroot, nrev, err := r.Commit(ctx, p.repomgr.kmgr.SignForUser)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
rslice, err := ds.CloseWithRoot(ctx, nroot, nrev)
|
||||
if err != nil {
|
||||
return fmt.Errorf("close with root: %w", err)
|
||||
}
|
||||
|
||||
var oldroot *cid.Cid
|
||||
if head.Defined() {
|
||||
oldroot = &head
|
||||
}
|
||||
|
||||
if p.repomgr.events != nil {
|
||||
p.repomgr.events(ctx, &RepoEvent{
|
||||
User: p.uid,
|
||||
OldRoot: oldroot,
|
||||
NewRoot: nroot,
|
||||
Rev: nrev,
|
||||
Since: &rev,
|
||||
Ops: ops,
|
||||
RepoSlice: rslice,
|
||||
})
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
+14
-4
@@ -71,11 +71,21 @@ func NewHoldServer(cfg *Config) (*HoldServer, error) {
|
||||
var xrpcHandler *pds.XRPCHandler
|
||||
var s3Service *s3.S3Service
|
||||
if cfg.Database.Path != "" {
|
||||
holdDID := pds.GenerateDIDFromURL(cfg.Server.PublicURL)
|
||||
slog.Info("Initializing embedded PDS", "did", holdDID)
|
||||
|
||||
ctx := context.Background()
|
||||
var err error
|
||||
|
||||
holdDID, err := pds.LoadOrCreateDID(ctx, pds.DIDConfig{
|
||||
DID: cfg.Database.DID,
|
||||
DIDMethod: cfg.Database.DIDMethod,
|
||||
PublicURL: cfg.Server.PublicURL,
|
||||
DBPath: cfg.Database.Path,
|
||||
SigningKeyPath: cfg.Database.KeyPath,
|
||||
RotationKeyPath: cfg.Database.RotationKeyPath,
|
||||
PLCDirectoryURL: cfg.Database.PLCDirectoryURL,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to resolve hold DID: %w", err)
|
||||
}
|
||||
slog.Info("Initializing embedded PDS", "did", holdDID)
|
||||
|
||||
if cfg.Database.Path != ":memory:" {
|
||||
// File mode: open centralized shared DB (supports embedded replica sync)
|
||||
|
||||
Reference in New Issue
Block a user