mirrors/at-container-registry
1
0
Fork 0
mirror of https://tangled.org/evan.jarrett.net/at-container-registry synced 2026-09-02 00:06:58 +00:00
Code Issues Packages Projects Releases Wiki Activity

add SBOM package diffing, verify hold-service captain records

Browse Source 
- diff view gains a Packages tab with added/removed/changed/unchanged
  package tables and purl-derived type/license/upstream links
- captain records verified against the DID's atcr_hold service before
  caching (processor + batch backfill), preventing forged holds
- fix empty-handle updates clobbering cached handles and colliding on
  the UNIQUE constraint
- move fillPrevCIDs into repo.go; DirectRepoOperator is now canonical,
  repomgr kept as a test oracle
- surface read-only crew status in hold selector
- reconcile docs
This commit is contained in:
Evan Jarrett
2026-06-13 12:49:03 -05:00
parent ab4a4ebf9d
commit 6758996300
59 changed files with 3747 additions and 10638 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+11 -10
View File
@@ -26,8 +26,8 @@ go build -o bin/oauth-helper ./cmd/oauth-helper
# Build scanner (separate module) # Build scanner (separate module)
cd scanner && go build -o ../bin/atcr-scanner ./cmd/scanner && cd .. cd scanner && go build -o ../bin/atcr-scanner ./cmd/scanner && cd ..
# Build hold with billing support (optional build tag) # Build appview with billing support (optional build tag; billing lives in pkg/billing/, appview-side only)
go build -tags billing -o bin/atcr-hold ./cmd/hold go build -tags billing -o bin/atcr-appview ./cmd/appview
# Tests # Tests
go test ./... # all tests go test ./... # all tests
@@ -72,20 +72,21 @@ ATCR uses **distribution/distribution** as a library, extending it via middlewar
### Four Components ### Four Components
1. **AppView** (`cmd/appview`) — OCI Distribution API server. Resolves identities, routes manifests to PDS, routes blobs to hold service, validates OAuth, issues registry JWTs. Includes web UI for browsing. 1. **AppView** (`cmd/appview`) — OCI Distribution API server. Resolves identities, routes manifests to PDS, routes blobs to hold service, validates OAuth, issues registry JWTs. Includes web UI for browsing.
2. **Hold Service** (`cmd/hold`) — BYOS blob storage. Embedded PDS with captain/crew/stats/scan records (all ATProto records in CAR store), S3-compatible storage, presigned URLs. Supports did:web (default) or did:plc identity with auto-recovery. Optional subsystems: admin UI, quotas, billing (Stripe), GC, scan dispatch, Bluesky status posts. 2. **Hold Service** (`cmd/hold`) — BYOS blob storage. Embedded PDS with captain/crew/stats/scan records (all ATProto records in CAR store), S3-compatible storage, presigned URLs. Supports did:web (default) or did:plc identity with auto-recovery. Optional subsystems: admin UI, quotas, GC, scan dispatch, Bluesky status posts.
3. **Scanner** (`scanner/cmd/scanner`) — Vulnerability scanning. Connects to hold via WebSocket, generates SBOMs (Syft), scans vulnerabilities (Grype). Priority queue with tier-based scheduling. 3. **Scanner** (`scanner/cmd/scanner`) — Vulnerability scanning. Connects to hold via WebSocket, generates SBOMs (Syft), scans vulnerabilities (Grype). Priority queue with tier-based scheduling.
4. **Credential Helper** (`cmd/credential-helper`) — Docker credential helper implementing ATProto OAuth flow, exchanges OAuth token for registry JWT. 4. **Credential Helper** (`cmd/credential-helper`) — Docker credential helper implementing ATProto OAuth flow, exchanges OAuth token for registry JWT.
### Request Flow Summary ### Request Flow Summary
**Push:** Client pushes to `atcr.io/<identity>/<image>:<tag>`. Registry middleware resolves identity → DID → PDS, discovers hold DID (from sailor profile `defaultHold` → legacy `io.atcr.hold` records → AppView default). Blobs go to hold via XRPC multipart upload (presigned S3 URLs). Manifests stored in user's PDS as `io.atcr.manifest` records with `holdDid` reference. **Push:** Client pushes to `atcr.io/<identity>/<image>:<tag>`. Registry middleware resolves identity → DID → PDS, discovers hold DID (from sailor profile `defaultHold` → AppView default). Blobs go to hold via XRPC multipart upload (presigned S3 URLs). Manifests stored in user's PDS as `io.atcr.manifest` records with `holdDid` reference.
**Pull:** AppView fetches manifest from user's PDS. The manifest's `holdDid` field tells where blobs were stored. Blobs fetched from that hold via presigned download URLs. Pull always uses the historical hold from the manifest, even if the user changed their default since pushing. **Pull:** AppView fetches manifest from user's PDS. The manifest's `holdDid` field tells where blobs were stored. Blobs fetched from that hold via presigned download URLs. Pull always uses the historical hold from the manifest, even if the user changed their default since pushing.
**Hold discovery priority** (in `findHoldDID()`, `pkg/appview/middleware/registry.go`): **Hold discovery priority** (in `findHoldDIDAndProfile()`, `pkg/appview/middleware/registry.go`):
1. Sailor profile's `defaultHold` (user preference) 1. Sailor profile's `defaultHold` (user preference)
2. User's `io.atcr.hold` records (legacy) 2. AppView's default hold (`server.managed_holds[0]`, the fallback)
3. AppView's `default_hold_did` (fallback)
After discovery, `resolveSuccessor()` applies a single-hop redirect: if the chosen hold's captain record declares a `successor` DID (migration redirect), blobs route to the successor instead. Single-hop only — successor chains are not followed.
### Name Resolution ### Name Resolution
@@ -178,7 +179,7 @@ The credential helper never manages OAuth tokens directly — AppView owns the O
ATCR uses **Viper** for config. YAML primary, env vars override. Generate defaults with `config init`. ATCR uses **Viper** for config. YAML primary, env vars override. Generate defaults with `config init`.
**Env var convention:** Prefix + YAML path with `_` separators: **Env var convention:** Prefix + YAML path with `_` separators:
- AppView: `ATCR_` (e.g., `ATCR_SERVER_DEFAULT_HOLD_DID`) - AppView: `ATCR_` (e.g., `ATCR_SERVER_MANAGED_HOLDS`)
- Hold: `HOLD_` (e.g., `HOLD_SERVER_PUBLIC_URL`) - Hold: `HOLD_` (e.g., `HOLD_SERVER_PUBLIC_URL`)
- S3: standard AWS names (`AWS_ACCESS_KEY_ID`, `S3_BUCKET`, `S3_ENDPOINT`) - S3: standard AWS names (`AWS_ACCESS_KEY_ID`, `S3_BUCKET`, `S3_ENDPOINT`)
- Scanner: `SCANNER_` prefix (env-only, no Viper) - Scanner: `SCANNER_` prefix (env-only, no Viper)
@@ -194,7 +195,7 @@ See `config-appview.example.yaml` and `config-hold.example.yaml` for all options
- **Hold DID lookups use database** (`manifests` table), not in-memory cache — persistent across restarts - **Hold DID lookups use database** (`manifests` table), not in-memory cache — persistent across restarts
- **Context keys** (`auth.method`, `puller.did`) exist because `Repository()` receives `context.Context` from the distribution library interface — context values are the only way to pass data from HTTP middleware into the distribution middleware layer. Both are copied into `RegistryContext` inside `Repository()`. - **Context keys** (`auth.method`, `puller.did`) exist because `Repository()` receives `context.Context` from the distribution library interface — context values are the only way to pass data from HTTP middleware into the distribution middleware layer. Both are copied into `RegistryContext` inside `Repository()`.
- **OAuth key types**: AppView uses P-256 (ES256) for OAuth, not K-256 like PDS keys - **OAuth key types**: AppView uses P-256 (ES256) for OAuth, not K-256 like PDS keys
- **Confidential vs public clients**: Production uses P-256 key at `/var/lib/atcr/oauth/client.key` (auto-generated); localhost is always public client - **Confidential vs public clients**: Production uses a P-256 OAuth key and an RSA JWT signing key, both stored in the appview SQLite DB `crypto_keys` table (keys `oauth_p256` and `jwt_rsa`, auto-generated on first boot — see `pkg/appview/crypto_keys.go`). Only the JWT cert (`auth.cert_path`) is written to disk, regenerated each boot for the distribution library. Localhost is always a public client.
- **Hold stats are ATProto records in CAR store** — `io.atcr.hold.stats` records are stored via `repomgr.PutRecord()`, not in SQLite. Lost if CAR store is lost without backup. - **Hold stats are ATProto records in CAR store** — `io.atcr.hold.stats` records are stored via `repomgr.PutRecord()`, not in SQLite. Lost if CAR store is lost without backup.
- **PLC auto-update on boot** — When using did:plc, `LoadOrCreateDID()` calls `EnsurePLCCurrent()` every startup. If local signing key or URL doesn't match plc.directory, it auto-updates (requires rotation key on disk). - **PLC auto-update on boot** — When using did:plc, `LoadOrCreateDID()` calls `EnsurePLCCurrent()` every startup. If local signing key or URL doesn't match plc.directory, it auto-updates (requires rotation key on disk).
- **Hold CAR store is the source of truth** — Captain, crew, layer, stats, scan records, Bluesky posts, profiles are all ATProto records in the CAR store. SQLite holds only the records index and events. - **Hold CAR store is the source of truth** — Captain, crew, layer, stats, scan records, Bluesky posts, profiles are all ATProto records in the CAR store. SQLite holds only the records index and events.
@@ -216,7 +217,7 @@ See `config-appview.example.yaml` and `config-hold.example.yaml` for all options
**Changing name resolution:** **Changing name resolution:**
1. Modify `pkg/atproto/resolver.go` for DID/handle resolution 1. Modify `pkg/atproto/resolver.go` for DID/handle resolution
2. Update `pkg/appview/middleware/registry.go` if changing routing 2. Update `pkg/appview/middleware/registry.go` if changing routing
3. `findHoldDID()` checks: sailor profile → `io.atcr.hold` records (legacy) → default hold DID 3. `findHoldDIDAndProfile()` checks: sailor profile `defaultHold` → AppView default hold (`server.managed_holds[0]`), then `resolveSuccessor()` applies a single-hop successor redirect
**Working with OAuth client:** **Working with OAuth client:**
- Self-contained: pass `baseURL`, handles client ID/redirect URI/scopes - Self-contained: pass `baseURL`, handles client ID/redirect URI/scopes
Dockerfile.hold
+5 -20
View File
@@ -1,9 +1,5 @@
FROM mirror.gcr.io/library/golang:1.26.2-trixie AS builder FROM mirror.gcr.io/library/golang:1.26.2-trixie AS builder
# Build argument to enable Stripe billing integration
# Usage: docker build --build-arg BILLING_ENABLED=true -f Dockerfile.hold .
ARG BILLING_ENABLED=false
ENV DEBIAN_FRONTEND=noninteractive ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \ RUN apt-get update && \
@@ -21,22 +17,11 @@ COPY . .
RUN npm ci RUN npm ci
RUN go generate ./... RUN go generate ./...
# Conditionally add billing tag based on build arg RUN CGO_ENABLED=1 go build \
RUN if [ "$BILLING_ENABLED" = "true" ]; then \ -ldflags="-s -w -linkmode external -extldflags '-static'" \
echo "Building with Stripe billing support"; \ -tags sqlite_omit_load_extension \
CGO_ENABLED=1 go build \ -trimpath \
-ldflags="-s -w -linkmode external -extldflags '-static'" \ -o atcr-hold ./cmd/hold
-tags "sqlite_omit_load_extension,billing" \
-trimpath \
-o atcr-hold ./cmd/hold; \
else \
echo "Building without billing support"; \
CGO_ENABLED=1 go build \
-ldflags="-s -w -linkmode external -extldflags '-static'" \
-tags sqlite_omit_load_extension \
-trimpath \
-o atcr-hold ./cmd/hold; \
fi
RUN CGO_ENABLED=0 go build \ RUN CGO_ENABLED=0 go build \
-ldflags="-s -w" \ -ldflags="-s -w" \
docker-compose.yml
-1
View File
@@ -88,7 +88,6 @@ services:
dockerfile: Dockerfile.dev dockerfile: Dockerfile.dev
args: args:
AIR_CONFIG: .air.hold.toml AIR_CONFIG: .air.hold.toml
BILLING_ENABLED: "true"
image: atcr-hold-dev:latest image: atcr-hold-dev:latest
container_name: atcr-hold container_name: atcr-hold
ports: ports:
docs/ADMIN_PANEL.md
-1403
View File
File diff suppressed because it is too large Load Diff
docs/APPVIEW-UI-FUTURE.md
+84 -54
View File
@@ -16,15 +16,19 @@ These features were implemented but weren't in the original future features list
| Feature | Location | Notes | | Feature | Location | Notes |
|---------|----------|-------| |---------|----------|-------|
| **Billing (Stripe)** | `pkg/hold/billing/` | Checkout sessions, customer portal, subscription webhooks, tier upgrades. Build with `-tags billing`. | | **Billing (Stripe)** | `pkg/billing/` | Checkout sessions, customer portal, subscription webhooks, tier upgrades. Build with `-tags billing`. |
| **Garbage collection** | `pkg/hold/gc/` | Mark-and-sweep for orphaned blobs. Preview (dry-run) and execute modes. Triggered from hold admin UI. | | **Garbage collection** | `pkg/hold/gc/` | Mark-and-sweep for orphaned blobs. Preview (dry-run) and execute modes. Triggered from hold admin UI. |
| **libSQL embedded replicas** | AppView + Hold | Sync to Turso, Bunny DB, or self-hosted libsql-server. Configurable sync interval. | | **libSQL embedded replicas** | AppView + Hold | Sync to Turso, Bunny DB, or self-hosted libsql-server. Configurable sync interval. |
| **Hold successor/migration** | `pkg/hold/` | Promote a hold as successor to migrate users to new storage. | | **Hold successor/migration** | `pkg/hold/` | Promote a hold as successor to migrate users to new storage. |
| **Relay management** | Hold admin | Manage firehose relay connections from admin panel. | | **Relay management** | Hold admin | Manage firehose relay connections from admin panel. |
| **Data export** | `pkg/appview/handlers/export.go` | GDPR-compliant export of all user data from AppView + all holds where user is member/captain. | | **Data export** | `pkg/appview/handlers/export.go` | GDPR-compliant export of all user data from AppView + all holds where user is member/captain. |
| **Dark/light mode** | AppView UI | System preference detection, toggle, localStorage persistence. | | **Dark/light mode** | AppView UI | System preference detection, toggle, localStorage persistence. |
| **Credential helper install page** | `/install` | Install scripts for macOS/Linux/Windows, version API. | | **Credential helper install page** | `/install` | Install scripts for macOS/Linux/Windows, version API, Homebrew formula (`Formula/`), self-updating from tangled releases. |
| **Stars** | AppView UI | Star/unstar repos stored as `io.atcr.star` ATProto records, counts displayed. | | **Stars** | AppView UI | Star/unstar repos stored as `io.atcr.sailor.star` ATProto records, counts displayed, starred-repos page at `/u/{handle}/starred`. |
| **Label service** | `pkg/labeler/`, `cmd/labeler/` | Standalone labeler for takedowns by DID/handle/repo/AT URI with audit trail. Holds listen for takedown labels; GC defers deletion for a grace period in case of reversal. |
| **Helm chart UI** | AppView UI | Chart-aware digest page: Chart.yaml metadata, dependencies, helm install/pull command switcher (`handlers/digest_content.go`, `holdclient/helm_config.go`). |
| **AI Image Advisor** | `pkg/appview/handlers/image_advisor.go` | Claude-powered image analysis (config + SBOM + vulns) for paid users. Gated on billing + `ClaudeAPIKey`. Suggestions cached in `advisor_suggestions` table. CLI companion at `cmd/image-advisor`. |
| **Go vanity import paths** | `pkg/appview/middleware/goimport.go` | `go install atcr.io/...` meta tags, browser visits redirect to source repo. Seamark-branded credential helper variant (`cmd/credential-helper/seamark`) and theme (`themes/seamark/`). |
--- ---
@@ -43,20 +47,17 @@ These features were implemented but weren't in the original future features list
- Automatic platform detection from manifest metadata - Automatic platform detection from manifest metadata
- Validate that all manifests are for the same image (different platforms) - Validate that all manifests are for the same image (different platforms)
### Layer Inspection & Visualization — NOT STARTED ### Layer Inspection & Visualization — PARTIAL
DB stores layer metadata (digest, size, media type, layer index) but there's no UI for any of this. **Layer details — DONE:**
- Digest page shows per-layer Dockerfile commands (from OCI config history), sizes, media types, empty-layer toggle (`handlers/digest.go`, `partials/layers-section.html`)
- Layer diff between two tags/digests: shared/rebuilt/added/removed via LCS on layer commands, with size delta summary (`handlers/diff.go`, `/diff/{handle}/{repo}?from=&to=`)
- Multi-arch aware: diff resolves platform children, intersects common platforms
**Layer details page:** **NOT STARTED:**
- Show Dockerfile command that created each layer (if available in history) - Compression ratio display
- Display layer size and compression ratio - File changes within each layer (added/modified/deleted files)
- Show file changes in each layer (added/modified/deleted files) - Layer deduplication stats (shared layers across images, storage savings)
- Visualize layer hierarchy (parent-child relationships)
**Layer deduplication stats:**
- Show which layers are shared across images
- Calculate storage savings from layer sharing
- Identify duplicate layers with different digests (potential optimization)
### Image Operations — PARTIAL (delete only) ### Image Operations — PARTIAL (delete only)
@@ -80,7 +81,7 @@ DB stores layer metadata (digest, size, media type, layer index) but there's no
- Rollback functionality - Rollback functionality
- Audit log of image operations - Audit log of image operations
### Vulnerability Scanning — DONE (backend) / NOT STARTED (UI) ### Vulnerability Scanning — DONE (backend + UI)
**Backend — DONE:** **Backend — DONE:**
- Separate scanner service (`scanner/` module) with Syft (SBOM) + Grype (vulnerabilities) - Separate scanner service (`scanner/` module) with Syft (SBOM) + Grype (vulnerabilities)
@@ -90,23 +91,26 @@ DB stores layer metadata (digest, size, media type, layer index) but there's no
- Automatic scanning dispatched by hold on manifest push - Automatic scanning dispatched by hold on manifest push
- See `docs/SBOM_SCANNING.md` - See `docs/SBOM_SCANNING.md`
**AppView UI — NOT STARTED:** **AppView UI — DONE:**
- Display CVE count by severity (critical, high, medium, low) - CVE count by severity badge (critical, high, medium, low) — `handlers/scan_result.go`, `partials/vuln-badge.html`
- Show detailed CVE information (description, CVSS score, affected packages) - Detailed CVE view: description, severity, affected packages, fix versions, NVD/GitHub advisory links — `handlers/vuln_details.go`, `partials/vuln-details.html`
- Filter images by vulnerability status - Vulnerability diff across tags/versions: fixed vs new vs unchanged, summarized by severity — `handlers/diff.go`
- Subscribe to CVE notifications for your images - Scan-completion webhooks (`scan:first`, `scan:all`, `scan:changed`) — see Webhooks section
- Compare vulnerability status across tags/versions
### Image Signing & Verification — NOT STARTED **NOT STARTED:**
- Filter images by vulnerability status (in search/browse)
- Subscribe to CVE notifications for your images (beyond scan webhooks)
Concept doc exists at `docs/SIGNATURE_INTEGRATION.md` but no implementation. ### Image Signing & Verification — NOT STARTED (concept + examples only)
Consolidated research/POC doc at `docs/research/IMAGE_SIGNING.md` plus example verify scripts and trust policy template in `examples/verification/` (reference an unbuilt `atcr-verify` CLI). No cosign/sigstore integration or active signing implementation.
- Sign images - Sign images
- Display signature verification status - Display signature verification status
- Display signature metadata - Display signature metadata
- Require signatures for protected repositories - Require signatures for protected repositories
### SBOM (Software Bill of Materials) — DONE (backend) / NOT STARTED (UI) ### SBOM (Software Bill of Materials) — DONE (backend) / PARTIAL (UI)
**Backend — DONE:** **Backend — DONE:**
- Syft generates SPDX JSON format SBOMs - Syft generates SPDX JSON format SBOMs
@@ -114,11 +118,13 @@ Concept doc exists at `docs/SIGNATURE_INTEGRATION.md` but no implementation.
- Blobs in S3, metadata in hold's PDS - Blobs in S3, metadata in hold's PDS
- Accessible via ORAS CLI and hold XRPC endpoints - Accessible via ORAS CLI and hold XRPC endpoints
**UI — NOT STARTED:** **UI — DONE:**
- Display package list from SBOM - Package list with names, versions, licenses, package types — `handlers/sbom_details.go`, `partials/sbom-details.html`
- Show license information - Export: copy as CSV, download raw SPDX JSON
- Compare SBOMs across versions: Packages tab on the diff page with changed/added/removed/unchanged sections — `computeSbomDiff` in `handlers/diff.go`
**NOT STARTED:**
- Link to upstream package sources - Link to upstream package sources
- Compare SBOMs across versions
--- ---
@@ -223,8 +229,9 @@ Hold management is implemented as a separate admin panel on the hold service its
### Social Features — PARTIAL (stars only) ### Social Features — PARTIAL (stars only)
**Stars — DONE:** **Stars — DONE:**
- Star/unstar repositories stored as `io.atcr.star` ATProto records - Star/unstar repositories stored as `io.atcr.sailor.star` ATProto records
- Star counts displayed on repository pages - Star counts displayed on repository pages
- Starred repositories page at `/u/{handle}/starred`
**NOT STARTED:** **NOT STARTED:**
- Follow other sailors - Follow other sailors
@@ -270,16 +277,28 @@ Hold management is implemented as a separate admin panel on the hold service its
- Overview of your images, holds, activity - Overview of your images, holds, activity
- Quick stats, recent activity, alerts - Quick stats, recent activity, alerts
### Pull Analytics — NOT STARTED ### Pull Analytics — PARTIAL
- Pull count per image/tag **DONE:**
- Pull count by client, geography, over time - Pull/push counts per repository stored in AppView DB (`repository_stats`), with daily time-series snapshots (`repository_stats_daily`)
- Pull counts displayed on repo cards, repository pages, and OpenGraph metadata
**NOT STARTED:**
- Growth/time-series charts (daily data is collected but not visualized)
- Per-tag breakdown
- Pull count by client, geography
- User analytics (authenticated vs anonymous) - User analytics (authenticated vs anonymous)
### Alerts & Notifications — NOT STARTED ### Alerts & Notifications — PARTIAL
- Alert types (quota exceeded, vulnerability detected, hold down, etc.) **DONE:**
- Notification channels (email, webhook, ATProto, Slack/Discord) - Storage quota alerts in settings UI (warning states at 80%, 95%, 100% usage) — `partials/storage_stats.html`
- Quota threshold webhooks and scan-completion webhooks, with Discord/Slack formatting — see Webhooks section
**NOT STARTED:**
- Email notifications
- ATProto/DM notification channel
- Hold-down alerts
--- ---
@@ -296,15 +315,24 @@ Hold management is implemented as a separate admin panel on the hold service its
- Interactive API explorer - Interactive API explorer
- Code examples, SDKs - Code examples, SDKs
### Webhooks — NOT STARTED ### Webhooks — DONE
- Repository-level webhook registration - Webhook registration UI in settings — `handlers/webhooks.go`, `partials/webhooks_list.html`
- Events: manifest.pushed, tag.created, scan.completed, etc. - Triggers: `push`, `scan:first`, `scan:all`, `scan:changed`, `quota` (configurable threshold percent) — `pkg/appview/webhooks/`
- Test, retry, delivery history - HMAC signing, retry with backoff, test delivery, Discord/Slack auto-detection and formatting
- Tier limits: free 1 webhook (`push` + `scan:first`), paid per plan, captain unlimited
- See `docs/WEBHOOKS.md`
### CI/CD Integration — NOT STARTED **NOT STARTED:**
- Pull-event webhooks (scalability concern, needs batching/throttling — see WEBHOOKS.md)
- Delivery history UI
- GitHub Actions, GitLab CI, CircleCI example workflows ### CI/CD Integration — PARTIAL
**DONE:**
- Example GitHub Actions and GitLab CI workflows in `examples/plugins/ci-cd/` (signature verification + deploy; reference the unbuilt `atcr-verify` CLI)
**NOT STARTED:**
- Pre-built actions/plugins - Pre-built actions/plugins
- Build status badges - Build status badges
@@ -328,6 +356,7 @@ Hold management is implemented as a separate admin panel on the hold service its
- Install page with credential helper setup - Install page with credential helper setup
- Learn more page - Learn more page
- Internal developer docs (`docs/`) - Internal developer docs (`docs/`)
- Signup flow: PDS provider picker with curated list, branded OAuth handoff interstitial — `handlers/signup.go`
**NOT STARTED:** **NOT STARTED:**
- Interactive onboarding wizard - Interactive onboarding wizard
@@ -399,7 +428,7 @@ Hold management is implemented as a separate admin panel on the hold service its
### Billing — DONE ### Billing — DONE
- Stripe integration (`pkg/hold/billing/`, requires `-tags billing` build tag) - Stripe integration (`pkg/billing/`, requires `-tags billing` build tag)
- Checkout sessions, customer portal, subscription webhooks - Checkout sessions, customer portal, subscription webhooks
- Tier upgrades/downgrades - Tier upgrades/downgrades
@@ -433,20 +462,21 @@ These remain future ideas with no implementation:
2. ~~Vulnerability scanning integration~~ — backend complete 2. ~~Vulnerability scanning integration~~ — backend complete
3. ~~Hold management dashboard~~ — implemented on hold admin panel 3. ~~Hold management dashboard~~ — implemented on hold admin panel
4. ~~Basic search~~ — working 4. ~~Basic search~~ — working
5. ~~Scan results UI in AppView~~ — badges, CVE details, diff across versions
6. ~~SBOM display UI in AppView~~ — package list, licenses, CSV/SPDX export
7. ~~Webhooks~~ — push/scan/quota triggers, Discord/Slack, tier limits
8. ~~Layer inspection UI~~ (was medium) — layer details + diff on digest page
**Remaining high priority:** **Remaining high priority:**
1. Scan results UI in AppView (backend exists, just needs frontend) 1. Enhanced search (filters, sorting, advanced queries, filter by vuln status)
2. SBOM display UI in AppView (backend exists, just needs frontend) 2. Richer sailor profiles (bio, stats, pinned repos)
3. Webhooks for CI/CD integration
4. Enhanced search (filters, sorting, advanced queries)
5. Richer sailor profiles (bio, stats, pinned repos)
**Medium priority:** **Medium priority:**
1. Layer inspection UI 1. Pull analytics charts (daily time-series data already collected, needs UI)
2. Pull analytics and monitoring 2. API documentation (Swagger/OpenAPI)
3. API documentation (Swagger/OpenAPI) 3. Tag management (promotion, protection, aliases)
4. Tag management (promotion, protection, aliases) 4. Onboarding wizard / getting started guide (signup flow exists)
5. Onboarding wizard / getting started guide 5. Crew invitations (invite by handle, invitation links, self-request flow)
**Low priority / long-term:** **Low priority / long-term:**
1. Team/organization accounts 1. Team/organization accounts
@@ -463,4 +493,4 @@ These remain future ideas with no implementation:
**Note:** This is a living document. Features may be added, removed, or reprioritized based on user feedback, technical feasibility, and ATProto ecosystem evolution. **Note:** This is a living document. Features may be added, removed, or reprioritized based on user feedback, technical feasibility, and ATProto ecosystem evolution.
*Last audited: 2026-02-12* *Last audited: 2026-06-11*
docs/ATCR_VERIFY_CLI.md
-728
View File
@@ -1,728 +0,0 @@
# atcr-verify CLI Tool
## Overview
`atcr-verify` is a command-line tool for verifying ATProto signatures on container images stored in ATCR. It provides cryptographic verification of image manifests using ATProto's DID-based trust model.
## Features
- ✅ Verify ATProto signatures via OCI Referrers API
- ✅ DID resolution and public key extraction
- ✅ PDS query and commit signature verification
- ✅ Trust policy enforcement
- ✅ Offline verification mode (with cached data)
- ✅ Multiple output formats (human-readable, JSON, quiet)
- ✅ Exit codes for CI/CD integration
- ✅ Kubernetes admission controller integration
## Installation
### Binary Release
```bash
# Linux (x86_64)
curl -L https://github.com/atcr-io/atcr/releases/latest/download/atcr-verify-linux-amd64 -o atcr-verify
chmod +x atcr-verify
sudo mv atcr-verify /usr/local/bin/
# macOS (Apple Silicon)
curl -L https://github.com/atcr-io/atcr/releases/latest/download/atcr-verify-darwin-arm64 -o atcr-verify
chmod +x atcr-verify
sudo mv atcr-verify /usr/local/bin/
# Windows
curl -L https://github.com/atcr-io/atcr/releases/latest/download/atcr-verify-windows-amd64.exe -o atcr-verify.exe
```
### From Source
```bash
git clone https://github.com/atcr-io/atcr.git
cd atcr
go install ./cmd/atcr-verify
```
### Container Image
```bash
docker pull atcr.io/atcr/verify:latest
# Run
docker run --rm atcr.io/atcr/verify:latest verify IMAGE
```
## Usage
### Basic Verification
```bash
# Verify an image
atcr-verify atcr.io/alice/myapp:latest
# Output:
# ✓ Image verified successfully
# Signed by: alice.bsky.social (did:plc:alice123)
# Signed at: 2025-10-31T12:34:56.789Z
```
### With Trust Policy
```bash
# Verify against trust policy
atcr-verify atcr.io/alice/myapp:latest --policy trust-policy.yaml
# Output:
# ✓ Image verified successfully
# ✓ Trust policy satisfied
# Policy: production-images
# Trusted DID: did:plc:alice123
```
### JSON Output
```bash
atcr-verify atcr.io/alice/myapp:latest --output json
# Output:
{
"verified": true,
"image": "atcr.io/alice/myapp:latest",
"digest": "sha256:abc123...",
"signature": {
"did": "did:plc:alice123",
"handle": "alice.bsky.social",
"pds": "https://bsky.social",
"recordUri": "at://did:plc:alice123/io.atcr.manifest/abc123",
"commitCid": "bafyreih8...",
"signedAt": "2025-10-31T12:34:56.789Z",
"algorithm": "ECDSA-K256-SHA256"
},
"trustPolicy": {
"satisfied": true,
"policy": "production-images",
"trustedDID": true
}
}
```
### Quiet Mode
```bash
# Exit code only (for scripts)
atcr-verify atcr.io/alice/myapp:latest --quiet
echo $? # 0 = verified, 1 = failed
```
### Offline Mode
```bash
# Export verification bundle
atcr-verify export atcr.io/alice/myapp:latest -o bundle.json
# Verify offline (in air-gapped environment)
atcr-verify atcr.io/alice/myapp:latest --offline --bundle bundle.json
```
## Command Reference
### verify
Verify ATProto signature for an image.
```bash
atcr-verify verify IMAGE [flags]
atcr-verify IMAGE [flags] # 'verify' subcommand is optional
```
**Arguments:**
- `IMAGE` - Image reference (registry/owner/repo:tag or @digest)
**Flags:**
- `--policy FILE` - Trust policy file (default: none)
- `--output FORMAT` - Output format: text, json, quiet (default: text)
- `--offline` - Offline mode (requires --bundle)
- `--bundle FILE` - Verification bundle for offline mode
- `--cache-dir DIR` - Cache directory for DID documents (default: ~/.atcr/cache)
- `--no-cache` - Disable caching
- `--timeout DURATION` - Verification timeout (default: 30s)
- `--verbose` - Verbose output
**Exit Codes:**
- `0` - Verification succeeded
- `1` - Verification failed
- `2` - Invalid arguments
- `3` - Network error
- `4` - Trust policy violation
**Examples:**
```bash
# Basic verification
atcr-verify atcr.io/alice/myapp:latest
# With specific digest
atcr-verify atcr.io/alice/myapp@sha256:abc123...
# With trust policy
atcr-verify atcr.io/alice/myapp:latest --policy production-policy.yaml
# JSON output for scripting
atcr-verify atcr.io/alice/myapp:latest --output json | jq .verified
# Quiet mode for CI/CD
if atcr-verify atcr.io/alice/myapp:latest --quiet; then
echo "Deploy approved"
fi
```
### export
Export verification bundle for offline verification.
```bash
atcr-verify export IMAGE [flags]
```
**Arguments:**
- `IMAGE` - Image reference to export bundle for
**Flags:**
- `-o, --output FILE` - Output file (default: stdout)
- `--include-did-docs` - Include DID documents in bundle
- `--include-commit` - Include ATProto commit data
**Examples:**
```bash
# Export to file
atcr-verify export atcr.io/alice/myapp:latest -o myapp-bundle.json
# Export with all verification data
atcr-verify export atcr.io/alice/myapp:latest \
--include-did-docs \
--include-commit \
-o complete-bundle.json
# Export for multiple images
for img in $(cat images.txt); do
atcr-verify export $img -o bundles/$(echo $img | tr '/:' '_').json
done
```
### trust
Manage trust policies and trusted DIDs.
```bash
atcr-verify trust COMMAND [flags]
```
**Subcommands:**
**`trust list`** - List trusted DIDs
```bash
atcr-verify trust list
# Output:
# Trusted DIDs:
# - did:plc:alice123 (alice.bsky.social)
# - did:plc:bob456 (bob.example.com)
```
**`trust add DID`** - Add trusted DID
```bash
atcr-verify trust add did:plc:alice123
atcr-verify trust add did:plc:alice123 --name "Alice (DevOps)"
```
**`trust remove DID`** - Remove trusted DID
```bash
atcr-verify trust remove did:plc:alice123
```
**`trust policy validate`** - Validate trust policy file
```bash
atcr-verify trust policy validate policy.yaml
```
### version
Show version information.
```bash
atcr-verify version
# Output:
# atcr-verify version 1.0.0
# Go version: go1.21.5
# Commit: 3b5b89b
# Built: 2025-10-31T12:00:00Z
```
## Trust Policy
Trust policies define which signatures to trust and what to do when verification fails.
### Policy File Format
```yaml
version: 1.0
# Global settings
defaultAction: enforce # enforce, audit, allow
requireSignature: true
# Policies matched by image pattern (first match wins)
policies:
- name: production-images
description: "Production images must be signed by DevOps or Security"
scope: "atcr.io/*/prod-*"
require:
signature: true
trustedDIDs:
- did:plc:devops-team
- did:plc:security-team
minSignatures: 1
maxAge: 2592000 # 30 days in seconds
action: enforce
- name: staging-images
scope: "atcr.io/*/staging-*"
require:
signature: true
trustedDIDs:
- did:plc:devops-team
- did:plc:developers
minSignatures: 1
action: enforce
- name: dev-images
scope: "atcr.io/*/dev-*"
require:
signature: false
action: audit # Log but don't fail
# Trusted DID registry
trustedDIDs:
did:plc:devops-team:
name: "DevOps Team"
validFrom: "2024-01-01T00:00:00Z"
expiresAt: null
contact: "devops@example.com"
did:plc:security-team:
name: "Security Team"
validFrom: "2024-01-01T00:00:00Z"
expiresAt: null
did:plc:developers:
name: "Developer Team"
validFrom: "2024-06-01T00:00:00Z"
expiresAt: "2025-12-31T23:59:59Z"
```
### Policy Matching
Policies are evaluated in order. First match wins.
**Scope patterns:**
- `atcr.io/*/*` - All ATCR images
- `atcr.io/myorg/*` - All images from myorg
- `atcr.io/*/prod-*` - All images with "prod-" prefix
- `atcr.io/myorg/myapp` - Specific repository
- `atcr.io/myorg/myapp:v*` - Tag pattern matching
### Policy Actions
**`enforce`** - Reject if policy fails
- Exit code 4
- Blocks deployment
**`audit`** - Log but allow
- Exit code 0 (success)
- Warning message printed
**`allow`** - Always allow
- No verification performed
- Exit code 0
### Policy Requirements
**`signature: true`** - Require signature present
**`trustedDIDs`** - List of trusted DIDs
```yaml
trustedDIDs:
- did:plc:alice123
- did:web:example.com
```
**`minSignatures`** - Minimum number of signatures required
```yaml
minSignatures: 2 # Require 2 signatures
```
**`maxAge`** - Maximum signature age in seconds
```yaml
maxAge: 2592000 # 30 days
```
**`algorithms`** - Allowed signature algorithms
```yaml
algorithms:
- ECDSA-K256-SHA256
```
## Verification Flow
### 1. Image Resolution
```
Input: atcr.io/alice/myapp:latest
↓
Resolve tag to digest
↓
Output: sha256:abc123...
```
### 2. Signature Discovery
```
Query OCI Referrers API:
GET /v2/alice/myapp/referrers/sha256:abc123
?artifactType=application/vnd.atproto.signature.v1+json
↓
Returns: List of signature artifacts
↓
Download signature metadata blobs
```
### 3. DID Resolution
```
Extract DID from signature: did:plc:alice123
↓
Query PLC directory:
GET https://plc.directory/did:plc:alice123
↓
Extract public key from DID document
```
### 4. PDS Query
```
Get PDS endpoint from DID document
↓
Query for manifest record:
GET {pds}/xrpc/com.atproto.repo.getRecord
?repo=did:plc:alice123
&collection=io.atcr.manifest
&rkey=abc123
↓
Get commit CID from record
↓
Fetch commit data (includes signature)
```
### 5. Signature Verification
```
Extract signature bytes from commit
↓
Compute commit hash (SHA-256)
↓
Verify: ECDSA_K256(hash, signature, publicKey)
↓
Result: Valid or Invalid
```
### 6. Trust Policy Evaluation
```
Check if DID is in trustedDIDs list
↓
Check signature age < maxAge
↓
Check minSignatures satisfied
↓
Apply policy action (enforce/audit/allow)
```
## Integration Examples
### CI/CD Pipeline
**GitHub Actions:**
```yaml
name: Deploy
on:
push:
branches: [main]
jobs:
verify-and-deploy:
runs-on: ubuntu-latest
steps:
- name: Install atcr-verify
run: |
curl -L https://github.com/atcr-io/atcr/releases/latest/download/atcr-verify-linux-amd64 -o atcr-verify
chmod +x atcr-verify
sudo mv atcr-verify /usr/local/bin/
- name: Verify image signature
run: |
atcr-verify ${{ env.IMAGE }} --policy .github/trust-policy.yaml
- name: Deploy to production
if: success()
run: kubectl set image deployment/app app=${{ env.IMAGE }}
```
**GitLab CI:**
```yaml
verify:
stage: verify
image: atcr.io/atcr/verify:latest
script:
- atcr-verify ${IMAGE} --policy trust-policy.yaml
deploy:
stage: deploy
dependencies:
- verify
script:
- kubectl set image deployment/app app=${IMAGE}
```
**Jenkins:**
```groovy
pipeline {
agent any
stages {
stage('Verify') {
steps {
sh 'atcr-verify ${IMAGE} --policy trust-policy.yaml'
}
}
stage('Deploy') {
when {
expression { currentBuild.result == 'SUCCESS' }
}
steps {
sh 'kubectl set image deployment/app app=${IMAGE}'
}
}
}
}
```
### Kubernetes Admission Controller
**Using as webhook backend:**
```go
// webhook server
func (h *Handler) ValidatePod(w http.ResponseWriter, r *http.Request) {
var admReq admissionv1.AdmissionReview
json.NewDecoder(r.Body).Decode(&admReq)
pod := &corev1.Pod{}
json.Unmarshal(admReq.Request.Object.Raw, pod)
// Verify each container image
for _, container := range pod.Spec.Containers {
cmd := exec.Command("atcr-verify", container.Image,
"--policy", "/etc/atcr/trust-policy.yaml",
"--quiet")
if err := cmd.Run(); err != nil {
// Verification failed
admResp := admissionv1.AdmissionReview{
Response: &admissionv1.AdmissionResponse{
UID: admReq.Request.UID,
Allowed: false,
Result: &metav1.Status{
Message: fmt.Sprintf("Image %s failed signature verification", container.Image),
},
},
}
json.NewEncoder(w).Encode(admResp)
return
}
}
// All images verified
admResp := admissionv1.AdmissionReview{
Response: &admissionv1.AdmissionResponse{
UID: admReq.Request.UID,
Allowed: true,
},
}
json.NewEncoder(w).Encode(admResp)
}
```
### Pre-Pull Verification
**Systemd service:**
```ini
# /etc/systemd/system/myapp.service
[Unit]
Description=My Application
After=docker.service
[Service]
Type=oneshot
ExecStartPre=/usr/local/bin/atcr-verify atcr.io/myorg/myapp:latest --policy /etc/atcr/policy.yaml
ExecStartPre=/usr/bin/docker pull atcr.io/myorg/myapp:latest
ExecStart=/usr/bin/docker run atcr.io/myorg/myapp:latest
Restart=on-failure
[Install]
WantedBy=multi-user.target
```
**Docker wrapper script:**
```bash
#!/bin/bash
# docker-secure-pull.sh
IMAGE="$1"
# Verify before pulling
if ! atcr-verify "$IMAGE" --policy ~/.atcr/trust-policy.yaml; then
echo "ERROR: Image signature verification failed"
exit 1
fi
# Pull if verified
docker pull "$IMAGE"
```
## Configuration
### Config File
Location: `~/.atcr/config.yaml`
```yaml
# Default trust policy
defaultPolicy: ~/.atcr/trust-policy.yaml
# Cache settings
cache:
enabled: true
directory: ~/.atcr/cache
ttl:
didDocuments: 3600 # 1 hour
commits: 600 # 10 minutes
# Network settings
timeout: 30s
retries: 3
# Output settings
output:
format: text # text, json, quiet
color: auto # auto, always, never
# Registry settings
registries:
atcr.io:
insecure: false
credentialsFile: ~/.docker/config.json
```
### Environment Variables
- `ATCR_CONFIG` - Config file path
- `ATCR_POLICY` - Default trust policy file
- `ATCR_CACHE_DIR` - Cache directory
- `ATCR_OUTPUT` - Output format (text, json, quiet)
- `ATCR_TIMEOUT` - Verification timeout
- `HTTP_PROXY` / `HTTPS_PROXY` - Proxy settings
- `NO_CACHE` - Disable caching
## Library Usage
`atcr-verify` can also be used as a Go library:
```go
import "github.com/atcr-io/atcr/pkg/verify"
func main() {
verifier := verify.NewVerifier(verify.Config{
Policy: policy,
Timeout: 30 * time.Second,
})
result, err := verifier.Verify(ctx, "atcr.io/alice/myapp:latest")
if err != nil {
log.Fatal(err)
}
if !result.Verified {
log.Fatal("Verification failed")
}
fmt.Printf("Verified by %s\n", result.Signature.DID)
}
```
## Performance
### Typical Verification Times
- **First verification:** 500-1000ms
- OCI Referrers API: 50-100ms
- DID resolution: 50-150ms
- PDS query: 100-300ms
- Signature verification: 1-5ms
- **Cached verification:** 50-150ms
- DID document cached
- Signature metadata cached
### Optimization Tips
1. **Enable caching** - DID documents change rarely
2. **Use offline bundles** - For air-gapped environments
3. **Parallel verification** - Verify multiple images concurrently
4. **Local trust policy** - Avoid remote policy fetches
## Troubleshooting
### Verification Fails
```bash
atcr-verify atcr.io/alice/myapp:latest --verbose
```
Common issues:
- **No signature found** - Image not signed, check Referrers API
- **DID resolution failed** - Network issue, check PLC directory
- **PDS unreachable** - Network issue, check PDS endpoint
- **Signature invalid** - Tampering detected or key mismatch
- **Trust policy violation** - DID not in trusted list
### Enable Debug Logging
```bash
ATCR_LOG_LEVEL=debug atcr-verify IMAGE
```
### Clear Cache
```bash
rm -rf ~/.atcr/cache
```
## See Also
- [ATProto Signatures](./ATPROTO_SIGNATURES.md) - How ATProto signing works
- [Integration Strategy](./INTEGRATION_STRATEGY.md) - Overview of integration approaches
- [Signature Integration](./SIGNATURE_INTEGRATION.md) - Tool-specific guides
- [Trust Policy Examples](../examples/verification/trust-policy.yaml)
docs/ATPROTO_SIGNATURES.md
-501
View File
@@ -1,501 +0,0 @@
# ATProto Signatures for Container Images
## Overview
ATCR container images are **already cryptographically signed** through ATProto's repository commit system. Every manifest stored in a user's PDS is signed with the user's ATProto signing key, providing cryptographic proof of authorship and integrity.
This document explains:
- How ATProto signing works
- Why additional signing tools aren't needed
- How to bridge ATProto signatures to the OCI/ORAS ecosystem
- Trust model and security considerations
## Key Insight: Manifests Are Already Signed
When you push an image to ATCR:
```bash
docker push atcr.io/alice/myapp:latest
```
The following happens:
1. **AppView stores manifest** as an `io.atcr.manifest` record in alice's PDS
2. **PDS creates repository commit** containing the manifest record
3. **PDS signs the commit** with alice's ATProto signing key (ECDSA K-256)
4. **Signature is stored** in the repository commit object
**Result:** The manifest is cryptographically signed with alice's private key, and anyone can verify it using alice's public key from her DID document.
## ATProto Signing Mechanism
### Repository Commit Signing
ATProto uses a Merkle Search Tree (MST) to store records, and every modification creates a signed commit:
```
┌─────────────────────────────────────────────┐
│ Repository Commit │
├─────────────────────────────────────────────┤
│ DID: did:plc:alice123 │
│ Version: 3jzfkjqwdwa2a │
│ Previous: bafyreig7... (parent commit) │
│ Data CID: bafyreih8... (MST root) │
│ ┌───────────────────────────────────────┐ │
│ │ Signature (ECDSA K-256 + SHA-256) │ │
│ │ Signed with: alice's private key │ │
│ │ Value: 0x3045022100... (DER format) │ │
│ └───────────────────────────────────────┘ │
└─────────────────────────────────────────────┘
│
↓
┌─────────────────────┐
│ Merkle Search Tree │
│ (contains records) │
└─────────────────────┘
│
↓
┌────────────────────────────┐
│ io.atcr.manifest record │
│ Repository: myapp │
│ Digest: sha256:abc123... │
│ Layers: [...] │
└────────────────────────────┘
```
### Signature Algorithm
**Algorithm:** ECDSA with K-256 (secp256k1) curve + SHA-256 hash
- **Curve:** secp256k1 (same as Bitcoin, Ethereum)
- **Hash:** SHA-256
- **Format:** DER-encoded signature bytes
- **Variant:** "low-S" signatures (per BIP-0062)
**Signing process:**
1. Serialize commit data as DAG-CBOR
2. Hash with SHA-256
3. Sign hash with ECDSA K-256 private key
4. Store signature in commit object
### Public Key Distribution
Public keys are distributed via DID documents, accessible through DID resolution:
**DID Resolution Flow:**
```
did:plc:alice123
↓
Query PLC directory: https://plc.directory/did:plc:alice123
↓
DID Document:
{
"@context": ["https://www.w3.org/ns/did/v1"],
"id": "did:plc:alice123",
"verificationMethod": [{
"id": "did:plc:alice123#atproto",
"type": "Multikey",
"controller": "did:plc:alice123",
"publicKeyMultibase": "zQ3shokFTS3brHcDQrn82RUDfCZESWL1ZdCEJwekUDdo1Ko4Z"
}],
"service": [{
"id": "#atproto_pds",
"type": "AtprotoPersonalDataServer",
"serviceEndpoint": "https://bsky.social"
}]
}
```
**Public key format:**
- **Encoding:** Multibase (base58btc with `z` prefix)
- **Codec:** Multicodec `0xE701` for K-256 keys
- **Example:** `zQ3sh...` decodes to 33-byte compressed public key
## Verification Process
To verify a manifest's signature:
### Step 1: Resolve Image to Manifest Digest
```bash
# Get manifest digest
DIGEST=$(crane digest atcr.io/alice/myapp:latest)
# Result: sha256:abc123...
```
### Step 2: Fetch Manifest Record from PDS
```bash
# Extract repository name from image reference
REPO="myapp"
# Query PDS for manifest record
curl "https://bsky.social/xrpc/com.atproto.repo.listRecords?\
repo=did:plc:alice123&\
collection=io.atcr.manifest&\
limit=100" | jq -r '.records[] | select(.value.digest == "sha256:abc123...")'
```
Response includes:
```json
{
"uri": "at://did:plc:alice123/io.atcr.manifest/abc123",
"cid": "bafyreig7...",
"value": {
"$type": "io.atcr.manifest",
"repository": "myapp",
"digest": "sha256:abc123...",
...
}
}
```
### Step 3: Fetch Repository Commit
```bash
# Get current repository state
curl "https://bsky.social/xrpc/com.atproto.sync.getRepo?\
did=did:plc:alice123" --output repo.car
# Extract commit from CAR file (requires ATProto tools)
# Commit includes signature over repository state
```
### Step 4: Resolve DID to Public Key
```bash
# Resolve DID document
curl "https://plc.directory/did:plc:alice123" | jq -r '.verificationMethod[0].publicKeyMultibase'
# Result: zQ3shokFTS3brHcDQrn82RUDfCZESWL1ZdCEJwekUDdo1Ko4Z
```
### Step 5: Verify Signature
```go
// Pseudocode for verification
import "github.com/bluesky-social/indigo/atproto/crypto"
// 1. Parse commit
commit := parseCommitFromCAR(repoCAR)
// 2. Extract signature bytes
signature := commit.Sig
// 3. Get bytes that were signed
bytesToVerify := commit.Unsigned().BytesForSigning()
// 4. Decode public key from multibase
pubKey := decodeMultibasePublicKey(publicKeyMultibase)
// 5. Verify ECDSA signature
valid := crypto.VerifySignature(pubKey, bytesToVerify, signature)
```
### Step 6: Verify Manifest Integrity
```bash
# Verify the manifest record's CID matches the content
# CID is content-addressed, so tampering changes the CID
```
## Bridging to OCI/ORAS Ecosystem
While ATProto signatures are cryptographically sound, the OCI ecosystem doesn't understand ATProto records. To make signatures discoverable, we create **ORAS signature artifacts** that reference the ATProto signature.
### ORAS Signature Artifact Format
```json
{
"schemaVersion": 2,
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"artifactType": "application/vnd.atproto.signature.v1+json",
"config": {
"mediaType": "application/vnd.oci.empty.v1+json",
"digest": "sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a",
"size": 2
},
"subject": {
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:abc123...",
"size": 1234
},
"layers": [
{
"mediaType": "application/vnd.atproto.signature.v1+json",
"digest": "sha256:sig789...",
"size": 512,
"annotations": {
"org.opencontainers.image.title": "atproto-signature.json"
}
}
],
"annotations": {
"io.atcr.atproto.did": "did:plc:alice123",
"io.atcr.atproto.pds": "https://bsky.social",
"io.atcr.atproto.recordUri": "at://did:plc:alice123/io.atcr.manifest/abc123",
"io.atcr.atproto.commitCid": "bafyreih8...",
"io.atcr.atproto.signedAt": "2025-10-31T12:34:56.789Z",
"io.atcr.atproto.keyId": "did:plc:alice123#atproto"
}
}
```
**Key elements:**
1. **artifactType**: `application/vnd.atproto.signature.v1+json` - identifies this as an ATProto signature
2. **subject**: Links to the image manifest being signed
3. **layers**: Contains signature metadata blob
4. **annotations**: Quick-access metadata for verification
### Signature Metadata Blob
The layer blob contains detailed verification information:
```json
{
"$type": "io.atcr.atproto.signature",
"version": "1.0",
"subject": {
"digest": "sha256:abc123...",
"mediaType": "application/vnd.oci.image.manifest.v1+json"
},
"atproto": {
"did": "did:plc:alice123",
"handle": "alice.bsky.social",
"pdsEndpoint": "https://bsky.social",
"recordUri": "at://did:plc:alice123/io.atcr.manifest/abc123",
"recordCid": "bafyreig7...",
"commitCid": "bafyreih8...",
"commitRev": "3jzfkjqwdwa2a",
"signedAt": "2025-10-31T12:34:56.789Z"
},
"signature": {
"algorithm": "ECDSA-K256-SHA256",
"keyId": "did:plc:alice123#atproto",
"publicKeyMultibase": "zQ3shokFTS3brHcDQrn82RUDfCZESWL1ZdCEJwekUDdo1Ko4Z"
},
"verification": {
"method": "atproto-repo-commit",
"instructions": "Fetch repository commit from PDS and verify signature using public key from DID document"
}
}
```
### Discovery via Referrers API
ORAS artifacts are discoverable via the OCI Referrers API:
```bash
# Query for signature artifacts
curl "https://atcr.io/v2/alice/myapp/referrers/sha256:abc123?\
artifactType=application/vnd.atproto.signature.v1+json"
```
Response:
```json
{
"schemaVersion": 2,
"mediaType": "application/vnd.oci.image.index.v1+json",
"manifests": [
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:sig789...",
"size": 1234,
"artifactType": "application/vnd.atproto.signature.v1+json",
"annotations": {
"io.atcr.atproto.did": "did:plc:alice123",
"io.atcr.atproto.signedAt": "2025-10-31T12:34:56.789Z"
}
}
]
}
```
## Trust Model
### What ATProto Signatures Prove
✅ **Authenticity**: Image was published by the DID owner
✅ **Integrity**: Image manifest hasn't been tampered with since signing
✅ **Non-repudiation**: Only the DID owner could have created this signature
✅ **Timestamp**: When the image was signed (commit timestamp)
### What ATProto Signatures Don't Prove
❌ **Safety**: Image doesn't contain vulnerabilities (use vulnerability scanning)
❌ **DID trustworthiness**: Whether the DID owner is trustworthy (trust policy decision)
❌ **Key security**: Private key wasn't compromised (same limitation as all PKI)
❌ **PDS honesty**: PDS operator serves correct data (verify across multiple sources)
### Trust Dependencies
1. **DID Resolution**: Must correctly resolve DID to public key
- **Mitigation**: Use multiple resolvers, cache DID documents
2. **PDS Availability**: Must query PDS to verify signatures
- **Mitigation**: Embed signature bytes in ORAS blob for offline verification
3. **PDS Honesty**: PDS could serve fake/unsigned records
- **Mitigation**: Signature verification prevents this (can't forge signature)
4. **Key Security**: User's private key could be compromised
- **Mitigation**: Key rotation via DID document updates, short-lived credentials
5. **Algorithm Security**: ECDSA K-256 must remain secure
- **Status**: Well-studied, same as Bitcoin/Ethereum (widely trusted)
### Comparison with Other Signing Systems
| Aspect | ATProto Signatures | Cosign (Keyless) | Notary v2 |
|--------|-------------------|------------------|-----------|
| **Identity** | DID (decentralized) | OIDC (federated) | X.509 (PKI) |
| **Key Management** | PDS signing keys | Ephemeral (Fulcio) | User-managed |
| **Trust Anchor** | DID resolution | Fulcio CA + Rekor | Certificate chain |
| **Transparency Log** | ATProto firehose | Rekor | Optional |
| **Offline Verification** | Limited* | No | Yes |
| **Decentralization** | High | Medium | Low |
| **Complexity** | Low | High | Medium |
*Can be improved by embedding signature bytes in ORAS blob
### Security Considerations
**Threat: Man-in-the-Middle Attack**
- **Attack**: Intercept PDS queries, serve fake records
- **Defense**: TLS for PDS communication, verify signature with public key from DID document
- **Result**: Attacker can't forge signature without private key
**Threat: Compromised PDS**
- **Attack**: PDS operator serves unsigned/fake manifests
- **Defense**: Signature verification fails (PDS can't sign without user's private key)
- **Result**: Protected
**Threat: Key Compromise**
- **Attack**: Attacker steals user's ATProto signing key
- **Defense**: Key rotation via DID document, revoke old keys
- **Result**: Same as any PKI system (rotate keys quickly)
**Threat: Replay Attack**
- **Attack**: Replay old signed manifest to rollback to vulnerable version
- **Defense**: Check commit timestamp, verify commit is in current repository DAG
- **Result**: Protected (commits form immutable chain)
**Threat: DID Takeover**
- **Attack**: Attacker gains control of user's DID (rotation keys)
- **Defense**: Monitor DID document changes, verify key history
- **Result**: Serious but requires compromising rotation keys (harder than signing keys)
## Implementation Strategy
### Automatic Signature Artifact Creation
When AppView stores a manifest in a user's PDS:
1. **Store manifest record** (existing behavior)
2. **Get commit response** with commit CID and revision
3. **Create ORAS signature artifact**:
- Build metadata blob (JSON)
- Upload blob to hold storage
- Create ORAS manifest with subject = image manifest
- Store ORAS manifest (creates referrer link)
### Storage Location
Signature artifacts follow the same pattern as SBOMs:
- **Metadata blobs**: Stored in hold's blob storage
- **ORAS manifests**: Stored in hold's embedded PDS
- **Discovery**: Via OCI Referrers API
### Verification Tools
**Option 1: Custom CLI tool (`atcr-verify`)**
```bash
atcr-verify atcr.io/alice/myapp:latest
# → Queries referrers API
# → Fetches signature metadata
# → Resolves DID → public key
# → Queries PDS for commit
# → Verifies signature
```
**Option 2: Shell script (curl + jq)**
- See `docs/SIGNATURE_INTEGRATION.md` for examples
**Option 3: Kubernetes admission controller**
- Custom webhook that runs verification
- Rejects pods with unsigned/invalid signatures
## Benefits of ATProto Signatures
### Compared to No Signing
✅ **Cryptographic proof** of image authorship
✅ **Tamper detection** for manifests
✅ **Identity binding** via DIDs
✅ **Audit trail** via ATProto repository history
### Compared to Cosign/Notary
✅ **No additional signing required** (already signed by PDS)
✅ **Decentralized identity** (DIDs, not CAs)
✅ **Simpler infrastructure** (no Fulcio, no Rekor, no TUF)
✅ **Consistent with ATCR's architecture** (ATProto-native)
✅ **Lower operational overhead** (reuse existing PDS infrastructure)
### Trade-offs
⚠️ **Custom verification tools required** (standard tools won't work)
⚠️ **Online verification preferred** (need to query PDS)
⚠️ **Different trust model** (trust DIDs, not CAs)
⚠️ **Ecosystem maturity** (newer approach, less tooling)
## Future Enhancements
### Short-term
1. **Offline verification**: Embed signature bytes in ORAS blob
2. **Multi-PDS verification**: Check signature across multiple PDSs
3. **Key rotation support**: Handle historical key validity
### Medium-term
4. **Timestamp service**: RFC 3161 timestamps for long-term validity
5. **Multi-signature**: Require N signatures from M DIDs
6. **Transparency log integration**: Record verifications in public log
### Long-term
7. **IANA registration**: Register `application/vnd.atproto.signature.v1+json`
8. **Standards proposal**: ATProto signature spec to ORAS/OCI
9. **Cross-ecosystem bridges**: Convert to Cosign/Notary formats
## Conclusion
ATCR images are already cryptographically signed through ATProto's repository commit system. By creating ORAS signature artifacts that reference these existing signatures, we can:
- ✅ Make signatures discoverable to OCI tooling
- ✅ Maintain ATProto as the source of truth
- ✅ Provide verification tools for users and clusters
- ✅ Avoid duplicating signing infrastructure
This approach leverages ATProto's strengths (decentralized identity, built-in signing) while bridging to the OCI ecosystem through standard ORAS artifacts.
## References
### ATProto Specifications
- [ATProto Repository Specification](https://atproto.com/specs/repository)
- [ATProto Data Model](https://atproto.com/specs/data-model)
- [ATProto DID Methods](https://atproto.com/specs/did)
### OCI/ORAS Specifications
- [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec)
- [OCI Referrers API](https://github.com/opencontainers/distribution-spec/blob/main/spec.md#listing-referrers)
- [ORAS Artifacts](https://oras.land/docs/)
### Cryptography
- [ECDSA (secp256k1)](https://en.bitcoin.it/wiki/Secp256k1)
- [Multibase Encoding](https://github.com/multiformats/multibase)
- [Multicodec](https://github.com/multiformats/multicodec)
### Related Documentation
- [SBOM Scanning](./SBOM_SCANNING.md) - Similar ORAS artifact pattern
- [Signature Integration](./SIGNATURE_INTEGRATION.md) - Practical integration examples
docs/BILLING.md
+94 -56
View File
@@ -1,33 +1,40 @@
# Hold Service Billing Integration # Billing Integration
Optional Stripe billing integration for hold services. Allows hold operators to charge for storage tiers via subscriptions. Optional Stripe billing integration. Allows charging for subscription tiers, which map to storage quotas and feature gates on managed holds.
## Overview ## Overview
- **Compile-time optional**: Build with `-tags billing` to enable Stripe support - **Compile-time optional**: Build the appview with `-tags billing` to enable Stripe support
- **Hold owns billing**: Each hold operator has their own Stripe account - **AppView owns billing**: All Stripe interaction (checkout, customer portal, webhook handling) lives in the appview (`pkg/billing/`)
- **AppView aggregates UI**: Fetches subscription info from holds, displays in settings - **Holds enforce quota**: On a subscription change, the appview pushes a tier update to each managed hold; the hold maps the tier rank to its own quota tier and enforces it
- **Customer-DID mapping**: DIDs stored in Stripe customer metadata (no extra database) - **Customer-DID mapping**: User DIDs are stored in Stripe customer metadata (no extra database)
## Architecture ## Architecture
``` ```
User → AppView Settings UI → Hold XRPC endpoints → Stripe User → AppView Settings UI → AppView (pkg/billing) → Stripe
↑
Stripe webhook → POST /api/stripe/webhook (AppView)
↓ ↓
Stripe webhook → Hold → Update crew tier io.atcr.hold.updateCrewTier (signed appview token) → Hold → update crew tier / enforce quota
``` ```
The appview is the sole billing authority: it creates checkout and portal sessions, receives Stripe webhooks at `POST /api/stripe/webhook`, and resolves the subscription's price ID to a tier rank. On a subscription change it calls each managed hold's `io.atcr.hold.updateCrewTier` endpoint (`pkg/appview/holdclient/tier_update.go`), authenticated with a short-lived JWT signed by the appview's P-256 key. The hold verifies that token against its configured appview DID and only then updates the crew member's quota tier (`pkg/hold/pds/xrpc.go`, `HandleUpdateCrewTier`). Holds never talk to Stripe and trust nothing but a valid appview-signed token; their job is quota enforcement, not payment.
## Building with Billing Support ## Building with Billing Support
Billing lives entirely in the AppView (`pkg/billing/`). The hold binary does not need a special build tag.
```bash ```bash
# Without billing (default) # AppView without billing (default)
go build ./cmd/hold go build -o bin/atcr-appview ./cmd/appview
# With billing # AppView with billing
go build -tags billing ./cmd/hold go build -tags billing -o bin/atcr-appview ./cmd/appview
# Docker with billing # Docker (Dockerfile.appview does not include -tags billing by default;
docker build --build-arg BILLING_ENABLED=true -f Dockerfile.hold . # build locally with the tag if you need billing support)
go build -tags billing -o bin/atcr-appview ./cmd/appview
``` ```
## Configuration ## Configuration
@@ -43,35 +50,67 @@ STRIPE_WEBHOOK_SECRET=whsec_xxx # from Stripe Dashboard or CLI
STRIPE_PUBLISHABLE_KEY=pk_live_xxx # for client-side (not currently used) STRIPE_PUBLISHABLE_KEY=pk_live_xxx # for client-side (not currently used)
``` ```
### quotas.yaml ### Billing tiers (appview config)
Stripe tiers are configured as a **list** under the `billing:` section of the appview config (`pkg/billing/config.go`). Position in the list determines tier rank (0-based, lowest to highest). Billing auto-enables when a Stripe secret key is set and at least one tier is configured.
```yaml ```yaml
tiers:
swabbie:
quota: 2GB
description: "Starter storage"
# No stripe_price = free tier
deckhand:
quota: 5GB
description: "Standard storage"
stripe_price_yearly: price_xxx # Price ID from Stripe
bosun:
quota: 10GB
description: "Mid-level storage"
stripe_price_monthly: price_xxx
stripe_price_yearly: price_xxx
defaults:
new_crew_tier: swabbie
plankowner_crew_tier: deckhand # Early adopters get this free
billing: billing:
enabled: true # Can also be set via STRIPE_SECRET_KEY env var (takes precedence).
stripe_secret_key: sk_live_xxx
# Can also be set via STRIPE_WEBHOOK_SECRET env var (takes precedence).
webhook_secret: whsec_xxx
currency: usd currency: usd
success_url: "{hold_url}/billing/success" success_url: "{base_url}/settings/billing"
cancel_url: "{hold_url}/billing/cancel" cancel_url: "{base_url}/settings/billing"
tiers:
- name: Free
description: Get started with basic storage
features: []
stripe_price_monthly: "" # empty = free tier
stripe_price_yearly: ""
max_webhooks: 1
webhook_all_triggers: false
ai_advisor: false
supporter_badge: false
- name: Supporter
description: Support the project
stripe_price_yearly: price_xxx
max_webhooks: 1
webhook_all_triggers: true
ai_advisor: true
supporter_badge: true
- name: Pro
description: More storage with scan-on-push
stripe_price_monthly: price_xxx
stripe_price_yearly: price_xxx
max_webhooks: 10
webhook_all_triggers: true
ai_advisor: true
supporter_badge: true
```
### Quota tiers (hold config)
Storage quotas are configured separately, in the `quota:` section of each **hold's** config (`pkg/hold/quota/config.go`). These are also a position-ranked list. The appview pushes a tier *rank* to the hold via `updateCrewTier`; the hold maps that rank onto its own quota tier list, so the billing tier names and quota tier names do not need to match (only ranks line up). Real quota tier names are `deckhand`, `bosun`, `quartermaster` (there is no "swabbie" tier).
```yaml
quota:
tiers:
- name: free
quota: 5GB
scan_on_push: false
- name: deckhand
quota: 5GB
scan_on_push: false
- name: bosun
quota: 50GB
scan_on_push: true
- name: quartermaster
quota: 100GB
scan_on_push: true
defaults:
new_crew_tier: deckhand
``` ```
### Stripe Price IDs ### Stripe Price IDs
@@ -173,9 +212,9 @@ endpoint or auditing an existing one.
| `invoice.payment_failed` | Log only (Stripe Smart Retries handle retry + customer email) | | `invoice.payment_failed` | Log only (Stripe Smart Retries handle retry + customer email) |
| `charge.dispute.created` | Log only (Stripe emails the account owner by default) | | `charge.dispute.created` | Log only (Stripe emails the account owner by default) |
## Plankowners (Grandfathering) ## Plankowners (planned)
Early adopters can be marked as "plankowners" to get a paid tier for free: `io.atcr.hold.crew` records carry a `plankowner` boolean flag (`Plankowner` on `CrewRecord` in `pkg/atproto/lexicon.go`) intended to mark early adopters:
```json ```json
{ {
@@ -188,25 +227,23 @@ Early adopters can be marked as "plankowners" to get a paid tier for free:
} }
``` ```
Plankowners: The flag exists on the record, but automated grandfathering behavior is **not implemented**. There is no `plankowner_crew_tier` config field, and nothing currently grants a paid tier for free or treats plankowners differently from other crew members at billing time. Their assigned `tier` is whatever is set on the crew record. Treat this section as a placeholder for future grandfathering logic.
- Get `plankowner_crew_tier` (e.g., deckhand) without paying
- Still see upgrade options in UI if they want to support
- Can upgrade to higher tiers normally
## Customer-DID Mapping ## Customer-DID Mapping
DIDs are stored in Stripe customer metadata: The user's DID is stored in Stripe customer metadata (set by `getOrCreateCustomer` in `pkg/billing/billing.go`):
```json ```json
{ {
"metadata": { "metadata": {
"user_did": "did:plc:xxx", "user_did": "did:plc:xxx"
"hold_did": "did:web:hold.example.com"
} }
} }
``` ```
The hold uses an in-memory cache (10 min TTL) to reduce Stripe API calls. On webhook events, the cache is invalidated for the affected customer. Only `user_did` is stored. The appview resolves the customer for a DID by searching Stripe customer metadata, and reads `user_did` back from webhook events to know which user to update.
The appview uses an in-memory customer cache (10 min TTL) to reduce Stripe API calls. On webhook events, the cache is invalidated for the affected user.
## Production Checklist ## Production Checklist
@@ -216,7 +253,7 @@ The hold uses an in-memory cache (10 min TTL) to reduce Stripe API calls. On web
- URL: `https://your-appview.com/api/stripe/webhook` - URL: `https://your-appview.com/api/stripe/webhook`
- Events: `checkout.session.completed`, `customer.subscription.created`, `customer.subscription.updated`, `customer.subscription.paused`, `customer.subscription.resumed`, `customer.subscription.deleted`, `invoice.payment_failed` - Events: `checkout.session.completed`, `customer.subscription.created`, `customer.subscription.updated`, `customer.subscription.paused`, `customer.subscription.resumed`, `customer.subscription.deleted`, `invoice.payment_failed`
- [ ] Set `STRIPE_WEBHOOK_SECRET` from Dashboard webhook settings - [ ] Set `STRIPE_WEBHOOK_SECRET` from Dashboard webhook settings
- [ ] Update `quotas.yaml` with live price IDs - [ ] Update the appview config `billing.tiers` with live price IDs
- [ ] Build appview with `-tags billing` - [ ] Build appview with `-tags billing`
- [ ] Test with a real payment (can refund immediately) - [ ] Test with a real payment (can refund immediately)
@@ -232,11 +269,12 @@ The hold uses an in-memory cache (10 min TTL) to reduce Stripe API calls. On web
### Tier not updating after payment ### Tier not updating after payment
- Check appview logs for webhook processing errors - Check appview logs for webhook processing errors
- Verify price ID in `quotas.yaml` matches Stripe - Verify the price ID in the appview config `billing.tiers` matches Stripe
- Ensure `billing.enabled: true` in appview config - Confirm the appview was built with `-tags billing` (otherwise `/api/stripe/webhook` returns 404)
- Confirm appview was built with `-tags billing` (otherwise `/api/stripe/webhook` returns 404) - Confirm each managed hold has the appview DID configured (so it accepts the signed `updateCrewTier` call) and has matching quota tier ranks
### "Billing not enabled" error ### "Billing not enabled" error
- Build with `-tags billing` There is no `billing.enabled` flag. Billing auto-enables when all of the following hold (see `Manager.Enabled()` in `pkg/billing/billing.go`):
- Set `billing.enabled: true` in `quotas.yaml` - The appview was built with `-tags billing`
- Ensure `STRIPE_SECRET_KEY` is set - A Stripe secret key is set (via `STRIPE_SECRET_KEY` env var or `billing.stripe_secret_key`)
- At least one tier is configured under `billing.tiers`
docs/BILLING_REFACTOR.md
-348
View File
@@ -1,348 +0,0 @@
# Billing & Webhooks Refactor: Move to AppView
## Motivation
The current billing model is **per-hold**: each hold operator runs their own Stripe integration, manages their own tiers, and users pay each hold separately. This creates problems:
1. **Multi-hold confusion**: A user on 3 holds could have 3 separate Stripe subscriptions with no unified view
2. **Orphaned subscriptions**: Users can end up paying for holds they no longer use after switching their active hold
3. **Complex UI**: The settings page needs to surface billing per-hold, with separate "Manage Billing" links for each
4. **Captain-only billing**: Only hold captains can set up Stripe. Self-hosted hold operators who want to charge users would need their own Stripe account per hold
The proposed model is **per-appview**: a single Stripe integration on the appview, one subscription per user, covering all holds that appview manages.
## Current Architecture
```
User ──Settings UI──→ AppView ──XRPC──→ Hold ──Stripe API──→ Stripe
↑
Stripe Webhooks
```
### What lives where today
| Component | Location | Notes |
|-----------|----------|-------|
| Stripe customer management | Hold (`pkg/hold/billing/`) | Build tag: `-tags billing` |
| Stripe checkout/portal | Hold XRPC endpoints | Authenticated via service token |
| Stripe webhook receiver | Hold (`stripeWebhook` endpoint) | Updates crew tier on subscription change |
| Tier definitions + pricing | Hold config (`quotas.yaml`, `billing` section) | Captain configures |
| Quota enforcement | Hold (`pkg/hold/quota/`) | Checks tier limit on push |
| Storage quota calculation | Hold PDS layer records | Deduped per-user |
| Subscription UI | AppView handlers | Proxies all calls to hold |
| Webhook management (scan) | Hold PDS + SQLite | URL/secret in SQLite, metadata in PDS record |
| Webhook dispatch | Hold (`scan_broadcaster.go`) | Sends on scan completion |
| Sailor webhook record | User's PDS | Links to hold's private webhook record |
## Proposed Architecture
```
User ──Settings UI──→ AppView ──Stripe API──→ Stripe
│ ↑
│ Stripe Webhooks
│
├──XRPC──→ Hold A (quota enforcement, scan results)
├──XRPC──→ Hold B
└──XRPC──→ Hold C
AppView signs attestation
│
└──→ Hold stores in PDS (trust anchor)
```
### What moves to AppView
| Component | From | To | Notes |
|-----------|------|----|-------|
| Stripe customer management | Hold | AppView | One customer per user, not per hold |
| Stripe checkout/portal | Hold | AppView | Single subscription covers all holds |
| Stripe webhook receiver | Hold | AppView | AppView updates tier across all holds |
| Tier definitions + pricing | Hold config | AppView config | AppView defines billing tiers |
| Scan webhooks (storage + dispatch) | Hold | AppView | AppView has user context, scan data comes via Jetstream/XRPC |
### What stays on the hold
| Component | Notes |
|-----------|-------|
| Quota enforcement | Hold still checks tier limit on push |
| Storage quota calculation | Layer records stay in hold PDS |
| Tier definitions (quota only) | Hold defines storage limits per tier, no pricing |
| Scan execution + results | Scanner still talks to hold, results stored in hold PDS |
| Crew tier field | Source of truth for enforcement, updated by appview |
## Billing Model
### One subscription, all holds
A user pays the appview once. Their subscription tier applies across every hold the appview manages.
```
AppView billing tiers: [Free] [Tier 1] [Tier 2]
│ │ │
▼ ▼ ▼
Hold A tiers (3GB/10GB/50GB): deckhand bosun quartermaster
Hold B tiers (5GB/20GB/∞): deckhand bosun quartermaster
```
### Tier pairing
The appview defines N billing slots. Each hold defines its own tier list with storage quotas. The appview maps its billing slots to each hold's lowest N tiers by rank order.
- AppView doesn't need to know tier names — just "slot 1, slot 2, slot 3"
- Each hold independently decides what storage limit each tier gets
- The settings UI shows the range: "5-10 GB depending on region" or "minimum 5 GB"
### Hold captains who want to charge
If a hold captain wants to charge their own users (not through the shared appview), they spin up their own appview instance with their own Stripe account. The billing code stays the same — it just runs on their appview instead of the shared one.
## AppView-Hold Trust Model
### Problem
The appview needs to tell holds "user X is tier Y." The hold needs to trust that instruction. If domains change, the hold needs to verify the appview's identity.
### Attestation handshake
1. **Hold config** already has `server.appview_url` (preferred appview)
2. **AppView config** gains a `managed_holds` list (DIDs of holds it manages)
3. On first connection, the appview signs an attestation with its private key:
```json
{
"$type": "io.atcr.appview.attestation",
"appviewDid": "did:web:atcr.io",
"holdDid": "did:web:hold01.atcr.io",
"issuedAt": "2026-02-23T...",
"signature": "<signed with appview's P-256 key>"
}
```
4. The hold stores this attestation in its embedded PDS
5. On subsequent requests, the hold can challenge the appview: present the attestation, appview proves it holds the matching private key
6. If the appview's domain changes, the attestation (tied to DID, not URL) remains valid
### Trust verification flow
```
AppView boots → checks managed_holds list
→ for each hold:
→ calls hold's describeServer endpoint to verify DID
→ signs attestation { appviewDid, holdDid, issuedAt }
→ sends to hold via XRPC
→ hold stores in PDS as io.atcr.hold.appview record
Hold receives tier update from appview:
→ checks: does this request come from my preferred appview?
→ verifies: signature on stored attestation matches appview's current key
→ if valid: updates crew tier
→ if invalid: rejects, logs warning
```
### Key material
- **AppView**: P-256 key (already exists at `/var/lib/atcr/oauth/client.key`, used for OAuth)
- **Hold**: K-256 key (PDS signing key)
- Attestation is signed by appview's P-256 key, verifiable by anyone with the appview's public key (available via DID document)
## Webhooks: Move to AppView
### Why move
Scan webhooks currently live on the hold, but:
- The webhook payload needs user handles, repository names, tags — all resolved by the appview
- The hold only has DIDs and digests
- The appview already processes scan records via Jetstream (backfill + live)
- Webhook secrets shouldn't need to live on every hold the user pushes to
### New flow
```
Scanner completes scan
→ Hold stores scan record in PDS
→ Jetstream delivers scan record to AppView
→ AppView resolves user handle, repo name, tags
→ AppView dispatches webhooks with full context
```
### What changes
| Aspect | Current (hold) | Proposed (appview) |
|--------|---------------|-------------------|
| Webhook storage | Hold SQLite + PDS record | AppView DB + user's PDS record |
| Webhook secrets | Hold SQLite (`webhook_secrets` table) | AppView DB |
| Dispatch trigger | `scan_broadcaster.go` on scan completion | Jetstream processor on `io.atcr.hold.scan` record |
| Payload enrichment | Hold fetches handle from appview metadata | AppView has full context natively |
| Discord/Slack formatting | Hold (`webhooks.go`) | AppView (same code, moved) |
| Tier-based limits | Hold quota manager | AppView billing tier |
| XRPC endpoints | Hold (`listWebhooks`, `addWebhook`, etc.) | AppView API endpoints (already exist as proxies) |
### Webhook record changes
The `io.atcr.sailor.webhook` record in the user's PDS stays. It already stores `holdDid` and `triggers`. The `privateCid` field (linking to hold's internal record) becomes unnecessary since appview owns the full webhook now.
The `io.atcr.hold.webhook` record in the hold's PDS is no longer needed. Webhooks are appview-scoped, not hold-scoped.
### Migration path
1. AppView gains webhook storage in its own DB (new table)
2. AppView gains webhook dispatch in its Jetstream processor
3. Hold's webhook endpoints deprecated (return 410 Gone after transition period)
4. Existing hold webhook records migrated via one-time script reading from hold XRPC + user PDS
## Config Changes
### AppView config additions
```yaml
server:
# Existing
default_hold_did: "did:web:hold01.atcr.io"
# New
managed_holds:
- "did:web:hold01.atcr.io"
- "did:plc:abc123..."
# New section
billing:
enabled: true
currency: usd
success_url: "{base_url}/settings/billing"
cancel_url: "{base_url}/settings/billing"
tiers:
- name: "Free"
# No stripe_price = free tier
- name: "Standard"
stripe_price_monthly: price_xxx
stripe_price_yearly: price_yyy
- name: "Pro"
stripe_price_monthly: price_xxx
stripe_price_yearly: price_yyy
```
### AppView environment additions
```bash
STRIPE_SECRET_KEY=sk_live_xxx
STRIPE_WEBHOOK_SECRET=whsec_xxx
```
### Hold config changes
```yaml
# Removed
billing:
# entire section removed from hold config
# Stays (quota enforcement only)
quota:
tiers:
- name: deckhand
quota: 5GB
- name: bosun
quota: 50GB
- name: quartermaster
quota: 100GB
defaults:
new_crew_tier: deckhand
```
The hold no longer has Stripe config. It just defines storage limits per tier and enforces them.
## AppView DB Schema Additions
```sql
-- Webhook configurations (moved from hold SQLite)
CREATE TABLE webhooks (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_did TEXT NOT NULL,
url TEXT NOT NULL,
secret_hash TEXT, -- bcrypt hash of HMAC secret
triggers INTEGER NOT NULL DEFAULT 1, -- bitmask: first=1, all=2, changed=4
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
UNIQUE(user_did, url)
);
-- Billing: track which holds have been attested
CREATE TABLE hold_attestations (
hold_did TEXT PRIMARY KEY,
attestation_cid TEXT NOT NULL, -- CID of attestation record in hold's PDS
issued_at DATETIME NOT NULL,
verified_at DATETIME
);
```
Stripe customer/subscription data continues to live in Stripe (queried via API, cached in memory). No local subscription table needed — same pattern as current hold billing, just on appview.
## Implementation Phases
### Phase 1: Trust foundation
- Add `managed_holds` to appview config
- Implement attestation signing (appview) and storage (hold)
- Add attestation verification to hold's tier-update endpoint
- New XRPC endpoint on hold: `io.atcr.hold.updateCrewTier` (appview-authenticated)
### Phase 2: Billing migration
- Move Stripe integration from hold to appview (reuse `pkg/hold/billing/` code)
- AppView billing uses `-tags billing` build tag (same pattern)
- Implement tier pairing: appview billing slots mapped to hold tier lists
- New appview endpoints: checkout, portal, stripe webhook receiver
- Settings UI: single subscription section (not per-hold)
### Phase 3: Webhook migration ✅
- Add webhook + scans tables to appview DB
- Implement webhook dispatch in appview's Jetstream processor
- Move Discord/Slack formatting code to `pkg/appview/webhooks/`
- Deprecate hold webhook XRPC endpoints (X-Deprecated header)
- Webhooks now user-scoped (global across all holds) in appview DB
- Scan records cached from Jetstream for change detection
### Phase 4: Cleanup ✅
- Removed hold webhook XRPC endpoints, dispatch code, and `webhooks.go`
- Removed `io.atcr.hold.webhook` and `io.atcr.sailor.webhook` record types + lexicons
- Removed `webhook_secrets` SQLite schema from scan_broadcaster
- Removed `MaxWebhooks`/`WebhookAllTriggers` from hold quota config
- Removed sailor webhook from OAuth scopes
## Settings UI Impact
The storage tab simplifies significantly:
```
┌──────────────────────────────────────────────────────┐
│ Active Hold: [▼ hold01.atcr.io (Crew) ] │
└──────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────┐
│ Subscription: Standard ($5/mo) [Manage Billing] │
│ Storage: 3-5 GB depending on region │
└──────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────┐
│ ★ hold01.atcr.io [Active] [Crew] [Online] │
│ Tier: bosun · 281.5 MB / 5.0 GB (5%) │
│ ▸ Webhooks (2 configured) │
└──────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────┐
│ Other Holds Role Status Storage │
│ hold02.atcr.io Crew ● 230 MB / 3 GB │
│ hold03.atcr.io Owner ● No data │
└──────────────────────────────────────────────────────┘
```
Key changes:
- **One subscription section** at the top (not per-hold)
- **Webhooks section** under active hold card (managed by appview now)
- **No "Paid" badge per hold** — subscription is global
- **Storage range** shown on subscription card ("3-5 GB depending on region")
- **Per-hold quota** still shown (each hold enforces its own limit for the user's tier)
## Open Questions
1. **Tier list endpoint**: Holds need a new XRPC endpoint that returns their tier list with quotas (without pricing). The appview calls this to build the "3-5 GB depending on region" display. Something like `io.atcr.hold.listTiers`.
2. **Existing Stripe customers**: Holds with existing Stripe subscriptions need a migration plan. Options: honor existing subscriptions until they expire, or bulk-migrate customers to appview's Stripe account.
3. **Webhook delivery guarantees**: Moving dispatch to appview adds latency (scan record → Jetstream → appview → webhook). For time-sensitive notifications, consider the hold sending a lightweight "scan completed" signal directly to appview via XRPC rather than waiting for Jetstream propagation.
4. **Self-hosted appviews**: The attestation model assumes one appview per set of holds. If multiple appviews try to manage the same hold, the hold should only trust the most recent attestation (or maintain a list).
docs/BYOS.md
+71 -46
View File
@@ -18,10 +18,13 @@ ATCR supports "Bring Your Own Storage" (BYOS) for blob storage. Users can:
│ - Profile management │ │ - Profile management │
└────────────┬─────────────────────────────┘ └────────────┬─────────────────────────────┘
│ │
│ Hold discovery priority: │ Hold discovery (findHoldDIDAndProfile):
│ 1. io.atcr.sailor.profile.defaultHold (DID) │ 1. io.atcr.sailor.profile.defaultHold (DID)
│ 2. io.atcr.hold records (legacy) │ 2. AppView default hold (server.managed_holds[0])
│ 3. AppView default_hold_did │
│ Then resolveSuccessor: if the chosen hold's
│ captain record sets a successor DID, apply a
│ single-hop redirect to it (hold migration).
▼ ▼
┌──────────────────────────────────────────┐ ┌──────────────────────────────────────────┐
│ User's PDS │ │ User's PDS │
@@ -57,23 +60,31 @@ Each hold is a full ATProto actor with:
"$type": "io.atcr.hold.captain", "$type": "io.atcr.hold.captain",
"owner": "did:plc:alice123", "owner": "did:plc:alice123",
"public": false, "public": false,
"allowAllCrew": false,
"enableBlueskyPosts": false,
"deployedAt": "2025-10-14T...", "deployedAt": "2025-10-14T...",
"region": "iad", "region": "iad",
"provider": "fly.io" "successor": ""
} }
``` ```
`region` and `successor` are optional. `successor` holds the DID of a replacement hold; when set, the AppView applies a single-hop redirect to it during hold discovery (see the Architecture diagram above).
**Crew records** (`io.atcr.hold.crew/{rkey}`): **Crew records** (`io.atcr.hold.crew/{rkey}`):
```json ```json
{ {
"$type": "io.atcr.hold.crew", "$type": "io.atcr.hold.crew",
"member": "did:plc:bob456", "member": "did:plc:bob456",
"role": "admin", "role": "captain",
"permissions": ["blob:read", "blob:write"], "permissions": ["blob:read", "blob:write"],
"tier": "bosun",
"plankowner": false,
"addedAt": "2025-10-14T..." "addedAt": "2025-10-14T..."
} }
``` ```
Authorization is driven by the `permissions` array (`blob:read`, `blob:write`, `crew:admin`), not the `role` string. `blob:write` implicitly grants `blob:read` (you can't push without being able to pull). `tier` and `plankowner` are optional and feed quota limits.
### Sailor Profile (User's PDS) ### Sailor Profile (User's PDS)
Users set their preferred hold in their sailor profile: Users set their preferred hold in their sailor profile:
@@ -91,28 +102,43 @@ Users set their preferred hold in their sailor profile:
### Configuration ### Configuration
Hold service is configured entirely via environment variables: The hold service is configured with Viper: a YAML file is the primary source, and
environment variables override individual fields. Env var names are `HOLD_` plus the
YAML path with `_` separators (e.g. `server.public_url` → `HOLD_SERVER_PUBLIC_URL`).
S3 credentials use the standard AWS names.
Generate a fully commented config and run with it:
```bash ```bash
# Hold identity (REQUIRED) ./bin/atcr-hold config init config-hold.yaml
HOLD_PUBLIC_URL=https://hold.example.com # edit config-hold.yaml, then:
HOLD_OWNER=did:plc:your-did-here ./bin/atcr-hold serve --config config-hold.yaml
# S3 storage backend (REQUIRED)
AWS_ACCESS_KEY_ID=your_access_key
AWS_SECRET_ACCESS_KEY=your_secret_key
AWS_REGION=us-east-1
S3_BUCKET=my-blobs
# Access control
HOLD_PUBLIC=false # Require authentication for reads
HOLD_ALLOW_ALL_CREW=false # Only explicit crew members can write
# Embedded PDS
HOLD_DATABASE_PATH=/var/lib/atcr-hold/hold.db
HOLD_DATABASE_KEY_PATH=/var/lib/atcr-hold/keys
``` ```
Key fields (YAML on the left, env override on the right):
```yaml
server:
public_url: https://hold.example.com # HOLD_SERVER_PUBLIC_URL (REQUIRED)
public: false # HOLD_SERVER_PUBLIC (allow anonymous reads)
registration:
owner_did: did:plc:your-did-here # HOLD_REGISTRATION_OWNER_DID
allow_all_crew: false # HOLD_REGISTRATION_ALLOW_ALL_CREW
database:
path: /var/lib/atcr-hold # HOLD_DATABASE_PATH (carstore + SQLite)
key_path: "" # HOLD_DATABASE_KEY_PATH (defaults to {path}/signing.key)
storage:
bucket: my-blobs # S3_BUCKET (REQUIRED)
region: us-east-1 # AWS_REGION
endpoint: "" # S3_ENDPOINT (for non-AWS providers)
```
S3 credentials are read from the standard AWS env vars (`AWS_ACCESS_KEY_ID`,
`AWS_SECRET_ACCESS_KEY`).
### Running Locally ### Running Locally
For local development, use Minio as an S3-compatible storage: For local development, use Minio as an S3-compatible storage:
@@ -124,16 +150,16 @@ docker run -p 9000:9000 -p 9001:9001 minio/minio server /data --console-address
# Build # Build
go build -o bin/atcr-hold ./cmd/hold go build -o bin/atcr-hold ./cmd/hold
# Run (with env vars or .env file) # Run (env overrides shown; a YAML config works too)
export HOLD_PUBLIC_URL=http://localhost:8080 export HOLD_SERVER_PUBLIC_URL=http://localhost:8080
export HOLD_OWNER=did:plc:your-did-here export HOLD_REGISTRATION_OWNER_DID=did:plc:your-did-here
export AWS_ACCESS_KEY_ID=minioadmin export AWS_ACCESS_KEY_ID=minioadmin
export AWS_SECRET_ACCESS_KEY=minioadmin export AWS_SECRET_ACCESS_KEY=minioadmin
export S3_BUCKET=test export S3_BUCKET=test
export S3_ENDPOINT=http://localhost:9000 export S3_ENDPOINT=http://localhost:9000
export HOLD_DATABASE_PATH=/tmp/atcr-hold/hold.db export HOLD_DATABASE_PATH=/tmp/atcr-hold
./bin/atcr-hold ./bin/atcr-hold serve
``` ```
On first run, the hold service creates: On first run, the hold service creates:
@@ -150,11 +176,11 @@ app = "my-atcr-hold"
primary_region = "ord" primary_region = "ord"
[env] [env]
HOLD_PUBLIC_URL = "https://my-atcr-hold.fly.dev" HOLD_SERVER_PUBLIC_URL = "https://my-atcr-hold.fly.dev"
AWS_REGION = "us-east-1" AWS_REGION = "us-east-1"
S3_BUCKET = "my-blobs" S3_BUCKET = "my-blobs"
HOLD_PUBLIC = "false" HOLD_SERVER_PUBLIC = "false"
HOLD_ALLOW_ALL_CREW = "false" HOLD_REGISTRATION_ALLOW_ALL_CREW = "false"
[http_service] [http_service]
internal_port = 8080 internal_port = 8080
@@ -176,7 +202,7 @@ fly deploy
# Set secrets # Set secrets
fly secrets set AWS_ACCESS_KEY_ID=... fly secrets set AWS_ACCESS_KEY_ID=...
fly secrets set AWS_SECRET_ACCESS_KEY=... fly secrets set AWS_SECRET_ACCESS_KEY=...
fly secrets set HOLD_OWNER=did:plc:your-did-here fly secrets set HOLD_REGISTRATION_OWNER_DID=did:plc:your-did-here
``` ```
## Request Flow ## Request Flow
@@ -227,22 +253,21 @@ fly secrets set HOLD_OWNER=did:plc:your-did-here
3. Manifest contains: 3. Manifest contains:
- holdDid: "did:web:alice-storage.fly.dev" - holdDid: "did:web:alice-storage.fly.dev"
4. AppView caches hold DID for 10 minutes (covers pull operation) 4. Client requests blob: GET /v2/alice/myapp/blobs/sha256:abc123
5. Client requests blob: GET /v2/alice/myapp/blobs/sha256:abc123 5. AppView reads the hold DID from the manifest's holdDid field (per request)
6. AppView uses cached hold DID from manifest 6. AppView gets service token from alice's PDS
(validated service tokens are cached ~45s to absorb a burst of blob requests)
7. AppView gets service token from alice's PDS 7. AppView calls hold XRPC:
8. AppView calls hold XRPC:
GET /xrpc/com.atproto.sync.getBlob?did={userDID}&cid=sha256:abc123 GET /xrpc/com.atproto.sync.getBlob?did={userDID}&cid=sha256:abc123
Authorization: Bearer {serviceToken} Authorization: Bearer {serviceToken}
Response: { "url": "https://s3.../presigned-download" } Response: { "url": "https://s3.../presigned-download" }
9. AppView redirects client to presigned S3 URL 8. AppView redirects client to presigned S3 URL
10. Client downloads directly from S3 9. Client downloads directly from S3
``` ```
**Key insight:** Pull uses the `holdDid` stored in the manifest, ensuring blobs are fetched from where they were originally pushed. **Key insight:** Pull uses the `holdDid` stored in the manifest, ensuring blobs are fetched from where they were originally pushed.
@@ -251,8 +276,8 @@ fly secrets set HOLD_OWNER=did:plc:your-did-here
### Read Access ### Read Access
- **Public hold** (`HOLD_PUBLIC=true`): Anonymous + authenticated users - **Public hold** (`server.public: true`): Anonymous + authenticated users
- **Private hold** (`HOLD_PUBLIC=false`): Authenticated users with crew membership - **Private hold** (`server.public: false`): Authenticated users with crew membership
### Write Access ### Write Access
@@ -290,7 +315,7 @@ atproto put-record \
--value '{ --value '{
"$type": "io.atcr.hold.crew", "$type": "io.atcr.hold.crew",
"member": "did:plc:bob456", "member": "did:plc:bob456",
"role": "admin", "role": "crew",
"permissions": ["blob:read", "blob:write"] "permissions": ["blob:read", "blob:write"]
}' }'
``` ```
@@ -318,9 +343,9 @@ Hold service requires S3-compatible storage. Supported providers:
```bash ```bash
# 1. Deploy hold service # 1. Deploy hold service
export HOLD_PUBLIC_URL=https://team-hold.fly.dev export HOLD_SERVER_PUBLIC_URL=https://team-hold.fly.dev
export HOLD_OWNER=did:plc:admin export HOLD_REGISTRATION_OWNER_DID=did:plc:admin
export HOLD_PUBLIC=false # Private export HOLD_SERVER_PUBLIC=false # Private
export AWS_ACCESS_KEY_ID=... export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=... export AWS_SECRET_ACCESS_KEY=...
export S3_BUCKET=team-blobs export S3_BUCKET=team-blobs
docs/CREDENTIAL_HELPER.md
+218 -105
View File
@@ -2,11 +2,15 @@
## Overview ## Overview
The ATCR credential helper is distributed as pre-built binaries for all major platforms using GoReleaser and GitHub Actions. This document outlines the complete distribution pipeline. The ATCR credential helper is distributed as pre-built binaries for Linux,
macOS, and Windows. Builds are produced with GoReleaser and the resulting
artifacts are published to the project's Tangled repository as ATProto records
(`sh.tangled.repo.artifact`) on the repo owner's PDS. There is no GitHub
release pipeline. This document describes the actual distribution flow.
## Why Go is Ideal for Credential Helpers ## Why Go is Ideal for Credential Helpers
Go is actually the **perfect choice** for Docker credential helpers: Go is a natural fit for Docker credential helpers:
1. **Cross-compilation** - Single command builds for all platforms 1. **Cross-compilation** - Single command builds for all platforms
2. **Static binaries** - No runtime dependencies (unlike Node.js, Python, Ruby) 2. **Static binaries** - No runtime dependencies (unlike Node.js, Python, Ruby)
@@ -16,22 +20,50 @@ Go is actually the **perfect choice** for Docker credential helpers:
- docker-credential-ecr-login (AWS) - docker-credential-ecr-login (AWS)
- docker-credential-pass (community) - docker-credential-pass (community)
## Multi-Brand Structure
The credential helper is built as multiple brand-specific binaries from a
single shared implementation:
- **`pkg/credhelper/`** holds the entire implementation (Docker protocol
commands, device-flow auth, config storage, self-update). It exposes
`credhelper.Run(credhelper.Config{...})`.
- **`cmd/credential-helper/atcr/`** and **`cmd/credential-helper/seamark/`**
are thin `main` packages (each with its own `go.mod`) that supply a per-brand
`Config` and call `Run`. They differ only in brand identity:
| Field | atcr binary | seamark binary |
|---|---|---|
| Binary name | `docker-credential-atcr` | `docker-credential-seamark` |
| Default registry | `atcr.io` | `seamark.cr` |
| Config dir (under `$HOME`) | `~/.atcr` | `~/.seamark` |
| Secret prefix | `atcr_device_` | `seamark_device_` |
Both brands point `ReleasesBaseURL` at the same Tangled repo
(`https://tangled.org/did:plc:e3kzdezk5gsirzh7eoqplc64`) for self-update and
download.
Each brand module is independently installable via `go install` (see
[From Source](#5-from-source)). The atcr module path is
`atcr.io/cmd/credential-helper/atcr`; the seamark module path is
`seamark.dev/cmd/credential-helper/seamark`.
## Supported Platforms ## Supported Platforms
| Platform | Arch | Format | Status | | Platform | Arch | Format | Status |
|----------|------|--------|--------| |----------|------|--------|--------|
| Linux | amd64 | tar.gz | ✅ | | Linux | amd64 | tar.gz | Available |
| Linux | arm64 | tar.gz | ✅ | | Linux | arm64 | tar.gz | Available |
| macOS | amd64 (Intel) | tar.gz | ✅ | | macOS | amd64 (Intel) | tar.gz | Available |
| macOS | arm64 (Apple Silicon) | tar.gz | ✅ | | macOS | arm64 (Apple Silicon) | tar.gz | Available |
| Windows | amd64 | zip | ✅ | | Windows | amd64 | tar.gz | Available |
| Windows | arm64 | zip | ✅ | | Windows | arm64 | tar.gz | Available |
## Distribution Methods ## Distribution Methods
### 1. GitHub Releases (Automated) ### 1. Tangled Releases (Automated)
**Trigger:** Push a git tag (e.g., `v1.0.0`) **Trigger:** Push a version tag (e.g. `v1.0.0`).
```bash ```bash
git tag v1.0.0 git tag v1.0.0
@@ -39,31 +71,48 @@ git push origin v1.0.0
``` ```
**What happens:** **What happens:**
1. GitHub Actions runs (`.github/workflows/release.yml`) 1. The Tangled CI workflow `.tangled/workflows/release-credential-helper.yml`
2. GoReleaser builds binaries for all platforms runs on tags matching `v*`.
3. Creates GitHub release with: 2. It installs `goat` (ATProto CLI) and `goreleaser`, then logs into the repo
- Pre-built binaries (tar.gz/zip) owner's PDS once with `goat account login`.
- Checksums file 3. `goreleaser release --clean` builds binaries for all platforms.
- Changelog 4. GoReleaser's `release` block is disabled (`release.disable: true`), so no
4. Updates Homebrew tap (if configured) GitHub/forge release is created. Instead, a custom `publishers` block runs
`./scripts/publish-artifact.sh` for each built archive and the checksums
file.
5. `publish-artifact.sh` uploads each artifact as a PDS blob (`goat blob
upload`) and creates an `sh.tangled.repo.artifact` record referencing it.
The record uses a deterministic rkey derived from `(tag, artifact name)` so
retries are idempotent (Tangled's PDS returns HTTP 500, not 409, on
duplicate rkey).
**Workflow file:** `.github/workflows/release.yml` **Workflow file:** `.tangled/workflows/release-credential-helper.yml`
**Config file:** `.goreleaser.yaml` **Config file:** `.goreleaser.yaml`
**Publisher script:** `scripts/publish-artifact.sh`
The artifacts become downloadable from the Tangled repo's tag download path
(see [Manual Download](#4-manual-download)).
### 2. Install Scripts ### 2. Install Scripts
Both scripts are served by the AppView from its static directory
(`pkg/appview/public/static/`) at `/static/install.sh` and
`/static/install.ps1`. They resolve the latest version by following the
`{repo}/tags/latest` redirect chain on Tangled, then download the matching
archive from the tag download path.
**Linux/macOS:** `install.sh` **Linux/macOS:** `install.sh`
- Detects OS and architecture - Detects OS and architecture
- Downloads latest release from GitHub - Resolves the latest tag from Tangled and downloads the archive
- Installs to `/usr/local/bin` (or custom dir) - Installs to `/usr/local/bin` (override with `INSTALL_DIR`)
- Makes executable and verifies installation - Makes executable and verifies installation
**Windows:** `install.ps1` **Windows:** `install.ps1`
- Detects architecture - Detects architecture
- Downloads latest release - Resolves the latest tag from Tangled and downloads the archive
- Installs to `C:\Program Files\ATCR` (or custom dir) - Installs to `%ProgramFiles%\ATCR` (override with `ATCR_INSTALL_DIR`)
- Adds to system PATH - Adds to system PATH (requires Administrator to modify the machine PATH)
- Requires Administrator privileges - Uses the bundled `tar.exe` to extract the `.tar.gz`
**Usage:** **Usage:**
```bash ```bash
@@ -71,55 +120,71 @@ git push origin v1.0.0
curl -fsSL https://atcr.io/static/install.sh | bash curl -fsSL https://atcr.io/static/install.sh | bash
# Windows (PowerShell) # Windows (PowerShell)
iwr -useb https://atcr.io/install.ps1 | iex iwr -useb https://atcr.io/static/install.ps1 | iex
``` ```
### 3. Homebrew (macOS) Pin a specific version by setting `ATCR_VERSION` (e.g. `ATCR_VERSION=v1.0.0`)
before running either script.
**Setup required:** ### 3. Homebrew (macOS) — Not Currently Available
1. Create `atcr-io/homebrew-tap` repository
2. Set `HOMEBREW_TAP_TOKEN` secret in GitHub Actions
3. GoReleaser automatically updates formula on release
**Usage:** Homebrew distribution is **not available yet**. A `brews:` block exists in
```bash `.goreleaser.yaml` but is commented out. If/when it is enabled, the formula
brew tap atcr-io/tap would live in the project's Tangled repo under `Formula/` and pull artifacts
brew install docker-credential-atcr from the Tangled tag download path. There is no published tap to `brew tap`
``` today; use one of the other methods.
**Benefits:**
- Automatic updates via `brew upgrade`
- Handles PATH configuration
- Native macOS experience
### 4. Manual Download ### 4. Manual Download
Users can download directly from GitHub Releases: Download the archive directly from the Tangled repo's tag download path. The
artifacts are published as `sh.tangled.repo.artifact` records on the repo
owner's PDS and served by Tangled at:
```
https://tangled.org/<did-or-handle>/<repo>/tags/<version>/download/<artifact>
```
```bash ```bash
# Example: Linux amd64 # Example: Linux amd64
REPO=https://tangled.org/did:plc:e3kzdezk5gsirzh7eoqplc64
VERSION=v1.0.0 VERSION=v1.0.0
curl -LO https://github.com/atcr-io/atcr/releases/download/${VERSION}/docker-credential-atcr_${VERSION#v}_Linux_x86_64.tar.gz curl -LO "${REPO}/tags/${VERSION}/download/docker-credential-atcr_${VERSION#v}_Linux_x86_64.tar.gz"
tar -xzf docker-credential-atcr_${VERSION#v}_Linux_x86_64.tar.gz tar -xzf docker-credential-atcr_${VERSION#v}_Linux_x86_64.tar.gz
sudo install -m 755 docker-credential-atcr /usr/local/bin/ sudo install -m 755 docker-credential-atcr /usr/local/bin/
``` ```
Tangled redirects DID to handle for the repo URL, so `curl -L` is needed to
follow the redirect.
### 5. From Source ### 5. From Source
For users with Go installed: For users with Go installed. Each brand is a separate installable module:
```bash ```bash
go install atcr.io/cmd/credential-helper@latest # atcr.io brand
sudo mv $(go env GOPATH)/bin/credential-helper /usr/local/bin/docker-credential-atcr go install atcr.io/cmd/credential-helper/atcr@latest
sudo mv "$(go env GOPATH)/bin/atcr" /usr/local/bin/docker-credential-atcr
# seamark.dev brand
go install seamark.dev/cmd/credential-helper/seamark@latest
sudo mv "$(go env GOPATH)/bin/seamark" /usr/local/bin/docker-credential-seamark
``` ```
**Note:** This requires Go 1.26+ and compiles locally. The installed binary takes the name of the leaf package directory (`atcr` /
`seamark`); rename it to `docker-credential-<brand>` so Docker can discover it.
**Note:** This requires Go 1.26+ and compiles locally. Locally (inside the repo
workspace) `go.work` resolves the `atcr.io` dependency; standalone installs
resolve the `require atcr.io vX.Y.Z` line pinned in each brand's `go.mod`.
## Release Process ## Release Process
### Creating a New Release ### Creating a New Release
1. **Update version** (if using version consts anywhere) 1. **Bump the pinned `atcr.io` version** in each brand's `go.mod`
(`cmd/credential-helper/atcr/go.mod`,
`cmd/credential-helper/seamark/go.mod`) if `go install` consumers need the
new code.
2. **Commit and tag:** 2. **Commit and tag:**
```bash ```bash
@@ -131,12 +196,11 @@ sudo mv $(go env GOPATH)/bin/credential-helper /usr/local/bin/docker-credential-
``` ```
3. **Wait for CI:** 3. **Wait for CI:**
- GitHub Actions builds and releases automatically - The Tangled workflow builds and publishes artifacts automatically.
- Check: https://github.com/atcr-io/atcr/actions
4. **Verify release:** 4. **Verify release:**
- Visit: https://github.com/atcr-io/atcr/releases - Confirm the `tags/latest` redirect resolves to the new tag and the archive
- Test install script: downloads:
```bash ```bash
ATCR_VERSION=v1.0.0 curl -fsSL https://atcr.io/static/install.sh | bash ATCR_VERSION=v1.0.0 curl -fsSL https://atcr.io/static/install.sh | bash
docker-credential-atcr version docker-credential-atcr version
@@ -144,7 +208,8 @@ sudo mv $(go env GOPATH)/bin/credential-helper /usr/local/bin/docker-credential-
### Version Information ### Version Information
GoReleaser injects version info at build time: GoReleaser injects version info at build time via ldflags into the brand
`main` packages:
```go ```go
var ( var (
@@ -167,20 +232,28 @@ docker-credential-atcr v1.0.0 (commit: abc123, built: 2025-01-15T10:30:00Z)
Key sections: Key sections:
**Builds:** **Builds:**
- Binary name: `docker-credential-atcr` (Windows: `.exe` auto-added) - One `credential-helper` build with `dir: ./cmd/credential-helper/atcr`,
binary name `docker-credential-atcr` (Windows: `.exe` auto-added)
- Targets: Linux, macOS, Windows (amd64, arm64) - Targets: Linux, macOS, Windows (amd64, arm64)
- CGO disabled for static binaries - CGO disabled for static binaries
- Ldflags inject version info - Ldflags inject version/commit/date
**Archives:** **Archives:**
- Format: tar.gz (Linux/macOS), zip (Windows) - Format: tar.gz for all platforms
- Naming: `docker-credential-atcr_VERSION_OS_ARCH.tar.gz` - Naming: `docker-credential-atcr_VERSION_OS_ARCH.tar.gz`
- Includes: LICENSE, README, INSTALLATION.md - Includes: LICENSE, README, INSTALLATION
**Homebrew:** **Release:**
- Auto-updates tap repository - `release.disable: true` — no GitHub/forge release is created.
- Formula includes version check
- Installs to Homebrew prefix **Publishers:**
- A custom `atproto-pds` publisher runs `./scripts/publish-artifact.sh` for
each artifact, forwarding the Tangled-provided `TANGLED_REF_NAME`,
`TANGLED_REPO_DID`, and `REPO_URL` env vars (GoReleaser publishers run in a
sanitized sub-shell, so these must be forwarded explicitly).
**Brews:**
- Present but commented out (Homebrew not currently enabled).
**Changelog:** **Changelog:**
- Auto-generated from commits - Auto-generated from commits
@@ -202,8 +275,14 @@ Docker looks for binaries named `docker-credential-*` in PATH:
} }
``` ```
3. Docker looks for `docker-credential-atcr` in PATH 3. Docker looks for `docker-credential-atcr` in PATH
4. Calls: `docker-credential-atcr get` (with `atcr.io` on stdin) 4. Calls `docker-credential-atcr get` with `atcr.io` on stdin (a plain string,
5. Helper returns credentials (JSON on stdout) not JSON)
5. Helper returns credentials as JSON on stdout
The `credHelpers` map value is the binary-name suffix after
`docker-credential-` (so `atcr` for `docker-credential-atcr`). The
`configure-docker` command (and the prompt at the end of `login`) writes this
entry automatically.
### PATH Requirements ### PATH Requirements
@@ -212,31 +291,84 @@ Docker looks for binaries named `docker-credential-*` in PATH:
- Check with: `which docker-credential-atcr` - Check with: `which docker-credential-atcr`
**Windows:** **Windows:**
- Common locations: `C:\Windows\System32`, `C:\Program Files\ATCR` - Common locations: `C:\Windows\System32`, `%ProgramFiles%\ATCR`
- Check with: `where docker-credential-atcr` - Check with: `where docker-credential-atcr`
## CI/CD Secrets ## CI/CD Secrets
### Required GitHub Secrets ### Required Tangled Secret
1. **`GITHUB_TOKEN`** (automatic) - **`PUBLISH_APP_PASSWORD`** — an ATProto app password for the account that
- Provided by GitHub Actions owns the repo's artifact records. The workflow runs `goat account login -u
- Used to create releases "$TANGLED_REPO_DID" -p "$PUBLISH_APP_PASSWORD"` once before GoReleaser, and
every `publish-artifact.sh` invocation reuses that session.
2. **`HOMEBREW_TAP_TOKEN`** (manual setup) There is no `GITHUB_TOKEN` or `HOMEBREW_TAP_TOKEN` — the project does not
- Personal access token with `repo` scope release through GitHub.
- Used to update Homebrew tap
- Can skip if not using Homebrew
### Setup Instructions ## Helper Behavior
```bash The helper implements the standard Docker credential helper protocol plus a
# Create PAT with repo scope at: few user-facing commands. Implementation lives in `pkg/credhelper/`.
# https://github.com/settings/tokens
# Add to repository secrets: ### Docker Protocol Commands (hidden)
# https://github.com/atcr-io/atcr/settings/secrets/actions
``` Called by Docker, not users (`pkg/credhelper/protocol.go`):
- **`get`** — reads the server URL from stdin, resolves the stored account,
validates the device secret against the AppView, and returns
`{ServerURL, Username, Secret}` JSON. If the OAuth session has expired it
prints the login URL and fails so Docker re-prompts; on a generic invalid
result it removes the bad account.
- **`store`** — reads `{ServerURL, Username, Secret}` from stdin. Only stores
the credential if `Secret` carries the brand's secret prefix (e.g.
`atcr_device_`); other secrets (e.g. an app password from `docker login`) are
ignored.
- **`erase`** — removes the active (or sole) account for the server URL.
- **`list`** — returns `{ "host": "username", ... }` for all stored registries.
### Device-Flow Authentication
`login` (`pkg/credhelper/cmd_login.go`, `pkg/credhelper/device_auth.go`) runs
an OAuth-style device authorization flow against the AppView:
1. `POST {appview}/auth/device/code` with `{"device_name": "<hostname>"}`.
Response: `device_code`, `user_code`, `verification_uri`, `expires_in`,
`interval`.
2. The helper shows the `user_code`, then opens (or prints)
`{verification_uri}?user_code=<code>` for the user to approve in a browser.
3. The helper polls `POST {appview}/auth/device/token` with
`{"device_code": ...}` every `interval` seconds until `expires_in`.
`authorization_pending` means keep polling; any other error aborts. On
success the response carries `device_secret`, `handle`, and `did`.
4. The account (handle, did, device secret) is saved to the brand config dir,
and the user is offered automatic Docker configuration.
### Credential Validation
`get` validates a stored device secret by calling `GET
{appview}/auth/token?service={appview}` with HTTP Basic auth
(`handle:device_secret`) and a 5s timeout (`validateCredentials` in
`device_auth.go`):
- `200` → valid.
- `401` with body `{"error":"oauth_session_expired", "login_url": ...}` →
expired; the user is told to re-login.
- `401` otherwise → invalid; the account is removed.
- Network errors or other status codes → treated as valid (don't re-auth on
transient server issues).
### Other User Commands
- **`login [registry]`** — device-flow auth (default registry from brand Config).
- **`logout [registry]`** — remove a stored account.
- **`status`** — show configured registries and accounts.
- **`switch`** — change the active account for a registry.
- **`configure-docker`** — write `credHelpers` entries to
`~/.docker/config.json` for all configured registries.
- **`update [--check]`** — self-update by resolving `{ReleasesBaseURL}/tags/latest`
and downloading the matching archive. `get` also performs a cached
(24h) background update check and prints a notice if a newer version exists.
## Testing the Distribution ## Testing the Distribution
@@ -251,10 +383,10 @@ Docker looks for binaries named `docker-credential-*` in PATH:
2. **Test specific platform:** 2. **Test specific platform:**
```bash ```bash
goreleaser build --snapshot --clean --single-target goreleaser build --snapshot --clean --single-target
./dist/docker-credential-atcr_*/docker-credential-atcr version ./dist/credential-helper_*/docker-credential-atcr version
``` ```
3. **Test full release (dry run):** 3. **Test full release without publishing:**
```bash ```bash
goreleaser release --snapshot --clean --skip=publish goreleaser release --snapshot --clean --skip=publish
``` ```
@@ -279,33 +411,14 @@ Docker looks for binaries named `docker-credential-*` in PATH:
### Package Managers ### Package Managers
**Linux:** - Homebrew (enable the commented `brews:` block in `.goreleaser.yaml`)
- `.deb` packages for Debian/Ubuntu (via GoReleaser) - `.deb` / `.rpm` packages (via GoReleaser nfpm)
- `.rpm` packages for RHEL/Fedora/CentOS - Arch AUR, Chocolatey, Scoop, Winget
- AUR package for Arch Linux
**macOS:**
- Official Homebrew core (requires popularity/maturity)
**Windows:**
- Chocolatey package
- Scoop manifest
- Winget package
### Docker Distribution ### Docker Distribution
Could also distribute the credential helper via container: The credential helper could also ship as a container image for CI/CD use, but
native binaries remain the primary distribution method.
```bash
docker run --rm atcr.io/credential-helper:latest version
# Install from container
docker run --rm -v /usr/local/bin:/install \
atcr.io/credential-helper:latest \
cp /usr/local/bin/docker-credential-atcr /install/
```
**Note:** Not recommended as primary method (users want native binaries), but useful for CI/CD pipelines.
## Troubleshooting ## Troubleshooting
@@ -357,4 +470,4 @@ uname -m
- [Docker Credential Helpers Spec](https://github.com/docker/docker-credential-helpers) - [Docker Credential Helpers Spec](https://github.com/docker/docker-credential-helpers)
- [GoReleaser Documentation](https://goreleaser.com) - [GoReleaser Documentation](https://goreleaser.com)
- [GitHub Actions: Publishing](https://docs.github.com/en/actions/publishing-packages) - [Tangled](https://tangled.org)
docs/CREDENTIAL_HELPER_V2.md
-165
View File
@@ -1,165 +0,0 @@
# Credential Helper Rewrite
## Context
The current credential helper (`cmd/credential-helper/main.go`, ~1070 lines) is a monolithic single-file binary with a manual `switch` dispatch. It has no help text, hangs silently when run without stdin, embeds interactive device auth inside the Docker protocol `get` command (blocking pushes for up to 2 minutes while polling), and only supports one account per registry. Users want multi-account support (e.g., `evan.jarrett.net` and `michelle.jarrett.net` on the same `atcr.io`) and multi-registry support (e.g., `atcr.io` + `buoy.cr`).
## Approach
Rewrite using **Cobra** (already a project dependency) for the CLI framework and **charmbracelet/huh** for interactive prompts (select menus, confirmations, spinners). Separate Docker protocol commands (machine-readable, hidden) from user-facing commands (interactive, discoverable). Model after `gh auth` UX patterns.
**Smart account auto-detection**: The `get` command inspects the parent process command line (`/proc/<ppid>/cmdline` on Linux, `ps` on macOS) to determine which image Docker is pushing/pulling. Since ATCR URLs are `host/<identity>/repo:tag`, we can extract the identity and auto-select the matching account — no prompts, no manual switching needed in the common case.
## Command Tree
```
docker-credential-atcr
├── get (Docker protocol — stdin/stdout, hidden, smart account detection)
├── store (Docker protocol — stdin, hidden)
├── erase (Docker protocol — stdin, hidden)
├── list (Docker protocol extension, hidden)
├── login (Interactive device flow with huh prompts)
├── logout (Remove account credentials)
├── status (Show all accounts with active indicators)
├── switch (Switch active account — auto-toggle for 2, select for 3+)
├── configure-docker (Auto-edit ~/.docker/config.json credHelpers)
├── update (Self-update, existing logic preserved)
└── version (Built-in via cobra)
```
## Smart Account Resolution (`get` command)
The `get` command resolves which account to use with this priority chain — fully non-interactive:
```
1. Parse parent process cmdline → extract identity from image ref
docker push atcr.io/evan.jarrett.net/test:latest
→ parent cmdline contains "evan.jarrett.net" → use that account
2. Fall back to active account (set by `switch` command)
3. Fall back to sole account (if only one exists for this registry)
4. Error with helpful message:
"Multiple accounts for atcr.io. Run: docker-credential-atcr switch"
```
**Parent process detection** (in `helpers.go`):
- Linux: read `/proc/<ppid>/cmdline` (null-separated args)
- macOS: `ps -o args= -p <ppid>`
- Windows: best-effort via `wmic` or skip (fall to active account)
- Parse image ref: find the arg matching `<registry-host>/<identity>/...`, extract `<identity>`
- Graceful failure: if parent isn't Docker, cmdline unreadable, or image ref not parseable → fall through to active account
## File Structure
```
cmd/credential-helper/
main.go — Cobra root command, version vars, subcommand registration
config.go — Config types, load/save/migrate, getConfigPath
device_auth.go — authorizeDevice(), validateCredentials() HTTP logic
protocol.go — Docker protocol: get, store, erase, list (all hidden)
cmd_login.go — login command (huh prompts + device flow)
cmd_logout.go — logout command (huh confirm)
cmd_status.go — status display
cmd_switch.go — switch command (huh select)
cmd_configure.go — configure-docker (edit ~/.docker/config.json)
cmd_update.go — update command (moved from existing code)
helpers.go — openBrowser, buildAppViewURL, isInsecureRegistry, parentCmdline, etc.
```
## Config Format (`~/.atcr/device.json`)
```json
{
"version": 2,
"registries": {
"https://atcr.io": {
"active": "evan.jarrett.net",
"accounts": {
"evan.jarrett.net": {
"handle": "evan.jarrett.net",
"did": "did:plc:abc123",
"device_secret": "atcr_device_..."
},
"michelle.jarrett.net": {
"handle": "michelle.jarrett.net",
"did": "did:plc:def456",
"device_secret": "atcr_device_..."
}
}
},
"https://buoy.cr": {
"active": "evan.jarrett.net",
"accounts": { ... }
}
}
}
```
**Migration**: `loadConfig()` auto-detects and migrates from old formats:
- Legacy single-device `{handle, device_secret, appview_url}` → v2
- Current multi-registry `{credentials: {url: {...}}}` → v2
- Writes back migrated config on first load
## Key Behavioral Changes
| Command | Current | New |
|---------|---------|-----|
| `get` | Opens browser, polls 2min if no creds | Smart detection → active account → error |
| `get` (multi-account) | N/A (single account only) | Auto-detects identity from parent cmdline |
| `get` (no stdin) | Hangs forever | Detects terminal, prints help, exits 1 |
| `get` (OAuth expired) | Auto-opens browser, polls | Prints login URL, exits 1 |
| `store` | No-op | Stores if secret is device secret (`atcr_device_*`) |
| `erase` | Removes all creds for host | Removes active account only |
| No args | Prints bare usage | Prints full cobra help with all commands |
## Dependencies
- `github.com/spf13/cobra` — already in go.mod
- `github.com/charmbracelet/huh` — new (pure Go, CGO_ENABLED=0 safe)
No changes to `.goreleaser.yaml` needed.
## Implementation Order
### Phase 1: Foundation
1. `helpers.go` — move utility functions verbatim + add `getParentCmdline()` and `detectIdentityFromParent(registryHost)`
2. `config.go` — new config types + migration from old formats
3. `main.go` — Cobra root command, register all subcommands
### Phase 2: Docker Protocol (must work for existing users)
4. `device_auth.go` — extract `authorizeDevice()` + `validateCredentials()`
5. `protocol.go` — `get`/`store`/`erase`/`list` using new config with smart account resolution
### Phase 3: User Commands
6. `cmd_login.go` — interactive device flow with huh spinner
7. `cmd_status.go` — display all registries/accounts
8. `cmd_switch.go` — huh select for account switching
9. `cmd_logout.go` — huh confirm for removal
10. `cmd_configure.go` — Docker config.json manipulation
11. `cmd_update.go` — move existing update logic
### Phase 4: Polish
12. Add `huh` to go.mod
13. Delete old `main.go` contents (replaced by new files)
## What to Keep vs Rewrite
**Keep** (move to new files): `openBrowser()`, `buildAppViewURL()`, `isInsecureRegistry()`, `getDockerInsecureRegistries()`, `readDockerDaemonConfig()`, `stripPort()`, `isTerminal()`, `authorizeDevice()` HTTP logic, `validateCredentials()`, all update/version check functions.
**Rewrite**: `main()`, `handleGet()` (split into non-interactive `get` with smart detection + interactive `login`), `handleStore()` (implement actual storage), `handleErase()` (multi-account aware), config types and loading.
**New**: `list`, `login`, `logout`, `status`, `switch`, `configure-docker` commands. Config migration. Parent process identity detection. huh integration.
## Verification
1. Build: `go build -o bin/docker-credential-atcr ./cmd/credential-helper`
2. Help works: `bin/docker-credential-atcr --help` shows all user commands
3. Protocol works: `echo "atcr.io" | bin/docker-credential-atcr get` returns credentials or helpful error
4. No hang: `bin/docker-credential-atcr get` (no stdin pipe) detects terminal, prints help, exits
5. Smart detection: `docker push atcr.io/evan.jarrett.net/test:latest` auto-selects `evan.jarrett.net`
6. Login flow: `bin/docker-credential-atcr login` triggers device auth with huh prompts
7. Status: `bin/docker-credential-atcr status` shows configured accounts
8. Config migration: Place old-format `~/.atcr/device.json`, run any command, verify auto-migration
9. GoReleaser: `CGO_ENABLED=0 go build ./cmd/credential-helper` succeeds
docs/DEVELOPMENT.md
+226 -611
View File
@@ -1,724 +1,339 @@
# Development Workflow for ATCR # Development Workflow for ATCR
## The Problem ## Goal
**Current development cycle with Docker:** Run the ATCR services (AppView, Hold, Labeler) locally with hot reload so that
1. Edit CSS, JS, template, or Go file Go, template, CSS, and JS changes show up after a fast incremental rebuild
2. Run `docker compose build` (rebuilds entire image) instead of a full production image rebuild.
3. Run `docker compose up` (restart container)
4. Wait **2-3 minutes** for changes to appear
5. Test, find issue, repeat...
**Why it's slow:** The mechanism is **Air** (`github.com/air-verse/air`) running inside a
- All assets embedded via `embed.FS` at compile time development container. Air watches the mounted source tree and rebuilds the
- Multi-stage Docker build compiles everything from scratch relevant binary on change. Production images are unaffected — they use the
- No development mode exists multi-stage `Dockerfile.appview` / `Dockerfile.hold` / `Dockerfile.scanner`
- Final image uses `scratch` base (no tools, no hot reload) builds with embedded assets.
## The Solution
**Development setup combining:**
1. **Dockerfile.devel** - Development-focused container (golang base, not scratch)
2. **Volume mounts** - Live code editing (changes appear instantly in container)
3. **DirFS** - Skip embed, read templates/CSS/JS from filesystem
4. **Air** - Auto-rebuild on Go code changes
**Results:**
- CSS/JS/Template changes: **Instant** (0 seconds, just refresh browser)
- Go code changes: **2-5 seconds** (vs 2-3 minutes)
- Production builds: **Unchanged** (still optimized with embed.FS)
## How It Works ## How It Works
All UI assets are embedded into the binary via `//go:embed` in
`pkg/appview/ui.go` (`//go:embed public` and `//go:embed templates/**/*.html`).
There is **no** filesystem-vs-embed toggle and no `ATCR_DEV_MODE` switch — the
binary always serves embedded assets. Hot reload therefore works by having Air
**rebuild the binary** whenever a watched file changes, not by reading templates
off disk at request time.
When Air rebuilds the AppView binary it runs a `pre_cmd` of
`go generate ./pkg/appview/...`. The `//go:generate` directive in
`pkg/appview/ui.go` shells out to `npm run build:appview`, which regenerates the
CSS bundle (`pkg/appview/public/css/style.css`), the JS bundle
(`pkg/appview/public/js/bundle.min.js`), and the icon sprite
(`pkg/appview/public/icons.svg`) before they are re-embedded into the new
binary.
> Do **not** run `npm run css:build` / `npm run js:build` manually. The
> `go generate` step driven by Air handles asset builds. Editing a source asset
> (CSS/JS/template) and saving triggers an Air rebuild, which regenerates and
> re-embeds the assets automatically.
### Architecture Flow ### Architecture Flow
``` ```
┌─────────────────────────────────────────────────────┐ ┌─────────────────────────────────────────────────────┐
│ Your Editor (VSCode, etc) │ │ Your editor │
│ Edit: style.css, app.js, *.html, *.go files │ │ Edit: *.go, templates/*.html, src/css/*, src/js/* │
└─────────────────┬───────────────────────────────────┘ └─────────────────┬───────────────────────────────────┘
│ (files saved to disk) │ (files saved to disk)
▼ ▼
┌─────────────────────────────────────────────────────┐ ┌─────────────────────────────────────────────────────┐
│ Volume Mount (docker-compose.dev.yml) │ │ Volume mount (docker-compose.yml) │
│ volumes: │ │ volumes: │
│ - .:/app (entire codebase mounted) │ │ - .:/app:z (entire codebase mounted) │
└─────────────────┬───────────────────────────────────┘ └─────────────────┬───────────────────────────────────┘
│ (changes appear instantly in container) │ (changes appear in container)
▼ ▼
┌─────────────────────────────────────────────────────┐ ┌─────────────────────────────────────────────────────┐
│ Container (golang:1.25.7 base, has all tools) │ │ Container (mirror.gcr.io/library/golang:1.26.2) │
│ │ │ │
│ ┌──────────────────────────────────────┐ │ │ ┌──────────────────────────────────────┐ │
│ │ Air (hot reload tool) │ │ │ │ Air (github.com/air-verse/air) │ │
│ │ Watches: *.go, *.html, *.css, *.js │ │ │ │ poll = true, poll_interval = 500 │ │
│ │ │ │ │ │ Watches: *.go *.html *.css *.js │ │
│ │ On change: │ │ │ │ │ │
│ │ - *.go → rebuild binary (2-5s) │ │ │ │ On change: │ │
│ │ - templates/css/js → restart only │ │ │ │ 1. pre_cmd: go generate (npm build) │ │
│ └──────────────────────────────────────┘ │ │ │ 2. cmd: go build → ./tmp/atcr-* │ │
│ │ │ │ │ 3. restart binary (entrypoint) │ │
│ ▼ │ │ └──────────────────────────────────────┘ │
│ ┌──────────────────────────────────────┐ │ │ │ │
│ │ ATCR AppView (ATCR_DEV_MODE=true) │ │ │ ▼ │
│ │ │ │ │ ATCR AppView (serves embedded assets from │
│ │ ui.go checks DEV_MODE: │ │ │ the freshly built binary) │
│ │ if DEV_MODE: │ │
│ │ templatesFS = os.DirFS("...") │ │
│ │ publicFS = os.DirFS("...") │ │
│ │ else: │ │
│ │ use embed.FS (production) │ │
│ │ │ │
│ │ Result: Reads from mounted files │ │
│ └──────────────────────────────────────┘ │
└─────────────────────────────────────────────────────┘ └─────────────────────────────────────────────────────┘
``` ```
### Change Scenarios Polling (`poll = true`, `poll_interval = 500`) is **required**: inotify/fsnotify
events do not propagate reliably across Docker bind mounts, so Air polls the
mounted tree every 500ms instead.
#### Scenario 1: Edit CSS/JS/Templates ## Files Involved
```
1. Edit pkg/appview/public/css/style.css in VSCode
2. Save file
3. Change appears in container via volume mount (instant)
4. App uses os.DirFS → reads new file from disk (instant)
5. Refresh browser → see changes
```
**Time:** **Instant** (0 seconds)
**No rebuild, no restart!**
#### Scenario 2: Edit Go Code | File | Purpose |
``` |------|---------|
1. Edit pkg/appview/handlers/home.go | `Dockerfile.dev` | Single dev image used by all three services. `golang:1.26.2-trixie` base with Air, Node/npm, and SQLite installed. Source comes from a volume mount, not `COPY`. Accepts an `AIR_CONFIG` build arg to select which `.air.*.toml` to run. |
2. Save file | `docker-compose.yml` | The dev compose file (this *is* the primary compose file — there is no separate `docker-compose.dev.yml`). Defines `atcr-appview`, `atcr-hold`, `atcr-labeler`, and `victorialogs`, all on a fixed `172.28.0.0/24` network. |
3. Air detects .go file change | `.air.toml` | AppView Air config (default `AIR_CONFIG`). |
4. Air runs: go build -o ./tmp/atcr-appview ./cmd/appview | `.air.hold.toml` | Hold Air config (selected via `AIR_CONFIG=.air.hold.toml`). |
5. Air kills old process and starts new binary | `.air.labeler.toml` | Labeler Air config (selected via `AIR_CONFIG=.air.labeler.toml`). |
6. App runs with new code
```
**Time:** **2-5 seconds**
**Fast incremental build!**
## Implementation ## `Dockerfile.dev`
### Step 1: Create Dockerfile.devel
Create `Dockerfile.devel` in project root:
```dockerfile ```dockerfile
# Development Dockerfile with hot reload support # Development image with Air hot reload
FROM golang:1.25.7-trixie FROM mirror.gcr.io/library/golang:1.26.2-trixie
# Install Air for hot reload ARG AIR_CONFIG=.air.toml
RUN go install github.com/cosmtrek/air@latest
# Install SQLite (required for CGO in ATCR) ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y \ ENV AIR_CONFIG=${AIR_CONFIG}
sqlite3 \
libsqlite3-dev \ RUN apt-get update && \
&& rm -rf /var/lib/apt/lists/* apt-get install -y --no-install-recommends sqlite3 libsqlite3-dev curl nodejs npm && \
rm -rf /var/lib/apt/lists/* && \
go install github.com/air-verse/air@latest
WORKDIR /app WORKDIR /app
# Copy dependency files and download (cached layer) # Copy go.mod first for layer caching
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
# Note: Source code comes from volume mount # For development: source mounted as volume, Air handles builds
# (no COPY . . needed - that's the whole point!) CMD ["sh", "-c", "air -c ${AIR_CONFIG}"]
# Air will handle building and running
CMD ["air", "-c", ".air.toml"]
``` ```
### Step 2: Create docker-compose.dev.yml Note the Air install path is `github.com/air-verse/air@latest`. The old
`github.com/cosmtrek/air` module is archived and must not be used.
Create `docker-compose.dev.yml` in project root: ## `.air.toml` (AppView)
```yaml This is the real file — keep it in sync rather than copying a hand-written
version: '3.8' version. Load-bearing settings:
services:
atcr-appview:
build:
context: .
dockerfile: Dockerfile.devel
volumes:
# Mount entire codebase (live editing)
- .:/app
# Cache Go modules (faster rebuilds)
- go-cache:/go/pkg/mod
# Persist SQLite database
- atcr-ui-dev:/var/lib/atcr
environment:
# Enable development mode (uses os.DirFS)
ATCR_DEV_MODE: "true"
# AppView configuration
ATCR_HTTP_ADDR: ":5000"
ATCR_BASE_URL: "http://localhost:5000"
ATCR_DEFAULT_HOLD_DID: "did:web:hold01.atcr.io"
# Database
ATCR_UI_DATABASE_PATH: "/var/lib/atcr/ui.db"
# Auth
ATCR_AUTH_KEY_PATH: "/var/lib/atcr/auth/private-key.pem"
# Jetstream (optional)
# JETSTREAM_URL: "wss://jetstream2.us-east.bsky.network/subscribe"
# ATCR_BACKFILL_ENABLED: "false"
ports:
- "5000:5000"
networks:
- atcr-dev
# Add other services as needed (postgres, hold, etc)
# atcr-hold:
# ...
networks:
atcr-dev:
driver: bridge
volumes:
go-cache:
atcr-ui-dev:
```
### Step 3: Create .air.toml
Create `.air.toml` in project root:
```toml ```toml
# Air configuration for hot reload
# https://github.com/cosmtrek/air
root = "." root = "."
testdata_dir = "testdata"
tmp_dir = "tmp" tmp_dir = "tmp"
[build] [build]
# Arguments to pass to binary (AppView needs "serve") # Use polling for Docker volume mounts (inotify doesn't work across mounts)
args_bin = ["serve"] poll = true
poll_interval = 500
# Where to output the built binary # Pre-build: generate assets if missing (each string is a shell command)
bin = "./tmp/atcr-appview" pre_cmd = ["go generate ./pkg/appview/..."]
cmd = "go build -tags billing -buildvcs=false -o ./tmp/atcr-appview ./cmd/appview"
# Build command entrypoint = ["./tmp/atcr-appview", "serve", "--config", "config-appview.example.yaml"]
cmd = "go build -o ./tmp/atcr-appview ./cmd/appview" include_ext = ["go", "html", "css", "js"]
exclude_dir = ["bin", "tmp", "vendor", "deploy", "docs", ".git", "dist", "node_modules", "scanner", "pkg/hold", "pkg/labeler"]
# Delay before rebuilding (ms) - debounce rapid saves exclude_regex = ["_test\\.go$", "cbor_gen\\.go$", "\\.min\\.js$", "public/css/style\\.css$", "public/icons\\.svg$"]
delay = 1000 delay = 3000
stop_on_error = true
# Directories to exclude from watching send_interrupt = true
exclude_dir = [ kill_delay = 3000
"tmp",
"vendor",
"bin",
".git",
"node_modules",
"testdata"
]
# Files to exclude from watching
exclude_file = []
# Regex patterns to exclude
exclude_regex = ["_test\\.go"]
# Don't rebuild if file content unchanged
exclude_unchanged = false
# Follow symlinks
follow_symlink = false
# Full command to run (leave empty to use cmd + bin)
full_bin = ""
# Directories to include (empty = all)
include_dir = []
# File extensions to watch
include_ext = ["go", "html", "css", "js"]
# Specific files to watch
include_file = []
# Delay before killing old process (s)
kill_delay = "0s"
# Log file for build errors
log = "build-errors.log"
# Use polling instead of fsnotify (for Docker/VM)
poll = false
poll_interval = 0
# Rerun binary if it exits
rerun = false
rerun_delay = 500
# Send interrupt signal instead of kill
send_interrupt = false
# Stop on build error
stop_on_error = false
[color]
# Colorize output
app = ""
build = "yellow"
main = "magenta"
runner = "green"
watcher = "cyan"
[log]
# Show only app logs (not build logs)
main_only = false
# Add timestamp to logs
time = false
[misc]
# Clean tmp directory on exit
clean_on_exit = false
[screen]
# Clear screen on rebuild
clear_on_rebuild = false
# Keep scrollback
keep_scroll = true
``` ```
### Step 4: Modify pkg/appview/ui.go Key points that differ from a naive config:
Add conditional filesystem loading to `pkg/appview/ui.go`: - `poll = true` / `poll_interval = 500` — needed for Docker bind mounts.
- `pre_cmd` runs `go generate ./pkg/appview/...`, which regenerates and
re-embeds CSS/JS/icons before the build.
- `cmd` builds with `-tags billing` (AppView dev runs with billing support) and
`-buildvcs=false`.
- `entrypoint` is the full argv for the built binary: it runs
`serve --config config-appview.example.yaml`. (The example config is the dev
base config; env vars in `docker-compose.yml` override it.)
- The `exclude_regex` deliberately ignores the *generated* asset outputs
(`*.min.js`, `public/css/style.css`, `public/icons.svg`) so regeneration does
not trigger an infinite rebuild loop.
```go `.air.hold.toml` and `.air.labeler.toml` are analogous: they build
package appview `./cmd/hold` / `./cmd/labeler`, generate `./pkg/hold/...` (the labeler has no
generate step), and exclude the other services' packages from watching.
import ( ## Configuration via Environment Variables
"embed"
"html/template"
"io/fs"
"log"
"net/http"
"os"
)
// Embedded assets (used in production) `docker-compose.yml` sets a base config file per service via the Air
//go:embed templates/**/*.html `entrypoint` (`config-appview.example.yaml`, `config-hold.example.yaml`,
var embeddedTemplatesFS embed.FS `config-labeler.example.yaml`) and overrides specific values with environment
variables. Viper maps env var names from the YAML path, prefixed with the
service prefix and joined with `_`.
//go:embed static Real AppView env vars (note these are the *Viper-mapped* names, not invented
var embeddedpublicFS embed.FS shorthand):
// Actual filesystems used at runtime (conditional) | Env var | Maps to |
var templatesFS fs.FS |---------|---------|
var publicFS fs.FS | `ATCR_SERVER_ADDR` | `server.addr` (listen address, e.g. `:5000`) |
| `ATCR_SERVER_BASE_URL` | `server.base_url` |
| `ATCR_SERVER_MANAGED_HOLDS` | `server.managed_holds` — comma-separated DID list; **the first entry is the default blob-storage hold**. Viper splits on commas. |
| `ATCR_AUTH_CERT_PATH` | `auth.cert_path` |
| `ATCR_JETSTREAM_BACKFILL_ENABLED` | `jetstream.backfill_enabled` |
| `ATCR_LABELER_DID` | `labeler.did` |
| `ATCR_SERVER_TEST_MODE` | `server.test_mode` |
| `ATCR_LOG_LEVEL` | `log.level` |
func init() { There is **no** `ATCR_DEV_MODE` variable anywhere in the codebase. Likewise
// Development mode: read from filesystem for instant updates `ATCR_HTTP_ADDR`, `ATCR_BASE_URL`, `ATCR_DEFAULT_HOLD_DID`, `ATCR_AUTH_KEY_PATH`,
if os.Getenv("ATCR_DEV_MODE") == "true" { and `ATCR_BACKFILL_ENABLED` are *not* real — use the Viper-mapped names above.
log.Println("🔧 DEV MODE: Using filesystem for templates and static assets")
templatesFS = os.DirFS("pkg/appview/templates")
publicFS = os.DirFS("pkg/appview/static")
} else {
// Production mode: use embedded assets
log.Println("📦 PRODUCTION MODE: Using embedded assets")
templatesFS = embeddedTemplatesFS
publicFS = embeddedpublicFS
}
}
// Templates returns parsed HTML templates Hold and Labeler use the `HOLD_` and `LABELER_` prefixes respectively
func Templates() *template.Template { (e.g. `HOLD_SERVER_PUBLIC_URL`, `HOLD_SERVER_APPVIEW_DID`,
tmpl, err := template.ParseFS(templatesFS, "templates/**/*.html") `LABELER_LABELER_PUBLIC_URL`). See `docker-compose.yml` for the dev values.
if err != nil {
log.Fatalf("Failed to parse templates: %v", err)
}
return tmpl
}
// StaticHandler returns a handler for static files S3/Storj credentials and shared secrets are loaded from an external
func StaticHandler() http.Handler { `../atcr-secrets.env` file referenced via `env_file:` in `docker-compose.yml`.
sub, err := fs.Sub(publicFS, "static")
if err != nil {
log.Fatalf("Failed to create static sub-filesystem: %v", err)
}
return http.FileServer(http.FS(sub))
}
```
**Important:** Update the `Templates()` function to NOT cache templates in dev mode:
```go
// Templates returns parsed HTML templates
func Templates() *template.Template {
// In dev mode, reparse templates on every request (instant updates)
// In production, this could be cached
tmpl, err := template.ParseFS(templatesFS, "templates/**/*.html")
if err != nil {
log.Fatalf("Failed to parse templates: %v", err)
}
return tmpl
}
```
If you're caching templates, wrap it with a dev mode check:
```go
var templateCache *template.Template
func Templates() *template.Template {
// Development: reparse every time (instant updates)
if os.Getenv("ATCR_DEV_MODE") == "true" {
tmpl, err := template.ParseFS(templatesFS, "templates/**/*.html")
if err != nil {
log.Printf("Template parse error: %v", err)
return template.New("error")
}
return tmpl
}
// Production: use cached templates
if templateCache == nil {
tmpl, err := template.ParseFS(templatesFS, "templates/**/*.html")
if err != nil {
log.Fatalf("Failed to parse templates: %v", err)
}
templateCache = tmpl
}
return templateCache
}
```
### Step 5: Add to .gitignore
Add Air's temporary directory to `.gitignore`:
```
# Air hot reload
tmp/
build-errors.log
```
## Usage ## Usage
### Starting Development Environment ### Start the dev environment
`docker-compose.yml` is the dev compose file, so no `-f` flag is needed:
```bash ```bash
# Build and start dev container # Build and start everything (appview, hold, labeler, victorialogs)
docker compose -f docker-compose.dev.yml up --build docker compose up --build
# Or run in background # Or in the background
docker compose -f docker-compose.dev.yml up -d docker compose up -d
# View logs # Tail a single service
docker compose -f docker-compose.dev.yml logs -f atcr-appview docker compose logs -f atcr-appview
``` ```
You should see Air starting: Services bind to fixed ports on the host:
- AppView: http://localhost:5000
- Hold: http://localhost:8080
- Labeler: http://localhost:5002
- Victoria Logs: http://localhost:9428
On a clean start you should see Air bootstrap, run the pre-build generate step,
build, and launch the binary, e.g.:
``` ```
atcr-appview | 🔧 DEV MODE: Using filesystem for templates and static assets
atcr-appview |
atcr-appview | __ _ ___
atcr-appview | / /\ | | | |_)
atcr-appview | /_/--\ |_| |_| \_ , built with Go
atcr-appview |
atcr-appview | watching . atcr-appview | watching .
atcr-appview | !exclude tmp atcr-appview | !exclude tmp
atcr-appview | running pre_cmd: go generate ./pkg/appview/...
atcr-appview | building... atcr-appview | building...
atcr-appview | running... atcr-appview | running...
atcr-appview | <appview startup logs: server listening on :5000 ...>
``` ```
### Development Workflow ### Daily workflow
#### 1. Edit Templates/CSS/JS (Instant Updates) - **Edit Go code** → save → Air rebuilds (`go build`) and restarts the binary in
a few seconds.
- **Edit a template** (`pkg/appview/templates/**/*.html`) → save → Air rebuilds
so the new template is re-embedded.
- **Edit CSS source** (`pkg/appview/src/css/main.css`) → save → the `pre_cmd`
`go generate` regenerates `pkg/appview/public/css/style.css` via `npm run
build:appview`, then the binary rebuilds.
- **Edit JS source** (`pkg/appview/src/js/main.js`) → save → `go generate`
regenerates `pkg/appview/public/js/bundle.min.js`, then the binary rebuilds.
```bash Important asset-source vs. generated-output distinctions:
# Edit any template, CSS, or JS file
vim pkg/appview/templates/pages/home.html
vim pkg/appview/public/css/style.css
vim pkg/appview/public/js/app.js
# Save file → changes appear instantly | You edit (source) | Do NOT edit (generated) |
# Just refresh browser (Cmd+R / Ctrl+R) |-------------------|--------------------------|
``` | `pkg/appview/src/css/main.css` | `pkg/appview/public/css/style.css` |
| `pkg/appview/src/js/main.js` | `pkg/appview/public/js/bundle.min.js` |
| icon references in templates | `pkg/appview/public/icons.svg` |
**No rebuild, no restart!** Air might restart the app, but it's instant since no compilation is needed. Refresh the browser after the rebuild completes.
#### 2. Edit Go Code (Fast Rebuild) ### Stop the dev environment
```bash
# Edit any Go file
vim pkg/appview/handlers/home.go
# Save file → Air detects change
# Air output shows:
# building...
# build successful in 2.3s
# restarting...
# Refresh browser to see changes
```
**2-5 second rebuild** instead of 2-3 minutes!
### Stopping Development Environment
```bash ```bash
# Stop containers # Stop containers
docker compose -f docker-compose.dev.yml down docker compose down
# Stop and remove volumes (fresh start) # Stop and wipe volumes (fresh DB / PDS / labeler state)
docker compose -f docker-compose.dev.yml down -v docker compose down -v
``` ```
## Production Builds ## Local Development (No Docker)
**Production builds are completely unchanged:** For a tighter loop you can run a single service on the host. The `make dev`
target runs the AppView under Air using `.air.toml`:
```bash ```bash
# Production uses normal Dockerfile (embed.FS, scratch base) make dev
docker compose build
# Or specific service
docker compose build atcr-appview
# Run production
docker compose up
``` ```
**Why it works:** `make dev` ensures Air is installed (`go install github.com/air-verse/air@latest`),
- Production doesn't set `ATCR_DEV_MODE=true` builds the generated assets, and runs `air -c .air.toml`.
- `ui.go` defaults to embedded assets when env var is unset
- Production Dockerfile still uses multi-stage build to scratch
- No development dependencies in production image
## Comparison You can also run Air directly, or skip hot reload entirely:
| Change Type | Before (docker compose) | After (dev setup) | Improvement |
|-------------|------------------------|-------------------|-------------|
| Edit CSS | 2-3 minutes | **Instant (0s)** | ♾️x faster |
| Edit JS | 2-3 minutes | **Instant (0s)** | ♾️x faster |
| Edit Template | 2-3 minutes | **Instant (0s)** | ♾️x faster |
| Edit Go Code | 2-3 minutes | **2-5 seconds** | 24-90x faster |
| Production Build | Same | **Same** | No change |
## Advanced: Local Development (No Docker)
For even faster development, run locally without Docker:
```bash ```bash
# Set environment variables # Air, AppView config
export ATCR_DEV_MODE=true
export ATCR_HTTP_ADDR=:5000
export ATCR_BASE_URL=http://localhost:5000
export ATCR_DEFAULT_HOLD_DID=did:web:hold01.atcr.io
export ATCR_UI_DATABASE_PATH=/tmp/atcr-ui.db
export ATCR_AUTH_KEY_PATH=/tmp/atcr-auth-key.pem
# Or use .env file
source .env.appview
# Run with Air
air -c .air.toml air -c .air.toml
# Or run directly (no hot reload) # No hot reload — build and run once
go run ./cmd/appview serve go build -tags billing -o bin/atcr-appview ./cmd/appview
./bin/atcr-appview serve --config config-appview.example.yaml
``` ```
**Advantages:** Running on the host requires a working toolchain for the build:
- Even faster (no Docker overhead) Go 1.26.2 (see `go.work`), Node/npm (for the `go generate` asset step), and
- Native debugging with delve SQLite headers. Override config values with the `ATCR_*` env vars listed above,
- Direct filesystem access or edit your local config file.
- Full IDE integration
**Disadvantages:** ## Production Builds (Unchanged)
- Need to manage dependencies locally (SQLite, etc)
- May differ from production environment Production images use the multi-stage Dockerfiles and embed all assets at
compile time:
```bash
make docker # build appview + hold + scanner images
make docker-appview # just the appview image
```
These do not involve Air, do not bind-mount source, and serve embedded assets
exactly as the dev binary does — the only difference is that the dev container
rebuilds on change.
## Troubleshooting ## Troubleshooting
### Air Not Rebuilding ### Air not rebuilding
**Problem:** Air doesn't detect changes
**Solution:**
```bash
# Check if Air is actually running
docker compose -f docker-compose.dev.yml logs atcr-appview
# Check .air.toml include_ext includes your file type
# Default: ["go", "html", "css", "js"]
# Restart container
docker compose -f docker-compose.dev.yml restart atcr-appview
```
### Templates Not Updating
**Problem:** Template changes don't appear
**Solution:**
```bash
# Check ATCR_DEV_MODE is set
docker compose -f docker-compose.dev.yml exec atcr-appview env | grep DEV_MODE
# Should output: ATCR_DEV_MODE=true
# Check templates aren't cached (see Step 4 above)
# Templates() should reparse in dev mode
```
### Go Build Failing
**Problem:** Air shows build errors
**Solution:**
```bash
# Check build logs
docker compose -f docker-compose.dev.yml logs atcr-appview
# Or check build-errors.log in container
docker compose -f docker-compose.dev.yml exec atcr-appview cat build-errors.log
# Fix the Go error, save file, Air will retry
```
### Volume Mount Not Working
**Problem:** Changes don't appear in container
**Solution:**
```bash
# Verify volume mount
docker compose -f docker-compose.dev.yml exec atcr-appview ls -la /app
# Should show your source files
# On Windows/Mac, check Docker Desktop file sharing settings
# Settings → Resources → File Sharing → add project directory
```
### Permission Errors
**Problem:** Cannot write to /var/lib/atcr
**Solution:**
```bash
# In Dockerfile.devel, add:
RUN mkdir -p /var/lib/atcr && chmod 777 /var/lib/atcr
# Or use named volumes (already in docker-compose.dev.yml)
volumes:
- atcr-ui-dev:/var/lib/atcr
```
### Slow Builds Even with Air
**Problem:** Air rebuilds slowly
**Solution:**
```bash
# Use Go module cache volume (already in docker-compose.dev.yml)
volumes:
- go-cache:/go/pkg/mod
# Increase Air delay to debounce rapid saves
# In .air.toml:
delay = 2000 # 2 seconds
# Or check if CGO is slowing builds
# AppView needs CGO for SQLite, but you can try:
CGO_ENABLED=0 go build # (won't work for ATCR, but good to know)
```
## Tips & Tricks
### Browser Auto-Reload (LiveReload)
Add LiveReload for automatic browser refresh:
```bash ```bash
# Install browser extension docker compose logs atcr-appview
# Chrome: https://chrome.google.com/webstore/detail/livereload # Confirm Air is running and polling. poll=true is required for bind mounts;
# Firefox: https://addons.mozilla.org/en-US/firefox/addon/livereload-web-extension/ # without it, saved files are never detected.
docker compose restart atcr-appview
# Add livereload to .air.toml (future Air feature)
# Or use a separate tool like browsersync
``` ```
### Database Resets Confirm your file type is in `include_ext` (`go`, `html`, `css`, `js`) and that
you are editing a *source* file, not a generated output excluded by
`exclude_regex`.
Development database is in a named volume: ### Go build failing
```bash ```bash
# Reset database (fresh start) docker compose logs atcr-appview
docker compose -f docker-compose.dev.yml down -v # Air prints build errors inline and (with stop_on_error=true) holds the old
docker compose -f docker-compose.dev.yml up # binary until the build succeeds again. Fix the error and save.
# Or delete specific volume
docker volume rm atcr_atcr-ui-dev
``` ```
### Multiple Environments ### Volume mount not working
Run dev and production side-by-side:
```bash ```bash
# Development on port 5000 docker compose exec atcr-appview ls -la /app
docker compose -f docker-compose.dev.yml up -d # Should show your source tree. On macOS/Windows check Docker Desktop file
# sharing for the project directory.
# Production on port 5001
docker compose up -d
# Now you can compare behavior
``` ```
### Debugging with Delve ### Asset changes not showing
Add delve to Dockerfile.devel: CSS/JS/icon changes only take effect after the `go generate` pre-build runs and
the binary rebuilds. If a save did not trigger a rebuild, you likely edited a
```dockerfile generated output file (excluded from watching) instead of its source under
RUN go install github.com/go-delve/delve/cmd/dlv@latest `pkg/appview/src/`.
# Change CMD to use delve
CMD ["dlv", "debug", "./cmd/appview", "--headless", "--listen=:2345", "--api-version=2", "--accept-multiclient", "--", "serve"]
```
Then connect with VSCode or GoLand.
## Summary
**Development Setup (One-Time):**
1. Create `Dockerfile.devel`
2. Create `docker-compose.dev.yml`
3. Create `.air.toml`
4. Modify `pkg/appview/ui.go` for conditional DirFS
5. Add `tmp/` to `.gitignore`
**Daily Development:**
```bash
# Start
docker compose -f docker-compose.dev.yml up
# Edit files in your editor
# Changes appear instantly (CSS/JS/templates)
# Or in 2-5 seconds (Go code)
# Stop
docker compose -f docker-compose.dev.yml down
```
**Production (Unchanged):**
```bash
docker compose build
docker compose up
```
**Result:** 100x faster development iteration! 🚀
docs/DIRECT_HOLD_ACCESS.md
+2 -2
View File
@@ -247,7 +247,7 @@ Each step after #3 requires generating a fresh DPoP proof JWT, which is why libr
### "Invalid token" or "Token expired" ### "Invalid token" or "Token expired"
Service tokens are only valid for ~60 seconds. Get a fresh one: Service tokens are requested with a 5-minute (300 second) expiry, though the PDS may grant less. Get a fresh one:
```bash ```bash
SERVICE_TOKEN=$(curl -s "$PDS/xrpc/com.atproto.server.getServiceAuth?aud=$HOLD_DID" \ SERVICE_TOKEN=$(curl -s "$PDS/xrpc/com.atproto.server.getServiceAuth?aud=$HOLD_DID" \
-H "Authorization: Bearer $ACCESS_JWT" | jq -r '.token') -H "Authorization: Bearer $ACCESS_JWT" | jq -r '.token')
@@ -290,7 +290,7 @@ curl -s "https://bsky.social/xrpc/com.atproto.repo.listRecords?repo=$DID&collect
## Security Notes ## Security Notes
- **App passwords** are scoped tokens that can be revoked without changing your main password - **App passwords** are scoped tokens that can be revoked without changing your main password
- **Service tokens** are short-lived (60 seconds) and scoped to a specific hold - **Service tokens** are short-lived (requested with a 5-minute expiry; the PDS may grant less) and scoped to a specific hold
- **Never share** your app password or access tokens - **Never share** your app password or access tokens
- Service tokens can only be used for the specific hold they were requested for (`aud` claim) - Service tokens can only be used for the specific hold they were requested for (`aud` claim)
docs/HOLD_AS_CA.md
-756
View File
@@ -1,756 +0,0 @@
# Hold-as-Certificate-Authority Architecture
## ⚠️ Important Notice
This document describes an **optional enterprise feature** for X.509 PKI compliance. The hold-as-CA approach introduces **centralization trade-offs** that contradict ATProto's decentralized philosophy.
**Default Recommendation:** Use [plugin-based integration](./INTEGRATION_STRATEGY.md) instead. Only implement hold-as-CA if your organization has specific X.509 PKI compliance requirements.
## Overview
The hold-as-CA architecture allows ATCR to generate Notation/Notary v2-compatible signatures by having hold services act as Certificate Authorities that issue X.509 certificates for users.
### The Problem
- **ATProto signatures** use K-256 (secp256k1) elliptic curve
- **Notation** only supports P-256, P-384, P-521 elliptic curves
- **Cannot convert** K-256 signatures to P-256 (different cryptographic curves)
- **Must re-sign** with P-256 keys for Notation compatibility
### The Solution
Hold services act as trusted Certificate Authorities (CAs):
1. User pushes image → Manifest signed by PDS with K-256 (ATProto)
2. Hold verifies ATProto signature is valid
3. Hold generates ephemeral P-256 key pair for user
4. Hold issues X.509 certificate to user's DID
5. Hold signs manifest with P-256 key
6. Hold creates Notation signature envelope (JWS format)
7. Stores both ATProto and Notation signatures
**Result:** Images have two signatures:
- **ATProto signature** (K-256) - Decentralized, DID-based
- **Notation signature** (P-256) - Centralized, X.509 PKI
## Architecture
### Certificate Chain
```
Hold Root CA Certificate (self-signed, P-256)
└── User Certificate (issued to DID, P-256)
└── Image Manifest Signature
```
**Hold Root CA:**
```
Subject: CN=ATCR Hold CA - did:web:hold01.atcr.io
Issuer: Self (self-signed)
Key Usage: Digital Signature, Certificate Sign
Basic Constraints: CA=true, pathLen=1
Algorithm: ECDSA P-256
Validity: 10 years
```
**User Certificate:**
```
Subject: CN=did:plc:alice123
SAN: URI:did:plc:alice123
Issuer: Hold Root CA
Key Usage: Digital Signature
Extended Key Usage: Code Signing
Algorithm: ECDSA P-256
Validity: 24 hours (short-lived)
```
### Push Flow
```
┌──────────────────────────────────────────────────────┐
│ 1. User: docker push atcr.io/alice/myapp:latest │
└────────────────────┬─────────────────────────────────┘
↓
┌──────────────────────────────────────────────────────┐
│ 2. AppView stores manifest in alice's PDS │
│ - PDS signs with K-256 (ATProto standard) │
│ - Signature stored in repository commit │
└────────────────────┬─────────────────────────────────┘
↓
┌──────────────────────────────────────────────────────┐
│ 3. AppView requests hold to co-sign │
│ POST /xrpc/io.atcr.hold.coSignManifest │
│ { │
│ "userDid": "did:plc:alice123", │
│ "manifestDigest": "sha256:abc123...", │
│ "atprotoSignature": {...} │
│ } │
└────────────────────┬─────────────────────────────────┘
↓
┌──────────────────────────────────────────────────────┐
│ 4. Hold verifies ATProto signature │
│ a. Resolve alice's DID → public key │
│ b. Fetch commit from alice's PDS │
│ c. Verify K-256 signature │
│ d. Ensure signature is valid │
│ │
│ If verification fails → REJECT │
└────────────────────┬─────────────────────────────────┘
↓
┌──────────────────────────────────────────────────────┐
│ 5. Hold generates ephemeral P-256 key pair │
│ privateKey := ecdsa.GenerateKey(elliptic.P256()) │
└────────────────────┬─────────────────────────────────┘
↓
┌──────────────────────────────────────────────────────┐
│ 6. Hold issues X.509 certificate │
│ Subject: CN=did:plc:alice123 │
│ SAN: URI:did:plc:alice123 │
│ Issuer: Hold CA │
│ NotBefore: now │
│ NotAfter: now + 24 hours │
│ KeyUsage: Digital Signature │
│ ExtKeyUsage: Code Signing │
│ │
│ Sign certificate with hold's CA private key │
└────────────────────┬─────────────────────────────────┘
↓
┌──────────────────────────────────────────────────────┐
│ 7. Hold signs manifest digest │
│ hash := SHA256(manifestBytes) │
│ signature := ECDSA_P256(hash, privateKey) │
└────────────────────┬─────────────────────────────────┘
↓
┌──────────────────────────────────────────────────────┐
│ 8. Hold creates Notation JWS envelope │
│ { │
│ "protected": {...}, │
│ "payload": "base64(manifestDigest)", │
│ "signature": "base64(p256Signature)", │
│ "header": { │
│ "x5c": [ │
│ "base64(userCert)", │
│ "base64(holdCACert)" │
│ ] │
│ } │
│ } │
└────────────────────┬─────────────────────────────────┘
↓
┌──────────────────────────────────────────────────────┐
│ 9. Hold returns signature to AppView │
└────────────────────┬─────────────────────────────────┘
↓
┌──────────────────────────────────────────────────────┐
│ 10. AppView stores Notation signature │
│ - Create ORAS artifact manifest │
│ - Upload JWS envelope as layer blob │
│ - Link to image via subject field │
│ - artifactType: application/vnd.cncf.notary... │
└──────────────────────────────────────────────────────┘
```
### Verification Flow
```
┌──────────────────────────────────────────────────────┐
│ User: notation verify atcr.io/alice/myapp:latest │
└────────────────────┬─────────────────────────────────┘
↓
┌──────────────────────────────────────────────────────┐
│ 1. Notation queries Referrers API │
│ GET /v2/alice/myapp/referrers/sha256:abc123 │
│ → Discovers Notation signature artifact │
└────────────────────┬─────────────────────────────────┘
↓
┌──────────────────────────────────────────────────────┐
│ 2. Notation downloads JWS envelope │
│ - Parses JSON Web Signature │
│ - Extracts certificate chain from x5c header │
└────────────────────┬─────────────────────────────────┘
↓
┌──────────────────────────────────────────────────────┐
│ 3. Notation validates certificate chain │
│ a. User cert issued by Hold CA? ✓ │
│ b. Hold CA cert in trust store? ✓ │
│ c. Certificate not expired? ✓ │
│ d. Key usage correct? ✓ │
│ e. Subject matches policy? ✓ │
└────────────────────┬─────────────────────────────────┘
↓
┌──────────────────────────────────────────────────────┐
│ 4. Notation verifies signature │
│ a. Extract public key from user certificate │
│ b. Compute manifest hash: SHA256(manifest) │
│ c. Verify: ECDSA_P256(hash, sig, pubKey) ✓ │
└────────────────────┬─────────────────────────────────┘
↓
┌──────────────────────────────────────────────────────┐
│ 5. Success: Image verified ✓ │
│ Signed by: did:plc:alice123 (via Hold CA) │
└──────────────────────────────────────────────────────┘
```
## Implementation
### Hold CA Certificate Generation
```go
// cmd/hold/main.go - CA initialization
func (h *Hold) initializeCA(ctx context.Context) error {
caKeyPath := filepath.Join(h.config.DataDir, "ca-private-key.pem")
caCertPath := filepath.Join(h.config.DataDir, "ca-certificate.pem")
// Load existing CA or generate new one
if exists(caKeyPath) && exists(caCertPath) {
h.caKey = loadPrivateKey(caKeyPath)
h.caCert = loadCertificate(caCertPath)
return nil
}
// Generate P-256 key pair for CA
caKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
return fmt.Errorf("failed to generate CA key: %w", err)
}
// Create CA certificate template
serialNumber, _ := rand.Int(rand.Reader, new(big.Int).Lsh(big.NewInt(1), 128))
template := &x509.Certificate{
SerialNumber: serialNumber,
Subject: pkix.Name{
CommonName: fmt.Sprintf("ATCR Hold CA - %s", h.DID),
},
NotBefore: time.Now(),
NotAfter: time.Now().AddDate(10, 0, 0), // 10 years
KeyUsage: x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign,
BasicConstraintsValid: true,
IsCA: true,
MaxPathLen: 1, // Can only issue end-entity certificates
}
// Self-sign
certDER, err := x509.CreateCertificate(
rand.Reader,
template,
template, // Self-signed: issuer = subject
&caKey.PublicKey,
caKey,
)
if err != nil {
return fmt.Errorf("failed to create CA certificate: %w", err)
}
caCert, _ := x509.ParseCertificate(certDER)
// Save to disk (0600 permissions)
savePrivateKey(caKeyPath, caKey)
saveCertificate(caCertPath, caCert)
h.caKey = caKey
h.caCert = caCert
log.Info("Generated new CA certificate", "did", h.DID, "expires", caCert.NotAfter)
return nil
}
```
### User Certificate Issuance
```go
// pkg/hold/cosign.go
func (h *Hold) issueUserCertificate(userDID string) (*x509.Certificate, *ecdsa.PrivateKey, error) {
// Generate ephemeral P-256 key for user
userKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
return nil, nil, fmt.Errorf("failed to generate user key: %w", err)
}
serialNumber, _ := rand.Int(rand.Reader, new(big.Int).Lsh(big.NewInt(1), 128))
// Parse DID for SAN
sanURI, _ := url.Parse(userDID)
template := &x509.Certificate{
SerialNumber: serialNumber,
Subject: pkix.Name{
CommonName: userDID,
},
URIs: []*url.URL{sanURI}, // Subject Alternative Name
NotBefore: time.Now(),
NotAfter: time.Now().Add(24 * time.Hour), // Short-lived: 24 hours
KeyUsage: x509.KeyUsageDigitalSignature,
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageCodeSigning},
BasicConstraintsValid: true,
IsCA: false,
}
// Sign with hold's CA key
certDER, err := x509.CreateCertificate(
rand.Reader,
template,
h.caCert, // Issuer: Hold CA
&userKey.PublicKey,
h.caKey, // Sign with CA private key
)
if err != nil {
return nil, nil, fmt.Errorf("failed to create user certificate: %w", err)
}
userCert, _ := x509.ParseCertificate(certDER)
return userCert, userKey, nil
}
```
### Co-Signing XRPC Endpoint
```go
// pkg/hold/oci/xrpc.go
func (s *Server) handleCoSignManifest(ctx context.Context, req *CoSignRequest) (*CoSignResponse, error) {
// 1. Verify caller is authenticated
did, err := s.auth.VerifyToken(ctx, req.Token)
if err != nil {
return nil, fmt.Errorf("authentication failed: %w", err)
}
// 2. Verify ATProto signature
valid, err := s.verifyATProtoSignature(ctx, req.UserDID, req.ManifestDigest, req.ATProtoSignature)
if err != nil || !valid {
return nil, fmt.Errorf("ATProto signature verification failed: %w", err)
}
// 3. Issue certificate for user
userCert, userKey, err := s.hold.issueUserCertificate(req.UserDID)
if err != nil {
return nil, fmt.Errorf("failed to issue certificate: %w", err)
}
// 4. Sign manifest with user's key
manifestHash := sha256.Sum256([]byte(req.ManifestDigest))
signature, err := ecdsa.SignASN1(rand.Reader, userKey, manifestHash[:])
if err != nil {
return nil, fmt.Errorf("failed to sign manifest: %w", err)
}
// 5. Create JWS envelope
jws, err := s.createJWSEnvelope(signature, userCert, s.hold.caCert, req.ManifestDigest)
if err != nil {
return nil, fmt.Errorf("failed to create JWS: %w", err)
}
return &CoSignResponse{
JWS: jws,
Certificate: encodeCertificate(userCert),
CACertificate: encodeCertificate(s.hold.caCert),
}, nil
}
```
## Trust Model
### Centralization Analysis
**ATProto Model (Decentralized):**
- Each PDS is independent
- User controls which PDS to use
- Trust user's DID, not specific infrastructure
- PDS compromise affects only that PDS's users
- Multiple PDSs provide redundancy
**Hold-as-CA Model (Centralized):**
- Hold acts as single Certificate Authority
- All users must trust hold's CA certificate
- Hold compromise = attacker can issue certificates for ANY user
- Hold becomes single point of failure
- Users depend on hold operator honesty
### What Hold Vouches For
When hold issues a certificate, it attests:
✅ **"I verified that [DID] signed this manifest with ATProto"**
- Hold validated ATProto signature
- Hold confirmed signature matches user's DID
- Hold checked signature at specific time
❌ **"This image is safe"**
- Hold does NOT audit image contents
- Certificate ≠ vulnerability scan
- Signature ≠ security guarantee
❌ **"I control this DID"**
- Hold does NOT control user's DID
- DID ownership is independent
- Hold cannot revoke DIDs
### Threat Model
**Scenario 1: Hold Private Key Compromise**
**Attack:**
- Attacker steals hold's CA private key
- Can issue certificates for any DID
- Can sign malicious images as any user
**Impact:**
- **CRITICAL** - All users affected
- Attacker can impersonate any user
- All signatures become untrustworthy
**Detection:**
- Certificate Transparency logs (if implemented)
- Unusual certificate issuance patterns
- Users report unexpected signatures
**Mitigation:**
- Store CA key in Hardware Security Module (HSM)
- Strict access controls
- Audit logging
- Regular key rotation
**Recovery:**
- Revoke compromised CA certificate
- Generate new CA certificate
- Re-issue all active certificates
- Notify all users
- Update trust stores
---
**Scenario 2: Malicious Hold Operator**
**Attack:**
- Hold operator issues certificates without verifying ATProto signatures
- Hold operator signs malicious images
- Hold operator backdates certificates
**Impact:**
- **HIGH** - Trust model broken
- Users receive signed malicious images
- Difficult to detect without ATProto cross-check
**Detection:**
- Compare Notation signature timestamp with ATProto commit time
- Verify ATProto signature exists independently
- Monitor hold's signing patterns
**Mitigation:**
- Audit trail linking certificates to ATProto signatures
- Public transparency logs
- Multi-signature requirements
- Periodically verify ATProto signatures
**Recovery:**
- Identify malicious certificates
- Revoke hold's CA trust
- Switch to different hold
- Re-verify all images
---
**Scenario 3: Certificate Theft**
**Attack:**
- Attacker steals issued user certificate + private key
- Uses it to sign malicious images
**Impact:**
- **LOW-MEDIUM** - Limited scope
- Affects only specific user/image
- Short validity period (24 hours)
**Detection:**
- Unexpected signature timestamps
- Images signed from unknown locations
**Mitigation:**
- Short certificate validity (24 hours)
- Ephemeral keys (not stored long-term)
- Certificate revocation if detected
**Recovery:**
- Wait for certificate expiration (24 hours)
- Revoke specific certificate
- Investigate compromise source
## Certificate Management
### Expiration Strategy
**Short-Lived Certificates (24 hours):**
**Pros:**
- ✅ Minimal revocation infrastructure needed
- ✅ Compromise window is tiny
- ✅ Automatic cleanup
- ✅ Lower CRL/OCSP overhead
**Cons:**
- ❌ Old images become unverifiable quickly
- ❌ Requires re-signing for historical verification
- ❌ Storage: multiple signatures for same image
**Solution: On-Demand Re-Signing**
```
User pulls old image → Notation verification fails (expired cert)
→ User requests re-signing: POST /xrpc/io.atcr.hold.reSignManifest
→ Hold verifies ATProto signature still valid
→ Hold issues new certificate (24 hours)
→ Hold creates new Notation signature
→ User can verify with fresh certificate
```
### Revocation
**Certificate Revocation List (CRL):**
```
Hold publishes CRL at: https://hold01.atcr.io/ca.crl
Notation configured to check CRL:
{
"trustPolicies": [{
"name": "atcr-images",
"signatureVerification": {
"verificationLevel": "strict",
"override": {
"revocationValidation": "strict"
}
}
}]
}
```
**OCSP (Online Certificate Status Protocol):**
- Hold runs OCSP responder: `https://hold01.atcr.io/ocsp`
- Real-time certificate status checks
- Lower overhead than CRL downloads
**Revocation Triggers:**
- Key compromise detected
- Malicious signing detected
- User request
- DID ownership change
### CA Key Rotation
**Rotation Procedure:**
1. **Generate new CA key pair**
2. **Create new CA certificate**
3. **Cross-sign old CA with new CA** (transition period)
4. **Distribute new CA certificate** to all users
5. **Begin issuing with new CA** for new signatures
6. **Grace period** (30 days): Accept both old and new CA
7. **Retire old CA** after grace period
**Frequency:** Every 2-3 years (longer than short-lived certs)
## Trust Store Distribution
### Problem
Users must add hold's CA certificate to their Notation trust store for verification to work.
### Manual Distribution
```bash
# 1. Download hold's CA certificate
curl https://hold01.atcr.io/ca.crt -o hold01-ca.crt
# 2. Verify fingerprint (out-of-band)
openssl x509 -in hold01-ca.crt -fingerprint -noout
# Compare with published fingerprint
# 3. Add to Notation trust store
notation cert add --type ca --store atcr-holds hold01-ca.crt
```
### Automated Distribution
**ATCR CLI tool:**
```bash
atcr trust add hold01.atcr.io
# → Fetches CA certificate
# → Verifies via HTTPS + DNSSEC
# → Adds to Notation trust store
# → Configures trust policy
atcr trust list
# → Shows trusted holds with fingerprints
```
### System-Wide Trust
**For enterprise deployments:**
**Debian/Ubuntu:**
```bash
# Install CA certificate system-wide
cp hold01-ca.crt /usr/local/share/ca-certificates/atcr-hold01.crt
update-ca-certificates
```
**RHEL/CentOS:**
```bash
cp hold01-ca.crt /etc/pki/ca-trust/source/anchors/
update-ca-trust
```
**Container images:**
```dockerfile
FROM ubuntu:22.04
COPY hold01-ca.crt /usr/local/share/ca-certificates/
RUN update-ca-certificates
```
## Configuration
### Hold Service
**Environment variables:**
```bash
# Enable co-signing feature
HOLD_COSIGN_ENABLED=true
# CA certificate and key paths
HOLD_CA_CERT_PATH=/var/lib/atcr/hold/ca-certificate.pem
HOLD_CA_KEY_PATH=/var/lib/atcr/hold/ca-private-key.pem
# Certificate validity
HOLD_CERT_VALIDITY_HOURS=24
# OCSP responder
HOLD_OCSP_ENABLED=true
HOLD_OCSP_URL=https://hold01.atcr.io/ocsp
# CRL distribution
HOLD_CRL_ENABLED=true
HOLD_CRL_URL=https://hold01.atcr.io/ca.crl
```
### Notation Trust Policy
```json
{
"version": "1.0",
"trustPolicies": [{
"name": "atcr-images",
"registryScopes": ["atcr.io/*/*"],
"signatureVerification": {
"level": "strict",
"override": {
"revocationValidation": "strict"
}
},
"trustStores": ["ca:atcr-holds"],
"trustedIdentities": [
"x509.subject: CN=did:plc:*",
"x509.subject: CN=did:web:*"
]
}]
}
```
## When to Use Hold-as-CA
### ✅ Use When
**Enterprise X.509 PKI Compliance:**
- Organization requires standard X.509 certificates
- Existing security policies mandate PKI
- Audit requirements for certificate chains
- Integration with existing CA infrastructure
**Tool Compatibility:**
- Must use standard Notation without plugins
- Cannot deploy custom verification tools
- Existing tooling expects X.509 signatures
**Centralized Trust Acceptable:**
- Organization already uses centralized trust model
- Hold operator is internal/trusted team
- Centralization risk is acceptable trade-off
### ❌ Don't Use When
**Default Deployment:**
- Most users should use [plugin-based approach](./INTEGRATION_STRATEGY.md)
- Plugins maintain decentralization
- Plugins reuse existing ATProto signatures
**Small Teams / Startups:**
- Certificate management overhead too high
- Don't need X.509 compliance
- Prefer simpler architecture
**Maximum Decentralization Required:**
- Cannot accept hold as single trust point
- Must maintain pure ATProto model
- Centralization contradicts project goals
## Comparison: Hold-as-CA vs. Plugins
| Aspect | Hold-as-CA | Plugin Approach |
|--------|------------|----------------|
| **Standard compliance** | ✅ Full X.509/PKI | ⚠️ Custom verification |
| **Tool compatibility** | ✅ Notation works unchanged | ❌ Requires plugin install |
| **Decentralization** | ❌ Centralized (hold CA) | ✅ Decentralized (DIDs) |
| **ATProto alignment** | ❌ Against philosophy | ✅ ATProto-native |
| **Signature reuse** | ❌ Must re-sign (P-256) | ✅ Reuses ATProto (K-256) |
| **Certificate mgmt** | 🔴 High overhead | 🟢 None |
| **Trust distribution** | 🔴 Must distribute CA cert | 🟢 DID resolution |
| **Hold compromise** | 🔴 All users affected | 🟢 Metadata only |
| **Operational cost** | 🔴 High | 🟢 Low |
| **Use case** | Enterprise PKI | General purpose |
## Recommendations
### Default Approach: Plugins
For most deployments, use plugin-based verification:
- **Ratify plugin** for Kubernetes
- **OPA Gatekeeper provider** for policy enforcement
- **Containerd verifier** for runtime checks
- **atcr-verify CLI** for general purpose
See [Integration Strategy](./INTEGRATION_STRATEGY.md) for details.
### Optional: Hold-as-CA for Enterprise
Only implement hold-as-CA if you have specific requirements:
- Enterprise X.509 PKI mandates
- Cannot use plugins (restricted environments)
- Accept centralization trade-off
**Implement as opt-in feature:**
```bash
# Users explicitly enable co-signing
docker push atcr.io/alice/myapp:latest --sign=notation
# Or via environment variable
export ATCR_ENABLE_COSIGN=true
docker push atcr.io/alice/myapp:latest
```
### Security Best Practices
**If implementing hold-as-CA:**
1. **Store CA key in HSM** - Never on filesystem
2. **Audit all certificate issuance** - Log every cert
3. **Public transparency log** - Publish all certificates
4. **Short certificate validity** - 24 hours max
5. **Monitor unusual patterns** - Alert on anomalies
6. **Regular CA key rotation** - Every 2-3 years
7. **Cross-check ATProto** - Verify both signatures match
8. **Incident response plan** - Prepare for compromise
## See Also
- [ATProto Signatures](./ATPROTO_SIGNATURES.md) - How ATProto signing works
- [Integration Strategy](./INTEGRATION_STRATEGY.md) - Overview of integration approaches
- [Signature Integration](./SIGNATURE_INTEGRATION.md) - Tool-specific integration guides
docs/HOLD_DISCOVERY.md
+181 -1643
View File
File diff suppressed because it is too large Load Diff
docs/HOLD_XRPC_ENDPOINTS.md
+39 -14
View File
@@ -13,10 +13,12 @@ This document lists all XRPC endpoints implemented in the Hold service (`pkg/hol
| `/xrpc/com.atproto.repo.describeRepo` | GET | Repository information | | `/xrpc/com.atproto.repo.describeRepo` | GET | Repository information |
| `/xrpc/com.atproto.repo.getRecord` | GET | Retrieve a single record | | `/xrpc/com.atproto.repo.getRecord` | GET | Retrieve a single record |
| `/xrpc/com.atproto.repo.listRecords` | GET | List records in a collection (paginated) | | `/xrpc/com.atproto.repo.listRecords` | GET | List records in a collection (paginated) |
| `/xrpc/com.atproto.sync.listBlobs` | GET | List blob CIDs for an account |
| `/xrpc/com.atproto.sync.listRepos` | GET | List all repositories | | `/xrpc/com.atproto.sync.listRepos` | GET | List all repositories |
| `/xrpc/com.atproto.sync.getRecord` | GET | Get record as CAR file | | `/xrpc/com.atproto.sync.getRecord` | GET | Get record as CAR file |
| `/xrpc/com.atproto.sync.getRepo` | GET | Full repository as CAR file | | `/xrpc/com.atproto.sync.getRepo` | GET | Full repository as CAR file |
| `/xrpc/com.atproto.sync.getRepoStatus` | GET | Repository hosting status | | `/xrpc/com.atproto.sync.getRepoStatus` | GET | Repository hosting status |
| `/xrpc/com.atproto.sync.getLatestCommit` | GET | Current commit CID and revision |
| `/xrpc/com.atproto.sync.subscribeRepos` | GET | WebSocket firehose | | `/xrpc/com.atproto.sync.subscribeRepos` | GET | WebSocket firehose |
| `/xrpc/com.atproto.identity.resolveHandle` | GET | Resolve handle to DID | | `/xrpc/com.atproto.identity.resolveHandle` | GET | Resolve handle to DID |
| `/xrpc/app.bsky.actor.getProfile` | GET | Get actor profile | | `/xrpc/app.bsky.actor.getProfile` | GET | Get actor profile |
@@ -37,19 +39,40 @@ This document lists all XRPC endpoints implemented in the Hold service (`pkg/hol
|----------|--------|-------------| |----------|--------|-------------|
| `/xrpc/com.atproto.repo.deleteRecord` | POST | Delete a record | | `/xrpc/com.atproto.repo.deleteRecord` | POST | Delete a record |
| `/xrpc/com.atproto.repo.uploadBlob` | POST | Upload ATProto blob | | `/xrpc/com.atproto.repo.uploadBlob` | POST | Upload ATProto blob |
| `/xrpc/io.atcr.hold.purgeManifest` | POST | Purge layer/scan/image-config records for a manifest (eager delete + takedown). Idempotent. |
### Auth Required (Service Token or DPoP) ### Inline Auth (per-manifest caller check)
`/xrpc/io.atcr.hold.purgeManifest` (POST) does not use a router middleware. Auth is validated inline by `ValidateManifestPurger`, which accepts either a Bearer service token or a DPoP token, then checks the caller's role:
- Hold captain (any manifest)
- Crew member with `crew:admin` permission (any manifest)
- Crew member whose DID matches the manifest URI's DID (own manifests only)
Idempotent. Does not delete S3 blobs — GC handles those.
### Auth Required (Service Token)
The `requireAuth` middleware validates Bearer service tokens only. `requestCrew` additionally accepts DPoP tokens when called directly (the handler falls back to `ValidateDPoPRequest` if no user is in context).
| Endpoint | Method | Description | | Endpoint | Method | Description |
|----------|--------|-------------| |----------|--------|-------------|
| `/xrpc/io.atcr.hold.requestCrew` | POST | Request crew membership | | `/xrpc/io.atcr.hold.requestCrew` | POST | Request crew membership (service token or DPoP) |
| `/xrpc/io.atcr.hold.exportUserData` | GET | GDPR data export (returns user's records) | | `/xrpc/io.atcr.hold.exportUserData` | GET | GDPR data export (returns user's records; service token only) |
| `/xrpc/io.atcr.hold.deleteUserData` | DELETE | GDPR data deletion (deletes crew, layer, and stats records; service token only) |
### Appview Token Required ### Appview Token Required
`/xrpc/io.atcr.hold.updateCrewTier` (POST) validates the caller inline via `ValidateAppviewToken`. Returns 503 if the appview DID is not configured on the hold, or 401 on token validation failure.
| Endpoint | Method | Description | | Endpoint | Method | Description |
|----------|--------|-------------| |----------|--------|-------------|
| `/xrpc/io.atcr.hold.updateCrewTier` | POST | Update a crew member's tier (appview-only) | | `/xrpc/io.atcr.hold.updateCrewTier` | POST | Update a crew member's tier (appview JWT, ES256) |
### Scanner WebSocket
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/xrpc/io.atcr.hold.subscribeScanJobs` | GET (WebSocket) | Scanner job subscription. Auth via `?secret=` query param or `X-Scanner-Secret` header (shared secret). Supports `?cursor=` for backfill. |
--- ---
@@ -60,8 +83,7 @@ All require `blob:write` permission via service token:
| Endpoint | Method | Description | | Endpoint | Method | Description |
|----------|--------|-------------| |----------|--------|-------------|
| `/xrpc/io.atcr.hold.initiateUpload` | POST | Start multipart upload | | `/xrpc/io.atcr.hold.initiateUpload` | POST | Start multipart upload |
| `/xrpc/io.atcr.hold.getPartUploadUrl` | POST | Get presigned URL for part | | `/xrpc/io.atcr.hold.getPartUploadUrl` | POST | Get presigned S3 URL for a part; the client PUTs the part bytes directly to S3 |
| `/xrpc/io.atcr.hold.uploadPart` | PUT | Direct buffered part upload |
| `/xrpc/io.atcr.hold.completeUpload` | POST | Finalize multipart upload | | `/xrpc/io.atcr.hold.completeUpload` | POST | Finalize multipart upload |
| `/xrpc/io.atcr.hold.abortUpload` | POST | Cancel multipart upload | | `/xrpc/io.atcr.hold.abortUpload` | POST | Cancel multipart upload |
| `/xrpc/io.atcr.hold.notifyManifest` | POST | Notify manifest push (creates layer records + optional Bluesky post) | | `/xrpc/io.atcr.hold.notifyManifest` | POST | Notify manifest push (creates layer records + optional Bluesky post) |
@@ -73,19 +95,20 @@ All require `blob:write` permission via service token:
| Endpoint | Method | Auth | Description | | Endpoint | Method | Auth | Description |
|----------|--------|------|-------------| |----------|--------|------|-------------|
| `/xrpc/io.atcr.hold.initiateUpload` | POST | blob:write | Start multipart upload | | `/xrpc/io.atcr.hold.initiateUpload` | POST | blob:write | Start multipart upload |
| `/xrpc/io.atcr.hold.getPartUploadUrl` | POST | blob:write | Get presigned URL for part | | `/xrpc/io.atcr.hold.getPartUploadUrl` | POST | blob:write | Get presigned S3 URL for a part; client PUTs bytes directly to S3 |
| `/xrpc/io.atcr.hold.uploadPart` | PUT | blob:write | Direct buffered part upload |
| `/xrpc/io.atcr.hold.completeUpload` | POST | blob:write | Finalize multipart upload | | `/xrpc/io.atcr.hold.completeUpload` | POST | blob:write | Finalize multipart upload |
| `/xrpc/io.atcr.hold.abortUpload` | POST | blob:write | Cancel multipart upload | | `/xrpc/io.atcr.hold.abortUpload` | POST | blob:write | Cancel multipart upload |
| `/xrpc/io.atcr.hold.notifyManifest` | POST | blob:write | Notify manifest push | | `/xrpc/io.atcr.hold.notifyManifest` | POST | blob:write | Notify manifest push/pull (creates layer records, increments stats, optional Bluesky post) |
| `/xrpc/io.atcr.hold.requestCrew` | POST | auth | Request crew membership | | `/xrpc/io.atcr.hold.requestCrew` | POST | service token or DPoP | Request crew membership |
| `/xrpc/io.atcr.hold.exportUserData` | GET | auth | GDPR data export | | `/xrpc/io.atcr.hold.exportUserData` | GET | service token | GDPR data export |
| `/xrpc/io.atcr.hold.deleteUserData` | DELETE | service token | GDPR data deletion (crew, layer, stats records) |
| `/xrpc/io.atcr.hold.getQuota` | GET | none | Get user quota info | | `/xrpc/io.atcr.hold.getQuota` | GET | none | Get user quota info |
| `/xrpc/io.atcr.hold.getLayersForManifest` | GET | none | Get layer records for a manifest AT-URI | | `/xrpc/io.atcr.hold.getLayersForManifest` | GET | none | Get layer records for a manifest AT-URI |
| `/xrpc/io.atcr.hold.image.getConfig` | GET | none | Get OCI image config record for a manifest digest | | `/xrpc/io.atcr.hold.image.getConfig` | GET | none | Get OCI image config record for a manifest digest |
| `/xrpc/io.atcr.hold.purgeManifest` | POST | owner/crew admin | Purge layer/scan/image-config records for a single manifest URI. Called by appview on UI delete; called internally on takedown receipt. Does not delete S3 blobs (GC handles those). | | `/xrpc/io.atcr.hold.purgeManifest` | POST | inline (service token or DPoP; captain, crew:admin, or manifest owner) | Purge layer/scan/image-config records for a single manifest URI. Called by appview on UI delete; called internally on takedown receipt. Does not delete S3 blobs (GC handles those). |
| `/xrpc/io.atcr.hold.listTiers` | GET | none | List hold's available tiers with quotas and features (scanOnPush) | | `/xrpc/io.atcr.hold.listTiers` | GET | none | List hold's available tiers with quotas and features (scanOnPush) |
| `/xrpc/io.atcr.hold.updateCrewTier` | POST | appview token | Update crew member's tier | | `/xrpc/io.atcr.hold.updateCrewTier` | POST | appview token (ES256 JWT; 503 if appview DID not configured) | Update crew member's tier |
| `/xrpc/io.atcr.hold.subscribeScanJobs` | GET (WebSocket) | shared secret (`?secret=` or `X-Scanner-Secret`) | Scanner job subscription; supports `?cursor=` for backfill |
--- ---
@@ -100,10 +123,12 @@ All require `blob:write` permission via service token:
| /xrpc/com.atproto.repo.listRecords | | /xrpc/com.atproto.repo.listRecords |
| /xrpc/com.atproto.repo.deleteRecord | | /xrpc/com.atproto.repo.deleteRecord |
| /xrpc/com.atproto.repo.uploadBlob | | /xrpc/com.atproto.repo.uploadBlob |
| /xrpc/com.atproto.sync.listBlobs |
| /xrpc/com.atproto.sync.listRepos | | /xrpc/com.atproto.sync.listRepos |
| /xrpc/com.atproto.sync.getRecord | | /xrpc/com.atproto.sync.getRecord |
| /xrpc/com.atproto.sync.getRepo | | /xrpc/com.atproto.sync.getRepo |
| /xrpc/com.atproto.sync.getRepoStatus | | /xrpc/com.atproto.sync.getRepoStatus |
| /xrpc/com.atproto.sync.getLatestCommit |
| /xrpc/com.atproto.sync.getBlob | | /xrpc/com.atproto.sync.getBlob |
| /xrpc/com.atproto.sync.subscribeRepos | | /xrpc/com.atproto.sync.subscribeRepos |
| /xrpc/com.atproto.identity.resolveHandle | | /xrpc/com.atproto.identity.resolveHandle |
docs/IMAGE_SIGNING.md
-505
View File
@@ -1,505 +0,0 @@
# Image Signing with ATProto
ATCR provides cryptographic verification of container images through ATProto's native signature system. Every manifest stored in a PDS is cryptographically signed, providing tamper-proof image verification.
## Overview
**Key Fact:** Every image pushed to ATCR is automatically signed via ATProto's repository commit signing. No additional signing tools or steps are required.
When you push an image:
1. Manifest stored in your PDS as an `io.atcr.manifest` record
2. PDS signs the repository commit containing the manifest (ECDSA K-256)
3. Signature is part of the ATProto repository chain
4. Verification proves the manifest came from your DID and hasn't been tampered with
**This document explains:**
- How ATProto signatures work for ATCR images
- How to verify signatures using standard and custom tools
- Integration options for different use cases
- When to use optional X.509 certificates (Hold-as-CA)
## ATProto Signature Model
### How It Works
ATProto uses a **repository commit signing** model similar to Git:
```
1. docker push atcr.io/alice/myapp:latest
↓
2. AppView stores manifest in alice's PDS as io.atcr.manifest record
↓
3. PDS creates repository commit containing the new record
↓
4. PDS signs commit with alice's private key (ECDSA K-256)
↓
5. Commit becomes part of alice's cryptographically signed repo chain
```
**What this proves:**
- ✅ Manifest came from alice's PDS (DID-based identity)
- ✅ Manifest content hasn't been tampered with
- ✅ Manifest was created at a specific time (commit timestamp)
- ✅ Manifest is part of alice's verifiable repository history
**Trust model:**
- Public keys distributed via DID documents (PLC directory, did:web)
- Signatures use ECDSA K-256 (secp256k1)
- Verification is decentralized (no central CA required)
- Users control their own DIDs and can rotate keys
### Signature Metadata
In addition to ATProto's native commit signatures, ATCR creates **ORAS signature artifacts** that bridge ATProto signatures to the OCI ecosystem:
```json
{
"$type": "io.atcr.atproto.signature",
"version": "1.0",
"subject": {
"digest": "sha256:abc123...",
"mediaType": "application/vnd.oci.image.manifest.v1+json"
},
"atproto": {
"did": "did:plc:alice123",
"handle": "alice.bsky.social",
"pdsEndpoint": "https://bsky.social",
"recordUri": "at://did:plc:alice123/io.atcr.manifest/abc123",
"commitCid": "bafyreih8...",
"signedAt": "2025-10-31T12:34:56.789Z"
},
"signature": {
"algorithm": "ECDSA-K256-SHA256",
"keyId": "did:plc:alice123#atproto",
"publicKeyMultibase": "zQ3shokFTS3brHcDQrn82RUDfCZESWL1ZdCEJwekUDdo1Ko4Z"
}
}
```
**Stored as:**
- OCI artifact with `artifactType: application/vnd.atproto.signature.v1+json`
- Linked to image manifest via OCI Referrers API
- Discoverable by standard OCI tools (ORAS, Cosign, Crane)
## Verification
### Quick Verification (Shell Script)
For manual verification, use the provided shell scripts:
```bash
# Verify an image
./examples/verification/atcr-verify.sh atcr.io/alice/myapp:latest
# Output shows:
# - DID and handle of signer
# - PDS endpoint
# - ATProto record URI
# - Signature verification status
```
**See:** [examples/verification/README.md](../examples/verification/README.md) for complete examples including:
- Standalone verification script
- Secure pull wrapper (verify before pull)
- Kubernetes webhook deployment
- CI/CD integration examples
### Standard Tools (Discovery Only)
Standard OCI tools can **discover** ATProto signature artifacts but cannot **verify** them (different signature format):
```bash
# Discover signatures with ORAS
oras discover atcr.io/alice/myapp:latest \
--artifact-type application/vnd.atproto.signature.v1+json
# Fetch signature metadata
oras pull atcr.io/alice/myapp@sha256:sig789...
# View with Cosign (discovery only)
cosign tree atcr.io/alice/myapp:latest
```
**Note:** Cosign/Notary cannot verify ATProto signatures directly because they use a different signature format and trust model. Use integration plugins or the `atcr-verify` CLI tool instead.
## Integration Options
ATCR supports multiple integration approaches depending on your use case:
### 1. **Plugins (Recommended for Kubernetes)** ⭐
Build plugins for existing policy/verification engines:
**Ratify Verifier Plugin:**
- Integrates with OPA Gatekeeper
- Verifies ATProto signatures using Ratify's plugin interface
- Policy-based enforcement for Kubernetes
**OPA Gatekeeper External Provider:**
- HTTP service that verifies ATProto signatures
- Rego policies call external provider
- Flexible and easy to deploy
**Containerd 2.0 Bindir Plugin:**
- Verifies signatures at containerd level
- Works with any CRI-compatible runtime
- No Kubernetes required
**See:** [docs/SIGNATURE_INTEGRATION.md](./SIGNATURE_INTEGRATION.md) for complete plugin implementation examples
### 2. **CLI Tool (atcr-verify)**
Standalone CLI tool for signature verification:
```bash
# Install
go install github.com/atcr-io/atcr/cmd/atcr-verify@latest
# Verify image
atcr-verify atcr.io/alice/myapp:latest --policy trust-policy.yaml
# Use in CI/CD
atcr-verify $IMAGE --quiet && kubectl apply -f deployment.yaml
```
**Features:**
- Trust policy management (which DIDs to trust)
- Multiple output formats (text, JSON, SARIF)
- Offline verification with cached DID documents
- Library usage for custom integrations
**See:** [docs/ATCR_VERIFY_CLI.md](./ATCR_VERIFY_CLI.md) for complete CLI specification
### 3. **External Services**
Deploy verification as a service:
**GitHub Actions:**
```yaml
- name: Verify image signature
uses: atcr-io/atcr-verify-action@v1
with:
image: atcr.io/alice/myapp:${{ github.sha }}
policy: .atcr/trust-policy.yaml
```
**GitLab CI, Jenkins, CircleCI:**
- Use `atcr-verify` CLI in pipeline
- Fail build if verification fails
- Enforce signature requirements before deployment
### 4. **X.509 Certificates (Hold-as-CA)** ⚠️
Optional approach where hold services issue X.509 certificates based on ATProto signatures:
**Use cases:**
- Enterprise environments requiring PKI compliance
- Tools that only support X.509 (legacy systems)
- Notation integration (P-256 certificates)
**Trade-offs:**
- ❌ Introduces centralization (hold acts as CA)
- ❌ Trust shifts from DIDs to hold operator
- ❌ Requires hold service infrastructure
**See:** [docs/HOLD_AS_CA.md](./HOLD_AS_CA.md) for complete architecture and security considerations
## Integration Strategy Decision Matrix
Choose the right integration approach:
| Use Case | Recommended Approach | Priority |
|----------|---------------------|----------|
| **Kubernetes admission control** | Ratify plugin or Gatekeeper provider | HIGH |
| **CI/CD verification** | atcr-verify CLI or GitHub Actions | HIGH |
| **Docker/containerd** | Containerd bindir plugin | MEDIUM |
| **Policy enforcement** | OPA Gatekeeper + external provider | HIGH |
| **Manual verification** | Shell scripts or atcr-verify CLI | LOW |
| **Enterprise PKI compliance** | Hold-as-CA (X.509 certificates) | OPTIONAL |
| **Legacy tool support** | Hold-as-CA or external bridge service | OPTIONAL |
**See:** [docs/INTEGRATION_STRATEGY.md](./INTEGRATION_STRATEGY.md) for complete integration planning guide including:
- Architecture layers and data flow
- Tool compatibility matrix (16+ tools)
- Implementation roadmap (4 phases)
- When to use each approach
## Trust Policies
Define which signatures you trust:
```yaml
# trust-policy.yaml
version: 1.0
trustedDIDs:
did:plc:alice123:
name: "Alice (DevOps Lead)"
validFrom: "2024-01-01T00:00:00Z"
expiresAt: null
did:plc:bob456:
name: "Bob (Security Team)"
validFrom: "2024-06-01T00:00:00Z"
expiresAt: "2025-12-31T23:59:59Z"
policies:
- name: production-images
scope: "atcr.io/*/prod-*"
require:
signature: true
trustedDIDs:
- did:plc:alice123
- did:plc:bob456
minSignatures: 1
action: enforce # reject if policy fails
- name: dev-images
scope: "atcr.io/*/dev-*"
require:
signature: false
action: audit # log but don't reject
```
**Use with:**
- `atcr-verify` CLI: `atcr-verify IMAGE --policy trust-policy.yaml`
- Kubernetes webhooks: ConfigMap with policy
- CI/CD pipelines: Fail build if policy not met
## Security Considerations
### What ATProto Signatures Prove
✅ **Identity:** Manifest signed by specific DID (e.g., `did:plc:alice123`)
✅ **Integrity:** Manifest content hasn't been tampered with
✅ **Timestamp:** When the manifest was signed
✅ **Authenticity:** Signature created with private key for that DID
### What They Don't Prove
❌ **Vulnerability-free:** Signature doesn't mean image is safe
❌ **Authorization:** DID ownership doesn't imply permission to deploy
❌ **Key security:** Private key could be compromised
❌ **PDS trustworthiness:** Malicious PDS could create fake records
### Trust Dependencies
When verifying signatures, you're trusting:
1. **DID resolution** (PLC directory, did:web) - public key is correct for DID
2. **PDS integrity** - PDS serves correct records and doesn't forge signatures
3. **Cryptographic primitives** - ECDSA K-256 remains secure
4. **Your trust policy** - DIDs you've chosen to trust are legitimate
### Best Practices
**1. Use Trust Policies**
Don't blindly trust all signatures - define which DIDs you trust:
```yaml
trustedDIDs:
- did:plc:your-org-team
- did:plc:your-ci-system
```
**2. Monitor Signature Coverage**
Track which images have signatures:
```bash
atcr-verify --check-coverage namespace/production
```
**3. Enforce in Production**
Use Kubernetes admission control to block unsigned images:
```yaml
# Ratify + Gatekeeper or custom webhook
enforceSignatures: true
failurePolicy: Fail
```
**4. Verify in CI/CD**
Never deploy unsigned images:
```yaml
# GitHub Actions
- name: Verify signature
run: atcr-verify $IMAGE || exit 1
```
**5. Plan for Compromised Keys**
- Rotate DID keys periodically
- Monitor DID documents for unexpected key changes
- Have incident response plan for key compromise
## Implementation Status
### ✅ Available Now
- **ATProto signatures**: All manifests automatically signed by PDS
- **ORAS artifacts**: Signature metadata stored as OCI artifacts
- **OCI Referrers API**: Discovery via standard OCI endpoints
- **Shell scripts**: Manual verification examples
- **Documentation**: Complete integration guides
### 🔄 In Development
- **atcr-verify CLI**: Standalone verification tool
- **Ratify plugin**: Kubernetes integration
- **Gatekeeper provider**: OPA policy enforcement
- **GitHub Actions**: CI/CD integration
### 📋 Planned
- **Containerd plugin**: Runtime-level verification
- **Hold-as-CA**: X.509 certificate generation (optional)
- **Web UI**: Signature viewer in AppView
- **Offline bundles**: Air-gapped verification
## Comparison with Other Signing Solutions
| Feature | ATCR (ATProto) | Cosign (Sigstore) | Notation (Notary v2) |
|---------|---------------|-------------------|---------------------|
| **Signing** | Automatic (PDS) | Manual or keyless | Manual |
| **Keys** | K-256 (secp256k1) | P-256 or RSA | P-256, P-384, P-521 |
| **Trust** | DID-based | OIDC + Fulcio CA | X.509 PKI |
| **Storage** | ATProto PDS | OCI registry | OCI registry |
| **Centralization** | Decentralized | Centralized (Fulcio) | Configurable |
| **Transparency Log** | ATProto firehose | Rekor | Configurable |
| **Verification** | Custom tools/plugins | Cosign CLI | Notation CLI |
| **Kubernetes** | Plugins (Ratify) | Policy Controller | Policy Controller |
**ATCR advantages:**
- ✅ Decentralized trust (no CA required)
- ✅ Automatic signing (no extra tools)
- ✅ DID-based identity (portable, self-sovereign)
- ✅ Transparent via ATProto firehose
**ATCR trade-offs:**
- ⚠️ Requires custom verification tools/plugins
- ⚠️ K-256 not supported by Notation (needs Hold-as-CA)
- ⚠️ Smaller ecosystem than Cosign/Notation
## Why Not Use Cosign Directly?
**Question:** Why not just integrate with Cosign's keyless signing (OIDC + Fulcio)?
**Answer:** ATProto and Cosign use incompatible authentication models:
| Requirement | Cosign Keyless | ATProto |
|-------------|---------------|---------|
| **Identity protocol** | OIDC | ATProto OAuth + DPoP |
| **Token format** | JWT from OIDC provider | DPoP-bound access token |
| **CA** | Fulcio (Sigstore CA) | None (DID-based PKI) |
| **Infrastructure** | Fulcio + Rekor + TUF | PDS + DID resolver |
**To make Cosign work, we'd need to:**
1. Deploy Fulcio (certificate authority)
2. Deploy Rekor (transparency log)
3. Deploy TUF (metadata distribution)
4. Build OIDC provider bridge for ATProto OAuth
5. Maintain all this infrastructure
**Instead:** We leverage ATProto's existing signatures and build lightweight plugins/tools for verification. This is simpler, more decentralized, and aligns with ATCR's design philosophy.
**For tools that need X.509 certificates:** See [Hold-as-CA](./HOLD_AS_CA.md) for an optional centralized approach.
## Getting Started
### Verify Your First Image
```bash
# 1. Check if image has ATProto signature
oras discover atcr.io/alice/myapp:latest \
--artifact-type application/vnd.atproto.signature.v1+json
# 2. Pull signature metadata
oras pull atcr.io/alice/myapp@sha256:sig789...
# 3. Verify with shell script
./examples/verification/atcr-verify.sh atcr.io/alice/myapp:latest
# 4. Use atcr-verify CLI (when available)
atcr-verify atcr.io/alice/myapp:latest --policy trust-policy.yaml
```
### Deploy Kubernetes Verification
```bash
# 1. Choose an approach
# Option A: Ratify plugin (recommended)
# Option B: Gatekeeper external provider
# Option C: Custom admission webhook
# 2. Follow integration guide
# See docs/SIGNATURE_INTEGRATION.md for step-by-step
# 3. Enable for namespace
kubectl label namespace production atcr-verify=enabled
# 4. Test with sample pod
kubectl run test --image=atcr.io/alice/myapp:latest -n production
```
### Integrate with CI/CD
```bash
# GitHub Actions
- name: Verify signature
run: |
curl -LO https://github.com/atcr-io/atcr/releases/latest/download/atcr-verify
chmod +x atcr-verify
./atcr-verify ${{ env.IMAGE }} --policy .atcr/trust-policy.yaml
# GitLab CI
verify_image:
script:
- wget https://github.com/atcr-io/atcr/releases/latest/download/atcr-verify
- chmod +x atcr-verify
- ./atcr-verify $IMAGE --policy .atcr/trust-policy.yaml
```
## Documentation
### Core Documentation
- **[ATProto Signatures](./ATPROTO_SIGNATURES.md)** - Technical deep-dive into signature format and verification
- **[Signature Integration](./SIGNATURE_INTEGRATION.md)** - Tool-specific integration guides (Ratify, Gatekeeper, Containerd)
- **[Integration Strategy](./INTEGRATION_STRATEGY.md)** - High-level overview and decision matrix
- **[atcr-verify CLI](./ATCR_VERIFY_CLI.md)** - CLI tool specification and usage
- **[Hold-as-CA](./HOLD_AS_CA.md)** - Optional X.509 certificate approach
### Examples
- **[examples/verification/](../examples/verification/)** - Shell scripts, Kubernetes configs, trust policies
- **[examples/plugins/](../examples/plugins/)** - Plugin skeletons for Ratify, Gatekeeper, Containerd
### External References
- **ATProto:** https://atproto.com/specs/repository (repository commit signing)
- **ORAS:** https://oras.land/ (artifact registry)
- **OCI Referrers API:** https://github.com/opencontainers/distribution-spec/blob/main/spec.md#listing-referrers
- **Ratify:** https://ratify.dev/ (verification framework)
- **OPA Gatekeeper:** https://open-policy-agent.github.io/gatekeeper/
## Support
For questions or issues:
- GitHub Issues: https://github.com/atcr-io/atcr/issues
- Documentation: https://docs.atcr.io
- Security: security@atcr.io
## Summary
**Key Points:**
1. **Automatic signing**: Every ATCR image is automatically signed via ATProto's native signature system
2. **No additional tools**: Signing happens transparently when you push images
3. **Decentralized trust**: DID-based signatures, no central CA required
4. **Standard discovery**: ORAS artifacts and OCI Referrers API for signature metadata
5. **Custom verification**: Use plugins, CLI tools, or shell scripts (not Cosign directly)
6. **Multiple integrations**: Kubernetes (Ratify, Gatekeeper), CI/CD (atcr-verify), containerd
7. **Optional X.509**: Hold-as-CA for enterprise PKI compliance (centralized)
**Next Steps:**
1. Read [examples/verification/README.md](../examples/verification/README.md) for practical examples
2. Choose integration approach from [INTEGRATION_STRATEGY.md](./INTEGRATION_STRATEGY.md)
3. Implement plugin or deploy CLI tool from [SIGNATURE_INTEGRATION.md](./SIGNATURE_INTEGRATION.md)
4. Define trust policy for your organization
5. Deploy to test environment first, then production
docs/INTEGRATION_STRATEGY.md
-692
View File
@@ -1,692 +0,0 @@
# ATCR Signature Verification Integration Strategy
## Overview
This document provides a comprehensive overview of how to integrate ATProto signature verification into various tools and workflows. ATCR uses a layered approach that provides maximum compatibility while maintaining ATProto's decentralized philosophy.
## Architecture Layers
```
┌─────────────────────────────────────────────────────────┐
│ Layer 4: Applications & Workflows │
│ - CI/CD pipelines │
│ - Kubernetes admission control │
│ - Runtime verification │
│ - Security scanning │
└──────────────────────┬──────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────┐
│ Layer 3: Integration Methods │
│ - Plugins (Ratify, Gatekeeper, Containerd) │
│ - CLI tools (atcr-verify) │
│ - External services (webhooks, APIs) │
│ - (Optional) X.509 certificates (hold-as-CA) │
└──────────────────────┬──────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────┐
│ Layer 2: Signature Discovery │
│ - OCI Referrers API (GET /v2/.../referrers/...) │
│ - ORAS artifact format │
│ - artifactType: application/vnd.atproto.signature... │
└──────────────────────┬──────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────┐
│ Layer 1: ATProto Signatures (Foundation) │
│ - Manifests signed by PDS (K-256) │
│ - Signatures in ATProto repository commits │
│ - Public keys in DID documents │
│ - DID-based identity │
└─────────────────────────────────────────────────────────┘
```
## Integration Approaches
### Approach 1: Plugin-Based (RECOMMENDED) ⭐
**Best for:** Kubernetes, standard tooling, production deployments
Integrate through plugin systems of existing tools:
#### Ratify Verifier Plugin
- **Use case:** Kubernetes admission control via Gatekeeper
- **Effort:** 2-3 weeks to build
- **Maturity:** CNCF Sandbox project, growing adoption
- **Benefits:**
- ✅ Standard plugin interface
- ✅ Works with existing Ratify deployments
- ✅ Policy-based enforcement
- ✅ Multi-verifier support (can combine with Notation, Cosign)
**Implementation:**
```go
// Ratify plugin interface
type ReferenceVerifier interface {
VerifyReference(
ctx context.Context,
subjectRef common.Reference,
referenceDesc ocispecs.ReferenceDescriptor,
store referrerStore.ReferrerStore,
) (VerifierResult, error)
}
```
**Deployment:**
```yaml
apiVersion: config.ratify.deislabs.io/v1beta1
kind: Verifier
metadata:
name: atcr-verifier
spec:
name: atproto
artifactType: application/vnd.atproto.signature.v1+json
parameters:
trustedDIDs:
- did:plc:alice123
```
See [Ratify Integration Guide](./SIGNATURE_INTEGRATION.md#ratify-plugin)
---
#### OPA Gatekeeper External Provider
- **Use case:** Kubernetes admission control with OPA policies
- **Effort:** 2-3 weeks to build
- **Maturity:** Very stable, widely adopted
- **Benefits:**
- ✅ Rego-based policies (flexible)
- ✅ External data provider API (standard)
- ✅ Can reuse existing Gatekeeper deployments
**Implementation:**
```go
// External data provider
type Provider struct {
verifier *atproto.Verifier
}
func (p *Provider) Provide(ctx context.Context, req ProviderRequest) (*ProviderResponse, error) {
image := req.Keys["image"]
result, err := p.verifier.Verify(ctx, image)
return &ProviderResponse{
Data: map[string]bool{"verified": result.Verified},
}, nil
}
```
**Policy:**
```rego
package verify
violation[{"msg": msg}] {
container := input.review.object.spec.containers[_]
startswith(container.image, "atcr.io/")
response := external_data({
"provider": "atcr-verifier",
"keys": ["image"],
"values": [container.image]
})
response.verified != true
msg := sprintf("Image %v has no valid ATProto signature", [container.image])
}
```
See [Gatekeeper Integration Guide](./SIGNATURE_INTEGRATION.md#opa-gatekeeper-external-provider)
---
#### Containerd 2.0 Image Verifier Plugin
- **Use case:** Runtime verification at image pull time
- **Effort:** 1-2 weeks to build
- **Maturity:** New in Containerd 2.0 (Nov 2024)
- **Benefits:**
- ✅ Runtime enforcement (pull-time verification)
- ✅ Works for Docker, nerdctl, ctr
- ✅ Transparent to users
- ✅ No Kubernetes required
**Limitation:** CRI plugin integration still maturing
**Implementation:**
```bash
#!/bin/bash
# /usr/local/bin/containerd-verifiers/atcr-verifier
# Binary called by containerd on image pull
# Containerd passes image info via stdin
read -r INPUT
IMAGE=$(echo "$INPUT" | jq -r '.reference')
DIGEST=$(echo "$INPUT" | jq -r '.descriptor.digest')
# Verify signature
if atcr-verify "$IMAGE@$DIGEST" --quiet; then
exit 0 # Verified
else
exit 1 # Failed
fi
```
**Configuration:**
```toml
# /etc/containerd/config.toml
[plugins."io.containerd.image-verifier.v1.bindir"]
bin_dir = "/usr/local/bin/containerd-verifiers"
max_verifiers = 5
per_verifier_timeout = "10s"
```
See [Containerd Integration Guide](./SIGNATURE_INTEGRATION.md#containerd-20)
---
### Approach 2: CLI Tool (RECOMMENDED) ⭐
**Best for:** CI/CD, scripts, general-purpose verification
Use `atcr-verify` CLI tool directly in workflows:
#### Command-Line Verification
```bash
# Basic verification
atcr-verify atcr.io/alice/myapp:latest
# With trust policy
atcr-verify atcr.io/alice/myapp:latest --policy trust-policy.yaml
# JSON output for scripting
atcr-verify atcr.io/alice/myapp:latest --output json
# Quiet mode for exit codes
atcr-verify atcr.io/alice/myapp:latest --quiet && echo "Verified"
```
#### CI/CD Integration
**GitHub Actions:**
```yaml
- name: Verify image
run: atcr-verify ${{ env.IMAGE }} --policy .github/trust-policy.yaml
```
**GitLab CI:**
```yaml
verify:
image: atcr.io/atcr/verify:latest
script:
- atcr-verify ${IMAGE} --policy trust-policy.yaml
```
**Universal Container:**
```bash
docker run --rm atcr.io/atcr/verify:latest verify IMAGE
```
**Benefits:**
- ✅ Works everywhere (not just Kubernetes)
- ✅ Simple integration (single binary)
- ✅ No plugin installation required
- ✅ Offline mode support
See [atcr-verify CLI Documentation](./ATCR_VERIFY_CLI.md)
---
### Approach 3: External Services
**Best for:** Custom admission controllers, API-based verification
Build verification as a service that tools can call:
#### Webhook Service
```go
// HTTP endpoint for verification
func (h *Handler) VerifyImage(w http.ResponseWriter, r *http.Request) {
image := r.URL.Query().Get("image")
result, err := h.verifier.Verify(r.Context(), image)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
json.NewEncoder(w).Encode(map[string]any{
"verified": result.Verified,
"did": result.Signature.DID,
"signedAt": result.Signature.SignedAt,
})
}
```
#### Usage from Kyverno
```yaml
verifyImages:
- imageReferences:
- "atcr.io/*/*"
attestors:
- entries:
- api:
url: http://atcr-verify.kube-system/verify?image={{ image }}
```
**Benefits:**
- ✅ Flexible integration
- ✅ Centralized verification logic
- ✅ Caching and rate limiting
- ✅ Can add additional checks (vulnerability scanning, etc.)
---
### Approach 4: Hold-as-CA (OPTIONAL, ENTERPRISE ONLY)
**Best for:** Enterprise X.509 PKI compliance requirements
⚠️ **WARNING:** This approach introduces centralization trade-offs. Only use if you have specific X.509 compliance requirements.
Hold services act as Certificate Authorities that issue X.509 certificates for users, enabling standard Notation verification.
**When to use:**
- Enterprise requires standard X.509 PKI
- Cannot deploy custom plugins
- Accept centralization trade-off for tool compatibility
**When NOT to use:**
- Default deployments (use plugins instead)
- Maximum decentralization required
- Don't need X.509 compliance
See [Hold-as-CA Architecture](./HOLD_AS_CA.md) for complete details and security implications.
---
## Tool Compatibility Matrix
| Tool | Discover | Verify | Integration Method | Priority | Effort |
|------|----------|--------|-------------------|----------|--------|
| **Kubernetes** | | | | | |
| OPA Gatekeeper | ✅ | ✅ | External provider | **HIGH** | 2-3 weeks |
| Ratify | ✅ | ✅ | Verifier plugin | **HIGH** | 2-3 weeks |
| Kyverno | ✅ | ⚠️ | External service | MEDIUM | 2 weeks |
| Portieris | ❌ | ❌ | N/A (deprecated) | NONE | - |
| **Runtime** | | | | | |
| Containerd 2.0 | ✅ | ✅ | Bindir plugin | **MED-HIGH** | 1-2 weeks |
| CRI-O | ⚠️ | ⚠️ | Upstream contribution | MEDIUM | 3-4 weeks |
| Podman | ⚠️ | ⚠️ | Upstream contribution | MEDIUM | 3-4 weeks |
| **CI/CD** | | | | | |
| GitHub Actions | ✅ | ✅ | Custom action | **HIGH** | 1 week |
| GitLab CI | ✅ | ✅ | Container image | **HIGH** | 1 week |
| Jenkins/CircleCI | ✅ | ✅ | Container image | HIGH | 1 week |
| **Scanners** | | | | | |
| Trivy | ✅ | ❌ | N/A (not verifier) | NONE | - |
| Snyk | ❌ | ❌ | N/A (not verifier) | NONE | - |
| Anchore | ❌ | ❌ | N/A (not verifier) | NONE | - |
| **Registries** | | | | | |
| Harbor | ✅ | ⚠️ | UI integration | LOW | - |
| **OCI Tools** | | | | | |
| ORAS CLI | ✅ | ❌ | Already works | Document | - |
| Notation | ⚠️ | ⚠️ | Hold-as-CA | OPTIONAL | 3-4 weeks |
| Cosign | ❌ | ❌ | Not compatible | NONE | - |
| Crane | ✅ | ❌ | Already works | Document | - |
| Skopeo | ⚠️ | ⚠️ | Upstream contribution | LOW | 3-4 weeks |
**Legend:**
- ✅ Works / Feasible
- ⚠️ Partial / Requires changes
- ❌ Not applicable / Not feasible
---
## Implementation Roadmap
### Phase 1: Foundation (4-5 weeks) ⭐
**Goal:** Core verification capability
1. **atcr-verify CLI tool** (Week 1-2)
- ATProto signature verification
- Trust policy support
- Multiple output formats
- Offline mode
2. **OCI Referrers API** (Week 2-3)
- AppView endpoint implementation
- ORAS artifact serving
- Integration with existing SBOM pattern
3. **CI/CD Container Image** (Week 3)
- Universal verification image
- Documentation for GitHub Actions, GitLab CI
- Example workflows
4. **Documentation** (Week 4-5)
- Integration guides
- Trust policy examples
- Troubleshooting guides
**Deliverables:**
- `atcr-verify` binary (Linux, macOS, Windows)
- `atcr.io/atcr/verify:latest` container image
- OCI Referrers API implementation
- Complete documentation
---
### Phase 2: Kubernetes Integration (3-4 weeks)
**Goal:** Production-ready Kubernetes admission control
5. **OPA Gatekeeper Provider** (Week 1-2)
- External data provider service
- Helm chart for deployment
- Example policies
6. **Ratify Plugin** (Week 2-3)
- Verifier plugin implementation
- Testing with Ratify
- Documentation
7. **Kubernetes Examples** (Week 4)
- Deployment manifests
- Policy examples
- Integration testing
**Deliverables:**
- `atcr-gatekeeper-provider` service
- Ratify plugin binary
- Kubernetes deployment examples
- Production deployment guide
---
### Phase 3: Runtime Verification (2-3 weeks)
**Goal:** Pull-time verification
8. **Containerd Plugin** (Week 1-2)
- Bindir verifier implementation
- Configuration documentation
- Testing with Docker, nerdctl
9. **CRI-O/Podman Integration** (Week 3, optional)
- Upstream contribution (if accepted)
- Policy.json extension
- Documentation
**Deliverables:**
- Containerd verifier binary
- Configuration guides
- Runtime verification examples
---
### Phase 4: Optional Features (2-3 weeks)
**Goal:** Enterprise features (if demanded)
10. **Hold-as-CA** (Week 1-2, optional)
- Certificate generation
- Notation signature creation
- Trust store distribution
- **Only if enterprise customers request**
11. **Advanced Features** (Week 3, as needed)
- Signature transparency log
- Multi-signature support
- Hardware token integration
**Deliverables:**
- Hold co-signing implementation (if needed)
- Advanced feature documentation
---
## Decision Matrix
### Which Integration Approach Should I Use?
```
┌─────────────────────────────────────────────────┐
│ Are you using Kubernetes? │
└───────────────┬─────────────────────────────────┘
│
┌────────┴────────┐
│ │
YES NO
│ │
↓ ↓
┌──────────────┐ ┌──────────────┐
│ Using │ │ CI/CD │
│ Gatekeeper? │ │ Pipeline? │
└──────┬───────┘ └──────┬───────┘
│ │
┌────┴────┐ ┌────┴────┐
YES NO YES NO
│ │ │ │
↓ ↓ ↓ ↓
External Ratify GitHub Universal
Provider Plugin Action CLI Tool
```
#### Use OPA Gatekeeper Provider if:
- ✅ Already using Gatekeeper
- ✅ Want Rego-based policies
- ✅ Need flexible policy logic
#### Use Ratify Plugin if:
- ✅ Using Ratify (or planning to)
- ✅ Want standard plugin interface
- ✅ Need multi-verifier support (Notation + Cosign + ATProto)
#### Use atcr-verify CLI if:
- ✅ CI/CD pipelines
- ✅ Local development
- ✅ Non-Kubernetes environments
- ✅ Want simple integration
#### Use Containerd Plugin if:
- ✅ Need runtime enforcement
- ✅ Want pull-time verification
- ✅ Using Containerd 2.0+
#### Use Hold-as-CA if:
- ⚠️ Enterprise X.509 PKI compliance required
- ⚠️ Cannot deploy plugins
- ⚠️ Accept centralization trade-off
---
## Best Practices
### 1. Start Simple
Begin with CLI tool integration in CI/CD:
```bash
# Add to .github/workflows/deploy.yml
- run: atcr-verify $IMAGE --policy .github/trust-policy.yaml
```
### 2. Define Trust Policies
Create trust policies early:
```yaml
# trust-policy.yaml
policies:
- name: production
scope: "atcr.io/*/prod-*"
require:
signature: true
trustedDIDs: [did:plc:devops-team]
action: enforce
```
### 3. Progressive Rollout
1. **Week 1:** Add verification to CI/CD (audit mode)
2. **Week 2:** Enforce in CI/CD
3. **Week 3:** Add Kubernetes admission control (audit mode)
4. **Week 4:** Enforce in Kubernetes
### 4. Monitor and Alert
Track verification metrics:
- Verification success/failure rates
- Policy violations
- Signature coverage (% of images signed)
### 5. Plan for Key Rotation
- Document DID key rotation procedures
- Test key rotation in non-production
- Monitor for unexpected key changes
---
## Common Patterns
### Pattern 1: Multi-Layer Defense
```
1. CI/CD verification (atcr-verify)
↓ (blocks unsigned images from being pushed)
2. Kubernetes admission (Gatekeeper/Ratify)
↓ (blocks unsigned images from running)
3. Runtime verification (Containerd plugin)
↓ (blocks unsigned images from being pulled)
```
### Pattern 2: Trust Policy Inheritance
```yaml
# Global policy
trustedDIDs:
- did:plc:security-team # Always trusted
# Environment-specific policies
staging:
trustedDIDs:
- did:plc:developers # Additional trust for staging
production:
trustedDIDs: [] # Only global trust (security-team)
```
### Pattern 3: Offline Verification
```bash
# Build environment (online)
atcr-verify export $IMAGE -o bundle.json
# Air-gapped environment (offline)
atcr-verify $IMAGE --offline --bundle bundle.json
```
---
## Migration Guide
### From Docker Content Trust (DCT)
DCT is deprecated. Migrate to ATCR signatures:
**Old (DCT):**
```bash
export DOCKER_CONTENT_TRUST=1
docker push myimage:latest
```
**New (ATCR):**
```bash
# Signatures created automatically on push
docker push atcr.io/myorg/myimage:latest
# Verify in CI/CD
atcr-verify atcr.io/myorg/myimage:latest
```
### From Cosign
Cosign and ATCR signatures can coexist:
**Dual signing:**
```bash
# Push to ATCR (ATProto signature automatic)
docker push atcr.io/myorg/myimage:latest
# Also sign with Cosign (if needed)
cosign sign atcr.io/myorg/myimage:latest
```
**Verification:**
```bash
# Verify ATProto signature
atcr-verify atcr.io/myorg/myimage:latest
# Or verify Cosign signature
cosign verify atcr.io/myorg/myimage:latest --key cosign.pub
```
---
## Troubleshooting
### Signatures Not Found
**Symptom:** `atcr-verify` reports "no signature found"
**Diagnosis:**
```bash
# Check if Referrers API works
curl "https://atcr.io/v2/OWNER/REPO/referrers/DIGEST"
# Check if signature artifact exists
oras discover atcr.io/OWNER/REPO:TAG
```
**Solutions:**
1. Verify Referrers API is implemented
2. Re-push image to generate signature
3. Check AppView logs for signature creation errors
### DID Resolution Fails
**Symptom:** Cannot resolve DID to public key
**Diagnosis:**
```bash
# Test DID resolution
curl https://plc.directory/did:plc:XXXXXX
# Check DID document has verificationMethod
curl https://plc.directory/did:plc:XXXXXX | jq .verificationMethod
```
**Solutions:**
1. Check internet connectivity
2. Verify DID is valid
3. Ensure DID document contains public key
### Policy Violations
**Symptom:** Verification fails with "trust policy violation"
**Diagnosis:**
```bash
# Verify with verbose output
atcr-verify IMAGE --policy policy.yaml --verbose
```
**Solutions:**
1. Add DID to trustedDIDs list
2. Check signature age vs. maxAge
3. Verify policy scope matches image
---
## See Also
- [ATProto Signatures](./ATPROTO_SIGNATURES.md) - Technical foundation
- [atcr-verify CLI](./ATCR_VERIFY_CLI.md) - CLI tool documentation
- [Signature Integration](./SIGNATURE_INTEGRATION.md) - Tool-specific guides
- [Hold-as-CA](./HOLD_AS_CA.md) - X.509 certificate approach (optional)
- [Examples](../examples/verification/) - Working code examples
docs/KNOWN_RELAYS.md
+7 -2
View File
@@ -27,16 +27,21 @@ Last verified: 2026-02-08
| Microcosm France | `https://relay3.fr.hose.cam` | Yes | No (404) | | | Microcosm France | `https://relay3.fr.hose.cam` | Yes | No (404) | |
| Upcloud | `https://relay.upcloud.world` | Yes | No (404) | | | Upcloud | `https://relay.upcloud.world` | Yes | No (404) | |
| Blacksky | `https://atproto.africa` | Down (502) | Down (502) | Was offline as of 2026-02-08 | | Blacksky | `https://atproto.africa` | Down (502) | Down (502) | Was offline as of 2026-02-08 |
| Hayes | `https://relay.hayescmd.net` | | | |
| Xero | `https://relay.xero.systems` | | | |
| Feeds Blue | `https://relay.feeds.blue` | | | |
| Waow | `https://relay.waow.tech` | | | |
| Bassh | `https://relay.bas.sh` | | | |
## ATCR Usage ## ATCR Usage
### Hold service (`requestCrawl`) ### Hold service (`requestCrawl`)
The hold announces its embedded PDS to relays on startup via `com.atproto.sync.requestCrawl`. Currently configured as a single relay in `server.relay_endpoint`. All healthy relays above accept `requestCrawl`. The hold announces its embedded PDS to relays on startup via `com.atproto.sync.requestCrawl`. On startup, `requestCrawls()` fans out to every relay in `KnownRelays` (all 15 entries hardcoded in `pkg/atproto/relays.go`) plus any additional entries in `server.relay_endpoints` (a list; defaults to `relay1.us-east.bsky.network` and `relay1.us-west.bsky.network`). Per-relay failures are logged but never block startup. All healthy relays above accept `requestCrawl`.
### Appview backfill (`listReposByCollection`) ### Appview backfill (`listReposByCollection`)
The appview uses `com.atproto.sync.listReposByCollection` to discover DIDs with `io.atcr.*` records during backfill. Only Bluesky's regional relays support this endpoint. The appview defaults to `relay1.us-east.bsky.network`. The appview uses `com.atproto.sync.listReposByCollection` to discover DIDs with `io.atcr.*` records during backfill. Only Bluesky's regional relays support this endpoint. The appview's `jetstream.relay_endpoints` defaults to both `relay1.us-east.bsky.network` and `relay1.us-west.bsky.network` with failover between them.
## Why most relays lack `listReposByCollection` ## Why most relays lack `listReposByCollection`
docs/OAUTH.md
+62 -57
View File
@@ -31,7 +31,7 @@ ATCR supports two OAuth client types depending on the deployment environment:
**Example:** **Example:**
```go ```go
// Automatically uses public client for localhost // Automatically uses public client for localhost
config := oauth.NewClientConfigWithScopes("http://127.0.0.1:5000", scopes) clientApp, err := oauth.NewClientApp("http://127.0.0.1:5000", store, scopes, keyPath, clientName)
``` ```
### Confidential Clients (Production) ### Confidential Clients (Production)
@@ -39,25 +39,22 @@ config := oauth.NewClientConfigWithScopes("http://127.0.0.1:5000", scopes)
**When:** `baseURL` is a public domain (not localhost) **When:** `baseURL` is a public domain (not localhost)
**Configuration:** **Configuration:**
- Client ID: `{baseURL}/client-metadata.json` (metadata endpoint) - Client ID: `{baseURL}/oauth-client-metadata.json` (metadata endpoint)
- Client authentication: P-256 (ES256) private key JWT assertion - Client authentication: P-256 (ES256) private key JWT assertion
- Private key stored at `/var/lib/atcr/oauth/client.key` - Private key loaded from the AppView SQLite database (`crypto_keys` table, key name `oauth_p256`)
- Auto-generated on first run with 0600 permissions - Auto-generated and stored in the database on first run
- Upgraded via `config.SetClientSecret(privateKey, keyID)` - Configured internally via indigo's `config.SetClientSecret(privateKey, keyID)`
**Example:** **Example:**
```go ```go
// 1. Create base config (public) // Production AppView loads the P-256 key from the database and passes it in:
config := oauth.NewClientConfigWithScopes("https://atcr.io", scopes) oauthKey, err := loadOAuthKey(database) // reads crypto_keys "oauth_p256", generates if absent
// 2. Load or generate P-256 key clientApp, err := oauth.NewClientAppWithKey(
privateKey, err := oauth.GenerateOrLoadClientKey("/var/lib/atcr/oauth/client.key") "https://atcr.io", store, scopes, oauthKey, clientName,
)
// 3. Generate key ID // NewClientAppWithKey derives the key ID and upgrades to a confidential client
keyID, err := oauth.GenerateKeyID(privateKey) // internally; localhost base URLs still produce a public client.
// 4. Upgrade to confidential
err = config.SetClientSecret(privateKey, keyID)
``` ```
## Key Management ## Key Management
@@ -72,26 +69,23 @@ ATCR uses **P-256 (NIST P-256, ES256)** keys for OAuth client authentication. Th
- Compatible with indigo's `SetClientSecret()` API - Compatible with indigo's `SetClientSecret()` API
**Key Storage:** **Key Storage:**
- Default path: `/var/lib/atcr/oauth/client.key` - Stored in the AppView SQLite database, `crypto_keys` table, under key name `oauth_p256`
- Configurable via: `ATCR_OAUTH_KEY_PATH` environment variable - Format: raw P-256 private key bytes (`atcrypto.PrivateKeyP256.Bytes()`)
- File permissions: `0600` (owner read/write only) - Loaded by `loadOAuthKey()` in `pkg/appview/crypto_keys.go`, wired up in `pkg/appview/server.go` via `oauth.NewClientAppWithKey()`
- Directory permissions: `0700` (owner access only) - There is no `ATCR_OAUTH_KEY_PATH` and no on-disk key file in production. (The on-disk `GenerateOrLoadClientKey()` path in `pkg/auth/oauth/keys.go` exists in the library but is not used by the production AppView; it is reached only through the `NewClientApp()` helper, which AppView does not call.)
- Format: Raw binary bytes (not PEM)
**Key Lifecycle:** **Key Lifecycle:**
1. On first production startup, AppView checks for key at configured path 1. On startup, AppView calls `loadOAuthKey()` to read the `oauth_p256` row from the database
2. If missing, generates new P-256 key using `atcrypto.GeneratePrivateKeyP256()` 2. If present, parses it with `atcrypto.ParsePrivateBytesP256()` and logs `"Loaded OAuth P-256 key from database"`
3. Saves raw key bytes to disk with restrictive permissions 3. If absent, generates a new P-256 key with `atcrypto.GeneratePrivateKeyP256()`, stores it in `crypto_keys`, and logs `"Generated new OAuth P-256 key and stored in database"`
4. Logs generation event: `"Generated new P-256 OAuth client key"` 4. The key is held in memory for the lifetime of the process
5. On subsequent startups, loads existing key
6. Logs load event: `"Loaded existing P-256 OAuth client key"`
**Key Rotation:** **Key Rotation:**
To rotate the OAuth client key: To rotate the OAuth client key:
1. Stop the AppView service 1. Stop the AppView service
2. Delete or rename the existing key file 2. Delete the `oauth_p256` row from the `crypto_keys` table
3. Restart AppView (new key will be generated automatically) 3. Restart AppView (a new key will be generated and stored automatically)
4. Note: Active OAuth sessions may need re-authentication 4. Note: the client metadata JWKS changes with the key, so active OAuth sessions may need re-authentication
### Key ID Generation ### Key ID Generation
@@ -133,6 +127,13 @@ sequenceDiagram
AppView->>User: Issue registry JWT AppView->>User: Issue registry JWT
``` ```
> Note: the Docker credential helper does not redirect directly into the OAuth
> flow shown above. It uses a device authorization flow (`POST /auth/device/code`,
> `POST /auth/device/token` — see `pkg/credhelper/device_auth.go` and the routes
> in `pkg/appview/routes/routes.go`). The browser-based OAuth exchange depicted
> here happens on AppView during device approval, after which the helper polls
> the device-token endpoint for the resulting registry JWT.
### Key Steps ### Key Steps
1. **Identity Resolution** 1. **Identity Resolution**
@@ -165,7 +166,8 @@ sequenceDiagram
7. **Registry JWT Issuance** 7. **Registry JWT Issuance**
- AppView validates OAuth session - AppView validates OAuth session
- Issues short-lived registry JWT (15 minutes) - Issues short-lived registry JWT (5 minutes, not configurable)
- The JWT's `exp` is further bound to the service-auth expiry the PDS grants at `/auth/token` time, so the registry JWT and service token invalidate concurrently
- JWT contains validated DID from PDS session - JWT contains validated DID from PDS session
## DPoP Implementation ## DPoP Implementation
@@ -226,11 +228,10 @@ Indigo manages:
### Environment Variables ### Environment Variables
**ATCR_OAUTH_KEY_PATH** > The OAuth client P-256 signing key is **not** configured via an environment
- Path to OAuth client P-256 signing key > variable. It lives in the AppView SQLite database (`crypto_keys` table, key
- Default: `/var/lib/atcr/oauth/client.key` > name `oauth_p256`) and is auto-generated on first run. See
- Auto-generated on first run (production only) > [Key Management](#key-management).
- Format: Raw binary P-256 private key
**ATCR_BASE_URL** **ATCR_BASE_URL**
- Public URL of AppView service - Public URL of AppView service
@@ -244,11 +245,11 @@ Indigo manages:
### Client Metadata Endpoint ### Client Metadata Endpoint
Production deployments serve OAuth client metadata at `{baseURL}/client-metadata.json`: Production deployments serve OAuth client metadata at `{baseURL}/oauth-client-metadata.json`:
```json ```json
{ {
"client_id": "https://atcr.io/client-metadata.json", "client_id": "https://atcr.io/oauth-client-metadata.json",
"client_name": "ATCR Registry", "client_name": "ATCR Registry",
"client_uri": "https://atcr.io", "client_uri": "https://atcr.io",
"redirect_uris": ["https://atcr.io/auth/oauth/callback"], "redirect_uris": ["https://atcr.io/auth/oauth/callback"],
@@ -275,26 +276,31 @@ For localhost, the client ID is query-based and no metadata endpoint is used.
## Scope Management ## Scope Management
ATCR requests the following OAuth scopes: ATCR requests the following OAuth scopes (see `GetDefaultScopes()` in `pkg/auth/oauth/client.go`):
**Base scopes:** **Base scopes:**
- `atproto`: Basic ATProto access - `atproto`: Basic ATProto access
**Blob scopes (for layer/manifest media types):** **Permission-set (for ATProto collections):**
- `include:io.atcr.authFullApp` — a Lexicon permission-set (defined in `lexicons/io/atcr/authFullApp.json`) that the PDS expands into repo access for these collections:
- `io.atcr.manifest` — manifest records
- `io.atcr.repo.page` — repository page records
- `io.atcr.sailor.profile` — user profile records
- `io.atcr.sailor.star` — star records
- `io.atcr.tag` — tag records
**RPC scope:**
- `rpc:com.atproto.repo.getRecord?aud=*`: Read access to any user's records (kept separate because permission-sets are namespace-limited)
**Blob scopes (not supported in Lexicon permission-sets, so listed explicitly):**
- `blob:application/vnd.oci.image.manifest.v1+json` - `blob:application/vnd.oci.image.manifest.v1+json`
- `blob:application/vnd.docker.distribution.manifest.v2+json` - `blob:application/vnd.docker.distribution.manifest.v2+json`
- `blob:application/vnd.oci.image.index.v1+json` - `blob:application/vnd.oci.image.index.v1+json`
- `blob:application/vnd.docker.distribution.manifest.list.v2+json` - `blob:application/vnd.docker.distribution.manifest.list.v2+json`
- `blob:application/vnd.cncf.oras.artifact.manifest.v1+json` - `blob:application/vnd.cncf.oras.artifact.manifest.v1+json`
- `blob:application/vnd.cncf.helm.config.v1+json`
**Repo scopes (for ATProto collections):** - `blob:application/vnd.cncf.helm.chart.content.v1.tar+gzip`
- `repo:io.atcr.manifest`: Manifest records - `blob:image/*` (image avatars)
- `repo:io.atcr.tag`: Tag records
- `repo:io.atcr.star`: Star records
- `repo:io.atcr.sailor.profile`: User profile records
**RPC scope:**
- `rpc:com.atproto.repo.getRecord?aud=*`: Read access to any user's records
Scopes are automatically invalidated on startup if they change, forcing users to re-authenticate. Scopes are automatically invalidated on startup if they change, forcing users to re-authenticate.
@@ -309,7 +315,7 @@ Scopes are automatically invalidated on startup if they change, forcing users to
- Used for PDS API requests (manifests, service tokens) - Used for PDS API requests (manifests, service tokens)
**Registry JWTs (issued to Docker clients):** **Registry JWTs (issued to Docker clients):**
- Short-lived (15 minutes) - Short-lived (5 minutes, not configurable); `exp` is bound to the PDS-granted service-auth expiry
- Signed by AppView's JWT signing key - Signed by AppView's JWT signing key
- Contain validated DID from OAuth session - Contain validated DID from OAuth session
- Used for OCI Distribution API requests - Used for OCI Distribution API requests
@@ -340,10 +346,9 @@ Scopes are automatically invalidated on startup if they change, forcing users to
### Common Issues ### Common Issues
**"Failed to initialize OAuth client key"** **"failed to query crypto_keys" / "failed to parse OAuth key from database"**
- Check that `/var/lib/atcr/oauth/` directory exists and is writable - Verify the AppView SQLite database is accessible and writable (the `oauth_p256` key lives in the `crypto_keys` table)
- Verify directory permissions are 0700 - A parse error suggests a corrupt `oauth_p256` row; delete the row to force regeneration (re-authentication required)
- Check disk space
**"OAuth session not found"** **"OAuth session not found"**
- User needs to re-authenticate (session expired or invalidated) - User needs to re-authenticate (session expired or invalidated)
@@ -358,7 +363,7 @@ Scopes are automatically invalidated on startup if they change, forcing users to
**"Client authentication failed"** **"Client authentication failed"**
- Confidential client key may be corrupted - Confidential client key may be corrupted
- Key ID may not match public key - Key ID may not match public key
- Try rotating the client key (delete and regenerate) - Try rotating the client key (delete the `oauth_p256` row from `crypto_keys` and restart to regenerate)
### Debugging ### Debugging
@@ -370,10 +375,10 @@ export ATCR_LOG_LEVEL=debug
``` ```
Look for log messages: Look for log messages:
- `"Generated new P-256 OAuth client key"` - Key was auto-generated - `"Generated new OAuth P-256 key and stored in database"` - Key was auto-generated and saved to `crypto_keys`
- `"Loaded existing P-256 OAuth client key"` - Key was loaded from disk - `"Loaded OAuth P-256 key from database"` - Key was loaded from the database
- `"Configured confidential OAuth client"` - Production confidential client active - `"Configured confidential OAuth client"` - Production confidential client active
- `"Localhost detected - using public OAuth client"` - Development public client active - `"Using public OAuth client (localhost development)"` - Development public client active
### Testing OAuth Flow ### Testing OAuth Flow
docs/QUOTAS.md
+169 -257
View File
@@ -12,6 +12,7 @@ This document describes ATCR's storage quota implementation using ATProto record
- [Delete Flow](#delete-flow) - [Delete Flow](#delete-flow)
- [Garbage Collection](#garbage-collection) - [Garbage Collection](#garbage-collection)
- [Configuration](#configuration) - [Configuration](#configuration)
- [Quota API Endpoints](#quota-api-endpoints)
- [Future Enhancements](#future-enhancements) - [Future Enhancements](#future-enhancements)
## Overview ## Overview
@@ -136,172 +137,117 @@ Using the example above:
### Implementation ### Implementation
```go Usage is computed **hold-side**, where the layer records actually live. There is no
// pkg/hold/quota/quota.go `QuotaManager` type — the pieces are:
type QuotaManager struct { - **`quota.Manager`** (`pkg/hold/quota/config.go`) resolves a crew tier name to a
pds *pds.Server // Hold's embedded PDS byte limit (tier → limit only). It does not compute usage. `Manager.IsEnabled()`
} reports whether any tiers are configured.
- **`recordsIndex.QuotaForDID(collection, userDID)`** runs the deduplicating SQL
// GetUsage calculates a user's current quota usage aggregation over the denormalized `digest`/`size` columns in the records index,
func (q *QuotaManager) GetUsage(ctx context.Context, userDID string) (int64, error) { returning `(uniqueBlobs, totalSize)`.
// List all layer records for this user - **`HoldPDS.GetQuotaForUser`** (`pkg/hold/pds/layer.go`) wraps `QuotaForDID` and
records, err := q.pds.ListRecords(ctx, LayerCollection, userDID) returns a `QuotaStats`.
if err != nil { - **`HoldPDS.GetQuotaForUserWithTier`** (`pkg/hold/pds/layer.go`) layers the tier
return 0, err limit on top: captain (owner) is always unlimited; otherwise it looks up the
} crew member's tier and asks `quota.Manager` for the limit.
// Deduplicate by digest
uniqueLayers := make(map[string]int64) // digest -> size
for _, record := range records {
var layer LayerRecord
if err := json.Unmarshal(record.Value, &layer); err != nil {
continue
}
if layer.UserDID == userDID {
uniqueLayers[layer.Digest] = layer.Size
}
}
// Sum unique layer sizes
var total int64
for _, size := range uniqueLayers {
total += size
}
return total, nil
}
// CheckQuota returns true if user has space for additional bytes
func (q *QuotaManager) CheckQuota(ctx context.Context, userDID string, additional int64, limit int64) (bool, int64, error) {
current, err := q.GetUsage(ctx, userDID)
if err != nil {
return false, 0, err
}
return current+additional <= limit, current, nil
}
```
### Quota Response
```go ```go
type QuotaInfo struct { // pkg/hold/pds/layer.go
Used int64 `json:"used"` // Current usage (deduplicated)
Limit int64 `json:"limit"` // User's quota limit // QuotaStats represents storage quota information for a user.
Available int64 `json:"available"` // Remaining space type QuotaStats struct {
UserDID string `json:"userDid"`
UniqueBlobs int `json:"uniqueBlobs"`
TotalSize int64 `json:"totalSize"`
Limit *int64 `json:"limit,omitempty"` // nil = unlimited
Tier string `json:"tier,omitempty"` // e.g. "deckhand", "bosun"
} }
// GetQuotaForUser deduplicates layer records via SQL aggregation.
func (p *HoldPDS) GetQuotaForUser(ctx context.Context, userDID string) (*QuotaStats, error)
// GetQuotaForUserWithTier adds the tier-resolved Limit (captain = unlimited).
func (p *HoldPDS) GetQuotaForUserWithTier(ctx context.Context, userDID string, quotaMgr *quota.Manager) (*QuotaStats, error)
``` ```
`QuotaStats` is the JSON shape returned by the public `io.atcr.hold.getQuota`
endpoint, which the appview calls during the push gate (see [Push Flow](#push-flow)).
## Push Flow ## Push Flow
### Step-by-Step: User Pushes Image ### Where Quota Is Enforced
Quota is enforced in the appview's **push authorizer at `/auth/token` time**, before
the push begins — not inside the manifest store. When a Docker client requests a push
token, the authorizer (`pkg/appview/authgate/push_authorizer.go`, `checkQuota`) calls
`atproto.FetchQuotaStats` against the hold's public `io.atcr.hold.getQuota` endpoint
and rejects the token request when the user's current usage already meets or exceeds
their limit.
``` ```
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ Client │ │ AppView │ │ Hold │ │ User PDS │ │ Client │ │ AppView │ │ Hold │
│ (Docker) │ │ │ │ Service │ │ │ │ (Docker) │ │ /auth/ │ │ getQuota │
└──────────┘ └──────────┘ └──────────┘ └──────────┘ │ │ │ token │ │ │
│ │ │ │ └──────────┘ └──────────┘ └──────────┘
│ 1. Upload blobs │ │ │ │ │ │
├─────────────────────>│ │ │ │ 1. Request push token│ │
│ │ 2. Route to hold │ │ ├─────────────────────>│ │
│ ├─────────────────────>│ │ │ │ 2. checkQuota: │
│ │ │ 3. Store in S3 │ │ │ FetchQuotaStats │
│ │ │ │ │ ├─────────────────────>│
│ 4. PUT manifest │ │ │ │ │ 3. {totalSize,limit} │
├─────────────────────>│ │ │ │ │<─────────────────────┤
│ │ │ │ │ │ │
│ │ 5. Calculate quota │ │ │ │ 4. if limit != nil │
│ │ impact for new │ │ │ │ && totalSize >= │
│ │ layers │ │ │ │ limit → reject │
│ │ │ │ │ │ (denied: quota) │
│ │ 6. Check quota limit │ │ │ │ │
│ ├─────────────────────>│ │ │ 5. 200 token / 401 │ │
│ │<─────────────────────┤ │ │<─────────────────────┤ │
│ │ │ │ │ │ │
│ │ 7. Store manifest │ │ │ 6. Push blobs + manifest (if token granted) │
│ ├──────────────────────┼─────────────────────>│ ├─────────────────────>│ ... │
│ │ │ │
│ │ 8. Create layer │ │
│ │ records │ │
│ ├─────────────────────>│ │
│ │ │ 9. Write to │
│ │ │ hold's PDS │
│ │ │ │
│ 10. 201 Created │ │ │
│<─────────────────────┤ │ │
``` ```
### Behavior
- **Check is current-usage vs. limit, evaluated before the push.** The condition is
`stats.TotalSize >= *stats.Limit`. It does **not** pre-add the incoming layer sizes;
a push is allowed to start whenever existing usage is strictly below the limit, even
if that push will push the user over. Overage is caught on the *next* push attempt.
- **Unlimited tiers skip the check.** `stats.Limit` is `nil` for captains and for holds
with no quota tiers configured, in which case no limit is enforced.
- **Fails open on errors.** If the hold's `getQuota` endpoint is unreachable or returns
an error, `checkQuota` logs a warning and **allows the push**. This is deliberate: the
endpoint is public/unauthenticated, and a hold outage should not produce spurious
`denied: quota` errors unrelated to the user's actual usage.
### Implementation ### Implementation
```go ```go
// pkg/appview/storage/routing_repository.go // pkg/appview/authgate/push_authorizer.go
func (r *RoutingRepository) PutManifest(ctx context.Context, manifest distribution.Manifest) error { func (a *Authorizer) checkQuota(ctx context.Context, userDID, holdDID string) error {
// Parse manifest to get layers stats, err := atproto.FetchQuotaStats(ctx, a.httpClient, holdDID, userDID)
layers := extractLayers(manifest)
// Get user's current unique layers from hold
existingLayers, err := r.holdClient.GetUserLayers(ctx, r.userDID)
if err != nil { if err != nil {
return err // Fail open: a hold outage must not block unrelated pushes.
} slog.Warn("push gate: quota call failed; allowing push", "did", userDID, "error", err)
existingSet := makeDigestSet(existingLayers) return nil
// Calculate quota impact (only new unique layers)
var quotaImpact int64
for _, layer := range layers {
if !existingSet[layer.Digest] {
quotaImpact += layer.Size
}
} }
// Check quota if stats.Limit != nil && stats.TotalSize >= *stats.Limit {
ok, current, err := r.quotaManager.CheckQuota(ctx, r.userDID, quotaImpact, r.quotaLimit) return fmt.Errorf("quota exceeded: %s / %s used by %s. Delete images to free space",
if err != nil { formatGB(stats.TotalSize), formatGB(*stats.Limit), a.identityLabel(ctx, userDID))
return err
} }
if !ok {
return fmt.Errorf("quota exceeded: used=%d, impact=%d, limit=%d",
current, quotaImpact, r.quotaLimit)
}
// Store manifest in user's PDS
manifestURI, err := r.atprotoClient.PutManifest(ctx, manifest)
if err != nil {
return err
}
// Create layer records in hold's PDS
for _, layer := range layers {
record := LayerRecord{
Type: "io.atcr.hold.layer",
Digest: layer.Digest,
Size: layer.Size,
MediaType: layer.MediaType,
Manifest: manifestURI,
UserDID: r.userDID,
CreatedAt: time.Now().Format(time.RFC3339),
}
if err := r.holdClient.CreateLayerRecord(ctx, record); err != nil {
log.Printf("Warning: failed to create layer record: %v", err)
// Continue - reconciliation will fix
}
}
return nil return nil
} }
``` ```
### Quota Check Timing Layer records that back the usage figure are created hold-side as part of the blob
upload / push pipeline, not by the appview's manifest store.
Quota is checked when the **manifest is pushed** (after blobs are uploaded):
- Blobs upload first via presigned URLs
- Manifest pushed last triggers quota check
- If quota exceeded, manifest is rejected (orphaned blobs cleaned by GC)
This matches Harbor's approach and is the industry standard.
## Delete Flow ## Delete Flow
@@ -334,63 +280,25 @@ When a user deletes a manifest:
│<─────────────────────┤ │ │ │<─────────────────────┤ │ │
``` ```
### Implementation ### Hold Service: Delete Layer Record
The deletion primitive is hold-side: `HoldPDS.DeleteLayerRecord` in
`pkg/hold/pds/layer.go`. It removes a single layer record by its rkey from both the
repo (the MST/CAR store) and the records index (the index delete is best-effort —
failures are logged, since a backfill resync will reconcile it).
```go ```go
// pkg/appview/handlers/manifest.go // pkg/hold/pds/layer.go
func (h *ManifestHandler) DeleteManifest(w http.ResponseWriter, r *http.Request) { // DeleteLayerRecord deletes a layer record by rkey from the repo (MST)
userDID := auth.GetDID(r.Context()) // and the records index.
repository := chi.URLParam(r, "repository") func (p *HoldPDS) DeleteLayerRecord(ctx context.Context, rkey string) error
digest := chi.URLParam(r, "digest")
// Get manifest URI before deletion
manifestURI := fmt.Sprintf("at://%s/%s/%s", userDID, ManifestCollection, digest)
// Delete manifest from user's PDS
if err := h.atprotoClient.DeleteRecord(ctx, ManifestCollection, digest); err != nil {
http.Error(w, "failed to delete manifest", 500)
return
}
// Delete associated layer records from hold's PDS
if err := h.holdClient.DeleteLayerRecords(ctx, manifestURI); err != nil {
log.Printf("Warning: failed to delete layer records: %v", err)
// Continue - reconciliation will clean up
}
w.WriteHeader(http.StatusNoContent)
}
``` ```
### Hold Service: Delete Layer Records To remove all layer records for a deleted manifest, callers identify the rkeys whose
records reference that manifest and call `DeleteLayerRecord` for each. Orphan cleanup
```go also happens automatically during garbage collection (see below), so a missed deletion
// pkg/hold/pds/xrpc.go is eventually reconciled rather than leaking quota permanently.
func (s *Server) DeleteLayerRecords(ctx context.Context, manifestURI string) error {
// List all layer records
records, err := s.ListRecords(ctx, LayerCollection, "")
if err != nil {
return err
}
// Delete records matching this manifest
for _, record := range records {
var layer LayerRecord
if err := json.Unmarshal(record.Value, &layer); err != nil {
continue
}
if layer.Manifest == manifestURI {
if err := s.DeleteRecord(ctx, LayerCollection, record.RKey); err != nil {
log.Printf("Failed to delete layer record %s: %v", record.RKey, err)
}
}
}
return nil
}
```
### Quota After Deletion ### Quota After Deletion
@@ -411,77 +319,68 @@ Orphaned blobs accumulate when:
### GC Process ### GC Process
```go GC is implemented in `pkg/hold/gc/gc.go`. The public entrypoint
// pkg/hold/gc/gc.go `GarbageCollector.Run` takes a single-run lock and delegates to `doRun`, which works
in multiple phases (see `(*GarbageCollector).doRun`):
func (gc *GarbageCollector) Run(ctx context.Context) error { 1. **Analyze records** (`analyzeRecords`) — build the set of referenced layer digests,
// Step 1: Get all referenced digests from layer records identify orphaned layer records (rkeys), and find manifest layers missing a record.
records, err := gc.pds.ListRecords(ctx, LayerCollection, "") 2. **Reconcile missing records** — create layer records for referenced layers that lost
if err != nil { their record, so usage accounting stays correct.
return err 3. **Delete orphaned layer records** (`deleteOrphanedRecords`) — remove layer records no
} longer referenced by any manifest.
4. **Delete orphaned blobs** (`deleteOrphanedBlobs`) — walk S3 and delete blobs not in the
referenced set.
referenced := make(map[string]bool) A read-only `Preview` / `doPreview` path runs phases 1 and the blob scan without deleting
for _, record := range records { anything, for the admin panel. A grace period (`gcGracePeriod`, 7 days) protects recently
var layer LayerRecord created records from collection.
if err := json.Unmarshal(record.Value, &layer); err != nil {
continue
}
referenced[layer.Digest] = true
}
log.Printf("Found %d referenced blobs", len(referenced))
// Step 2: Walk S3 blobs and delete unreferenced
var deleted, reclaimed int64
err = gc.driver.Walk(ctx, "/docker/registry/v2/blobs", func(fi storagedriver.FileInfo) error {
if fi.IsDir() {
return nil
}
digest := extractDigestFromPath(fi.Path())
if !referenced[digest] {
size := fi.Size()
if err := gc.driver.Delete(ctx, fi.Path()); err != nil {
log.Printf("Failed to delete %s: %v", digest, err)
return nil
}
deleted++
reclaimed += size
log.Printf("GC: deleted %s (%d bytes)", digest, size)
}
return nil
})
log.Printf("GC complete: deleted %d blobs, reclaimed %d bytes", deleted, reclaimed)
return err
}
```
### GC Schedule ### GC Schedule
```bash GC is toggled by a single config field; the interval is **not** configurable — it is the
# Environment variable hardcoded `gcInterval` constant (24h) in `pkg/hold/gc/config.go`.
GC_ENABLED=true
GC_INTERVAL=24h # Daily by default ```yaml
# config-hold.yaml
gc:
enabled: true # env: GC_ENABLED
``` ```
There is no `GC_INTERVAL` setting.
## Configuration ## Configuration
### Hold Service Environment Variables ### Enabling Quotas
```bash Quotas are **enabled by the presence of quota tiers** in the hold config — there is no
# .env.hold `QUOTA_ENABLED` flag and no global `QUOTA_DEFAULT_LIMIT`. `quota.Manager.IsEnabled()`
returns true whenever a config with at least one tier was loaded; with no tiers, every
user is unlimited.
# Quota Configuration ```yaml
QUOTA_ENABLED=true # config-hold.yaml
QUOTA_DEFAULT_LIMIT=10737418240 # 10GB in bytes
# Garbage Collection quota:
GC_ENABLED=true tiers:
GC_INTERVAL=24h - name: deckhand
quota: 5GB
- name: bosun
quota: 50GB
scan_on_push: true
- name: quartermaster
quota: 100GB
scan_on_push: true
defaults:
new_crew_tier: deckhand
gc:
enabled: true # env: GC_ENABLED (interval is a hardcoded 24h, not configurable)
``` ```
The only quota/GC environment variable is `GC_ENABLED` (bound to `gc.enabled`). Tier
limits are human-readable sizes (`5GB`, `50GB`, `1TB`) parsed by `quota.ParseHumanBytes`.
### Quota Limits by Bytes ### Quota Limits by Bytes
| Size | Bytes | | Size | Bytes |
@@ -492,13 +391,26 @@ GC_INTERVAL=24h
| 50 GB | 53687091200 | | 50 GB | 53687091200 |
| 100 GB | 107374182400 | | 100 GB | 107374182400 |
## Future Enhancements ## Quota API Endpoints
### 1. Quota API Endpoints The per-user quota endpoint is **implemented**. It is public (no auth) and is what the
appview's push gate calls (see [Push Flow](#push-flow)).
``` ```
GET /xrpc/io.atcr.hold.getQuota?did={userDID} - Get user's quota usage GET /xrpc/io.atcr.hold.getQuota?userDid={did}
GET /xrpc/io.atcr.hold.getQuotaBreakdown - Storage by repository → {"userDid": "...", "uniqueBlobs": 10, "totalSize": 1073741824, "limit": ..., "tier": "..."}
```
- Endpoint constant: `HoldGetQuota` in `pkg/atproto/endpoints.go`
- Handler: `HandleGetQuota`, registered in `pkg/hold/pds/xrpc.go`
- Client helper: `atproto.FetchQuotaStats` in `pkg/atproto/quota.go`
## Future Enhancements
### 1. Quota Breakdown Endpoint
```
GET /xrpc/io.atcr.hold.getQuotaBreakdown - Storage by repository (not yet implemented)
``` ```
### 2. Quota Alerts ### 2. Quota Alerts
@@ -570,6 +482,6 @@ Pull rate limits (Docker Hub style):
--- ---
**Document Version:** 2.0 **Document Version:** 2.1
**Last Updated:** 2026-01-04 **Last Updated:** 2026-06-11
**Model:** Per-user layer tracking with ATProto records **Model:** Per-user layer tracking with ATProto records
docs/REBRAND.md
-558
View File
@@ -1,558 +0,0 @@
# Website Visual Improvement Plan
## Goal
Create a fun, personality-driven container registry that embraces its nautical theme while being clearly functional. Think GitHub's Octocat or DigitalOcean's Sammy - playful but professional.
## Brand Identity (from seahorse logo)
- **Primary Teal**: #4ECDC4 (body color) - the "ocean" feel
- **Dark Teal**: #2E8B8B (mane/fins) - depth and contrast
- **Mint Background**: #C8F0E7 - light, airy, underwater
- **Coral Accent**: #FF6B6B (eye) - warmth, CTAs, highlights
- **Nautical theme to embrace:**
- "Ship" containers (not just push)
- "Holds" for storage (like a ship's cargo hold)
- "Sailors" are users, "Captains" own holds
- Seahorse mascot as the friendly guide
## Design Direction: Fun but Functional
- Softer, more rounded corners
- Playful color combinations (teal + coral)
- Mascot appearances in empty states, loading, errors
- Ocean-inspired subtle backgrounds (gradients, waves)
- Friendly copy and microcopy throughout
- Still clearly a container registry with all the technical info
## Current State
- Pure CSS with custom properties for theming
- Basic card designs for repositories
- Simple hero section with terminal mockup
- Existing badges: Helm charts, multi-arch, attestations
- Existing stats: stars, pull counts
## Layout Wireframes
### Current Homepage Layout
```
┌─────────────────────────────────────────────────────────────────┐
│ [Logo] [Search] [Theme] [User] │ Navbar
├─────────────────────────────────────────────────────────────────┤
│ │
│ ship containers on the open web. │ Hero
│ ┌─────────────────────────┐ │
│ │ $ docker login atcr.io │ │
│ └─────────────────────────┘ │
│ [Get Started] [Learn More] │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ Benefits
│ │ Docker │ │ Your Data │ │ Discover │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
├─────────────────────────────────────────────────────────────────┤
│ │
│ Featured │
│ ┌─────────────────────────────────────────────────────────────┐│
│ │ [icon] user/repo ★ 12 ↓ 340 ││ WIDE cards
│ │ Description text here... ││ (current)
│ └─────────────────────────────────────────────────────────────┘│
│ ┌─────────────────────────────────────────────────────────────┐│
│ │ [icon] user/repo2 ★ 5 ↓ 120 ││
│ └─────────────────────────────────────────────────────────────┘│
│ │
│ What's New │
│ (similar wide cards) │
└─────────────────────────────────────────────────────────────────┘
```
### Proposed Layout: Tile Grid
```
┌─────────────────────────────────────────────────────────────────┐
│ [Logo] [Search] [Theme] [User] │
├─────────────────────────────────────────────────────────────────┤
│ │
│ ship containers on the open web. │
│ ┌─────────────────────────┐ │
│ │ $ docker login atcr.io │ │
│ └─────────────────────────┘ │
│ [Get Started] [Learn More] │
│ │
│ ┌────────────┐ ┌────────────┐ ┌────────────┐ │
│ │ Docker │ │ Your Data │ │ Discover │ │
│ └────────────┘ └────────────┘ └────────────┘ │
├─────────────────────────────────────────────────────────────────┤
│ │
│ Featured [View All] │
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐│
│ │ [icon] │ │ [icon] │ │ [icon] ││ 3 columns
│ │ user/repo │ │ user/repo2 │ │ user/repo3 ││ ~300px each
│ │ Description... │ │ Description... │ │ Description... ││
│ │ ────────────────││ │ ────────────────││ │ ────────────────│││
│ │ ★ 12 ↓ 340 │ │ ★ 5 ↓ 120 │ │ ★ 8 ↓ 89 ││
│ └──────────────────┘ └──────────────────┘ └──────────────────┘│
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐│
│ │ ... │ │ ... │ │ ... ││
│ └──────────────────┘ └──────────────────┘ └──────────────────┘│
│ │
├─────────────────────────────────────────────────────────────────┤
│ │
│ What's New │
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐│
│ │ ... │ │ ... │ │ ... ││ Same tile
│ └──────────────────┘ └──────────────────┘ └──────────────────┘│ layout
└─────────────────────────────────────────────────────────────────┘
```
### Unified Tile Card (Same for Featured & What's New)
```
┌─────────────────────────────┐
│ ┌────┐ user/repo [Helm] │ Icon + name + type badge
│ │icon│ :latest │ Tag (if applicable)
│ └────┘ │
│ │
│ Description text that │ Description (2-3 lines max)
│ wraps nicely here... │
│ │
│ sha256:abcdef12 │ Digest (truncated)
│ ───────────────────────────│ Divider
│ ★ 12 ↓ 340 1 day ago │ Stats + timestamp
└─────────────────────────────┘
Card anatomy:
┌─────────────────────────────┐
│ HEADER │ - Icon (48x48)
│ - icon + name + badge │ - user/repo
│ - tag (optional) │ - :tag or :latest
├─────────────────────────────┤
│ BODY │ - Description (clamp 2-3 lines)
│ - description │ - sha256:abc... (monospace)
│ - digest │
├─────────────────────────────┤
│ FOOTER │ - ★ star count
│ - stats + time │ - ↓ pull count
│ │ - "2 hours ago"
└─────────────────────────────┘
```
### Both Sections Use Same Card (Different Sort)
```
Featured (by stars/curated): What's New (by last_push):
┌─────────────────────────┐ ┌─────────────────────────┐
│ user/repo │ │ user/repo │
│ :latest │ │ :v1.2.3 │ ← latest tag
│ Description... │ │ Description... │
│ │ │ │
│ sha256:abc123 │ │ sha256:def456 │ ← latest digest
│ ───────────────────────│ │ ───────────────────────│
│ ★ 12 ↓ 340 1 day ago │ │ ★ 5 ↓ 89 2 hrs ago │ ← last_push time
└─────────────────────────┘ └─────────────────────────┘
Same card component, different data source:
- Featured: GetFeaturedRepos() (curated or by stars)
- What's New: GetRecentlyUpdatedRepos() (ORDER BY last_push DESC)
```
### Card Dimensions Comparison
```
Current: █████████████████████████████████████████ (~800px+ wide)
Proposed: ████████████ ████████████ ████████████ (~280-320px each)
Card 1 Card 2 Card 3
```
### Mobile Responsive Behavior
```
Desktop (>1024px): [Card] [Card] [Card] 3 columns
Tablet (768-1024px): [Card] [Card] 2 columns
Mobile (<768px): [Card] 1 column (full width)
```
### Playful Elements
```
Empty State (no repos):
┌─────────────────────────────────────────┐
│ │
│ 🐴 (seahorse) │
│ "Nothing here yet!" │
│ │
│ Ship your first container to get │
│ started on your voyage. │
│ │
│ [Start Shipping] │
└─────────────────────────────────────────┘
Error/404:
┌─────────────────────────────────────────┐
│ │
│ 🐴 (confused seahorse) │
│ "Lost at sea!" │
│ │
│ We couldn't find that container. │
│ Maybe it drifted away? │
│ │
│ [Back to Shore] │
└─────────────────────────────────────────┘
Hero with subtle ocean feel:
┌─────────────────────────────────────────┐
│ ≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋ │ Subtle wave pattern bg
│ │
│ ship containers on the │
│ open web. 🐴 │ Mascot appears!
│ │
│ ┌─────────────────────┐ │
│ │ $ docker login ... │ │
│ └─────────────────────┘ │
│ │
│ ≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋≋ │
└─────────────────────────────────────────┘
```
### Card with Personality
```
┌───────────────────────────────────┐
│ ┌──────┐ │
│ │ icon │ user/repo │
│ │ │ :latest [⚓ Helm] │ Anchor icon for Helm
│ └──────┘ │
│ │
│ A container that does amazing │
│ things for your app... │
│ │
│ sha256:abcdef12 │
│ ─────────────────────────────────│
│ ★ 12 ↓ 340 1 day ago │
│ │
│ 🐴 Shipped by alice.bsky.social │ Playful "shipped by" line
└───────────────────────────────────┘
(optional: "Shipped by" could be subtle or only on hover)
```
## Design Improvements
### 1. Enhanced Card Design (Priority: High)
**Files:** `pkg/appview/public/css/style.css`, `pkg/appview/templates/components/repo-card.html`
- Add subtle gradient backgrounds on hover
- Improve shadow depth (layered shadows for modern look)
- Add smooth transitions (transform, box-shadow)
- Better icon styling with ring/border accent
- Enhanced badge visibility with better contrast
- Add "Updated X ago" timestamp to cards
- Improve stat icon/count alignment and spacing
### 2. Hero Section Polish (Priority: High)
**Files:** `pkg/appview/public/css/style.css`, `pkg/appview/templates/pages/home.html`
- Add subtle background pattern or gradient mesh
- Improve terminal mockup styling (better shadows, glow effect)
- Enhance benefit cards with icons and better spacing
- Add visual separation between hero and content
- Improve CTA button styling with better hover states
### 3. Typography & Spacing (Priority: High)
**Files:** `pkg/appview/public/css/style.css`
- Increase visual hierarchy with better font weights
- Add more breathing room (padding/margins)
- Improve heading styles with subtle underlines or accents
- Better link styling with hover states
- Add letter-spacing to badges for readability
### 4. Badge System Enhancement (Priority: Medium)
**Files:** `pkg/appview/public/css/style.css`, templates
- Create unified badge design language
- Add subtle icons inside badges (already using Lucide)
- Improve color coding: Helm (blue), Attestation (green), Multi-arch (purple)
- Add "Official" or "Verified" badge styling (for future use)
- Better hover states on interactive badges
### 5. Featured Section Improvements (Priority: Medium)
**Files:** `pkg/appview/templates/pages/home.html`, `pkg/appview/public/css/style.css`
- Add section header with subtle styling
- Improve grid responsiveness
- Add "View All" link styling
- Better visual distinction from "What's New" section
### 6. Navigation Polish (Priority: Medium)
**Files:** `pkg/appview/public/css/style.css`, nav templates
- Improve search bar visibility and styling
- Better user menu dropdown aesthetics
- Add subtle border or shadow to navbar
- Improve mobile responsiveness
### 7. Loading & Empty States (Priority: Low)
**Files:** `pkg/appview/public/css/style.css`
- Add skeleton loading animations
- Improve empty state illustrations/styling
- Better transition when content loads
### 8. Micro-interactions (Priority: Low)
**Files:** `pkg/appview/public/css/style.css`, `pkg/appview/public/js/app.js`
- Add subtle hover animations throughout
- Improve button press feedback
- Star button animation on click
- Copy button success animation
## Implementation Order
1. **Phase 1: Core Card Styling**
- Update `.featured-card` with modern shadows and transitions
- Enhance badge styling in `style.css`
- Add hover effects and transforms
2. **Phase 2: Hero & Featured Section**
- Improve hero section gradient/background
- Polish benefit cards
- Add section separators
3. **Phase 3: Typography & Spacing**
- Update font weights and sizes
- Improve padding throughout
- Better visual rhythm
4. **Phase 4: Navigation & Polish**
- Navbar improvements
- Loading states
- Final micro-interactions
## Key CSS Changes
### Tile Grid Layout
```css
.featured-grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(280px, 1fr));
gap: 1.5rem;
}
/* Already exists but updating min-width */
.featured-card {
min-height: 200px;
display: flex;
flex-direction: column;
justify-content: space-between;
}
```
### Enhanced Shadow System (Multi-layer for depth)
```css
--shadow-card: 0 1px 3px rgba(0,0,0,0.08), 0 4px 12px rgba(0,0,0,0.05);
--shadow-card-hover: 0 8px 25px rgba(78,205,196,0.15), 0 4px 12px rgba(0,0,0,0.1);
--shadow-nav: 0 2px 8px rgba(0,0,0,0.1);
```
### Card Design Enhancement
```css
.featured-card {
transition: transform 0.2s ease, box-shadow 0.2s ease, border-color 0.2s ease;
border: 1px solid var(--border);
}
.featured-card:hover {
transform: translateY(-4px);
box-shadow: var(--shadow-card-hover);
border-color: var(--primary); /* teal accent on hover */
}
```
### Icon Container Styling
```css
.featured-icon-placeholder {
background: linear-gradient(135deg, var(--primary) 0%, var(--primary-dark) 100%);
box-shadow: 0 2px 8px rgba(78,205,196,0.3);
}
```
### Badge System (Consistent, Accessible)
```css
.badge-helm {
background: #0d6cbf;
color: #fff;
}
.badge-multi {
background: #7c3aed;
color: #fff;
}
.badge-attestation {
background: #059669;
color: #fff;
}
/* All badges: */
font-weight: 600;
letter-spacing: 0.02em;
text-transform: uppercase;
font-size: 0.7rem;
padding: 0.25rem 0.5rem;
border-radius: 4px;
```
### Hero Section Enhancement
```css
.hero-section {
background:
linear-gradient(135deg, var(--hero-bg-start) 0%, var(--hero-bg-end) 50%, rgba(78,205,196,0.1) 100%),
url('/static/wave-pattern.svg'); /* subtle wave pattern */
background-size: cover, 100% 50px;
background-position: center, bottom;
background-repeat: no-repeat, repeat-x;
}
.benefit-card {
border: 1px solid transparent;
border-radius: 12px; /* softer corners */
transition: all 0.2s ease;
}
.benefit-card:hover {
border-color: var(--primary);
transform: translateY(-4px);
}
```
### Playful Border Radius (Softer Feel)
```css
:root {
--radius-sm: 6px; /* was 4px */
--radius-md: 12px; /* was 8px */
--radius-lg: 16px; /* new */
}
.featured-card { border-radius: var(--radius-md); }
.benefit-card { border-radius: var(--radius-md); }
.btn { border-radius: var(--radius-sm); }
.hero-terminal { border-radius: var(--radius-lg); }
```
### Fun Empty States
```css
.empty-state {
text-align: center;
padding: 3rem;
}
.empty-state-mascot {
width: 120px;
height: auto;
margin-bottom: 1.5rem;
animation: float 3s ease-in-out infinite;
}
@keyframes float {
0%, 100% { transform: translateY(0); }
50% { transform: translateY(-10px); }
}
.empty-state-title {
font-size: 1.5rem;
font-weight: 600;
color: var(--fg);
}
.empty-state-text {
color: var(--secondary);
margin-bottom: 1.5rem;
}
```
### Typography Refinements
```css
.featured-title {
font-weight: 600;
letter-spacing: -0.01em;
}
.featured-description {
line-height: 1.5;
opacity: 0.85;
}
```
## Data Model Change
**Current "What's New":** Shows individual pushes (each tag push is a separate card)
**Proposed "What's New":** Shows repos ordered by last update time (same as Featured, different sort)
**Tracking:** `repository_stats` table already has `last_push` timestamp!
```sql
SELECT * FROM repository_stats ORDER BY last_push DESC LIMIT 9;
```
**Unified Card Data:**
| Field | Source |
|-------|--------|
| Handle, Repository | users + manifests |
| Tag | Latest tag from `tags` table |
| Digest | From latest tag or manifest |
| Description, IconURL | repo_pages or annotations |
| StarCount, PullCount | stars count + repository_stats |
| LastUpdated | `repository_stats.last_push` |
| ArtifactType | manifests.artifact_type |
## Files to Modify
| File | Changes |
|------|---------|
| `pkg/appview/public/css/style.css` | Rounded corners, shadows, hover, badges, ocean theme |
| `pkg/appview/public/wave-pattern.svg` | NEW: Subtle wave pattern for hero background |
| `pkg/appview/templates/components/repo-card.html` | Add Tag, Digest, LastUpdated fields |
| `pkg/appview/templates/components/empty-state.html` | NEW: Reusable fun empty state with mascot |
| `pkg/appview/templates/pages/home.html` | Both sections use repo-card grid |
| `pkg/appview/templates/pages/404.html` | Fun "Lost at sea" error page |
| `pkg/appview/db/queries.go` | New `GetRecentlyUpdatedRepos()` query; add fields to `RepoCardData` |
| `pkg/appview/handlers/home.go` | Replace `GetRecentPushes` with `GetRecentlyUpdatedRepos` |
| `pkg/appview/templates/partials/push-list.html` | Delete or repurpose (no longer needed) |
## Dependencies
**Mascot Art Needed:**
- `seahorse-empty.svg` - Friendly pose for "nothing here yet" empty states
- `seahorse-confused.svg` - Lost/confused pose for 404 errors
- `seahorse-waving.svg` (optional) - For hero section accent
**Can proceed without art:**
- CSS changes (colors, shadows, rounded corners, gradients)
- Card layout and grid changes
- Data layer changes (queries, handlers)
- Wave pattern background (simple SVG)
**Blocked until art is ready:**
- Empty state component with mascot
- 404 page redesign with mascot
- Hero mascot integration (optional)
## Implementation Phases
### Phase 1: CSS & Layout (No art needed)
1. Update border-radius variables (softer corners)
2. New shadow system
3. Card hover effects with teal accent
4. Tile grid layout (`minmax(280px, 1fr)`)
5. Wave pattern SVG for hero background
### Phase 2: Card Component & Data
1. Update `repo-card.html` with new structure
2. Add `Digest`, `Tag`, `CreatedAt` fields
3. Update queries for latest manifest info
4. Replace push list with card grid
### Phase 3: Hero & Section Polish
1. Hero gradient + wave pattern
2. Benefit card improvements
3. Section headers and spacing
4. Mobile responsive breakpoints
### Phase 4: Mascot Integration (BLOCKED - needs art)
1. Empty state component with mascot
2. 404 page with confused seahorse
3. Hero mascot (optional)
### Phase 5: Testing
1. Dark mode verification
2. Mobile responsive check
3. All functionality works (stars, links, copy)
## Verification
1. **Visual check on homepage** - cards have depth and polish
2. **Hover states** - smooth transitions on cards, buttons, badges
3. **Dark mode** - all changes work in both themes
4. **Mobile** - responsive at all breakpoints
5. **Functionality** - stars, search, navigation all work
6. **Performance** - no jank from CSS transitions
7. **Accessibility** - badge text readable (contrast check)
docs/REPOMGR_MIGRATION.md
-148
View File
@@ -1,148 +0,0 @@
# Incremental Migration: Vendored repomgr → Direct `indigo/repo`
## Context
The hold PDS uses a vendored copy of indigo's `repomgr` (~1450 lines in `pkg/hold/pds/repomgr.go`). Upstream, repomgr is [soft-deprecated](https://github.com/bluesky-social/indigo/pull/1102#issuecomment-2985956040) — bnewbold recommends using `indigo/repo` directly (as [cocoon](https://github.com/haileyok/cocoon) does). The vendored copy already has custom patches (PutRecord, UpsertRecord, prevData) and will continue to drift. This migration defines a clean interface, then swaps the implementation behind it.
## Phase 0: Save plan to docs, remove dead code, define interface ✅
**Goal:** Persist this migration plan as a reference doc. Shrink repomgr.go from ~1450 lines to ~750 by removing dead code. Fix import.go encapsulation. Define `RepoOperator` interface so all code accesses repomgr through it. No behavior change.
**Completed 2026-02-28.** repomgr.go: 1453 → 871 lines (-40%). import.go: 189 → 88 lines (-53%). New repo_operator.go: 82 lines.
### Step 1: Save plan to docs ✅
- Write this plan to `docs/REPOMGR_MIGRATION.md`
### Step 2: Dead code deleted from `repomgr.go` ✅
- `HandleExternalUserEvent()` + `handleExternalUserEventNoArchive()` + `handleExternalUserEventArchive()`
- `ImportNewRepo()` + `processNewRepo()` + `walkTree()` + `processOp()` + `stringOrNil()`
- `CheckRepoSig()`
- `TakeDownRepo()`, `ResetRepo()`, `VerifyRepo()`
- `GetProfile()`
- `CarStore()`
- `NextTID()` / `nextTID()` (removed entirely — `BatchWrite` uses `rm.clk.Next()` directly)
- `noArchive` field removed from struct
- 12 unused imports cleaned up
### Step 3: Fixed `import.go` encapsulation ✅
Added `BulkUpsert()` method to `RepoManager`. Rewrote `ImportFromCAR` to call `p.repomgr.BulkUpsert()` instead of reaching into `p.repomgr.lockUser`, `p.repomgr.cs`, `p.repomgr.kmgr`, `p.repomgr.events`. Removed the 88-line `bulkImportRecords()` private method.
### Step 4: Defined `RepoOperator` interface ✅
New file: `pkg/hold/pds/repo_operator.go` — interface with 16 methods, compile-time check, shared types (`RepoEvent`, `RepoOp`, `EventKind`, `BulkRecord`).
### Step 5: Updated callers to use interface ✅
- `pkg/hold/pds/server.go` — `repomgr *RepoManager` → `repomgr RepoOperator`, `RepomgrRef()` returns `RepoOperator`
- Downstream callers (`hold/server.go`, `admin/handlers_relays.go`, tests) unchanged — they go through `RepomgrRef()` which returns the interface
- `var _ RepoOperator = (*RepoManager)(nil)` compile-time check in `repo_operator.go`
### Verification: ✅
- `make lint` — 0 issues (also fixed pre-existing unchecked error in `events.go`)
- `make test` — all tests pass
---
## Phase 1: Test hardening against the interface ✅
**Goal:** Write tests against `RepoOperator` that verify current behavior while `RepoManager` is the only implementation. These become the regression safety net when swapping to the new implementation in Phase 3.
**Completed 2026-02-28.** New `repo_operator_test.go`: 38 subtests covering all CRUD, read, event emission, error paths, and edge cases. `runRepoOperatorTests(t, setup)` pattern ready for Phase 2's `DirectRepoOperator`.
### Gaps covered ✅
- `CreateRecord` — round-trip, TID 13-char rkey format, no-panic without event handler
- `UpdateRecord` — CID changes, new data returned, non-existent record error, hydrated events
- `PutRecord` — explicit rkey, duplicate rkey error, hydrated events
- `UpsertRecord` — create path (created=true), update path (created=false, CID changes)
- `DeleteRecord` — delete + verify gone, non-existent rkey error
- `BatchWrite` — create+delete batch, update write type, auto-rkey (nil Rkey), delete-not-found error, empty write elem error, multi-op event emission with hydration, update hydration
- `BulkUpsert` — create + re-upsert with changed data, multi-op event emission
- `GetRecord` — CID match, CID mismatch error, not-found error
- `GetRecordProof` — head CID + proof blocks, not-found error, no-repo error
- `GetRepoRoot` — defined CID after init, changes after write
- `GetRepoRev` — non-empty, changes after write
- `ReadRepo` — non-empty CAR output, incremental export with `since`
- `InitNewActor` — empty DID error, zero user error, event emission with hydration
- Event emission — create/update/delete events verified: prevData, ops, oldRoot, newRoot, rev, since, repoSlice, hydration
### Coverage ✅
All RepoOperator methods 81–100%. Remaining uncovered lines are internal infrastructure error guards (`GetUserRepoRev`, `NewDeltaSession`, `OpenRepo`, `Commit`, `CloseWithRoot`) — not reachable without mocking the carstore.
### Files modified ✅
- `pkg/hold/pds/repo_operator_test.go` — new file, 38 subtests
### Verification ✅
- `make lint` — 0 issues
- `make test` — all tests pass
---
## Phase 2: Build new implementation ✅
**Goal:** Create `DirectRepoOperator` using `indigo/repo` directly (cocoon pattern).
**Completed 2026-02-28.** New `pkg/hold/pds/repo.go`: 548 lines (vs 852 in repomgr.go, ~36% reduction). All 37 subtests pass identically for both implementations. Race detector, shuffled order, and parallel execution all clean.
### New file: `pkg/hold/pds/repo.go` ✅
Key differences from vendored repomgr:
- **Single `sync.Mutex`** instead of per-user lock map (`lklk` + `userLocks` + `userLock` struct + reference counting)
- **No OpenTelemetry tracing** (`otel.Tracer` calls removed)
- **No `gorm` dependency** (`RepoHead` struct removed, `gorm.io/gorm` dropped from go.mod)
- **`openWriteSession` / `commitWrite` helpers** extract the repeated 6-step write pattern
Core mutation pattern (same as current, just cleaner):
1. Lock → get rev → open delta session → open repo
2. Capture `r.DataCid()` for prevData
3. Perform operation(s)
4. `r.Commit()` → `ds.CloseWithRoot()` → emit event → unlock
### Shared types moved to `repo_operator.go` ✅
- `KeyManager` interface and `ActorInfo` struct moved from `repomgr.go`
- Both implementations import from the same location
### Test wiring ✅
- `setupTestDirectRepoOperator` — creates carstore + key manager directly (no `NewHoldPDS`)
- `runRepoOperatorTests` refactored to accept optional `freshSetup` for `InitNewActor_EventEmission`
- `TestDirectRepoOperator` runs all 37 subtests identically
### Verification ✅
- `go build ./cmd/hold` — compiles
- `TestRepoManager` — 37/37 subtests pass
- `TestDirectRepoOperator` — 37/37 subtests pass
- `-race -shuffle=on -count=5 -parallel=8` — all clean
- `make lint` — 0 issues
- `make test` — all tests pass
---
## Phase 3: Config flag + switchover
**Goal:** Feature flag to select implementation, default old.
### Changes:
- `pkg/hold/config.go` — add `UseDirectRepo bool` to DatabaseConfig
- `pkg/hold/pds/server.go` — select implementation based on config in `NewHoldPDS`/`NewHoldPDSWithDB`
- Regenerate example configs
### Verification:
- Deploy with `use_direct_repo: false` (default)
- Test with `use_direct_repo: true` in staging
- `make lint && make test`
---
## Phase 4: Remove vendored repomgr
**Goal:** After production validation, delete the old code.
- Delete `repomgr.go`
- Remove config flag, make `DirectRepoOperator` the only implementation
- Rename `repo_direct.go` → `repo_operator_impl.go`
- Regenerate example configs
- `make lint && make test`
---
## Decision log
- **`indigo/repo` over `atproto/repo`**: `atproto/repo` has the MST primitives (`Insert`, `Remove`, `ApplyOp`) but no high-level PDS API (`OpenRepo`, `CreateRecord`, `Commit(signFn)`). Its own doc.go says "does not yet work for implementing a repository host (PDS)." `indigo/repo` is what the reference PDS and cocoon use. The `RepoOperator` interface means we can swap later if `atproto/repo` adds PDS support.
docs/SBOM_SCANNING.md
+338 -558
View File
@@ -1,220 +1,244 @@
# SBOM Scanning # SBOM Scanning and Vulnerability Analysis
ATCR supports optional Software Bill of Materials (SBOM) generation for container images stored in holds. This feature enables automated security scanning and vulnerability analysis while maintaining the decentralized architecture. ATCR generates Software Bills of Materials (SBOMs) and scans container images for
vulnerabilities. Scanning runs in a separate `atcr-scanner` service that connects to
a hold over a WebSocket, so the hold itself never runs Syft or Grype. Results are
stored as `io.atcr.hold.scan` records in the hold's embedded PDS.
## Overview ## Overview
When enabled, holds automatically generate SBOMs for uploaded container images in the background. The scanning process: - **Separate scanner binary**: Scanning is performed by `atcr-scanner` (the `scanner/`
Go module), not by the hold. The scanner connects out to the hold and pulls jobs.
- **Syft for SBOMs, Grype for vulnerabilities**: Each job runs Syft to produce an
SPDX-JSON SBOM, then Grype to scan that SBOM for CVEs. Grype is enabled by default.
- **WebSocket dispatch**: The hold pushes jobs to connected scanners over
`/xrpc/io.atcr.hold.subscribeScanJobs`. A shared secret authenticates the scanner.
- **ATProto result storage**: Results land as `io.atcr.hold.scan` records in the
hold's embedded PDS, with the SBOM and full Grype report uploaded as PDS blobs.
- **Tier-gated scan-on-push plus proactive rescans**: Pushes from eligible tiers
trigger an immediate scan; the hold also discovers never-scanned manifests and
re-scans stale ones on an interval.
- **Async execution**: Scanning happens after upload completes (non-blocking) ### Tools
- **ORAS artifacts**: SBOMs stored as OCI Registry as Storage (ORAS) artifacts
- **ATProto integration**: Scan results stored as `io.atcr.manifest` records in hold's embedded PDS
- **Tool agnostic**: Results accessible via XRPC, ATProto queries, and direct blob URLs
- **Opt-in**: Disabled by default, enabled per-hold via configuration
### Default Scanner: Syft - [Anchore Syft](https://github.com/anchore/syft) generates the SBOM. Output format is
SPDX JSON, hardcoded in `scanner/internal/scan/syft.go` (not configurable).
- [Anchore Grype](https://github.com/anchore/grype) scans the SBOM for known
vulnerabilities and produces critical/high/medium/low/total counts plus a full
JSON report with CVE detail.
ATCR uses [Anchore Syft](https://github.com/anchore/syft) for SBOM generation: ## Architecture
- Industry-standard SBOM generator
- Supports SPDX and CycloneDX formats
- Comprehensive package detection (OS packages, language libraries, etc.)
- Active maintenance and CVE database updates
Future enhancements may include [Grype](https://github.com/anchore/grype) for vulnerability scanning and [Trivy](https://github.com/aquasecurity/trivy) for comprehensive security analysis. Three pieces cooperate:
## Trust Model ```
io.atcr.hold.subscribeScanJobs (WebSocket, ?secret=...)
┌───────────┐ ◄──────────────────────────────────────────── ┌──────────────┐
│ Hold │ job: {seq, manifestDigest, repo, tier, │ atcr-scanner │
│ (Scan │ config, layers, holdEndpoint, ...} │ (Syft + │
│ Broadcaster)│ ────────────────────────────────────────────► │ Grype) │
│ │ │ │
│ │ result/error/skipped: {seq, sbom, │ │
│ │ ◄──── vulnReport, summary{critical,high,...}} └──────────────┘
└─────┬─────┘
│ stores io.atcr.hold.scan record + SBOM/vuln blobs
▼
Hold embedded PDS (CAR store)
```
### Same Trust as Docker Hub 1. **Hold (`pkg/hold/pds/scan_broadcaster.go`)** owns the `ScanBroadcaster`. It
persists pending jobs in SQLite (`scan_jobs` table), accepts scanner WebSocket
connections, and dispatches jobs **round-robin** across all connected scanners
using a competing-consumer pattern. It re-dispatches timed-out jobs, and (when a
rescan interval is set) runs background discovery and stale-scan loops. On receiving
a result, the hold uploads the SBOM and vuln report as blobs and writes the
`io.atcr.hold.scan` record.
2. **Scanner (`scanner/` module)** dials the hold's WebSocket, acks jobs, runs the
Syft → Grype pipeline, and sends back a result, error, or skipped message. It keeps
a local **priority queue** so paid tiers jump ahead of free ones (see Scheduling).
3. **AppView** reads the scan records and blobs from the hold's PDS to render
vulnerability badges, SBOM details, and download links in the web UI.
SBOM scanning follows the same trust model as Docker Hub or other centralized registries: ### Why the hold's PDS?
**Docker Hub model:**
- Docker Hub scans your image on their infrastructure
- Results stored in their database
- You trust Docker Hub's scanner version and scan integrity
**ATCR hold model:**
- Hold scans image on their infrastructure
- Results stored in hold's embedded PDS
- You trust hold operator's scanner version and scan integrity
The security comes from **reproducibility** and **transparency**, not storage location:
- Anyone can re-scan the same digest and verify results
- Multiple holds scanning the same image provide independent verification
- Scanner version and scan timestamp are recorded in ATProto records
### Why Hold's PDS?
Scan results are stored in the **hold's embedded PDS** rather than the user's PDS: Scan results are stored in the **hold's embedded PDS** rather than the user's PDS:
**Advantages:** - No OAuth/service-token plumbing: the hold owns and signs its own records.
1. **No OAuth expiry issues**: Hold owns its PDS, no service tokens needed - Hold-scoped metadata (scanner version, scan time) stays with the operator.
2. **Hold-scoped metadata**: Scanner version, scan time, hold configuration - Different holds can independently scan the same image for cross-verification.
3. **Multiple perspectives**: Different holds can scan the same image independently - The user's PDS stays lean: SBOM and Grype JSON live in hold blob storage.
4. **Simpler auth**: Hold writes directly to its own PDS
5. **Keeps user PDS lean**: Potentially large SBOM data doesn't bloat user's repo
**Security properties:** The trust model is the same as Docker Hub: you trust the hold operator's scanner
- Same trust level as trusting hold to serve correct blobs version and scan integrity. The hold's DID signs the records, and anyone can re-scan a
- DID signatures prove which hold generated the SBOM digest to verify the result.
- Reproducible scans enable independent verification
- Multiple holds scanning same digest → compare results for tampering detection
## ORAS Manifest Format
SBOMs are stored as ORAS artifacts that reference their subject image using the OCI referrers specification.
### Example Manifest Record
```json
{
"$type": "io.atcr.manifest",
"repository": "alice/myapp",
"digest": "sha256:4a5e...",
"holdDid": "did:web:hold01.atcr.io",
"holdEndpoint": "https://hold01.atcr.io",
"schemaVersion": 2,
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"artifactType": "application/spdx+json",
"subject": {
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:abc123...",
"size": 1234
},
"config": {
"mediaType": "application/vnd.oci.empty.v1+json",
"digest": "sha256:44136f...",
"size": 2
},
"layers": [
{
"mediaType": "application/spdx+json",
"digest": "sha256:def456...",
"size": 5678,
"annotations": {
"org.opencontainers.image.title": "sbom.spdx.json"
}
}
],
"manifestBlob": {
"$type": "blob",
"ref": { "$link": "bafyrei..." },
"mimeType": "application/vnd.oci.image.manifest.v1+json",
"size": 789
},
"ownerDid": "did:plc:alice123",
"scannedAt": "2025-10-20T12:34:56.789Z",
"scannerVersion": "syft-v1.0.0",
"createdAt": "2025-10-20T12:34:56.789Z"
}
```
### Key Fields
- `artifactType`: Distinguishes SBOM artifact from regular image manifest
- `application/spdx+json` for SPDX format
- `application/vnd.cyclonedx+json` for CycloneDX format
- `subject`: Reference to the original image manifest
- `ownerDid`: DID of the image owner (for multi-tenant holds)
- `scannedAt`: ISO 8601 timestamp of when scan completed
- `scannerVersion`: Tool version for reproducibility tracking
### SBOM Blob
The actual SBOM document is stored as a blob in the hold's storage backend and referenced in the manifest's `layers` array. The blob contains the full SPDX or CycloneDX JSON document.
## Configuration ## Configuration
SBOM scanning is configured via environment variables on the hold service. ### Hold side
### Environment Variables The hold's scanner integration is configured under `scanner:` in the hold config
(`pkg/hold/config.go`). Env-var prefix is `HOLD_`.
```bash | YAML key | Env var | Default | Meaning |
# Enable SBOM scanning (opt-in) |--------------------------|-------------------------------|---------|---------|
HOLD_SBOM_ENABLED=true | `scanner.secret` | `HOLD_SCANNER_SECRET` | `""` | Shared secret a scanner must present (as `?secret=`) on the WebSocket. **Empty disables scanning entirely** — no scanner can connect and no jobs are dispatched. |
| `scanner.rescan_interval`| `HOLD_SCANNER_RESCAN_INTERVAL`| `168h` | Minimum interval between re-scans of the same manifest. When > 0 the hold runs proactive discovery + stale-scan loops. Set to `0` to disable proactive scanning (push-triggered scans still work). |
# Number of concurrent scan workers (default: 2) ```yaml
# Higher values = faster scanning, more CPU/memory usage # config-hold.yaml
HOLD_SBOM_WORKERS=4 scanner:
secret: "a-long-random-shared-secret"
# SBOM output format (default: spdx-json) rescan_interval: 168h
# Options: spdx-json, cyclonedx-json
HOLD_SBOM_FORMAT=spdx-json
# Future: Enable vulnerability scanning with Grype
# HOLD_VULN_ENABLED=true
``` ```
### Example Configuration Whether a push triggers an immediate scan is decided by the quota tier (see
[Scan-on-push tier gate](#scan-on-push-tier-gate)).
### Scanner side
The scanner is configured via Viper (`scanner/internal/config/config.go`); it accepts
a YAML file or pure env vars with the `SCANNER_` prefix. Run with
`SCANNER_HOLD_URL=... SCANNER_HOLD_SECRET=... atcr-scanner serve`.
| YAML key | Env var | Default | Meaning |
|---------------------|------------------------------|----------------------------------|---------|
| `hold.url` | `SCANNER_HOLD_URL` | — (**required**) | WebSocket URL of the hold, e.g. `ws://localhost:8080` or `wss://hold01.atcr.io`. `http(s)` is auto-converted to `ws(s)`. |
| `hold.secret` | `SCANNER_HOLD_SECRET` | — (**required**) | Must match the hold's `scanner.secret`. Sent as `?secret=`. |
| `scanner.workers` | `SCANNER_SCANNER_WORKERS` | `1` | Number of concurrent scan workers. |
| `scanner.queue_size`| `SCANNER_SCANNER_QUEUE_SIZE` | `100` | Max depth of the local priority queue. |
| `vuln.enabled` | `SCANNER_VULN_ENABLED` | `true` | Run Grype after Syft. When false, only the SBOM is produced (no counts). |
| `vuln.db_path` | `SCANNER_VULN_DB_PATH` | `/var/lib/atcr-scanner/vulndb` | Directory for the Grype vulnerability database. |
| `vuln.tmp_dir` | `SCANNER_VULN_TMP_DIR` | `/var/lib/atcr-scanner/tmp` | Directory for layer extraction and DB download. Also exported as `TMPDIR`; point it at a large partition, **not** tmpfs. |
| `vuln.max_image_size`| `SCANNER_VULN_MAX_IMAGE_SIZE`| `2147483648` (2 GiB) | Max total compressed image size. Larger images are skipped with an error. `0` = no limit. |
| `server.addr` | `SCANNER_SERVER_ADDR` | `:9090` | Listen address for the scanner's health endpoint. |
Both `hold.url` and `hold.secret` are required; `LoadConfig` errors out if either is
empty.
```bash ```bash
# .env.hold # Minimal scanner invocation (env-only)
HOLD_PUBLIC_URL=https://hold01.atcr.io SCANNER_HOLD_URL=wss://hold01.atcr.io \
S3_BUCKET=my-hold-blobs SCANNER_HOLD_SECRET=a-long-random-shared-secret \
AWS_ACCESS_KEY_ID=your-access-key ./bin/atcr-scanner serve
AWS_SECRET_ACCESS_KEY=your-secret-key
HOLD_OWNER=did:plc:xyz123
HOLD_DATABASE_DIR=/var/lib/atcr-hold
# Enable SBOM scanning
HOLD_SBOM_ENABLED=true
HOLD_SBOM_WORKERS=2
HOLD_SBOM_FORMAT=spdx-json
``` ```
## Scanning Workflow ## Scanning Workflow
### 1. Upload Completes ### 1. Push → scan-on-push tier gate
When a container image is successfully pushed to a hold: When an image is pushed and the manifest is recorded, the hold's OCI XRPC handler
(`pkg/hold/oci/xrpc.go`) decides whether to enqueue a scan. Multi-arch manifest lists
and artifacts with a `subject` (attestations) are skipped — they have no scannable
content. For everything else, the tier of the pusher decides:
``` - **Captain / owner**: always scanned.
1. Client: docker push atcr.io/alice/myapp:latest - **Quotas disabled** (`quotaMgr == nil` or quotas not enabled): all pushes scanned
2. AppView routes blobs to hold service (backwards compatible).
3. Hold receives multipart upload via XRPC - **Quotas enabled**: scanned only if the pusher's tier has `scan_on_push: true`.
4. Hold completes upload and stores blobs
5. Hold checks: HOLD_SBOM_ENABLED=true? In the default config (`pkg/hold/config.go`), `bosun` and `quartermaster` have
6. If yes: enqueue scan job (non-blocking) `scan_on_push: true`; `deckhand` does not. So a free-tier (deckhand) push is **not**
7. Upload completes immediately scanned on push — it gets picked up later by the proactive discovery loop.
### 2. Dispatch
The `ScanBroadcaster.Enqueue` inserts the job into the `scan_jobs` SQLite table
(status `pending`) and immediately tries to dispatch it round-robin to one of the
connected scanners. Jobs survive hold restarts. If no scanner is connected, the job
waits; newly connected scanners drain pending jobs. Assigned-but-unacked jobs time out
after 5 minutes and are re-dispatched; jobs stuck in `processing` for 10 minutes are
marked failed (scanner likely crashed).
### 3. Scan pipeline (scanner)
For each job (`scanner/internal/scan/worker.go`):
1. **Artifact-type check** — if `config.mediaType` is in `unscannableConfigTypes` the
job returns a `SkipError` and the scanner sends a `skipped` message (see below).
2. **Size check** — if total compressed size exceeds `vuln.max_image_size`, the job
fails.
3. **Build OCI layout** — layers are fetched from the hold via presigned URLs and
assembled into an OCI image layout in `vuln.tmp_dir`.
4. **Syft** — generates the SBOM and encodes it to SPDX JSON.
5. **Grype** (if `vuln.enabled`) — scans the SBOM, producing the full JSON report and
a severity summary (critical/high/medium/low/total).
The scanner then sends one of three messages back over the WebSocket: `result`
(SBOM + optional vuln report + summary), `error`, or `skipped` (with a reason).
### 4. Result storage (hold)
On `result` (`scan_broadcaster.go` `handleResult`):
1. Upload the SBOM bytes as a PDS blob (`application/spdx+json`).
2. Upload the Grype report as a PDS blob (`application/vnd.atcr.vulnerabilities+json`).
3. Create an `io.atcr.hold.scan` record (`CreateScanRecord`) keyed by the manifest
digest, referencing both blobs and carrying the severity counts.
4. Mark the `scan_jobs` row `completed`.
On `error`, a failed scan record is written (`NewFailedScanRecord`) and the job is
marked `failed`. On `skipped`, a skipped record is written (`NewSkippedScanRecord`)
and the job is marked `completed`.
## Scan Record Schema
Results are `io.atcr.hold.scan` records in the hold's embedded PDS
(`pkg/atproto/lexicon.go`, `ScanRecord`). The record key is the manifest digest hex
(without the `sha256:` prefix), so there is exactly one scan record per manifest and
re-scans upsert it.
```json
{
"$type": "io.atcr.hold.scan",
"manifest": "at://did:plc:alice123/io.atcr.manifest/abc123...",
"repository": "myapp",
"userDid": "did:plc:alice123",
"sbomBlob": {
"$type": "blob",
"ref": { "$link": "bafkrei..." },
"mimeType": "application/spdx+json",
"size": 51234
},
"vulnReportBlob": {
"$type": "blob",
"ref": { "$link": "bafkrei..." },
"mimeType": "application/vnd.atcr.vulnerabilities+json",
"size": 18567
},
"critical": 2,
"high": 15,
"medium": 42,
"low": 8,
"total": 67,
"scannerVersion": "atcr-scanner-v1.0.0",
"scannedAt": "2026-06-11T12:34:56Z",
"status": "ok",
"reason": ""
}
``` ```
### 2. Background Scanning | Field | Notes |
|------------------|-------|
| `manifest` | AT-URI of the scanned manifest in the user's PDS. |
| `userDid` | DID of the image owner. |
| `sbomBlob` | Reference to the SPDX-JSON SBOM in hold blob storage. Absent for failed/skipped scans. |
| `vulnReportBlob` | Reference to the full Grype JSON report. Absent if Grype disabled or scan failed/skipped. |
| `critical`/`high`/`medium`/`low`/`total` | Vulnerability counts from Grype. Zero on failed/skipped scans. |
| `scannerVersion` | Scanner identifier for reproducibility (currently `atcr-scanner-v1.0.0`). |
| `scannedAt` | RFC3339 scan completion timestamp. |
| `status` | `ok`, `failed`, or `skipped`. |
| `reason` | Populated for `failed` (error text) and `skipped` (why it was bypassed). |
Scan workers process jobs from the queue: ### Status field
``` | Status | Meaning | Rescan behavior |
1. Worker pulls job from queue |-------------|-------------------------------------------------------------------------|-----------------|
2. Extracts image layers from storage | `ok` (or empty) | Scanner produced an SBOM; counts and SBOM blob populated. | Re-scanned on the rescan interval (default 7d). |
3. Runs Syft on extracted filesystem | `failed` | Scanner ran but errored (network, OOM, parse failure). No SBOM/counts. | Re-scanned on the rescan interval — failures may be transient. |
4. Generates SBOM in configured format | `skipped` | Scanner intentionally bypassed the artifact (helm chart, in-toto, DSSE). `reason` explains why. | **Never re-queued.** Won't change without a code change in the scanner. |
5. Uploads SBOM blob to storage
6. Creates ORAS manifest record in hold's PDS
7. Job complete
```
### 3. Result Storage Records written before the `status` field existed have an empty status. The appview
treats empty + nil-blob + zero-count as failed (legacy fallback).
SBOM results are stored in two places:
1. **SBOM blob**: Full JSON document in hold's blob storage
2. **ORAS manifest**: Metadata record in hold's embedded PDS
- Collection: `io.atcr.manifest`
- Record key: SBOM manifest digest
- Contains reference to subject image
## Scan Record Status
Every scan attempt produces an `io.atcr.hold.scan` record. The `status` field
tells the appview how to render the result:
| Status | Meaning | Stale-loop behavior |
|-------------|--------------------------------------------------------------------------------------------------|--------------------------------------------------|
| `ok` (or empty) | Scanner produced an SBOM. Vulnerability counts populated; SBOM blob populated. | Re-scanned on the rescan interval (default 7d). |
| `failed` | Scanner ran but errored (network, OOM, parse failure). No SBOM, no counts. | Re-scanned on the rescan interval — failures may be transient. |
| `skipped` | Scanner intentionally bypassed the artifact (helm chart, in-toto attestation, DSSE envelope). The `reason` field explains why. | **Never re-queued.** A skipped record won't change without a code change in the scanner. |
Records written before the `status` field existed have an empty status. The
appview treats empty + nil-blob + zero-count as failed (legacy fallback).
### Unscannable artifact types ### Unscannable artifact types
@@ -222,395 +246,151 @@ The scanner skips artifacts whose config media type appears in
`unscannableConfigTypes` (`scanner/internal/scan/worker.go`). Currently: `unscannableConfigTypes` (`scanner/internal/scan/worker.go`). Currently:
- `application/vnd.cncf.helm.config.v1+json` — Helm charts. Rendered with a - `application/vnd.cncf.helm.config.v1+json` — Helm charts. Rendered with a
helm-aware digest page (`pkg/appview/handlers/digest.go`) that shows helm-aware digest page (`pkg/appview/handlers/digest.go`) that shows Chart.yaml
Chart.yaml metadata instead of layers / vulns / SBOM. metadata instead of layers / vulns / SBOM.
- `application/vnd.in-toto+json` — in-toto attestations. - `application/vnd.in-toto+json` — in-toto attestations.
- `application/vnd.dsse.envelope.v1+json` — DSSE envelopes (SLSA provenance). - `application/vnd.dsse.envelope.v1+json` — DSSE envelopes (SLSA provenance).
For these types the appview's vuln/SBOM tabs render For these types the appview's vuln/SBOM tabs render *"Vulnerability scanning isn't
*"Vulnerability scanning isn't applied to this artifact type."* — no retry hint. applied to this artifact type."* — no retry hint.
To add a new unscannable type: append the media type to To add a new unscannable type: append the media type to `unscannableConfigTypes`.
`unscannableConfigTypes`. Existing records won't auto-rewrite — run Existing records won't auto-rewrite — run the backfill tool (below) once to convert
`atcr-hold scan-backfill` once to convert any pre-existing failure records any pre-existing failure records into skipped records.
into skipped records (see below).
## Scheduling and Priority
### Scanner-side priority queue
Each scanner keeps a local priority heap (`scanner/internal/queue/priority_queue.go`).
Jobs are ordered by tier priority, FIFO within a tier (lower number = higher priority):
| Tier | Priority |
|-----------------|----------|
| `owner` | 0 |
| `quartermaster` | 1 |
| `bosun` | 2 |
| anything else (`deckhand`) | 3 |
So when a scanner has a backlog, owner and paid-tier jobs are processed before
free-tier ones.
### Hold-side dispatch
The hold dispatches jobs **round-robin** across connected scanners (no priority at the
hold level — that is the scanner's job). Each scanner pulls its assigned jobs into its
own priority queue. With multiple scanners, the competing-consumer pattern spreads load.
### Proactive scanning
When `scanner.rescan_interval > 0`, the hold runs three background loops:
- **Discovery loop**: every 4 hours (and on scanner reconnect), queries relays for
DIDs with `io.atcr.manifest` records, walks each user's PDS, and queues manifests
that belong to this hold but have no scan record yet. These are dispatched at the
`deckhand` tier.
- **Stale-scan loop**: walks the local scan records and re-queues any `ok`/`failed`
record older than `rescan_interval`. Skipped records are left alone.
- **Dispatch loop**: drains the unscanned queue (higher priority) before the stale
queue, throttled to one proactive job at a time so push-triggered scans aren't
starved.
## Accessing Results
There is **no** `io.atcr.hold.getSBOM` XRPC endpoint. Results are read directly from
the hold's PDS using standard ATProto XRPC, and the appview UI wraps these calls.
### From the AppView web UI
The appview exposes HTMX endpoints that render scan data on repository/digest pages
(`pkg/appview/routes/routes.go`, handlers in `pkg/appview/handlers/`):
- `GET /api/scan-result` — vulnerability badge for a digest (`scan_result.go`).
- `GET /api/scan-results` — batch badges for a tag list (`scan_result.go`).
- `GET /api/vuln-details` — full vulnerability detail modal (`vuln_details.go`).
- `GET /api/sbom-details` — SBOM summary modal (`sbom_details.go`).
- `GET /api/scan-download?digest=...&holdEndpoint=...&type=sbom|vuln` — downloads the
raw SBOM or Grype JSON as a file (`scan_download.go`).
These handlers resolve the hold, fetch the `io.atcr.hold.scan` record, and pull the
SBOM/vuln blobs.
### Directly from the hold's PDS
The appview handlers do exactly this under the hood:
```bash
# 1. Fetch the scan record (rkey = manifest digest hex, no "sha256:" prefix)
curl "https://hold01.atcr.io/xrpc/com.atproto.repo.getRecord?\
repo=did:web:hold01.atcr.io&\
collection=io.atcr.hold.scan&\
rkey=abc123..."
# Response value contains sbomBlob.ref.$link, vulnReportBlob.ref.$link, and counts.
# 2. Download the SBOM blob by its CID
curl "https://hold01.atcr.io/xrpc/com.atproto.sync.getBlob?\
did=did:web:hold01.atcr.io&\
cid=bafkrei..." > sbom.spdx.json
# 3. Scan locally with another tool if desired
grype sbom:./sbom.spdx.json
osv-scanner --sbom sbom.spdx.json
```
You can also list all scan records on a hold via
`com.atproto.repo.listRecords?repo=<holdDid>&collection=io.atcr.hold.scan`.
## Backfill and Rescan
### Rescans
Re-scanning is automatic when `scanner.rescan_interval > 0` — the stale-scan loop
re-queues records older than the interval (default 7 days). Failed scans are retried;
skipped scans are not.
### Backfill tool ### Backfill tool
`atcr-hold scan-backfill --config <path>` walks every scan record on the `atcr-hold scan-backfill --config <path>` walks every `io.atcr.hold.scan` record and
hold and rewrites legacy ones (empty status + nil blob + zero counts) using rewrites legacy ones (empty status + nil SBOM blob + zero counts) by assigning a status
the manifest's layer media types as a signal: from the manifest's layer media types:
- Layer media type contains `helm.chart.content`, `in-toto`, or - Layer media type contains `helm.chart.content`, `in-toto`, or `dsse.envelope`
`dsse.envelope` → `status="skipped"`. → `status="skipped"`.
- Otherwise → `status="failed"`. - Otherwise → `status="failed"`.
The tool is idempotent and preserves the original `scannedAt`, so it can be The tool is idempotent and preserves each record's original `scannedAt`. It opens the
re-run safely. Run once per hold after upgrading. hold's CAR store directly, so the hold service must be **stopped** first (the embedded
PDS holds an exclusive lock). For zero-downtime backfill on a running hold, use the
## Accessing SBOMs admin endpoint `POST /admin/api/scan-backfill` instead.
Multiple methods for discovering and retrieving SBOM data. ## Troubleshooting
### 1. XRPC Query Endpoint - **No scans happening at all.** Check that `scanner.secret` is set on the hold (empty
disables scanning) and that a scanner is connected. Scanner connection failures log
Query for SBOMs by image digest: `dial failed` / `WebSocket read error`.
- **Scanner connects then immediately disconnects.** Usually a secret mismatch —
```bash `SCANNER_HOLD_SECRET` must equal the hold's `scanner.secret`.
# Get SBOM for a specific image - **Free-tier pushes never get scanned on push.** Expected: `deckhand` has
curl "https://hold01.atcr.io/xrpc/io.atcr.hold.getSBOM?\ `scan_on_push: false` by default. They are picked up by the discovery loop instead
digest=sha256:abc123&\ (requires `rescan_interval > 0`).
ownerDid=did:plc:alice123&\ - **Large images skipped.** Total compressed size exceeds `vuln.max_image_size`
repository=alice/myapp" (2 GiB default). Raise it or set `0` for no limit.
- **Layer extraction or Grype DB download fails mid-process.** `vuln.tmp_dir` is too
# Response: ORAS manifest JSON small or on tmpfs. Point it at a large persistent partition; the scanner sets
{ `TMPDIR` to this directory.
"manifest": { - **SBOM present but no vulnerability counts.** `vuln.enabled` is false on the scanner,
"schemaVersion": 2, or the Grype DB failed to initialize (check startup logs).
"mediaType": "application/vnd.oci.image.manifest.v1+json", - **Helm/attestation artifacts show "scanning isn't applied".** Expected — these are
"artifactType": "application/spdx+json", in `unscannableConfigTypes` and recorded as `skipped`.
"subject": { "digest": "sha256:abc123...", ... },
"layers": [ { "digest": "sha256:def456...", ... } ]
},
"scannedAt": "2025-10-20T12:34:56.789Z",
"scannerVersion": "syft-v1.0.0"
}
```
### 2. ATProto Repository Queries
Use standard ATProto XRPC to list all SBOMs:
```bash
# List all SBOM manifests in hold's PDS
curl "https://hold01.atcr.io/xrpc/com.atproto.repo.listRecords?\
repo=did:web:hold01.atcr.io&\
collection=io.atcr.manifest"
# Filter by artifactType (requires AppView indexing)
# Returns all SBOM artifacts
```
### 3. Direct SBOM Blob Download
Download the full SBOM JSON file:
```bash
# Get SBOM blob CID from manifest layers[0].digest
SBOM_DIGEST="sha256:def456..."
# Request presigned download URL
curl "https://hold01.atcr.io/xrpc/com.atproto.sync.getBlob?\
did=did:web:hold01.atcr.io&\
cid=$SBOM_DIGEST"
# Response: presigned S3 URL or direct blob
{
"url": "https://s3.amazonaws.com/bucket/blob?signature=...",
"expiresAt": "2025-10-20T12:49:56Z"
}
# Download SBOM JSON
curl "$URL" > sbom.spdx.json
```
### 4. ORAS CLI Integration
Use the ORAS CLI to discover and pull SBOMs:
```bash
# Discover referrers (SBOMs) for an image
oras discover atcr.io/alice/myapp:latest
# Output shows SBOM artifacts:
# digest: sha256:abc123...
# referrers:
# - artifactType: application/spdx+json
# digest: sha256:4a5e...
# Pull SBOM artifact
oras pull atcr.io/alice/myapp@sha256:4a5e...
# Downloads sbom.spdx.json to current directory
```
### 5. AppView Web UI (Future)
Future enhancement: AppView web interface will display SBOM information on repository pages:
- Link to SBOM JSON download
- Vulnerability count (if Grype enabled)
- Scanner version and scan timestamp
- Comparison across multiple holds
## Tool Integration
### SPDX/CycloneDX Tools
Any tool that understands SPDX or CycloneDX formats can consume the SBOMs:
**Example tools:**
- [OSV Scanner](https://github.com/google/osv-scanner) - Vulnerability scanning
- [Grype](https://github.com/anchore/grype) - Vulnerability scanning
- [Dependency-Track](https://dependencytrack.org/) - Software composition analysis
- [SBOM Quality Score](https://github.com/eBay/sbom-scorecard) - SBOM completeness
**Usage:**
```bash
# Download SBOM
curl "https://hold01.atcr.io/xrpc/io.atcr.hold.getSBOM?..." | \
jq -r '.manifest.layers[0].digest' | \
# ... fetch blob ... > sbom.spdx.json
# Scan with OSV
osv-scanner --sbom sbom.spdx.json
# Scan with Grype
grype sbom:./sbom.spdx.json
```
### OCI Registry API
ORAS manifests are fully OCI-compliant and discoverable via standard registry APIs:
```bash
# Discover referrers for an image
curl -H "Accept: application/vnd.oci.image.index.v1+json" \
"https://atcr.io/v2/alice/myapp/referrers/sha256:abc123"
# Returns referrers index with SBOM manifests
{
"schemaVersion": 2,
"mediaType": "application/vnd.oci.image.index.v1+json",
"manifests": [
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:4a5e...",
"artifactType": "application/spdx+json"
}
]
}
```
### Programmatic Access
Use the ATProto SDK to query SBOMs:
```go
import "github.com/bluesky-social/indigo/atproto"
// List all SBOMs for a hold
records, err := client.RepoListRecords(ctx,
"did:web:hold01.atcr.io",
"io.atcr.manifest",
100, // limit
"", // cursor
)
// Filter for SBOM artifacts
for _, record := range records.Records {
manifest := record.Value.(ManifestRecord)
if manifest.ArtifactType == "application/spdx+json" {
// Process SBOM manifest
}
}
```
## Future Enhancements
### Vulnerability Scanning (Grype)
Add vulnerability scanning to SBOM generation:
```bash
# Configuration
HOLD_VULN_ENABLED=true
HOLD_VULN_DB_UPDATE_INTERVAL=24h
# Extended manifest with vulnerability count
{
"artifactType": "application/spdx+json",
"annotations": {
"io.atcr.vuln.critical": "2",
"io.atcr.vuln.high": "15",
"io.atcr.vuln.medium": "42",
"io.atcr.vuln.low": "8",
"io.atcr.vuln.scannedWith": "grype-v0.74.0",
"io.atcr.vuln.dbVersion": "2025-10-20"
}
}
```
### Multi-Scanner Support (Trivy)
Support multiple scanner backends:
```bash
HOLD_SBOM_SCANNER=trivy # syft (default), trivy, grype
HOLD_TRIVY_SCAN_TYPE=os,library,config,secret
```
### Multi-Hold Verification
Compare SBOMs from different holds for the same image:
```bash
# Alice pushes to hold1 and hold2
docker push atcr.io/alice/myapp:latest
# Both holds scan independently
# Compare results:
atcr-cli compare-sboms \
--image atcr.io/alice/myapp:latest \
--holds hold1.atcr.io,hold2.atcr.io
# Output: Package count differences, version mismatches, etc.
```
### Signature Verification (Cosign)
Sign SBOMs with Sigstore Cosign:
```bash
HOLD_SBOM_SIGN=true
HOLD_COSIGN_KEY_PATH=/var/lib/atcr/cosign.key
# SBOM artifacts get signed
# Verification:
cosign verify --key cosign.pub atcr.io/alice/myapp@sha256:4a5e...
```
## Security Considerations
### Reproducibility
SBOMs should be reproducible for the same image digest:
**Best practices:**
- Pin scanner versions in production holds
- Record scanner version in manifest annotations
- Document vulnerability database versions
- Re-scan periodically to catch new CVEs
**Validation:**
```bash
# Compare SBOMs from different holds
diff <(curl hold1/sbom.json | jq -S) \
<(curl hold2/sbom.json | jq -S)
# Differences indicate:
# - Different scanner versions
# - Different scan times (new CVEs discovered)
# - Potential tampering (investigate)
```
### Multiple Hold Verification
Running multiple holds provides defense in depth:
1. User pushes to hold1 (uses hold1 by default)
2. User also pushes to hold2 (backup/verification)
3. Both holds scan independently
4. Compare SBOM results:
- Similar results = confidence in accuracy
- Divergent results = investigate discrepancy
### Transparency
Hold operators should publish scanning policies:
- Scanner version and update schedule
- Vulnerability database update frequency
- SBOM format and schema version
- Data retention policies
### Trust Anchors
Users can verify scanner integrity:
1. **Scanner version**: Check `scannerVersion` field matches expected version
2. **DID signature**: ATProto record signed by hold's DID
3. **Timestamp**: Check `scannedAt` for stale scans
4. **Reproducibility**: Re-scan locally and compare results
## Example Workflows
### Enable Scanning on Your Hold
```bash
# 1. Configure hold with SBOM enabled
cat > .env.hold <<EOF
HOLD_PUBLIC_URL=https://myhold.example.com
S3_BUCKET=my-blobs
AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key
HOLD_OWNER=did:plc:myid
# Enable SBOM scanning
HOLD_SBOM_ENABLED=true
HOLD_SBOM_WORKERS=2
HOLD_SBOM_FORMAT=spdx-json
EOF
# 2. Start hold service
./bin/atcr-hold
# 3. Push an image
docker push atcr.io/alice/myapp:latest
# 4. Wait for background scan (check logs)
# 2025-10-20T12:34:56Z INFO Scanning image sha256:abc123...
# 2025-10-20T12:35:12Z INFO SBOM generated sha256:def456...
# 5. Query for SBOM
curl "https://myhold.example.com/xrpc/io.atcr.hold.getSBOM?..."
```
### Consume SBOMs in CI/CD
```yaml
# .github/workflows/security-scan.yml
name: Security Scan
on: push
jobs:
scan:
runs-on: ubuntu-latest
steps:
- name: Pull image
run: docker pull atcr.io/alice/myapp:latest
- name: Get SBOM from hold
run: |
IMAGE_DIGEST=$(docker inspect atcr.io/alice/myapp:latest \
--format='{{.RepoDigests}}')
curl "https://hold01.atcr.io/xrpc/io.atcr.hold.getSBOM?\
digest=$IMAGE_DIGEST&\
ownerDid=did:plc:alice123&\
repository=alice/myapp" \
-o sbom-manifest.json
SBOM_DIGEST=$(jq -r '.manifest.layers[0].digest' sbom-manifest.json)
curl "https://hold01.atcr.io/xrpc/com.atproto.sync.getBlob?\
did=did:web:hold01.atcr.io&\
cid=$SBOM_DIGEST" \
| jq -r '.url' | xargs curl -o sbom.spdx.json
- name: Scan with Grype
uses: anchore/scan-action@v3
with:
sbom: sbom.spdx.json
fail-build: true
severity-cutoff: high
```
## References ## References
- [ORAS Specification](https://oras.land/) - [Syft](https://github.com/anchore/syft)
- [OCI Artifacts](https://github.com/opencontainers/artifacts) - [Grype](https://github.com/anchore/grype)
- [SPDX Specification](https://spdx.dev/) - [SPDX Specification](https://spdx.dev/)
- [CycloneDX Specification](https://cyclonedx.org/) - [Hold XRPC Endpoints](./HOLD_XRPC_ENDPOINTS.md)
- [Syft Documentation](https://github.com/anchore/syft) - [Quotas](./QUOTAS.md)
- [ATProto Specification](https://atproto.com/) - [ATProto Specification](https://atproto.com/)
</content>
</invoke>
docs/SIGNATURE_INTEGRATION.md
-1210
View File
File diff suppressed because it is too large Load Diff
docs/TROUBLESHOOTING.md
+28 -24
View File
@@ -42,15 +42,13 @@ curl -I https://your-pds.example.com | grep -i date
docker logs atcr-appview 2>&1 | grep "Configured confidential OAuth client" docker logs atcr-appview 2>&1 | grep "Configured confidential OAuth client"
``` ```
Example log output: Example log output (the startup message logs `key_id` and `key_path` only; system time is not included):
``` ```
level=INFO msg="Configured confidential OAuth client" level=INFO msg="Configured confidential OAuth client" key_id=did:key:z... key_path=/var/lib/atcr/oauth/client.key
key_id=did:key:z...
system_time_unix=1731844215
system_time_rfc3339=2025-11-17T14:30:15Z
timezone=UTC
``` ```
To verify clock skew separately, compare the output of `date +%s` on the AppView host with the `Date:` header returned by the PDS (see step 3 above).
**Solution:** **Solution:**
1. **Enable NTP synchronization** (recommended): 1. **Enable NTP synchronization** (recommended):
@@ -155,9 +153,9 @@ docker logs atcr-appview 2>&1 | grep "use_dpop_nonce" | wc -l
timedatectl status timedatectl status
``` ```
3. Look for session lock acquisition in logs: 3. Look for session lock acquisition in logs (requires debug-level logging):
```bash ```bash
docker logs atcr-appview 2>&1 | grep "Acquired session lock" ATCR_LOG_LEVEL=debug docker logs atcr-appview 2>&1 | grep "Acquired session lock"
``` ```
**Solution:** **Solution:**
@@ -229,7 +227,7 @@ error_description: Client metadata endpoint returned 404
``` ```
**Root Cause:** **Root Cause:**
PDS cannot fetch OAuth client metadata from `{ATCR_BASE_URL}/client-metadata.json` PDS cannot fetch OAuth client metadata from `{ATCR_SERVER_BASE_URL}/client-metadata.json`
**Diagnosis:** **Diagnosis:**
@@ -243,16 +241,16 @@ PDS cannot fetch OAuth client metadata from `{ATCR_BASE_URL}/client-metadata.jso
docker logs atcr-appview 2>&1 | grep "client-metadata" docker logs atcr-appview 2>&1 | grep "client-metadata"
``` ```
3. Verify `ATCR_BASE_URL` is set correctly: 3. Verify `ATCR_SERVER_BASE_URL` is set correctly:
```bash ```bash
echo $ATCR_BASE_URL echo $ATCR_SERVER_BASE_URL
``` ```
**Solution:** **Solution:**
1. Ensure `ATCR_BASE_URL` matches your public URL: 1. Ensure `ATCR_SERVER_BASE_URL` matches your public URL:
```bash ```bash
export ATCR_BASE_URL=https://atcr.example.com export ATCR_SERVER_BASE_URL=https://atcr.example.com
``` ```
2. Verify reverse proxy (nginx, Caddy, etc.) routes `/.well-known/*` and `/client-metadata.json`: 2. Verify reverse proxy (nginx, Caddy, etc.) routes `/.well-known/*` and `/client-metadata.json`:
@@ -339,18 +337,24 @@ Slow Docker push/pull operations, high CPU usage on AppView
**Solution:** **Solution:**
1. For production, migrate to PostgreSQL (recommended): 1. For production, enable libSQL embedded-replica mode to offload reads to a remote sync target (Turso cloud or self-hosted libsql-server):
```bash ```bash
export ATCR_UI_DATABASE_TYPE=postgres export ATCR_UI_LIBSQL_SYNC_URL=libsql://your-db.turso.io
export ATCR_UI_DATABASE_URL=postgresql://user:pass@localhost/atcr export ATCR_UI_LIBSQL_AUTH_TOKEN=your-auth-token
export ATCR_UI_LIBSQL_SYNC_INTERVAL=60s # optional, default 60s
``` ```
2. Or increase SQLite busy timeout: Or in YAML (`config-appview.yaml`):
```go ```yaml
// In code: db.SetMaxOpenConns(1) for SQLite ui:
libsql_sync_url: libsql://your-db.turso.io
libsql_auth_token: your-auth-token
libsql_sync_interval: 60s
``` ```
3. Vacuum the database to reclaim space: PostgreSQL is not supported; the AppView uses SQLite/libSQL only.
2. Vacuum the database to reclaim space:
```bash ```bash
sqlite3 /var/lib/atcr/ui.db "VACUUM;" sqlite3 /var/lib/atcr/ui.db "VACUUM;"
``` ```
@@ -380,9 +384,9 @@ docker logs atcr-appview 2>&1 | grep "OAuth callback failed"
docker logs atcr-appview 2>&1 | grep "OAuth authentication failed during service token request" docker logs atcr-appview 2>&1 | grep "OAuth authentication failed during service token request"
``` ```
**Clock diagnostics:** **Clock diagnostics (confidential OAuth client startup):**
```bash ```bash
docker logs atcr-appview 2>&1 | grep "system_time" docker logs atcr-appview 2>&1 | grep "Configured confidential OAuth client"
``` ```
**DPoP nonce issues:** **DPoP nonce issues:**
@@ -410,7 +414,7 @@ curl http://localhost:8080/.well-known/did.json
If issues persist after following this guide: If issues persist after following this guide:
1. **Check GitHub Issues**: https://github.com/ericvolp12/atcr/issues 1. **Check Issues**: https://tangled.org/evan.jarrett.net/at-container-registry/issues
2. **Collect logs**: Include output from `docker logs` for AppView and Hold services 2. **Collect logs**: Include output from `docker logs` for AppView and Hold services
3. **Include diagnostics**: 3. **Include diagnostics**:
- `timedatectl status` output - `timedatectl status` output
@@ -428,6 +432,6 @@ If issues persist after following this guide:
| `use_dpop_nonce` | OAuth/DPoP | Concurrent requests or clock skew | Fix NTP, wait for auto-retry | | `use_dpop_nonce` | OAuth/DPoP | Concurrent requests or clock skew | Fix NTP, wait for auto-retry |
| `server_error` (500) | PDS | PDS internal error | Check PDS logs | | `server_error` (500) | PDS | PDS internal error | Check PDS logs |
| `invalid_grant` | OAuth | Expired auth code | Retry OAuth flow | | `invalid_grant` | OAuth | Expired auth code | Retry OAuth flow |
| `unauthorized_client` | OAuth | Client metadata unreachable | Check ATCR_BASE_URL and firewall | | `unauthorized_client` | OAuth | Client metadata unreachable | Check ATCR_SERVER_BASE_URL and firewall |
| `RecordNotFound` | ATProto | Manifest doesn't exist | Verify repository name | | `RecordNotFound` | ATProto | Manifest doesn't exist | Verify repository name |
| Connection refused | Hold/S3 | Network/credentials | Check S3 config and connectivity | | Connection refused | Hold/S3 | Network/credentials | Check S3 config and connectivity |
docs/appview.md
+24 -18
View File
@@ -71,22 +71,25 @@ This creates a fully-commented YAML file with all available options and their de
### 3. Set the required field ### 3. Set the required field
Edit `config-appview.yaml` and set `server.default_hold_did` to your hold service's DID: Edit `config-appview.yaml` and set `server.managed_holds` to the list of hold DIDs this AppView manages. The first entry is used as the default blob-storage hold when a user has no hold selected:
```yaml ```yaml
server: server:
default_hold_did: "did:web:127.0.0.1:8080" # local dev managed_holds:
# default_hold_did: "did:web:hold01.example.com" # production - "did:web:127.0.0.1:8080" # local dev
# managed_holds:
# - "did:web:hold01.example.com" # production
``` ```
This is the **only required configuration field**. To find a hold's DID, visit its `/.well-known/did.json` endpoint. This is the **only required configuration field**. To find a hold's DID, visit its `/.well-known/did.json` endpoint. The env var equivalent is `ATCR_SERVER_MANAGED_HOLDS` (comma-separated list of DIDs).
For production, also set your public URL: For production, also set your public URL:
```yaml ```yaml
server: server:
base_url: "https://registry.example.com" base_url: "https://registry.example.com"
default_hold_did: "did:web:hold01.example.com" managed_holds:
- "did:web:hold01.example.com"
``` ```
### 4. Run ### 4. Run
@@ -124,7 +127,7 @@ AppView uses YAML configuration with environment variable overrides. The generat
YAML paths map to env vars with `ATCR_` prefix and `_` separators: YAML paths map to env vars with `ATCR_` prefix and `_` separators:
``` ```
server.default_hold_did → ATCR_SERVER_DEFAULT_HOLD_DID server.managed_holds → ATCR_SERVER_MANAGED_HOLDS (comma-separated)
server.base_url → ATCR_SERVER_BASE_URL server.base_url → ATCR_SERVER_BASE_URL
ui.database_path → ATCR_UI_DATABASE_PATH ui.database_path → ATCR_UI_DATABASE_PATH
jetstream.backfill_enabled → ATCR_JETSTREAM_BACKFILL_ENABLED jetstream.backfill_enabled → ATCR_JETSTREAM_BACKFILL_ENABLED
@@ -134,7 +137,7 @@ jetstream.backfill_enabled → ATCR_JETSTREAM_BACKFILL_ENABLED
| Section | Purpose | Notes | | Section | Purpose | Notes |
|---------|---------|-------| |---------|---------|-------|
| `server` | Listen address, public URL, hold DID, OAuth key, branding | Only `default_hold_did` is required | | `server` | Listen address, public URL, managed holds, branding | Only `managed_holds` is required |
| `ui` | Database path, theme, libSQL sync | All have defaults; auto-creates DB on first run | | `ui` | Database path, theme, libSQL sync | All have defaults; auto-creates DB on first run |
| `auth` | JWT signing key/cert paths | Auto-generated on first run | | `auth` | JWT signing key/cert paths | Auto-generated on first run |
| `jetstream` | Real-time ATProto event streaming, backfill sync | Runs automatically; backfill enabled by default | | `jetstream` | Real-time ATProto event streaming, backfill sync | Runs automatically; backfill enabled by default |
@@ -144,16 +147,16 @@ jetstream.backfill_enabled → ATCR_JETSTREAM_BACKFILL_ENABLED
### Auto-generated files ### Auto-generated files
On first run, AppView auto-generates these under `/var/lib/atcr/`: On first run (and each boot), AppView auto-generates these under `/var/lib/atcr/`:
| File | Purpose | | File | Purpose |
|------|---------| |------|---------|
| `ui.db` | SQLite database (OAuth sessions, stars, pull counts, device approvals) | | `ui.db` | SQLite database (OAuth sessions, stars, pull counts, device approvals, crypto keys) |
| `auth/private-key.pem` | RSA private key for signing registry JWTs | | `auth/private-key.crt` | X.509 certificate regenerated every boot from the RSA key stored in `ui.db` |
| `auth/private-key.crt` | X.509 certificate for JWT verification |
| `oauth/client.key` | P-256 private key for OAuth client authentication |
**Persist `/var/lib/atcr/` across restarts.** Losing the auth keys invalidates all active sessions; losing the database loses OAuth state and UI data. The RSA key (for registry JWT signing) and the P-256 key (for OAuth client authentication) are both stored in the `crypto_keys` table inside `ui.db` and generated on first run. The cert file is derived from the DB key on every boot so the distribution library can read it from disk.
**Persist `ui.db` across restarts.** Losing the database loses both crypto keys (invalidating all active sessions) as well as OAuth state and UI data. The `auth/` directory is transient and recreated automatically each boot.
## Deployment ## Deployment
@@ -166,7 +169,7 @@ On first run, AppView auto-generates these under `/var/lib/atcr/`:
**Port:** `5000` (HTTP) **Port:** `5000` (HTTP)
**Volume:** `/var/lib/atcr` (auth keys, database, OAuth keys) **Volume:** `/var/lib/atcr` (database; cert is regenerated each boot)
**Health check:** `GET /health` returns `{"status":"ok"}` **Health check:** `GET /health` returns `{"status":"ok"}`
@@ -195,7 +198,7 @@ services:
command: ["serve", "--config", "/config.yaml"] command: ["serve", "--config", "/config.yaml"]
environment: environment:
ATCR_SERVER_BASE_URL: https://registry.example.com ATCR_SERVER_BASE_URL: https://registry.example.com
ATCR_SERVER_DEFAULT_HOLD_DID: did:web:hold.example.com ATCR_SERVER_MANAGED_HOLDS: did:web:hold.example.com
volumes: volumes:
- ./config-appview.yaml:/config.yaml:ro - ./config-appview.yaml:/config.yaml:ro
- atcr-appview-data:/var/lib/atcr - atcr-appview-data:/var/lib/atcr
@@ -224,7 +227,8 @@ Open to all ATProto users:
# config-appview.yaml # config-appview.yaml
server: server:
base_url: "https://registry.example.com" base_url: "https://registry.example.com"
default_hold_did: "did:web:hold01.example.com" managed_holds:
- "did:web:hold01.example.com"
jetstream: jetstream:
backfill_enabled: true backfill_enabled: true
``` ```
@@ -239,7 +243,8 @@ Restricted to crew members only:
# config-appview.yaml # config-appview.yaml
server: server:
base_url: "https://registry.internal.example.com" base_url: "https://registry.internal.example.com"
default_hold_did: "did:web:hold.internal.example.com" managed_holds:
- "did:web:hold.internal.example.com"
``` ```
The linked hold service should have `server.public: false` and `registration.allow_all_crew: false`, with an explicit `registration.owner_did` set to the organization's DID. The linked hold service should have `server.public: false` and `registration.allow_all_crew: false`, with an explicit `registration.owner_did` set to the organization's DID.
@@ -250,7 +255,8 @@ The linked hold service should have `server.public: false` and `registration.all
# config-appview.yaml # config-appview.yaml
log_level: debug log_level: debug
server: server:
default_hold_did: "did:web:127.0.0.1:8080" managed_holds:
- "did:web:127.0.0.1:8080"
test_mode: true # allows HTTP for DID resolution test_mode: true # allows HTTP for DID resolution
``` ```
docs/hold.md
+3 -4
View File
@@ -90,7 +90,6 @@ docker run -d \
- **`/var/lib/atcr-hold`** — Persistent volume for the embedded PDS (carstore database + signing keys). Back this up. - **`/var/lib/atcr-hold`** — Persistent volume for the embedded PDS (carstore database + signing keys). Back this up.
- **Port 8080** — Default listen address. Put a reverse proxy (Caddy, nginx) in front for TLS. - **Port 8080** — Default listen address. Put a reverse proxy (Caddy, nginx) in front for TLS.
- The image is built `FROM scratch` — the binary includes SQLite statically linked. - The image is built `FROM scratch` — the binary includes SQLite statically linked.
- Optional: `docker build --build-arg BILLING_ENABLED=true` to include Stripe billing support.
## Configuration ## Configuration
@@ -123,15 +122,15 @@ See [BYOS.md](BYOS.md) for the full authorization model.
| Admin panel | Enabled | `admin.enabled` | Web UI for crew, settings, and storage management | | Admin panel | Enabled | `admin.enabled` | Web UI for crew, settings, and storage management |
| Quotas | Disabled | `quota.tiers` | Tier-based storage limits (e.g., deckhand=5GB, bosun=50GB) | | Quotas | Disabled | `quota.tiers` | Tier-based storage limits (e.g., deckhand=5GB, bosun=50GB) |
| Garbage collection | Disabled | `gc.enabled` | Nightly cleanup of orphaned blobs and records | | Garbage collection | Disabled | `gc.enabled` | Nightly cleanup of orphaned blobs and records |
| Vulnerability scanner | Disabled | `scanner.secret` | Requires separate scanner service; see [SBOM_SCANNING.md](SBOM_SCANNING.md) | | Vulnerability scanner | Disabled | `scanner.secret`, `scanner.rescan_interval` | Requires separate scanner service; see [SBOM_SCANNING.md](SBOM_SCANNING.md) |
| Billing (Stripe) | Disabled | Build flag + env | Build with `--build-arg BILLING_ENABLED=true`; see [BILLING.md](BILLING.md) | | Labeler | Disabled | `labeler.did`, `labeler.subscribe_url` | Consumes takedown labels from an ATProto labeler; purges affected records and gates GC blob cleanup |
| Bluesky posts | Disabled | `registration.enable_bluesky_posts` | Posts push notifications from hold's identity | | Bluesky posts | Disabled | `registration.enable_bluesky_posts` | Posts push notifications from hold's identity |
## Hold Identity ## Hold Identity
**did:web (default)** — Derived from `server.public_url` with zero setup. `https://hold.example.com` becomes `did:web:hold.example.com`. The DID document is served at `/.well-known/did.json`. Tied to domain ownership — if you lose the domain, you lose the identity. **did:web (default)** — Derived from `server.public_url` with zero setup. `https://hold.example.com` becomes `did:web:hold.example.com`. The DID document is served at `/.well-known/did.json`. Tied to domain ownership — if you lose the domain, you lose the identity.
**did:plc (portable)** — Set `database.did_method: plc` in config. Registered with plc.directory. Survives domain changes. Requires a rotation key (auto-generated at `{database.path}/rotation.key`). Use `database.did` to adopt an existing DID for recovery or migration. **did:plc (portable)** — Set `database.did_method: plc` in config. Registered with plc.directory. Survives domain changes. Requires a rotation key. If `database.rotation_key` is not set, a new K-256 key is generated **in memory only** and logged once via `slog.Warn` — it is never written to disk. You must copy it from the startup logs into `database.rotation_key` in your config immediately, or you will lose the ability to update or recover the DID. Only the signing key (`database.key_path`, default `{database.path}/signing.key`) is persisted to disk automatically. Use `database.did` to adopt an existing DID for recovery or migration.
## Verification ## Verification
docs/research/IMAGE_SIGNING.md
+564
View File
@@ -0,0 +1,564 @@
# Image Signing with ATProto (Research / POC)
> STATUS: RESEARCH / DESIGN PROPOSAL. Last reviewed 2026-06-11.
>
> **Nothing in this document is implemented.** As of this review there is no
> signature artifact creation, no OCI Referrers API endpoint in the AppView, no
> `cmd/atcr-verify` binary, no `pkg/verify` package, no hold-as-CA code, no
> `io.atcr.hold.coSignManifest` / `io.atcr.hold.reSignManifest` XRPC endpoints,
> and no `HOLD_COSIGN_*` configuration. The CLI examples, install instructions,
> `oras`/`cosign` command transcripts, Kubernetes manifests, and version output
> shown below are **illustrative of a proposed design**, not working features.
>
> This file consolidates what were previously six separate docs
> (`IMAGE_SIGNING.md`, `ATPROTO_SIGNATURES.md`, `SIGNATURE_INTEGRATION.md`,
> `HOLD_AS_CA.md`, `ATCR_VERIFY_CLI.md`, `INTEGRATION_STRATEGY.md`) into a single
> research note. Several of the originals presented features as shipped
> ("Available Now", install commands, version strings); those claims have been
> removed or relabeled as proposals. See "Corrections from review" at the end for
> the specific factual fixes applied during consolidation.
## Table of contents
1. [What is real today](#1-what-is-real-today)
2. [Proposed design: ORAS signature artifacts + Referrers API](#2-proposed-design-oras-signature-artifacts--referrers-api)
3. [Alternative proposal: Hold-as-CA (X.509)](#3-alternative-proposal-hold-as-ca-x509)
4. [Proposed tool: atcr-verify CLI](#4-proposed-tool-atcr-verify-cli)
5. [Integration notes (hypothetical examples)](#5-integration-notes-hypothetical-examples)
6. [Trust model and security considerations](#6-trust-model-and-security-considerations)
7. [Comparison with other signing solutions](#7-comparison-with-other-signing-solutions)
8. [Proposed implementation roadmap](#8-proposed-implementation-roadmap)
9. [Corrections from review](#9-corrections-from-review)
---
## 1. What is real today
This section describes mechanisms that genuinely exist in the protocol layer
ATCR builds on. It is the accurate core extracted from the original
`ATPROTO_SIGNATURES.md`.
### Manifests are signed by the user's PDS
When a manifest is stored in ATCR, it lands in the user's PDS as an
`io.atcr.manifest` record. ATProto's repository model means that record is part
of a commit that the PDS signs:
1. AppView stores the manifest as an `io.atcr.manifest` record in the user's PDS.
2. The PDS creates a repository commit containing the new record in its Merkle
Search Tree (MST).
3. The PDS signs that commit with the repository's signing key (ECDSA over the
secp256k1 / K-256 curve).
4. The signature lives in the commit object and becomes part of the user's
verifiable repository chain.
This is a real, existing property of ATProto repositories. It is **not** an
ATCR feature that needed to be built; it is inherited from the PDS.
### Signature algorithm
- Curve: secp256k1 (K-256), the same curve used by Bitcoin/Ethereum.
- Hash: SHA-256.
- Signing: serialize commit data as DAG-CBOR, hash with SHA-256, sign with the
K-256 private key. ATProto uses "low-S" signatures (per BIP-0062).
Note that ATProto repository keys are K-256. This is distinct from the **P-256
(ES256)** key the AppView uses for OAuth, and distinct from the P-256 curve that
Notation expects (see the Hold-as-CA section for why that matters).
### Public key distribution
Public keys are published in DID documents and obtained through DID resolution:
```
did:plc:alice123
-> Query PLC directory: https://plc.directory/did:plc:alice123
-> DID document verificationMethod (id "#atproto")
publicKeyMultibase: zQ3sh... (multibase base58btc, multicodec 0xE701 for K-256)
```
For `did:web` identities the document is served from the web origin instead of
the PLC directory.
### What can be verified today, and how
Given an image and the manifest record behind it, a verifier can prove the
manifest came from a specific DID and was not tampered with, using only existing
ATProto endpoints:
1. Resolve the image reference to a manifest digest.
2. Find the corresponding `io.atcr.manifest` record in the user's PDS
(`com.atproto.repo.getRecord` / `listRecords`).
3. Fetch the repository (`com.atproto.sync.getRepo`) and extract the signed
commit covering that record.
4. Resolve the DID to its public key from the DID document.
5. Verify the commit signature against the public key using a K-256 verifier
(e.g. `github.com/bluesky-social/indigo/atproto/crypto`).
Sketch of the verification core (existing libraries, no new ATCR code):
```go
import "github.com/bluesky-social/indigo/atproto/crypto"
// pubKey parsed from the DID document's publicKeyMultibase (K-256)
// commit parsed from the CAR returned by com.atproto.sync.getRepo
bytesToVerify := commit.Unsigned().BytesForSigning()
err := pubKey.Verify(bytesToVerify, commit.Sig)
```
This is the foundation every proposal below builds on. None of the bridging,
discovery, or tooling that would make this convenient for OCI consumers exists
yet.
---
## 2. Proposed design: ORAS signature artifacts + Referrers API
> Proposal. Not implemented. There is no Referrers API endpoint in the AppView
> and no code that creates signature artifacts.
The OCI ecosystem does not understand ATProto records. The proposal here is to
publish a small **ORAS signature artifact** that points at the existing ATProto
signature, so that standard OCI tooling can at least *discover* it via the OCI
Referrers API, and a custom verifier can then follow the pointer to the real
ATProto signature.
### Proposed artifact format
The signature artifact would be an OCI image manifest with a `subject` field
referencing the image manifest, an `artifactType` of
`application/vnd.atproto.signature.v1+json`, and a single layer carrying the
metadata blob:
```json
{
"schemaVersion": 2,
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"artifactType": "application/vnd.atproto.signature.v1+json",
"config": {
"mediaType": "application/vnd.oci.empty.v1+json",
"digest": "sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a",
"size": 2
},
"subject": {
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:<image-manifest-digest>",
"size": 1234
},
"layers": [
{
"mediaType": "application/vnd.atproto.signature.v1+json",
"digest": "sha256:<metadata-blob-digest>",
"size": 512,
"annotations": { "org.opencontainers.image.title": "atproto-signature.json" }
}
],
"annotations": {
"io.atcr.atproto.did": "did:plc:alice123",
"io.atcr.atproto.pds": "https://bsky.social",
"io.atcr.atproto.recordUri": "at://did:plc:alice123/io.atcr.manifest/<rkey>",
"io.atcr.atproto.commitCid": "bafyreih8...",
"io.atcr.atproto.signedAt": "2025-10-31T12:34:56.789Z",
"io.atcr.atproto.keyId": "did:plc:alice123#atproto"
}
}
```
The metadata blob would carry enough to find and verify the underlying ATProto
signature:
```json
{
"$type": "io.atcr.atproto.signature",
"version": "1.0",
"subject": { "digest": "sha256:<image-manifest-digest>", "mediaType": "application/vnd.oci.image.manifest.v1+json" },
"atproto": {
"did": "did:plc:alice123",
"handle": "alice.bsky.social",
"pdsEndpoint": "https://bsky.social",
"recordUri": "at://did:plc:alice123/io.atcr.manifest/<rkey>",
"recordCid": "bafyreig7...",
"commitCid": "bafyreih8...",
"commitRev": "3jzfkjqwdwa2a",
"signedAt": "2025-10-31T12:34:56.789Z"
},
"signature": {
"algorithm": "ECDSA-K256-SHA256",
"keyId": "did:plc:alice123#atproto",
"publicKeyMultibase": "zQ3shokFTS3brHcDQrn82RUDfCZESWL1ZdCEJwekUDdo1Ko4Z"
},
"verification": {
"method": "atproto-repo-commit",
"instructions": "Fetch repository commit from PDS and verify signature using public key from DID document"
}
}
```
### Proposed discovery via the Referrers API
If the AppView implemented the OCI Referrers API, discovery would look like:
```
GET /v2/<owner>/<repo>/referrers/sha256:<image-manifest-digest>
?artifactType=application/vnd.atproto.signature.v1+json
```
returning an OCI image index listing matching signature artifacts. This endpoint
does **not** exist yet.
### Why standard tools cannot verify these
Even with discovery in place, standard OCI tools (`cosign verify`, `notation
verify`) could not *verify* an ATProto signature: they expect their own
signature formats and trust models. They would only be able to *list* the
artifact. Verification would require a custom tool (see the proposed
`atcr-verify` CLI) or a plugin.
### Proposed storage approach
The proposal is for the AppView, after storing a manifest, to read back the
commit CID/revision, build the metadata blob, and create the ORAS artifact
linked via `subject`. This is design intent only; the code path does not exist.
> Correction (do not confuse with how SBOMs/scans work today): the original docs
> claimed signature artifacts would follow "the same pattern as SBOMs" and that
> SBOMs are stored as ORAS artifacts. That is wrong. SBOMs and scan results are
> stored as `io.atcr.hold.scan` ATProto records in the hold's CAR store (see
> `pkg/hold/pds/scan.go`, `CreateScanRecord` -> `repomgr.UpsertRecord`), with the
> SBOM carried as a blob reference inside the record. There is no ORAS/Referrers
> machinery behind scans. If signature artifacts were ever built, they would be
> new infrastructure, not a reuse of the scan pipeline.
---
## 3. Alternative proposal: Hold-as-CA (X.509)
> Alternative proposal, separate from the ORAS-artifact design above. Not
> implemented. There is no CA code, no `io.atcr.hold.coSignManifest` or
> `io.atcr.hold.reSignManifest` endpoint, and no `HOLD_COSIGN_*` / `HOLD_CA_*`
> configuration in the hold.
This is a distinct, optional design for environments that require standard X.509
PKI (for example, tools that only speak Notation/Notary v2). It deliberately
trades away decentralization, so it is presented as an alternative rather than
the recommended path.
### The problem it addresses
- ATProto repository signatures use K-256.
- Notation supports only P-256/P-384/P-521.
- A K-256 signature cannot be converted to a P-256 one (different curves), so a
second signature with a P-256 key would be required.
### The proposed mechanism
A hold would act as a Certificate Authority:
1. User pushes an image; the manifest is signed by the PDS with K-256 (as today).
2. The AppView asks the hold to co-sign (proposed `io.atcr.hold.coSignManifest`).
3. The hold verifies the ATProto K-256 signature is valid for the user's DID.
4. The hold mints an ephemeral P-256 key pair and issues a short-lived X.509
certificate (subject `CN=<did>`, SAN `URI:<did>`) signed by the hold's CA key.
5. The hold signs the manifest digest with the P-256 key and wraps it in a
Notation JWS envelope (with the cert chain in the `x5c` header).
6. The signature is published as an ORAS artifact
(`artifactType: application/vnd.cncf.notary.signature`).
Proposed certificate chain:
```
Hold Root CA (self-signed, P-256, ~10y, CA=true pathLen=1)
-> User certificate (CN=<did>, SAN URI:<did>, P-256, ~24h, Code Signing)
-> Manifest signature
```
### What is signed, clarified
> Correction: the original `HOLD_AS_CA.md` was internally inconsistent about what
> the P-256 signature covers. The intended design is that the hold signs a hash
> derived from the **manifest content/digest**, producing a Notation-style
> detached signature whose payload is the manifest descriptor. The illustrative
> Go in the original that did `SHA256([]byte(req.ManifestDigest))` (hashing the
> digest *string*) was a sketch, not a spec. A real implementation would sign
> over the manifest bytes / OCI descriptor per the Notation signing spec, not the
> ASCII of the digest string.
### Proposed (non-existent) configuration
These environment variables are **proposed names only**; none are read by the
hold today:
```
HOLD_COSIGN_ENABLED=true
HOLD_CA_CERT_PATH=/var/lib/atcr/hold/ca-certificate.pem
HOLD_CA_KEY_PATH=/var/lib/atcr/hold/ca-private-key.pem
HOLD_CERT_VALIDITY_HOURS=24
HOLD_OCSP_ENABLED=true
HOLD_CRL_ENABLED=true
```
### Why this is the non-default proposal
| Aspect | Hold-as-CA | ORAS-artifact + custom verifier |
|---|---|---|
| Standard tool compatibility | Notation works unchanged | Requires a custom verifier/plugin |
| Decentralization | Centralized (hold is the CA) | Decentralized (DID-based) |
| ATProto alignment | Against the model | Native |
| Signature reuse | Must re-sign with P-256 | Reuses existing K-256 commit signature |
| Compromise blast radius | Hold key compromise affects all users | Metadata only |
| Operational overhead | High (CA, CRL/OCSP, trust distribution) | Low |
Threat highlights for the CA design: compromise of the hold's CA private key is
catastrophic (an attacker could mint certificates for any DID); a malicious hold
operator could issue certificates without genuinely verifying the ATProto
signature. Mitigations discussed in the original (HSM-backed CA key, short cert
lifetimes, transparency logging, cross-checking against the ATProto signature)
remain valid design considerations but are unbuilt.
---
## 4. Proposed tool: atcr-verify CLI
> Proposal / specification. Not implemented. There is no `cmd/atcr-verify`
> directory and no `pkg/verify` package. There are no release binaries, no
> container image, and no version output. The commands, flags, and transcripts
> below describe a tool that does not exist yet.
A standalone verifier is the keystone of the decentralized (non-CA) approach: it
would discover the signature artifact, follow it to the ATProto record, resolve
the DID, fetch the commit, and verify the K-256 signature, then apply a trust
policy.
### Proposed verification flow
1. Resolve the image reference to a manifest digest.
2. Query the Referrers API for `application/vnd.atproto.signature.v1+json`
artifacts and fetch the metadata blob.
3. Resolve the DID to a public key.
4. Query the PDS for the manifest record and its commit.
5. Verify the K-256 commit signature against the public key.
6. Evaluate a trust policy (trusted DIDs, max age, minimum signatures).
### Proposed surface (illustrative only)
```bash
# proposed, not real
atcr-verify atcr.io/alice/myapp:latest
atcr-verify atcr.io/alice/myapp:latest --policy trust-policy.yaml
atcr-verify atcr.io/alice/myapp:latest --output json
atcr-verify atcr.io/alice/myapp:latest --quiet # exit code only
```
Proposed flags: `--policy`, `--output {text,json,quiet}`, `--cache-dir`,
`--no-cache`, `--timeout`, `--verbose`. Proposed exit codes: `0` verified,
`1` failed, `2` bad args, `3` network error, `4` policy violation.
Proposed subcommands: `verify` (default), `export` (write a bundle), `trust`
(manage trusted DIDs), `version`.
### Proposed trust policy format
```yaml
version: 1.0
defaultAction: enforce # enforce | audit | allow
requireSignature: true
policies:
- name: production-images
scope: "atcr.io/*/prod-*"
require:
signature: true
trustedDIDs: [did:plc:devops-team, did:plc:security-team]
minSignatures: 1
maxAge: 2592000 # 30 days, seconds
action: enforce
- name: dev-images
scope: "atcr.io/*/dev-*"
require:
signature: false
action: audit
trustedDIDs:
did:plc:devops-team:
name: "DevOps Team"
validFrom: "2024-01-01T00:00:00Z"
expiresAt: null
```
### Offline verification is a proposal, not a current capability
> Correction: the original docs contradicted each other on offline verification.
> One described full `--offline --bundle` support as a feature; another listed
> ATProto offline verification as "Limited" with a footnote that it "can be
> improved by embedding signature bytes in the ORAS blob". The accurate
> statement: **offline verification does not exist.** It is *possible in
> principle* if the signature artifact embeds the commit bytes and the relevant
> DID document so a verifier need not reach the PLC directory or PDS at run time.
> Treat any `--offline`/`export bundle` mention as a future design idea.
### Proposed library usage
If built, the verifier package would import as `atcr.io/pkg/verify` (the module
is `atcr.io`, not `github.com/atcr-io/atcr`). The original docs used the wrong
module path throughout; see corrections.
---
## 5. Integration notes (hypothetical examples)
> Everything in this section is hypothetical. The `atcr-verify` binary,
> container image, Referrers API, and signature artifacts they assume do not
> exist. The examples are retained as design sketches for the integration
> surfaces that would matter if the foundation were built. The Ratify and
> Gatekeeper plugin skeletons referenced here live under `examples/plugins/`
> and are likewise unbuilt scaffolding.
### Tool reality check
Standard OCI tools could, *given the proposed artifacts*, discover but not verify
ATProto signatures:
| Tool | Could discover | Could fetch | Could verify |
|---|---|---|---|
| `oras discover` | yes | - | no |
| `oras pull` / `crane manifest` | - | yes | no |
| `cosign tree` | yes (as artifacts) | - | no |
| `cosign verify` / `notation verify` | - | - | no (different format) |
| proposed `atcr-verify` | yes | yes | yes |
### Kubernetes (proposed)
Several admission-control paths were sketched, all depending on the proposed
verifier:
- A validating admission webhook that runs verification per container image and
rejects pods on failure.
- A Ratify verifier plugin (`CanVerify` on
`application/vnd.atproto.signature.v1+json`, then resolve DID, fetch commit,
verify K-256, check trust policy). Skeleton under
`examples/plugins/ratify-verifier/`.
- An OPA Gatekeeper external data provider that calls a verification service and
returns `verified: true/false` for Rego policies. Skeleton under
`examples/plugins/gatekeeper-provider/`.
- A Containerd 2.0 bindir image-verifier that shells out to `atcr-verify` at pull
time.
### CI/CD (proposed)
GitHub Actions / GitLab CI snippets would install the (nonexistent) `atcr-verify`
binary or use a (nonexistent) `atcr.io/atcr/verify` image and fail the build on a
bad signature. They are placeholders until the CLI exists.
---
## 6. Trust model and security considerations
This section is accurate at the conceptual level and applies to any design built
on ATProto signatures.
### What an ATProto signature proves
- Authenticity: the manifest record was committed by the DID owner's PDS.
- Integrity: the manifest content has not been altered since signing
(content-addressed; tampering changes the CID).
- Timestamp: when the commit was made.
### What it does not prove
- That the image is free of vulnerabilities (that is what scanning is for).
- That the DID owner is *authorized* to deploy anywhere.
- That the signing key was not compromised.
- That the PDS is honest about anything other than the signed bytes (it cannot
forge a signature without the private key, but availability and record
selection still depend on it).
### Trust dependencies
1. DID resolution returns the correct public key for the DID.
2. The PDS is reachable to fetch the commit (unless offline verification is
built per the proposal in section 4).
3. K-256 remains secure.
4. The verifier's trust policy lists only legitimately trusted DIDs.
### Threat sketches
- MITM on PDS queries: defeated by signature verification (attacker cannot forge
the signature) plus TLS.
- Compromised/malicious PDS serving fake manifests: signature verification fails.
- Key compromise: same posture as any PKI; rotate keys via DID document updates.
- Replay/rollback: check that the commit is in the current repository DAG and
inspect timestamps.
- DID takeover via rotation keys: serious but requires compromising rotation
keys, which is harder than the signing key.
---
## 7. Comparison with other signing solutions
| Feature | ATCR (ATProto) | Cosign (Sigstore) | Notation (Notary v2) |
|---|---|---|---|
| Signing | Inherent (PDS commit) | Manual or keyless | Manual |
| Keys | K-256 (secp256k1) | P-256 or RSA | P-256/P-384/P-521 |
| Trust | DID-based | OIDC + Fulcio CA | X.509 PKI |
| Storage of signature | ATProto commit (today); ORAS artifact (proposed) | OCI registry | OCI registry |
| Centralization | Decentralized | Centralized (Fulcio) | Configurable |
| Transparency log | ATProto firehose | Rekor | Configurable |
| Offline verification | Not built (possible per proposal) | No | Yes |
### Why not just use Cosign keyless?
ATProto and Cosign keyless use incompatible identity models. Cosign keyless
needs OIDC + Fulcio + Rekor + TUF; ATProto uses DPoP-bound OAuth and DID-based
PKI with no CA. Bridging would mean operating Fulcio/Rekor/TUF and an OIDC bridge
for ATProto OAuth. The ATProto-native approach reuses the existing commit
signature instead. (For tools that strictly require X.509, the Hold-as-CA
alternative in section 3 exists as a deliberate, centralized fallback.)
---
## 8. Proposed implementation roadmap
> Aspirational ordering, not a commitment. Effort estimates are from the
> original design docs and are unvalidated.
1. **Foundation**: build the `atcr-verify` CLI (K-256 commit verification, trust
policy, output formats) and the AppView OCI Referrers API + signature-artifact
creation. Without these two, nothing else is testable.
2. **Kubernetes**: OPA Gatekeeper external provider and Ratify verifier plugin.
3. **Runtime**: Containerd 2.0 bindir verifier.
4. **Optional/enterprise**: Hold-as-CA (only if X.509 compliance is actually
demanded), plus advanced features (transparency log, multi-signature,
offline bundles).
---
## 9. Corrections from review
The following factual problems in the original six docs were fixed during
consolidation:
- **Module path.** Originals used `github.com/atcr-io/atcr` for imports, `go
install`, `git clone`, and release URLs. The actual Go module is `atcr.io`
(see `go.mod`). The GitHub-style paths and release-download instructions were
removed or relabeled as hypothetical.
- **SBOM storage.** Originals claimed SBOMs are stored as ORAS artifacts and that
signature artifacts would "follow the same pattern as SBOMs." False. SBOMs and
scan results are `io.atcr.hold.scan` ATProto records in the hold's CAR store
(`pkg/hold/pds/scan.go`), with the SBOM as a blob reference inside the record.
There is no ORAS/Referrers pipeline for scans.
- **"Available Now" claims.** `IMAGE_SIGNING.md` listed ATProto signature
artifacts, ORAS artifacts, and the OCI Referrers API under "Available Now."
None of these exist. All such claims were removed; the only thing real today is
the PDS commit signature (section 1).
- **Fabricated CLI artifacts.** `ATCR_VERIFY_CLI.md` shipped install commands,
`docker pull` instructions, and a `version` output (`atcr-verify version
1.0.0`, a fake commit hash and build date). Removed; the CLI is a spec.
- **Offline verification contradiction.** Resolved: offline verification is a
proposal, not a current capability (section 4).
- **What the CA signs.** The Hold-as-CA Go sketch hashed the digest *string*; the
intended design signs over the manifest descriptor/bytes per Notation. Noted in
section 3.
- **Cross-references.** Links that pointed at the now-deleted sibling docs were
consolidated into this single file. External example scaffolding
(`examples/verification/`, `examples/plugins/`) still exists but is unbuilt;
its READMEs were updated to point here.
examples/plugins/README.md
+1 -4
View File
@@ -464,10 +464,7 @@ kubectl logs -n gatekeeper-system deployment/ratify
### Documentation ### Documentation
- [ATProto Signatures](../../docs/ATPROTO_SIGNATURES.md) - Technical deep-dive - [Image Signing (Research / POC)](../../docs/research/IMAGE_SIGNING.md) - Consolidated design notes; nothing implemented yet
- [Signature Integration](../../docs/SIGNATURE_INTEGRATION.md) - Tool-specific guides
- [Integration Strategy](../../docs/INTEGRATION_STRATEGY.md) - High-level overview
- [atcr-verify CLI](../../docs/ATCR_VERIFY_CLI.md) - CLI tool specification
### Examples ### Examples
examples/plugins/gatekeeper-provider/README.md
+1 -2
View File
@@ -491,8 +491,7 @@ The provider should only be accessible from Gatekeeper. Options:
- [Gatekeeper Documentation](https://open-policy-agent.github.io/gatekeeper/) - [Gatekeeper Documentation](https://open-policy-agent.github.io/gatekeeper/)
- [External Data Provider](https://open-policy-agent.github.io/gatekeeper/website/docs/externaldata/) - [External Data Provider](https://open-policy-agent.github.io/gatekeeper/website/docs/externaldata/)
- [ATCR Signature Integration](../../../docs/SIGNATURE_INTEGRATION.md) - [ATCR Image Signing (Research / POC)](../../../docs/research/IMAGE_SIGNING.md)
- [ATCR Integration Strategy](../../../docs/INTEGRATION_STRATEGY.md)
## Support ## Support
examples/plugins/ratify-verifier/README.md
+1 -2
View File
@@ -294,8 +294,7 @@ Consider implementing rate limiting for:
- [Ratify Documentation](https://ratify.dev/) - [Ratify Documentation](https://ratify.dev/)
- [Ratify Plugin Development](https://ratify.dev/docs/plugins/verifier/overview) - [Ratify Plugin Development](https://ratify.dev/docs/plugins/verifier/overview)
- [ATCR Signature Integration](../../../docs/SIGNATURE_INTEGRATION.md) - [ATCR Image Signing (Research / POC)](../../../docs/research/IMAGE_SIGNING.md)
- [ATCR Integration Strategy](../../../docs/INTEGRATION_STRATEGY.md)
## Support ## Support
examples/verification/README.md
+2 -3
View File
@@ -352,9 +352,8 @@ WantedBy=multi-user.target
## See Also ## See Also
- [ATProto Signatures](../../docs/ATPROTO_SIGNATURES.md) - Technical details - [Image Signing (Research / POC)](../../docs/research/IMAGE_SIGNING.md) - Consolidated design notes; nothing implemented yet
- [Signature Integration](../../docs/SIGNATURE_INTEGRATION.md) - Integration guide - [SBOM Scanning](../../docs/SBOM_SCANNING.md)
- [SBOM Scanning](../../docs/SBOM_SCANNING.md) - Similar ORAS pattern
## Support ## Support
pkg/appview/db/queries.go
+6
View File
@@ -574,6 +574,12 @@ func UpdateUserLastSeen(db DBTX, did string) error {
// UpdateUserHandle updates a user's handle when an identity change event is received // UpdateUserHandle updates a user's handle when an identity change event is received
// This is called when Jetstream receives an identity event indicating a handle change // This is called when Jetstream receives an identity event indicating a handle change
func UpdateUserHandle(db DBTX, did string, newHandle string) error { func UpdateUserHandle(db DBTX, did string, newHandle string) error {
// Never blank a cached handle. handle is NOT NULL UNIQUE, so writing "" both
// discards good data and collides the moment a second user does the same.
// Callers that see an empty/unresolved handle should leave the cache untouched.
if newHandle == "" {
return nil
}
_, err := db.Exec(` _, err := db.Exec(`
UPDATE users SET handle = ?, last_seen = ? WHERE did = ? UPDATE users SET handle = ?, last_seen = ? WHERE did = ?
`, newHandle, time.Now(), did) `, newHandle, time.Now(), did)
pkg/appview/handlers/diff.go
+251 -65
View File
@@ -4,6 +4,7 @@ import (
"fmt" "fmt"
"log/slog" "log/slog"
"net/http" "net/http"
"sort"
"strings" "strings"
"sync" "sync"
@@ -26,16 +27,27 @@ type VulnDiffEntry struct {
Vuln vulnMatch Vuln vulnMatch
} }
// SbomDiffEntry represents one package categorized by diff status.
type SbomDiffEntry struct {
Status string // "added", "removed", "changed", "unchanged"
Package sbomPackage // the "to" package (or the "from" package for "removed")
PrevVersion string // set for "changed" — the old version
}
// DiffSummary is the top-line summary for the banner and diff page. // DiffSummary is the top-line summary for the banner and diff page.
type DiffSummary struct { type DiffSummary struct {
SizeDelta int64 // bytes, positive = "to" is larger SizeDelta int64 // bytes, positive = "to" is larger
LayerCountFrom int LayerCountFrom int
LayerCountTo int LayerCountTo int
VulnFixedCount int VulnFixedCount int
VulnNewCount int VulnNewCount int
VulnFixedBySev vulnSummary VulnFixedBySev vulnSummary
VulnNewBySev vulnSummary VulnNewBySev vulnSummary
HasVulnData bool HasVulnData bool
PkgAddedCount int
PkgRemovedCount int
PkgChangedCount int
HasSbomData bool
} }
// layerKey returns the matching key for a layer — digest for real layers, command for empty layers. // layerKey returns the matching key for a layer — digest for real layers, command for empty layers.
@@ -151,8 +163,95 @@ func computeVulnDiff(fromMatches, toMatches []vulnMatch) []VulnDiffEntry {
return result return result
} }
// computeSbomDiff compares two package lists. Identical entries
// (name+type+version) match first as "unchanged"; leftovers pair by
// name+type as "changed" (version bump); the rest are "added"/"removed".
// Multiset counting keeps duplicate identical entries on one side from all
// matching a single entry on the other, and the name+type grouping keeps a
// deb package from pairing against a binary of the same name.
func computeSbomDiff(fromPkgs, toPkgs []sbomPackage) []SbomDiffEntry {
exactKey := func(p sbomPackage) string {
return p.Name + "\x00" + p.Type + "\x00" + p.Version
}
nameKey := func(p sbomPackage) string {
return p.Name + "\x00" + p.Type
}
// Exact pass: count from-side entries, consume per to-side match.
fromCounts := make(map[string]int, len(fromPkgs))
for _, p := range fromPkgs {
fromCounts[exactKey(p)]++
}
var result []SbomDiffEntry
var toLeft []sbomPackage
for _, p := range toPkgs {
if k := exactKey(p); fromCounts[k] > 0 {
fromCounts[k]--
result = append(result, SbomDiffEntry{Status: "unchanged", Package: p})
} else {
toLeft = append(toLeft, p)
}
}
fromGroups := make(map[string][]sbomPackage)
for _, p := range fromPkgs {
if k := exactKey(p); fromCounts[k] > 0 {
fromCounts[k]--
fromGroups[nameKey(p)] = append(fromGroups[nameKey(p)], p)
}
}
// Version-change pass: pair leftovers that share name+type. Versions are
// sorted lexically per group — determinism matters more than semver
// correctness for pairing multiple installed versions.
toGroups := make(map[string][]sbomPackage)
for _, p := range toLeft {
toGroups[nameKey(p)] = append(toGroups[nameKey(p)], p)
}
byVersion := func(pkgs []sbomPackage) {
sort.Slice(pkgs, func(i, j int) bool { return pkgs[i].Version < pkgs[j].Version })
}
for k, toGroup := range toGroups {
fromGroup := fromGroups[k]
byVersion(fromGroup)
byVersion(toGroup)
paired := min(len(fromGroup), len(toGroup))
for i := range paired {
result = append(result, SbomDiffEntry{Status: "changed", Package: toGroup[i], PrevVersion: fromGroup[i].Version})
}
for _, p := range toGroup[paired:] {
result = append(result, SbomDiffEntry{Status: "added", Package: p})
}
for _, p := range fromGroup[paired:] {
result = append(result, SbomDiffEntry{Status: "removed", Package: p})
}
delete(fromGroups, k)
}
for _, fromGroup := range fromGroups {
for _, p := range fromGroup {
result = append(result, SbomDiffEntry{Status: "removed", Package: p})
}
}
// Map iteration above is unordered — sort for stable rendering.
sort.Slice(result, func(i, j int) bool {
a, b := result[i].Package, result[j].Package
if a.Name != b.Name {
return a.Name < b.Name
}
if a.Type != b.Type {
return a.Type < b.Type
}
if a.Version != b.Version {
return a.Version < b.Version
}
return result[i].Status < result[j].Status
})
return result
}
// computeDiffSummary derives the top-line summary from layer and vuln diffs. // computeDiffSummary derives the top-line summary from layer and vuln diffs.
func computeDiffSummary(fromLayers, toLayers []LayerDetail, vulnDiff []VulnDiffEntry, hasVulnData bool) DiffSummary { func computeDiffSummary(fromLayers, toLayers []LayerDetail, vulnDiff []VulnDiffEntry, hasVulnData bool, sbomDiff []SbomDiffEntry, hasSbomData bool) DiffSummary {
var fromSize, toSize int64 var fromSize, toSize int64
for _, l := range fromLayers { for _, l := range fromLayers {
fromSize += l.Size fromSize += l.Size
@@ -166,6 +265,7 @@ func computeDiffSummary(fromLayers, toLayers []LayerDetail, vulnDiff []VulnDiffE
LayerCountFrom: len(fromLayers), LayerCountFrom: len(fromLayers),
LayerCountTo: len(toLayers), LayerCountTo: len(toLayers),
HasVulnData: hasVulnData, HasVulnData: hasVulnData,
HasSbomData: hasSbomData,
} }
for _, entry := range vulnDiff { for _, entry := range vulnDiff {
@@ -179,6 +279,17 @@ func computeDiffSummary(fromLayers, toLayers []LayerDetail, vulnDiff []VulnDiffE
} }
} }
for _, entry := range sbomDiff {
switch entry.Status {
case "added":
summary.PkgAddedCount++
case "removed":
summary.PkgRemovedCount++
case "changed":
summary.PkgChangedCount++
}
}
return summary return summary
} }
@@ -200,6 +311,23 @@ func addToSevCount(s *vulnSummary, severity string) {
s.Total++ s.Total++
} }
// sbomDiffStatus maps one side's SBOM fetch result to a status the template
// can branch on: "ok", "no-data" (no scan record or SBOM blob),
// "not-applicable" (scanner skipped this artifact type), or
// "hold-unreachable" (couldn't resolve or reach the hold).
func sbomDiffStatus(d *sbomDetailsData) string {
switch {
case d == nil:
return "hold-unreachable"
case d.Status == atproto.ScanStatusSkipped:
return "not-applicable"
case d.Error != "":
return "no-data"
default:
return "ok"
}
}
// ManifestDiffHandler renders the full diff page comparing two manifests. // ManifestDiffHandler renders the full diff page comparing two manifests.
type ManifestDiffHandler struct { type ManifestDiffHandler struct {
BaseUIHandler BaseUIHandler
@@ -263,6 +391,7 @@ func (h *ManifestDiffHandler) ServeHTTP(w http.ResponseWriter, r *http.Request)
manifest *db.ManifestWithMetadata manifest *db.ManifestWithMetadata
layers []LayerDetail layers []LayerDetail
vulnData *vulnDetailsData vulnData *vulnDetailsData
sbomData *sbomDetailsData
err error err error
} }
@@ -294,23 +423,39 @@ func (h *ManifestDiffHandler) ServeHTTP(w http.ResponseWriter, r *http.Request)
var layers []LayerDetail var layers []LayerDetail
var vulnData *vulnDetailsData var vulnData *vulnDetailsData
var sbomData *sbomDetailsData
hold, holdErr := ResolveHold(r.Context(), h.ReadOnlyDB, holdEndpoint) hold, holdErr := ResolveHold(r.Context(), h.ReadOnlyDB, holdEndpoint)
if holdErr == nil { if holdErr == nil {
config, err := holdclient.FetchImageConfig(r.Context(), hold.URL, layerDigest) // Parallelize the three hold fetches. They're independent and
if err == nil { // each takes a network round-trip; serial runs add up on slow links.
layers = buildLayerDetails(config.History, dbLayers) var fwg sync.WaitGroup
} else { fwg.Add(3)
layers = buildLayerDetails(nil, dbLayers) go func() {
} defer fwg.Done()
config, err := holdclient.FetchImageConfig(r.Context(), hold.URL, layerDigest)
vd := FetchVulnDetails(r.Context(), hold.DID, layerDigest) if err == nil {
vulnData = &vd layers = buildLayerDetails(config.History, dbLayers)
} else {
layers = buildLayerDetails(nil, dbLayers)
}
}()
go func() {
defer fwg.Done()
vd := FetchVulnDetails(r.Context(), hold.DID, layerDigest)
vulnData = &vd
}()
go func() {
defer fwg.Done()
sd := FetchSbomDetails(r.Context(), hold.DID, layerDigest)
sbomData = &sd
}()
fwg.Wait()
} else { } else {
layers = buildLayerDetails(nil, dbLayers) layers = buildLayerDetails(nil, dbLayers)
} }
return manifestData{manifest: m, layers: layers, vulnData: vulnData} return manifestData{manifest: m, layers: layers, vulnData: vulnData, sbomData: sbomData}
} }
// First fetch both top-level manifests to check for multi-arch // First fetch both top-level manifests to check for multi-arch
@@ -470,7 +615,19 @@ func (h *ManifestDiffHandler) ServeHTTP(w http.ResponseWriter, r *http.Request)
vulnDiff = computeVulnDiff(fromData.vulnData.Matches, toData.vulnData.Matches) vulnDiff = computeVulnDiff(fromData.vulnData.Matches, toData.vulnData.Matches)
} }
summary := computeDiffSummary(fromData.layers, toData.layers, vulnDiff, hasVulnData) // SBOM status mirrors the scan status, with one extra case: the scanner
// records status="skipped" for artifact types it doesn't scan, which the
// template surfaces as "not applicable" rather than "not scanned yet".
fromSbomStatus := sbomDiffStatus(fromData.sbomData)
toSbomStatus := sbomDiffStatus(toData.sbomData)
var sbomDiff []SbomDiffEntry
hasSbomData := fromSbomStatus == "ok" && toSbomStatus == "ok"
if hasSbomData {
sbomDiff = computeSbomDiff(fromData.sbomData.Packages, toData.sbomData.Packages)
}
summary := computeDiffSummary(fromData.layers, toData.layers, vulnDiff, hasVulnData, sbomDiff, hasSbomData)
// Determine tag labels // Determine tag labels
fromTag := fromDigest fromTag := fromDigest
@@ -495,6 +652,21 @@ func (h *ManifestDiffHandler) ServeHTTP(w http.ResponseWriter, r *http.Request)
} }
} }
// Split packages by status for template
var addedPackages, removedPackages, changedPackages, unchangedPackages []SbomDiffEntry
for _, entry := range sbomDiff {
switch entry.Status {
case "added":
addedPackages = append(addedPackages, entry)
case "removed":
removedPackages = append(removedPackages, entry)
case "changed":
changedPackages = append(changedPackages, entry)
case "unchanged":
unchangedPackages = append(unchangedPackages, entry)
}
}
title := fmt.Sprintf("Diff: %s → %s - %s/%s - %s", fromTag, toTag, owner.Handle, repo, h.ClientShortName) title := fmt.Sprintf("Diff: %s → %s - %s/%s - %s", fromTag, toTag, owner.Handle, repo, h.ClientShortName)
description := fmt.Sprintf("Comparing %s to %s in %s/%s", fromTag, toTag, owner.Handle, repo) description := fmt.Sprintf("Comparing %s to %s in %s/%s", fromTag, toTag, owner.Handle, repo)
meta := NewPageMeta(title, description). meta := NewPageMeta(title, description).
@@ -503,52 +675,66 @@ func (h *ManifestDiffHandler) ServeHTTP(w http.ResponseWriter, r *http.Request)
data := struct { data := struct {
PageData PageData
Meta *PageMeta Meta *PageMeta
Owner *db.User Owner *db.User
Repository string Repository string
FromManifest *db.ManifestWithMetadata FromManifest *db.ManifestWithMetadata
ToManifest *db.ManifestWithMetadata ToManifest *db.ManifestWithMetadata
FromTag string FromTag string
ToTag string ToTag string
Summary DiffSummary Summary DiffSummary
LayerDiff []LayerDiffEntry LayerDiff []LayerDiffEntry
FixedVulns []vulnMatch FixedVulns []vulnMatch
NewVulns []vulnMatch NewVulns []vulnMatch
UnchangedVulns []vulnMatch UnchangedVulns []vulnMatch
HasVulnData bool HasVulnData bool
FromScanStatus string FromScanStatus string
ToScanStatus string ToScanStatus string
FromFailed bool AddedPackages []SbomDiffEntry
ToFailed bool RemovedPackages []SbomDiffEntry
IsMultiArch bool ChangedPackages []SbomDiffEntry
CommonPlatforms []db.PlatformInfo UnchangedPackages []SbomDiffEntry
SelectedPlatform string HasSbomData bool
FromDigest string FromSbomStatus string
ToDigest string ToSbomStatus string
FromFailed bool
ToFailed bool
IsMultiArch bool
CommonPlatforms []db.PlatformInfo
SelectedPlatform string
FromDigest string
ToDigest string
}{ }{
PageData: NewPageData(r, &h.BaseUIHandler), PageData: NewPageData(r, &h.BaseUIHandler),
Meta: meta, Meta: meta,
Owner: owner, Owner: owner,
Repository: repo, Repository: repo,
FromManifest: fromData.manifest, FromManifest: fromData.manifest,
ToManifest: toData.manifest, ToManifest: toData.manifest,
FromTag: fromTag, FromTag: fromTag,
ToTag: toTag, ToTag: toTag,
Summary: summary, Summary: summary,
LayerDiff: layerDiff, LayerDiff: layerDiff,
FixedVulns: fixedVulns, FixedVulns: fixedVulns,
NewVulns: newVulns, NewVulns: newVulns,
UnchangedVulns: unchangedVulns, UnchangedVulns: unchangedVulns,
HasVulnData: hasVulnData, HasVulnData: hasVulnData,
FromScanStatus: fromScanStatus, FromScanStatus: fromScanStatus,
ToScanStatus: toScanStatus, ToScanStatus: toScanStatus,
FromFailed: fromFailed, AddedPackages: addedPackages,
ToFailed: toFailed, RemovedPackages: removedPackages,
IsMultiArch: isMultiArch, ChangedPackages: changedPackages,
CommonPlatforms: commonPlatforms, UnchangedPackages: unchangedPackages,
SelectedPlatform: selectedPlatform, HasSbomData: hasSbomData,
FromDigest: fromDigest, FromSbomStatus: fromSbomStatus,
ToDigest: toDigest, ToSbomStatus: toSbomStatus,
FromFailed: fromFailed,
ToFailed: toFailed,
IsMultiArch: isMultiArch,
CommonPlatforms: commonPlatforms,
SelectedPlatform: selectedPlatform,
FromDigest: fromDigest,
ToDigest: toDigest,
} }
if err := h.Templates.ExecuteTemplate(w, "diff", data); err != nil { if err := h.Templates.ExecuteTemplate(w, "diff", data); err != nil {
pkg/appview/handlers/diff_test.go
+211 -1
View File
@@ -338,7 +338,7 @@ func TestComputeDiffSummary(t *testing.T) {
{Status: "unchanged", Vuln: vulnMatch{Severity: "Low"}}, {Status: "unchanged", Vuln: vulnMatch{Severity: "Low"}},
} }
summary := computeDiffSummary(fromLayers, toLayers, vulnDiff, true) summary := computeDiffSummary(fromLayers, toLayers, vulnDiff, true, nil, false)
if summary.SizeDelta != 1000 { if summary.SizeDelta != 1000 {
t.Errorf("expected size delta 1000, got %d", summary.SizeDelta) t.Errorf("expected size delta 1000, got %d", summary.SizeDelta)
@@ -375,6 +375,8 @@ func TestComputeDiffSummary_NoVulnData(t *testing.T) {
[]LayerDetail{{Size: 200}}, []LayerDetail{{Size: 200}},
nil, nil,
false, false,
nil,
false,
) )
if summary.HasVulnData { if summary.HasVulnData {
@@ -391,6 +393,8 @@ func TestComputeDiffSummary_SmallerImage(t *testing.T) {
[]LayerDetail{{Size: 2000}}, []LayerDetail{{Size: 2000}},
nil, nil,
false, false,
nil,
false,
) )
if summary.SizeDelta != -6000 { if summary.SizeDelta != -6000 {
@@ -400,3 +404,209 @@ func TestComputeDiffSummary_SmallerImage(t *testing.T) {
t.Error("unexpected layer counts") t.Error("unexpected layer counts")
} }
} }
// sbomStatusCounts tallies a diff result by status for compact assertions.
func sbomStatusCounts(diff []SbomDiffEntry) map[string]int {
counts := make(map[string]int)
for _, e := range diff {
counts[e.Status]++
}
return counts
}
func TestComputeSbomDiff_Identical(t *testing.T) {
pkgs := []sbomPackage{
{Name: "openssl", Version: "3.0.1", Type: "deb"},
{Name: "zlib", Version: "1.2.13", Type: "deb"},
}
diff := computeSbomDiff(pkgs, pkgs)
if len(diff) != 2 {
t.Fatalf("expected 2 entries, got %d", len(diff))
}
for _, e := range diff {
if e.Status != "unchanged" {
t.Errorf("expected unchanged, got %q for %s", e.Status, e.Package.Name)
}
}
}
func TestComputeSbomDiff_VersionBump(t *testing.T) {
from := []sbomPackage{{Name: "openssl", Version: "3.0.1", Type: "deb"}}
to := []sbomPackage{{Name: "openssl", Version: "3.0.2", Type: "deb"}}
diff := computeSbomDiff(from, to)
if len(diff) != 1 {
t.Fatalf("expected 1 entry, got %d", len(diff))
}
if diff[0].Status != "changed" {
t.Errorf("expected changed, got %q", diff[0].Status)
}
if diff[0].PrevVersion != "3.0.1" {
t.Errorf("expected PrevVersion 3.0.1, got %q", diff[0].PrevVersion)
}
if diff[0].Package.Version != "3.0.2" {
t.Errorf("expected Package.Version 3.0.2, got %q", diff[0].Package.Version)
}
}
func TestComputeSbomDiff_AddedRemovedEmpty(t *testing.T) {
diff := computeSbomDiff(nil, nil)
if len(diff) != 0 {
t.Errorf("expected 0 entries for empty inputs, got %d", len(diff))
}
diff = computeSbomDiff(nil, []sbomPackage{{Name: "curl", Version: "8.0", Type: "deb"}})
if len(diff) != 1 || diff[0].Status != "added" {
t.Errorf("expected single added entry, got %+v", diff)
}
diff = computeSbomDiff([]sbomPackage{{Name: "curl", Version: "8.0", Type: "deb"}}, nil)
if len(diff) != 1 || diff[0].Status != "removed" {
t.Errorf("expected single removed entry, got %+v", diff)
}
}
func TestComputeSbomDiff_SameNameDifferentType(t *testing.T) {
// A deb package and a binary share a name; the deb version bump must
// not pair against the binary entry.
from := []sbomPackage{
{Name: "curl", Version: "8.0", Type: "deb"},
{Name: "curl", Version: "8.0", Type: ""},
}
to := []sbomPackage{
{Name: "curl", Version: "8.1", Type: "deb"},
{Name: "curl", Version: "8.0", Type: ""},
}
diff := computeSbomDiff(from, to)
if len(diff) != 2 {
t.Fatalf("expected 2 entries, got %d", len(diff))
}
counts := sbomStatusCounts(diff)
if counts["changed"] != 1 || counts["unchanged"] != 1 {
t.Errorf("expected 1 changed + 1 unchanged, got %v", counts)
}
for _, e := range diff {
if e.Status == "changed" {
if e.Package.Type != "deb" || e.PrevVersion != "8.0" || e.Package.Version != "8.1" {
t.Errorf("changed entry paired wrong: %+v", e)
}
}
}
}
func TestComputeSbomDiff_Multiplicity(t *testing.T) {
// from {1.0, 1.1} vs to {1.1, 1.2}: 1.1 matches exactly, 1.0 pairs
// with 1.2 as a version change.
from := []sbomPackage{
{Name: "libssl", Version: "1.0", Type: "deb"},
{Name: "libssl", Version: "1.1", Type: "deb"},
}
to := []sbomPackage{
{Name: "libssl", Version: "1.1", Type: "deb"},
{Name: "libssl", Version: "1.2", Type: "deb"},
}
diff := computeSbomDiff(from, to)
if len(diff) != 2 {
t.Fatalf("expected 2 entries, got %d", len(diff))
}
counts := sbomStatusCounts(diff)
if counts["unchanged"] != 1 || counts["changed"] != 1 {
t.Errorf("expected 1 unchanged + 1 changed, got %v", counts)
}
for _, e := range diff {
if e.Status == "changed" && (e.PrevVersion != "1.0" || e.Package.Version != "1.2") {
t.Errorf("expected 1.0 to pair with 1.2, got %+v", e)
}
}
// Asymmetric counts: two from-side versions, one to-side version.
from = []sbomPackage{
{Name: "libssl", Version: "1.0", Type: "deb"},
{Name: "libssl", Version: "1.1", Type: "deb"},
}
to = []sbomPackage{{Name: "libssl", Version: "1.2", Type: "deb"}}
diff = computeSbomDiff(from, to)
counts = sbomStatusCounts(diff)
if counts["changed"] != 1 || counts["removed"] != 1 {
t.Errorf("expected 1 changed + 1 removed, got %v", counts)
}
}
func TestComputeSbomDiff_DuplicateEntriesOneSide(t *testing.T) {
// Two identical entries on the from side must not both match the
// single to-side entry (multiset, not set).
from := []sbomPackage{
{Name: "foo", Version: "1.0", Type: "npm"},
{Name: "foo", Version: "1.0", Type: "npm"},
}
to := []sbomPackage{{Name: "foo", Version: "1.0", Type: "npm"}}
diff := computeSbomDiff(from, to)
if len(diff) != 2 {
t.Fatalf("expected 2 entries, got %d", len(diff))
}
counts := sbomStatusCounts(diff)
if counts["unchanged"] != 1 || counts["removed"] != 1 {
t.Errorf("expected 1 unchanged + 1 removed, got %v", counts)
}
}
func TestComputeSbomDiff_DeterministicOrder(t *testing.T) {
from := []sbomPackage{
{Name: "zlib", Version: "1.2", Type: "deb"},
{Name: "curl", Version: "8.0", Type: "deb"},
{Name: "gone", Version: "0.1", Type: "npm"},
}
to := []sbomPackage{
{Name: "curl", Version: "8.1", Type: "deb"},
{Name: "zlib", Version: "1.2", Type: "deb"},
{Name: "fresh", Version: "2.0", Type: "npm"},
}
want := []SbomDiffEntry{
{Status: "changed", Package: sbomPackage{Name: "curl", Version: "8.1", Type: "deb"}, PrevVersion: "8.0"},
{Status: "added", Package: sbomPackage{Name: "fresh", Version: "2.0", Type: "npm"}},
{Status: "removed", Package: sbomPackage{Name: "gone", Version: "0.1", Type: "npm"}},
{Status: "unchanged", Package: sbomPackage{Name: "zlib", Version: "1.2", Type: "deb"}},
}
for run := range 5 {
diff := computeSbomDiff(from, to)
if len(diff) != len(want) {
t.Fatalf("run %d: expected %d entries, got %d", run, len(want), len(diff))
}
for i := range want {
if diff[i] != want[i] {
t.Errorf("run %d entry %d: expected %+v, got %+v", run, i, want[i], diff[i])
}
}
}
}
func TestComputeDiffSummary_PackageCounts(t *testing.T) {
sbomDiff := []SbomDiffEntry{
{Status: "added"},
{Status: "added"},
{Status: "removed"},
{Status: "changed"},
{Status: "unchanged"},
}
summary := computeDiffSummary(nil, nil, nil, false, sbomDiff, true)
if summary.PkgAddedCount != 2 {
t.Errorf("expected 2 added, got %d", summary.PkgAddedCount)
}
if summary.PkgRemovedCount != 1 {
t.Errorf("expected 1 removed, got %d", summary.PkgRemovedCount)
}
if summary.PkgChangedCount != 1 {
t.Errorf("expected 1 changed, got %d", summary.PkgChangedCount)
}
if !summary.HasSbomData {
t.Error("expected HasSbomData to be true")
}
}
pkg/appview/handlers/purl.go
+199
View File
@@ -0,0 +1,199 @@
package handlers
import (
"net/url"
"strings"
)
// purlInfo is the subset of a package URL (pkg:type/namespace/name@version)
// needed for display: the ecosystem type and an upstream link.
type purlInfo struct {
Type string // purl type, lowercased (e.g. "deb", "npm", "golang")
Namespace string // may be empty or multi-segment (golang)
Name string
Version string
Qualifiers map[string]string
}
// parsePurl parses a package URL (https://github.com/package-url/purl-spec):
// pkg:type/namespace/name@version?qualifiers#subpath
// Returns nil for anything malformed — callers fall back to supplier sniffing.
func parsePurl(s string) *purlInfo {
rest, ok := strings.CutPrefix(s, "pkg:")
if !ok {
return nil
}
if i := strings.IndexByte(rest, '#'); i >= 0 {
rest = rest[:i]
}
qualifiers := map[string]string{}
if i := strings.IndexByte(rest, '?'); i >= 0 {
for kv := range strings.SplitSeq(rest[i+1:], "&") {
if k, v, ok := strings.Cut(kv, "="); ok && k != "" {
if uv, err := url.QueryUnescape(v); err == nil {
v = uv
}
qualifiers[strings.ToLower(k)] = v
}
}
rest = rest[:i]
}
typ, rest, ok := strings.Cut(strings.TrimPrefix(rest, "/"), "/")
if !ok || typ == "" {
return nil
}
version := ""
if i := strings.LastIndexByte(rest, '@'); i >= 0 {
version = rest[i+1:]
rest = rest[:i]
}
namespace := ""
name := rest
if i := strings.LastIndexByte(rest, '/'); i >= 0 {
namespace = rest[:i]
name = rest[i+1:]
}
if name == "" {
return nil
}
unescape := func(s string) string {
if u, err := url.PathUnescape(s); err == nil {
return u
}
return s
}
return &purlInfo{
Type: strings.ToLower(typ),
Namespace: unescape(namespace),
Name: unescape(name),
Version: unescape(version),
Qualifiers: qualifiers,
}
}
// purlDisplayType maps purl types to the short labels the UI shows. Types
// without a friendlier label (deb, npm, gem, apk, rpm, nuget, ...) pass
// through unchanged; "generic" (binary catalogers) maps to empty so the
// column shows "-" rather than a meaningless badge.
func purlDisplayType(t string) string {
switch t {
case "golang":
return "go"
case "pypi":
return "python"
case "cargo":
return "rust"
case "maven":
return "java"
case "composer":
return "php"
case "generic":
return ""
default:
return t
}
}
// purlURL returns the canonical upstream page for a package, or "" when the
// ecosystem has no stable public index or the purl type is unknown.
func purlURL(p *purlInfo) string {
// Syft stamps "UNKNOWN" when a manifest has no version field. Published
// packages always have one (registries require it), so these entries are
// almost always phantom nested manifests (e.g. a dist/package.json inside
// another package's tarball) that don't exist upstream — don't link them.
if strings.EqualFold(p.Version, "unknown") {
return ""
}
esc := url.PathEscape
switch p.Type {
case "npm":
// Scoped names (@scope/name) are accepted literally by npmx.dev;
// versions use the /v/{version} path form (name@version 404s).
name := p.Name
if p.Namespace != "" {
name = p.Namespace + "/" + p.Name
}
if p.Version != "" {
return "https://npmx.dev/package/" + name + "/v/" + esc(p.Version)
}
return "https://npmx.dev/package/" + name
case "pypi":
if p.Version != "" {
return "https://pypi.org/project/" + esc(p.Name) + "/" + esc(p.Version) + "/"
}
return "https://pypi.org/project/" + esc(p.Name) + "/"
case "gem":
return "https://rubygems.org/gems/" + esc(p.Name)
case "golang":
path := p.Name
if p.Namespace != "" {
path = p.Namespace + "/" + p.Name
}
if p.Version != "" {
return "https://pkg.go.dev/" + path + "@" + esc(p.Version)
}
return "https://pkg.go.dev/" + path
case "cargo":
if p.Version != "" {
return "https://crates.io/crates/" + esc(p.Name) + "/" + esc(p.Version)
}
return "https://crates.io/crates/" + esc(p.Name)
case "nuget":
if p.Version != "" {
return "https://www.nuget.org/packages/" + esc(p.Name) + "/" + esc(p.Version)
}
return "https://www.nuget.org/packages/" + esc(p.Name)
case "maven":
if p.Namespace == "" {
return ""
}
u := "https://central.sonatype.com/artifact/" + esc(p.Namespace) + "/" + esc(p.Name)
if p.Version != "" {
u += "/" + esc(p.Version)
}
return u
case "composer":
if p.Namespace == "" {
return ""
}
return "https://packagist.org/packages/" + esc(p.Namespace) + "/" + esc(p.Name)
case "deb":
// Trackers are keyed by source package; Syft records it in the
// "upstream" qualifier for binary packages (e.g. libssl3t64 →
// openssl). Fall back to the binary name, which usually matches.
name := p.Name
if up := p.Qualifiers["upstream"]; up != "" {
if i := strings.IndexByte(up, '@'); i >= 0 {
up = up[:i]
}
name = up
}
switch p.Namespace {
case "debian":
return "https://tracker.debian.org/pkg/" + esc(name)
case "ubuntu":
return "https://launchpad.net/ubuntu/+source/" + esc(name)
}
return ""
case "apk":
return "https://pkgs.alpinelinux.org/packages?name=" + url.QueryEscape(p.Name)
case "rpm":
if p.Namespace == "fedora" {
return "https://packages.fedoraproject.org/pkgs/" + esc(p.Name) + "/"
}
return ""
case "github":
if p.Namespace == "" {
return ""
}
return "https://github.com/" + esc(p.Namespace) + "/" + esc(p.Name)
}
return ""
}
pkg/appview/handlers/purl_test.go
+178
View File
@@ -0,0 +1,178 @@
package handlers
import "testing"
func TestParsePurl(t *testing.T) {
tests := []struct {
name string
in string
want *purlInfo
}{
{
name: "deb with qualifiers",
in: "pkg:deb/ubuntu/libssl3t64@3.0.13-0ubuntu3.9?arch=amd64&distro=ubuntu-24.04&upstream=openssl",
want: &purlInfo{Type: "deb", Namespace: "ubuntu", Name: "libssl3t64", Version: "3.0.13-0ubuntu3.9"},
},
{
name: "scoped npm",
in: "pkg:npm/%40discordjs/builders@1.14.1",
want: &purlInfo{Type: "npm", Namespace: "@discordjs", Name: "builders", Version: "1.14.1"},
},
{
name: "unscoped npm",
in: "pkg:npm/lodash@4.18.1",
want: &purlInfo{Type: "npm", Namespace: "", Name: "lodash", Version: "4.18.1"},
},
{
name: "golang multi-segment namespace",
in: "pkg:golang/github.com/moby/sys/user@v0.1.0",
want: &purlInfo{Type: "golang", Namespace: "github.com/moby/sys", Name: "user", Version: "v0.1.0"},
},
{
name: "no version",
in: "pkg:gem/rails",
want: &purlInfo{Type: "gem", Namespace: "", Name: "rails", Version: ""},
},
{
name: "subpath stripped",
in: "pkg:golang/golang.org/x/sys@v0.1.0#unix",
want: &purlInfo{Type: "golang", Namespace: "golang.org/x", Name: "sys", Version: "v0.1.0"},
},
{name: "not a purl", in: "https://example.com/foo", want: nil},
{name: "missing name", in: "pkg:npm", want: nil},
{name: "empty", in: "", want: nil},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := parsePurl(tt.in)
if tt.want == nil {
if got != nil {
t.Fatalf("expected nil, got %+v", got)
}
return
}
if got == nil {
t.Fatalf("expected %+v, got nil", tt.want)
}
if got.Type != tt.want.Type || got.Namespace != tt.want.Namespace ||
got.Name != tt.want.Name || got.Version != tt.want.Version {
t.Errorf("expected %+v, got %+v", tt.want, got)
}
})
}
}
func TestParsePurl_Qualifiers(t *testing.T) {
p := parsePurl("pkg:deb/debian/libpam0g@1.7.0-5?arch=amd64&upstream=pam%401.7.0")
if p == nil {
t.Fatal("expected parse to succeed")
}
if p.Qualifiers["upstream"] != "pam@1.7.0" {
t.Errorf("expected upstream qualifier pam@1.7.0, got %q", p.Qualifiers["upstream"])
}
if p.Qualifiers["arch"] != "amd64" {
t.Errorf("expected arch qualifier amd64, got %q", p.Qualifiers["arch"])
}
}
func TestPurlDisplayType(t *testing.T) {
tests := map[string]string{
"golang": "go",
"pypi": "python",
"cargo": "rust",
"maven": "java",
"composer": "php",
"generic": "",
"deb": "deb",
"npm": "npm",
"nuget": "nuget",
"apk": "apk",
}
for in, want := range tests {
if got := purlDisplayType(in); got != want {
t.Errorf("purlDisplayType(%q): expected %q, got %q", in, want, got)
}
}
}
func TestPurlURL(t *testing.T) {
tests := []struct {
name string
in string
want string
}{
{
name: "scoped npm",
in: "pkg:npm/%40discordjs/builders@1.14.1",
want: "https://npmx.dev/package/@discordjs/builders/v/1.14.1",
},
{
name: "ubuntu deb uses upstream source",
in: "pkg:deb/ubuntu/libssl3t64@3.0.13?upstream=openssl",
want: "https://launchpad.net/ubuntu/+source/openssl",
},
{
name: "debian deb without upstream falls back to binary name",
in: "pkg:deb/debian/bash@5.2.37-2",
want: "https://tracker.debian.org/pkg/bash",
},
{
name: "debian upstream with version suffix",
in: "pkg:deb/debian/libpam0g@1.7.0-5?upstream=pam%401.7.0",
want: "https://tracker.debian.org/pkg/pam",
},
{
name: "golang",
in: "pkg:golang/github.com/moby/sys/user@v0.1.0",
want: "https://pkg.go.dev/github.com/moby/sys/user@v0.1.0",
},
{
name: "nuget",
in: "pkg:nuget/Serilog@4.3.0",
want: "https://www.nuget.org/packages/Serilog/4.3.0",
},
{
name: "pypi",
in: "pkg:pypi/requests@2.31.0",
want: "https://pypi.org/project/requests/2.31.0/",
},
{
name: "apk",
in: "pkg:apk/alpine/musl@1.2.4-r2",
want: "https://pkgs.alpinelinux.org/packages?name=musl",
},
{
name: "deb with unknown distro namespace",
in: "pkg:deb/somedistro/foo@1.0",
want: "",
},
{
name: "unknown type",
in: "pkg:conan/openssl@3.0.0",
want: "",
},
{
name: "UNKNOWN version gets no link",
in: "pkg:npm/web-streams-ponyfill-es6@UNKNOWN",
want: "",
},
{
name: "lowercase unknown version also gets no link",
in: "pkg:pypi/somepkg@unknown",
want: "",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
p := parsePurl(tt.in)
if p == nil {
t.Fatal("expected parse to succeed")
}
if got := purlURL(p); got != tt.want {
t.Errorf("expected %q, got %q", tt.want, got)
}
})
}
}
pkg/appview/handlers/sbom_details.go
+56 -8
View File
@@ -26,12 +26,19 @@ type spdxDocument struct {
} }
type spdxPackage struct { type spdxPackage struct {
SPDXID string `json:"SPDXID"` SPDXID string `json:"SPDXID"`
Name string `json:"name"` Name string `json:"name"`
VersionInfo string `json:"versionInfo"` VersionInfo string `json:"versionInfo"`
Supplier string `json:"supplier"` Supplier string `json:"supplier"`
LicenseConcluded string `json:"licenseConcluded"` LicenseConcluded string `json:"licenseConcluded"`
DownloadLocation string `json:"downloadLocation"` LicenseDeclared string `json:"licenseDeclared"`
DownloadLocation string `json:"downloadLocation"`
ExternalRefs []spdxExternalRef `json:"externalRefs"`
}
type spdxExternalRef struct {
ReferenceType string `json:"referenceType"`
ReferenceLocator string `json:"referenceLocator"`
} }
// sbomDetailsData is the template data for the sbom-details partial. // sbomDetailsData is the template data for the sbom-details partial.
@@ -50,7 +57,8 @@ type sbomPackage struct {
Name string Name string
Version string Version string
License string License string
Type string // Derived from supplier (e.g., "deb", "npm") Type string // Derived from the purl external ref (e.g., "deb", "npm"); supplier sniffing as fallback
URL string // Upstream package page derived from the purl; empty when unknown
} }
func (h *SbomDetailsHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) { func (h *SbomDetailsHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
@@ -191,6 +199,11 @@ func FetchSbomDetails(ctx context.Context, holdEndpoint, digest string) sbomDeta
license := p.LicenseConcluded license := p.LicenseConcluded
if license == "NOASSERTION" || license == "" { if license == "NOASSERTION" || license == "" {
license = p.LicenseDeclared
}
// LicenseRef-... values are opaque document-local identifiers
// (hashes), useless as display names.
if license == "NOASSERTION" || license == "" || strings.HasPrefix(license, "LicenseRef-") {
license = "-" license = "-"
} }
@@ -199,11 +212,46 @@ func FetchSbomDetails(ctx context.Context, holdEndpoint, digest string) sbomDeta
supplier = "" supplier = ""
} }
// The purl external ref is authoritative for ecosystem type and
// upstream location. The supplier heuristic is a last-resort
// fallback: deb suppliers are maintainer names, so substring
// matching misfires (a maintainer named Santiago tags the package
// as "go").
pkgType := ""
pkgURL := ""
purlFound := false
for _, ref := range p.ExternalRefs {
if ref.ReferenceType != "purl" {
continue
}
if purl := parsePurl(ref.ReferenceLocator); purl != nil {
purlFound = true
pkgType = purlDisplayType(purl.Type)
pkgURL = purlURL(purl)
}
break
}
if !purlFound {
pkgType = extractPackageType(supplier)
}
if pkgURL == "" && (strings.HasPrefix(p.DownloadLocation, "https://") || strings.HasPrefix(p.DownloadLocation, "http://")) {
pkgURL = p.DownloadLocation
}
// Entries without a real version (registries require one to publish)
// are almost always phantom nested manifests — e.g. a dist/package.json
// inside another package's tarball — that don't exist upstream. The
// purl often omits the version entirely in this case, so check the
// SPDX versionInfo rather than the purl.
if p.VersionInfo == "" || strings.EqualFold(p.VersionInfo, "unknown") {
pkgURL = ""
}
packages = append(packages, sbomPackage{ packages = append(packages, sbomPackage{
Name: p.Name, Name: p.Name,
Version: p.VersionInfo, Version: p.VersionInfo,
License: license, License: license,
Type: extractPackageType(supplier), Type: pkgType,
URL: pkgURL,
}) })
} }
pkg/appview/handlers/settings.go
+5 -1
View File
@@ -29,7 +29,8 @@ type HoldDisplay struct {
Region string `json:"region"` Region string `json:"region"`
Membership string `json:"membership"` Membership string `json:"membership"`
Permissions []string `json:"permissions,omitempty"` Permissions []string `json:"permissions,omitempty"`
Status string `json:"status"` // "" = unknown, "online", "offline" ReadOnly bool `json:"readOnly"` // crew member without blob:write — pushes would be rejected
Status string `json:"status"` // "" = unknown, "online", "offline"
IsActive bool `json:"isActive"` IsActive bool `json:"isActive"`
} }
@@ -218,6 +219,9 @@ func (h *SettingsHandler) buildHoldsData(ctx context.Context, userDID, defaultHo
} }
} }
// Owners hold all permissions implicitly; crew need blob:write to push.
display.ReadOnly = hold.Membership == "crew" && !slices.Contains(display.Permissions, "blob:write")
if h.HealthChecker != nil { if h.HealthChecker != nil {
if status := h.HealthChecker.GetStatus(ctx, hold.HoldDID); status != nil { if status := h.HealthChecker.GetStatus(ctx, hold.HoldDID); status != nil {
if status.Reachable { if status.Reachable {
pkg/appview/handlers/upgrade_banner.go
+1 -1
View File
@@ -185,7 +185,7 @@ func (h *UpgradeBannerHandler) ServeHTTP(w http.ResponseWriter, r *http.Request)
} }
} }
summary := computeDiffSummary(currentLayers, newerLayers, vulnDiff, hasVulnData) summary := computeDiffSummary(currentLayers, newerLayers, vulnDiff, hasVulnData, nil, false)
slog.Debug("Upgrade banner: computed summary", "hasVulnData", hasVulnData, slog.Debug("Upgrade banner: computed summary", "hasVulnData", hasVulnData,
"layersFrom", summary.LayerCountFrom, "layersTo", summary.LayerCountTo, "sizeDelta", summary.SizeDelta) "layersFrom", summary.LayerCountFrom, "layersTo", summary.LayerCountTo, "sizeDelta", summary.SizeDelta)
pkg/appview/jetstream/backfill.go
+1 -1
View File
@@ -308,7 +308,7 @@ func (b *BackfillWorker) backfillRepo(ctx context.Context, did, collection strin
case atproto.StatsCollection: case atproto.StatsCollection:
recordCount, procErr = b.batchStats(ctx, did, allRecords) recordCount, procErr = b.batchStats(ctx, did, allRecords)
case atproto.CaptainCollection: case atproto.CaptainCollection:
recordCount, procErr = b.batchCaptains(did, allRecords) recordCount, procErr = b.batchCaptains(ctx, did, allRecords)
case atproto.CrewCollection: case atproto.CrewCollection:
recordCount, procErr = b.batchCrew(did, allRecords) recordCount, procErr = b.batchCrew(did, allRecords)
default: default:
pkg/appview/jetstream/backfill_batch.go
+19 -1
View File
@@ -423,9 +423,12 @@ func (b *BackfillWorker) batchStats(ctx context.Context, holdDID string, records
} }
// batchCaptains decodes captain records and writes them in one upsert. // batchCaptains decodes captain records and writes them in one upsert.
func (b *BackfillWorker) batchCaptains(holdDID string, records []atproto.Record) (int, error) { // Records whose publishing DID does not advertise an atcr_hold service in its
// DID document are skipped — only real holds may enter the discovery cache.
func (b *BackfillWorker) batchCaptains(ctx context.Context, holdDID string, records []atproto.Record) (int, error) {
captains := make([]db.HoldCaptainRecord, 0, len(records)) captains := make([]db.HoldCaptainRecord, 0, len(records))
now := time.Now() now := time.Now()
verified := make(map[string]bool)
for i := range records { for i := range records {
r := &records[i] r := &records[i]
var cr atproto.CaptainRecord var cr atproto.CaptainRecord
@@ -443,6 +446,21 @@ func (b *BackfillWorker) batchCaptains(holdDID string, records []atproto.Record)
if recordHoldDID == "" { if recordHoldDID == "" {
recordHoldDID = holdDID recordHoldDID = holdDID
} }
isHold, ok := verified[recordHoldDID]
if !ok {
var err error
isHold, err = atproto.HasHoldService(ctx, recordHoldDID)
if err != nil {
slog.Warn("Backfill skipping captain, hold DID unresolvable", "uri", r.URI, "error", err)
isHold = false
} else if !isHold {
slog.Info("Backfill skipping captain from non-hold DID", "hold_did", recordHoldDID)
}
verified[recordHoldDID] = isHold
}
if !isHold {
continue
}
captains = append(captains, db.HoldCaptainRecord{ captains = append(captains, db.HoldCaptainRecord{
HoldDID: recordHoldDID, HoldDID: recordHoldDID,
OwnerDID: cr.Owner, OwnerDID: cr.Owner,
pkg/appview/jetstream/backfill_batch_test.go
+119
View File
@@ -0,0 +1,119 @@
package jetstream
import (
"context"
"encoding/json"
"testing"
"time"
"atcr.io/pkg/atproto"
"github.com/bluesky-social/indigo/atproto/identity"
"github.com/bluesky-social/indigo/atproto/syntax"
)
// countingDirectory wraps fakeDirectory and counts LookupDID calls per DID,
// so tests can assert verification results are memoized within a batch.
type countingDirectory struct {
fakeDirectory
lookups map[string]int
}
func (d *countingDirectory) LookupDID(ctx context.Context, did syntax.DID) (*identity.Identity, error) {
d.lookups[did.String()]++
return d.fakeDirectory.LookupDID(ctx, did)
}
func TestBatchCaptains_VerifiesHoldService(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
execStatements(t, db, `
CREATE TABLE hold_captain_records (
hold_did TEXT PRIMARY KEY,
owner_did TEXT NOT NULL,
public BOOLEAN NOT NULL,
allow_all_crew BOOLEAN NOT NULL,
deployed_at TEXT,
region TEXT,
successor TEXT,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
`)
realHold := "did:web:realhold.example.com"
notAHold := "did:plc:notahold"
unresolvable := "did:plc:unresolvable"
dir := &countingDirectory{
fakeDirectory: fakeDirectory{byDID: map[string]*identity.Identity{
realHold: holdIdentity(realHold, "https://realhold.example.com"),
notAHold: {
DID: syntax.DID(notAHold),
Services: map[string]identity.ServiceEndpoint{
"atproto_pds": {Type: "AtprotoPersonalDataServer", URL: "https://pds.example.com"},
},
},
}},
lookups: map[string]int{},
}
atproto.SetDirectory(dir)
defer atproto.SetDirectory(nil)
worker := &BackfillWorker{db: db}
captainValue, _ := json.Marshal(map[string]any{
"$type": "io.atcr.hold.captain",
"owner": "did:plc:owner123",
"public": true,
"allowAllCrew": true,
"enableBlueskyPosts": false,
"deployedAt": time.Now().Format(time.RFC3339),
})
captainRecord := func(holdDID string) atproto.Record {
return atproto.Record{
URI: "at://" + holdDID + "/io.atcr.hold.captain/self",
Value: captainValue,
}
}
records := []atproto.Record{
captainRecord(realHold),
captainRecord(realHold), // duplicate DID: exercises the verification memo
captainRecord(notAHold),
captainRecord(unresolvable),
}
count, err := worker.batchCaptains(context.Background(), realHold, records)
if err != nil {
t.Fatalf("batchCaptains failed: %v", err)
}
if count != 2 {
t.Errorf("batchCaptains processed %d records, want 2 (both from the real hold)", count)
}
rows, err := db.Query(`SELECT hold_did FROM hold_captain_records`)
if err != nil {
t.Fatalf("Failed to query captain records: %v", err)
}
defer rows.Close()
var cached []string
for rows.Next() {
var did string
if err := rows.Scan(&did); err != nil {
t.Fatalf("Failed to scan captain record: %v", err)
}
cached = append(cached, did)
}
if len(cached) != 1 || cached[0] != realHold {
t.Errorf("cached captain records = %v, want only %q", cached, realHold)
}
// Each DID should be verified exactly once per batch, regardless of how
// many of its records appear.
for _, did := range []string{realHold, notAHold, unresolvable} {
if got := dir.lookups[did]; got != 1 {
t.Errorf("LookupDID(%s) called %d times, want 1 (memoized)", did, got)
}
}
}
pkg/appview/jetstream/processor.go
+32 -8
View File
@@ -615,14 +615,20 @@ func (p *Processor) ProcessIdentity(ctx context.Context, did string, newHandle s
return nil return nil
} }
// Update handle in database // Update handle in database. An empty handle means the identity event carried
if err := db.UpdateUserHandle(p.db, did, newHandle); err != nil { // an invalidated/unresolvable handle (expired, DNS/_atproto TXT removed, PDS down
slog.Warn("Failed to update user handle in database", // during resolution). Don't overwrite the cached handle in that case, and don't
"component", "processor", // blank it to "" (which trips UNIQUE(handle) the moment a second user does the same).
"did", did, // Fall through to invalidate the cache so the next lookup re-resolves.
"handle", newHandle, if newHandle != "" {
"error", err) if err := db.UpdateUserHandle(p.db, did, newHandle); err != nil {
// Continue to invalidate cache even if DB update fails slog.Warn("Failed to update user handle in database",
"component", "processor",
"did", did,
"handle", newHandle,
"error", err)
// Continue to invalidate cache even if DB update fails
}
} }
// Invalidate cached identity data to force re-resolution on next lookup // Invalidate cached identity data to force re-resolution on next lookup
@@ -896,6 +902,24 @@ func (p *Processor) ProcessCaptain(ctx context.Context, holdDID string, recordDa
return fmt.Errorf("failed to unmarshal captain record: %w", err) return fmt.Errorf("failed to unmarshal captain record: %w", err)
} }
// Verify the publishing DID actually runs a hold before caching: any
// account can publish a captain record, but only real holds advertise the
// atcr_hold service in their DID document. Unverified records would
// otherwise surface in every user's hold picker (allowAllCrew=true) or,
// via forged crew records, in targeted users' member-hold lists.
// Skip rather than fail — unresolvable DIDs are retried by periodic backfill.
isHold, err := atproto.HasHoldService(ctx, holdDID)
if err != nil {
slog.Warn("Skipping captain record, hold DID unresolvable",
"component", "processor", "hold_did", holdDID, "error", err)
return nil
}
if !isHold {
slog.Info("Skipping captain record from non-hold DID",
"component", "processor", "hold_did", holdDID)
return nil
}
// Convert to db struct and upsert // Convert to db struct and upsert
record := &db.HoldCaptainRecord{ record := &db.HoldCaptainRecord{
HoldDID: holdDID, HoldDID: holdDID,
pkg/appview/jetstream/processor_test.go
+127
View File
@@ -4,14 +4,57 @@ import (
"context" "context"
"database/sql" "database/sql"
"encoding/json" "encoding/json"
"fmt"
"strings" "strings"
"testing" "testing"
"time" "time"
"atcr.io/pkg/atproto" "atcr.io/pkg/atproto"
"github.com/bluesky-social/indigo/atproto/identity"
"github.com/bluesky-social/indigo/atproto/syntax"
_ "github.com/tursodatabase/go-libsql" _ "github.com/tursodatabase/go-libsql"
) )
// fakeDirectory is an in-memory identity.Directory for tests. Lookups of
// unregistered identifiers fail, mirroring unresolvable identities.
type fakeDirectory struct {
byDID map[string]*identity.Identity
}
func (d *fakeDirectory) LookupDID(_ context.Context, did syntax.DID) (*identity.Identity, error) {
if ident, ok := d.byDID[did.String()]; ok {
return ident, nil
}
return nil, fmt.Errorf("%w: %s", identity.ErrDIDNotFound, did)
}
func (d *fakeDirectory) LookupHandle(_ context.Context, handle syntax.Handle) (*identity.Identity, error) {
return nil, fmt.Errorf("%w: %s", identity.ErrHandleNotFound, handle)
}
func (d *fakeDirectory) Lookup(ctx context.Context, atid syntax.AtIdentifier) (*identity.Identity, error) {
if did, err := atid.AsDID(); err == nil {
return d.LookupDID(ctx, did)
}
return nil, fmt.Errorf("%w: %s", identity.ErrHandleResolutionFailed, atid)
}
func (d *fakeDirectory) Purge(_ context.Context, _ syntax.AtIdentifier) error {
return nil
}
// holdIdentity builds an identity advertising the atcr_hold service, as real
// holds publish in their DID documents (see pds.HoldServices).
func holdIdentity(did, url string) *identity.Identity {
return &identity.Identity{
DID: syntax.DID(did),
Services: map[string]identity.ServiceEndpoint{
"atproto_pds": {Type: "AtprotoPersonalDataServer", URL: url},
"atcr_hold": {Type: "AtcrHoldService", URL: url},
},
}
}
// execStatements splits a multi-statement SQL string and executes each statement individually. // execStatements splits a multi-statement SQL string and executes each statement individually.
// go-libsql does not support multi-statement Exec like mattn/go-sqlite3. // go-libsql does not support multi-statement Exec like mattn/go-sqlite3.
func execStatements(t *testing.T, db *sql.DB, schema string) { func execStatements(t *testing.T, db *sql.DB, schema string) {
@@ -754,6 +797,7 @@ func TestProcessRecord_RoutesCorrectly(t *testing.T) {
allow_all_crew BOOLEAN NOT NULL, allow_all_crew BOOLEAN NOT NULL,
deployed_at TEXT, deployed_at TEXT,
region TEXT, region TEXT,
successor TEXT,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
); );
CREATE TABLE hold_crew_members ( CREATE TABLE hold_crew_members (
@@ -770,6 +814,13 @@ func TestProcessRecord_RoutesCorrectly(t *testing.T) {
); );
`) `)
// Register the hold DID so captain verification resolves locally instead
// of hitting the network.
atproto.SetDirectory(&fakeDirectory{byDID: map[string]*identity.Identity{
"did:web:hold.example.com": holdIdentity("did:web:hold.example.com", "https://hold.example.com"),
}})
defer atproto.SetDirectory(nil)
processor := NewProcessor(db, false, nil) processor := NewProcessor(db, false, nil)
ctx := context.Background() ctx := context.Background()
var err error var err error
@@ -831,6 +882,82 @@ func TestProcessRecord_RoutesCorrectly(t *testing.T) {
} }
} }
func TestProcessCaptain_VerifiesHoldService(t *testing.T) {
db := setupTestDB(t)
defer db.Close()
execStatements(t, db, `
CREATE TABLE hold_captain_records (
hold_did TEXT PRIMARY KEY,
owner_did TEXT NOT NULL,
public BOOLEAN NOT NULL,
allow_all_crew BOOLEAN NOT NULL,
deployed_at TEXT,
region TEXT,
successor TEXT,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
`)
holdDID := "did:web:realhold.example.com"
userDID := "did:plc:notahold"
atproto.SetDirectory(&fakeDirectory{byDID: map[string]*identity.Identity{
holdDID: holdIdentity(holdDID, "https://realhold.example.com"),
userDID: {
DID: syntax.DID(userDID),
Services: map[string]identity.ServiceEndpoint{
"atproto_pds": {Type: "AtprotoPersonalDataServer", URL: "https://pds.example.com"},
},
},
}})
defer atproto.SetDirectory(nil)
processor := NewProcessor(db, false, nil)
ctx := context.Background()
captainData, _ := json.Marshal(map[string]any{
"$type": "io.atcr.hold.captain",
"owner": "did:plc:owner123",
"public": true,
"allowAllCrew": true,
"enableBlueskyPosts": false,
"deployedAt": time.Now().Format(time.RFC3339),
})
captainCount := func(did string) int {
t.Helper()
var n int
if err := db.QueryRow(`SELECT COUNT(*) FROM hold_captain_records WHERE hold_did = ?`, did).Scan(&n); err != nil {
t.Fatalf("Failed to count captain records: %v", err)
}
return n
}
// A DID advertising the atcr_hold service is cached.
if err := processor.ProcessCaptain(ctx, holdDID, captainData); err != nil {
t.Fatalf("ProcessCaptain failed for real hold: %v", err)
}
if captainCount(holdDID) != 1 {
t.Error("Captain record from a real hold should be cached")
}
// A resolvable DID without the atcr_hold service is skipped silently.
if err := processor.ProcessCaptain(ctx, userDID, captainData); err != nil {
t.Fatalf("ProcessCaptain should skip non-hold DIDs without error: %v", err)
}
if captainCount(userDID) != 0 {
t.Error("Captain record from a non-hold DID should not be cached")
}
// An unresolvable DID is skipped silently (periodic backfill retries).
if err := processor.ProcessCaptain(ctx, "did:plc:unresolvable", captainData); err != nil {
t.Fatalf("ProcessCaptain should skip unresolvable DIDs without error: %v", err)
}
if captainCount("did:plc:unresolvable") != 0 {
t.Error("Captain record from an unresolvable DID should not be cached")
}
}
func TestProcessRecord_SkipsInvalidRecords(t *testing.T) { func TestProcessRecord_SkipsInvalidRecords(t *testing.T) {
db := setupTestDB(t) db := setupTestDB(t)
defer db.Close() defer db.Close()
pkg/appview/public/icons.svg
+2
View File
@@ -43,6 +43,8 @@
<symbol id="loader-2" viewBox="0 0 24 24"><path d="M21 12a9 9 0 1 1-6.219-8.56"/></symbol> <symbol id="loader-2" viewBox="0 0 24 24"><path d="M21 12a9 9 0 1 1-6.219-8.56"/></symbol>
<symbol id="moon" viewBox="0 0 24 24"><path d="M20.985 12.486a9 9 0 1 1-9.473-9.472c.405-.022.617.46.402.803a6 6 0 0 0 8.268 8.268c.344-.215.825-.004.803.401"/></symbol> <symbol id="moon" viewBox="0 0 24 24"><path d="M20.985 12.486a9 9 0 1 1-9.473-9.472c.405-.022.617.46.402.803a6 6 0 0 0 8.268 8.268c.344-.215.825-.004.803.401"/></symbol>
<symbol id="package" viewBox="0 0 24 24"><path d="M11 21.73a2 2 0 0 0 2 0l7-4A2 2 0 0 0 21 16V8a2 2 0 0 0-1-1.73l-7-4a2 2 0 0 0-2 0l-7 4A2 2 0 0 0 3 8v8a2 2 0 0 0 1 1.73z"/><path d="M12 22V12"/><polyline points="3.29 7 12 12 20.71 7"/><path d="m7.5 4.27 9 5.15"/></symbol> <symbol id="package" viewBox="0 0 24 24"><path d="M11 21.73a2 2 0 0 0 2 0l7-4A2 2 0 0 0 21 16V8a2 2 0 0 0-1-1.73l-7-4a2 2 0 0 0-2 0l-7 4A2 2 0 0 0 3 8v8a2 2 0 0 0 1 1.73z"/><path d="M12 22V12"/><polyline points="3.29 7 12 12 20.71 7"/><path d="m7.5 4.27 9 5.15"/></symbol>
<symbol id="package-minus" viewBox="0 0 24 24"><path d="M12 22V12"/><path d="M16 17h6"/><path d="M21 13V8a2 2 0 0 0-1-1.73l-7-4a2 2 0 0 0-2 0l-7 4A2 2 0 0 0 3 8v8a2 2 0 0 0 1 1.729l7 4a2 2 0 0 0 2 .001l1.675-.955"/><path d="M3.29 7 12 12l8.71-5"/><path d="m7.5 4.27 8.997 5.148"/></symbol>
<symbol id="package-plus" viewBox="0 0 24 24"><path d="M12 22V12"/><path d="M16 17h6"/><path d="M19 14v6"/><path d="M21 10.535V8a2 2 0 0 0-1-1.73l-7-4a2 2 0 0 0-2 0l-7 4A2 2 0 0 0 3 8v8a2 2 0 0 0 1 1.729l7 4a2 2 0 0 0 2 .001l1.675-.955"/><path d="M3.29 7 12 12l8.71-5"/><path d="m7.5 4.27 8.997 5.148"/></symbol>
<symbol id="pause" viewBox="0 0 24 24"><rect x="14" y="3" width="5" height="18" rx="1"/><rect x="5" y="3" width="5" height="18" rx="1"/></symbol> <symbol id="pause" viewBox="0 0 24 24"><rect x="14" y="3" width="5" height="18" rx="1"/><rect x="5" y="3" width="5" height="18" rx="1"/></symbol>
<symbol id="pencil" viewBox="0 0 24 24"><path d="M21.174 6.812a1 1 0 0 0-3.986-3.987L3.842 16.174a2 2 0 0 0-.5.83l-1.321 4.352a.5.5 0 0 0 .623.622l4.353-1.32a2 2 0 0 0 .83-.497z"/><path d="m15 5 4 4"/></symbol> <symbol id="pencil" viewBox="0 0 24 24"><path d="M21.174 6.812a1 1 0 0 0-3.986-3.987L3.842 16.174a2 2 0 0 0-.5.83l-1.321 4.352a.5.5 0 0 0 .623.622l4.353-1.32a2 2 0 0 0 .83-.497z"/><path d="m15 5 4 4"/></symbol>
<symbol id="play" viewBox="0 0 24 24"><path d="M5 5a2 2 0 0 1 3.008-1.728l11.997 6.998a2 2 0 0 1 .003 3.458l-12 7A2 2 0 0 1 5 19z"/></symbol> <symbol id="play" viewBox="0 0 24 24"><path d="M5 5a2 2 0 0 1 3.008-1.728l11.997 6.998a2 2 0 0 1 .003 3.458l-12 7A2 2 0 0 1 5 19z"/></symbol>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 25 KiB

After

Width:  |  Height:  |  Size: 26 KiB

pkg/appview/server.go
+1 -1
View File
@@ -607,7 +607,7 @@ func NewAppViewServer(cfg *Config, branding *BrandingOverrides) (*AppViewServer,
"device_token", "/auth/device/token", "device_token", "/auth/device/token",
"oauth_authorize", "/auth/oauth/authorize", "oauth_authorize", "/auth/oauth/authorize",
"oauth_callback", "/auth/oauth/callback", "oauth_callback", "/auth/oauth/callback",
"oauth_metadata", "/client-metadata.json") "oauth_metadata", "/oauth-client-metadata.json")
} }
// Health check endpoint (for Docker health checks / load balancers) // Health check endpoint (for Docker health checks / load balancers)
pkg/appview/templates/pages/diff.html
+27 -1
View File
@@ -24,7 +24,7 @@
{{ end }} {{ end }}
{{ if and (not .FromFailed) (not .ToFailed) (eq .FromDigest .ToDigest) }} {{ if and (not .FromFailed) (not .ToFailed) (eq .FromDigest .ToDigest) }}
{{ template "alert" (dict "Type" "info" "Message" "These manifests are identical. No layers or vulnerabilities changed.") }} {{ template "alert" (dict "Type" "info" "Message" "These manifests are identical. No layers, vulnerabilities, or packages changed.") }}
{{ end }} {{ end }}
<!-- Summary Card --> <!-- Summary Card -->
@@ -81,6 +81,32 @@
</div> </div>
{{ end }} {{ end }}
{{ end }} {{ end }}
{{ if .HasSbomData }}
<!-- Packages changed -->
{{ if gt .Summary.PkgChangedCount 0 }}
<div class="stat bg-warning/10 rounded-lg p-3">
<div class="stat-title text-xs">Pkgs changed</div>
<div class="stat-value text-sm text-warning"><span aria-hidden="true">~</span>{{ .Summary.PkgChangedCount }}</div>
</div>
{{ end }}
<!-- Packages added -->
{{ if gt .Summary.PkgAddedCount 0 }}
<div class="stat bg-success/10 rounded-lg p-3">
<div class="stat-title text-xs">Pkgs added</div>
<div class="stat-value text-sm text-success">+{{ .Summary.PkgAddedCount }}</div>
</div>
{{ end }}
<!-- Packages removed -->
{{ if gt .Summary.PkgRemovedCount 0 }}
<div class="stat bg-error/10 rounded-lg p-3">
<div class="stat-title text-xs">Pkgs removed</div>
<div class="stat-value text-sm text-error">-{{ .Summary.PkgRemovedCount }}</div>
</div>
{{ end }}
{{ end }}
</div> </div>
{{ if .IsMultiArch }} {{ if .IsMultiArch }}
pkg/appview/templates/partials/diff-content.html
+175 -3
View File
@@ -48,9 +48,11 @@
{{ end }} {{ end }}
</div> </div>
<!-- Vulnerability Diff (Right) --> <!-- Vulnerability + Package Diff (Right) -->
<div class="card bg-base-200 shadow-sm border border-base-300 p-6 space-y-4 min-w-0"> <div class="card bg-base-200 shadow-sm border border-base-300 p-6 min-w-0">
<h2 class="text-lg font-semibold">Vulnerabilities</h2> <div role="tablist" class="tabs tabs-bordered">
<input type="radio" id="diff-tab-vulns" name="diff-scan-tabs" role="tab" class="tab" aria-label="Vulnerabilities" aria-controls="diff-panel-vulns" checked="checked" />
<div id="diff-panel-vulns" role="tabpanel" aria-labelledby="diff-tab-vulns" class="tab-content pt-4 space-y-4">
{{ if not .HasVulnData }} {{ if not .HasVulnData }}
{{/* Branch on per-side scan status so users can tell "not scanned {{/* Branch on per-side scan status so users can tell "not scanned
@@ -198,6 +200,176 @@
{{ end }} {{ end }}
{{ end }} {{ end }}
</div>
<input type="radio" id="diff-tab-sbom" name="diff-scan-tabs" role="tab" class="tab" aria-label="Packages" aria-controls="diff-panel-sbom" />
<div id="diff-panel-sbom" role="tabpanel" aria-labelledby="diff-tab-sbom" class="tab-content pt-4 space-y-4">
{{ if not .HasSbomData }}
{{/* Mirror the vuln branching so users can tell "not scanned yet"
from "hold offline" from "this artifact type isn't scanned". */}}
{{ if or (eq .FromSbomStatus "hold-unreachable") (eq .ToSbomStatus "hold-unreachable") }}
<div class="alert alert-warning" role="status">
{{ icon "wifi-off" "size-4 shrink-0" }}
<span>We couldn't reach the hold to fetch SBOM data. Try again in a moment.</span>
</div>
{{ else if or (eq .FromSbomStatus "not-applicable") (eq .ToSbomStatus "not-applicable") }}
<p class="text-base-content/60">SBOMs aren't generated for this artifact type.</p>
{{ else if or (eq .FromSbomStatus "no-data") (eq .ToSbomStatus "no-data") }}
<p class="text-base-content/60">An SBOM isn't available for both manifests yet. Package comparison will appear after both scans complete.</p>
{{ else }}
<p class="text-base-content/60">SBOM data isn't available for both manifests.</p>
{{ end }}
{{ else }}
<!-- Changed Packages -->
{{ if .ChangedPackages }}
<div class="collapse collapse-arrow bg-warning/5 border border-warning/20 rounded-lg">
<input type="checkbox" checked aria-label="Toggle changed packages ({{ len .ChangedPackages }})" />
<div class="collapse-title font-medium text-sm flex items-center gap-2">
{{ icon "package" "size-4 text-warning" }}
Changed ({{ len .ChangedPackages }})
</div>
<div class="collapse-content">
<div class="overflow-x-auto">
<table class="table table-xs w-full">
<caption class="sr-only">Packages whose version changed between manifests</caption>
<thead>
<tr class="text-xs">
<th scope="col">Package</th>
<th scope="col">Type</th>
<th scope="col">Version</th>
</tr>
</thead>
<tbody>
{{ range .ChangedPackages }}
<tr>
<td class="text-xs truncate max-w-xs" title="{{ .Package.Name }}">{{ if .Package.URL }}<a href="{{ .Package.URL }}" target="_blank" rel="noopener noreferrer" class="link link-primary">{{ .Package.Name }}</a>{{ else }}{{ .Package.Name }}{{ end }}</td>
<td>{{ if .Package.Type }}<span class="badge badge-xs badge-ghost">{{ .Package.Type }}</span>{{ end }}</td>
<td class="text-xs font-mono truncate max-w-48" title="{{ .PrevVersion }} to {{ .Package.Version }}">{{ .PrevVersion }} <span aria-hidden="true">→</span><span class="sr-only">to</span> {{ .Package.Version }}</td>
</tr>
{{ end }}
</tbody>
</table>
</div>
</div>
</div>
{{ end }}
<!-- Added Packages -->
{{ if .AddedPackages }}
<div class="collapse collapse-arrow bg-success/5 border border-success/20 rounded-lg">
<input type="checkbox" checked aria-label="Toggle added packages ({{ len .AddedPackages }})" />
<div class="collapse-title font-medium text-sm flex items-center gap-2">
{{ icon "package-plus" "size-4 text-success" }}
Added ({{ len .AddedPackages }})
</div>
<div class="collapse-content">
<div class="overflow-x-auto">
<table class="table table-xs w-full">
<caption class="sr-only">Packages only present in the newer manifest</caption>
<thead>
<tr class="text-xs">
<th scope="col">Package</th>
<th scope="col">Version</th>
<th scope="col">Type</th>
<th scope="col">License</th>
</tr>
</thead>
<tbody>
{{ range .AddedPackages }}
<tr>
<td class="text-xs truncate max-w-xs" title="{{ .Package.Name }}">{{ if .Package.URL }}<a href="{{ .Package.URL }}" target="_blank" rel="noopener noreferrer" class="link link-primary">{{ .Package.Name }}</a>{{ else }}{{ .Package.Name }}{{ end }}</td>
<td class="text-xs font-mono truncate max-w-40" title="{{ .Package.Version }}">{{ .Package.Version }}</td>
<td>{{ if .Package.Type }}<span class="badge badge-xs badge-ghost">{{ .Package.Type }}</span>{{ end }}</td>
<td class="text-xs truncate max-w-40" title="{{ .Package.License }}">{{ .Package.License }}</td>
</tr>
{{ end }}
</tbody>
</table>
</div>
</div>
</div>
{{ end }}
<!-- Removed Packages -->
{{ if .RemovedPackages }}
<div class="collapse collapse-arrow bg-error/5 border border-error/20 rounded-lg">
<input type="checkbox" checked aria-label="Toggle removed packages ({{ len .RemovedPackages }})" />
<div class="collapse-title font-medium text-sm flex items-center gap-2">
{{ icon "package-minus" "size-4 text-error" }}
Removed ({{ len .RemovedPackages }})
</div>
<div class="collapse-content">
<div class="overflow-x-auto">
<table class="table table-xs w-full">
<caption class="sr-only">Packages only present in the older manifest</caption>
<thead>
<tr class="text-xs">
<th scope="col">Package</th>
<th scope="col">Version</th>
<th scope="col">Type</th>
<th scope="col">License</th>
</tr>
</thead>
<tbody>
{{ range .RemovedPackages }}
<tr>
<td class="text-xs truncate max-w-xs" title="{{ .Package.Name }}">{{ if .Package.URL }}<a href="{{ .Package.URL }}" target="_blank" rel="noopener noreferrer" class="link link-primary">{{ .Package.Name }}</a>{{ else }}{{ .Package.Name }}{{ end }}</td>
<td class="text-xs font-mono truncate max-w-40" title="{{ .Package.Version }}">{{ .Package.Version }}</td>
<td>{{ if .Package.Type }}<span class="badge badge-xs badge-ghost">{{ .Package.Type }}</span>{{ end }}</td>
<td class="text-xs truncate max-w-40" title="{{ .Package.License }}">{{ .Package.License }}</td>
</tr>
{{ end }}
</tbody>
</table>
</div>
</div>
</div>
{{ end }}
<!-- Unchanged Packages -->
{{ if .UnchangedPackages }}
<div class="collapse collapse-arrow bg-base-200/50 border border-base-300 rounded-lg">
<input type="checkbox" aria-label="Toggle unchanged packages ({{ len .UnchangedPackages }})" />
<div class="collapse-title font-medium text-sm text-base-content/60">
Unchanged ({{ len .UnchangedPackages }})
</div>
<div class="collapse-content">
<div class="overflow-x-auto overflow-y-auto max-h-128">
<table class="table table-xs w-full">
<caption class="sr-only">Packages identical in both manifests</caption>
<thead>
<tr class="text-xs">
<th scope="col">Package</th>
<th scope="col">Version</th>
<th scope="col">Type</th>
<th scope="col">License</th>
</tr>
</thead>
<tbody>
{{ range .UnchangedPackages }}
<tr>
<td class="text-xs truncate max-w-xs" title="{{ .Package.Name }}">{{ if .Package.URL }}<a href="{{ .Package.URL }}" target="_blank" rel="noopener noreferrer" class="link link-primary">{{ .Package.Name }}</a>{{ else }}{{ .Package.Name }}{{ end }}</td>
<td class="text-xs font-mono truncate max-w-40" title="{{ .Package.Version }}">{{ .Package.Version }}</td>
<td>{{ if .Package.Type }}<span class="badge badge-xs badge-ghost">{{ .Package.Type }}</span>{{ end }}</td>
<td class="text-xs truncate max-w-40" title="{{ .Package.License }}">{{ .Package.License }}</td>
</tr>
{{ end }}
</tbody>
</table>
</div>
</div>
</div>
{{ end }}
{{ if and (not .ChangedPackages) (not .AddedPackages) (not .RemovedPackages) (not .UnchangedPackages) }}
<p class="text-base-content/60">No packages found in either manifest</p>
{{ end }}
{{ end }}
</div>
</div>
</div> </div>
</div> </div>
{{ end }} {{ end }}
pkg/appview/templates/partials/hold_selector.html
+1 -1
View File
@@ -12,7 +12,7 @@
<optgroup label="Your Holds"> <optgroup label="Your Holds">
{{ range .MemberHolds }} {{ range .MemberHolds }}
<option value="{{ .DID }}" {{ if .IsActive }}selected{{ end }}> <option value="{{ .DID }}" {{ if .IsActive }}selected{{ end }}>
{{ .DisplayName }}{{ if eq .Membership "owner" }} (Owner){{ else }} (Crew){{ end }}{{ if .Region }} &middot; {{ .Region }}{{ end }} {{ .DisplayName }}{{ if eq .Membership "owner" }} (Owner){{ else if .ReadOnly }} (Crew, read-only){{ else }} (Crew){{ end }}{{ if .Region }} &middot; {{ .Region }}{{ end }}
</option> </option>
{{ end }} {{ end }}
</optgroup> </optgroup>
pkg/appview/templates/partials/sbom-details.html
+1 -1
View File
@@ -46,7 +46,7 @@
<tbody> <tbody>
{{ range .Packages }} {{ range .Packages }}
<tr> <tr>
<td class="text-xs truncate" title="{{ .Name }}">{{ .Name }}</td> <td class="text-xs truncate" title="{{ .Name }}">{{ if .URL }}<a href="{{ .URL }}" target="_blank" rel="noopener noreferrer" class="link link-primary">{{ .Name }}</a>{{ else }}{{ .Name }}{{ end }}</td>
<td class="font-mono text-xs truncate" title="{{ .Version }}">{{ .Version }}</td> <td class="font-mono text-xs truncate" title="{{ .Version }}">{{ .Version }}</td>
<td class="text-xs truncate" title="{{ .License }}"> <td class="text-xs truncate" title="{{ .License }}">
{{ if eq .License "-" }} {{ if eq .License "-" }}
pkg/atproto/resolver.go
+26
View File
@@ -118,6 +118,32 @@ func ResolveHoldDIDToURL(ctx context.Context, did string) (string, error) {
return "", fmt.Errorf("no hold or PDS service endpoint found for DID %s", did) return "", fmt.Errorf("no hold or PDS service endpoint found for DID %s", did)
} }
// HasHoldService reports whether a DID's identity document advertises an
// #atcr_hold service endpoint, i.e. whether the DID actually runs a hold
// service. Any ATProto account can publish io.atcr.hold.captain records, but
// only real holds publish the atcr_hold service in their DID document — use
// this to verify captain records discovered on the network before caching.
// Resolution goes through the shared identity directory (cached, 24h TTL).
func HasHoldService(ctx context.Context, did string) (bool, error) {
didParsed, err := syntax.ParseDID(did)
if err != nil {
return false, fmt.Errorf("invalid hold DID %q: %w", did, err)
}
ident, err := GetDirectory().LookupDID(ctx, didParsed)
if err != nil {
// In test mode, local did:web identifiers (HTTP, IP:port) are not
// resolvable by the indigo directory at all — trust them, matching
// the ResolveHoldDIDToURL fallback.
if testMode && strings.HasPrefix(did, "did:web:") {
return true, nil
}
return false, fmt.Errorf("failed to resolve DID %s: %w", did, err)
}
return ident.GetServiceEndpoint("atcr_hold") != "", nil
}
// NormalizeDID ensures did:web DIDs use %3A encoding for port separators // NormalizeDID ensures did:web DIDs use %3A encoding for port separators
// per the did:web spec. Other DID methods are returned as-is. // per the did:web spec. Other DID methods are returned as-is.
// e.g., "did:web:172.28.0.3:8080" → "did:web:172.28.0.3%3A8080" // e.g., "did:web:172.28.0.3:8080" → "did:web:172.28.0.3%3A8080"
pkg/atproto/resolver_test.go
+138
View File
@@ -2,10 +2,148 @@ package atproto
import ( import (
"context" "context"
"fmt"
"strings" "strings"
"testing" "testing"
"github.com/bluesky-social/indigo/atproto/identity"
"github.com/bluesky-social/indigo/atproto/syntax"
) )
// stubDirectory is a minimal in-memory identity.Directory; lookups of
// unregistered identifiers fail, mirroring unresolvable identities.
type stubDirectory struct {
byDID map[string]*identity.Identity
}
func (d *stubDirectory) LookupDID(_ context.Context, did syntax.DID) (*identity.Identity, error) {
if ident, ok := d.byDID[did.String()]; ok {
return ident, nil
}
return nil, fmt.Errorf("%w: %s", identity.ErrDIDNotFound, did)
}
func (d *stubDirectory) LookupHandle(_ context.Context, handle syntax.Handle) (*identity.Identity, error) {
return nil, fmt.Errorf("%w: %s", identity.ErrHandleNotFound, handle)
}
func (d *stubDirectory) Lookup(ctx context.Context, atid syntax.AtIdentifier) (*identity.Identity, error) {
if did, err := atid.AsDID(); err == nil {
return d.LookupDID(ctx, did)
}
return nil, fmt.Errorf("%w: %s", identity.ErrHandleResolutionFailed, atid)
}
func (d *stubDirectory) Purge(_ context.Context, _ syntax.AtIdentifier) error {
return nil
}
func TestHasHoldService(t *testing.T) {
holdDID := "did:web:hold.example.com"
pdsOnlyDID := "did:plc:ordinaryaccount"
SetDirectory(&stubDirectory{byDID: map[string]*identity.Identity{
holdDID: {
DID: syntax.DID(holdDID),
Services: map[string]identity.ServiceEndpoint{
"atproto_pds": {Type: "AtprotoPersonalDataServer", URL: "https://hold.example.com"},
"atcr_hold": {Type: "AtcrHoldService", URL: "https://hold.example.com"},
},
},
pdsOnlyDID: {
DID: syntax.DID(pdsOnlyDID),
Services: map[string]identity.ServiceEndpoint{
"atproto_pds": {Type: "AtprotoPersonalDataServer", URL: "https://pds.example.com"},
},
},
}})
defer SetDirectory(nil)
ctx := context.Background()
t.Run("DID with atcr_hold service", func(t *testing.T) {
isHold, err := HasHoldService(ctx, holdDID)
if err != nil {
t.Fatalf("HasHoldService(%q) unexpected error: %v", holdDID, err)
}
if !isHold {
t.Errorf("HasHoldService(%q) = false, want true", holdDID)
}
})
t.Run("DID without atcr_hold service", func(t *testing.T) {
isHold, err := HasHoldService(ctx, pdsOnlyDID)
if err != nil {
t.Fatalf("HasHoldService(%q) unexpected error: %v", pdsOnlyDID, err)
}
if isHold {
t.Errorf("HasHoldService(%q) = true, want false", pdsOnlyDID)
}
})
t.Run("invalid DID", func(t *testing.T) {
if _, err := HasHoldService(ctx, "not-a-did"); err == nil {
t.Error("HasHoldService(not-a-did) expected error, got nil")
}
})
t.Run("unresolvable DID", func(t *testing.T) {
if _, err := HasHoldService(ctx, "did:plc:unknown000000000000"); err == nil {
t.Error("HasHoldService(unresolvable) expected error, got nil")
}
})
}
// TestHasHoldServiceTestMode covers the test-mode fallback: local did:web
// identifiers (HTTP, IP:port) that the indigo directory cannot resolve are
// trusted, matching the ResolveHoldDIDToURL fallback. Resolvable DIDs are
// still checked normally, and the fallback never applies to other methods.
func TestHasHoldServiceTestMode(t *testing.T) {
pdsOnlyDID := "did:plc:ordinaryaccount"
SetDirectory(&stubDirectory{byDID: map[string]*identity.Identity{
pdsOnlyDID: {
DID: syntax.DID(pdsOnlyDID),
Services: map[string]identity.ServiceEndpoint{
"atproto_pds": {Type: "AtprotoPersonalDataServer", URL: "https://pds.example.com"},
},
},
}})
defer SetDirectory(nil)
SetTestMode(true)
defer SetTestMode(false)
ctx := context.Background()
t.Run("trusts unresolvable did:web", func(t *testing.T) {
localDID := "did:web:172.28.0.3%3A8080"
isHold, err := HasHoldService(ctx, localDID)
if err != nil {
t.Fatalf("HasHoldService(%q) unexpected error: %v", localDID, err)
}
if !isHold {
t.Errorf("HasHoldService(%q) = false, want true (test-mode did:web fallback)", localDID)
}
})
t.Run("rejects unresolvable did:plc", func(t *testing.T) {
if _, err := HasHoldService(ctx, "did:plc:unknown000000000000"); err == nil {
t.Error("HasHoldService(unresolvable did:plc) expected error even in test mode, got nil")
}
})
t.Run("still checks resolvable DIDs", func(t *testing.T) {
isHold, err := HasHoldService(ctx, pdsOnlyDID)
if err != nil {
t.Fatalf("HasHoldService(%q) unexpected error: %v", pdsOnlyDID, err)
}
if isHold {
t.Errorf("HasHoldService(%q) = true, want false (fallback must not bypass a successful lookup)", pdsOnlyDID)
}
})
}
func TestResolveHoldURL(t *testing.T) { func TestResolveHoldURL(t *testing.T) {
ctx := context.Background() ctx := context.Background()
pkg/hold/admin/public/icons.svg
+2
View File
@@ -43,6 +43,8 @@
<symbol id="loader-2" viewBox="0 0 24 24"><path d="M21 12a9 9 0 1 1-6.219-8.56"/></symbol> <symbol id="loader-2" viewBox="0 0 24 24"><path d="M21 12a9 9 0 1 1-6.219-8.56"/></symbol>
<symbol id="moon" viewBox="0 0 24 24"><path d="M20.985 12.486a9 9 0 1 1-9.473-9.472c.405-.022.617.46.402.803a6 6 0 0 0 8.268 8.268c.344-.215.825-.004.803.401"/></symbol> <symbol id="moon" viewBox="0 0 24 24"><path d="M20.985 12.486a9 9 0 1 1-9.473-9.472c.405-.022.617.46.402.803a6 6 0 0 0 8.268 8.268c.344-.215.825-.004.803.401"/></symbol>
<symbol id="package" viewBox="0 0 24 24"><path d="M11 21.73a2 2 0 0 0 2 0l7-4A2 2 0 0 0 21 16V8a2 2 0 0 0-1-1.73l-7-4a2 2 0 0 0-2 0l-7 4A2 2 0 0 0 3 8v8a2 2 0 0 0 1 1.73z"/><path d="M12 22V12"/><polyline points="3.29 7 12 12 20.71 7"/><path d="m7.5 4.27 9 5.15"/></symbol> <symbol id="package" viewBox="0 0 24 24"><path d="M11 21.73a2 2 0 0 0 2 0l7-4A2 2 0 0 0 21 16V8a2 2 0 0 0-1-1.73l-7-4a2 2 0 0 0-2 0l-7 4A2 2 0 0 0 3 8v8a2 2 0 0 0 1 1.73z"/><path d="M12 22V12"/><polyline points="3.29 7 12 12 20.71 7"/><path d="m7.5 4.27 9 5.15"/></symbol>
<symbol id="package-minus" viewBox="0 0 24 24"><path d="M12 22V12"/><path d="M16 17h6"/><path d="M21 13V8a2 2 0 0 0-1-1.73l-7-4a2 2 0 0 0-2 0l-7 4A2 2 0 0 0 3 8v8a2 2 0 0 0 1 1.729l7 4a2 2 0 0 0 2 .001l1.675-.955"/><path d="M3.29 7 12 12l8.71-5"/><path d="m7.5 4.27 8.997 5.148"/></symbol>
<symbol id="package-plus" viewBox="0 0 24 24"><path d="M12 22V12"/><path d="M16 17h6"/><path d="M19 14v6"/><path d="M21 10.535V8a2 2 0 0 0-1-1.73l-7-4a2 2 0 0 0-2 0l-7 4A2 2 0 0 0 3 8v8a2 2 0 0 0 1 1.729l7 4a2 2 0 0 0 2 .001l1.675-.955"/><path d="M3.29 7 12 12l8.71-5"/><path d="m7.5 4.27 8.997 5.148"/></symbol>
<symbol id="pause" viewBox="0 0 24 24"><rect x="14" y="3" width="5" height="18" rx="1"/><rect x="5" y="3" width="5" height="18" rx="1"/></symbol> <symbol id="pause" viewBox="0 0 24 24"><rect x="14" y="3" width="5" height="18" rx="1"/><rect x="5" y="3" width="5" height="18" rx="1"/></symbol>
<symbol id="pencil" viewBox="0 0 24 24"><path d="M21.174 6.812a1 1 0 0 0-3.986-3.987L3.842 16.174a2 2 0 0 0-.5.83l-1.321 4.352a.5.5 0 0 0 .623.622l4.353-1.32a2 2 0 0 0 .83-.497z"/><path d="m15 5 4 4"/></symbol> <symbol id="pencil" viewBox="0 0 24 24"><path d="M21.174 6.812a1 1 0 0 0-3.986-3.987L3.842 16.174a2 2 0 0 0-.5.83l-1.321 4.352a.5.5 0 0 0 .623.622l4.353-1.32a2 2 0 0 0 .83-.497z"/><path d="m15 5 4 4"/></symbol>
<symbol id="play" viewBox="0 0 24 24"><path d="M5 5a2 2 0 0 1 3.008-1.728l11.997 6.998a2 2 0 0 1 .003 3.458l-12 7A2 2 0 0 1 5 19z"/></symbol> <symbol id="play" viewBox="0 0 24 24"><path d="M5 5a2 2 0 0 1 3.008-1.728l11.997 6.998a2 2 0 0 1 .003 3.458l-12 7A2 2 0 0 1 5 19z"/></symbol>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 25 KiB

After

Width:  |  Height:  |  Size: 26 KiB

pkg/hold/pds/repo.go
+33 -1
View File
@@ -9,7 +9,6 @@ package pds
// - No gorm dependency // - No gorm dependency
// //
// Implements the RepoOperator interface (see repo_operator.go). // Implements the RepoOperator interface (see repo_operator.go).
// See docs/REPOMGR_MIGRATION.md for the migration plan.
import ( import (
"context" "context"
@@ -105,6 +104,39 @@ func (d *DirectRepoOperator) openWriteSession(ctx context.Context, user models.U
}, nil }, nil
} }
// fillPrevCIDs uses the MST diff between oldRoot and the current repo state to
// stamp Prev on existing RepoOps in-place. Required for Sync 1.1 inductive
// firehose: update/delete ops must carry the previous record CID.
//
// Call after r.Commit() and before ds.CloseWithRoot() — the delta session still
// holds both old and new blocks at that point. No-op for first commits.
func fillPrevCIDs(ctx context.Context, r *repo.Repo, oldRoot cid.Cid, ops []RepoOp) error {
if !oldRoot.Defined() || len(ops) == 0 {
return nil
}
diff, err := r.DiffSince(ctx, oldRoot)
if err != nil {
return fmt.Errorf("DiffSince: %w", err)
}
prevByPath := make(map[string]cid.Cid, len(diff))
for _, d := range diff {
// mst.DiffOp.Op is "add" | "mut" | "del" — only the latter two have an OldCid.
if d.Op == "mut" || d.Op == "del" {
prevByPath[d.Rpath] = d.OldCid
}
}
for i := range ops {
if ops[i].Kind == EvtKindCreateRecord {
continue
}
path := ops[i].Collection + "/" + ops[i].Rkey
if pc, ok := prevByPath[path]; ok {
ops[i].Prev = &pc
}
}
return nil
}
// commitWrite commits a write session and emits an event if configured. // commitWrite commits a write session and emits an event if configured.
func (d *DirectRepoOperator) commitWrite(ctx context.Context, ws *writeSession, user models.Uid, ops []RepoOp) (cid.Cid, string, error) { func (d *DirectRepoOperator) commitWrite(ctx context.Context, ws *writeSession, user models.Uid, ops []RepoOp) (cid.Cid, string, error) {
nroot, nrev, err := ws.r.Commit(ctx, d.kmgr.SignForUser) nroot, nrev, err := ws.r.Commit(ctx, d.kmgr.SignForUser)
pkg/hold/pds/repo_operator.go
+3 -3
View File
@@ -14,9 +14,9 @@ import (
) )
// RepoOperator defines the interface for ATProto repository operations. // RepoOperator defines the interface for ATProto repository operations.
// RepoManager implements this interface. Future implementations (e.g., using // DirectRepoOperator (repo.go) is the production implementation; RepoManager
// indigo/repo directly) can be swapped in behind this interface. // (repomgr.go) is the legacy vendored implementation, retained so the shared
// See docs/REPOMGR_MIGRATION.md for the migration plan. // test suite can cross-validate both.
type RepoOperator interface { type RepoOperator interface {
// Record CRUD // Record CRUD
CreateRecord(ctx context.Context, user models.Uid, collection string, rec cbg.CBORMarshaler) (string, cid.Cid, error) CreateRecord(ctx context.Context, user models.Uid, collection string, rec cbg.CBORMarshaler) (string, cid.Cid, error)
pkg/hold/pds/repomgr.go
+2 -34
View File
@@ -8,7 +8,8 @@ package pds
// - Added prevData support for Sync 1.1 // - Added prevData support for Sync 1.1
// //
// Implements the RepoOperator interface (see repo_operator.go). // Implements the RepoOperator interface (see repo_operator.go).
// See docs/REPOMGR_MIGRATION.md for planned migration to indigo/repo directly. // Superseded in production by DirectRepoOperator (repo.go); kept as a
// cross-validation oracle for the shared RepoOperator test suite.
import ( import (
"context" "context"
@@ -98,39 +99,6 @@ func (rm *RepoManager) lockUser(ctx context.Context, user models.Uid) func() {
} }
} }
// fillPrevCIDs uses the MST diff between oldRoot and the current repo state to
// stamp Prev on existing RepoOps in-place. Required for Sync 1.1 inductive
// firehose: update/delete ops must carry the previous record CID.
//
// Call after r.Commit() and before ds.CloseWithRoot() — the delta session still
// holds both old and new blocks at that point. No-op for first commits.
func fillPrevCIDs(ctx context.Context, r *repo.Repo, oldRoot cid.Cid, ops []RepoOp) error {
if !oldRoot.Defined() || len(ops) == 0 {
return nil
}
diff, err := r.DiffSince(ctx, oldRoot)
if err != nil {
return fmt.Errorf("DiffSince: %w", err)
}
prevByPath := make(map[string]cid.Cid, len(diff))
for _, d := range diff {
// mst.DiffOp.Op is "add" | "mut" | "del" — only the latter two have an OldCid.
if d.Op == "mut" || d.Op == "del" {
prevByPath[d.Rpath] = d.OldCid
}
}
for i := range ops {
if ops[i].Kind == EvtKindCreateRecord {
continue
}
path := ops[i].Collection + "/" + ops[i].Rkey
if pc, ok := prevByPath[path]; ok {
ops[i].Prev = &pc
}
}
return nil
}
func (rm *RepoManager) CreateRecord(ctx context.Context, user models.Uid, collection string, rec cbg.CBORMarshaler) (string, cid.Cid, error) { func (rm *RepoManager) CreateRecord(ctx context.Context, user models.Uid, collection string, rec cbg.CBORMarshaler) (string, cid.Cid, error) {
ctx, span := otel.Tracer("repoman").Start(ctx, "CreateRecord") ctx, span := otel.Tracer("repoman").Start(ctx, "CreateRecord")
defer span.End() defer span.End()